[Dnsmasq-discuss] dnsmasq 2.86 seems to stop reading from one of its dns sockets after a period of time under load
Geert Stappers
stappers at stappers.nl
Sun Apr 24 09:50:27 UTC 2022
On Wed, Apr 20, 2022 at 07:38:52AM -0700, Tom Keddie via Dnsmasq-discuss wrote:
> Hi All,
>
> We've been trying to track down some connectivity issues on unreleased
> firmware for some of the openwrt based routers we sell
Welcome to this mailinglist.
> and have narrowed it to a dns issue.
I do read that as a _possible_ DNS issue. ;-)
> We're seeing dnsmasq 2.86 stop reading from one of its sockets after a
> variable period of time with many (~45) clients. We also saw this problem
> with dnsmasq 2.85 but on the single socket it opens.
>
> Upstream dns still works as does resolution on 127.0.0.1 but resolution on
> the bridged lan interface, 192.168.1.1, is backed up and not responding.
>
> We do see some topology change events on the bridge but the interface has
> been up since first boot and still responds to ping.
>
> Any suggestions are welcome.
I do read that as
"You have my awareness that I aim for win-win"
> I did go through the pending commits for 2.87
> but didn't see anything relevant.
> I've included everything that I think might be useful here.
It lacks the "why". I'll try to explain what I mean / miss
> Thanks,
> Tom Keddie
>
> root:~# netstat -anu
I see the `netstat -anu`, but I don't see why this command
and neither why this command as first.
> Active Internet connections (servers and established)
> Proto Recv-Q Send-Q Local Address Foreign Address State
> udp 0 0 127.0.0.1:50032 0.0.0.0:*
> udp 0 0 127.0.0.1:38000 0.0.0.0:*
> udp 0 0 127.0.0.1:48000 0.0.0.0:*
> udp 0 0 127.0.0.1:37000 0.0.0.0:*
> udp 0 0 127.0.0.1:58000 0.0.0.0:*
> udp 0 0 0.0.0.0:51376 0.0.0.0:*
> udp 0 0 127.0.0.1:58032 0.0.0.0:*
> udp 0 0 127.0.0.1:46000 0.0.0.0:*
> udp 0 0 127.0.0.1:45000 0.0.0.0:*
> udp 0 0 127.0.0.1:44000 0.0.0.0:*
> udp 0 0 127.0.0.1:45032 0.0.0.0:*
> udp 0 0 127.0.0.1:43000 0.0.0.0:*
> udp 0 0 127.0.0.1:44032 0.0.0.0:*
> udp 0 0 127.0.0.1:42000 0.0.0.0:*
> udp 0 0 127.0.0.1:52000 0.0.0.0:*
> udp 0 0 127.0.0.1:42032 0.0.0.0:*
> udp 0 0 71.192.91.235:53 0.0.0.0:*
> udp 0 0 127.0.0.1:53 0.0.0.0:*
> udp 183552 213248 192.168.1.1:53 0.0.0.0:*
> udp 0 0 127.0.0.1:51000 0.0.0.0:*
> udp 0 0 127.0.0.1:52032 0.0.0.0:*
> udp 0 0 0.0.0.0:67 0.0.0.0:*
> udp 0 0 127.0.0.1:50000 0.0.0.0:*
>
>
> root:~# nslookup minim.co 75.75.75.75
Showing that 75.75.75.75 is working as DNS
(75.75.75.75 is configured as upstream DNS)
> Server: 75.75.75.75
> Address: 75.75.75.75:53
>
> Non-authoritative answer:
> Name: minim.co
> Address: 18.215.222.49
>
> Non-authoritative answer:
>
> root:~# nslookup minim.co 75.75.76.76
Showing that 75.75.76.76 is working as DNS
(75.75.76.76 is configured as upstream DNS)
> Server: 75.75.76.76
> Address: 75.75.76.76:53
>
> Non-authoritative answer:
>
> Non-authoritative answer:
> Name: minim.co
> Address: 18.215.222.49
>
> root:~# nslookup minim.co 127.0.0.1
Showing that 127.0.0.1 is working as DNS
> Server: 127.0.0.1
> Address: 127.0.0.1:53
>
> Non-authoritative answer:
> Name: minim.co
> Address: 18.215.222.49
>
> Non-authoritative answer:
>
> root:~# nslookup minim.co 192.168.1.1
The non-working DNS
> ;; connection timed out; no servers could be reached
>
> root:~# ps w | grep dns
Probably: dnsmasq is running and the configuration file it is using.
> 6268 dnsmasq 2260 S /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf.cfg01411c -k -x /var/run/dnsmasq/dnsmasq.cfg01411c.pid
> root:~# cat /var/etc/dnsmasq.conf.cfg01411c
Show content of configuration file
> # auto-generated config file from /etc/config/dhcp
> conf-file=/etc/dnsmasq.conf
> dhcp-authoritative
> domain-needed
> localise-queries
> read-ethers
> enable-ubus=dnsmasq
> expand-hosts
> bind-dynamic
> local-service
> edns-packet-max=1232
> domain=lan
> local=/lan/
> addn-hosts=/tmp/hosts
> dhcp-leasefile=/tmp/dhcp.leases
> resolv-file=/tmp/resolv.conf.d/resolv.conf.auto
> stop-dns-rebind
> rebind-localhost-ok
> dhcp-broadcast=tag:needs-broadcast
> conf-dir=/tmp/dnsmasq.d
> user=dnsmasq
> group=dnsmasq
> dhcp-host=00:0a:cd:32:6e:e9,192.168.1.193,Multisensor-Database
> dhcp-ignore-names=tag:dhcp_bogus_hostname
> conf-file=/usr/share/dnsmasq/dhcpbogushostname.conf
> bogus-priv
> conf-file=/usr/share/dnsmasq/rfc6761.conf
> dhcp-range=set:lan,192.168.1.10,192.168.1.254,255.255.255.0,24h
>
> root:~# cat /tmp/resolv.conf.d/resolv.conf.auto
Showing "/etc/resolv.conf"
> # Interface wan
> nameserver 75.75.75.75
> nameserver 75.75.76.76
> search hsd1.ma.comcast.net.
> root:~# cat /tmp/dhcp.leases
Might be to back the "many (~45) clients" statement
> 1650547228 16:27:11:bd:95:42 192.168.1.208 OnePlus-Nord-N200-5G 01:16:27:11:bd:95:42
> 1650510491 48:98:ca:9b:e3:cc 192.168.1.157 KNOK 01:48:98:ca:9b:e3:cc
> 1650541991 12:0d:6e:11:0e:f1 192.168.1.222 Dennis-s-S21-Ultra 01:12:0d:6e:11:0e:f1
> 1650535710 72:20:71:2f:c8:2f 192.168.1.123 * 01:72:20:71:2f:c8:2f
> 1650533880 26:26:22:df:cb:e3 192.168.1.236 Galaxy-S10 01:26:26:22:df:cb:e3
> 1650535136 00:0a:cd:32:6e:e9 192.168.1.193 Multisensor-Database 01:00:0a:cd:32:6e:e9
> 1650491917 80:c1:6e:f3:f7:4c 192.168.1.182 * 01:80:c1:6e:f3:f7:4c
> 1650534737 a4:83:e7:5a:49:33 192.168.1.99 ZoomDev 01:a4:83:e7:5a:49:33
> 1650534463 6c:94:f8:a1:b4:fb 192.168.1.174 Denniss-iPad 01:6c:94:f8:a1:b4:fb
> 1650548813 0c:cb:85:f7:e5:81 192.168.1.62 MH7601 *
> 1650490643 0c:cb:85:f7:e5:80 192.168.1.61 * *
> 1650548775 0c:cb:85:f7:e5:7d 192.168.1.58 * *
> 1650537688 08:5b:d6:81:99:99 192.168.1.155 DESKTOP-QPFCJMS 01:08:5b:d6:81:99:99
> 1650546045 0a:35:2a:f3:bb:6a 192.168.1.161 Pixel-3-XL 01:0a:35:2a:f3:bb:6a
> 1650534279 48:98:ca:2a:5c:d0 192.168.1.206 CORI 01:48:98:ca:2a:5c:d0
> 1650532675 4c:66:41:63:a3:ce 192.168.1.212 Galaxy-S7 01:4c:66:41:63:a3:ce
> 1650532683 00:b5:d0:fc:cd:b5 192.168.1.16 Galaxy-Tab-A-2016 01:00:b5:d0:fc:cd:b5
> 1650544456 00:51:ed:3b:53:b2 192.168.1.243 LGwebOSTV 01:00:51:ed:3b:53:b2
> 1650547147 40:06:a0:a7:dc:a4 192.168.1.41 4006A0A7DCA4-mysimplelink *
> 1650545297 34:20:03:ed:e2:e3 192.168.1.190 * *
> 1650546997 5c:f3:70:26:7b:3f 192.168.1.30 DESKTOP-R3VI2U2 01:5c:f3:70:26:7b:3f
> 1650490270 0c:cb:85:f7:e5:7c 192.168.1.57 * *
> 1650546786 34:97:f6:b6:9b:01 192.168.1.127 DESKTOP-MVQ4Q7J 01:34:97:f6:b6:9b:01
> 1650548017 5c:f3:70:26:7b:67 192.168.1.70 DESKTOP-59AGI93 01:5c:f3:70:26:7b:67
> 1650546628 28:b2:bd:1a:03:06 192.168.1.186 kaplan-laptop 01:28:b2:bd:1a:03:06
> 1650545301 34:20:03:ed:d1:40 192.168.1.84 * *
> 1650546024 38:f9:d3:5e:02:e1 192.168.1.165 * 01:38:f9:d3:5e:02:e1
> 1650546052 8c:85:90:4f:35:12 192.168.1.185 DVTs-MBP 01:8c:85:90:4f:35:12
> 1650545303 34:20:03:ed:f1:6f 192.168.1.139 * *
> 1650547347 60:dd:8e:d3:d1:6e 192.168.1.211 DVT-Test 01:60:dd:8e:d3:d1:6e
> 1650545313 34:20:03:ed:d4:f1 192.168.1.78 * *
> 1650545301 34:20:03:ed:ea:22 192.168.1.244 * *
> 1650533137 48:43:dd:6b:86:50 192.168.1.221 * *
> 1650533886 dc:f5:05:13:2f:51 192.168.1.184 P12100384 01:dc:f5:05:13:2f:51
> 1650545293 34:20:03:ed:8d:7a 192.168.1.125 * *
> 1650533065 b0:fc:0d:7a:80:b2 192.168.1.67 amazon-533fd135f 01:b0:fc:0d:7a:80:b2
> 1650533016 d0:c5:d3:65:17:47 192.168.1.60 C72926881 01:d0:c5:d3:65:17:47
> 1650510564 b4:e6:2d:03:b5:32 192.168.1.130 Etekcity-Outlet *
> 1650546031 d4:25:8b:94:c3:de 192.168.1.251 * 01:d4:25:8b:94:c3:de
> 1650545211 48:98:ca:f9:3a:d2 192.168.1.252 C58804806 01:48:98:ca:f9:3a:d2
> 1650546123 b8:31:b5:ba:64:c9 192.168.1.18 XBOXONE 01:b8:31:b5:ba:64:c9
> 1650534777 48:98:ca:f4:c2:bf 192.168.1.232 C58213759 01:48:98:ca:f4:c2:bf
> 1650533023 bc:d7:d4:32:0f:ea 192.168.1.89 RokuUltra *
> 1650541196 60:f8:1d:ab:eb:40 192.168.1.249 build-001 01:60:f8:1d:ab:eb:40
> 1650550011 3c:06:30:3b:74:04 192.168.1.54 MacBook-Pro 01:3c:06:30:3b:74:04
> root:~# ip addr
Yes, but why?
What are you trying to tell?? As in: What do you expect us to see???
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> valid_lft forever preferred_lft forever
> 2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop qlen 1000
> link/ether 22:ca:00:55:d9:ee brd ff:ff:ff:ff:ff:ff
> 3: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop qlen 32
> link/ether ba:84:9c:1e:ec:ac brd ff:ff:ff:ff:ff:ff
> 4: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop qlen 32
> link/ether 8e:9e:7d:ee:3a:6c brd ff:ff:ff:ff:ff:ff
> 5: gre0 at NONE: <NOARP> mtu 1476 qdisc noop qlen 1000
> link/gre 0.0.0.0 brd 0.0.0.0
> 6: gretap0 at NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop qlen 1000
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 7: erspan0 at NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop qlen 1000
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 8: sit0 at NONE: <NOARP> mtu 1480 qdisc noop qlen 1000
> link/sit 0.0.0.0 brd 0.0.0.0
> 9: ip6gre0 at NONE: <NOARP> mtu 1448 qdisc noop qlen 1000
> link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
> 10: archer: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
> 11: bcmsw: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
> link/ether 0c:cb:85:f7:e5:a8 brd ff:ff:ff:ff:ff:ff
> 12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
> link/ether 0c:cb:85:f7:e5:a9 brd ff:ff:ff:ff:ff:ff
> inet 71.192.91.235/22 brd 71.192.91.255 scope global eth0
> valid_lft forever preferred_lft forever
> 13: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-lan qlen 1000
> link/ether 0c:cb:85:f7:e5:a8 brd ff:ff:ff:ff:ff:ff
> 14: wl0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-lan qlen 1000
> link/ether 0c:cb:85:f7:e5:ab brd ff:ff:ff:ff:ff:ff
> 15: wl1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-lan qlen 1000
> link/ether 0c:cb:85:f7:e5:aa brd ff:ff:ff:ff:ff:ff
> 18: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
> link/ether 0c:cb:85:f7:e5:a8 brd ff:ff:ff:ff:ff:ff
> inet 192.168.1.1/24 brd 192.168.1.255 scope global br-lan
> valid_lft forever preferred_lft forever
> 19: wl1.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-lan qlen 1000
> link/ether 72:cb:85:f7:e5:ab brd ff:ff:ff:ff:ff:ff
> 29: wds1.1.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
> master br-lan qlen 1000
> link/ether 72:cb:85:f7:e5:ab brd ff:ff:ff:ff:ff:ff
> 30: wds1.1.2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-lan qlen 1000
> link/ether 72:cb:85:f7:e5:ab brd ff:ff:ff:ff:ff:ff
> root:~# brctl show
Why?
> bridge name bridge id STP enabled interfaces
> br-lan 7000.0ccb85f7e5a8 yes eth1
> wds1.1.1
> wds1.1.2
> wl0
> wl1
> wl1.1
> root:~# dmesg | grep br-lan
Why?
> [ 2257.008455] br-lan: port 1(eth1) entered blocking state
> [ 2257.013706] br-lan: port 1(eth1) entered listening state
> [ 2259.046965] br-lan: port 1(eth1) entered learning state
> [ 2261.094953] br-lan: port 1(eth1) entered forwarding state
> [ 2261.100376] br-lan: topology change detected, propagating
> [ 2271.342450] br-lan: port 1(eth1) entered disabled state
> [ 2279.536780] br-lan: port 1(eth1) entered blocking state
> [ 2279.542056] br-lan: port 1(eth1) entered listening state
> [ 2281.574810] br-lan: port 1(eth1) entered learning state
> [ 2283.622807] br-lan: port 1(eth1) entered forwarding state
> [ 2283.628240] br-lan: topology change detected, propagating
> [ 4835.714957] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [ 6689.781276] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [10165.333204] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [20921.866775] br-lan: port 5(wds1.1.1) entered disabled state
> [20921.883309] br-lan: port 5(wds1.1.1) entered disabled state
> [20922.512373] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [20923.718768] br-lan: port 5(wds1.1.1) entered blocking state
> [20923.725229] br-lan: port 5(wds1.1.1) entered disabled state
> [20923.737857] br-lan: port 5(wds1.1.1) entered blocking state
> [20923.743492] br-lan: port 5(wds1.1.1) entered listening state
> [20925.766397] br-lan: port 5(wds1.1.1) entered learning state
> [20927.814380] br-lan: port 5(wds1.1.1) entered forwarding state
> [20927.820154] br-lan: topology change detected, propagating
> [20930.280812] br-lan: port 5(wds1.1.1) received tcn bpdu
> [20930.285982] br-lan: topology change detected, propagating
> [20948.714635] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [26002.362865] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [26003.804245] br-lan: port 6(wds1.1.2) entered disabled state
> [26003.821039] br-lan: port 6(wds1.1.2) entered disabled state
> [26005.660527] br-lan: port 6(wds1.1.2) entered blocking state
> [26005.667707] br-lan: port 6(wds1.1.2) entered disabled state
> [26005.680231] br-lan: port 6(wds1.1.2) entered blocking state
> [26005.685845] br-lan: port 6(wds1.1.2) entered listening state
> [26007.703587] br-lan: port 6(wds1.1.2) entered learning state
> [26009.751572] br-lan: port 6(wds1.1.2) entered forwarding state
> [26009.757342] br-lan: topology change detected, propagating
> [26010.603790] br-lan: port 6(wds1.1.2) received tcn bpdu
> [26010.608954] br-lan: topology change detected, propagating
> [26903.288550] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [29308.403115] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [44074.437481] br-lan: port 5(wds1.1.1) entered disabled state
> [44074.453464] br-lan: port 5(wds1.1.1) entered disabled state
> [44075.721563] br-lan: port 5(wds1.1.1) entered blocking state
> [44075.727494] br-lan: port 5(wds1.1.1) entered disabled state
> [44075.740713] br-lan: port 5(wds1.1.1) entered blocking state
> [44075.746331] br-lan: port 5(wds1.1.1) entered listening state
> [44077.811280] br-lan: port 5(wds1.1.1) entered learning state
> [44079.859259] br-lan: port 5(wds1.1.1) entered forwarding state
> [44079.865029] br-lan: topology change detected, propagating
> [44082.572293] br-lan: port 5(wds1.1.1) received tcn bpdu
> [44082.577454] br-lan: topology change detected, propagating
> [44221.999541] br-lan: port 6(wds1.1.2) entered disabled state
> [44222.022059] br-lan: port 6(wds1.1.2) entered disabled state
> [44223.231419] br-lan: port 6(wds1.1.2) entered blocking state
> [44223.237146] br-lan: port 6(wds1.1.2) entered disabled state
> [44223.249414] br-lan: port 6(wds1.1.2) entered blocking state
> [44223.255030] br-lan: port 6(wds1.1.2) entered listening state
> [44225.265897] br-lan: port 6(wds1.1.2) entered learning state
> [44227.313875] br-lan: port 6(wds1.1.2) entered forwarding state
> [44227.319647] br-lan: topology change detected, propagating
> [44228.227914] br-lan: port 6(wds1.1.2) received tcn bpdu
> [44228.233077] br-lan: topology change detected, propagating
> [44228.625036] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [55550.135997] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [56437.513600] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [56900.600612] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [57338.311331] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [57399.986991] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> [57545.345542] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [58224.520851] br-lan: received packet on wl0 with own address as source address (addr:0c:cb:85:f7:e5:ab, vlan:0)
> [61335.029068] br-lan: received packet on wl1 with own address as source address (addr:0c:cb:85:f7:e5:aa, vlan:0)
> root:~# ping 192.168.1.1
ping --count 5 192.168.1.1 # to show interface still responds to ping.
> PING 192.168.1.1 (192.168.1.1): 56 data bytes
> 64 bytes from 192.168.1.1: seq=0 ttl=64 time=0.181 ms
> ^C
> --- 192.168.1.1 ping statistics ---
> 1 packets transmitted, 1 packets received, 0% packet loss
> round-trip min/avg/max = 0.181/0.181/0.181 ms
Advice for digging deeper:
watch 'netstat -anu | grep 192.168.1.1:53'
--log-queries=extra
Groeten
Geert Stappers
--
Silence is hard to parse
More information about the Dnsmasq-discuss
mailing list