[Dnsmasq-discuss] rr_on_list stuck in infinite loop, dnsmasq unresponsive

[Simon Kelley]

https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=89aad014685161318318737dc0e350ee4dae982d

should fix this.


Simon.

On 19/02/2024 23:16, Simon Kelley wrote:
> Wow, excellent bug report, thank you.
> 
> Took me straight to the stupid error.
> 
> src/dbus.c around line 834. The code block controlled by "if (!done)" 
> should include the line "done = 1;" Same thing below for filter AAAA.
> 
> I'll push the patch directly.
> 
> Cheers,
> 
> Simon.
> 
> 
> On 19/02/2024 21:29, Clayton Craft wrote:
>> It seems like sometimes the rrlist given to rr_on_list can be a 
>> circular linked
>> list:
>>
>>      0x0000aaaabbf57044 in rr_on_list (list=0xaaaabbfe1990 <list>, 
>> rr=5) at util.c:120
>>      120       while (list)
>>      (gdb) p list
>>      $1 = (struct rrlist *) 0xaaaabbfe1990 <list>
>>      (gdb) p list.next
>>      $3 = (struct rrlist *) 0xaaaabbfe1980 <list>
>>      (gdb) p list.next.next
>>      $4 = (struct rrlist *) 0xaaaabbfe1990 <list>
>>
>> This causes rr_on_list to get stuck in an infinite loop, and dnsmasq 
>> to consume
>> 100% CPU and be completely unresponsive. I generated a flamegraph of 
>> dnsmasq and
>> found that it was stuck here, then attached gdb to confirm.
>>
>> The conditions to trigger this aren't well understood by me... it 
>> seems to
>> happen when we use dnsmasq's dbus interface to toggle filtering 
>> ("set_filter A
>> bool"). But I've had trouble reproducing it manually. We hit it a lot 
>> when using
>> a NetworkManager dispatcher script to apply filtering in dnsmasq 
>> conditionally.
>>
>> The issue does *not* go away when I revert 3de7289.
>>
>> If rrlist is a circular list, then rr_on_list should have an escape if 
>> head was
>> already visited or ? I don't understand the dnsmasq code enough to 
>> have any real
>> suggestions for how to proceed with fixing this :)
>>
>> Full bt:
>>
>> (gdb) bt full
>> #0  0x0000aaaad9077028 in rr_on_list (
>>      list=0xaaaad9101990 <list>, rr=5) at util.c:120
>> No locals.
>> #1  0x0000aaaad90cd718 in rrfilter (header=0xffffa56e3580,
>>      plen=0xfffff663bc80, mode=2) at rrfilter.c:233
>>          pstart = 0xffffa56e35ac "\300\f"
>>          type = 5
>>          class = 1
>>          rrs = 0xffffa5566b40
>>          rr_sz = 12
>>          p = 0xffffa56e35c0 "\3008"
>>          rr_found = 0
>>          i = 0
>>          rdlen = 8
>>          qtype = 28
>>          qclass = 1
>>          chop_an = 0
>>          chop_ns = 0
>>          chop_ar = 0
>> #2  0x0000aaaad9089c40 in process_reply (
>>      header=0xffffa56e3580, now=1708377610,
>>      cache_secure=0, bogusanswer=0, ad_reqd=0, do_bit=0, 
>> added_pheader=0, query_source=0xffffa56d1240,
>>      limit=0xffffa56e3a50 "", ede=-1) at forward.c:848
>>          pheader = 0x0
>>          sizep = 0x0
>>          ipsets = 0x0
>>          nftsets = 0x0
>>          is_sign = 0
>>          rcode = 0
>>          plen = 281473457238616
>> #3  0x0000aaaad908b68c in return_reply (now=1708377610, 
>> forward=0xffffa56d1240, header=0xffffa56e3580, n=92, status=524288)
>>      at forward.c:1382
>>          check_rebind = 0
>>          no_cache_dnssec = 0
>>          cache_secure = 0
>>          bogusanswer = 0
>>          nn = 187650762387748
>>          ede = -1
>> #4  0x0000aaaad908b21c in reply_query (fd=10, now=1708377610) at 
>> forward.c:1301
>>          header = 0xffffa56e3580
>>          serveraddr = {sa = {sa_family = 2, sa_data = 
>> "\0005\300\000\000\001\000\000\000\000\000\000\000"}, in = {
>>              sin_family = 2, sin_port = 13568, sin_addr = {s_addr = 
>> 16777408}, sin_zero = "\000\000\000\000\000\000\000"},
>>            in6 = {sin6_family = 2, sin6_port = 13568, sin6_flowinfo = 
>> 16777408, sin6_addr = {__in6_union = {
>>                  __s6_addr = 
>> "\000\000\000\000\000\000\000\000\224\202\t٪\252\000", __s6_addr16 = 
>> {0, 0, 0, 0, 33428, 55561,
>>                    43690, 0}, __s6_addr32 = {0, 0, 3641279124, 
>> 43690}}}, sin6_scope_id = 0}}
>>          forward = 0xffffa56d1240
>>          addrlen = 16
>>          n = 92
>>          server = 0xffffa5566e70
>>          hash = 0xffffa55059b0
>>          first = 32
>>          last = 33
>>          c = 32
>> #5  0x0000aaaad90982c4 in check_dns_listeners (now=1708377610) at 
>> dnsmasq.c:1831
>>          serverfdp = 0x0
>>          listener = 0xe0af57cb29b80071
>>          rfl = 0xfffff663bf80
>>          i = 0
>>          pipefd = {-653704516, 43690}
>> #6  0x0000aaaad9096c84 in main (argc=12, argv=0xfffff663c1d8) at 
>> dnsmasq.c:1269
>>          timeout = -1
>>          now = 1708377610
>>          sigact = {__sa_handler = {sa_handler = 0x1, sa_sigaction = 
>> 0x1}, sa_mask = {__bits = {0, 187651085026544,
>>                281473457229824, 281474815476000, 281473456612332, 
>> 187651085026504, 281473457229824, 281474815476032,
>>                281473456612716, 187651085026504, 281473457238016, 
>> 281474815476096, 281473456885404, 281473457238016,
>>                281474815476184, 224}}, sa_flags = 0, sa_restorer = 
>> 0xffffa56eb000 <env_alloced_n>}
>>          if_tmp = 0x0
>>          piperead = 7
>>          pipefd = {7, 8}
>>          err_pipe = {0, -1}
>>          ent_pw = 0xffffa56eb4e0 <pw>
>>          script_uid = 0
>>          script_gid = 0
>>          gp = 0xffffa56eb4a0 <gr>
>>          i = 20
>>          max_fd = 1024
>>          baduser = 0x0
>>          log_err = 0
>>
>> -Clayton
>>
>>
>> _______________________________________________
>> Dnsmasq-discuss mailing list
>> Dnsmasq-discuss at lists.thekelleys.org.uk
>> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss