[Dnsmasq-discuss] [PATCH] Optimize speed on massive server=/.../... records

[Simon Kelley]

Patch tweaked and applied.

Given the rate of good changes coming in, I'm not going to make the 
final 2.88 release this weekend. Let's give it  a few more days to settle.


Cheers,

Simon.


On 25/11/2022 13:11, Petr Menšík wrote:
> When looking what this change did, I have noticed mark_servers() cleanup 
> of local_domains is using serv->next after it has freed serv. Use 
> additional variable just like in cleanup_servers().
> 
> Patch attached.
> 
> On 11/21/22 23:22, Simon Kelley wrote:
>> Thanks for this. It was in my mind that vary large number of domains 
>> would be --local=/domain/ or --address=/domain/, not forwarding to 
>> servers.
>>
>> I've applied something that looks very like your patch, but with 
>> cosmetic code changes.
>>
>> Cheers,
>>
>> Simon.
>>
>>
>> On 20/11/2022 05:50, Ye Zhou wrote:
>>> Hi all,
>>>
>>> I'm attaching a patch to optimize a speed issue introduced in version 
>>> 2.86.
>>>
>>> I have two ISP upstreams and need to forward different sites to 
>>> different ISP's DNS providers. For example:
>>>
>>> server=/meituan.com/114.114.114.114 <http://meituan.com/114.114.114.114>
>>> ... (lots of records)
>>> server=/taobao.com/223.5.5.5 <http://taobao.com/223.5.5.5>
>>> ... (lots of records)
>>>
>>> It works well before v2.86. Since v2.86 the configuration load time 
>>> becomes extremely long (more than 1 minutes to load all server 
>>> records). The time consuming part is inside the rewritten 
>>> domain-match.c. When adding a new server record, the code will 
>>> traverse all existing records so the configuration load becomes 
>>> quadratic time complexity. The issue still persists on v2.88rc3.
>>> This patch will optimize the config load time by bypassing the time 
>>> consuming code block. During the config load mark_servers() will 
>>> never be called so does not need to waste time on the record 
>>> traversal and re-order part.
>>>
>>> Before:
>>> dnsmasq --test  77.50s user 0.30s system 99% cpu 1:17.84 total
>>> After:
>>> dnsmasq --test  0.16s user 0.02s system 99% cpu 0.188 total
>>>
>>> https://gist.github.com/zhouye/adfd509f51645d314f53992331449c45 
>>> <https://gist.github.com/zhouye/adfd509f51645d314f53992331449c45>
>>>
>>>
>>> _______________________________________________
>>> Dnsmasq-discuss mailing list
>>> Dnsmasq-discuss at lists.thekelleys.org.uk
>>> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
>>
>> _______________________________________________
>> Dnsmasq-discuss mailing list
>> Dnsmasq-discuss at lists.thekelleys.org.uk
>> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
> 
> 
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss