[Dnsmasq-discuss] Extremely long startup times when using libidn2 (and proposed fix)

[Simon Kelley]

On 06/09/2021 14:27, Gustaf Ullberg wrote:
> Hi Simon and dnsmasq contributors,
> 
> I am running dnsmasq with a blocklist from
> https://github.com/notracking/hosts-blocklists/blob/master/dnsmasq/dnsmasq.blacklist.txt
> <https://github.com/notracking/hosts-blocklists/blob/master/dnsmasq/dnsmasq.blacklist.txt>
> 
> I have noticed that building dnsmasq with libidn2 support (which my
> distro does) can cause extreme slowdowns. The slowdowns seem to come
> from the call to idn2_to_ascii_lz in canonicalise() being very slow.
> 
> idn2_to_ascii_lz is run on every domain name in the blocklist to encode
> special characters, and this is surprisingly slow even when there are no
> special characters. I developed a patch (attached to this email) that
> checks a domain name for other characters than . - a-z 0-9. If any such
> character is found, the domain name will be encoded. If no such
> character is found the domain name will not be encoded (as encoding
> won't change it). This removes most of the overhead of using libidn2.
> Unless you find any problems with this approach, I wish the patch can be
> mainlined.
> 
> Some benchmarks on a Raspberry Pi (slow, but probably not an uncommon
> device for running dnsmasq) running ArchLinux and dnsmasq git master:
> 
> # Without libidn2: Acceptable speed
>> make
>> time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
> 
> real 0m3.699s
> user 0m3.468s
> sys 0m0.200s
> 
> 
> 
> # With libidn2: To slow to be usable
>> make COPTS="-DHAVE_LIBIDN2"
>> time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
> 
> real 1m6.921s
> user 0m59.509s
> sys 0m0.606s
> 
> 
> # With libidn2 and attached patch: Back to acceptable speed
>> git am 0001-Avoid-IDN-translations-when-not-needed.patch
>> make COPTS="-DHAVE_LIBIDN2"
>> time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
> 
> real 0m3.903s
> user 0m3.643s
> sys 0m0.219s


I see a similar ratio (but much faster absolute execution times) on my
Pentium laptop with the latest code.

Petr's implementation looks better, but the idea is a good one.

Will add to the 2.87 cycle.


Cheers,

Simon.

> 
> Best regards,
> Gustaf
> 
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
>