[Dnsmasq-discuss] Extremely long startup times when using libidn2 (and proposed fix)

Petr Menšík pemensik at redhat.com
Wed Sep 8 13:03:41 UTC 2021


I think your check should also accept uppercase ASCII letters. Anyway,
similar check is already done in check_names, which is there to skip
names containing underscore with older libidn2 versions. I guess it
could return 2 also in case ascii-only characters were detected, instead
of checking the name again in another loop.

Attached alternative change, which would process only names not only
ascii names. Changes check_names to return 2 when IDN should be used.
Printing ascii names should be safe, even when they contain characters
not allowed by hostnames. Such as _, +, = or whatever garbage is
present. As long as it is readable in logs, it should not matter.

How many lines does your dnsmasq.blacklist.txt contain? Those
differences are significant. Maybe bug should be filled on libidn2.
Conversion from ascii-only name to ascii name should not take too long
even if it was called.

On 9/6/21 3:27 PM, Gustaf Ullberg wrote:
> Hi Simon and dnsmasq contributors,
>
> I am running dnsmasq with a blocklist from
> https://github.com/notracking/hosts-blocklists/blob/master/dnsmasq/dnsmasq.blacklist.txt
> <https://github.com/notracking/hosts-blocklists/blob/master/dnsmasq/dnsmasq.blacklist.txt>
>
> I have noticed that building dnsmasq with libidn2 support (which my
> distro does) can cause extreme slowdowns. The slowdowns seem to come
> from the call to idn2_to_ascii_lz in canonicalise() being very slow.
>
> idn2_to_ascii_lz is run on every domain name in the blocklist to
> encode special characters, and this is surprisingly slow even when
> there are no special characters. I developed a patch (attached to this
> email) that checks a domain name for other characters than . - a-z
> 0-9. If any such character is found, the domain name will be encoded.
> If no such character is found the domain name will not be encoded (as
> encoding won't change it). This removes most of the overhead of using
> libidn2. Unless you find any problems with this approach, I wish the
> patch can be mainlined.
>
> Some benchmarks on a Raspberry Pi (slow, but probably not an uncommon
> device for running dnsmasq) running ArchLinux and dnsmasq git master:
>
> # Without libidn2: Acceptable speed
> > make
> > time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
>
> real 0m3.699s
> user 0m3.468s
> sys 0m0.200s
>
>
>
> # With libidn2: To slow to be usable
> > make COPTS="-DHAVE_LIBIDN2"
> > time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
>
> real 1m6.921s
> user 0m59.509s
> sys 0m0.606s
>
>
> # With libidn2 and attached patch: Back to acceptable speed
> > git am 0001-Avoid-IDN-translations-when-not-needed.patch
> > make COPTS="-DHAVE_LIBIDN2"
> > time ./src/dnsmasq -C dnsmasq.blacklist.txt --test
> dnsmasq: syntax check OK.
>
> real 0m3.903s
> user 0m3.643s
> sys 0m0.219s
>
> Best regards,
> Gustaf
>
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss

-- 
Petr Menšík
Software Engineer
Red Hat, http://www.redhat.com/
email: pemensik at redhat.com
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20210908/5235440a/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Alternative-change-to-skip-ascii-only-names-IDN-proc.patch
Type: text/x-patch
Size: 2840 bytes
Desc: not available
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20210908/5235440a/attachment.bin>


More information about the Dnsmasq-discuss mailing list