[Dnsmasq-discuss] Feature request = block-conf

[Simon Kelley]

On 27/03/2022 08:52, Ercolino de Spiacico wrote:
>  >> [ -f list.of.domain ] && { create the formatted configuration }
>  >>
>  >> would always fail at dnsmasq level if the list.of.domains doesn't exist
>  >> despite the file existence condition defined. I'm not sure this is 
> meant
>  >> to be and/or if there's a smart workaround that can be used.
>  >
>  > I don't understand this, could you give more details.
> 
> 
> Ok I have progressed on this point since. Basically I worked out the 
> dnsmasq is checking the script exit code. as I had something like this 
> in the script:
> 
> [ -f list.of.domain ] && { create the formatted configuration }
> 
> I just had to "force" the exit 0 to the command so:
> 
> [ -f list.of.domain ] && { create the formatted configuration } || { 
> exit 0; }
> 
> This makes the script returning exit 0 regardless and have --test 
> ignoring (rightly) any issue with that part of the script. Wanted so 
> good enough for me.
> 
> 
> 
> 
> 
>  > Note that having loaded the config, dnsmasq has to sort all the domains,
>  > which will be part of the time taken. Doing that once makes the lookups
>  > much faster.
> 
> 
> As part of the adblock.domain formatting I feed it to dnsmasq after a 
> sort -u so somehow it's already sorted, Could this internal sorting be 
> optionally skipped?
> 
> 

Not safely. If it's not sorted or sorted wrong, dnsmasq will break.

> 
> 
> 
>  > How long are you without DNS service?
>  >
> 
> 
> It's proportional to the number of records in the adblock file. For 
> large lists, on a fast router it can easy go into the 15+ seconds. This 
> happens relatively frequently on a router where external conditions 
> restart dnsmasq "just in case". So answering your question in detail, 
> given a relatively basic scripting config:
> 
> #######
> DOMAINS="/tmp/adblock.domains"
> 
> /usr/bin/wget --no-check-certificate -T 15 -q -U "Mozilla/5.0 (X11; 
> Linux x86_64; rv:10.0) Gecko/20100101 Firefox/98.0.1" -O- 
> https://hosts.oisd.nl | grep -Ev 
> '^#.*|^!.*|^::|^\s*?$|^([a-f0-9:]+:+)+[a-f0-9]+' | grep -Eo 
> '((([a-zA-Z]{1,2})|([0-9]{1,2})|([a-zA-Z0-9]{1,2})|([a-zA-Z0-9][a-zA-Z0-9-]{1,61}[a-zA-Z0-9]))\.)+[a-zA-Z]{2,6}' 
> | sort -u > $DOMAINS
> 
> gzip /tmp/adblock.domains
> 
> echo "script-conf=/tmp/script.sh" >> /etc/dnsmasq.conf
> 
> echo -e "#!/bin/sh\nset -e\n[ -f ${DOMAINS} ] && /bin/zcat ${DOMAINS}.gz 
> | /bin/sed -e \"s:^:address=/:\" -e \"s:$:/:\" || exit 0" > /tmp/script.sh
> 
> chmod 777 /tmp/script.sh
> #######
> 
> At this point restarting should give you a magnitude of time. Of course 
> on a fast PC/VM this might not be that much, on a router where in the 
> best of the cases you have 2x 800/1000MHz the delay is surely noticeable.
> 
> This Internet list above (https://hosts.oisd.nl) is 40MB uncompressed, 
> the regex extracts domains-only so shrinking it to 60% of its original 
> size and the gzip compression shrinks much further. Decompressing and 
> scripting it up of course takes time.
> 
> 
All of that looks like stuff which can be done before stopping dnsmasq, 
right. SO how long it takes makes no difference to how long DNS and DHCP 
service is interrupted for?
> 
> 
> 
>  > One possible solution is to add an option to dnsmasq which causes it to
>  > send SIGTERM is a process-id _after_ reading config and _before_ opening
>  > network sockets. That would delay stopping the old dnsmasq process until
>  > the new one is almost ready to go. Would require some clever scripting
>  > in the init system or systemd to make it work.
> 
> 
> Right! Could you please share more details on this idea? It could be a 
> smart workaround indeed.
> 


When dnsmasq starts up, it does roughly the following things:

1) Read config files (including execute script-conf
2) organise the data from the config into in-memory data structures
3) Open listening sockets on DNS and DHCP ports.
4) Enter the event loop.

1) and 2) are likely to take an appreciaable amount of time with big 
block lists. 3) and 4) never do.

A restart of dnsmasq  consists of

1) send SIGTERM to existing dnsmasq process to cause it to halt.
2) Run new dnsmasq instance, which goes through the four steps above.

If reading the congfig takes time, that causes service interruption. 
Note that you can't do those two steps in the opposite order, since the 
old dnsmasq process will still be listening on the DNS and DHCP ports, 
and the new one will fail to start up.

If we add an option to dnsmasq which takes a process-ID and sends a 
SIGTERM to that process-ID as step 2.5, the old dnsmasq process can 
continue to run during the parsing of the options, then the it gets toen 
down before opening listening sockets.

The restart now just runs the new dnsmasq instance, passing the PID of 
the old dnsmasq instance if it exists.


Cheers,

Simon.

> Thanks
> 
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
>