[Dnsmasq-discuss] Feature request = block-conf

[Ercolino de Spiacico]

 > OK, but as you rightly pointed out a couple of messages ago, all these
 > operations are performed before the service dnsmasq restart.
 >
 > Because of this I have performed an additional test to complete the
 > picture:
 >
 > my usual 650K list of domains is now unzipped and I have simply scripted
 > the usual formatting with address=/domain/. So no compression involved
 > this time. e.g.
 >
 > /bin/cat /mnt/USB/adblock/adblock.domains | /bin/sed -e "s:^:address=/:"
 > -e "s:$:/:"
 >
 >
 > The service dnsmasq restart went from 15.something sec of the script+zip
 > version to 14.4 sec of this script only version.
 > root at sparrow:/tmp/mnt/USB/adblock# time service dnsmasq restart
 > 
................................................................................................................................................
 > Done.
 > real    0m 14.43s
 > user    0m 0.00s
 > sys     0m 0.00s
 >
 > What I was trying to say is that feeding pre-formatted config (as per
 > prev message posted here) takes 4.2 sec on the very same file.
 >
 > And you're right the aim here is to minimize storage hence RAM demand,
 > but the dnsmasq restart time must also stay within acceptable levels as
 > nobody wants the DNS resolution to be down for too long.
 >
 > So to summarize: Because the conf-script option works great on RAM
 > reduction but takes a big hit on the restart time, I was suggesting if a
 > different approach could/should be considered. Since we learnt from this
 > last test that the zip operation add only 1-2 sec of delay on the restart
 >
 > root at sparrow:/mnt/USB/adblock# time gzip -d 
adblock.domains.unloaded.gz
 > real    0m 1.76s
 > user    0m 0.67s
 > sys     0m 0.18s
 >
 > perhaps allowing the conf-file to process zipped content (internal zcat
 > or something) would suffice to achieve the desired result to minimize
 > storage demand and retain restart time within acceptable levels.
 > Allowing something like conf-block and import domain only is indeed
 > nice-to-have but secondary in my opinion compared to support for
 > compressed config:
 >
 > root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded
 > -rwxrwxrwx    1 root     root       14.3M Mar 31 18:42
 > root at sparrow:/mnt/USB/adblock# gzip adblock.domains.unloaded
 > root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded.gz
 > -rwxrwxrwx    1 root     root        4.2M Mar 31 18:42


Simon, it is not my intention to put pressure or else but I just wanted 
to give a final feedback on this topic. The conf-script is not usable on 
a router for large lists/config. It simply takes dnsmasq far too long to 
restart. There are a series of event that restart dnsmasq from a simple 
Save Config from the GUI to an automatic event e.g. a multiWAN swap due 
to external factors and anything in between. on a 1M record blocklist it 
takes dnsmasq almost 30 before before it can resolve domains again.

On one side I'm grateful this was considered by the dnsmasq team, on the 
other side I still believe (as per message above) that allowing dnsmasq 
to read natively a conf-file in .gz would actually be the simplest and 
possibly most effective of the solutions to resolve the large list 
issue. On plain config (conf-file) even a huge file takes dnsmasq 5-6 
seconds to restart. The gzip operation on such file are about 1 sec on 
top which is totally an acceptable compromise. Finally the size of a 1M 
records conf-file compresses down from 40M to 8Mb.
A win win.

I'm not sure if you still have interest in tackling this topic but I 
felt a feedback is expected as a minimum given all the work done so far. 
It would be good to know either ways if you don't mind.

Thanks!