[Dnsmasq-discuss] Feature request = block-conf

[Ercolino de Spiacico]

> My understanding is that the script-conf feature was to be used to 
> reduce filesystem storage usage, ie instead of a conf-file full of lines 
> like
> 
> address=/domain1.com/
> address=/domain2.com/
> 
> Juts the list of domains could be stored, in compressed form, and then 
> they would be decompressed and decorated with the address=/...../ stuff 
> on the fly before being fed into the dnsmasq configuration parser.
> 
> The  scripts you posted earlier seemed to
> 
> 1) Download the block list.
> 2) compress it
> 3) feed it to dnsmasq which decompresses it
> 
> Which is the worst of all possible worlds, since it uses more storage 
> for the compressed AND uncompressed versions, and more CPU to the 
> compression and decompression. It also make the down-time depend on how 
> fast the block list downloads.


OK, but as you rightly pointed out a couple of messages ago, all these 
operations are performed before the service dnsmasq restart.

Because of this I have performed an additional test to complete the 
picture:

my usual 650K list of domains is now unzipped and I have simply scripted 
the usual formatting with address=/domain/. So no compression involved 
this time. e.g.

/bin/cat /mnt/USB/adblock/adblock.domains | /bin/sed -e "s:^:address=/:" 
-e "s:$:/:"


The service dnsmasq restart went from 15.something sec of the script+zip 
version to 14.4 sec of this script only version.
root at sparrow:/tmp/mnt/USB/adblock# time service dnsmasq restart
................................................................................................................................................
Done.
real    0m 14.43s
user    0m 0.00s
sys     0m 0.00s

What I was trying to say is that feeding pre-formatted config (as per 
prev message posted here) takes 4.2 sec on the very same file.

And you're right the aim here is to minimize storage hence RAM demand, 
but the dnsmasq restart time must also stay within acceptable levels as 
nobody wants the DNS resolution to be down for too long.

So to summarize: Because the conf-script option works great on RAM 
reduction but takes a big hit on the restart time, I was suggesting if a 
different approach could/should be considered. Since we learnt from this 
last test that the zip operation add only 1-2 sec of delay on the restart

root at sparrow:/mnt/USB/adblock# time gzip -d adblock.domains.unloaded.gz
real    0m 1.76s
user    0m 0.67s
sys     0m 0.18s

perhaps allowing the conf-file to process zipped content (internal zcat 
or something) would suffice to achieve the desired result to minimize 
storage demand and retain restart time within acceptable levels. 
Allowing something like conf-block and import domain only is indeed 
nice-to-have but secondary in my opinion compared to support for 
compressed config:

root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded
-rwxrwxrwx    1 root     root       14.3M Mar 31 18:42
root at sparrow:/mnt/USB/adblock# gzip adblock.domains.unloaded
root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded.gz
-rwxrwxrwx    1 root     root        4.2M Mar 31 18:42


My 2 cents