[Dnsmasq-discuss] Feature request = block-conf

[Simon Kelley]

On 30/03/2022 12:13, Ercolino de Spiacico wrote:
>> It looks like your script which downloads the blocked domains file and 
>> compresses it takes 15s, then dnsmasq takes 15s to uncompress the list 
>> and load it into memory and sort.
>>
>> The first delay can be solved by doing the download before stopping 
>> the old dnsmasq process. The second is amenable the new option to 
>> SIGTERM the old dnsmasq _after_ parsing the new config.
> 
> I run some additional test to verify the impact. I got the very same 
> list (650K records) unzipped and pre-formatted (address=/$domain/) to be 
> used by the standard conf-file directive. As we used to do before the 
> conf-script was introduced. Here the result:
> 
> root at sparrow:/tmp/mnt/USB/adblock# time service dnsmasq restart
> ..........................................
> Done.
> real    0m 4.21s
> user    0m 0.00s
> sys     0m 0.01s
> 
> The amazing part is that there's no measurable delay in the change of 
> ownership from root to nobody. So 4.2 seconds and that's it which is 
> completely acceptable for this list size. But then we're back to square 
> 1 before this conversation ever happened. So I'm starting to conclude 
> that the scripting part (script-conf) and it's interpretation 
> internally, despite being flexible in terms of wanted output, it doesn't 
> scale well on large block-files. Fair enough admittedly this has never 
> been the reason why dnsmasq was originally created.
> 
> This also make me think further about two (potentially even overlapping) 
> ways to progress:
> 
> A- Going back to my original idea to have a new conf-block (or 
> block-conf whichever) directive where any domain defined in the target 
> file will be blocked (only domains without formatting) dnsmasq would do 
> an internal interpretation of this list of domains based on the 
> directive. So smaller list size and no extra processing needed. This 
> perhaps could be an option to have the cake and eat it?
> 
> B- Allow the conf-block= and/or the existing conf-file= to read a zipped 
> source e.g.
> conf-block=z,/bla/bla/file.gz
> conf-file=z,/bla/bla/file.gz
> Again no scripting/manipulation but taking advantage of the on-the-fly 
> decompression
> 
> 

My understanding is that the script-conf feature was to be used to 
reduce filesystem storage usage, ie instead of a conf-file full of lines 
like

address=/domain1.com/
address=/domain2.com/

Juts the list of domains could be stored, in compressed form, and then 
they would be decompressed and decorated with the address=/...../ stuff 
on the fly before being fed into the dnsmasq configuration parser.

The  scripts you posted earlier seemed to

1) Download the block list.
2) compress it
3) feed it to dnsmasq which decompresses it

Which is the worst of all possible worlds, since it uses more storage 
for the compressed AND uncompressed versions, and more CPU to the 
compression and decompression. It also make the down-time depend on how 
fast the block list downloads.

Simon.