[Dnsmasq-discuss] About resolution performance and adblock

[Ercolino de Spiacico]

In the context of Adblock, I noticed that our adblock script can handle 
relatively well about 10MB of blockfile which is about 7.8% of the 
device RAM (128MB), after that the resolution time increases 
exponentially to the point where the DNS resolution times-out and more 
importantly the device becomes unstable.

I was trying to understand the root cause on why we couldn't have a 
larger blockfile which is compiled in local=/example.com/# format. The 
RAM, cache and buffer demand stay relatively low when the system becomes 
unstable.

In the aim to investigate this further, I run a test creating a bogus 
blockfile of the size of about 100MB (and 3.6M lines/domains). Here are 
the grep results to fetch info from it:

root at router:/mnt/USB/adblock# time grep 'mytestdomain.com' 
/mnt/USB/adblock/test2
mytestdomain.com
real    0m 2.44s
user    0m 0.49s
sys     0m 0.43s

root at router:/mnt/USB/adblock# cp /mnt/USB/adblock/test2 /tmp/test2

root at router:/mnt/USB/adblock# time grep 'mytestdomain.com' /tmp/test2
mytestdomain.com
real    0m 0.65s
user    0m 0.39s
sys     0m 0.20s

What I'm trying to demonstrate here is that a USB2 device can extract a 
domain via grep in 2.44sec, and if that file was to be placed in RAM 
(/tmp is mapped in RAM on devices with squashfs) it's just 0.65sec. 
Admittedly /tmp compresses the content so the 100MB uses about 38MB, 
still the point on performance is valid and tells me we could fit 
200-250MB blockfile if ever needed, looking at RAM capacity only.

As a point of discussion/improvement, I believe dnsmasq uploads the 
custom config (so the blockfile in this case) into RAM, why do we 
experience poor resolution performance and system instability at just 10MB?

Considering the system grep is so fast, could this be an alternative 
method for dnsmasq to address locally defined domains? If given the 
possibility, I would be very happy to map a file in RAM knowing that 
this is handled differently from the "standard" conf-file.

I suppose the first step would be to fully understand where the 
limitation we currently have comes from.

Then, I'm not suggesting we should re-invent the wheel, but perhaps 
there's a margin for a new directive whose behavior is a simple grep 
against a mapped file to be used as an authority for those domains? 
Might be restricted to blocking only (returning NX or 0.0.0.0 or 
127.0.0.1)? Not sure what the secondary implications of such an idea 
would be, but I'll be glad to hear some comments/opinions on this topic.


Thanks