[Dnsmasq-discuss] Dnsmasq with Gigantic hosts file

[Jan 'RedBully' Seiffert]

Jason wrote:
> Hello all,
> 
Hi

> I'm considering replacing squidguard, a filtering redirector for the
> squid cache proxy, with a BIG hosts file and dnsmasq (716,093
> entries!).  Currently, my network serves less than 10 clients(dns only,
> no dhcp), but may reach 200 in the next few years.  Will this work. 
> It'd be nice to consolidate the functions and have the option to
> shutdown the squid cache.

If you are talking about a traditional hosts file:
AFAIK dnsmasq uses a hash table for such lookups, so basically even a
large number should not be a problem. I for example have 10.000
"bad-hosts" in an extra file read by dnsmasq (50k where also already
mentioned here on the list), and my DNS lookups are "lightning fast" (I
also run dnsmasq with nice -1 + some traffic shaping rules prioritizing
DNS packets, but thats to reduce latencies).
But 700k may drive it to it's limits...
First you will need a little RAM for it (my dnsmasq uses 2MB ATM, so
_very_ roughly * 70 ~= 140MB), but secondly scalability-issues in the
hash table may surface (hash collision, but you may fix it locally by
setting a higher number of hash buckets in the source code and recompile).

But the ultimate answer to this is only known by Simon.

I also have a patch installed to use RegEx on host names, so you can do
things like:
address=/:(double|fast|value|(euros|pay)4)[kc]lick?\.(com|net|tld):/127.0.0.1

This can compress a list of "bad hosts" in traditional hosts file format
and match things you never thought of in the first place.
Downside is, it internally uses a linked list, so it scales linear with
the number of rules (for every lookup!) + RegEx overhead (but you can
squeeze a lot in one RegEx ;).
But i'm still testing if it runs smoothly before letting it escape in to
the wild :-/

> Also, squidguard can block a specific url
> inside a domain, ie, www.goodsite.com/badarea/badstuff.html.  Can
> dnsmasq emulate this?  I know this is outside the design intent of
> dnsmasq, but it's not a bad application.

As Richard Voigt already mentioned, the URL is never transfered to your
DNS resolver, only the hostname.

IMHO the right tool for such a task is a proxy, like squid. I mean you get:
* Content cache to speed things up and save bandwidth (esp. with 200
clients)
* Filtering on URL, source, target, authentication, foo and bar
* Possibility to pass every file through clamav, for example

But since dnsmasq is so easy to set up and well performing you may split
up your task:
Basic host based blocking (and get a DNS cache for free ;) -> dnsmasq
Advanced URL filtering and other magic -> squid

But if this complication is wise, the question may has to be: Whats the
problem with your current setup?
(Oh, and no, i personally do not use squid, so i cannot help you on
problems with squid)

> Jason
> 
Greetings
	Jan

PS: Jippie, first Post on the list.
I want to thank Simon for this great program.
And sorry for my bad English.
-- 
error compiling committee.c: too many arguments to function