[Dnsmasq-discuss] Google's DNS servers

[Tom Metro]

Perette Barella wrote:
> Plus then you've got more complexity down the road (more stuff to
> install, need to adapt future code changes to it/it to future code
> changes, maintain configurations that work with it, etc.).

If it proved to be useful, and was integrated into Dnsmasq, there could 
be as few as 2 config settings. One to enable predictive caching. 
Another to set the limit on how many records to cache (either record 
count or percentage of cache to dedicate to pre-seeded records).

The database used to persistently store the list of most frequently 
queried domains could be just a text file (similar to the lease file), 
so no database configuration required.

(There's no need to be able to access random records from the database 
for reading (cache seeding would iterate over them sequentially). You 
might, however, want to be able to write random records to the database 
if you gather statics in real time, rather than analyzing query logs 
after the fact. It's a trade-off between one approach that requires a 
separate analysis process and depends on query logging (or the 
equivalent) or keeping a larger database that can handle random updates 
without showing down queries.)


> All that introduces processing overhead (and memory, etc.).

Obviously there will be a hit on memory...or at least the memory 
available for caching non-seeded records will be reduced. Processing 
overhead might be minimal - at least the impact imposed on Dnsmasq's 
code to respond to and issue queries. Statistical analysis could be 
performed by a separate process. Pre-seeding at startup would need to be 
a separate process or thread.

Simon would know best, but my guess is that it would probably be most 
memory efficient to have Dnsmasq extend its cached record structure in 
memory to add an "auto refresh" flag, which causes the record expiration 
code to refresh that record, rather than expiring it. (Alternatively 
pre-seeding and refreshing could be relegated to a separate process, 
perhaps receiving record expiration events from Dnsmasq via dbus, but 
the second process would then need to have a redundant copy of the 
domains flagged for pre-seeding in memory, or an indexed database. The 
prototype I suggested would use the latter.)


> ...introduce a lot of complexity because you need algorithms making
> somewhat subjective calls on what to keep in caches, etc.

I don't think it would be all that complex or subjective. Caching 
algorithms are fairly well understood, and the rules for what to keep 
should be fairly simple. But it wouldn't be a trivial change, if integrated.

  -Tom

-- 
Tom Metro
Venture Logic, Newton, MA, USA
"Enterprise solutions through open source."
Professional Profile: http://tmetro.venturelogic.com/