[Dnsmasq-discuss] Google's DNS servers

[Simon Kelley]

Tom Metro wrote:
> Perette Barella wrote:
>> Plus then you've got more complexity down the road (more stuff to
>> install, need to adapt future code changes to it/it to future code
>> changes, maintain configurations that work with it, etc.).
> 
> If it proved to be useful, and was integrated into Dnsmasq, there could 
> be as few as 2 config settings. One to enable predictive caching. 
> Another to set the limit on how many records to cache (either record 
> count or percentage of cache to dedicate to pre-seeded records).
> 
> The database used to persistently store the list of most frequently 
> queried domains could be just a text file (similar to the lease file), 
> so no database configuration required.
> 
> (There's no need to be able to access random records from the database 
> for reading (cache seeding would iterate over them sequentially). You 
> might, however, want to be able to write random records to the database 
> if you gather statics in real time, rather than analyzing query logs 
> after the fact. It's a trade-off between one approach that requires a 
> separate analysis process and depends on query logging (or the 
> equivalent) or keeping a larger database that can handle random updates 
> without showing down queries.)
> 
> 
>> All that introduces processing overhead (and memory, etc.).
> 
> Obviously there will be a hit on memory...or at least the memory 
> available for caching non-seeded records will be reduced. Processing 
> overhead might be minimal - at least the impact imposed on Dnsmasq's 
> code to respond to and issue queries. Statistical analysis could be 
> performed by a separate process. Pre-seeding at startup would need to be 
> a separate process or thread.
> 
> Simon would know best, but my guess is that it would probably be most 
> memory efficient to have Dnsmasq extend its cached record structure in 
> memory to add an "auto refresh" flag, which causes the record expiration 
> code to refresh that record, rather than expiring it. (Alternatively 
> pre-seeding and refreshing could be relegated to a separate process, 
> perhaps receiving record expiration events from Dnsmasq via dbus, but 
> the second process would then need to have a redundant copy of the 
> domains flagged for pre-seeding in memory, or an indexed database. The 
> prototype I suggested would use the latter.)
> 
> 
>> ...introduce a lot of complexity because you need algorithms making
>> somewhat subjective calls on what to keep in caches, etc.
> 
> I don't think it would be all that complex or subjective. Caching 
> algorithms are fairly well understood, and the rules for what to keep 
> should be fairly simple. But it wouldn't be a trivial change, if integrated.
> 
>   -Tom
> 

See my previous post on possible algorithms, which covers some of this.

I see this a completely orthogonal to using persistent storage. You
don't need PS to do predictive cache refresh, and you don't need cache
refresh to do persistent storage.

I comtinue to contend that persistent storage is a bad idea.


Cheers,

Simon.