[Dnsmasq-discuss] Google's DNS servers

[Simon Kelley]

Tom Metro wrote:
> I read an article about how Google is providing DNS servers that they 
> believe will respond to queries faster than the typical ISP's servers. 
> They say they're offering them as a service to the net, and claim that 
> they're frequently purging the logs and not correlating the data with 
> other Google services.
> 
> The article said that they weren't releasing the code, but instead were 
> documenting the concepts (there's probably a white paper somewhere that 
> goes into details) and hoped that existing software would adopt the 
> concepts.
> 
> The main concept mentioned in the article was doing a statistical 
> analysis of the queried domains, and then pre-seeding the cache with 
> those domains. It also sounded like for those popular domains, when 
> their TTL expires, they fetch an update immediately, rather than waiting 
> for the next request for it to arrive. (The article also mentioned some 
> security enhancements, such as adding junk data to outbound queries as a 
> sort of security token to thwart forged replies.)
> 
> Anyone thought about building something like this around Dnsmasq? 
> Probably make sense for the initial implementation to be built external 
> to Dnsmasq. Say a program that analyzes the query log daily (obviously 
> query logging would have to be turned on), and updates a database. 
> Another that fetches the domains in the database, which would be called 
> from the Dnsmasq startup script. I'm not sure what the best way would be 
> to trigger a query on TTL expiration, unless TTLs were stored in the 
> database, and a daemon process fired queries at the appropriate time. 
> But that's starting to feel a bit redundant. Might be simpler to just 
> re-seed the cache daily.
> 
> What I wonder is whether all this added complexity would really be worth 
> while for a small scale installation. With tens of queries per minute, 
> instead of millions, could you even spot much difference in the 
> benchmarks? Would the difference be perceptible to end-users?
> 
>   -Tom
> 



An interesting idea. The following thoughts occur, in no particular order.

1) There's a difference between doing this a recursive server(Google) 
and a cache (dnsmasq.) Image a server caches a record and it has a TTL 
of 60s. At 59s (say) the server spots the record is about to expire and 
attempts to refresh it. A recursive server will talk to an authoritative 
server for the domain which will will give new data, valid for another 
60s (probably). Dnsmasq will talk to an upstream recursive server, 
probably the same one it talked to last time. Said server will also have 
cached the data from last time, and will therefore return it with the 
remaining time-to-live of 1s. Oops. So, to make this work, refreshes 
have to happen _after_ the TTL expires, plus a few seconds slop to avoid 
  clock-drift. That's OK as long as take it into account.


2) Records get evicted from the dnsmasq cache for two reasons, either 
the TTL expires, or sapce is needed for another record, and the 
least-recently-used record is dumped. Clearly only the first of should 
result in a refresh, and the act of doing a refresh shouldn't move the 
record back up the LRU order. (The second might be an implementation 
problem, I think newly inserted records have to go at the head of the 
LRU list with current code)

3) The most valuable records to refresh are those with short TTLs and 
those which are accessed often. One could combine this by marking a 
record for refresh iff is gets accessed _and_ it has less the n seconds 
before it expires. That would refresh records which are being accessed 
regularly and the those being accessed at all with short TTL values.


So, something like:

On reading a record if the expire time is less than 60s in the future, 
mark it for refresh.

On expiring a record, if it's marked, attempt to refresh it and clear 
the "refresh" flag.

This scheme means that if a record is refreshed and then not accessed 
again, it will drop out the next time the TTL expires. That's important, 
you don't what records to persist forever,

The time should be tunable somehow.


Cheers,

Simon.