[Dnsmasq-discuss] Two questions about the cache and how dnsmasq forwards queries
Simon Kelley
simon at thekelleys.org.uk
Fri Feb 12 11:47:25 GMT 2010
SamLT wrote:
> On Thu, Feb 11, 2010 at 09:18:38PM +0000, Simon Kelley wrote:
>> SamLT wrote:
>>> Hello,
>>>
>>>
>>> I have a slow connection and every bit saved is precious! So I installed
>>> dnsmasq, but when I run dnsmasq -dq and watch what's happening, I can see
>>> two things that surprise me:
>>>
>>> -> only few queries really benefit from the dnsmasq cache, and I think it
>>> happens when the name is actually a CNAME, but I'm not sure yet.
>>> eg:
>>> * dig www.google.com
>>> * dnsmasq forwards the query then fills its cache with www.google.com
>>> being a CNAME to www.l.google.com, and an entry with the IP
>>> www.l.google.com resolved to
>>>
>>> * but if I issue again the dig command, dnsmask forwards again the
>>> query even though the entries haven't yet expired
>>>
>>> Note: I've just tried on an other computer(at home, other OS, other ISP...)
>>> and I don't see this behaviour, I leave the question untouched, since I may
>>> have missed a configuration option? I'll investigate on this further
>>> tomorow and report back.
>>>
>>>
>> That behaviour is not expected, but note that if any of the links in
>> the CNAME chain expire, the query will have to be forwarded again.
>> It's also possible that the cache entries get thrown out if dnsmasq
>> is busy and entries are not used. It's quite subtle, so you may need
>> to do more experiments to understand what's happening.
>
> The CNAME link was expired. There is still something a little weird though:
>
> Here is a part of the log (dnsnasq -dq --log-facility=/tmp/test.log):
> Feb 12 08:58:47 dnsmasq[11135]: query[A] www.google.com from 192.168.50.126
> Feb 12 08:58:47 dnsmasq[11135]: forwarded www.google.com to 196.32.200.12
> Feb 12 08:58:48 dnsmasq[11135]: reply www.google.com is <CNAME>
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.99
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.103
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.104
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.105
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.106
> Feb 12 08:58:48 dnsmasq[11135]: reply www.l.google.com is 74.125.159.147
>
> then if I dump the cache with a USR1 signal:
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.99 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.103 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.104 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.105 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.106 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.l.google.com 74.125.159.147 4F Fri Feb 12 09:03:26 2010
> Feb 12 09:01:23 dnsmasq[11135]: www.google.com www.l.google.com CF Fri Feb 12 08:58:48 2010
>
>
> As you can see on the last line, the CNAME entry is expired as soon as it's
> recieved. Which brings up two more questions:
> -> Why does dnsmasq keep it in its cache although it's already expired?
The code frees cache entries lazily: If the expiry-time is later than
NOW, the record will never be used. It will hang around in memory until
it's purged, which can happen if the record looked up, or dnsmasq runs
out of space and does a global garbage-collect. The cache dump code
doesn't look at the expiry time, so you still see those records. It's
arguable that the cache dump should suppress those records, butu it's a
debugging aid, and old records can be useful when debugging.
> -> Is my ISP doing something weird? Is it normal that the CNAME link is
> expired _that_ soon?
>
>
Imagine this: www.google.com has a time-to-live of 60s. Someone using
your ISP looks up www.google.com and your ISPs nameservers go to
Google's nameservers and get the record, with a time to live of 60s.
That record stays in the ISP nameserver cache. 59 seconds later _you_
look up www.google.com. The record is still in the cache, it had a TTL
of 60s and was collected 59s ago, so it has 1s left to live. The
namesevers give you the answer you want, with a TTL of 1s.
For popular domains, you would expect any busy caching server to return
TTL values evenly distributed between the TTL in the authoritative
server and 1s. For rare domains, which probably aren't cached when you
first look them up, the TTL will normally be the full length given by
the authoritative server.
>
>
>
> As for the second part of my original mail:
>
>>> -> dnsmasq is supposed to help me saving bandwidth, but when it forwards a
>>> query, it forwards it to every nameserver (well, most of the time at
>>> least), isn't there a way to make it forward to only one server at a time,
>>> and to try an other one only if there was a problem (not found, server
>>> down..)
>>>
>> It will normally forward to one nameserver, but every 10 seconds or
>> 50 queries it sends to all, to find which is fastest. If you are
>> testing and sending a test query every 10 seconds or so, then it
>> might look like most queries go to all servers, but under load
>> that's not true.
>>
>
> Well, I can't say that's what I see.
> I've attached a small awk script to parse the log:
>
> dnsmasq -dq --log-facility=/tmp/test.log
> awk -f dns.awk test.log | less
>
> Here is part of the output:
> name | nb | forwarded | answered with cache
> * users.conduit.com | 3 | 4 | 0
> * www.google-analytics.com | 61 | 86 | 0
> * weather.partners.msn.com | 14 | 23 | 0
> * js.wp.bandoo.com | 7 | 11 | 0
> * hst.tradedoubler.com | 1 | 2 | 0
> [ .... snip .... ]
> * fr.mail.yahoo.com | 6 | 11 | 0
> watson.microsoft.com | 4 | 2 | 2
> * estb.msn.com | 6 | 11 | 0
> * fr.ard.yahoo.com | 22 | 39 | 0
> * 89221e8c4a08c8f8.users.storage.live.com | 3 | 5 | 0
> 6f75271f741d160d.users.storage.live.com | 1 | 1 | 0
> * images.logicimmo.com | 2 | 4 | 0
> logv3.xiti.com | 1 | 1 | 0
>
> total: | 8824 | 14329 | 0
>
>
> (I think it's self explanatory, but I may be wrong)
>
> As you can see on the last line, dnsmasq received 8824 queries, and
> forwarded 14329. This is due to dnsmasq often forwarding the queries to
> the two nameservers I have configured instead of just one. I could of
> course, configure dnsmasq to only forward to one nameserver, but it isn't a
> solution.
>
>
> Am I doing something wrong? I'm using dnsmasq-2.52-1 on a debian testing system.
>
>
The "answered from cache" numbers seem to be very low. Could you look at
the logs and try and get a better idea of exactly what's happening? Do
you clients have their own caches? If so that would obviously
dramatically reduce the cache hit-rate.
Cheers,
Simon.
More information about the Dnsmasq-discuss
mailing list