[Dnsmasq-discuss] Dnsmasq with Gigantic hosts file

[Jan Seiffert]

2012/1/11 Preston Crow <pc-dnsmasq at crowcastle.net>:
> I'm running dnsmasq with a large hosts file, and it's taking about a minute
> to start up, which doesn't seem right.  Specifically, for the first minute
> while it is initializing, it does not respond to DNS requests.  If I attach
> to it with strace, I see it doing a sequence of 4K reads of the hosts file,
> and I can see it slowing down as it progresses.  After every 8 reads, there
> is one brk() syscall.
>
> I assume that the slow part is in processing the hosts that it's reading.
>
> My setup is using only DNS, no DHCP.  I have two hosts files, the first is
> short, only 1116 bytes, but the second is over 1MB and includes over 50,000
> entries.
>

I can only offer 450kbyte, 15k entries, but it reads it in, don't know, seconds?
Something must be "different".

> If you're curious, the second file is a copy of the NIS hosts file,

NIS ...
hmmm, all hosts of the same domain?

Simon, i already forgot how the cache works, could this be a massive
hash collision?

[snip]
> (It's a
> new high-end laptop with tons of memory, fast CPU, and an SSD, so it I
> shouldn't hit any issues there.)
>
i have dnsmasq on an Sempron, so it is bigger/faster than one of those
Atom/ARM, but still small compared with your laptop.
Frequency scaling setting it to slowest frequency?

> From what I've read, including a thread with the same subject line in the
> archives, the startup time should be just a second or two, not a minute,
> though that was five years ago:
> http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2007q1/001134.html
>

It should not get worse, and this thread back then was more about
lookup performance.

> My config file has:
>
> resolv-file=/etc/resolv.conf.upstream
> server=/{some domain}/ip.number.here
> ... (26 server lines total)
> no-dhcp-interface=
> addn-hosts=/etc/hosts.nis
>
>
> Any suggestions for how to fix this?
>

I wish i would. I would now grab for callgrind (valgrind tool) &
kcachegrind out of my toolbox.
But that output is tricky to interpret.

Greetings
Jan

-- 
Remember to eat a healthy breakfast, for tonight we dine in hell!