[Dnsmasq-discuss] excessive cpu consumption in 2.39rc1

[Simon Kelley]

Carlos Carvalho wrote:
> Simon Kelley (simon at thekelleys.org.uk) wrote on 14 April 2007 09:08:
>  >Carlos Carvalho wrote:
>  >> I really have a problem with this new code. The server is loaded and
>  >> dnsmasq takes a long time to answer. I've got some angry users
>  >> today...
>  >
>  >Dnsmasq tries to limit queue overflows by delaying  in the log() routine 
>  >for longer and longer times as the queue length increases. That would 
>  >explain the long query times, but it's rather strange that it's worse 
>  >than with log-max=0, since the queue will only build up (thus causing 
>  >delays) when syslogd is slow, and if syslogd is slow, then logging will 
>  >block in the log-max=0 case, so that should be slow too.....
> 
> I don't know if it was worse, it looked as bad. I had to restore
> service quickly, so I just turned logging off.
> 
>  >> I tried log-max=1000 to no avail. I had to disable logging to restore
>  >> the service (make it fast enough).
>  >
>  >I recall that you do lots of logging: log-queries and log-dhcp? Is it 
>  >faster with log-max=0?
> 
> No, that's why I had to turn off logging.

OK, just to be clear: when your sever is loaded, is it too slow to have
 logging on with 2.38 too? I hope so, otherwise there's an effect here
which I don't understand.
> 
>  >> It seems that log-max is limited to 25. This shouldn't happen...
>  >> 
>  >> Another alternative would be to disable the new code. In my case
>  >> there's no risk of deadlock because the server that runs syslog (and
>  >> dnsmasq) doesn't use dnsmasq itself, it's only for the clients.
>  >> However setting log-max high enough should have the same effect.
>  >> 
>  >
>  >log-max=0 does effectively disable the new behaviour, and I'm starting 
>  >to thing that would be a better default, for precisely this sort of reason.
> 
> In summary, the new code seems to be working correctly. Thanks for it.
> 
> However I have two requests. First, the limiting of log-max to 25 is
> against my principles. One must not change a user setting if the
> implementation doesn't require it, and if it's done it must be
> documented in BIG LETTERS.
> 
> Second, the increasing delays when there are queued log messages
> should be documented and only happen above a certain fraction of
> log-max queue usage. This is so that one can set a large log-max to
> have fast response even with transient slow logging; that's what I
> tried putting log-max=1000.
> 

Ok, I've put up

http://www.thekelleys.org.uk/dnsmasq/release-candidates/dnsmasq-2.39rc2.tar.gz

which makes basically these changes.

1) async logging is off by default, it can be enabled with --log-async
2) When enabled, the queue defaults to 5, but can be increased with
   --log-async=<queue size>
3) The queue size is now limited to 100 (I still contend that very large
   queues are useless.) the actual queue in use and sync/async are
   logged at startup.
4) The delay is tuned as you suggest: it only starts when the last 8
   queue entries are used.


Thanks, Carlos, for the useful suggestions. Everybody please test.


Cheers,

Simon.