[Dnsmasq-discuss] How does DNSMASQ handle large concurrent configure file updating request requests

[Simon Kelley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 15/01/15 07:47, Yongkang You wrote:
> Hi Simon,
> 
> Thanks for replying. Batching the changes is a very good idea, but
> it is not easy to implement in IaaS, since the VMs creation
> processes are handled separately.
> 
>> From my observation, the burst 10k SIGUSER1 signals will cause
>> dnsmasq
> occupying 100% CPUs for several hours. The workload comes from 10k
> reload events, although the 10k DHCP config entries have been
> successfully updated in first tens of minutes.
> 

Wow, that's a lot of CPU. How are you synchronising changes to the
config files which you are re-reading?  That last thing you want is
race conditions where changes are missed, or half-written files read.

Is there one file per VM (ie ~10000 files, just some of which change)?
we might have to get more smart about only readng stuff which changes,
especially as I think there's processing going on at the moment which
is O(n^2) in the number of configs.


Cheers,

Simon.

> I didn't read the code, but just guess. Do you think if there is
> any possibility that the event queue doesn't accept other reload
> events, when last reload event is not finished.
> 
> On Thu, Jan 15, 2015 at 6:10 AM, Simon Kelley
> <simon at thekelleys.org.uk> wrote:
> 
> Dnsmasq  checks for incoming network packets and signals in the
> same event loop, so the maximum delay to DHCP packets will be the
> time to read a configuration file once, even if you send 10000
> SIGUSER1 signals, because the code will interleave handling DHCP
> and DNS packets with the config file reading. Batching changes
> would be good, if you can do it, but I'd not go looking for trouble
> here.
> 
> 
> Cheers,
> 
> Simon.
> 
> On 14/01/15 15:49, Brian Haley wrote:
>>>> On 01/14/2015 04:48 AM, Yongkang You wrote:
>>>>> Hi,
>>>>> 
>>>>> We have a cloud environment which uses DNSMASQ as DHCP/DNS 
>>>>> server; in the cloud, it usually starts lots of VMs
>>>>> concurrently, for example, 10000 VM; for each VM, we will
>>>>> configure a static DHCP entry in DNSMASQ configuration and
>>>>> send SIGUSER1 to reload. we observe DNSMASQ becomes very
>>>>> busy of reloading, for example, it seems to reload 10000
>>>>> times if we send for 10000 SIGUSER1.
>>>>> 
>>>>> Is this expected behavior? Is there any way to avoid this? 
>>>>> because it keep reloading, we don't know if this will
>>>>> effect VM getting IP address.
>>>> 
>>>> Just for completeness, is this on Openstack?  I can see that
>>>> code doesn't buffer anything when a port is plugged-in, so
>>>> each triggers a reload.  Could maybe tweak things there to
>>>> wait a few seconds between the HUPs.  I'm not sure it's a bug
>>>> in dnsmasq though.
>>>> 
>>>> -Brian
>>>> 
>>>> _______________________________________________
>>>> Dnsmasq-discuss mailing list
>>>> Dnsmasq-discuss at lists.thekelleys.org.uk 
>>>> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
>>>>
>
>>>> 
>> 
>> _______________________________________________ Dnsmasq-discuss
>> mailing list Dnsmasq-discuss at lists.thekelleys.org.uk 
>> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
>> 
> 
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iEYEARECAAYFAlS3xvsACgkQKPyGmiibgrfFuQCcDSNn8XmzOShyy6aDb6cIUMB8
WvUAnAi3WeYQyPRL7N0Xt3RfUvusk6q3
=zrnG
-----END PGP SIGNATURE-----