[Dnsmasq-discuss] [PATCH] fix for netlink ENOBUF problem

Simon Kelley simon at thekelleys.org.uk
Thu Jul 7 15:32:11 BST 2016


On 06/07/16 14:55, Ivan Kokshaysky wrote:
> On Mon, Jul 04, 2016 at 01:58:43PM -0400, wkitty42 at gmail.com wrote:
>> On 07/04/2016 11:29 AM, Ivan Kokshaysky wrote:
>>> To fix that we need to purge the netlink buffer on ENOBUF error. With the
>>> appended patch dnsmasq is running flawlessly for about a month.
>>
>> why are the messages not removed from the buffer when they are processed? or are 
>> they and there's simply too many messages coming in to handle?
> 
> Good questions, thanks.
> 
> It's certainly possible to drop these messages during normal processing -
> if the message process ID is correct, but sequence number is wrong,
> the message must be dropped (instead of putting it on async queue
> like it happens now, and it's a real bug, BTW). 

So that real bug relates to this:

   if (h->nlmsg_seq != seq ||
       h->nlmsg_pid != netlink_pid ||
       h->nlmsg_type == NLMSG_ERROR)
	  {
	    /* May be multicast arriving async */
	    nl_async(h);
	  }

Which should be something like

if (h->nlmsg_pid != netlink_pid ||
    h->nlmsg_type == NLMSG_ERROR)
     {
        /* May be multicast arriving async */
        nl_async(h);
     }
else if (h->nlmsg_seq != seq)
     {
        /* drop message */
     }
else
     /* process message */

Also, I wonder if the loop which drops messages should check for async
stuff and handle it - that won't get re-tried by the kernel, will it?

Also, also, you still have the sleep(1) in there. Is that actually
necessary? Handling things this way looks better than just sleeping and
hoping?


Cheers,

Simon.


In fact I tried this
> approach first and it sort of worked, but then I went to "flush" because
> it was much easier to debug and also had some minor advantages:
> - less risk to hit ENOBUFS again on the next netlink request
>   as the buffer is just flushed;
> - no additional checks in a fast code path.
> 
> As for too many async messages coming in, I doubt that. In our config
> all interfaces are static and never change their state. Of course, there are
> lots of events like new neighbor appearance and so on, but as far as
> I can see they are masked out and should not disturb dnsmasq.
> 
>> how large is the buffer? can it be made larger to handle the larger amount of 
>> message traffic?
> 
> It's rather large, some 120 kB by default, IIRC. Even with thousands of
> interfaces it's enough to keep dnsmasq happy most of the time. I'm not
> a netlink specialist, but from what I read enlarging socket buffer is
> generally not considered like a very good idea, as it won't eliminate
> ENOBUFS, just delay it.
> 
>> what problem(s) will requesting devices run into when there is no response to 
>> their query when the message is flushed?
> 
> Not a big deal, I think. After ENOBUFS our view on the system state
> is not valid anymore. We don't know if the device in questions hasn't
> requested again while we were out of buffer space. As netlink(7) says:
> 
> "However, reliable transmissions from kernel to user are impossible
>  in any case. The kernel can't send a netlink message if the socket
>  buffer is full: the message will be dropped and the kernel and the
>  user-space process will no longer have the same view of kernel state.
>  It is up to the application to detect when this happens (via the ENOBUFS
>  error returned by recvmsg(2)) and resynchronize."
> 
>> would fixing/solving (one of?) the above be better than flushing?
> 
> Personally I like flushing more, because I think it fits in "resynchronize"
> mentioned above. But I'd also be fine with a proper check for pid/seq in
> the normal message processing path. It's up to authors to decide :)
> 
> Ivan.
> 
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20160707/811644d8/attachment.sig>


More information about the Dnsmasq-discuss mailing list