<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1362709909;
mso-list-type:hybrid;
mso-list-template-ids:1758482586 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am experiencing coredump in dnsmasq on OpenStack environment.<o:p></o:p></p>
<p class="MsoNormal">This is my setup:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>RH 7.1 <o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>OpenStack Liberty release<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>dnsmasq-utils-2.66-14.el7_1.x86_64<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>we are using dhcp on InfiniBand network (using client id and there is no MAC)
<o:p></o:p></p>
<p class="MsoNormal">Some times when spawning a VM the dnsmasq crashes see [1] and [2]<o:p></o:p></p>
<p class="MsoNormal">Just to point out when spawning a VM the neutron-dhcp-agent (which manage the dnsmasq instances for OpenStack) send SIGHUP to reload the config files
<o:p></o:p></p>
<p class="MsoNormal">And after that I see <o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 neutron-dhcp-agent: 2015-10-27 12:11:18.213 44868 ERROR neutron.agent.linux.external_process [-] respawning dnsmasq for uuid 82acf0a3-ec07-4009-84b5-74f750c89dc6<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: started, version 2.66 cachesize 150<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: warning: no upstream servers configured<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: DHCP, static leases only on 192.168.111.0, lease time 1d<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/addn_hosts - 6 addresses<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/host<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/opts<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 kernel: dnsmasq[41374]: segfault at 7a ip 00007f886e5501e8 sp 00007fff9c540b80 error 4 in dnsmasq[7f886e51e000+43000]<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This is how the neutron-dhcp-agent spawn the dnsmasq <o:p></o:p></p>
<p class="MsoNormal">dnsmasq --no-hosts --no-resolv --strict-order --except-interface=lo --pid-file=/var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/host --addn-hosts=/var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/addn_hosts
--dhcp-optsfile=/var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/opts --dhcp-leasefile=/var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/leases --dhcp-match=set:ipxe,175 --bind-interfaces --interface=tap04c60fe7-62 --dhcp-range=set:tag0,192.168.111.0,static,86400s
--dhcp-lease-max=256 --conf-file= --domain=openstacklocal --dhcp-broadcast<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And I also attached the config files: opts, leases, addn_hosts, host<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Please note that it is not happened on the Ethernet network only InfiniBand, so I guess the crash (and as it seems in the logs) related to the client id.<o:p></o:p></p>
<p class="MsoNormal">It will be great if you can help me debug this issue. <o:p></o:p></p>
<p class="MsoNormal">Thank,<o:p></o:p></p>
<p class="MsoNormal"> Moshe Levi. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] - /var/log/messages<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 neutron-dhcp-agent: 2015-10-27 12:11:18.213 44868 ERROR neutron.agent.linux.external_process [-] respawning dnsmasq for uuid 82acf0a3-ec07-4009-84b5-74f750c89dc6<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: started, version 2.66 cachesize 150<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: warning: no upstream servers configured<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: DHCP, static leases only on 192.168.111.0, lease time 1d<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/addn_hosts - 6 addresses<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/host<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 dnsmasq-dhcp[41374]: read /var/lib/neutron/dhcp/82acf0a3-ec07-4009-84b5-74f750c89dc6/opts<o:p></o:p></p>
<p class="MsoNormal">Oct 27 12:11:18 r-smg37 kernel: dnsmasq[41374]: segfault at 7a ip 00007f886e5501e8 sp 00007fff9c540b80 error 4 in dnsmasq[7f886e51e000+43000]<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[2]: CoreDump<o:p></o:p></p>
<p class="MsoNormal">[root@r-smg37 ~(keystone_admin)]# gdb /usr/sbin/dnsmasq /root/core-dnsmasq-11-99-40-46692-1445940738<o:p></o:p></p>
<p class="MsoNormal">GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7<o:p></o:p></p>
<p class="MsoNormal">Copyright (C) 2013 Free Software Foundation, Inc.<o:p></o:p></p>
<p class="MsoNormal">License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html><o:p></o:p></p>
<p class="MsoNormal">This is free software: you are free to change and redistribute it.<o:p></o:p></p>
<p class="MsoNormal">There is NO WARRANTY, to the extent permitted by law. Type "show copying"<o:p></o:p></p>
<p class="MsoNormal">and "show warranty" for details.<o:p></o:p></p>
<p class="MsoNormal">This GDB was configured as "x86_64-redhat-linux-gnu".<o:p></o:p></p>
<p class="MsoNormal">For bug reporting instructions, please see:<o:p></o:p></p>
<p class="MsoNormal"><http://www.gnu.org/software/gdb/bugs/>...<o:p></o:p></p>
<p class="MsoNormal">Reading symbols from /usr/sbin/dnsmasq...Reading symbols from /usr/lib/debug/usr/sbin/dnsmasq.debug...done.<o:p></o:p></p>
<p class="MsoNormal">done.<o:p></o:p></p>
<p class="MsoNormal">[New LWP 46692]<o:p></o:p></p>
<p class="MsoNormal">[Thread debugging using libthread_db enabled]<o:p></o:p></p>
<p class="MsoNormal">Using host libthread_db library "/lib64/libthread_db.so.1".<o:p></o:p></p>
<p class="MsoNormal">Core was generated by `dnsmasq --no-hosts --no-resolv --strict-order --except-interface=lo --pid-file='.<o:p></o:p></p>
<p class="MsoNormal">Program terminated with signal 11, Segmentation fault.<o:p></o:p></p>
<p class="MsoNormal">#0 find_config (configs=0x7f4b5b014350, context=context@entry=0x0, clid=0x7f4b5b00bc00 "\377", clid_len=20, hwaddr=hwaddr@entry=0x7f4b5b00bb90 "", hw_len=0, hw_type=32, hostname=hostname@entry=0x0)<o:p></o:p></p>
<p class="MsoNormal"> at dhcp-common.c:319<o:p></o:p></p>
<p class="MsoNormal">319 if (!(context->flags & CONTEXT_V6) && *clid == 0 && config->clid_len == clid_len-1 &&<o:p></o:p></p>
<p class="MsoNormal">(gdb) bt<o:p></o:p></p>
<p class="MsoNormal">#0 find_config (configs=0x7f4b5b014350, context=context@entry=0x0, clid=0x7f4b5b00bc00 "\377", clid_len=20, hwaddr=hwaddr@entry=0x7f4b5b00bb90 "", hw_len=0, hw_type=32, hostname=hostname@entry=0x0)<o:p></o:p></p>
<p class="MsoNormal"> at dhcp-common.c:319<o:p></o:p></p>
<p class="MsoNormal">#1 0x00007f4b5952575b in lease_update_from_configs () at lease.c:193<o:p></o:p></p>
<p class="MsoNormal">#2 0x00007f4b59521c2c in clear_cache_and_reload (now=1445940738) at dnsmasq.c:1236<o:p></o:p></p>
<p class="MsoNormal">#3 0x00007f4b5950c334 in async_event (now=1445940738, pipe=10) at dnsmasq.c:1049<o:p></o:p></p>
<p class="MsoNormal">#4 main (argc=<optimized out>, argv=<optimized out>) at dnsmasq.c:852<o:p></o:p></p>
</div>
</body>
</html>