[Dnsmasq-discuss] Malformed RRSIG Can Crash dnsmasq

[Daniel Anderson]

A malformed DNSSEC RRSIG response can crash DNSSEC-enabled dnsmasq during RSA signature verification.

Code location: src/dnssec.c, validate_rrset() RRSIG parsing; crash reaches src/crypto.c, dnsmasq_rsa_verify().

dnsmasq parses the RRSIG signer name using the full DNS packet length instead of the declared RRSIG RDATA boundary. It then computes:

sig_len = rdlen - (p - psav);

If the RRSIG RDATA contains only the fixed 18-byte prefix and the next packet byte is parseable as a DNS name, p advances beyond the RDATA and sig_len becomes negative. That value is later passed as a size_t into GMP mpz_import().

I reproduced this with a controlled upstream DNS server returning crash. A plus a malformed RRSIG. An ASAN build crashed in __gmpz_import() through dnsmasq_rsa_verify(); a normal non-ASAN build also crashed with Bus error: 10.

Suggested fix: after signer-name parsing, reject if p > psav + rdlen.

I am attaching the POC and the patch.  Copy files to a director on your computer and run using ```./run-live-poc.sh dnssec2```

Note: This POC showcases two bugs and I’ll be filing the next bug in a few minutes. Also, I found this bug while testing the N184 Bug and Vulnerability Scanning Suite (https://github.com/MillaFleurs/N184).

Thank you,

Dan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dnssec_upstream.py
Type: text/x-python-script
Size: 10495 bytes
Desc: not available
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20260511/05969b84/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: README.md
Type: text/markdown
Size: 1385 bytes
Desc: not available
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20260511/05969b84/attachment.md>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: run-live-poc.sh
Type: application/octet-stream
Size: 4747 bytes
Desc: not available
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20260511/05969b84/attachment.obj>