[Dnsmasq-discuss] DNS TTL && IPSET TIMEOUT

[Ed W]

Hi

Oooh! I really like the idea of this! However, I would prefer it to be optional... (or have some way
to set a minimum  timeout)

I'm using ipsets for a couple of purposes, one is stuff like categorising IPs to upstream usage.
However, in practice most large service providers have stupidly low timeouts set on their DNS (1 min
or 3 mins), and this means in practice we see very little caching and in some of my use cases, DNS
can be a major fraction of all bandwidth used. So for one of my use cases I don't mind if the IP
list is a little stale (few hours say) since a) it's unlikely to actually happen in practice and b)
having a few extra open IP addresses in the firewall isn't a probable issue since they are unlikely
to leak to some other use case in those few hours and c) most big name apps seem to trigger a fresh
DNS check when something goes wrong, so there is a good chance of stale stuff sorting itself out
without the user particularly noticing more than a minor glitch

For this reason and more I've set min TTLs on a bunch of major app service entries. Bandwidth costs
are high for my use case and extra DNS traffic can be thousands of dollars a month

Please do continue this development though!

Thanks

Ed W


On 17/09/2020 08:06, Roderick Groesbeek wrote:
>
> Hi List,
>
>  
>
> Ipset supports a concept of ‘aging’ entries, like:
>
> ~~
>
> Examples from ipset(8):
>
> ipset create test hash:ip timeout 300
>
> ipset add test 192.168.0.1 timeout 60
>
> ipset -exist add test 192.168.0.1 timeout 600
>
> ~~
>
>  
>
> Dnsmasq supports a concept of adding entries to ipset
>
> ~~
>
> ipset=/.wearetriple.com/p1_afkl_http_https_test
>
> ~~
>
>  
>
>  
>
> However the timeout functionality is not implemented in the current DNSMASQ implementation.
>
> Using the DNS TTL as an IPSET TIMEOUT would seem natural…
>
>  
>
>  
>
>  
>
> Would this be right construct, to support that behavior?
>
> ATTR_DATA followed by the IP or TIMEOUT value in the nested construct?
>
>  
>
> ~~
>
>   proto = IPSET_PROTOCOL;
>
>   add_attr(nlh, IPSET_ATTR_PROTOCOL, sizeof(proto), &proto);
>
>   add_attr(nlh, IPSET_ATTR_SETNAME, strlen(setname) + 1, setname);
>
>  
>
>   nested[0] = (struct my_nlattr *)(buffer + NL_ALIGN(nlh->nlmsg_len));
>
>   nlh->nlmsg_len += NL_ALIGN(sizeof(struct my_nlattr));
>
>   nested[0]->nla_type = NLA_F_NESTED | IPSET_ATTR_DATA;
>
>  
>
>   nested[1] = (struct my_nlattr *)(buffer + NL_ALIGN(nlh->nlmsg_len));
>
>   nlh->nlmsg_len += NL_ALIGN(sizeof(struct my_nlattr));
>
>   nested[1]->nla_type = NLA_F_NESTED | IPSET_ATTR_IP;
>
>   add_attr(nlh,
>
>            (af == AF_INET ? IPSET_ATTR_IPADDR_IPV4 : IPSET_ATTR_IPADDR_IPV6) | NLA_F_NET_BYTEORDER,
>
>            addrsz, ipaddr);
>
>  
>
>   nested[2] = (struct my_nlattr *)(buffer + NL_ALIGN(nlh->nlmsg_len));
>
>   nlh->nlmsg_len += NL_ALIGN(sizeof(struct my_nlattr));
>
>   nested[2]->nla_type = NLA_F_NESTED | IPSET_ATTR_DATA;
>
>  
>
>   nested[3] = (struct my_nlattr *)(buffer + NL_ALIGN(nlh->nlmsg_len));
>
>   nlh->nlmsg_len += NL_ALIGN(sizeof(struct my_nlattr));
>
>   nested[3]->nla_type = NLA_F_NESTED | IPSET_ATTR_TIMEOUT;
>
>   add_attr(nlh, IPSET_ATTR_TIMEOUT | NLA_F_NET_BYTEORDER,  sizeof(attl), &attl);
>
>  
>
>   nested[3]->nla_len = (void *)buffer + NL_ALIGN(nlh->nlmsg_len) - (void *)nested[3];
>
>   nested[2]->nla_len = (void *)buffer + NL_ALIGN(nlh->nlmsg_len) - (void *)nested[2];
>
>   nested[1]->nla_len = (void *)buffer + NL_ALIGN(nlh->nlmsg_len) - (void *)nested[1];
>
>   nested[0]->nla_len = (void *)buffer + NL_ALIGN(nlh->nlmsg_len) - (void *)nested[0];
>
>  
>
>   while (retry_send(sendto(ipset_sock, buffer, nlh->nlmsg_len, 0,
>
>                            (struct sockaddr *)&snl, sizeof(snl))));
>
> ~~
>
>  
>
> sendto(3, {{len=88, type=NFNL_SUBSYS_IPSET<<8|IPSET_CMD_ADD, flags=NLM_F_REQUEST, seq=0, pid=0},
> {nfgen_family=AF_INET, version=NFNETLINK_V0, res_id=htons(0), [{{nla_len=5,
> nla_type=NFNETLINK_V1}, "\x06"}, {{nla_len=28, nla_type=0x2},
> "\x70\x31\x5f\x61\x66\x6b\x6c\x5f\x68\x74\x74\x70\x5f\x68\x74\x74\x70\x73\x5f\x74\x65\x73\x74\x00"},
> {{nla_len=32, nla_type=NLA_F_NESTED|0x7},
> "\x1c\x00\x01\x80\x08\x00\x01\x40\x5d\xbb\x0d\xc8\x10\x00\x06\x80\x0c\x00\x06\x40\x0e\x00\x00\x00\x00\x00\x00\x00"}]},
> 88, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 88
>
>  
>
>  
>
> Met vriendelijke groet / Best regards,
>
>  
>
> *Roderick Groesbeek*
>
> CTO
>
>
>
> 	
> 	
> 	
>
> *M:*+31 6250 67917
>
> 	
>
> *O:*+31 72 512 95 16 <tel:+31725129516>
>
> *E:*r.groesbeek at wearetriple.com <mailto:r.groesbeek at wearetriple.com>
>
> 	
>
> *W:*www.wearetriple.com <http://www.wearetriple.com/>
>
>
> Triple  |  Keesomstraat 10E  |  1821 BS  |  Alkmaar  |  The Netherlands
>
>  
>
>  
>
>
> _______________________________________________
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss at lists.thekelleys.org.uk
> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/attachments/20200928/7273bb9a/attachment.html>