[Dnsmasq-discuss] [PATCH] Optimize speed on massive server=/.../... records

[Geert Stappers]

On Sun, Nov 20, 2022 at 01:50:10PM +0800, Ye Zhou wrote:
> Hi all,
> 
> I'm attaching a patch to optimize a speed issue introduced in version 2.86.
 
Now is the patch attached 

> I have two ISP upstreams and need to forward different sites to different
> ISP's DNS providers. For example:
> 
> server=/meituan.com/114.114.114.114
> ... (lots of records)
> server=/taobao.com/223.5.5.5
> ... (lots of records)
> 
> It works well before v2.86. Since v2.86 the configuration load time becomes
> extremely long (more than 1 minutes to load all server records). The time
> consuming part is inside the rewritten domain-match.c. When adding a new
> server record, the code will traverse all existing records so the
> configuration load becomes quadratic time complexity. The issue still
> persists on v2.88rc3.
> This patch will optimize the config load time by bypassing the time
> consuming code block. During the config load mark_servers() will never be
> called so does not need to waste time on the record traversal and re-order
> part.
> 
> Before:
> dnsmasq --test  77.50s user 0.30s system 99% cpu 1:17.84 total
> After:
> dnsmasq --test  0.16s user 0.02s system 99% cpu 0.188 total
> 
> https://gist.github.com/zhouye/adfd509f51645d314f53992331449c45

<screenshot documents="What the F">
stappers at alpaca:~/src/dnsmasq
$ wget -O fastreload.patch https://gist.github.com/zhouye/adfd509f51645d314f53992331449c45
--2022-11-20 08:23:57--  https://gist.github.com/zhouye/adfd509f51645d314f53992331449c45
Resolving gist.github.com (gist.github.com)... 140.82.121.3
Connecting to gist.github.com (gist.github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'fastreload.patch'

fastreload.patch          [ <=>                             ]  91.26K  --.-KB/s    in 0.07s

2022-11-20 08:23:59 (1.30 MB/s) - 'fastreload.patch' saved [93447]

stappers at alpaca:~/src/dnsmasq
$ file fastreload.patch
fastreload.patch: HTML document, UTF-8 Unicode text, with very long lines
stappers at alpaca:~/src/dnsmasq
$
<screenshot>

curl --silent https://gist.githubusercontent.com/zhouye/adfd509f51645d314f53992331449c45/raw/3d948c9b4e7eac419a3c669ffba59b84341b4577/dnsmasq-server-fix.diff


diff --git a/src/dnsmasq.h b/src/dnsmasq.h
index 90dc986..5aaffcc 100644
--- a/src/dnsmasq.h
+++ b/src/dnsmasq.h
@@ -1144,6 +1144,7 @@ extern struct daemon {
   struct iname *if_names, *if_addrs, *if_except, *dhcp_except, *auth_peers, *tftp_interfaces;
   struct bogus_addr *bogus_addr, *ignore_addr;
   struct server *servers, *servers_tail, *local_domains, **serverarray;
+  int servers_flag;
   struct rebind_domain *no_rebind;
   int server_has_wildcard;
   int serverarraysz, serverarrayhwm;
diff --git a/src/domain-match.c b/src/domain-match.c
index 76a1109..56ae015 100644
--- a/src/domain-match.c
+++ b/src/domain-match.c
@@ -552,6 +552,7 @@ void mark_servers(int flag)
   struct server *serv, **up;
 
   daemon->servers_tail = NULL;
+  daemon->servers_flag = flag;
   
   /* mark everything with argument flag */
   for (serv = daemon->servers; serv; serv = serv->next)
@@ -602,6 +603,8 @@ void cleanup_servers(void)
          daemon->servers_tail = serv;
        }
     }
+
+  daemon->servers_flag = 0;
 }
 
 int add_update_server(int flags,
@@ -663,29 +666,32 @@ int add_update_server(int flags,
     }
   else
     { 
-      /* Upstream servers. See if there is a suitable candidate, if so unmark
-        and move to the end of the list, for order. The entry found may already
-        be at the end. */
-      struct server **up, *tmp;
-      
-      for (serv = daemon->servers, up = &daemon->servers; serv; serv = tmp)
+      if (daemon->servers_flag)
        {
-         tmp = serv->next;
-         if ((serv->flags & SERV_MARK) &&
-             hostname_isequal(alloc_domain, serv->domain))
+         /* Upstream servers. See if there is a suitable candidate, if so unmark
+           and move to the end of the list, for order. The entry found may already
+           be at the end. */
+         struct server **up, *tmp;
+         
+         for (serv = daemon->servers, up = &daemon->servers; serv; serv = tmp)
            {
-             /* Need to move down? */
-             if (serv->next)
+             tmp = serv->next;
+             if ((serv->flags & SERV_MARK) &&
+                 hostname_isequal(alloc_domain, serv->domain))
                {
-                 *up = serv->next;
-                 daemon->servers_tail->next = serv;
-                 daemon->servers_tail = serv;
-                 serv->next = NULL;
+                 /* Need to move down? */
+                 if (serv->next)
+                   {
+                     *up = serv->next;
+                     daemon->servers_tail->next = serv;
+                     daemon->servers_tail = serv;
+                     serv->next = NULL;
+                   }
+                 break;
                }
-             break;
+             else
+               up = &serv->next;
            }
-         else
-           up = &serv->next;
        }
 
       if (serv)


Groeten
Geert Stappers
-- 
Silence is hard to parse