Header Shadow Image


DD-WRT: Network Performance Tuning

It became apparent that with the growing push for more content on web pages and general media content, my router began to perform rather inadequately.  Inadequately to the point where it was rebooting spontaneously.  Below is what the situation looked like.  Below is an example of high SIRQ's inundating the environment: 

Mem: 116784K used, 396932K free, 432K shrd, 84K buff, 13164K cached
CPU0:  0.0% usr  0.0% sys  0.0% nic  0.0% idle  0.0% io  0.0% irq  100% sirq
CPU1:  0.0% usr  2.5% sys  0.0% nic 96.0% idle  0.0% io  0.0% irq  1.2% sirq
Load average: 3.53 1.16 0.49 4/112 9210
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 1068     1 root     R     1496  0.2   0 35.9 syslogd -Z -L -R 192.168.0.14
 2866     2 root     SW       0  0.0   1  0.6 [kworker/1:0]
 6865     1 root     S     1524  0.3   1  0.2 watchquagga -dz -r %s -d zebra ospfd bgpd ripd
  206     2 root     SW       0  0.0   1  0.1 [kswapd0]
 8677  2731 root     R     1520  0.3   1  0.0 top -d 10

Mem: 113748K used, 399968K free, 288K shrd, 8084K buff, 13796K cached
CPU0:  0.0% usr  0.0% sys  0.0% nic  0.0% idle  0.0% io  0.0% irq  100% sirq
CPU1:  0.0% usr  0.0% sys  0.0% nic 83.3% idle  0.0% io  0.0% irq 16.6% sirq
Load average: 0.92 0.60 0.24 3/115 2585
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    3     2 root     RW       0  0.0   0 40.9 [ksoftirqd/0]
  735     1 root     D     1624  0.3   1  3.1 watchdog
 2585  2580 root     R     1512  0.2   1  0.1 top -d 10
  721     2 root     RW       0  0.0   1  0.1 [dhd_watchdog_th]

Also the Destination Host Unreachable due to the load:

[1668315026.612617] 64 bytes from 192.168.0.16: icmp_seq=2996 ttl=64 time=14.4 ms
[1668315027.996485] 64 bytes from 192.168.0.16: icmp_seq=2997 ttl=64 time=396 ms
[1668315057.607704] From 192.168.0.114 icmp_seq=3024 Destination Host Unreachable
[1668315057.607796] From 192.168.0.114 icmp_seq=3025 Destination Host Unreachable
[1668315057.607808] From 192.168.0.114 icmp_seq=3026 Destination Host Unreachable

As such, decided to set of on a mission to improve the performance of all my network routers.  Here are the settings used:

Shortcut Forwarding Engine: CTF ( If this results in OOM, use SFE )
STP: Disabled

MTU:    Manual    1452

# nvram show|grep -Ei ctf_fa_cap=1
size: 55223 bytes (75849 left)
ctf_fa_cap=1

NOTE: The last option may or may not be ideal for your router, depending if Flow Acceleration (FA) module is included in your setup and your router supports it.

Additionally, also scan the DD-WRT remote logs (You did setup rsyslog to a remote server right?) which can tell you, amongst other things excessive requests or packet storms and how many DNS queries occurred in 5 minutes (this is alot):

# grep -Ei "DPT=53" dd-wrt-inet.mds.xyz.log|wc -l
4155
#

Or 30K in 3 hours.  That's alot:

# grep -Ei "DPT=53" dd-wrt-inet.mds.xyz.log | wc -l
30022
#

To solve the above DNS queries problem, you can either tune the DNS masquerade on DD-WRT, if you use it, or adjust the DNS caching on your internal DNS servers.  Here's a Windows Server example:

PS C:\Users\Administrator.WINAD01.000> Get-DnsServerCache
MaxTTL                           : 1.00:00:00
MaxNegativeTTL                   : 00:15:00
MaxKBSize                        : 0
EnablePollutionProtection        : True
LockingPercent                   : 100
StoreEmptyAuthenticationResponse : True

PS C:\Users\Administrator.WINAD01.000> Set-DnsServerCache -MaxKBSize 65536
PS C:\Users\Administrator.WINAD01.000> Get-DnsServerCache
MaxTTL                           : 1.00:00:00
MaxNegativeTTL                   : 00:15:00
MaxKBSize                        : 65536
EnablePollutionProtection        : True
LockingPercent                   : 100
StoreEmptyAuthenticationResponse : True

PS C:\Users\Administrator.WINAD01.000> Set-DnsServerCache -MaxKBSize 65536 -MaxTtl 0x15180
WARNING: The input value for the setting MaxTtl is lesser than a second and will be ignored.  The input value must be
in the format DD.HH:MM:SS where DD is days, HH is hours, MM is minutes and SS is seconds.

PS C:\Users\Administrator.WINAD01.000> Set-DnsServerCache -MaxKBSize 65536 -MaxTtl 2.00:00:00
PS C:\Users\Administrator.WINAD01.000> Get-DnsServerCache


MaxTTL                           : 2.00:00:00
MaxNegativeTTL                   : 00:15:00
MaxKBSize                        : 65536
EnablePollutionProtection        : True
LockingPercent                   : 100
StoreEmptyAuthenticationResponse : True

PS C:\Users\Administrator.WINAD01.000>

To set it to 2 days and something other then 0, which effectively, it seems, would turn this off.  Likewise for FreeIPA / IDM, use the following to adjust the DNS cache: 

NOTE: A word about OOM when using CTF.  Appears these OOM messages followed by reboots on one of the routers prompted me to change back to SFE:

# cat dd-wrt-roma.mds.xyz.log|grep -Ei oom_kill
Nov 21 02:54:27 dd-wrt-roma.mds.xyz kernel: [20094.748505] [<80014094>] (dump_header) from [<800b7344>] (oom_kill_process+0xec/0x3cc)
Nov 21 02:54:27 dd-wrt-roma.mds.xyz kernel: [20094.766870] [<800b7258>] (oom_kill_process) from [<800b78f0>] (out_of_memory+0x260/0x344)
Nov 21 04:06:13 dd-wrt-roma.mds.xyz kernel: [ 4191.173207] [<80014094>] (dump_header) from [<800b7344>] (oom_kill_process+0xec/0x3cc)
.
.
.
Nov 23 22:15:56 dd-wrt-roma.mds.xyz kernel: [47881.130510] [<80014094>] (dump_header) from [<800b7344>] (oom_kill_process+0xec/0x3cc)
Nov 23 22:15:56 dd-wrt-roma.mds.xyz kernel: [47881.148868] [<800b7258>] (oom_kill_process) from [<800b78f0>] (out_of_memory+0x260/0x344)
#
# cat dd-wrt-inet.mds.xyz.log|grep -Ei oom_kill
Nov 25 03:21:21 dd-wrt-inet.mds.xyz kernel: [172011.430393] [<80014094>] (dump_header) from [<800b7344>] (oom_kill_process+0xec/0x3cc)
Nov 26 04:07:38 dd-wrt-inet.mds.xyz kernel: [89063.941579] [<80014094>] (dump_header) from [<800b7344>] (oom_kill_process+0xec/0x3cc)
Nov 26 04:07:38 dd-wrt-inet.mds.xyz kernel: [89063.941594] [<800b7258>] (oom_kill_process) from [<800b78f0>] (out_of_memory+0x260/0x344)
#

EDIT: Nov 27th 2022

Appears that networking topology has alot to do with the performance as well as DNS caching above.  See posts below:

https://i0.wp.com/www.microdevsys.com/WordPressImages/NetworkTopologyNov27-2022.PNG?ssl=1

Cheers,
TK

REF: 
https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1275963#1275963
https://wiki.dd-wrt.com/wiki/index.php/Hardware#Flow_Acceleration.2C_SFE_and_Cut-Through_Forwarding
https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=329793&sid=86de7e78395bca1b71a162cf5b82c1ef
https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=85265
https://learn.microsoft.com/lb-lu/powershell/module/dnsserver/set-dnsservercache?view=winserver2012-ps
https://learn.microsoft.com/lb-lu/powershell/module/dnsserver/get-dnsservercache?view=winserver2012-ps

https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1276305#1276305

https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1276303#1276303 

Leave a Reply

You must be logged in to post a comment.


     
  Copyright © 2003 - 2013 Tom Kacperski (microdevsys.com). All rights reserved.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License