How We Test Hardware Routers – Version 4

Photo of author

Tim Higgins

This process applies to routers tested after 15 May 2016. The previous process can be found here.

Updated 1/29/17 – Removed UDP tests due to process error

It has been just over six years (!) since we last revised our wired router test process. Since then, many routers now routinely max out our simple throughput tests. So it’s time to work these products a lot harder.

The new process does away with the pair of computers and IxChariot, instead using QA Cafe’s NTA1000, and flagship CDRouter router test solution.

QA Cafe NTA1000

QA Cafe NTA1000

CDRouter is a Linux-based router test suite that includes over 4000 functional and performance tests. The video below provides a short overview of CDRouter.

CDRouter Overview


We test all routers by directly connecting WAN and LAN ports to the NTA1000’s Gigabit Ethernet ports. Note the same test setup is used for both wired-only and wireless routers, i.e. the wireless portion of wireless routers is not involved in this testing.

We do not test LAN – LAN, i.e. the router’s switch, performance. The switch chips used in today’s routers all support non-blocking wire-speed performance.

Router Test Setup

Router Test Setup

All routers are prepared for test as follows:

  • Upgraded to latest firmware
  • Reset to factory defaults
  • QoS features disabled (if present)
  • Remote administration disabled (if enabled)
  • WAN type set to DHCP client
  • DoS protection enabled (if present)
  • LAN DHCP server range set to – 30
  • LAN DHCP server lease time set to 120 minutes (if available)
  • All ALG passthroughs enabled
  • Remote management (HTTP / HTTPS) disabled (if enabled, if available)
  • Local HTTPS management enabled (if available)
  • TCP port 21 forwarded to
  • UDP port 988 forwarded to
  • Triggered port forwarding (if available) set for Dialpad
    • Trigger port = 7175
    • Trigger type = UDP
    • Opened ports = 51200-51201,51210
    • Opened port type = TCP, UDP
  • Triggered port timeout set to 60 seconds (if available)


We run these tests, selected from the CDRouter perf.tcl IPV4 performance test module:

  • TCP download throughput test WAN to LAN
  • TCP upload throughput test LAN to WAN
  • TCP bidirectional throughput test
  • UDP download throughput test WAN to LAN
  • UDP upload throughput test LAN to WAN
  • UDP bidirectional throughput test
Update 1/27/17: UDP benchmarks were removed when it was found incorrect versions of the UDP benchmark tests were run for many of the products tested.

Each test is run with 10 simultaneous connections with each connection presenting a 95 Mbps load. Each test runs for 5 minutes. Note that CDRouter uses iperf to measure throughput.

For TCP throughput tests, we report throughput and retries. For UDP throughput tests, we report throughput and % packet loss. CDRouter reports up and download throughput separately for the bidirectional tests. The Router Charts bidirectional benchmarks present the sum of the separate results.

We have retired the Maximum Simultaneous Connection test. We now test whether 3,000 simultaneous TCP and UDP sessions are supported in separate tests. This new test is actually harder than the previous maximum connection test because it opens the connections over a longer period of time, then goes back and checks they are still open.

Functional Tests

CDRouter’s primary focus is router functional testing. We run 245 tests, checking DNS, DHCP, UPnP, Xbox, NAT, firewall and other functions. We also run nmap scans from LAN and WAN to check for open ports. You can download a PDF of the full test list to see exactly what we run. Note we start the test by throwing a series of DoS attacks at the router, giving it 30 seconds to recover from each one.

In the Router Charts, we report the results in the Router Functional benchmark as a % of tests passing. The list of failing tests with explanation is noted in the product review.

Related posts

LAN Party How To – Part 3: LAN Management and Troubleshooting

Chris Dickens' series has so far shown you how to organize a smooth-running LAN party and get it powered properly. This time he offers tips on choosing networking gear, installing it and handling the problems that come with the territory.

Doing More with IPv6

In Part 2 of our series, we dig deeper into IPv6 addressing, debunk some firewall and IPsec myths and get some IPv6 VLANs working.

Measuring Network Performance – Iperf

Updated - Iperf is a network performance tool used for measuring the bandwidth and the performance of TCP and UDP data flows between two endpoints.