I had two requirements for the P2P stress test. The first was that the test had to be done without requiring connection to the Internet. While this might seem strange for applications that depend on interaction with numerous partners scattered across the world, it's the only way that I can ensure that I'm testing the router and not dozens of Internet connections and their associated ISP's.
The second requirement for the P2P test was that it had to force the router to establish and maintain more than 100 simultaneous connections. Both P2P and gaming applications open multiple simultaneous connections through a router's firewall, with each session requiring CPU cycles and, more importantly, a chunk of not-so-plentiful memory.
For example, a search for available gaming servers might involve launching a few hundred server queries. Although the queries themselves are pretty small and don't require a lot of router throughput, the sheer number of them can cause routers with buggy code to lock up and require rebooting. On the other hand, the bandwidth of most broadband connections limits the number of useful simultaneous BitTorrent or eDonkey sessions to a few dozen (or even fewer). But each connection is a steady stream taking as much bandwidth as the router (and its Internet connection) can provide.
Given these requirements and the tools available to me, I chose Ixia's IxChariot as my instrument of router torture because it could handle up to 180 simultaneous connections and is a breeze to set up and modify. And its plots of throughput vs. time are handy to monitor both the in-process and finished test runs. I used the standard throughput.scr script with TCP/IP with the only modification to change the file_size parameter (the size of the transferred file) from its default of 100,000 to 1,000,000 Bytes.
Two computers were involved in the test, which were connected into my lab's switch. The LAN-side machine also ran the IxChariot console and was a Dell Inspiron 4100 notebook with a 1 GHz Intel Celeron CPU, 576 MB of RAM and integrated 3Com 3C920 Ethernet adapter, running WinXP SP2 Home. The WAN-side machine was an HP Pavilion 716n with a 2.4 GHz Intel Pentium 4 CPU, 504 MB of RAM and Intel PRO / 1000 MT desktop Ethernet adapter, also running WinXP SP2 Home.
The WAN-side machine and WAN port on the router under test were assigned static IP addresses so that the test scripts could easily be re-used. The LAN-side machine was allowed to pick up whatever IP address was handed out by the router under test. Because IxChariot has known problems with NAT + SPI routers, I found that I had to both put the LAN side machine in DMZ in order to get the tests to run correctly. I also had to disable the SPI features of each router if it allowed me to do so.
For each router, I upgraded to the latest firmware and reset the router to factory defaults. Next, I opened the ports that IxChariot needed, put the LAN machine in DMZ and disabled the firewall SPI features if I could. I then attempted to run five tests with 2, 64, 128, 160 and 180 simultaneous connections.
In each case, I evenly split the connections between upload and download and ran the tests for one minute. While an even split between upload and download connections isn't exactly the scenario that a router would encounter in a P2P application, I figured it would suit my purposes.