Updated August 2003
We use Ixia‘s free Qcheck utility and Chariot network peformance evaluation program to put HomePlug power line network gear through their paces. We use the simple test setup pictured below to run our tests under four different conditions.
Test Conditions
Testing is done in a wood frame two floor residence of recent-construction with 200Amp split-phase electrical service, no whole-house surge protection installed. A HomePlug to Ethernet Bridge (the “reference bridge”) is used to provide a HomePlug connection into our Ethernet test LAN and is plugged directly into a wall outlet in our office test area. There is an assortment of networking gear and computers plugged into three APC UPS units on the same power circuit, and usually two to three computers running. A HomePlug adapter or Bridge (from the same manufacturer unless other wise noted) is attached to a client laptop and measurements are taken at four locations.
- Condition #1: Client in same room as reference bridge, both plugged directly into same power outlet (same branch circuit, same mains phase).
- Condition #2: Client in upper floor room directly above, connected to different branch circuit, same mains phase.
- Condition #3: Client in upper floor room, opposite end of residence, approximately 60 feet away, different branch circuit, different mains phase.
- Condition #4: Client on upper floor external porch, opposite end of residence, approximately 60 feet away, different branch circuit, different mains phase.
Tests Run
1) Transfer Rate – More commonly known as Throughput, this test is a measure of how fast data moves between the two test partnets. The test sends a file from client to client, measures how much time it takes, and calculates the result in Mbps (Megabits per second). Our test sends one MegaByte of data (the most that Qcheck will do). Higher numbers are better.
2) Response Time – This test measures the delay (also known as lag, or latency) in the test setup, and is essentially what you’d measure by using the ping command. This test sends a small packet of data from one test partner to the other, and measures the time it takes to receive a reply. Our test setup runs the test 10 times in a row and calculates the average and maximum times. Lower numbers are better, especially for gaming and any voice or video applications, but anything under 10ms (milliseconds) is fine.
3) UDP Stream – This test measures how well a continuous stream of data is handled. In addition to giving an indication of whether you’ll have trouble listening to Internet audio or watching video program streams, it tends to show whether there is a high bit error rate, or poor speed autoranging algorithm. It uses the connectionless UDP protocol, which has less overhead and error recovery mechanisms than the TCP protocol (picture a fire hose being turned on vs. a water bucket brigade). Our test setup pushes data at a 500 kbps (0.5Mbps) rate for 10 seconds and results in two numbers. You want the Actual Throughput number to be as close to 500 kbps as possible and the Lost Data to be ideally zero.
Tests are run from the Qcheck console in the laptop, with data flow from the laptop to the Ethernet LAN client, which is running the Qcheck endpoint.
We also sometimes use Ixia’s Chariot network peformance evaluation program to run longer transfer rate/throughput tests and then plot the results. This can be helpful to get a better idea of how a network performs with large file transfers, and also the variation in throughput during an extended period of time.We use the Throughput.scr script (which is an adaptation of the Filesndl.scr long file send script) unmodified, and run the script for one minute in batch mode and don’t poll the endpoint. We then use the Chariot Comparison program to generate comparison plots and statistics for various groups of tests.
Equipment Used
We tend to use the same computers to run the tests, with the Ethernet client running Win98SE or WinXP, and having 300MHz or better processor speeds and memory configurations in excess of 256MB. The test laptop, however, has a 266MHz Pentium, with 96MB of memory, running WinXP Home. The test machines have no other applications running during testing.