How We Test: SmallNetBuilder’s Wireless Testbed – Revision 10

Photo of author

Tim Higgins

Introduction

Updated 7/7/17 – Reflect proper octoScope Pal product names

The Wi-Fi landscape has shifted significantly since SNB’s Wi-Fi test process was last changed only a year ago. Distributed Wi-Fi (DW) / mesh systems have come on strong, challenging the single point routers that have formed the bedrock of the consumer Wi-Fi market to date. And MU-MIMO has moved down-market into less expensive products that lack connectorized antennas, making our direct-connect test method less practical.

The past year has also shown that having multiple test setups increased test time and was more prone to errors. The return to walking a laptop around to test DW products also increased test time. So some streamlining was in order.

The Revision 10 Testbed

We once again worked with our test partner octoScope to design the testbed for the Revision 10 process. The new testbed got moved up all the way to Revision 10 to line up with the name of the process it is used with.

The new testbed can handle both single point routers and DW products of up to three modules. The top box is unchanged; it’s still octoScope’s Box-38 anechoic RF test chamber equipped with turntable and four octoScope high-gain antennas. Below that, however, everything is new.

SmallNetBuilder Wireless Testbed Revision 10

SmallNetBuilder Wireless Testbed Revision 10

The Multipath Emulator (MPE) has been replaced with octoScope’s MPE2. The block diagram below shows that the multipath emulator can be bypassed to better model behavior in line-of-sight conditions. This also has the benefit of reducing path loss, which helps 5 GHz tests to reach higher maximum link rates. The built-in quadAttenuator has the same 63 dB attenuation range of the quadAttenuator modules. The ability to stack the MPE2’s quadAttenuator with an outboard quadAttenuator provides additional attenuation needed to counteract the additional gain from the high-gain antennas that is not needed for 2.4 GHz testing.

octoScope MPE2 block diagram

octoScope MPE2 block diagram

Below the MPE2 are three new octoScope Box-18 RF chambers and a new TS (Test System) -MESH module detailed in the diagram below. The variable attenuation symbols represent octoScope quadAttenuators. The blocks marked 1:2 and 1:4 are octoScope RF splitter / combiners.

octoScope TS-MESH block diagram

octoScope TS-MESH block diagram
Updated 7/7/17

The Pals are octoScope’s partner devices that can function in AP, STA or Monitor mode and support up to 32 virtual STAs. Each is based on a Qualcomm Atheros chipset, has an onboard processor and runs both an iperf3 server and IxChariot endpoint.

The Pal-245 is a selectable dual-band 3×3 device that supports up to three transmit / receive chains, 20, 40 and 80 MHz bandwidth and 256 QAM on both bands. It does not support MU-MIMO or 80+80 or 160 MHz bandwidth.

The Pal-24 and Pal-5 are 2.4 and 5 GHz 4×4 devices, respectively, that support up to four transmit / receive chains, 20, 40, 80, 80+80 and 160 MHz channel bandwidth and 256 QAM. Pal-5 also supports 1×1 and 2×2 MU-MIMO.

A key feature of all Pals is they can report / log all the status information you see in the screenshot below including beacon and data RSSI, BSSID, channel congestion and transmit and receive link rates.

octoScope Pal web interface

octoScope Pal web interface

Here’s a physical view of the TS-MESH module. That’s a TL-SG1008PE PoE+ switch powering the quadAttens and Pals in the TS-MESH. There is also a TP-Link TL-SG108E not shown that was added to power other octoScope modules and support network connection for other parts of the system.

Closer view of TS-MESH module

Closer view of TS-MESH module

Computer

Since a computer is no longer needed on the client end (each Pal has its own processor), we’ve reassigned the two Dell Optiplex SFF machines to wired router test duties. A Lenovo T450S laptop (Intel Core i5-5200 @ 2.2 GHz, 8 GB RAM, Windows 10 Pro 64-bit) now runs the whole shebang. The inboard Intel gigabit Ethernet adapter is used for test traffic; a multi-homed Anker Y 3461 USB 3.0 Gigabit Ethernet adapter handles control and house LAN connection.

Box-38 Detail

The signals from the device under test (DUT) are picked up by an octoScope High Gain MIMO Antenna Array shown on the right in the photo below. This consists of 4 log-periodic antenna elements mounted on a plastic rail. These antennas are optimized for 2-6 GHz operation and offer 10 dB of gain in this band.

Only the center two antennas are used for throughput vs. attenuation testing with the octoScope Pal set to 2×2 mode. All four antennas are used when testing maximum wireless throughput with the Pal set to 4×4 mode.

ASUS RT-AC88U router inside Box-38 RF chamber

ASUS RT-AC88U router inside Box-38 RF chamber

The antennas are equally spaced with the midline of the turntable bisecting the space between the center two antennas. And yes, the bamboo cable rod is still in service! The antennas have a new mounting system that is much easier to adjust.

Box-38 antenna detail

Box-38 antenna detail

Box 18 Detail

octoScope’s Box-18 is a lot smaller inside. So to maximize room for product under test, the chamber is equipped with simple unity-gain dual-band dipole antennas. Even though signal levels are lower than with octoScope’s high gain antennas, the antennas are so close to the product under test that there is still plenty of signal. Each antenna is on a carrier that can be easily slid on its mounting rail and locked down using a thumb screw.

eero inside Box-18 RF chamber

eero inside Box-18 RF chamber

Traffic Generation

We have long used Ixia’s IxChariot as our throughput traffic generation and measurement tool. I’m thankful that Ixia has provided IxChariot for over 15 years at no charge. But it’s now time to make a change for at least some of our test processes. The V10 process uses the open source iperf3 tool. The main reason for the change is better correlation with other octoScope users, who prefer iperf3’s free price tag.

That concludes the tour of the Revision 10 testbed. How We Test Wireless Products – Revison 10 (coming soon) describes how we use it.

Related posts

How We Test Wireless LAN Products – Five Location Open Air Method

In late 2005, I moved into a new home and had to establish new test locations. Since this essentially wiped the slate clean in terms of being able to compare wireless test results going forward to those from my previous reviews, I also decided to establish new test methodology and add a scoring system.

How To Crack WEP – Part 1: Setup & Network Recon

It's common knowledge that WEP can be "easily" cracked. But did you ever wonder how easy it is? In Part 1 of a two-part series, Humphrey Cheung takes you step-by-step through putting together a WEP-cracking setup and using Kismet to find vulnerable wireless LANs.

How Powerline Can Solve Your Wi-Fi Woes

There's a better way to get WiFi Everywhere™.