Wireless Features

 

Introduction

In Part 1, I provided details on how my OFDMA benchmark test was developed. In this second and final part, I'll show the results for two Wi-Fi 5 (802.11ac) and six Wi-Fi 6 (802.11ax) routers that were run through the process.

I used the last variation of the test described in Part 1. In sum:

• iperf3 TCP/IP traffic run simultaneously to four STAs (Intel AX200, Win 10, 21.80.2.1 driver)
• bitrate (-b): 50Mbps
• length (-l): 256 Bytes
• DSCP values (--dscp), one per STA: 0 (CS0), 96 (CS3), 160 (CS5), 192 (CS6)
• 200ms interval ping run concurrently from AP to STA on each pair. Ping is always DUT to STA for both uplink and downlink traffic.
• congestion measured using an octoScope Pal6 (Qualcomm AX based) associated to the DUT, running 1bps of traffic so that stats could be recorded.

Table 1 shows the routers tested, with key attributes.

Product	Streams	Platform	OFDMA				Firmware	Notes
			2.4 GHz DL	2.4 GHz UL	5 GHz DL	5 GHz UL
ASUS RT-AX58U / RT-AX3000 [$180]	2 (4, 5 GHz Rx only)	Broadcom	Yes	Yes	Yes	Yes	3.0.0.4.384_8601	BCM43684: 5 GHz BCM6750: 2.4 GHz
ASUS RT-AX88U [$350]	4	Broadcom	Yes	Yes	Yes	Yes	3.0.0.4.384_9107	BCM43684: both radios
ASUS GT-AX11000 [$450]	4	Broadcom	Yes	Yes	Yes	Yes	3.0.0.4.384_9165	BCM43684: all radios
Evenroute IQrouter V3 [$139]	2	Mediatek	N/A	N/A	N/A	N/A	v3.3.2	MT7610E: 5 GHz MT7603E: 2.4 GHz Wi-Fi 5 router. No OFDMA
NETGEAR R7800 [$170]	4	Qualcomm	N/A	N/A	N/A	N/A	V1.0.2.68	QCA9984: both radios Wi-Fi 5 router. No OFDMA
NETGEAR RAX15/20 [$100]	2	Broadcom	Yes	Yes	Yes	Yes	V1.0.1.56	BCM6755: both radios
NETGEAR RAX45/50 [$330]	2 -2.4 GHz 4 - 5 GHz	Broadcom	Yes	Yes	Yes	Yes	V1.0.2.20	BCM43684: 5 GHz BCM6750: 2.4 GHz
NETGEAR RAX120 [$400]	4	Qualcomm	Yes	No	Yes	No	V1.0.1.122	Qualcomm QCN5054 (5 GHz x2) Qualcomm QCN5024 (2.4 GHz) Chipset not capable of UL OFDMA

Table 1: Routers tested

Since there are so many routers, including all products produces pretty unreadable plots. So I'll be showing separate plots for the ASUS and NETGEAR routers. Both sets of plots will include two Wi-Fi 5 routers and octoScope's Pal6 configured as a four-stream AX AP for reference.

The NETGEAR R7800 is a four-stream Wi-Fi 5 (802.11ac) router that is my "go-to" AC reference router. Whenever I have a product that is behaving oddly, I put the R7800 back in the chamber to ensure that it's the product, not the testbed that is misbehaving.

The OpenWRT-based Evenroute IQrouter V3 was included because it has the codel and airtime fairness work of the MakeWiFiFast group baked into its wireless drivers for its Mediatek chipsets. IQrouter's feature set mainly focuses on optimizing wired routing performance and, specifically, reducing the effects of bufferbloat on the router's ISP connection. But since it also has some Wi-Fi latency reduction, it's included to see how it stacks up against the R7800 and the AX products.

(Not so) funny story; I actually wrote this article twice. The first version, which posted for about 15 minutes before I took it down, was based on bad data. Shortly after the article posted, I went back to the testbed to start to reconfigure it and found one of the channels for one of the STApals was not connected. Yup, just hanging in the breeze. After some verbal self-abuse and a few more dents in the lab wall, I started out to retake the data.

But while I was at it, I figured I might as well make the test configuration better support MU-MIMO. After its years of not adding value to Wi-Fi performance, I'm told that MU-MIMO actually works now, at least the 802.11ac downlink form of it. The 802.11ax standard has added an uplink form of MU-MIMO, but it wasn't supported in the first AX chipsets.

MU-MIMO uses beamforming, which requires spatial diversity, i.e. devices must be physically separated. The testbed didn't support this because all four STApals used the same two antennas.

OFDMA testbed - before

The figure below shows the testbed's new configuration. Pairs of STApals each have their own attenuator and are connected to their own pair of antennas, situated at corners of the 38" octoBox. The unused ports of the 1:4 RF splitter/combiner all have 50 ohm terminators. Ideally, each STApal would have its own antenna pair sitting in a box corner. But even this non-ideal configuration revealed performance differences when MU-MIMO was enabled and disabled in test runs.

OFDMA testbed - after

On to the results.

Latency - ASUS

We'll start with a summary of 80% probability downlink latencies with OFDMA off and on in Table 2.

Product	80% latency - OFDMA off	80% latency - OFDMA on	% Change
Evenroute IQrouter V3	10.1	10.1	-
NETGEAR R7800	8.9	8.9	-
octoScope Pal6	24.5	7.0	-71
ASUS GT-AX11000	5.7	4.2	-26
ASUS RT-AX58U / RT-AX3000	4.6	6.2	+35
ASUS RT-AX88U	4.9	3.4	-31

Table 2: ASUS router OFDMA latency summary - downlink

The first plot shows downlink latency with OFDMA disabled for the ASUS routers and three "reference" products, using an 80% probability benchmark. While all the ASUS AX products are clustered fairly closely together, the reference octoScope Pal6 is not. Note the two AC reference products, the IQrouter v3 and NETGEAR R7800, have similar behavior.

ASUS Latency CDF - OFDMA off - downlink

I rechecked the Pal6 results at least half a dozen times. Some of the results are shown below, with the value used above in bold black.

Pal6 Latency CDF - OFDMA off - downlink - retests

The downlink latency with OFDMA enabled plot confirms the results in the table above. The Pal6 is most improved with latency decreasing 71%. The RT-AX58U is the only router to get worse, increasing 35%. Still, all the ASUS routers still have latencies better than the AC reference routers.

ASUS Latency CDF - OFDMA on - downlink

Turning to uplink latencies, the 80% probability latencies are summarized with OFDMA off and on in Table 3. Once again, the Pal6's latency is most improved, decreasing 57%. And, once again, the RT-AC58U is the only one of the three ASUS routers where latency gets worse.

Product	80% latency - OFDMA off	80% latency - OFDMA on	% Change
Evenroute IQrouter V3	22.6	22.6	-
NETGEAR R7800	24.9	24.9	-
octoScope Pal6	25.3	11.0	-57
ASUS GT-AX11000	17.7	15.9	-10
ASUS RT-AX58U / RT-AX3000	15.1	18.1	+20
ASUS RT-AX88U	17.8	14.9	-16

Table 3: ASUS router OFDMA latency summary - uplink

The uplink latency with OFDMA disabled plot shows all the ASUS routers clearly differentiated from the "reference" products.

ASUS Latency CDF - OFDMA off - uplink

The uplink latency with OFDMA enabled plot shows the big change in Pal6 latency. Once again, the AX products generally have lower latencies than the AC.

ASUS Latency CDF - OFDMA on - uplink

Latency -NETGEAR

Table 4 has a summary of 80% probability downlink latencies with OFDMA off and on for the three AX NETGEAR routers tested. Like the three ASUS routers, all three NETGEAR OFDMA off latencies are lower than the two AC and Pal6 reference products. Latencies for the Broadcom-based RAX15 and RAX45 are about half the Qualcomm-based RAX120, the only Qualcomm-based router in this test.

Product	80% latency - OFDMA off	80% latency - OFDMA on	% Change
Evenroute IQrouter V3	10.1	10.1	-
NETGEAR R7800	8.9	8.9	-
octoScope Pal6	24.5	7.0	-71
NETGEAR RAX15	4.2	3.3	-21
NETGEAR RAX45	4.9	4.9	0
NETGEAR RAX120	7.9	7.7	-3

Table 4: NETGEAR router OFDMA latency summary - downlink

Here's the downlink latency with OFDMA disabled plot...

NETGEAR Latency CDF - OFDMA off - downlink

...and downlink latency with OFDMA enabled, below. There's not much else to say.

NETGEAR Latency CDF - OFDMA on - downlink

Turning to uplink latencies, the 80% probability latencies are summarized with OFDMA off and on in Table 5. Latency improvement is much better than downlink for the RAX120. But enabling OFDMA seems to seriously break the RAX15. To check, I repeated the test, with similar results.

Product	80% latency - OFDMA off	80% latency - OFDMA on	% Change
Evenroute IQrouter V3	22.6	22.6	-
NETGEAR R7800	24.9	24.9	-
octoScope Pal6	25.3	11.0	-57
NETGEAR RAX15	13.9	145.5	+947
NETGEAR RAX45	15.9	13	-18
NETGEAR RAX120	29.4	22	-25

Table 5: NETGEAR router OFDMA latency summary - uplink

The uplink latency with OFDMA disabled plot shows a definite shift to the right (higher latencies) vs. the downlink plot, just as we saw with the ASUS routers. Note the RAX120 with higher latency than the R7800.

NETGEAR Latency CDF - OFDMA off - uplink

The uplink latency with OFDMA enabled plot shows the RAX120 now better than the R7800 and about even with the IQrouter v3. The RAX15 is, as they say, off the chart, but not in a good way.

NETGEAR Latency CDF - OFDMA on - uplink

Conclusion: My takeaway for latency is that it appears OFDMA can provide some reduction in latency. But I suspect the effect would not be noticeable in the real world. It's not like switching OFDMA on is taking 100's of milliseconds of delay and cutting it to 1 ms.