Multi-client Streaming Test
The pain many Wi-Fi users feel is from everyone in their household trying to watch streaming video at the same time. Success depends on the services you're using, the protocols and stream rates they use, how big your internet pipe is and, yes, the robustness of your Wi-Fi network. I first considered using streaming video to load up eero. But the thought of trying to watch all those screens for signs of buffering or pixelation made me rethink that approach.
Instead, I turned to IxChariot and used UDP streaming. While I wouldn't have anything to watch, this method would keep track of lost packets. It's been almost 10 years (yikes!) since I last looked at the relationship of packet loss to video quality. But this study showed, depending on encoding method, 1% of UDP packet loss would make video unwatchable and small problems may be seen with as little as 0.05% packet loss!
The mix of eight dual-band N and AC class devices in Table 3 was assembled for this test.
|Device||Type||Connection - Utility test||Connection - Back Lab test||Comment|
|Acer Aspire S7 Ultrabook||2x2 N||Ch 36||Ch 36||Intel Wireless N 7260|
|Lenovo x220i notebook||3x3 N||Ch 1||Ch 1||Intel Centrino Ultimate-N 6300|
|Lenovo T450s notebook||2x2 AC||Ch 1||Ch 1||Intel Wireless AC 7265|
|Moto X Gen 2 smartphone||1x1 AC||Ch 36||Ch 1|
|Dell Venue Tablet||1x1 AC||Ch 1||Ch 1|
|iPod Touch Gen 5 (A1509)||1x1 N||Ch 36||Ch 36||Broadcom BCM4334 in Murata 339S0171 Wi-Fi module|
|iPad Air (A1474)||2x2 N||Ch 36||Ch 36||Broadcom BCM4324 in USI 339S0213 Wi-Fi Module|
|Galaxy Tab S 10.5 (SM-T800)||2x2 AC||Ch 1||Ch 1|
Table 3: Streaming test client devices
An IxChariot script was configured using the IPTV video portion script (IPTVv.scr) and UDP. The only thing changed was the send data rate from its default of 1.451 Mbps. All eight devices received this same stream for 2 minutes. All devices could connect to either band. They connected as shown in Table 4 and stayed this way from run to run. No other networks were in range for this test.
UDP Streaming test IxChariot script configuration
For the baseline test, one eero ("Root") was set up in an adjacent Utility room with a sheetrock wall between it and all the devices sitting on a wire-frame rack approximately 13 feet away. Packet loss for each device is shown in the IxChariot plot below.
Only the iPod Touch, Lenovo x220i and iPad lost packets, with the Lenovo having the highest data loss of almost 1.5%, which should have rendered the video unwatchable. Please read iPad2 as iPad Air in all the plots; I incorrectly identified it at test time.
UDP Streaming test - Utility room - 5 Mbps
Increasing the stream rate to 10 Mbps significantly increased the Lenovo's error rate to 12%. The two Apple devices had no errors in this test.
UDP Streaming test - Utility room - 10 Mbps
I decided to move the Root eero to the same room as the devices (Back Lab) and rerun the test, this time for 5 minutes. Once the eero was powered back up, I again noted where devices connected and found only the Moto X smartphone had moved from channel 36 to 1.
This small change seemed to have an unusually large effect on error rate with multiple devices losing over 10% of streamed data. Without changing anything, I reran the test about 20 minutes later and found performance had improved significantly. Although the plot below shows all devices having some packet loss, only the problem child Lenovo x220i had data loss over 1%.
UDP Streaming test - Back Lab - 10 Mbps
Although I can't say I pushed eero to its limit, my conclusion for these tests is that the single eero handled itself pretty well. It moved an aggregate capacity of 80 Mbps spread across a mix of eight devices with potentially noticeable data loss on only one or two devices. If TCP/IP, HLS or another protocol that ensures packet delivery were used, I doubt problems would be seen in any of the video streams.
Multi Hop Tests
eero's key claim to fame is its wireless mesh connected multi access point architecture. So this next set of tests set out to explore the throughput a few different eero configurations would yield. Since I was looking at throughput for a single device, I went back to my old standby, IxChariot's TCP/IP throughput script. The test client was a Lenovo T450s notebook with a built-in Intel Wireless AC 7265 2x2 wireless adapter.
I tested four configurations shown in the floor plan diagrams below, with the test client connected using zero to two wireless "hops". The three eeros were creatively named "Root, "One" and "Two".
Key parameters of the four test configurations are summarized in Table 4. For each test, I made sure the test laptop was connected to the eero shown before starting the test. I had no way of tracking inter-eero connections or traffic
|Test name||eeros used||Connection||Link rate (Mbps)||Hops|
|root||Root||Root, Ch 1||300||0|
|root-one||Root, One||One, Ch 36||867||1|
|root-one-two_butler||Root, One, Two||Two, Ch 36||867||2|
|root-one-two_master||Root, One, Two||Two, Ch 1||300||2|
Table 4: Wireless Hop Test parameters
The IxChariot plot below combines the downlink results for the four runs. What's interesting here is that the one-hop "root-one" throughput is pretty close to the two-hop root-one-two_butler test. I'm not sure whether it makes a difference, but note the test laptop connected via 5 GHz for both these tests. Also note the "Two" eero was located pretty much right above the "One" eero for the root-one-two_butler test.
The root-one-two_master test shows the importance of getting your eeros placed correctly. Everything else in this test is the same; only the location of the "Two" eero changed.
Multi-hop test - downlink
The uplink plot shows a very different story. Throughput for three of the four tests is highly variable and at best only half the 179 Mbps obtained in the best-case zero-hop downlink root test.
Multi-hop test - uplink
Mesh wireless is attractive to home users for the same reason wireless extenders have been flying off store shelves; no cabling required. So why should you pay $400 for two eeros with a very limited routing feature set, when you can buy the top-ranked AC1200 class TP-LINK Archer C5 router for around $80 and the top-ranked NETGEAR EX6150 AC1200 extender for around $100?
The answer boils down to ease of use and reliability. While wireless extender setup has become a lot easier, it still requires judgement by the installer, who has only fairly crude tools to help her. The other problem with extenders is that the only "tools" provided to monitor network performance post-installation are user complaints and confusion on which SSID to connect to. Although backhaul management is provided in some newer extenders, it's fairly crude and will only improve if and when new firmware is available and installed.
In contrast, eero is designed as a multi-AP system that has had much effort put into learning how to manage backhaul traffic, plus a purpose-built dual-band radio designed to complement those algorithms. Since the system intelligence primarily lives in eero's cloud, it has access to more computing power than economically feasible to put into a consumer unit and its algorithms can be constantly tweaked as the system learns. Of course, this cloud dependence is also eero's greatest weakness, since an eero APs essentially turns into a glossy white objet d'art, should eero go belly-up or be acquired.
Although some folks think router hacking and tweaking is their idea of a good time, most of us just want our Wi-Fi to work. And we've shown we are willing to throw more and more money at achieving this result. You may think it's crazy to pay $500 in hope of getting better Wi-Fi, but that's what some folks were doing this past holiday season when ASUS RT-AC5300s were in short supply. And while those days seem behind us, price trackers still show RT-AC5300 ASPs around $375.
Given my albeit limited previous experience with business-grade Meraki mesh wireless, I was prepared to not be impressed by eero. But I am. Yes, it's twice as expensive as it needs to be, but that's just a matter of eero seizing an opportunity while it can. With AC1200 class routers now down around $100, I'm sure eero has room to move on price. If they got a three-pack down to $300 - $350 and a single to $125, eero could sell a lot more and secure its place as a mesh-wireless market leader.
And they'll want to do that before Luma ships its mesh solution later this "spring" and D-Link starts shipping its "DKT-891 Unified Home Networking Kit" based on Qualcomm's Wi-Fi SON technology in around the same timeframe. Both will appeal to the informed Wi-Fi buyer who is already sold on the multi-AP approach and both have more routing and security features to offer than eero currently does.
Price aside, eero is not going to be for everyone. If you have used your current' router's administration interface for anything other than setup, you probably won't be able to deal with eero's very basic feature set. You'll also want to place the Ethernet-connected eero between the other two, to minimize the number of throughput-sapping hops your devices see. And if you don't have many dual-band devices, they'll compete for 2.4 GHz bandwidth that eero could otherwise use for backhaul.
Just as every auto maker is hard at work at producing vehicles that can drive themselves or at least take over more mundane driving tasks, consumer router makers do realize the days of the big honkin' router are coming to a close. Sure, they'll keep making them, as long as uninformed consumers keep buying them. However at some point folks are going to get wise to the fact that to get fast, reliable Wi-Fi you need multiple access points and the smarts to make them work smoothly together.
Quite frankly, most of us don't have those smarts, but there are folks who do. The trick is to let those folks do the Wi-Fi driving. eero is worth taking for a spin.