|At a glance|
|Product||Luma Surround Wi-Fi System (WRTQ-329ACN) [Website]|
|Summary||Multi AP system using Wi-Fi mesh.|
|Pros||• Internet content filtering
• Keeps Wi-Fi running without internet connection
• Protects against malware, spyware & more
|Cons||• Less flexible radio configuration than eero
• Unimpressive Wi-Fi performance
Updated 8/2/16 – Wireless retest done.
Updated 7/30/16 – Wireless tests invalid. Retest in progress
Luma is the second "mesh" Wi-Fi product to become available, trailing first-to-market competitor eero that started shipping in March. These systems basically use multiple access points (APs) to provide more consistent and higher bandwidth Wi-Fi over a larger area than a single wireless router can. At least that’s what they are intended to do.
To keep installation simple, "mesh" Wi-Fi uses wireless connections among the modules spread throughout your home. And there’s the rub. The "backhaul" connections between APs compete for the same bandwidth as the devices you’re trying to get good Wi-Fi to. And for cost reasons, these products don’t use dedicated radios to get packets from your devices, then AP-to-AP and finally to the single unit connected via Ethernet cable to your existing router or modem. How well all these connections are managed is each manufacturer’s "secret sauce" that is still being cooked up, using your home as the test kitchen.
Since many of you are trying to decide whether to try Luma or eero (or whether you should give Luma a shot after eero has disappointed), I’ll be making frequent comparisons between them as we go along.
Luma and eero took different approaches for physical design; Luma is a hexagon that stands upright while eero is a rounded square that lies down. The photo shows their relative sizes, which are pretty similar. Luma’s official size is 4.6" W x 4.13" H x 1.18" D; eero’s is 4.75" W x 4.75" H x 1.34" D.
Luma and eero side-by-side
Luma’s big ol’ light ring lights up only during setup or when something is wrong. So it won’t serve as a nightlight or bother your spouse with unwanted decoration. But you can’t tell at a glance whether the thing is on, either.
Luma and eero in normal operating positions
The rear panel has a reset pinhole, power port, two Gigabit Ethernet ports and a USB 2.0 port that serves no purpose right now; just like eero. Luma’s Ethernet ports are purpose-specific; the IN gets connected to your modem, while the OUT would connect to a device or switch if you have more then one Ethernet device to connect. eero automatically figures out what to do with each of its ports depending on what it senses is connected.
Luma and eero back panels
Like eero, Luma wants you to buy the three-pack that comes with three power supplies, but only one Ethernet cable. This will run you only $399 vs. eero’s $499. Both companies also sell single units; Luma for $149 and eero for $199.
Since Luma is still busy fulfilling pre-orders until August 26 according to this this blog post, you can’t buy it and expect delivery before then. Don’t go looking for Luma in Best Buy yet either, because all available stock is going to filling preorders. I should note I purchased my Luma three-pack (as I did with eero) last November to ensure I’d get product to review sooner vs. later.
Luma’s hardware design is simpler and has fewer radio connection options than eero’s. Luma’s FCC docs indicate it is manufactured by Taiwan ODM Gemtek, which has requested short-term non-disclosure for the internal photos. So I opened up an unused Luma for a look-see.
The design is a straightforward 2×2 AC1200 class router, with two internal dual 2.4 GHz printed circuit antennas positioned on the upper left and right hexagon panels and two 5 GHz printed circuit antennas on the other side of the board on the lower left and right hexagon panels. A single antenna for the Bluetooth 4.0 LE radio sits on the bottom panel.
The RF front-ends and RAM are in RF cans; the flash memory, processor / radio SoC and Ethernet switch are not. The latter two components are physically coupled to a heatsink plate by the two thermal pads shown in the photo below.
The next shot shows the RF cans removed, but you can’t make out most part numbers. The 5 GHz radio connections are at the top of the photo and connect to the antennas on the other side of the board via black cables. The 2.4 GHz connections are right below the 5 GHz and connect to their antennas via grey cables.
In contrast, eero has a tri-radio design with one 2.4 GHz and separate 5 GHz low and high-band radios. This gives is more flexibility to manage device and backhaul connections than Luma.
The component summary below shows the key components of both Luma and eero. Luma appears to have a cost advantage over eero, perhaps why it has been more aggressive in lowering its price.
|CPU||Qualcomm IPQ4018 2×2 a/b/g/n/ac SoC||Qualcomm dual-core IPQ8062 @ 1 GHz|
|Switch||QCA8075||Qualcomm Atheros QCA8337|
|RAM||256 MB||512 MB|
|Flash||128 MB GigaDevice 5F1GQ4UCYIG
2 MB GigaDevice serial flash 25Q16CS1G
|4 GB / 8 MB|
|2.4 GHz Radio||2.4 GHz: in SoC + Skyworks RFX8425 2.4 GHz RF front end (x2)||– QCA9982 2×2 MU-MIMO 802.11abgnac radio
– RFMD RFFM4204 2.4 GHz Front End (x2)
|5 GHz radio||5 GHz: in SoC + Skyworks SKY85716-11 5 GHz RF front end (x2)||– QCA9982 2×2 MU-MIMO 802.11abgnac radio
– RFMD RFPA5522 5 GHz power amp (x4)
|Bluetooth||CSR8510 Bluetooth 4.0 SoC||Atheros AR3012 Bluetooth 4.0 SoC|
Table 1: Component summary and comparison
Luma, however, probably spent more than it should have for its light ring. Two cables plus a separate board with multiple components costs more than eero’s single on-board LED. Not money well spent, IMO.
Luma light board and Bluetooth antenna
Like eero, Luma is designed for people who just want good, fast Wi-Fi and don’t really ever want to twiddle the settings that router geeks love so much. Like eero, Luma uses Android and iOS apps running on a phone or tablet with Bluetooth 4.0 LE radio for setup and administration. If you try to hit Luma’s default 192.168.55.1 IP address with a web browser, you get the screen below. A service scan on Luma’s LAN IP address using Fing revealed these TCP services (ports): DNS (53); HTTP (80); UPnP (5000); and (5001).
Luma doesn’t support web administration
I’m not going to walk through setting up Luma. I’ll just do what Luma does; send you to the video it created.
Luma setup video
There is also this setup guide if you don’t want to watch the video.
I have a few comments about setup, however. First, Luma takes an old school approach to administration authentication, requiring you to set up a password. I prefer eero’s emailed / texted one time code. I really didn’t like that Luma requires locations services be enabled on your phone for setup. If you don’t want to share your location, you can’t proceed with setup.
I first tried setting up with a second generation Moto X smartphone but kept getting stuck looking at Luma’s spinning blue circle. Despite multiple attempts, and rebooting my phone, router and Luma, I just couldn’t get the job done. So I switched to a second generation (ASUS) Nexus 7 tablet, which completed the process without a problem.
Luma supports only DHCP WAN types. If your provider requires your router to use PPPoE, PPTP, L2TP or IPv6 WAN connections, you’ll have to leave your current router in place. Luma’s FAQ aren’t very helpful when it comes to how to handle this situation. There is only one FAQ that refers to bridging, but offers no help in how to do it. Fortunately, Luma doesn’t skimp on other methods of obtaining help, offering 7 day toll-free phone help, a ticket system or plain old email.
Finally, if you connect more than one Luma to your existing network via Ethernet, it will designate one as the internet gateway / router and the others as additional APs, just as eero does. In other words, if you give Luma an Ethernet connection, it will figure out how best to use it. You can also use an Ethernet port on any Luma to connect an Ethernet device or a switch to support multiple Ethernet devices.
Luma’s app has some of the eero app’s features plus a unique one of its own. Setup and admin features are very sparse. About all you can do is see Luma’s LAN IP address and set Wi-Fi network names for the main WLAN and single guest network. That’s it. No port forwarding, QoS, DMZ, DHCP reservations or other things you may depend on. You can’t even check your WAN IP address or see the firmware revision Luma is running.
Luma says these and other features are coming in future updates, for what it’s worth. In the meantime, if you need anything more than basic connectivity, you’ll need to use your current router. When you do that, unfortunately, Luma’s routing functions still run and will create a double-NAT situation, with Luma devices on their own separate network from devices connected to your main router. There is no way to switch Luma to AP-only mode.
Like eero, Luma’s opening screen greets you with a measure of your internet connection up and download speeds, which you can refresh by dragging down from the top of the window below the menu bar icons.
Luma Home and Menus
Where Luma differentiates itself is in its security and filtering features. I submitted a few rounds of questions to Luma about these features. The Q’s and A’s follow.
Q: Can you provide specifics about Luma’s security features? What exploits are detected, what database is used?
A: The types of exploits detected are Ad fraud, botnet, malware, spyware, spam, and Phishing. We use several third party malware databases and and build our own based on our customer network.
Q: Can you provide specifics about Luma’s internet filtering? What database is used? Examples of sites filtered for the R, PG-13, PG and G classes?
A: We use a couple of third party URL filtering databases. We have tens of millions of websites classified into categories and we then map those categories into PG style ratings. For example, violence, weapons and sexual categories are allowed for R but not PG. G is a whitelist only approach that only provides access to a set of kids websites like PBS Kids and Nick for Kids.
Q: What is done locally and what is done in the cloud?
A: All network routing and policy enforcement is done locally. The cloud provides policy decisions, malware database updates, and wireless and routing optimization updates. For filtering and security, a portion of the database is cached locally and the Luma queries the cloud for real-time decisions for other traffic.
Q: How is the connection secured between Luma and Luma cloud?
A: The connection between Luma and the Luma cloud is two-authenticated SSL connection.
Q: Does all user traffic pass through Luma’s cloud, including local traffic?
A: The user traffic does not pass through Lumas cloud. The local Lumas query the cloud based on the URLs that the user is visiting and the cloud provides a policy decision in real-time back to the local Luma.
Q: What information about user traffic does Luma store and for how long?
A: Luma only stores the information that is needed to provide reporting back to the user. If no reporting is enabled, then no traffic information is stored.
Security features automatically apply to all devices connected to Luma. To use Filtering features, you need to define Connections, which are relationships among People, Devices and Filtering levels.The composite screenshots below provide the gist of this.
Luma Connections and People screens
Good luck assigning devices; you currently can’t rename them and Luma doesn’t look up MAC addresses to assign more useful descriptions or icons.
Luma Devices and Assign screens
Security and filtering services pretty much operate as a black box. The only way you know if they are working is when something shows up on the Security screen in the Luma app. There is no other reporting at this point. I don’t have a zoo of bad stuff to test with so couldn’t really give this an even cursory test and Luma didn’t suggest any way of testing this feature.
The screenshot below shows the simple Security screen and the overlay that pops up when you tap the icon at screen lower right.
Luma Security and pop-up control & Filtering screens
Pause Internet did just that, leaving local Wi-Fi up, but cutting all internet access. Invite to WiFi just shares the guest network name and password in clear text via the various connection methods supported by your device (text, email, Android beam, Bluetooth, etc.). The enigmatic Filtering screen is the last panel; a level set here is taken as the default for all connections unless otherwise adjusted for a Person.
I set my restrictions / filtering level to G, which Luma says is "whitelist only" and tried an assortment of sites on my Android tablet using Chrome. SmallNetBuilder.com was accessible, as was Disney.com. Playboy.com and Cosmopolitan.com were blocked as was Yahoo.com. Google.com was not blocked, nor was access to browsing YouTube.com when I clicked on the YouTube link from the Chrome browser. But when I tried to play a few Suicide Squad trailers, YouTube reported "Connection to server lost".
I didn’t always get the Luma Website Not Allowed page when trying to access a blocked site. Sometimes I just got a Chrome This Site Can’t Be Reached page.
My bottom line for both security and filtering features is they’re not ready for prime time and need much more transparency and documentation. Don’t-bother-me-with-the-details may be a good method for setting up and managing a Wi-Fi network. But when it comes to web filtering and understanding the level of protection a product provides, details are paramount. Per person internet time limits would also be a welcome addition to the feature set.
So much for features, let’s see how fast Luma is! I was pleasantly surprised to be able to run the new Version 4 router performance tests on Luma. This was possible because Luma stays up and running indefinitely without an internet connection once it is set up. You can’t change settings and the Filtering and Security features won’t work, but Wi-Fi will stay up and, as I discovered, routing functions still run.
I also tried to retest eero, but could not find a way to configure it so that it had the constant internet connection it requires to function while also connected to the QACafe NTA1000 that now runs our routing performance tests.
I didn’t bother trying to run the functional test suite on either product. It requires configuring router settings neither Luma nor eero provide access to.
As noted earlier, Luma doesn’t report firmware version, so I don’t have that information to report. Table 2 summaries the new test results. I threw in the Linksys EA9500 AC5300 class router for comparison to show how Luma is no slouch in the wired routing department. As you can see, Luma did quite well. The TCP and UDP connection tests run as part of the performance test suite and turned in expected results. So far, only NETGEAR routers pass the UDP connection test because they appear to leave UDP sessions open longer.
|Test Description||Luma||Linksys EA9500|
|WAN – LAN TCP (Mbps)||941||918|
|LAN – WAN TCP (Mbps)||941||917|
|Total Simultaneous TCP (Mbps)||1764||1524|
|WAN – LAN UDP (Mbps)||960||950|
|LAN – WAN UDP (Mbps)||950||947|
|Total Simultaneous UDP (Mbps)||1898||1721|
Table 2: Routing performance comparison
Luma is not Wi-Fi Certified and does not support Wi-Fi Protected Setup (WPS). So any client you connect to it must be able to have its Wi-Fi network password entered. It supports WPA2/AES PSK wireless encryption only.
Since we’re talking about wireless features, I’ll share a few other things I learned from Luma:
- Band steering is supported
- Devices are not load balanced across multiple Lumas
- 802.11v, k and r are all supported.
I didn’t bother asking Luma whether they could provide a special firmware load for testing that would allow me to set channels and bandwidth. I used inSSIDer to verify channels used (11 for 2.4 GHz, 40 for 5 GHz) and link rate to determine channel bandwidth. The starting link rate for 5 GHz varied among 526.5, 650 and 702 Mbps, indicating 2×2 operation in 802.11ac 80 MHz bandwidth mode. The 2.4 GHz starting link rate was 144.4 Mbps, indicating a 2×2 802.11n connection with 20 MHz bandwidth. The test client was connected using WPA2/AES encryption.
My previous experience with eero had it drop all Wi-Fi connections within 5 minutes of being disconnected from the internet. So I kept Luma internet connected during wireless testing to be safe and had my test client connect to Luma on its LAN side, just like all other wireless routers I test.
Luma was centered on the test chamber turntable in normal operating position, as shown in the photo below. The 0° position for the router had the front facing the chamber antennas. I put Luma on a short platform to get it more in line with the chamber’s antennas.
Luma in test chamber
I tested using the new V9 test process. Because I knew you’d ask, I also retested eero using the same process. The original eero test results have been replaced by the new due to the test process change, which significantly changed the results. But now both eero and Luma are more fairly compared because the new test reflect both products in 20 MHz bandwidth mode in 2.4 GHz. I have not retested any other AC1200 class routers with the V9 process, so can’t fairly compare eero and Luma to anything else right now.
After the original review posted, I had a discussion with a Luma engineer to discuss the wireless test results, which were not in line with Luma’s. I agreed to do some further testing and in fact found a problem with my test results. The short story is the Intel test client was moving between bands during testing, paying no attention to the band preference and roaming aggressivness settings in the driver advanced properties.
Luma was able to temporarily set up separate SSIDs for each band so that I could retest the product. A helpful reader also tipped me to the Mandatory Access Point advanced properties in the Intel PROset Wi-Fi utility that I was able to use to ensure the client would not change bands for another eero retest. I also used the 802.11n Channel Width for 2.4 GHz advanced Windows property to successfully get 20 MHz bandwidth mode measurements for eero. The bottom line is I now have wireless test results that fairly compare the two products. So let’s see how they compare.
Intel STA test settings
I ran Benchmark Summaries for both routers, shown below. 2.4 GHz average downlink performance is similar, but eero outperforms Luma on 2.4 GHz average uplink and both up and down average 5 GHz throughput. I’ve removed wired routing performance from these summaries, to not clutter the view.
Luma & eero Benchmark Summaries
The 2.4 GHz downlink profile shows both Luma and eero with similar range, with eero holding connection 3 dB longer. eero also has higher maximum throughput (90 Mbps vs. 73).
2.4 GHz Downlink Throughput vs. Attenuation
The 2.4 GHz uplink plot shows a much wider performance gap, with eero taking the high road, starting at 73 Mbps vs. Luma’s 43 and staying above Luma throughout the test run.
2.4 GHz Uplink Throughput vs. Attenuation
5 GHz downlink shows similar performance, with Luma performing slightly better. Luma’s starting throughput is higher (376 vs. 333 Mbps) and it stays connected 6 dB longer, indicating better range.
5 GHz Downlink Throughput vs. Attenuation
For 5 GHz uplink, Luma’s maximum throughput is about half eero’s and stays below eero throughout the test run. Luma’s main advantage in this benchmark is that it stays connected 6 dB longer.
5 GHz Uplink Throughput vs. Attenuation
The takeaway is that downlink performance is similar for Luma and eero, but eero is significantly better on uplink. Luma also may have a slight advantage in 5 GHz range over eero.
I didn’t put Luma through the multi-client or multi-hop tests I ran for eero. I’m working on better test processes to assess both multi-hop throughput and total wireless capacity and will do a head-to-head test of both Luma and eero when I have the tests worked out.
Even if I had run the tests, I doubt the results would change my bottom line on Luma, which is to wait before you buy. Luma has a lot of work to do in its feature set, documentation of the Security and filtering features and uplink Wi-Fi performance. Even if Luma gets its Wi-Fi tuned, however, it still has a disadvange of having only two radios vs. eero’s three. eero’s second 5 GHz radio gives it more flexibility to balance backhaul and client traffic.
It remains to be seen whether either of these "mesh" multi-AP systems or the others in the pipeline can lure enough buyers away from the traditional BHR (big honkin’ router) high-margin cash generators the home Wi-Fi industry has trained us to salivate over. The clock is ticking and both startups are under the gun to ship product and grow as big as they can before NETGEAR, Linksys and other established players come out with their multi-AP solutions, which we know they have to be working on. D-Link could be first with its "DKT-891 Unified Home Networking Kit" based on Qualcomm’s Wi-Fi SON technology, even though the Q2 ship date it gave when the product was announced at CES 2016 has come and gone.
I still hope these home networking upstarts survive long enough to light a fire under their larger competitors. We need to move closer to the multi-AP systems that are needed to provide good, fast Wi-Fi all over our homes in the 5 GHz band where you need to be to take advantage of higher throughput, MU-MIMO and other fun yet to come. For now, however, it’s pretty clear Luma has its work cut out for it to overtake eero.