SmallNetBuilder

Gigabit Gear

Manufacturers haven't made gigabit standard (they say it's because gigabit parts cost significantly more and they don't think they can recoup the cost in the form of more expensive products). But prices have come down to the point where, even though they are still expensive relative to 10/100 gear, they are cheap enough that you don't have to agonize over the decision.

PCI gigabit NICs can be found for around $15 (vs. $5 for 10/100) and the Intel Pro/1000's are a popular choice. The PRO/1000 GT Desktop for PCI slots pictured below, runs a little less than $30. If your computer has a PCIe X1 slot, the PRO/1000 PT (~$35) is a better choice, as we'll see shortly.

Figure 1: Intel PRO/1000 GT gigabit NIC

5 port gigabit switches start around $30 (vs. $10 for 10/100). Table 1, taken from the 8 Port Gigabit Switch Roundup, shows that 8 port gigabit switches are even a reasonable buy. (It also shows that it pays to shop around!)

Table 1: Switch Feature Summary

Go for PCIe

Even today's most inexpensive computers have no problem meeting the rough rule of thumb of 1Hz of processor speed for each bit per second (bps) of TCP/IP network traffic. You don't need to worry about the processor, as long as it is clocked at 1 GHz or higher. What matters more, however, is whether it has either a PCIe (PCI Express) X1 slot or an onboard NIC that connects via PCIe to the CPU.

The reason for this is shown in Figures 2 and 3. They show the results of an IxChariot throughput test between the machines listed in Table 2. The computers were connected via a NETGEAR GS108 8 port gigabit switch, which automatically supports up to 9K jumbo frames. I used the default settings provided by the latest Intel drivers, changing only the jumbo frame setting.

8/20/2008: Corrected system info

Table 2: Gigabit throughput test computers

Figure 2 shows throughput with an Intel PRO/1000 MT PCI adapter in each machine combining runs without jumbo frames enabled and with 4k and 9K jumbo frames used. You see average throughput of only around 530 Mbps for a 30 second test with no jumbo frames. Performance improves to just shy of 600 Mbps (around 11%) with 4K jumbo frames with increased variation, but actually drops back down to around 540 Mbps when 9K jumbo frames are enabled. I also ran tests in the other direction, with similar results.

Figure 2: 1000 Mbps throughput - PCI adapters - no, 4k and 9K jumbo frames

Figure 3 shows throughput using onboard Ethernet that uses Intel's 82566DM-2 PCIe gigabit Ethernet controller. Quite a difference, eh? The PCIe NICs got me much closer to gigabit wire speed, yielding slightly over 900 Mbps. Enabling 4K jumbo frames didn't really make a difference, but using 9K jumbo frames actually significantly reduced throughput to around 650 Mbps—almost 30%

Figure 3: 1000 Mbps throughput - PCIe adapters - no, 4k and 9K jumbo frames

I also ran simultaneous bi-directional tests and got around 640 Mbps total throughput using the PCI adapters, which is consistent with the unidirectional throughput. But the PCIe bidirectional test came in around 1.3 Gbps total, instead of the 1.8 that you might expect from the unidirectional results.

These results confirm the discussion in our Forums about gigabit Ethernet. If you want to achieve top speeds, PCIe adapters or on-board NICs are the way to go. Table 3 lists some of the Ethernet controller chips commonly seen and whether they connect via PCI or PCIe bus.

Table 3: Gigabit Ethernet controllers

Closing Thoughts

If you're moving a lot of large files around, moving to gigabit Ethernet is a no-brainer. Otherwise, 100 Mbps networks will continue to do you just fine. But if you do decide to make the move to gigabit, you should make sure your gigabit adapters use a PCIe connection to get the most from your investment.