Those of you who have been following this series know that I have been unable to achieve the goal of 100 MB/s read and write throughput for any NAS that I have put together. Try as I might, I have been unable to achieve non-cached performance better than around 70 MB/s.
Well, folks, I have found out why.
The key to unlocking higher performance can be summarized in three words: larger record size.
The journey to this conclusion started with SmallNetBuilder Forum posts by Dennis Wood, detailing his success at coaxing much higher performance from his QNAP TS-509 Pro than I measured using iozone. Dennis has been hammering on the TS-509 Pro and running more tests in more configurations than I want to even think about!
Some of the performance gains have been from firmware updates that QNAP has issued in response to performance and other problems that Dennis found and some have been from increasing the 509 Pro's RAM to its 4 GB maximum. But Dennis' best results seem to have been consistently achieved when using systems running Vista SP1 with a three-drive RAID 0 array. With this configuration, Dennis has been able, in some of his tests, to achieve over 100 MB/s file copy performance.
Iozone is very careful about disk use and does not access the drive on the system that it is running on when performing its reads and writes to the filesystem under test. So while drive speed is a factor in real-life file copy and filesystem performance, it isn't a factor in iozone testing. So using a RAID 0 array on the iozone system won't help to achieve higher performance.
But Dennis' success got me thinking about what Vista SP1 was doing that iozone was not. Vista SP1's higher file copy performance has also been discussed in the SNB Forum. Multiple posts have mentioned Mark Russinovich's much-referenced Inside Vista SP1 File Copy Improvements blog post, which contains an excellent description of Microsoft's journey to improved filecopy performance.
Part of the article describes how Vista removed XP's limit of 64 KB blocks for network reads and writes, increasing them in some cases to up to 2 MB! In the article, Process Monitor traces were used to illustrate some of the points so that the actual read and write block sizes could be seen. That inspired Forum member 00Roush to run some experiments of his own using Process Monitor, iozone and XP file copies, which peeled the onion a bit more.
00Roush's experiments inspired me to run my own tests with Process Monitor, which were quite helpful. I first, however, had to create a test NAS that I was sure was capable of > 100 MB/s performance, at least with Vista SP1 file copies.
Table 1 summarizes what I came up with, thanks to Intel who supplied the CPU and motherboard, Western Digital who provided the VelociRaptor drives and Microsoft who sent a copy of Vista Home Premium.
|CPU||Intel Core2 Duo E8500 @ 3.16GHz|
|RAM||Corsair XMS2 TWIN2X2048-6400C4 2 GB DDR2 800 (PC2 6400)|
|Hard Drives||Western Digital WD3000HLFS 300 GB VelociRaptor Qty. 3|
|Ethernet||Intel 82567LF-2 Network Connection (Onboard PCIe gigabit)|
|OS||Vista Home Premium SP1|
Table 1: Test NAS
The Intel DG45FC motherboard doesn't have an IDE interface so that I could set up the OS on its own drive. So I just configured the three WD VelociRaptor drives into a RAID 0 volume using the BIOS RAID utility and the suggested default 128 KB stripe size before installing Vista and upgrading it to SP1.
With the target NAS in place, I now had to choose a testbed system. I ended up using the "Big NAS" machine based on an ASUS P5E-VM DO motherboard that I have been using in the Fast NAS series as the test NAS. Table2 summarizes the system used, with some tweaks from its previous role.
|CPU||Intel Core 2 Duo E7200 @ 2.53 GHz|
|Motherboard||ASUS P5E-VM DO|
|RAM||Kingston ValueRAM 2 GB PC2 - 5300 (KVR667D2N5)|
|Hard Drives||Seagate Barracuda 7200.11 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s (ST31000340AS) Qty. 2|
|Ethernet||Intel 82566DM (Onboard PCIe gigabit)|
|OS||Vista Home Premium SP1|
Table 2: NAS Test Bed
RAM has been expanded to 2 GB to better reflect systems in use today. But to ensure that I capture non-cached performance, I ran iozone tests out to 4 GB filesize. The other change is that I configured the two Seagate 7200.11 drives into a RAID 0 volume using the Intel BIOS RAID utility, again using the default suggested 128 KB stripe size. This was so that hard drive performance would not limit drag-and-drop file copy performance.