Like every other website on the planet, SmallNetBuilder uses cookies. Our cookies track login status, but we only allow admins to log in anyway, so those don't apply to you. Any other cookies you pick up during your visit come from advertisers, which we don't control.
If you continue to use the site, you agree to tolerate our use of cookies. Thank you!

Router Charts

Click for Router Charts

Router Ranker

Click for Router Ranker

NAS Charts

Click for NAS Charts

NAS Ranker

Click for NAS Ranker

More Tools

Click for More Tools

NAS Features

Tags:

To recap, we use iozone as our primary tool to capture NAS performance data. iozone tests overall filesystem performance, which includes both host and client-side caches. Generally, when tested file sizes are below the NAS and iozone host machine RAM capacities, throughput will be be higher than the maximum speed supported by the network connection used in the test.

With our first iozone test machine, which was purposely limited to only 512 MB of RAM, higher-than-network-speeds dominated the small file size (64 KB to 16 MB) results, but had only a minor influence on the large file size (32 MB to 1 GB) tests, which are most often used to compare NAS performance.

Figure 1 compares small file size mode write test results for the same product—NETGEAR's high-performance ReadyNAS Pro—with the old (blue line) and new (red line) test methods. Only three data points taken with the old test method are below the 125 MB/s maximum speed of the gigabit Ethernet connection used in the test, while there are no points below 125 MB/s in the new test method results.

Old and New Test Method Comparison - small file size mode

Figure 1: Old and New Test Method Comparison - small file size mode

Figure 2 compares write results taken in the large file size mode where there are no test points above the 125 MB/s network speed with the old method vs. three with the new. But even with just three points above network link speed, the resulting average speed is 162 MB/s, which causes confusion.

Old and New Test Method Comparison - large file size mode

Figure 2: Old and New Test Method Comparison - large file size mode

We have always recommended using read test results as the best indication of relative NAS Performance and still do. But some NASes have large differences in read and write performance, even when cached results are removed. And other NASes show average large file size write speeds in excess of network connection speeds due to unusually high results for file sizes up to 512 MB in some cases. So clearly a change was needed.

iozone provides a -c switch that includes the time it takes to close a file in the throughput calculation. Figure 3 shows the effect of using the -c switch, which removes all write cache behavior.

Effect of iozone -c switch on write

Figure 3: Effect of iozone -c switch on write

By contrast, and as expected, using the -c switch has no effect on read behavior (Figure 4). (The variation shown is typical for multiple test runs.)

Effect of iozone -c switch on read

Figure 4: Effect of iozone -c switch on read

But changing our test method to use the -c option would add another barrier to product comparison by introducing yet another test method change. It also would remove cache effects that are within network connection speed.

So, instead, we have added a new filter to the NAS Charts, the Cached Performance Filter. When it is enabled (which is the default), it removes all points above 12.5 MB/s for the 100 Mbps benchmarks and 125 MB/s for the 1000 Mbps benchmarks from the value plotted in the Average benchmark Charts. Like the Small File Size mode, the Cached Performance Filter is cookie-based and remains in effect for the browser session as you move from chart to chart.

It has no effect on the Throughput vs. File Size and Benchmark Chart views, so you will still see all data points taken when you switch to those views, even with the filter on. It also does not effect the File Copy benchmark charts, because that data is not obtained using iozone.

The filter does effect the Performance values shown in the Price vs. Performance plots for all benchmarks except File Copy.

When it is in effect, the filter is applied to all results, obtained with both old and new test methods. It has little effect on read results, since reads generally do not show extreme cached behavior. It has the most effect on write results for the small file size mode, which generally represent mostly cached behavior. Note that in some cases, the filter will remove a product from the chart in small file size mode if all data points are above maximum network connection speed.

We think that this is a reasonable compromise to solve the data distortion caused by abnormally high cached results.

Please drop us a line or post a comment to share your thoughts about this change or if you have a question.

More NAS

Wi-Fi System Tools
Check out our Wi-Fi System Charts, Ranker and Finder!

Support Us!

If you like what we do and want to thank us, just buy something on Amazon. We'll get a small commission on anything you buy. Thanks!

Over In The Forums

As you already noticed entware-ng (http://pkg.entware.net/binaries/mipsel) doesn't get updated anymore.The repo on github is archived (read-only) and ...
Hi,I have RT-AC86U on latest merlin FW 384.7. At the same time i have 2 x 1Gpbs connection from 2 diff ISPs.I have configured load-balancing ratio of ...
Any ideas on how to get overclocking working on RT-AC68U running 384.7? I am using JFFS scripts to set clkfreq to 1200,666. This is working (verified ...
I'm trying to connect a computer to my Asus ac87 using OpenVPN but running into some issues. One of my computers keeps disconnecting form the VPN conn...
Hi All,Not sure if what I want to achieve *can* be achieved, but thought I'd run this past the more experienced heads on this forum .....I have an ASU...

Don't Miss These

  • 1
  • 2
  • 3