Qualification processes tended to reflect the reasons given for the need to qualify. Synology was the least forthcoming with specifics, saying that the information is proprietary. But they said they test "several behavioral and physical attributes such as RAID, hibernation, stress and power behavior, thermal generation and length of time the drive lasts under a data burn".
When I asked if they could provide specifics about problems or disqualifiers, Synology said that they have "yet to find a specific characteristic that disqualifies a drive" and that drives may have "new behaviors" including power requirements, disk parameter changes, and "green drive" spin up/down rates.
2/27/2009: Since no other vendor provided as much qual process detail as NETGEAR, I have revised the description at NETGEAR's request.
NETGEAR, on the other hand, was the most forthcoming with information about their qual process. They provided a detailed list of 14 tests included in their process, which include checks for proper XRAID and XRAID2 initialization and expansion, operating temperature and noise level, hot-swap, drive spin-down and restart and long-term data integrity.
I asked for more detail about the XRAID and XRAID2 checks and found they include verifying that a secure erase command (done on previously-used drives) properly executes in a live RAID environment. They also check that partitions are created properly and that the RAID, LVM and file system layers are created correctly during the actual expansion process.
QNAP's test process includes:
- Make a RAID array
- Checking HDD standby & resuming from standby
- Hot-swap check
- Checking HDD SMART and volume information/ status
- File copy test
The SMART test mainly checks that the information is correctly passed from the hard drive. QNAP doesn't do anything with the information in its NAS OS besides report it. They also use "some popular PC-based SMART applications" to verify that SMART information is reported properly. They also said that they found in one case that their SMART testing found a drive that unmounted during the test.
Other issues that QNAP's qual process has found are abnormal volume status reports, drives that won't go into standby or come out of standby and drives that are consistently recognized during boot. The latter issue is caused by drives that take longer to spin up than the mount delay built into the NAS.
Thecus' main qualification process seems to focus on RAID creation, migration and rebuild. Drives are put into a system and the following checks are run:
- Create a RAID 5 array
- Perform RAID migration to stress every disk drive
- Force a degraded RAID (pull and reinsert a drive) and let it auto rebuild
- Power cycle the NAS 50 times and verify that RAID status comes up healthy
- Create RAID 0 with highest capacity HDD to check RAID capacity limit
Thecus also does an "eat your own dog food" test (my description, not Thecus') to ensure RAID stability by fully loading up a NAS with the highest capacity drives available (1.5 TB currently) and using it as their main file server.
So that hot new 1.5 TB drive isn't on your NAS' approved drive list. But you bought it anyway, installed it and it seems to work fine. So, no problem, right? Or even worse, you stuck to the drives on the approved list, but still had a mysterious RAID array failure. So why bother with the list?
The main reason is that drives on the list have made it through a series of checks and tests, while drives not on the list have not. Although I have shown that there is a wide variation in qual processes, NAS vendors do know where to look for problems and have better access to drive vendor technical resources than you or I. So by using an approved drive you are taking advantage of that knowledge and vetting process and improving your odds of trouble-free operation.
The main risk of using unapproved drives that all vendors pointed to is over the long term. Many drives can seem to work fine while they are new. But over time as they age or as volumes become more fragmented, unapproved drives can cause increased errors, slower performance and even RAID rebuilds if a drive drops out of an array. If you're lucky, the errors will be recoverable. But if you're not, and you don't have a backup, your data may be gone.
The bottom line is that qualified or approved drive lists are not a guarantee of trouble-free operation. But they are a no-cost way of raising your odds of having a trouble-free NAS.