SmallNetBuilder

RAID Fail Test

The pull-the-drive test on the ShareSpace produced mixed results. After waiting for the 16 hour RAID 5 build to complete, I started a filecopy of a large folder of images, waited about 30 seconds for the copy to get underway, then pulled a drive. I had previously set up email alerts and when I checked my mail within 10 seconds or so of the drive pull, I found an email titled "HDD Status" with the following message:

HDD 2 is absent.
on Sun Sep 28 11:54:50 EDT 2008

In the meantime, the folder copy process continued uninterrupted. I had logged out of the web GUI, so then logged back in to check for failure notifications. I was pleased to find a clear indication of drive failure as shown in Figure 10. This message was presented on every page in the Web GUI until I clicked the Acknowledge button to remove it. WD deserves kudos for this, since virtually every other NAS we have tested makes the user hunt to find any indication of drive failure.

Figure 10: Drive failure alert

Since the ShareSpace does not support hot-swapping, I powered it down, re-inserted the drive, then turned it back on. When I logged back in, however, I found no indication of a problem with the RAID volume, no prompting to perform a drive rebuild and no rebuild automatically in progress. The RAID 5 volume was available, however, and I was able to resume the folder copy.

I navigated to the Volumes and RAID Management > Manage Volume screen which showed the drive as missing with a Degraded status. As I was browsing through the screens looking for a way to rebuild the volume, however, a message similar to the one shown in Figure 10 appeared, announcing that the volume was degraded. The only option presented, however, was the Acknowledge button; there was no option to start a volume rebuild.

Figure 11: Degraded message and new icons

Browsing to the Storage > Disk Manager page (Figure 11, above) showed two new icons next to the failed drive (yes, that's a different drive... I did this experiment twice). The left icon is "Safely Remove Disk" and the right was "Clean Disk". There was no guidance in the User Manual, so I tried the "Safely Remove Disk" option, which resulted in the drive being removed from the array.

So now my only option was to go back to the Manage Volume or Manage RAID pages and see what happened when I chose the RAID 5 options. Unfortunately, when I tried both options, I was warned via a pop-up that I was going to wipe all data and start a 16 - 17 hour rebuild process.

After checking with WD, it turns out that I encountered a problem similar to the one I found with the D-Link DNS-321 pull-a-drive test. In a nutshell, the odd behavior was due to my inserting the same drive (which normally would not be the case) instead of a fresh WD drive. Had I done that, WD said that the system would have automatically started a rebuild upon start up.

There actually is a way to recover from my odd situation. All I had to do was choose the "Clean Disk" icon and reboot to shart an automatic rebuild. (If you do as I did, and choose the "Safely Remove Disk" option, just reboot and you'll get the two icons back and you can choose the "Clean Disk" option.)

Be forewarned, however, that recovery can take a very long time if you are actively changing contents on the damaged volume. I started copying a large folder during recovery and got an estimated recovery time of over 35 hours at one point! Recovery time may not seem important as long as the recovery is successful. But remember that until the recovery is complete, your volume will go bye-bye if you get another drive failure.