Hardware

Once I made the decision to use a 16-bay chassis, I selected the Ci Design SR316 as the base of the system. It holds up to 16 hot-swap Serial ATA (SATA) drives, a floppy drive, CD-ROM, a triple-redundant power supply, and a variety of motherboards.

For the motherboard, I selected the Tyan Thunder i7501 Pro with the optional SCSI controller. I had used Tyan motherboards in the past and was mostly satisfied with them (anyone remember the Tomcat COAST fiasco?). The Tyan mobo supports 2 Intel Xeon processors and I selected the 3.06GHz / 533MHz models as the "sweet spot" on the price/performance curve. I used two processors so I could feel free to run local programs on the server without impacting its file-serving performance. I often do CPU-intensive tasks such as converting CD images to MP3s, checksumming files, etc.

The motherboard also has three on-board Ethernet ports, one which supports 10/100 Mbit and two that support 10/100/1000 Mbit. I felt that Gigabit Ethernet performance was necessary for this project, and this board had two ports already installed, which was ideal.

For memory, I use Kingston products exclusively and selected two of the KVR266X72RC25L/2G modules. This memory was on Tyan's approved memory list and 4GB would give me lots of memory for disk buffers. While the motherboard supports up to 12GB of memory (6 of these 2GB modules), operating system support for more than 4GB was not common at the time.

One of the most important items for a file server is the disk controller. I picked the 3Ware/AMCC 9500S-8MI controllers. Each controller supports eight drives and connects to the chassis backplane via a Multilane Interconnect (MI) cable. This cable takes the place of four normal SATA cables, reducing the total disk cable count from 16 to four.

Finally, I added internal SCSI cables and external terminators from CS Electronics. The folks at CS are wonderful, building and shipping whatever custom cables I've needed in only a day or two.

The 16 drives in the system are split into two independent groups of eight, with one group on one controller and the other group on the second controller. Within the groups of eight, seven drives are used as a RAID 5 set with the eighth drive being a "hot spare" in case any of the other seven drives fail.

Software

As I mentioned above, I'm using FreeBSD 6.x as the operating system. I also use Samba as my CIFS server software, and rdiff-backup for synchronizing my offsite backup RAIDzillas to the local units. There is a large variety of other software installed as well. One of the nice things about the RAIDzilla as opposed to a off-the-shelf NAS is that I can run a huge amount of software on them locally without needing to build a customized NAS OS (if the sources are even available) and cross-compiling the applications.

What I'd do differently this time (two years later - 2007)

If I were building these from scratch now, the major change I'd make would be to use a single 16-channel RAID card, so I'd have one parity disk and one spare drive in the chassis, instead of the two of each I have now.

I'd also use more modern hardware, such as a PCI Express motherboard and RAID card, and go with larger-capacity drives (750GB or 1TB).

Since support for systems with > 4GB of memory is now common, I might also experiment with a huge memory size. It should be possible to easily run 16GB or even 32GB on server-class motherboards today.

Tags: DIY, NAS, RAID,

Related Articles: