Like every other website on the planet, SmallNetBuilder uses cookies. Our cookies track login status, but we only allow admins to log in anyway, so those don't apply to you. Any other cookies you pick up during your visit come from advertisers, which we don't control.
If you continue to use the site, you agree to tolerate our use of cookies. Thank you!

Wi-Fi Router Charts

Click for Wi-Fi Router Charts

Mesh System Charts

Click for Wi-Fi Mesh System Charts



Build Your Own Fibre Channel SAN For Less Than $1000

Sitting in the corner of the office is a Thecus N5200, dusty and filled to capacity – been around for more than five years. It was one of the first five disk consumer NASes, very quick for its time, as Tim’s review stated eloquently in late 2006:

My bottom line is that if you're looking for the fastest NAS for file serving and backup and don't mind a funky user interface, sparse documentation, and immature firmware then you might take a chance on the 5200…

Thecus has succeeded in raising the feature bar for "prosumer" NASes to include five bays, RAID 6 and 10 and speeds that make good use of a gigabit LAN connection. But its reach has exceeded its grasp in producing an all-around polished product that anyone could feel comfortable buying.

When initially unboxed and loaded up with five new shiny and quick drives, a home network RAID 5 NAS was then an uncommon sight. Flash forward to now and the dusty Thecus is slow, stores a little more than a single 2 TB drive, and with the rise of home theater automation, is now not unusual to see a NAS in someone's home. In other words, ripe for replacement.

Home media requirements have also started to exceed what most NASes can provide. Storage guru Tom Coughlin of Coughlin Associates predicts that by 2015 you’ll have a terabyte in your pocket and a petabyte at home. And there are now folks with multiple NASes at home. My own home network has well over 11 TB and growing, much of it unprotected by either redundancy or backups. Do you have the means to back up that much data?

To solve this, to quote Roy Scheider in Jaws, “We are going to need a bigger boat.”

A Fibre Channel SAN/DAS/NAS

The plan is to first build a disk array RAID NAS, then configure it as a SAN node and use fibre channel to connect it directly to a DAS server, which will attach the newly available storage to the network. A terabyte version of Tinkers to Evers to Chance – a perfect double play.

Block diagram of the Fibre Channel SAN/DAS/NAS

Figure 1: Block diagram of the Fibre Channel SAN/DAS/NAS

So what is the difference between a SAN and a NAS and what are the advantages of a SAN over a NAS? The primary difference is that in a SAN, the disks in the RAID array are shared at a block level, but with NAS they are shared at the filesystem level.

A SAN takes SCSI commands off the wire and talks directly to the disk. In contrast, a NAS utilizes file level protocols such as CIFS and NFS. This means that with a SAN, filesystem maintenance is offloaded to the client. In addition, SANs generally uses a wire protocol designed for disk manipulation, whereas NAS uses TCP/IP, a general purpose protocol.

Comparing the Ethernet to fibre equivalent, i.e. 10GbE to 10gig fiber channel, significantly more payload is passed with fibre because of the lack of inherent TCP/IP overhead. Combine that with a fat cable like fiber and you have one lean, mean transport layer.

Additionally, if you are running your NAS in a strictly Windows environment, you have the added overhead of converting the SMB protocol (via Samba) to the native filesystem, typically EXT3/4, XFS, etc. On a SAN, no such conversion takes place because you are talking directly to the server-assisted RAID controller. Less overhead, better performance.

The downside of this is that TCP/IP is ubiquitous, and fibre channel is expensive, requiring special hardware and software. Hardware being Fibre Channel host bus adaptors (HBA) and fiber switches and cables and software such as the SAN management suite from SANMelody. SAN infrastructure can run into the tens of thousands; a new single 10GB fibre channel HBA can be as much as $1500. The software is often twice as much.

That is unless you are willing to go open source, buy used equipment, and work with outdated interfaces. If you are, you can get the necessary hardware and software for a song.

Right now there is a glut of used fibre channel hardware on the market. You can pick up a 2 GB fibre channel HBA for $10 on EBay, $50 for 4 GB. The catch is that it is PCI-X hardware. PCI-X was an extended version of PCI for servers, providing a 64-bit interface at up to commonly 133 MHz.

The half-duplex PCI-X bus has been superseded by the faster full-duplex PCIe bus. This, plus the economy, and the move away from fibre, means that what was an exclusively enterprise technology is now available for your home network at reasonable prices; one-tenth its original cost.

Openfiler logoFor software, we are going with Openfiler, which dates back to 2004 and is based on rPath, a branch of RedHat linux. The compelling feature of Openfiler is SAN and fibre connect Host Bus Adapters (HBA) support. Openfiler is also a complete NAS solution.

The Plan

We are going to take a low risk approach to achieve our goals, which are:

1) Build a Fibre Connect SAN for less than $1K
On the cheap, one thousand dollars, excluding disks and embellishments. Think of the build as sort of a wedding: something old, something new, something borrowed, and something black. We’ll buy older generation used PCI-X server hardware including motherboard, CPUs, memory and interface cards mostly from EBay. The new components will be the power supply and our disks. Something borrowed are pieces from previous builds, the system drive, fans, and cables. Black of course is the hot swap system chassis.

2) Exceed the capacity of any consumer grade NAS
The largest consumer grade NAS is currently eight bay QNAP TS-809 Pro which will support 3TB drives. In RAID 5, that is about 18-19TB. We are shooting for a max RAID 5 capacity of 35TB.

3) Beat the Charts
Our goal is to exceed the current SNB performance chart leader, both as fibre channel and as a logical NAS, as measured by Intel’s NAS Performance Toolkit for RAID 5.

Low risk means putting the pieces together, insuring they work, and then bringing up a NAS array. Once we have an operational RAID array, we’ll look at NAS performance. We'll then add the fiber HBAs and configure the array as a SAN. Even if we can’t get fibre connect to light, we’ll still have a respectable NAS.

Old ShuckPart of the fun in these builds is in the naming. We’re calling our San/NAS array Old Shuck, after the wickedly fast ghostly demon dog of English and Irish legend (with eyes in the 6500 angstrom range). We did toy with calling our build Balto, after the famous Alaskan hero sled dog, but that was not nearly as cool.

Support Us!

If you like what we do and want to thank us, just buy something on Amazon. We'll get a small commission on anything you buy. Thanks!

Don't Miss These

  • 1
  • 2