Like every other website on the planet, SmallNetBuilder uses cookies. Our cookies track login status, but we only allow admins to log in anyway, so those don't apply to you. Any other cookies you pick up during your visit come from advertisers, which we don't control.
If you continue to use the site, you agree to tolerate our use of cookies. Thank you!

Wi-Fi Router Charts

Click for Wi-Fi Router Charts

Mesh System Charts

Click for Wi-Fi Mesh System Charts



Build Your Own Fibre Channel SAN For Less Than $1000

So Verizon has this whole pitch, bringing fiber to the home. Well, we are going to one-up that, by doing fiber in the home. Taking advantage of the economy, demand for SAN hardware has slowed at the same time that less expensive PCIe components have made obsolete a whole swath of perfectly good hardware. Combine that with marketplaces like EBay, forcing what were local equipment liquidators to compete globally on price and we can score data center gear for less than a tenth of its original cost.

In Part 1, we built Old Shuck, an inexpensive twelve drive, twenty bay NAS for less than $700. A beast that was able to pretty much hold its own against equipment that costs twice as much and holds less. This time, we are going to blow the top off that, converting our NAS to a 4 Gbps fibre channel SAN, connecting to a DAS server running Windows and out to the network from there (Figure 1).

Block diagram of the Fibre Channel SAN/DAS/NAS

Figure 1: Block diagram of the Fibre Channel SAN/DAS/NAS

We need to purchase the needed components, Fiber HBAs and the cable to connect them, then install the boards and configure them. We will then reconfigure our Old Shuck NAS array to support SCSI and Fibre Connect – Old Shuck will morph into a shining SAN. Once we get it up and running, we will do benchmarking, just to see how hardcore the performance really is.

Inevitably with these sorts of articles, someone will point out that you just don’t need this kind of equipment, performance or capacity in your home. While that may be true, I’d argue that, quoting an old Richard Pryor joke, the same folks would never drop acid and try to watch The Exorcist, just to see what it is like. There is a thrill to accomplishing something like this from the ground up, it is a cool bit of kit, the geek’s version of the old quote – “You can never be too thin or too rich” – you can never have too much storage or too much performance. Double that if you can do it on the cheap.

Brief Fibre Channel Intro

Fibre Channel (FC) has been around since the early nineties, as a replacement for the awkward supercomputer based HIPPI protocol, and has become the de facto standard for connecting high speed storage arrays to host servers.

Fibre channel (which does not require an actual fiber cable, it can use copper), accepts SCSI block commands allowing direct read/write access to served storage. Each node requires a FC Host Bus Adapter (HBA), and can be connected to each other in three different topologies: looped like token ring, switched like modern Ethernet, and point to point. We are going to be focused on the least expensive, point to point, which requires no switch or multiport cards, hence less dollars.

Each HBA has a unique identifier (like an Ethernet’s MAC address) called a WWN (World-Wide Number) and come in speeds ranging from 1 Gbps to 20 Gbps. We are going to be focused on 4 Gbps (800MB/s), which offers the best bang for the buck, and high compatibility – the current sweet spot.

In the terminology, which iSCSI adopted, the array is the target node, and the DAS server is the initiator. Since there is no inherent routing, like with TCP/IP (and less overhead), each node needs to be configured separately.

The Parts

As we said, to accomplish a fiber connected SAN, we need two fibre channel HBAs, one for our SAN node, the other for our DAS server. And we’ll need a cable to connect the two. Our budget for these pieces is $300, two hundred for the PCIe card, and $75 for the PCI-X card, and leaving $25 for the cable. A tight fit.

Linux best supports QLogic 2xxx cards right now, so our SAN node needs a 4 Gbps QLogic 2xxx PCI-X card (not 2xx since the popular 220 cards have issues). Our DAS node, running Windows 7, doesn’t have the same vendor restrictions. For probably superstitious compatibility reasons, it would be ideal to get the same card for our DAS. But with a PCIe interface, the PCIe card nervously promises to be our most expensive item, and most likely to push us over budget.

Hitting eBay, we immediately found a QLogic QLE2460 4Gb PCI-X HBA within our budget. Score! Now we need to find a QLE2460 PCIe card. All of the “Buy it Now” 2460’s are outside our budget, so we join several auctions only to get sniped at a little over the $200 mark (Arrgh!).The second time around we get lucky with a price that falls just under our limit, if you include shipping.

The cable is easy, we need a LC/LC (the newer type of fiber cable connector) patch cable, say 6 feet and searching we find a 3 meter cable for less than we thought we could.

What we got:

QLogic QLE2460 4Gb PCI-X HBA $54, EBay Buy it Now
QLogic QLE2460 4Gb PCIe HBA $197, EBay Auction
Fiber Optic Patch Cable Cord 50/125 LC-LC 10M 33FT $10,
Total $261
Table 1: Fibre Channel components

Great, we end up about $100 under budget, $56 under from the NAS parts, and $39 from here. Now I can sit back and wait for the sweet sound of the UPS van’s horn.

With the parts I have, I start building our NAS array, everything goes great until I get to the performance and stress testing phase. I’m seeing an odd message, something like “CPU #2 Now running within proper temperature range.” No previous indication that it wasn’t in range, but it appears as though we’re running hot. Darn.

Turns out that under stress our Norco case can be a little less than adequate at cooling, especially with passive cooling on the CPUs. This might be real trouble, isn’t like there’s a high demand for S604 CPU coolers. But lo, the builder’s favorite vendor, NewEgg, has active coolers for Nocona CPUs which will work fine on my 3.8 Ghz Irwindales. For what they are, they’re not cheap, $72 for two and a stall in performance testing. 

QLogic QLE2460 4Gb PCI-X HBA $54, EBay Buy it Now
QLogic QLE2460 4Gb PCIe HBA $197, EBay Auction
Fiber Optic Patch Cable Cord 50/125 LC-LC 10M 33FT $10,
Previous Total $261
Dynatron H6HG 60mm 2 Ball CPU Cooler (X2) $72, NewEgg
Total $333
Table 2: Fibre Channel components, revised

Over budget on Part 2 by $33. But we now have all the components we need – done with the shopping.

Putting it All Together

We now have our FC HBAs, one for each end of the connection, and the cable to connect them. Bringing up the SAN is straightforward, but will require working at the shell level, and familiarity with a Unix editor.

We have four main steps needed to complete our big conversion:

  • Install – Install our two HBAs, one under Windows, the other under Openfiler.
  • Configure – Configure Openfiler to run as a SAN FC Target
  • Convert – Change over our RAID array to run as SCSI array.
  • Start-up – Connect Windows DAS server as an FC Initiator

Support Us!

If you like what we do and want to thank us, just buy something on Amazon. We'll get a small commission on anything you buy. Thanks!

Don't Miss These

  • 1
  • 2