Backup

It was actually one of the Media editors who created the backup script that divided backups into 4 GB files that were copied hourly. This made it less likely that a file would be in the process of being edited while a backup was taking place. This method had a downside that the backups were not necessarily of the most recently edited material.

If the backup file size had been 50 or 100 GB, and an editor was working on that 100GB file at the time, then the entire 100GB would be unable to be copied. During the backup, each 4GB file was copied, locally, to a new name, sent to the Media Backup server, and removed. The script also had the provision that if a file could not be copied, it was put into a list of files to be attempted at the end of the backup. And if the file could still not be copied, then it was skipped until the next backup.

With this setup, the Media editors were capturing, editing, and making backups over the network while I added the rest of the file servers. The Dell PowerConnect 5324 switched were handling the load well with an average transfer rate still around 1.33 GB per minute.

On the Media side, there were ten users with two workstations each for a total of twenty clients. You'll remember from Part 1 that I had chosen to organize the Media group as five clients per switch, one switch per NIC, and four client NICs per server (with the additional built-in NIC being connected to the Domain Controller’s switch). From Part 1's Comments it appears that my rationale for this organization was misunderstood, so I’d like to flesh this out a bit.

There were actually two Media fileservers (we’ll call them Media and Media Backup). The primary Media server is where the video/audio files were being transferred to and from and where audio/video capturing/encoding takes place. Remember also that each Media user had two machines. The first machine was connected to a capture device that also encoded the files to the desired format and size. This machine then automatically named and transferred each file to the Media fileserver into a folder. Once an entire reel or cassette was captured and named, the finished folder was then sent to the Media Backup server.

The second Media workstation was where the actual editing was taking place. This involved an eight hour series of transfers from the primary Media server to the ten media editing stations. Then the saved projects would be sent from the editing stations back to the Media server with an hourly backup being sent to the Media Backup server. Each transfer on the editing workstations was a minimum of 50 GB.

With ten editing stations, there was a minimum of 500 GB being transferred at a time, with the overall transfer rate of each workstation being 1.33 GB per minute. This means that each switch was being hit with about 13.3 GB per minute from the Media editing workstations alone. We can pretty much double that figure with the transfer from the capture/encoding workstations to the Media fileserver.

Figure 3 shows a hypthetical scenario in which we're only using one NIC. As you can see, the two workstations used by each Media client are hitting the switch with around 26.6 GB per minute. But with only one gigabit connection on the Media fileserver, throughput is choked down to 1.33 GB per minute—the effective transfer rate of a gigabit NIC.

So while the main network, from the Firewall to the Domain Controller/DHCP Server down to the clients, was functioning efficiently, the Media editors would experience lag at the file server because a single NIC card is incapable of processing 26.6 GB per minute.

Figure 3: Hypothetical 1 NIC scenario.

Figure 4 shows what was actually implemented. With the addition of 3 NIC cards, throughput increases to around 5.32 GB per minute (give or take a few MB in the name of CPU and Memory overhead). I opted use the network load balancing built into Windows to handle that issue. I suppose I could've writen a Java port listener to hand out routes "round robin" to each NIC... but life's too short.

Figure 4: Four NIC card solution.

Note that I've also attached four NICs to the Media Backup server even though only one nic (the built-in NIC) is connected to the regular Media server through the Server Switch. This is, just in case there's a problem with the Media's main server, all I'll have to do is move the connections over to the backup server. And if you're wondering why I didn't use 10 GB Ethernet, unfortunately, it was neither affordable, practical nor widely available at this time.

Tags: gigabit, LAN,

Related Articles: