Like every other website on the planet, SmallNetBuilder uses cookies. Our cookies track login status, but we only allow admins to log in anyway, so those don't apply to you. Any other cookies you pick up during your visit come from advertisers, which we don't control.
If you continue to use the site, you agree to tolerate our use of cookies. Thank you!

Router Charts

Click for Router Charts

Router Ranker

Click for Router Ranker

NAS Charts

Click for NAS Charts

NAS Ranker

Click for NAS Ranker

More Tools

Click for More Tools

Security How To

Performance

First, a bit of review. Cerberus was introduced in Build Your Own IDS Firewall With pfSense as an inexpensive build (around $350) for an IDS Firewall. The build list is in Table 2.

CPU Intel Atom D525 (Pineview-D) Dual Core, 1.8GHz (13W) processor Incl in mobo
Motherboard Supermicro X7SPA-H-D525 Mini-ITX Server $180
RAM 2 x non-ECC DDR3 1066MHz SO-DIMM (running @800MHz) $50
Storage WD Scorpio Blue 2.5” 250Gig drive $40
Ethernet Intel 10/100/1000 PCIe NIC $30*
Case Antec Mini-Skeleton-90 $90
DVD Sony DVD-ROM *
Table 2: Cerberus component list

That previous article explained the whole decision process, the components and why. On top of that hardware we installed pfSense, Snort, and IP Blocklist – all to provide an extraordinary level of protection for a home network.

As an IDS Firewall, Cerberus made a good showing, not a speed demon, but in the top third of SNB’s router performance charts. Running iPerf as the server on Cerberus, directly over gigabit LAN to jPerf, Figure 5 shows an average throughput of 236 Mbps, with a peak of 253 Mbps with a fair amount of CPU headroom left over.

Running iperf on Cerberus as IDS

Figure 5: Running iperf on Cerberus as IDS

In our goal to convert Cerberus to a UTM, we poured on a whole lot of additional functionality. We added Squid and Squid Guard for caching and content filtering, we expanded Snort to cover three interfaces instead of just the single WAN interface, added HAVP and its scanning engine ClamAV for anti-virus, and instituted QOS and set-up multiple WAN load balancing and fail-over.

And finally we added some minor packages, SpamD for anti-spam, and DNS Blacklist and Country Block for targeted content filtering, BandwidthD, Lightsquid and Darkstat for reporting. In all, a complete package, our UTM.

So how did Cerberus the UTM fare performance-wise? Let's look at Figure 6, running the same iperf test, under the same conditions that we used for our IDS Firewall.

Running iperf on Cerberus as UTM

Figure 6: Running iperf on Cerberus as UTM

This time, I measured an average throughput of 203 Mbps, with a peak of 231 Mbps; CPU hit a utilization of just over 80% with using about 93% of available memory. Not too shabby, only a 14% drop in performance, but without CPU headroom. This shows how much we overestimated the processing requirements of pfSense; a dual core Atom 510 would probably been sufficient vs. the D525.

Conclusion

Without a doubt, Cerberus has been transformed. Take a look at the packages and features we have enabled in the summary Table 3.

Package/Feature Pros Cons
Snort IPS/IDS Comprehensive, Quick Rules engine supporting dynamic rules High Memory Demands, Requires both thoughtful configuration and administration
Squid Proxy Server Fast capable proxy server, allows for traffic throttling Not just point and shoot, doesn’t work with QOS
HAVP/ClamAV Anti-Virus Non-Blocking, Easy to set-up Not comprehensive, non-commercial AV scanning
pfSense QOS Wizard-based setup, queue based administration Limited Level-7 Support
pfSense Multi-Wan Load Balancing and Failover Provides for resilient failover Not integrated with QOS or packages, uses simple load balancing algorithm, complex non-intuitive set-up
Squid Guard Content Filtering Full featured content filtering down to who and when, ability to use external well maintained lists Difficult install, no stock blacklist, poor documentation
IP Blocklist Dynamic list based blocking Slow, manually updated list administration has bugs,  lists can be a mixed bag
DNS Blacklist Quick and simple category-based host blocking Static list requires manual updating
Country Block Easy and quick blocking of country CIDRs Geared more towards anti-spam
SpamD Anti-Spam Simple, clever spam protection Not integrated into pfSense, set-up requires hacking
Reporting: RRD, BandwidthD, LightSquid Comprehensive and easy to set up, dynamically updated Not fully integrated into webGui
Table 3: Cerberus UTM packages

So can Cerberus take home the UTM Crown? Have we hit our target? Let's take a look at the big picture. The first step is reviewing the summary of grades from Part One:

Function Grade
Intrusion Prevention & Detection A
Anti-Virus C-
Content Filtering B
Anti-Spam D
Traffic Control B
Enterprise Capabilities C
Overall Grade C+
Table 4: Cerberus UTM grading

I do feel this is an accurate grade, based on functional capabilities. But the overall grade does not reflect what you personally might need from a UTM - in that case the grade drops to that of your most urgent requirement. If you are being pummeled with spam, or run an environment with a lot of unknown users, where anti-virus is significant, the grade you give pfSense drops dramatically. If home network protection is most important, the grade gets much better.

We could stop now, and say Cerberus is a UTM, sort-of. But that would be disingenuous, because of what we learned in the upgrade process. There are three other important aspects of our system in grading whether we hit our goal. These are:  our installation experience; how well the system performs; and finally, the degree of integration, i.e. how well do the pieces work together.

The installation experience varied greatly, spanning the spectrum from seamlessly simple, with the installation of HAVP, our anti-virus solution, to the convolutions of origami we saw with installing SquidGuard, the cornerstone of content filtering. None of the more significant packages was what would be called turnkey.

It is understood that difference between an amateur and a professional is consistency - a professional chef makes the same dish over and over and it tastes the same, we cook at home, the meal can vary dramatically. PfSense’s install processes are not consistent.

pfSense Installation Process Grade:  C-

Performance is the bright spot, even with several layers on top of our TCP/IP stack, a multitude of processes poking and prodding packet after packet, Snort, QOS, load balancing, and a couple proxy servers, Cerberus still rendered excellent performance.

pfSense Performance Grade: B

Now the big one, the degree of integration: the pieces just don’t meld together to form one appliance. Squid doesn’t work with QoS, HTTP traffic will remain unmetered. The reporting tools, LightSquid and BandwidthD, are only partially integrated into the webGUI. And most significantly, virtually none of the packages are compatible with the critical enterprise aspect of running multiple WAN connections, not the built-in QoS, not any of the various proxy servers.

pfSense Integration Grade: F

If a UTM is defined by the six functional groups we identified in Part 1 of this article, then yes, pfSense and Cerberus is a UTM, all the boxes are checked. But if a UTM is an appliance where all the pieces work together, are really unified, then no, we can’t say that Cerberus is a UTM. The whole must be bigger than the sum of the parts, or a checklist of functionality.

What we learned in this upgrade is that pfSense is a patchwork of packages, some excellent, others not so much. But overall, the pieces don’t gel. The updated scorecard in Table 5 calculates out to a C. But it feels more like a Fail, or if you are charitable, an Incomplete.

Function Grade
Intrusion Prevention & Detection A
Anti-Virus C-
Content Filtering B
Anti-Spam D
Traffic Control B
Enterprise Capabilities C
Installation Process C-
Performance B
Integration F
Total Grade C
Table 5: Cerberus final UTM grading

This judgment, our final grade, only applies to our well-formed definition of what a UTM is, and does not imply that pfSense is not suitable for solving your problem, especially if you don’t need Multi-Wan. If all you want to do is protect your home network, Cerberus is an all-star.

However, there is hope on the horizon. While writing this article, pfSense moved the long awaited Version 2.0 out of beta. 2.0 is reported to sport fully integrated multi-wan support, and expanded support for packages like SpamD. So we may get to do this all over again!


In writing this, I’d like to thank Tessa Maish for her keen editing eye, and Tim Higgins for his invaluable input.

More Stuff

Wi-Fi System Tools
Check out our Wi-Fi System Charts, Ranker and Finder!

Support Us!

If you like what we do and want to thank us, just buy something on Amazon. We'll get a small commission on anything you buy. Thanks!

Over In The Forums

My WIFI has failed,stopped/quit twice in the last two days.Suddenly, theres no SSID broadcast and all wifi devices fail to see the WIFI and the router...
Buonasera! Ho impostato un profilo VPN. se inserisco le policy rules per inseire alcuni device sotto tunnel VPN ed altri liberi in WAN, succede che co...
I am about to run a script for the first time to configure a new bridge and move the guest wifi/new vlan to it. I am not quite getting which interface...
Using an Asus Rt-AC68p router with latest Asuswrt-Merlin installed. I have a mobile hotspot (At&t) connected to its USB port and USB activated in the ...
I have a pair of RT-AC66U_B1 routers with OpenVPN set up between them -- a client (Maryland, 192.168.58.XXX) and server (Virginia, 192.168.60.xxx). Vi...

Don't Miss These

  • 1
  • 2
  • 3