Build Your Own UTM With pfSense – Part 3

Photo of author

Tim Higgins

Introduction

In Part One of this series, we established a working definition of our target, i.e. what has to be done, and in what order, to Cerberus the lowly IDS firewall to make it a UTM Appliance. In Part Two, we started the conversion by installing and configuring multi-WAN support, Squid, IDS and anti-virus features. This time, we’ll add and configure Content Filtering, Traffic Control, Load Balancing and Failover.

Content Filtering

As introduced in the first part of this article, pfSense has several packages for content filtering, from the simple to the sublime. When setting up Cerberus in the previous article, Build Your Own IDS Firewall With pfSense, we installed the first of these, IP Blocklist, which blocks IP addresses based on lists downloaded from a clearinghouse of list maintainers, i.e. iBlocklist.com. There you will find a large assortment of list flavors: Adult Sites, Compromised Sites, Torrent Sites, etc.

In addition to IP Blocklist, there are two very simple packages to install: Country Block and DNS Blacklist. Country Block is geared towards blocking the countries responsible for the highest volume of Spam, but can be used to block the Individual countries. It uses the national CIDR ranges from CountryIPBlocks.net.

Once installed, it is simple to configure. Select the countries you wish to block from a list of all countries. At the top, you’ll find a list of countries responsible for the largest volume of spam. Enable the service, select the countries you want to block, commit your selections, and save. Done.

Country Block configuration

Figure 1: Country Block configuration

Incoming traffic is blocked by default, but this can be changed along with logging on the Settings tab. You can also limit blocking to a particular interface, but it defaults to all interfaces.

The other simple package, DNS Blacklist, allows you to block specific categories of domain names. The package forces DNS to resolve all domains listed in the selected categories to Google’s IP address. The categorized domain list is originally from the Université Toulouse 1 Capitole, and has been wrapped into the release. This means the lists are static, and are not updated regularly, limiting overall usefulness, unless you choose to update them manually.

DNS Blacklist configuration

Figure 2: DNS Blacklist configuration

DNS Blacklist offers a very lightweight alternative to the content filtering heavyweight, Squid Guard. It uses DNSMasq as a DNS Forwarder, so requires no proxy server or complex indexing.

SquidGuard

The other alternative to content filtering is SquidGuard, a full bodied content filtering system that has more controls than a Gemini space capsule and is just as hard to get in and out of. To complicate this further, the SquidGuard tutorial on pfSense.org has gone 404.

Even with the difficulties of configuring SquidGuard, the functionality is compelling. You can choose what to block, for whom to block it, from what time to what time should the whole block thing happen, per entry.

The initial setup is a bit convoluted and requires a bit of dancing. First, you should select the blacklist provider you want to use. A meta-list is available from SquidGuard.org. The recommended set of lists is Shalla’s Blacklists (List Archive: http://www.shallalist.de/Downloads/shallalist.tar.gz ).

Starting with the General tab (Services->Proxy Filter), enable the blacklist and paste the URL of your list archive, a tarball, into the value for Blacklist URL. Go ahead and save without enabling SquidGuard yet.

SquidGuard General Settings

Figure 3: SquidGuard General Settings

Now move to the Common ACL tab. The common access control list handles filtering policy for everyone, and by default, web access is denied. We need to set it to ALLOW before enabling SquidGuard, otherwise we would lose all web access.

Expand the Target Rules List, there should be one entry, Default Access, set this to ALLOW and save.

SquidGuard Common ACL setting

Figure 4: SquidGuard Common ACL setting

We are still not ready to turn the key yet. We need to go get our blacklists, so move to the Blacklist tab. If the URL field doesn’t contain your selected list URL, copy it from the General tab and download the list. It will be downloaded and loaded into SquidGuard database. Wait for the download to complete; this may take up to ten minutes, depending on the list archive.

Blacklist download

Figure 5: Blacklist download

Once we verify we have a blacklist, we will be ready to kick-start this beast. Return to the Common ACL Tab and expand the Target Rules List. It should look like this now:

SquidGuard Common ACL Target rules

Figure 6: SquidGuard Common ACL Target rules

Now return to the General Settings tab, check all the logging you can, and check Enable. Save these changes and wait for the SquidGuard Service State to change to Started.

To verify that it is up and running, check the Filter Log under the Logging tab. If all looks good, go to the Common ACL tab and set the blacklist blk_BL_hobby_pets to DENY and Save. Return to the General Settings tab and click Apply. Now, try to go to the French Bulldog Club.

You should see:

URL denied

Figure 7: URL denied

This is just the tip of the iceberg for SquidGuard. For example, it would be possible to redirect any references to the Fox News site to that of the NY Times, from 9 AM to 9:10 AM…on only Karl the programmer’s machine. Or more importantly, ensure that your kids are actually using the Internet to do their homework after school, instead of Facebook.

Traffic Control

Though Traffic control is central to pfSense, there are some serious limitations in the current version. Traffic shaping in Version 1.2.3 doesn’t handle either Squid HTTP traffic or failover. (Squid uses your loopback interface, which is not shaped, but there is a workaround). Version 2, to be released soon, supposedly does.

Traffic shaping can be effective on a single WAN system or multi-WAN, but just on a single WAN interface with static routing. For example, you can direct all file transfer protocols (P2P, FTP, etc) through your secondary WAN interface, and leaving HTTP on the primary interface.

I will introduce traffic shaping. But full traffic shaping is complex, requiring specific details of not only your traffic, but of use patterns. This kind of traffic shaping is outside the scope of this article; more details can be found in the pfSense forums.

The Wizard sets up initial traffic queues and rules that can then be tuned; it uses your actual bandwidth figures to allocate traffic across the defined queues. So, before you start, you will need to gather your bandwidth figures, both up and down, using any number of sources (DSLReports, for example).

The first time you go to the Traffic Shaper (Firewall->Traffic Shaper) you will be presented with the wizard interface, which will step you through setting up traffic queues for the traffic you want to shape.

Traffic Shaper Wizard

Figure 8: Traffic Shaper Wizard

Here are the options for types of traffic that can be prioritized:

Traffic Type Description
VoIP Higher priority for VOIP traffic, generic or Vonage, Voice Plus, Asterisk
Peer To Peer Allocate Bandwidth to generic P2P traffic, or Disable and Lower priority for about 20 protocols of P2P traffic
Gaming Increase priority for about 20 Games, including BattleNet, WOW, Xbox360
Other Set priority for about eight categories including VPN, IM, HTTP, and Multimedia
Table 1: Traffic Shaper options

You can also define a Penalty Box, a specific IP or alias to limit if traffic levels are high.

Once you finish the wizard, it will generate traffic queues, which are essentially separate sets of routing rules. When you return to the Traffic Shaper, you will now have three tabs: Rules;Queues; and a tab for rerunning the wizard.

Traffic Shaper queues

Figure 9: Traffic Shaper queues

The values and order of the rules can all be tuned to prioritize traffic. By editing a queue, you can change the traffic percentage, and the corresponding priority of the traffic.

To verify that traffic is moving through your queues, go to the Queue Status page (Status->Queues). The various bar graphs should dynamically show changes in traffic patterns after a short delay. Attention should be paid to any drops, which indicate traffic problems.

Traffic Shaper queue status

Figure 10: Traffic Shaper queue status

Both Squid and Snort offer traffic control facilities. Squid offers both transfer caps and throttling under the Traffic Management tab of the Squid page (Services->Proxy Server). These settings are straightforward, and allow for throttling of particular categories of downloads.

Squid traffic management

Figure 11: Squid traffic management

Snort, on the other hand, offers rules for blocking certain protocol traffic , such as IM Traffic (emerging and snort chat.rules) and P2p traffic (snort and emerging p2p.rules).

Load Balancing & Failover

Now we are going to set up load balancing and failover. Let’s look at the diagram from the pfSense tutorial again, and gather our required parameters before we begin.

pfSense block diagram

Figure 12: pfSense block diagram

We need our interface IP gateway addresses and the address for a ISP DNS server used on the corresponding interface. We will be using the DNS address as the monitor address, to verify the interface is up and running via a simple ping to that address. The values in Table 2 are actual addresses I used for Cerberus. Your values may be different.

Interface IP address DNS address
Gateway Primary ISP
(WAN)
192.168.100.100 68.105.28.12

Gateway Secondary ISP (OPT1)

192.168.0.2 205.171.3.25
Table 2: IP address assignment

There are five steps to setting up failover and load balancing, one of which we have already accomplished.

  1. Set up Multi-WAN Configuration – done in Part 2.
  2. Set up Required Values – List DNS Servers, Turn on Sticky Sessions
  3. Define Failover Gateways – One for each WAN connection
  4. Set Up Load Balancing Gateway – Handles Round Robin Traffic Assignment
  5. Define Rules for LAN Traffic – Direct LAN Traffic to Load Balancer

We will also need to test load balancing and failover and write a rule for outbound HTTPS traffic. This rule will serve as an example of traffic that needs to bypass the load balancer and travel directly out a single selected ISP interface.

Since we have already set up Cerberus for multi-WAN, we’ll jump to step two, setting values. We need to do two things here; the first is make sure the two DNS addresses we are going to be using (68.105.28.12, 205.171.3.25) are listed under General Setup.

DNS address assignment

Figure 13: DNS address assignment

In Advanced Setup, we want to turn on sticky connections, so traffic started on a particular ISP WAN interface stays there, preventing sites that use your IP Address, such as your bank, from getting confused.

Enable Sticky Connections

Figure 14: Enable Sticky Connections

I also recommend editing your Snort Whitelist (Services->Snort), ensuring DNS servers are automatically added. Depending on your ISP, DNS irregularities may cause Snort to block them, giving you a false failure.

Snort Whitelist auto-add DNS servers

Figure 15: Snort Whitelist auto-add DNS servers

The next step is setting up the failover gateways in the Load Balancer (Services->Load Balancer). Each failover gateway has a pool of interfaces, each with a monitoring IP. We have two pairs of Interface and Monitor IPs that need to be added to each pool. The only difference between the two gateways is the order of these pairs.

Pair One is the Primary ISP, and the WAN DNS Server: [ WAN, 68.105.28.12 ]

Pair Two is the Secondary ISP, and the OPT1 DNS Server: [ OPT1, 205.171.3.25 ]

The first pair in each gateway is the opposing interface, the one that it fails over to. The second is its own Interface. So the pools look like:

Failover gateway address pool

Figure 16: Failover gateway address pool

Here is the pool setup for the Primary ISP, note the the Secondary ISP Failover gateway only differs in pair order:

Primary Failover pool IP setup

Figure 17: Primary Failover pool IP setup

With the failover gateways up, we can define the load balancer gateway – this looks just like our 2ndWanFailover gateway, except the behavior is Load Balancing instead of Failover.

Load Balancer gateway setup

Figure 18: Load Balancer gateway setup

With that, we have completed our Gateway setup:

Gateway setup complete

Figure 19: Gateway setup complete

Load Balancing & Failover – more

The final step is to start routing traffic through the load balancer. For that ,we need to define three firewall rules:

Rule Explanation Order
Primary ISP Traffic Drive traffic to your Primary ISP First
Secondary ISP Traffic Traffic Destined for Second ISP Second
Load Balancer Traffic Direct Traffic across ISPs Last
Table 3: Load balancer rules

To define the rules, go to the Rules page (Firewall->Rules). The rules handle outbound LAN traffic, so go to the LAN tab. Let’s first add the new rules, then delete existing rules.

Action PASS
Interface LAN
Protocol ANY
Source LAN subnet
Destination Network, 192.168.100.0/24
Log Yes ( For Testing)
Gateway Default
Table 4: Primary ISP Traffic rules

For the secondary rules, we just change the destination:

Action PASS
Interface LAN
Protocol ANY
Source LAN subnet
Destination Secondary ISP Subnet
Log Yes ( For Testing)
Gateway Default
Table 5: Secondary ISP Traffic rules

For the Load Balancing Rule, we want any traffic that doesn’t have a determined destination to go through the load balance gateway:

Action PASS
Interface LAN
Protocol ANY
Source LAN subnet
Destination ANY
Log No
Gateway LoadBalance
Table 6: Load balancer Traffic rules

This is what your finished rules will look like:

Firewall rules complete

Figure 20: Firewall rules complete

To verify that everything started properly, go to the Load Balancer status page (Status->Load Balancer). It should be all green:

Load Balancer ready

Figure 21: Load Balancer ready

Before we go any further, we should test load balancing and failover. Remember, Squid and HAVP are not multi-WAN enabled. These packages use a single interface and bypass the load balancer to push traffic out the interface you configured it to use, in our case the WAN PrimaryISP interface. So to test failover we’ll take down the SecondaryISP by simply disconnecting the cable. The system log should record the failure:

Log showing failover event

Figure 22: Log showing failover event

If the failure is not logged, or shows the wrong interface, most likely you’ve confused your pairs, using the wrong DNS address.

To test load-balancing, use a protocol other than HTTP, say FTP, POP, IM, etc. that doesn’t go through Squid. You should see the rule trigger on the balancer gateway in the firewall log:

Load Balancer rule trigger

Figure 23: Load Balancer rule trigger

You can also check your States Table (Diagnostics->States). It should list some states associated with your Secondary ISP if load balancing is working.

Your network should now be ready for the next unplanned outage by your ISP.

Sticky Connections solves most requirements for persistent sessions, but you may want to do your own pre-emptive load balancing, especially if you will be running a proxy server such as HAVP and Squid. The template for these rules is:

Action PASS
Interface LAN
Protocol TCP
Source LAN subnet
Destination ANY
Destination Port HTTPS
Log No
Gateway 2ndWANFail
Table 7: HTTPS Rule for Balancer Bypass

The HTTPS Destination Port in Table 7 can be FTP, SMTP, etc. This rule needs to be at the top of the list of rules—the load balancer rule should always be last. Using a failure gateway, traffic will, of course, fail over. If that isn’t what you want, change the gateway address to use the direct Gateway instead. In our example, that is Opt1/192.168.0.2 or WAN/192.168.100.100.

P2P traffic is much the same. You will have to use a static port and the destination port will need to agree with the configuration of your BitTorrent client (uTorrent uses 2000-3000). For incoming connections, you’ll need to define a port forwarding rule on your NAT, instead of using UPnP. More details are available in this pfSense tutorial.

That’s all for this time. We’ll try to wrap this up next time and run some performance tests to see if our hardware platform can handle all the extra duties we have piled onto it.

Related posts

How To Set Up a Site-to-Site VPN with OpenVPN

Brandon Teska walks you through using this award-winning open source SSL VPN to securely link two networks over the Internet.

Build Your Own UTM With pfSense – Part 4

In the fourth and final part of our series, we finish the upgrade of our pfSense-based box from IDS to UTM.

The Smart Traveler’s Guide to Data Theft Protection

Your identity and other personal data can be much more vulnerable while on the road. Derek Boiko-Weyrauch shows you how not to be an easy mark.