FreeBSD: Block Brute-force Attacks Using Sshguard and IPFW Firewall

There is an old saying that the only safe computer is one that’s disconnected from the network, turned off, and locked in an underground bunker—and even then you can’t be sure!

Since most of us can’t afford to keep our servers in an underground bunker, the least little thing that could have been done in order to keep their threat exposure at rock-bottom is protecting them by running a combination of a firewall and an intrusion prevention system or IPS (a.k.a intrusion detection and prevention systems or IDPS). Surely, that alone proved insufficient and other security measures and best practices should also be considered.

This blog post covers setting up a basic secure and stateful IPFW firewall on FreeBSD along with Sshguard by iXsystems Inc as intrusion prevention system.

IPFW

Traditionally FreeBSD has three firewalls built into its base system: PF, IPFW, and IPFILTER, also known as IPF. In my estimation, IPFW would be the natural choice on FreeBSD if we set aside the pros and cons of each. In contrast to the other two, IPFW was originally written for FreeBSD and its main development platform - if we do not count the DragonFly’s fork - is still FreeBSD. This means that the latest features are always available on FreeBSD. On the contrary, this is not true for PF or IPF on FreeBSD. So, that’s why I chose to go with IPFW.

Before I begin, I have to mention that this guide was written for FreeBSD 10.1-RELEASE and 10-STABLE, and it may not work with older releases. I cannot verify this since all my servers and workstations are either running FreeBSD 10.1-RELEASE or 10-STABLE at the time of writing. So, you are on your own if you are trying this on an older release.

OK, in order to configure our firewall we have to modify /etc/rc.conf. First, you should make sure no other firewall is running by looking for pf_enable=“YES” or ipfilter_enable=“YES” inside /etc/rc.conf. If you have any of them, you should disable them by either setting their value to “NO” or removing them completely. After that we can enable and configure our IPFW firewall inside /etc/rc.conf:

/etc/rc.conf
1
2
3
4
5
6
firewall_enable="YES"
firewall_quiet="YES"
firewall_type="workstation"
firewall_myservices="11011 domain http https imap imaps pop3 pop3s smtp smtps"
firewall_allowservices="any"
firewall_logdeny="YES"

I’m not going verbose except for firewall_myservices which requires explanation. If you would like to play with these options and you are on a SSH session, please be wary of the fact that even the slightest change in the above setup may drop the connection, therefore, close the session and effectively lock you out of the server. For example firewall_quiet=“NO” alone is enough for such a scenario. So, please take a look at FreeBSD Handbook or /etc/rc.conf man page before any modification in case you are not sure what you’re doing. I also highly recommend this great article by Digital Ocean which did a good job of summing it all up in a novice-friendly way.

By default the above setup blocks all inbound connections on all ports for both TCP and UDP. firewall_myservices is a white-space separated list of TCP ports. So, you have to address a specific port here if you have an obligation to allow inbound TCP connection for that port. You must have noticed by now, instead of specifying port numbers I mentioned them by name in the firewall_myservices list. It’s possible to either use port numbers or address a port by name if a known or standard service uses that specific port.

For example, let’s say you are running dns/bind910 on your server and you know it’s listening for dns queries on port 53. You can search for the port name which FreeBSD recognizes by looking up the port number inside /etc/services.

$ cat /etc/services | grep -w "53"
domain		 53/tcp	   #Domain Name Server
domain		 53/udp	   #Domain Name Server

In the above example the port name for both TCP and UDP is domain. Now, let’s consider another example: You are running a mail server and instead of specifying the port name or protocol you would like to directly use port numbers in firewall_myservices list. So, we do the exact opposite for our outgoing server:

$ cat /etc/services | grep -w "smtp"
smtp		 25/tcp	   mail		#Simple Mail Transfer
smtp		 25/udp	   mail		#Simple Mail Transfer
smtps		465/tcp	   #smtp protocol over TLS/SSL (was ssmtp)
smtps		465/udp	   #smtp protocol over TLS/SSL (was ssmtp)

As you can see the equal port numbers are 25 and 465. Depending on how you did configured your mail server (e.g. 25 for PLAIN and STARTTLS, 465 for TLS/SSL) you can pick one or both 25 and 465.

As you may have noticed already, I did mixed numbers and names in the firewall_myservices list. That’s because I used a non-default port instead of 22 for SSH. It is perfectly fine to mix-up numbers and names in the list.

Now it’s time to start the firewall by running the following command:

$ service ipfw start

After starting the firewall you can always check the current rules by using the following command:

$ ipfw list

Now, you may ask there is one little issue with configuring IPFW through firewall_myservices. This list is TCP only and there is no way to configure UDP ports through /etc/rc.conf. OK, we address that by extending the default configuration through our own script without touching any files from the base system which is the proper way to do so. If you take a look at /etc/rc.firewall script, I’m sure you’ll figure out how FreeBSD applies firewall_* options. So, we adopt the same approach by writing our own script. Therefore, we create and put the following script inside /usr/local/etc/ in the first step:

/usr/local/etc/rc.firewall
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/bin/sh

#  (The MIT License)
#
#  Copyright (c) 2015 Mamadou Babaei
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the "Software"), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


. /etc/rc.subr

load_rc_config firewall

fw_add="/sbin/ipfw -q add"

case ${1} in
quietstart)
        for i in ${firewall_allowservices} ; 
        do
            for j in ${firewall_myservices_tcp} ;
            do
                ${fw_add} pass tcp from ${i} to me ${j}
            done
        done

        for i in ${firewall_allowservices} ;
        do
            for j in ${firewall_myservices_udp} ;
            do
                ${fw_add} pass udp from ${i} to me ${j}
            done
        done
    ;;
quietstop)
    ;;
*)
    echo "Error: unknown parameter '${1}'"
    ;;
esac

Then we have to fix its permissions to make it read-only and executable by anyone:

$ chmod u-w,ugo+x /usr/local/etc/rc.firewall

In the end, we have to apply our own configuration for both TCP and UDP ports inside /etc/rc.conf:

/etc/rc.conf
1
2
3
4
5
6
7
8
9
firewall_enable="YES"
firewall_quiet="YES"
firewall_type="workstation"
#firewall_myservices="11011 domain http https imap imaps pop3 pop3s smtp smtps"
firewall_allowservices="any"
firewall_logdeny="YES"
firewall_coscripts="/usr/local/etc/rc.firewall"
firewall_myservices_tcp="11011 domain http https imap imaps pop3 pop3s smtp smtps"
firewall_myservices_udp="domain"

Then it’s time to restart IPFW firewall and check whether our script did its job properly or not:

$ service ipfw restart
$ ipfw list

firewall_myservices_tcp and firewall_myservices_tcp are our own extended options. Note that we’ll leave alone firewall_myservices despite the fact that it’s still functional. For the sake of clarity we choose to use our own firewall_myservices_tcp and avoid using or mixing it with firewall_myservices. So, from now on we abandon firewall_myservices.

Last but not least, a recommended best practice is putting a limit on the logging of denials per IP address. This will prevent a single, persistent user to fill up our logs. To do so we should adjust net.inet.ip.fw.verbose_limit by issuing the following command:

$ sysctl net.inet.ip.fw.verbose_limit=5

This adjusts the number of logs per IP to 5. In order to make that permanent and apply it on future reboots we should add it to /etc/sysctl.conf:

/etc/sysctl.conf
net.inet.ip.fw.verbose_limit=5

Sshguard

Do not let the name fool you. Sshguard protects many services out of the box, including but not limited to: OpenSSH, Sendmail, Exim, Dovecot, Cucipop, UW IMAP, vsftpd, ProFTPD, Pure-FTPd and FreeBSD ftpd.

Sshguard monitors servers from their logging activity. When logs convey that someone is doing a Bad Thing, sshguard reacts by blocking he/she/it for a bit. Sshguard has a touchy personality: when a naughty tyke insists disturbing your host, it reacts firmer and firmer.

Sshguard supports many services out of the box, recognizes several log formats, and can operate many firewall systems. Many users appreciate its ease of use, compatibility and feature richness.

Currently, FreeBSD Ports provides five variants of Sshguard:

  • security/sshguard Protects your host from brute force attacks against ssh and other services.
  • security/sshguard-ipfilter Utilizes IPF to protect your host from brute force attacks against ssh and other services
  • security/sshguard-ipfw Utilizes IPFW to protect your host from brute force attacks against ssh and other services
  • security/sshguard-null Do-nothing backend for applying detection but not prevention.
  • security/sshguard-pf Utilizes PF to protect your host from brute force attacks against ssh and other services.

So, we choose security/sshguard-ipfw since we just setup up an IPFW firewall. To install Sshguard for IPFW from Ports collection:

$ cd /usr/ports/security/sshguard-ipfw/
$ make config-recursive
$ make install clean

Or if you are using pkgng instead of Ports:

$ pkg install security/sshguard-ipfw

Finally, to enable Sshguard for a typical usage we have to modify our /etc/r.conf as follows:

/etc/rc.conf
sshguard_enable="YES"

But, Sshguard on FreeBSD gives you more options to tweak. The valid options are:

  • sshguard_enable [BOOLEAN, DEFAULT: NO]: Set it to YES to enable sshguard.
  • sshguard_pidfile [STRING, OPTION: -i, DEFAULT: /var/run/sshguard.pid]: Path to PID file.
  • sshguard_watch_logs [STRING, DEFAULT: /var/log/auth.log:/var/log/maillog]: Colon splitted list of logs to watch.
  • sshguard_blacklist [STRING, OPTION: -b, DEFAULT: 40:/var/db/sshguard/blacklist.db]: [threshold:]/path/to/blacklist.
  • sshguard_safety_thresh [INTEGER, OPTION: -a, DEFAULT: 40]: Safety threshold.
  • sshguard_pardon_min_interval [INTEGER, OPTION: -p, DEFAULT: 420]: Minimum pardon interval in seconds.
  • sshguard_prescribe_interval [INTEGER, OPTION: -s, DEFAULT: 1200]: Prescribe interval in seconds.
  • sshguard_whitelistfile [STRING, OPTION: -w, DEFAULT: /usr/local/etc/sshguard.whitelist]: Path to the whitelist.
  • sshguard_flags [STRING]: Set additional command line arguments.

Please note that some of the above options directly map to their command line options. So, it won’t do any harm to consult the manual page sshguard(8) for detailed information.

Anyway, let’s start Sshguard:

$ service sshguard start

Now, we can test our setup from another machine with different IP to see if it works. I did tried it without any luck. Unfortunately, it did not worked as expected since IPFW runs a first-match-win policy. This is due to the fact that the rules generated by Sshguard has lower priority than the ones generated by our script that we wrote earlier. According to the official documentation of Sshguard, it adds blocking rules with IDs from 55000 to 55050 by default:

With IPFW, sshguard adds blocking rules with IDs from 55000 to 55050 by default. If a pass rule appears before these, it is applied because IPFW runs a first-match-win policy.

If you have an allow policy higher than 55050 in your IPFW chain, move it to a lower priority.

So we have to modify our own script to adapt to the new situation:

/usr/local/etc/rc.firewall
#!/bin/sh

#  (The MIT License)
#
#  Copyright (c) 2015 Mamadou Babaei
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the "Software"), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


. /etc/rc.subr

load_rc_config firewall

fw_add="/sbin/ipfw -q add"

if [ -z "${firewall_myservices_rules_id_start}" ] ;
then
    firewall_myservices_rules_id_start=56000
fi

if [ -z "${firewall_myservices_rules_id_step}" ] ;
then
    firewall_myservices_rules_id_step=10
fi

case ${1} in
quietstart)
        rule_id=${firewall_myservices_rules_id_start}

        for i in ${firewall_allowservices} ;
        do
            for j in ${firewall_myservices_tcp} ;
            do
                ${fw_add} ${rule_id} pass tcp from ${i} to me ${j}
                rule_id=`expr $rule_id + ${firewall_myservices_rules_id_step}`
            done
        done

        for i in ${firewall_allowservices} ;
        do
            for j in ${firewall_myservices_udp} ;
            do
                ${fw_add} ${rule_id} pass udp from ${i} to me ${j}
                rule_id=`expr $rule_id + ${firewall_myservices_rules_id_step}`
            done
        done
    ;;
quietstop)
    ;;
*)
    echo "Error: unknown parameter '${1}'"
    ;;
esac

So, we added firewall_myservices_rules_id_start and firewall_myservices_rules_id_step as a way to control the ID assignment of the rules. Now our final configuration for IPFW + Sshguard looks like this one:

/etc/rc.conf
firewall_enable="YES"
firewall_quiet="YES"
firewall_type="workstation"
firewall_allowservices="any"
firewall_logdeny="YES"
firewall_coscripts="/usr/local/etc/rc.firewall"
firewall_myservices_rules_id_start="56000"
firewall_myservices_rules_id_step="10"
firewall_myservices_tcp="11011 domain http https imap imaps pop3 pop3s smtp smtps"
firewall_myservices_udp="domain"

sshguard_enable="YES"
sshguard_pidfile="/var/run/sshguard.pid"
sshguard_watch_logs="/var/log/auth.log:/var/log/maillog"
sshguard_blacklist="40:/var/db/sshguard/blacklist.db"
sshguard_safety_thresh="40"
sshguard_pardon_min_interval="420"
sshguard_prescribe_interval="1200"
sshguard_whitelistfile="/usr/local/etc/sshguard.whitelist"
sshguard_flags=""

After making the final modification we have to restart IPFW once more and check out the rules:

$ service ipfw restart
$ ipfw list

From now on, our rules IDs for ports specified in both ${firewall_myservices_tcp} and ${firewall_myservices_udp} should start from 56000 with a 10 step gap in between. e.g. 56000, 56010, 56020….

Ultimately, at anytime you can check the blocked IP’s by issuing the following command:

$ ipfw list | awk '{ if ( $1 >= 55000 && $1 <= 55050 ) print $5 }'

Or directly checking the blacklist file:

$ cat /var/db/sshguard/blacklist.db

Unban

If you or some other host get banned, you can wait to get unbanned automatically or use the following set of instructions:

First check if you are banned by Sshguard:

$ ipfw list | awk '{ if ( $1 >= 55000 && $1 <= 55050 ) print $5 }'

Or

$ cat /var/db/sshguard/blacklist.db

If the answer is positive, the are two more steps that you should take to get unbanned. First you have to remove the host from Sshguard blacklist database, e.g. /var/db/sshguard/blacklist.db if you are using the default option. Let’s say we would like to unblock the penultimate IP in the following database which is 192.168.10.101:

$ cat /var/db/sshguard/blacklist.db

# SSHGuard blacklist file ( http://www.sshguard.net/ ).
# Format of entries: BLACKLIST_TIMESTAMP|SERVICE|ADDRESS_TYPE|ADDRESS
1437762353|100|4|10.12.0.4
1437850200|100|4|192.168.10.17
1437893500|260|4|10.10.0.18
1437903401|260|4|10.10.0.23
1437997365|100|4|192.168.10.101
1438253201|260|4|10.10.0.17

$ sed -i '' '/192.168.10.101/ d' /var/db/sshguard/blacklist.db

The next step involves removing the related rule from IPFW:

$ ipfw list | grep "192.168.10.101"

55037 deny ip from 192.168.10.101 to me

$ ipfw -q delete deny ip from 192.168.10.101 to me

Or simply delete the rule using its ID:

$ ipfw list | grep "192.168.10.101"

55037 deny ip from 192.168.10.101 to me

$ ipfw -q delete 55037

Sshguard Won’t Start

Remove both PID file and blacklist file, then start Sshguard service again:

$ rm /var/run/sshguard.pid /var/db/sshguard/blacklist.db
$ service sshguard start

Source Code

Check out the source code on GitLab

Check out the source code on GitHub