fail2ban or How I Learned to Love netfilter

fail2ban or How I Learned to Love netfilter

If you have ports open to the internet you need to use Fail2ban. It has been around for a long time and it's pretty ubiquitous in Linux systems exposed to the public. This article is going to walk you through the process I use to setup fail2ban on the machines which have ports open to the public.

The first step is to install the package:

sudo apt update
sudo apt install fail2ban

This should install and start the fail2ban service. On a systemd based system, you can check it's status with the sudo systemctl status fail2ban command. Now that it's installed, let's open up the configuration file and take a look.

Hopefully you get the general idea of how the configuration works from inspecting those files. The next step is to create a local copy of data that might get overwritten in an upgrade (as a side note, that's also why the directories with someserver.d exist in many modern software packages).

The command below uses a bash shortcut to copy jail.conf to jail.local (the .conf file can get overwritten when changes are made in that file provided by the package manager, i.e. yum & apt. For more information about how this command works, see any bash user's guide.

sudo cp /etc/fail2ban/jail.{conf,local}

Now for configuring the local jails, which are just (a) script(s) that create the rules fail2ban follows. Using SSH ( secure shell) as an example, mainly due to how common SSH is and the fact that almost everyone needs to do something to stop brute force attacks on SSH if you have it open to the internet. If you prefer using a different editor, please feel free to use it instead.

sudo vi /etc/fail2ban/jail.local

In this file you will want to configure some "Miscellaneous Options", should you find the need. Specifically the length of time an IP address will be blocked by iptablesas well as the how far back in time to look for previous login attempts. I personally leave these settings at their default values.

bantime = 10m

findtime = 10m

Ten minutes for both, so any attempts from the same IP address over a 10 minute period will be counted against the total. Which brings us to:

maxretry = 5

This is the number of failed attempts before a ban is enacted. I sometimes set this value lower if the system will only be accessed by administrators and power users or higher if the users will be general users that don't access remote servers often. Like most configurable settings, this is something you will need to determine yourself and adjust as needed.

The next section is "Actions". I usually don't change many of these but I do configure the emails sent to root to be forwarded to my user's mailbox or the admin team's alias' mailbox . You can take a look at the "actions" in this section and see what the pre-configured options are then determine if you need to change them for your own needs. There is a short description above each and are fairly self-explanatory. I don't recommend editing these unless you have a good reason and are prepared to spend a non-trivial amount of time configuring them.

Finally - "Jails". This is where you define what happens for each service you want fail2ban protection on. Typically SSH is the first jail you will come to in the file, so lets take a look.

[sshd]

# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
#mode   = normal
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

This jail is ready to be enabled as it is. I usually enable and add other setting changes in the /etc/fail2ban/jail.d directory so I can easily add and remove them but it is fine to do it in the jail.local file as well. In my case I typically run Debian and /etc/fail2ban/jail.d/defaults-debian.conf exists by default so I edit as shown below to make these changes. Again, if vim is not your cup of tea, you can use your preferred editor.

sudo vi /etc/fail2ban/jail.d/defaults-debian.conf

[sshd]
enabled = true

[ssh-repeater]
enabled = true

It feels complicated but it's really pretty simple so far. Now, let's get into something a bit more interesting.

I sometimes see a number of the same IP addresses getting banned over and over again. And I wondered if anyone else had the same problem, that's when I found this: https://stuffphilwrites.com/2013/03/permanently-ban-repeat-offenders-fail2ban/ (There has been an update to Phil's site that indicates there is an easier option which you may want to implement, I've not tried it so I cannot speak to it but it does look very easy to implement - I may update this post in the future.)

So below I have a jail that "permanently" blocks IP addresses once they are repeatedly caught by the SSH jail. The setup is a bit more complicated because we are adding something completely new to the fail2ban service and need to tell it what to do. In /etc/fail2ban/action.d we need to create a file named iptables-repeater.conf and place the following in it, elevated privileges (sudo) will be required to create and/or edit the file.

# Fail2ban configuration file
#
# Author: Phil Hagen <phil@identityvector.com>
#

[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart = iptables -N fail2ban-REPEAT-<name>
              iptables -A fail2ban-REPEAT-<name> -j RETURN
              iptables -I INPUT -j fail2ban-REPEAT-<name>
              # set up from the static file
              cat /etc/fail2ban/ip.blocklist.<name> |grep -v ^\s*#|awk '{print $1}' | while read IP; do iptables -I fail2ban-REPEAT-<name> 1 -s $IP -j DROP; done

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop = iptables -D INPUT -j fail2ban-REPEAT-<name>
             iptables -F fail2ban-REPEAT-<name>
             iptables -X fail2ban-REPEAT-<name>

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck = iptables -n -L INPUT | grep -q fail2ban-REPEAT-<name>

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionban = iptables -I fail2ban-REPEAT-<name> 1 -s <ip> -j DROP
            # also put into the static file to re-populate after a restart
            ! grep -Fq <ip> /etc/fail2ban/ip.blocklist.<name> && echo "<ip> # fail2ban/$( date '+%%Y-%%m-%%d %%T' ): auto-add for repeat offender" >> /etc/fail2ban/ip.blocklist.<name>

# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionunban = /bin/true

[Init]

# Defaut name of the chain
#
name = REPEAT

Now in the jail.local file we took a look at above, add the following just below the SSH section. Like the rest of the edits, this one requires root permissions to edit.

[ssh-repeater]
filter   = sshd
action   = iptables-repeater[name=ssh]
           sendmail-whois[name=SSH-repeater, dest=root, sender=root]
logpath  = %(sshd_log)s
maxretry = 21
findtime = 31536000
bantime  = 31536000

As you can see, it sets the bantime & findtime to a very big number (31536000 is the number of seconds in a year). I've made a change to the jail above from the blog post I got it from, specifically the logpath has been changed because I'm on debian while Phil is using Red Hat (other debian based systems, like ubuntu, will need this change as well).

To apply the changes run the following on systemd systems

sudo systemctl restart fail2ban

That's it, now you can be fairly sure you won't get brute forced from the bad actors on the internet. You may want to check out the article where I go over fail2ban-client and what to do when you need to unban an IP address or want to manually ban one.