New firewall system for Sympl using nftables

Feature Discussion
1

I’ve been working for some time on replacing the iptables based Symbiosis/Sympl firewall with a version that drives nftables. The system is written in Python 3. It’s now in a late beta state and I’ve had it running for some months on a couple of machines. It’s doing its job.

It’s called nftfw, and you can find it on github:

https://github.com/pcollinson/nftfw

along with some suggested changes to your mail system

https://github.com/pcollinson/sympl-email-changes

that will make exim4 and dovecot report more information to the firewall to keep the bad guys out. This is based on the files on your sympl system, and so the complete fileset can be dropped in.

There’s a bunch of documentation and explanations of how nftfw works and how to use it. There are installation instructions, and hints in what you need to install, and latterly a test suite.

What’s the deal?

Nftfw uses the same admin interface as Symbiosis/Sympl (because that’s the point), you can drive it from the files installed on your system, but it’s probably a good idea to use a separate set of control files because you’ll get more out of the system.

Nftfw used nftables sets to lookup ip addresses that you want to block. Adding a new ip address to these sets doesn’t mean a complete reload of the firewall, only the sets are reloaded. Sets mean that the ruleset is fairly slim.

The system allows you to provide feedback from firewall rejections to ensure that frequent visitors remain locked out.

The patterns.d directory can contain settings that tell ntffw how to extract log entries made by the nftables system in the logs, and it uses those to update its database. When the bad guys knock at your door again, they stay knocking.

Nftfw allows you to see what is happening on the firewall, who you are locking out. why they are locked out and how long it’s been since they started attacking you. If you install geoip2, you can see what countries they come from.

The listing program can also be inserted into a web page so you can view what’s happening remotely. Do make sure that you password protect the page.

There are several improvements. You can put a file expansion glob statement into the pattern files so one rule set can be used for all your websites under /srv. You can create testing pattern files to check if your regular expressions are working.

If you are interested in installng nftfw, please read the Installation document at least once before you start - the link is at the bottom of the nftfw github Readme page.

What do I want?

Well, please look at the last few paras on the Readme on nftfw github front page. I’m looking for people who are interested in helping to get this tested and in use more widely. Paul Cammish is interested in the
project for Sympl, but I think if it can be used on people’s systems it will help in making it a robust and tested alternative to the current system.

Over to you…

1 Like
2

I have two or three test instances of Sympl that I could try it on, though they’re not really doing much work at present.

3

Well can I encourage you to try, hopefully it will give you the confidence to run it in production.

I think that any system offering email is under attack from the moment it appears as a public address on the net. Today I have 146 IP live addresses in my firewall on my Mythic Beasts v-machine - most are email attackers. 7 of them are active in the first 12 hours of today, 3 of which are robots that send packets every minute or so, and have been for some number of days. The ‘leader’ is a system in Panama, probably a botnet member, who’s been at it since 24/April.

nftables sets are the way to deal with this number of IP addresses and not the long lists of tests that the Symbiosis system uses. Also, nftfw checks for established, related and invalid packets very early on, the Symbiosis firewall puts every packet through the blacklist test BEFORE checking for existing connections - loadsa processing for every packet.

The machine sitting at this end of my broadband has a web server, with nothing very interesting on it for external people. Apart from the front page, it’s entirely password controlled. The firewall there has had incidents from 529 distinct IP addresses in the last 90 days.

So there’s a lot of activity - and it only takes one ‘easy’ password…

4

Thank you, this looks like great work.
I’m interested because I’m already using nftables. SSH access is kept out by whitelisting the 2 addresses I use, but I have fail2ban monitoring failed connection attempts to exim4 and dovecot. F2B has a “recidive” feature which bans persistent offenders for a day instead of the normal 10 minutes. My figures are very different; I’ve never seen more than 20 IP addresses in the “recidive” list.
It looks like your system replaces fail2ban as well as having some nice usability features. If it turns out to be easier to set up than fail2ban that would be an advantage.
So, I have a secondary server on which I could install Sympl, transfer a couple of low-importance domains and try all this stuff out. Don’t hold your breath, but I’ll see what happens.

5

Actually it’s standalone - and doesn’t need sympl to run. I have it on my home machine on a vanilla Debian Buster system.

6

This is great, and I’ll be looking at adding it to all to the
standard Sympl install when possible.

7

Looks great I’ll attempt to get it up and running.

8

OK, first feedback thoughts. I think I’ve installed it OK on a default Sympl/Buster system, though I’ve not yet tried to reboot, or even to test much. Just want to give my initial impression.

The instructions are a little too informative.

Assume I know less than you think, tell me what to do. Yes, it’s good to explain what it does, but make the default path through the task as clear as possible.

As far as I know, only two things went wrong:

  1. At the stage of “sudo update_alternatives --config iptables”, “update_alternatives” was not found. I think I installed iptables (or something: the postman came to the door at that point) and then I was back on track.
  2. Right at the end, in the README for the cronfiles, it has the line:
    sudo chmod 644 /etc.incron.d/nftfw
    which, of course, should be
    sudo chmod 644 /etc/incron.d/nftfw
9

I concur with the HairyDog!

I understand the square root of nothing about Linux so could you kindly flesh out the installation instructions please Sir?

Something like this would be more user friendly: Enabling SNI for Exim on Symbiosis - Documentation | Bytemark

10

The other bit that needs feedback is the Geolocation section. I went to the website and created an account, but got lost at that point, so I skipped the whole section.
Please make it clearer what to do at that point, with nice little screen-scrape examples like
$ sudo apt install magic-fix

11

update_alternatives is a typo - should have a hyphen. Thanks I’ll fix both of them.

If you were running legacy iptables, and

$ sudo iptables -V
iptables v1.8.2 (nf_tables)

doesn’t say ‘nf_tables’ but says ‘legacy’, then you need to revisit that part of the document and redo:

$ sudo iptables-save > ipsaved
$ sudo ip6tables-save > ip6saved

$ sudo update-alternatives --config iptables
$ sudo update-alternatives --config ip6tables

$ sudo iptables-restore < ipsaved
$ sudo ip6tables-restore < ip6saved

$ sudo iptables-legacy -F
$ sudo ip6tables-legacy -F

If the -V does say that then this isn’t a problem.
You are asking for less text, it seems that @compassweb wants more :grinning:.

Would a simple unadorned list suit your needs?? The problem is that it’s complicated and I was at all times trying to prevent people from killing their system…

1 Like
12

I think if you need an idea of what it’s trying to do, then you need to read some of the early part of the User’s Guide which attempts to explain what it does. Starting from the installation script may not be the place for you. It’s very hard to know where to pitch all this.

It seems you want more text, @Hairydog wants less… it’s hard to please all of the people all of the time :grinning:

13

I don’t think that either compassweb or I actually want different things. I know that I am asking for a simple “default” path through the process, where you tell me the recommended defaults to use. I suspect that he is asking the same, wanting actual examples of commands.

Yes there are alternative settings people can use if they know what they are doing, but my feeling is that I understand enough to go along with what is happening, but not enough to choose a different route, so please don’t offer me choices without making it easiest to do the “right” thing.

By the way, by not grasping what you meant about geolocation, I get:

$ nftfwls
Traceback (most recent call last):
File “/usr/local/bin/nftfwls”, line 6, in
from nftfw.nftfwls import main
File “/usr/local/lib/python3.7/dist-packages/nftfw/nftfwls.py”, line 12, in
from geoipcountry import GeoIPCountry
File “/usr/local/lib/python3.7/dist-packages/nftfw/geoipcountry.py”, line 9, in
from geoip2.database import Reader
ModuleNotFoundError: No module named ‘geoip2’

But I don’t know what to do to fix it.

14

Fixes are now on GitHub…

If you did a clone - you can cd to the directory you installed in and do

git pull

It will now work without the geoip2 package installed

Oh and you’ll need to rerun

pip3 install .
15

I had another go, and now get

sudo nftfwls
±------±-----±--------±-------±------±---------±--------+
| IP(0) | Port | Ct/Incd | Latest | First | Duration | Pattern |
±------±-----±--------±-------±------±---------±--------+
±------±-----±--------±-------±------±---------±--------+

16

Well that’s what it looks like if you have nothing in the database.

17

To be honest @compassweb, you’re probably best off not replacing odd bits of Sympl which are somewhat needed (ie: the firewall) unless you really need nftables.

This will likely find it’s way into the core Sympl packages sometime in the future, but if you’re making changes to it outside that, you may end up with a broken system and the need to reinstall, or potentially worse if the firewall isn’t working how you expect!

18

It is quite possible. This is a new server with almost nothing on it, and nearly all the attempts were on port 22, which I’ve switched off.

I’ve played some more with the Maxmind stuff, which did not play nicely at all. I seem to have v3.1.1 of geoipupdate, and their instructions about how to upgrade to v4 just take you round in circles.
So I decided to stick with 3.1.1 but it wouldn’t accept the API keys. I tried several times, but there was nothing wrong.
Then I tried clicking the pencil icon next to the line on the Licence Keys page. Didn’t visibly do anything,but suddenly the keys are accepted and the updates seem to have downloaded.

19

It may take time for them to update the databases. Sounds as if you’ve been installing things from code? Version 3.1.1 is delivered by Debian, I’ve had no problem with it. There was some change in the keys in an earlier version, I seem to remember.

20

New version with documentation changes - I’ve separated out the Geolocation install into a new document.