How to Secure NGINX with Custom Fail2ban Filters

Fail2Ban is a great security measure to deploy for your web application server. It comes with a range of features, default filters, and actions that can immediately impact banning bad web bots, draining your system resources, and stopping attacks, which is the most crucial part of any website.

However, most people system admins and website owners are looking for sometimes a bit more extra than what fail2ban has to offer. In the following tutorial, you will learn how to create and use custom filters on your Nginx server, which can be fine-tuned to suit your own needs and expanded later.

Make sure to test and use with caution any new fail2ban filters. Note that settings will need to be adjusted to your needs.

Advertisement

Add New Fail2Ban Jails

The tutorial assumes you have Fail2ban installed and are familiar with configuring jails and settings. Below are some additional jails you will need to tweak to your liking if you are using further ban actions like Cloudflare or reporting to AbuseIPDB.org etc.

Add the following jails you wish to use in your /fail2ban/jail.local file.

sudo nano /etc/fail2ban/jail.local
[nginx-403]
 enabled = true
 port     = http,https
 filter = nginx-403
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 4

[nginx-404]
 enabled = true
 port     = http,https
 filter = nginx-404
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 30 # <--- monitor and adjust, all servers are different.

[nginx-noagent]
 enabled = true
 port     = http,https
 filter = nginx-noagent
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[nginx-noauth]
 enabled = true
 filter = nginx-noauth
 action = iptables-allports
 logpath = %(nginx_error_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

[nginx-no-x-spam]
 enabled = true
 filter = nginx-no-x-spam
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

[nginx-noscript]
 enabled = true
 action = iptables-allports
 filter = nginx-noscript
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[nginx-noproxy]
 enabled = true
 action = iptables-allports
 filter = nginx-noproxy
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 0

[nginx-nowordpress]
 enabled = true
 action = iptables-allports
 filter = nginx-nowordpress
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[portscan-block]
 enabled = true
 action = iptables-allports
 filter = portscan-block
 logpath = /var/log/ufw.log  <--- this has to be direct to your UFW logs.
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

Create new Fail2Ban Filters

Next, you need to make a new filter file for each of these filters and put .conf at the end of the file.

Note, the created file is something you need to do yourself.

create file /location/fail2ban/filter.d/nginx-403.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 403

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-404.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 404

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noagent.conf

 [Definition]

 failregex = ^<HOST> -.*"-" "-"$
                   ^<HOST> -.*"-" "curl.*"$

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noauth.conf

 [Definition]

 failregex = no user/password was provided for basic authentication.client:              user . was not found in.client:              user . password mismatch.*client: 

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-no-x-spam.conf

 [Definition]

 failregex = {"log":"<HOST> .* ".*\\x.*" .*$

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noscript.conf

 [Definition]

 failregex =     ^<HOST> -.*GET.*(\.asp|\.exe|\.pl|\.cgi|\.scgi)

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noproxy.conf

 [Definition]

 failregex = ^<HOST> -.*GET http.*
                   ^<HOST> -.*CONNECT.*

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-nowordpress.conf

 [Definition]
            
 failregex = ^<HOST> -.*"(GET|POST|HEAD) /+(?i)(wp(-|/)|xmlrpc.php|\?author=1)
                   ^<HOST> -.* "(GET|POST|HEAD|PROPFIND) /+(?i)    (a2billing|admin|apache|axis|blog|cfide|cgi|cms|config|etc|.git|hnap|inc|jenkins|jmx-|joomla|lib|linuxsucks|msd|muieblackcat|mysql|myadmin|n0w|owa-autodiscover|pbxip|pma|recordings|sap|sdk|script|service|shell|sqlite|vmskdl44rededd|vtigercrm|w00tw00t|webdav|websql|wordpress|xampp|xxbb)
             ^<HOST> -.* "(GET|POST|HEAD) /[^"]+.(asp|cgi|exe|jsp|mvc|pl)( |\?)
             ^<HOST> -.*(?i)(/bash|burger-imperia|changelog|hundejo|hvd-store|jorgee|masscan|pizza-imperia|pizza-tycoon|servlet|testproxy|uploadify)

 ignoreregex =

 journalmatch = _SYSTEMD_UNIT=nginx.service + _COMM=nginx

create file /location/fail2ban/filter.d/portscan-block.conf

[Definition]

failregex = .*\[UFW BLOCK\] IN=.* SRC=<HOST>

ignoreregex = SRC=(10.|172.1[6-9].|172.2[0-9].|172.3[0-1].|192.168.|fe\w:). DST=(static.ip.address.here|224.0.0.). PROTO=(2|UDP)(\s+|.* DPT=(1900|3702|5353|5355) LEN=\d*\s+)$

WebExploits Filter

A top-rated fail2ban filter is blocking URL malicious bot scanners. However, this filter is not recommended for users new to Linux servers, and fail2ban especially does not have the time to monitor and adjust. The biggest issue with this filter is that it can be some real users, but mostly it will ban SEO good bots that may accidentally scan the same trap.

You can download several community-made and updated lists, but the best way is to monitor your logs and add your own slowly when you notice scans. Over time you will amass an extensive list custom-made by you and will work with zero positives.

First, add the jail for the web exploits script.

Example:

[webexploits]
enabled   = false
logpath = %(nginx_access_log)s
filter = webexploits
port    = http,https
bantime = 1440m
findtime = 1440m
maxretry = 0

Next, create the filter, but this is where it gets interesting.

Example:

create file /location/fail2ban/filter.d/webexploits.conf

[Definition]

failregex = 
            ^<HOST> -.*(GET|POST|HEAD).*(/data/admin/)
            ^<HOST> -.*(GET|POST|HEAD).*(/dnt-policy.txt)
            ^<HOST> -.*(GET|POST|HEAD).*(/gpc.json)
            ^<HOST> -.*(GET|POST|HEAD).*(/wlwmanifest.xml)
            ^<HOST> -.*(GET|POST|HEAD).*(/adminer)
            ^<HOST> -.*(GET|POST|HEAD).*(/_adminer)
            ^<HOST> -.*(GET|POST|HEAD).*(/user/login)
            ^<HOST> -.*(GET|POST|HEAD).*(/.env)

ignoreregex =

As above, those are just some typical examples. Then add your own, and modify the filter in the following format.

            ^<HOST> -.*(GET|POST|HEAD).*(/URL-TO-ADD-HERE)

From the example, the URL-TO-ADD-HERE is the extension where you see malicious users or bots scanning, such as WordPress XMLRPC.

            ^<HOST> -.*(GET|POST|HEAD).*(/xmlrpc.php)

You can also modify the GET|POST|HEAD options, but you want to ban anyone touching those URLs, which shouldn’t need to be changed.

Some example lists are here on this Github.

DO NOT JUST RANDOMLY COPY AND PASTE.

This requires testing, and I suggest adding your own from your logs. This can seriously backfire if you do not take the time and dedication to correct this. However, it can reduce bots on huge websites wasting resources scanning multiple URLs.

Make sure to test your filters as below will cover.

Nginx Secure Pages & Honeypot

Another interesting feature you can do with Nginx is making a 444 filter, then making geo locations to, for example, hide pages such as login that no good bot or the real user should be touching.

First, add the jail.

Example:

[nginx-444]
 enabled = true
 port     = http,https
 filter = nginx-444
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 1

Next, add the jail this time, making it an instant ban. I would advise adding the page to your robots.txt as an exclusion so that good SEO bots like Google will not touch it. You could make fake pages and exclude them also in robots.txt as honeypots. This example works for actual pages or honeypots.

create file /location/fail2ban/filter.d/nginx-444.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 444

 ignoreregex =

Next, create a file and whitelist your IP address or the IP ranges you want to be excluded that can reach the page.

Example location:

sudo nano /etc/nginx/whitelist.txt

An example of adding an IP address is below. Take note of the false at the end.

IP Address false;

or another example

195.168.50.1 false;

Add the following: once you have added your IP address in your nginx.conf file.

geo $remote_addr $exclude_trafic {
 default true;
include /etc/nginx/whitelist.txt;
}

Notice exclude all traffic is defaulting true.

Now, in your server block, add the following file.

if ( $allowed_trafic = 'true'){
    return 444;
}

FOR PHP such as WordPress locking off wp-login.php example.

  location ~ /wp-login.php {
if ( $exclude_trafic = 'true'){
    return 444;
} 
 fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/run/php/php8.1-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param SCRIPT_NAME $fastcgi_script_name;
    include fastcgi_params;
    include snippets/fastcgi-php.conf;
    fastcgi_intercept_errors on;
}

Note, this may need tinkering depending on your PHP version and Distribution, more importantly. Still, the example defaults to exclude all traffic from that page unless you are in the whitelist with false.

This is mostly not needed if you are behind Cloudflare or similar services, but this can come in handy more for honeypots on fake pages, be sure to add in your HTML links to it so it will appear to bot scrapers only and exclude it in your robots.txt so real bots that abide by your rules won’t touch it (malicious bots don’t obey robots.txt).

Advertisement

Test New Fail2ban Jails/Filters

To finish it off, make sure to restart fail2ban in your operating system via its command.

For most Linux distributions, the following command should work if running systemd.

sudo systemctl restart fail2ban

Other non-systemd distributions or other systems restart your system or fail2ban service.

Once fail2ban has been restarted, run the following fail2ban-client status command to view the jail.

sudo fail2ban-client status nginx-noscripts

Example output:

Status for the jail: nginx-noscripts
 |- Filter
 | |- Currently failed: 0
 | |- Total failed: 0
 | - File list: /var/log/nginx/access.log - Actions
 |- Currently banned: 95
 |- Total banned: 107
 `- Banned IP list:

As the output above has shown, you have 95 IP addresses banned, with a total of 107 including active and inactive historical bans.

Next, to test a fail2ban filter, use the following example command.

Example:

fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf

The above will parse your log file and print out bans that will go live if you enable the filter. This should be done on filters that can cause false positives, such as web exploits.

Comments and Conclusion

In the tutorial, you have learned how to configure some additional jails to your fail2ban configuration for your Nginx server. Overall, you can achieve some excellent server-side security, filters can be adjusted and tweaked, and what was shown is just a scrape of what more can be done to suit specific situations and needs.

The only drawback with fail2ban is running clusters of servers, for now, this isn’t viable unless you use Cloudflare and block them at the reverse proxy, for example, but for single server operators, you can practically lock your server like fort Knox.

Subscribe
Notify of
17 Comments
Inline Feedbacks
View all comments
Guest
Wednesday, August 11, 2021 11:25 pm

Hi theгe! Would you mind іf I share your blog with my myspace
group? There’s a lot of folks that I think would really appreciate your
content. Please let me know. Thanks

michael
Guest
Sunday, November 28, 2021 7:41 am

Hallo,
habe bereits Fail2Ban laufen mit 13 Jails.
Habe jetzt Deinen nginx-nowordpress Jail in meine Jail.local hinzugefügt und auch die nginx-nowordpress erstellt.
Nach service fail2ban restart habe ich aber auf einmal nur noch 12 Jails, eigentlich müssten es aber 14 sein.

2 Jails die nicht in der Jail.local sind, sondern in xy.local und xyz.local sind also weg…

michael
Guest
Reply to  Joshua James
Monday, November 29, 2021 4:10 am

Hi,

I checked the no-wordpress Jail again. Removed “journalmatch = _SYSTEMD_UNIT=nginx.service + _COMM=nginx” from the filter. => After that one of my already existing jails does not work anymore. Disabled your no-wordpress jail => all my jails working again.

Tried your portscan-block jail incl. the filter. => Fail2ban does not restart anymore. Have to disable your jail again.

michael
Guest
Reply to  Joshua James
Monday, November 29, 2021 4:31 am

Tried your no-wordpress jail again, removed “journalmatch = _SYSTEMD_UNIT=nginx.service + _COMM=nginx” from the filter. => one of my already existing jails does not work anymore. Disabled your no-wordpress jail, all my jails working again.

Tried also portscan-block jail => fail2ban does not start anymore. Have to disable your jail again and everything is fine again

michael
Guest
Reply to  Joshua James
Monday, November 29, 2021 5:34 am

Thanks Joshua, I will prepare this the next days

michael
Guest
Reply to  Joshua James
Tuesday, November 30, 2021 5:39 am

I have sent you a mail with the files:) Many Thanks!

michael
Guest
Sunday, November 28, 2021 8:43 am

Wow, that was a fast reply, thanks.
I’m on Ubuntu 20.4 LTS with Fail2Ban V 1.57 (the latest you get with apt update).
No, I have 11 of 13 jails in jail.local. Only 2 Jails are in separate jails for whatever reason…

Thanks for checking

Michael

michael
Guest
Sunday, November 28, 2021 8:50 am

Sorry fail2ban-client -V show me 0.11.1

michael
Guest
Sunday, November 28, 2021 9:15 am

Yes with Nginx, I will try to change the syntax for the filter as proposed in your first quote…

But for now I have to go to bed:)

Thanks for the fast help so far:)

Techieferret
Guest
Wednesday, January 19, 2022 2:14 am

You got an error. In [nginx-no-x-spam], you’re using filter = nginx-nologin instead of nginx-no-x-spam.

adplus-dvertising
17
0
Would love your thoughts, please comment.x
()
x