Install/Enable Secure NGINX with Custom Fail2ban Filters

Advertisement

Fail2Ban is a great security measure to deploy for your web application server. It comes with features, default filters, and actions that can immediately impact banning bad web bots, draining your system resources, and stopping attacks, which is the most crucial part of any website.

However, most people system admins and website owners are looking for sometimes a bit more extra than what fail2ban has to offer. In the following tutorial, you will learn how to create and use custom filters on your Nginx server, which can be fine-tuned to suit your needs and expanded later.

Make sure to test and use with caution any new fail2ban filters. Note that settings will need to be adjusted to your needs.

Add New Fail2Ban Jails

The tutorial assumes you have Fail2ban installed and are familiar with configuring jails and settings. Below are some additional jails you will need to tweak to your liking if you are using further ban actions like Cloudflare or reporting to AbuseIPDB.org etc.

Add the following jails you wish to use in your /fail2ban/jail.local file.

Advertisement

sudo nano /etc/fail2ban/jail.local

Now, copy and paste which jails you prefer to use. Remember to see the next section for the actual filter.

[nginx-403]
 enabled = true
 port     = http,https
 filter = nginx-403
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 4

[nginx-404]
 enabled = true
 port     = http,https
 filter = nginx-404
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 30 # <--- monitor and adjust, all servers are different.

[nginx-noagent]
 enabled = true
 port     = http,https
 filter = nginx-noagent
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[nginx-noauth]
 enabled = true
 filter = nginx-noauth
 action = iptables-allports
 logpath = %(nginx_error_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

[nginx-no-x-spam]
 enabled = true
 filter = nginx-no-x-spam
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

[nginx-noscript]
 enabled = true
 action = iptables-allports
 filter = nginx-noscript
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[nginx-noproxy]
 enabled = true
 action = iptables-allports
 filter = nginx-noproxy
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 0

[nginx-nowordpress]
 enabled = true
 action = iptables-allports
 filter = nginx-nowordpress
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 3

[portscan-block]
 enabled = true
 action = iptables-allports
 filter = portscan-block
 logpath = /var/log/ufw.log  <--- this has to be direct to your UFW logs.
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 5

Create new Fail2Ban Filters

Next, you need to make a new filter file for each of these filters and put .conf at the end of the file.

Advertisement

Note that the created file is something you need to do yourself.

create file /location/fail2ban/filter.d/nginx-403.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 403

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-404.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 404

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noagent.conf

 [Definition]

 failregex = ^<HOST> -.*"-" "-"$
                   ^<HOST> -.*"-" "curl.*"$

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noauth.conf

 [Definition]

 failregex = no user/password was provided for basic authentication.client:              user . was not found in.client:              user . password mismatch.*client: 

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-no-x-spam.conf

 [Definition]

 failregex = {"log":"<HOST> .* ".*\\x.*" .*$

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noscript.conf

 [Definition]

 failregex =     ^<HOST> -.*GET.*(\.asp|\.exe|\.pl|\.cgi|\.scgi)

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-noproxy.conf

 [Definition]

 failregex = ^<HOST> -.*GET http.*
                   ^<HOST> -.*CONNECT.*

 ignoreregex =

create file /location/fail2ban/filter.d/nginx-nowordpress.conf

 [Definition]
            
 failregex = ^<HOST> -.*"(GET|POST|HEAD) /+(?i)(wp(-|/)|xmlrpc.php|\?author=1)
                   ^<HOST> -.* "(GET|POST|HEAD|PROPFIND) /+(?i)    (a2billing|admin|apache|axis|blog|cfide|cgi|cms|config|etc|.git|hnap|inc|jenkins|jmx-|joomla|lib|linuxsucks|msd|muieblackcat|mysql|myadmin|n0w|owa-autodiscover|pbxip|pma|recordings|sap|sdk|script|service|shell|sqlite|vmskdl44rededd|vtigercrm|w00tw00t|webdav|websql|wordpress|xampp|xxbb)
             ^<HOST> -.* "(GET|POST|HEAD) /[^"]+.(asp|cgi|exe|jsp|mvc|pl)( |\?)
             ^<HOST> -.*(?i)(/bash|burger-imperia|changelog|hundejo|hvd-store|jorgee|masscan|pizza-imperia|pizza-tycoon|servlet|testproxy|uploadify)

 ignoreregex =

 journalmatch = _SYSTEMD_UNIT=nginx.service + _COMM=nginx

create file /location/fail2ban/filter.d/portscan-block.conf

[Definition]

failregex = .*\[UFW BLOCK\] IN=.* SRC=<HOST>

ignoreregex = SRC=(10.|172.1[6-9].|172.2[0-9].|172.3[0-1].|192.168.|fe\w:). DST=(static.ip.address.here|224.0.0.). PROTO=(2|UDP)(\s+|.* DPT=(1900|3702|5353|5355) LEN=\d*\s+)$

WebExploits Filter

A top-rated fail2ban filter is blocking URL malicious bot scanners. However, this filter is not recommended for users new to Linux servers, and fail2ban especially does not have the time to monitor and adjust. The biggest issue with this filter is that it can be some real users, but mostly it will ban SEO good bots that may accidentally scan the same trap.

You can download several community-made and updated lists, but the best way is to monitor your logs and add your own slowly when you notice scans. Over time you will amass an extensive list custom-made and will work with zero positives.

First, add the jail for the web exploits script.

Advertisement

Example:

[webexploits]
enabled   = false
logpath = %(nginx_access_log)s
filter = webexploits
port    = http,https
bantime = 1440m
findtime = 1440m
maxretry = 0

Next, create the filter, but this is where it gets interesting.

Example:

create file /location/fail2ban/filter.d/webexploits.conf

[Definition]

failregex = 
            ^<HOST> -.*(GET|POST|HEAD).*(/data/admin/)
            ^<HOST> -.*(GET|POST|HEAD).*(/dnt-policy.txt)
            ^<HOST> -.*(GET|POST|HEAD).*(/gpc.json)
            ^<HOST> -.*(GET|POST|HEAD).*(/wlwmanifest.xml)
            ^<HOST> -.*(GET|POST|HEAD).*(/adminer)
            ^<HOST> -.*(GET|POST|HEAD).*(/_adminer)
            ^<HOST> -.*(GET|POST|HEAD).*(/user/login)
            ^<HOST> -.*(GET|POST|HEAD).*(/.env)

ignoreregex =

As above, those are just some typical examples. Then add your own, and modify the filter in the following format.

Advertisement
            ^<HOST> -.*(GET|POST|HEAD).*(/URL-TO-ADD-HERE)

From the example, the URL-TO-ADD-HERE is the extension where you see malicious users or bots scanning, such as WordPress XMLRPC.

            ^<HOST> -.*(GET|POST|HEAD).*(/xmlrpc.php)

You can also modify the GET|POST|HEAD options, but you want to ban anyone touching those URLs, which shouldn’t need to be changed.

Advertisement

Some example lists are here on this Github.

DO NOT JUST RANDOMLY COPY AND PASTE.

This requires testing, and I suggest adding your own from your logs. This can seriously backfire if you do not take the time and dedication to correct this. However, it can reduce bots on huge websites wasting resources scanning multiple URLs.

Make sure to test your filters as below will cover.

Nginx Secure Pages & Honeypot

Another interesting feature you can do with Nginx is making a 444 filter, then making geo locations to, for example, hide pages such as login that no good bot or the real user should be touching.

First, add the jail.

Advertisement
Advertisement

Example:

[nginx-444]
 enabled = true
 port     = http,https
 filter = nginx-444
 action = iptables-allports
 logpath = %(nginx_access_log)s
 bantime = 1440m # 1 day
 findtime = 1440m # 1 day
 maxretry = 1

Next, add the jail this time, making it an instant ban. I would advise adding the page to your robots.txt as an exclusion so that good SEO bots like Google will not touch it. You could make fake pages and exclude them also in robots.txt as honeypots. This example works for actual pages or honeypots.

create file /location/fail2ban/filter.d/nginx-444.conf

 [Definition]

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*" 444

 ignoreregex =

Next, create a file and whitelist your IP address or the IP ranges you want to be excluded that can reach the page.

Example location:

sudo nano /etc/nginx/whitelist.txt

An example of adding an IP address is below. Take note of the false at the end.

IP Address false;

or another example

195.168.50.1 false;

Once you have added your IP address in your nginx.conf file, add the following.

Advertisement

geo $remote_addr $exclude_trafic {
 default true;
include /etc/nginx/whitelist.txt;
}

Notice exclude all traffic is defaulting true.

Now, in your server block, add the following file.

Advertisement
if ( $allowed_trafic = 'true'){
    return 444;
}

FOR PHP such as WordPress locking off wp-login.php example.

  location ~ /wp-login.php {
if ( $exclude_trafic = 'true'){
    return 444;
} 
 fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/run/php/php8.1-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param SCRIPT_NAME $fastcgi_script_name;
    include fastcgi_params;
    include snippets/fastcgi-php.conf;
    fastcgi_intercept_errors on;
}

Note that this may need tinkering depending on your PHP version and distribution. Still, the example defaults to exclude all traffic from that page unless you are in the whitelist with false.

This is mostly not needed if you are behind Cloudflare or similar services, but this can come in handy more for honeypots on fake pages, be sure to add in your HTML links to it so it will appear to bot scrapers only and exclude it in your robots.txt so real bots that abide by your rules won’t touch it (malicious bots don’t obey robots.txt).

Test New Fail2ban Jails/Filters

To finish it off, restart fail2ban in your operating system via its command.

Advertisement

For most Linux distributions, the following command should work if running systemd.

sudo systemctl restart fail2ban

Other non-systemd distributions or other systems restart your system or fail2ban service.

Once fail2ban has been restarted, run the following fail2ban-client status command to view the jail.

sudo fail2ban-client status nginx-noscripts

Example output:

Advertisement
Status for the jail: nginx-noscripts
 |- Filter
 | |- Currently failed: 0
 | |- Total failed: 0
 | - File list: /var/log/nginx/access.log - Actions
 |- Currently banned: 95
 |- Total banned: 107
 `- Banned IP list:

As the output above has shown, you have 95 IP addresses banned, with 107 including active and inactive historical bans.

Next, to test a fail2ban filter, use the following example command.

Advertisement

Example:

fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf

The above will parse your log file and print out bans that will go live if you enable the filter. This should be done on filters that can cause false positives, such as web exploits.

Comments and Conclusion

In the tutorial, you have learned how to configure some additional jails to your fail2ban configuration for your Nginx server. Overall, you can achieve some excellent server-side security, filters can be adjusted and tweaked, and what was shown is just a scrape of what more can be done to suit specific situations and needs.

The only drawback with fail2ban is running clusters of servers, for now, this isn’t viable unless you use Cloudflare and block them at the reverse proxy, for example, but for single server operators, you can practically lock your server like fort Knox.

Share on: