Mitchell Krog created NGINX Ultimate Bad Bot & Referrer Blocker, a Linux Specialist focusing on security, and given his work, you can tell he is one of the good guys. NGINX Ultimate Bad Bot Blocker is, to date, one of the best open free source pieces of software you can deploy to your website free of charge with a continuous update of sometimes twice a day. The work from Mitchell and the community is outstanding.
NGINX Ultimate Bad Bot & Referrer Blocker is jam-packed with features, a run down below:
- Bad Bots Blocker
- Bad User-Agents Blocker
- Spam Referrer Blocker
- Adware Blocker
- Ransomware Blocker
- Clickjacking Blocker
- SEO Companies and BAD IP Blocker
- Nginx Rate Limiting Built-in anti-DDoS System
- WordPress Theme Detector Blocking
In the following tutorial, you will learn how to install Nginx Bad Bot Blocker on Ubuntu 20.04 LTS Focal Fossa using the command line terminal. The tutorial focuses the same as stated in the official documentation but with a more precise focus on the Focal Fossa distribution.
Table of Contents
First, before you begin, ensure all your system packages are up-to-date to avoid any conflicts during the installation and setup of the Nginx bad bot blocker.
sudo apt update && sudo apt upgrade -y
Install Required Packages
The following packages are required to install and run the software; use the following command to install or verify that they are present.
sudo apt install curl -y
If you are unsure, run the command regardless, as it will cause no harm to your system.
Install Nginx Bad Bot Blocker
The first step to installing the Bad Bot Blocker is downloading the handy installation script from the official Github. This script will handle the process automatically, and in most cases, little intervention is required.
Using the wget command, download the following.
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker
Next, navigate to the directory where the executable is located as follows.
Set the installation script to be executable using the following command.
sudo chmod +x install-ngxblocker
Now check to ensure the permission setting was successful before executing the script.
If done correctly, you should have x at the end of your permissions, along with install-ngxblocker being green.
Now, execute the script with the following command.
Note use: You can use the sudo command in front of the command if you have issues executing.
The required files have been downloaded to the required directories on your NGINX install from the repository.
Next, set NGINX Bad Bot Blocker & Referrer set up and update scripts and executable by the following commands:
sudo chmod +x /usr/local/sbin/setup-ngxblocker sudo chmod +x /usr/local/sbin/update-ngxblocker
Check chmod permissions, make sure it’s correct.
Install Nginx Bad Bot Blocker now with executing the setup script.
sudo ./setup-ngxblocker -x
The output shows that the script has been added to NGINX vhost configuration files (server blocks). Also, another significant part of this script install will automatically whitelist your IP address in the whitelist-ips.conf file.
Any further required IP addresses needing whitelisting can be added later. An example of what it may look like is below.
INFO: /etc/nginx/conf.d/* detected => /etc/nginx/nginx.conf inserting: include /etc/nginx/bots.d/blockbots.conf; => /etc/nginx/sites-available/example.com.vhost inserting: include /etc/nginx/bots.d/ddos.conf; => /etc/nginx/sites-available/example.com.vhost
Next, confirm that the bad bot blocker has been included in the nginx.conf configuration file using any text editor.
sudo nano /etc/nginx/nginx.conf
In the configuration file, the following should be added.
# Bad Bot Blocker include /etc/nginx/bots.d/ddos.conf; include /etc/nginx/bots.d/blockbots.conf;
Note that this can appear anywhere in the configuration file. The automatic installation and setup scripts do their best to add the required includes in the best spot. Still, it can be randomized sometimes, but you can adjust it manually in the nginx.conf file.
To finish the installation part, we will do the following commands to test and then restart the NGINX server:
sudo nginx -t
Example output if successful:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful
Proceed to restart the NGINX server:
sudo systemctl restart nginx
Congratulations, you have successfully installed Nginx Bad Bot Blocker on Ubuntu 20.04.
Create Cron Job for Automatic Updates
The sources of bad user agents, IP addresses, domains, and more are often updated twice daily. Hence, it is crucial to set an automatic job to keep checking and updating to protect against the latest threats. The best way to do this is using a cron job.
First, open crontab in your terminal:
sudo crontab -e
Add the following, which is set to 8 hours intervals which are highly recommended given the update script. It’s very lightweight.
00 */8 * * * sudo /usr/local/sbin/update-ngxblocker -e firstname.lastname@example.org
Note, if unsure about cron time settings, visit CronTab.Guru is an excellent calculator/test which you can copy/paste.
Configure Bad Bot Blocker
Nginx Bad Bot Blocker gives you the ability to customize itself in many different ways, such as adding your blacklists, whitelists, user agents, etc.
For example, modifying the following file /etc/nginx/bots.d/blacklist-ips.conf and adding a custom-made blacklist.
Another great benefit is making whitelists for yourself and whitelists specific Search Engine Crawlers to make 100% sure they are not blocked. You can do this by adding single IP addresses or subnets.
Below is the list of options and their paths you can customize by adding additional data. The most significant point is that updates do not affect these configuration files, so you will not lose the data you added when Nginx Bad Bot Blocker updates itself.
Conclusion and Comments
In the tutorial, you have learned how to install Nginx Bad Bot Blocker on Ubuntu 20.04 LTS server. Overall, this is great for servers with dedicated backend access and is very light and customizable, and if your site has a bot plague problem, this will surely help clean it up.