Mitchell Krog created NGINX Ultimate Bad Bot & Referrer Blocker, a Linux Specialist focusing on security, and given his work, you can tell he is one of the good guys. NGINX Ultimate Bad Bot Blocker is, to date, one of the best open free source pieces of software you can deploy to your website free of charge with a continuous update of sometimes twice a day. The work from Mitchell and the community is outstanding.
NGINX Ultimate Bad Bot & Referrer Blocker is jam-packed with features, a run down below:
- Bad Bots Blocker
- Bad User-Agents Blocker
- Spam Referrer Blocker
- Adware Blocker
- Ransomware Blocker
- Clickjacking Blocker
- SEO Companies and BAD IP Blocker
- Nginx Rate Limiting Built-in anti-DDoS System
- WordPress Theme Detector Blocking
In the following tutorial, you will learn how to install Nginx Bad Bot Blocker on Ubuntu 20.04 LTS Focal Fossa.
- Recommended OS: Ubuntu 20.04 LTS or higher.
- User account: A user account with sudo or root access.
- Required Packages: wget
Update Operating System
Update your Ubuntu operating system to make sure all existing packages are up to date:
sudo apt update && sudo apt upgrade -y
The tutorial will be using the sudo command and assuming you have sudo status.
To verify sudo status on your account:
Example output showing sudo status:
[joshua@ubuntu ~]$ sudo whoami root
To set up an existing or new sudo account, visit our tutorial on How to Add a User to Sudoers on Ubuntu.
To use the root account, use the following command with the root password to log in.
Install CURL Package
To successfully use Nginx Bad Bot Blocker after installing, you need to make sure you have curl installed on your system.
Run the following command to install.
sudo apt install curl -y
If you are unsure, run the command regardless, as it will cause no harm to your system.
Install Nginx Bad Bot Blocker
The first step to installing the Bad Bot Blocker is downloading the handy installation script from the official Github. This script will handle the process automatically, and in most cases, little intervention afterward is required.
Using the wget command, download the following.
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker
Next, navigate to the directory and set the installation script to be executable.
cd /usr/local/sbin sudo chmod +x install-ngxblocker
Now check to make sure the permission setting was successful before executing the script.
If done correctly, you should have x at the end of your permissions, along with install-ngxblocker being green.
Now, execute the script with the following command:
sudo ./install-ngxblocker -x
The required files have been downloaded to the required directories on your NGINX install from the repository.
Next, set NGINX Bad Bot Blocker & Referrer set up and update scripts and executable by the following commands:
sudo chmod +x /usr/local/sbin/setup-ngxblocker sudo chmod +x /usr/local/sbin/update-ngxblocker
Check chmod permissions, make sure it’s correct.
Install Nginx Bad Bot Blocker now with executing the setup script.
sudo ./setup-ngxblocker -x
The output shows that the script has been added to NGINX vhost configuration files (server blocks). Also, another significant part of this script install will automatically whitelist your IP address in the whitelist-ips.conf file.
Any further required IP addresses needing whitelisting can be added later. Example what it may look like below:
INFO: /etc/nginx/conf.d/* detected => /etc/nginx/nginx.conf inserting: include /etc/nginx/bots.d/blockbots.conf; => /etc/nginx/sites-available/example.com.vhost inserting: include /etc/nginx/bots.d/ddos.conf; => /etc/nginx/sites-available/example.com.vhost
Next, confirm that the bad bot blocker has been included in the nginx.conf configuration file using any text editor.
sudo nano /etc/nginx/nginx.conf
In the configuration file, the following should be added.
# Bad Bot Blocker include /etc/nginx/bots.d/ddos.conf; include /etc/nginx/bots.d/blockbots.conf;
Note, this can appear anywhere in the configuration file, the automatic installation and set up scripts do their best to add the required includes in the best spot. Still, it can be randomized sometimes, but you can adjust it manually in the nginx.conf file.
To finish the installation part, we will do the following commands to test and then restart the NGINX server:
sudo nginx -t
Example output if successful:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful
Proceed to restart the NGINX server:
sudo systemctl restart nginx
Congratulations, you have successfully installed Nginx Bad Bot Blocker on Ubuntu 20.04.
How to set Cron Job for Automatic Updates
The sources of bad user-agents, IP addresses, domains, and more are often updated twice daily. Hence, it is crucial to set an automatic job to keep checking and updating to protect against the latest threats. The best way to do this is using a cron job.
First, open crontab in your terminal:
sudo crontab -e
Add the following, which is set to 8 hours intervals which are highly recommended given the update script. It’s very lightweight.
00 */8 * * * sudo /usr/local/sbin/update-ngxblocker -e email@example.com
Note, if unsure about cron time settings, visit CronTab.Guru is an excellent calculator/test which you can copy/paste.
How to Configure Bad Bot Blocker
Nginx Bad Bot Blocker gives you the ability to highly customize itself in many different ways, such as adding your blacklists, whitelists, user-agents, and so on.
For example, modifying the following file /etc/nginx/bots.d/blacklist-ips.conf and adding a custom-made blacklist.
Another great benefit is making whitelists for yourself and whitelist specific Search Engine Crawlers to make 100% sure they are not blocked. You can do this by adding single IP addresses or subnets.
Below is the list of options and their paths you can customize by adding additional data. The most significant point is that updates do not affect these configuration files, so you will not lose the data you added when Nginx Bad Bot Blocker updates itself.
Conclusion and Comments
In the tutorial, you have learned how to install Nginx Bad Bot Blocker on Ubuntu 20.04 LTS server. Overall, this is great for servers with dedicated backend access and is very light and customizable, and if your site has a bot plague problem, this will surely help clean it up.