You might have received the following notice in your Really Simple SSL Dashboard about suspected bots triggering large numbers of “404 Not Found” errors on your site:
This article explains why the plugin has built-in detection for large amounts of 404 pages being triggered, and the reasons why these are unlikely to be triggered by legitimate (human) visitors; but rather by bots.
Finally, we will cover how to configure the Firewall in Really Simple SSL Pro to block bots that send large amounts of 404 requests to your website.
Why does Really Simple SSL detect excessive amounts of 404 pages being triggered?
When a legitimate visitor browses a website, they typically follow links provided on the site or through search engines. While legitimate users could definitely encounter an occasional “404 Not Found” error due to visiting an outdated or incorrect link, they are unlikely to run into excessive amounts of 404 pages, especially within a short timeframe (e.g. 2-10 seconds).
On the contrary, bots and automated scanning tools tend to generate high volumes of 404 errors when they scan websites for vulnerabilities. These tools work by trying to access several pages and files on a site, including pages that do not exist on the website, whereby each failed attempt would generate a 404 Not Found error. This is done in an attempt to discover vulnerabilities in the site’s security which can potentially be exploited.
This is why Really Simple SSL detects excessive amounts of 404 pages being triggered, as it is a good indication that a malicious actor is looking for vulnerable parts of your website. Ideally, we would stop those bots from searching our sites, saving some server resources for legitimate users as well.
How to block users that trigger excessive amounts of 404 pages on a site?
The “404 Blocking” firewall rule in Really Simple SSL Pro (SSL & Security -> Settings -> Firewall -> Rules) can pro-actively block IP addresses that exceed the acceptable amount of 404 Not Found pages within a certain timeframe.
The Threshold setting determines the acceptable amount of 404 errors (for example: 10 errors) that can be triggered within a certain timeframe (for example: 10 seconds). If this amount of 404 pages is exceeded, the offending IP address will be blocked for the duration as selected under Lockout Duration.
If you are concerned about locking out legitimate visitors (who trigger large amounts of 404 errors by accident), you can additionally enable the “Trigger Captcha on Lockout” setting. This allows legitimate users who have accidentally been blocked to unblock their IP address by completing a CAPTCHA.
