There are lots of different types of bots bouncing around the Internet, some benign, some annoying and many others malicious. They all take up resources, though, and Cloudflare is implementing a new system that will help enterprises prevent malicious bots from causing trouble on their networks and consuming valuable computational resources.
The new feature is called “bot fight mode” and Cloudflare enterprise customers can flip a switch in the dashboard to enable it now. When it’s turned on, the system uses Cloudflare’s bot-detection technology to identify malicious bots and block their activity. The system relies on a variety of different technologies that Cloudflare has developed over the years, including a method for identifying legitimate bots (such as search engine crawlers), custom rules for identifying malicious bots, and the company’s Bot Activity Detector that uses past traffic patterns to find malicious bots.
“In addition, Gatebot, our DDoS mitigation system, fingerprints DDoS bots and blocks their traffic at the packet level. Beyond Gatebot, customers also have access to our Firewall Rules where they can write granular rules to block very specific attack types,” said John Graham-Cumming, CTO at Cloudflare.
“Another model allows us to determine whether an IP address belongs to a VPN endpoint, a home broadband subscriber, a company using NAT or a hosting or cloud provider. It’s this last group that 'Bot Cleanup' targets.”
Identifying malicious bots is only part of the challenge, however. The next step is doing something with them. Cloudflare’s new system will take a number of different actions once it identifies a malicious bot, including sending them into a so-called tarpit that forces them to use more compute power, which wastes the bot operator’s time and money. The idea is to increase the cost to the point that it is no longer worth the operator’s time to launch the attacks.
“The cost of launching a bot attack consists of the expense of CPU time that powers the attack. If our models show that the traffic is coming from a bot, and it’s on a hosting or a cloud provider, we’ll deploy CPU intensive code to make the bot writer expend more CPU and slow them down. By forcing the attacker to use more CPU, we increase their costs during an attack and deter future ones,” Graham-Cumming said.
“This is one of the many so-called ‘tarpitting’ techniques we're now deploying across our network to change the economics of running a malicious bot. Every minute we tie malicious bots up is a minute they're not harming the Internet as a whole. This means we aren't just protecting our customers but everyone online currently terrorized by malicious bots.”
In addition to sending the malicious bots into a blind alley, if the bot is hosted on the infrastructure of one of the Cloudflare Bandwidth Alliance members, Cloudflare will share the IP address of the bot with that provider in order to take it offline. That alliance includes some of the larger cloud providers in the world, such as Microsoft Azure and IBM Cloud. The combination of technological and policy defenses has proven to be one of the more effective methods for defeating a number of online threats, including DDoS attacks and some types of botnets.
The technical part of this approach--forcing the bots to use more CPU power to waste their resources--also means that those bots will be consuming more electricity. Cloudflare decided to address that problem by pledging to plant trees to offset the increased carbon dioxide production.