Amidst the rising of bots, Cloudflare has rolled out “Friendly Bots” – a bot validation service. With this feature, Cloudflare aims to accelerate and improvise manual bot validation.
Cloudflare Rolls Out Friendly Bots
Sharing the details via a recent post, Cloudflare explained how its new “Friendly Bots” feature will jazz up bot validation.
Briefly, Cloudflare conducts manual verification for bots submitted by users. To proceed with such requests, Cloudflare requires various details, mentioning which, the post reads,
We ask for some standard bits of information: your bot’s name, its public documentation, and its user agent (or regex).
Also, the service requires other information to go with its bot validation methods. These include validating IPs, after receiving details about the IP list that the bot would use, the rDNS (reverse DNS lookup via IP address), User Agent + ANS validation, and machine learning. The latter comes into play when the other methods fail the purpose. But machine learning validation requires substantial traffic to detect a “usable pattern” for bot profiling.
The process is tedious, and so, often reserved for larger bots. Thus, it becomes difficult for small-scale bots to pass the verification in time. As explained,
In the past, we’ve struggled to verify bots that did not crawl the web at a large scale. Why? Our system relies on a cache of verified traffic, ensuring that certain IPs or other data have widely shown good behavior on the Internet. This means that bots were sometimes difficult to verify if they did not make thousands of requests to Cloudflare. With Friendly Bots, we’ve eliminated that requirement, introducing a new, dynamic cache that optimizes for fun-sized projects.
Cloudflare has now launched “Friendly Bots” for those bots that lie between “good” and “bad”. It allows users to “instantly auto-validate” traffic by providing details.
Working And Benefits
Explaining how it works, Cloudflare explained that users can submit the required bot details via the dashboard. The service will then start running the validation mechanisms at a data center. Thus, it will facilitating a secure and scalable bot validation.
Alongside accelerating the process, the new feature will also let users know about any issues delaying the bot verification.
Cloudflare Radar To Show “Verified Bots”
Apart from the above-described update, Cloudflare has also announced displaying a list of “verified Bots with Cloudflare Radar. The list will show real-time updates.