Cloudflare Launches AI-Powered Feature to Deceive Web Crawlers

Cloudflare has launched a new feature using AI to generate random pages to deceive web crawling bots. Instead of blocking bots, Cloudflare lures them into a "maze" of realistic but irrelevant pages, thereby wasting the computational resources of crawlers. This approach aims to prevent bot operators from being alerted to their detection. The generated pages contain neutral scientific information to avoid spreading disinformation. According to Cloudflare, AI crawlers generate more than 50 billion requests daily, accounting for nearly 1% of total web traffic. This method could lead to an arms race between honeypots and crawlers, increasing energy consumption.