Cloudflare, one of the largest internet infrastructure companies, has introduced AI Labyrinth, a new tool to fight web-crawling bots.
The company on Saturday, March 22, 2025, revealed that AI Labyrinth is designed to prevent web-crawling bots from scraping sites for AI training data without permission.
In a blog post, Cloudflare stated that when it detects “inappropriate bot behavior,” the free, opt-in tool lures crawlers down a path of links leading to AI-generated decoy pages that “slow down, confuse, and waste the resources” of those acting in bad faith.
Related: Gemini Live expands Android support with Astra-powered camera, screen sharing
Cloudflare highlighted that it sees over 50 billion web crawler requests per day, and although it has tools for spotting and blocking the malicious ones, this often prompts attackers to switch tactics in “a never-ending arms race.”
Rather than block bots, AI Labyrinth fights back by making them process data that has nothing to do with a given website’s actual data.
Moreover, it functions as “a next-generation honeypot,” drawing in AI crawlers that keep following links to fake pages deeper, whereas a regular human being wouldn’t.
Website administrators can choose to use AI Labyrinth by navigating to the Bot Management section of their site’s Cloudflare dashboard’s settings and toggling it on.
According to Ars Technica, AI Labyrinth sounds similar to Nepenthes, a tool that’s designed to sideline crawlers for “months” in a hell of AI-generated junk data.
Related: WhatsApp announces new motion photo sharing feature in Android
