‘No real human would go four links deep into a maze of AI-generated nonsense’: Cloudflare’s AI Labyrinth uses decoy pages to trap web-crawling bots and feed them slop ‘as a defensive weapon’

The web is plagued by bots. That’s nothing new of course, but now we’re in the midst of our much-loved AI revolution (you do love it, right?) many websites are continually crawled by bots aiming to scrape them of their precious data to train AI content. Cloudflare thinks it might have the solution, however, as its newly-announced AI Labyrinth tool aims to take the fight to the nefarious bots by “using generative AI as a defensive weapon.”

Cloudflare says that AI crawlers generate more than 50 billion requests to its network every day—and while tools exist to block them, these methods can alert attackers that they’ve been noticed, causing them to shift approach (via The Verge).

AI Labyrinth, however, links detected bots to a series of AI-generated pages that are convincing enough to draw them in, but contain no useful information.

Why? Well, because they were generated by AI, of course. Essentially this creates an ouroboros of AI slop in, AI slop out, to the point where the bot wastes precious time and resources churning through useless content instead of scraping something created by an actual human being.

“As an added benefit, AI Labyrinth also acts as a next-generation honeypot. No real human would go four links deep into a maze of AI-generated nonsense,” says Cloudflare.

“Any visitor that does is very likely to be a bot, so this gives us a brand-new tool to identify and fingerprint bad bots, which we add to our list of known bad actors.”

It’s bots, bots all the way down. The AI-generated “poisoned” content is integrated in the form of hidden links on existing pages, meaning a human is unlikely to find them but a web crawler will.

To double down on the human-first angle, Cloudflare also says these links will only be added to pages viewed by suspected AI scrapers, so the rest of us shouldn’t even notice it’s working away in the background, fighting evil bots like some sort of Batman-esque caped crusader.

Enabling the tool is a simple matter of ticking a checkbox in Cloudflare’s settings page, and ta-da, off to work the AI Labyrinth goes. Cloudflare says this is merely the first iteration of this particular tech and encourages its users to opt in to the system so it can be refined in future.

I do have a question, though. Given AI is now, let’s face it, bloody everywhere, are we really sure that making its training process worse isn’t going to have longer-term effects? Far be it from me to take the side of the nefarious crawlers, but I wonder if this will simply lead to a glut of even-more-terrible AI models in future if their training data is hamstrung from the start.

Ah, screw it, I’ve talked myself out of my own counter argument. Something needs to be done about relentless permission-free data scraping from genuine human endeavour, and I salute the clever thinking behind this particular defensive tool.

If I could make one suggestion, however, could we perhaps add a Minotaur? All good labyrinths need one, and then I can write something like “Cloudflare has grabbed the bull by the horns and…”

Fill in your own headline there. Or, y’know, get an AI to do it for you. Kidding, kidding. I probably shouldn’t be feeding the AI any more of my terrible jokes anyway.