Can Cloudflare CEO Matthew Prince save the web from AI?
TL;DR
The Verge's Decoder talks with Cloudflare CEO Matthew Prince about how AI is reshaping the web. Cloudflare measured in June that bots now make up more than half of all internet traffic, driven by AI crawlers scraping sites and a growing wave of AI agents acting for users. Sitting between websites and those agents, Cloudflare lets site owners block AI tools, allow them, or only admit crawlers willing to pay for access. The episode is part of a two-part series on the future of business.
Nauti's Take
For publishers this is a real opportunity: controlling which AI crawlers get in through Cloudflare gives them bargaining power and possibly revenue for their content for the first time. The risk is concentration, since a single infrastructure provider effectively helps decide which agents can see the web.
Site owners should review their bot settings now, and teams building agents should budget for pay-per-crawl access.
Sources