Topic
Web Infrastructure
3 stories related to this topic, newest first.
Semaforai17 hrs ago
Cloudflare Introduces Paywall Letting Sites Charge AI Crawlers for Content Access
Cloudflare introduced a system letting its customers set prices for AI crawlers accessing their sites. The move addresses rising automated traffic that now exceeds half of all web requests on its network.
Engadgettechnology5 days ago
Cloudflare to Block Mixed-Use Web Crawlers Serving AI Companies
Cloudflare will automatically block web crawlers that both index sites for search engines and train AI models. The change gives website owners more control over how their content is used by AI systems.
CoinDeskai62 days ago
Coinbase Engineer Proposes Micropayments for AI Agents at Consensus Miami
Erik Reppel, head of engineering at Coinbase Developer Platform and founder of the x402 payments protocol, told attendees at Consensus Miami 2026 that autonomous AI agents ignore online advertising and could upend the internet's ad-driven economy. He advocated using the HTTP 402…