Anthropic's Web Crawl-to-Referral Ratio Fell to 2,800-to-1 in Early July, Still Highest Among Peers

Anthropic's automated bots crawled webpages 2,800 times for every one referral returned during the week of July 1 through July 7, 2026, according to Cloudflare data reported by Business Insider. The same period placed OpenAI second in the crawl-to-refer ratio, followed by Perplexity, Microsoft and Google. DuckDuckGo recorded three scrapes for every referral.

Anthropic's ratio marked an improvement from roughly 8,800-to-1 in early April 2026. During the first seven days of May the figure reached 24,700-to-1. Business Insider reported that Anthropic has disputed Cloudflare's methodology, stating it could not verify the calculations and that new search features are increasing referrals.