Cloudflare AI Crawl Control review (2026): free crawler controls, no citations
Cloudflare AI Crawl Control is free on every Cloudflare plan and shows which AI bots hit which paths. It cannot show citations. We compare it with Promptwatch Agent Analytics at $245/mo.
Cloudflare AI Crawl Control is a feature of the Cloudflare dashboard, formerly called AI Audit. If your site runs behind Cloudflare, it is already there. It shows which AI crawlers request your pages, lets you allow or block each one, and in a closed beta lets you charge them per request.
It is free on every Cloudflare plan. It does not run prompts or report citations. That one sentence decides where it sits on our board: first place for crawler access control at zero cost, and no place at all for AI visibility measurement.
What you get per plan
Read on developers.cloudflare.com/ai-crawl-control, cloudflare.com/ai-crawl-control and cloudflare.com/plans on 20 October 2026.
| Cloudflare plan | Price | AI Crawl Control |
|---|---|---|
| Free | $0 | Dashboard, allow and block per crawler, user-agent detection |
| Pro | $20 | Adds referral metrics and custom block responses |
| Business | $200/mo annual, $250 monthly | Paid-plan features |
| Contract | Custom | Enterprise; pay per crawl beta via account executive |
The docs say it works on all plans with zero configuration. Referral metrics, custom block responses and detection beyond user-agent strings need a paid plan.
What the dashboard shows
The Overview tab gives total and allowed requests, unsuccessful requests, the most common status code, the most popular path, and crawlers grouped by operator: OpenAI, Anthropic, Google, Microsoft, Meta, ByteDance and others.
The Metrics tab charts requests and data transfer by crawler, category, operator or host. It shows status codes from 2xx to 5xx and compares requested and served content types (Accept and Content-Type headers). A most-popular-paths table lists requests and referrals per path, with grouping such as /blog/*. Robots.txt health and violations are tracked too.
Data exports to CSV and is queryable through the GraphQL Analytics API.
What it does well
Zero cost, zero setup. For a Cloudflare site there is nothing to install.
Status codes are the practical win. If GPTBot or ClaudeBot gets a 403 on your pricing page, this is where you find out.
Measurement and control share one screen. Blocking a crawler writes a Cloudflare WAF custom rule you can extend. Paid plans can return a custom 403 or 402 with a message.
Pay per crawl is an interesting option for publishers, though still a closed beta.
What to check before relying on it
Coverage stops at Cloudflare. Traffic that bypasses Cloudflare is invisible.
Free-plan detection uses user-agent strings only, so only well-known self-identifying bots are detected. Bot Management detection IDs need an upgrade.
Referral data is paid-plan only.
Cloudflare's own pages disagree on pay per crawl's status: the overview calls it a private beta, the crawler page a closed beta.
Where it falls short of a full platform
AI Crawl Control answers "what did the bots fetch?" It does not answer "did that fetch turn into an answer that named us?" or "what did that answer send us?" There is no prompt tracking, no citation data and no conversion tracking.
A crawler log without the citation side is half a diagnosis. You can see how often ClaudeBot fetched your pricing page last week. You cannot see whether Claude mentioned you even once.
AI Crawl Control vs Promptwatch Agent Analytics
This is the head-to-head most Cloudflare users should run.
| AI Crawl Control | Promptwatch Agent Analytics | |
|---|---|---|
| Price | Free on any Cloudflare plan | Included from Professional, $245/mo |
| Log sources | Cloudflare only | Cloudflare, AWS CloudFront, Fastly, Vercel, Netlify, Akamai, Google Cloud CDN, custom HTTP |
| Bot requests and errors | Yes | Yes, with error tracking |
| Block or allow bots | Yes, via WAF rule | Not published |
| Crawl-to-citation path | No | Yes |
| Prompt tracking | No | Yes, daily across nine LLMs plus AI Overviews and AI Mode |
| Citation analytics | No | Page, domain, Reddit, YouTube, offsite |
| AI referrals and conversions | Referrals on paid plans | Visitor analytics with conversions |
Promptwatch accepts Cloudflare as a log source. That means you do not have to choose. Keep AI Crawl Control for access control, the allow and block switches and the WAF rules. Send the same logs to Promptwatch to see what the crawling earned.
Promptwatch Professional costs $245 a month and includes 25M crawler logs, 150 prompts, 18,000 responses, 1M visitor events, 15 AEO articles and 2 seats. Agent Analytics covers ChatGPTBot, ClaudeBot, PerplexityBot, GoogleOther and the Meta AI crawler in real time. The crawl-to-citation path shows which fetched pages ended up cited. Visitor analytics, by script or GTM template, ties AI-referred sessions to conversions. Business at $579 a month raises crawler logs to 100M. Agency plans include crawler logs from Kick-off at $199 a month.
Note the naming: Cloudflare's $20 plan is called Pro. Promptwatch's crawler-log tier is called Professional.
Verdict
Rank: best free crawler control, not a visibility tool. If you are on Cloudflare, turn it on and check your status codes today. When you need to know whether the crawling produced citations and revenue, connect the same Cloudflare logs to Promptwatch Professional.
FAQ
Is Cloudflare AI Crawl Control free?
Yes. It is available on all Cloudflare plans, including Free, with no configuration. Referral metrics and custom block responses need a paid plan.
Is AI Crawl Control the same as Cloudflare AI Audit?
Yes. Cloudflare's docs describe it as formerly AI Audit.
Does AI Crawl Control show whether ChatGPT cites my site?
No. It shows crawler requests, status codes, paths and, on paid plans, referrals. For citations joined to crawler logs, see promptwatch.com.