Is my website AI-ready?

What “AI-ready” should mean

Different tools mean different things by it. Cloudflare’s “Is Your Site Agent-Ready?” scanner checks emerging standards for AI agents in five groups: discoverability (robots.txt, sitemap, link headers), Markdown content negotiation, bot access rules such as Content Signals and Web Bot Auth, protocol discovery (MCP Server Card, Agent Skills, OAuth) and agentic commerce (isitagentready.com, read 15 Oct 2026). That’s useful if you want agents to sign in, call your API or buy from you.

It doesn’t tell you whether ChatGPT, Claude or Perplexity can read your page text and quote it correctly, which is what most small sites want first. For them, ready means three things in this order: bots can read you, assistants can quote you, and then the extras. Start with the AI crawler checker, which requests your home page as GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and seven other bots.

The twelve checks

Group 1: can AI bots read your pages?

Any fail here outranks everything below. Checks 1, 3 and 4 have longer explanations on My vibe-coded site is invisible to ChatGPT, and failures on a specific URL on ChatGPT can’t access my website.

Group 2: can assistants quote you correctly?

These four decide whether an assistant can lift a correct sentence about you. Check 8 fails more often than people expect: our own llms.txt once contradicted a page, with two prices for one feature.

Group 3: extras, useful but not decisive

More on llms.txt: What is llms.txt?, or generate one with the llms.txt generator.

Example: run the checks in one go

Groups 1 and 3 can be scripted; group 2 needs your eyes.

SITE="https://yoursite.com"
for b in "OAI-SearchBot/1.4" "ChatGPT-User/1.0" "ClaudeBot/1.0" "PerplexityBot/1.0"; do
  printf "%-20s " "$b"
  curl -s -o /dev/null -D - -A "Mozilla/5.0 (compatible; $b)" "$SITE/" | head -n 1
done
echo "robots rules:"; curl -s "$SITE/robots.txt" | grep -i -A2 "searchbot\|perplexitybot"
echo "words:  $(curl -s "$SITE/" | sed 's/<[^>]*>/ /g' | wc -w)"
echo "404:    $(curl -s -o /dev/null -D - "$SITE/no-such-page-4821" | head -n 1)"
echo "urls:   $(curl -s "$SITE/sitemap.xml" | grep -c '<loc>')"
echo "llms:   $(curl -s -o /dev/null -D - "$SITE/llms.txt" | head -n 1)"
echo "jsonld: $(curl -s "$SITE/" | grep -c 'application/ld+json')"

Remember that a borrowed user agent only reveals user-agent rules; CDNs that verify bots by IP need their own logs checked.

What to fix first

1. Any fail in checks 1 and 2: nothing else counts until bots get in.
2. Check 3: if they get in and see no text, nothing else counts either.
3. Checks 4 and 5, then 8 and 9: they decide which pages and facts are real.
4. Checks 6 and 7: the pages and sentences assistants actually quote.
5. Checks 10 to 12, when the rest pass.

Passing all twelve makes you readable and quotable. It doesn’t make you recommended: that also needs pages for buyer situations and mentions on other sites, covered on Why doesn’t ChatGPT recommend my brand?.

Where Katman fits, and what it can’t do

Katman MCP runs inside your AI coding tool. katman_audit_site runs groups 1 and 3 on every page of your live site, including the bot test through your CDN, and reads each page’s first paragraph and JSON-LD. katman_lint_content checks group 2 against your research file, and katman_generate writes robots.txt, llms.txt and Organization JSON-LD from one source of facts, so they can’t drift apart. Katman Pro is $99 a month for up to 3 sites, with fix prompts and page briefs uncapped. Or have us do it: Growth writes 30 pages a month for you. Plans are on the pricing page.

It doesn’t change your CDN settings or edit your site by itself; your coding tool makes the code changes. And nobody can promise an assistant will recommend you, however ready the site is.

Written by the Katman team, Rnify Limited. Cloudflare and Google pages checked 15 October 2026.