Why can’t ChatGPT access my website?

What “I can’t access that site” usually means

When someone pastes your link or asks ChatGPT to look at your site, it opens the page with the ChatGPT-User agent. OpenAI says this agent acts on a user’s request, isn’t used for automatic crawling, and that robots.txt rules may not apply to it (OpenAI, “Overview of OpenAI crawlers”, read 15 Oct 2026). So for a pasted link, robots.txt is rarely the culprit. Something on the way answered instead of your page.

ChatGPT describes the failure in different words, and some failures don’t look like failures at all:

Run the AI crawler checker first: it requests your home page as ChatGPT-User and eleven other bots and shows which ones your CDN turns away. Then test the exact URL that failed.

Five tests for the failing URL

An example run, with the URL ChatGPT couldn’t open:

URL="https://yoursite.com/pricing"
UA="Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot"

# 1. Status line: anything but 200 is a block or an error
curl -s -o /dev/null -D - -A "$UA" "$URL" | head -n 1
# 2. Challenge page: a cf-mitigated header means Cloudflare served a check
curl -s -o /dev/null -D - -A "$UA" "$URL" | grep -i "cf-mitigated"
# 3. Login or preview wall: a redirect to a sign-in page
curl -s -o /dev/null -D - -A "$UA" "$URL" | grep -i "^location"
# 4. Words visible without JavaScript: dozens means an empty shell
curl -s -A "$UA" "$URL" | sed 's/<[^>]*>/ /g' | wc -w
# 5. Slow answers: more than a few seconds is a risk
time curl -s -o /dev/null -A "$UA" "$URL"

A good result: HTTP/2 200, no cf-mitigated header, no redirect to a login, hundreds of words, and an answer in about a second.

One limit: a borrowed user agent only reveals rules that look at the user agent. If your CDN verifies bots against OpenAI’s published addresses (openai.com/chatgpt-user.json), your curl is treated as a fake while the real bot may pass, or the reverse. Your CDN’s security events or server logs show what happened to real ChatGPT-User requests.

Fix it by cause

A prompt for your coding tool

For the rendering and redirect causes, hand your AI coding tool one measurable job. An example:

ChatGPT can't read https://yoursite.com/pricing. Make every public page
return its full text in the first HTML response, without JavaScript and
without a redirect to a login. Leave the app screens behind auth.
Accept when this prints HTTP/2 200 and the page's H1:
curl -s -D - -A "ChatGPT-User/1.0" https://yoursite.com/pricing | head -n 40

CDN settings live outside your repo, so change those in the dashboard and rerun the five tests.

Where Katman fits, and what it can’t do

Katman MCP runs inside your AI coding tool. katman_audit_site requests your live pages as ChatGPT-User, OAI-SearchBot and the other AI bots through your CDN, reports status codes, challenges and soft 404s, and counts the words each page shows without JavaScript. katman_scan_code tells you whether your framework renders on the server or only in the browser, and katman_prompt returns the rendering fix with an acceptance check. Katman Pro is $99 a month for up to 3 sites, with fix prompts and page briefs uncapped. Or have us do it: Growth writes 30 pages a month for you. Plans are on the pricing page.

It can’t change your Cloudflare or Vercel settings for you, and it doesn’t edit your site by itself: your coding tool makes the code changes. Once ChatGPT can open your pages, being found in search is the next step: How to show up in ChatGPT search.

Written by the Katman team, Rnify Limited. OpenAI, Cloudflare, Vercel and Vercel/MERJ pages checked 15 October 2026.