katman / LLM visibility test

DATED TEST · 2026-09-02

Do LLMs know we exist?

We asked six major models six buyer-intent questions about our own category — once closed-book (training data only) and once with web access — and recorded every answer. This is the same observation protocol we sell, run on ourselves, published raw.

Problem-based test (v2) — the honest baseline

We ran the v2 protocol on ourselves: diagnosed the problems katman solves from its own pages, generated the queries a real buyer would type (brand name excluded) and asked 6 models all of them. Result: 0 brand hits in 108 answers. The problem space is owned by Profound, Otterly.ai and Peec AI — every one of them lives in directories and listicles we haven't entered yet. This is our own medicine, and it is exactly what the 30-day plan targets: external validation, not more pages.

Share of voice on our own problem queries

Per-query results

What the data says

The brand query (katman.pro) is known by most models — including closed-book. For a site that launched weeks ago, that means crawlers and mentions are being indexed fast.

Category queries are the opposite: with one exception (Llama 3.3 listing us among AEO tools with web access), we are absent from the answers. That is not a bug — it is the third condition doing its job. Models recommend what third parties validate, and our external footprint is days old. The gap is exactly what the audit's job cards target: directories, community mentions, citable comparison pages.

Method: one request per cell via OpenRouter, temperature 0, max 500 tokens. "Mention" = the response contains katman / katman.pro. Web access via OpenRouter's online plugin. Failed calls (provider rate limits) are shown as err, not counted as absence. This page will be re-run and re-dated.