On this page
You can spend an afternoon typing your category into ChatGPT and screenshotting whether your name comes up. That is the manual version of an AI visibility checker, and it is where most founders start. The problem is not that it is crude. The problem is that the answer changes every time you ask, so one check tells you almost nothing you can act on. This post covers what these tools actually measure, where quick one-shot checkers break down, and how to get a number worth tracking.
What an AI visibility checker actually does
Strip away the dashboard and every checker does the same three things:
- Takes a prompt set. A list of questions your buyers would plausibly type: “best invoicing tool for freelancers”, “alternatives to [competitor]”, “what software helps with X”.
- Sends each prompt to one or more assistants. ChatGPT, Claude, Perplexity, Gemini, sometimes Google AI Overviews on top.
- Reads the responses looking for your brand. Then reports how often you showed up.
That output is your AI visibility: the share of relevant AI answers where your product gets named. Everything that separates a useful checker from a toy is in the details of those three steps, especially step two.
The reason this matters at all is that the buyer journey moved. Someone evaluating tools used to open Google, scan ten blue links, and click three. Now a meaningful share of that research happens inside a conversation where the assistant names four products and the user picks one. If you are not among the four, you were never in consideration, and no analytics dashboard will tell you why.
Why a single check is not a measurement
Ask ChatGPT the same question three times and you can get three different lists. That is not a bug in the checker, it is how these models work: responses are sampled, not looked up. Your product can appear in run one, vanish in run two, and appear in third place in run three.
This is the single biggest reason a one-time check misleads people. Founders run a checker, see their name, and conclude they are visible. Or they miss once and conclude they are invisible. Both are coin flips reported as facts.
The fix is multi-sampling: run each prompt several times, then report a percentage. “Mentioned in 7 of 12 runs” is a measurement. “Mentioned” is an anecdote.
What a good checker measures beyond yes or no
Mention rate is the headline number, but it is the least actionable one on its own. The checkers worth your time also capture:
- Position in the answer. Being named first in a list of five is not the same as being the afterthought in the last sentence. Buyers rarely evaluate past the top two or three.
- Sentiment and framing. “X is a solid choice for small teams” and “X is limited compared to Y” are both mentions. Only one helps you.
- Cited sources. Which URLs the assistant actually pulled from when it answered. This is the most directly actionable output a checker produces, because it tells you which pages to get onto, and citation tracking deserves its own attention.
- Per-provider breakdown. You will routinely see 40% visibility on Perplexity and near zero on a model leaning harder on training data. An averaged single score hides that split, and the two failures need completely different fixes.
- Competitor comparison on the same prompts. Your number in isolation is meaningless. Your number next to the three products that keep beating you is a to-do list.
One-shot checkers vs continuous trackers
The market splits cleanly into two shapes, and choosing the wrong one wastes either money or months.
| One-shot checker / grader | Continuous tracker | |
|---|---|---|
| Typical use | First diagnostic, “am I invisible?” | Weekly trend, competitor benchmarking |
| Prompt count | A handful, often auto-generated | Dozens, curated by you |
| Repetition | Usually one pass | Repeated runs on a schedule |
| Answers | ”Here is a snapshot" | "Here is what changed and why” |
| Cost | Free or near free | Subscription |
Free graders such as the HubSpot AEO Grader or Knowatoa sit firmly in the left column and are a reasonable first stop. Enterprise platforms like Profound or Brandlight sit in the right column with pricing built for brand teams rather than a solo founder. Most indie SaaS products need the right column at an indie price, which is the gap tools like AskAiRank are built for. If you are weighing a free grader against ongoing tracking, the side-by-side breakdown of AskAiRank and the HubSpot AEO Grader lays out where the free option stops being enough.
How to run a credible check yourself this week
You do not need to buy anything to get a defensible baseline. You do need to be systematic about it.
- Write 20 to 30 prompts. Mix four types: category questions (“best X for Y”), comparison questions (“A vs B”), problem questions (“how do I stop Z”), and a couple of branded ones (“is [your product] any good”). Write them the way a buyer types, not the way you describe your product.
- Pick at least two assistants. ChatGPT and Perplexity is the minimum useful pair, because one leans on training data and the other on live retrieval.
- Run each prompt three times per assistant. Yes, this is tedious. It is also the entire difference between a number and a guess.
- Record four columns per run: mentioned yes/no, position in the list, how you were described, and which sources were cited.
- Repeat the exact same set for two competitors. Same prompts, same runs. Their mention rate is your benchmark.
- Save the prompt set verbatim. Next month you re-run the identical list. Changing prompts between checks makes the comparison worthless.
The step-by-step walkthrough in how to track your brand in ChatGPT covers the ChatGPT half of this in more detail, including how to keep the sessions clean so your own history does not skew the answers.
Reading the result without fooling yourself
Three traps catch almost everyone on their first check.
Flattering prompt sets. If most of your prompts contain your product name, congratulations, you will score high and learn nothing. The prompts that matter are the ones where you are not mentioned in the question.
Logged-in sessions. Running checks from an account that has discussed your product for months gives you personalized answers. Use a fresh session or an incognito window so you see what a stranger sees.
Treating the score as a revenue metric. Visibility is a leading indicator, like rankings were. A rising mention rate means more buyers hear your name during evaluation. Whether that converts still depends on your landing page and your pricing.
One more: do not chase a number across tools. Every checker uses its own prompts, providers, and sampling, so a 40 in one product and a 40 in another are not the same quantity. Pick one method and stay with it.
What to do next
Run the manual version this week on 20 prompts, two assistants, three runs each, and two competitors. It costs you an afternoon and gives you a real baseline instead of a vibe. If the numbers look bad, that is the useful outcome: you now know which prompts you lose and which sources the assistants trust instead of you.
Then decide whether you want to repeat that afternoon every month by hand. If not, that is exactly the point where an automated checker earns its keep, and you can see how AskAiRank handles the scheduling, sampling, and competitor side of it for you.
Frequently asked questions
Answers about what an AI visibility checker measures and when a one-time check is enough.
For a first look, yes. A free checker tells you whether you are completely invisible or occasionally mentioned, and that alone is worth knowing. It stops being useful the moment you want to answer 'did last month's work help', because a single run has too much randomness in it to show a trend.
LLM responses are non-deterministic, so asking the same question twice can produce two different lists of recommended tools. That is normal model behavior, not a broken tool. The fix is repetition: run each prompt several times and report a mention rate instead of a yes or no.
As a rough floor, 20 to 30 buyer-realistic prompts sampled a few times each across more than one provider. Fewer than that and your result mostly reflects which questions you happened to pick. A 5-prompt check is a spot check, useful for a sanity test and not much else.
Often not directly, since many answers do not include a clickable link to you. What it does is put your name in front of a buyer at the exact moment they are asking for a recommendation, which usually surfaces later as direct visits and signups that say they heard about you from an assistant.
Check at least one retrieval-heavy assistant like Perplexity alongside ChatGPT. They fail differently: retrieval-based answers reward fresh, crawlable, citable pages, while training-data-heavy answers reward being an established named entity. Seeing only one of the two hides half the problem.
