Multi-sampling is the technique of running each prompt in your tracking set multiple times per measurement session and averaging the results. It is the standard approach to producing reliable Visibility Scores given the inherent non-determinism of AI language model outputs.
Why multi-sampling matters
AI language models are probabilistic. Given the same prompt, a model will produce different responses on different runs because of temperature (controlled randomness) in its generation process. If you run a prompt once and your brand is not mentioned, you do not know whether that is because the model rarely mentions you, or because you happened to get an unlucky sample.
Multi-sampling resolves this ambiguity. If you run a prompt five times and your brand appears in three of those five responses, you have a 60% mention rate for that prompt. That figure is far more meaningful than a binary “yes/no” from a single run.
The cost-accuracy trade-off
Each sample costs money (API calls) and time. More samples = more accuracy, but diminishing returns set in quickly:
| Samples | Variance | Cost relative to 1 sample |
|---|---|---|
| 1 | High | 1x |
| 3 | Medium-low | 3x |
| 5 | Low | 5x |
| 10 | Very low | 10x |
The standard-deviation of a visibility estimate drops roughly by the square root of the sample count. Going from 1 to 3 samples roughly halves the standard deviation. Going from 3 to 5 samples reduces it by a further ~18%. Going from 5 to 10 samples reduces it by another ~29% but at 2x the cost.
For most indie SaaS founders, 3 samples strikes the right balance: meaningfully more reliable than single-run measurements, at a cost that is justified by the business value of the data.
Multi-sampling in AskRank
AskRank runs:
- Free/Starter: 1 sample per prompt per run
- Pro: 3 samples per prompt per run
- Team/Agency: 5 samples per prompt per run
The Visibility Score displayed on your dashboard reflects the averaged result across all samples. You can also see the per-sample raw responses in the detail view if you want to inspect the variance directly.
Multi-sampling vs multi-prompt
Multi-sampling (running the same prompt multiple times) should not be confused with having a large, diverse prompt set. Both are important:
- A large prompt set covers a broad range of query angles - it tells you which types of questions your brand does or does not appear in
- Multi-sampling gives you accurate measurements for each of those prompts
The combination of a thoughtfully designed prompt library and adequate sampling per prompt is what produces trustworthy visibility data.