Twenty vendors, twenty definitions
In August 2026 the IAB published Measuring Visibility in the AI Era. Its central observation is not a metric. It is an admission: more than twenty companies now sell AI visibility measurement tools, each using different methodologies that can produce different answers for the same brand, with no common definition of a “mention” and no standard for what constitutes a “citation”. That is a trade body, not a competitor, describing the state of the field.
Look at the published definitions and the problem is immediate. Peec defines visibility as responses mentioning your brand divided by total responses. Rankscale defines share of voice as mentions of you as a proportion of all mentions across you and your competitors — a different denominator producing a different number from the same data. Semrush publishes a 0–100 benchmark score without publishing the formula. Several vendors describe what they track without publishing a calculation at all.
Insight
Work an example. A hundred tracked responses. Sixty mention some brand. Fifty mention yours. Your visibility is either 50% or 83.3% depending entirely on which denominator the vendor chose — and in several cases the vendor has not told you which.
Compare that with the metric this one is named after. Share of voice in media and PR measurement has had a single formula for decades: your metric divided by the total market metric, times a hundred. One number, one method, comparable across agencies. AI share of voice has no such property today.
The measurement problem sits underneath all of it
Search results are broadly stable. Ask the same question twice and you get the same page. Generative engines are not stable, and this is not a bug anyone intends to fix.
General non-determinism is documented. A 2024 study across five major models found that at temperature zero — the nominally deterministic setting — accuracy varied by up to 15 points across repeated runs, with best-to-worst gaps reaching 70 points on individual tasks. A January 2026 paper confirmed drift persists at temperature zero and varies by model size and deployment.
Brand-specific variance is not documented. We could not find a peer-reviewed study quantifying run-to-run variance in brand mentions specifically. What exists is practitioner testing — one published experiment found the same model answering the same prompt minutes apart repeated only about 40% of its cited domains on the second run, across twenty prompts. Small, not peer-reviewed, and a signal rather than a statistic.
The IAB names the problem without solving it, and its recommendation is the right one: report visibility as a range, not a single number. Which has a consequence most dashboards quietly ignore. A visibility score that moved from 41% to 44% has probably not moved. Without a stated interval you cannot distinguish a real improvement from the engine having a different afternoon.
How many runs is enough
There is no established answer, and you should be suspicious of anyone who gives you one without showing the arithmetic.
What can be said is ordinary statistics. Visibility is a proportion, and the precision of a proportion estimate depends on sample size predictably: roughly 300 observations per period gives about five percentage points either way, and detecting a move from 20% to 30% with confidence needs a few hundred answers in each period. Practitioner guidance converging on fifty prompts across several engines, read weekly rather than daily, is consistent with that maths — though it assumes runs are independent, which nobody has demonstrated.
The IAB sets a floor rather than an optimum: under fifty queries is “exploratory” and cannot characterise a category. It distinguishes directional measurement from decision-grade measurement, and the distinction is useful precisely because most marketing is done on directional data without saying so.
Insight
Where this leaves a one-off audit, including ours. A single pass over twenty buying-intent questions across four engines is, by the IAB's own definition, directional rather than decision-grade. It will tell you reliably whether you are broadly present or broadly absent, which domains dominate citations in your category, and where the obvious gaps are — differences large enough to survive the noise. It will not tell you that your visibility rose three points. The useful version of a one-off is as a baseline you re-field identically later. The second run is what makes the first one mean something.
Prompt-set design is where the bias actually lives
This receives far less attention than it deserves, because it is the easiest place to produce a flattering number without technically lying.
“Best shoes”, “best running shoes” and “is X better than Y for marathon training” are three different questions that will return three different brand sets. A prompt set weighted towards phrasings a brand already performs well on produces a visibility score that is real, reproducible and meaningless.
There is no published standard for prompt-set construction. No sampling frame, no intent taxonomy, no exclusion criteria, no peer-reviewed work on selection bias in this setting. What exists is vendor how-to content and commercial tiering — plans capping tracked prompts at 50, 150 or 350, which tells you what vendors charge for rather than what is statistically sufficient.
Hallucination as a brand metric, which barely exists yet
The IAB folds accuracy into its framework under “Portrayal” — sentiment and factual correctness, hallucinations included. Treating it as a distinct, largely unmeasured dimension is the right call, because that is what it is.
General hallucination benchmarks are real and methodologically transparent. Vectara's leaderboard measures factual consistency in summarisation, with the best models around 3% and several well-known ones above 10%. That is a genuine benchmark of a genuine phenomenon — and it is summarisation of supplied documents, not statements about companies. The two are not interchangeable and the industry conflates them constantly.
What does not exist, as far as we could establish: any published, methodologically transparent study of how often an engine states something false about a named company or product. Headline percentages circulate with no traceable methodology behind them. They should not be repeated, including by us.
This matters commercially because brand hallucination is the failure mode with actual legal and revenue exposure — wrong pricing, a discontinued product presented as current, a compliance claim you never made. It is measurable per brand by fielding factual questions and checking answers against ground truth. It is simply not measured in general, so there is no benchmark to compare yourself against.
What defensible measurement looks like
Strip out everything unproven and a short specification remains.
Our own Signal Score is built to that shape — twenty buying-intent questions across four engines, scored on eight signals and read against the competitors in the same category — and the honest description of it is the one above. It is a baseline instrument. Its value comes from being re-fielded identically at day 45, where the comparison is like-for-like and the movement means something. Read once, in isolation, it is an anecdote with a number attached, and we would rather say so than let a client over-read it.
The question to ask before buying any of this
Not “what is our AI share of voice”. The honest answer today is that the term does not have a fixed meaning.
Ask instead: against which questions, on which engines, how many times, with which denominator, and compared with what baseline? A vendor who can answer all five is measuring something. A vendor who cannot is selling a number.
IAB, Measuring Visibility in the AI Era (3 Aug 2026) · Peec AI brand-metrics documentation (undated; read 4 Oct 2026) · Semrush AI SEO metrics knowledge base (undated; read 4 Oct 2026) · Rankscale AI-visibility guide (updated 7 Jul 2026) · Ahrefs Brand Radar methodology (published 29 Oct 2025, modified 16 Jun 2026) · Meltwater, How to Measure Share of Voice (10 Apr 2026) · Non-Determinism of “Deterministic” LLM Settings (arXiv:2408.04667) · Quantifying non-deterministic drift (arXiv:2601.19934) · Aggarwal et al., GEO, KDD 2024 · Vectara Hallucination Leaderboard (live page, last update 11 May 2026; read 4 Oct 2026)