If the prompt set, platform scope, competitor list, or classifier changes between runs, you have created a new measurement version, not a clean trend line. Version control matters more than any single score.
What you should leave with
- A baseline is a versioned method snapshot, not a one-off screenshot of answers.
- Freeze prompts, platforms, competitors, region, and score weights before the first run.
- Repeat volatile, high-value prompts; label confidence instead of trusting a single pass.
- Separate real score movement from method drift using a documented change taxonomy.
What is an AI visibility baseline?
An AI visibility baseline is the first approved, versioned measurement of brand recommendations, competitors, citations, errors, and prompt coverage under a documented method. It becomes the reference point every later run is compared against, not a one-time snapshot.
A screenshot of a chatbot answer proves a moment happened; it proves nothing about trend. A baseline is different: it is a stored configuration plus a stored result, both versioned, so a rerun six weeks later is actually comparable.
A usable baseline record stores the prompt set, platforms tested, competitor list, capture date, region, answer mode, number of runs per prompt, classification policy, source-crediting policy, and score weights. Miss one field and the next comparison is guesswork dressed as analytics.
Evidence used in this section
What needs to be frozen before the baseline run?
Freeze the prompt set, competitor set, platform scope, region, repeat policy, entity matching rules, answer-role definitions, and score weights before collecting answers. Any change to these after the fact invalidates a clean comparison later.
Teams that skip this step usually discover the problem in month two, when someone quietly added three prompts or dropped a competitor and the score moved for reasons that have nothing to do with the market. Freezing first prevents that.
Build the prompt set deliberately rather than improvising it; see /build-buyer-intent-prompt-set for buyer-intent coverage, and lock scoring definitions using /ai-visibility-score-methodology before the first data point is captured.
- Prompt set finalized and versioned | Competitor list agreed and dated | Platform scope defined (which assistants, which modes) | Region and language locked | Repeat count per prompt set | Entity matching rules documented | Answer-role definitions written down | Score weights fixed
How many runs are needed?
Use one run only for a directional preview; repeat high-value and volatile prompts when the baseline will guide budget, reporting, or client promises. A single pass tells you what happened once, not what usually happens.
Think in terms of prompt x platform x repeat observations. A ten-prompt, three-platform baseline with three repeats each is ninety observations, enough to separate a stable pattern from a one-off fluke on a single volatile prompt.
Label confidence rather than pretending certainty: high confidence for repeated, stable answers; low confidence for single-run or contradictory results. The repeated-measurement literature supports this discipline directly, since AI answers vary run to run even with identical inputs.
Evidence used in this section
Why do AI visibility scores change?
Scores change because the answer changed, the source environment changed, a competitor changed, the platform changed, or the measurement method changed. Only the first four count as market movement; the fifth is drift you introduced.
Treating every score movement as market news is the single most common reporting mistake in this space. A dropped competitor from the list, a reworded prompt, or a new classifier will move a number without a single customer-facing thing having changed.
Build the habit of labeling the cause before reporting the number. The full change taxonomy and worked examples live at /why-ai-visibility-scores-change, which should sit next to every monthly report a team sends.
| Change type | Example | How to report it |
|---|---|---|
| Answer changed | Assistant now recommends brand in top 3 | Report as movement |
| Source environment changed | New review site got indexed | Report as movement, note cause |
| Competitor changed | Rival launched a new page | Report as movement, flag competitor |
| Platform changed | Assistant updated its model | Report as movement, flag platform |
| Method changed | Prompt set or classifier edited | Do not report as trend; new baseline |
When should you retest after a fix?
Retest after the intended signal is live and discoverable: a page published, profile corrected, source inclusion approved, crawl path fixed, or material claim updated. Retesting too early just measures the old, unfixed state.
Publishing is not the same as being discoverable. A new page needs a working crawl path and, ideally, sitemap inclusion before an assistant's underlying sources can reflect it; checking crawler access first avoids a wasted retest cycle.
Run the same unchanged prompt set, compare answers side by side, classify what moved against the taxonomy above, and explicitly record no-change results. A quiet non-result is still a data point worth keeping.
- STEP 1
Confirm the fix is actually implemented | Check discoverability via sitemap and crawl access | Rerun the frozen, unchanged prompt set | Compare new answers against baseline answers | Classify each movement by cause | Record no-change results, not just wins
Confirm the fix is actually implemented | Check discoverability via sitemap and crawl access | Rerun the frozen, unchanged prompt set | Compare new answers against baseline answers | Classify each movement by cause | Record no-change results, not just wins
Evidence used in this section
How often should monitoring run?
Monthly monitoring is enough for many active programs; weekly checks fit volatile, high-value categories; quarterly checks can fit slower markets. Cadence should follow decision value, not dashboard anxiety.
There is no single correct interval for every brand, and any page claiming otherwise is selling comfort, not method. The right cadence depends on how fast your category moves and how much money rides on the answer.
Use a free run for a first directional look, a paid audit to lock the versioned baseline, and event-driven retests around real fixes rather than a rigid calendar. Full cadence guidance sits at /how-often-run-ai-visibility-audit.
| Stage | Purpose | Typical trigger |
|---|---|---|
| Free preview | Directional first look | Curiosity, early scoping |
| Paid baseline | Versioned reference point | Before committing to tracking |
| Monthly monitoring | Ongoing trend visibility | Active program, moderate volatility |
| Event-driven retest | Confirm a specific fix | Post-publication, post-launch |
| Quarterly refresh | Slower categories | Low volatility markets |
Questions that change the decision
Frequently asked questions
Is a free audit a baseline?
A free audit is a directional first look, not a versioned baseline. It skips repeat runs and locked configuration. Use the free audit as a preview, then the paid audit to lock a retestable version.
Can I compare this month with last month if prompts changed?
Not cleanly. Changing prompts creates a new measurement version, so any score difference mixes real movement with method drift. Keep the prompt set frozen, or treat the new run as a fresh baseline.
How long should I wait after publishing a fix?
Wait until the fix is confirmed discoverable, not just published. Check crawl access and sitemap inclusion first; retesting before content is reachable just remeasures the old, unfixed state and wastes the cycle.
What if the score moves but traffic does not?
That is a corroborated pattern worth watching, not proof of impact. AI citation and referral traffic can lag or stay indirect; treat the score change as inferred signal, not a guaranteed traffic outcome.
Should agencies report every fluctuation?
No. Reporting single-run noise as trend erodes client trust fast. Classify each change by cause first, report confirmed movement, and flag low-confidence or method-driven shifts separately instead of alarming clients.
Primary sources and research
Platform documentation supports factual statements. Where we describe an audit method or prioritization rule, that is AnswerMentions' operating judgment and is labeled as such.
- [1]Google Search Central: AI features and your websiteGoogle explains how AI features can surface links and how crawlable, eligible web content remains part of the Search foundation.
- [2]arXiv: repeated measurement of AI search resultsThe paper supports repeated observations and careful treatment of answer variability instead of relying on one run.
- [3]NIST AI Risk Management FrameworkNIST frames AI measurement as a governed, monitored process with transparency, validity, and risk awareness.
- [4]Google Search Central: sitemaps overviewGoogle documents how sitemaps help discovery of important canonical URLs, which matters before retesting published fixes.
- [5]Google Search Central: Google crawlers and fetchersGoogle documents crawler access patterns, useful when checking whether corrected evidence is accessible before retesting.