Key takeaways
- The same question can get different answers, so one answer is not a measurement.
- Asking several times and reporting a range shows what is real and what is chance.
- A change inside the range is normal; a lasting move outside it is worth acting on.
If your AI visibility figure goes up one week and down the next with nothing changed, the figure is probably measuring chance. Assistants do not give the same answer twice, so a score built on one answer per question bounces around, however carefully it is calculated.
Why answers vary
Assistants generate text, and the generation includes some randomness. They also draw on different sources from one request to the next, and the models behind them are updated. Two identical questions can name different businesses.
Why one answer misleads
Suppose an assistant would name you in about half of its answers to a question. Ask once and you will see either yes or no. Over several weeks that single answer swings between the two, and the average of a handful of questions swings with it.
What fixes it
- Ask each question several times each week, not once
- Count the share of answers that name you, not a single yes or no
- Report a range around the share, so the uncertainty is visible
- Weight each question equally, whatever the number of answers it received
Reading the result
If this week's range overlaps last week's, treat the change as noise. If the range moves clear of the old one and stays there for a few weeks, something real has changed. And when an assistant changes its underlying model, every answer can shift at once; that is worth noting separately.
Our AI visibility service reports ranges for exactly this reason, and the measurement guide shows the method.