You run a North London business. You’ve invested in your website, fine-tuned your content, optimised for AI search. Then you check your AI visibility score. It’s gone up 15 points. You’re excited. You check again three days later. It’s down 8 points. You’re confused. Which number is real?
According to new research published in a peer-reviewed study cited by Search Engine Journal, the answer is uncomfortable: neither. Or both. Or it doesn’t matter the way you think it does.
Here’s the secret most North London business owners and even some agencies don’t talk about: AI visibility rankings aren’t stable measurements. They’re mostly statistical noise.
What the Research Actually Shows
A new paper on AI visibility metrics demonstrates that these scores fluctuate significantly between runs, even when nothing on your website changes. The variations are so large that a single reading can mislead you into believing your strategy is working, or failing, when the truth is far more complex.
The researchers offer a practical solution: a stopping rule for when your rankings are finally trustworthy enough to act on. This is the insider insight nobody’s discussing openly: you need to run multiple measurements and wait for convergence before you trust the data.
For a North London business owner trying to make decisions about your digital strategy, this changes everything. You’re not chasing a real number. You’re trying to read a signal buried in noise.
Why This Matters More Than You Think
AI search is different from traditional Google rankings in one critical way: Google’s rankings are derived from a stable, indexed database. Your position for “web design Finchley” might shift a few places week to week, but the underlying index is the same every time Google calculates it.
AI visibility, on the other hand, is measured by running your content through an AI model and checking whether it gets cited, ranked, or recommended. But AI models don’t always produce identical outputs from identical inputs. There’s built-in variation. It’s how these systems work.
So when you measure AI visibility today and see one score, then measure again tomorrow and see a different score, you’re not necessarily seeing a real change in your content’s performance. You’re seeing the statistical margin of error in the measurement itself.
This is the secret your competitors might not have caught yet: most businesses are making strategic decisions based on noise, not signal.
The Stopping Rule: How to Know When You Can Trust Your Data
The researchers behind this study weren’t content to just identify the problem. They offered a mathematical approach: run your AI visibility measurement multiple times, and only treat the result as stable when it converges, when successive runs show minimal variation.
For North London businesses, this translates to practical advice:
- Run your AI visibility check at least three times over a period of days, not all on the same day.
- Look for a pattern, not a single spike. If your score jumps 20 points one day, drops 12 the next, then rises 18, you haven’t moved, you’re in noise territory.
- Wait for convergence. Only when your scores cluster in a narrow range (ideally within 5 points of each other) can you trust that a real change has occurred.
- Change only one variable at a time. If you optimise your schema markup, wait for convergence. Then optimise your content structure. Then measure again. This isolates which changes actually moved the needle.
This approach sounds laborious compared to the quick “check once, act fast” mentality that dominates digital marketing. But it’s honest. It reflects reality.
What This Means for Your North London Business Strategy
The larger insight here is uncomfortable for agencies selling “AI search optimisation” and for business owners desperate for quick wins: AI visibility metrics are not yet reliable enough to be your primary decision-making tool.
This doesn’t mean AI search doesn’t matter. It does. But it means you need to shift how you measure success.
Instead of obsessing over AI visibility scores, focus on what actually drives business value:
- Are you getting cited in AI search results? This matters more than a raw visibility number. Citations are harder to fake and more meaningful for your brand.
- Are customers finding you through AI-powered tools? Track this in your analytics. Look for traffic from ChatGPT, Perplexity, Claude, and other AI search interfaces.
- Is your content ranking in traditional Google search? If you’re not visible in regular search, AI search won’t save you. The foundations matter first.
- Are your trust signals strong? Author credentials, real reviews, clear contact information, and verifiable claims, these help both Google and AI systems. Build these first.
We see this play out with clients across Finchley, Barnet, Muswell Hill, and beyond. The businesses that win aren’t the ones chasing the latest AI visibility metric. They’re the ones building genuine authority: consistent messaging, real case studies, genuine expertise demonstrated in their content, and transparent business information.
The Hard Truth About Measurement
Here’s what this research really exposes: the rush to measure and optimise for AI search has outpaced our ability to measure it reliably. We want to treat AI visibility like traditional rankings, a single number that tells us how we’re doing.
But it’s not that simple. And pretending it is will cost you time and money chasing phantom improvements.
The secret isn’t a new tactic or a hidden ranking factor. The secret is patience and rigour. Run your measurements multiple times. Wait for the noise to settle. Only then make changes. Only then measure again.
This is what separates businesses that actually improve from those that confuse activity with progress.
What to Do Right Now
If you’re a North London business owner wondering how to navigate AI search in 2026, here’s the practical path forward:
- Stop checking your AI visibility score daily. It’s not actionable data.
- Establish a measurement baseline by running your AI visibility check 3-5 times and averaging the results.
- Focus on the fundamentals: clear website structure, authoritative content, real trust signals, and accurate business information.
- After implementing changes, run the measurement again multiple times before deciding whether it worked.
- Prioritise AI search citations over raw visibility numbers. Citations are what actually drive customer discovery.
- If you’re working with an agency, ask them to explain their measurement methodology. If they can’t articulate the stopping rule and convergence principle, they’re guessing.
Your competitors are probably still obsessing over single AI visibility scores. They’re making changes based on noise. You now know better. That’s your advantage.
The research is clear: AI visibility rankings aren’t stable until you test multiple times and wait for convergence. Most businesses aren’t doing this. Most agencies aren’t communicating it. And most decision-makers don’t understand what they’re actually measuring.
But you do. Use that knowledge to make smarter decisions about your North London business’s digital strategy in 2026.
Frequently Asked Questions
Why do AI visibility scores change if I haven’t changed my website?
AI models introduce variation by design. Each time you run a measurement, the model may produce slightly different results due to sampling, temperature settings, or other internal processes. This is normal and expected. It doesn’t indicate a real change in your visibility.
How many times should I measure before I trust the data?
The research suggests running measurements until results converge, typically 3-5 runs over a few days. Only when successive measurements cluster within a narrow range (ideally 5 points or fewer) can you trust that a real change has occurred.
Should I abandon AI search optimisation altogether?
No. AI search matters. But shift your focus from chasing visibility scores to building genuine authority: strong trust signals, real expertise, cited content, and verifiable claims. These serve both traditional search and AI search equally well.
What’s more important: AI visibility rankings or Google rankings?
For most North London businesses in 2026, traditional Google search still drives more traffic. Focus there first. AI search is growing but not yet the primary driver for most SMEs. Build a solid foundation in Google Search, then optimise for AI as a secondary priority.
How do I know if my AI search strategy is actually working?
Track AI-generated traffic in your analytics. Look for visits from ChatGPT, Perplexity, and other AI interfaces. Monitor whether you’re being cited in AI search results. These are real signals. A single visibility score is not.
