AI Visibility Data Collection Methodology

This page documents how Spyglasses collects and processes the data behind every number in your AI Visibility dashboard — which platforms we query, how we query them, what we count as a mention or a citation, and how each headline metric is calculated.

We publish this for the same reason we publish our AI Placement Value Score, AI Placement Quality Score, and Citation Optimizer methodologies: a visibility metric you can't audit is a metric you can't act on. Where a number has a caveat, we state the caveat. Where a signal is unavailable on a platform, we say it's unavailable rather than filling the gap with an estimate.

How Spyglasses Collects Data From Each AI Platform

Spyglasses tracks five AI answer platforms. We collect from each one in the way that most closely reproduces what a real prospect would see, which means two different collection modes depending on what the platform allows.

Table: Spyglasses AI platform data collection methods. For each tracked AI platform, this table lists how Spyglasses runs the prompt (anonymous web interface vs. official API), which model version answers, how location targeting is applied, and whether the platform exposes the grounding searches it ran.

Why two collection modes

ChatGPT, Gemini, and AI Overviews are run through the platform's own interface, logged out and unauthenticated. No account, no history, no memory, no personalization. This is deliberate: an answer generated for a signed-in user with months of chat history is not the answer a new prospect sees. A logged-out session is the closest available proxy for a first-time buyer researching your category. Because we use the public interface, these platforms answer with whatever model they currently serve to free-tier users — we do not and cannot pin a version. When the platform silently upgrades its default model, your data reflects that change, because your prospects experience that change.

Prompts on these three platforms can optionally be run through a residential connection in a location you specify, so a query about "dentist near me" resolves against the market you actually operate in. Claude and Perplexity are queried over an API and so cannot be geo-located this way — for those platforms the location is supplied as context in the prompt instead, which is a weaker signal. See Locations for how to configure this.

Claude and Perplexity are run through their official APIs. Neither platform offers a logged-out free tier we can run a prompt through reliably — Claude requires an account, and Perplexity restricts anonymous use. Rather than skip these platforms or scrape them in ways that would be fragile and against their terms, we query them through the documented API.

An API call has no notion of "a person typing a question into a box," so we supply a system prompt that establishes that context. It frames the model as an assistant giving recommendations to someone researching a purchase, adapts its emphasis to the category (software, retail, local services, and so on), and instructs it to search the web and cite what it finds. This is intended to reproduce the behavior of the prospects your clients most want to reach: people asking an assistant which solution to buy. It is not a neutral, context-free API call, and we would rather say so than imply an equivalence with the UI-based platforms that doesn't exist.

Platforms and models we don't track by default

If you need a platform we don't cover in the base product — Copilot, DeepSeek, a specific model version, or a reasoning tier above the free default — we can add it for your account, in some cases for an additional fee. Contact us to scope it.

How Often Prompts Run

Tracked prompts run once per day, at 06:30 UTC. The nightly run covers ChatGPT, Gemini, and Google AI Overviews — the three platforms we can run through an anonymous interface at daily volume.

Claude and Perplexity run inside AI Visibility Reports, which you trigger on demand or schedule. Because these platforms are billed per API call, running them nightly against every tracked prompt is not something we do by default. If you want daily Claude or Perplexity tracking, that's one of the extensions we can enable per account.

A weekly maintenance pass (Sundays, 07:00 UTC) refreshes SERP rankings for your grounding searches, harvests Gemini fan-out queries, and re-scores brand accuracy.

The same prompt does not produce the same answer twice

This is the single most important thing to understand about all AI visibility data, ours included. These models are non-deterministic. Run the identical prompt twice and you may be mentioned once and not the other time — with no change to your website, your competitors, or the model.

This means a single day's result is a sample, not a measurement. A brand that appears in 6 of 10 runs has a true mention rate somewhere near 60%, but a single run tells you only "yes" or "no." Confidence comes from accumulating observations over time.

We publish the statistics for this openly, including the formula, in our AI visibility data reliability calculator. We use the normal-approximation binomial confidence interval:

margin of error = z × √( p(1 − p) / n )

n = number of prompts × number of runs (30/month at daily cadence)
p = observed mention rate
z = 1.645 at 90% confidence

Ten prompts tracked daily produces 300 observations per platform per month, for a margin of error of roughly ±4.7% at 90% confidence. Three prompts tracked daily produces 90 observations and a margin of roughly ±8.7% — directional, but not a number you should report as precise.

We show you this before you spend anything. When you add prompts to a project, Spyglasses displays the resulting observation count, margin of error, and a quality rating (Reliable under ±8%, Directional under ±15%, Exploratory above that) while you are still choosing how many prompts to track — not after you've paid for a number that can't support the conclusion you want to draw from it.

Our Commitment to Data Provenance

Every prompt Spyglasses runs and analyzes on your behalf is one you explicitly asked us to run.

Some vendors in this category populate dashboards from prompt databases rather than from prompts their customers requested. Those databases are typically sourced one of two ways:

• Harvested clickstream data — acquired from third-party tools such as free VPNs, browser extensions, or "free" analytics products that record what their users type into AI assistants, and resell it.
• Synthetic prompts generated at volume from proxy data — inflating a small seed set into thousands of prompts derived from signals like Google's "People Also Ask," then presenting the resulting volume as observed demand.

Spyglasses will never acquire, use, resell, or advocate for either. The first is built on data collected from people who did not meaningfully consent to having their AI conversations sold. The second dresses up a guess as a measurement. Neither belongs in a dataset a client is going to make budget decisions from.

Everything downstream of that commitment is auditable:

• Raw prompt data is always available for inspection in the platform. For any prompt, on any run, you can read the full answer text the AI returned, every citation it produced, and every brand it named.
• You can export it on demand, at any time, in full.
• We do not filter, curate, or reweight the prompt set to improve how your results look.

How We Detect Brand Mentions

Brand mention detection is deterministic string matching — your brand name, plus any number of aliases you configure, matched case-insensitively at word boundaries against the AI's answer.

We do not use an LLM to judge whether your brand was mentioned. This is a deliberate trade. An LLM judge can catch a paraphrase that a string match misses, but it introduces non-determinism into the one layer of the pipeline that most needs to be reproducible: two runs of the same judge over the same answer can disagree, which means your share of voice could move without your visibility moving. Deterministic matching means any mention we counted, you can find yourself in the raw answer text — and any mention we missed, you can capture by adding an alias.

• Matching is case-insensitive and respects word boundaries, so "Acme" does not match "Acmeworks."
• Aliases are yours to control. Product names, legal entity names, common misspellings, abbreviations, former names. If AI describes you in a way our matcher misses, an alias fixes it permanently.
• Competitors work identically — each competitor has its own name and alias list.

Changing an alias recalculates your history immediately

When you add or remove an alias, rename your brand, or change your competitor set, Spyglasses recalculates your historical brand mention metrics in real time. Every stored metric row — mentions, share of voice, citation counts, per-platform breakdowns — is rewritten against the new alias set.

This works because we retain every raw AI answer. Recalculation re-matches the new alias list against the stored answers; it does not re-run any prompts and does not cost you anything. The practical effect is that adding an alias six months in retroactively corrects six months of history, rather than creating a discontinuity in your chart on the day you added it.

Spyglasses Core Metric Definitions

The atomic unit of measurement is a platform execution: one prompt, run against one platform, once. A report covering 10 prompts across 5 platforms produces 50 platform executions.

Table: Definitions of core Spyglasses AI visibility metrics. For each headline metric in the Spyglasses dashboard, this table gives the precise formula and the unit it is calculated over.

Three of these need a caveat stated plainly.

Share of Voice is a presence rate, not a zero-sum split

Our Share of Voice answers: "of all the times we asked, how often did AI name you?" The denominator is the number of prompts we ran, not the number of brands named.

This means Share of Voice does not sum to 100% across you and your competitors. If an AI answer recommends five tools and you're one of them, you get credit for that execution — and so do the other four. It is entirely possible for you and every competitor to each hold 90% share of voice, if AI keeps naming all of you.

Some vendors define share of voice as a zero-sum split of total mentions (your mentions ÷ everyone's mentions). That number has a real weakness: it falls when a competitor is mentioned more, even if you are mentioned in exactly as many answers as before. We chose the presence-rate definition because "am I in the answer?" is the question that maps to a decision, and because it can't be gamed by a competitor's noise. If you're comparing a Spyglasses Share of Voice against another tool's, confirm which definition they use — they are not interchangeable.

Share of Influence measures narrative, not naming

Share of Voice asks whether you were named. Share of Influence asks whether your framing of the category showed up before any brand was named — the awareness stage, where an AI is explaining what matters in a category rather than recommending vendors.

It is calculated by embedding each of your positive key messages and each sentence of the AI's answer using text-embedding-3-small, scoring them with cosine similarity, and counting a match when a sentence clears a 0.50 similarity threshold. The threshold is calibrated: at 0.40, sentences that merely shared a topic with a key message were matching. At 0.50, a sentence has to actually express the idea. Share of Influence is then the unweighted mean of the match rates across all of your positive key messages.

This is a genuinely different signal from mentions. A brand can have low Share of Voice and high Share of Influence — AI has absorbed your argument about what matters in the category, but recommends someone else. That gap is usually the most actionable thing on the dashboard.

Citation rate can exceed 100%

The numerator counts citations; the denominator counts executions. An answer that cites two different pages on your domain contributes two citations from one execution. A citation rate above 100% means AI is citing you more than once per answer, on average — which is a good outcome, not a bug.

What we do not measure

We do not run sentiment analysis on how AI answers describe your brand. We could, but a sentiment classifier over an AI answer is an LLM judging an LLM, and we haven't been able to make it stable enough that a score change reliably means something changed in the world. We would rather ship no number than an unstable one. (Sentiment is scored on PR placements, where the input is a fixed human-written article rather than a regenerated answer — see the AI Placement Quality Score methodology.)

We do not publish a position-in-answer ranking. We capture a rank when a platform exposes one, but availability differs so much between platforms and detection paths that a cross-platform "average position" would be comparing incompatible things.

How We Handle Citations

Every source an AI platform surfaces is stored, classified, and — where we can reach it — crawled.

Cited, evaluated, or search result

Not every source an AI looks at makes it into the answer. We record which:

The distinction is derived per platform from what that platform actually reports: Claude and the AI Overviews panel tell us directly which sources they cited; for ChatGPT and Gemini we check each source URL against the links present in the answer text. Where a platform doesn't distinguish (Perplexity self-reports a single citation list), we say so rather than guess.

"Evaluated only" is the most underused signal in the product. It identifies pages AI is reading but not crediting — usually a shorter path to a citation than creating something new.

What we store per citation

For every citation: the URL, page title, whether it's your domain, which brand the answer attributed it to, the citation type above, the page type and content format, and — once crawled — the author, publish date, and word count. Media type and publisher authority are stored at the domain level.

For how cited pages are classified into media types, authority tiers, and content formats, see the Citation Intelligence documentation.

We crawl cited pages and scan them for your brand

When we detect a citation, we make a best-effort attempt to fetch the page and scan its content for mentions of your brand and your competitors, using the same alias list. We try a direct fetch first and fall back to a rendering service for pages that block it. Some pages can't be reached at all — hard paywalls, aggressive bot protection — and we mark those rather than pretending we read them.

This distinction matters, and we keep the two strictly separate in every metric. But the gap between them is useful in its own right: it's coverage you have already earned, that AI is already reading, and is not yet crediting you for. In the dashboard this appears as hidden earned media, and it's often the cheapest visibility win available — the page already exists and already mentions you, so converting it into an attributed citation is an editing problem, not a PR problem.

Grounding Searches and SERP Rankings

When an AI assistant answers a question, it usually doesn't search for your prompt verbatim. It decomposes the question into its own web searches — grounding searches, sometimes called fan-out queries — and builds its answer from those results. These are the queries that actually determine whether you get cited.

We capture grounding searches for ChatGPT, Gemini, and Claude. All three expose the searches they ran. Google AI Overviews and Perplexity do not expose them, so we have none to report — we would rather show you nothing than infer a query list the platform never gave us.

Because the AI reads the search results for these queries, its top results are the pool it can cite from. So for every grounding search we capture, we run our own SERP and record where you, your competitors, and every other domain rank:

Claude grounds its web searches in Brave rather than Google, so a Google ranking tells you little about whether Claude can find you. We run Brave for Claude's searches specifically. Brave's API returns at most 20 results, so Claude's SERP coverage is top 20 rather than top 30. If a search produced by Claude was also produced by ChatGPT, we run both engines against it.

A domain that doesn't rank gets recorded with a null position — "not in the top 30" is itself a finding, and it's what drives gap analysis: grounding searches where competitors rank and you don't.

SERP rankings are re-checked weekly, up to 50 grounding searches per property per week. Properties with more than 50 tracked grounding searches have them refreshed in rotation rather than all at once.

How We Measure Brand Accuracy

Share of voice tells you whether AI names you. Brand accuracy tells you whether what it says about you is correct — the failure mode where you appear in every answer and are described as something you are not.

The eight-call design

We ask four AI platforms to describe your brand, twice each:

• Without web browsing — the model answers from its training data alone. This is what it "remembers" about you, and it's what a user gets when the assistant doesn't search.
• With web browsing — the model researches you first. This is what it says when it can read your site.

Four platforms × two modes = eight independent brand descriptions per scoring run. The two modes usually disagree, and the disagreement is the point: a brand that scores well with browsing and badly without has a training-data problem, not a website problem, and no amount of on-site optimization will fix it quickly.

Table: Spyglasses brand accuracy scoring platforms and models. These are the four AI platforms and specific model versions Spyglasses uses to score brand accuracy. Unlike the free-tier UI platforms, these models are pinned so that accuracy scores remain comparable over time.

How the score is calculated

Each of the eight descriptions is scored against your Brand Snapshot — the structured self-description Spyglasses builds by crawling your own website, and which you can edit at any time. This is important, and worth being explicit about: we are not scoring AI against some objective truth about your company. We are scoring it against what your own website says about you, as corrected by you.

Five fields are compared and weighted:

Category and pricing model are scored deterministically against a fixed decision table. Features, ICP, and differentiators are compared semantically — "helps teams collaborate on documents" and "document collaboration for teams" should score as a match, not a miss. That comparison is made by an LLM judge (gpt-5-nano) against a fixed rubric, with a deterministic string-similarity fallback if the judge fails.

The weighted result is a 0–100 score. Two behaviors worth knowing:

• If a model doesn't know your brand at all, the score is 0, not a low-but-nonzero number. An AI that has never heard of you is a categorically different problem from an AI that describes you inaccurately, and averaging them together would hide it.
• Penalties are applied for confident wrongness — describing you as being in the wrong industry, or attributing features you couldn't possibly have. Hallucinating a feature you don't have is worse than omitting one you do.

Known Limitations

We would rather you learn these here than discover them in a client meeting.

A daily result is a sample. See the reliability section above. Never report a single day's share of voice as a fact.

Cross-platform comparison is asymmetric. Claude and Perplexity are collected via API with a system prompt; ChatGPT, Gemini, and AI Overviews are collected from anonymous UI sessions without one. Trends within a platform are more trustworthy than differences between platforms.

Model versions shift underneath us on the UI platforms — by design. We report what the platform currently serves, which means an unannounced model swap by OpenAI or Google will show up in your data as a step change that has nothing to do with your marketing. We use annotations to mark these when we detect them.

Logged-out is a proxy, not a mirror. Real users have accounts, chat history, memory, and personalization. We measure the unpersonalized baseline because it's the only thing that's measurable and comparable — but it is not the only answer any real person sees.

Deterministic matching misses paraphrase. If AI refers to you by a name that isn't in your alias list, we won't count it. This is why the alias list exists, and why we recalculate history when you change it.

Not every cited page can be crawled. Paywalls and bot protection mean content-mention coverage is a floor, not a total.

These metrics are directional signals, not guarantees. A high share of voice means AI is naming you today, in the answers we sampled, for the prompts you chose. It does not guarantee any specific person asking any specific question will see you.

Related

• Is Your AI Visibility Data Reliable? — The statistical confidence calculator, with the formula
• Citation Intelligence — How cited pages are classified
• Key Messages — How Share of Influence is configured
• Locations — Location-targeted prompt runs
• AI Placement Quality Score — How individual PR placements are scored
• AI Placement Value Score — How publishers are scored
• Citation Optimizer — How content is scored and rewritten for citation