On this page
Your dashboard records no mention of your brand. You open ChatGPT, ask what looks like the same question, and your product appears in the answer.
That difference deserves a closer look. It does not, on its own, show which observation is wrong.
A monitoring result describes a recorded question, provider request and point in time. Your manual chat describes another interaction. To compare them usefully, start with the two answers and the conditions that produced them.
First, compare one answer with one answer
A dashboard percentage may summarize dozens of responses. A manual question produces one response. Even if the question is relevant to the monitored set, it cannot validate or invalidate the entire summary by itself.
In Prerender Buddy, open the selected site's collection and find the exact monitored question. Then inspect the successful provider answer behind the result. Confirm the collection date: the view may be showing an earlier usable collection rather than the latest attempted run.
If you need a general process for building that baseline, start with checking ChatGPT brand mentions. Here, the task is narrower: explaining a disagreement between two observations.
The text may be similar without asking the same thing
Small wording differences can change the task.
Consider these illustrative questions:
- “Which booking tools work for small tour operators?”
- “Which booking tools work for small kayak tour operators in Croatia?”
- “Would HarborBookings work for my kayak business?”
The first asks about a category. The second adds a business type and market. The third supplies a brand that the answer can discuss directly. A mention in the third answer does not establish that the brand would appear in an unbranded discovery question.
Copy the exact monitored wording before comparing results. Include any market or language context recorded with it. Otherwise, you may be comparing different buyer needs rather than different answers to the same need.
A consumer chat and an API request have different context
Prerender Buddy records controlled calls to configured provider APIs. It does not read private conversations in the consumer ChatGPT app.
An existing chat can contain earlier messages that narrow the question. Search may also use information beyond the sentence you just typed. OpenAI explains that ChatGPT can rewrite a prompt into search queries, use approximate location and, when enabled, use relevant saved memories in query rewriting. Those details can change the search context. OpenAI's ChatGPT search guide
For example, your earlier messages might establish that you run a small agency and need a tool compatible with your current stack. A standalone monitored question may not contain either constraint.
A fresh conversation is useful for reducing prior-chat context, but do not assume it reproduces an API request exactly. Record the relevant settings you can see, and leave unknown settings unknown.
Search and model configuration can differ
A provider name identifies a service, not a complete experiment. Model selection, search configuration and supplied location can affect what a request asks the system to do.
OpenAI's API documentation describes configurable web-search behavior, including location and domain filtering. It also distinguishes the broader set of sources consulted during a search from inline citations in the answer. These are reasons to preserve request context and source relationships when comparing observations. OpenAI's web-search documentation
This does not mean every control is enabled in PB or that you should try to reproduce hidden ChatGPT settings. It means a monitoring provider label should not be read as a promise that the consumer app will return the identical answer.
The same discipline applies when comparing other providers: check their actual recorded context instead of treating all answers as interchangeable.
Timing and variation matter
A Monday collection and a Friday manual chat are observations at different times. A source may have changed, a different page may have been retrieved, or the answer may have selected a different set of examples.
Repeated requests can also produce different wording or selections even when the visible inputs stay the same. One changed answer is a reason to inspect the evidence, not enough to establish a lasting visibility change.
Treat the specific cause as a hypothesis unless the evidence establishes it. A different source list is visible evidence; a claim about why the provider chose that list usually goes beyond what the result shows.
Repeat comparable checks on a consistent schedule. Do not keep rephrasing a question until you get a favorable answer and then compare that selected screenshot with an unedited monitoring baseline.
Check what “visible” means in each result
An answer can name your product without linking to your website. It can cite your documentation without recommending the product. A provider can also return a source that is never explicitly cited in the final answer.
Those outcomes should be recorded separately. PB's overview counts successful responses that mention or recommend the target brand for its Visibility percentage. Its Sources cited count uses distinct explicitly cited URLs.
For an illustrative collection with eight successful responses, three mentioning or recommending the brand would produce a rounded Visibility result of 38%. A ninth manual answer that recommends the product is additional evidence; it does not make the original three-of-eight calculation incorrect.
Check the difference between mentions, citations and recommendations before treating different labels as contradictory results.
Failed checks are not negative answers
A failed provider request does not show that the brand was absent. Neither does a queued run, an unavailable provider or a response that could not be processed.
In PB's main summary, failed and unfinished runs are excluded from the successful-response denominator. Always inspect completion coverage alongside the score. A partial collection may calculate correctly while still giving you less evidence than expected.
If a compact table labels a cell as a miss, open its detail and check whether the underlying run actually succeeded. An error message is a collection problem to investigate, not an answer about your brand.
A practical comparison record
Keep the evidence side by side:
| Detail | Monitored observation | Manual observation |
|---|---|---|
| Question | Exact recorded wording | Exact submitted wording |
| Date and time | Collection/run time | Time of the manual chat |
| Surface | Provider API | Consumer app or another interface |
| Model and search | Recorded configuration, where available | Visible settings; unknown where not exposed |
| Market/context | Stored language, country and prompt context | Relevant location, memory and prior-chat context |
| Run outcome | Successful, failed or incomplete | Answer received or error encountered |
| Brand result | Mention, recommendation or no detected target | What the answer actually says |
| Sources | Explicit citations and other source types separately | Links shown in or alongside the answer |
This record often reveals that two apparently conflicting results describe different conditions. If it does not, you have a specific example that can be investigated.
When the mismatch needs product review
Ask for investigation when the recorded evidence contradicts its displayed interpretation. Examples include:
- The saved answer clearly names the configured brand, but the result says it was absent.
- A failed run is presented as a successful negative answer.
- An unrelated business with a similar name is counted as your brand.
- A returned source is labeled as an explicit answer citation.
- The screen shows the wrong site's collection or does not make its date clear.
Share the question, provider, collection date and private-safe evidence. The useful report is “this answer was classified incorrectly,” rather than “ChatGPT gave me a different answer.”
For your next review, open one collection in AI Visibility, inspect a question and its recorded provider answer, then compare it with the manual result. Keep both observations. The value of monitoring is a consistent record you can investigate over time, with enough context to understand what changed.