A client report shows an AI mention rate rising from 60% to 70%. The team appears to have made progress. Then someone opens the collection ledger. Completed answers fell from 80 to 50, while the number mentioning the brand fell from 48 to 35. The higher percentage describes a smaller, less complete set of answers.
Before changing a content budget or renewing a monitoring contract, reconcile the planned observations, complete answers, verified mentions and unresolved results. A completion rate shows how much of the intended collection reached the reporting stage. The mention rate describes the answers that passed the declared inclusion rules. Their relationship determines what a reader can infer from the headline.
Xindar publishes this guide and provides Semvi, its AI visibility monitoring product. All numerical examples below are fictional. The formulas and acceptance exercises are proposed methods that a buyer can reproduce with its own permitted records.
Give the planned observation a stable identity
Define one observation before collection begins. It might be one exact English buyer question, submitted once on a specified consumer surface under recorded account and regional conditions, within a scheduled sampling window. Assign that event a stable identifier.
Keep a separate record of its attempts and retained answer. An interrupted browser capture can leave the intended observation unresolved even after the platform produced an answer. Reading that original conversation later can recover the evidence. Sending another question creates an additional attempt whose treatment needs a declared rule.
Also define what qualifies as a complete response. A finished answer that declines a comparison can still meet a completeness rule. Its content receives the applicable classification. A partial capture leaves the classification unfinished.
Set a reporting cutoff. That lets two people reconstruct which answer records and review decisions entered the same report. The related guide to AI visibility report cutoffs explains how late evidence changes a report version.
Read the four counts together
The following fictional study plans 100 comparable observations in each of two periods. Each cell is intended to supply one complete answer with a binary, verified-entity mention label. The unresolved cells in this example have technical collection gaps. The question set, surface and classification rule are held constant.
| Measure | Earlier period | Later period |
|---|---|---|
| Planned observations | 100 | 100 |
| Complete, eligible answers | 80 | 50 |
| Answers mentioning the verified brand | 48 | 35 |
| Unresolved observations | 20 | 50 |
| Mention rate among complete answers | 48/80 = 60% | 35/50 = 70% |
| Collection completion | 80/100 = 80% | 50/100 = 50% |
The observed mention rate rose by ten percentage points. Collection completion fell by 30 percentage points. The report contains 13 fewer observed mentions.
Those facts can coexist. The outstanding question concerns the unavailable labels. If collection disproportionately loses questions on which the brand tends to be absent, the retained answers can become more favorable. If the losses concentrate elsewhere, the effect can run in another direction.
Survey methodology offers a relevant principle. The U.S. Census Bureau explains that nonresponse can bias an estimate when the responding and nonresponding units differ in relevant characteristics. Applying that principle to an AI monitoring panel requires a study-specific examination of the missing records. Census Bureau response-rate definitions
Calculate what the unresolved labels could change
Let `N` be the prelisted observations, `C` the complete eligible answers, `M` their verified mentions and `U = N − C` the unresolved observations. For this calculation, assume every unresolved cell has a well-defined potential binary label under the same collection protocol.
The observed complete-answer rate is `M/C`. If `x` of the `U` unavailable labels would be positive, the completed-panel rate would be `(M + x)/N`.
Because `x` can range from zero to `U`, the data permit the following range of completions.
`Lower endpoint = M/N`
`Upper endpoint = (M + U)/N`
For the earlier period, those endpoints are 48% and 68%. For the later period, they are 35% and 85%.
| Quantity | Calculation | Result |
|---|---|---|
| Earlier completed-panel range | 48/100 through 68/100 | 48%–68% |
| Later completed-panel range | 35/100 through 85/100 | 35%–85% |
| Smallest compatible later-minus-earlier change | 35% − 68% | −33 percentage points |
| Largest compatible later-minus-earlier change | 85% − 48% | +37 percentage points |
The endpoints enumerate extreme assignments to the missing binary labels. They describe the stated finite panel under the assumptions above. Statistical uncertainty about a wider population requires its own sampling model and treatment of repeated observations.
The later 70% figure therefore leaves a wide range of full-panel changes compatible with the available record. Recovering the missing evidence can narrow that range. Adding an unsupported assumption about the missing answers can also narrow it mathematically, but the assumption then becomes part of the conclusion.
Ask how much missing information would reverse the decision
An analyst can turn the same example into a decision question. Let `x_old` and `x_new` be the positive labels among the earlier and later unresolved cells.
The later full-panel rate exceeds the earlier rate when `35 + x_new > 48 + x_old`. Equivalently, the later missing set needs at least 14 more positives than the earlier missing set.
Suppose ten earlier missing cells eventually prove positive. The earlier completed count would then be 58 mentions. The later missing set would need at least 24 positives to produce an increase, giving 59 mentions. With 23, the periods would tie at 58.
This sensitivity calculation identifies which unknown facts matter to the decision. It supplies a concrete question for evidence recovery. It supplies no prediction of how the unresolved answers will be classified.
The Bureau of Labor Statistics adds a useful caution from survey work. An overall response rate alone gives limited information about nonresponse bias, while patterns across groups can help investigate the collection process. BLS response-rate methodology
Examine where completion was lost
A shared dashboard can conceal an uneven loss by engine, market, client or question type. Examine the planned and completed counts within the groups relevant to the business decision.
Consider another fictional panel with 50 planned observations on each of two engines. Engine A delivers all 50 answers, with 40 mentions. Engine B delivers ten answers, with one mention and 40 unresolved cells.
The combined observed rate is `41/60`, approximately 68.3%. Yet Engine B accounts for half the planned panel and only one-sixth of the completed answers. That imbalance deserves attention before the percentage becomes a client success claim.
An equal-engine descriptive index would average 80% and 10%, producing 45%. Its weights express the chosen reporting design. The ten observed Engine B answers still leave uncertainty about that engine's other 40 cells. For the original 100-cell panel, the extreme completed-panel range is 41%–81%.
Breakdowns can identify where to investigate. They cannot by themselves establish why a failure occurred or what the unavailable answer contained. Preserve authentication, transport, capture and review statuses at their actual level of evidence.
Give absent features their own category
Some monitoring plans require a feature that a consumer product displays selectively. Google explains that AI Overviews appear when its systems consider generative AI especially helpful. A successfully loaded Search page can therefore provide direct evidence that no Overview appeared for that observation. Google AI Overviews guidance
Retain that observation with an explicit no-Overview status. An AIO-content metric can report the complete Overviews it actually collected alongside the feature-appearance count. A separate study of brand exposure across Search sessions would need its own outcome definition and denominator.
The missing-label endpoints above require a defined binary outcome for every included cell. Apply them to the appropriate unresolved set after establishing eligibility. Keep completed pages with absent features, technical failures and unknown eligibility visible in their own counts.
Timing also needs a documented rule. Otterly's current monitoring FAQ says daily prompts run without a fixed hour and fresh results appear after the run completes. That narrow timing statement illustrates why an early report can contain pending work. A supplier's own schedule and customer agreement determine the applicable cutoff. Otterly monitoring interval FAQ
Put the demonstration in the purchasing brief
Ask a monitoring provider to reconstruct one report from a small set of disposable, authorized test records. Agree the expected outcome before running the exercise.
| Acceptance exercise | Evidence the buyer should receive |
|---|---|
| One complete answer mentions the target | Complete answer, verified entity and positive label |
| One complete answer omits the target | Complete answer and reviewed negative label |
| One planned cell remains technically unresolved | Original cell identifier, attempt status and unknown label |
| One answer arrives after the cutoff | Original report and an explained later report version |
| A question has two attempts | Both attempt records and the rule selecting the reported observation |
| One engine loses most of its cells | Planned and completed counts by engine beside the combined figure |
| A file contains an empty mention field | A data dictionary explaining the field and its status column |
Recalculate the counts from the supplied file. Confirm which statuses enter each denominator. Inspect the unresolved records individually if the result will influence a renewal, performance fee or public claim.
Set any completion threshold according to the decision, collection constraints and agreed service scope. Document that threshold before the results arrive. A percentage chosen after seeing an attractive outcome gives the reviewer little protection against selective reporting.
Apply this brief when evaluating Semvi by Xindar. Its published product reference describes monitoring and evidence scope. A demonstration should establish the actual status fields, calculation rules and export behavior needed for this particular acceptance exercise.
Questions buyers ask
Can I still report the rate among complete answers?
Yes. Name the inclusion rule and show the numerator, completed-answer denominator, planned count and unresolved count. The resulting percentage describes the observed complete answers under that rule.
Should an unresolved mention field be filled with zero?
Preserve its unknown state and the collection status that explains it. Assign a negative label after a complete eligible answer has been reviewed under the declared entity rule.
What happens when there are zero complete answers?
Report zero completed answers and the actual operational statuses. The ratio `M/C` has a zero denominator and is unavailable. If every planned binary label remains unresolved under the hypothetical completion model, its endpoints span 0%–100%.
Do the endpoints establish the true market-wide mention rate?
They enumerate possible completions of the prelisted binary labels. Market-wide inference additionally depends on which buyer questions were selected, their intended population, repeated observations and collection conditions.
Can reweighting repair every missing-data problem?
A weighting model requires relevant information about the missingness and its assumptions. Preserve the observed counts, report the chosen weights and explain how groups with unavailable outcomes are treated. Inspect the evidence supporting the model before interpreting its adjusted result.
Sources and method
Public sources were reviewed on November 13–14, 2026. Census and BLS explain principles from survey methodology. Google describes selective AI Overview appearance, and the Otterly citation supports the specific timing statement attributed to its FAQ. The finite-panel formulas, fictional counts, sensitivity exercise and purchasing tests are Xindar's editorial analysis. Vendor account operation, collection reliability and customer outcomes require evidence from the applicable service and authorized demonstration.
