Which page changes deserve an AI citation test?
Start with missing facts that a reader needs to make a decision. A controlled citation study supports testing relevance and factual completeness before assuming a layout change will help. Its experiment begins after the sources have already been supplied to the model, so it does not show how to get a page discovered by a search engine.
What the experiment actually compared
In What Gets Cited, published in May 2026, researchers supplied pairs of product-review texts to six language models. The pairs differed in one content factor, and the researchers reversed their order to account for position preference. They recorded which source was cited first. They did not call a search engine.
Relevant content and explicit decision facts, including price, generally favored citation selection in this setup. Formatting effects were less consistent. These are conditional citation preferences, not measured gains in traffic or discovery. The paper also reports statistical fitting warnings for some estimates, so its largest ratios should not become marketing promises. Its synthetic rewrites and anonymized brands further limit transfer to real sites.
Build a test around one missing answer
Consider an illustrative comparison page for appointment software. It describes an attractive dashboard, but it never says whether the quoted price covers one staff member or the entire business. A potential buyer asks which plan suits a small team. The page does not contain a complete answer to that question.
A useful proposed edit would state the billing unit, the included features, and the conditions that change the price. Attach a source or a checked date where the information can change. Leave genuine uncertainty visible. The goal is to make the answer accurate enough to use.
Keep a record of the original page and the edited version. Write the buyer question before making the edit. If the question changes at the same time as the page, the comparison becomes harder to interpret.
For an initial controlled test, give the answering system both versions in separate trials. Keep the competing material and instructions fixed. Reverse source order. Inspect whether the answer uses the newly supplied fact correctly, as well as whether it includes a citation.
This proposed test checks whether the edit helps when the page is available. A separate live check is needed to see whether a search product finds and cites the page.
Keep the outcomes separate
A citation can accompany an incorrect explanation. An accurate explanation can produce no visit. Record the text of the answer, the cited URL, and any observable visit as separate outcomes. Do not combine them into a single success label.
For a live check, save the exact question, product, date, and response. Repeat the same check after the edit has become accessible. Treat any apparent improvement as a result to investigate. Search conditions and competing pages can also change.
- Choose one buyer question that the page currently answers poorly.
- Add the missing verified facts and test whether answers use them accurately.
- Check live citations and website visits separately before calling the edit successful.