A web-extraction test found that a short instruction made models far less likely to fill missing fields with plausible but wrong values. Across 16 models, 405 of 573 missing fields received invented answers without the instruction. With it, 116 of 574 did. That is a drop from 70.7% to 20.2% in this test, though one in five missing fields still drew an answer.
The instruction was: “Use null for any field whose value is not on the page. Do not guess.” The benchmark was published September 27 by Earn an Honest Dollar, an agent-services marketplace. It compared each model’s results with and without that sentence, using a plain HTTP fetch and page text as input.
The paired pages expose borrowed answers
The test used 42 pairs across seven page types. Each pair differed by one row: the requested field appeared on one page and was absent from the other. Both pages retained a tempting nearby value, such as an old price where the current price was missing, a fact-checker’s name where the author was missing, or an update date where the publication date was missing. A correct extractor would return the value when present and null when absent.
The old-price trap was especially stark. Without the instruction, all 16 models used the old price as the current price on that page. With it, one did. The result shows how a small prompt change affected this particular failure mode. It also shows why returning a non-null value still requires evidence from the right field.
The benchmark ran each contestant once on synthetic pages. Its model table generally scores 36 missing-field cases per run after exclusions and errors. There are no repeat runs to show how stable the rates are, and real websites can have messier labels, layouts and navigation. The three paid extraction APIs in the table were tested only with the null rule, so their results cannot establish how much that instruction changed their behavior.
Checking returned values is a separate step
The publisher also asked two cheap models whether extracted values were supported by the page. In its scored sample, GPT-6 Luna caught 38 of 49 invented values and rejected none of 47 correct values. The other checker caught 23 of 49 invented values and rejected none of 48 correct values. Those counts came from the same limited test, and the missed invented values matter if an extraction result will trigger a purchase, payment or publication.
