AI · Feature
If a language model gives different answers to the same question when asked repeatedly, this can be a warning sign. Semantic entropy examines such differences at the level of meaning. The approach helps with certain fabricated answers, but it is not a universal truth detector. In particular, consistently repeated errors remain a central limitation of this form of uncertainty measurement in practical applications.
Not every variation in wording means uncertainty
A question can have several equivalent answers without the model wavering on the substance. The formulations Paris is the capital of France and France’s capital is Paris differ linguistically, not factually. The Nature paper by Farquhar and colleagues therefore uses semantic entropy: several generated answers are grouped by meaning rather than simply counting different word sequences. If the outputs are spread across contradictory meanings, this indicates uncertainty. The researchers used this approach to examine a particular subset of hallucinations, known as confabulations. These are arbitrary, false generations that may vary when the query is repeated. The method is interesting because it does not require a known model answer for every individual question in order to produce a warning signal.
Why consistent answers can still be wrong
A model can repeat the same error every time. This produces little semantic variation, even though the statement is factually incorrect. Misleading wording in a question can also reliably steer answers in the same direction. Semantic entropy therefore measures not truth itself, but a form of instability. There is also additional computational effort: several answers must be generated and the relationships between their meanings assessed. That assessment can itself be wrong, for example when there are subtle differences between quantities or conditions. The NIST risk profile for generative AI accordingly treats unsupported, convincingly presented content as a broader problem. A single uncertainty score replaces neither source checking nor task-specific tests and should not be mistaken for blanket approval for sensitive decisions, even if the user interface displays very precise numbers.
Turning warning signals into useful workflow responses
The practical benefit lies primarily in graduated responses. When uncertainty is high, a system could search for additional sources, ask a targeted follow-up question or flag the answer for human review. This requires thresholds to be tested on real tasks. Too many warnings make the tool cumbersome; too few create false reassurance. A particularly revealing question is how many genuinely incorrect answers get through despite an unremarkable score. The interface should explain clearly to users what was measured: for example, substantial differences between several attempts at an answer, not a supposedly objective probability of truth. Semantic entropy is therefore useful as an additional checking mechanism. As a substitute for robust evidence, it would create precisely the illusion of certainty that it is actually meant to question critically.
Sources & further reading
- Farquhar et al.: Detecting hallucinations using semantic entropynature.com
- NIST: Generative Artificial Intelligence Profile (2024)doi.org
Written in our own words with AI assistance, based on the sources listed above. The text distinguishes research findings, technical prototypes and established knowledge. This feature is not a breaking-news report.