Before LLMs, a control in an enterprise system was applied to a deterministic output. A rule checked a value. A threshold blocked a transaction. The output was fixed, the check was fixed, and a failure meant the rule had not been followed. The problem was bounded.
LLMs changed this. The output is now probabilistic. The same input can produce different outputs on different runs. A model can produce text that looks correct, reads fluently, and is factually wrong. It can extract information while silently dropping the qualifier that made that information meaningful. It can summarise a document while omitting the single clause that matters most.
In this environment, a control cannot simply check a value. It has to verify a relationship: between the output and the source, between what was extracted and what was said, between what the model produced and what a second independent model finds when it checks the same material. The four mechanisms described here each do that. They are not principles. They are implementable, testable, and each produces a log entry that can be audited.
Policy documents and oversight principles describe controls. Mechanisms implement them.
Four technical controls
Groundedness is often described as hallucination prevention. In practice, it is a provenance control.
The purpose of the control is not to determine whether a statement is true. It is to determine whether a statement can be supported by the material provided to the model. Each material assertion in the output is evaluated against the source context. Statements that cannot be attributed to the retrieved documents, structured data, or other approved sources are flagged before delivery.
The control does not ask whether an unsupported statement is plausible. It asks whether the system can explain where the statement came from. A contract may contain a force majeure clause that is common in similar agreements. If the retrieved contract does not contain such a clause, the model has no basis for asserting its existence. The statement may be correct. It is still ungrounded.
Groundedness therefore creates a provenance record. Every output receives an auditable score describing the degree to which its assertions can be supported by the input material. The threshold is a governance parameter set by the organisation based on the risk tolerance of the use case.
In regulated industries, the groundedness score should be a logged, auditable property of every output. Not just a runtime check but a persistent record that compliance functions can inspect. Confidence is an internal property of the model. Provenance is an observable property of the output. For governed AI systems, provenance matters more.
A user asks: "Show me all active contracts with counterparties in Germany expiring before Q3." That request is natural language. The retrieval system needs structured inputs: a table name, a set of filters, a date range, a field list. The LLM's job in this control is to resolve the user's intent into those typed query variables. Nothing more.
Once the variables are resolved, the retrieval runs deterministically. The query structure, traversal logic, and scope boundaries are fixed in code. The model does not decide what to retrieve. It does not expand the scope because something seems relevant. It resolved the inputs; the system does the rest. The output is entirely a function of what the database returned, not of what the model chose to say about it.
This is a different role for the LLM than most people assume. It is not generating the answer. It is acting as an intent parser that produces structured inputs to a governed retrieval process. The separation matters: the LLM touches only the query variables, and those variables are typed and validated before the query runs. An entity name that cannot be resolved against a known identifier is rejected before retrieval begins. A date range that falls outside the permitted scope is rejected. The model's output is constrained before it can cause a retrieval error.
The governance property this creates is a clean audit boundary. Every retrieval result is traceable to a specific set of resolved variables, and those variables are loggable, inspectable, and replayable. If a retrieval returns an unexpected result, the investigation starts with the resolved variables, not with the model's reasoning. Was "Germany" correctly resolved to the right jurisdiction identifier? Was the date range interpreted correctly? These are answerable questions with a logged record.
The groundedness control then sits downstream: it verifies that whatever is communicated about the retrieval result is traceable to what the retrieval actually returned. The two controls work together. The LLM resolves the query. The retrieval runs deterministically. The output is checked against the retrieval. The model's probabilistic nature is contained to the one step where natural language interpretation is genuinely needed.
A model should not be assumed to reliably detect its own errors. The same training distribution that produced an error shapes how the model evaluates its own output. Errors that flow logically from a flawed premise are particularly hard to catch, because the flaw is in the premise, not in the reasoning from it.
An adversarial model is a second model deployed specifically to find fault with the output of the first. It is optimised for one task: given this source material and this generated output, identify any statement that contradicts, overstates, or cannot be supported by the source. Its objective differs from that of the generating model. That opposition is what makes it an effective check.
The adversarial model's findings are not binary pass or fail. They are a structured log of potential inconsistencies that a human reviewer can assess. The control pre-filters the output for the specific failure modes the adversarial model is calibrated to detect, reducing the cognitive load on the reviewer rather than asking them to approach every output cold.
Calibrate for recall, not precision. A well-tuned adversarial model will over-flag: it raises concerns that a reviewer determines are not material. That is the correct trade-off. False positives increase reviewer workload. False negatives reach downstream processes unchecked.
Extraction is one of the most common enterprise AI tasks and one of the most poorly controlled. The extracted output looks right: the correct entities are present, the numbers match, the formatting is clean. What is not visible is whether the extraction preserved the semantic relationships that give those elements their meaning.
A liability cap expressed as a percentage of revenue may become a fixed number. A regulatory obligation limited to one jurisdiction may be extracted as a general requirement. The words remain. The meaning changes.
One way to detect this form of drift is to compare the semantic representations of the source passage and the extracted output using an independent encoder model. Large differences may indicate that information has been compressed, simplified, or omitted. Encoder similarity should not be interpreted as proof that meaning has been preserved. It is a signal that helps identify outputs that deserve further review.
The encoder used for this check should be distinct from the model that performed the extraction. A model that produced an extraction with meaning loss is likely to embed it close to the source, because the same distributional patterns that caused the loss shape the embedding. An independent encoder, trained specifically on semantic similarity tasks, provides a more reliable signal.
The purpose of the control is not to certify semantic equivalence. It is to identify extraction outputs whose meaning may have drifted sufficiently to justify escalation. The threshold for escalation is a governance decision. Different document types, use cases, and tolerances for false positives will produce different thresholds. Like groundedness scores, semantic fidelity measurements become part of the output's audit trail.
Where the risk actually sits in a summarisation pipeline
A common assumption is that summarisation pipelines carry lower governance risk because their outputs are informational rather than decisional. This is wrong. The risk does not disappear. It moves upstream, into a question none of the four controls above can answer: who decided what belongs in the summary at all?
When a model summarises a document it makes continuous implicit decisions about what to include. Groundedness confirms that what appears in the summary is present in the source. It says nothing about what was present in the source but did not appear. Encoder fidelity confirms that extracted content preserved its meaning. It cannot detect content that was never extracted.
The model's default selection logic uses salience through repetition as a proxy for importance. Content that appears frequently, uses prominent language, or mirrors the model's training distribution will surface. A liability clause that appears once, in plain language, in a subordinate section may be the most material term in the document. It will not be the most salient to the model. Frequency is not materiality. The model cannot know the difference unless it has been told.
The key question in a summarisation pipeline is not whether the output is grounded. It is who decided what belongs in the summary in the first place.
Summarisation pipelines therefore need a fifth governance layer that sits upstream of the four controls: a topic importance specification. This is a governed definition of what the summary must cover, expressed as mandatory extraction targets rather than free-form instructions. For a contract summary that might be: payment terms, termination rights, liability caps, governing law, data processing obligations, change of control provisions. Each is a typed slot that must be present in the output regardless of how frequently or prominently it appears in the source.
Without this specification, the model is not summarising. It is selecting, using distributional priors about what summaries look like, calibrated on training data that has nothing to do with the enterprise's definition of what matters. The output passes a groundedness check. It passes an encoder fidelity check. And it systematically omits the terms a practitioner would consider essential. The controls make the extraction verifiable. The specification makes the extraction meaningful.
What these controls produce
These controls address different failure modes and are designed to be layered. Groundedness addresses unsupported assertions. Deterministic retrieval constrains what information enters the pipeline. Encoder similarity provides a signal for semantic drift. Adversarial verification looks for internal inconsistencies that the generating model may not detect.
Each mechanism produces evidence. Retrieval variables, groundedness scores, semantic similarity measures, and adversarial findings are all loggable artefacts attached to the output.
No single control proves correctness. That is not their purpose. Their purpose is to make failures observable and outputs explainable. A governed AI output is not one that is guaranteed to be correct. It is one whose provenance, constraints, and verification signals are visible and available for inspection. The uncertainty does not disappear. It becomes observable.