The pattern is familiar enough by now to have a rhythm. A lawyer under deadline asks a chatbot for supporting authority. The answer arrives fluent and confident, citations formatted exactly right, quotations that sound the way judges sound. The brief gets filed. Then opposing counsel or a clerk goes looking for the cases and finds nothing, because there is nothing to find. Sanctions follow, with a published order that reads like the last dozen published orders.
The names change and the jurisdictions vary, but the failure keeps recurring, and it keeps recurring among competent, otherwise careful lawyers using tools that sound authoritative because sounding authoritative is what they are built to do. When a mistake repeats across people who know better, the cause is not character. It is architecture.
Firms have responded with policies, and policies help. But a policy against citing unverified authority is a promise to be careful, and the recent record of sanction orders is a record of what happens to promises under deadline pressure. The durable fix changes what the tool can do, not what the lawyer intends.
Why models fabricate citations
A language model generates the most plausible continuation of the text in front of it. Fluency is the product. A legal citation, from the model's side of the table, is just unusually constrained plausible text: a pattern of party names, reporters, volumes, page numbers, and years that it has seen millions of times. When the training data contains the real case, plausible and true happen to coincide. When it does not, the model produces something citation-shaped anyway, because producing text is the only thing it does.
This is why fabricated citations look so good. The model is not failing at its job when it invents a case. It is succeeding at the wrong job. Fluency is not retrieval, and no amount of confident formatting converts one into the other. The same mechanism produces invented quotations from real cases, which are harder to catch and just as sanctionable.
Why a better model is not a strategy
Each model generation hallucinates less, and it is tempting to conclude the problem will age out. Two facts of arithmetic say otherwise.
First, low rates multiplied by high stakes still produce unacceptable risk. A system that fabricates one citation in two hundred, deployed across a firm that files thousands of documents a year, produces fabricated authority on a schedule. No partner would accept those odds from an associate, and a sanctions order carries no footnote for error rates. The exposure is not linear, either: one fabricated citation can undo a decade of credibility with a bench that remembers.
Second, the verification burden moves in the wrong direction. As the model gets more reliable, readers verify less. Trust grows exactly as scrutiny shrinks, which means the residual errors land precisely when nobody is checking. Reliability that depends on the reader's vigilance is a control that weakens with use.
A legal AI that cannot say it does not know is not safe to sign your name over.
What grounding actually means
Grounding is an architecture, not a system prompt asking the model to be accurate. Four properties define it, and a buyer can test every one of them in a demo:
- Retrieval comes first. The system searches a controlled corpus before any text is generated. The answer starts from what was found, not from what the model remembers.
- Generation is constrained to what was retrieved. The model composes from the retrieved passages. It is not free to supplement them from training data when the passages run thin.
- Every assertion links to a source. Each claim traces to a passage the attorney can open and read. An answer without provenance is a hypothesis wearing a suit.
- The system can refuse. When the corpus does not support an answer, the honest output is that it does not know. Refusal behavior is a feature to test for during procurement, not an inconvenience to prompt around.
None of these properties are exotic. They are engineering choices, and a vendor has either made them or has not. The demo question is simple: ask the system something its corpus cannot support, and watch what happens. The good ones decline. The dangerous ones improvise.
Two corpora, two problems
Grounding for legal work splits into two domains worth keeping distinct.
For public law, the controlled corpus is an authoritative legal database: published decisions, statutes, regulations. Grounding there means a cited case resolves to a real, current document in that database, every time. The lawyer's traditional duties still apply, checking that authority remains good law among them; grounding does not replace judgment about how strong an authority is, it guarantees the authority is real. The existence question, the one that produces sanctions, closes by construction.
For firm knowledge, the corpus is the firm's own documents, matters, and negotiated positions, and the firm controls it completely. The risk profile shifts. The question is rarely whether a document exists, but whether the answer faithfully reflects what the documents say, across versions, amendments, and ethical walls. Control of the corpus is the quiet advantage: the firm decides what is in scope, who can retrieve what, and how answers must cite. This is the domain where Reframe operates, grounded retrieval over the firm's own knowledge, inside the firm's own boundary.
The graph advantage
Passage-level retrieval grounds an answer to loose text: somewhere in these ten paragraphs is support for the claim. That is far better than nothing, but legal work wants more precision, because the questions that matter are about instruments and versions, not paragraphs. Grounding to a graph of entities and clauses makes the citation structural: clause 8.3 of the executed stock purchase agreement, as modified by the second amendment, on this matter. Not "a document in the data room."
Clause-level provenance changes verification from a reading task into a glance, and it survives the questions that follow: which version, which amendment, which party. The same structure powers audit. When a client asks why the firm's answer said eighteen months, the trail runs from the sentence to the clause to the executed document. We compare the two retrieval architectures in RAG versus knowledge graphs for legal work; the short version is that they compose, and the graph is what makes citations precise.
Designing the verification workflow
Grounding produces checkable answers. Workflow design decides whether anyone actually checks. The pattern that works is boring and specific. Citations render as one-click links, so verification costs seconds; add friction beyond a click and busy people stop clicking. Answers expose their coverage: assertions supported by sources look different from the system's own inferences, and unsupported claims do not ship. Confidence displays help only when they are tied to source coverage; a percentage the model invents is another fluent guess. And retrieval is logged, what was searched, what was returned, what was shown, so the firm can audit an answer after the fact, inside the same security and audit model that governs the documents themselves.
The same standard extends to drafting surfaces, where inserted language should carry its precedent sources with it, as we describe in our piece on drafting in Word. Partners never see most of this design work. They just notice that checking an answer is faster than doubting it, and adoption follows that arithmetic: attorneys keep using the tool whose answers survive checking, and they quietly abandon the one that wastes a skeptic's time.
The professional framing is already in the rules. A lawyer who signs work supervises the research behind it, whether the researcher is a first-year or a model. Competence with these tools does not mean re-running every search. It means insisting on systems whose output can be checked cheaply, then checking it. Hope is not a control. Retrieval, provenance, refusal behavior, and a one-click path to the source: that is the cure, it is the standard Reframe builds to, and it is available now.