Somewhere in your firm today, an associate is pasting text into a chatbot. The useful response from leadership is neither a ban nor a shrug. It is a clear view of what actually differs between a public assistant and a private system, and a policy that draws the line where the risk really sits.
Start by being fair to the public tools, because a policy built on pretending they are useless will be ignored.
What public chatbots are genuinely good for
Public assistants are strong at tasks whose inputs and outputs are public knowledge. Explaining an unfamiliar doctrine before a client call. Summarizing a public filing. Turning a dense regulation into plain English. Producing a first outline for a memo, or a checklist of issues to consider in a kind of deal you rarely touch.
They are also good drafting scaffolds. Given a non-confidential prompt, they produce serviceable first-pass prose that a lawyer then rewrites: client-friendly explanations, cover emails, plain-language summaries of public material. For lawyers learning a new area, they compress hours of orientation into minutes, provided every proposition gets verified against real sources afterward.
The right mental model for a public assistant is a well-read stranger: enormously broad, fast, occasionally wrong with total confidence, and owing no duty to your client. Plenty of useful work fits that description. The problems begin when the stranger is handed things a stranger should never see.
Gap one: confidentiality and privilege
Typing client facts into a consumer chatbot sends them outside the firm's boundary, to be processed on infrastructure the firm does not control, under terms the firm did not negotiate. That sentence alone should settle most of the debate, but it is worth being precise about the professional duties in play.
ABA Model Rule 1.6 requires lawyers to make reasonable efforts to prevent unauthorized disclosure of information relating to a representation. Comment 8 to Rule 1.1 folds technological competence into the duty of competence itself: a lawyer is expected to understand the benefits and risks of the tools they use. The ABA's 2024 formal opinion on generative AI, and the steady stream of state bar guidance following it, points the same direction: understand where the data goes before you send it.
Privilege adds a second layer. Whether sharing privileged material with a model provider risks waiver is a question courts are still working through, and no firm wants to be the test case. Enterprise tiers of public tools improve the contractual position: no training on your data, shorter retention, better terms. They do not change the architecture. The data still leaves, and the boundary still sits outside the firm. The question for leadership therefore shifts from "is this tool safe" to "whose infrastructure processes our client's facts, under whose keys, with what record." Asked that way, consumer tools answer themselves.
None of this makes public tools forbidden. It makes them a decision. A lawyer who understands where the data goes can route public questions to public tools in good conscience. The duty is to know the difference before the paste, not after.
Gap two: retention and training
Most of the difference between consumer and enterprise tiers lives in unglamorous data-handling terms. Before anyone at the firm relies on any tier of any public tool, someone should be able to answer these questions in writing:
- Retention. How long are prompts and outputs kept, where, and in what jurisdiction?
- Training. Is customer content used to improve models, and is exclusion the default or a setting someone must find?
- Human review. Can the vendor's employees or contractors read conversations, under what circumstances, and with what logging?
- Deletion. Can the firm delete on demand, and can the vendor demonstrate that deletion actually happened?
- Legal process. If a third party subpoenas the vendor, does the firm find out before or after the data moves?
Enterprise agreements answer some of these well. The point is that with any public tool, the answers are promises about someone else's infrastructure. Our checklist of SOC 2 questions for legal AI vendors goes deeper on how to pressure-test promises like these.
Gap three: grounding
Public models know public law imperfectly and know your firm's documents not at all. Both halves of that sentence matter.
The first half is why fabricated citations keep appearing in sanctions orders: a model trained on the open internet reproduces the shape of authority without the substance, and it does so most fluently exactly when it is wrong. The second half is quieter but costs more. The questions that carry economic weight at a firm are firm-specific: what did we negotiate last time, what is our position on this clause, which matters touch this counterparty. Ask a public tool what indemnity cap your client accepted in its last three acquisitions and it will either decline or invent. Neither is the answer a partner needs at six in the evening before a call.
A private system inverts this. It retrieves from the firm's own knowledge, filtered by the asker's permissions, and cites the documents behind every answer. The difference is not model quality. It is context: the same class of model, grounded in a structured graph of the firm's own matters and documents, produces answers an attorney can verify instead of merely admire.
Gap four: auditability
When a client's outside counsel guidelines ask how AI was used on their matters, or an insurer asks the same question at renewal, or a court asks how a brief was prepared, the firm needs an answer better than "we do not know what was asked."
Public tools produce no matter-level record. Conversations sit in individual accounts, invisible to the firm, unlinked to matters. A private system logs who asked what, on which matter, which sources were retrieved, and what came back. That record is what turns AI use from an unknowable liability into a supervisable activity, which is what the supervision duties in Model Rules 5.1 and 5.3 already expect of firm leadership.
The same record has a second use that gets less attention: it is how the firm learns. Which questions attorneys ask, which answers get used, where the system falls short. Without it there is no feedback loop, and the tool never improves for the way your firm actually works.
What "private" should actually mean, and where to draw the line
Vendors use "private AI" loosely, so pin the term down. It should mean a single-tenant deployment dedicated to your firm, running inside your own cloud boundary; keys and network in the firm's hands; retrieval over firm knowledge with document-level permissions and ethical walls enforced at query time; and a complete audit trail. Our piece on tenant isolation unpacks each layer, and our security overview describes how Reframe implements them inside a firm's own tenant.
Then write the policy as a line, not a ban. A workable sketch:
- Green: public tools permitted. Public-knowledge work containing no client information: doctrine explainers, public filings, style rewrites of non-confidential text. Verification of anything factual is mandatory.
- Yellow: permitted with care. Abstracted questions where a pattern is discussed without identifying facts. Honest abstraction is harder than people think, so this lane needs training and spot checks.
- Red: private system only. Anything touching client facts, matter facts, deal terms, names, or documents. This is most real work, which is why the private lane has to be good, not merely compliant.
Two implementation notes make the policy stick. First, the private lane has to live where work already happens: in the DMS, in Word, in the research workflow, or it will be routed around. Second, tell attorneys plainly what the private system logs and why. A record framed as protection for the attorney, the client, and the firm reads very differently from one framed as monitoring, and adoption follows the framing.
The firms handling this well run both lanes deliberately: public tools for public knowledge, a private system like Reframe for everything that touches a client. The line holds because the private lane is genuinely better for real work, not because a memo said so.