Most legal AI procurements reach the security section the same way. The vendor's deck shows a SOC 2 logo, someone from IT nods, and the conversation moves on to pricing. That moment, repeated across hundreds of firms this year, is exactly what the badge is designed to produce. It ends the inquiry right before the useful questions begin.
A SOC 2 report is not a certification, and it is not a grade. It is an auditor's opinion that controls chosen by the vendor operated within a scope defined by the vendor over a period selected by the vendor. Each of those qualifiers is a place where the assurance you think you are getting can quietly narrow. For a firm about to route client confidences through someone else's software, the distance between the badge and the report is the entire exercise.
Type I, Type II, and the scope games
Start with the basic question: which report is it? A Type I says the controls were suitably designed as of a single date. A Type II says they operated effectively over a review period, typically six to twelve months. A vendor holding only a Type I has, in effect, been audited for one day. Plenty of young AI companies run on a Type I for a year or more while the Type II remains perpetually in progress. Ask for the Type II, and ask for the exact period it covers. A report whose window closed eighteen months ago describes a different company than the one you are buying from.
Scope is the larger game. SOC 2 lets the vendor decide which systems and services the auditor examines, and it is common for the product you are actually buying to sit partly outside the audited boundary. The core application is in scope while the new agent framework, the fine-tuning pipeline, or the analytics layer is not. Read the system description and confirm that the product on your order form, including its model endpoints, is the system that was audited.
Then find the subservice organizations. Nearly every AI vendor runs on cloud infrastructure and calls external model APIs. The report either includes those providers in scope or, far more often, carves them out and relies on their separate attestations. Carve-outs are normal, but they mean the report in your hands says nothing about the most important dependencies in the stack. Ask which subservice organizations are carved out, and request their reports as well.
Get the report under NDA, then actually read it
SOC 2 reports are confidential documents, which is why the public sees only the badge. Ask for the full report under NDA. Reframe shares its own report this way as a matter of routine, and any serious vendor will do the same. One that hesitates to share it with a law firm, an institution professionally organized around confidentiality, is telling you something about its posture.
Two sections repay close reading. Section 3 is the vendor's own description of the system: the architecture, the data flows, the boundary. It is where you learn what the vendor believes it is responsible for. Section 4 lists every control the auditor tested and the result of each test, including exceptions. Read the exceptions one by one. A failed access review or a missed offboarding at a vendor whose engineers can reach production data is not a paperwork problem.
Do not skip the complementary user entity controls, the CUECs. These are the controls the auditor assumed the customer operates. If the report assumes that you enforce multifactor authentication, review user access quarterly, and configure retention appropriately, then part of the assurance is conditional on your firm actually doing those things. Hand the CUEC list to whoever will administer the tool and confirm that each item has a named owner.
The questions SOC 2 was never designed to answer
The trust services criteria predate this generation of AI systems. Nothing in them forces a vendor to answer the questions that matter most for a legal AI product, so ask them directly:
- Where does inference run? Is the model endpoint inside the audited boundary, or does every prompt leave for a third-party API? If it leaves, under what agreement, to which region, and with what retention on the other side?
- What crosses the boundary during embedding and generation? Documents are chunked, embedded, retrieved, and assembled into prompts. Ask for a data flow diagram that shows every hop client text takes, in transit and at rest.
- What is retained, and for how long? Prompts, outputs, embeddings, and logs each have their own retention story. Zero-retention claims often describe the model provider while the vendor's own logging quietly keeps everything.
- Is customer data used for training? Get the answer in the contract as a warranty, not in a marketing FAQ, and make sure the language covers fine-tuning and evaluation as well as training.
- Who are the subprocessors, and how do you learn about changes? You want a current list, advance notice of additions, and a right to object, not a webpage you are expected to check on your own.
Testing, incidents, and keys
Ask about penetration testing cadence, then ask the sharper question: did the most recent test include AI-specific attack paths? Prompt injection through ingested documents, data exfiltration through model outputs, cross-tenant leakage through shared vector stores. A test that never touched the model layer evaluated the previous decade's product.
Incident notification deserves a number, not an adverb. Prompt notification is a mood; seventy-two hours is a commitment. Your own engagement letters and outside counsel guidelines may put you on a notification clock with clients, so the vendor's SLA has to be short enough for you to keep your own promises.
Key custody is the quiet question underneath all of it. Who holds the encryption keys for your data at rest, and can you revoke them? If the vendor holds the keys in the vendor's cloud, every other control operates inside that fact. Customer-managed keys change the failure modes materially, and vendors that support them will say so without being asked twice.
The deployment question that reframes the exercise
One question changes the weight of every other answer: where does the platform actually run?
When the product runs in the vendor's cloud, the vendor's SOC 2 is your primary assurance about runtime security, and everything above is load-bearing. When the platform deploys inside your firm's own tenant, the roles split. The vendor's report still matters, but it now covers the build pipeline, the release process, and support access. The runtime, the data, the keys, the network boundary, and the audit logs sit inside controls your firm already operates and your clients already examine. You are no longer extending trust to someone else's environment; you are extending your own.
That is the architecture Reframe ships: the platform, including the Context Graph and the model endpoints it calls, runs inside the customer's own cloud boundary. The controls are laid out on our security page, and the path from kickoff to a live system is described in our delivery process. Why tenant residency settles most of the other questions gets a fuller treatment in our piece on tenant isolation, and the same logic drives the comparison of private legal AI and public chatbots.
A checklist you can lift
For your next vendor questionnaire, in roughly the order the answers eliminate candidates:
- Report: Type II, current period, full copy provided under NDA, exceptions reviewed, CUECs assigned to named owners at the firm.
- Scope: the purchased product and its model endpoints appear in the audited system description; carved-out subservice organizations are identified and their reports requested.
- Data flow: a diagram covering ingestion, embedding, retrieval, inference, and logging, with regions named.
- Retention: durations and deletion mechanics for prompts, outputs, embeddings, and logs.
- Training: a contractual warranty that customer data is not used for training, fine-tuning, or evaluation.
- Subprocessors: current list, advance change notification, right to object.
- Testing: annual penetration testing that includes prompt injection and output exfiltration scenarios.
- Incidents: a notification SLA expressed in hours, aligned with your commitments to clients.
- Keys: customer-managed key support, with custody and revocation documented.
- Deployment: whether the platform can run inside your own tenant, and which controls become yours when it does.
None of this is adversarial. Mature vendors answer these questions quickly because they have heard them before. The ones hearing them for the first time are telling you, precisely and at no charge, where they sit on the security curve. The badge starts the conversation. The reading is the diligence.