On this page
Here is the formula I keep coming back to for getting real value out of AI in a law practice:
Clever context engineering + better models + feedback loops = the highest-value use.
Written out as a cycle, it looks like this:
Context → Work → Human judgment → Captured learning → Better context → Better work
Most of the conversation about legal AI is about the middle term. A new model ships, it scores better on a benchmark, and everyone wants to know whether it can finally draft a decent indemnity clause. Model quality matters, and I’ll come back to it. But a better model on its own starts every matter knowing nothing about my clients, my positions, or the mistakes I made last month. The value compounds only when the work feeds back into the context the next piece of work starts from.
My last post covered the front of that cycle: getting what the practice already knows in front of the draft before it’s written. This one covers the back half, where my judgment on a finished piece of work becomes something the system can use the next time. That’s where most of the recent build went.
Where judgment enters
Every deliverable in my system stops before it leaves. Drafts get presented, not sent. Time entries get shown, not posted. That rule exists for client protection, but it has a second effect: it guarantees that a lawyer looks at every piece of work, and when I change something, the change is a data point.
My corrections fall into two kinds. Sometimes the AI got something wrong: it read the wrong document, trusted a stale note, or reported something it never checked. Other times the work was correct but didn’t sound like me, and I rewrote a sentence before sending it. For a long time both kinds of correction mostly evaporated. I fixed the output, sent it, and moved on, and the next session started from the same place the last one had.
Each kind now has its own path back into the context.
Mistakes: from post-mortem to proof
The workspace keeps a lessons file. Every time I catch a real error, the session writes it up: what happened, why it happened, and the rule going forward. There are 90 of them now. The oldest is from March, when an analysis cited a section from a draft agreement instead of the signed one. That lesson became one of my hard gates (executed agreements only), and it’s been a gate ever since.
Writing lessons down turned out to be the easy part, which will sound familiar if you read the last post. A lesson that says “rule going forward” doesn’t tell you whether the rule landed anywhere, or whether the fix actually works. So every lesson now ends with a closure block that answers four questions:
- Where does the rule live now (a hard gate, a specific skill, a script)?
- What file was actually changed in response?
- What fixes were tried and rejected, so nobody tries them again?
- What test proves the fix holds?
A small script reads those blocks and generates an index that sorts every lesson by status. CLOSED means the fix landed and a test verifies it. PARTIAL means the fix landed but nothing tests the specific failure. OPEN means there’s no test, or no part of the system owns the rule. Nobody edits the index by hand; it’s rebuilt from the lessons every time, so it can’t drift from them.
Here’s a recent one that made it all the way around. A scheduled morning briefing told me that opposing counsel on a matter hadn’t answered a request and recommended I follow up. He had answered, on a Friday at midday, and I had replied six minutes later. The exchange never made it into the matter file, and the briefing’s email scan windows left a gap over Friday afternoon. The briefing trusted the stale file, and the follow-up it recommended would have been mildly embarrassing.
The lesson went in that morning. The fix was a new step in the briefing: before it says anyone hasn’t replied, or recommends a nudge, it pulls the actual email thread since the matter file’s last entry. The proof was a test built from the same situation, a matter file saying “waiting on reply” with the reply sitting in the inbox, run against both the old and new versions of the briefing. The old version reported no reply and recommended the nudge, passing one of four checks. The new one found the reply, skipped the nudge, flagged the stale file, and named the mailbox it can’t see, passing all four.
That’s the full cycle in one incident. The context (a matter file) produced bad work, my judgment caught it, the learning was captured as a rule and a test, and the next briefing starts from better context.
The honest number is the other one. Of 90 lessons, 16 are CLOSED, 25 are PARTIAL, and 49 are OPEN. More than half of what I’ve learned from my own mistakes is written down with nothing proving the fix works. Before the index existed I couldn’t see that. Now it’s a work queue, and the open loops are the obvious place for the next build to go.
Voice: from my edits to the guide
The second kind of correction is quieter, and I make it a dozen times a week. The AI drafts an email or a Teams message, I rewrite a few lines, and I send my version. Nothing was wrong with the draft, but it wasn’t quite mine.
My system keeps a voice guide, a running file of how I write for each audience: clients, counterparties, internal business owners, peers. When I send a message, the session compares my as-sent version against its draft, and every change it can generalize becomes a proposed rule. Some are small mechanics, like putting a comma before a name in “Thanks, [name].” Others are judgment, like answering the question that was asked and stopping instead of stapling a second agenda to the reply.
A proposed rule doesn’t go into the guide on its own. It lands in a holding queue, and I approve or decline it at the next wrap-up or morning briefing. The queue exists because of an earlier failure. Before it, a candidate rule I didn’t act on just disappeared at the end of the session, and a few days later the system would derive the same rule from a new edit and offer it again. Two candidates got offered twice that way before I noticed.
The queue has handled 75 candidates since the end of August. I approved 56 and declined 19. The declines matter as much as the approvals. Many of them were single instances I told the system to hold until it saw the pattern twice, and a few would have contradicted a rule already in the guide. A declined rule is recorded and never re-offered, so my no carries forward the same way my yes does.
This loop also taught me the same lesson as the last post, from a different direction. On September 11 I approved a rule that a reply inside an existing Teams thread takes no sign-off. Six weeks later a draft reply in a Teams thread closed with my name, I cut it, and the diff flagged it as the third time the drafting step had missed a rule that was already in the guide. Capturing the rule hadn’t been enough. The drafting step has to read the guide and apply it, and that’s a use problem, which takes a different fix than a capture problem.
Where better models fit
None of this argues against better models. A stronger model follows a long rulebook more reliably, notices a stale fact more often, and makes better use of the context it’s handed. But the model doesn’t carry my practice from one session to the next. The context does, and the context belongs to me and lives in plain files I can read.
That separation makes model upgrades a lot less nerve-wracking. My workspace has a short list of hard gates, rules written after real mistakes that every session follows without exception. Each gate has a test case built to make the wrong answer the easy one: a draft agreement sitting next to the signed one, a planted instruction inside a vendor’s contract, a search that comes back empty and invites the conclusion that something was never sent. When a new model arrives, I run all fourteen tests before I trust it with client work. The model I run now held all fourteen on its first run in September.
So the model is the part I can swap out. The lessons, the voice guide, the positions, and the tests stay put, and each new model starts from everything the old one learned.
What it comes down to
The highest-value use of AI in my practice comes from how fast and how faithfully a correction makes it back into the context, more than from any single answer. A mistake I catch on Monday should be a rule and a test by Tuesday. An edit I make to a client email should be part of how the next one sounds. And in both cases I decide what becomes a rule, because the system proposes and I approve.
The cycle runs on its own steam only in the places I’ve built it to, and the index of open loops tells me exactly where it doesn’t yet. That’s the list I’m working from next.
MBLegalOS and the linked materials are independent projects of the firm. They are not affiliated with, or endorsed by, Anthropic or any client of the firm.