How to measure whether a legal AI tool is actually saving time

Every legal AI business case I’ve seen, including a few I’ve written, rests on the same claim: this tool will save your lawyers time. And almost every firm that buys on that basis then fails to check whether it did. Six months later someone asks whether the investment paid off, and the evidence consists of a vendor slide, a handful of enthusiastic comments from the pilot group and a usage chart showing that people logged in. None of that tells you whether anyone got their evening back. Measuring time saved properly isn’t hard, but it does require deciding to do it before the tool arrives, and being honest about where the time actually goes.

Ignore the vendor’s number

Start by setting aside whatever figure appeared in the pitch. Vendor time-saving numbers are usually measured on the task as the tool sees it - the minutes between uploading a document and getting an output - in conditions chosen to flatter it. That isn’t a lie, but it isn’t your number. It leaves out everything around the tool: finding the documents, getting them in, checking the output, correcting it and turning it into something a client can read. In legal work, those surrounding steps are often most of the job.

Your number has to be measured on your work, by your people, end to end.

Measure the task, not the tool

The most common mistake is trying to measure a tool in general. “Has the review tool saved us time?” is unanswerable. “How long does it take to produce a first-draft lease report for a portfolio acquisition?” is not. Pick two or three specific, repeatable tasks the tool is meant to help with, define where each one starts and ends, and measure the whole thing.

Crucially, you need a baseline. Before the tool goes live, time those tasks done the existing way, on real matters, by the people who will eventually use it. It doesn’t need to be a scientific study. A few weeks of honest recording across a handful of matters is enough to give you something to compare against. Without it, every later claim is guesswork dressed up as a result.

Count the checking time

This is where most measurements quietly fall apart. A lawyer using AI output doesn’t stop at the output. They check it - and they should, because their name is on the work. If checking a tool’s answer takes as long as producing the answer by hand, the saving is zero, however fast the tool was.

So measure verification explicitly. How long does a lawyer spend confirming, correcting or redoing what the tool produced? Is that falling as they learn which outputs to trust, or staying stubbornly flat? This is also the most useful diagnostic you’ll get, because high checking time usually points to a design problem: answers without sources, confidence without calibration, output in the wrong shape. Those are fixable, but only if you can see them.

Follow the time

Time saved in one place has a habit of reappearing somewhere else. The trainee finishes the first draft faster, but the associate supervising them now spends longer reviewing it because they don’t trust the process. The deal team moves quicker, but the knowledge team spends hours each week maintaining the tool’s templates. The work isn’t gone; it has moved upstairs or sideways, to where nobody was measuring.

The only way to catch this is to measure across the whole chain of people who touch the task, not just the person using the tool. If the total time across everyone involved hasn’t fallen, the tool hasn’t saved anything. It has redistributed effort, often towards more expensive people.

Use the matter record, not the survey

Surveys are tempting because they’re cheap, and close to useless because they measure enthusiasm. People who like a tool report that it saves them time; people who don’t, don’t. Neither tells you much.

Far better evidence already exists in the systems firms run on. Time recording, for all its flaws, will show whether hours on comparable matters are falling. Turnaround times show whether clients are getting work back sooner. Write-offs show whether time is being recorded and then not billed, which is often where inefficiency hides. None of these is perfect on its own, but together, set against your baseline, they tell a story that’s much harder to argue with than a survey.

Then ask what happened to the time

Even a genuine saving only matters if something good happens with it. Did quality hold, or did errors creep in that will cost more later? Did the time go into better work, more matters or simply fewer late nights - any of which can be a win, provided you know which one you’re getting? And the question I’d weigh above all the others: when the pressure is on, do people still choose the tool, or do they revert to doing it by hand? Voluntary use under pressure is the clearest signal that the saving is real, because nobody reaches for something that slows them down.

Time saved is the most quoted benefit in legal AI and the least verified. It doesn’t have to be. Pick the tasks, take a baseline, count the checking, follow the work wherever it moves and judge the result from the matter record rather than from opinion. It’s a modest amount of discipline. And it’s the difference between a tool that feels like it’s working and one you can show is.