Evidence for Reform

Context

In the US, if you get arrested and you cannot afford a lawyer, the state has to give you one. That's the law. We've all watched enough TV to be familiar with the Miranda Warning:

"You have the right to remain silent. Anything you say can and will be used against you in a court of law. You have the right to talk to a lawyer for advice before we ask you any questions. You have the right to have a lawyer with you during questioning. If you cannot afford a lawyer, one will be appointed for you before any questioning if you wish. If you decide to answer questions now without a lawyer present, you have the right to stop answering at any time."

(Cue the obligatory Law & Order's bum bum )

Each state has their own way of handling these cases. Now, you may have heard of public defenders across the country being so inudated with cases that they're unable to fairly represent their defendants. Most of that has to do with policy and budget failures, but technology also plays a role. Lawyers and other admin are forced to dedicate attention to poor systems, and that can take away from their capacity to represent their defendants.

That's where our work comes in.

The US Digital Response brought in two researchers to assist Nevada's Department of Indigent Defense Services (DIDs) in modernizing their tools.

Problem

To do its job, the agency needs a computer system. Something that tracks every case, every lawyer, and every hour billed.

The system they had, LegalServer, was built for a different kind of law, not criminal defense--but they used it because that's what was available.

Everyone who worked for DIDs knew something was wrong. They just couldn't say what was wrong. All they knew was that everything took too long, reports were unbelievably painful to build, and people had to keep fixing the same problems by hand (pen and paper).

Before they spent money on a new system, they wanted to know:

• What was actually broken?

• What would this new system have to do fit the needs of its users?

Here's the seven-phase process we followed throughout this project:

1. Absorb. Before talking to anyone, I read everything. The consent decree. The statute the agency reports under. Their past reports to the legislature. I wanted to understand what the agency was legally required to prove before I asked anyone what was hard about their day.
   

2. Kickoff. I use this time to gather important information from the stakeholders. Questions like 'What does success look like?' and 'Who can help us best understand the current system?'
   

3. Wall of knowledge. Everything we already knew went up in one place. What the system does. Who touches it. What the agency has already tried. This is where you find out how much of what "everyone knows" is actually true.
   
   

   
   ↗ Click to expand

   
   

4. Research. Five sessions. Twelve staff. Every role that touches the system, from the people who open cases to the person who cuts the checks.
   

5. Synthesis. We mapped every workflow, step by step, and marked every place the work snagged. That produced six journey maps, fifty-seven phases, and three hundred twenty-three friction points. The number is not the finding. The pattern underneath it is.
   

6. Alignment. We brought the maps back to the staff who gave us the information. Did we get this right? What did we miss? Nothing goes into a final report that the people who do the work have not seen first.
   

7. Recommendations and handoff. We turned the findings into thirty user stories, ranked not by how annoying a problem was, but by what happens if it fails. Then we handed the whole thing over in a form the agency could use without us.

What we did

Five interview sessions. Twelve people. Every role that touches the system.

We started at the top and worked down.

• The executive director

• Two deputy directors

• The person who watches the budget

• The person who administers the system

• The attorney who handles oversight

• The coordinators who open cases and assign them

• The staff who take in bills and review them

• The person who pays them out.

We watched instead of only asking

Most of these sessions were screen shares. People opened the system and did their job while we watched.

When you ask someone how they do their work, they tell you the process they are supposed to follow. When you watch them, you see the one they actually use. The gap between those two is where the real problems live.

We were able to catch some really interesting workarounds and obstructions like:

• The spreadsheet someone keeps on the side because the system cannot produce the report she needs

• The email folder that is functioning as an audit trail

• Someone opened four screens to answer one question.

Nobody had told us about any of that, because none of it registers as a problem to the person doing it. It is just Tuesday for them. That's why behavioral data is so important in cases like this.

What came out of it

We mapped six journeys.

1. Case opening and assignment

2. Time entry and billing review

3. Post-conviction and prison payments

4. Reporting and compliance

5. System administration.

6. Expert and Investigator pay

Inside those six journeys were 57 distinct phases of work. Inside those phases, 323 points where the work snagged.

The number is not the finding

323 sounds like a lot. It is a lot. But a friction count is not an argument. Any sufficiently complicated system will generate a big number if you look closely enough.

The finding is what those 323 points had in common.

They were not scattered. They clustered. And when we looked at what the clusters were made of, almost all of them traced back to the same root: work happens somewhere other than the system, and the system finds out about it later, if at all.

Essentially, it was the same problem in 323 different costumes.

The system could tell you where a case was right now. It could not tell you how it got there.

Think about the difference. A photo shows you one moment. A video shows you what happened. The agency had a photo. The court was asking for a video.

That is the whole problem. Everything the agency has to prove is about what happened over time. Did this person really get defended? Did that lawyer really do the work? A system that only stores "here is where things stand today" cannot answer those questions.

So the staff rebuilt that history by hand. Every three months. From memory and old emails.

I. They had a filing cabinet, not a case management system

This is the most important thing we found--the workd doesn't happen in the system, it happens somewhere else. Then somebody types a version of it into the system afterward so there is a record.

So where does the work actually happen? In email. On paper. In spreadsheets. In whatever software the lawyer already uses. The system is just where evidence of that work gets dropped off later by hand, and not even all of it.

That is not a case management system. That is a filing cabinet with a login screen.

II. It knows where things stand. It does not know what happened.

This part took us longer to see clearly. Ask the system about a case and it can tell you a few things. What its status is. What kind of case it is. Which lawyer has it.

Ask what happened along the way and it has nothing.

Were the charges changed? When? Did the case move to a different court? What did the lawyer actually do between the first hearing and the plea? None of that is in there.

The system was never built to record change. It was built to record a condition. When a case ends, somebody picks one option from a dropdown menu. That is the whole story the system keeps about how it finished.

III. Why that gap is fatal here

For most organizations, this would be annoying. For this one it is the entire problem, because everything the agency has to prove is a claim about a sequence of events.

A federal court wants to know whether people got a real defense. The legislature wants to know whether the money bought anything. These are questions about what happened over time. Their current system cannot help them answer that question.

The agency cannot answer either question from a system that only knows about right now.

So somebody rebuilds the proof by hand, over and over. It's not that the agency is failing to keep records. It's that its keeping the wrong kind of record.

IV. The bind underneath it

Two things are true at once. Both are reasonable. The system cannot satisfy both.

The agency needs a complete record. As far as a court is concerned, if it is not in the system, it did not happen.

Lawyers will not type the same thing twice. They already keep their own records. The system gives them no way to bring those records in. So it asks them to do the work over again. Fewer than half of them do.

V. What that meant for what we told them

The agency came to us with one question. Should we fix the system we have, or buy a new one?

Our research changed the question. Neither answer matters if the new system asks for the same thing twice.

They didn't need a prettier screen. What they need is a system that records the work as the work happens, instead of asking someone to write it all down again afterward. That is a different kind of purchase. And a very different conversation to have with the companies selling these systems.

I. What we recommended

The agency came in with one question: fix what we have, or buy something new?

The research changed the question. Before either answer makes sense, one requirement has to be settled: whatever the agency ends up with cannot ask attorneys to enter their work twice. If it does, the outcome is the same regardless of which path they take.

With that settled, the recommendations sequence in three tiers.

First: stop the harms that cannot be undone

Nothing else on this list matters as much as this.

The system must not allow a case to be assigned to an attorney who is not qualified to take it. Right now the check depends on a category that a person types in by hand, which means the check is only as good as the typing. That has to change. The gate has to hold even when the data is wrong.

The same is true of conflict detection. It has to catch the conflict before the assignment, not months later during an audit.

These are not efficiency improvements. They are the difference between a system that protects people and one that does not.

Second: make compliance provable

The agency has to be able to show what happened, not just what is.

That means the system records change. When a charge is amended, when jurisdiction moves, when an attorney is substituted, the system should know, and it should know without anyone having to remember to tell it.

It also means the quarterly report stops being a construction project. If the agency has to rebuild its evidence by hand every three months, the evidence is only as reliable as the person doing the rebuilding, and there is no way to audit it.

Third: reduce the time tax

This one seems pretty self-evident.

Fewer screens. Reports that run themselves. Search that works. The daily friction that makes this job harder than it needs to be.

II. Two things procurement will not solve

Both of these will still be true the day the new system launches.

The knowledge is in people, not in the system. Critical parts of how this agency works exist in one person's head. What the categories really mean. Which counties do things differently. Why that field is filled in that way. If those people leave, the knowledge leaves. No software purchase fixes this. It has to be written down, and that is a decision the agency has to make on its own.

The data-entry compact is broken. Attorneys were asked to do double work and they declined. Whatever replaces the current system arrives into that history. Trust has to be rebuilt, and it will not be rebuilt by an interface. It gets rebuilt when the agency asks for less and gives back more.

III. What we handed over

Thirty user stories, ranked by consequence. Six journey maps. A friction inventory covering every phase of every workflow. And a baseline: what the agency looks like today, so it can tell whether whatever it buys next actually helped.

That last one matters more than it sounds. Most organizations replace a system and never find out if it worked, because they never measured what they had.

What came of it

The engagement ended with a handoff, not a launch. That is worth saying plainly, because the timelines here are not software timelines. Government procurement takes years. Eight weeks of research does not end with a new system.

What it can end with is an agency that knows what to buy, and knows why.

The agency said the work met their needs

In their closing feedback, the department rated the engagement 5 out of 5 on whether the results met their needs, 5 out of 5 on their confidence to continue the work without us, and 5 out of 5 on the change in their capacity to do it.

The confidence rating is the one I care about. A research engagement that leaves an organization dependent on the researcher has not built anything. The point was to hand them something they could use after we were gone.

The unexpected outcome was internal

The most interesting thing the agency said had nothing to do with the deliverables.

They told us that the interviews themselves changed how their team talked to each other. Staff who had been working in separate parts of the same process started to see how their piece connected to everyone else's. Their director described it as galvanizing, and named it as the capacity change that mattered most.

That was not something we designed for. But it makes sense in hindsight. When you map a workflow end to end and show it to the people inside it, you are showing most of them their own work in context for the first time. Nobody had drawn the whole thing before.

The billing reviewer had never seen what case assignment actually involves. The person who opens cases had never seen what happens to the data downstream. The map was the first shared picture of the service they all run together.

What they said they would do next

They described the deliverables as an interim discussion document. Something to work through in stages rather than act on all at once.

They plan to keep meeting. They plan to tackle the findings iteratively. And the director connected it back to the mission in a way that stuck with me: better data means better support for the attorneys, which means better defense for the people relying on them.

What I don't know

Whether they replace the system. Whether the double-entry requirement makes it into a procurement document. Whether the qualification gate gets fixed.

I would like to tell you this research changed a system. It has not, yet, and it may not. What it did was give a small department, with no research function of its own, a clear and defensible account of its own operation, and enough confidence in that account to act on it.

This endeavour is what discovery research is for AND it was free.

Team: US Digital Response

Co-Researcher: Elizabeth Cedar Louis
Project Manager: Meghan Marie Fowler-Finn