How to prepare a legacy codebase for agentic engineering - Part 1

Most teams prepare a legacy codebase for AI agents by cleaning it. We tried that. Here is what we found when we first measured it.

Plenty of teams want to point AI coding agents at their real systems. The years-old codebase that actually runs the business.

They first just use agents on the unmodified codebase and are surprised that the results are subpar and mirror the mistakes of the existing code. Then the next instinct is nearly always the same. The legacy code is messy, so let’s clean it up before we let an agent loose in it.

We also started with that. It turns out to be half right, and the half that is wrong is the half most teams start with. Here is what we found out. The next post in this series is about what to do instead.

What does "agent-ready" mean

Code that compiles is not enough. Generated code also has to follow the rules your team lives by. Security. Data integrity. Architectural boundaries. The rules that keep a codebase safe to ship. So the question for an old codebase is not “is this code clean?” The question is “will an agent working here follow our rules?” That second question has an answer you can measure.

Turning it into a number

We took the engineering rules we always apply and built a small eval consisting of a fixed set of coding tasks. Each task scored on one thing: did the agent follow the rule that applies here? We ran it in two modes. Writing brand-new code, and extending modules that already exist. The tasks are frozen, and that is the point. The same prompts run before a change and after it, so the scores can be compared. Without that you only have opinions about whether anything got better.

The failures were not random

Take the brand-new-code runs first. Most rules passed on the first try. Five out of our seven, every time. Two failed again and again. Those two were not a random pair. They were the rules that describe how the system is put together, not how a single line of code should look. Things like “every endpoint that touches tenant data has to check per-tenant access”, and “don’t wrap two separate databases in one transaction.” Those numbers only hold for new code. When the agent extended a module that already existed, the score stopped tracking the rule and started tracking the code around it. That turns out to be the whole story of part 2, so park it for now.

Five things the hard rules have in common

We tried to look at the two rules which constantly failed and here are five hypothesis why they failed - we can’t verify any of them, but here they are:

They fail by leaving something out. Most rules ask for a visible action that the task already hints at. Validate this input. Delete this record. The hard rules ask the agent to add something nobody requested, to close a gap you cannot see in the diff. Models are good at doing the thing you asked for. They are weak at remembering the thing you did not ask for.

They need knowledge of the whole system. You can satisfy an easy rule by reading one file. A hard rule needs the architecture in your head. A multi-tenant data model. Two separate datastores. None of that is in the task description.

They get applied by feel, not by principle. The agent guarded the endpoints that sounded dangerous. Admin screens. Deletes. It skipped the routine reads and list views. That is pattern matching on surface cues, not applying the underlying rule.

The common default is your anti-pattern. Public code shows authentication almost everywhere. It rarely shows the app-specific authorization that has to come after it. So for a routine endpoint, the model’s instinct is the exact thing you don’t want.

There is no single fix to copy. The right mechanism was different in every case. And the hardest one was not written down anywhere.

If you want to guess which of your rules an agent will break, look for these five traits. You can usually spot them before you run anything.

Where to start

Build the small eval before you start any cleanup. It costs far less than a cleanup sweep, and it turns a vague worry into a number you can move.

You will probably find what we found: most of your rules are fine, and one or two are not. That is a much smaller problem to solve than “our codebase is messy.”

Next: we hand-cleaned a large batch of old violations, re-ran the eval, and the agent’s new code did not improve at all. Part 2 covers why, and what actually worked.