Field notes / 2026 · AI systems
I bet a compliance migration on a model
The cleanup nobody would fund with humans was the one that mattered most. The only way to afford it was to let an AI do work I’d normally never let it touch.
Live question
When the work is too big for humans and too sensitive for a model, which fear do you let win?
The most important cleanup in the codebase is the one no one will pay for. Account scoping, making it provably impossible for one user’s data to surface in another’s session, a child’s, a teen’s, an incognito tab, is exactly the kind of work that gets deferred forever, because it adds no feature and the cost is enormous. Estimated by hand, it ran to dozens of engineer-years. That headcount doesn’t exist. It never will.
So I proposed doing it with an internal LLM tool, fine-tuned on our own model, running the migration at a scale humans couldn’t afford. The pitch had an obvious internal contradiction I had to sit with. The reason this migration matters is that it’s safety-critical (get a boundary wrong and you’ve leaked a child’s data), and the thing I was proposing to entrust it to is a system that does not understand boundaries, only patterns that resemble them.
The model could write the change. It couldn’t care whether the change was right, and the whole program was about caring.
I’d watched the other failure mode for years: the demo that dazzles and then falls apart the moment real conditions arrive. So I refused to treat the model’s output as done. The bet wasn’t ‘let the AI do it.’ The bet was that the AI could do the volume, the thousands of mechanical, error-prone edits, if I spent the saved time building the thing that decides whether each edit is safe. Verification was where I spent the savings. The model writes; the harness, the tests, the human review on the boundary cases decide.
That reframing is the part I’d defend. Targeting a large cut to the cost only counts if the cut comes from the labor that was always tedious, never from the judgment that was always the point. The moment you let the savings come out of scrutiny instead of toil, you haven’t modernized the codebase. You’ve automated the introduction of the exact bug the project exists to prevent.
What’s still live for me is calibration. I can review the boundary cases I think to look at. The model is most dangerous precisely where it’s most fluent — where it produces a change so plausible that nobody flags it for review. I don’t yet have a reliable way to find the confident, wrong edit hiding inside ten thousand confident, right ones. Until I do, I’m treating every claimed efficiency as a hypothesis, not a result — and I notice how badly I want to believe the number.