Explorations / Real systems
Reality is the dependency nobody declares
A clean agent meets a legacy desktop app with no API, a form that moved overnight, a PDF that is secretly a photo. The demo was never the hard part.
What it taught me
In productionEvery system that works in the demo meets a reality it was never told about.
Still open
How much of “AI doesn’t work in production” is the model, and how much is the world being messier than any training set?
From the bench
The model’s fine. The world isn’t.
The hardest integration I’m working on drives a legacy desktop app a human normally clicks through — no API, just a login screen. Every time it stalls, my instinct is to blame the model. Almost every time, the model was fine. The form moved overnight. A label changed. The “obvious” button is obvious to a person and invisible to a parser. The PDF I was extracting from turned out to be a photo of a document.
Production runs in a world that changes while you’re not looking.
What that costs
Build for the day the world moved
Most of “AI doesn’t work in production” isn’t the model failing. It’s the model meeting a reality nobody wrote down. The demo passes because the demo is a world you curated: clean inputs, a fixed layout, the happy path. Ship it, and it meets inputs no one anticipated, on a surface that updated itself since you last looked.
- No declared interface
- A lot of consequential software has no API. The contract is a screen drawn for human eyes.
- Silent drift
- Forms move, labels change, a field becomes required. Nothing tells you; the agent just starts being wrong.
- Inputs that lie
- A PDF that’s a photo, a number stored as text, a date in a format you’ve never seen. The world is not tidy.
The demo is a world you control; production is one that moves
Fig. 01 — The agent was fine; the world it met was never declared
the demothe world that movedSo the lesson keeps repeating: build for the day the world is different from yesterday, because it will be. Less “handle the input,” more “notice the ground moved and fail loudly instead of confidently.”
What’s still open
Permanent mess, or a closing gap?
The question I can’t settle: how much of this is the model, and how much is just that the world is messier than any training set? Some of it is clearly a modeling gap that better systems will close. But some of it feels permanent — reality is under no obligation to stay still or stay clean, and a good chunk of real engineering has always been absorbing that on the system’s behalf.
My working guess is that the messiness doesn’t go away; it moves. The frontier isn’t a model that finally handles a tidy world. It’s a system honest enough to know when the world stopped matching its assumptions — and to stop, rather than press on, confident, into a reality it was never told about.