The two gates
I build agents for production. The failures I see do not come from a lack of ambition. They come from the opposite: a system given more autonomy than the job needed, then shipped before anyone proved it could hold. So I stopped starting from the model. I start from two gates. The first decides what to build. The second decides whether it is ready to run. Nothing goes live until it clears both. I did not arrive at this from theory. I arrived at it from a system that nearly got shut down in its first day.
The weekend it almost died
A major US airline put a customer service agent into production. Within a day of go-live it was being pulled back toward shutdown. The failure was not a crash. The agent answered a policy question fluently, confidently, and wrong. It was reaching into the model’s parametric knowledge, the things it had absorbed in training, and answering from there instead of from the airline’s current approved policy. In a federally regulated business, a fluent wrong answer is worse than no answer, because fluency is what makes people trust it.
I spent that Friday night and the Saturday after inside the system. The fix was not more intelligence. It was less latitude. I forced every answer through retrieval, so the agent could only speak from the airline’s live approved source, never from what the model happened to remember. What it did not have grounding for, it did not say. The confident wrong answers stopped because the path that produced them was closed. Then I found an observability metric that had been lying about what the system was doing and fixed it at the source, because a number you cannot trust is worse than no number.
That weekend bought the time to do the rest properly. I split the work into delegate agents behind an orchestrator instead of one agent trying to hold everything. I drew a hard trust boundary around the federally regulated data so nothing crossed it by accident. I built the tools once and let every agent share them rather than rebuild. That agent now handles hundreds of thousands of automated customer actions a week and deflects roughly 20 percent of call-center volume. The full account is in the case study.
What the scar taught
The lesson was not add retrieval. It was that the failure and the fix lived in two different places, and I had confused them.
The agent almost died because of what it was allowed to do at run time. But most of what saved it was decided before any of it was built: whether to use one agent or several, where the trust boundary went, what the tools were. I had been treating those as one decision. They are two, and they happen at different times. So I started running two gates, one before I build and one before I run, and I built each one into something I can run again instead of relearning it over a broken launch.
The design gate: decide what to build
Most agent projects fail at a decision made before any code is written. Someone assumes the answer is an agent, because agents are the interesting thing to build, and reaches for the most autonomous version of it. The design gate refuses that. It asks a narrower question first: given this task, what is the least autonomy that does the job. A workflow. A hybrid. A single model call. One agent with tools. Or, only when the work genuinely demands it, several agents behind an orchestrator.
I turned that gate into a tool, agent, or workflow, with a Claude skill behind it. You describe the problem and it returns a verdict, and the verdict is deterministic code, not a model reasoning about itself in a loop. A routing decision you can settle with rules should be settled with rules, because rules do not drift and you can read them a year later and know exactly what they did. The airline system is multi-agent because the design gate said it needed to be, and I can show that reasoning rather than assert it.
A demo is not evidence.
The ship gate: prove it is ready
Passing the design gate means you built the right thing. It says nothing about whether the thing works. That is a separate question, and it is the one that gets skipped, because by ship time everyone is tired and the demo looks good.
So the second gate scores a built agent across seven dimensions before go-live: where its answers actually come from, what it is allowed to touch, whether you can see what it did after the fact, how it fails, and whether any of it holds under load. This is the agent production readiness diagnostic, with its own skill. Grounding is one of those dimensions. It is the check that would have caught the airline failure on a Tuesday afternoon instead of a Friday night. Every failure I fixed that weekend was a ship-gate dimension. None of them showed up in the demo. All of them showed up in production.
Why I work this way now
Two gates, in order. The design gate keeps me from building more autonomy than the job needs. The ship gate keeps me from trusting a thing I have not proven. One decides what to build. The other decides whether it is ready. Between them sits most of the distance between an agent that demos well and an agent that runs.
I did not name this after myself, and I am not going to. It is not a framework. It is two questions I now ask every time, written down so I ask them the same way each time. The airline system is the reason I ask them at all. Everything since has just been me refusing to relearn that Friday night.
The whole method, both gates plus a loop that turns incidents into regression evals, installs as a Claude Code plugin.