Multi-agent customer operations platform

A production customer service agent was a day from being shut down. I found the failure, closed the path that produced it, and designed the delegate-agent architecture the platform runs on now.

Overview

A major US airline launched a production customer service agent to deflect call-center volume across loyalty, baggage, and back-office requests. I led the engagement as the forward deployed engineer: finding the failure that nearly ended the launch, then designing the delegate-agent architecture it now runs on.

The turning point

Within a day of launch, the agent gave customers guidance that no longer matched a policy the airline had recently changed. The error reached real customers, and the executive team moved to pull the platform out of production. I stepped in over a Friday night and Saturday to find the failure and fix it before the decision became permanent.

The cause was not a weak model. The agent was answering from what the model already held rather than from the airline's current, approved knowledge. The fix was to force every answer through retrieval, so the agent could only respond from sources the airline controlled and kept current.

Decisions

• Ground every answer in retrieval, not the model's own knowledge. The go-live failure traced directly to the agent answering from parametric knowledge. I enforced retrieval-grounded responses so the agent could only draw from approved, current sources.
• Split one agent into delegate agents behind an orchestrator. Instead of a single monolithic agent, I designed specialized delegates for loyalty and back-office requests, coordinated by an orchestrator, so each domain could be governed, tested, and scaled on its own.
• Draw an explicit trust boundary for regulated data. I authored the trust-boundary analysis for federally regulated customer data moving through the platform, and set the deployment path that kept that data inside the required controls.
• Build tools once, share them across agents and partners. I defined a reusable tool strategy so capabilities were implemented a single time and shared across delegates and SI partners, rather than rebuilt per agent.
• Fix the observability gap at the source. When the analytics layer produced unreliable KPIs, I built the query and computed the metric directly so leadership had a trustworthy read on production.

Outcomes

• The program survived the go-live escalation and stayed in production. The weekend grounding fix reversed a decision that had already been made to shut it down.
• In production, the platform handles hundreds of thousands of automated customer actions a week and deflects roughly 20 percent of call-center volume.
• I co-authored the roadmap and statement of work for the next-generation agent architecture, now in flight.
• I became the technical point of contact leadership escalated to for the hardest production and trust decisions on the account.

What carried forward

The incident is a compact version of why enterprise agents fail in production. Not because the model is weak, but because it is allowed to answer from the wrong source. Reliability comes from grounding answers in current, approved knowledge and enforcing trust boundaries deterministically in the request path, not from trusting the model to police itself.

The approach it taught me is in the two gates.