Cutting an enterprise agent from two minutes to fifteen seconds

An EU enterprise in aviation services had a stalled agent deployment that took one to two minutes per answer. Replacing a single mega-prompt with six routed skill prompts over an MCP tool layer cut that to roughly fifteen seconds and reduced prompt tokens by about 66 percent.

Problem

An earlier preview-stack deployment had not made it through scope and security review. Every request carried one mega-prompt with every MCP tool attached, which meant ten to fifteen thousand prompt tokens and one to two minutes of waiting for an answer. The client needed ITSM self service for several thousand internal users, and nobody waits two minutes to raise an incident.

Approach

I split the single prompt into six modular skill prompts behind a router, so each skill sees only the tools it needs. In front of the agent sits a Redis semantic cache, seeded with fifty to sixty of the questions people actually asked during requirements gathering. Common questions never reach the agent at all.

Architecture

Cache first, then a router, then one of four skills over a shared MCP tool layer covering incident creation, knowledge base retrieval, service requests, and ITSM FAQ with dynamic category and location resolution.

The hard part

Security was the reason the earlier attempt stalled, so guardrails came before features: PII redaction, prompt injection defence, and content filtering, on a fully private Azure network with no public exposure. The async backend adds rate limiting, circuit breakers, and per user token and cost tracking through Application Insights, which is what makes future per user usage caps possible rather than theoretical.

Outcome

Latency dropped from one to two minutes to roughly fifteen seconds, and to one to three seconds on cached questions, which are about sixty percent of traffic. Average prompt tokens fell from ten to fifteen thousand down to roughly five thousand. Shipped through Azure DevOps CI/CD, in pilot with fifty internal users, with a rollout target of several thousand.

Next

Automating Medicare claim payments at 95 percent accuracy