The pitch for AI agents is straightforward: instead of prompting a model and reading its response, you give it a goal and let it act - browsing, writing, executing code, calling APIs - until the task is done. The gap between that pitch and production reality is where things get uncomfortable.
Salesforce, Microsoft, and ServiceNow all shipped agent-based products in 2025. Actual enterprise adoption followed, not just in controlled pilots but in live customer-facing workflows. And the operational question nobody had a clean answer to was: when an agent does something wrong, how do you even reconstruct what happened?
The Audit Problem Is Worse Than It Looks
Traditional software has logs. An agent taking ten sequential actions across three external services generates something closer to an improvised chain of decisions, and the trace of that chain depends entirely on what the vendor chose to record. Right now, that’s inconsistent at best. Some platforms expose structured logs with intermediate steps; others surface only the final output. When an agent misclassifies a support ticket, routes a payment to the wrong account, or drafts an email that goes out before review, reconstructing the decision sequence is often closer to forensic guesswork than debugging.
This isn’t a hypothetical edge case. It’s the normal failure mode for any sufficiently complex automated system, and agents are more complex than most.
The Guardrail Framing Is Doing a Lot of Work
“Guardrails” has become the industry’s favourite way to gesture at safety without specifying anything. Vendors list it as a feature. Procurement teams check it as a box. But a guardrail that prevents an agent from accessing certain file paths is not the same thing as a mechanism that explains, after the fact, why the agent made the choice it did. Containment and interpretability are different properties, and conflating them is creating a false sense of oversight.
LLM observability tooling - companies like Langfuse, Arize, and Weights & Biases with their LLM monitoring products - is genuinely improving. But it’s largely being adopted by ML teams, not the operations or compliance teams who will actually be on the hook when something goes wrong.
The Deployment Incentive Is Running Ahead
The commercial pressure to ship agents is real and not irrational. Productivity gains from well-scoped agents in narrow tasks - document summarisation, ticket triage, form extraction - are demonstrable. The problem is that “well-scoped” keeps drifting once agents are embedded and extended.
What’s less clear is whether the organisations deploying these systems have any shared understanding of what acceptable failure looks like, or who owns it when it happens. That’s not a technical question. It’s an organisational one, and most companies haven’t answered it yet.