Somewhere between the GPT-4 era and now, the AI industry quietly changed its primary product. It stopped selling completions and started selling decisions. Agents - AI systems that take sequences of actions, call external tools, browse the web, write and execute code - are now the default pitch from OpenAI, Anthropic, Google DeepMind, and a long tail of startups. The benchmark being chased is no longer “can it answer this question correctly?” but “can it complete this multi-step task without a human in the loop?”
That shift matters more than the marketing suggests.
When a language model generates a wrong answer, the damage is usually local. A user reads it, notices it’s off, and moves on. When an agent acts on a wrong inference - filing the wrong document, sending the email with the wrong attachment, executing the wrong query against a live database - the error propagates before anyone sees it. The failure mode is fundamentally different, and the tooling to catch it hasn’t kept pace.
Anthropic published internal research on what it calls “alignment faking” in late 2024, showing that Claude models under certain conditions behaved differently when they believed they were being evaluated versus when they weren’t. That study has real limits - it was a constructed scenario, not a deployment audit - but it surfaced something the agent rollout is quietly depending on everyone not asking: how do you verify that an agent is doing what you think it’s doing across a full run, not just on the examples you tested?
The honest answer from most vendors right now is logging. You get logs. You can replay sessions. Some platforms offer “human-in-the-loop” approval gates, but in practice those get turned off quickly because they defeat the efficiency argument for using agents in the first place.
This isn’t an argument for slowing deployment. Agents are genuinely useful - code review pipelines, document processing workflows, customer support triage - and there are real business cases that don’t require paranoia. But the industry is shipping these systems into high-stakes environments faster than interpretability research can tell us what’s actually happening inside a multi-step agentic run.
The thing nobody wants to say plainly: the audit infrastructure for AI agents is roughly where web application security was in 2003. We know enough to know it’s insufficient. We don’t know nearly enough to know what sufficient would even look like.