The benchmarks are losing the race. Anthropic, OpenAI, and Google have all shipped agentic products in 2025 and 2026 - systems that don’t just answer questions but take multi-step actions across tools, browsers, and APIs. The problem is that the standard evaluation frameworks were designed for models that sit still and respond. Agents move, and movement breaks the rubric.
Take something like WebArena, one of the more credible benchmarks for evaluating agents acting in browser environments. It was a meaningful signal when it launched. But once labs start optimizing their training pipelines against a specific benchmark, the score stops telling you what it originally measured. This isn’t a new observation - Goodhart’s Law has been embarrassing ML benchmarks for years - but the speed at which agentic evals are getting saturated is new. The gap between benchmark performance and real-world reliability has become genuinely difficult to interpret.
The deployment gap
What makes this worse is that agents are being deployed in production contexts - customer support pipelines, internal tooling, code review automation - where the failure mode isn’t a wrong answer on a test. It’s a wrong action in a system that doesn’t have an undo button. A model that hallucinates a citation is annoying. An agent that misreads a file path and overwrites production data is a different category of problem.
Enterprises adopting these tools are largely evaluating them the same way they evaluated software a decade ago: pilot programs, internal audits, gut feel from the team that ran the proof of concept. That’s not cynicism - it’s the only practical option when the tooling to do better doesn’t exist yet.
What’s actually being built
A small number of companies are working on agent-specific evals that test for things like task abandonment behavior (does the agent stop when it should?), error recovery, and scope adherence. This work is happening, but it’s scattered and mostly unpublished. The AI safety community has been thinking about some of these properties for years under different vocabulary, though the operational focus - not “will this agent cause harm in theory” but “will it stay inside the box a sysadmin draws for it” - is still underserved.
The labs shipping these products know the eval problem exists. The honest ones say so. But shipping has its own momentum, and the enterprise demand for automation is real enough that the evaluation infrastructure is being built after the fact, in the field, by the people already depending on these systems.