The scoreboard is owned by the players
Something worth paying attention to: nearly every headline benchmark that AI companies cite when launching a new model was either created by the industry itself, heavily influenced by it, or has been so thoroughly optimised against that passing it signals very little about real-world capability. MMLU, HumanEval, MATH - these have been treated as objective measures of intelligence for years. At this point they’re closer to known quantities that labs train toward.
This isn’t a conspiracy. It’s a structural problem with no obvious villain. When a lab needs to evaluate its own model, it uses available benchmarks. When those benchmarks become famous, they attract training data that resembles them. When every frontier model scores above 90% on a test, the test stops being useful - but the press release still leads with it.
Goodhart’s Law, applied
Goodhart’s Law - roughly, that when a measure becomes a target, it ceases to be a good measure - has never had a cleaner case study than AI evaluation. OpenAI, Anthropic, Google DeepMind, and Meta all report benchmark performance prominently in their model release announcements. All of them also contribute to or sit on the advisory boards of organisations that design those benchmarks. The incentive structure makes rigorous, independent evaluation structurally difficult to sustain.
Arc Prize, which runs the ARC-AGI benchmark, has made a genuine effort to stay ahead of this. But even ARC-AGI - originally designed to resist pattern-matching - has been scrutinised for whether top models are approaching it through brute compute rather than the compositional reasoning it was meant to test.
What third-party testing actually looks like
MLCommons and HELM at Stanford represent more independent approaches, though neither has achieved the cultural weight of the numbers labs put in their own announcements. Chatbot Arena, run by researchers originally at UC Berkeley’s LMSYS group, uses human preference voting at scale - which has its own biases, but at least doesn’t hand the pen to the people being graded.
The problem is that independent evaluation is slow, expensive, and doesn’t ship on the same news cycle as a model launch.
So the marketing cadence wins. A new model drops, the lab posts a comparison table where their model leads every row, and that table gets screenshotted and shared before anyone has run the thing on a task they actually care about.
The benchmark is the ad. The ad looks like a benchmark. That distinction is eroding fast.