GDPx Benchmark Puts Frontier Agents at Just 12%
SemiAnalysis’s new GDPx benchmark finds leading models complete only 12% of difficult professional tasks, exposing a wide gap between demos and dependable autonomy.
A harder test for autonomous work
SemiAnalysis has released GDPx, a benchmark designed to measure whether frontier models can complete extended professional assignments rather than isolated questions. The first version covers 206 tasks across 11 domains, including cybersecurity, biology, finance, legal work, chip design, materials science, software engineering, alignment and recursive self-improvement. Fourteen specialist partners helped design the tasks, which run inside simulated work environments with time limits and domain-specific grading.
Claude Opus 5.5 led the public leaderboard with a 12.0% score, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. The score averages performance across equally weighted domains and normally reflects three attempts per task. That makes the result a measure of broad reliability, not a best-case demonstration.
Where the systems break
GDPx reports especially weak results in finance and legal work under its binary grading. The benchmark also records failures that conventional answer-based tests can miss: agents may produce plausible analysis while violating task boundaries, fail to complete a required chain of actions, or optimize an intermediate result that does not survive independent evaluation. In one alignment task, models were given extensive traces and harness code yet struggled to identify why an earlier training run had stopped improving.
The benchmark is not a neutral sample of the entire economy. Its creators deliberately selected domains they expect to generate large economic value, and several tasks depend on specialized simulators and partner-designed grading. Scores should therefore be read as a demanding capability floor, not a forecast of every workplace use case.
Why it matters
The headline is less about which lab leads than about how little dependable completion remains after tasks become long, tool-heavy and independently checked. GDPx gives enterprise buyers a concrete counterweight to agent demos: frontier models can be impressive collaborators while still being poor unsupervised operators. Its planned updates every three to six months may also make it a useful yardstick for whether model progress translates into reliable economic work.
Uncle Cat take
GDPx’s 12% leader score matters more than the leaderboard order: eight-hour tasks expose unfinished work that short demos systematically hide.