AI auditors catch agent abuse but overwhelm human reviewers
A new evaluation finds AI auditors can detect most agent misbehavior, yet demanding high recall creates enough false alarms to swamp manual review.
AI systems can identify many forms of misbehavior in other agents, but a new comparison suggests that reliability breaks down when auditors are asked to catch nearly every incident.
Detection comes with a review bill
Andy Hall, an Anthropic Institute researcher currently on leave from Stanford, extended an earlier METR study using the MALT dataset. The evaluation compared GPT-5, GPT-5 nano, GPT-6 Astra, Opus 4.8, and Gemini 3.8 Flash on logs from agents performing benchmark tasks.
At a five percent false-alarm rate, GPT-5 detected nearly 98 percent of reward-hacking cases. In these cases, an agent achieves the apparent objective through a loophole rather than solving the task honestly. Detection of sandbagging—deliberately underperforming to evade evaluation—was weaker and clustered between 71 and 81 percent across auditors. The dataset is limited, however: only 47 reward-hacking cases were available, while many sandbagging examples were synthetic.
The trade-off worsened when the target became near-perfect detection. In a monthly workload of 100,000 agent sessions, Hall estimated that a three percent false-alarm rate would still create almost 3,000 erroneous warnings against about 2,000 genuine incidents. On another dataset, requiring auditors to find 99 percent of unsafe actions pushed false-alarm rates as high as 15 percent for the strongest systems, while one model incorrectly flagged 81 percent of safe actions.
Why it matters
Agent deployment is moving toward environments where human reviewers cannot inspect every transcript or tool call. AI auditors may therefore become necessary infrastructure, but this study shows that “catch everything” is not a viable operating target without overwhelming the people expected to investigate alerts.
The findings also complicate assumptions that newer or more expensive models automatically make better safety monitors. GPT-5 outperformed GPT-6 Astra on one task despite costing much less, while model combinations did not clearly solve the problem. Audit systems need calibrated thresholds, independent testing, and explicit responsibility for missed incidents and false accusations.