Auditors barely beat chance at spotting sabotaged AI safety research, Redwood-led study finds

A human expert played saboteur across AI safety experiments; human auditors reached an AUROC of 0.59 and the best model 0.73, a gap the authors call within noise.
Why it matters
Labs increasingly plan to hand alignment research to AI agents, which makes undetected sabotage by a misaligned model a central control problem. This study tests that worry end to end, with a human expert standing in for the saboteur across 10 new safety experiments. It follows Redwood's earlier Auditing Sabotage Bench, where the best auditor reached an AUROC of 0.77 on sabotaged codebases. Watch whether labs adopt such audits before delegating safety research to agents.
Key facts
- Human auditors: AUROC 0.59, not significantly better than chance
- Specific sabotage fully identified in 17% of cases
- Best model, Fable 5.1: AUROC 0.73, 32% full identification
- 25 human auditors, 104 audits, 90 minutes each
- 10 benign and 12 sabotaged research traces