FrontierPhysics benchmark: no AI model passes more than 46 of 168 research-task runs
Forty-two physicists wrote 56 tasks across nine fields; 22 models in 26 configurations were tested, and the task set is public.
Why it matters
FrontierPhysics asks agents to finish end-to-end physics research tasks rather than quiz-style answers. In 168 runs, no model cleared more than 46 tasks, according to the arXiv preprint. That gap shows current agents still fail long scientific workflows even when they score well on short benchmarks. The paper sits with a wave of harder science evals that puncture leaderboard optimism. Watch follow-on leaderboards, which labs attempt the suite, and whether tool-using agents close the gap.
Key facts
- arXiv:2611.06910
- 168 evaluation runs
- No model passed more than 46
- End-to-end physics research tasks
- Preprint, not peer reviewed