Google's new Physical-Engineering's Last Exam finds every frontier model scores below 50%
Built with Mercor, the 85-task benchmark tests agents on real mechanical, electrical, civil and structural engineering projects; the best mean score on the private set is 48.3%.
Why it matters
Google says frontier models now score above 90% on many academic benchmarks, but no existing test checked whether agents can produce safe, code-compliant designs end to end. PLE gives each agent a project folder averaging 61 files, including CAD drawings and BIM models, and awards points only if every hard safety check passes. The top configuration met 91% of individual constraints yet only 41.3% of its deliverables passed every check. Google found model choice mattered far more than the agent harness. Watch the open leaderboard at ple-bench.com.
Key facts
- 85 tasks: 44 expert-authored, 41 co-authored
- 14 frontier models tested on seven agent harnesses
- Best mean score on the private set: 48.3%
- Only 41.3% of the top agent's deliverables passed every hard check
- 29 public tasks released with grading tools
- Built with Mercor