Google publishes the framework it uses to test its AI models for health accuracy, safety and usefulness
A whitepaper written with Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran combines closed-question benchmarks, autoraters and expert human review; Google gives no new performance scores for its own models.
Why it matters
Health questions are one of the main things people bring to chatbots, yet there is no agreed way to test whether answers are safe. Google's paper maps evaluation to stages of a product's life, from base model to post-deployment monitoring, and to three domains: accuracy, safety and usefulness. It is a method, not a results table, so outsiders still cannot compare Google's models with rivals'. Watch whether health systems, procurement teams or regulators adopt it, which the authors say is the aim.
Key facts
- Three domains: accuracy, safety, usefulness
- Methods: closed-question benchmarks, autoraters, expert review
- Written with Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran
- Paper posted to arXiv on 3 Nov 2026