CellType's BioSimBench finds frontier models predict only part of how cells respond to drugs and knockdowns
The benchmark scores predictions against held-out lab experiments; CellType says its own 27B model trained in BioSimGym beats the best frontier model on every task.
Why it matters
Most biology benchmarks for language models test written knowledge; BioSimBench asks models to predict which genes go up or down after a drug, gene knockdown or immune signal, and grades them against measured lab results, with agreement between repeat experiments as the ceiling. Its five tracks share one leaderboard. The benchmark is built by a company whose own model tops it, so independent submissions will matter. Watch which labs submit models and whether the gap to experimental reproducibility narrows.
Key facts
- Five tracks, one leaderboard
- Tasks: drugs, gene knockdowns and immune signals
- Graded against held-out lab experiments
- CellType's 27B BioSimGym model beats the best frontier model on every task
- Models trained on NVIDIA Blackwell GPUs