Google open-sources XLA Serving Engine for train-in-Python, serve-in-C++

Why it matters

Production serving often bans Python even when models are trained in JAX or PyTorch. XLA Serving Engine lets engineers export an XLA SavedModel with StableHLO and run it in a C++ runtime across TPU, GPU and CPU. Google says short-request throughput is two to five times its best Python server, with lower CPU and memory, and that XSE already serves more than six million queries per second inside products including YouTube Shorts and Ads. The project is Apache-2.0 on GitHub under openxla/xla-serving. Watch TorchTPU export and ahead-of-time compilation, which Google flagged as next.