ByteDance Seed releases Cola2, a 9B continuous-latent language model on Hugging Face
Model card reports training on 346.03B source tokens and a 60.10 eight-task average while the latent DiT remains unconverged; weights require trust_remote_code.
Why it matters
ByteDance Seed published Cola2 on Hugging Face on November 6, 2026: a roughly 9B-parameter base model that compresses text into continuous semantic latents, generates latent blocks with a Diffusion Transformer, then decodes to tokens. The card reports a 60.10 average across eight tasks after 346.03B source tokens and notes the latent DiT is still unconverged. Authors frame the release as an architecture and efficiency probe rather than a leaderboard chase; Llama-3-8B’s reported average is higher after far more pretraining tokens. Inference needs Transformers remote code and is nondeterministic without a seeded generator.
Key facts
- Hugging Face repo ByteDance-Seed/Cola2 created 2026-11-06T06:59:36Z
- ≈9B params; trained on 346.03B source tokens
- Reported 60.10 average across eight evaluation tasks
- Architecture: compression VAE + block-causal DiT + token decoder
- Requires trust_remote_code; base model, not instruction-tuned