i14 launches Sugar Glider, constant-memory language models that serve 32 times more million-token requests
In its whitepaper i14 says Sugarglider-120B, built on Nvidia's Nemotron 3 Super, matches its parent on benchmarks while needing 1/32 of the memory at one million tokens; the results are self-reported.
Why it matters
Long contexts are expensive because a transformer's key-value cache grows with every token, which limits how many long requests a GPU server can hold. i14's whitepaper says Sugarglider keeps memory and per-token compute constant, and that on four H200s the 120B model fits 453 million-token requests against 14 for Nemotron 3 Super. It reports 73.3 on the 13-task RULER at one million tokens, which it calls the best result for any constant-memory model. The figures are the company's own; watch for independent tests and wider access.
Key facts
- Built on Nvidia Nemotron 3 Nano 4B and Nemotron 3 Super 120B-A12B
- 1/32 of the parent's memory at one million tokens
- 453 vs 14 concurrent million-token requests on four H200s
- 73.3 on the 13-task RULER at one million tokens
- Memory budget is set at inference time