NVIDIA open-sources Reshard collective to speed RL weight refit up to 5.7×
![[object Object]](https://developer-blogs.nvidia.com/wp-content/uploads/2026/11/green-cube.webp)
NCCL M2N Reshard moves only intersecting shards between training and inference layouts; NeMo RL benchmarks show 2.7–5.7× vs gather-and-broadcast on H100 and GB200.
Why it matters
NVIDIA described Reshard, a new collective that treats RL weight refit as a DTensor-to-DTensor transfer so only the theoretical minimum shard intersections move between training and inference layouts. An open NCCL M2N Reshard reference groups NVLink-domain replicas to cut redundant network copies. In NeMo RL on DGX H100 and GB200, Reshard cut refit latency 2.7×–5.7× versus gather-and-broadcast across models including Qwen 3 235B and DeepSeek v3 671B, with a hybrid path keeping small non-FFN weights on the old route.
Key facts
- New Reshard collective for DTensor-to-DTensor refit
- Open NCCL M2N Reshard reference implementation
- 2.7×–5.7× faster than gather-and-broadcast in NeMo RL
- Benchmarked on DGX H100 and GB200
- Hybrid path: FFN projections via Reshard; small weights legacy
- Aimed at trillion-parameter RL post-training payloads