Memory offloading

Technique · Architecture & Efficiency · 1 item since Apr 2026

Moving model state between GPU and CPU memory so large models train on less hardware.

Organizations: Lehigh University University of Notre Dame

Trends

Efficiency methods cut the compute and memory models need

Items

Apr 6
MegaTrain trains 100B+ models on one GPU
Notre Dame and Lehigh keep all parameters and optimizer states in host memory and stream them through the GPU, training models up to 120B parameters on a single H200 with 1.5TB of host RAM, at 1.84x the throughput of DeepSpeed ZeRO-3 offload at 14B scale.