Speculative decoding
Technique · Architecture & Efficiency · 3 items since Apr 2026
A small draft model proposes tokens that the large model verifies in one pass, speeding generation.
Related
- Organizations
- Techniques
Trends
Items
- Jul 10
NVIDIA's Nemotron-Labs-Diffusion
A tri-mode language model unifies autoregressive, diffusion, and self-speculation decoding; the 8B variant decodes ~6x more tokens per forward pass than Qwen3-8B at comparable accuracy, for ~4x higher throughput on a GB200.
- Jul 10
DeepSeek's DSpark speculative decoding
A semi-autoregressive drafter with confidence-scheduled, load-aware verification speeds per-user generation 60-85% at matched throughput in DeepSeek-V4's live serving system.
- Apr 10
Together AI's ATLAS adaptive speculator
A speculative-decoding system that keeps learning from live request distributions to update its draft model, delivering up to 4x faster LLM inference in production with no changes to the base model.