Diffusion language models

Related

Organizations

Google DeepMindH CompanyInclusion AINational University of SingaporeNVIDIAUniversity of Illinois Urbana-Champaign

Techniques

Speculative decodingUnified multimodal models

Trends

• Efficiency methods cut the compute and memory models need
• Multimodal understanding and generation merge into single models
• Open-weight models diversify in architecture, modality, and size
• Open releases add training data, recipes, and agent-ready vision

Items

1. Oct 2

   UIUC's hierarchical continuous diffusion LM

   HC-DLM makes a continuous latent trajectory the only persistent generative state, reading tokens out at each step and feeding them back as scaffolding, and beats discrete and continuous diffusion baselines on Sudoku, Countdown, and LM1B at matched size.

2. Sep 3

   H Company's NeoMME multimodal encoders

   260M and 800M multimodal, multilingual encoders trained from scratch with a masked discrete-diffusion objective in a single bidirectional Transformer, with no separate vision tower or causal LM, released as Apache-2.0 checkpoints with day-zero Transformers support.

3. Jul 10

   NVIDIA's Nemotron-Labs-Diffusion

   A tri-mode language model unifies autoregressive, diffusion, and self-speculation decoding; the 8B variant decodes ~6x more tokens per forward pass than Qwen3-8B at comparable accuracy, for ~4x higher throughput on a GB200.

4. Jun 10

   Google DeepMind's DiffusionGemma

   An experimental open 26B-A4B MoE (3.8B active) that generates text by discrete diffusion, emitting 256-token blocks in parallel with bidirectional attention, for up to 4-5x faster decoding (over 1,000 tokens/sec on one H100) under Apache 2.0.

5. Apr 22

   Inclusion AI's LLaDA2.0-Uni

   A single discrete-diffusion LLM with a MoE backbone matches specialized VLMs on multimodal understanding while generating and editing high-fidelity images, the first open architecture for natively interleaved generation and reasoning.

6. Apr 9

   DMax speeds up diffusion LLM decoding

   NUS recasts masked-diffusion decoding as progressive self-refinement so the model can correct its own predictions, raising tokens per forward pass on GSM8K from 2.04 to 5.47 with no accuracy loss and reaching 1,338 tokens/s on two H200s.