DiffusionGemma

Trends

• Open-weight models diversify in architecture, modality, and size

Items

1. Jun 10

   Google DeepMind's DiffusionGemma

   An experimental open 26B-A4B MoE (3.8B active) that generates text by discrete diffusion, emitting 256-token blocks in parallel with bidirectional attention, for up to 4-5x faster decoding (over 1,000 tokens/sec on one H100) under Apache 2.0.