Open-weight models diversify in architecture, modality, and size

Organizations

Updates

1. Sep 11

   DeepSeek's MIT-licensed V4.1-Flash (552B, Causal Encoder-Decoder, 1M context) claims to surpass V4-Pro and will absorb its traffic, and SenseTime's 8B SenseNova-U1.5 unifies image understanding and generation without an encoder or VAE, with its training code to be open-sourced.

   DeepSeek-V4.1-Flash · SenseTime's SenseNova-U1.5

2. Jun 12

   Google DeepMind's DiffusionGemma decodes text by block diffusion at up to 4-5x speed, Cohere's North Mini Code fits a 30B/3B-active agentic coder on one H100, and the 20B Harness-1 search agent beat GPT-5.4 on recall (73% vs 70.9%), all under Apache 2.0; Microsoft's MIT-licensed 3.8B Lens matched larger image models at a fifth of the compute.

   Google DeepMind's DiffusionGemma · Cohere's North Mini Code · Harness-1 open search agent beats GPT-5.4 · Microsoft Research's Lens image model

3. Jun 5

   MiniMax M3 paired a 1M-token context and native multimodality with 59.0% on SWE-Bench Pro (weights pledged), NVIDIA released Cosmos 3 world models at 64B and 16B, JetBrains shipped Apache-2.0 Mellum2 (12B, 2.5B active), and H Company added quantized Holo3.1 computer-use checkpoints.

   MiniMax M3 · NVIDIA Cosmos 3 · JetBrains' Mellum2 · H Company's Holo3.1

4. May 1

   NVIDIA's Nemotron 3 Nano Omni, a 30B-A3B hybrid MoE adding native audio to images and video, delivers 9x the throughput of comparable open omni models, and Mistral Medium 3.5 ships open 128B multimodal weights.

   NVIDIA's Nemotron 3 Nano Omni · Mistral Medium 3.5 and Vibe remote agents

5. Apr 10New

   Llama 4 Scout and Maverick (17B active of 109B and 400B), GLM-5.1 (40B active of 744B), and Holo3 all shipped as MoE in one week, making active-parameter efficiency the main axis, though Gemma 4's strongest model is a 31B dense that ranks #3 on Arena.

   Meta's Llama 4 Scout and Maverick · Z.ai's GLM-5.1 · H Company's Holo3

   Against Google DeepMind's Gemma 4

Predictions

1. Sep 2026

   MoE becomes the default architecture for open-weight models above 10B total parameters, and leaderboards report active-parameter efficiency as a primary comparison metric. Apr 10

   Confirmed. Eight of the ten top open-weight models above 10B on the Artificial Analysis index are mixture-of-experts. Source

2. Nov 2026

   Hybrid MoE with tokenization-level modality fusion and token compression becomes the dominant open-weight multimodal architecture, displacing encoder-decoder pipelines for production omni-modal agents. May 1

3. Dec 2026

   The open-weight tier is judged mainly on sustained agentic and multimodal task completion rather than single-turn benchmarks, and at least one Western enterprise standardizes on a self-hosted open-weight model over a closed frontier API for cost and data residency. Jun 5

4. Dec 2026

   Open-weight selection hinges on architecture and per-token economics rather than headline benchmarks, with at least one production team standardizing on diffusion or small-active-parameter MoE for latency- and cost-sensitive workloads. Jun 12

5. Mar 2027

   Flash-tier open weights become the default substrate for cost-sensitive work, and flagship-into-flash deprecations signal that architecture, not raw scale, drives the next capability step. Sep 11