On-policy distillation
Technique · Training · 7 items since Jul 2026
A student learns from teacher scores on its own sampled outputs, not on teacher-written examples.
Related
- Organizations
Trends
Items
- Aug 8
Shanghai AI Lab's Intern-S2-Preview
A 397B scientific agentic foundation-model series combining multimodal pretraining, multi-task and agentic RL, and on-policy distillation, with native time-series forecasting and a Memory Decoder that specializes the frozen backbone without modifying it.
- Aug 1
Tsinghua's AgentOPSD critic-free turn-level credit
Converts sparse outcome rewards into turn-level credit by aggregating token-level teacher-student log-probability gaps into a Bayesian belief, reaching 89.1% success on ALFWorld with Qwen2.5-7B without extra critics or rollouts.
- Jul 24
Microsoft Research's LLM-as-a-Coach
An LLM judge becomes a coach whose textual feedback is distilled into the policy through on-policy context distillation, beating rubric-based RL on non-verifiable tasks while mitigating reward hacking.
- Jul 17
SEED self-evolves on-policy distillation
Agents turn their own completed trajectories into natural-language hindsight skills, then distill the resulting probability shift into a dense token-level signal optimized jointly with outcome-based RL, improving performance and sample efficiency on text and vision tasks.
- Jul 17
Two failure modes of on-policy distillation
A systematic study casts OPD as an exploration catalyst, names Student-Teacher Mismatch and Length Exploitation as its failure modes, and shows lightweight advantage clipping and log-scale compression beat OPD and RLVR baselines on seven benchmarks.
- Jul 10
Tencent Hunyuan's TurnOPD
Turn-aware on-policy distillation for long-horizon agents, using adaptive rollout-depth budgets and a progressive turn-normalized loss, improves the accuracy-versus-wall-clock frontier over vanilla OPD on ALFWorld, WebShop, and Multi-Hop Search.
- Jul 3
Shanghai AI Lab's Agents-A1
An open-weight 35B MoE agent matches or beats trillion-parameter models such as Kimi-K2.6 and DeepSeek-V4-Pro on long-horizon benchmarks by scaling agent horizon (45K-token trajectories) and domain-routed on-policy distillation rather than parameters.