⚗️ Awesome On-Policy Distillation

A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation (OPD) of large language models.

On-policy distillation trains a student on samples from its own evolving policy, while a teacher (external, privileged, or self-conditioned) provides dense supervision on those same samples.

On-policy distillation (OPD) trains a student on trajectories sampled from its own policy while a teacher scores the student-visited prefixes with dense token-level guidance. This on-policy data collection reduces the train-inference distribution gap that affects off-policy KD/SFT on fixed traces. Depending on the estimator, OPD looks like GKD on student rollouts or policy-gradient/RL with teacher-defined per-token KL/log-prob rewards, making the natural contrast sparse outcome-reward RL rather than RL as a whole. As of 2026, OPD is a standard post-training primitive at Alibaba (Qwen3), DeepSeek (V4), Xiaomi (MiMo), Zhipu (GLM-5), NVIDIA (Nemotron-Cascade 2), and others.

Shipping today? Jump to Frameworks and Implementations. New to OPD? Read Start Here.

Contents

• Start Here
• Surveys and Essays
• Core OPD Papers
  • Foundations
  • Gap-Bridging
  • Stability and Objective Design
  • Self-Distillation
  • Context and Experience Internalization
  • Efficiency, Systems, and Privacy Variants
• Taxonomy
• Adjacent and Enabling Work
• Domain Extensions
  • Agents and Tool-Use
  • Multimodal and Vision-Language
  • Speech and Audio
  • Diffusion, Flow, and Generative Media
  • Embodied, Robotics, and Control
  • Speculative Decoding
• Technical Reports and Industrial Recipes
• Frameworks and Implementations
• Acknowledgments
• Contributing
• Citation

Start Here

A fast path through the field:

1. Survey. OPD Survey — taxonomy, methods, and open problems in one place.
2. Foundations. MiniLLM, GKD, and ExOPD — the core student-rollout plus teacher-supervision loop, including its dense KL-constrained RL framing.
3. Practical intuition. Thinking Machines blog — the clearest end-to-end explanation of why and when OPD applies.
4. When OPD works and when it breaks. Revisiting OPD, Entropy-Aware OPD, and Rethinking OPD — failure modes (instability, diversity collapse, tokenizer mismatch) and success conditions (compatible thinking patterns, novel teacher capability).
5. No teacher logits. Black-Box OPD — discriminator-based reward when the teacher is API-only.
6. No teacher at all. OPSD and SDFT — same model as student and self-teacher.
7. Context and experience. OPCD and OEL — distill prompts and deployment traces into weights.
8. Industrial recipes. Qwen3, DeepSeek-V4, MiMo-V2-Flash, GLM-5 — how labs ship OPD in production.

Key decision: access to teacher logits? Yes → white-box (GKD, Veto, Entropy-Aware OPD). No → black-box (GAD, OVD) or self-distillation (OPSD, SDFT).

Surveys and Essays

Surveys and Position Papers

• A Survey of On-Policy Distillation for Large Language Models (2026) — First dedicated OPD survey; organizes methods by feedback signal, teacher access mode, and loss scope.
• A Brief Overview: On-Policy Self-Distillation in Large Language Models (2026) — Beginner-oriented overview of on-policy self-distillation, cataloguing privileged-context designs where a single model is both teacher and student.
• Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation (2026) — Reframes SFT/RL/OPD by training-state source rather than loss, explaining why OPD's student-sampled states beat a degraded teacher.
• On the Geometry of On-Policy Distillation (2026) — Parameter-space diagnostics placing OPD in a relaxed off-principal regime with early subspace locking, a distinct update geometry between SFT and RLVR.
• Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation (2026) — Analyzes OPD checkpoint deltas, showing dense teacher supervision still yields sparse, off-principal weight edits resembling RLVR rather than dense supervised rewriting. (Code)
• A Formula-Driven Survey and Research Agenda for On-Policy Distillation (2026) — Formula-driven OPD survey and research agenda splitting temporal credit from vocabulary routing.
• Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning (2026) — Unifies post-training by data-policy mismatch, with a dedicated framework for on-policy teacher-guided transfer.
• Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling (2026) — Test-time scaling analysis finds OPD improves sampling efficiency while often shrinking the student's high-budget capability boundary.
• One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation (2026) — Reviews OPSD around collapse as a single symptom, unifying vocabulary across papers rather than reporting new experiments.
• Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives (2026) — Shows forward and reverse KL learn arithmetic versus geometric multi-teacher aggregates, explaining OPD's forgetting benefits and fragility.
• Understanding the Synergy between SFT, RLVR, and OPD in LLM Post-Training (2026) — Controlled study showing SFT and RLVR stages reshape student-teacher compatibility, making OPD's effectiveness depend on pipeline order.
• Rethinking Self-Distillation for Multi-Teacher Capability Merging (2026) — Controlled comparison finding multi-teacher OPD matches off-policy merging accuracy at far higher compute once training choices are equalized.
• On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics (2026) — Controlled study finding KL direction and learning rate, more than rollout policy, govern distillation performance and forgetting.
• Train4Merge: A Controlled Single-Teacher Study of RL vs. SFT Teachers for OPD-Based Model Merging (2026) — Controlled study showing RL-trained teachers guide OPD students better than equally strong SFT teachers, especially in agentic tasks.
• From Gradients to Capabilities: Understanding Multi-Teacher On-Policy Distillation (2026) — Analyzes how loss averaging, Adam momentum, and BF16 rounding mute differences between multi-teacher OPD signals.
• Divergence controls entropy in distillation (2026) — Proves forward KL inflates and reverse KL deflates student entropy, attributing OPD's lower entropy to the divergence, not sampling.
• No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation (2026) — Finds OPD task vectors compose with their RL teachers' and across tasks better than RL task vectors.

Essays, Blog Posts, and Walkthroughs

• Thinking Machines: On-Policy Distillation (2025) — Best single-article introduction. Covers concepts, intuition, and practical use cases.
• Unlocking On-Policy Distillation for Any Model Family (GOLD) (2025) — Cross-tokenizer OPD walkthrough with TRL code.
• The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind (2025) — Latent Space talk tracing distillation from logits to on-policy student rollouts, framed as the DAgger-style fix for exposure bias.
• Distilling 100B+ Models 40x Faster with TRL (2026) — HF engineering walkthrough of TRL's DistillationTrainer scaling tricks; ~40× speedup, validated on Qwen3-235B → Qwen3-4B math.
• Multi-Teacher On-Policy Distillation: A New Post-Training Primitive (2026) — Yumo Xu surveys MOPD as a post-training primitive across MiMo-V2-Flash, GLM-5, Nemotron-Cascade 2, DeepSeek-V4.
• On-Policy Distillation: Theory & Practice in Model Merging (2026) — ByteDance Seed framing OPD as entropy-regularized RL; cross-tokenizer pitfalls and reward hacking in agent merging.
• On SFT, RL, and on-policy distillation (2026) — Will Brown's essay on OPD via SFT-vs-RL compounding and gradient geometry; pointers toward an optimal teacher.
• SFT, RL, and OPD Through a Distributional Lens (2026) — wh's distributional-geometry framing; experiment shows OPD students from SFT and RL teachers converge and forget less.
• On Policy Self Distillation (2026) — KL-geometry study showing OPSD inverts OPD's per-token sign and suffers larger KL shocks that GEPA hint evolution roughly halves.
• What Apple found out about On-Policy Distillation (2026) — AVB's tutorial-style breakdown of "Unmasking OPD"; training-free gradient-alignment for predicting student-teacher fit.
• OPD深度解析：从数学推导到DeepSeek V4、SWIFT与verl实践 / OPD Deep Dive: From Mathematical Derivation to DeepSeek V4, SWIFT, and verl Practice (2026) — Chinese-language Zhihu deep-dive deriving OPD's sequence- and token-level reverse-KL; maps variants to MiniLLM, GKD, verl, DeepSeek V4.
• 重温 On-Policy Distillation / Revisiting On-Policy Distillation (2026) — Chinese-language notes deriving OPD as both a SeqKD student-rollout mirror and RL with token-level teacher supervision.
• The Imitation Game: State of Policy Distillation in Language Model training (2026) — Long-form OPD/OPSD survey with a four-axis failure-modes taxonomy; argues hybrid OPSD and cross-tokenizer OPD as the highest-leverage open problems.
• Scaling SDPO (2026) — Trajectory field note extending self-distillation policy optimization to stale off-policy production rollouts for single-trajectory continual learning on live interactions.
• My Continuous Learning Pipeline with Self-Distillation on Agent Traces (2026) — HuggingFace continuous-learning walkthrough that self-distills coding-agent traces with TRL's SDPO trainer, using an evaluator model's targeted feedback as privileged context.
• On-Policy Distillation: Promise, Pitfalls, and Prospects (2026) — Li Jiang organizes recent OPD failure analyses into three connected mechanisms — local teacher noise, horizon coverage decay, and myopic per-token supervision.
• Solving OPSD (basically) (2026) — Continuation arguing OPSD's positive teacher-agreement pressure carries the useful signal while negative pressure drives length collapse, so keeping only positive matches OPD.
• Why On-Policy Distillation Works and Naive Self-Distillation Doesn't (2026) — Frames on-policy distillation as KL-regularized RL toward a reward-tilted teacher, explaining why naive self-distillation distills a feedback-hallucinating template.
• Self-Distilled Reasoner: On-Policy Self-Distillation (2026) — Author walkthrough deriving on-policy self-distillation from a privileged self-teacher as dense token-wise distribution matching, with a policy-gradient interpretation.
• On-policy Distillation (2026) — Accessible long-form guide framing on-policy distillation as teacher-graded student trajectories that fix exposure bias, built around the GKD loop.
• Understanding Self-Distillation and Privileged Information Distillation (2026) — Walkthrough deriving self-distillation and privileged-information distillation through an RL-as-variational-inference lens, spanning reverse-KL self-teachers to reward-tilted joint training.

Core OPD Papers

The papers that define on-policy distillation for LLMs.

Scope rule: A paper belongs here if its primary contribution is a new component of the OPD training loop itself — an objective, divergence formulation, stability fix, teacher access-mode variant, self-distillation variant, context-internalization mechanism, or systems/efficiency/privacy constraint applied to that loop — with student rollouts central to the learning signal, evaluated on LLM text generation or reasoning. Operational test: if removing the OPD-loop component leaves a working contribution (a working RL recipe, preference method, or KD baseline), the OPD piece is auxiliary → Adjacent. Papers that enable OPD (cross-tokenizer alignment, calibration), compose with OPD as one component of a larger RL/preference structure, or apply OPD to non-text-reasoning substrates live in Adjacent and Enabling Work or Domain Extensions.

Foundations

• MiniLLM: On-Policy Distillation of Large Language Models (2023) — Reverse-KL framing for generative LMs; the paper that named the field. (Code)
• GKD: On-Policy Distillation of Language Models — Learning from Self-Generated Mistakes (2023) — Unifying formulation spanning on-/off-policy mixtures with flexible divergences.
• Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models (2026) — Controlled generalization study finding OPD transfers teacher reasoning behavior, with transfer strength hinging on teacher-student shared origin.
• Scaling Properties of Same-Family On-Policy Distillation (2026) — Fits power laws for same-family OPD across weak-to-strong and strong-to-weak setups, finding students can exceed compact RL teachers.

Gap-Bridging

• Speculative Knowledge Distillation (2024) — Interleaved teacher/student sampling mitigates poor student rollout quality. (Code)
• Black-Box On-Policy Distillation of Large Language Models (2025) — GAD: discriminator-based reward on student rollouts; no teacher logits required. (Code)
• SOD: Step-wise On-policy Distillation for Small Language Model Agents (2026) — Reweights teacher guidance by step-level divergence to avoid tool-induced cascade drift.
• MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate (2026) — Multi-agent debate consensus as the OPD teacher; extends to agentic tasks via step-level sampling.
• ROPD: Rubric-based On-policy Distillation (2026) — Black-box OPD using prompt-specific rubrics distilled from teacher-student contrasts to score rollouts. (Code)
• Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation (2026) — Backtracks straying student rollouts to the last safe state for teacher correction, targeting the reversed exposure bias on-policy distillation introduces. (Code)
• Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation (2026) — Counteraction-aware multi-teacher OPD that decouples conflicting recovery and preservation gradients, recovering general capability from proxy prompts without teacher-aligned prompt coverage.
• Trust-Region Behavior Blending for On-Policy Distillation (2026) — Warmup samples early prefixes from a teacher-blended behavior policy within a student-centered KL trust region, annealed to zero by warmup's end.
• Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance (2026) — Spreads teacher guidance across a near-future token window, using trajectory drift to find true reasoning forks rather than high-loss single tokens.
• Trust Region On-Policy Distillation (2026) — Restricts reverse-KL distillation to teacher-reliable trust regions on student rollouts, applying forward-KL to mismatched outlier tokens instead.
• OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification (2026) — Replaces teacher logits with chunk-level semantic verification from Monte Carlo rollouts, enabling on-policy distillation from black-box teachers.
• Trajectory-Refined Distillation (2026) — Teacher revises the student's failed rollout within on-policy support before distilling, correcting prefix failure at the trajectory rather than token level. (Code)
• SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models (2024) — Teacher selectively intervenes during student generation at high token-probability discrepancies, curbing teacher misguidance over long student rollouts.
• Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients (2026) — Embeds teacher candidate answers into RL prompts so the student learns on-policy, recovering signal on hard questions where every rollout fails.
• SGPO: Strategy-Guided Policy Optimization for LLM Reasoning (2026) — Distills reusable strategy descriptions from teacher responses via forward-KL on student trajectories, replacing instance-level trajectory imitation.
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration (2026) — Distills multiple per-domain RL teachers into one student on its own rollouts, integrating their specialized capabilities into a single model.
• Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selection (2026) — Student perplexity steers the teacher's chunk-by-chunk generation toward learnable reasoning paths, selecting branches during generation instead of filtering afterward.
• Weak-to-Strong Generalization via Direct On-Policy Distillation (2026) — Applies a weak teacher's pre/post-RL log-ratio as a dense reward on student rollouts, transferring the RL-induced shift rather than the final policy.
• CoPD: Co-Evolving Policy Distillation (2026) — Parallel expert training with bidirectional OPD; experts co-evolve as mutual teachers during RLVR.
• CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation (2026) — Schedules a per-token forward-to-reverse-KL mixture on student rollouts, gating the transition on student coverage of teacher tokens rather than time.
• Weak-to-Strong On-Policy Distillation (2026) — Synthesizes a proxy teacher from two weak models' logit difference anchored on the student's base, lifting a student stronger than both. (Code)
• Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold (2026) — Reliability-gated multi-teacher on-policy distillation whose teachers are deliberately trained on examples the accumulated teacher union fails to cover.
• Woodpecker Distillation: Weak Models Diagnose Reasoning Bugs in Strong Models (2026) — Builds a corrective teacher from how successful versus failed weak-model patches shift continuations at the strong model's prefixes, never imitating the patches.
• RoCo-ACE: Rollout-Conditioned Online Distillation for Retention-Aware Knowledge Injection (2026) — Contrasts rollout-conditioned teacher guidance with retention anchors, injecting knowledge without erasing existing capabilities.
• Pass the Baton: Trajectory-Relayed On-Policy Distillation (2026) — Lets the teacher repair failed prefixes before returning control, keeping supervision near the student's reachable trajectory. (Code)
• CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning (2026) — Finds the first causal error with a teacher, then repairs short continuations through on-policy reinforcement.
• Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning (2026) — Supervises complete teacher continuations from student-visited steps, giving long-horizon repairs where token-level OPD offers only fragmented corrections.
• Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation (2026) — Amplifies the student's verifier-driven policy gradient along a weak teacher's RL-induced shift, letting the student surpass its supervisor. (Code)
• Learning to Revise Reasoning with Segment-wise On-Policy Distillation (2026) — Replaces uncertain student reasoning segments with teacher redrafts, teaching explicit revision rather than token-wise correction.
• Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs (2026) — Admits only teachers whose answers pass a verifier per sample, merging self-anchors and verified feedback for multi-domain distillation. (Code)
• Interactive-Policy Distillation with Bidirectional Propose-and-Verify (2026) — Alternates student and teacher as proposer and verifier to build mixed-source rollouts, bridging on- and off-policy distillation.
• CataOPD: Catalytic On-Policy Distillation for Large Language Model Reasoning (2026) — Uses the teacher as a catalyst eliciting verified student-produced trajectories on all-fail problems, then internalizes them guidance-free.
• Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation (2026) — Replaces domain-routed teacher selection with label-free token-level weighting by each specialist's calibrated displacement from a shared reference.
• Lexicographic Multi-Objective On-Policy Distillation (2026) — Integrates reward-specialized teachers under a strict priority order, projecting out corrections that oppose higher-priority objectives.
• Not Every Token Is Worth Distilling: Selective Supervision for Direct-OPD (2026) — Masks Direct-OPD supervision at states where post- and pre-RL teachers barely diverge, keeping only high-divergence states.
• Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning (2026) — Licenses each expert to teach only tokens where its counterfactual gain on an answer criterion is verified.
• SAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation (2026) — Routes token supervision through maximal-coupling accept and correction events of a KL-constrained teacher-guided rollout. (Code)
• Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation (2026) — Rescales each domain teacher's feedback by its measured spread, preventing one domain from dominating multi-teacher updates.
• How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy Distillation (2026) — Trains the teacher with RL to continue correctly from student prefixes before distilling, adapting it to student-induced states.
• From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation (2026) — Adapts when the teacher takes over and how long it generates using local policy disagreement aggregated within reasoning paragraphs. (Code)
• MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation (2026) — Routes supervision across the whole teacher pool per token without domain labels, scoring teachers by expressed specialization. (Code)
• On the Off-Policy Teacher in On-Policy Distillation (2026) — Co-trains the teacher with RLVR to continue from student prefixes, fixing degradation from supervising off-policy prefixes.
• Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction (2026) — Shapes critic rewards for group-relative geometry in adversarial black-box OPD, avoiding collapsed or fragile GRPO advantage groups. (Code)
• Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation (2026) — Samples rollouts from a token-level interpolation of student and teacher distributions, tuning quality versus learnability continuously.
• Persistent Negatives for Adversarial Black-Box On-Policy Distillation (2026) — Stabilizes adversarial black-box OPD by training the discriminator on a live pool of historical teacher-student comparisons.
• Learning from Teacher Continuations at Student States (2026) — Trains the student by cross-entropy on teacher continuations of its own fresh prefixes, needing no teacher token probabilities.
• Latent-MOPD: Latent Multi-Teacher On-Policy Distillation (2026) — Integrates specialists through both hidden states and token predictions, shifting each teacher's supervision from representations to outputs. (Code)
• No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation (2026) — Shows multi-teacher OPD hinges on student initialization, searching merge weights beyond convex averaging to start distillation.
• Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation (2026) — Couples a fast student updated by each teacher with an EMA slow model, preserving general capabilities during multi-teacher OPD.
• Self-Specialized Teachers for Domain Post-Training (2026) — Trains a base-model copy into a base-aware domain teacher, then distills it on student prefixes to adapt without forgetting.
• Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation (2026) — Lets the teacher continue briefly past student prefixes, choosing continuation length by a variance-bias trade-off from uncertainty contraction.

Stability and Objective Design

• DistiLLM: Towards Streamlined Distillation for Large Language Models (2024) — Skew-KL divergence with adaptive off-policy use of student-generated outputs; foundational OPD objective formulation. (Code)
• DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (2025) — Contrastive extension of skew-KL; student-generated outputs collected per epoch. (Code)
• Veto: Stable On-Policy Distillation through Adaptive Target Reformulation (2026) — Intermediate target distribution in logit space stabilizes training.
• Entropy-Aware On-Policy Distillation of Language Models (2026) — Forward-KL on high-entropy teacher tokens preserves output diversity.
• ExOPD: Learning beyond Teacher via Generalized On-Policy Distillation with Reward Extrapolation (2026) — Casts OPD as dense KL-constrained RL; reward scaling enables teacher-surpassing behavior. (Code)
• Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention (2026) — Black-box oracle edits student failures into proximal rollouts for reward-based distillation that preserves prior knowledge. (Code)
• REOPOLD: Scaling Reasoning Efficiently via Relaxed On-Policy Distillation (2026) — Relaxes imitation with reward clipping, entropy-based dynamic sampling, and explore-to-refine training.
• PACED: Distillation at the Frontier of Student Competence (2026) — Pass-rate weighting focuses learning on the student's competence frontier.
• Revisiting On-Policy Distillation — Empirical Failure Modes and Simple Fixes (2026) — Truncated reverse-KL with teacher top-K support matching; fixes imbalanced signals and tokenizer mismatch. (Code)
• Rethinking On-Policy Distillation — Phenomenology, Mechanism, and Recipe (2026) — Identifies compatible thinking patterns and novel teacher capability as OPD success conditions.
• The Illusion of Certainty — Decoupling Capability and Calibration in OPD (2026) — Diagnoses OPD-induced overconfidence; CaOPD replaces confidence targets with student-grounded empirical success rates.
• Demystifying OPD — Length Inflation and Stabilization Strategies (2026) — Repetition-driven length inflation in iterative OPD; Stable-OPD adds divergence constraints and a rollout-mixture anchor.
• SCOPE: Signal-Calibrated On-Policy Distillation with Dual-Path Adaptive Weighting (2026) — Routes correct student rollouts to student-PPL-weighted MLE and incorrect to teacher-PPL-weighted KL; dual-path OPD loss design.
• HPD: Hybrid Policy Distillation for LLMs (2026) — Unified reweighted-log-likelihood framework combining forward/reverse KL with off-policy and on-policy sampling. (Code)
• Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe (2026) — Offline difficulty-aware and online correctness-aware data balancing with outcome-guided margin calibration. (Code)
• AOPD: Asymmetric On-Policy Distillation (2026) — Replaces ineffective negative reinforcement with localized teacher-distribution matching in non-positive advantage regions.
• vOPD: On-Policy Distillation with a Control Variate Baseline (2026) — Closed-form per-token reverse-KL value baseline; unbiased lower-variance single-sample estimator with no extra critic.
• Unmasking On-Policy Distillation — Where It Helps, Where It Hurts, and Why (2026) — Training-free gradient-alignment diagnostic; best teacher flips with student capacity and task; wrong demos hurt self-distillation except on hard math.
• The Many Faces of On-Policy Distillation — Pitfalls, Mechanisms, and Fixes (2026) — Names three failure modes (student-prefix teacher-state mismatch, biased Top-K gradients, PI-free OPSD aggregation) and three stabilizers (stop-grad Top-K KL, RLVR teachers, SFT-stabilized students).
• Rock Tokens — Deciphering High-Loss Tokens in On-Policy Distillation (2026) — High-loss tokens (up to 18%) persist after apparent convergence; masking them streamlines alignment. (Code)
• BRTS: On-Policy Distillation with Best-of-N Teacher Rollout Selection (2026) — Auxiliary teacher-context branch alongside standard OPD; selects best-of-N teacher rollouts by correctness then student-alignment. (Code)
• Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation (2026) — Dynamic release rule truncates dense supervision where the teacher's local margin collapses; counters suffix degradation in strong-to-weak OPD.
• Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for LLM Post-Training (2026) — Sparse-to-dense post-training workflow framing OPD as the dense teacher-induced reward between GRPO stages.
• The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs (2026) — Reward-extrapolation OPD collapses past a clip threshold on near-deterministic structured outputs, mapping where teacher-surpassing reward scaling stops working.
• MOPD: Multi-Rollout On-Policy Distillation via Peer Successes and Failures (2026) — Conditions the teacher on successful and failed peer rollouts from the student's local group, sharpening token-level supervision over independent per-rollout distillation.
• Teacher-Guided Policy Optimization for LLM Distillation (2026) — Feeds teacher tokens conditioned on the student's rollout as explicit on-policy-SFT targets, replacing reverse-KL's uninformative negative feedback under large teacher gaps.
• Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation (2026) — Applies OPD loss only to "teachable" tokens where the teacher's corrective mass lands within the student's support, separating learnable from incompatible disagreement.
• AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment (2026) — Reflection-bottlenecked privileged self-distillation converting diagnostics into ReLU-gated token-level advantages, preventing the late-stage collapse of raw-oracle conditioning.
• Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation (2026) — Rewards the student's top-K candidate tokens by the teacher confidence they induce one step ahead, countering supervision-fidelity decay over long reasoning chains.
• OPD+: Rethinking the Advantage Design for On-Policy Distillation (2026) — Corrects on-policy distillation's biased stop-gradient advantage estimator, generalizing the objective to any f-divergence beyond the usual reverse KL.
• SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment (2026) — Confines reverse-KL on-policy distillation to a mined sparse subset of safety tokens, aligning behavior while sidestepping the alignment tax.
• FiRe-OPD: Filter, Then Reweight — Rethinking Optimization Granularity in On-Policy Distillation (2026) — Drops low teacher-likelihood trajectories, then soft-reweights remaining tokens by teacher confidence and student confusion, replacing hard token selection. (Code)
• Reinforcement Learning from Rich Feedback with Distributional DAgger (2026) — Distills a feedback-conditioned teacher via forward cross-entropy on student-visited states, provably guaranteeing the monotonic policy improvement reverse-KL self-distillation lacks. (Code)
• OPRD: On-Policy Representation Distillation (2026) — Supervises student rollouts by aligning intermediate hidden states with the teacher across layers, replacing high-variance output-space logit matching. (Code)
• Physics-Guided Policy Optimization with Self-Distillation (2026) — Modulates self-distillation step size by student-teacher mutual information, stabilizing privileged-feedback updates that otherwise collapse late in training.
• Constitutional On-Policy Safe Distillation (2026) — Recalibrates a constitution-conditioned teacher via cross-SFT cold-start before on-policy distillation, preventing reverse-KL collapse into over-cautious safety refusals.
• When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation (2026) — Gates each historical-teacher refresh on consolidation evidence, replacing clock-driven updates to prevent state-oblivious collapse in self on-policy distillation.
• SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling (2026) — Routes each distillation token by sign agreement between a binary verifier and the teacher, extrapolating consensus tokens and interpolating conflicts.
• Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation (2026) — Unifies SFT, DAgger, offline RL, and on-policy distillation by decomposing the distillation KL along prefix-source and direction axes.
• Escaping the KL Agreement Trap in On-Policy Distillation (2026) — Terminates student rollouts trapped in persistent low-KL teacher agreement, filtering degenerate supervision that yields no corrective on-policy signal.
• Validity-Calibrated Reasoning Distillation (2026) — Reweights per-token distillation updates by the relative local validity of teacher and student next-step proposals, instead of uniform trajectory imitation.
• PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation (2026) — Replaces on-policy distillation's unbounded log-ratio reward with a natively bounded Box–Cox power transformation, curbing high-variance gradients that post-hoc scaling cannot. (Code)
• On the Position Bias of On-Policy Distillation (2026) — Importance-Weighted OPD upweights each token by its cumulative student-teacher prefix discrepancy, correcting position bias where later rollout tokens drift off-teacher. (Code)
• Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation (2026) — Finds decision tokens via student entropy and evidence tokens via hidden-state similarity plus teacher-student divergence for selective OPD loss.
• ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation (2026) — Reweights each student trajectory by a student-teacher log-ratio score over pivotal tokens, emphasizing likely-negative rollouts without observing the final answer. (Code)
• Blockwise Policy-Drift Gating for On-Policy Distillation (2026) — Reweights OPD position losses with student-only block-level policy-drift gates, stabilizing reused rollouts without altering teacher targets.
• SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision (2026) — Uses joint teacher-student entropy as a competence probe to tailor token-level supervision, addressing the competence-dependent degradation uniform on-policy distillation ignores.
• DOPD: Dual On-Policy Distillation (2026) — Routes each token's supervision between a privileged teacher and privileged student by advantage gap, disentangling closable capability gaps from un-replicable information asymmetry.
• Reward-Gated On-Policy Distillation (2026) — Gates teacher distillation on each rollout by agreement between verifier reward sign and teacher–student likelihood gap, trusting only directionally reliable teacher signal.
• Trust Region Policy Distillation (2026) — Interpolates teacher and student probabilities into a proximal teacher, replacing on-policy distillation's unbounded log-ratio reward with a provably variance-bounded signal.
• Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation (2026) — Anchors suppression-prone tokens to a frozen-base reference prior via an asymmetric divergence gate, preventing OPSD's collapse of native reasoning.
• On-Policy Delta Distillation (2026) — Rewards on-policy distillation by a delta signal, the teacher minus its pre-tuning base, transferring reasoning traces rather than inherited preferences.
• Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations (2026) — Diagnoses length-exploitation shortcuts in token-level on-policy distillation, regulating the advantage with hard clipping or order-preserving log-scale compression.
• Distilled Reinforcement Learning for LLM Post-training (2026) — Weights student rollouts by clipped teacher-to-student token ratios inside the RL objective, replacing on-policy distillation's unconditional logit matching.
• Outcome-Confounded Local Supervision in On-Policy Distillation (2026) — Crosses token-level teacher-student divergence with final-answer correctness, finding agreement-on-failure dominates because local divergence cannot localize where trajectories became unrecoverable.
• β-OPSD: Deriving with Policy Optimization, Training with Self-Distillation (2026) — Distills a scheduled reference-to-teacher logit interpolant derived as the optimal policy of a KL-regularized family generalizing vanilla self-distillation.
• Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Models (2026) — Subtracts the cross-rollout-predictable part of teacher-reference disagreement, relaxing the requirement that the distillation teacher generated the fine-tuning data.
• SAF-OPD: Stable Advantage Fusion for On-Policy Distillation (2026) — Rescales the token-level distillation advantage into a bounded, time-decaying signal before fusing it with GRPO's, preventing fixed-coefficient entropy collapse.
• Not Every Divergence Should Be Suppressed: Counterfactual Recoverability in On-Policy Distillation (2026) — Labels each student error state as recoverable or not by replaying teacher-continuation and rollback branches, replacing divergence-based supervision selection.
• SPOT: Sparse Probing and Outcome Calibration for On-Policy Distillation (2026) — Reshapes teacher targets at sparsely probed positions using verifier-scored student continuations, correcting reverse-KL's neglect of plausible alternatives.
• On-Policy Delta Distillation for Multilingual Math Reasoning (2026) — Empirical study of delta-signal on-policy distillation for multilingual math reasoning, finding English-only training transfers accuracy but drags responses toward English.
• DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models (2026) — Weights self-distillation tokens by backward multi-step aggregation of divergence gaps, restoring the order dependence vanilla OPSD's uniform coefficients discard. (Code)
• Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance (2026) — Applies teacher guidance only to zero-reward rollout groups, restoring gradients where group-relative reinforcement learning stalls.
• DAPD: Dual-Anchored Policy Distillation (2026) — Dual self-teacher anchors constrain privileged targets, preventing unreachable guidance from destabilizing student rollouts. (Code)
• When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation (2026) — Filters teacher signals by input grounding plus distributional extremity, suppressing confident but spurious guidance. (Code)
• Privileged Likelihood Is Not Automatically Value: Three Checks for Token Credit in On-Policy Self-Distillation (2026) — Audits privileged likelihood as token credit, exposing hindsight signals with weak outcome alignment.
• Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation (2026) — Jointly adapts privileged exposure plus token weighting under a capacity budget to match evolving student learnability. (Code)
• Mismatch Matters: On-Policy Distillation Beyond Token Agreement (2026) — Routes directional mismatch through bounded excess suppression plus top-K deficit recovery to avoid degenerate agreement. (Code)
• SR-OPSD: Self-Referenced On-Policy Self-Distillation (2026) — Projects a reference-anchored self-teacher through Rényi divergence to decouple target placement from rollout projection geometry.
• WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training (2026) — Co-trains rollout and auxiliary policies through geometric-mixture reverse KL to buffer teacher corrections from visited states.
• ReOrder-OPD: Reliability-Aware Prompt Ordering for On-Policy Distillation (2026) — Schedules prompts by predicted teacher-continuation reliability before fresh student-rollout OPD.
• Simple-OPD: Demystifying Warm-up for On-policy Distillation (2026) — Teacher-generated reasoning warm-up initializes OPD for faster convergence with less out-of-domain degradation. (Code)
• REOPD: Reliability-Adaptive Reward Extrapolation for On-Policy Distillation (2026) — Reliability-weighted token extrapolation with batch-adaptive budgets prevents fixed-coefficient reward hacking across domains.
• CROP: Task Relevance via Counterfactuals for Selective On-Policy Distillation (2026) — Selects token-level supervision by paraphrase-calibrated counterfactual sensitivity, targeting task-relevant corrections.
• Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress (2026) — Suppresses distillation rewards that disagree with an independent reasoning-progress ranking, dropping teacher feedback that penalizes genuine but divergent reasoning.
• On-policy Distillation with Verifiable Reward (2026) — ReLU-gates the sampled-token OPD reward by trajectory correctness, merging dense teacher guidance with verifiable rewards without extra hyperparameters. (Code)
• A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation (2026) — Factorizes the per-token reverse-KL gradient into teacher-student gap and student surprise, motivating a bounded surprise-aware reweighting rule.
• Rethinking Reverse KL as Adaptive Entropy Distillation (2026) — Decomposes on-policy reverse KL into teacher-fitting and student-entropy terms, using teacher entropy to calibrate per-token imitation strength.
• Tail-Aware Top-k On-Policy Distillation (2026) — Adds a tail token carrying the mass outside the teacher's top-k, stopping the entropy inflation top-k normalization induces. (Code)
• Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning (2026) — Spreads a group-normalized teacher-verifier disagreement residual across tokens by their OPD advantages, correcting supervision that token-level teacher support misjudges. (Code)
• Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation (2026) — Traces multi-teacher OPD's integration gap to token-budget misallocation rather than gradient conflict, rebalancing with token-share and staleness-aware controls.
• DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance (2026) — Isolates a prohibition's causal effect from two identical-weight teachers differing only in prohibition visibility, turning their token disagreement into supervision.
• SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation (2026) — Drops the lowest-entropy student tokens from the distillation loss, stabilizing multi-turn OPSD better than scaling by final-answer correctness.
• SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning (2026) — Aligns student explorations to teacher symbolic milestones by longest common subsequence, giving dense process rewards without a trained verifier. (Code)
• STAR-OPD: Structured Aspect-Cascade-Aware On-Policy Reward Distillation for ABSA Quadruple Extraction (2026) — Rewards student rollouts by cascade-aware set-structured criteria, correcting the structurally invalid extraction states off-policy distillation never supervises.
• Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation (2026) — Verifies teacher reliability per prompt with verifier-scored probes, routing unreliable prompts from dense OPD to verifier-grounded GRPO.
• DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation (2026) — Applies bounded teacher guidance only to failed trajectories within GRPO, gating supervision by problem difficulty and outcome.
• Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement (2026) — Finds OPD gains come mostly from suppressing low-probability tokens, motivating a teacher-free self-adaptation alternative.
• Reward-Aligned Reweighting for On-Policy Distillation (2026) — Continuously reweights token-level teacher supervision by outcome agreement and disagreement magnitude, instead of uniform imitation or hard filtering.
• Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models (2026) — Learns per-token OPD weights through bilevel optimization that maximizes the student's expected reward, with closed-form weight updates. (Code)
• ReTaCo: Residual-Target Control for On-Policy Distillation (2026) — Groups omitted vocabulary into one residual symbol so top-k OPD keeps the teacher as a stationary point.
• Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models? (2026) — Shows OPD works with only sign rewards toward the teacher on a few high-disagreement tokens, questioning KL's necessity. (Code)
• Calibrating Teacher-Student Discrepancy for On-Policy Distillation (2026) — Removes the teacher's own self-deviation from teacher-student discrepancy via privileged interventions before distilling.
• Gains and Collapse in On-Policy Distillation: A Reinforcement Learning Perspective (2026) — Explains OPD collapse into long repetitive outputs as reward hacking of the teacher's implicit reward model. (Code)
• When Teacher Guidance Misleads: Reward-Aligned On-Policy Distillation (2026) — Keeps only trajectories whose teacher-induced updates move the student toward correct outcomes, filtering misaligned guidance.
• An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning (2026) — Connects reverse-KL OPD to KL-regularized RL, adding optimistic exploration and off-policy rollout reuse from value-based RL.
• Unbiased Top-k Estimation for On-Policy Distillation (2026) — Corrects top-k OPD's truncation bias with an unbiased estimator covering probability mass outside the selected tokens.
• The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation (2026) — Extrapolates the teacher's RL-induced residual in representation space at every layer, avoiding noisy output-space reward extrapolation. (Code)
• TV-Regulated OPD: Direction Matters in On-Policy Distillation (2026) — Shapes OPD advantages with bounded total-variation signals after finding token-advantage sign alone matches standard OPD.
• Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation (2026) — Interprets token-level OPD as a temporal approximation of sequence-level reverse KL, using discounted credit with bounded variance.
• When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation (2026) — Traces OPD length inflation to teacher-student termination-token mismatch, fixed by treating equivalent EOS tokens as one stopping action.
• Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers (2026) — Screens problems by joint student-teacher success, routing them to gated forward or reverse KL with an entropy brake.
• A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients (2026) — Parameterizes per-token forward/reverse-KL gating with multi-channel signals and bias, subsuming entropy- and gap-gated OPD variants.
• DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation (2026) — Sets feedback direction by student outcome and teacher role by joint teacher-student outcome, covering all four combinations.
• Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution (2026) — Trains a model to sample a frozen teacher's sequence-level power distribution in one pass via student-proposed sequential Monte Carlo. (Code)
• LastOPD: Taming Collapse in Latent On-Policy Distillation (2026) — Applies latent supervision only at the last-layer state during a brief crossfade into token-level OPD, avoiding late latent-alignment collapse. (Code)
• Expected Reasoning-Step Return Unifies On-Policy Learning from Rewards and Teachers (2026) — Estimates expected task return of student and teacher-proposed reasoning steps via student rollouts, choosing which signal guides each step.
• Distillation as Probability Transport: Routed On-Policy Distillation (2026) — Recasts OPD as probability transport, pairing student-excess sources with teacher-deficit destinations under a bounded potential.
• Teaching a Moving Student: Rethinking the Curriculum of On-Policy Distillation (2026) — Adds a gradient-triggered curriculum that revisits initial-student rollouts, since evolving students abandon still-informative early prefixes.
• Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation (2026) — Restores teacher diversity in sampled-token OPD by steering updates away from entropy-contracting positions using a first-order influence proxy.
• GVPO++: Group Variance Policy Optimization for LLM Post-Training and On-Policy Distillation (2026) — Extends a KL-constrained group-variance policy optimizer to OPD without importance sampling, supporting a broad family of objectives.

Self-Distillation

• OPSD: Self-Distilled Reasoner (2026) — Single model as both teacher and student via privileged information; no external teacher.
• SDFT: Self-Distillation Enables Continual Learning (2026) — Demonstration-conditioned self-teaching for continual learning with less forgetting. (Code)
• SDPO: Reinforcement Learning via Self-Distillation (2026) — Converts textual feedback into dense self-teacher signals for RL-like training. (Code)
• Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs? (2026) — Traces failures to suppression of epistemic verbalization; task coverage determines whether conciseness helps. (Code)
• OPSDC: On-Policy Self-Distillation for Reasoning Compression (2026) — Compresses verbose reasoning using concise privileged self-teachers. (Code)
• BIRD: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning (2026) — SFT-bootstraps the rollout distribution toward cleaner prefixes before on-policy reverse-KL self-distillation, fixing OPSD's cold-start prefix bottleneck. (Code)
• GATES: Self-Distillation under Privileged Context with Consensus Gating (2026) — Consensus-gated asymmetric-context self-distillation without labels or rewards.
• HDPO: Hybrid Distillation Policy Optimization via Privileged Self-Distillation (2026) — Privileged self-distillation on cliff prompts where RL gradients vanish; recovers KL-regularized optimal policy.
• RLSD: Self-Distilled RLVR (2026) — Self-distillation as token-level credit assignment within GRPO; OPSD-style matching leaks privileged information.
• SDZero: Self-Revision Turns Binary Rewards into Dense Supervision (2026) — Generator-reviser dual roles; reviser converts binary feedback into token-level supervision with no external teacher.
• OPSDL: On-Policy Self-Distillation for Long-Context Language Models (2026) — Short-context distribution of the same model as co-evolving reverse-KL teacher under long context.
• PBSD: Preference-Based Self-Distillation — Beyond KL Matching via Reward Regularization (2026) — DPO-style preference learning between context-augmented teacher positives and on-policy student negatives.
• UniSD: Towards a Unified Self-Distillation Framework for Large Language Models (2026) — Unifies self-distillation across supervision reliability, representation alignment, and training stability. (Code)
• OPSD Compresses What RLVR Teaches — A Post-RL Compaction Stage (2026) — Correct-only OPSD preserves accuracy and shortens responses; proposes SFT → RLVR → OPSD as post-RL compaction.
• ATESD: Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning (2026) — Treats teacher reveal ratio as a learnable control variable via Beta-policy controller with discounted learning-progress reward.
• OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering (2026) — Contrasts averaged teacher logits over correct vs. incorrect rollouts to form outcome-guided steering on anchor logits.
• RLRT: Rebellious Student — Reversing Teacher Signals for Reasoning Exploration (2026) — Upweights student tokens that diverged from teacher but still succeeded as a "valuable exploration" signal added to GRPO.
• EGRSD: Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning (2026) — Teacher-entropy confidence gate over RLSD's direction-magnitude signal; causal-lookahead variant preserves transient pivot tokens.
• CREDIT: From Generic Correlation to Input-Specific Credit in On-Policy Self Distillation (2026) — Recasts the self-distillation token reward as Bayesian filtering; batch-contrastive teacher baseline strips input-generic shortcuts.
• OPHSD: Training with Harnesses — On-Policy Harness Self-Distillation for Complex Reasoning (2026) — Generalizes self-distillation privileged context from a static variable (reference solution, environment trace) to a harness-driven workflow (draft-verify, plan-solve); harness is a removable training scaffold. (Code)
• MixSD: Mixed Contextual Self-Distillation for Knowledge Injection (2026) — Per-token Bernoulli mix of fact-conditioned and naive-conditioned base-model samples; replaces SFT for knowledge injection without collapsing held-out capability.
• AntiSD: Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information (2026) — Identifies the OPSD token reward as a PMI that suppresses deliberation tokens, then reverses its sign under an entropy-triggered gate. (Code)
• TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment (2026) — Routes self-distillation KL only to annotator-marked spans to cure the all-token "distillation tax" of SDPO/SRPO.
• AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals (2026) — Multi-view privileged self-distillation that gates teacher-specific residuals so they can adjust update magnitude but cannot reverse the cross-view consensus direction. (Code)
• It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs (2026) — Two same-model teachers conditioned on self-generated feedback supply complementary reverse-KL targets on the student's own rollouts, intersected as a product-of-experts.
• VPD: Learning from Language Feedback via Variational Policy Distillation (2026) — Variational-EM self-distillation refines a feedback-conditioned self-teacher in the E-step before distilling it back via token-level KL in the M-step.
• RMSD: Bringing Capabilities in Distribution via Relevance-Masked Self-Distillation (2026) — Applied Compute's OPSD variant masking the reverse-KL loss to LLM-judge-selected behavior-relevant tokens; preserves capabilities where SFT collapses.
• SPD: Self-Policy Distillation via Capability-Selective Subspace Projection (2026) — Decode-time KV-subspace projection biases self-rollout generation toward capability-relevant directions, then LoRA-SFTs on those rollouts without any external verifier or teacher.
• Multilingual Safety Alignment via Self-Distillation (2026) — Same-model OPSD transfers English safety reasoning to low-resource languages without any response data.
• COPSD: Crosslingual On-Policy Self-Distillation for Multilingual Reasoning (2026) — Uses English translations and reference solutions as privileged teacher context for low-resource multilingual reasoning OPSD. (Code)
• EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation (2026) — Guides a fraction of student rollouts with the privileged context, then distills only positive-evidence tokens, internalizing rare identities OPSD never samples.
• When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning (2026) — Weights OPSD token supervision by within-sequence position, the strongest tested predictor of privileged-teacher reliability, rather than ambiguous teacher entropy.
• Ditto: Reinforcing Human Behavior Simulation via Verbal Feedback (2026) — Jointly GRPO-optimizes a draft rollout and its judge-feedback-conditioned refinement so the policy internalizes verbal guidance, targeting subjective human-simulation rather than verifiable rewards.
• OISD: On-Policy Internal Self-Distillation of Language Models (2026) — Distills the detached final layer into an intermediate layer across model depth via advantage-weighted Jensen–Shannon alignment — needs no privileged context.
• ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains (2026) — Reflection-guided OPSD restricting self-teacher distillation to a rollout's erroneous span, targeting cross-domain reasoning generalization.
• SGSD: Skill-Conditioned Gated Self-Distillation for LLM Reasoning (2026) — Skill-conditioned OPSD whose retrieved-skill teachers are outcome-validated before distillation, extending privileged self-distillation to unreliable experience-derived context. (Code)
• Distilling LLM Feedback for Lean Theorem Proving (2026) — Distills a self-teacher conditioned on LLM-generated critique of the student's attempt, injecting external knowledge through natural-language feedback rather than logits or solutions.
• CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO (2026) — Answer-free correctness-conditioned self-teacher bidirectionally flips GRPO token-advantage signs, unlike the privileged-context teachers of related self-distillation methods.
• SC-SDPO: Restoring the Sweet Spot via Pass-Rate Weighted Self-Distillation (2026) — Reweights SDPO's self-distillation loss by an on-the-fly pass-rate term, restoring the difficulty sweet spot that pure self-distillation discards.
• Self-Supervised On-Policy Distillation for Reasoning Language Models (2026) — Conditions a self-teacher on a successful peer completion to densely supervise failed on-policy prefixes within each GRPO group.
• Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation (2026) — On-policy self-distillation for safety using a privileged-context self-teacher, with flip-rate prompt search selecting contexts that activate latent refusal.
• Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning (2026) — Distills a temperature-scaled copy of the model's own logits to restore entropy in RL-collapsed policies before continued training.
• SDPG: Self-Distilled Policy Gradient (2026) — Couples GRPO verifier rewards with exact full-vocabulary privileged self-distillation, casting its reverse-KL term as an equivalent centered log-ratio policy gradient. (Code)
• Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning (2026) — Routes signed-KL self-distillation by student entropy, pulling low-entropy tokens toward the privileged self-teacher while pushing high-entropy tokens away to preserve exploration.
• AR-OPD: Anchored Residual Guidance for Privileged On-Policy Distillation (2026) — Splits privileged supervision into a locally reachable partial-oracle anchor plus a scaled full-view residual, curbing hindsight leakage from monolithic full-privilege imitation.
• The Role of Feedback Alignment in Self-Distillation (2026) — Compares self-teacher context forms, showing step-aligned critique matched to the solver's reasoning trace localizes distillation credit at errors unlike reference solutions.
• RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation (2026) — Contrasts the privileged-hint self-teacher gap under correct versus wrong hints, cancelling style-token drift to concentrate distillation signal on task-bearing tokens. (Code)
• RGSD: Rubric-Guided Self-Distillation (2026) — Conditions the frozen self-teacher on per-prompt grading rubrics, distilling dense token-level targets into the prompt-only student to replace verifier-scored rubric reinforcement learning.
• PAINT: Partial-Solution Adaptive Interpolated Training for Self-Distilled Reasoners (2026) — Privileged self-distillation re-scoring student rollouts under an adaptively masked reference solution, calibrating only high-entropy-mismatch tokens. (Code)
• OPCoD: Be My Tutor — On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback (2026) — Bidirectional co-distillation where paired students condition each self-teacher on peer natural-language feedback, targeting mutual cross-domain Pareto improvement without external teachers.
• Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation (2026) — Splices a student's own wrong rollouts with self-generated correct references into explicit error-correction trajectories, replacing logit-level self-distillation.
• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation (2026) — Conditions the privileged teacher on learned rubrics to give criterion-aware token-level guidance on student rollouts, replacing reference-trajectory supervision.
• Self-Distillation as a Performance Recovery Mechanism for LLMs (2026) — Anchors a degraded model to a frozen earlier checkpoint via on-policy self-distillation, recovering capabilities lost to fine-tuning or compression.
• On-Policy Self-Distillation Reduces Output Diversity (2026) — Sampled-demonstration teacher tilts the base distribution by pointwise conditional mutual information, collapsing rollout diversity unlike on-policy RL.
• PHF: Privileged Hidden Flow for On-Policy Self-Distillation (2026) — Distills how the privileged teacher's hidden states move along the rollout, supervising internal computation that output-only self-distillation leaves untouched.
• Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation (2026) — Student matches a frozen self-teacher's answer distribution while conditioned only on a truncated reasoning prefix, building robustness to inference-time truncation.
• Purified OPSD: On-Policy Self-Distillation Without Losing How to Think (2026) — Subtracts a reference-only teacher probe to isolate the transferable residual, distilling a closed-form PMI target that preserves long-CoT reflection.
• DemoPSD: Disagreement-Modulated Policy Self-Distillation (2026) — Targets a disagreement-modulated reverse-KL barycenter of privileged teacher and student, pulling supervision toward the student where high divergence signals privileged-information leakage.
• Neuron-OPSD: Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation (2026) — Derives both training-data selection and teacher context from internal neuron activations, driving annotation-free on-policy self-distillation without external oracles.
• Denser ≠ Better: Limits of On-Policy Self-Distillation for Continual Post-Training (2026) — Empirical-theoretical critique showing dense on-policy self-distillation forgets more than GRPO during continual post-training, as teacher projection induces excess parameter drift. (Code)
• Rethinking On-Policy Self-Distillation for Thinking Models (2026) — Diagnostic study showing privileged self-distillation degrades thinking models on long reasoning, as teacher context lowers fork rates at high-entropy positions.
• CANON: Consensus as Privileged Context for Label-Free Self-Distillation (2026) — Turns agreement among a model's own samples into privileged context for a frozen self-teacher, densely supervising fresh rollouts without labels.
• Enhancing Rubric-based RL via Self-Distillation (2026) — Distills a rubric-conditioned self-teacher into the policy via forward-KL on the model's own rollouts, rescuing criteria that scalar reward aggregation suppresses.
• PromptSD: One Student, Many Teachers — Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context (2026) — On-policy self-distillation via a soft-prompt teacher, routing each example in a merged multi-task corpus to its own per-task prompt-teacher.
• H²SD: Hybrid Hindsight Self-Distillation (2026) — Correctness-aware on-policy self-distillation that reweights updates on correct rollouts but applies reference-hint reverse-KL correction where the student's reasoning fails.
• Not All Tokens Deserve Equal Credit: Counterfactual Sensitivity Credit Reallocation for Long-CoT Reasoning (2026) — Downweights credit for tokens whose likelihoods shift most under opposite correctness assertions, recasting privileged self-teacher shifts as sensitivity rather than supervision.
• Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation (2026) — Traces self-distillation's failure on hard tasks to privileged-information bias pulling per-token targets toward one reference solution rather than correctness.
• On-Policy Self-Distillation without Any Supervision (2026) — Uses a majority-vote consensus rollout as privileged teacher context for distillation onto disagreeing rollouts, removing OPSD's last dependence on external supervision.
• RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer (2026) — Routes multilingual token supervision by how much an English reference shifts the teacher view, targeting reasoning pivots rather than surface realization. (Code)
• Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning (2026) — Replaces privileged solution traces with problem-structure guidance, preserving reasoning cues while reducing shortcut imitation.
• Rubrics as Privileged Information for Open-Ended Generation (2026) — Conditions a self-teacher on task rubrics, transferring open-ended evaluation criteria through student-generated responses.
• Learning from Consensus and Disagreement: Unsupervised On-Policy Self-Distillation with Minority-Trajectory Contrast (2026) — Builds privileged context from rollout consensus with minority-trajectory calibration to resist false consensus.
• Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation (2026) — Conditions the self-teacher on other-problem solutions to isolate context-induced behavior from target-specific privilege. (Code)
• PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation (2026) — Adapts a privileged teacher from completed student trajectories before distilling onto unchanged prefixes.
• Adaptive Supervised Anchoring for On-Policy Self-Distillation (2026) — Adds alignment-weighted canonical supervision beside rollout-conditioned self-distillation to restore task signal after prefix drift.
• Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation (2026) — Skill-conditioned self-teaching uses annealed tilted cross-entropy to internalize abstract guidance without privileged inference context.
• I-SDPO: Instance-Level Adaptive Self-Distillation Policy Optimization (2026) — Routes all-incorrect rollout groups to privileged self-distillation, withdrawing imitation once rewards become informative.
• Rethinking Privileged Information in On-Policy Self-Distillation (2026) — Finds OPSD gains persist without a correct privileged reference, so performance and alignment alone cannot show the reference was learned.
• DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation (2026) — Alternates student updates with teacher moves toward the student on the same trajectory, replacing OPSD's fixed privileged teacher without extra rollouts.
• Mitigating Rubric Interference in LLM Judges via On-Policy Self-Distillation (2026) — Anchors multi-rubric judging to the model's own single-rubric verdicts through on-policy self-distillation, removing interference without external supervision.
• SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation (2026) — Scores injected-input rollouts against the initialization model reading clean input, giving token-level defense signal where sequence-level preference training fails. (Code)
• Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation (2026) — Decomposes MoE self-distillation into routing and content terms, showing router movement barely shifts outputs while content supervision does.
• E²-OPSD: Taming Entropy Overshoot in On-Policy Self-Distillation (2026) — Diagnoses entropy overshoot in OPSD and fixes it with exemplar-guided teachers plus entropy-aware distillation.
• Negative Self-Distillation: Learning to Reason by Avoiding Flaws (2026) — Pushes the student away from a self-generated flawed-reasoner teacher instead of imitating privileged solutions.
• VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation (2026) — Adapts the privileged teacher toward verified student rollouts at high-divergence positions, making OPSD supervision bidirectional.
• SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation (2026) — Builds contrastive self-teachers pairing the reference answer with in-group mistakes, using their log-probability difference as dense credit.
• Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning (2026) — Supervises mostly verified on-policy scaffolds with model-generated attempts as teacher context, needing only final-answer labels.
• Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models (2026) — Trains the policy ahead to obtain a future self-teacher, then restores and distills the restarted student from it.
• Outcome-Guided On-Policy Self-Distillation (2026) — Adapts the divergence objective and distillation positions by binary outcome and cumulative teacher entropy along each rollout.
• Diagnosing On-Policy Self-Distillation for Reasoning Language Models (2026) — Controlled diagnosis showing OPSD helps only in narrow reasoning-mode compatibility regimes, otherwise causing length growth or collapse.
• Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation (2026) — Decouples step-level credit direction and magnitude via belief-margin probing and information gain to calibrate privileged supervision.
• RISE: Recursive Improvement via Self-Extrapolating Policy Distillation (2026) — Builds a synthetic teacher by extrapolating the model's own RLVR checkpoint displacement, converting sparse outcome updates into dense targets.
• On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation (2026) — Separates correctness signals from behavioral shifts in self-distillation by combining attractive and repulsive privileged teachers.
• Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning (2026) — Adds solution-conditioned attention distillation to OPSD, projecting teacher attention onto student-visible positions.
• What Does Privileged Information Add to On-Policy Self-Distillation? (2026) — Isolates the reference's contribution beyond reference-free self-distillation, finding much of OPSD's gain needs no privileged information.
• EOPSA: Efficient On-Policy Self-Distilled Safety Alignment (2026) — Concentrates safety self-distillation on early rollout tokens and safety-relevant signals, avoiding late-prefix collapse and stylistic dilution.
• Learning from Evolving Errors: Adaptive Iterative Repair for On-Policy Distillation (2026) — Iteratively generates repair guidance for each newly observed student error, supervising error-aligned regions rather than whole solutions.
• Learning from a Thoughtful Teacher: Adaptive On-Policy Self-Distillation for Mathematical Reasoning (2026) — Reveals only the affordable subgraph of a reasoning-DAG solution to the self-teacher, ordered by the student's evolving capability.
• When Clipping Reverses Correction: Failure Dynamics of Pointwise Forward-KL On-Policy Self-Distillation (2026) — Proves pointwise forward-KL clipping in OPSD can reverse correction, explaining persistent repetition in clipped runs.
• Activation-Conditioned Self-Distillation (2026) — Steers a frozen self-teacher with a verified-success activation vector, replacing reference-solution conditioning for dense supervision.
• Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation (2026) — Builds a pool of parameter-perturbed frozen experts covering more reference-aligned corrections, routed online at student-visited states.
• What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation (2026) — Compares abstracted privileged contexts against full solutions, finding strategy or category hints beat complete references with fewer tokens.
• Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention (2026) — Trains every combination of rollout source, teacher coupling, and KL direction to model self-distillation's acquisition-retention trade-off.
• Recursive Self-Improvement via On-Policy Distillation for Reasoning (2026) — Periodically refreshes the frozen privileged self-teacher with the student's improvements, making OPSD recursive.
• FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience (2026) — Calibrates trajectory-level self-guidance scores with verifier group advantages, reversing guidance on negative-advantage rollouts.
• Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models (2026) — Clusters executed solver objectives across rollouts into a majority pseudo-reference for self-distillation, improving OR models without verified answers.
• TISD: On-Policy Self-Distillation with Trajectory Intervention (2026) — Branches at peak teacher-student disagreement, regenerating the continuation from the teacher-preferred token before distilling. (Code)
• OPSRD: On-Policy Self-Role Distillation (2026) — Uses a fixed expert role prompt as privileged self-teacher context, needing no reference solutions. (Code)
• Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation (2026) — Repairs the student's own failed reasoning at its root cause and uses the repair as privileged guidance. (Code)
• Aha-Flow Distillation: Flow Markers Matter in LLM Reasoning (2026) — Pairs solution-based and rewritten flow-marker reasoning as dual privileged contexts in OPSD, shaping confident reasoning style. (Code)
• Fisher-Informed Recalibration for Feedback-Based On-Policy Self-Distillation of LLMs (2026) — Replaces self-distillation with reweighted on-policy SFT on correct outputs and Fisher-recalibrates feedback targets on incorrect ones.
• SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation (2026) — Adds a Fisher-conditioned low-rank hidden-state channel to OPSD, projecting privileged residuals onto a structured subspace.

Context and Experience Internalization

• OPCD: On-Policy Context Distillation for Language Models (2026) — Context-conditioned teacher on student rollouts; distills system prompts and experiential knowledge. (Code)
• OEL: Online Experiential Learning for Language Models (2026) — Deployment loop using OPCD for consolidating interaction traces into weights. (Code)
• Aligning Language Models from User Interactions (2026) — Hindsight self-distillation from user follow-ups; same model conditioned on the follow-up serves as the teacher.
• MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation (2026) — History-cleaned OPSD distilling assistant-stripped reference distributions onto the student's own sharded rollouts, fixing lost-in-conversation self-contamination.
• Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap (2026) — View-asymmetric self-distillation aligning on-policy multi-turn trajectories to the same model's single-turn behavior, needing no external teacher.
• Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models (2026) — Canonical-context OPSD aligning multi-turn student trajectories to a full-context frozen self-teacher, countering self-anchored drift across turns.
• Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight (2026) — Distills a weak-critic-conditioned self-teacher into a critique-free student, letting a weaker overseer improve a stronger model without test-time critiques.
• Reasoning Compression with Mixed-Policy Distillation (2026) — A larger teacher rewrites student-sampled reasoning into concise traces for KL alignment, transferring brevity instead of enforcing length penalties.
• When Context Returns: Toward Robust Internalization in On-Policy Distillation (2026) — Stop-gradient consistency regularizer anchoring the student's no-context output, making distilled privileged context safely removable when reintroduced at inference.
• PMD: Procedural Memory Distillation — Online Reflection for Self-Improving Language Models (2026) — Accumulates cross-episode procedural memory conditioning a self-teacher over the policy's own rollouts, internalizing recurring strategies into a memory-free model beyond episode-local RLVR.
• Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories (2026) — Consolidates in-context memory into freshly added expert weights during a sleep phase, distilling a pre-update self over the model's own rollouts.
• LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks (2026) — Repurposes the judge into a coach whose experiential knowledge conditions the teacher, replacing scalar rubric rewards on non-verifiable tasks.
• Sample-Efficient Learning from Agent Experience (2026) — Distills an experience-conditioned self-teacher's one-step decisions branched from the agent's own recorded trajectories, requiring no additional environment interaction.
• Masked Distillation: Internalizing the Chain-of-Thought in Language Models (2026) — Distills a CoT-conditioned teacher onto a student's answer-only rollouts, tuning a suffix scaffold that interpolates between full and no internalization.
• Flux-OPD: On-Policy Distillation with Evolving Contexts (2026) — Distills contextual difference signals into a context-free teacher anchor, letting contexts re-extracted from student trajectories evolve as in-training supervision.
• Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation (2026) — Turns failed trajectories into reflective feedback, enabling self-distillation despite rare task success.
• Self-Improving Large Language Models via Progressive Experience Evolution (2026) — Evolves reusable experience from student trajectories, then distills an experience-conditioned self-teacher into the unassisted policy. (Code)
• Latent On-Policy Self-Distillation (2026) — Learns privileged experience context as latent tokens to self-distill agent rollouts without designer-specified artifacts. (Code)
• Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing (2026) — Injects privileged tokens into off-topic student rollouts to internalize composable knowledge edits. (Code)
• EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning (2026) — Admits only experience-conditioned rollouts with positive marginal gain, then reverse-KL distills their behavior into the scaffold-free policy. (Code)
• OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation (2026) — Uses rubrics first as privileged teacher context for on-policy distillation, then as RL rewards for open-ended tasks.
• Inductive Feedback for Mixed-Policy Distillation (2026) — Treats verbal feedback as confirmation evidence over next tokens, isolating feedback-motivated signal from the teacher's inherent preferences.
• Rubric-Aware On-Policy Self-Distillation for LLM Personalization (2026) — Turns user-specific rubrics into token-level supervision by distilling a rubric-informed self-teacher onto rubric-free student rollouts. (Code)
• Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following (2026) — Ablates each constraint from the teacher's context in turn, summing leave-one-out likelihood shifts into per-constraint distillation signals. (Code)
• Training LLM Judges from Language Feedback via Position-Selective Self-Distillation (2026) — Trains LLM judges by self-distilling feedback-conditioned teachers only at positions where feedback sharpens criterion choice.
• Learning from Think-Mode Advantage via On-Policy Distillation (2026) — Routes think-trace-conditioned teacher supervision per student response by trace-response divergence, distilling think-mode advantage.

Efficiency, Systems, and Privacy Variants

• Prefix OPD: Fast and Effective On-policy Distillation from Reasoning Prefixes (2026) — Distills only reasoning prefixes, cutting training FLOPs 2×-47×.
• OVD: On-policy Verbal Distillation (2026) — Trajectory-level verbal scoring instead of token-level logit matching; relaxes alignment requirements.
• pi-Distill: Privileged Information Distillation for Language Models (2026) — Training-time privileged information in agentic settings where only actions are observable.
• Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline OPD (2026) — Precomputes teacher log-probs once over SFT rollouts; 4× speedup via teacher-consistency condition.
• DP-OPD: Differentially Private On-Policy Distillation for Language Models (2026) — Student-rollout OPD with DP-SGD on student updates; first OPD recipe with sample-level differential privacy.
• TIP: Token Importance in On-Policy Distillation (2026) — Selective training on high-entropy and confidently-wrong low-entropy tokens; matches full-token baselines at lower memory.
• Nitrobrew: Communication- and Memory-Efficient On-Policy Distillation (2026) — Hidden-state teacher→student transport plus tile-wise online divergence kernel; 1.5-3× throughput.
• NPD: Near-Policy Distillation via Asynchronous Generation and Selective Packing (2026) — Decouples generation from training; sparse updates plus Δ-IFD filtering; 8.1× speedup over on-policy baselines.
• Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning (2026) — Top-k overlap monitors prefix drift; attenuates unreliable rewards and truncates drifted rollouts.
• EffOPD: Learning to Foresee — Unlocking Efficiency of On-Policy Distillation (2026) — Adaptively extrapolates along the current update step for ~3× training acceleration with no extra trainables.
• Less is More: Early Stopping Rollout for On-Policy Distillation (2026) — Restricts rollout and reverse-KL loss to the first response tokens, where teacher supervision is strongest before it decays toward the student baseline.
• ADWIN: Adaptive Windows for Horizon-Aware On-Policy Distillation (2026) — Trains OPD on short teacher-anchored prefix windows whose horizon is adapted online by delayed full-rollout probes auditing prefix–full gradient alignment.
• Are Full Rollouts Necessary for On-Policy Distillation? (2026) — Controls OPD rollout horizon by progressively expanding or permanently truncating student rollouts, distilling only reliable early segments to cut compute.
• f-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Control (2026) — Scores per-sample freshness from rollout–supervision drift to stabilize asynchronous on-policy distillation where generation and training are decoupled.
• Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts (2026) — Prefix-overlap scores fixed-length student rollouts to allocate long-continuation budget only to teacher-compatible candidates, cutting long-horizon OPD compute.
• AsyncOPD: How Stale Can On-Policy Distillation Be? (2026) — Asynchronous OPD pipeline decoupling rollout generation from learner updates, mapping how stale rollouts degrade reverse-KL while forward-KL stays robust.
• LoRi: Low-Rank Distillation for Implicit Reasoning (2026) — Aligns the student's self-generated latent reasoning trajectory to the teacher's hidden states in a shared low-rank Tucker subspace, enabling length-invariant implicit-CoT distillation. (Code)
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation (2026) — Recovers structurally pruned LLMs by distilling the frozen pre-pruning model on student rollouts, growing rollout length as teacher-confirmed repetition subsides.
• Adaptive FastOPD: Progress-Aware Rollout Horizon Expansion for Efficient On-Policy Distillation (2026) — Expands rollout horizons with training progress, concentrating early compute on short prefixes before longer reasoning becomes learnable.
• Spend Teacher Tokens Where They Matter: Success-Referenced On-Policy Distillation (2026) — Spends teacher compute only on failed rollouts diverging from a successful sibling, sharply cutting teacher-input tokens. (Code)
• REVO: Rollout-Efficient Off-Policy Distillation via Variance-Guided Reuse (2026) — Reuses each student rollout for multiple updates with prefix weighting and resampling, prioritizing tokens by log-ratio variance. (Code)
• CLOOPD: Closing the Learner Loop in On-Policy Distillation (2026) — Freezes each teacher-scored batch and allocates multiple student update passes under a token budget, decoupling signal acquisition from realization.
• ResOPD: Tail Residualization for Sparse On-Policy Distillation (2026) — Aggregates unobserved vocabulary into a coarse tail event, giving unbiased low-variance reverse-KL gradients under sparse teacher payloads.
• Flash-OPD: Fast On-Policy Distillation (2026) — Stops each rollout at its first passage of accumulated low teacher-student compatibility, matching horizon to trajectory-specific reliability.
• Rethinking On-Policy Distillation of Large Language Models II: One Training Example (2026) — Shows single-query OPD recovers most full-data gains, explained by rapid state coverage of on-policy rollouts. (Code)
• What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection (2026) — Finds one-shot OPD effective on any example, with harder problems helping via longer chains of thought rather than entropy.
• Data-Free On-Policy Distillation: How Far Can We Go Without External Data? (2026) — Shows a handful of prompts nearly match full-data OPD, then lets the system self-generate its training questions without seed data.
• Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation (2026) — Selects which logits to differentiate by full-vocabulary teacher correction magnitude, with signed residual compensation preserving total correction.
• Leaky Students: Membership Inference against On-Policy Distillation (2026) — First membership-inference study of OPD, showing fresh student trajectories leak which records supplied privileged teacher context.
• D³-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation (2026) — Reschedules multi-teacher OPD domain mixtures online from each domain's reverse-KL convergence, at zero added overhead.
• Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers (2026) — Uses teacher-successful problems as a cheap learnability reference to retain useful supervision from imperfect teachers in offline OPD.
• Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning (2026) — Adds an OPD stage generating through the quantized forward path, correcting quantization-amplified exposure bias in low-bit reasoning.
• SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models (2026) — Stabilizes OPD for migrating ANN teachers into spiking language models, correcting prefix-source mismatch and spiking-dynamics drift.
• 1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation (2026) — Selects sparse OPD tokens by an information-efficiency ratio of gradient signal to sampling noise. (Code)

Taxonomy

Cross-cutting views over the Core papers — representative examples per category, illustrative rather than exhaustive. For the full set, see the Core OPD Papers subsections. Many entries span multiple categories.

By Teacher Type