OPD-Models

Checkpoints trained for the on-policy distillation (OPD) experiments of our paper. All other teachers and students used in the paper are public Hub checkpoints. Each model lives in its own subfolder:

Subfolder	Role in OPD	Init	Training	Weights
math_grpo500_qwen3_4b	teacher of Qwen/Qwen3-4B	Qwen/Qwen3-4B	GRPO, 500 steps × 128 prompts × 8 samples, lr 1e-6, on DeepMath-103K difficulty ≥ 6 (57,046 problems); reward: last \boxed{} checked with math-verify	BF16
code_grpo300_qwen3_4b	teacher of Qwen/Qwen3-4B	Qwen/Qwen3-4B	GRPO, 300 steps × 128 × 8, lr 1e-6, on Eurus-2-RL-Data code problems with stdio tests (22,618); reward: 1 if all tests pass	BF16
qwen3_1p7b_base_sft554	student of code_grpo400_qwen3_1p7b	Qwen/Qwen3-1.7B-Base (+ chat template)	direct-code SFT, 1 epoch, lr 3e-6, step 554	FP32 (as used in training)
code_grpo400_qwen3_1p7b	teacher of qwen3_1p7b_base_sft554	qwen3_1p7b_base_sft554	GRPO, 400 steps × 128 × 8, lr 5e-7, on open-r1/verifiable-coding-problems-python (6,519); reward: fraction of (≤ 10) tests passed	BF16

The two Qwen3-4B teachers were trained and used with enable_thinking=False.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "wyy1112/OPD-Models"
sub = "math_grpo500_qwen3_4b"
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
model = AutoModelForCausalLM.from_pretrained(repo, subfolder=sub, torch_dtype="auto")

To download a single model for vLLM / verl:

hf download wyy1112/OPD-Models --include "math_grpo500_qwen3_4b/*" --local-dir OPD-Models