Instructions to use cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100")
model = AutoModelForCausalLM.from_pretrained("cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100

SGLang

How to use cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100 with Docker Model Runner:
```
docker model run hf.co/cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100
```

Qwen2.5-7B-Instruct — Direct-OPD transfer of the OpenThinker3 reasoning-SFT shift, 100 steps

Artifact of the Direct-OPD SFT-transfer experiment (direct-opd-sft-transfer), extension condition openthinker. H2 — with a chat→reasoning SFT pair the student gains on at least one held-out benchmark (paired CI excluding 0).

The training signal is the token-level shift log pi_post - log pi_pre, evaluated on the student's own sampled tokens. Neither teacher is imitated; only the difference between them is.

role	model	what it is
`pi_pre` — teacher_ref (`TEACHER_REF_MODEL_PATH`)	`Qwen/Qwen2.5-1.5B-Instruct` @ `989aa7980e4cf806f80c7fef2b1adb7bc71aa306`	the pre-shift reference: the instruct model the reasoning SFT started from
`pi_post` — teacher (`REWARD_MODEL_PATH`)	`open-thoughts/OpenThinker3-1.5B` @ `0ee90a38b29bfac8b8b005da9ae32c59e2943785`	the post-shift model: that same 1.5B checkpoint after SFT-only training (7 epochs on OpenThoughts3-1.2M). No RL.
student init	`Qwen/Qwen2.5-7B-Instruct` @ `a09a35458c702b33eeacc393d103063234e8bc28`	non-thinking instruct model, 7.6 B params (5.1x the teachers)

Because pi_post was fine-tuned from pi_pre, this shift isolates reasoning SFT alone — unlike the experiment's first run, whose base→SFT pair also bundled "became a chat model".

Root = step 100. checkpoint-{20,40,60,80,100}/ = intermediate merged checkpoints. Weights are bf16 (verl's FSDP->HF merge downcasts the fp32 masters).

Training configuration

code: https://github.com/BytedTsinghua-SIA/Direct-OPD @ 3a9d6bd37b00a38e7a9b2959239e4631e5324aea + logs/phase4_seed.patch (seed 42 shim)
data: cmpatino/direct-opd-sft-deepmath-pilot-data @ 22625ae5db434947195bf862c429cd94504a4809 :: opd_train.parquet (6,400 AIME-decontaminated prompts, one pass)
100 steps x 64 prompts x 4 rollouts, lr 2e-7, token-mean loss, fp32 masters
KL coefficient: adaptive, init 2.5, clamped to [0.5, 2.5], eps 0.01 (x(1−eps) after a negative mean shift reward, x(1+eps) after a positive one)
sequence budget 768 prompt + 3328 response = 4096 (both teachers allow 32,768 positions, so this is a deliberate rollout budget, not a context limit)
top-k 16 only_stu, student_p weighting, T=1.0 for student and teachers, reward_model.model.input_tokenizer=null (teachers score the student's rendered ids verbatim)
4 x A100-80GB, FSDP full-shard, gradient checkpointing + activation offload, actor optimizer offload True, log-prob token budgets ref/rollout 8192/8192
driver + full env block: logs/run_manifest.json, console log logs/train.log.gz

Caveats

No in-training validation (test_freq=-1); all evaluation is external and pre-registered.
Bit-exact reproducibility is not attainable (vLLM continuous batching, dynamic micro-batching, FSDP reduction order). Seeding fixes data order and engine seeds only.
Prompts are rendered with the student's chat template in training and in evaluation (verl RLHFDataset, rl_dataset.py:363).
The trio's tokenizers agree over the whole defined id range (P0 gate: ordinary ids [0, 151642] identical, added tokens identical, chat templates byte-identical), so the cross-tokenizer caveat that applied to this experiment's first run does not apply here.
Qwen2.5-7B-Instruct's template injects its default system prompt in training and in evaluation (same apply_chat_template(..., add_generation_prompt=True) call) — consistent by construction.

Downloads last month: 3

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for cmpatino/Qwen2.5-7B-Instruct-DirectOPD-OpenThinker3Shift-lr2e-7-100

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Finetuned

(3032)

this model