Instructions to use ModalityDance/OPTD-Qwen3-4B-Mid with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ModalityDance/OPTD-Qwen3-4B-Mid with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ModalityDance/OPTD-Qwen3-4B-Mid")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ModalityDance/OPTD-Qwen3-4B-Mid")
model = AutoModelForCausalLM.from_pretrained("ModalityDance/OPTD-Qwen3-4B-Mid", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ModalityDance/OPTD-Qwen3-4B-Mid with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ModalityDance/OPTD-Qwen3-4B-Mid"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModalityDance/OPTD-Qwen3-4B-Mid",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ModalityDance/OPTD-Qwen3-4B-Mid

SGLang

How to use ModalityDance/OPTD-Qwen3-4B-Mid with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ModalityDance/OPTD-Qwen3-4B-Mid" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModalityDance/OPTD-Qwen3-4B-Mid",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ModalityDance/OPTD-Qwen3-4B-Mid" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModalityDance/OPTD-Qwen3-4B-Mid",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ModalityDance/OPTD-Qwen3-4B-Mid with Docker Model Runner:
```
docker model run hf.co/ModalityDance/OPTD-Qwen3-4B-Mid
```

OPTD-Qwen3-4B-Mid

arXiv · Hugging Face Paper · Project Page · All OPTD models

An OPTD model trained from Qwen/Qwen3-4B with specialists in mathematics and coding. This is a complete model that can be used directly for text generation.

OPTD teaches a student both the teacher’s final prediction and how its predictions change across layers, using responses generated by the student. This checkpoint accompanies On-Policy Transition Distillation.

Mid identifies the placement of transition supervision during training. Mid and Late use different sets of layers; these names do not denote release versions or a quality ranking.

Use non-thinking mode (enable_thinking=False) for this Qwen3-4B model, as in the paper.

Quick start

Install PyTorch for your hardware, Transformers, and Accelerate:

python -m pip install torch transformers accelerate

The example below uses BF16. Teacher models and the OPTD training framework are not required for inference.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ModalityDance/OPTD-Qwen3-4B-Mid"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
messages = [{"role": "user", "content": "Solve 2x + 5 = 17."}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = tokenizer(text, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs, max_new_tokens=1024, do_sample=True, temperature=0.7, top_p=0.95
    )
response = output[0, inputs.input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

The short generation limit is for the example. Paper evaluation uses benchmark-specific token limits and sample counts.

Download

hf download ModalityDance/OPTD-Qwen3-4B-Mid --local-dir models/optd-qwen3-4b-mid

Install huggingface_hub to use the hf command. To load a downloaded copy, replace model_id with its local directory.

Training and release details

Configuration and file contents

Student initialization: Qwen/Qwen3-4B.
Training updates: 50.
Intermediate layers (zero-based): 7 / 13 / 19.
Training domains: mathematics and coding.
Full BF16 safetensors weights, tokenizer, and generation configuration are included.
training_metadata.json records the training configuration; checksums.json records checkpoint/tokenizer SHA256 hashes.
Optimizer state is not included.

Evaluation

Compare this checkpoint using the paper’s prompt mode, dataset revisions, sample counts, and graders. The example above demonstrates inference rather than reproducing benchmark scores. Generated answers can be incorrect; execute generated code only in an isolated sandbox.

Citation

XXXX.XXXXX and PROJECT_PAGE_URL are placeholders for the arXiv id and the project page.

@misc{optd2026,
  title         = {On-Policy Transition Distillation},
  author        = {Authors},
  year          = {2026},
  eprint        = {XXXX.XXXXX},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/XXXX.XXXXX}
}

License and attribution

The underlying student and specialist models retain their respective upstream license terms. Consult the linked base model's license and notices before redistribution or deployment. No new license grant for upstream model weights is implied by this model card.