Instructions to use zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6")
model = AutoModelForCausalLM.from_pretrained("zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6

SGLang

How to use zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6 with Docker Model Runner:
```
docker model run hf.co/zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6
```

Qwen2.5-Math-1.5B-GRPO-Staleness-6

Final step-1000 full-parameter GRPO checkpoint of Qwen/Qwen2.5-Math-1.5B, trained on the shared 17,005-row DAPO math dataset.

Training

Staleness cap: 6 (max_off_policy_steps). This limits rollout policy age during training, not decoding length. The run used 2 trainer GPUs and 1 inference GPUs. The model started from the pinned base model with fresh optimizer state; higher-cap runs are not continuations of lower-cap models.

PrimeRL v0.9.0; 1,000 updates; batch size 64; group size 8; seed 42; AdamW learning rate 1e-6; 30 warmup updates; PPO clip 0.2; no reference KL penalty. Training uses a 4,096-token total context and up to 3,072 completion tokens. The deterministic reward scores mathematical equivalence of the terminal answer. See training-config.json. GPU topology varies across some arms, so comparisons are not pure staleness ablations.

Export and stopping

Full saved-dtype model weights are exported losslessly to sharded Safetensors. Export checks include step-1000 provenance, finite tensors, strict reload, tied embeddings, tokenizer round-trip, and identical CPU probe logits before and after serialization. Optimizer state remains in the source checkpoint. See export-manifest.json.

The training tokenizer identifies <|im_end|> (151645) as EOS. For inference compatibility, generation_config.json stops on both 151645 and <|endoftext|> (151643). Model weights and tokenizer are unchanged by this stopping configuration. Use explicit stop IDs if your serving engine ignores the generation configuration. The native positional configuration is preserved; this release does not configure or validate 8K context extension.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = 'zbeeb/Qwen2.5-Math-1.5B-GRPO-Staleness-6'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
messages = [{'role': 'user', 'content': 'Solve the following math problem. Explain your reasoning. End with either \\boxed{...} or a final line `Final answer: ...`.\n\nCompute 2 + 2.'}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
assert inputs["input_ids"].shape[-1] <= 1024
output = model.generate(**inputs, max_new_tokens=3072, do_sample=False, eos_token_id=[151645, 151643], pad_token_id=tokenizer.pad_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Final training-run evaluations

Scores use final policy 1000. Greedy rows use one completion per question. Sampled rows use eight completions per question at temperature 0.6 and report mean answer accuracy, not pass@8. MATH500, AMC and AIME use 3,072 output tokens; Minerva and OlympiadBench use 2,048. All nine final sets have zero recorded evaluation errors. These are training-run results, not a fresh benchmark of the exported artifact or an 8K comparison.

Benchmark	Completions	Accuracy	Truncated
aime24-pass1	30	13.33%	16.67%
aime24-sampled	240	10.42%	10.00%
aime25-pass1	30	6.67%	20.00%
aime25-sampled	240	5.00%	5.00%
aime26-sampled	240	7.50%	11.25%
amc23-pass1	40	45.00%	2.50%
math500-pass1	500	62.40%	2.40%
minerva-pass1	272	15.44%	19.85%
olympiadbench-pass1	675	29.63%	8.00%

Machine-readable results and dataset revisions: evaluation-results.json. Training data was filtered against these evaluations; this does not establish absence of pretraining contamination or all near-duplicates. The 3B model uses general Qwen2.5, whereas 1.5B uses Qwen2.5-Math; cross-family differences are not attributable solely to size.