Instructions to use zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2")
model = AutoModelForCausalLM.from_pretrained("zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2

SGLang

How to use zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2 with Docker Model Runner:
```
docker model run hf.co/zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen2.5-Math-7B-GRPO-Staleness-2

Final step-1000 full-parameter GRPO checkpoint of Qwen/Qwen2.5-Math-7B, trained on the shared 17,005-row Staleness GRPO DAPO Math dataset. Part of the Staleness collection by zbeeb.

Training

The configured staleness cap was 2 (max_off_policy_steps = 2), the maximum permitted policy age for training rollouts.

The run used Prime RL v0.9.0, GRPO with group size 8 and batch size 64, PPO clipping at 0.2, AdamW at learning rate 1e-6, 30 warmup steps, and seed 42. Training used a 4,096-token total context and a maximum of 3,072 completion tokens. The reward checks mathematical equivalence of the terminal answer; it uses no LLM judge or separate format reward. See training-config.json for settings and pinned provenance.

The 1.5B and 7B models start from Qwen2.5-Math; the 3B model starts from Qwen2.5. Differences across these models cannot be attributed to parameter count alone.

Checkpoint format

This repository contains lossless model weights in their saved floating-point dtype exported from the final distributed training checkpoint as sharded Safetensors, plus the tokenizer used during training. Optimizer and scheduler state remain in the original training checkpoint. The model can be loaded in BF16 for inference, as it was during training evaluation.

The training tokenizer uses <|im_end|> (151645) as EOS. The exported model and generation configurations use the same EOS so generation stops at the end of the assistant turn. The model architecture and original context configuration are preserved; the training and reported evaluations used a total context of 4,096 tokens.

Export validation checks the saved training step, finite tensors, tensor keys and shapes, strict Transformers reload, tokenizer round-trip, tied embeddings where applicable, and identical CPU probe logits before and after serialization. File hashes are recorded in export-manifest.json.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zbeeb/Qwen2.5-Math-7B-GRPO-Staleness-2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
messages = [{'role': 'user', 'content': 'Solve the following math problem. Explain your reasoning. End with either \\boxed{...} or a final line `Final answer: ...`.\n\nCompute 2 + 2.'}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
assert inputs["input_ids"].shape[-1] <= 1024
output = model.generate(**inputs, max_new_tokens=3072, do_sample=False, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Final training-run evaluation

Self-reported results for policy step 1000 with the run's deterministic final-answer grader. Greedy rows use one completion per problem. The sampled rows use eight completions per problem at temperature 0.6 and report mean answer accuracy, not pass@8. MATH-500, AMC and AIME use a 3,072-token completion limit; Minerva and OlympiadBench use 2,048.

Benchmark	Completions	Accuracy	Truncated
MATH-500	500	74.60%	2.0%
AMC23	40	55.00%	7.5%
AIME24	30	26.67%	10.0%
AIME25	30	10.00%	10.0%
Minerva Math	272	26.10%	0.7%
OlympiadBench	675	40.15%	6.2%
AIME24 sampled mean	240	21.25%	14.2%
AIME25 sampled mean	240	11.67%	12.1%
AIME26 sampled mean	240	10.42%	13.3%

Exact source revisions and machine-readable results are included in evaluation-results.json and the dataset provenance. The broader post-training benchmark sweep has not yet run. These are final-answer scores and do not establish proof quality. Training data was filtered against the retained evaluations, but this does not establish absence of pretraining contamination or prove that every near-duplicate was removed. Long completions can be truncated.

Data and license

Training data: zbeeb/Staleness-GRPO-DAPO-Math-17k. Base-model revision: b101308fe89651ea5ce025f25317fea6fc07e96e. This checkpoint modifies the base model through 1,000 GRPO training updates. The upstream license is included unchanged in LICENSE.