Instructions to use zbeeb/Qwen2.5-3B-Base-GRPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zbeeb/Qwen2.5-3B-Base-GRPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zbeeb/Qwen2.5-3B-Base-GRPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zbeeb/Qwen2.5-3B-Base-GRPO")
model = AutoModelForCausalLM.from_pretrained("zbeeb/Qwen2.5-3B-Base-GRPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zbeeb/Qwen2.5-3B-Base-GRPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zbeeb/Qwen2.5-3B-Base-GRPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-3B-Base-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zbeeb/Qwen2.5-3B-Base-GRPO

SGLang

How to use zbeeb/Qwen2.5-3B-Base-GRPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zbeeb/Qwen2.5-3B-Base-GRPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-3B-Base-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zbeeb/Qwen2.5-3B-Base-GRPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zbeeb/Qwen2.5-3B-Base-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zbeeb/Qwen2.5-3B-Base-GRPO with Docker Model Runner:
```
docker model run hf.co/zbeeb/Qwen2.5-3B-Base-GRPO
```

Qwen2.5-3B-Base-GRPO

Built with Qwen. This is the final base → GRPO checkpoint in the OpenR1 SFT to GRPO token-level study. GRPO started directly from the original Qwen2.5-3B base weights, without an SFT stage.

The exact initial checkpoint is Qwen/Qwen2.5-3B at revision 3aab1f1954e9cc14eb9509a215f9e5ca08227a9b. Its weight checksums were matched against the recorded training inputs.

Completion

The run completed 1,000 optimizer updates, with 799 nonzero-gradient updates and 201 zero-gradient updates. All 1,000 steps have stable token exports from all four trainer ranks, and the independent selected-token probability checks passed. Zero-gradient counts refer to the gradient norm before clipping; Adam momentum and weight decay can still change weights.

Benchmark evaluations and fixed-probe evaluations were disabled for this GRPO run. This release includes no new evaluation scores or claims of improved benchmark performance.

GRPO recipe

Setting	Value
Framework	Prime RL 0.9.0
Prime revision	`ab5de8fff44b2c4a5c85e24b6e6e3f7d57eee7b1`
RL dataset	zbeeb/Staleness-GRPO-DAPO-Math-17k
Dataset revision	`53064564abf94eac096877a61d63e92ac4217433`
Dataset rows	17,005
Optimizer updates	1,000
Response batch	32 responses, four prompts × eight responses
Learning rate	1e-06
Schedule	30-update warmup, then constant
Optimizer	AdamW; weight decay 0.01; gradient clipping 1.0
Policy loss	PPO ratio clipping at 0.2; no reference-policy KL penalty
Advantage	Reward minus group mean
Zero-advantage groups	Retained
Maximum off-policy age	8 updates
Sampling	Temperature 1; top-p 1
Maximum response	3,072 tokens
Context budget	4,096 tokens
Seed	42
Topology	Four trainer GPUs and four inference GPUs on one A100 node
Saved tensor dtypes	F32

Files

The saved final weights, tokenizer, chat template, and generation configuration are preserved byte for byte. training-config.json records the resolved configuration with credentials and local paths removed. provenance.json, learning-signal.json, token-logs.json, and export-manifest.json contain initialization checksums, completion evidence, source hashes, gradient counts, and export checksums.

study-metrics-rank-*.jsonl includes all 1,000 updates for each trainer rank. metrics.jsonl.gz contains sanitized native run metrics. Full token-level Parquet exports, rollouts, and system logs remain preserved in the experiment storage.

Load

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "zbeeb/Qwen2.5-3B-Base-GRPO"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, dtype=torch.bfloat16, device_map="auto")

License and attribution

The upstream Qwen Research License is retained verbatim in LICENSE, and Notice records attribution and the training modifications. The upstream model is Qwen/Qwen2.5-3B at revision 3aab1f1954e9cc14eb9509a215f9e5ca08227a9b.

Downloads last month: 23

Safetensors

Model size

3B params

Tensor type

F32

Model tree for zbeeb/Qwen2.5-3B-Base-GRPO

Base model

Qwen/Qwen2.5-3B

Finetuned

(608)

this model

zbeeb
/

Qwen2.5-3B-Base-GRPO