Instructions to use zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B")
model = AutoModelForCausalLM.from_pretrained("zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B

SGLang

How to use zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B with Docker Model Runner:
```
docker model run hf.co/zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

TactLM-Math-ProcessGRPO-Qwen2.5-0.5B

TactLM-Math-ProcessGRPO-Qwen2.5-0.5B is Qwen2.5-0.5B-Instruct trained with GRPO on high-school math. Each reasoning step is credited by how much it raised a frozen copy of the model's chance of finishing the problem correctly. It has 494M unique parameters and also runs fully in the browser via the WebLLM build.

This is an independent run from the base Qwen model. It does not continue the TactLM instruction-following checkpoint; the two projects share a model family name, not training lineage. See the separate Math Reasoning collection.

Model details

Developed by: Zayd Syed
Base model: Qwen/Qwen2.5-0.5B-Instruct
Parameters: 494M unique, BF16 weights (tied input/output embeddings)
Language: English
License: Apache 2.0, same as the base model
Demo: runs in the browser at models.zaydsyed.com

How it was trained

Final-answer rewards give every reasoning step of a solution the same credit, so a solution with five good steps and one slip is punished on all six. The process-reward method splits each sampled solution into reasoning steps (its paragraphs). It scores each step by the change in a frozen copy's success rate (16 continuations) when finishing the solution from just before vs just after that step. Each step's tokens get the solution's outcome advantage plus 3 × that step's progress. Progress sums to the final outcome, so only the placement of credit changes.

Data: 8,478 high-school problems with numeric answers from MATH, GSM8K and NuminaMath. They were filtered to problems the base model solves 1–7 times in 8, with answer keys verified by two independent solvers.
Training: GRPO with Dr. GRPO loss, 8 problems × 8 attempts per training step, learning rate 1e-6, 3,072-token cap, one A100.
Checkpoint: chosen by validation accuracy (training step 280), before any benchmark was run.

A controlled comparison on one seed (same data order, batch and settings): per-step credit learned faster than final-answer-only GRPO, with mean validation accuracy over training of 0.498 vs 0.455. A control that shuffled the same credit across a solution's reasoning steps reached only 0.465, so most of the gain comes from where the credit lands.

Results

Greedy decoding, up to 3,072 new tokens, with the prompt the model was trained on: one worked example, then "Please reason step by step, and put your final answer within \boxed{}". Answers are compared as exact numbers. IFEval and MMLU-Redux use the TactLM instruction-following run's protocol. Every model was evaluated in the same run.

Model	GSM8K	GSM-Symbolic	MATH-500†	SVAMP	Held-out test	Fresh multi-step	IFEval	MMLU-Redux
Qwen2.5-0.5B-Instruct	47.4	40.1	32.1	55.3	57.6	30.9	26.4	41.5
Same data, final-answer reward only	49.2	42.3	33.4	58.3	61.8	28.4	27.2	42.7
ProcessGRPO (this model)	49.7	43.6	33.7	64.0	65.7	36.1	27.0	43.0

† Numeric-answer subset of MATH-500 (389 problems); Qwen2.5 has seen some of MATH.

"Held-out test" is 510 problems from the training distribution that were sealed before training. "Fresh multi-step" is 700 newly generated word problems with 2–8 chained operations.

Change from the base, paired bootstrap 95% confidence intervals (percentage points):

GSM8K	GSM-Symbolic	SVAMP	Held-out test	Fresh multi-step
+2.4 [−0.3, +4.9]	+3.4 [+2.1, +4.7]	+8.7 [+2.7, +14.7]	+8.0 [+3.1, +12.9]	+5.3 [+2.0, +8.7]

The gains concentrate on longer chains of reasoning. On fresh problems with 5 and 6 operations, this model solves 26% and 19%, against the base's 20% and 11%. The final-answer-only model fell below the base there (14% and 9%). Instruction following and general knowledge are unchanged.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "zaydsyed/TactLM-Math-ProcessGRPO-Qwen2.5-0.5B"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype="auto")
question = "Victor caught 26 shrimp and Austin caught 8 fewer. Brian caught half of their total. They sell shrimp at $7 for every 11 and split the money equally. How much does each boy make?"
messages = [{"role": "user", "content": question + "\nPlease reason step by step, and put your final answer within \\boxed{}."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
print(tokenizer.decode(model.generate(inputs, max_new_tokens=1024)[0][inputs.shape[1]:], skip_special_tokens=True))

Use the step-by-step instruction above; the model was trained with it. Low temperature (0–0.3) works best for math.

Limitations

It's a 0.5B model. It is often confidently wrong, and it isn't for anything with real consequences.
Trained only on high-school math with numeric answers. Calculus, proofs and open-ended questions are outside its training.
It can loop on problems beyond its level, repeating a line until it hits the token limit. In training this happened in about 2% of attempts, mostly on the hardest problems. A little sampling randomness (temperature around 0.3) helps it escape.
It merges reasoning steps. Late in training the model stopped separating reasoning steps with blank lines, which weakens the per-step credit, and validation peaked as this happened. The released checkpoint is from before most of that drift. Each step still sits on its own line, so answers read normally.
Single training seed; the comparison above was not replicated across seeds.