Instructions to use sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH")
model = AutoModelForCausalLM.from_pretrained("sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH

SGLang

How to use sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH with Docker Model Runner:
```
docker model run hf.co/sunblaze-ucb/Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH
```

Llama-3.2-3B-Instruct-GRPO-MATH-1EPOCH

This model is a GRPO-fine-tuned version of meta-llama/Llama-3.2-3B-Instruct trained on the MATH dataset for one epoch. It was developed as part of the research presented in the paper Learning to Reason without External Rewards.

The official code implementation is available in the Intuitor GitHub repository.

Model Description

The model explores Reinforcement Learning from Internal Feedback (RLIF), a framework that enables Large Language Models (LLMs) to learn from intrinsic signals—such as self-certainty—without the need for external rewards or labeled data. This specific checkpoint serves as a baseline for the Intuitor method described in the paper, which uses a model's own confidence as its sole reward signal in Group Relative Policy Optimization (GRPO).

Paper: Learning to Reason without External Rewards
Repository: sunblaze-ucb/Intuitor

Citation

@article{zhao2025learning,
  title={Learning to Reason without External Rewards},
  author={Zhao, Xuandong and Kang, Zhewei and Feng, Aosong and Levine, Sergey and Song, Dawn},
  journal={arXiv preprint arXiv:2505.19590},
  year={2025}
}