Instructions to use tinyopsec/GenPRM-7B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use tinyopsec/GenPRM-7B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/tinyopsec/GenPRM-7B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use tinyopsec/GenPRM-7B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tinyopsec/GenPRM-7B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tinyopsec/GenPRM-7B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Ollama
How to use tinyopsec/GenPRM-7B-GGUF with Ollama:
```
ollama run hf.co/tinyopsec/GenPRM-7B-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use tinyopsec/GenPRM-7B-GGUF with Docker Model Runner:
```
docker model run hf.co/tinyopsec/GenPRM-7B-GGUF:Q4_K_M
```

Lemonade

How to use tinyopsec/GenPRM-7B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.GenPRM-7B-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

GenPRM-7B-GGUF

GGUF quantizations of GenPRM/GenPRM-7B, a generative process reward model for mathematical reasoning.

Model Details

GenPRM-7B is a Qwen2-based process reward model trained from the DeepSeek-R1-Distill-Qwen-7B base model. It performs explicit chain-of-thought reasoning and code verification before producing process judgments. It supports test-time scaling through parallel generation and majority voting, and can be used as a verifier or critic.

Quantizations

File	Bits	Size	Recommended Memory	Use Case
`model_f16.gguf`	16-bit	15.2 GB	—	Maximum quality
`model_q8_0.gguf`	8-bit	8.1 GB	10+ GB	Near-F16 quality
`model_q6_k.gguf`	6-bit	6.25 GB	8+ GB	High quality
`model_q5_k_m.gguf`	5-bit	5.44 GB	7+ GB	Quality / size balance
`model_q5_k_s.gguf`	5-bit	5.32 GB	7+ GB	Compact 5-bit
`model_q4_k_m.gguf`	4-bit	4.68 GB	6+ GB	Recommended general use
`model_q4_k_s.gguf`	4-bit	4.46 GB	6+ GB	Compact 4-bit
`model_q3_k_l.gguf`	3-bit	4.09 GB	5+ GB	Low-memory use
`model_q3_k_m.gguf`	3-bit	3.81 GB	5+ GB	Smaller deployment
`model_q3_k_s.gguf`	3-bit	3.49 GB	5+ GB	Maximum compression
`model_q2_k.gguf`	2-bit	3.02 GB	4+ GB	Minimum memory

Actual requirements depend on context length, KV cache, and runtime configuration.

Hardware Compatibility

Bits	Quant	Size
2-bit	Q2_K	3.02 GB
3-bit	Q3_K_S	3.49 GB
3-bit	Q3_K_M	3.81 GB
3-bit	Q3_K_L	4.09 GB
4-bit	Q4_K_S	4.46 GB
4-bit	Q4_K_M	4.68 GB
5-bit	Q5_K_S	5.32 GB
5-bit	Q5_K_M	5.44 GB
6-bit	Q6_K	6.25 GB
8-bit	Q8_0	8.1 GB
16-bit	F16	15.2 GB

Usage

llama.cpp

llama-cli -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Server:

llama-server -hf tinyopsec/GenPRM-7B-GGUF:Q4_K_M

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="model_q4_k_m.gguf",
    n_ctx=8192,
)

output = llm(
    "Review the following mathematical solution step by step.",
    max_tokens=2048,
)
print(output)

LM Studio

Download the desired GGUF file and load it through LM Studio.

Ollama

ollama run hf.co/tinyopsec/GenPRM-7B-GGUF:Q4_K_M

Original Model

GenPRM/GenPRM-7B

GenPRM was introduced in GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning. The model uses explicit reasoning and verification for process supervision and supports both verifier and critic applications.

Features

Generative process reward model
Mathematical reasoning and process supervision
Explicit chain-of-thought reasoning
Code verification
Relative Progress Estimation (RPE)
Parallel test-time scaling
Majority-vote verification
Critique and refinement support
Qwen2 architecture
GGUF format for llama.cpp-compatible runtimes

Citation

@article{zhao2025genprm,
  title   = {GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning},
  author  = {Jian Zhao and Runze Liu and Kaiyan Zhang and Zhimu Zhou and Junqi Gao and Dong Li and Jiafei Lyu and Zhouyi Qian and Biqing Qi and Xiu Li and Bowen Zhou},
  journal = {arXiv preprint arXiv:2504.00891},
  year    = {2025}
}

Model tree for tinyopsec/GenPRM-7B-GGUF

Base model

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Finetuned

GenPRM/GenPRM-7B

Quantized

(2)

this model

Paper for tinyopsec/GenPRM-7B-GGUF

GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning

Paper • 2504.00891 • Published Apr 1, 2025 • 15

tinyopsec
/

GenPRM-7B-GGUF