abenzerps/MiniCPM5-2B-GGUF

MiniCPM5-2B GGUF

GGUF quantizations of OpenBMB/MiniCPM5-2B, a 2B dense Llama-based model for local deployment, coding, reasoning, long-context work, and tool use. The source checkpoint supports a native context length of 131,072 tokens (128K).

Benchmarks

Benchmark results reported by OpenBMB for the original MiniCPM5-2B checkpoint.

Capability comparison reported by OpenBMB for the original MiniCPM5-2B checkpoint.

GGUF files

Quantization	File	Size
Q2_K	MiniCPM5-2B-Q2_K.gguf	1.04 GB
Q3_K_M	MiniCPM5-2B-Q3_K_M.gguf	1.29 GB
Q4_0	MiniCPM5-2B-Q4_0.gguf	1.49 GB
Q4_K_S	MiniCPM5-2B-Q4_K_S.gguf	1.50 GB
Q4_K_M	MiniCPM5-2B-Q4_K_M.gguf	1.56 GB
Q5_K_M	MiniCPM5-2B-Q5_K_M.gguf	1.81 GB
Q6_K	MiniCPM5-2B-Q6_K.gguf	2.07 GB
Q8_0	MiniCPM5-2B-Q8_0.gguf	2.68 GB
IQ2_M	MiniCPM5-2B-IQ2_M.gguf	0.97 GB
IQ3_M	MiniCPM5-2B-IQ3_M.gguf	1.23 GB
IQ4_XS	MiniCPM5-2B-IQ4_XS.gguf	1.42 GB

The model is text-only. No vision projector or MTP files are included. The IQ files use an importance matrix generated from WikiText-2 and are intended for recent llama.cpp builds. SHA-256 checksums are provided in SHA256SUMS.txt.

Chat template

The GGUF files embed the upstream chat template. chat_template.jinja is provided as an external copy for runtimes that require a separate template file.

Usage

Use a current llama.cpp build with MiniCPM5 support. The example below uses an 8K context; increase -c up to 131072 when sufficient memory is available.

llama-cli \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja \
  --temp 1.0 --top-p 0.95 \
  -p "Explain why reproducible builds matter."

For an OpenAI-compatible server:

llama-server \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja --host 0.0.0.0 --port 8080

Tool-call behavior depends on the serving runtime's parser and API integration; use the embedded template and verify tool calls in the target application.

Source

• Model: OpenBMB/MiniCPM5-2B
• Source revision: 3497c460c89e00520c3cfa2e73f49ab7647f1177
• Conversion: the original Q4_0–Q8_0 files use upstream llama.cpp commit f114f91f9ed6792cf402437e3874adad98902744; the additional Q2_K, Q3_K_M, Q4_K_S, IQ2_M, IQ3_M, and IQ4_XS files use upstream commit 67672dc5b76f8bc17785a19d3dc6d1463fc2902c
• License: Apache-2.0
• Checksums: SHA256SUMS.txt