Instructions to use HanzoHuang/MiniCPM5-2B-RKNN3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with RKLLM:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf HanzoHuang/MiniCPM5-2B-RKNN3
# Run inference directly in the terminal:
llama cli -hf HanzoHuang/MiniCPM5-2B-RKNN3

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf HanzoHuang/MiniCPM5-2B-RKNN3
# Run inference directly in the terminal:
llama cli -hf HanzoHuang/MiniCPM5-2B-RKNN3

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf HanzoHuang/MiniCPM5-2B-RKNN3
# Run inference directly in the terminal:
./llama-cli -hf HanzoHuang/MiniCPM5-2B-RKNN3

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf HanzoHuang/MiniCPM5-2B-RKNN3
# Run inference directly in the terminal:
./build/bin/llama-cli -hf HanzoHuang/MiniCPM5-2B-RKNN3

Use Docker

docker model run hf.co/HanzoHuang/MiniCPM5-2B-RKNN3

LM Studio
Jan

vLLM

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HanzoHuang/MiniCPM5-2B-RKNN3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HanzoHuang/MiniCPM5-2B-RKNN3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HanzoHuang/MiniCPM5-2B-RKNN3

Ollama
How to use HanzoHuang/MiniCPM5-2B-RKNN3 with Ollama:
```
ollama run hf.co/HanzoHuang/MiniCPM5-2B-RKNN3
```
Unsloth Desktop

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf HanzoHuang/MiniCPM5-2B-RKNN3

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "HanzoHuang/MiniCPM5-2B-RKNN3"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use HanzoHuang/MiniCPM5-2B-RKNN3 with Docker Model Runner:
```
docker model run hf.co/HanzoHuang/MiniCPM5-2B-RKNN3
```

Lemonade

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull HanzoHuang/MiniCPM5-2B-RKNN3

Run and chat with the model

lemonade run user.MiniCPM5-2B-RKNN3-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf HanzoHuang/MiniCPM5-2B-RKNN3

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default HanzoHuang/MiniCPM5-2B-RKNN3

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use HanzoHuang/MiniCPM5-2B-RKNN3 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf HanzoHuang/MiniCPM5-2B-RKNN3

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "HanzoHuang/MiniCPM5-2B-RKNN3" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiniCPM5-2B for RKNN3 / RKLLM3 Server

RKNN3-converted openbmb/MiniCPM5-2B LLM artifacts for the Rockchip RK1820 and RK1828 AI coprocessors, ready for deployment with Rockchip's RKLLM3 Server (rkllm3-server) or the RKNN3 Runtime examples.

These are hardware-specific deployment artifacts, not a Transformers checkpoint and not an RKLLM model. They were produced with rknn3-toolkit v1.1.0 for the new RKNN3 toolchain and are intended for use with a compatible RKNN3 Runtime, firmware, demo, or rkllm3-server.

Base model

Upstream model: openbmb/MiniCPM5-2B
Model type: causal language model (LlamaForCausalLM)
Parameters: 2.52B total, 1.98B excluding embeddings
Upstream context length: 131,072 tokens
Upstream license: Apache-2.0

The usable context length of an RKNN3 deployment is determined by the converted model and runtime configuration. Do not assume that it is identical to the upstream checkpoint's maximum context length.

Conversion

Item	Value
Toolkit	rknn3-toolkit v1.1.0
Conversion reference	rknn3-model-zoo MiniCPM5 example
Targets	RK1820 and RK1828
Quantization	W4A16, `normal`, `group32`
Output format	Weight-separated RKNN3 (`.rknn` + `.weight`)

RKNN3 is a separate toolchain from RKLLM Toolkit, RKNN-Toolkit, and RKNN-Toolkit2. Use RKNN3 v1.1.0 runtime components and RK182x firmware that are compatible with these converted artifacts.

Files

.
├── MiniCPM5-2B-w4a16.weight
├── MiniCPM5-2B.embed.bin
├── MiniCPM5-2B.tokenizer.gguf
├── RK1820/
│   └── MiniCPM5-2B-RK1820-w4a16.rknn
└── RK1828/
    └── MiniCPM5-2B-RK1828-w4a16.rknn

The tokenizer, FP16 embedding table, and W4A16 weight file are shared by both targets. The RK1820 and RK1828 conversions produced byte-identical .weight files, so this repository stores one copy at the root. The .rknn files differ and you must select the one matching your coprocessor.

Target	RKNN graph	Shared weight	Tokenizer	Embedding
RK1820	MiniCPM5-2B-RK1820-w4a16.rknn	MiniCPM5-2B-w4a16.weight	MiniCPM5-2B.tokenizer.gguf	MiniCPM5-2B.embed.bin
RK1828	MiniCPM5-2B-RK1828-w4a16.rknn	MiniCPM5-2B-w4a16.weight	MiniCPM5-2B.tokenizer.gguf	MiniCPM5-2B.embed.bin

Keep all four files from this repository revision together. A model graph or weight from a different conversion may be incompatible even when its filename looks similar.

Download

Install the current Hugging Face CLI and download the complete repository:

hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  --local-dir MiniCPM5-2B-RKNN3

To download only the RK1820 runtime set:

hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  MiniCPM5-2B-w4a16.weight \
  MiniCPM5-2B.tokenizer.gguf \
  MiniCPM5-2B.embed.bin \
  --local-dir MiniCPM5-2B-RKNN3

Replace RK1820 with RK1828 and use the corresponding filename for an RK1828 deployment.

Run with rkllm3-server

The official rkllm3-server accepts the graph, separated weight, tokenizer, and embedding as explicit paths. For RK1820:

/usr/bin/rkllm3-server \
  -m MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  --weight MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  --vocab MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  --embed MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin

For RK1828, change only the graph path:

-m MiniCPM5-2B-RKNN3/RK1828/MiniCPM5-2B-RK1828-w4a16.rknn

Use the server binary, RKNN3 Runtime libraries, transfer proxy, and RK182x firmware supplied for the same compatible RKNN3 release. RK1820/RK1828 operate as coprocessors and require a supported host platform and connection method.

Run the rknn3-model-zoo C++ demo

Build and deploy the official MiniCPM5 example as described by the rknn3-model-zoo. From the deployed demo directory, an RK1820 invocation is:

export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH

./rknn_minicpm5_demo \
  /path/to/MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin \
  0xff \
  "Explain the theory of relativity"

For RK1828, select the graph under RK1828/. The 0xff argument is the NPU core mask used in Rockchip's example; adjust it for your deployment when necessary.

Compatibility and limitations

These files require the RKNN3 software stack. They cannot be loaded by Transformers, llama.cpp, the legacy RKLLM Runtime, RKNN-Toolkit, or RKNN-Toolkit2.
The .rknn graph is target-specific. Do not use the RK1820 graph on RK1828 or the RK1828 graph on RK1820.
The root .weight, .tokenizer.gguf, and .embed.bin files are shared by the two graphs in this repository revision.
Quantization can change generation quality relative to the upstream BF16 model. Validate accuracy and performance for your application.
Runtime behavior depends on the installed RKNN3 Runtime, host software, transfer method, and RK182x firmware.

References and acknowledgements

Thanks to OpenBMB, Rockchip, and the RKNN community.

Downloads last month: 132

GGUF

Model size

0 params

Architecture

llama

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for HanzoHuang/MiniCPM5-2B-RKNN3

Base model

openbmb/MiniCPM5-2B

Quantized

(79)

this model