oliver0804/MiniCPM5-2B-RK3588-rkllm-w8a8

MiniCPM5-2B — RKLLM w8a8 for the RK3588 NPU

openbmb/MiniCPM5-2B converted to Rockchip RKLLM w8a8 format for the RK3588 / RK3588S NPU. Converted with rkllm-toolkit 1.3.1; requires the librkllmrt.so 1.3.1 runtime (older runtimes fail to load the file). No architecture changes were made.

Benchmark

Radxa ROCK 5C (RK3588S, 8 GB LPDDR4x), clocks locked with the official fix_freq_rk3588.sh (CPU 1.8/2.3 GHz, DDR 2112 MHz, NPU 1 GHz), greedy decoding, thinking disabled, max_new_tokens 768, fixed 7-question zh-TW benchmark (word problem, facts, numeric comparison, letter counting, syllogism, code generation, constrained formatting):

score	generate	prefill	memory
6 / 7	10.9 tok/s	189 tok/s	~2.5 GB

The only miss is "9.11 vs 9.9". Answers occasionally mix in simplified-Chinese characters. For reference on the same board and benchmark: Gemma-4-E2B-it 6/7 @ 9.3 tok/s (2.6 GB), Qwen2.5-1.5B 4.5/7 @ 15.8 tok/s (1.8 GB).

Usage

Load with any RKLLM 1.3.1 runtime from airockchip/rknn-llm, e.g. the C++ demo:

./llm_demo MiniCPM5-2B_RK3588_w8a8.rkllm 2048 4096

The file is self-contained (weights + tokenizer); just point your runtime at it.

License

Apache-2.0, inherited from the base model (see LICENSE).