HelloSun/openPangu-R-7B-2512-Q4KM-gguf

openPangu-R-7B-2512-Q4_K_M (GGUF)

Powered by openPangu. openPangu is a trademark of Huawei Technologies Co., Ltd.

Q4_K_M GGUF conversion of openpangu/openPangu-R-7B-2512. This is an unofficial community conversion, not released or endorsed by Huawei.

Important

• Upstream llama.cpp does not support this model. The GGUF uses the architecture name openpangu-r, which exists only in the patch in this repository. The patch targets llama.cpp v0.7.0 (latest stable release).
• Tested on CPU only (x86-64, AVX-512). GPU backends were not tested.

Quick start

bash build_openpangu_r.sh          # clones llama.cpp v0.7.0, applies the patch, builds, downloads the GGUF

Options: SKIP_MODEL=1 skips the download; HF_TOKEN is only needed if this repository is private. The script checks the sha256 of the patch and the GGUF.

Run (raw prompt, Chinese chat template of the model):

./llama.cpp/build/bin/llama-completion -m models/openPangu-R-7B-2512-Q4_K_M.gguf -no-cnv --temp 0 -n 128 --no-display-prompt \
  -p "[unused9]系统：你是一个有帮助的助手。[unused10][unused9]用户：用一句话介绍什么是大语言模型。 /no_think[unused10][unused9]助手：[unused16]"

Prompt format: [unused9]系统：…[unused10][unused9]用户：…[unused10][unused9]助手：[unused16]. Append /no_think to a user message for fast (non-thinking) answers. The answer follows [unused17]; generation ends with [unused10].

Conversion

1. convert_hf_to_gguf.py (patched) with --outtype f16 from the BF16 safetensors. The MTP module (layer 27) and the second final norm are dropped.
2. llama-quantize <f16.gguf> <out.gguf> Q4_K_M. Sinks and the conv weight are kept in F32.

Verification (done in this environment)

• Reference check: a CPU fp32 reference was written from the HF modeling code. For a 33-token prompt, the top-10 logits match llama.cpp F16 (max |Δlogit| ≈ 0.01). For a 484-token prompt (sliding-window layers active), the top-10 match with max |Δlogit| ≈ 0.03. Turning the sliding window off in the reference changes the logits by about 6, so the window is verified.
• Greedy decoding: llama.cpp F16 matches the fp32 reference token by token (11 tokens checked).
• Batching: F16 gives the same results with prompt decoding in one batch, in chunks of 4, and token by token.
• Long context: a fact placed about 400 tokens before the question was answered correctly (7312).
• Multi-turn: a second user turn appended to the same context is answered in context.

Sample outputs (Q4_K_M):

prompt	output
用一句话介绍什么是大语言模型（快思考）	大语言模型是一种基于大量文本数据训练而成的人工智能模型，能够理解和生成人类语言，广泛应用于自然语言处理任务。
写一首关于秋天的四行短诗（快思考）	秋风轻拂叶纷飞，/ 金黄铺满小径长。/ 寒蝉低唱夕阳下，/ 一缕炊烟入梦乡。
一个班有30名学生，其中2/5是女生，女生有多少人？（慢思考）	12人
123 + 456 等于多少？（慢思考）	579

Note: in fast mode the model once answered 123 + 456 = 679. The fp32 reference gives the same answer for that prompt, so this is a model behavior, not a conversion or implementation error. Thinking mode answers 579.

Known limitations

• One sequence per batch. A batch with more than one sequence aborts. The conv history of the merge layers is kept inside the model and follows one sequence decoded contiguously. Prompt-cache reuse, seq_rm, and multi-slot serving are not supported.
• The MTP (next-token) head is not included and not used.
• Hadamard-rotated (quantized) KV cache is not supported; use the default f16 KV cache.
• llama-server was not validated.

License and notices

The model and derivative works are licensed under the OPENPANGU MODEL LICENSE AGREEMENT VERSION 1.0 (see LICENSE).

• EU restriction (Section 3): you must not access, download, install, run, deploy, integrate, modify, or otherwise use this model within the European Union.
• Acknowledgment (Section 4.2): "Powered by openPangu" and the trademark notice "openPangu is a trademark of Huawei Technologies Co., Ltd." must appear on pages, manuals, or materials that describe features of the model.
• The model is provided "AS IS" without warranty. Outputs are generated automatically and may be inaccurate.

Files and checksums

file	sha256
openPangu-R-7B-2512-Q4_K_M.gguf	efeb81edbf64ad9d7a80254332f3128d86deec589ee39315a3c1f7ec1aa5f10a
openpangu-r-llama.cpp-v0.7.0.patch	b5f302da0cece9c6bc717982d820f8b3e6ca0970f8f32b549efb41ef0d6b8773
LICENSE	747083c8c24995c75ed641d5c031a95a859b1ec37cd3deeb033425a75249a3cb