Qwen3.5-4B-GGUF

GGUF builds of Qwen/Qwen3.5-4B for llama.cpp. Muniment builds these files from the official Transformers weights and ships them in its products. No prebuilt or third-party GGUF is an input to this build.

Files

File	Quantization	Size in bytes	sha256
qwen3.5-4b-Q4_K_M.gguf	Q4_K_M	2783446784	5ca0d868d45462e33c7671740bbd97b1ec4d38834827609fcd1a3f726cf49649

Provenance

Field	Value
Source repository	https://huggingface.co/Qwen/Qwen3.5-4B
Source revision	851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
llama.cpp repository	https://github.com/ggml-org/llama.cpp
llama.cpp revision	178a6c44937154dc4c4eff0d166f4a044c4fceba
High-precision intermediate	qwen3.5-4b-BF16.gguf
Converter dependency lock	evals/routing/qwen3.5-4b-requirements.lock (muniment-qa)
Python	3.10.18

Build

Run these commands from a new, empty working directory. The revision assertions must pass before any artifact is produced.

set -eu

QWEN_REV=851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
LLAMA_CPP_REV=178a6c44937154dc4c4eff0d166f4a044c4fceba

mkdir qwen3.5-4b-source
git -C qwen3.5-4b-source init
git -C qwen3.5-4b-source remote add origin https://huggingface.co/Qwen/Qwen3.5-4B
git -C qwen3.5-4b-source fetch --depth 1 origin "$QWEN_REV"
git -C qwen3.5-4b-source checkout --detach "$QWEN_REV"
test "$(git -C qwen3.5-4b-source rev-parse HEAD)" = "$QWEN_REV"
git -C qwen3.5-4b-source lfs pull origin

mkdir llama.cpp
git -C llama.cpp init
git -C llama.cpp remote add origin https://github.com/ggml-org/llama.cpp.git
git -C llama.cpp fetch --depth 1 origin "$LLAMA_CPP_REV"
git -C llama.cpp checkout --detach "$LLAMA_CPP_REV"
test "$(git -C llama.cpp rev-parse HEAD)" = "$LLAMA_CPP_REV"

python3 -m venv llama.cpp/.venv
llama.cpp/.venv/bin/python -m pip install \
  -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txt
cmake -S llama.cpp -B llama.cpp/build \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_BUILD_TESTS=OFF
cmake --build llama.cpp/build --config Release \
  --target llama-quantize llama-server --parallel

mkdir artifacts
llama.cpp/.venv/bin/python llama.cpp/convert_hf_to_gguf.py \
  qwen3.5-4b-source \
  --outfile artifacts/qwen3.5-4b-BF16.gguf \
  --outtype bf16
llama.cpp/build/bin/llama-quantize \
  artifacts/qwen3.5-4b-BF16.gguf \
  artifacts/qwen3.5-4b-Q4_K_M.gguf \
  Q4_K_M

Reproduced

This recipe was rebuilt from source in a fresh scratch directory by muniment-qa run 30313497452, which produced the same 2783446784 bytes and the same sha256. Byte-identical output depends on the pinned revisions AND on the pinned converter dependency closure. Two builds from the same revisions with floating packages produced different bytes.

Verify

sha256sum -c SHA256SUMS
sha256sum qwen3.5-4b-Q4_K_M.gguf
# 5ca0d868d45462e33c7671740bbd97b1ec4d38834827609fcd1a3f726cf49649
stat --format='%s' qwen3.5-4b-Q4_K_M.gguf
# 2783446784

Run

llama-server \
  --model qwen3.5-4b-Q4_K_M.gguf \
  --alias qwen3.5-4b-Q4_K_M \
  --ctx-size 32768

Licence and statement of changes

This repository distributes a modified form of Qwen/Qwen3.5-4B, which is licensed under Apache-2.0. The same licence applies here.

Changes made to the original work:

1. Converted the Transformers weights to GGUF at BF16 precision with convert_hf_to_gguf.py from the pinned llama.cpp revision.
2. Quantized that BF16 intermediate to Q4_K_M with llama-quantize from the same pinned revision.

No weights were retrained, merged, pruned, or otherwise altered. Model behavior follows the upstream model, subject to quantization loss.