Qwen3.8-27B-UD3-Q5_K_M-MLX

Native MLX affine re-encoding of the official Unsloth Dynamic v3.0 Q5_K_M per-tensor allocation for Qwen3.8-27B.

This is a 19.72 GB, text-only MLX model with 5.774 effective bits per quantized weight. It preserves the model's MTP head and is intended for mlx-serve on Apple Silicon.

Before downloading: use this model with mlx-serve, not stock mlx-lm. Stock mlx-lm 0.31.3 can load the files but produces corrupted output for this converted layout. See Runtime compatibility below.

Genuine Unsloth Dynamic v3.0: the per-tensor widths are read from the official unsloth/Qwen3.8-27B-GGUF UD-Q5_K_M checkpoint's own ggml type table. They are not re-derived and do not use an older Dynamic 2.0 class map.

What this model is

The upstream GGUF determines the bit width of each tensor. maccelerate reads that allocation and quantizes the corresponding clean bf16 Qwen weights onto MLX's affine format at the same width.

This is the same per-tensor bit-width allocation as the source GGUF, re-encoded for MLX. It is not a lossless GGUF conversion and does not claim numerical parity with llama.cpp's K-quant or IQ codebooks.

Claim	Status
Same upstream per-tensor allocation	Yes
Quantized from clean bf16 weights	Yes
Exact numerical parity with the GGUF	No

Allocation

MLX width	Tensors
8-bit	124
6-bit	160
5-bit	189
4-bit	33
Total	506

Weight class	Parameters	On disk
MLP gate + up	11.41B	8.01 GB
MLP down	5.70B	4.19 GB
GDN in-proj QKV	2.52B	1.82 GB
Attention Q/K/V/O	1.68B	1.29 GB
GDN out-proj	1.51B	1.18 GB
GDN in-proj Z	1.51B	1.11 GB
LM head	1.27B	1.03 GB
Token embeddings	1.27B	0.72 GB
MTP head	0.37B	0.31 GB
Total quantized	27.32B	19.72 GB

Widths are not a house preset: each tensor retains the width selected by the upstream Dynamic allocator. For 4- and 8-bit tensors, the conversion uses the official imatrix in an activation-weighted search. MLX's weighted packing path does not implement 5- or 6-bit output, so those tensors use mx.quantize. Norms, biases, convolution weights, and SSM state remain bf16. The vision tower is deliberately omitted.

Serving

Apple Silicon requirements

• Apple Silicon Mac running macOS.
• Allow at least 20 GB of disk for the model files, plus working space for the download.
• The released artifact is 19.72 GB. Actual unified-memory needs also depend on the KV cache, context length, and concurrent requests; no minimum-memory configuration has been validated for this Q5 build yet.

Run it with mlx-serve

Install and run mlx-serve according to its README, download this repository locally, then point it at that directory:

mlx-serve --model /path/to/Qwen3.8-27B-UD3-Q5_K_M-MLX --kv-quant 8

The pack is mixed-width, so mlx-serve's uniform-width NAX MTP profile does not apply; the generic profile supports its retained MTP weights.

Runtime compatibility

This release is not supported by stock mlx-lm. With mlx-lm 0.31.3, loading can succeed but greedy output is corrupted: it applies Qwen3.8's norm transformation a second time because it treats the presence of MTP weights as evidence that every trunk norm needs adjustment. Use mlx-serve for this model.

Reproducibility and generation code

This checkpoint was generated with maccelerate, the open-source converter that reads a Dynamic GGUF's per-tensor allocation and re-encodes the matching clean bf16 model for MLX. Its repository contains the conversion workflow, validation code, and the mlx-lm compatibility finding.

Provenance

Item	Value
Source GGUF	unsloth/Qwen3.8-27B-GGUF, Qwen3.8-27B-UD-Q5_K_M.gguf
Source GGUF SHA-256	2de73110cb254cbf09b54b717578dadff12ef1194e7271527e68202f39ba4bfd
bf16 source	Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
imatrix	imatrix_unsloth.gguf
Converter	maccelerate
Output	10 SafeTensors shards; 19.72 GB

The uploaded artifact includes manifest.json, recording its structural checks and shard hashes.

License

The model weights, source GGUF, and conversion code are licensed under Apache-2.0. See the included LICENSE and the upstream model cards for attribution and terms.