Qwen3.8-27B-UD3-Q4_K_M-MLX

Native MLX affine re-encoding of the official Unsloth Dynamic v3.0 Q4_K_M per-tensor allocation for Qwen3.8-27B.

This is a 16.72 GB, text-only MLX model with 4.894 effective bits per quantized weight. It preserves the model's MTP head and is intended for mlx-serve on Apple Silicon.

Before downloading: use this model with mlx-serve, not stock mlx-lm. Stock mlx-lm 0.31.3 can load the files but produces corrupted output for this converted layout. See Runtime compatibility below.

Genuine Unsloth Dynamic v3.0: the per-tensor widths are read from the official unsloth/Qwen3.8-27B-GGUF UD-Q4_K_M checkpoint's own ggml type table. They are not re-derived and do not use an older Dynamic 2.0 class map.

What this model is

The upstream GGUF determines the bit width of each tensor. maccelerate reads that allocation and quantizes the corresponding clean bf16 Qwen weights onto MLX's affine format at the same width.

This is the same per-tensor bit-width allocation as the source GGUF, re-encoded for MLX. It is not a lossless GGUF conversion and does not claim numerical parity with llama.cpp's K-quant or IQ codebooks.

Claim	Status
Same upstream per-tensor allocation	Yes
Quantized from clean bf16 weights	Yes
Exact numerical parity with the GGUF	No

Allocation

MLX width	Tensors
8-bit	106
6-bit	30
5-bit	131
4-bit	228
3-bit	11
Total	506

Weight class	Parameters	On disk
MLP gate + up	11.41B	6.67 GB
MLP down	5.70B	3.43 GB
GDN in-proj QKV	2.52B	1.49 GB
Attention Q/K/V/O	1.68B	1.08 GB
LM head	1.27B	1.03 GB
GDN out-proj	1.51B	1.02 GB
GDN in-proj Z	1.51B	0.90 GB
Token embeddings	1.27B	0.72 GB
MTP head	0.37B	0.31 GB
Total quantized	27.32B	16.71 GB

Widths are not a house preset: each tensor retains the width selected by the upstream Dynamic allocator. For 4- and 8-bit tensors, the conversion uses the official imatrix in an activation-weighted search. MLX's weighted packing path does not implement 3-, 5-, or 6-bit output, so those tensors use mx.quantize. Norms, biases, convolution weights, and SSM state remain bf16. The vision tower is deliberately omitted.

Serving

Apple Silicon requirements

• Apple Silicon Mac running macOS.
• Allow at least 17 GB of disk for the model files, plus working space for the download.
• The released artifact is 16.72 GB. Actual unified-memory needs also depend on the KV cache, context length, and concurrent requests; no minimum-memory configuration has been validated for this Q4 build yet.

Run it with mlx-serve

Install and run mlx-serve according to its README, download this repository locally, then point it at that directory:

mlx-serve --model /path/to/Qwen3.8-27B-UD3-Q4_K_M-MLX --kv-quant 8

The pack is mixed-width, so mlx-serve's uniform-width NAX MTP profile does not apply; the generic profile supports its retained MTP weights.

Runtime compatibility

This release is not supported by stock mlx-lm. With mlx-lm 0.31.3, loading can succeed but greedy output is corrupted: it applies Qwen3.8's norm transformation a second time because it treats the presence of MTP weights as evidence that every trunk norm needs adjustment. Use mlx-serve for this model.

Reproducibility and generation code

This checkpoint was generated with maccelerate, the open-source converter that reads a Dynamic GGUF's per-tensor allocation and re-encodes the matching clean bf16 model for MLX. Its repository contains the conversion workflow, validation code, and the mlx-lm compatibility finding.

Provenance

Item	Value
Source GGUF	unsloth/Qwen3.8-27B-GGUF, Qwen3.8-27B-UD-Q4_K_M.gguf
Source GGUF SHA-256	322e194ff79741c7baa497c240f677f54b201b0efab44ca8e50f122b39123482
bf16 source	Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
imatrix	imatrix_unsloth.gguf
Converter	maccelerate
Output	8 SafeTensors shards; 16.72 GB

The uploaded artifact includes manifest.json, recording its structural checks and shard hashes.

License

The model weights, source GGUF, and conversion code are licensed under Apache-2.0. See the included LICENSE and the upstream model cards for attribution and terms.