Qwen3.8-27B-UD3-Q6_K_M-MLX

Native MLX affine re-encoding of the official Unsloth Dynamic v3.0 Q6_K_M per-tensor allocation for Qwen3.8-27B.

This is a 22.94 GB, text-only MLX model with 6.716 effective bits per quantized weight. It preserves the model's MTP head and is intended for mlx-serve on Apple Silicon.

Before downloading: use this model with mlx-serve, not stock mlx-lm. Stock mlx-lm 0.31.3 can load the files but produces corrupted output for this converted layout. See Runtime compatibility below.

Genuine Unsloth Dynamic v3.0: the per-tensor widths are read from the official unsloth/Qwen3.8-27B-GGUF UD-Q6_K_M checkpoint's own ggml type table. They are not re-derived and do not use an older Dynamic 2.0 class map.

What this model is

The upstream GGUF determines the bit width of each tensor. maccelerate reads that allocation and quantizes the corresponding clean bf16 Qwen weights onto MLX's affine format at the same width.

This means the model uses the same per-tensor bit-width allocation as the source GGUF, re-encoded for MLX. It is not a lossless GGUF conversion and does not claim numerical parity with llama.cpp's K-quant or IQ codebooks.

Claim	Status
Same upstream per-tensor allocation	Yes
Quantized from clean bf16 weights	Yes
Exact numerical parity with the GGUF	No

Allocation

MLX width	Tensors
8-bit	201
6-bit	231
5-bit	71
4-bit	3
Total	506

Widths are not a house preset: each tensor retains the width selected by the upstream Dynamic allocator.

Weight class	Parameters	On disk
MLP gate + up	11.41B	9.30 GB
MLP down	5.70B	4.70 GB
GDN in-proj QKV	2.52B	2.08 GB
Attention Q/K/V/O	1.68B	1.47 GB
GDN out-proj	1.51B	1.38 GB
LM head	1.27B	1.35 GB
Token embeddings	1.27B	1.03 GB
MTP head	0.37B	0.31 GB
Total quantized	27.32B	22.94 GB

For 4- and 8-bit tensors, the conversion uses the official imatrix in an activation-weighted search. MLX's weighted packing path does not implement 5- or 6-bit output, so those tensors use mx.quantize. Norms, biases, convolution weights, and SSM state remain bf16. The vision tower is deliberately omitted.

Serving

Apple Silicon requirements

• Apple Silicon Mac running macOS.
• Allow at least 23 GB of disk for the model files, plus working space for the download.
• The reference run used an M4 Max with 64 GB unified memory. The model alone reached a 23.40 GB peak Metal footprint in a compatibility probe; generation, KV cache, context length, and concurrent requests need additional memory. Check your intended context length and concurrency before treating a smaller-memory Mac as suitable.

Run it with mlx-serve

Install and run mlx-serve according to its README, download this repository locally, then point it at that directory:

mlx-serve --model /path/to/Qwen3.8-27B-UD3-Q6_K_M-MLX --kv-quant 8

--kv-quant 8 is the reference configuration, not a guarantee for every context length or workload. The pack is mixed-width, so mlx-serve's uniform-width NAX MTP profile does not apply; the generic profile supports its retained MTP weights.

Runtime compatibility

This release is not supported by stock mlx-lm. With mlx-lm 0.31.3, loading can succeed but its greedy output is corrupted: it applies Qwen3.8's norm transformation a second time because it treats the presence of MTP weights as evidence that every trunk norm needs adjustment. Use mlx-serve for this model.

Reproducibility and generation code

This checkpoint was generated with maccelerate, the open-source converter that reads a Dynamic GGUF's per-tensor allocation and re-encodes the matching clean bf16 model for MLX. Its repository contains the conversion workflow, validation code, and the mlx-lm compatibility finding.

For a reproducible rebuild, use maccelerate-ai/maccelerate at 58458a509866, along with the source revisions below. The repository's Qwen3.8 reference-build instructions document the required GGUF, bf16 source, and imatrix inputs.

Provenance

Item	Value
Source GGUF	unsloth/Qwen3.8-27B-GGUF, Qwen3.8-27B-UD-Q6_K_M.gguf
Source GGUF revision	4ca720788d1e01f1bff70c033e0d0028fd02e502
Source GGUF SHA-256	493301830a596b8ad56dc1329f80bbcb578c8e910da395feafdc9cd8263430bb
bf16 source	Qwen/Qwen3.8-27B
bf16 source revision	1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
imatrix	imatrix_unsloth.gguf, 10,248,192 tokens
Converter	maccelerate at 58458a509866
Output	11 SafeTensors shards; 22.94 GB

The uploaded artifact should retain its generated manifest.json, which records the source file/revision, allocation, structural validation, and shard hashes for the exact build.

License

The model weights, source GGUF, and conversion code are licensed under Apache-2.0. See the included LICENSE and the upstream model cards for attribution and terms.