seabit-ai/Qwen3.8-27B-MTP-draft

Qwen3.8-27B MTP draft head, split out for MLX

The multi-token-prediction (MTP) head that ships inside Qwen/Qwen3.8-27B (its 18th safetensors shard: 15 mtp.* tensors — one full-attention decoder layer, a fusion projection and three norms), saved as a standalone drafter in the layout that mlx-vlm loads with load_drafter(...) and --draft-kind mtp. It shares the target model's embeddings and LM head, so it is 810 MB in bf16 and needs no weights of its own beyond this file.

Why this exists: the MLX conversions of Qwen3.8-27B on the Hub (lmstudio-community, mlx-community) drop the MTP tensors, so speculative decoding with the model's own draft head was not available to MLX users without re-downloading the original 54 GB.

Provenance

• Source: Qwen/Qwen3.8-27B, file model-00018-of-00018.safetensors (bf16), unchanged values — only renamed (mtp. prefix removed) and re-saved with MLX metadata.
• Tool: python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split --model <snapshot> --output <dir> (mlx-vlm 0.6.12). config.json is what the tool writes: model_type: qwen3_5_mtp, block_size: 3, the target's text_config.
• Checksums: SHA256SUMS.

Use

Works with any MLX conversion of Qwen3.8-27B (4/5/6/8-bit) — compatibility is by hidden_size and vocabulary, which the quantizations keep.

With lmk: lmk up fetches it for qwen3.8-27b-4bit (and the 5-, 6- and 8-bit versions); set speculative_decoding: true in ~/.lmk/config.yaml. Measured on an M3 Ultra: greedy decoding 1.2× (prose) to 1.7× (code, copy-editing) with identical output on code.

With mlx-vlm directly:

from mlx_vlm import load
from mlx_vlm.generate import generate
from mlx_vlm.speculative.drafters import load_drafter
model, processor = load("lmstudio-community/Qwen3.8-27B-MLX-4bit")
draft, kind = load_drafter("seabit-ai/Qwen3.8-27B-MTP-draft")
generate(model, processor, "Write a quicksort in Python.", draft_model=draft, draft_kind=kind, temperature=0)

License

Apache-2.0, as the source weights. Copyright of the weights remains with the Qwen team; this repository only repackages one shard of them.