Qwen3.6-27B-MTP-4bit-test

This is not a standalone model. It holds only the Multi-Token Prediction (MTP) drafter head, so it carries no token embeddings and no lm_head and it cannot generate text alone. Load it as the draft model beside a Qwen3.6-27B target, which supplies both.

Qwen ships this head inside Qwen/Qwen3.6-27B as 15 tensors under the mtp. prefix, but the MLX converter deletes them when it builds the main model, so the head is published separately and the runtime loads it as its own model with model_type: qwen3_5_mtp.

Use with mlx-vlm

pip install -U mlx-vlm

mlx_vlm generate \
  --model vvsotnikov/Qwen3.6-27B-4bit-test \
  --draft-model vvsotnikov/Qwen3.6-27B-MTP-4bit-test \
  --prompt "Write a quicksort in Python." \
  --max-tokens 256 --temperature 0.6

--draft-kind mtp is detected from model_type, so you do not need to pass it. The drafter proposes tokens each step while the target verifies them, and only accepted tokens reach the output, so quality follows the target rather than the drafter.

How this was produced

# 1. split the mtp.* tensors out of the base checkpoint
python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
  --model Qwen/Qwen3.6-27B --output Qwen3.6-27B-MTP-bf16

# 2. quantize the result
mlx_vlm convert --hf-path Qwen3.6-27B-MTP-bf16 \
  --mlx-path Qwen3.6-27B-MTP-4bit -q --q-bits 4 --q-group-size 64

Round-to-nearest is the only option here, because AWQ needs a forward pass and a drafter cannot run one on its own. The split step also adds 1.0 to every RMSNorm weight, since Qwen stores the scale minus one.

Verification

This build is bit-identical to mlx-community/Qwen3.6-27B-MTP-4bit across all 31 tensors.

Check	Result
model_type	qwen3_5_mtp
block_size	3, from mtp_num_hidden_layers + 2
Tensors	31, being 8 quantized projections and 7 dense norms
RMSNorm shift	applied exactly once, checked against the source
Acceptance	93.5 percent of drafted tokens, 2.86 accepted per round

Pairing rules

Use a drafter and a target that come from ONE checkpoint, because the drafter binds to the target's embeddings at runtime and a mismatched pair either fails a hidden-size check or drafts badly. A bf16 drafter is also published as Qwen3.6-27B-MTP-bf16-test, which matters for Mixture-of-Experts targets, where quantized MTP weights are reported to cut acceptance from about 80 percent to about 10 percent. This 27B target is dense, so the 4-bit drafter performs well on it.

License and attribution

The weights derive from Qwen/Qwen3.6-27B under Apache 2.0, so the original license and its terms carry over. Refer to the upstream model card for details.