Qwen3.8-9B-Distill-MLX-4bit-mtp

MLX conversion of empero-ai/Qwen3.8-9B-Distill (Qwen3.5-9B distilled from Qwen3.8, Apache-2.0) with the native Qwen3.5 MTP head embedded.

MTP

This checkpoint carries the model's built-in Multi-Token Prediction head (15 tensors language_model.mtp.*, 1 layer, shared embeddings) in model-mtp.safetensors. The config.json declares mtp_num_hidden_layers: 1 (inherited from the Qwen3.5 base).

Default mlx-lm converters strip mtp.* weights during sanitize(); this checkpoint was produced with a converter that preserves them, so the oMLX Lightning MTP toggle works out of the box.

Usage

With oMLX (0.6.x): enable Lightning MTP in the model settings (mtp_enabled: true). Requires mlx-lm 0.31.3+.

import mlx_lm
model, tokenizer = mlx_lm.load("nicolasramos/Qwen3.8-9B-Distill-MLX-4bit-mtp")

4bit quantization: group_size 64, affine. The MTP fusion projection (mtp.fc) is kept in full precision.