deanzchen/Qwen3.6-27B-Q5-MTPLX

Qwen3.6 27B Q5 + MTPLX MTP

This artifact combines a uniform 5-bit MLX Qwen3.6 27B trunk with the calibrated MTPLX MTP sidecar used by the optimized-speed release.

Components

• Base model: Qwen/Qwen3.6-27B
• Q5 trunk: mlx-community/Qwen3.6-27B-5bit, revision 46f9b268ffe53528a8ad4ce8f684a480b3ef0d18
• MTP sidecar: Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed, revision be5190f2349594ec941753efc90a4ca5641af174
• Trunk quantization: MLX affine 5-bit, group size 64
• MTP sidecar path: mtp/weights.safetensors
• MTPLX discovery metadata: mtplx_runtime.json

The Q5 trunk and optimized MTP sidecar are family-compatible but do not have a published exactness contract for this combined artifact. MTPLX therefore marks it family-compatible-unverified and requires explicit unsafe confirmation.

Run with MTPLX

export MTPLX_VLLM_METAL_PAGED_TURBOQUANT=1
export MTPLX_VLLM_METAL_PAGED_TURBOQUANT_K_QUANT=q8_0
export MTPLX_VLLM_METAL_PAGED_TURBOQUANT_V_QUANT=q4_0
export MTPLX_PREFILL_CHUNK_SIZE=512
export MTPLX_PREFILL_CHUNK_SIZE_DENSE=512
export MTPLX_PREFILL_CHUNK_SIZE_REPAGE=512

mtplx serve \
  --model deanzchen/Qwen3.6-27B-Q5-MTPLX \
  --profile sustained \
  --generation-mode mtp \
  --depth 2 \
  --context-window 262144 \
  --prefill-chunk-tokens 512 \
  --temperature 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --draft-temperature 0.7 \
  --draft-top-p 0.95 \
  --draft-top-k 20 \
  --reasoning auto \
  --preserve-thinking scoped \
  --ssd-session-cache off \
  --no-stats-footer \
  --unsafe-force-unverified \
  --yes

TurboQuant normally activates after the sustained cache crosses 131,072 tokens. The configured 262,144-token maximum is a capacity target; this Q5 combination has been exercised to 132,515 tokens, not to the full maximum.

Local M5 Max measurements

Hardware: M5 Max, 40 GPU cores, 48 GB unified memory. Runtime: MTPLX 2.0.2, MLX 0.31.2, MLX-LM 0.31.3.

Mode	Decode speed
Autoregressive	25.15 tok/s
MTP depth 1	41.29 tok/s
MTP depth 2	49.91 tok/s
MTP depth 3	44.95 tok/s

Depth 2 was selected. The matched short run used approximately 22.1 GB peak memory. A 132,515-token activation probe reached approximately 35.7 GB and confirmed TurboQuant K8/V4, but initial decode at that extreme context was only about 0.08 tok/s.

These are local measurements, not broad hardware claims.

The current CLI wrapper defers full verification-hidden materialization and evaluates the committed hidden-state slice after target distributions. Five token-identical D2 runs averaged 50.78 tok/s; the persisted wrapper's final gate measured 49.82 tok/s with 100% acceptance.

For repeatable expectations and regression thresholds, see MTPLX_PERFORMANCE_BASELINE.json. An 8-token smoke test is informational; use the 128- or 256-token direct tuner workload as the performance gate.