rapid-mlx/Qwen3.8-27B-4bit-MTP-fp16-MLX

Qwen3.8-27B 4-bit MLX + MTP — FP16 sibling

This is the M1/M2-oriented FP16 sibling of rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX. It preserves the 4-bit target weights and colocated native Multi-Token Prediction (MTP) sidecar, while storing every non-quantized BF16 tensor as FP16. M1 and M2 chips do not accelerate BF16 matrix multiplication; M3 and newer Macs should normally use the original BF16 checkpoint.

Rapid-MLX

rapid-mlx serve qwen3.8-27b-4bit-fp16 --no-think

Or use the repository ID directly:

rapid-mlx serve rapid-mlx/Qwen3.8-27B-4bit-MTP-fp16-MLX --no-think

--no-spec-decode remains available when a workload does not benefit from MTP. Rapid does not silently substitute this checkpoint for the regular qwen3.8-27b-4bit alias.

Conversion and validation

• Source revision: aa985c29ff5b334cbfdcbbc787d47e66e9d9e456
• 1,705 BF16 tensors converted to FP16.
• 506 non-BF16 tensors preserved with identical names, shapes, dtypes, and values, including all packed U32 quantized weights.
• All four safetensors files were read back and verified; the MTP sidecar loaded all 31/31 expected tensors in Rapid-MLX.
• On an M3 Ultra, two of three fixed greedy prompts were byte-identical to the BF16 source. The third differed by one semantically equivalent phrase. This is a normal FP16/BF16 numerical boundary, so universal byte identity is not claimed.
• On the FP16 target, greedy output with MTP enabled was byte-identical to its autoregressive output in the paired smoke test.

The machine-readable fp16-conversion.json records the conversion contract and counts. M1/M2 prefill measurements will be added after the paired hardware qualification completes.

Contents

• model-*.safetensors: Qwen3.8-27B MLX 4-bit target weights with FP16 non-quantized tensors
• mtp/model.safetensors: matching 4-bit native MTP drafter with FP16 non-quantized tensors
• tokenizer, processor, generation, and model configuration files

Upstream model licenses and limitations continue to apply.