Ingot-27B-DFlash2-MLX

The Ingot-27B-DFlash2 speculative-decoding drafter for Apple Silicon: MLX weights quantized to 8 bits, and a runner that pairs it with an Ingot king's MLX pack (dendriteholdings/Ingot-<N>-27B-MLX-2bit).

The drafter only proposes tokens; the king checks every one, so generation gets faster without changing which model answers.

Files

file	what
model.safetensors	the drafter, MLX affine 8-bit, group 64 (2.04 GB)
config.json	DFlash 2 architecture and the quantization settings
ingot_dflash_mlx.py	runner: generate for one prompt, serve for an OpenAI-compatible API
requirements.txt	pinned dependencies

Usage

Apple Silicon, Python 3.12:

hf download dendriteholdings/Ingot-27B-DFlash2-MLX --local-dir ingot-dflash2-mlx
python3.12 -m venv venv && . venv/bin/activate
pip install -r ingot-dflash2-mlx/requirements.txt

python ingot-dflash2-mlx/ingot_dflash_mlx.py generate --drafter ingot-dflash2-mlx \
  --king dendriteholdings/Ingot-XVI-27B-MLX-2bit --prompt "Write a bubble sort in C++."

python ingot-dflash2-mlx/ingot_dflash_mlx.py serve --drafter ingot-dflash2-mlx \
  --king dendriteholdings/Ingot-XVI-27B-MLX-2bit --port 8080
# OpenAI API at http://127.0.0.1:8080/v1 (chat completions, streaming or not)

--king takes any Ingot king's MLX pack, by repo id or local folder. The king is downloaded on first use (about 8.6 GB). Options: --temp (default 0), --block (draft block size, default 4), --max-tokens and --show-thinking for generate, --host for serve.

How it works

The kings use Prism's prism_hadamard_qwen35 MLX layout, which mlx-lm cannot load. The runner loads the king through MTPLX and runs z-lab's DFlash MLX speculative loop on it. It also casts the drafter's inputs to bfloat16 and its output to float16, because the king computes in float16 and MLX would otherwise run the whole drafter in float32. Draft blocks of 4 are the fastest setting we measured.

Results

Mac Studio M3 Ultra, one stream, 8 chat and code prompts, 256 greedy tokens:

target	plain tok/s	with drafter tok/s	speedup	tokens per step
Ingot-IV-27B MLX	59.0	72.8	1.23x	2.97
Ingot-XVI-27B MLX	59.0	72.6	1.23x	2.99

Code answers reach about 90 tok/s. Memory: about 10 GB for king and drafter together.

On M1 to M4 GPUs the gain is limited by the cost of verifying several tokens at once with 2-bit weights: longer draft blocks get more tokens accepted but cost more to check than they return. M5 GPUs have matrix units that should change this; not tested yet. On NVIDIA GPUs the GGUF drafter in llama.cpp is 1.8x faster than plain decoding; see Ingot-27B-DFlash2.

Limitations

• Apple Silicon only. One request at a time, text only, no prompt cache.
• The runner depends on an internal MTPLX function; requirements.txt pins MTPLX 2.12.2 and the z-lab dflash commit it was tested with.
• Batched verification rounds differently from plain decoding, so a greedy output can differ at near-tied tokens. The model is the same.
• Measured on an M3 Ultra only.

Training

Fine-tuned from z-lab/Qwen3.8-27B-DFlash2 on answers generated by Ingot-IV-27B; the drafter trained on king IV works unchanged on later kings. Details on Ingot-27B-DFlash2.

Citation

@article{chen2026dflash,
  title   = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026},
  url     = {https://arxiv.org/abs/2602.06036}
}

Upstream drafter by z-lab, Apache 2.0.