Ingot-II-27B-MLX-2bit

A public mirror of a model that won on Ingot, a Bittensor subnet where miners quantize a fixed teacher model and the validator crowns the copy that stays closest to it. This is the MLX pack of dendriteholdings/Ingot-II-27B-Q2: the same ternary weights, written for Apple Silicon.

It is Prism ML's Ternary Bonsai 2 27B, refined: the same ternary codes, with the per-block scales and the norm weights re-tuned by the miner to bring it closer to Qwen3.8-27B.

• Submitted by miner hotkey 5Gjbk5EZP1wm9G2CrtqmMT19TmjwNZtn6E6i6Z5zPM7h2DZR (uid 147)
• Crowned from submission 3f79f454-232a-4acf-b8dd-296f199c9dfb

The weights are the miner's upload converted without loss: every 128-weight block keeps its ternary codes and fp16 scale as MLX affine 2-bit values (scale d, bias −d), and every weight decodes to the same number the gguf kernel uses. Credit belongs to the miner, and to Prism ML for Bonsai 2. The vision tower, config, tokenizer and Hadamard sign vectors are Prism's prism-ml/Ternary-Bonsai-2-27B-mlx-2bit, which the competition does not change.

Files

file	bits per weight	size
model.safetensors	2.25 (2-bit codes, fp16 scale and bias per 128 weights)	8.60 GB

How close it is to the teacher

Per answer token, against the BF16 teacher reading the same answer: KL(teacher ‖ model) of the next-token distributions, and the hidden-state distance |h − h_teacher| / |h_teacher| averaged over every 8th layer. Lower is closer.

	KL	hidden-state distance
this model, in its winning eval	0.11245	0.3864
the king it beat, same eval	0.11556	0.3960

Speed

Decode tokens per second at batch 1 on a Mac Studio M3 Ultra (80-core GPU, 512 GB), measured on Ingot king IV. Every q2 king has the same layout and size, so the numbers hold for all of them within run-to-run noise. "Plain" is one token per forward pass. The server numbers are 256 generated tokens on a code prompt, timed over HTTP; the in-process numbers are 128 tokens with no server.

engine	mode	decode tok/s
MTPLX 2.12 (mlx 0.32) server	plain	55.3
mlx-lm 0.31 graph with MTPLX's ternary kernels, in-process	plain	59.9

For comparison, the gguf twin of these weights decodes at 56.4 tok/s on the same Mac in the Dendrite llama.cpp fork and at 137 tok/s on an RTX PRO 6000; see its card.

Running it

The pack uses Prism's prism_hadamard_qwen35 layout, which mlx-lm does not load. MTPLX does, and serves it as an OpenAI-compatible API:

pip install mtplx
mtplx serve --model dendriteholdings/Ingot-II-27B-MLX-2bit --generation-mode ar --port 8080 --no-auth --yes

This repo carries no MTP draft head, so MTPLX decodes one token per step. The gguf twin runs in the Dendrite llama.cpp fork; see its card.

Faster generation

We trained a DFlash 2 speculative-decoding drafter for this model family, dendriteholdings/Ingot-27B-DFlash2-MLX. It proposes several tokens at a time and the model verifies each one; on the M3 Ultra above it raises greedy single-stream decode from 59 to about 73 tok/s, and code answers reach about 90. Its repo has the runner:

hf download dendriteholdings/Ingot-27B-DFlash2-MLX --local-dir ingot-dflash2-mlx
pip install -r ingot-dflash2-mlx/requirements.txt
python ingot-dflash2-mlx/ingot_dflash_mlx.py serve --drafter ingot-dflash2-mlx --king dendriteholdings/Ingot-II-27B-MLX-2bit --port 8080