Binade/gemma-4-12B-it

Gemma 4 12B, losslessly compressed with Binade

google/gemma-4-12B-it packed with Binade: 16.10 GB instead of 23.92 GB (10.746 bits per weight instead of 16 in the packed tensors), every weight restored bit for bit, and outputs bit-identical to the BF16 model, not just close.

Binade codes each weight's exponent by its frequency rank and keeps the sign and mantissa as they are. Its kernels decode the weights inside the matrix products and add the products in exactly the order the framework's own BF16 kernels do, so the logits and generated tokens match the original. The paper explains how.

Exactness

check	result
every tensor unpacked and compared with the original checkpoint	677 tensors, 0 values differ (gemma-4-12B-it/roundtrip_v2.md)
logits at 16,352 WikiText-2 positions, Apple M4 Max, against BF16 in memory	16,352 / 16,352 bit-identical (gemma-4-12B-it/exactness_scale.md)
32 greedy generations of 128 tokens, Apple M4 Max	32 / 32 identical, tokens and log-probabilities
the same logits test on NVIDIA RTX PRO 6000 Blackwell Server Edition	16,352 / 16,352 bit-identical, 32 / 32 generations (cuda/gemma-4-12B-it/exactness_scale.md)
the same logits test on NVIDIA L4 (BF16 does not fit this GPU)	16,352 / 16,352 bit-identical (cuda-l4/gemma-4-12B-it/exactness_scale.md)
MLX 8-bit quantization, for comparison	0 / 16,352 positions identical, argmax agreement 85.0%

Size and speed

	BF16	Binade	MLX 8-bit (lossy)
checkpoint	23.92 GB	16.10 GB
greedy decoding, Apple M4 Max	19.2 tok/s	24.7 tok/s	35.0 tok/s

Decoding speed: gemma-4-12B-it/decode_speed.md. NVIDIA results: https://github.com/GuyShafir/binade/tree/main/results.

Use

pip install binade               # Apple Silicon
pip install "binade[cuda]"       # Linux + CUDA 12

from mlx_lm import generate
from binade.mlx.r4 import load

model, tokenizer, _ = load("Binade/gemma-4-12B-it")
messages = [{"role": "user", "content": "Why is the sky blue?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt, max_tokens=200))

To get the original BF16 checkpoint back, bit for bit: binade unpack Binade/gemma-4-12B-it --out gemma-4-12B-it-bf16.

Requirements and limits

• MLX 0.32.2 and mlx-lm 0.31.3, which pip install binade installs. Exactness is tied to MLX's own kernels; the tests in the repository check it.
• Apple Silicon runs the fused prompt kernels on M3 and M4 GPUs. On NVIDIA they are verified on the RTX PRO 6000, L4 and A100; on other GPUs prompts rebuild BF16 weights, which is exact but slower.
• The checkpoint keeps every tensor of the original, the vision and audio weights included (binade unpack restores them), but the runtime loads the text model through mlx-lm, so it serves text. Exactness was tested on text generation.
• The Hub tags this repository 8-bit, and counts its parameters by bytes, because the packed bytes are stored as uint8 tensors. The weights are the full BF16 weights: binade unpack restores every one bit for bit.

License

The weights keep the base model's license: apache-2.0 (terms). Binade itself is Apache-2.0.

Citation

@misc{shafir2026binade,
  title  = {Binade: Bit-Exact Compressed LLM Inference on Apple Silicon},
  author = {Guy Shafir and Tulber},
  year   = {2026},
  url    = {https://github.com/GuyShafir/binade}
}

Packed by Binade format 2 at commit bd1ba8b of the development repository.