bonsai-repro-1.7B-ternary

Independent reproduction of the PrismML Bonsai ternary recipe on Qwen/Qwen3-1.7B, trained with train_bonsai_repro.py (see training/).

What this checkpoint is. Every decoder linear layer and the tied embedding / LM head is ternary in a blockwise signed-Hadamard rotated basis (block 1024): per group of 128 input weights, 42 are zero and the other 86 are ±s with a per-group FP16-representable scale s. The weights are stored here dequantized in bf16 and folded back to the original basis so that the model loads with stock transformers; the ternary structure is exact and can be recovered by re-applying the rotation.

Training. Straight-through QAT from the base model's weights, KL(teacher ‖ student) distillation against the bf16 base model, 20000 steps × 8192 tokens, lr 6e-05 (cosine), norms trainable, data = FineWeb-Edu.

Final metrics (wikitext-2 test, 16×512 tokens): ppl 35.0, KL to teacher 0.651 nats/token, top-1 agreement 66.5 %, sign flips vs. init 3.4e-04, mask change 9.44 %.

Benchmarks (lm-evaluation-harness 0.4.13, HF backend, bf16)

acc_norm for ARC-C / HellaSwag / PIQA (0-shot), acc for Winogrande (0-shot), MMLU 5-shot, GSM8K 5-shot strict-match.

Model	ARC-C	HellaSwag	PIQA	Winogrande	MMLU	GSM8K
Qwen3-1.7B (FP teacher)	43.0	60.5	72.2	60.5	60.2	68.8
prism-ml/Ternary-Bonsai-1.7B (released)	39.4	52.5	70.3	59.0	49.1	59.1
this checkpoint (r9, fully ternary incl. embeddings)	33.5	46.5	66.1	58.3	34.5	54.7

Raw lm-eval JSONs are under training/evals/. Earlier ablation runs (r1–r8) are described in the accompanying training report in the source repository.

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("alllwang/bonsai-repro-1.7B-ternary", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("alllwang/bonsai-repro-1.7B-ternary")