OpenBonsai Qwen3.5-9B (ternary, PTQ1_0)

Research preview. The model is useful for experiments, not a finished product. Its math and especially its code ability are still well below the original model (see Results).

Independent project, not affiliated with or endorsed by Prism ML. Bonsai is Prism ML's product.

This is Qwen/Qwen3.5-9B compressed to ternary weights by OpenBonsai, an open (Apache-2.0) compression framework.

Weights: {-1, 0, +1}, in groups of 128, each group with one fp16 scale.
File: OpenBonsai-Qwen3.5-9B-PTQ1_0.gguf, 1,997,486,976 bytes (about 2 GB), SHA-256 4f2f3d1069a505059126e13ed1d2d7928e715930fe83f6d284426c4b19e35532.
Layout: the same tensor layout as Prism ML's Ternary-Bonsai-2-9B. All 202 large matrices are ternary, including the embedding and output head.
Text only: no vision tower and no MTP layers.

Running it

It needs Prism's llama.cpp fork. Upstream llama.cpp does not support PTQ1_0. It was tested at commit c9c34bc, CPU build; the CUDA build is untested.

llama-cli -m OpenBonsai-Qwen3.5-9B-PTQ1_0.gguf -p "Explain why the sky is blue." -n 256

The chat template and tokenizer are embedded in the GGUF.

How it was made

One RTX 5090, about 4.5 hours in total, with openbonsai run --config examples/qwen3_5-9b-reasoning.yaml.

Self-generated traces: the BF16 model answers 1,520 prompts (1.74M tokens). The prompts come from the training splits of GSM8K, MATH, MBPP and OpenCodeInstruct, plus general chat, with thinking on and off. This is the AYOT idea from ScaleQ-1.58 (arXiv 2608.01078). Test problems from the evaluation sets are refused by a contamination guard.
Rotation: sign + Hadamard rotation in Prism's layout.
Ternary rounding: sequential GPTQ, on 256 × 2,048 tokens. Half of these are the traces; the rest is WikiText-2 train and UltraChat.
Ternary embedding and head: also via GPTQ.
Scale distillation: the per-group scales are distilled end to end against the BF16 model's top-64 outputs, on 3.1M tokens (42% traces), with the codes frozen.
Export and checks: a verified PTQ1_0 GGUF export, which Prism's runtime loads.

This is not Prism ML's recipe, which has not been published. Only the file format is shared. Method notes are in the repository.

Results

All rows are one run each, on small evaluation sets.

Reasoning: greedy decoding with the model's chat template.
- GSM8K uses a frozen 100-problem subset and HumanEval a frozen 50-problem subset, with a 512-token cap and thinking off.
- "On" means thinking on, using the first 50 and 25 of those subsets with a 4,096-token cap.
Dev: 2,048 WikiText-2 test tokens and 20 ARC-Easy questions.
EAR: the expected acceptance rate against BF16 over 4,088 tokens, Σ min(p, q).

Model	GSM8K off	HumanEval off	GSM8K on	HumanEval on	Dev NLL ↓	ARC /100	EAR ↑
Qwen3.5-9B BF16	77/100	44/50	36/50	22/25	2.423	97	1
Ternary-Bonsai-2-9B (Prism ML)	77/100	45/50	46/50	24/25	3.191	–	0.676
This model	69/100	19/50	25/50	10/25	2.960	84	0.706
Earlier version (generic calibration)	30/100	6/50	19/50	3/25	2.887	89	0.737

Calibrating on the model's own math and code answers more than doubled every reasoning score over the earlier version, at a small cost in WikiText NLL and ARC.
Code is the clearest weakness: 19/50 against 44–45 for BF16 and Prism's model.
With thinking on, this model often reasons longer than the 4,096-token cap. It hit the cap on 23 of 50 GSM8K problems, against 14 for BF16 and 4 for Prism's model.
Prism's runtime: perplexity on WikiText-2 test, 8 × 512 tokens, is 12.59 for this model. Prism's Ternary-Bonsai-2-9B scores 16.51 and the earlier OpenBonsai version 11.93.
WikiText is partly in-domain. The calibration data includes WikiText-2 train, so WikiText numbers favor this model. ARC, GSM8K and HumanEval test problems are not in its data.
Non-English output is weak. The calibration data is almost all English.

License and attribution

Apache-2.0, following the base model Qwen/Qwen3.5-9B.
The PTQ1_0 layout follows Prism ML's published Ternary Bonsai format.
The method builds on ScaleQ-1.58 / CAT-Q, GPTQ, EfficientQAT-style distillation and Hadamard-rotation quantization work.
Calibration prompts come from GSM8K (MIT), MATH (MIT), MBPP (CC-BY-4.0), OpenCodeInstruct (CC-BY-4.0), UltraChat (MIT) and WikiText-2 (CC BY-SA).