Bonsai-27B for hipfire (.hfq)

Prism ML's Bonsai-27B low-bit models, repackaged into hipfire's .hfq container so they load directly in hipfire on RDNA GPUs.

The quantized weight values are unchanged. The Q1_0 / Q2_0 low-bit blocks are copied byte-verbatim from the upstream GGUFs — this is a container and layout conversion, not a requantization. See Modifications for exactly what differs.

Created using Bonsai by Prism ML.

file	quant	size	KLD ↓	PPL	decode	prefill
bonsai-27b.bq1	BQ1G128 — binary, 1.14 bpw	3.80 GB	0.629	17.76	~28.2 tok/s	~285 tok/s
bonsai-27b.tq2	TQ2G128 — ternary, 2.125 bpw	7.16 GB	0.536	16.69	~22.4 tok/s	~258 tok/s

Throughput measured on gfx1151 (Strix Halo), kv q8, medians after warmup; prefill from a ~6.5k-token prompt. For reference, qwen3.6-27b.mq4 on the same box and prompt runs 13.3 tok/s decode and 245.1 prefill.

Usage

hipfire pull bonsai:27b-bq1        # 3.8 GB
hipfire run  bonsai:27b-bq1 "The capital of France is"

hipfire pull bonsai:27b-tq2        # 7.2 GB, higher quality

bonsai aliases to bonsai:27b-tq2.

License and attribution

Licensed under the Apache License 2.0. The full text is in LICENSE; the upstream attribution notices are retained verbatim in NOTICE, reproduced here:

This software is copyright 2026-present Prism ML, Inc. It is available under the Apache 2.0 license. If you publicly deploy or redistribute this software, we would appreciate attribution such as: "Created using Bonsai by Prism ML."

This software is built from Qwen3.6-27B, Copyright 2026 Alibaba Cloud, which is available under the Apache 2.0 License: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE

The chain is therefore: Qwen3.6-27B (Alibaba Cloud, Apache-2.0) → Bonsai-27B (Prism ML, Apache-2.0) → this .hfq conversion (hipfire project, Apache-2.0). All three attributions must be retained in any further redistribution.

Modifications (Apache-2.0 §4(b) notice)

Apache-2.0 requires derivatives to state what was changed. hipfire's qwen35 runtime was written against the HF safetensors convention, while these GGUFs come from Prism ML's own converter (conversion/qwen.py) — so the conversion is essentially the inverse of that converter, putting tensors back into the layout our runtime expects.

#	change	why
1	Repackaged GGUF → hipfire .hfq container	hipfire's loader format
2	Quantized weight values unchanged — Q1_0/Q2_0 blocks copied byte-verbatim	no requantization is performed
3	Linear-attention value heads permuted tiled → grouped	GQA layout differs: 16 key heads, 48 value heads (r = 3); index map inv(g) = (g % r) · nk + g / r. Reordering only — no value changes.
4	ssm_a stored as A_log = ln(−A)	HF stores A_log and the runtime computes A = −exp(A_log). The upstream converter stores the already-evaluated (negative) A, so we invert it.
5	RMSNorm weights (except ssm_norm): baked +1 removed	The upstream converter bakes +1 into every RMSNorm weight except ssm_norm; hipfire's runtime adds QWEN35_NORM_BIAS = 1.0 to exactly that same set at load. Passing through unmodified double-adds → norm weights ≈2× → the whole forward runs ~2× hot.
6	conv1d kept at full precision (F16)	Depthwise Gated-DeltaNet conv kernels are tiny and precision-critical; also k_dim = conv_kernel is never 256-aligned, so the 4-bit fallback cannot take them.

⚠️ If you read these tensors with other tooling, change 5 is the one that matters: the RMSNorm weights are stored one less than the true value, because hipfire re-adds the +1 at load time. Anything that does not apply that bias will get incorrect norms.

Every file records the same information in its own metadata under hipfire_provenance — source GGUF, upstream URL, license, tool version, git commit, build timestamp, and the modifications list — so an artifact can always be traced back to how it was produced:

$ python3 benchmarks/quality-baselines/harness/hfq_provenance.py bonsai-27b.bq1
bonsai-27b.bq1  (3.80 GB, mtime 2026-08-18 09:55:47)
    built    2026-08-18 09:55:47  by hipfire-quantize 0.3.0 @ unknown
    source   Bonsai-27B-Q1_0.gguf
    format   BQ1G128
    upstream https://huggingface.co/prism-ml/Bonsai-27B-gguf
    license  Apache-2.0
    changes  (6) — Apache-2.0 §4(b) notice: ...

Quality

KLD is measured against a hipfire-native qwen3.6-27b.mq4 reference (build_kld_ref_native, top-256), 8 chunks × 512 ctx of wikitext2, kv asym3, per-token scoring. Lower is closer to the 4-bit teacher, which scores 0.000 / PPL 7.42 by construction. Both variants generate coherently.

For context, from the same sweep on the same axis: our own 2-bit PTQ of Qwen3.6-27B (Lloyd-Max codebook) scores 0.612 at 8.58 GB, and plain 3-bit scores 0.277 at 11 GB. Bonsai's binary quant at 3.80 GB is the best size/quality point available to us — 2.3× smaller than our best PTQ at statistically indistinguishable quality.

Port fidelity

The conversion is faithful, not approximate. Scoring bonsai-27b.tq2 against Prism ML's own llama.cpp fork running the same GGUF, on llama's token stream with f32 KV:

mean KLD = 0.000153 (per-chunk 1.07e-4 … 1.91e-4)

Corroborated on realized tokens — llama.cpp's cumulative PPL after 4 chunks is 14.1907, hipfire scores 14.1456. The difference is numerical noise (GPU vs CPU reduction order, fp16 storage).

Upstream

• 1-bit: prism-ml/Bonsai-27B-gguf
• Ternary: prism-ml/Ternary-Bonsai-27B-gguf
• Base model: Qwen/Qwen3.6-27B

If you can run Prism ML's own releases, prefer them — this repo exists only to save hipfire users a conversion step.