TokForge-Bonsai-2-9B-MNN (text only)

Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Text only

This bundle carries the language model only. Prism ML ships the vision part as a separate file, and it is not included here. In TokForge, photos are described by the app's built-in image helper instead, and the model answers from that description.

What this is

Ternary Bonsai 2 9B by Prism ML, built from Qwen3.5 9B (32 layers (24 linear attention, 8 full attention), hidden size 4096). Its large weight matrices are ternary: every weight is -1, 0 or +1 times one scale per group of 128. This repo holds the same model converted to MNN for phones, 6.52 GB of model files. In the TokForge catalog it is listed for phones with 12 GB of RAM or more.

The weights are exact

• Prism stores each ternary weight matrix in a rotated basis: the runtime multiplies each layer's input by a fixed Walsh-Hadamard matrix (blocks of 4096) and a fixed +1/-1 sign pattern before the matrix product. This bundle keeps the weights in that rotated basis and puts the same rotation into the graph, using ordinary MNN operations (two small matrix products of 64 x 64, a transpose and a sign multiply), so the model computes the same function as Prism's own runtime.
• Every ternary weight is stored as a 4-bit symmetric value with one scale per block of 64 (each block sits inside one of Prism's groups of 128): the codes are -7, 0 and +7 and the scale is the group scale divided by 7, so every weight decodes back to exactly -1, 0 or +1 times its group scale (up to float32 rounding of the scale). We read every ternary matrix back from llm.mnn.weight and compared it with Prism's values: all of them match.
• The output head is its own 4-bit table (exact, as above). The embedding lookup table is separate (embeddings_bf16.bin, BF16) and holds the embedding rows already turned back from Prism's rotated storage into the model's own basis, rounded to BF16 as in other MNN bundles (the rows are not ternary in that basis).
• Not ternary in Prism's file and therefore not exact here: the two small gate projections of each linear-attention layer (stored as 32-bit floats by Prism, 4-bit here, as in other Qwen3.5 MNN bundles). Norms stay in float.
• The ternary values use 4 bits here instead of about 2 in Prism's file, so the download is about twice the size of the PQ2_0 file.

Files

file	bytes	sha256
config.json	527	6d8e2686a9668fd30afd13628e9757f458c514d2531fe0f361d74202aa47bfe4
llm_config.json	8,264	c827323b99f819830fed416e2582fbaaa0ff2a302ce85ae764e63074df5f39bd
llm.mnn	4,340,872	15368e3b65a0eec0400cd0687c96f4782670d9ddffe7cd4bd54f9b94ead3125d
llm.mnn.json	10,550,820	df9424b6ed04a8fd1f6c4d2a72caa2f6ea61e7e25aa45fc2d443e793cfd29b37
llm.mnn.weight	4,465,980,778	0bc1e9f574a3967e096e5b0aa40adedd924e207ad5a1fa1b07950d19d170cbeb
embeddings_bf16.bin	2,034,237,440	fe93d832ba64f58b0f9b3336a8c97dcf88f9093bdcd8f7dc1ac55fb5d924dcd6
tokenizer.mtok	7,342,047	6a320c5d71a1196119d993d4a825b2eee9b90727a89c1fc2dc42c7f74d19146d
LICENSE	10,174	69849221bfb90053de2134ef5e6d540287b4b98062326492f1f96f5da685524b
NOTICE	341	52d2dc5ac7c274fd2c8cfb0900e04e37c1589e4f0675ea676c0b4d10bc1f9d46

config.json names tokenizer.mtok (MNN's binary tokenizer). llm.mnn.json is the readable form of the graph. SHA256SUMS lists the same hashes.

How it was made

• Source: prism-ml/Ternary-Bonsai-2-9B-gguf at commit 5b2abb02362efbed46bbe204014b02837778b2c2, file Ternary-Bonsai-2-9B-PQ2_0.gguf (2,416,851,840 bytes, sha256 as in the repo's SHA256SUMS).
• The 2-bit packed ternary codes and their scales were unpacked; packing them again gives the original tensors byte for byte. The tensors were then laid out as a Qwen3.5 checkpoint (the inverse of llama.cpp's conversion), with the weights kept in Prism's rotated basis. The Qwen3.5 9B tokenizer and chat template are used unchanged (Prism's file carries the same ones).
• Converted with MNN's llmexport (TokForge's pinned build plus a small TokForge change that adds the rotation to the graph): --quant_bit 4 --quant_block 64 --lm_quant_bit 4 --lm_quant_block 64 --sym, text only.
• Graph: the plain LinearAttention layout. llm_config.json declares model_type: qwen3_5.

Accuracy on the host

Measured on an x86 build of the MNN engine source that TokForge ships (CPU, with this bundle's own settings), against Prism's PQ2_0 GGUF run on TokForge's llama.cpp build (CPU). Three long real-text prompts in the model's chat template, scored on the first reply token and 16 more teacher-forced steps each: 51 positions in total.

measure	result
first reply token matches	3 of 3
top-1 token matches	51 of 51
KL divergence, mean / max	0.0000 / 0.0000

Adding two short chat prompts (85 positions): top-1 85 of 85, KL max 0.0014.

License

Apache-2.0, as Prism ML's Bonsai 2 and Qwen3.5. LICENSE holds the license text and NOTICE credits the sources and states the changes.

Credits

• Ternary Bonsai 2 9B by Prism ML (Hugging Face). Apache-2.0.
• Qwen3.5 9B by the Qwen team, Alibaba Cloud (Hugging Face). Apache-2.0.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).