BitMTP-Bonsai-1.7B-D8

Bonsai-1.7B checkpoint trained with the joint NTP + MTP objective at block size D=8.

Code and installation

Format and use

This repository contains a latent QAT checkpoint, configuration, and matching tokenizer. Convert it with BitMTP to group-128 ternary runtime weights before inference.

After installing BitMTP, run:

python -m bonsai_sft.download_model \
  --model bonsai_1p7_d8 --output weights/BitMTP-Bonsai-1.7B-D8

python -m bonsai_sft.gpu.convert_bonsai_mtp_to_gpu \
  --src weights/BitMTP-Bonsai-1.7B-D8 --out weights/BitMTP-Bonsai-1.7B-D8-packed --quant ternary

python -m bonsai_sft.gpu.run_generation \
  --ckpt weights/BitMTP-Bonsai-1.7B-D8-packed --hf-ckpt weights/BitMTP-Bonsai-1.7B-D8 \
  --block-size 8 --mode hybrid --threshold 0.9 --min-chain 6 --draft-precision fp8 --verify-precision native \
  --chat --prompt 'Write a short greeting.' --max-new-tokens 128

The example targets H20. See the platform guides for RTX 3090 and Jetson Orin NX.

Attribution

Created using Bonsai by Prism ML, based on Qwen3. BitMTP modifies the model through supervised fine-tuning and joint NTP + MTP training. The upstream Apache-2.0 license and notices are included in LICENSE and NOTICE.txt.