BitMTP-Bonsai-4B-D8

Bonsai-4B checkpoint trained with the joint NTP + MTP objective at block size D=8.

Code and installation

Format and use

This repository contains a latent QAT checkpoint, configuration, and matching tokenizer. Convert it with BitMTP to group-128 ternary runtime weights before inference.

After installing BitMTP, run:

python -m bonsai_sft.download_model \
  --model bonsai_4b_d8 --output weights/BitMTP-Bonsai-4B-D8

python -m bonsai_sft.gpu.convert_bonsai_mtp_to_gpu \
  --src weights/BitMTP-Bonsai-4B-D8 --out weights/BitMTP-Bonsai-4B-D8-packed --quant ternary

python -m bonsai_sft.gpu.run_generation \
  --ckpt weights/BitMTP-Bonsai-4B-D8-packed --hf-ckpt weights/BitMTP-Bonsai-4B-D8 \
  --block-size 8 --mode hybrid --threshold 0.9 --min-chain 6 --draft-precision fp8 --verify-precision native \
  --chat --prompt 'Write a short greeting.' --max-new-tokens 128

The example targets H20.

Attribution

Created using Bonsai by Prism ML, based on Qwen3. BitMTP modifies the model through supervised fine-tuning and joint NTP + MTP training. The upstream Apache-2.0 license and notices are included in LICENSE and NOTICE.txt.