BitMTP-Bonsai-1.7B-SFT

Supervised fine-tuned Bonsai-1.7B checkpoint used for the SFT AR reference in BitMTP.

Code and installation

Format and use

This repository contains a latent QAT checkpoint, configuration, and matching tokenizer. Convert it with BitMTP to group-128 ternary runtime weights before inference.

After installing BitMTP, run:

python -m bonsai_sft.download_model \
  --model bonsai_1p7_sft --output weights/BitMTP-Bonsai-1.7B-SFT

python -m bonsai_sft.gpu.convert_bonsai_mtp_to_gpu \
  --src weights/BitMTP-Bonsai-1.7B-SFT --out weights/BitMTP-Bonsai-1.7B-SFT-packed --quant ternary

python -m bonsai_sft.gpu.run_generation \
  --ckpt weights/BitMTP-Bonsai-1.7B-SFT-packed --hf-ckpt weights/BitMTP-Bonsai-1.7B-SFT \
  --block-size 8 --mode ar --draft-precision native --verify-precision native \
  --chat --prompt 'Write a short greeting.' --max-new-tokens 128

The example targets H20. See the platform guides for RTX 3090 and Jetson Orin NX.

Attribution

Created using Bonsai by Prism ML, based on Qwen3. BitMTP modifies the model through supervised fine-tuning. The upstream Apache-2.0 license and notices are included in LICENSE and NOTICE.txt.