BitMTP-Bonsai-4B-SFT

Supervised fine-tuned Bonsai-4B checkpoint used for the SFT AR reference in BitMTP.

Code and installation

Format and use

This repository contains a latent QAT checkpoint, configuration, and matching tokenizer. Convert it with BitMTP to group-128 ternary runtime weights before inference.

After installing BitMTP, run:

python -m bonsai_sft.download_model \
  --model bonsai_4b_sft --output weights/BitMTP-Bonsai-4B-SFT

python -m bonsai_sft.gpu.convert_bonsai_mtp_to_gpu \
  --src weights/BitMTP-Bonsai-4B-SFT --out weights/BitMTP-Bonsai-4B-SFT-packed --quant ternary

python -m bonsai_sft.gpu.run_generation \
  --ckpt weights/BitMTP-Bonsai-4B-SFT-packed --hf-ckpt weights/BitMTP-Bonsai-4B-SFT \
  --block-size 8 --mode ar --draft-precision native --verify-precision native \
  --chat --prompt 'Write a short greeting.' --max-new-tokens 128

The example targets H20.

Attribution

Created using Bonsai by Prism ML, based on Qwen3. BitMTP modifies the model through supervised fine-tuning. The upstream Apache-2.0 license and notices are included in LICENSE and NOTICE.txt.