Bonsai 27B 1bit WebGPU
An Intel GPU optimized mirror of webml-community/bonsai-webgpu-kernels. Credit goes to webml-community and its contributors for the original browser app and WebGPU inference implementation, and to Prism ML for Bonsai 27B.
The 27-billion-parameter model uses packed 1-bit weights and runs locally on your GPU through WebGPU. Approximately 3.8 GB of model data downloads from Hugging Face; prompts and replies stay on your device. The optimizations target Intel GPUs, with performance depending on the GPU, driver and browser.
Included optimizations
1. Full fp16 pipeline
Prefill and decode use fp16 activations, intermediate tensors and
linear-attention recurrent state on GPUs supporting shader-f16.
Weights and KV caches use fp16 where supported; the Q1 weights remain packed
1-bit values. Numerically sensitive reductions and selected state buffers
retain fp32.
2. Tiled symmetric LUT4 kernel
The prefill kernel reads packed 1-bit weights directly. Each workgroup uses 512 threads to compute a 32 × 512 output tile, sharing vector LUT entries across four prompt rows. Input features are processed in 64-feature batches to reuse workgroup storage.
For a four-bit mask $b$, the centered LUT value and its complement have opposite signs:
The kernel stores only masks 0–7 and reconstructs 8–15 by negation, halving LUT storage in both fp16 and fp32. Unsupported devices or shapes fall back to other matmul kernels.