IDEA: Bitnet 1.58 (a4.8) version in future variants would be so incredible!
Hy3 is incredible — thanks for releasing the weights. Would the team consider releasing a BitNet 1.58 (a4.8) variant in a future update? For anyone unfamiliar: ternary weights {-1, 0, +1} at 1.58 bits/param plus 4-bit activations means matmul becomes pure integer addition, no floating-point multipliers needed. Microsoft's bitnet.cpp already runs a 100B BitNet model on a single CPU at reading speed (5–7 tok/s) with 2–6× speedups on consumer hardware. The BitNet 1.58 paper (arXiv:2402.17764) showed perplexity parity with FP16 at 7B scale, and the a4.8 paper (arXiv:2411.04965) pushed activations down further with minimal quality loss. Training recipe is public at microsoft/unilm. A BitNet-native Hy3 would run offline on ordinary laptops and phones — no GPU needed — which is exactly the kind of accessibility a model this good deserves. Just putting it out there!
Currently, the AngelSlim team is supporting the low-bit PTQ solution. The 1.58-bit solution is not favorable for inference speed and training resource consumption. AngelSlim is expected to explore a lower-bit quantization solution that is more suitable for production and deployment. Sherry 1.25-bit and 2-bit solutions are within the consideration scope.
Currently, the AngelSlim team is supporting the low-bit PTQ solution. The 1.58-bit solution is not favorable for inference speed and training resource consumption. AngelSlim is expected to explore a lower-bit quantization solution that is more suitable for production and deployment. Sherry 1.25-bit and 2-bit solutions are within the consideration scope.
Im literally mind blown 🤯