BitNet b1.58 2B4T Technical Report
hongyuw
Abstract
We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures.
Community
We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures.
🔥 Inference weights: https://huggingface.co/microsoft/bitnet-b1.58-2B-4T
🔥 Training weights (bf16): https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16
🧰 Inference code: https://github.com/microsoft/bitnet
📄 Technical report will be available soon.
Try it out via https://bitnet-demo.azurewebsites.net, we are cooking larger models🥳🥳
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models (2025)
- Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing (2025)
- RWKV-7"Goose"with Expressive Dynamic State Evolution (2025)
- Evaluating Compact LLMs for Zero-Shot Iberian Language Tasks on End-User Devices (2025)
- PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing (2025)
- Overcoming Vocabulary Constraints with Pixel-level Fallback (2025)
- Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 5
Browse 5 models citing this paperDatasets citing this paper 0
No dataset linking this paper