BitNet b1.58 2B4T Technical Report
Authors: name: Shuming Ma; name: Hongyu Wang; name: Shaohan Huang; name: Xingxing Zhang; name: Ying Hu; name: Ting Song; name: Yan Xia; name: Furu Wei
Published: 2025-04-16 17:51:43

Abstract: We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
trillion tokens, the model has been rigorously evaluated across benchmarks
covering language understanding, mathematical reasoning, coding proficiency,
and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
achieves performance on par with leading open-weight, full-precision LLMs of
similar size, while offering significant advantages in computational
efficiency, including substantially reduced memory footprint, energy
consumption, and decoding latency. To facilitate further research and adoption,
the model weights are released via Hugging Face along with open-source
inference implementations for both GPU and CPU architectures.