Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
Authors: name: Jamba Team, name: Barak Lenz, name: Alan Arazi, name: Amir Bergman, name: Avshalom Manevich, name: Barak Peleg, name: Ben Aviram, name: Chen Almagor, name: Clara Fridman, name: Dan Padnos, name: Daniel Gissin, name: Daniel Jannai, name: Dor Muhlgay, name: Dor Zimberg, name: Edden M Gerber, name: Elad Dolev, name: Eran Krakovsky, name: Erez Safahi, name: Erez Schwartz, name: Gal Cohen, name: Gal Shachaf, name: Haim Rozenblum, name: Hofit Bata, name: Ido Blass, name: Inbal Magar, name: Itay Dalmedigos, name: Jhonathan Osin, name: Julie Fadlon, name: Maria Rozman, name: Matan Danos, name: Michael Gokhman, name: Mor Zusman, name: Naama Gidron, name: Nir Ratner, name: Noam Gat, name: Noam Rozen, name: Oded Fried, name: Ohad Leshno, name: Omer Antverg, name: Omri Abend, name: Opher Lieber, name: Or Dagan, name: Orit Cohavi, name: Raz Alon, name: Ro'i Belson, name: Roi Cohen, name: Rom Gilad, name: Roman Glozman, name: Shahar Lev, name: Shaked Meirom, name: Tal Delbari, name: Tal Ness, name: Tomer Asida, name: Tom Ben Gal, name: Tom Braude, name: Uriya Pumerantz, name: Yehoshua Cohen, name: Yonatan Belinkov, name: Yuval Globerson, name: Yuval Peleg Levy, name: Yoav Shoham

We present Jamba-1.5, new instruction-tuned large language models based on
our Jamba architecture. Jamba is a hybrid Transformer-Mamba mixture of experts
architecture, providing high throughput and low memory usage across context
lengths, while retaining the same or better quality as Transformer models. We
release two model sizes: Jamba-1.5-Large, with 94B active parameters, and
Jamba-1.5-Mini, with 12B active parameters. Both models are fine-tuned for a
variety of conversational and instruction-following capabilties, and have an
effective context length of 256K tokens, the largest amongst open-weight
models. To support cost-effective inference, we introduce ExpertsInt8, a novel
quantization technique that allows fitting Jamba-1.5-Large on a machine with 8
80GB GPUs when processing 256K-token contexts without loss of quality. When
evaluated on a battery of academic and chatbot benchmarks, Jamba-1.5 models
achieve excellent results while providing high throughput and outperforming
other open-weight models on long-context benchmarks. The model weights for both
sizes are publicly available under the Jamba Open Model License and we release
ExpertsInt8 as open source.