BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Published on Nov 9, 2022


Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.



Paper author
edited Mar 17, 2023

Amazing stuff as always @mrm8488 !
Is the model up somewhere? ๐Ÿค—๐ŸŒธ๐ŸŒฟ

Paper author

Also, for people interested in the BLOOMZ follow-up paper definitely go have a look there!


Is there any planned support for Italian language and can I contribute?

Paper author

Hey @lucabianchi ,

you could get in contact with @malteos , because he did that with German:

And he's very open for adapting new languages!

Paper author

@librarian-bot recommend

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend


Sign up or log in to comment

Models citing this paper 8

Browse 8 models citing this paper

Datasets citing this paper 1

Spaces citing this paper 474

Collections including this paper 12