Nemotron-4 15B Technical Report
Authors: name: Jupinder Parmar; name: Shrimai Prabhumoye; name: Joseph Jennings; name: Mostofa Patwary; name: Sandeep Subramanian; name: Dan Su; name: Chen Zhu; name: Deepak Narayanan; name: Aastha Jhunjhunwala; name: Ayush Dattagupta; name: Vibhu Jawa; name: Jiwei Liu; name: Ameya Mahabaleshwarkar; name: Osvald Nitski; name: Annika Brundyn; name: James Maki; name: Miguel Martinez; name: Jiaxuan You; name: John Kamalu; name: Patrick LeGresley; name: Denys Fridman; name: Jared Casper; name: Ashwath Aithal; name: Oleksii Kuchaiev; name: Mohammad Shoeybi; name: Jonathan Cohen; name: Bryan Catanzaro
Published: 2024-02-26 18:43:45

Abstract: We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual
language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates
strong performance when assessed on English, multilingual, and coding tasks: it
outperforms all existing similarly-sized open models on 4 out of 7 downstream
evaluation areas and achieves competitive performance to the leading open
models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best
multilingual capabilities of all similarly-sized models, even outperforming
models over four times larger and those explicitly specialized for multilingual
tasks.