Nemotron-4 15B Technical Report
Authors: name: Jupinder Parmar, name: Shrimai Prabhumoye, name: Joseph Jennings, name: Mostofa Patwary, name: Sandeep Subramanian, name: Dan Su, name: Chen Zhu, name: Deepak Narayanan, name: Aastha Jhunjhunwala, name: Ayush Dattagupta, name: Vibhu Jawa, name: Jiwei Liu, name: Ameya Mahabaleshwarkar, name: Osvald Nitski, name: Annika Brundyn, name: James Maki, name: Miguel Martinez, name: Jiaxuan You, name: John Kamalu, name: Patrick LeGresley, name: Denys Fridman, name: Jared Casper, name: Ashwath Aithal, name: Oleksii Kuchaiev, name: Mohammad Shoeybi, name: Jonathan Cohen, name: Bryan Catanzaro
Organization: NVIDIA

We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual
language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates
strong performance when assessed on English, multilingual, and coding tasks: it
outperforms all existing similarly-sized open models on 4 out of 7 downstream
evaluation areas and achieves competitive performance to the leading open
models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best
multilingual capabilities of all similarly-sized models, even outperforming
models over four times larger and those explicitly specialized for multilingual
tasks.