Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open
  Generative Large Language Models
Authors: name: Neha Sengupta; name: Sunil Kumar Sahu; name: Bokang Jia; name: Satheesh Katipomu; name: Haonan Li; name: Fajri Koto; name: Osama Mohammed Afzal; name: Samta Kamboj; name: Onkar Pandit; name: Rahul Pal; name: Lalit Pradhan; name: Zain Muhammad Mujahid; name: Massa Baali; name: Alham Fikri Aji; name: Zhengzhong Liu; name: Andy Hock; name: Andrew Feldman; name: Jonathan Lee; name: Andrew Jackson; name: Preslav Nakov; name: Timothy Baldwin; name: Eric Xing
Published: 2023-08-30 17:07:17

Abstract: We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric
foundation and instruction-tuned open generative large language models (LLMs).
The models are based on the GPT-3 decoder-only architecture and are pretrained
on a mixture of Arabic and English texts, including source code in various
programming languages. With 13 billion parameters, they demonstrate better
knowledge and reasoning capabilities in Arabic than any existing open Arabic
and multilingual models by a sizable margin, based on extensive evaluation.
Moreover, the models are competitive in English compared to English-centric
open models of similar size, despite being trained on much less English data.
We provide a detailed description of the training, the tuning, the safety
alignment, and the evaluation of the models. We release two open versions of
the model -- the foundation Jais model, and an instruction-tuned Jais-chat
variant -- with the aim of promoting research on Arabic LLMs. Available at
https://huggingface.co/inception-mbzuai/jais-13b-chat