InkubaLM: A small language model for low-resource African languages
Authors: name: Atnafu Lambebo Tonja, name: Bonaventure F. P. Dossou, name: Jessica Ojo, name: Jenalea Rajab, name: Fadel Thior, name: Eric Peter Wairagala, name: Aremu Anuoluwapo, name: Pelonomi Moiloa, name: Jade Abbott, name: Vukosi Marivate, name: Benjamin Rosman

High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.