InkubaLM: A small language model for low-resource African languages
Authors: name: Atnafu Lambebo Tonja; name: Bonaventure F. P. Dossou; name: Jessica Ojo; name: Jenalea Rajab; name: Fadel Thior; name: Eric Peter Wairagala; name: Aremu Anuoluwapo; name: Pelonomi Moiloa; name: Jade Abbott; name: Vukosi Marivate; name: Benjamin Rosman
Published: 2024-08-30 05:42:31

Abstract: High-resource language models often fall short in the African context, where
there is a critical need for models that are efficient, accessible, and locally
relevant, even amidst significant computing and data constraints. This paper
introduces InkubaLM, a small language model with 0.4 billion parameters, which
achieves performance comparable to models with significantly larger parameter
counts and more extensive training data on tasks such as machine translation,
question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM
outperforms many larger models in sentiment analysis and demonstrates
remarkable consistency across multiple languages. This work represents a
pivotal advancement in challenging the conventional paradigm that effective
language models must rely on substantial resources. Our model and datasets are
publicly available \url{https://huggingface.co/lelapa} to encourage
research and development on low-resource languages.