Cheetah: Natural Language Generation for 517 A frican Languages (2024)

Abstract: Low-resource African languages pose unique challenges for natural language processing (NLP) tasks, including natural language generation (NLG). In this paper, we develop Cheetah, a massively multilingual NLG language model for African languages. Cheetah supports 517 African languages and language varieties, allowing us to address the scarcity of NLG resources and provide a solution to foster linguistic diversity. We demonstrate the effectiveness of Cheetah through comprehensive evaluations across six generation downstream tasks. In five of the six tasks, Cheetah significantly outperforms other models, showcasing its remarkable performance for generating coherent and contextually appropriate text in a wide range of African languages. We additionally conduct a detailed human evaluation to delve deeper into the linguistic capabilities of Cheetah. The findings of this study contribute to advancing NLP research in low-resource settings, enabling greater accessibility and inclusion for African languages in a rapidly expanding digital landscape. We will publicly release our models for research.

## introduction the linguistic diversity present in african languages poses unique challenges for nlg systems. with over 2, 000 languages spoken across the african continent @xcite , the need for effective nlg solutions that can accommodate this rich linguistic ecosystem cannot be over-emphasized. this is especially important because traditional nlg approaches have primarily focused on high-resource languages, such as english and french due to the availability of large-scale datasets and resources. consequently, low-resource ⋆ authors contributed equally. languages, including numerous african languages, have been marginalized in nlg research and development. developing robust nlg systems for the diverse needs of african communities is challenging due to the scarcity of extensive language datasets, limited linguistic research, and variations across these languages. to address these challenges, recent advancements in language modeling and transfer learning techniques have shown promise in supporting nlg in low-resource languages. pretrained language models, such as @xcite @xcite , mt5 @xcite , and mt0 @xcite , have demonstrated remarkable capabilities in understanding and generating human-like text. these models capture the statistical regularities and syntactic structures of the languages they are trained on, making them valuable starting points for supporting nlg in low-resource settings. in this paper, we present a pioneering work on nlg in african languages by introducing cheetah: a novel language model (lm) specifically designed to support 517 african languages and language varieties. to the best of our knowledge, cheetah supports the largest number of african languages and language varieties. leveraging a vast corpus of text data collected from diverse sources, cheetah learns intricate linguistic information that characterizes each african language. the contributions of this work are three fold. first, we address the scarcity of nlg resources for african languages by providing a comprehensive language model that covers a wide range of languages spoken on the continent. second, we demonstrate the efficacy of our approach through extensive evaluations across six downstream task clusters. each cluster includes multiple languages, showcasing the model's ability to generate coherent and contextually appropriate text in different african languages. third, we perform fine grained human analysis of cheetah using a controlled machine translation (mt) test set. this uncovers model behaviour that is not visible with automatic metrics. by supporting nlg in african languages, we foster linguistic diversity, empower african communities to express themselves in their native languages, and bridge the digital divide. this paper serves as a foundational step towards promoting afrocentric nlp @xcite ) that prioritizes inclusivity and cultural preservation in language technology, emphasizing the importance of catering to the unique linguistic needs of diverse populations. the rest of the paper is organized as follows: in section 2, we discuss related work. in section, 4 we describe afronlg, the benchmark we create for evaluation. we provide details of cheetah in section 3. we present performance of cheetah in section 5 and compare it to other multilingual models. we present controlled test sets in section 5.1. we conclude in section 6, and outline a number of limitations and use cases for our work in section 7 and section 8. ## literature review one of the challenges in nlg is to generate coherent and semantically meaningful text. various approaches have been proposed, including templatebased @xcite , rule-based @xcite , and statistical approaches @xcite . more recently, deep learning approaches @xcite including the transformer model @xcite have achieved sota results in various nlg tasks such as text summarization @xcite and machine translation @xcite . while these models have shown impressive results, they often require a large amount of training data and computing resources. however, only a few african languages have benefited from these advancements due to inadequate data. to address this issue, researchers have proposed transfer learningbased approaches, where a pretrained model is finetuned for a specific nlg task. transfer learning @xcite @xcite has enabled the use of low-resource languages on various nlp tasks. due to lack of adequate (or good quality) pretraining data @xcite , transfer learning is often the most accessible method for only a few low-resource languages leaving behind a vast majority of extremely low-resource languages. this is because about 90% of the world's languages is claimed to be either leftbehinds, in that it is probably impossible to build nlp resources for them, or scraping-bys with no labelled datasets @xcite . for the leftbehinds, labelled and unlabelled data are unavailable and even transfer learning approaches are beyond reach while the scraping-by languages have no labelled data with which to evaluate model performance. ## evaluation and results we evaluate cheetah on six task clusters of afronlg benchmark and compare to performance on mt0, mt5, afri-mt5, and afriteva. we report results in ## conclusion we introduced cheetah, a massively multilingual language model designed for african natural language generation. we also propose a new african language generation benchmark, dubbed afronlg, that is both sizeable and diverse. we evaluate cheetah on afronlg comparing it to three other models, two multilingual and one dedicated to african languages. the performance of cheetah surpasses that of all other models we evaluate. this is demonstrated by its superior afronlg score, which is approximately three times better than the combined performance of other models. furthermore, cheetah outperforms all other models across 48 out of 65 test sets spanning six task clusters. we further analyze our model's robustness to lexical complexity and carry out human evaluation to inspect the model's perform on a controlled test set. again, our results underscore superiority of our model.