Towards Afrocentric NLP for A frican Languages: Where We Are and Where We Can Go (2022)

Abstract: Aligning with ACL 2022 special Theme on “Language Diversity: from Low Resource to Endangered Languages”, we discuss the major linguistic and sociopolitical challenges facing development of NLP technologies for African languages. Situating African languages in a typological framework, we discuss how the particulars of these languages can be harnessed. To facilitate future research, we also highlight current efforts, communities, venues, datasets, and tools. Our main objective is to motivate and advocate for an Afrocentric approach to technology development. With this in mind, we recommend what technologies to build and how to build, evaluate, and deploy them based on the needs of local African communities.

## introduction language is the foundation on which communication rests, allowing us to share ideas and interact with one another. cultures are built on this foundation. we cannot understand, nurture, or help a culture thrive without understanding and nurturing the language carrying it. language, in turn, is incubated and evolved by culture @xcite . each culture is thus naturally best expressed using the language in which it evolved, which encodes knowledge about people, their traditions, wisdom, environment, and how they interact with the sum of the concepts that belong to their own culture. technology is an element of culture that arguably both shapes and is shaped by it. technology interacts in complex ways with other elements of culture such as gender, race, and class. natural language processing (nlp) technologies are no exception, and play an increasingly important role in today's world. modern nlp technologies, however, have primarily been developed in western societies. as such, they often function within contexts of values, norms, and beliefs that reflect these societies and serve their needs. on the other hand, the very methods employed to develop most of these technologies and the knowledge on which they rest also derive from the same western-centric approaches. this poses challenges to the extension and use of these technologies in communities with different social fabrics that speak different languages. the scale of this problem is huge, because the majority of the world's 7000+ living languages @xcite are not nlp-supported. apart from perhaps two dozens of popular languages, most languages of the world are under-resourced, indigenous, and/or endangered. most african languages fall within this category and are the focus of this paper (figure 1 ). our goal is to discuss the major linguistic and sociopolitical challenges facing development of nlp technologies for african languages. 1 in doing so, we both motivate and advocate for an afrocentric approach to technology development where what technologies to build and how to build, evaluate, and deploy them arise from the needs of local african communities. we start by typologically situating african languages and providing illustrating examples as to what makes them challenging from a computational linguistics perspective ( § 2). next, we discuss consequences of the literacy situation in africa on nlp ( § 3). we then further explain why the classical binary approach to technology development of feature engineering vs. end-toend solutions familiar to most nlp researchers is not ideal for the african context ( § 4). we follow by data quality ( § 5). to facilitate future work, we also point to ongoing community efforts, venues, and datasets ( § 6). we conclude in § 7. ## why typology matters although it has been argued that the best way to achieve cross-linguistically useful nlp is to leverage findings of typological research @xcite , most nlp work remains indo-eurocentric in terms of algorithms for preprocessing, training, and evaluation. this is a mismatch to the fact that every nlp approach requires either explicit or implicit representative linguistic knowledge @xcite @xcite . knowledge of linguistic typology can indeed be very useful for both language-specific and languageindependent nlp @xcite , including for african languages. this knowledge can be useful for determining which languages may be treated together (e.g., in multilingual models) and/or which methods are best suited for a language-specific task (e.g., a method can be deemed potentially useful if it has been applied successfully on a language with a similar typology). to illustrate what typological information can concretely mean for african languages, it may be useful here to list a number of the most notable typological features prevalent in african languages across several language families including afro-asiatic, austronesian, niger-congo, nilo-saharan, indo-european and creole. these features include use of tone, open syllables, vowel harmony, splitting verbs, serial verb construction, reduplication, use of very few or no adjectives (closed class of adjectives @xcite ), and a large number of ideophones. we will discuss three of these features which we judge as largely absent from most of the top 10 nlp-popular languages. 2 we provide a list indicating presence of one or more of these features in over 100 african languages in appendix table b .1. ## a tale of two approaches there are two main approaches for developing nlp systems. we discuss each of these visa-vis the situation for african languages here, giving relevant recommendations. feature engineering. feature engineering requires domain knowledge, which is lacking for many african languages due to the aforementioned literacy situation. this negatively impacts use of written african languages in many domains of human endeavor, let alone nlp research. weak literacy simply means unavailability and inaccessibility of linguists, annotators, language experts, and computational linguists with expertise in african languages. it also manifests itself in lack of grammars, primers, teaching aids, and dictionaries @xcite . as it turns out, grammatical information is either fully lacking or under-documented for almost half of africa's languages. this makes africa the second least known contin-ent (after oceania, dominated by the new guinea area) @xcite . in appendix table f .2, we list available linguistic resources for all african languages we could trace. ## garbage in, garbage out a manual evaluation of 205 datasets involving african languages such as those in ccaligned (el-kishky et al., 2020), paracrawl @xcite , wikimatrix @xcite , oscar @xcite , and mc4 @xcite show that at least 15 corpora were completely erroneous, a significant fraction contained less than 50% of correct data, and 82 corpora were mislabelled or used ambiguous language codes @xcite . the inaccuracy is due to a lack, or poor quality of language identification tools, dictionaries, and text pre-processing piplelines, for many low resource languages including african languages represented in these datasets. furthermore, available resources are rarely evaluated especially when crawled as part of a multilingual dataset. furthermore, alabi et al. (2020) find that, fasttext embeddings for yorùbá has an estimated 135k out of 150k words belonging to other languages such as english, french, and arabic. new embedding models created by alabi et al. (2020) with a curated high quality dataset outperform the off-the-shelf fasttext embeddings even though the curated set has fewer words. results of these few studies paint a gloomy picture for most current multilingual datasets involving african languages, and models derived from them. inconsistent orthographies also contribute to the data quality problem @xcite . in many cases, orthographies may not be standardized and will have significant spelling and punctuation variations across different domains. in some cases where standard orthographies exist, word lists or dictionaries do not necessarily represent the standardized orthography. using hausa as an example, all commercially published books and nearly all hausa language newspapers use the standard romanized orthography. standard romanized orthography is written without tones or any indication of vowel length @xcite . the orthography used in grammars, dictionaries, and pedagogical documents on the other hand, indicate tone and vowel length @xcite . furthermore, languages that have standard orthographies may also suffer from inconsistencies when orthographic conventions are not adhered to @xcite . this is evident in the methods and practices for content archiving of many african languages on the web. for example, all voa websites, omit tones for african languages whose standard orthographies require tone diacritics. bbc also does not adhere to the orthographic conventions for yorùbá texts except in the headlines, jw.org also does the same for some african languages. apart from the aforementioned issues, lack of constant and systematic use of african languages in contexts such as governance, law, technology, science, and education prevents african languages from expanding in vocabulary to accommodate new concepts that have become important parts of conversation elsewhere. as a result, it is not uncommon to have large amounts of foreign words in a dataset which are not adapted to the phonological or orthographic structure of the target african language. furthermore, terminologies continue to be employed inconsistently and spelt differently in many african venues. to provide a concrete example of the data quality problem for african languages, we perform a manual evaluation of flores-101 dataset @xcite for yorùbá. we find the following: (1) 5.29% spelling errors (2) 2.7% inconsistent spellings (3) 1.2% borrowed words not adapted to the orthographic conventions of target language and (4) 12.4% incorrect tone marks. detailed information is in appendix g. it is important to mention that a single error in assignment of diacritics, for instance, can result in significant semantic and syntactic differences in texts. the implication of inconsistencies in orthography is hence enormous for low resource african languages. such inconsistencies worsen the issue of data sparsity: when different spellings of the same word are employed, or when tone or other grammatical features are inconsistently marked, the same 'word' will have many more surface forms than what it actually should. data sparsity can in turn aggravate the situation for any work involving training with data from different domains (e.g., in domain adaptation). that is, reliability of models trained with erroneous data from a source domain will be diminished while transferring into a target domain. orthographic inconsistencies also affect results of search engines @xcite in that these engines would not recognize the relationship between a diacritized text and its undiacritized counterparts @xcite . again, this results in difficulty retrieving resources for many african languages. to optimize search for african languages that involve diacritics, some users employ normalized text which in turn further creates a mismatch between web documents and other standard offline documents (e.g., books) for many african languages. recommendations. (1) we recommend developing language identification tools that cover african languages. (2) development of dictionaries or even extended word lists will also help the community ensure data quality. (3) manual inspection of sizeable samples of multilingual datasets should also continue to be prioritized. (4) we also suggest orchestrated efforts to enforce consistency in orthography for the various languages. (5) linguistic rules may be appropriate for developing automatic data cleaning and pre-processing, but development of any such rules should be carried out carefully. we now briefly highlight community efforts invested in developing skills, datasets, and tools in the african nlp space. ## communities and resources the majority of existing resources for nlp are the initiative of various non-governmental organizations determined to develop datasets and tools for african languages. we list some of these efforts for nlp, but also within the larger contexts of artificial intelligence. we focus on communities and venues here and list recent funding initiatives in table d .1 (appendix). workshops. as far as we know, there are two main venues in the form of workshops supporting nlp for african languages, and african ai. these are africannlp and black-inai. we provide details about these venues in appendix e. communities. masakhane, black in ai, deep learning indaba, knowledge 4 all foundation ltd (k4a), zindi and alti are some of the active communities for research on nlp for african languages. more information about each of these communities is in section d. resources. the religious domain is currently the major source of data for a large number of african languages. top amongst religious resources is the bible corpus (available in over 1, 000 languages of africa @xcite ) and the jw300 website (with data for ∼ 100 low-resource african languages). religious sources are constantly updated with new data from the same languages and new languages are often added, making these sources increasingly useful. one issue of these datasets is that, although they are parallel, they may not be sentence aligned. regardless, these resources remain significantly inadequate. most other data available for african languages are raw and unlabelled. still, these can be useful in many applications (e.g., in training word embeddings or language models, for backtranslation). we provide more details about available resources (labelled, unlabelled, and raw) and tools in appendix f. recommendations. (1) to achieve afrocentric nlp, we recommend active interactions between differently existing communities, as well as encouraging new regional and thematically-defined communities. (2) we recommend extending these communities beyond ai, nlp, and machine learning to involve theoretical linguists, anthropologists, sociologists, field workers, and other scholars and practitioners with interest in african languages. (3) we believe acl and other similar organizations should continue to prioritize work on lowresource languages by securing dedicated tracks in their publication and dissemination venues. ## discussion and conclusion we discussed major challenges facing development of nlp technologies for african languages. one of the most important recommendations we would like to emphasize is to prioritize african nlp work based on the needs of african communities. for example, we believe development for data and tools for improving health and education should be a priority. we also caution against extractive practices, and encourage creation of opportunities, contexts, and venues for work on african languages and advocacy for reclaiming african language policies. in addition, data literacy and issues around data sovereignty and privacy should remain of highest importance. we highlighted various communities and venues here that we think should continue to be supported. david i adelani, dana ruiter, jesujoba o alabi, damilola adebonojo, adesina ayeni, mofe adeyemi, ayodele awokoya, and cristina españa-bonet. 2021a. menyo-20k: a multi-domain english-yorùbá corpus for machine translation and domain adaptation. arxiv preprint arxiv:2103.08647. david ifeoluwa adelani, jade abbott, graham neubig, daniel d'souza, julia kreutzer, constantine lignos, chester palen-michel, happy buzaaba, shruti rijhwani, sebastian ruder, stephen mayhew, israel abebe azime, shamsuddeen h. muhammad, chris chinenye emezue, joyce nakatumba-nabende, perez ogayo, aremu anuoluwapo, catherine gitau, derguene mbaye, jesujoba alabi, seid muhie yimam, tajuddeen rabiu gwadabe, ignatius ezeani, rubungo andre niyongabo, jonathan mukiibi, verrah otiende, iroro orife, davis david, samba ngom, tosin adewumi, paul rayson, mofetoluwa adeyemi, gerald muriuki, emmanuel anebi, chiamaka chukwuneke, nkiruka odu, eric peter wairagala, samuel oyerinde, clemencia siro, tobius saul bateesa, temilola oloyede, yvonne wambui, victor akinode, deborah nabagereka, maurice katusiime, ayodele awokoya, mouhamadane mboup, dibora gebreyohannes, henok tilaye, kelechi nwaike, degaga wolde, abdoulaye faye, blessing sibanda, orevaoghene ahia, bonaventure f. p. dossou, kelechi ogueji, thierno ibrahima diop, abdoulaye diallo, adewale akinfaderin, tendai marengereke, and salomey osei. 2021b. masakhaner: named entity recognition for african languages. transactions of the association for computational linguistics, 9:1116-1131. david ifeoluwa adelani, dana ruiter, jesujoba o alabi, damilola adebonojo, adesina ayeni, mofe adeyemi, ayodele awokoya, and c espana-bonet. 2021c. the effect of domain and diacritics in yorùbá-english neural machine translation. proceedings of machine translation summit (research track). european association for machine translation. željko agić and ivan vulić. 2019. jw300: a widecoverage parallel corpus for low-resource languages. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 3204-3210, florence, italy. association for computational linguistics. orevaoghene ahia and kelechi ogueji. 2020. towards supervised and unsupervised neural machine translation baselines for nigerian pidgin. arxiv preprint arxiv:2003.12660. thipe i modipa, febe de wet, and marelie h davel. 2013. implications of sepedi/english code switching for asr systems. conference proceedings of the 24th annual symposium of the pattern recognition association of south africa, johannesburg, south africa, 3 december 2013. jama hussein mohamud, lloyd acquaye thompson, aissatou ndoye, and laurent besacier. 2021. fast development of asr in african languages using self supervised speech representation learning. corr, abs/2103.08993. susanne mohr. 2018. the changing dynamics of language use and language attitudes in tanzania. language matters, 49(3):105-127. justin mott, ann bies, stephanie strassel, jordan kodner, caitlin richter, hongzhi xu, and mitch marcus. 2020. morphological segmentation for low resource languages. in proceedings of the 12th language resources and evaluation conference, pages 3996-4002. wilhelmina nekoto, vukosi marivate, tshinondiwa matsila, timi fasubaa, taiwo fagbohungbe, solomon oluwole akinola,