MMTA frica: Multilingual Machine Translation for A frican Languages (2021)

Abstract: In this paper, we focus on the task of multilingual machine translation for African languages and describe our contribution in the 2021 WMT Shared Task: Large-Scale Multilingual Machine Translation. We introduce MMTAfrica, the first many-to-many multilingual translation system for six African languages: Fon (fon), Igbo (ibo), Kinyarwanda (kin), Swahili/Kiswahili (swa), Xhosa (xho), and Yoruba (yor) and two non-African languages: English (eng) and French (fra). For multilingual translation concerning African languages, we introduce a novel backtranslation and reconstruction objective, BT&REC, inspired by the random online back translation and T5 modelling framework respectively, to effectively leverage monolingual data. Additionally, we report improvements from MMTAfrica over the FLORES 101 benchmarks (spBLEU gains ranging from +0.58 in Swahili to French to +19.46 in French to Xhosa).

## introduction despite the progress of multilingual machine translation (mmt) and the many efforts towards improving its performance for low-resource languages, african languages suffer from under-representation. for example, of the 2000 known african languages @xcite only 17 of them are available in the flores 101 large-scale multilingual translation task as at the time of this research. furthermore, most research that look into transfer learning of multlilingual models from high-resource to low-resource languages rarely work with als in the low-resource scenario. while the consensus is that the outcome of the research made using the low-resource non-african languages should be scalable to african languages, this cross-lingual generalization is not guaranteed @xcite and the extent to which it actually works remains largely understudied. transfer learning from african languages to african languages sharing the same language sub-class has been shown to give better translation quality than from high-resource anglo-centric languages @xcite calling for the need to investigate al←→al multilingual translation. this low representation of african languages goes beyond machine translation @xcite @xcite . the analysis conducted by @xcite revealed that low-resourcedness of african languages can be traced to the poor incorporation of african languages in the nlp research community @xcite . all these call for the inclusion of more african languages in multilingual nlp research and experiments. with the high linguistic diversity in africa, multilingual machine translation systems are very important for inter-cultural communication, which is in turn necessary for peace and progress. for example, one widely growing initiative to curb the large gap in scientific research in africa is to translate educational content and scientific papers to various african languages in order to reach far more african native speakers @xcite @xcite . we take a step towards addressing the underrepresentation of african languages in mmt and improving experiments by participating in the 2021 wmt shared task: large-scale multilingual machine translation with a major target of als←→als. in this paper, we focused on 6 african languages and 2 non-african languages (english and french). our main contributions are summarized below: language lang id (iso 639-3) family speakers region igbo ibo niger-congo-volta-niger 27m west fon (fongbe) fon niger-congo-volta-congo-gbe 1.7m west kinyarwanda kin niger-congo-bantu 12m east swahili swa niger-congo-bantu 98m southern, central & east xhosa xho niger-congo-nguni bantu 19.2m southern yorùbá yor niger-congo-volta-niger 42m west ## data methodology parallel corpora: as nmt models are very reliant on parallel data, we sought to gather more parallel sentences for each language direction in an effort to increase the size and domain of each language direction. to this end, our first source was jw300 @xcite , a parallel corpus of over 300 languages with around 100 thousand biblical domain parallel sentences per language pair on average. using opustools @xcite we were able to get only very trustworthy translations by setting @xmath0.5 (t is a threshold which indicates the confidence of the translations). we collected more parallel sentences from tatoeba foot_0 , kde4 foot_1 @xcite , and some english-based bilingual samples from multiparacrawl foot_2 . finally, following pointers from the native speakers of these focus languages in the masakhane community @xcite to existing research on machine translation for african languages which opensourced their parallel data, we assembled more parallel sentences mostly in the {en, f r}←→al direction. from all this we created mmtafrica test set (explained in more details in section 3.1), got 5, 424, 578 total training samples for all languages directions (a breakdown of data size for each language direction is provided in monolingual corpora: despite our efforts to gather several parallel data from various domains, we were faced with some problems: 1) there was a huge imbalance in parallel samples across the language directions. in we therefore set out to gather monolingual data from diverse sources. as our focus is on african languages, we collated monolingual data in only these languages. the monolingual sources and volume are summarized in language(id) monolingual source size xhosa (xho) the cc100-xhosa dataset created by conneau et al. (2019), and openslr (van niekerk et al., 2017) 158, 660 yoruba (yor) yoruba embeddings corpus (alabi et al., 2020) and menyo20k (adelani et al., 2021a) 45, 218 fon/fongbe (fon) ffr dataset (dossou and emezue, 2020), and fon french daily dialogues parallel data (dossou and emezue, 2021a) 42, 057 swahili/kiswahili (swa) (shikali and refuoe, 2019) 23, 170 kinyarwanda (kin) kinnews-and-kirnews (niyongabo et al., 2020) 7, 586 igbo (ibo) (ezeani et al., 2020) 7, 817 ## results and insights all evaluations were made using spbleu (sentencepiece @xcite ) + sacre-bleu @xcite ) as described in @xcite . we further evaluated on the chrf @xcite and ter metrics. ## conclusion and future work in this paper, we introduced mmtafrica, a multilingual machine translation model on 6 african languages, which outperformed the m2m mmt model @xcite . our results and analyses, including a new reconstruction objective, give insights on mmt for african languages for future research. moreover, we plan to launch the model on masakhane mt and ffrtranslate in order to get human evaluation feedback from the actual speakers of the languages in the masakhane community @xcite and beyond. in order to fully test the advantage of mmtafrica, we plan to finish comparing it on 11 https://en.wikipedia.org/wiki/benin (last accessed : 30.08.2021).