N gambay- F rench Neural Machine Translation (sba-Fr) (2023)

Abstract: In Africa, and the world at large, there is an increasing focus on developing Neural Machine Translation (NMT) systems to overcome language barriers. NMT for Low-resource language is particularly compelling as it involves learning with limited labelled data. However, obtaining a well-aligned parallel corpus for low-resource languages can be challenging. The disparity between the technological advancement of a few global languages and the lack of research on NMT for local languages in Chad is striking. End-to-end NMT trials on low-resource Chad languages have not been attempted. Additionally, there is a dearth of online and well-structured data gathering for research in Natural Language Processing, unlike some African languages. However, a guided approach for data gathering can produce bitext data for many Chadian language translation pairs with well-known languages that have ample data. In this project, we created the first sba-Fr Dataset, which is a corpus of Ngambay-to-French translations, and fine-tuned three pre-trained models using this dataset. Our experiments show that the M2M100 model outperforms other models with high BLEU scores on both original and original+synthetic data. The publicly available bitext dataset can be used for research purposes.

## introduction differential access to information is a pervasive issue in both developed and developing nations, reinforced by physical, social, and economic structures. the problem is especially acute in rural areas, where the lack of communication technology such as the internet can severely limit access to information. furthermore, automated translation tools face significant challenges in dealing with low-resource language pairs and morphologically rich languages, leading to limited cultural exchange and market integration for certain nations. a major contributor to this problem is the fact that internet research is primarily conducted in languages such as english, french, spanish, german, etc. resulting in limited data availability for other languages. as a result, machine translation (mt) is heavily dependent on parallel text or "bitext," leaving speakers of languages with limited data resources or parallel corpora at a disadvantage when it comes to building mt models @xcite . to make the recent successes of mt systems accessible and inclusive, research efforts should focus on identifying and closing the technological gap between these languages that lack digital or computational data resources. addressing this gap will require innovative approaches for data collection and processing, as well as the development of new mt models that can effectively operate with limited resources. the ngambay language is one of such marginalized and low-resource language facing the challenges of information access and automated translation. as an example of a morphologically rich language, ngambay encounters significant difficulties in finding adequate translation resources, limiting cultural exchange and economic integration opportunities. the scarcity of internet research conducted in languages like ngambay further exacerbates this problem, leaving speakers of such languages at a disadvantage in building mt models. bridging the technological gap for languages with limited digital and computational resources, like ngambay, is essential to ensure inclusivity and accessibility to the recent successes of mt systems. this research aims to contribute to the advancement of nmt for low-resource languages like ngambay, making strides toward more equitable access to information and linguistic inclusion. ## related work machine translation is a crucial subfield of natural language processing (nlp) that utilizes computers to translate natural languages. recently, end-to-end neural machine translation (nmt) has emerged as the new standard method in practical mt systems, leveraging transformer models with parallel computation and attention mechanism @xcite . although nmt models require extensive parallel data, which is typically only available for a limited number of language pairs @xcite , some research has been conducted on nmt using rare african languages such as swahili, hausa, yoruba, wolof, amharic, bambara, ghomala, ewe, fon, kinyarwanda, and others. (emezue and dossou, 2020) introduced the ffr dataset, a corpus of fon-to-french translations, which included the diacritical encoding process and their ffr v1.1 model, trained on the dataset. in their 2020 paper titled "neural machine translation for extremely low-resource african languages: a case study on bambara," (tapo et al., 2020) introduced the pioneering parallel dataset for machine translation of bambara to and from english and french. this dataset has served as a significant milestone as it has provided the foundation for benchmarking machine translation results involving the bambara language. the authors extensively address the unique challenges encountered when working with low-resource languages and propose effective strategies to overcome the scarcity of data in low-resource machine translation. their research sheds light on the potential solutions for improving machine translation in similar linguistic contexts. by tackling the data scarcity issue, (tapo et al., 2020)'s work contributes to the advancement of machine translation for under-resourced languages. @xcite have created a new african news corpus covering 16 languages, including eight that were not part of any existing evaluation dataset. they demonstrated that fine-tuning large pre-trained models with small amounts of high-quality translation data is the most effective strategy for transferring to additional languages and domains. @xcite , in their paper "participatory translations of oshiwambo", built a resource for language technology development and culture preservation, as well as providing socio-economic opportunities through language preservation. they created a diverse corpus of data spanning topics of cultural importance in the oshindonga dialect, translated to english, which is the largest parallel corpus for oshiwambo to-date @xcite . other works have also been conducted on african languages, and many of them have websites for data crawling, such as jw300 and bbc. however, there is currently no research related to the ngambay language or any other local language in chad, and it is difficult to find websites related to these languages, such as newspapers or other sources, such as jw300. ## ngambay lewis, simons, and fennig (2013) reported 896,000 ngambay speakers in chad and 57,000 in cameroon (wikipedia). according to @xcite j.h. greenberg's classification in the languages of africa places ngambay in the nilo-saharan family, chari-nil subfamily, central sudanese group, and bongo-baguinnian subgroup. tucker and bryan classify ngambay as bongo-baguinnian, sara group. lakka and mouroum, closely related to ngambay, share a fair amount of homogeneity, though they differ in vocabulary and pronunciation. @xcite states that ngambay is related to western saras, kaba, and laka. ngambay is spoken in eastern logone, tandjile, moyen-chari, mayo-kebbi, and chari-baguirmi prefectures. it is used as a lingua franca by other ethnic groups. in 1993, 812,003 ngambay lived in chad, with at least half in logone occidental. the ngambay people call their language "tàr ngàmbáí" or "tà ngàmbáí". protestant priests and missionaries helped many ngambay speakers learn to read and write. they translated the new testament and bible into ngambay, titled "testament ge cigi" and "maktub ge to qe kemee" respectively. ngambay hymns include "pa kula ronduba do mbaidombaije'g". it is worth noting that a monthly evangelical magazine called dannasur was published for several decades until its discontinuation in 1995, or possibly more recently. however, it is regrettable that the transcription of ngambay has not taken into account its distinctive feature of tones. several studies have already been conducted on this language, including the work of charles vandame (archbishop of n'djamena before) titled the ngambay-moundou, which was published in 1963 @xcite . ## problem of education the economic difficulties of recent years have had a significant impact on the education sector of chad, leading to stagnation or even a decline in the quality and effectiveness of the education system. school infrastructure has deteriorated rapidly, and there is a lack of motivated and qualified staff, with illiteracy remaining prevalent and gender disparities showing no signs of improvement. although the primary school enrollment rate is relatively high at 86.85%, only 41.32% of students complete primary school. when compared with niger, a neighbouring african country facing similar challenges, the data is disappointing, with niger having a primary school enrollment rate of 73.43% and nearly 72% of students completing primary school. a recent sectoral analysis of the chadian education system highlights several deficiencies, including low enrollment rates, a lack of textbooks and inadequate classroom equipment, unqualified teachers, and limited access to higher education. therefore, several changes are necessary to improve education in chad. the paqepp (projet d'amélioration de la qualité de l'éducation par une gestion de proximité) project, funded by the french development agency, aims to address these issues, involving 50 schools in moundou and n'djamena. the project was scheduled to run for four years, from 2017 to 2021, and involved more than 700 teachers and nearly 55,000 students. however, due to the global health crisis (covid-19), the project has been extended until 2023. one possible solution to address such problems is the development of efficient machine translation models that can be deployed on edge devices to help overcome language barriers, as many people face difficulties in accessing education. creating high-quality datasets for research in nmt is crucial for building these models. ## data creation in data creation, we utilized two sources. the first source was the sara bagirmi languages project which provided us with the fifth edition (2015) of the ngambaye to french dictionary in pdf format. however, due to the complexity of performing web scraping on a pdf, we manually created a parallel corpus of 1,176 sentences with short to medium lengths from the most commonly used sentences in daily life using a google form. the second source was youversion bible, an online bible translated into multiple languages, including ngambay. using r programming, we performed web scraping on the website, but the ngambay translation did not include all the verses like the french version. we extracted up to 34,647 sentences, but there were various grammatical errors, incorrect and incomplete translations, and inconsistencies. to ensure the quality of the data after crawling, we gave the dataset to native speakers of ngambay and other linguists, including the association of people translating the bible from french to ngambay in chad, to check for problematic translations, misspellings, and duplicated sentences following @xcite . after quality control, we combined the two bitext datasets, dropped inconsistent and incomplete translations, and ended up with 33,073 sentences for use in this project. the morphological characteristics of a language can have a significant impact on its sentence structure and complexity. our analysis revealed that the ngambay language has a relatively simple morphology compared to french, which contributes to shorter sentences and fewer words. in contrast, french has a highly inflected morphology, resulting in longer and more complex sentences with a larger vocabulary. these differences in morphology pose a challenge for machine translation systems, as they must be trained on parallel texts that are aligned at the sentence and word levels. given the complexity of french and the simplicity of ngambay, it is essential to develop effective strategies for handling the morphological variations in each language when building mt models. by understanding the unique features of each language, we can improve the accuracy and effectiveness of mt systems for languages with varying levels of complexity. ## models and methods we have used three transformer-based language models in our experiments: mt5 @xcite , byt5 @xcite , and m2m100 @xcite . transformers are a type of neural network architecture that has become popular in nlp since 2017 @xcite . they are used in many cutting-edge nlp applications. unlike rnns, transformers use a self-attention mechanism to weigh input sequence importance when making predictions. the transformer architecture consists of an encoder and decoder, which can be trained for nlp tasks such as machine translation, text classification, and language modelling. the encoder produces hidden representations from the input sequence, and the decoder uses them to generate the output sequence @xcite . ## results and discussion this section will detail our training process, specifically discussing the data augmentation method we used to enhance the performance of our pre-trained models. our source language is french (fr), while the target language is ngambay (sba). our experiment aimed to identify and select the model that performed best among the pre-trained models when trained on the original bitext data, then use the selected model to generate synthetic data. of the three pre-trained models we fine-tuned, m2m100 achieved the highest evaluation bleu score of 33.06, followed by byt5 with a score of 28.447 when trained on a sample of 21,166, as shown in ## conclusion the primary objective of this study is to demonstrate the possibility of gathering data on chadian languages, similar to how other african countries do, and utilizing this data to develop a machine translation (mt) system. specifically, the aim is to establish an mt system for the ngambay language as an example for other chadian languages. by doing so, we hope to set a benchmark for the accuracy of chadian mt systems. to achieve this goal, we constructed the first bitext dataset for ngambay-french and fine-tuned three transformerbased models (m2m100, byt5, and mt5). our experimental results indicate that m2m100 outperforms the other models and that monolingual source-side can enhance the performance of all models. we believe that such mt system can be integrated into electronic devices to overcome language barriers. however, this work has limitations that future studies can address