Feriji: A F rench- Z arma Parallel Corpus, Glossary & Translator (2024)

Abstract: Machine translation (MT) is a rapidly expanding field that has experienced significant advancements in recent years with the development of models capable of translating multiple languages with remarkable accuracy. However, the representation of African languages in this field still needs improvement due to linguistic complexities and limited resources. This applies to the Zarma language, a dialect of Songhay (of the Nilo-Saharan language family) spoken by over 5 million people across Niger and neighboring countries (Lewis et al., 2016). This paper introduces Feriji, the first robust French-Zarma parallel corpus and glossary designed for MT. The corpus, containing 61,085 sentences in Zarma and 42,789 in French, and a glossary of 4,062 words represents a significant step in addressing the need for more resources for Zarma. We fine-tune three large language models on our dataset, obtaining a BLEU score of 30.06 on the best-performing model. We further evaluate the models on human judgments of fluency, comprehension, and readability and the importance and impact of the corpus and models. Our contributions help to bridge a significant language gap and promote an essential and overlooked indigenous African language.

## introduction the field of mt has witnessed substantial progress, particularly with the development of sophisticated models capable of accurately translating multiple languages. these models sometimes even get closer to human proficiency @xcite . however, despite these advances, african languages still need representation in mt systems, primarily due to linguistic complexities and limited resources @xcite . one such under-represented language is zarma, spoken by over 5 million people, predominantly in niger @xcite . as a member of the songhay family within the nilo-saharan language group, zarma has received limited attention in natural language processing research. this lack of representation restricts zarma speakers' access to technology and hinders efforts to preserve and promote zarma. to address this challenge, we introduce feriji-the first parallel french-zarma corpus and glossary designed specifically for mt tasks. the corpus contains 61,085 sentences in zarma and 42,789 in french, representing a significant step towards enriching mt resources for the zarma language. the development of feriji involved extensive collection, alignment, and cleaning of texts, resulting in a resource that not only bridges a significant linguistic gap but also promotes the use of zarma in research contexts. we chose french as the source language because niger is a french-speaking country, and most information and resources are readily available in french rather than any other language. this makes french a practical choice for creating a resource that can effectively support the zarma-speaking community. this paper details the creation process of feriji, structure, and potential value for mt research, particularly for the zarma language. by providing this resource, we aim to facilitate further research in this area and enhance the integration of zarma into the global mt field. ## literature review advances in mt have been a significant focus within natural language processing (nlp). in recent years, we have seen the rise of neural machine translation (nmt) models capable of producing translations that approach-or even surpasshuman proficiency in many languages. models such as facebook's m2m-100 @xcite @xcite have revolutionized multilingual translation with their accuracy. however, the representation of african languages in mt remains a significant challenge, as highlighted in several studies @xcite . 1 african languages, numbering approximately 3,000, are diverse and complex, characterized by unique tonal nuances and dialects @xcite . representing these languages in mt systems is a substantial task, requiring extensive resources and expert input. the under-representation of african languages in mt systems is particularly concerning, given the literacy rates in sub-saharan africa. as of 2020, the literacy rate stood at 67.27%, while in niger, it is at 80.9% as of 2023 @xcite . this data indicates that a significant portion of the population relies on native languages for communication, unlike in regions with higher literacy rates. the comparatively high illiteracy rates further highlight the importance of including these native languages in initiatives through translation systems. efforts to address the under-representation of african languages include initiatives like the masakhane project, which focuses on strengthening nmt for african languages @xcite ; the aya model @xcite , a multi-task model covering 101 languages (over 50% of which are low-resource); and facebook's no language left behind (nllb) project @xcite , which aims to enable translation into over 60 african languages. unfortunately, no specific initiative has targeted zarma or any songhay language, leaving them largely unexplored in the mt field. this literature review highlights the importance of our work in contributing to the diversification of language resources in mt, particularly for lowresource languages such as zarma. ## feriji-based machine translation to evaluate the effectiveness of feriji, we finetuned three state-of-the-art language models on the french-to-zarma translation task: mt5-small @xcite , m2m100, and nllb-200-distilled-600m . we used a p100 gpu in the kaggle environment for training. ## human evaluation since the bleu metric alone cannot fully assess performance in our case, we conducted a human evaluation experiment to assess the quality of the translations produced by the nllb-200-dist and m2m100 models. we recruited five native zarma speakers to participate in the evaluation. each participant received a set of 100 sentences that both models had translated. participants rated each translation on a scale of 1 to 5 for fluency, accuracy, and readability, with 5 being the highest score: ## feriji translator the feriji translator (ft)-french to zarma translator-is a crucial component of the feriji project. it provides a means for non-native speakers to explore the zarma language and for zarma speakers to access textual resources available in french but not in zarma. we chose the m2m100 model for ft because it achieved the highest bleu score and performed well in our human evaluation, as shown in tables 3 and 5 . figure 2 shows the interface of the ft. ## community engagement and feedback following the release of the ft and its associated model pipeline, we surveyed to gather feedback from the zarma community about the feriji project. we selected 104 representative zarma speakers, including both native and non-native speakers. the demographics of the survey participants are illustrated in figures 3 and 4 . the survey results are summarized in section 7.1. in addition to the survey responses, the community raised concerns about two key areas: the fluency of the translations and the accessibility of the tool for illiterate zarma speakers. ## areas of application the feriji translator has potential applications that can benefit the zarma-speaking community. this section highlights some key areas where ft can be effectively used. ## ethical considerations the development and implementation of mt systems like feriji raise several ethical considerations that require careful attention. one primary concern is the potential for cultural insensitivity or misrepresentation, especially when working with languages deeply intertwined with cultural identities, such as zarma. as highlighted by @xcite , mt systems can inadvertently perpetuate stereotypes or misinterpret cultural nuances. this can significantly impact the perception and understanding of a language and its speakers. to mitigate this risk, we engaged closely with native zarma speakers and cultural experts throughout the development process to ensure that feriji is culturally sensitive and respectful. another critical aspect is data privacy and consent, mainly when sourcing texts from the community or online platforms. ## conclusion this paper introduced feriji, the first parallel french-zarma corpus and glossary designed for machine translation. feriji significantly contributes to the field by addressing the lack of resources for zarma, a language spoken by over 5 million people in niger and neighboring countries. feriji will be a valuable resource for researchers and developers working on zarma mt. we anticipate that feriji will contribute to the promotion of the zarma language and make it more accessible to people around the world.