K reol M orisien MT : A Dataset for Mauritian Creole Machine Translation (2022)

Abstract: In this paper, we describe KreolMorisienMT, a dataset for benchmarking machine translation quality of Mauritian Creole. Mauritian Creole (Kreol Morisien) is a French-based creole and a lingua franca of the Republic of Mauritius. KreolMorisienMT consists of a parallel corpus between English and Kreol Morisien, French and Kreol Morisien and a monolingual corpus for Kreol Morisien. We first give an overview of Kreol Morisien and then describe the steps taken to create the corpora. Thereafter, we benchmark Kreol Morisien ↔ English and Kreol Morisien ↔ French models leveraging pre-trained models and multilingual transfer learning. Human evaluation reveals our systems’ high translation quality.

## introduction creoles 1 are natural languages that develop from the simplifying and mixing of different languages into a new one within a fairly brief period of time. most creoles are highly related to a widely spoken language, and in this paper, we focus on mauritian creole, which is a french based creole. mauritian creole, or kreol morisien, is widely spoken in the republic of mauritius by approximately 1.2 million people. kreol morisien is an important language from the perspective of tourism because mauritius is a country well known for its tourism industry. therefore, enabling tourists and locals to easily communicate with each other should not only help the tourism industry, but also improve cultural understanding. machine translation of creoles is quite under researched, mainly due to the lack of publicly available datasets. although research has been conducted on kreol morisien translation in the past @xcite , datasets were not released publicly, making it difficult to reproduce and continue research. in this paper, we describe kreolmorisienmt, a dataset containing standardized evaluation sets for benchmarking kreol morisien↔english and kreol morisien↔french translation. we first give an overview of kreol morisien followed by the description of the dataset creation process. we then use the evaluation sets to benchmark strong neural machine translation (nmt) @xcite baselines trained using the created parallel corpora. we mainly rely on transfer learning @xcite through multilingual @xcite fine-tuning of pre-trained models based on mbart. by leveraging transfer learning, we can obtain a translation quality of about 23-25 bleu for kreol morisien-english and about 20-23 bleu for kreol morisien-french. we manually evaluate translations to better understand the impact of transfer learning. our results show that there is significant room for innovation for kreol morisien nmt and kreol morisien nlp in general. our datasets, models and human evaluation annotations are publicly available foot_0 . ## related work this paper mainly focuses on the creation of datasets for under resourced languages, specifically creoles, as well as leveraging multilingualism and transfer learning to improve translation quality. mauritius is a part of east africa, and kreol-morisienmt falls under the broad area of research focusing on african language machine translation. the masakhane foot_1 community heavily focuses on african language nlp @xcite , a heavily under resourced area. with regard to creole translation, haitian creole was the first creole language to receive substantial attention @xcite and was featured in a wmt shared task foot_2 . work on kreol morisien itself was focused on a bit later by @xcite , @xcite , and boodeea and pudaruth (2020) but unlike us, they did not release their datasets. motivated by work on cree @xcite , we decided to focus on the creation of publicly available standardized datasets for kreol morisien to/from english and french translation. on a related note, @xcite work on language models for nigerian pidgin and haitian creole. kreol morisien is a low-resource language where multilingualism @xcite and transfer learning approaches involving fine-tuning @xcite are most relevant. self-supervised pre-trained models such as mbart @xcite ) can be used, but they are not explicitly trained on kreol morisien. however, @xcite showed that mbart like pre-trained models can be useful for unseen related languages, and we explore this possibility in this paper. once strong baselines are trained, approaches such as back-translation @xcite may be used to further improve translation quality, but we do not explore this given our limited size of monolingual corpus for kreol morisien. ## kreol morisien kreol morisien is spoken in mauritius and rodrigues islands, and a variant is also spoken in seychelles. mauritius was colonized successively by the dutch, french and british. although the british took over the island from the french in the early 1800, french remained as a dominant language and as such kreol morisien shares many features with french. ## kreolmorisienmt kreolmorisienmt is a mixed-domain dataset which was either created by manual translation of parts of kreol morisien and english books or by manual alignment of content in books that were already translated. ## experiments we describe the experimental settings including datasets used, training details, and models. ## results fine-tuning: both unidirectional and multilingual fine-tuning of the french↔english model trained on the un corpus as well as the mbart-50 models leads to large improvements @xmath0. especially, the performance of fine-tuning the mbart-50 models is impressive. mbart-50's vocabulary does not explicitly cover kreol morisien, but models finetuned on them still are comparable to or even outperform the french↔english model, which does. this shows the impressive power of massively multilingual models. denoising vs translation pre-training: comparing the results of fine-tuning on the mbart-50 denoising model (mb-d) and its many-tomany translation version (mb-t) as well as the french↔english model (fr↔en), we can see that in the absence of kreol morisien monolingual corpora for denoising pre-training, it is better to consider translation models for fine-tuning. however, denoising models perform reasonably well. ## conclusion we have presented kreolmorisienmt, a dataset for machine translation between mauritian creole (kreol morisien) to/from english and french. our datasets contain dictionary and sentence pairs belonging to a mix of domains and their sizes range from roughly 17,000 to 23,000 pairs. we also provide a monolingual corpus for kreol morisien containing about 45,000 sentences. we conduct translation experiments using kreolmorisienmt in conjunction with large english-french corpora and mbart-50 pre-trained models, leading to improvements of up to 15 bleu, despite most of the training data being dictionary pairs. adequacy and fluency based human evaluation indicates high translation quality, despite bleu scores being in the range of 20 to 25, indicating the need for better metrics. in the future, we plan to expand kreol-morisienmt with additional data as well as on additional generation tasks for kreol morisien. the kreol morisien monolingual corpus will be used in the future to extend pre-trained denoising models via light-weight adapter pre-training @xcite . study in african languages. in findings of the association for computational linguistics: emnlp 2020, pages 2144-2160, online. association for computational linguistics.