Enhancing Translation for Indigenous Languages: Experiments with Multilingual Models (2023)

Abstract: This paper describes CIC NLP’s submission to the AmericasNLP 2023 Shared Task on machine translation systems for indigenous languages of the Americas. We present the system descriptions for three methods. We used two multilingual models, namely M2M-100 and mBART50, and one bilingual (one-to-one) — Helsinki NLP Spanish-English translation model, and experimented with different transfer learning setups. We experimented with 11 languages from America and report the setups we used as well as the results we achieved. Overall, the mBART setup was able to improve upon the baseline for three out of the eleven languages.

## introduction while machine translation systems have shown commendable performance in recent years, the performance is lagging for low-resource languages @xcite . since lowresource languages suffer from a lack of sufficient data @xcite , most models and methods that are developed for high-resource languages do not work well in lowresource settings. additionally, low-resource languages are linguistically diverse and have divergent properties from the mainstream languages in nlp studies @xcite . though low-resource languages lack sufficient data to train large models, some such languages still have a large number of native speakers @xcite . while the availability of language technologies such as machine translation systems can be helpful for such linguistic communities, they could also bring harm and exposure to exploitation @xcite . borrowing from human-computer interaction (hci) studies @xcite , we want to acknowledge our belief that low-resource language speakers should be empowered to create technologies that benefit their communities. many indigenous communi-ties have community-rooted efforts for preserving their languages and building language technologies for their communities foot_0 and we hope that methods from shared tasks like this will contribute to their efforts. improving machine translation systems for lowresource languages is an active research area and different approaches @xcite @xcite @xcite have been to improve the performance of systems geared forward low-resource languages. we participated in the americasnlp 2023 shared task in hopes of contributing new approaches for low-resource machine translation that are likely to be helpful for community members interested in developing and adapting these technologies for their languages. in recent years, large pre-trained models have been used for downstream nlp tasks, including machine translation @xcite because of the higher performance in downstream tasks compared to traditional approaches @xcite . one trend is to use these pre-trained models and fine-tune them on smaller data sets for specific tasks @xcite . this method has shown promising results in downstream nlp tasks for languages with low or limited resources @xcite . in our experiments, we used multilingual and bilingual models and employed different fine-tuning strategies for the eleven languages in the 2023 shared task @xcite . in this paper, we describe the system setups we used and the results we obtained from our experiments. one of our systems improves upon the baseline for three languages. we also reflect on the setups we experimented with but ended up not submitting in hopes that future work could improve upon them. ## languages and datasets in this section, we present the languages and datasets used in our shared task submission. ## models we experimented with two multilingual and one bilingual translation model with different transfer learning setups. we used m2m-100 and mbart50 for the multilingual experiment and the helsinki-nlp spanish-english model for the bilingual experiment. figure 1 shows the models used in this experiment. ## results we submitted three (two multilingual and one bilingual) systems, as shown in ## conclusion in this work, we present the system descriptions and results for our submission to the 2023 ameri-casnlp shared task on machine translation into indigenous languages. we used pre-trained models and tested different fine-tuning strategies for the eleven languages provided for the shared task. we used one bilingual (helsinki nlp english-spanish model) and two multilingual (m2m-100 and mbart50) models for our experiments. in addition to fine-tuning the individual languages' data, we concatenated the data from all eleven languages to create a spanish-all dataset and fine-tuned the m2m-100 model before fine-tuning for the individual languages. our mbaert50 model beat the strong baseline in three languages.