P lay G round Low Resource Machine Translation System for the 2023 A mericas NLP Shared Task (2023)

Abstract: This paper presents PlayGround’s submission to the AmericasNLP 2023 shared task on machine translation (MT) into indigenous languages. We finetuned NLLB-600M, a multilingual MT model pre-trained on Flores-200, on 10 low-resource language directions and examined the effectiveness of weight averaging and back translation. Our experiments showed that weight averaging, on average, led to a 0.0169 improvement in the ChrF++ score. Additionally, we found that back translation resulted in a 0.008 improvement in the ChrF++ score.

## introduction we participated in the americasnlp 2023 @xcite shared task with the goal of advancing previous studies @xcite on indigenous american languages. the task is to translate spanish into 10 indigenous languages, including ashaninka, aymara, bribri, guarani, hñähñu, nahuatl, quechua, raramuri, shipibo-konibo, and wixarika. additionally, there was another language, chatino foot_0 , for which we did not participate in. we started with the monolingual and bilingual data from @xcite and finetuned nllb-600m, a multilingual pre-trained mt model from meta's no language left behind (nllb) project @xcite both bilingually and multilingually. on top of that, we employed weight averaging and back translation. for back translation, we additionally filtered the back translated sentence pairs to improve the data quality. we demonstrate that training on model weights averaged from multiple checkpoints improves translation quality, as indicated by a 0.0169 increase in the chrf++ score on average, without requiring additional computation resources. additionally, we found that back translation can enhance translation quality for low-resource languages, although it is sensitive to the quality of synthetic data. to address this, we introduced a data filtering technique to improve the quality of synthetic data. with filtered back translation, our system achieved an average improvement of 0.008 in the chrf++ score. furthermore, our study reveals that multilingual finetuning achieves comparable translation quality to bilingual fine-tuning for low-resource languages. we selected the bilingual model with weight averaging and back translation as our final submission. the implementation of this study is available in our git repository foot_2 . ## results the results are presented in previous study @xcite has primarily focused on fine-tuning bilingual machine translation models. however, the results from our multi++ and bi++ models demonstrate the promising potential of multilingual fine-tuning @xcite . on average, the chrf++ score for multi++ is only 0.0012 lower than that of bi++. we also compared the effectiveness of weight averaging and back translation. weight averaging improved translations for all target languages. on average, multi+ achieved a chrf++ score that was 0.0169 higher than multi. these results indicate that our simple technique can enhance low-resource machine translation without requiring additional computational resources. however, the impact of back translation varied across languages, as observed in the results for multi+ and multi++. on average, the implementation of back translation resulted in a 0.008 im-provement in the chrf++ metric. for wixarika and aymara, there was a slight drop in the chrf++ scores after back translation. despite performing data filtering, the quality of synthetic data largely depends on the performance of the x-to-spanish model. in summary, our fine-tuning technique has shown improvements in performance. however, with further refinements and design enhancements, there is potential for our model to achieve higher levels of performance. ## conclusion in this paper, we presented our submission to the americasnlp 2023 shared task. our system utilized the nllb-600m pre-trained model to translate spanish into 10 indigenous languages. we also investigated the potential of multilingual translation models, which showed promising results. additionally, we found that averaging model weights from previous epochs proved to be an efficient and effective approach. while back translation demonstrated performance improvements, further methods are necessary to address noisy data. these findings highlight the positive outcomes of our study and provide valuable insights for future advancements in low-resource machine translation techniques.