The REPU CS ’ S panish– Q uechua Submission to the A mericas NLP 2021 Shared Task on Open Machine Translation (2021)

Abstract: We present the submission of REPUcs to the AmericasNLP machine translation shared task for the low resource language pair Spanish–Quechua. Our neural machine translation system ranked first in Track two (development set not used for training) and third in Track one (training includes development data). Our contribution is focused on: (i) the collection of new parallel data from different web sources (poems, lyrics, lexicons, handbooks), and (ii) using large Spanish–English data for pre-training and then fine-tuning the Spanish–Quechua system. This paper describes the new parallel corpora and our approach in detail.

## introduction repucs participated in the americasnlp 2021 machine translation shared task @xcite for the spanish-quechua language pair. quechua is one of the most spoken languages in south america @xcite , with several variants, and for this competition, the target language is southern quechua. a disadvantage of working with indigenous languages is that there are few documents per language from which to extract parallel or even monolingual corpora. additionally, most of these languages are traditionally oral, which is the case of quechua. in order to compensate the lack of data we first obtain a collection of new parallel corpora to augment the available data for the shared task. in addition, we propose to use transfer learning @xcite using large spanish-english data in a neural machine translation (nmt) model. to boost the performance of our transfer learning approach, we follow the work of @xcite , which demonstrated that sharing the source language and a vocabulary of subword 1 "research experience for peruvian undergraduates -computer science" is a program that connects peruvian students with researchers worldwide. the author was part of the 2021 cohort: https://www.repuprogram.org/repu-cs . ## @xmath0 quechua is the most widespread language family in south america, with more than 6 millions speakers and several variants. for the americasnlp shared task, the development and test sets were prepared using the standard southern quechua writing system, which is based on the quechua ayacucho (quy) variant (for simplification, we will refer to it as quechua for the rest of the paper). this is an official language in peru, and according to @xcite it is labelled as endangered. quechua is essentially a spoken language so there is a lack of written materials. moreover, it is a polysynthetic language, meaning that it usually express large amount of information using several morphemes in a single word. hence, subword segmentation methods will have to minimise the problem of addressing "rare words" for an nmt system. to the best of our knowledge, @xcite is one of the few studies that employed a sequence-to-sequence nmt model for southern quechua, and they focused on transfer learning with finnish, an agglutinative language similar to quechua. likewise, huarcaya taquiri (2020) used the jehovah witnesses dataset @xcite , together with additional lexicon data, to train an nmt model that reached up to 39 bleu points on quechua. however, the results in both cases were high because the development and test set are split from the same distribution (domain) as the training set. on the other hand, ortega and pillaipakkamnatt (2018) improved alignments for quechua by using finnish(an agglutinative language) as the pivot language. the corpus source is the parallel treebank of rios et al. @xcite ., so we deduce that they worked with quechua cuzco (quz). @xcite in the americasnlp shared task, new out-of-domain evaluation sets were released, and there were two tracks: using or not the validation set for training the final submission. we addressed both tracks by collecting more data and pre-training the nmt model with large spanish-english data. ## data and pre-processing in this competition we are going to use the ameri-casnlp shared task datasets and new corpora extracted from documents and websites in quechua. ## results and discussion the results from the transfer learning models and the baseline are shown in from input (es) el control de armas probablemente no es popular en texas. input (en) weapon control is probably not popular in texas. reference (quy) texaspiqa sutillapas arma controlayqa manachusmi hinachu apakun output texas llaqtapi armakuna controlayqa manam runakunapa runachu for instance, the subwords "arma", "mana", among others, have been correctly translated but are not grouped in the same words as in the reference. in addition, only the word "controlayqa" is translated correctly, which would explain the low results in bleu. decoding an agglutinative language is a very difficult task, and the low bleu scores cannot suggest a translation with proper adequacy and/or fluency (as we can also observe this from the example). nevertheless, bleu works at word-level so other character-level metrics should be considered to inspect agglutinative languages. this would be the case of chrf @xcite were there is an increase of around 3% when using the americasnlp altogether with the new extracted corpora. translations using the transfer learning model trained with all available quechua datasets were submitted for track 2 (development set not used for training). for the submission of track 1 (development set used for training) we retrained the best transfer learning model adding the validation to the training for 40 epochs. the official results of the competition are shown in ## conclusion in this paper, we focused on extracting new datasets for spanish-quechua, which helped to improve the performance of our model. moreover, we found that using transfer learning was beneficial to the results even without the additional data. by combining the new corpora in the fine-tuning step, we managed to obtain the first place on track 2 and the third place on track 1 of the americasnlp shared task. due to time constrains, the quechua cusco data was not used, but it can be beneficial for further work. in general, we found that the translating quechua is a challenging task for two reasons. firstly, there is a lack of data for all the variants of quechua, and the available documents are hard to extract. in this research, all the new datasets were extracted and aligned mostly manually. secondly, the agglutinative nature of quechua motivates more research about effective subword segmentation methods.