Multilingual Neural Machine Translation: Case-study for C atalan, S panish and P ortuguese R omance Languages (2020)

Abstract: In this paper, we describe the TALP-UPC participation in the WMT Similar Language Translation task between Catalan, Spanish, and Portuguese, all of them, Romance languages. We made use of different techniques to improve the translation between these languages. The multilingual shared encoder/decoder has been used for all of them. Additionally, we applied back-translation to take advantage of the monolingual data. Finally, we have applied fine-tuning to improve the in-domain data. Each of these techniques brings improvements over the previous one. In the official evaluation, our system was ranked 1st in the Portuguese-to-Spanish direction, 2nd in the opposite direction, and 3rd in the Catalan-Spanish pair.

## introduction research in the field of machine translation (mt) has been growing during these last years. from statistical approaches @xcite to neural ones @xcite , the progress has been impressive. even after having achieved exceptional results based only on attention mechanisms @xcite , there are still many challenges and improvements remaining, for instance, multilingual translation from languages other than english, which have lower resources, and domain adaptation. in order to tackle these challenges, the similar language task organized in the context of the conference on machine translation (wmt 2020) has provided an appropriate setting for them. within this task, the focus is the translation between languages that are different from english, and more specifically, the focus consists of translating languages that are from the same family. the families included are the following: south-slavic, indo-aryan, and romance. in our case, we have devoted the research to romance languages, which include spanish, portuguese, and catalan. the evaluation comprised all translation directions, but only provided parallel training data for spanish-portuguese and spanish-catalan. we approached the portuguese-catalan pair both from a pivot-based and zero-shot perspective. in this paper, we make use of the well-known multilingual shared encoder/decoder and we show its effectiveness when applied to languages of the same linguistic family. additionally, we benefited from back-translation and fine-tuning. ## background in this section, we show an overview of neuralbased multilingual machine translation and domain adaptation using fine-tuning. ## experimental framework in this section, we describe the datasets used for the task, the data preprocessing, the training, and the evaluation of the bilingual and multilingual systems. ## results the results show the improvements obtained by applying multilinguality, back-translation, and finetuning techniques. for the pair catalan-portuguese (ca-pt), in which there was no training data available. we have used the cascade technique, which consists of concatenating the translation of catalan-to-spanish and spanish-to-portuguese systems, and the other way around for the opposite direction. also, we have used the multilingual system to obtain zero-shot translation for this pair. ## discussion we will now discuss the results obtained for each system we have trained, comparing one against the others. ## conclusion we have observed how using a multilingual shared encoder/decoder in languages from the same family improves bilingual translation. this is due to a positive transfer among these languages while sharing vocabulary and embeddings. additionally, this multilingual shared system has been improved with both back-translation and fine-tuning methods.