Pivot Translation for Zero-resource Language Pairs Based on a Multilingual Pretrained Model (2023)

Abstract: A multilingual translation model enables a single model to handle multiple languages. However, the translation qualities of unlearned language pairs (i.e., zero-shot translation qualities) are still poor. By contrast, pivot translation translates source texts into target ones via a pivot language such as English, thus enabling machine translation without parallel texts between the source and target languages. In this paper, we perform pivot translation using a multilingual model and compare it with direct translation. We improve the translation quality without using parallel texts of direct translation by fine-tuning the model with machine-translated pseudo-translations. We also discuss what type of parallel texts are suitable for effectively improving the translation quality in multilingual pivot translation.

## introduction multilingual neural network models are models in which multiple languages are learned in a single model, and are useful for machine translation and cross-lingual language processing. multilingual models utilize resources of similar languages (e.g., those in the same language family) and thus provide a relatively high degree of accuracy for even low-resource languages. 1 machine translation is performed according to a combination of a source language and a target language, and therefore, language-specific models require a model for each possible combination of languages. by contrast, a multilingual model can handle all combinations of source and target languages and is therefore easier to manage. the potential usefulness of the multilingual model has led to the development of several encoder-decoder models pretrained using parallel corpora. for example, a multilingual translation model pretrained with the opus-100 corpus @xcite foot_1 has been developed. this is a multilingual model that translates between english and any of 100 languages (i.e., an english-centric model). the m2m-100 model @xcite ## multilingual pretrained model used in this study for this study, we newly trained an english-centric model to focus on translating zero-resource language pairs. we call this the "basic model." specifically, this model corresponds to the 103 languages covered by the cc-100 corpus @xcite , and the opus-100 corpus @xcite or the ccaligned v1 corpus @xcite . cc-100 is a monolingual corpus, and opus-100 and ccaligned are parallel corpora. all corpora are based on web crawl data. ## translation experiments in this study, we conducted translation experiments between japanese (ja) and languages other than english (xx). ## conclusions using the pivot translation, we can translate texts even for zero-resource language pairs. moreover, we can improve the translation quality without changing the zero-resource condition because we can generate pseudo-parallel corpora from monolingual corpora. in this study, we applied pivot translation to zero-resource language pairs using a multilingual pretrained model. the answers to the questions studied in this work are summarized as follows. a1 comparing pivot translation with direct translation, the quality of pivot translation is higher than that of direct translation when the parallel corpus size is very small. when the corpus size is large, the quality of direct translation increases. if we can acquire a corpus of medium size, we should prepare a direct parallel corpus to improve the translation quality. a2 comparing the first and second stages of pivot translation, it is better to enhance the second stage to improve quality. a3 it is possible to improve translation quality using pseudo-translations generated by pivot translation. a4 when generating pseudo-translations, it is better to generate them from monolingual corpora of the target language. the fact that zero-resource language pairs can be translated is helpful when we extend our machine translation to new languages. for example, we can check the quality of a newly created parallel corpus by back-translation, or we can post-edit pseudo-translations to create direct parallel corpora. we plan to extend multilinguality while appropriately using direct and pivot translation.