Triangular Transfer: Freezing the Pivot for Triangular Machine Translation (2022)

Abstract: Triangular machine translation is a special case of low-resource machine translation where the language pair of interest has limited parallel data, but both languages have abundant parallel data with a pivot language. Naturally, the key to triangular machine translation is the successful exploitation of such auxiliary data. In this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data. As we train auxiliary source-pivot and pivot-target translation models, we initialize some parameters of the pivot side with a pre-trained language model and freeze them to encourage both translation models to work in the same pivot language space, so that they can be smoothly transferred to the source-target translation model. Experiments show that our approach can outperform previous ones.

## introduction machine translation (mt) has achieved promising performance when large-scale parallel data is available. unfortunately, the abundance of parallel data is largely limited to english, which leads to concerns on the unfair deployment of machine translation service across languages. in turn, researchers are increasingly interested in non-english-centric machine translation approaches @xcite . triangular mt @xcite ) has the potential to alleviate some data scarcity conditions when the source and target languages both have a good amount of parallel data with a pivot language (usually english). @xcite have shown that transfer learning is an effective approach to triangular mt, surpassing generic approaches like multilingual mt. however, previous works have not fully exploited all types of auxiliary data ( in this work, we propose a transfer-learningbased approach that exploits all types of auxiliary data. during the training of auxiliary models on auxiliary data, we design parameter freezing mechanisms that encourage the models to compute the representations in the same pivot language space, so that combining parts of auxiliary models gives a reasonable starting point for finetuning on the source-target data. we verify the effectiveness of our approach with a series of experiments. ## approach we first present a preliminary approach that is a simple implementation of our basic idea, for ease of understanding. we then present an enhanced version that achieves better performance. for notation purpose, we use x, y, and z to represent source, target, and pivot language, respectively. ## conclusion in this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data, including both source-pivot and pivot-target parallel data, as well as involved monolingual data. we investigate different freezing strategies for training the auxiliary models to improve source-target translation, and achieve better performance than previous approaches.