Towards Higher P areto Frontier in Multilingual Machine Translation (2023)

Abstract: Multilingual neural machine translation has witnessed remarkable progress in recent years. However, the long-tailed distribution of multilingual corpora poses a challenge of Pareto optimization, i.e., optimizing for some languages may come at the cost of degrading the performance of others. Existing balancing training strategies are equivalent to a series of Pareto optimal solutions, which trade off on a Pareto frontierIn Pareto optimization, Pareto optimal solutions refer to solutions in which none of the objectives can be improved without sacrificing at least one of the other objectives. The set of all Pareto optimal solutions forms a Pareto frontier..In this work, we propose a new training framework, Pareto Mutual Distillation (Pareto-MD), towards pushing the Pareto frontier outwards rather than making trade-offs. Specifically, Pareto-MD collaboratively trains two Pareto optimal solutions that favor different languages and allows them to learn from the strengths of each other via knowledge distillation. Furthermore, we introduce a novel strategy to enable stronger communication between Pareto optimal solutions and broaden the applicability of our approach. Experimental results on the widely-used WMT and TED datasets show that our method significantly pushes the Pareto frontier and outperforms baselines by up to +2.46 BLEUOur code will be released upon acceptance..

## introduction multilingual neural machine translation (mnmt) is a popular paradigm that uses a unified model to handle the entire translation process for multiple language pairs @xcite @xcite . this paradigm is particularly effective at improving the performance of lowresource languages through transfer learning @xcite @xcite . besides, mnmt is highly deployable since only one model is required @xcite @xcite . however, the severely imbalanced distribution of multilingual training data puts the mnmt in 1 in pareto optimization, pareto optimal solutions refer to solutions in which none of the objectives can be improved without sacrificing at least one of the other objectives. the set a situation of pareto optimization (also known as multi-objective optimization). that is, when some languages are optimized, others degenerate. existing methods can be considered a set of pareto optimal solutions that trade off on a pareto frontier, which focus on balancing the performance across different languages by adjusting the sampling distribution @xcite @xcite . the widely-used temperature-based sampling @xcite is typical evidence of the claim above, which uses a hyper-parameter to smooth the training distribution over all language pairs to enhance the representation of low-source languages (lrls) while sacrificing the which of high-resource languages (hrls). despite the emergence of several sophisof all pareto optimal solutions forms a pareto frontier. 2 our code is publicly available at https://github.com/ orangeinsouth/pareto-mutual-distillation ticated dynamic sampling technologies designed to overcome the inflexibility of temperature-based sampling, their performance remains restricted to this pareto frontier @xcite @xcite . in this work, we propose a novel training framework, named pareto mutual distillation (pareto-md), to push the pareto frontier of multilingual models. specifically, pareto-md uses different training distributions that favor dissimilar subsets of languages to train two multilingual models simultaneously. these two models learn from each other at each training step with knowledge distillation. the underlying idea of pareto-md is to address shortcomings of individual pareto optimal solutions via access to a better one in terms of that shortcoming, thereby raising the pareto frontier, as fig. 1 depicts. to fully exploit the potential of our approach in multilingual settings, we further propose automatic pareto mutual distillation, which dynamically determines the contribution of distillation learning loss on each objective. these contributions, controlled by a set of distillation weights, adapt automatically to the evolving models, eliminating the need for manual hyper-parameter search. while our method applies essentially to any multi-objective optimization problem, we specifically demonstrate its benefit on multilingual machine translation. the experimental results on two widely-used datasets demonstrate the effectiveness of our method, which improves up to +2.46 bleu, and the further analysis shows the pareto frontier is pushed outwards visibly. ## preliminaries neural machine translation (nmt) is a classic nlp task that translates a sentence x in source language into a sentence y in target language @xcite @xcite . given a parallel corpus d " tpx, yq p x ˆyu, the nmt model is commonly trained by minimizing the negative log-likelihood loss: ## pareto mutual distillation in this section, we first introduce our training framework pareto-md ( §3.1). next, two strategies that determine the important distillation weights, uni-pmd and bi-pmd, are shown ( §3.2). to overcome the flaws of these two strategies above, auto-pmd is further proposed ( §3.3). ## related work for a long time, data imbalance has been a problem hindering multilingual models from performing evenly across different languages. existing methods pursue balanced performance via designing heuristics @xcite or automatic sampling strategies @xcite @xcite 2021; @xcite . for example, @xcite design a reinforce learning based method to automatically adjust the sampling probability of each language pair towards an overall optimal solution. @xcite vary the distribution via distributional robust optimization. however, their improvement is limited since increasing the training weights of some languages leads to relative decreases in the weights of other languages, resulting in a trade-off on the pareto frontier. different from their methods, we overcome this issue by training two models collaboratively. before our work, there were two approaches also based on knowledge distillation in mnmt. @xcite use pre-defined bilingual models to teach the multilingual model via knowledge distillation. @xcite propose language-specific self-distillation to remedy the convergence inconsistency problem in mnmt using self-distillation. our pareto-md is an extension of mutual distillation on the pareto optimization problems. ## conclusion in this work, we propose a training framework pareto-md to reach a higher pareto frontier for mnmt. the core of pareto-md is the synergy between diverse pareto optimal solutions via mutual distillation. besides, we design a novel strategy for deducing distillation weights automatically, achieving better performance and getting rid of hyperparameter searching. experimental results on the wmt and ted datasets show the effectiveness of our method. even though we experiment with training two models in this work, our method can naturally apply to train more models. in the fu-ture, we are interested in exploring how to apply our pareto-md to the training of large language models @xcite .