Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training (2021)

Abstract: Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model’s uncertainty on a small set of trusted clean data for multi-corpus machine translation. We experiments with two classes of uncertainty measures on multilingual (16 languages with 4 settings) and multi-domain settings (4 for in-domain and 2 for out-of-domain on English-German translation) and demonstrate our approach MultiUAT substantially outperforms its baselines, including both static and dynamic strategies. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.

## introduction text corpora are commonly collected from several different sources in different languages, raising the problem of learning a nlp system from the heterogeneous corpora, such as multilingual models @xcite @xcite @xcite and multi-domain models @xcite @xcite . a strong demand is to deploy a unified model for all the languages and domains, because a unified model is much more resource-efficient, and knowledge learned from high-resource languages/domains (hrls/hrds) can be transferred to low-resource languages/domains (lrls/lrds). one common issue on training models across corpora is that data from a variety of corpora are both heterogeneous (different corpora reveal different linguistic properties) and imbalance (the accessibility of training data varies across corpora). the standard practice to address this issue is to adjust the training data distribution heuristically by up-sampling the training data from lrls/lrds @xcite . @xcite rescale the training data distribution with a heuristic temperature term and demonstrate that the ideal temperature can substantially improve the overall performance. however, the optimal value for such heuristics is both hard to find and varies from one experimental setting to another @xcite . @xcite and @xcite hypothesize that the training data instances that are similar to the validation set can be more beneficial to the evaluation performance and propose a general reinforcement-learning framework differentiable data selection (dds) that automatically adjusts the importance of data points, whose reward is the cosine similarity of the gradients between a small set of trusted clean data and training data. they instantiate this framework on multilingual nmt, known as multidds, to dynamically weigh the importance of language pairs. both the hypothesis and the proposed approach rely on the assumption that knowledge learned from one corpus can always be beneficial to the other corpora. however, their assumption does not always hold. if the knowledge learned from one corpus is not able to be transferred easily or is useless to the other corpora, this approach fails. unlike cosine similarity, model uncertainty is free from the aforementioned assumption on cross-corpus transfer. from a bayesian viewpoint, the model parameters can be considered as a random variable that describes the dataset. if one dataset is well-described by the model parameters, its corresponding model uncertainty is low, and vice versa. this nature makes the model uncertainty an ideal option to weigh the datasets. in this work, we propose an approach mul-tiuat that leverages the model uncertainty as the reward to dynamically adjust the sampling probability distribution over multiple corpora. we consider the model parameter as a random variable that describes the multiple training corpora. if one corpus is well-described by the model compared with other corpora, we spare more training efforts to the other poorly described corpora. we conduct extensive experiments on multilingual nmt (16 languages with 4 settings) and multi-domain nmt (4 for indomain and 2 for out-of-domain), comparing our approach with heuristic static strategy and dynamic strategy. in multilingual nmt, we improve the overall performance from +0.83 bleu score to +1.52 bleu score among 4 settings, comparing with the best baseline. in multi-domain nmt, our approach improves the in-domain overall performance by +0.58 bleu score comparing with the best baseline and achieves the second best out-ofdomain overall performance. we also empirically illustrate the vulnerability of cosine similarity as the reward in the training among multiple corpora. ## preliminaries standard nmt a standard nmt model, parameterized by θ θ θ, is commonly trained on one language pair d o @xmath0, y y y) i } m @xmath1. the objective is to minimize the negative log-likelihood of the training data with respect to θ θ θ: multi-corpus nmt both multilingual nmt and multi-domain nmt can be summarized as multicorpus nmt that aims to build a unified translation system to maximize the overall performance across all the language pairs or domains. formally, let us assume we are given a number of datasets d @xmath2, in which d j @xmath3, y y y heuristic strategy for multi-corpus training in practice, equation 2 can be reviewed as training using mini-batch sampling according to the proportion of these corpora, however, this simple training method does not work well in real cases, where low-resource tasks are under-trained. a heuristic static strategy is to adjust the proportion exponentiated by a temperature term τ @xcite : and the loss function for multi-corpus training can be re-formulated as: specifically, τ = 1 or τ = ∞ is equivalent to proportional (equation 2 ) or uniform sampling respectively. differentiable data selection (dds) @xcite propose a general framework that automatically weighs training instances to improve the performance while relying on an independent set of held-out data d dev . their framework consists of two major components, the model θ θ θ and the scorer network ψ ψ ψ. the scorer network ψ ψ ψ is trained to assign a sampling probability to each training instance, denoted as p ψ ψ ψ (x x x, y y y), based on its contribution to the validation performance. the training instance that contributes more to the validation performance is assigned a higher probability and more likely to be used for updating the model θ θ θ. this strategy aims to maximize the overall performance over d dev and is expected to generalize well on the unseen d tst with the assumption of the independence and identical distribution between d dev and d tst . therefore, the objective can formulated as: and ψ ψ ψ and θ θ θ are updated iteratively using bilevel optimization @xcite . ## methodology in this work, we leverage the idea of dds under the multi-corpus scenarios. we utilize a differentiable domain/language scorer ψ ψ ψ to weigh the training corpora. to learn ψ ψ ψ, we exploit the model uncertainty to measure the model's ability over the target corpus. below, we elaborate on the details of our method. ## main results the summarized results for both multilingual and multi-domain nmt are presented in multilingual nmt overall, dynamic strategies (multidds-s and multiuat) demonstrate their superiority against heuristic static strategies. as shown in multi-domain nmt multiuat outperforms all its baselines on in-domain evaluation and achieves the second best performance on out-ofdomain evaluation. multiuat with pretp achieves the optimal balance on in-domain evaluation and the one with exptp achieves the second best performance on out-of-domain evaluation. however, multidds-s performs poorly on multi13 we report the multilingual results of the heuristic approach and multidds-s from @xcite in based on the detailed results in appendix a, we can observe that the higher sampling probability for certain domain is commonly but not always positively correlated to the corresponding indomain performance. uniformly sampling minibatches from domains does not result in the best performance on lrds, because the lrds with too much up-sampling are not able to fully leverage the knowledge from the hrds. ## analysis wang et al. (2020b) conduct exhaustive analyses on multilingual nmt and most of our observations are consistent with theirs. 14 hence, we focus more on analyzing the results on multi-domain nmt. ## related work multi-corpus nlp multilingual training has been particularly prominent in recent advances driven by the demand of training a unified model for all the languages @xcite @xcite . freitag and firat (2020) extend current english-centric training to a many-to-many setup without sacrificing the performance on englishcentric language pairs. wang et al. (2021) improve the multilingual training by adjusting gradient directions based on gradient similarity. existing works on multi-domain training commonly attempt to leverage architectural domain-specific components or auxiliary loss @xcite @xcite @xcite . these approaches commonly do not explore much on the training proportion across domains and are limited to in-domain prediction and less generalizable to unseen domains. @xcite dynamically balance the importance of tasks in multitask nmt to improve the low-resource nmt performance. @xcite leverage a pre-trained language model to select useful monolingual data from either source language or target language to perform unsupervised domain adaptation for nmt models. our work is directly related to @xcite and @xcite that leverage cosine similarity of gradients as a reward to dynamically adjust the data usage in the multilingual training. model uncertainty estimating the sequence and word-level uncertainty via monta carlo dropout @xcite has been investigated for nmt @xcite @xcite . @xcite exploit model uncertainty on back-translation to reduce the noise in the backtranslated corpus. @xcite and @xcite investigate to leverage model uncertainty to detect out-of-distribution translations. @xcite summarize several measures to estimate quality of translated sentences, including the model uncertainty. our work exploits the uncertainty measures as suggested by @xcite and @xcite . ## conclusion in this work, we propose multiuat, a general model-agnostic framework that learns to automatically balance the data usage to achieve better overall performance on multiple corpora based on model uncertainty. we run extensive experiments on both multilingual and multi-domain nmt, and empirically demonstrate the effectiveness of our approach. our approach substantially outperforms other baseline approaches. we empirically point out the vulnerability of a comparable approach multidds-s @xcite . we focus on the problem that dynamically balances text corpora collected from heterogeneous sources in this paper. however, the heterogeneity of text corpora is far beyond the languages and domains which are discussed in this work. for example, the quality of datasets is not covered. we leave the study on the quality of datasets to the future work.