Multi-Domain Adaptation in Neural Machine Translation with Dynamic Sampling Strategies (2022)

Abstract: Building effective Neural Machine Translation models often implies accommodating diverse sets of heterogeneous data so as to optimize performance for the domain(s) of interest. Such multi-source / multi-domain adaptation problems are typically approached through instance selection or reweighting strategies, based on a static assessment of the relevance of training instances with respect to the task at hand. In this paper, we study dynamic data selection strategies that are able to automatically re-evaluate the usefulness of data samples and to evolve a data selection policy in the course of training. Based on the results of multiple experiments, we show that such methods constitute a generic framework to automatically and effectively handle a variety of real-world situations, from multi-source domain adaptation to multi-domain learning and unsupervised domain adaptation.

## introduction a typical setting in machine translation (mt) is to collect the largest possible collection of parallel data for the chosen language pair, with the intent to achieve optimal performance for the task of interest. in such situations, the training data distribution is opportunistic, while the test data distribution is chosen and fixed; a key aspect of training is then to mitigate the detrimental effects of a mismatch between these distributions. single-source and multisource foot_0 domain adaptation (da) is a well-studied instance of this setting (see @xcite for a review), and so is multi-domain (md) learning @xcite @xcite . a related situation is multilingual mt @xcite @xcite , where the diversity of training data not only corresponds to variations in the topic, genre, or register but also in language. this problem is often approached by static instance selection or re-weighting strategies, where the available training data is used in proportion to its relevance for the testing conditions @xcite . finding the optimal balance of training data is however, a challenging task due, for instance, to the similarity between domains/languages, or to the regularization effects of out-of-domain data (miceli @xcite . a static policy may also be suboptimal when some target domains or languages are easier to train than others. finally, improving the performance of the mt system in one domain will often hurt that of another @xcite and improving model generalization across all domains @xcite may not achieve optimally for any particular domain. several recent proposals explore ways to instead consider dynamic data selection and sampling strategies: @xcite and @xcite construct a static curriculum, while @xcite and @xcite build curricula that automatically adapt to the training data. in this paper, we contribute to this line of research in several ways. ## learning with multiple data sources we conventionally define a domain d as a distribution d d (x) over some feature space x that is shared across domains @xcite : in machine translation, x is the representation space for input sentences; each domain corresponds to a specific source of data, and may differ from other data sources in terms of textual genre, thematic content @xcite , register, style @xcite , etc. translation in domain d is formalized by a translation function h d (y|x) pairing sentences in a source language with sentences in a target language @xmath0. h d is usually assumed to be deterministic (hence @xmath1. it is usual in mt to opportunistically collect corpora from several domains, which means that training instances are distributed according to a mixture in the sequel, boldface λ denotes a vector with λ(d) the d th component of λ. the main challenge in this situation is to make the best of heterogeneous data, with the aim to achieve optimal performance for the target test conditions. these might correspond to data from just one of the training domains, as in standard supervised domain adaptation; a more difficult case is when the test data is from one domain unseen in training (unseen domain adaptation); in multido-main adaptation finally, the test distribution is itself a mixture of domains, some of which may also be observed in training. we thus assume that the test distribution takes the form d t (x) = d λ t (d)d d (x) -with only one non-null component in the case of domain adaptation (see figure 1 ). these situations have been amply documented from a theoretical perspective @xcite @xcite . a general recommendation in the da setting is to adjust the sampling distribution used to optimize the system so as to compensate for the mismatch between d s (x) and d t (x). this can be approximated by reweighting instances, or more conveniently domains, which are selected during training with a probability λ l (d), with λ l (d) = λ s (d). a widely-used approach to supervised da is fine-tuning @xcite , where λ l varies during learning. with our notations, this approach first learns an initial parameter value with all the data (∀d, λ l (d) = λ s (d)), then continues training with only batches from the test domain d t (λ l (d) = i(@xmath2, with i(a) the indicator function for predicate a. this strategy is potentially suboptimal as some out-of-domain samples may contribute to the final performance due to e.g. domain similarity. optimizing the learning distribution in multidomain settings is even more challenging as the learner needs to take advantage of possible domains overlaps and also of the fact that some domains might be easier to learn than others. ## related work domain adaptation is an old problem that has been studied from many angles, both for smt and nmt. a survey of supervised and unsupervised da for nmt is in @xcite , where the authors distinguish between data-centric and model-centric da, a view also adopted in the recent survey of @xcite . our approach to da in this paper falls under the former category. we refer readers interested in da to these papers. multidomain nmt (mdmt) aims to develop systems that simultaneously bode well for several domains. like for da, techniques for supervised mdmt combine one or several ingredients: (a) the specialization of data representations @xcite or of sub-networks @xcite to differentiate the processing of each domain; (b) the use of adversarial techniques to neutralize differences between domains @xcite ; (c) the use of automatic domain identification e.g. @xcite . unsupervised mdmt is studied in @xcite , as an instance of unsupervised da. most approaches to adaptive/dynamic data selection take inspiration from @xcite , where the notion of curriculum learning is introduced. cl relies on the notion of the "easiness" of zhang et al. (2019) study supervised da and propose a curriculum approach which progressively augments the training data: early stages only use in-data, while less relevant foot_3 data are introduced in later stages. this is opposite to the policy of van der wees et al. (2017), whose gradual fine-tuning progressively increases the focus on in-domain data. @xcite use reinforcement learning to learn the curriculum strategy: in this work, complexity corresponds to difficulty levels which are binned using contrastive data selection. the reward is based on the increase of the devset loss that results from the current data selection strategy. this technique is applied to multilingual nmt in @xcite . @xcite propose another cl-based approach which relies on instance uncertainty as a measure of their difficulty and presents data samples starting with the easiest. another contribution of this work is a new stopping criterium. closest to our problems, @xcite adapt cl for multidomain nmt, where an optimal instance weighting scheme is found using bayesian optimization techniques. each step consists of (a) weighting instances based on relevance features, (b) fine-tuning a pretrained model using the weighted training set, and is applied to train a sequence of models. the one that maximizes the devset performance is finally retained. ## conclusion and outlook in this study, we have presented a generic framework to perform multiple adaptation tasks for machine translation, ranging from supervised domain adaptation to multidomain nmt and unseen domain adaptation. in our experiments, we have shown that the same algorithm, aimed at automatically finding an effective data sampling scheme during the course of training, can be used in all these situations. this algorithm, we believe, provides us with a more sound approach to (multi-domain) da than existing heuristics and dispenses with the costly search of optimal meta-parameters. another contribution of our work is an experimental comparison of recent approaches to dynamic data selection. our future work will continue developing this approach and improve its effectiveness. one issue that we have left unaddressed is reward normalization, which is especially important in the early stages of training @xcite . another area where we need to progress is the unsupervised learning setting of § 5.5, where our results lag behind supervised da. this might be due to the inability of our simplistic optimization strategy to handle situations where the number of clusters is large.