Efficient Machine Translation Domain Adaptation (2022)

Abstract: Machine translation models struggle when translating out-of-domain text, which makes domain adaptation a topic of critical importance. However, most domain adaptation methods focus on fine-tuning or training the entire or part of the model on every new domain, which can be costly. On the other hand, semi-parametric models have been shown to successfully perform domain adaptation by retrieving examples from an in-domain datastore (Khandelwal et al., 2021). A drawback of these retrieval-augmented models, however, is that they tend to be substantially slower. In this paper, we explore several approaches to speed up nearest neighbors machine translation. We adapt the methods recently proposed by He et al. (2021) for language modeling, and introduce a simple but effective caching strategy that avoids performing retrieval when similar contexts have been seen before. Translation quality and runtimes for several domains show the effectiveness of the proposed solutions.

## introduction modern neural machine translation models are mostly parametric @xcite , meaning that, for each input, the output depends only on a fixed number of model parameters, obtained using some training data, hopefully in the same domain. however, when running machine translation systems in the wild, it is often the case that the model is given input sentences or documents from domains that were not part of the training data, which frequently leads to subpar translations. one solution is training or fine-tuning the entire model or just part of it for each domain, but this can be expensive and may lead to catastrophic forgetting @xcite . recently, an approach that has achieved promising results is augmenting parametric models with a retrieval component, leading to semi-parametric models @xcite @xcite @xcite @xcite . these models construct a datastore based on a set of source / target sentences or word-level contexts (translation memories) and retrieve similar examples from this datastore, using this information in the generation process. this allows having only one model that can be used for every domain. however, the model's runtime increases with the size of the domain's datastore and searching for related examples on large datastores can be computationally very expensive: for example, when retrieving 64 neighbors from the datastore, the model may become two orders of magnitude slower @xcite . due to this, some recent works have proposed methods that aim to make this process more efficient. @xcite proposed constructing a different datastore for each source sentence, by first searching for the neighbors of the source tokens; and @xcite proposed several techniques -datastore pruning, adaptive retrieval, dimension reduction -for nearest neighbor language modeling. in this paper, we adapt several methods proposed by @xcite to machine translation, and we further propose a new approach that increases the model's efficiency: the use of a retrieval distributions cache. by caching the knn probability distributions, together with the corresponding decoder representations, for the previous steps of the generation of the current translation(s), the model can quickly retrieve the retrieval distribution when the current representation is similar to a cached one, instead of having to search for neighbors in the datastore at every single step. we perform a thorough analysis of the model's efficiency on a controlled setting, which shows that the combination of our proposed techniques results in a model, the efficient knn-mt, which is approx-imately twice as fast as the vanilla knn-mt. this comes without harming translation performance, which is, on average, more than 8 bleu points and 5 comet points better than the base mt model. in sum, this paper presents the following contributions: ## background when performing machine translation, the model is given a source sentence or document, @xmath0, . . . , x l ], on one language, and the goal is to output a translation of the sentence in the desired language, @xmath1, . . . , y n ]. this is usually done using a parametric sequence-to-sequence model @xcite , in which the encoder receives the source sentence as input and outputs a set of hidden states. then, at each step t, the decoder attends to these hidden states and outputs a probability distribution p nmt (y t |@xmath2, x) over the vocabulary. finally, these probability distributions are used to predict the output tokens, typically with beam search. ## efficient knn-mt in this section, we describe the approaches introduced by @xcite to speed-up the inference time for nearest neighbor language modeling, such as pruning the datastore ( §3.1) and reducing the representations dimension ( §3.2), which we adapt to machine translation. we further describe a novel method that allows the model to have access to examples without having to search them in the datastore at every step, by maintaining a cache of the past retrieval distributions, for the current translation(s) ( §3.3). ## conclusion in this paper we propose the efficient knn-mt, in which we combine several methods to improve the knn-mt generation speed. first, we adapted to machine translation methods that improve retrieval efficiency in language modeling @xcite . then we proposed a new method which consists on keeping in cache the previous retrieval distributions so that the model does not need to search for neighbors in the datastore at every step. through experiments on domain adaptation, we show that the combination of the proposed methods leads to a considerable speed-up (up to 2x) without harming the translation performance substantially.