Phrase-level Active Learning for Neural Machine Translation (2021)

Abstract: Neural machine translation (NMT) is sensitive to domain shift. In this paper, we address this problem in an active learning setting where we can spend a given budget on translating in-domain data, and gradually fine-tune a pre-trained out-of-domain NMT model on the newly translated data. Existing active learning methods for NMT usually select sentences based on uncertainty scores, but these methods require costly translation of full sentences even when only one or two key phrases within the sentence are informative. To address this limitation, we re-examine previous work from the phrase-based machine translation (PBMT) era that selected not full sentences, but rather individual phrases. However, while incorporating these phrases into PBMT systems was relatively simple, it is less trivial for NMT systems, which need to be trained on full sequences to capture larger structural properties of sentences unique to the new domain. To overcome these hurdles, we propose to select both full sentences and individual phrases from unlabelled data in the new domain for routing to human translators. In a German-English translation task, our active learning approach achieves consistent improvements over uncertainty-based sentence selection methods, improving up to 1.2 BLEU score over strong active learning baselines.

## introduction machine translation (mt) models are very sensitive to domain shift @xcite , and one typical way to address this problem is adding in-domain data to the mt training process @xcite . however, this data may not be available a priori, and hiring professional translators with knowledge of specific domains (such as medicine or law) is usually costly. as a result, active learning approaches @xcite @xcite have been widely adopted to reduce the annotation cost by translating a smaller representative subset of the in-domain data, with the hope that models trained on this translated subset approximate those trained on a much larger labeled set. in general, active learning (al) approaches iterate between two steps: data selection/annotation, and model update. with regards to data selection for machine translation, most existing works @xcite @xcite focus on selecting sentences that are most useful for training either phrase-based machine translation (pbmt) or neural machine translation (nmt) models. however, even the most informative sentences inevitably involve segments that the mt system can already translate well, and asking the translator to also translate these segments is not costeffective. there have been a few works used in conjunction with older pbmt models that ameliorate this problem through phrase-based selection techniques @xcite @xcite , which select only individual phrases, maximizing information gain. however, while these translated phrases can be easily integrated into pbmt by adding them to the existing phrase table, incorporating them into nmt models is less simple because nmt has no concept of a "phrase table" and must be trained on full sentences similar to those that must be translated. in this paper, we propose a method for incorporating phrase-based active learning into nmt. specifically, we first describe sentence-based and phrase-based selection strategies, then propose a hybrid strategy that combines both methods. we also describe several ways to incorporate this translated data into the training of nmt systems. we conduct experiments on german-english translation by adapting nmt models trained on wmt parallel data to the medicine and it domains. experimental results show that the hybrid selection strategy obtains more stable translation performance than either phrase-based or sentence-based selection strategy. ## problem definition in the setting of active learning for domain adaptation, we are given an out-of-domain labelled corpus (x, y) ∈ l and an in-domain unlabelled corpus @xmath0. we define a phrase as a contiguous sequence of words up to some length limit n , and denote a set of possible phrases in a sentence x by ∪ @xmath1,n ] n-gram(x), where we set @xmath2. to obtain translations of unlabelled data, we assume access to professional translators o(•) who can translate source-side sentences s and/or phrases p selected from u, i.e., o(x) ∀@xmath3, and o(p) ∀@xmath4,n ] n-gram(x). we assume that translating sentences or phrases requires cost c(•), and annotation must be performed within a fixed budget @xmath5. this active learning procedure consists of two main steps: selection/translation ( §3) and fine-tuning ( §4). ## training with sentences and phrases after data selection, we fine-tune the base nmt model on the newly translated data. this is essentially an extreme form of domain adaptation where we adapt a base nmt model trained on outof-domain data to a new domain. specifically, we adapt a strategy of mixed fine-tuning @xcite , which continues training a pretrained out-of-domain model on both in-domain data and a certain amount of out-of-domain data to prevent overfitting to relatively small in-domain data. compared to the standard domain adaptation setting where we have only a small number of in-domain sentences, our phrase-level active learning setting has the additional difficulty of having to use short translations of individual phrases. in the following, we describe both methods to choose which data to use in mixed fine-tuning, and how to incorporate phrasal translations. ## related work active learning for machine translation pioneering works on active learning for machine translation focus on selecting sentences that are most useful for training pbmt. this includes sentence selection strategies based on maximizing the percentage of unseen n-gram @xcite , ngram frequency, lexical diversity @xcite , or in-domain coverage @xcite . these sentence selection strategies have been used in active learning algorithms to deal with static data in the batch mode @xcite , or steaming data in the interactive setting @xcite @xcite . for phrase-level annotations, there have been a few works applying phrase-based selection @xcite to pbmt. while the annotated phrases can be easily integrated by adding them with estimated translation probability to the existing phrase table in pbmt, it it less trivial to integrate these phraselevel annotations in nmt. @xcite integrated the word-level translations to nmt by interpolating the probability of the nmt decoder with the estimated lexical probability. however, this approach requires a modification of the nmt model. our paper investigates data-driven approaches that augment the training data by leveraging annotated phrases and existing parallel data. ## discussion and future work in this paper, we investigate ways to incorporating phrasal translations into training nmt for domain adaptation in the active learning setting. we find that phrasal translation is particularly useful in the adaptation scenario where longer sentence context is not necessarily required to translate indomain words correctly. in contrast, nmt systems can benefit from learning sentence structure with sentence-based selection strategies. the hybrid selection strategies can combine the merits of both sentence-based and phrase-based selection strategies. nonetheless, there are several future directions. (1) it is worth exploring how different annotation strategies may result in a difference in cost or time. (2) although several findings could be generalized to other language pairs, testing our methods on morphologically rich languages is our next step. (3) our current hybrid strategy simply allocates the annotation budget evenly without assuming any prior knowledge of the strategies and the translation performance. techniques in multiarmed bandit problems (gittins et al., 2011) can be used to learn a good allocation strategy.