Analyzing the Use of Influence Functions for Instance-Specific Data Filtering in Neural Machine Translation (2022)

Abstract: Customer feedback can be an important signal for improving commercial machine translation systems. One solution for fixing specific translation errors is to remove the related erroneous training instances followed by re-training of the machine translation system, which we refer to as instance-specific data filtering. Influence functions (IF) have been shown to be effective in finding such relevant training examples for classification tasks such as image classification, toxic speech detection and entailment task. Given a probing instance, IF find influential training examples by measuring the similarity of the probing instance with a set of training examples in gradient space. In this work, we examine the use of influence functions for Neural Machine Translation (NMT). We propose two effective extensions to a state of the art influence function and demonstrate on the sub-problem of copied training examples that IF can be applied more generally than hand-crafted regular expressions.

## introduction neural machine translation (nmt) is the de facto standard for recent high-quality machine translation systems. nmt, however, requires abundant amount of bi-text for supervised training. one common approach to increase the amount of bi-text is via data augmentation @xcite @xcite . another approach is the use of web-crawled data @xcite but since crawled data is known to be notoriously noisy @xcite , a plethora of data filtering techniques @xcite @xcite , inter alia) have been proposed for retaining a cleaner portion of the bi-text for training. while standard data filtering techniques aim to improve the quality of the overall training data * work done during an internship at amazon. without targeting the translation quality of specific instances, instance-specific data filtering focuses on the improvement of translation quality toward a specific set of input sentences via removal of the related training data. in commercial mt, this selected set of sentences can be the problematic translations reported by customers. one simple approach of instance-specific data filtering in nmt is manual filtering. in manual filtering, human annotators identify translation errors on sentences reported by customer and designs filtering scheme, e.g., regular expressions to search related training examples for removal from the training set. in this work, we attempt to apply a more automatable technique called influence functions @xcite which is shown to be effective on image classification @xcite , and certain nlp tasks such as sentiment analysis, entailment and toxic speech detection @xcite ). given a probing example, influence functions (if) search for the influential training examples by measuring the similarity of the probing example with a set of training examples in gradient space. @xcite use a low-rank approximation of the hessian to speed up the computation of if and apply the idea of self-influence to nmt. however, self-influence measures if a training instance is an outlier rather than its similarity with another instance. @xcite question the back-tracing ability of if on the facttracing task. they compare if with heuristics used in information retrieval and attribute the worse performance of if to a problem called saturation. compared to fact-tracing, the target sides of machine translation can be more diverse which complicates the application of if. we apply an effective type of if called tracin @xcite to nmt for instance-specific data filtering and analyze its behaviour by constructing synthetic training examples containing simulated translation errors. in particular, we find that • the gradient similarity, also called the influence foot_0 , is highly sensitive to the network component. to the best of our knowledge, we are the first to investigate applying if for instance-specific data filtering to nmt. ## method influence functions if is a technique from robust statistics @xcite . it aims to trace a model's predictions back to the most responsible training examples without repeated re-training of the model, aka leave-one-out. koh and liang (2017) extend this idea from robust statistics to deep neural network that requires only the gradient of the loss functions l and hessian-vector products so that the influence i(z, z ) of two examples z and z is approximated as tracin, denoted by i tracin (z, z ), replaces the computationally costly hessian matrix with an identity matrix. the remained gradient dot product, or called the gradient similarity, is instead computed over c number of checkpoints, followed by averaging: in nmt, given the same source sentence, the magnitude of the gradient in general is positively correlated to the length of the target sentence. in order to reduce the effect of the target length, we normalize equation 2 by the product of ∇ θ l(z ) and ∇ θ l(z) , or equivalently, we compute the cosine similarity of ∇ θ l(z ) and ∇ θ l(z). given a probing instance z and its probing gradient ∇ θ l(z ), instances in the training set that yield a positive value of i tracin (z, z ) are called the positively influential training instances (+if-train) whereas those that yield a negative value of i tracin (z, z ) are called the negatively influential training instances (-iftrain). taking a gradient step on +iftrain reduces the loss on the probing example while taking a gradient step on -iftrain increases it. if can be used for data filtering by removing the +iftrain examples of low quality probing samples since their gradients have similar direction. conversely, if the probing sample is of high quality, removing -iftrain examples from the training data would be expected to increase translation quality w.r.t. the probing sample. ## experimental setting model configuration and training we use transformer base configuration as described in @xcite with default setting and implementation in fairseq. we use a sentence-piece model to create subword units of size 32k. unless otherwise specified, we pre-trained our nmt on europarl-v7 data and news commentary-v12 data in german-english direction from wmt17 for 100 epochs, about 112k updates, using adam optimizerion training of 16-bit foot_1 . the effective mini-batch size is 4096 x 16 tokens and it takes a p3.16xlarge 3 machine on aws 6 hours for training. we evaluate the mt model on the newstest2017 test set with a checkpoint averaged over the 10-best checkpoints, measured by the validation loss on the newstest2014-2016 dev set. on the test set, our nmt model with non-shared parameters with the two word embeddings and the output layer scores 29.99 bleu whereas the one with shared parameters scores 29.78 bleu. we use beam search with beam size of 5 in decoding. tracin we select 5 checkpoints, i.e., at epoch 5, 8, 15, 30 and 100 for computing tracin 4 . we select checkpoints which have relatively large changes in the validation loss, i.e., usually in the earlier phrase of training, and include the last one to cover information at the end of the training. we com-pute the per-sample gradient with a batch size of 1 parallelized over multiple processes with several g4dn.2x 3 machines on aws. ## experimental results this section describes our findings on the properties of applying if on nmt for instance-specific data filtering. ## conclusion we have analyzed the use of influence functions for nmt as instance-specific data filtering. by constructing synthetic instances, we found that 1) the gradient similarity is very sensitive to the selected network components, 2) vanilla influence functions are not sufficient for good retrieval performance, 3) our proposed contrastive-if can boost the retrieval performance regardless of the gradient component or parameter sharing, 4) finding an effective automation of if for instance-specific data filtering is difficult. this is because the proper choice of gradient component with respect to the type of error in the probing example is crucial for the effectiveness of influence functions. despite the reported effectiveness for certain classification tasks in previous literature, our results show that applying if to nmt poses some practical difficulties that we have not yet been able to solve.