Investigating Active Learning in Interactive Neural Machine Translation (2021)

Abstract: Interactive-predictive translation is a collaborative iterative process and where human translators produce translations with the help of machine translation (MT) systems interactively. Various sampling techniques in active learning (AL) exist to update the neural MT (NMT) model in the interactive-predictive scenario. In this paper and we explore term based (named entity count (NEC)) and quality based (quality estimation (QE) and sentence similarity (Sim)) sampling techniques – which are used to find the ideal candidates from the incoming data – for human supervision and MT model’s weight updation. We carried out experiments with three language pairs and viz. German-English and Spanish-English and Hindi-English. Our proposed sampling technique yields 1.82 and 0.77 and 0.81 BLEU points improvements for German-English and Spanish-English and Hindi-English and respectively and over random sampling based baseline. It also improves the present state-of-the-art by 0.35 and 0.12 BLEU points for German-English and Spanish-English and respectively. Human editing effort in terms of number-of-words-changed also improves by 5 and 4 points for German-English and Spanish-English and respectively and compared to the state-of-the-art.

## introduction neural machine translation (nmt) requires a significantly large amount of in-domain data for building the robust systems. absence of sufficient training samples often result in the generation of erroneous output samples. post-editing could be an effective solution in this situation, where human interference may help to rectify the errors in the output samples. however, there are two problems, viz. (i) post-editing a large number of output samples is time consuming and not very efficient in terms of productivity and (ii) not including all the post-edited examples might pose the risk of encountering the same mistakes in future. hence, there is a necessity that instead of post-editing all the output samples, we explore effective sampling techniques for selecting important samples for post-editing, and further these post-edited samples are used to update the model's parameter following an active learning technique that makes the translation model learns from these (new) samples. interactive mt (imt) is viewed as an effective mean to increase the productivity in the translation industry. in principle, imt aims to reduce human effort in automatic translation workflows by employing an iterative collaborative strategy with its two most important components, the human agent and the mt engine. as of today, nmt models @xcite represent state-of-the-art in mt research. this has led researchers to test interactive-predictive protocol on nmt too. papers @xcite that pursued this line of research suggest that nmt is superior than phrase-based statistical mt @xcite . so use of interactive nmt (inmt) for output sample correction can significantly reduce the overall translation time and active learning strategy can use human corrected samples for adapting the underlying nmt model so that in future, the model does not repeat previous errors and improves the translation quality. the contributions of our current work are stated as follows: ## related work in a case, where an mt model is not providing high quality translation due to low resource or out-of-domain scenarios, it could be beneficial to update the model with new samples while preserving the previous knowledge too. there has been some works which deal with the large input data streams but generally adopt the incremental learning approaches (e.g. updating the model as the labelled data become available) rather than the active learning approach (where labelled data stream is not guaranteed). in the literature @xcite , authors used incremental learning to update the translation model but these were with respect to the statistical machine translation (smt) model. @xcite applied incremental learning over the nmt model where they used the human post-edited data to update the initially trained models which make it very costly and time consuming due to human-edited data. @xcite ; ortiz-martínez (2016) used an interactive paradigm for updating the smt model on the iteratively corrected outputs. as for active learning, it has also been well adopted for model learning. the unbounded and unlabelled large data streams is well suited to the objective of active learning @xcite . this unbounded data stream scenario was explored by @xcite ; @xcite , where a pool of data was edited and the smt model was updated using this data. @xcite used the stream data to update the smt model. further, interactive paradigm of smt was introduced in @xcite ; @xcite . later, the nmt became more prominent and efficient in the interactive paradigm of mt @xcite . @xcite explored the application of active learning and imt on the nmt model. they performed the experiments over the attention based encoder-decoder nmt model @xcite . to handle the ## interactive neural machine translation in inmt @xcite , human translators correct errors in automatic translations in collaboration with the mt systems. here, users read tokens of the generated hypothesis from left to right and modifies (insert/replace) his/her choice of words in the hypothesis generated by the nmt model. from the start index to the right most token position where the user make change is considered as the 'validated prefix'. after the user makes any change, the model regenerates a new hypothesis by preserving the validated prefix and new tokens next to it. multiple attempts of token replacements may be required by a user to get the desired output as shown by an example in for an input-output sentence pair [x, y], where @xmath0, x 2 , ..., x m ) being a sequence of input tokens and @xmath1, y 2 , ..., y n ) being a sequence of output tokens, the probability of the ith translated word y i is calculated as in eq. ( 1 ): here, s i and c i are the i th decoder hidden state and context vector, respectively. as shown in eq. (1), in nmt, during decoding, next predicted output y i depends on model's previous output y 1 , ..., y i-1 . in inmt, y i will be generated by considering y * 1 , ..., y * i-1 as the previous tokens, where y * i-1 is actually the token of user's choice at sequence position i -1. eq. (2) shows the conditional probability of generating y i in the inmt scenario. p(y i |y * 1 , ..., y from figure 1 , we see that the sampling module selects and recommends the incoming inference samples to the inmt for supervision. the purpose of a sampling technique is to filter out the ideal candidate from the incoming inference samples for which the trained nmt model is most uncertain and by supervising that sample it should increase the nmt performance using the technique of al. let s be the input sentences for inference, b be the block of sentences that are taken from s iteratively. from the block b, c a chunk, the size of which depends on the percentage (%) of the samples from b are taken, is used to be supervised from the human. we take the size of b as 10,000 samples and the chunk size from b can be 20, 40, 60 and 80%. the amount of samples is measured by the count of sentence pairs. the sampling techniques which are implemented are pool based, and basically belong to two categories, namely uncertainty english-german english-spanish english-hindi train 1.26m (europarl) 1.9m (europarl) 1.6m (iitb corpus) dev 1,057 (europarl) 2000 (europarl) 599 (iitb corpus) testset 59,975 (newscommentary) 51,613 (newscommentary) 47,999 (ilci corpus) ## dataset we carried out experiments on three language pairs using three benchmark datasets. ## experimental setup our experiments were based on the transformer nmt model @xcite . we used 6 layered encoder-decoder stacks with 8 attention heads. embedding size and hidden sizes were set to 512, dropout rate was set to 0.1. feed-forward layer consists of 2,048 cells. adam optimizer @xcite was used for training with 8,000 warm up steps. we used the bpe @xcite with a vocabulary size of 40k. models were trained with opennmt toolkit foot_3 @xcite with batch size of 2,048 tokens till convergence and checkpoints were created after every 10,000 steps. during inference, beam size is set to 5. we measured bleu (calculated with multi-bleu.pl script) @xcite of the trained models on the test sets. ## results and analysis we evaluate the impact of the proposed sampling techniques for active learning in nmt in two different ways. firstly, we test whether the proposed techniques help the nmt model to improve its translation performance in terms of the bleu score. secondly, in order to see whether the proposed techniques are able to reduce the human efforts (number of token correction required) in correcting the hypothesis, we compare the performance of the proposed ## conclusion in this paper, we have explored the applicability of various sampling techniques in active learning to update the nmt models. we select the incoming source samples using the sampling techniques, correct them in an interactive nmt scenario and subsequently update the trained nmt model using the corrected parallel samples. it helps the model to adapt over the new parallel samples which results in improving the translation quality and reducing the human ef-fort for further hypothesis correction. we proposed term based (nec) and quality based (qe, sim emb , sim f uzzy ) sampling techniques to pick the source samples from a large block of input sentences for correction and subsequently updating the nmt models. since it is not feasible for a human to supervise (modify) a large set of input data coming for the translation, the proposed sampling techniques help to pick and recommend the suitable samples from large input data to the user for supervision. we measure the impact of sampling techniques by two criteria: first, improvement in translation quality in terms of bleu score and second, reduction in human effort (i.e. number of tokens in generated outputs needed to correct). we performed experiments over three language pairs i.e. english-german, english-spanish and english-hindi. we use different domain data for training and testing the nmt model to see if the nmt model trained over the data from one domain can successfully adapt to the different domain data. we empirically showed that the proposed term and quality based sampling techniques outperform the random sampling and outperformed the attention distraction sampling (ads) method