Instance Selection for Online Automatic Post-Editing in a multi-domain scenario (2016)

Abstract: In recent years, several end-to-end online translation systems have been proposed to successfully incorporate human post-editing feedback in the translation workflow. The performance of these systems in a multi-domain translation environment (involving different text genres, post-editing styles, machine translation systems) within the automatic post-editing (APE) task has not been thoroughly investigated yet. In this work, we show that when used in the APE framework the existing online systems are not robust towards domain changes in the incoming data stream. In particular, these systems lack in the capability to learn and use domain-specific post-editing rules from a pool of multi-domain data sets. To cope with this problem, we propose an online learning framework that generates more reliable translations with significantly better quality as compared with the existing online and batch systems. Our framework includes: i) an instance selection technique based on information retrieval that helps to build domain-specific APE systems, and ii) an optimization procedure to tune the feature weights of the log-linear model that allows the decoder to improve the post-editing quality.

## introduction nowadays, machine translation (mt) is a core element in the computer-assisted translation (cat) framework. the motivation for integrating mt in the cat framework lies in its capability to provide useful suggestions for unseen segments, which helps to increase the translators' productivity. however, it has been observed that mt is often prone to systematic errors that human post-editing has to correct before publication. the by-product of this "translation as post-editing" process is an increasing amount of parallel data consisting of mt output on one side and its corrected version on the other side. this data can be leveraged to develop automatic post-editing (ape) systems capable not only to spot recurring mt errors, but also to correct them. thus, integrating an ape system inside the cat framework can further improve the quality of the suggested segments, reduce the workload of human post-editors and increase the productivity of the translation industry. as pointed out in @xcite and @xcite , from the application point of view ape components would make it possible to: in the last decade several works have shown that the quality of the machine translated text can be improved significantly by post-processing the translations with an ape system @xcite @xcite @xcite @xcite . these systems mainly follow the phrase-based machine translation approach where the mt outputs (with optionally the source sentence) are used as the source language corpus and the post-edits are used as the target language corpus. a common trait of all these ape systems is that they were developed in a batch mode, which consists of training the models over a batch of parallel sentences, optimizing the parameters over a development set, and then decoding the test data with the tuned parameters. although these standard approaches showed promising results, they lack the ability to incorporate human feedback in a real-time translation workflow. this led to the development of online learning algorithms that can leverage the continuous streams of data arriving in the form of human post-editing feedback to dynamically update the models and tune the parameters on-the-fly within the cat framework. in recent years, several online systems have been proposed in mt (see section 2 for more details) to address the problem of incremental training of the models or on-the-fly optimization of feature weights. few online mt systems have also been applied to the ape scenario @xcite in a controlled working environment in which the systems are trained and evaluated on homogeneous/coherent data where the training and test sets share similar characteristics. moving from this controlled lab environment to real-world translation workflow, where training and test data can be produced by different mt systems, post-edited by various translators and belong to several text genres, makes the task more challenging, because the ape systems have to adapt to all these diversities in real-time. we define this scenario as a multi-domain translation environment (mdte), where a domain is made of segments belonging to the same text genre and the mt outputs are generated by the same mt system. to reproduce this scenario, in our experiments we run the online ape systems on the concatenation of two datasets belonging to different domains. a preliminary evaluation in the mdte scenario reveals that online systems are not robust enough to learn and adapt towards the dynamics of the data, mainly because they try to leverage all the seen data without considering the peculiarities of each domain. in the long-run, these systems tend to become more and more generic, which may not be useful and even harmful to automatically post-edit domain-specific segments. to address this problem, for the first time, we propose an online ape system that is able to efficiently work in a mdte scenario. our intuition is that an online ape model trained with few but relevant data (with respect to the segment to be post-edited) can be more reliable than using all the available data as-is. to validate this intuition, we propose an online ape system based on an instance selection (is) technique that is able to retrieve the most relevant training instances from a pool of multi-domain data for each segment to post-edit. the selected data are then used to train and tune the ape system on-thefly. the relevance of a training sample is measured by a similarity score that takes into account the context of the segment to be post-edited. this technique allows our online ape system to be flexible enough to decide if it has the correct knowledge for post-editing a sentence or if it is safer to keep the mt output untouched, avoiding possible damages of correction made with insufficient/unreliable knowledge. the results of our experiments with various data sets show that our online learning approach based on is is: i) able to outperform the batch and the other online ape techniques in the single domain scenario, and ii) robust enough to work in a mdte to generate reliable post-edits with significantly better performance than the existing online ape systems. ## online translation systems online translation systems aim to incorporate human post-editing feedback (or the corrected version of the mt output) into their models in real-time, as soon as it becomes available. this feedback helps the system to learn from the mistakes made in the past translations and to avoid repeating them in future translations. this continuous learning capability will eventually improve the quality of the translations and consequently increase the productivity of the translators/post-editors @xcite working with mt suggestions in a cat environment. the basic workflow of an online translation system goes through the following steps repeatedly: i) the system receives an input segment; ii) the input segment is translated and provided to the post-editor to fix any errors in it; and iii) the human post-edited version of the translation is incorporated back into the system, by stepwise updating the underlying models and parameters. in the ape context, the input is a machine-translated segment (optionally with its corresponding source segment), which is processed by the online ape system to fix errors, and then verified by the post-editors. several online translation systems have been proposed over the years @xcite @xcite @xcite @xcite . in this section, we describe two online systems that have been used in the ape task (pepr, and thot), and one in the mt scenario which is similar to our proposed system (realtime cdec): pepr: post-edit propagation: simard and foster (2013) proposed a method for post-edit propagation (pepr), which learns post-editors' corrections and applies them on-the-fly to further mt output. their proposal is based on a phrase-based smt system, used in an ape setting with online learning mechanism. to perform post-edit propagation, this system was trained incrementally using pairs of machine-translated (mt) and human post-edited (pe) segments as they were produced. when receiving a new pair (mt, pe), word alignments are obtained by using damerau-levenshtein distance. in the next step the phrase pairs are extracted and appended to the existing phrase table. the whole process is assumed to take place within the context of a single document. for every new document the ape system begins with an "empty" model. since the post-editing rules are learned for a given document they can be more precise and useful for that document, but the limitation is that knowledge gained after processing one document is not utilized for other similar documents. this limitation can be addressed by our system (section 3), in which we maintain one global knowledge base to store all the processed documents, still being able to retrieve post-editing rules specific to a document to be translated. thot: the thot toolkit @xcite is developed to support fully automatic and interactive statistical machine translation. foot_0 it was also used by @xcite in an online setting for the ape task, to perform large-scale experiments with several data sets for multiple language pairs, with base mt systems built using different technologies (rule-based mt, statistical mt). in the majority of their experiments online ape successfully improved the quality of the translations obtained from the base mt system by a significant margin. to update the underlying translation and language models with the user feedback, a set of sufficient statistics was maintained that can be incrementally updated. in the case of language model, only the n-gram counts are required to maintain sufficient statistics. to update the translation model, an incremental version of em algorithm is used to first obtain word alignment and then phrase pairs counts were extracted to update the sufficient statistics. other features like source/target phrase-length models or distortion model are implemented by means of geometric distributions with fixed parameters. the sentence length model is implemented by means of gaussian distributions. however, the feature weights of the log-linear model are static throughout the online learning process, as opposed to our method that updates the weights on-the-fly. also, this method learns post-editing rules from all the data processed in real-time, whereas, our approach learns from the most relevant data points. realtime cdec: @xcite proposed an online model adaptation method to leverage human post-edited feedback to improve the quality of an mt system in a real-time translation workflow. to build the translation models they use a static suffix array @xcite to index initial data (or a seed corpus), and a dynamic lookup table to store information from the post-edited feedback. to decode a sentence, the statistics of the translation options are computed both from the suffix array and from the lookup table. an incremental language model is maintained and updated with each incoming human post-edit. to update the feature weights they used an extended version of the margin-infused relaxed algorithm (mira) @xcite . the decoding is treated as simply the next iteration of mira, where a segment is first translated and then its corresponding reference/post-edition is provided to the model, and mira updates the parameters. while this system was earlier used in the context of mt, in this work we use it to investigate its applicability in online ape. a key difference between this approach and ours is the sampling technique. the former uses suffix-arrays to always retrieve the top k source phrases, whereas in our approach the number of samples (or the training instances) is dynamically set to use only the most relevant ones. another difference is visible in the parameter optimization step. realtime cdec optimizes the feature weights of the log-linear model after decoding each segment, whereas, our method optimizes the weights specifically for the segment to be post-edited. ## instance selection for online ape system the online systems described in section 2 compute and update the feature scores of the loglinear models based on all the previously seen data. this indicates that, in the long-run, the model will tend to become more and more generic, since the data processed in the online scenario may belong to multiple domains as explained in section 1. having a generic model might not be useful to retrieve the domain-specific post-editing rules needed to fix errors in a particular document. one solution is to build document-specific ape models as proposed by @xcite . in their approach, however, once the entire document is processed the models are reset back to their original state, due to which the knowledge gained from the current document is lost. to preserve all the knowledge gained in the online learning process, at the same time being able to apply specific post-editing rules when needed, we propose an instance selection technique for online ape. our proposed framework, as shown in figure 1 , uses a global knowledge base to preserve all the data points seen in the online process, and has the ability to retrieve specific data points whose context is similar to the segment to be post-edited. these data points are used to build reliable ape models. when there are no reliable data points in the knowledge base, the mt output is kept untouched, as opposed to the existing ape systems, which tend to figure 1: architecture of our online ape system always "translate" the given input segment independently from the reliability of the applicable correction rules. this approach of post-editing with reliable information only makes our system more precise compared with others (see results in section 5): that is when a post-editing rule is applied it is more likely to improve the quality of the translation. when no reliable knowledge is available for the correction, the mt output is left untouched. we propose online ape but we actually "emulate" it by processing the data points one at a time. our proposed algorithm assumes to have the following data to run the online experiments: i) source (src); ii) mt output (mt); and iii) human post-edits (pe) of the mt output. at the beginning the knowledge base of our online ape system is empty and it will be updated whenever a new instance (a tuple containing parallel segments from all the above mentioned documents) is processed. when the system receives an input (src, mt), it proceeds through the following steps: instance selection. initially, it selects the most relevant training instances from a pool of multi-domain data stored in our knowledge base. this will help to build a reliable ape model for each input segment processed in real-time. the relevance of the training instances with respect to the input segment is measured in terms of a similarity score based on the term frequency-inverse document frequency (tf-idf ), generally used in information retrieval. the larger the number of words in common between the training and the input sentences, the higher is the score. in our system, these scores are computed using the lucene library. foot_1 only those training instances that have a similarity score above a certain threshold (decided over a held-out development set) are used to build the system. in case there are no training instances available, we preserve the input segment as it is. indeed, we assume that ape with unreliable information can damage the mt segment instead of improving the translation quality. this is one of the main outcomes of the first ape pilot task organized last year within the wmt initiative @xcite and, as we will see from our results, it represents a major problem for the approaches that always translate the given input segments. the proposed instance selection technique (or sampling mechanism) differs from the one proposed in real-time cdec @xcite , which uses suffix-arrays to select the top k instances. in our approach the sample size is in fact dynamically set in order to select only the most similar ones. this allows us to build more reliable models (since the underlying data better resembles the test segment), and to gain speed when the sample size is small. the use of a tf-idf similarity measure was proposed before in the context of machine translation by @xcite to create a pseudo in-domain corpus from a big out-of-domain corpus. our work is the first to investigate it for the ape task in an online learning scenario. ## experiments and results our preliminary objective is to examine if the online learning methods are able to achieve results that are competitive with those of batch methods, which are potentially favored by the possibility to leverage all the training data at the same time. for this test, all the algorithms are evaluated in the classic in-domain setting, where training, development, and test sets are sampled from the same data set or domain. all the online ape methods are run in two modes; i) batch: the test set is not used in the learning process (to have a fair comparison with the batch ape), ii) online: the test set is leveraged in the online learning process. the experiments are performed for both the data sets (autodesk and wmt2016), and their corresponding results are reported in from the results of the in-domain experiments with the autodesk data set it is evident that our proposed online ape method performs not only better than cdec and thot (both in batch and online mode) but also better than the strong batch ape method. it achieves significant improvements of 0.54 bleu, 1.26 ter, and 17.9% precision over the batch ape, which already beats the other online methods. the improvement of our system can be attributed to its ability to learn from the most relevant data and to avoid over-correction by leaving the test segment untouched when no reliable information is found in the knowledge base. as discussed in section 4.1, several factors like sentence length, sparsity, and translation quality make the wmt2016 data set more challenging to improve for all the online ape methods. in particular, due to the higher translation quality of the mt segments, the room for improvement gets lower and the chances of damaging the correct parts are higher. this is visible from the low precision scores reported in building on these results, the main goal of this research is to examine the performance of the online ape methods in a mdte. this represents a more challenging condition since the system has to adapt to the dynamics of the data processed in a real-time scenario. to emulate this environment, all the online learning methods are trained and tuned on one data set (or domain) and evaluated on the other data set with the possibility to learn from it. in order to capture the peculiarities of the online learning methods over a long run with many data points, we use the training section of the second data set as a test set. the left side of ## performance analysis to understand and compare the behavior of different online learning approaches in the longrun, the plot in figure 2 shows the moving average ter (window of 750 data points) at each segment of the autodesk test set for the multi-domain experiment ( for the wmt2016 test set the moving average ter is shown in figure 3 . as said before, improving translation quality on this test set is more challenging, which is reflected in the graph. although none of the systems is able to improve over the mt baseline, our system manages to consistently stay close to the mt performance throughout the test set, whereas, all other systems show significant drops. this ensures that our approach is more robust against the domain-shift figure 3: moving average ter for the wmt2016 test set in a multi-domain scenario and even in this difficult scenario it is able to maintain stable performance close to the mt without a large deterioration. to gain further insights about the performance at the segment level, the plot in figure 4 compares our approach against thot for the first 300 segments of the autodesk test set used in the multi-domain experiment. it shows the differences between the segment-level ter of the mt (t er mt ) and our approach (t er our approach ), and mt and thot (t er t hot ) automatically post-edited segments. we notice that our approach modifies less segments compared with thot, because it builds a model only if it finds relevant data in the knowledge base, otherwise it leaves the mt segment untouched. these untouched mt segments, when modified by thot, often lead to deterioration rather than to improvements (as seen by many negative peaks for thot in the figure 4 ). this suggests that, compared with the other online approaches, the output obtained with our solution has a higher potential for being useful to human translators. such usefulness comes not only in terms of a more pleasant post-editing activity, but also in terms of time savings yield by overall better suggestions. ## conclusion we addressed the problem of building robust online ape systems in a multi-domain translation environment in which the system has to continuously adapt to the dynamics of diverse data processed in real-time. our evaluation revealed that the online systems that leverage all the available data without considering the peculiarities of each domain are not robust enough to work in a multi-domain translation environment, because they are unable to learn domain-specific postediting rules. to overcome this limitation, we proposed an online learning framework based on instance selection that has the capability to filter out the most relevant information from a pool of multi-domain data for learning domain-specific post-editing rules. when no reliable information is available our system leaves the mt segments untouched, these segments when automatically post-edited by other systems are often found to get deteriorated. therefore, the ape suggestions provided by our system to the translators/post-editors are more reliable with better translation quality. from our experiments in a simulated multi-domain environment, we learn that the postediting rules are not portable across domains which is revealed by the poor performance of the batch ape system that can leverage only the training data. in the case of online systems that leverage also the test set, it was still a challenging scenario (specially for the autodesk-wmt2016 data set). among all the online systems, our proposed approach has the highest improvement on the wmt2016-autodesk data set, and the least degradation on the autodesk-wmt2016 data set with respect to the mt quality. experiments in the in-domain setting confirmed that our approach for instance selection is also useful in a single domain scenario. it performed significantly better than the batch ape that already beats cdec and thot. one common observation from all the experiments in different working scenarios and with different data sets is that our system has the highest precision among all its competitors (mt, batch ape, cdec, and thot). this indicate that when our system automatically post-edits mt segments, it is more likely to improve the quality of the mt output, which makes it a viable solution to be deployed in a real-word cat framework.