Can Automatic Post-Editing Improve NMT ? (2020)

Abstract: Automatic post-editing (APE) aims to improve machine translations, thereby reducing human post-editing effort. APE has had notable success when used with statistical machine translation (SMT) systems but has not been as successful over neural machine translation (NMT) systems. This has raised questions on the relevance of APE task in the current scenario. However, the training of APE models has been heavily reliant on large-scale artificial corpora combined with only limited human post-edited data. We hypothesize that APE models have been underperforming in improving NMT translations due to the lack of adequate supervision. To ascertain our hypothesis, we compile a larger corpus of human post-edits of English to German NMT. We empirically show that a state-of-art neural APE model trained on this corpus can significantly improve a strong in-domain NMT system, challenging the current understanding in the field. We further investigate the effects of varying training data sizes, using artificial training data, and domain specificity for the APE task. We release this new corpus under CC BY-NC-SA 4.0 license at https://github.com/shamilcm/pedra .

## introduction automatic post-editing (ape) aims to reduce manual post-editing effort by automatically fixing errors in the machine-translated output. @xcite first proposed ape to cope with systematic errors in selecting appropriate articles for japanese to english translation. earlier application of statistical phrase-based models for ape treated it as a monolingual re-writing task without considering the source sentence @xcite . modern ape models take the source text and machinetranslated text as input and output the post-edited text in the target language (see figure 1 ). source text (english): will he send the gifts to the house? machine translated text (german): die geschenke in mein haus schicken? ape models are usually trained and evaluated in a black-box scenario where the underlying mt model and the decoding process are inaccessible making it difficult to improve the mt system directly. ape can be effective in this case to improve the mt output or to adapt its style or domain. recent advancement of ape has shown remarkable success on statistical machine translation (smt) outputs @xcite even when trained with limited number of post-edited training instances (generally "triplets" consisting of source, translated, and post-edited segments), with or without additional large-scale artificial data @xcite . substantial improvements have been reported especially on english-german (en-de) wmt ape shared tasks on smt @xcite , when models were trained with fewer than 25,000 human post-edited triplets. however, on nmt, strong ape models have failed to show any notable improvement @xcite @xcite when trained on similar-sized human post-edited data. this has led to questions regarding the usefulness of ape with current nmt systems that produce improved trans-lations compared to smt. @xcite concluded that the results of the wmt'18 ape (nmt) task "might constitute the end of neural automatic post-editing for strong neural in-domain systems" and that "neuralon-neural ape might not actually be useful". contrary to this belief, we hypothesize that a competitive neural ape model still has potential to further improve strong state-of-the-art in-domain nmt systems when trained on adequate human post-edited data. we compile a new large post-edited corpus, subedits, which consists of actual human postedits of translations of drama and movie subtitles produced by a strong in-domain proprietary nmt system. we use this corpus to train a state-of-theart neural ape model (correia and martins, 2019), with the goal of answering the following three research questions to better assess the relevance of ape going forward: nmt with adequate data size? spoilers: through automatic and human evaluation, we confirm our hypothesis that, in order to notably improve over the original nmt output ("do-nothing" baseline), state-of-the-art ape models need to be trained on a larger number of human post-edits, unlike the case with smt. training on datasets of sizes in the scale of those from the wmt ape tasks, even with large-scale indomain artificial ape corpora, leads to underperformance. our experimental results also highlight that ape models are highly sensitive to domain differences. to effectively exploit out-of-domain post-edited corpora such as subedits in other domains, it has to be carefully mixed with available in-domain data. ## subedits corpus human post-edited corpora of nmt outputs from previous wmt ape shared tasks usually consist of fewer than 25,000 instances. large-scale artificial corpora such as escape @xcite , do not adequately cater to the primary ape objective of correcting systematic errors of the mt outputs since the pseudo "post-edits" are independent human-translated references often differing greatly from the mt output. ## bert encoder-decoder ape model bert encoder-decoder ape (correia and martins, 2019) is a state-of-the-art neural ape model based on a transformer model @xcite with the encoder and decoder initialized with pre-trained multilingual bert @xcite weights and fine-tuned on post-editing data. a single encoder is used to encode both the source text and the machine-translated text by concatenating them with the separator token @xcite . the encoder component of the model is identical to the original transformer encoder initialized with pre-trained weights from the multilingual bert. for the decoder, correia and martins (2019) initialized the context attention weights with the corresponding bert selfattention weights. also, the weights of the selfattention layers of the encoder and decoder are tied. all other weights are initialized with corresponding weights from the same multilingual bert model as well. bert encoder-decoder ape was shown to outperform other state-of-the-art ape models @xcite on smt outputs even in the absence of additional large-scale artificial data that competing models have used. an improved variant of this model with additional in-domain artificial data, despite being the winning submission of the recent wmt'19 ape en-de (nmt) task 2 using moses normalize-punctuation.perl script. 3 https://github.com/saffsd/langid.py @xcite , only performed marginally better than the baseline nmt output. for the purpose of this study, we base our experiments on the bert encoder-decoder ape architecture (correia and martins, 2019). ## analysis 6.1 impact of ape with varying nmt quality to study the impact of ape with varying quality of nmt output, we conduct analysis on subsets of our development set with varying translation qualities (figure 4 ). we split the subedits development set into 10 subsets by aggregating those triplets with the nmt output @xmath0, 90 -81 ter, . . ., 20 -11 ter, @xmath1. they are ordered from left to right in the x-axis in figure 4 according to increasing mt quality. y-axis denotes the difference (∆) between the ter score of ape output and nmt output for each subset. the more negative ∆ter indicates a larger improvement due to ape. we find that on the lower quality subsets, ape improves over nmt substantially. this improvement margin reduces with improving nmt quality and can deteriorate the nmt output when nmt quality is at the highest. this experiment shows that ape contributes to improving overall nmt performance by predominantly fixing poorer quality nmt outputs. the ape model's error will dominate and ape can become counter-productive when nmt output is nearly perfect (i.e., when there are very few or no postedits done on them as indicated by sentence-level ter scores @xmath2. ape task remains relevant until nmt systems achieve this state, which is still not the case even for strong in-domain nmt systems as indicated by our experiments. ## related work until 2018, ape models were benchmarked on smt outputs through various wmt ape tasks @xcite @xcite . the scale of post-edited data provided by these tasks was in the order of 10,000 to 25,000 triplets. the largest collection of human post-edits, released by @xcite , however, was on smt and consisted of 30,000 to 410,000 triplets across 12 language pairs. on smt output, participating systems showed impressive gains even with small training datasets from wmt ape tasks @xcite . the results of subsequent ape (nmt) tasks were not as promising with only marginal improvements on english-german and no improvement on english-russian @xcite . previously, there was no study to assess the necessity of larger human post-edited training data on ape performance on nmt outputs which we address in this paper. ape models were predominantly trained on large-scale artificial data combined with a few thousand human post-edits. junczys-dowmunt and grundkiewicz (2016) proposed generation of large-scale artificial ape training data via round-trip translation approach inspired from back-translation @xcite . they combined artificial training data with real data provided by wmt ape tasks to train their model. using a similar approach of generating artificial ape data, @xcite trained a monolingual re-writing ape model trained on the generated artificial training data alone. contrary to the round-trip translation approach, largescale artificial ape data was generated by simply translating source sentences using nmt and smt systems and using the reference translations as the "pseudo" post-edits to create escape corpus @xcite . using the escape english-italian ape corpus, @xcite assessed the performance of an online ape model in a simulated environment where the ape model is updated at test time with new user inputs. they found that their online ape models trained on es-cape found it difficult to improve specialized indomain nmt systems. such an analysis by training on artificial corpora may not adequately assess the actual potential of ape since these corpora do not fully cater to the task and can be noisy. the "synthetic" post-edits are independent or loosely coupled with the mt outputs, and are often drastically different from the mt output. this makes analyzing ape performance over competitive nmt systems on actual post-edited data an important step in understanding the potential of ape research. contrary to previous conclusions, our analysis shows that a competitive in-domain nmt system can be markedly improved by a strong neural ape model when trained on sufficient human post-edited training data. ## conclusion ape has been an effective option to fix systematic mt errors and improve translations from blackbox mt services. however, on nmt outputs, ape has shown hardly any improvement since training has been done on limited human postedited data. the newly collected subedits corpus is the largest corpus of nmt human post-edits collected so far. we reassessed the usefulness of ape on nmt using this corpus. we showed that with a larger human post-edited corpus, a strong neural ape model can substantially improve a strong in-domain nmt system. while artificial ape corpora help, we showed that the ape model performs better when trained on adequate human post-edited data (subedits) compared to large-scale artificial corpora. finally, our experiments comparing in and out-domain ape show that domain-specificity of training affects ape performance drastically and a combination of in and out-of-domain data with certain upscaling alleviates the domain-shift problem for ape. we find that ape mostly contributes to improving nmt performance by fixing the poorer-quality outputs that still exist with strong in-domain nmt systems. we release the postediting datasets used in this paper (subescape and subedits) along with pre/post-processing scipts at pedra github repository ( https://github .