Quality Estimation-Assisted Automatic Post-Editing (2023)

Abstract: Automatic Post-Editing (APE) systems are prone to over-correction of the Machine Translation (MT) outputs. While Word-level Quality Estimation (QE) system can provide a way to curtail the over-correction, a significant performance gain has not been observed thus far by utilizing existing APE and QE combination strategies. In this paper, we propose joint training of a model on APE and QE tasks to improve the APE. Our proposed approach utilizes a multi-task learning (MTL) methodology, which shows significant improvement while treating both tasks as a ‘bargaining game’ during training. Moreover, we investigate various existing combination strategies and show that our approach achieves state-of-the-art performance for a ‘distant’ language pair, viz., English-Marathi. We observe an improvement of 1.09 TER and 1.37 BLEU points over a baseline QE-Unassisted APE system for English-Marathi, while also observing 0.46 TER and 0.62 BLEU points for English-German. Further, we discuss the results qualitatively and show how our approach helps reduce over-correction, thereby improving the APE performance. We also observe that the degree of integration between QE and APE directly correlates with the APE performance gain. We release our code and models publicly.

## introduction significant progress in machine translation (mt) and adopting mt systems in the translation industry has led to motivated research on auxiliary tasks. one auxiliary task in the mt field is automatic post-editing (ape), which is aimed toward the automatic identification and correction of translation errors. the ape task is motivated by the need to improve mt output in black-box conditions, where translation models are not accessible for modification or retraining @xcite , which can be costly in terms of time and effort. quality estimation (qe) is another auxiliary task that assesses the quality of mt output in the absence of reference translations @xcite . qe is performed at a word, sentence, and document level. a word-level qe system predicts an ok or bad tag for each token on both source and target segments, while a sentence-level qe assigns a rating (0-100), referred to as a direct assessment (da) score, to the translation denoting its overall quality. both qe and ape have been explored significantly as standalone tasks. over the years, predicting mt quality has been formulated differently (e.g. ranking, scoring, qualitative metrics) and has been dealt with at diverse levels of granularity. consistent advancements in both these tasks make qe and ape more appealing technologies. despite the growing interest in the two tasks and their intuitive relatedness, previous research in both areas has mostly followed separate paths. current qe integration strategies in the mt-ape pipeline have only been applied over statistical phrase-based mt system translations @xcite . our motivation for the current work is: the potential usefulness of leveraging the two technologies to develop a better ape system has been scarcely explored. also, there is no systematic analysis of ape performance when different qe and ape combination strategies are used to postedit translations obtained from a neural machine translation (nmt) system. we hypothesize that qe can help ape in the following manner: a sentencelevel qe output can provide an overall idea of how much editing is required to correct a translation, whereas a word-level qe output can help ape by identifying translation tokens that need editing. our goal is to reduce over-correction in ape by using qe. hence, our contributions are: ## related work during the past decade, there has been tremendous progress in the field of qe and ape, primarily due to the shared tasks organized annually by the conference on machine translation (wmt), since 2012 and 2015, respectively @xcite . @xcite and @xcite have shown that the ape systems have the potential to reduce human effort by correcting systematic and repetitive translation errors. recent ape approaches utilize transfer learning by adapting pretrained language or translation models to perform post-editing @xcite @xcite . also, the recent approaches use multilingual or cross-lingual models to get latent representations of the source and target sentences @xcite . @xcite have shown that gradually adapting pre-trained models to ape by using the curriculum training strategy (cts) improves performance. @xcite have demonstrated that augmenting the ape data by phrase-level ape triplets improves feature diversity and used the cts to train the ape system on high-quality data. recently, in the field of qe, neural-based systems such as deepquest @xcite and openkiwi @xcite have consistently outperformed other approaches in wmt quality estimation shared tasks @xcite . these systems utilize an encoder-decoder recurrent neural network (rnn) architecture, commonly called the 'predictor,' combined with a bidirectional rnn known as the 'estimator,' which generates quality estimates. however, one drawback of these architectures is that they require extensive predictor pre-training, relying on large parallel data and demanding computational resources @xcite . to address this limitation, transquest @xcite emerged as a solution, winning the wmt20 sentence-level qe (da prediction) shared task @xcite . transquest eliminates the need for a predictor by leveraging cross-lingual embeddings. the authors fine-tuned an xlm-roberta model for the sentence-level qe, demonstrating that a simple architecture can achieve stateof-the-art results. subsequently, the transquest framework has been extended to the word-level qe task @xcite . recently, @xcite showed that combining sentence-level and word-level qe systems can help alleviate the problem of inconsistent predictions. martins et al. (2017) used ape outputs to improve qe systems. @xcite used an ensemble of factored nmt models for word-level qe and ape tasks. @xcite compared three combination approaches and showed the potential of qe systems in improving ape on output obtained from a phrase-based mt system. we use these three approaches for comparison in the current work. the winning submission of the wmt22 ape shared task shows the usefulness of a sentence-level qe system in deciding whether ape has improved a translation or not @xcite . ## standalone ape and qe systems this section discusses the approaches used to develop standalone ape and qe systems. we follow the state-of-the-art approaches for training the systems as it allows us to investigate whether the findings of @xcite hold when neural ape and qe systems are used to post-edit and assess the quality of nmt-generated translations. ## progressively integrating qe with ape this section discusses three existing strategies for combining qe with ape. starting with lighter combination approaches, we move towards strongly coupled ones that help ape learn from the qe subtasks (sentence-and word-level qe). ## joint training over qe and ape this approach investigates the tightest coupling between the qe tasks (sentence-and word-level qe) and ape. we follow exactly the same steps in the cts (explained in section 3.1), except the last one, and train the model for the ape task. during the last (fine-tuning) stage of the cts, we jointly train the model on the ape and qe tasks using the real ape training data and the qe training data. to do so, we add the task-specific heads (refer section 3.2) on top of a shared representation layer that receives inputs from the final encoder layers (refer figure 1 ). the representation layer has 2x neurons for en-mr than for en-de. for en-de, its size is equal to the size of the final encoder layer. the following two multi-task learning (mtl) approaches are used to perform the experiments. ## results and analysis this section discusses an evaluation of the qe and ape combination strategies. the performance of the ape systems is reported on the wmt development sets in terms of ter @xcite and bleu @xcite scores. we perform a statistical significance test considering primary metric (ter) using william's significance test @xcite . for comparison, we use two ape baselines as follows: (i) a do-nothing baseline that does not modify original translations, as obtained from the en-mr en-de approach ter bleu ter bleu do-nothing 22.93 64.51 19.06 68.79 ape w/o qe 19.39 68.35 18.91 68.91 ape ->qe 19.01 68.74 18.84 69.00 in the mtl-based experiments, first, we use the ls-mtl approach to perform various experiments using a combination of tasks to see the improvements brought by a simple mtl approach. further, we try to optimize the performance on the best combination of tasks through the nash-mtl approach. ## conclusion and future work in this paper, we have investigated four strategies to integrate qe systems into the mt-ape pipeline, intending to improve ape performance when translations to be post-edited are obtained from an nmt system. experimental results highlight the complementary nature of ape and qe tasks, which can be best utilized through our proposed approach: joint training of a model that learns both tasks simultaneously. the quantitative results and our qualitative analysis suggest that the qe helps ape to address the over-correction problem. among the existing combination strategies, the qe as mt/ape selector improves performance over qe-unassisted ape by discarding low-quality ape outputs. ape guide is a more beneficial approach as we observe even better improvements using the tighter combination strategy. it provides ape with explicit information about the overall translation quality and erroneous segments. moreover, a comparison of these combination strategies shows that tighter coupling between qe and ape is increasingly beneficial for improving ape. we plan to conduct a thorough qualitative analysis of qe-assisted ape systems to gain a deeper understanding of the contribution of qe. we also aim to strengthen the coupling between ape and qe to enhance ape performance further. additionally, we intend to conduct a study to evaluate the potential of ape in improving qe systems.