L evenshtein Training for Word-level Quality Estimation (2021)

Abstract: We propose a novel scheme to use the Levenshtein Transformer to perform the task of word-level quality estimation. A Levenshtein Transformer is a natural fit for this task: trained to perform decoding in an iterative manner, a Levenshtein Transformer can learn to post-edit without explicit supervision. To further minimize the mismatch between the translation task and the word-level QE task, we propose a two-stage transfer learning procedure on both augmented data and human post-editing data. We also propose heuristics to construct reference labels that are compatible with subword-level finetuning and inference. Results on WMT 2020 QE shared task dataset show that our proposed method has superior data efficiency under the data-constrained setting and competitive performance under the unconstrained setting.

## introduction quality estimation (qe) is the task of estimating the quality of translation without access to a humangenerated reference. most recent advances on quality estimation @xcite @xcite , inter alia) focus on estimating the quality of translation on either the corpus or segment-level. however, in practice, the end-users of machine translation (mt) often call for quality signals on more fine-grained level-the level of individual words in a translation. such signals are not only useful for more fine-grained triage of translation quality, but also open up the potential for targeted post-processing and faster human post-editing. to automatically assess the quality of translations, it is natural to consider starting from a machine translation model, which has already acquired the translation knowledge. however, it is not @xcite vaccinations put ends to the pandemic ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? vaccinations put an end to the pandemic ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ok ok ok bad bad ok ok ok ok ok ok t arget post-edit ter (b) vaccinations put ends to the pandemic ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? no no yes no no no prev. t arget delet ion mask ins. w ord ins. 0 2 0 0 vaccinations put an end to the pandemic clear how to best transfer the translation knowledge to a word-level quality estimation setting, since autoregressive translation models only see the preceding context of the word they generate. hence, they are not well-equipped to perform word-level quality assessments or edits on an existing translation, as there are both preceding and succeeding context for a translated word. towards this goal, we leverage levenshtein transformer (levt, @xcite , a non-autoregressive neural machine translation (nmt) model trained to generate translations by starting with an empty output sequence and iteratively performing edits on the sequence. because of this special training and decoding procedure, the model should have already learned to edit an existing translation sentence without supervision from post-edited translations. we then use multi-stage transfer learning to teach the model to perform the actual qe task, first on artificially-crafted pseudo post-editing, then on real human post-edited data. we show that our method achieves better performance than the currently widely adopted predictor-estimator scheme @xcite under the data-constrained setting, while also being competitive when compared with the highranking submissions to the wmt 2020 word-level qe shared task @xcite under the unconstrained setting. ## transfer learning from translation to word-level quality estimation by training for the translation task, levt already acquired some initial knowledge of post-editing. however, there is still some train/inference time mismatch for the word-level qe task regarding (1) the target-side context and (2) the edit tags. in terms of target-side context, during training, the target-side context is a noisy version of the real target sentence in the training set; during inference, the target-side context is a translation output from an nmt system. in terms of edit tags, during training, we want the model to predict lcs edit tags that correct the noisy version of the target sentence; during inference, we want the model to predict ter-style (levenshtein) edit tags that correspond to human post-editing of nmt outputs. to address such mismatch, we adopt a two-step transfer learning scheme. for both stages of transfer learning, we need translation triplets (source, mt output, post-edited output) to perform finetuning. however, in practice, human post-editing resources are quite scarce. hence, like some previous work @xcite , we start by performing transfer learning on synthetic translation triplets, followed by real translation triplets constructed with human post-editing. synthetic data construction we explore four different methods for translation triplet synthesis: • src-mt-ref take a parallel dataset @xcite and translate the source sentence with an mt model (mt). with λ t and λ p as the interpolation weights. similarly, beam search can also be performed on top of the ensemble. the intuition behind this idea is that such ensemble should create a target sentence that is semantically equivalent to the target side of the parallel corpus, while being close to the original mt output as much as possible, imitating the way humans perform the post-editing task. compatibility with subwords to the best of our knowledge, previous work on word-level quality estimation either builds models that directly output word-level tags @xcite @xcite or uses simple heuristics to re-assign word-level tags to the first subword token @xcite . since levt predicts edits on a subword-level starting from translation training, we need to: (1) for inference, convert subwordlevel tags predicted by the model to word-level tags for evaluation, and (2) for both finetuning stages, build subword-level reference tags. for inference, the conversion can be easily done by heuristics. for finetuning, a naive subword-level tag reference can be built by running ter alignments on mt and post-edited text after subword tokenization. however, a preliminary analysis shows that such reference introduces a 10% error after being converted back to word-level. hence, we introduce another heuristic to create heuristic subwordlevel tag references. the high-level idea is to interpolate the word-level and the naive subword-level references to ensure that the interpolated subwordlevel tag reference can be perfectly converted back to the word-level references. details for the subword tag conversions can be found in appendix a and algorithm 1 and 2 in the appendix. ## conclusion in this work, we proposed to use levenshtein transformer to substitute the usual mlm-style training in the predictor-estimator framework as the initial training step. we also proposed a series of techniques to effectively transfer the translation knowledge to the word-level qe task, including data synthesis, heuristic subword-level reference, and incorporating pre-trained translation models. our results demonstrate superior data efficiency under the data-constrained setting and competitive performance under the unconstrained setting. we also hope this work can inspire further exploration for other uses of levenshtein transformer apart from the non-autoregressive translation.