I ntelli CAT : Intelligent Machine Translation Post-Editing with Quality Estimation and Translation Suggestion (2021)

Abstract: We present IntelliCAT, an interactive translation interface with neural models that streamline the post-editing process on machine translation output. We leverage two quality estimation (QE) models at different granularities: sentence-level QE, to predict the quality of each machine-translated sentence, and word-level QE, to locate the parts of the machine-translated sentence that need correction. Additionally, we introduce a novel translation suggestion model conditioned on both the left and right contexts, providing alternatives for specific words or phrases for correction. Finally, with word alignments, IntelliCAT automatically preserves the original document’s styles in the translated document. The experimental results show that post-editing based on the proposed QE and translation suggestions can significantly improve translation quality. Furthermore, a user study reveals that three features provided in IntelliCAT significantly accelerate the post-editing task, achieving a 52.9% speedup in translation time compared to translating from scratch. The interface is publicly available at https://intellicat.beringlab.com/ .

## introduction existing computer-aided translation (cat) tools incorporate machine translation (mt) in two ways: post-editing (pe) or interactive translation prediction (itp). pe tools @xcite provide a machine-translated document and ask the translator to edit incorrect parts. by contrast, itp tools @xcite @xcite aim to provide translation suggestions for the next word or phrase given a partial input from the translator. a recent study with human translators revealed that pe was 18.7% faster than itp in terms of translation time @xcite and required fewer edits @xcite . however, many translators still prefer itp over pe because of (1) high cognitive loads @xcite and (2) the lack of subsegment mt suggestions @xcite in pe. in this paper, we introduce intellicat 1 , a hybrid cat interface designed to provide pe-level efficiency while retaining the advantages of itp, such as subsegment translation suggestions. to mitigate the cognitive loads of human translators, intelli-cat aims to automate common post-editing tasks by introducing three intelligent features: (1) quality estimation, (2) translation suggestion, and (3) word alignment. quality estimation (qe) is the task of estimating the quality of mt output without reference translations @xcite . we integrate qe into the cat interface so that the human translator can easily identify which machine-translated sentences and which parts of the sentences require corrections. furthermore, for words that require postediting, our interface suggests possible translations to reduce the translators' cognitive load. finally, based on word alignments, the interface aligns the source and translated documents in terms of formatting by transferring the styles applied in the source document (e.g., bold, hyperlink, footnote, equation) to the translated document to minimize the post-editing time. our contributions are: the experimental results on the wmt 2020 english-german qe dataset show that post-editing with the proposed qe and translation suggestion models could significantly improve the translation quality (-6.01 ter and +6.15 bleu). moreover, the user study shows that the three features provided by intellicat significantly reduce postediting time (19.2%), which led to a 52.6% reduction in translation time compared to translating from scratch. finally, translators evaluate our interface to be highly effective, with a sus score of 88.61. translation suggestion translation suggestions from interactive translation prediction (itp) @xcite @xcite are conditioned only on the left context of the word to be inserted. therefore, itp has intrinsic limitations in post-editing tasks where the complete sentence is presented, and the right context of the words that need correction should also be considered. we propose a novel translation suggestion model in which suggestions are conditioned on both the left and right contexts of the words or phrases to be modified or inserted to provide more accurate suggestions when post-editing the complete sentence. cross-lingual language model cross-lingual language models (xlms), which are language models pre-trained in multiple languages, have led to advances in mt @xcite and related tasks such as qe @xcite , automatic post-editing @xcite , and parallel corpus filtering @xcite . accordingly, our qe and translation suggestion models are trained on top of xlm-r @xcite , an xlm that shows state-of-the-art performance for a wide range of cross-lingual tasks. to the best of our knowledge, intellicat is the first cat interface that leverages xlm to assist human post-editing for mt outputs. ## conclusion and future work in this paper, we introduce intellicat, an intelligent mt post-editing interface for document translation. the interface provides three neural networkbased features to assist post-editing: (1) sentencelevel and word-level qes, (2) alternative translation suggestions for words or phrases, and (3) automatic formatting of the translated document based on word alignments. the model evaluation shows that post-editing based on the proposed qe and translation suggestion models can significantly improve the quality of translation. moreover, the user study shows that these features significantly accelerate post-editing, achieving a 52.9% speedup in translation time compared to translating from scratch. finally, the usability of intellicat was evaluated as an "excellent" level, with a sus score of 88.61. in future work, we will build a pipeline that continuously improves the performance of neural models based on automatically collected triplets consisting of source, mt, and post-edited sentences. we will implement an automatic post-editing (chatterjee et al., 2020) model to continuously improve mt performance and apply online learning to qe models to continually enhance qe performance.