B i TIIMT : A Bilingual Text-infilling Method for Interactive Machine Translation (2022)

Abstract: Interactive neural machine translation (INMT) is able to guarantee high-quality translations by taking human interactions into account. Existing IMT systems relying on lexical constrained decoding (LCD) enable humans to translate in a flexible translation order beyond the left-to-right. However, they typically suffer from two significant limitations in translation efficiency and quality due to the reliance on LCD. In this work, we propose a novel BiTIIMT system, Bilingual Text-Infilling for Interactive Neural Machine Translation. The key idea to BiTIIMT is Bilingual Text-infilling (BiTI) which aims to fill missing segments in a manually revised translation for a given source sentence. We propose a simple yet effective solution by casting this task as a sequence-to-sequence task. In this way, our system performs decoding without explicit constraints and makes full use of revised words for better translation prediction. Experiment results show that BiTiIMT performs significantly better and faster than state-of-the-art LCD-based IMT on three translation tasks.

## introduction recent years have witnessed significant development in neural machine translation(nmt) @xcite . despite their success, their translation in quality still can not meet the requirements in industrial applications. on the other hand, interactive neural machine translation (imt) @xcite @xcite @xcite @xcite is able to guarantee high-quality translation: it is an iterative collaboration process between human and machine that involves multiple interactive steps to obtain a satisfactory translation. traditional imt generates a translation in the left-to-right completing paradigm @xcite @xcite @xcite where human translators are required to revise words in the translation prefix. this strict leftto-right manner limits its flexibility because some human translators may enjoy their translation manners beyond left-to-right. as a result, another part of works @xcite propose an alternative imt paradigm under which human translators can revise words at arbitrary positions of a translation. the essential technique to this paradigm is lexically constrained decoding (lcd) @xcite , which extends beam search in the decoding stage to include revised words as constraints. unfortunately, lcd-based imt suffers from two major shortcomings on efficiency and translation quality in practice. firstly, lcd-based imt usually involves multiple interactions between a human translator and machine and runs the lcd algorithm multiple times to translate a sentence. since each lcd run takes considerable time compared with nmt decoding, the human translator will encounter severe latency, leading to poor user experience. in addition, lcd is based on the standard translation model, which is defined on top of the prefix context, and thus cannot make use of the revised words to assist the model in predicting target words to their left. hence this characteristic limits its overall translation quality. this paper proposes a simple yet effective imt approach, bitiimt, which addresses the issues above. the core idea to bitiimt is the bilingual text-infilling (biti) task which extends textinfilling @xcite from monolingual setting to bilingual setting and aims to fill missing segments in a revised translation for a given source sentence. unlike @xcite carefully designing a model, we simply cast the bilingual text-infilling task as a sequence-to-sequence task and then employ the standard nmt model to perform this task. to train the model, we construct simulated data by randomly sampling revised sentences from a bilingual corpus and augment the simulated data with bilingual corpus for further improvements. in this way, our model is able to yield a valid output that can be seamlessly filled in the revised translation in an efficient way similar to the standard nmt decoding. moreover, the proposed model makes full use of all revised words to predict a target word and thus has the potential to obtain better translation than lcd. we conduct extensive experiments on wmt14 en-de, wmt14 en-fr, and zh-en tasks. our simulated experiments demonstrate that the proposed model indeed outperforms lcd in terms of translation quality and efficiency, and the proposed bitiimt is better than lcdimt according to both translation quality and human editing costs. the advantages of bitiimt over lcdimt are also verified in real-world imt experiments with human translators. this paper makes the following contributions: ## proposed bitiimt this section illustrates the overview of the proposed bitiimt system and accordingly presents its essential technique (i.e., bilingual text-infilling) to take human interactions into nmt. ## experiments following previous works @xcite @xcite , we experiment on two simulated scenarios and a real-world scenario. ## related work since the period of statistical machine translation (smt), imt has been widely exploited to reduce human effort by using human's feedback to improve translation quality @xcite @xcite @xcite @xcite . recently, with the development of nmt @xcite , researchers turned to employ imt on it @xcite . a classical type of imt uses a left-to-right sentence completing framework proposed in @xcite , in which human translators can only revise the translation generated by models from left to right. generally, the text portion from the beginning to the current modified part is called prefix, and the system will generate a new translation based on the given prefix @xcite @xcite . @xcite propose a pick-revise framework that enables translators to do revisions on arbitrary positions to improve efficiency. @xcite allow users to make any interaction at random positions by using lcd algorithms @xcite in the decoding stage which can integrate lexical constraints into translation. however, lcd can not achieve a win-win result in terms of decoding speed and translation quality. @xcite propose a bidirectional imt framework on top of lcd, which could fix mistakes by using two constrained decoding procedures with opposite directions. however, it needs to train two decoders, and in each con-methods bleu decoding time cost editing cost rounds other works @xcite @xcite apply alignment information to improve the decoding efficiency of lcd. @xcite use alignment extracted by vanilla transformer, which is poor as argued by @xcite . @xcite apply an external aligner to train the alignment module. these works can only perform constrained decoding based on a constraint pair, which means a burden for human translators. in order to address the issue of decoding speed for lcd, some works use a non-autoregressive approach to integrate constraints. @xcite propose levenshtein transformer @xcite to inject terminology constraints at inference time. @xcite propose a novel re-position operator to replace deletion in levenshtein transformer to exploit lexical constraints more effectively and efficiently. however, nonautoregressive models are still worse than autoregressive models in translation quality currently. compared to these efforts in nat, bitiimt is essentially based on an auto-regressive translation model. ## conclusion traditional imt systems often use lcd to incorporate manually revised words into translations. in this paper, we propose bitiimt, a novel imt method that outperforms lcd-based imt in both translation quality and efficiency. the key to biti-imt is the bilingual text-infilling task which extends text-infilling from a monolingual setting to a bilingual one. we cast this task as a sequence-tosequence task and propose a simple yet effective solution to address it. experiments show that biti-imt achieves a significantly improved efficiency in the area of imt.