Rethinking Word-Level Auto-Completion in Computer-Aided Translation (2023)

Abstract: Word-level auto-completion (WLAC) plays a crucial role in Computer-Assisted Translation. While previous studies have primarily focused on designing complex model architectures, this paper takes a different perspective by rethinking the fundamental question: what kind of words are good auto-completions? We introduce a measurable criterion to address this question and discover that existing WLAC models often fail to meet this criterion. Building upon this observation, we propose an effective approach to enhance WLAC performance by promoting adherence to the criterion. Notably, the proposed approach is general and can be applied to various encoder-based architectures. Through extensive experiments, we demonstrate that our approach outperforms the top-performing system submitted to the WLAC shared tasks in WMT2022, while utilizing significantly smaller model sizes.

## introduction in recent years, more and more researchers have studied computer-aided translation (cat) that aims to assist human translators to translate the input text @xcite @xcite @xcite . the wordlevel auto-completion (wlac) task @xcite is the core function of cat, which involves predicting the word being typed by the translator given the translation context, as illustrated in figure 1 . effective auto-completion has the potential to reduce keystrokes by at least 60% during the translation process @xcite . a figure 1 : an example of word-level auto completion. assume the human translator is going to input the golden translation. the auto-completion suggests the possible word candidates given the typed characters. it can be more accurate with the help of translation hypothesis from mt models. user survey indicates that 90.2% of participants find the word-level auto-suggestion feature helpful @xcite . therefore, wlac plays an important role in cat. there are many existing methods for modeling wlac, and they mainly differ in model architectures @xcite @xcite @xcite . for example, @xcite ; @xcite design a bert-like architecture to directly predict the target word while @xcite employ a model similar to the auto-regressive nmt to predict the bpe tokens of the target word. take figure 1 as an example, the task of wlac can be described as: given the source sentence (the chinese sentence) , the partial translated sentence ("that's one small") and the typed character sequence ("s"), predict the word w that the human translator is going to input ("step"). this paper goes beyond model architectures and reconsiders the essence of wlac, posing a fundamental question ( §3): what defines a correct word w? theo-retically, a good w should appear in the reference translation, as illustrated in figure 1 . however, in practice, this criterion is not directly applicable for improving the performance of wlac because the reference translation is unavailable during inference. therefore, we attempt to relax this criterion by replacing the reference with the output from a trained machine translation system. the experimental results show that the relaxed criterion is reasonable: the predicted w that are present in the machine-translated hypothesis exhibit significantly higher accuracy compared to those absent in the hypothesis. furthermore, we find that existing wlac models usually fail to meet the relaxed criterion (see based on the above finding, this paper presents a novel approach to enhance wlac systems, which can be applied to different model architectures ( §4). the key idea is to encourage wlac's output to satisfy the relaxed criterion as much as possible, i.e., enhancing the agreement between wlac and mt system such that the output w exists in the machinetranslated hypothesis. guided by this principle, two distinct methods are proposed. the first method achieves this agreement during the inference phase. this method requires running mt decoding during inference, which is inefficient. moreover, its performance is limited by the quality of mt. to alleviate both issues, we introduce the second method, which achieves agreement through joint training of wlac and mt. the intuition behind this method is that at the training stage, we generate the training data for the wlac task from parallel corpus, so the wlac's ground truth naturally exists in the reference translation. by jointly training the two models, we can implicitly learn the agreement between wlac and mt. notably, this method bypasses the need for mt decoding during inference, resulting in improved efficiency and removing the constraints imposed by mt's quality. the effectiveness of our proposed method is validated through experiments on the four language directions of the wlac shared task in wmt22 ( §5). our approach achieves substantial improvements across two distinct backbone models. notably, it surpasses the performance of the wmt22's best system with only 20% of the parameters and much simpler architecture. this paper makes the following contributions: the wlac task comes from a real translation scenario: a human translator is translating a source sentence, who has already translated part of the sentence, and is typing a new word. the input contains three parts: the source sentence s, the partial translation c, and the typed character sequence t. as shown in figure 1 , the wlac task is to predict the word w that the translator is going to input @xcite ,formulate as: in this section, we introduce two types of backbone models for the wlac task. these backbone models serve as the foundations for our proposed techniques and experiments in the subsequent sections. ## what is a good prediction in wlac? in this paper, we rethink the wlac task by posing a fundamental question: what is a "good" prediction given the source sentence s, the context c and a typed character sequence t? in the subsequent sections, we explore the criterion to assess a candidate predictions and use it to improve the performance during the inference. ## enhancing wlac with mt by agreement motivated by the findings from section 3, we propose two different approaches which improve the agreement between wlac and machine translation for overall improvements in wlac performance. ## analysis & discussion the experiment results have demonstrated the effectiveness of our method, but how does the mt task help wlac? in this section, we conduct de- ## conclusion this paper firstly rethinks the essence of the "correct" prediction for wlac and we propose agreement, a measurable criterion to judge whether a prediction word is good or not. surprisingly, we find that existing wlac models usually violate this criterion with a high probability. based on this findings, we present two effective strategies: joint inference and joint training with machine translation (mt), aimed at enhancing wlac performance by encouraging the model to adhere to the agreement criterion. extensive experiments show that the proposed approach surpasses the best system submitted to the wlac shared tasks in wmt2022, with much smaller model size. additionally, our result analysis highlights the effective integration of mt knowledge into the wlac task, resulting in improved performance. the error analysis also reveals the directions for further optimization, which we will leave as future work. and the generalizability of our findings to other languages may vary. while our main experiments have demonstrated the effectiveness of the proposed joint training method, it is essential to examine the improvement analysis and error analysis for specific linguistic phenomena, such as the prediction of german grammatical gender. to gain a comprehensive understanding of the effectiveness of the joint training approach, our future work includes extensive experiments on multiple languages. lastly, the current wlac task primarily focuses on high-resource languages. exploring the performance of our method in low-resource scenarios is also an important area for future investigation.