IIGROUP Submissions for WMT 22 Word-Level A uto C ompletion Task (2022)

Abstract: This paper presents IIGroup’s submission to the WMT22 Word-Level AutoCompletion(WLAC) Shared Task in four language directions. We propose to use a Generate-then-Rerank framework to solve this task. More specifically, the generator is used to generate candidate words and recall as many positive candidates as possible. To facilitate the training process of the generator, we propose a span-level mask prediction task. Once we get the candidate words, we take the top-K candidates and feed them into the reranker. The reranker is used to select the most confident candidate. The experimental results in four language directions demonstrate the effectiveness of our systems. Our systems achieve competitive performance ranking 1st in English to Chinese subtask and 2nd in Chinese to English subtask.

## introduction recent advances in neural machine translation @xcite allow us to generate high-quality translation results. however, as it's pointed out by @xcite that in some scenarios(e.g., legal instruments), the results of machine translation can't directly replace human translations due to their imperfections(e.g., terminology translation error). therefore, more and more researchers pay attention to computeraided translation(cat) @xcite @xcite , which focuses on leveraging the advantages of nmt systems to increase the effectiveness and efficiency of the human translation process. to further promote the development of cat, wmt22 proposes a novel task --word-level autocompletion(wlac). in the word-level auto-completion task, given a source sentence x, target context and human-typed characters t, an ideal system is expected to be able to predict the target word w that should be placed in the target context. we participate in the wmt22 shared word-level autocompletion task in four language directions: chinese ⇒ english, english ⇒ chinese, german ⇒ english and english ⇒ german and submit a system for each language direction. we develop a generate-then-rerank frameworkbased system for each language direction. based on the vanilla transformer architecture, we adopt a bidirectional decoder, which can predict the current target word by paying attention to the source sentence and both the left-side and right-side target context. the paper is organized as follows, section 2 gives the overview of the data used in the shared task and preprocessing operations for the data, while section 3 describes our training techniques, including model architecture, span-level mask prediction, etc. section 4 presents our experimental results. finally, our conclusions are summarized in section 5. ## data in this section, we first introduce the datasets used to train our systems, then we introduce how to prepare the simulated training data for the wlac shard task and describe the vocabulary for each language direction. ## word-level autocompletion systems in this section, we mainly introduce the generatethen-rerank framework. both the generator and the reranker's architecture are based on transformer @xcite with the modification that the decoder is bi-directional to leverage more context information. it is important to note that we borrow the idea from @xcite that we view wlac as a word prediction task and only use human-typed characters t as hard constraints. ## experimental results we report experimental results in four language directions: chinese ⇒ english, english ⇒ chinese, german ⇒ english and english ⇒ german. the performance of the generator is as expected, and as demonstrated in @xcite , without using the bi-directional decoder, the generator performs relatively poorly. additionally, we conduct an ablation study on chinese ⇒ english subtask to demonstrate the effectiveness of the span-level mask prediction, the model without leveraging the span-level mask prediction strategy performs poorly, with a drop of -10.1 in accuracy on the validation set. however, the performance of the reranker is not as expected. we conjecture that this is due to the insufficiency of the training procedure of the reranker. initializing reranker's weights with generator's weights or with plm's weight will boost the performance of reranker, we leave this as future work. ## conclusion this paper describes the iigroup's systems submitted to the word-level autocompletion task at wmt22. we adopt a generate-then-rerank framework. the experimental results demonstrate the effectiveness of the generator. however, due to the lack of computing power and time, the results of our experiments don't show the effectiveness of our reranker. we discuss this issue in section 4 and we will try to solve this in future work.