TMU NMT System with Automatic Post-Editing by Multi-Source L evenshtein Transformer for the Restricted Translation Task of WAT 2022 (2022)

Abstract: In this paper, we describe our TMU English–Japanese systems submitted to the restricted translation task at WAT 2022 (Nakazawa et al., 2022). In this task, we translate an input sentence with the constraint that certain words or phrases (called restricted target vocabularies (RTVs)) should be contained in the output sentence. To satisfy this constraint, we address this task using a combination of two techniques. One is lexical-constraint-aware neural machine translation (LeCA) (Chen et al., 2020), which is a method of adding RTVs at the end of input sentences. The other is multi-source Levenshtein transformer (MSLevT) (Wan et al., 2020), which is a non-autoregressive method for automatic post-editing. Our system generates translations in two steps. First, we generate the translation using LeCA. Subsequently, we filter the sentences that do not satisfy the constraints and post-edit them with MSLevT. Our experimental results reveal that 100% of the RTVs can be included in the generated sentences while maintaining the translation quality of the LeCA model on both English to Japanese (En→Ja) and Japanese to English (Ja→En) tasks. Furthermore, the method used in previous studies requires an increase in the beam size to satisfy the constraints, which is computationally expensive. In contrast, the proposed method does not require a similar increase and can generate translations faster.

## introduction we participated in the restricted translation task at wat 2022. in this task, we were given pairs of an input sentence and a list of restricted target vocabularies (rtvs), wherein words or phrases are stored in a random order. next, we were asked to generate a translated sentence for the input sentence that contained all the rtvs in the corresponding list. this setting is intended for cases where a user wishes to translate technical terms or proper nouns consistently by specifying these words in advance. previous studies have shown that neural machine translation (nmt) models @xcite @xcite exhibit high translation performance in machine translation. additionally, studies to control output in nmt under terminological constraints have been conducted @xcite @xcite . however, several of these studies were set up to be available a bilingual dictionary rather than only the desired words to output. in the previous year, the first shared task of restricted translation was performed, for which @xcite achieved the highest score @xcite . their proposed method combines a "soft" method (which does not ensure constraint satisfaction using data augmentation @xcite ) and a "hard" method (which ensures constraint word satisfaction using grid beam search @xcite ). their results revealed that certain constraint terms could be satisfied with only soft methods. we speculated whether the constraints could be satisfied by correcting those that were not satisfied by automatic post-editing. in this study, we tackled this task in two generation steps. first, we generated the translation by a soft method (lexical-constraint-aware nmt (leca)). next, we filtered the sentences that did not satisfy the constraints and post-edited those with multi-source levenshtein transformer (mslevt) @xcite . in general, hard methods employs a computationally expensive decoding algorithm compared with conventional beam search. we adopted mslevt, an efficient non-autoregressive model, as the automatic posteditting from the perspective of computational complexity. in addition, while performing post-editing in mslevt, rtvs were provided as initial values. subsequently, the sentences were generated with repeated modifications according to the levenshtein transformer process. the restriction of delete and insert operations to rtvs ensured that rtvs would appear in the output in the order provided as the initial value. consequently, we had to determine the order of the rtvs in advance. we used the cosine similarity of the embedding of each word in leca's generated text and rtvs, which were obtained using fasttext @xcite , to determine the order of the rtvs. we submitted the system outputs to the @xmath0. we successfully included 100% of the constraint words in the system's output without significantly compromising the bleu score of the leca model. we confirmed the effectiveness of the proposed method in reordering constraint words by calculating spearman's rank correlation coefficient for the reordered constraint words and the constraint words in the reference. ## system overview first, we used a baseline model called lexicalconstraint-aware nmt @xcite , for translation that considers constraint words. however, because this method did not ensure that constraint words would appear in the generated text, automatic post-processing correction was performed on the sentences that failed to satisfy the constraints in the leca output to ensure that the constraints were satisfied. the automatic correction was performed by reordering the rtvs using fasttext @xcite and then, using mslevt @xcite . ## conclusion we introduced an automatic post-editing approach for the restricted translation task of wat 2022. in our experiments, 100% of the rtvs could be included in the generated sentences while maintaining the translation quality of leca. furthermore, our method does not require any preliminary experiments to determine the beam size and can generate translations faster while satisfying constraints compared with existing methods using grid beam search.