NHK ’s Lexically-Constrained Neural Machine Translation at WAT 2021 (2021)

Abstract: This paper describes the system of our team (NHK) for the WAT 2021 Japanese-English restricted machine translation task. In this task, the aim is to improve quality while maintaining consistent terminology for scientific paper translation. This task has a unique feature, where some words in a target sentence are given in addition to a source sentence. In this paper, we use a lexically-constrained neural machine translation (NMT), which concatenates the source sentence and constrained words with a special token to input them into the encoder of NMT. The key to the successful lexically-constrained NMT is the way to extract constraints from a target sentence of training data. We propose two extraction methods: proper-noun constraint and mistranslated-word constraint. These two methods consider the importance of words and fallibility of NMT, respectively. The evaluation results demonstrate the effectiveness of our lexical-constraint method.

## introduction our team (nhk) participated in the restricted machine translation task 1 using the japanese-english dataset of the asian scientific paper excerpt corpus (aspec-je) @xcite at wat 2021 @xcite . in this task, the aim is to improve translation quality while preserving consistent terminology for translating scientific papers that include technical terms and proper nouns. in this task, a list of target words is given for each source sentence to appear in a target sentence. figure 1 shows the overview of this task. there are two evaluation criteria: the 1 https://sites.google.com/view/restricted-translation-task/ source sentence: この回路は，⼊⼒信号位相の変化により 共振周波数がシフトする帰還回路であり，２基のコイル の中央にある物体の磁気特性の変化を，⾼い感度と分解 能で検出することができる。 target-vocabulary list: {magnetic features, resonance frequency, feedback circuit, resolution} output sentence is required to contain all the target words in each target-vocabulary list. this is a feedback circuit shifting resonance frequency by change of input signal phase, which can detect change of magnetic features of an object present at a center of two coils on high sensitivity and resolution. translation accuracy via bilingual evaluation understudy @xcite (bleu score) and the consistency score of the ratio of sentences satisfying an exact match of given constraints (consistency score). the final ranking is determined by the combined score of both: calculating bleu with only the exact match sentences foot_0 . in related work @xcite @xcite @xcite , since it does not require higher computational complexity than the other methods using the grid beam search (gbs) decoding algorithm @xcite , we use the lexicalconstraint method of @xcite . this method concatenates a source sentence and constrained words with a special token to input them into an encoder of the neural machine translation (nmt). in addition to the merit of reducing the computational cost compared with gbs decoding, this method has two other merits: no need to modify the architecture of the nmt system or prepare any word alignment data. in this method for this task, one of the main problems is how to extract constraints from training data since only constrained word lists for dev, devtest, and test sets are provided to participants. in this paper, we propose extracting constraints from target sentences on the basis of propernoun and mistranslated-word constraints considering the importance of words and fallibility of nmt. the former constraint is a list of proper nouns extracted with named-entity recognition. the latter constraint is a list of words mistranslated or under-translated with vanilla nmt compared with a target sentence. we conducted experiments to evaluate the nmt using the proposed method and found that the proposed method outperformed a baseline lexical-constraint method. ## nmt with lexical constraint borrowing chen et al. (2020a)'s idea, we implemented a lexically-constrained nmt with encoder and decoder modules. we concatenated a source sentence and constrained words with a special token to input into the encoder, as illustrated in figure 2 . the key to the successful lexicallyconstrained nmt is the way to extract constraints from a target sentence. though the constraints are given for the dev, devtest, and test data, they are not given for the training data. in this paper, we focus on the way to extract a constraint from the target sentence in training data for the training phase. the simplest method of extracting a lexical constraint is randomly sampling words from the target sentence, as @xcite did. beyond the random sampling method, we propose two other directions with a focus on proper nouns and mistranslated words to extract the constrained words automatically from the target sentence. training phase: extracted constraints from training data by the proposed methods. translation phase: given constraints by organizers. figure 2 : overview of nmt using lexical-constraint method. @xmath0, x 2 , ..., x k ), @xmath1, c 2 , ..., c n ), and @xmath2, y 2 , ..., y j ) show source-, constraint-, and predicted-sequences, respectively. k and j are the lengths of source and target sentences. n is a number of constrained words. "|" is a special token for delimiter. during the training phase, constraints are extracted from training data by the proposed methods. during the translation phase, constraints are given by wat 2021 organizers. the proper-noun constraint is not enough to be sufficient to cover all constraints in this task. given constrained words including the proper-noun constraints accounted for 21% of the japanese dev data. to increase the number of appropriate constrained words, we extract mistranslated or dropped words by nmt as constraints. first, we trained an nmt model on parallel training data, and translated the source sentences in training data with this model. we then picked out the words that do not appear in the translated sentence but appear in the target sentence. both proper-noun and mistranslatedword constraints could cover 38% of constraints for the dev data. the remaining 62% constrained words could be translated correctly without adding them as constraints. ## conclusion we described our proposed method using lexical constraints for a japanese↔english restricted machine translation task with the asian scientific paper excerpt corpus (aspec). we proposed a method to extract appropriate constraints of the lexically-constrained neural machine translation (nmt) for this task. our proposed method using the proper-noun and mistranslated-word constraints improved translation performance compared with random-word constraint. for future work, we plan to apply the proposed constraints into nmt with a grid beam search decoding algorithm @xcite to compare the performance.