Machine Translation with Pre-specified Target-side Words Using a Semi-autoregressive Model (2021)

Abstract: We introduce our TMU Japanese-to-English system, which employs a semi-autoregressive model, to tackle the WAT 2021 restricted translation task. In this task, we translate an input sentence with the constraint that some words, called restricted target vocabularies (RTVs), must be contained in the output sentence. To satisfy this constraint, we use a semi-autoregressive model, namely, RecoverSAT, due to its ability (known as “forced translation”) to insert specified words into the output sentence. When using “forced translation,” the order of inserting RTVs is a critical problem. In this work, we aligned the source sentence and the corresponding RTVs using GIZA++. In our system, we obtain word alignment between a source sentence and the corresponding RTVs and then sort the RTVs in the order of their corresponding words or phrases in the source sentence. Using the model with sorted order RTVs, we succeeded in inserting all the RTVs into output sentences in more than 96% of the test sentences. Moreover, we confirmed that sorting RTVs improved the BLEU score compared with random order RTVs.

## introduction in this study, we tackle a machine translation task called "restricted translation." this task requires the output sentence to contain all the pre-specified restricted target vocabularies (rtvs) 1 . in other words, we are given a source sentence and a set of rtvs, and we are supposed to generate an output sentence that contains all the rtvs in the set foot_1 . since the emergence of neural machine translation (nmt) models @xcite @xcite , several studies have been conducted to explore nmt systems capable of decoding translations under terminological constraints @xcite @xcite . however, these previous studies were conducted under the condition that a bilingual dictionary is given. moreover, these challenges are limited to autoregressive nmt systems, and scant research has been conducted on non-autoregressive or semiautoregressive nmt systems, which have received more attention recently. to accomplish restricted translation, where only target terminologies are given, we used a semiautoregressive model called @xcite , which generates a sentence as a sequence of segments. in this model, the segments are generated simultaneously, and each segment is predicted token-by-token. ran et al. (2020) also attempted to force the model to generate a certain token at the beginning of a segment and showed that the model could generate valid sentences under the constraint. then, we considered whether this model could be applied to generate sentences containing rtvs. when tackling this task using this model, the insertion order of rtvs is a critical issue. to address this issue, we used giza++ @xcite to obtain word alignments and then identify the source position corresponding to the rtvs. subsequently, we inserted them in the order in which their corresponding source tokens appear. we confirmed that sorting rtvs with giza++ improved the bleu @xcite score. finally, by using this model, we achieved all the rtvs outputs in more than 96% of the test sentences. following @xcite , we used forward-translation to refine the latter half of the aspec training data. in the same manner as their method, we first trained a japanese-to-english translation model on the first 1.5m sentences of the aspec training data. subsequently, we used the trained model to translate the latter 1.5m japanese sentences of the aspec training data and obtained refined english sentences. finally, we combined the first 1.5m training data and the refined 1.5m training data and trained a japanese-to-english translation model. ## conclusions we introduced a semi-autoregressive approach to tackle the restricted translation task. in our experiments, we showed that recoversat could output almost all the rtvs. additionally, we used source sentence alignment to determine the insertion position and observed that it improved the bleu score. moreover, the importance of the order of the rtvs was confirmed by the fact that the score was considerably improved by inserting rtvs in the order in which they appear in the reference translations. however, there is still room for improvement in determining the insertion order. in future work, investigating how to determine the best order to insert rtvs will be necessary.