Smart-Start Decoding for Neural Machine Translation (2021)

Abstract: Most current neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to-left. In this work, we propose a novel method that breaks up the limitation of these decoding orders, called Smart-Start decoding. More specifically, our method first predicts a median word. It starts to decode the words on the right side of the median word and then generates words on the left. We evaluate the proposed Smart-Start decoding method on three datasets. Experimental results show that the proposed method can significantly outperform strong baseline models.

## introduction neural machine translation (nmt) has made remarkable progress in recent years. there has been much progress in encoder-decoder framework, including recurrent neural models @xcite , convolutional models @xcite and self-attention models @xcite . particularly, the transformer, only relying on selfattention networks, has achieved state-of-the-art performance on different benchmarks. most encoder-decoder frameworks generate target translation in a completely monotonic order from left to right (l2r) or from right to left (r2l). however, monotonic generation is not always the best translation order for the machine translation task. as shown in figure 1 , "乐 (happy)" needs to leverage the future context "开朗 (lively)" to make disambiguation of the translation in english sentence, because "乐" has two meanings: "happy to do something" and "le (person name)". in this example, the l2r baseline model produced an incorrect translation of "le (person name)" due to unseen future context. ## smart-start machine translation in this section, we present the details of the proposed hard and soft smart-start methods. our method first predicts a median word and then predicts the words on the right part, and then generates words on the left. ## experiments in this section, we evaluate our method on three popular benchmarks. ## related work neural machine translation (nmt) has attracted a lot of attention recently. the architecture of nmt models has evolved quickly so that there are many different models @xcite @xcite @xcite @xcite . asynchronous and synchronous bidirectional decoding model @xcite exploits the contexts generated in the r2l manner to help the l2r translation. previous non-monotonic methods @xcite @xcite @xcite jointly leverage l2r and r2l information. non-monotonic methods are also widely used in many tasks @xcite , such as parsing @xcite , image caption @xcite , and dependency parsing @xcite . similarly, insertion-based method @xcite predicts the next token and its position to be inserted. ## conclusion in this work, we propose a novel method that breaks up the limitation of these decoding orders, called smart-start decoding. our method predicts a median word and then generates the words on the right part. finally, it generates words on the left. experimental results show that our smart-start method significantly improves the quality of translation.