Transformer-based Double-token Bidirectional Autoregressive Decoding in Neural Machine Translation (2020)

Abstract: This paper presents a simple method that extends a standard Transformer-based autoregressive decoder, to speed up decoding. The proposed method generates a token from the head and tail of a sentence (two tokens in total) in each step. By simultaneously generating multiple tokens that rarely depend on each other, the decoding speed is increased while the degradation in translation quality is minimized. In our experiments, the proposed method increased the translation speed by around 113%-155% in comparison with a standard autoregressive decoder, while degrading the BLEU scores by no more than 1.03. It was faster than an iterative non-autoregressive decoder in many conditions.

## introduction most neural machine translation systems are based on an encoder-decoder architecture. although there are some frameworks, such as recurrent neural network-based translation @xcite and transformerbased translation @xcite , they employ autoregressive decoding for high-quality translation. however, autoregressive decoding requires a decoding time that depends on sentence length because it generates a single token in each step. to solve this problem, non-autoregressive decoding, which generates all tokens in one step, has been proposed @xcite @xcite . however, the translation quality of non-autoregressive decoding has not yet matched the quality of autoregressive decoding. to improve the quality, some methods, such as mask-predict @xcite , apply non-autoregressive decoding iteratively. this is a trade-off between quality and speed because the decoding time depends on the number of iterations. this paper presents an autoregressive decoder that simultaneously generates two tokens in each step, as an intermediate solution between autoregressive and non-autoregressive decoding. the proposed method is based on the transformer decoder and generates a token from the head and tail of a sentence (two tokens in total) in each step. although this is a simple extension of standard autoregressive decoding, it has some notable features as follows: in the following sections, we first briefly review autoregressive and non-autoregressive decoding. we then explain the proposed method and evaluate it, from the viewpoints of translation quality and speed. ## conclusions this paper presented a bidirectional decoding method that simultaneously generates two tokens. the proposed method achieves fast decoding while minimizing quality degradation by generating tokens that rarely depend on each other. it is faster than both the standard autoregressive decoder and the mask-predict method in many conditions.