Incorporating a Local Translation Mechanism into Non-autoregressive Translation (2020)

Abstract: In this work, we introduce a novel local autoregressive translation (LAT) mechanism into non-autoregressive translation (NAT) models so as to capture local dependencies among target outputs. Specifically, for each target decoding position, instead of only one token, we predict a short sequence of tokens in an autoregressive way. We further design an efficient merging algorithm to align and merge the output pieces into one final output sequence. We integrate LAT into the conditional masked language model (CMLM) (Ghazvininejad et al.,2019) and similarly adopt iterative decoding. Empirical results on five translation tasks show that compared with CMLM, our method achieves comparable or better performance with fewer decoding iterations, bringing a 2.5x speedup. Further analysis indicates that our method reduces repeated translations and performs better at longer sentences. Our code will be released to the public.

## introduction traditional neural machine translation (nmt) models @xcite @xcite @xcite commonly make predictions in an incremental token-by-token way, which is called autoregressive translation @xcite . although this strategy can capture the full translation history, it has relatively high decoding latency. to make the decoding more efficient, non-autoregressive translation (nat) @xcite is introduced to generate multiple tokens at once instead of one-by-one. however, with the conditional independence property @xcite , nat models do not directly consider the dependencies among output tokens, which may cause errors of repeated translation and figure 1 : an example of the lat mechanism. for each decoding position, a short sequence of tokens is generated in an autoregressive way. sop is the special startof-piece symbol. 'pos*' denotes the hidden state from the decoder at that position. in this work, we introduce a novel mechanism, i.e., local autoregressive translation (lat), to take local target dependencies into consideration. for a decoding position, instead of generating one token, we predict a short sequence of tokens (which we call a translation piece) for the current and next few positions in an autoregressive way. a simple example is shown in figure 1 . with this mechanism, there can be overlapping tokens between nearby translation pieces. we take advantage of these redundancies, and apply a simple algorithm to align and merge all these pieces to obtain the full translation output. specifically, our algorithm builds the output by incrementally aligning and merging adjacent pieces, based on the hypothesis that each local piece is fluent and there are overlapping tokens between adjacent pieces as aligning points. moreover, the final output se-quence is dynamically decided through the merging algorithm, which makes the decoding process more flexible. we integrate our mechanism into the conditional masked language model (cmlm) @xcite and similarly adopt iterative decoding, where tokens with low confidence scores are masked for prediction in more iterations. with evaluations on five translation tasks, i.e., wmt'14 en↔de, wmt'16 en↔ro and iwslt'14 @xmath0, we show that our method could achieve similar or better performance compared with cmlm and at models while gaining nearly 2.5 and 7 times speedups, respectively. furthermore, our method is shown to effectively reduce repeated translations and perform better at longer sentences. ## merging algorithm in decoding, the model generates local translation pieces for all decoding positions. we adopt a simple algorithm that incrementally builds the output through a piece-by-piece merging process. our hypothesis is that if the local autoregressive translator is well-trained, then 1) the token sequence inside each piece is fluent and well-translated, 2) there are going to study here will study in the s1 s2 in the above procedure, we need to specify the score of a span. through preliminary experiments, we find a simple but effective scheme. from the translation model, each token gets a model score of its log probability. for the score of a span, we average the scores of all the tokens inside. if the span is empty, we utilize a pre-defined value, which is empirically set to log 0.25. for aligned tokens, we choose the highest scores among them for later merging process (line 16). with this core merging operation, we apply a left-to-right scan to merge all the pieces in a pieceby-piece fashion. for each merging operation, we only take the last k tokens of s1 and the first k tokens of s2, while other tokens are directly copied. this ensures that the merging will only be local, to mitigate the risk of wrongly aligned tokens. here, k is again the local translation step size. our merging algorithm can be directly applied at the end of each iteration in the iterative decoding. however, since the output length of the merging algorithm is not always the same as the number of input pieces, we further adopt a length adjustment procedure for intermediate iterations. briefly speaking, we adjust the output length to the predicted length by adding or deleting certain amounts of special mask symbols. please refer to the apalthough our merging algorithm is actually autoregressive, it does not include any neural network computations and thus can run efficiently. in addition to efficiency, our method also makes the decoding more flexible, since the final output is dynamically created through the merging algorithm. ## conclusion in this work, we incorporate a novel local autoregressive translation mechanism (lat) into nonautoregressive translation, predicting multiple short sequences of tokens in parallel. with a simple and efficient merging algorithm, we integrate lat into the conditional masked language model (cmlm @xcite and similarly adopt iterative decoding. we show that our method could achieve similar results to cmlm with less decoding iterations, which brings a 2.5x speedup. moreover, analysis shows that lat can reduce repeated translations and perform better at longer sentences.