MR - P : A Parallel Decoding Algorithm for Iterative Refinement Non-Autoregressive Translation (2022)

Abstract: Non-autoregressive translation (NAT) predicts all the target tokens in parallel and significantly speeds up the inference process. The Conditional Masked Language Model (CMLM) is a strong baseline of NAT. It decodes with the Mask-Predict algorithm which iteratively refines the output. Most works about CMLM focus on the model structure and the training objective. However, the decoding algorithm is equally important. We propose a simple, effective, and easy-to-implement decoding algorithm that we call MaskRepeat-Predict (MR-P). The MR-P algorithm gives higher priority to consecutive repeated tokens when selecting tokens to mask for the next iteration and stops the iteration after target tokens converge. We conduct extensive experiments on six translation directions with varying data sizes. The results show that MR-P significantly improves the performance with the same model parameters. Specifically, we achieve a BLEU increase of 1.39 points in the WMT’14 En-De translation task.

## introduction the autoregressive neural machine translation (at) model based on encoder-decoder framework @xcite has achieved great success @xcite @xcite . the decoder predicts target tokens step by step conditioned on source tokens and previously predicted tokens. such dependency between target tokens inevitably leads to the decoding latency. non-autoregressive neural machine translation (nat) models @xcite remove the dependency between tokens in the target sentence and generate all tokens in parallel, significantly improving the inference speed. the assumption of conditional independence in target tokens makes it more difficult for nat mod-els to learn the target distribution. nat models' translation is often incomplete or repetitive, especially for long sentences. an approach for alleviating this problem is to iteratively refine the model output and make a trade-off between inference speed and model performance @xcite @xcite ). many refinement-based models are based on cmlm @xcite and use the mask-predict (m-p) @xcite algorithm for decoding. most works attempt to improve the model from the model structure and the training method. in this work, we propose a novel decoding algorithm for refinement-based models that we call maskrepeat-predict (mr-p). our algorithm prefers the consecutive repeated tokens when selecting tokens to mask. and the iteration will stop in advance when the target sentence converges, which reduces the number of iterations and avoid over-refinement.we verify the effectiveness of mr-p in six translation directions of three standard datasets with varying data sizes. under the same model parameters, the model's performance is significantly improved using the mr-p decoding algorithm. the main contributions of this work are as follows: ## methodology the mask-predict algorithm selects tokens according to the generation probabilities. there is a problem with this strategy. when the probabilities of consecutive repeated tokens are high, they will not be selected and remain in the results. as can be seen from ## conclusion