One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation (2022)

Abstract: Non-autoregressive neural machine translation (NAT) suffers from the multi-modality problem: the source sentence may have multiple correct translations, but the loss function is calculated only according to the reference sentence. Sequence-level knowledge distillation makes the target more deterministic by replacing the target with the output from an autoregressive model. However, the multi-modality problem in the distilled dataset is still nonnegligible. Furthermore, learning from a specific teacher limits the upper bound of the model capability, restricting the potential of NAT models. In this paper, we argue that one reference is not enough and propose diverse distillation with reference selection (DDRS) for NAT. Specifically, we first propose a method called SeedDiv for diverse machine translation, which enables us to generate a dataset containing multiple high-quality reference translations for each source sentence. During the training, we compare the NAT output with all references and select the one that best fits the NAT output to train the model. Experiments on widely-used machine translation benchmarks demonstrate the effectiveness of DDRS, which achieves 29.82 BLEU with only one decoding pass on WMT14 En-De, improving the state-of-the-art performance for NAT by over 1 BLEU.

## introduction non-autoregressive machine translation @xcite has received increasing attention in the field of neural machine translation for the property of parallel decoding. despite the significant speedup, nat suffers from the performance degradation compared to autoregressive models @xcite due to the multimodality problem: the source sentence may have multiple correct translations, but the loss is calculated only according to the reference sentence. the multi-modality problem will cause the inaccuracy of the loss function since nat has no prior knowledge about the reference sentence during the generation, where the teacher forcing algorithm @xcite ) makes autoregressive models less affected by feeding the golden context. how to overcome the multi-modality problem has been a central focus in recent efforts for improving nat models @xcite @xcite @xcite @xcite . a standard approach is to use sequence-level knowledge distillation @xcite , which attacks the multimodality problem by replacing the target-side of the training set with the output from an autoregressive model. the distilled dataset is less complex and more deterministic @xcite , which becomes a default configuration of nat. however, the multi-modality problem in the distilled dataset is still nonnegligible @xcite . furthermore, the distillation requires nat models to imitate the behavior of a specific autoregressive teacher, which limits the upper bound of the model capability and restricts the potential of developing stronger nat models. in this paper, we argue that one reference is not enough and propose diverse distillation with reference selection (ddrs) for nat. diverse distillation generates a dataset containing multiple reference translations for each source sentence, and reference selection finds the reference translation that best fits the model output for the training. as illustrated in figure 1 , diverse distillation provides candidate references "i must leave tomorrow" and "tomorrow i must leave", and reference selection selects the former which fits better with the model output. more importantly, nat with ddrs does not imitate the behavior of a specific teacher but learns selectively from multiple references, which improves the upper bound of the model capability and allows for developing stronger nat models. the object of diverse distillation is similar to the task of diverse machine translation, which aims to generate diverse translations with high translation quality @xcite @xcite @xcite . we propose a simple yet effective method called seeddiv, which directly uses the randomness in model training controlled by random seeds to produce diverse reference translations without losing translation quality. for reference selection, we compare the model output with all references and select the one that best fits the model output, which can be efficiently conducted without extra neural computations. the model learns from all references indiscriminately in the beginning, and gradually focuses more on the selected reference that provides accurate training signals for the model. we also extend the reference selection approach to reinforcement learning, where we encourage the model to move towards the selected reference that gives the maximum reward to the model output. we conduct experiments on widely-used machine translation benchmarks to demonstrate the effectiveness of our method. on the competitive task wmt14 en-de, ddrs achieves 27.60 bleu with 14.7× speedup and 28.33 bleu with 5.0× speedup, outperforming the autoregressive transformer while maintaining considerable speedup. when using the larger version of transformer, ddrs even achieves 29.82 bleu with only one decoding pass, improving the state-of-the-art performance level for nat by over 1 bleu. the vanilla-nat has to know the target length before constructing the decoder inputs. the target length t is set as the reference length during the training and obtained from a length predictor during the inference. the target length cannot be changed dynamically during the inference, so it often requires generating multiple candidates with different lengths and re-scoring them to produce the final translation @xcite . the length issue can be overcome by connectionist temporal classification (ctc, @xcite . ctc-based models usually generate a long alignment containing repetitions and blank tokens. the alignment will be post-processed by a collapsing function γ -1 to recover a normal sentence, which first collapses consecutive repeated tokens and then removes all blank tokens. ctc is capable of efficiently finding all alignments a which the reference sentence y can be recovered from, and marginalizing the log-likelihood with dynamic programming: log p(y|x, θ) = log @xmath0, θ). (3) due to the superior performance and the flexibility of generating predictions with variable length, ctc is receiving increasing attention in nonautoregressive translation @xcite @xcite @xcite . ## approach in this section, we first introduce the diverse distillation technique we use to generate multiple reference translations for each source sentence, and then apply reference selection to select the reference that best fits the model output for the training. ## conclusion in this paper, we propose diverse distillation with reference selection (ddrs) for nat. diverse distillation generates a dataset containing multiple references for each source sentence, and reference selection finds the best reference for the training. ddrs demonstrates its effectiveness on various benchmarks, setting new state-of-the-art performance levels for non-autoregressive translation.