Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation (2022)

Abstract: Non-autoregressive translation (NAT) model achieves a much faster inference speed than the autoregressive translation (AT) model because it can simultaneously predict all tokens during inference. However, its translation quality suffers from degradation compared to AT. And existing NAT methods only focus on improving the NAT model’s performance but do not fully utilize it. In this paper, we propose a simple but effective method called “Candidate Soups,” which can obtain high-quality translations while maintaining the inference speed of NAT models. Unlike previous approaches that pick the individual result and discard the remainders, Candidate Soups (CDS) can fully use the valuable information in the different candidate translations through model uncertainty. Extensive experiments on two benchmarks (WMT’14 EN–DE and WMT’16 EN–RO) demonstrate the effectiveness and generality of our proposed method, which can significantly improve the translation quality of various base models. More notably, our best variant outperforms the AT model on three translation tasks with 7.6× speedup.

## introduction autoregressive translation (at) models based on transformer @xcite @xcite , where each generation step depends on the previously generated tokens, achieve state-of-the-art (sota) performance on most datasets for machine translation tasks. at model can better model the process of translation generation but leads to a massive limitation of its inference speed. therefore, the non-autoregressive translation (nat) @xcite model is proposed, which is 15.6 times faster than at model. nat assumes that the generated tokens are conditionally independent 1 our code is released at https://github.com/ boom-r123/candidate_soups . figure 1 : efficiency (speedup) and translation quality (bleu) of nat models in the wmt'14 en-de translation dataset. a cross "×" represents our candidate soups (cds) variants. its base model is shown in the shape "•", and its correspondence is represented by an arrow. cds (me-nd) refers to the at model for rescoring that has m encoder layers and n decoder layers. several methods have been proposed to alleviate the multimodality problem and improve the performance of the nat model, such as the iterationbased nat model @xcite @xcite and the semiautoregressive translation model @xcite . most of the previous methods are modified from the model's perspective, either modifying the structure of the model @xcite @xcite or modifying the training method of the model @xcite . different from the previous methods, in this paper, we propose a simple but effective method: src die beschaffung des erforderlichen personalausweises kostet oft über hundert dollar. candidate 1 it often costs over a hundred dollars to obtain the require identity card . candidate 2 it often cost over a hundred dollars to obtain the required identity card . npd it often cost over a hundred dollars to obtain the required identity card . cds it often costs over a hundred dollars to obtain the required identity card . candidate soups, which can significantly improve the translation quality without any modification to the model. moreover, candidate soups is a general approach that can be used by any nat model that can generate multiple candidate results, such as vanilla nat @xcite , glat @xcite , etc. the conventional recipe for maximizing translation quality through candidate results is noisy parallel decoding (npd) @xcite , which regards each candidate translation as an independent individual and ultimately only selects one of them as the final result and discards others. therefore npd can not utilize the valuable information in all the candidate translations. for example, there are a total of two candidate translations. the first candidate translation has the wrongly translated word in the second half, and the second candidate translation has the wrongly translated word in the first half. using the npd algorithm, in this case, can not get the correct translations (figure 2 ). however, candidate soups will effectively use the valuable information of all the candidate translations to fuse the different candidate results and obtain a higher-quality translation (figure 2 ). specifically, candidate soups first finds the common subsequence among all candidate results. based on the uncertainty of the model, we consider the common subsequence to be the most confident part of the model's predictions, so we make it part of the final translation and use it to align the candidate results. for the remaining parts, we select the part with the highest average log-probability among all candidate results to add to the final translation. candidate soups can be regarded as an implicit model ensemble method, which generates multiple different results by introducing uncertainty in the inference process, and further enhances the translation quality by making full use of the information of multiple results. we conduct extensive experiments in two datasets commonly used in machine translation, wmt'14 en-de and wmt'16 en-ro. the results demonstrate that our proposed method can significantly improve the base models' translation quality on different tasks while maintaining the fast inference speed of the nat model. remarkably, our best variant achieves better performance than the at teacher model on three translation tasks with 7.6× speedup. figure 1 demonstrates the qualityspeed trade-off compared with at and recent nat models. and relevant background knowledge is introduced in appendix a. ## related work since the nat model was proposed, it has attracted the attention of many researchers due to its superior inference speed. however, its translation quality suffers from degradation compared to at model. therefore various methods have been proposed to bridge the performance gap between nat and at model. some researchers constrain the distribution of nat model outputs by introducing various latent variables @xcite @xcite . through latent variables, the diversity of the nat model output space can be significantly reduced so that the model can better handle the dependencies between output words and alleviate the multimodality problem. such methods can usually maintain the efficient inference speed of the nat model, but the improvement in translation quality is relatively small. some other researchers have proposed iterative decoding methods @xcite @xcite , which continuously optimize the model's output by introducing more information in the iterative process. for example, @xcite mask the partial token of the previous output result and then use it as the input of the decoder for the next round of iteration. although such models can achieve high performance, multiple iterations can also significantly affect the inference speed of the nat model. recently, @xcite borrowed ideas from curriculum learning and proposed a novel way to train nat models, which let the model starts from learning the generation of sequence fragments and gradually moving to whole sequences. @xcite proposes to predict the result at each decoder layer and input it to the next layer together with the output of the current decoder layer to modify the result of the subsequent prediction. the above methods are all improvements from the model perspective, and their purpose is to allow the model to generate higher quality translations. unlike previous work, candidate soups wants to explore how to make the most of an existing model, and it can be applied to all nat models that can generate multiple candidate results. ## candidate soups this section describes the details of the proposed method in the paper. we first show the problem definition and the general idea of candidate soups in section 3.1, then introduce implementation details of the candidate soups in section 3.2, followed by the example in section 3.3. ## experiments in this section, we first introduce the settings of our experiments in section 4.1, then report the main results in section 4.2. ablation experiments and analysis are presented in section 4.3. ## conclusion in this paper, we propose "candidate soups," which can discover and fuse valuable information from multiple candidate translations based on model uncertainty. this approach is general and can be applied to various nat models. extensive experimental results prove that the translation quality of the nat model can be significantly improved by using candidate soups, especially for long sentences that are difficult to translate. and the tradeoff between translation quality and inference speed is well controlled and balanced by candidate soups. furthermore, our best variant can achieve better results on three translation tasks than the at teacher while maintaining nat's high-speed inference.