Scheduled Sampling Based on Decoding Steps for Neural Machine Translation (2021)

Abstract: Scheduled sampling is widely used to mitigate the exposure bias problem for neural machine translation. Its core motivation is to simulate the inference scene during training by replacing ground-truth tokens with predicted tokens, thus bridging the gap between training and inference. However, vanilla scheduled sampling is merely based on training steps and equally treats all decoding steps. Namely, it simulates an inference scene with uniform error rates, which disobeys the real inference scene, where larger decoding steps usually have higher error rates due to error accumulations. To alleviate the above discrepancy, we propose scheduled sampling methods based on decoding steps, increasing the selection chance of predicted tokens with the growth of decoding steps. Consequently, we can more realistically simulate the inference scene during training, thus better bridging the gap between training and inference. Moreover, we investigate scheduled sampling based on both training steps and decoding steps for further improvements. Experimentally, our approaches significantly outperform the Transformer baseline and vanilla scheduled sampling on three large-scale WMT tasks. Additionally, our approaches also generalize well to the text summarization task on two popular benchmarks.

## introduction neural machine translation (nmt) has made promising progress in recent years @xcite @xcite . generally, nmt models are trained to maximize the likelihood of next token given previous golden tokens as inputs, i.e., teacher forcing @xcite . however, at the inference stage, golden tokens are unavailable. the model is exposed to an unseen data distribution generated by itself. this discrepancy between training and inference is named as the exposure bias problem @xcite . with the growth of decoding steps, such discrepancy becomes more problematic due to error accumulations @xcite (shown in figure 1 ). many techniques have been proposed to alleviate the exposure bias problem. to our knowledge, they mainly fall into two categories. the one is sentencelevel training, which treats the sentence-level metric (e.g., bleu) as a reward, and directly maximizes the expected rewards of generated sequences @xcite @xcite . although intuitive, they generally suffer from slow and unstable training due to the high variance of policy gradients and the credit assignment problem @xcite @xcite . another category is sampling-based approaches, aiming to simulate the data distribution of the inference scene during training. scheduled sampling @xcite is a representative method, which samples tokens between golden references and model predictions with a scheduled probability. @xcite further refine the sampling candidates by beam search. @xcite and @xcite extend scheduled sampling to the transformer with a novel two-pass decoder architecture. @xcite develop a more fine-grained sampling strategy according to the model confidence. although these sampling-based approaches have been shown effective and training efficient, there still exists an essential issue in their sampling strategies. in the real inference scene, the nature of sequential predictions quickly accumulates errors along with decoding steps, which yields higher error rates for larger decoding steps @xcite (figure 1 ). however, most sampling-based approaches are merely based on training steps and equally treat all decoding steps foot_1 . namely, they simulate an inference scene with uniform error rates along with decoding steps, which is inconsistent with the real inference scene. to alleviate this inconsistent issue, we propose scheduled sampling methods based on decoding steps, which increases the selection chance of predicted tokens with the growth of decoding steps. in this way, we can more realistically simulate the inference scene during training, thus better bridging the gap between training and inference. furthermore, we investigate scheduled sampling based on both training steps and decoding steps, which yields further improvements. it indicates that our proposals are complementary with existing studies. additionally, we provide in-depth analyses on the necessity of our proposals from the perspective of translation error rates and accumulated errors. experimentally, our approaches significantly outperform the transformer baseline by 1.08, 1.08, and 1.27 bleu points on wmt 2014 english-german, wmt 2014 english-french, and wmt 2019 chinese-english, respectively. when comparing with the stronger vanilla scheduled sampling method, our approaches bring further improvements by 0.58, 0.62, and 0.55 bleu points on these wmt tasks, respectively. moreover, our approaches generalize well to the text summarization task and achieve consistently better performance on two popular benchmarks, i.e., cnn/dailymail @xcite and gigaword @xcite . the main contributions of this paper can be summarized as follows foot_2 : ## experiments we validate our proposals on two important sequence generation tasks, i.e., machine translation and text summarization. ## analysis and discussion in this section, we provide in-depth analyses on the necessity of our proposals and conducts experiments on the validation set of wmt14 en-de with the transformer base model. ## conclusion in this paper, we propose scheduled sampling methods based on decoding steps from the perspective of simulating real translation error rates, and provide in-depth analyses on the necessity of our proposals. we also confirm that our proposals are complementary with existing studies (based on training steps). experiments on three large-scale wmt translation tasks and two text summarization tasks confirm the effectiveness of our approaches. in the future, we will investigate low resource settings which may suffer from a more serious error accumulation problem. in addition, more autoregressive-based tasks would be explored as future work.