Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training (2022)

Abstract: Most existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples. They require the model to translate both the authentic source sentence and its adversarial counterpart into the identical target sentence within the same training stage, which may be a suboptimal choice to achieve robust NMT. In this paper, we first conduct a preliminary study to confirm this claim and further propose an Iterative Scheduled Data-switch Training Framework to mitigate this problem. Specifically, we introduce two training stages, iteratively switching between authentic and adversarial examples. Compared with previous studies, our model focuses more on just one type of examples at each single stage, which can better exploit authentic and adversarial examples, and thus obtaining a better robust NMT model. Moreover, we introduce an improved curriculum learning method with a sampling strategy to better schedule the process of noise injection. Experimental results show that our model significantly surpasses several competitive baselines on four translation benchmarks. Our source code is available at https://github.com/DeepLearnXMU/RobustNMT-ISDST .

## introduction in recent years, neural machine translation (nmt) has achieved great success @xcite @xcite . usually, the nmt models are trained on clean parallel corpus and thus achieve promising performance under clean inputs. however, small perturbations, such as replacing words in the input sentences, can mislead the trained model to generate incorrect translations @xcite . in realworld scenarios, it is often required to deal with such sentences. thus, it has important academic value and application prospects to design a robust nmt model for both clean and noisy inputs. to reach this goal, some researchers explore data-oriented approaches focusing on constructing adversarial examples @xcite . generally, adversarial examples are used to augment the authentic dataset or fine-tune an nmt model pre-trained on the authentic dataset to improve robustness. although data-oriented approaches are simple and efficient, they leverage adversarial examples coarsely, as concluded by @xcite and @xcite , which can not reach the full potential of these examples. besides, researchers also study model-oriented approaches. some design additional model components to correct noisy inputs @xcite @xcite . there are more studies exploring training strategies for robust nmt, including multi-task learning @xcite , contrastive learning @xcite , and adversarial training @xcite . despite their success, there still exist two drawbacks: 1) most existing methods indiscriminately exploit authentic and adversarial examples within the same training stage, which is a suboptimal choice confirmed in our preliminary study; 2) previous studies on robust nmt adopt a constant noise ratio to construct adversarial examples during training, while the determination of noise ratio is a subtle process, i.e., too little noise may lead to poor robustness and too much noise may also hurt the model performance @xcite . therefore, dealing with both clean and noisy inputs well for nmt remains to be a significant but challenging task. in this paper, we first conduct a preliminary study, which reveals that indiscriminately exploit-ing authentic and adversarial examples within the same training stage is suboptimal. concretely, we find that this training strategy can not significantly reduce the source sentence representation (ssr) discrepancies 1 between authentic examples and the corresponding adversarial examples, resulting in a suboptimal model training which is reflected by lower model confidence foot_1 on examples. based on this observation, we further propose an iterative scheduled data-switch training framework for robust nmt. under this framework, we train the model in a two-stage scheme, iteratively switching between authentic and adversarial examples with their individual modified training objectives. during training, we introduce an additional kullback-leibler (kl) divergence loss, expecting the model to make similar predictions on authentic and adversarial datasets. by doing so, at each training stage, the model not only focuses on one of authentic and adversarial datasets but also avoids forgetting the knowledge from the other. therefore, our model is able to handle both clean and noisy inputs well. furthermore, we introduce curriculum learning (cl) to better schedule the process of noise injection. particularly, inspired by the baby step strategy @xcite in cl that gradually exposes more difficult examples to the model while still involving simple examples, we sample the noise ratio from a uniform distribution, where the sampling interval is progressively extended. compared with the naive cl strategy of continuously increasing the noise ratio, our strategy is re-sampling previous simple adversarial examples which is beneficial to the model generalization. in summary, our contributions are as follows: ## preliminary study indiscriminately exploiting authentic examples and their adversarial counterparts within the same training stage is an effective way to build a robust nmt model. however, it requires the model to overcome the ssr discrepancy between an authentic example (x, y) and its adversarial counterpart (x ′ , y), which increases the training difficulty to maximize p(y|x; θ) and p(y|x ′ ; θ) simultaneously. we argue it may be a better choice to exploit authentic and adversarial examples at two training stages, iteratively switching between two types of examples. in such a data-switch training manner, the model can better benefit from the knowledge of different stages to verify our hypothesis, we use transformer @xcite as our nmt model and conduct a preliminary experiment on the iwslt14 de⇒en dataset. to be specific, we train the three models: 1) transformer. we follow @xcite to train this model on the authentic dataset; 2) indisc-model. it indiscriminately exploits authentic and adversarial examples for training within the same stage. besides, following @xcite , we introduce a mean square error (mse) loss to enforce the corresponding encoder outputs to be similar; 3) switch-model. this model is trained at two training stages, iteratively switching between authentic and adversarial examples. we make an investigation through the two metrics: 1) the euclidean distances of the ssr between authentic examples and their adversarial counterparts; 2) the model confidence, i.e., log-likelihood values of target ground-truth sentences. ## methodology based on the observations in section 2, we further propose an iterative scheduled data-switch training framework for robust nmt. ## related work to build robust nmt models, researchers have proposed a range of methods, which can be mainly divided into two categories: data-oriented and modeloriented approaches. in the first category, how to construct adversarial examples is a non-trivial problem @xcite . usually, adversarial examples are used in two ways: one is to directly train a robust model using the dataset mixed with authentic and adversarial examples @xcite , and the other is to use adversarial examples to fine-tune the nmt model pre-trained on authentic examples @xcite @xcite . in the second category, some researchers design additional components for nmt model to correct noisy inputs @xcite @xcite or explore fault-tolerant neural networks @xcite . mean-while, more researchers resort to exploring training strategies, including multi-task learning @xcite , contrastive learning @xcite , and adversarial training @xcite @xcite . in this work, the proposed framework belongs to the second model-oriented category. in this regard, most existing methods indiscriminately exploit authentic and adversarial examples within the same training stage, which are suboptimal confirmed in our preliminary study. to mitigate this problem, we propose an iterative scheduled data-switch training framework for robust nmt, where we introduce two training stages, iteratively switching between authentic and adversarial examples. besides, inspired by the successful applications of curriculum learning (cl) in nmt @xcite @xcite , we use cl to better schedule the process of noise injection. particularly, we equip cl with a sampling strategy, which is beneficial to the model generalization. finally, note that @xcite introduce an alternated training to alleviate the performance drop caused by low-quality back-translation data. our work differs from theirs in three aspects: 1) we aim at building a robust nmt model dealing with clean and noisy inputs well, while @xcite try to prevent the model performance on clean test sets from being disturbed by synthetic data; 2) we introduce an improved cl method to better schedule the process of noise injection, which is beneficial to the model performance; 3) in addition to the conventional cross-entropy objective @xcite , we introduce an additional regularization term to cope with both clean and noisy inputs well. ## conclusion in this paper, we first conduct a preliminary study to reveal that indiscriminately exploiting authentic and adversarial examples for robust nmt is suboptimal. to achieve better robust nmt, we further propose an iterative scheduled data-switch training framework, where we train the model at two training stages, iteratively switching between authentic and adversarial examples. moreover, we introduce curriculum learning with a sampling strategy to schedule the process of noise injection at each training stage. extensive experiments show the superiority of our framework. in the future, we will introduce more types of real noise, such as asr errors, into our framework. besides, we plan to apply our framework to other natural language generation tasks, such as dialogue generation, so as to verify the generality of our framework.