Continual Learning for Neural Machine Translation (2021)

Abstract: Neural machine translation (NMT) models are data-driven and require large-scale training corpus. In practical applications, NMT models are usually trained on a general domain corpus and then fine-tuned by continuing training on the in-domain corpus. However, this bears the risk of catastrophic forgetting that the performance on the general domain is decreased drastically. In this work, we propose a new continual learning framework for NMT models. We consider a scenario where the training is comprised of multiple stages and propose a dynamic knowledge distillation technique to alleviate the problem of catastrophic forgetting systematically. We also find that the bias exists in the output linear projection when fine-tuning on the in-domain corpus, and propose a bias-correction module to eliminate the bias. We conduct experiments on three representative settings of NMT application. Experimental results show that the proposed method achieves superior performance compared to baseline models in all settings.

## introduction continual learning, which is also referred to as incremental learning or lifelong learning, is a learning paradigm that allows the agent to continuously learn from new knowledge without forgetting previously learned knowledge. humans naturally have the ability to continually acquire knowledge while preserving old knowledge throughout their lifespan. in real-world applications, data is usually given in a continuous stream form, and only part of the data is available at the beginning of training. therefore, the ability to learn from continuous streams of information is crucial for artificial intelligence systems. however, continual learning remains a big challenge for artificial intelligence systems and models since they suffer from the problem of catastrophic forgetting @xcite , i.e., the learning of new tasks may cause the model to forget the knowledge learned from previous tasks. this phenomenon typically leads to a significant performance decrease in previously learned tasks. one trivial solution to avoid catastrophic forgetting is to retrain from scratch by combining old and new tasks. however, this methodology is computationally inefficient and needs to store old data all the time. recently, continual learning has received increasing attention in the artificial intelligence filed. most of existing works focus on computer vision tasks @xcite @xcite @xcite @xcite . in the natural language processing area, several methods have been proposed to alleviate the problem of catastrophic forgetting for neural machine translation (nmt) models. for example, @xcite propose to ensemble models trained on different domains. however, this brings a storage issue: as the number of domains increases, the number of stored models also increases. @xcite and @xcite add an l2 or ewc regularization to each parameter to prevent the model's parameters from changing too much. however, for those transformer models with more than 100 million parameters, the time and space cost for computing l2 or ewc regularization is expensive. @xcite propose a regularized training objective that minimizes the cross-entropy between in domain model's output distribution and that of the out-of-domain model. this method can essentially be regarded as a kind of knowledge distillation. the above works assume that the training is divided into two stages, i.e., out-of-domain training and in-domain fine-tuning. in this work, we extend these works and propose a new continual learning framework for nmt models. we consider a more general scenario where the training is comprised of multiple stages. we propose a dynamic knowledge distillation-based method to alleviate the problem of catastrophic forgetting in a systematic and principled way. we also find that when fine-tuning on new data, there exists a strong bias towards the new words in the output embedding layer (i.e. the linear projection before the last softmax layer) of the decoder, which results in the bias in the generation that favors words from new data. to address this issue, we incorporate the model with a bias-correction module that normalizes the weights in the projection layer. the bias-correction module can effectively eliminate the bias of significant differences in magnitudes. we consider three continual learning scenarios: (1) in-domain multi-stage training, where m streams of data from the same domain are fed to the model sequentially, (2) domain-incremental training, where m streams of data from different domains are fed to the model sequentially, and (3) time-incremental training, where m streams of data from different time are fed to the model sequentially. experimental results show that the proposed method can effectively address the catastrophic forgetting issue and balance the weights in the projection layer, thus achieving superior results compared to the competitive models. in summary, the prime contributions of this paper are as follows: ## conclusion in this paper, we propose a new continual learning framework for neural machine translation. we first propose a dynamic knowledge distillationbased method to alleviate the problem of catastrophic forgetting in a multi-stage view, and then propose a bias-correction module to address the biased weights issue. to verify the effectiveness of the proposed method, we conduct experiments in three different settings: in-domain incremental training, time-incremental training, and domainincremental training. experimental results show that the proposed method can obtain superior performance compared to competitive models. in the future, we will apply the proposed method to other nlp tasks to test its robustness.