Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions (2022)

Abstract: This paper considers continual learning of large-scale pretrained neural machine translation model without accessing the previous training data or introducing model separation. We argue that the widely used regularization-based methods, which perform multi-objective learning with an auxiliary loss, suffer from the misestimate problem and cannot always achieve a good balance between the previous and new tasks. To solve the problem, we propose a two-stage training method based on the local features of the real loss. We first search low forgetting risk regions, where the model can retain the performance on the previous task as the parameters are updated, to avoid the catastrophic forgetting problem. Then we can continually train the model within this region only with the new training data to fit the new task. Specifically, we propose two methods to search the low forgetting risk regions, which are based on the curvature of loss and the impacts of the parameters on the model output, respectively. We conduct experiments on domain adaptation and more challenging language adaptation tasks, and the experimental results show that our method can achieve significant improvements compared with several strong baselines.

## introduction the current large-scale pretrained neural machine translation (nmt) models, such as gmnmt @xcite , mbart50-nn @xcite , and m2m-100 @xcite , are generally trained with large amounts of data from different domains and languages, so the model can learn good semantic representation and mapping relationship at the same time. based on such models, we hope that they can continually acquire new knowledge from new translation tasks, e.g., new domains and languages, while preserving the previously learned knowledge, i.e., continual learning (cl). however, catastrophic forgetting is the biggest challenge of continual learning, which means the model will forget the previously learned knowledge while learning new knowledge @xcite . one solution to avoid forgetting is to mix the previous and the new training data to train the model jointly. however, considering that the training data of the large-scale pre-trained nmt model is usually very large, this method will bring more training consumption and consume more energy. besides, we sometimes cannot access the previous data due to data privacy or storage limitation. to solve this problem, many researchers have made their attempts. some work avoids catastrophic forgetting by constructing pseudo data and mixing them with the new task data for joint training @xcite @xcite . however, these methods must require that the previous and new tasks are similar, and can not be directly applied to tasks with large differences, such as learning a totally different new language. some work introduces extra task-specific parameters and only updates these parameters with the new task data @xcite . on the one hand, this will increase the size of the model. on the other hand, the taskspecific parameters lead to model separation, which makes the model must know which task the input sentence belongs to, otherwise, the translation quality will degrade significantly @xcite . in contrast, the regularization-based methods @xcite don't have such limitations and are more flexible in practice. these methods perform multiobjective learning with an extra penalty item on the parameters, which aims to approximate the real loss on the previous task, and is usually in quadratic form. however, as illustrated in figure 1 (a ) and (b), the approximated loss (the red dashed line) is convex and symmetric, but the real loss (the red solid line) is not necessarily so in most cases, which may lead to the under or over constraint problems. besides, the multi-objective learning can only make the parameters converge to a region, where the gradients produced by the two losses are equal in size and opposite in direction, i.e., gradients reach balance. however, this does not guarantee that the values of the two losses are small in this region, which means the model may still suffer from the forgetting problem. instead of performing multi-objective learning with the approximated loss, we propose a two-stage continual learning method based on the local features of the real loss around the initial parameters. in the first stage, we aim to search the low forgetting risk regions as the parameter update regions, where the performance of the previous task will not degrade significantly when the parameters are updated. therefore, we can constrain the parameters to such regions to avoid catastrophic forgetting. in the second stage, the parameters will be freely updated in the searched regions only guided by the gradients of the new task, while the update outside these regions will be forbidden, as illustrated in figure 1 (c ). to achieve this, we can use some data related to the previous task to help us find such regions, e.g., the validation set data of the previous translation task, which is usually small-scale and easy to obtain. we propose two methods to search the low forgetting risk regions. the first method is to find out which parameters have the least impact on the previous task loss. we use the curvature of the loss curve as an indicator and the parameter with small curvature indicates that the loss curve is relatively flat, so the parameters can be updated more greedily. the second method is based on the impacts of parameters on the model output of the previous task. we define an objective function, which tries to maximize the size of update regions as much as possible while restricting the change of model output on the previous task, to help the model learn the update regions automatically. we conduct experiments on the domain adaptation and more challenging language adaptation tasks, and the results show that our method can achieve significant improvements compared with several strong baselines. our contribution can be summarized as follows: ## background in this section, we report the background knowledge used in this paper: the transformer @xcite model, multilingual nmt, hessian matrix, and fisher information matrix. ## experiments in this work, we perform continual learning on two tasks: the domain adaptation task and the more challenging language adaptation task. in the domain adaptation task, the language pairs of the new training data are already supported by the pretrained nmt model, and the goal is to enable the model to support the translation of new domains. in the language adaptation task, the goal is to enable the model to support the translation of new languages, which are not seen during pretraining. under the above two scenarios, we hope to retain the translation ability of the pre-trained mnmt model on the original translation task. ## conclusion in this work, we propose a continual learning method for nmt by constraining model parameters within low forgetting risk regions. we propose two methods to find such regions, the first is based on the curvature of the loss function and the second is based on the model output of the previous task. then we can continually train the model parameters within this region. the experimental results on the domain adaptation and language adaptation tasks prove that our method can achieve significant improvements over several strong baselines.