Mitigating the Diminishing Effect of Elastic Weight Consolidation (2022)

Abstract: Elastic weight consolidation (EWC, Kirkpatrick et al. 2017) is a promising approach to addressing catastrophic forgetting in sequential training. We find that the effect of EWC can diminish when fine-tuning large-scale pre-trained language models on different datasets. We present two simple objective functions to mitigate this problem by rescaling the components of EWC. Experiments on natural language inference and fact-checking tasks indicate that our methods require much smaller values for the trade-off parameters to achieve results comparable to EWC.

## introduction new training data may arrive after we have spent considerable time training our model on the data at hand. a simple method for exploiting both new and old training data is to mix them and retrain the model from scratch. however, this mix-and-retrain method is neither always practical nor economical, especially in academic environments where computational resources are limited. sequential training is a potential alternative approach but faces a difficult challenge called catastrophic forgetting in which the performance on old data drastically drops when we train a model on new data. there exists a line of work that has addressed this challenge @xcite @xcite @xcite . in this paper, we are particularly interested in elastic weight consolidation (ewc, @xcite , which has been shown to be helpful for domain adaptation @xcite . ewc adds a regularization term to the objective function to ensure that the model works well on both new and old data. we empirically find 1 our code is available at https://github.com/ nii-yamagishilab/ewc . we sequentially fine-tune bert (bidirectional encoder representations from transformers, @xcite ) on mnli @xcite and fever @xcite and evaluate performance on the balanced dev sets. ewc starts to increase the accuracy of the prior dataset (mnli) when increasing λ to 10 5 and yields the highest average accuracy at 10 7 . that ewc requires unexpectedly large values for the trade-off parameter (λ) between the regularizer and the loss to be effective when applying to pre-trained language models. figure 1 shows such a phenomenon in which ewc has no effect in preventing catastrophic forgetting of the prior dataset (mnli) with λ in the range of [10 0 , 10 4 ]. we have to scale λ up to [10 5 , 10 7 ], which is an unusual range of hyperparameters. to the best of our knowledge, this phenomenon has not been reported in the literature. we propose two simple objective functions for mitigating the diminishing effect of ewc. our objective functions rely on rescaling the components of ewc. specifically, the first objective function involves taking the square root of the regularization term, while the second one involves using the absolute value of the gradient instead of the squared gradient. both of our objective functions can reduce the values of the trade-off parameter λ by three to seven orders of magnitude while producing results similar to those of the original ewc. ## proposed method as shown in figure 1 , ewc requires extremely large values of λ to be effective. we analyze the components of ewc and find that this problem arises from the fisher approximation in eq. ( 5 ). the diagonal element f j corresponds to the j th element of the squared gradient with respect to θ 0 . since its training had already converged, the values of the gradient are typically small. when we square such a small decimal and combine it with the squared difference between the current and prior parameters, the final value can be vanishingly small. 2 we find that this issue is neither affected by datasets nor pre-trained language models. in appendix a, we further investigate this issue on another pre-trained language model. we propose scaling up the fisher approximation by taking the square root to resolve the issue above. we define the square root of ewc (rewc) as: where another solution is to use the absolute value of the gradient instead of the squared gradient. we define: note that diag(g) is positive semi-definite (like diag(f)) because all of its eigenvalues are greater than or equal to 0. replacing the squared difference with the absolute difference yields our absolute ewc (aewc): figure 2 shows the results of rewc and aewc based on the same setting as in figure 1 . ## conclusion without realizing the diminishing effect of ewc, we may fine-tune a pre-trained language model with a conventional range of hyperparameters and find no effect in combating catastrophic forgetting. we identified a possible cause of this issue and suggested two alternative objective functions, rewc and aewc, that yield results comparable to the original ewc. exploring more efficient ways for choosing an optimal λ is part of our future work.