Effect of scale on catastrophic forgetting in neural networks

Continual learning is a current challenge in designing machine learning systems : when trained on multiple tasks in sequence , models tend to suffer performance losses on earlier tasks—this is known as catastrophic forgetting ( McCloskey & Cohen , 1989 ) . Catastrophic forgetting occurs in many settings , from supervised learning for classification ( Zenke et al. , 2017 ) to reinforcement learning for games ( Kirkpatrick et al. , 2017 ) and downstream applications such as translation ( Bapna & Firat , 2019 ) or medical diagnoses ( Gupta et al. , 2021 ) . While many methods have been proposed to help mitigate forgetting , none of these have fully solved the problem ; see Kemker et al . ( 2017 ) for a review . Increasing model and dataset size have been key to the success of deep learning ( Kolesnikov et al. , 2020 ; BIG-bench collaboration , 2021 ; Radford et al. , 2019 ; Zhai et al. , 2021 ) . A recent trend has been the use of increasingly large models pretrained on diverse datasets , which are subsequently fine-tuned or evaluated without fine-tuning on downstream tasks of interest ; see Bommasani et al . ( 2021 ) and references therein . A particularly striking realization of this has been large pretrained language models , which show consistent improvement across scales ( Kaplan et al. , 2020 ) and can perform well on diverse tasks with no finetuning ( Brown et al. , 2020 ) . From a continual learning perspective , this behavior is especially dramatic as , without finetuning , any training signal for these tasks appears mixed into a drastically larger pretraining dataset , and thus more than likely appeared many gradient steps before evaluation . This observation in large part inspired our work . Here , we empirically study continual learning in large pretrained models . Our results suggest that pretrained models are significantly more resistant to catastrophic forgetting than their randomlyinitialized counterparts , and that this resistance improves with the scale of both the model and pretraining data . Most of our experiments concern vision—specifically image classification ; but we also conduct a preliminary investigation of language models . Our key observations are as follows : • Resistance to catastrophic forgetting improves with model and pretraining dataset scale ( Figure 1 ) . • Large , pretrained image-classification models are significantly more resistant to forgetting than models trained from random initialization ( Figure 5 ) . • Both supervised and unsupervised , SimCLR ( Chen et al. , 2020 ) , pretraining improve forgetting ( Supplement , Figure 11 ) . • We perform initial experiments aimed at exploring why scale and pretraining reduce catastrophic forgetting , showing that distinct class representations in pretrained models are more orthogonal than in trained-from-scratch models ( Figure 8 ) . • We observe similar phenomena in language models ( Figure 9 ) . 1 RELATED WORK . Catastrophic forgetting Catastrophic forgetting was identified as a problem for neural networks as early as the late 1980s , as it occurred in the single-hidden-layer networks that were studied at the time ( see Delange et al . ( 2021 ) for a review ) . Within the catastrophic forgetting literature , a number of potential solutions to the problem have been devised ; some involve replaying previous-task examples during training on later tasks ( Rebuffi et al. , 2016 ; Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2018 ; Shin et al. , 2017 ; Riemer et al. , 2018 ) , while others like elastic weight consolidation ( Kirkpatrick et al. , 2017 ) limit the extent to which parameters can change once learned ( Zenke et al. , 2017 ; Lee et al. , 2017 ; Farajtabar et al. , 2019 ) . Perhaps most similar to our work are proposed mitigation methods which exploit the overparameterization of modern neural networks to perform continual learning of multiple tasks , e.g . by using pruning in Mallya & Lazebnik ( 2018 ) or task-specific subnetworks in Wortsman et al . ( 2020 ) and recent works studying the impact of pretraining Anonymous ( 2022 ) and overparameterization via network width Mirzadeh et al . ( 2021 ) on forgetting . Large-scale models The observation—initially in language models ( Kaplan et al. , 2020 ) —that model performance scales predictably with model and dataset size spurred an increased interest in training and deploying increasingly large pretrained models . Large-scale models emerged in natural-language processing with models such as GPT ( Radford et al. , 2019 ) and BERT ( Devlin et al. , 2019 ) , which are pretrained in an unsupervised manner and thus able to make use of the large corpora of natural text available on the internet . More recently , image-classification models have also benefited from pretraining on large datasets ( Kolesnikov et al. , 2020 ) , both supervised and unsupervised ; a key example is the Vision Transformer architecture of Dosovitskiy et al . ( 2020 ) , which—despite not using the inductive bias provided by convolutions—attains downstream performance comparable to convolutional networks after pretraining on ImageNet-21k ( Deng et al. , 2009 ) or JFT-300M ( Sun et al. , 2017 ) . On many current NLP and computer vision benchmarks , large pretrained models achieve top performance . All of the experiments in our work follow the pretrain-then-finetune method ; recently , another paradigm , side-tuning , was proposed by Zhang et al . ( 2020 ) , which avoids forgetting altogether by leaving the pretrained model fixed and employing auxiliary networks for downstream tasks . 2 EXPERIMENTAL SETUP . Tasks : Most of the experiments in this paper are conducted in the standard , task split setting ( Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ) . We consider split CIFAR-10 task sequences – the ten class dataset is split and sequentially trained on two tasks of 5 classes each . We also consider sequential 10 and 50 class splits of CIFAR-100 . We also consider the more recent input-distribution-shift CIFAR-100 setup ( Ramasesh et al. , 2021 ) . In this case each task has the same target types made up a fixed subset of the 20 CIFAR-100 superclasses ( e.g . aquatic mammals , fish , ... ) . The sequence of tasks consists of modifying which subclasses are sampled from . For example Task A may have aquatic mammals drawn from beaver and dolphin , while Task B may have otter and whale . This setup allows one to probe forgetting without changing the model head , in a setting where task identity is not known to the model . For our language model experiments , we consider next token prediction generative tasks and use the IMDb Reviews ( Maas et al. , 2011 ) and english Wikipedia ( Foundation ) datasets . Vision models : In most of our experiments we use the Vision Transformer ( ViT ) and ResNet architectures . For the Vision Transformers , we use the ViT-S and ViT-B configurations from Dosovitskiy et al . ( 2020 ) and a smaller configuration we call ViT-xS – ranging from 5.7-86.7 million parameters . For ResNets , we use five models : ResNet26 , ResNet50 , ResNet101 , ResNet152 , and ResNet200 – ranging from 14.0-62.6 million parameters ; as in Kolesnikov et al . ( 2020 ) , these models use group normalization ( Wu & He , 2018 ) and weight standarization ( Qiao et al. , 2019 ) in place of batch normalization . Details can be found in Tables 1 and 2 in Appendix A.1.2 . Pretraining : Unless otherwise specified , we pretrain our image models in a supervised manner on the ImageNet21k dataset , which contains approximately 14 million images in about 26000 distinct categories using the Adam optimizer ( Kingma & Ba , 2017 ) . Further details are in the Appendix . Finetuning : When fine-tuning on multiple tasks in sequence , we use the SGD optimizer with momentum β = 0.9 . The first task training employs a cosine decay schedule with a linear warmup ; later tasks in the sequence are trained with fixed learning rates . For the split-CIFAR-10 task sequences , we use a multi-head setup in which each head is initialized to zeros . 3 RESULTS . 3.1 FORGETTING IMPROVES WITH MODEL SCALE . Our main finding is that for both pretrained Vision Transformers and pretrained ResNets , catastrophic forgetting is mitigated to a large extent by scale : that is , larger models suffer less from forgetting . This is shown in Figure 1 for the two-task split CIFAR-10 sequence : Task A { airplane , automobile , bird , cat , deer } → Task B { dog , frog , horse , ship , truck } . In order to make a fair comparison , we plot the test accuracies of the model on both tasks at different steps of Task B fine-tuning for a variety of learning rates . This allows us to see , for each model , the maximum performance achievable on the new task for a given performance on the initial task . We call this the forgetting frontier . Figure 1 shows that the forgetting frontier for both ViT and ResNet models improves significantly with scale . While a 5.7M-parameter ViT-xS model loses about 6.5 % in Task A accuracy over the course of learning to perform Task B , an 86.7M-parameter ViT-B model loses less than half a percent . To put this in context , in a previous studies of split-CIFAR-10 task sequences , a ResNet18 model lost close to 40 % accuracy with no mitigation methods , 5 % when using a replay buffer , and 20 % when using elastic weight consolidation ( see Figure 1 of Ramasesh et al . ( 2021 ) for an example ) . It is also important to note that this effect of scale and pretraining on improving forgetting frontiers does not seem to be architecture-dependent . Both vision transformers—based on self-attention— and ResNets—based on convolutions—exhibit improvements in their forgetting frontiers as they grow larger , as long as they are pretrained . We see this improvement in forgetting performance beyond the simple two-task split-CIFAR-10 scenario . Across a 10-task sequence of 10-class split-CIFAR-100 tasks ( Figure 2 left ) , sequential 50-class splits of CIFAR-100 ( Figure 2 right ) , and the CIFAR-100 single-head input-distributionshift task ( Figure 3 ) , large-scale pretrained models exhibit resistance to forgetting , and continual learning ability appears to improve systematically with scale . For detailed results , scaling plots , and forgetting frontiers in the 10-task scenario , see Section B.2 in the Supplement . 3.2 DEPENDENCE ON FINETUNING DATASET . It is interesting to study how the improved resistance to forgetting with scale depends on the choice of finetuning dataset . In particular , it is worth noting that , though quite different in format , CIFAR10/100 are semantically quite similar to subsets of ImageNet21k , our pretraining set . To empirically investigate the dependence on finetuning dataset we have run additional experiments similar to those presented in Figure 1 for the Oxford Pet and Flowers , Caltech Birds , Stanford Cars , street view house numbers ( SVHN ) and DomainNet Clipart datasets . Results are shown in Figure 4 and in Figure 17 . We note in particular that SVHN classes are numerals and so distinct from ImageNet21k classes , while the Clipart dataset consists of synthetic , rather than natural images . See the Supplement for further details . 3.3 FORGETTING IN PRETRAINED MODELS VS. MODELS TRAINED FROM SCRATCH . In the previous section , we saw that the amount of catastrophic forgetting in models pretrained on large datasets decreased with increasing model size , to the point that the largest models ( ViT-B , with 86.7M parameters and ResNet200 with 62.6M parameters ) suffer very little performance loss from second-task training . In this section , we investigate whether this improvement in forgetting is simply a consequence of the improvement in performance on Task A . That is , do pretraining and scale have benefits with respect to forgetting beyond simply improving the fine-tuned performance on downstream tasks ? To check this , we compare the performance and amount of forgetting in pretrained ResNet models to models trained from scratch . We handicap the pretrained ResNets so that their maximum accuracy on on Task A matches that of the corresponding trained-from-scratch ResNet ; this is done simply by reducing the number of steps for which we fine-tune . Figure 5 shows the results of this experiment , with additional results and details in Appendix C.1 . While the trained-from-scratch ResNets achieve the same performance as their pretrained counterparts on Task A , their Task A performance degrades significantly more than the pretrained ResNets when trained on Task B ; pretrained ResNets forget less , and this improves with model scale . This result suggests that pretraining and model scale confer benefits upon models which go beyond simply improving performance . From Figure 5 it is also clear ( see also Figure 19 in the appendix , which plots the same data in a different way ) that pretraining is required to see the benefits of scale . Simply scaling up the model size in trained-from-scratch ResNets does not yield clear improvements in their forgetting performance . We focused in this section exclusively on ResNets ( rather than Vision Transformers ) because obtaining decent performance on image classification tasks with Vision Transformers seems to require pretraining . In our experiments , for example , a trained-from-scratch ViT-S model was unable to achieve better than 75 % test accuracy on five-class CIFAR-10 , while the same model , pretrained , achieved better than 98 % accuracy .

Summary: Following recent works studying the effect of model scaling on tasks such as transfer learning, this paper asks whether scaling up the size of a pretrained network also helps alleviate or minimize catastrophic forgetting. The authors use split CIFAR-10/-100 datasets (early experiments on NLP tasks are also included) and demonstrate the accuracy lost by a pretrained network when learning a set of tasks sequentially reduces with increase in model size. This result is shown to hold for both ResNets and Vision transformers. The topic is timely and the paper is well written. My main apprehensions relate to the experimental setup--due to the use of only CIFAR datasets for evaluation, which have significant class overlap with ImageNet, it may not be justified to think of learning a subset of CIFAR classes as a new task. Overall, I believe some claims are not completely validated yet, but this can be addressed by using a more appropriate experimental setting.