ImpressLearn: Continual Learning via Combined Task Impressions

Keywords : Catastrophic forgetting , Continual learning , Neural networks , Masking 1 INTRODUCTION . Sequential learning without catastrophic forgetting has been an area of active research in machine learning for some time ( Maes et al. , 1996 ; Thrun & Pratt , 1998 ; Serra et al. , 2018 ) . A precondition for achieving artificial general intelligence ( AGI ) is that models be able to learn and remember a wide variety of tasks sequentially , without forgetting previously learned ones . In real-world scenarios , data from different tasks may not be available simultaneously , which makes it imperative to both allow continued learning as well as to avoid catastrophic forgetting of a potentially unbounded number of tasks ( see also The Sequential Learning Problem ( McCloskey & Cohen , 1989 ) , Constraints Imposed by Learning and Forgetting Functions ( Ratcliff , 1990 ) and Lifelong Learning Algorithms ( Thrun & Pratt , 1998 ) ) . Recently , some successful approaches to combat this problem use task specific submodels , which allow neural networks to context-switch between different learning tasks ( Wortsman et al. , 2020 ; Mallya et al. , 2018 ; Mancini et al. , 2018 ) . The underlying context for each task can be represented as ” filters ” or ” masks ” , altering the network connections for each task . Yet all of these approaches scale unfavorably with the number of unique tasks to be learned . Our contribution : We propose a novel method which exploits transfer learning and network masking to sequentially learn a theoretically unlimited number of tasks with much lower parameter overhead than prevailing benchmarks . Our method , termed ImpressLearn , uses elements from Supermasks in Superposition ( SupSup ) by Wortsman et al . ( 2020 ) . The SupSup approach leverages the observation that even within randomly weighted neural networks there exist task-specific supermasks—subnetworks produced by overlaying a binary mask that selectively removes connections—which achieve good performance on the task . These supermasks can be learned from task-specific data and stored , one mask per task . At inference , the appropriate task-specific mask is applied when task identity is known . When task-labels of a previously seen task are not known , the correct mask can be inferred via entropy considerations . A ” basis ” of masks : Our approach goes further , driven by two core ideas . In order to dramatically improve scaling of the parameters with the number of different tasks , we reduce the number of necessary masks to a constant number , independent of the number of different tasks . Our basis-masks are constructed from a small number of initial tasks . Leveraging transfer learning , this set of learned masks , each of which can be interpreted as an ” impression ” of a previously seen prior task , serves as a set of latent features for new learning objectives , encoding common structural information . Our second idea is to use the power of linear combinations to combine these impressions to quickly learn a real-valued mask that performs well for new tasks . Hence , other than a fixed number of basis-masks , only a small number of coefficients need to be learned and stored for each subsequent task . This greatly benefits scalability , a major drawback of previous approaches , as this in principle allows for an unlimited number of new tasks while the parameter overhead is several orders of magnitude smaller than storing even a compressed binary network mask . Specifically , ImpressLearn uses the first few tasks from a multi-task set to generate a relatively small number of binary masks . For subsequent tasks , these impressions are used akin to a basis : new tasks are learned through a simple linear combination of the basis-impressions . This might be reminiscent of associative learning where impressions of previous scenarios are combined to cope with new ones . In order to increase the expressivity of this approach , we allocate one real-valued coefficient for each layer of weights of the underlying model . Learning these linear combinations is fast and simple ; and storing the coefficients requires only trivial overhead per task . For instance , in the case of LeNet-300-100 on RotatedMNIST with a basis set of 10 masks , new tasks require only 3× 10 = 30 parameters while a new SupSup mask at e.g . 10 % sparsity requires ≥ 25K weight indices to specify . We provide ample empirical evidence of the efficacy of our ImpressLearn approach on a variety of benchmarks , outlining the radical savings in parameters needed to be stored per task compared to SupSup . Homogeneous masks : Somewhat surprisingly , we can even generate all basis-masks from the same initial task using different random seeds for the learning algorithm ( but the same random backbone network ) . We show that with a sufficiently large number of such ” homogeneous ” impressions , our algorithm learns linear combinations with close to benchmark accuracy on new tasks . We are reminded of an infant learning by taking different ” snapshots ” of the same object , to infer properties of another . This ” homogeneous impression ” method is particularly useful to address possible drifts in the data . Akin to ensembling , it leverages the power of linear combinations for transfer learning . An additional important advantage is that the homogeneous method has no limit on the number of basis-mask we can generate ab initio ( since this requires a single task only ) . To provide another benchmark for our approach , we also try our linear combination approach with a set of completely random masks of desired sparsity . We demonstrate on several benchmarks that if we chose a sufficiently large set of such random basis masks , our optimization still yields good accuracy . While naturally , combinations of random masks lag behind the heterogeneous and also the homogeneous setting , we show that there is a trade-off between number of masks and their task-specificity ( non-randomness ) . In settings where producing task-specific basis masks is costly and where we might not wish to perform the SupSup mask search , our linear combination optimization still can yield satisfactory results with a large number of random masks . Example : LeNet-300-100 on RotMNIST To illustrate our approach and its performance , Fig . 2 shows the accuracy of our ImpressLearn approach , compared to SupSup on the Rotated MNIST task set . This allows us to share some observations which generalize to a wide variety of benchmarks ( see Sec . 4 ) . First , to show the power of the linear combination approach beyond pure transfer learning , we evaluate the accuracy of a basis mask on tasks different from the one the mask was trained on : we get close to random accuracy ( ” X ” in Fig . 2 , leftmost box plot ) . Next we illustrate how a linear combination of a small number of masks from different tasks ( heterogeneous impressions ) achieves close to benchmark accuracy on new tasks . In this case , combining a few masks already gives performance close to , even exceeding , the SupSup benchmark . This is followed by the performance of a set of homogeneous masks , where all impressions are generated from the first task ( middle of Fig . 2 ) . We can see that when using homogeneous impressions , we need a larger number of basis masks to achieve accuracy comparable to the heterogeneous setting , but that ultimately here also we match SupSup accuracy , with a vastly smaller number of parameters than the SupSup approach . Lastly , in the right part of Fig . 2 we study how our linear combination approach performs on tasks from the basis set when task labels are not provided at inference ( basis-GN setting ) . Here , the our optimization finds the ” correct ” mask , or a linear combination of equal or better accuracy , providing an alternative to the entropy-based GN-inference of ( Wortsman et al. , 2020 ) . Unknown task labels at inference and a new GN algorithm : Wortsman et al . ( 2020 ) showed that in the so called GN regime where task labels are given at training but not at inference , entropy minimization starting from a uniform combination of ( binary ) masks quickly singles out the ” correct ” mask for a task . Even though our masks for non-basis tasks are real-valued linear combinations of binary masks , we show that in the GN setting a similar entropy minimization allows us to infer the correct real-valued mask with the same approach . Moreover , in what we call the basis-GN regime where the unlabeled tasks come from the set of basis tasks , we show that our ImpressLearn optimization finds a linear combination that either favors the correct mask or yields an even better accuracy than the corresponding basis mask without using the entropy of the output . This means that the optimization routine of ImpressLearn gives an alternative algorithm for inference in the case of unknown task labels at test time : it finds a linear combination of masks that often outperforms the corresponding basis mask . In Section 2 , we briefly review related work and general approaches to countering catastrophic forgetting , highlighting research that motivated our approach . In Section 3 , we lay out the ImpressLearn algorithm , including our modified objective function . In Section 4 , we demonstrate the effectiveness of ImpressLearn on a variety of benchmark data sets and architectures to show close-to-benchmark performance with a drastically reduced parameter count on new tasks , especially where the number of possible tasks is large . We discuss our results including the trade offs between homogeneous and heterogeneous masks . Finally , in Section 5 we talk about limitations of our work , and areas for future research . 2 RELATED WORK . In practice , intelligence systems should be able to learn a variety of tasks incrementally and without experiencing catastrophic forgetting—degrading performance on previously acquired skills ( McCloskey & Cohen , 1989 ) , while possibly transferring current knowledge to facilitate future training . Oftentimes tasks are diverse and not available concurrently , making joint training impractical . Conversely , ordinary finetuning ( continued training of a pre-trained network ) inevitably leads to catastrophic forgetting . Continual learning encompasses a broad spectrum of algorithms and architectures that address these issues and propose systems capable of learning from an incremental stream of tasks while minimizing catastrophic forgetting . Most naturally , these techniques are categorized into three groups described below ( Delange et al. , 2021 ; Wortsman et al. , 2020 ) . Regularization-based methods : This class of algorithms trains on new tasks by finetuning weights but attempts to retain performance on previously learned tasks through regularization . A number of studies assess the importance of individual parameters for previous tasks and penalize their displacement accordingly during optimization . Pioneering this approach , Serra et al . ( 2018 ) estimates parameter importance using a Laplace-approximated posterior distribution after training on earlier tasks . Zenke et al . ( 2017 ) impose a quadratic penalty proportional to the accumulated sensitivity of previous loss functions to perturbations in the corresponding parameter ; Aljundi et al . ( 2018 ) use the same strategy but accumulate sensitivity of the network output to parameter perturbations instead . In contrast , Li & Hoiem ( 2018 ) regularize by means of distilling current knowledge on the incoming task ’ s data and using it during finetuning . Regularization-based methods require no additional memory overhead per task and hence are advantageous in capacity constrained settings . However , the plasticity of a network decreases with more tasks , imposing a natural limit on new tasks , which creates a trade-off between learning new tasks and catastrophic forgetting . Replay methods : Techniques in this category preserve performance on prior tasks by replaying , rehearsing or otherwise utilizing representative samples from the corresponding data distributions . Most commonly , replay models store examples of seen data in a separate memory buffer ( Rebuffi et al. , 2017 ; Rolnick et al. , 2019 ; Riemer et al. , 2019 ) ; others maintain generators that approximate the original data distribution and provide pseudo-examples ( Atkinson et al. , 2021 ; Shin et al. , 2017 ) . While the majority of algorithms in this group replay stored examples during optimization to mimic joint training , Lopez-Paz & Ranzato ( 2017 ) uses them to constrain optimization space and ensure positive knowledge transfer . Replay methods require additional memory to store data samples or allocate generators , however , these costs are usually kept fixed . For this reason , like regularizationbased models , replay models exhibit poor stability-plasticity trade-off with more tasks , but often come with increased memory requirements when compared to regularization based methods . Parameter isolation methods : These methods allocate new parameters for incoming tasks and feature little to no interference between previously learned tasks . Rusu et al . ( 2016 ) allocate a new copy of the network and enable forward transfer learning with lateral connections going into new modules . Ren et al . ( 2017 ) combine individual learners in a decision tree and eliminate outdated models with tree pruning . A large body of recent algorithms piggyback on a single backbone network shared by all tasks . As such , Wen et al . ( 2020 ) ( BatchEnsemble ) operate on a fixed pretrained network and , for each incoming task , optimize for a rank one parameter mask applied to the backbone at inference . Mallya & Lazebnik ( 2018 ) ( PackNet ) use pruning to assign subsets of free parameters of a backbone network to individual tasks by issuing one binary parameter mask per task . Assigned parameters are forever frozen at their trained values , limiting capacity of the network for future tasks . In a subsequent study , Mallya et al . ( 2018 ) ( Piggyback ) lift this limitation by directly optimizing per-task binary masks and applying them to a fixed pretrained network . SupSup Our ImpressLearn algorithm is most closely related to yet another similar method called SupSup ( Wortsman et al. , 2020 ) . This algorithm trains individual per-task binary masks but applies them to a randomly-initialized network , leveraging the existence of supermasks ( Zhou et al. , 2019 ) . The mask optimization algorithm , edge-popup ( Ramanujan et al. , 2019 ) , uses a heaviside function to binarize mask values on the forward pass and employs a straight-through estimator when computing gradients . In addition , Wortsman et al . ( 2020 ) propose different training and inference modes depending on availability of task identifiers ; e.g. , GG refers to the scenario when task identifies are known during both training and inference , while in GN they are available only during training . For the GN case Wortsman et al . ( 2020 ) introduce a one-shot algorithm to infer task identity by minimizing the output entropy , starting from a uniform linear combination of masks and optimizing the coefficients . While at first glance this algorithm resembles ours , there are essential differences as we use optimization of a refined linear combination to learn new tasks . While SupSup and other related methods suffer no catastrophic forgetting regardless of the number of tasks , they are required to store the corresponding parameter mask for each task , which is costly . SupSup try to address this by storing masks as attractors of a Hopfield network , but it is unclear how feasible this approach is .

Summary: In the current paper, the authors propose a novel way to adapt an existing continual learning algorithm, leveraging principles from transfer learning.  More concretely, they learn an initial set of masks (or impressions) from a small number of basis tasks, and they use afterwards linear combinations of these masks in the learning process of new tasks. Therefore, their approach is able  to generalize to new tasks, allowing for scalable and parameter efficient continual learning (with much lower parameter overhead than existing methods).