ImpressLearn: Continual Learning via Combined Task Impressions

Keywords : Catastrophic forgetting , Continual learning , Neural networks , Masking 1 INTRODUCTION . Sequential learning without catastrophic forgetting has been an area of active research in machine learning for some time ( Maes et al. , 1996 ; Thrun & Pratt , 1998 ; Serra et al. , 2018 ) . A precondition for achieving artificial general intelligence ( AGI ) is that models be able to learn and remember a wide variety of tasks sequentially , without forgetting previously learned ones . In real-world scenarios , data from different tasks may not be available simultaneously , which makes it imperative to both allow continued learning as well as to avoid catastrophic forgetting of a potentially unbounded number of tasks ( see also The Sequential Learning Problem ( McCloskey & Cohen , 1989 ) , Constraints Imposed by Learning and Forgetting Functions ( Ratcliff , 1990 ) and Lifelong Learning Algorithms ( Thrun & Pratt , 1998 ) ) . Recently , some successful approaches to combat this problem use task specific submodels , which allow neural networks to context-switch between different learning tasks ( Wortsman et al. , 2020 ; Mallya et al. , 2018 ; Mancini et al. , 2018 ) . The underlying context for each task can be represented as ” filters ” or ” masks ” , altering the network connections for each task . Yet all of these approaches scale unfavorably with the number of unique tasks to be learned . Our contribution : We propose a novel method which exploits transfer learning and network masking to sequentially learn a theoretically unlimited number of tasks with much lower parameter overhead than prevailing benchmarks . Our method , termed ImpressLearn , uses elements from Supermasks in Superposition ( SupSup ) by Wortsman et al . ( 2020 ) . The SupSup approach leverages the observation that even within randomly weighted neural networks there exist task-specific supermasks—subnetworks produced by overlaying a binary mask that selectively removes connections—which achieve good performance on the task . These supermasks can be learned from task-specific data and stored , one mask per task . At inference , the appropriate task-specific mask is applied when task identity is known . When task-labels of a previously seen task are not known , the correct mask can be inferred via entropy considerations . A ” basis ” of masks : Our approach goes further , driven by two core ideas . In order to dramatically improve scaling of the parameters with the number of different tasks , we reduce the number of necessary masks to a constant number , independent of the number of different tasks . Our basis-masks are constructed from a small number of initial tasks . Leveraging transfer learning , this set of learned masks , each of which can be interpreted as an ” impression ” of a previously seen prior task , serves as a set of latent features for new learning objectives , encoding common structural information . Our second idea is to use the power of linear combinations to combine these impressions to quickly learn a real-valued mask that performs well for new tasks . Hence , other than a fixed number of basis-masks , only a small number of coefficients need to be learned and stored for each subsequent task . This greatly benefits scalability , a major drawback of previous approaches , as this in principle allows for an unlimited number of new tasks while the parameter overhead is several orders of magnitude smaller than storing even a compressed binary network mask . Specifically , ImpressLearn uses the first few tasks from a multi-task set to generate a relatively small number of binary masks . For subsequent tasks , these impressions are used akin to a basis : new tasks are learned through a simple linear combination of the basis-impressions . This might be reminiscent of associative learning where impressions of previous scenarios are combined to cope with new ones . In order to increase the expressivity of this approach , we allocate one real-valued coefficient for each layer of weights of the underlying model . Learning these linear combinations is fast and simple ; and storing the coefficients requires only trivial overhead per task . For instance , in the case of LeNet-300-100 on RotatedMNIST with a basis set of 10 masks , new tasks require only 3× 10 = 30 parameters while a new SupSup mask at e.g . 10 % sparsity requires ≥ 25K weight indices to specify . We provide ample empirical evidence of the efficacy of our ImpressLearn approach on a variety of benchmarks , outlining the radical savings in parameters needed to be stored per task compared to SupSup . Homogeneous masks : Somewhat surprisingly , we can even generate all basis-masks from the same initial task using different random seeds for the learning algorithm ( but the same random backbone network ) . We show that with a sufficiently large number of such ” homogeneous ” impressions , our algorithm learns linear combinations with close to benchmark accuracy on new tasks . We are reminded of an infant learning by taking different ” snapshots ” of the same object , to infer properties of another . This ” homogeneous impression ” method is particularly useful to address possible drifts in the data . Akin to ensembling , it leverages the power of linear combinations for transfer learning . An additional important advantage is that the homogeneous method has no limit on the number of basis-mask we can generate ab initio ( since this requires a single task only ) . To provide another benchmark for our approach , we also try our linear combination approach with a set of completely random masks of desired sparsity . We demonstrate on several benchmarks that if we chose a sufficiently large set of such random basis masks , our optimization still yields good accuracy . While naturally , combinations of random masks lag behind the heterogeneous and also the homogeneous setting , we show that there is a trade-off between number of masks and their task-specificity ( non-randomness ) . In settings where producing task-specific basis masks is costly and where we might not wish to perform the SupSup mask search , our linear combination optimization still can yield satisfactory results with a large number of random masks . Example : LeNet-300-100 on RotMNIST To illustrate our approach and its performance , Fig . 2 shows the accuracy of our ImpressLearn approach , compared to SupSup on the Rotated MNIST task set . This allows us to share some observations which generalize to a wide variety of benchmarks ( see Sec . 4 ) . First , to show the power of the linear combination approach beyond pure transfer learning , we evaluate the accuracy of a basis mask on tasks different from the one the mask was trained on : we get close to random accuracy ( ” X ” in Fig . 2 , leftmost box plot ) . Next we illustrate how a linear combination of a small number of masks from different tasks ( heterogeneous impressions ) achieves close to benchmark accuracy on new tasks . In this case , combining a few masks already gives performance close to , even exceeding , the SupSup benchmark . This is followed by the performance of a set of homogeneous masks , where all impressions are generated from the first task ( middle of Fig . 2 ) . We can see that when using homogeneous impressions , we need a larger number of basis masks to achieve accuracy comparable to the heterogeneous setting , but that ultimately here also we match SupSup accuracy , with a vastly smaller number of parameters than the SupSup approach . Lastly , in the right part of Fig . 2 we study how our linear combination approach performs on tasks from the basis set when task labels are not provided at inference ( basis-GN setting ) . Here , the our optimization finds the ” correct ” mask , or a linear combination of equal or better accuracy , providing an alternative to the entropy-based GN-inference of ( Wortsman et al. , 2020 ) . Unknown task labels at inference and a new GN algorithm : Wortsman et al . ( 2020 ) showed that in the so called GN regime where task labels are given at training but not at inference , entropy minimization starting from a uniform combination of ( binary ) masks quickly singles out the ” correct ” mask for a task . Even though our masks for non-basis tasks are real-valued linear combinations of binary masks , we show that in the GN setting a similar entropy minimization allows us to infer the correct real-valued mask with the same approach . Moreover , in what we call the basis-GN regime where the unlabeled tasks come from the set of basis tasks , we show that our ImpressLearn optimization finds a linear combination that either favors the correct mask or yields an even better accuracy than the corresponding basis mask without using the entropy of the output . This means that the optimization routine of ImpressLearn gives an alternative algorithm for inference in the case of unknown task labels at test time : it finds a linear combination of masks that often outperforms the corresponding basis mask . In Section 2 , we briefly review related work and general approaches to countering catastrophic forgetting , highlighting research that motivated our approach . In Section 3 , we lay out the ImpressLearn algorithm , including our modified objective function . In Section 4 , we demonstrate the effectiveness of ImpressLearn on a variety of benchmark data sets and architectures to show close-to-benchmark performance with a drastically reduced parameter count on new tasks , especially where the number of possible tasks is large . We discuss our results including the trade offs between homogeneous and heterogeneous masks . Finally , in Section 5 we talk about limitations of our work , and areas for future research . 2 RELATED WORK . In practice , intelligence systems should be able to learn a variety of tasks incrementally and without experiencing catastrophic forgetting—degrading performance on previously acquired skills ( McCloskey & Cohen , 1989 ) , while possibly transferring current knowledge to facilitate future training . Oftentimes tasks are diverse and not available concurrently , making joint training impractical . Conversely , ordinary finetuning ( continued training of a pre-trained network ) inevitably leads to catastrophic forgetting . Continual learning encompasses a broad spectrum of algorithms and architectures that address these issues and propose systems capable of learning from an incremental stream of tasks while minimizing catastrophic forgetting . Most naturally , these techniques are categorized into three groups described below ( Delange et al. , 2021 ; Wortsman et al. , 2020 ) . Regularization-based methods : This class of algorithms trains on new tasks by finetuning weights but attempts to retain performance on previously learned tasks through regularization . A number of studies assess the importance of individual parameters for previous tasks and penalize their displacement accordingly during optimization . Pioneering this approach , Serra et al . ( 2018 ) estimates parameter importance using a Laplace-approximated posterior distribution after training on earlier tasks . Zenke et al . ( 2017 ) impose a quadratic penalty proportional to the accumulated sensitivity of previous loss functions to perturbations in the corresponding parameter ; Aljundi et al . ( 2018 ) use the same strategy but accumulate sensitivity of the network output to parameter perturbations instead . In contrast , Li & Hoiem ( 2018 ) regularize by means of distilling current knowledge on the incoming task ’ s data and using it during finetuning . Regularization-based methods require no additional memory overhead per task and hence are advantageous in capacity constrained settings . However , the plasticity of a network decreases with more tasks , imposing a natural limit on new tasks , which creates a trade-off between learning new tasks and catastrophic forgetting . Replay methods : Techniques in this category preserve performance on prior tasks by replaying , rehearsing or otherwise utilizing representative samples from the corresponding data distributions . Most commonly , replay models store examples of seen data in a separate memory buffer ( Rebuffi et al. , 2017 ; Rolnick et al. , 2019 ; Riemer et al. , 2019 ) ; others maintain generators that approximate the original data distribution and provide pseudo-examples ( Atkinson et al. , 2021 ; Shin et al. , 2017 ) . While the majority of algorithms in this group replay stored examples during optimization to mimic joint training , Lopez-Paz & Ranzato ( 2017 ) uses them to constrain optimization space and ensure positive knowledge transfer . Replay methods require additional memory to store data samples or allocate generators , however , these costs are usually kept fixed . For this reason , like regularizationbased models , replay models exhibit poor stability-plasticity trade-off with more tasks , but often come with increased memory requirements when compared to regularization based methods . Parameter isolation methods : These methods allocate new parameters for incoming tasks and feature little to no interference between previously learned tasks . Rusu et al . ( 2016 ) allocate a new copy of the network and enable forward transfer learning with lateral connections going into new modules . Ren et al . ( 2017 ) combine individual learners in a decision tree and eliminate outdated models with tree pruning . A large body of recent algorithms piggyback on a single backbone network shared by all tasks . As such , Wen et al . ( 2020 ) ( BatchEnsemble ) operate on a fixed pretrained network and , for each incoming task , optimize for a rank one parameter mask applied to the backbone at inference . Mallya & Lazebnik ( 2018 ) ( PackNet ) use pruning to assign subsets of free parameters of a backbone network to individual tasks by issuing one binary parameter mask per task . Assigned parameters are forever frozen at their trained values , limiting capacity of the network for future tasks . In a subsequent study , Mallya et al . ( 2018 ) ( Piggyback ) lift this limitation by directly optimizing per-task binary masks and applying them to a fixed pretrained network . SupSup Our ImpressLearn algorithm is most closely related to yet another similar method called SupSup ( Wortsman et al. , 2020 ) . This algorithm trains individual per-task binary masks but applies them to a randomly-initialized network , leveraging the existence of supermasks ( Zhou et al. , 2019 ) . The mask optimization algorithm , edge-popup ( Ramanujan et al. , 2019 ) , uses a heaviside function to binarize mask values on the forward pass and employs a straight-through estimator when computing gradients . In addition , Wortsman et al . ( 2020 ) propose different training and inference modes depending on availability of task identifiers ; e.g. , GG refers to the scenario when task identifies are known during both training and inference , while in GN they are available only during training . For the GN case Wortsman et al . ( 2020 ) introduce a one-shot algorithm to infer task identity by minimizing the output entropy , starting from a uniform linear combination of masks and optimizing the coefficients . While at first glance this algorithm resembles ours , there are essential differences as we use optimization of a refined linear combination to learn new tasks . While SupSup and other related methods suffer no catastrophic forgetting regardless of the number of tasks , they are required to store the corresponding parameter mask for each task , which is costly . SupSup try to address this by storing masks as attractors of a Hopfield network , but it is unclear how feasible this approach is .

Summary: The paper studies continual learning and that context the important and prominent problem of catastrophic forgetting. Similar to related work the paper achieves this through learning task-dependent masks on the neural network representation (effectively generating different sub-networks for different tasks).  The main contribution w.r.t. related work is that the masks are constrained to a sub-space which is spanned by a "basis of masks". The experiments show some reduction in the number of required parameters over a single baseline method.