Addressing the Stability-Plasticity Dilemma via Knowledge-Aware Continual Learning

1 INTRODUCTION . Continual learning ( CL ) aims to build intelligent agents based on deep neural networks that can learn a sequence of tasks , use previous knowledge in future learning , and accumulate on it without forgetting . The main challenge in this paradigm is the stability-plasticity dilemma ( Mermillod et al. , 2013 ) . Optimizing all model weights on a new task , highest plasticity , causes catastrophic forgetting of previous tasks ( McCloskey & Cohen , 1989 ) . While fixing all weights , highest stability , hinders learning new tasks . Finding the right balance between stability and plasticity is challenging . This sharpens the community ’ s focus on the forgetting problem . The excessive focus on one aspect impedes building agents that balance between mitigating forgetting and exploiting relevant knowledge in future learning while considering the capacity constraints ( Dı́az-Rodrı́guez et al. , 2018 ) . Task-specific components methods ( Section 2 ) address the stability-plasticity dilemma by allocating different connections for each task . The newly added connections for a new task are flexible to learn while the connections of previously learned tasks are fixed . Despite that these methods are quite successful in mitigating forgetting , some limitations remain to be tackled to achieve the balance between the above-mentioned CL desiderata . First , new components ( connections/neurons ) are allocated in each layer for every new task . However , the new components might capture knowledge that already exists in the learned components of previous similar tasks . In this case , adding new components would be redundant and resource inefficient . Second , the topology of the new task is allocated using the unimportant components of previous tasks . The design choice of the new topology is made to protect previous knowledge but does not take into consideration the usefulness of this topology for the current task . This limits the forward transfer of useful knowledge for the new task and its speed of learning . Further discussions of these limitations are in Appendix A . 1The code will be made publicly available after the publication of this paper and now can be found in the supplementary material in the system . In response to these limitations , we study the core of the stability-plasticity dilemma in the CL paradigm . In particular , we address the following question : Which components are eligible to be reused , added , updated , or fixed when a CL agent faces a new task to achieve the balance between the CL desiderata ? With our proposed Knowledge-Aware contiuNal learner ( KAN ) , we find that considering the semantic similarity between old and new classes is crucial in addressing this question . With the awareness of existing knowledge , the CL agent could identify similar previous knowledge that could be reusable in learning a new task ( forward transfer ) and add the necessary components only to capture the specific knowledge that can not be explained with the existing knowledge ( resource efficiency ) . Moreover , the agent could protect the dissimilar previous knowledge and limit its transfer to the new task ( mitigating forgetting ) . To the best of our knowledge , the recent work , CAT ( Ke et al. , 2020 ) , is the only one that attempted to consider the task similarities to balance between forward transfer and mitigating forgetting . CAT was proposed for the task-Incremental Learning ( task-IL ) scenario , where each task is a separate classification problem . In this work , we aim to tackle the more challenging class-Incremental Learning ( class-IL ) scenario where a unified classifier is used for all seen classes . The inaccessibility of the task identity at inference in the latter scenario arises the following challenges : ( 1 ) ambiguities between old and new tasks which affect the balance between maintaining forgetting and forward transfer . ( 2 ) Inability to select the corresponding components to a task at inference ; all the existing components in the model are involved in making predictions . To fully analyze the stability-plasticity dilemma in class-IL , given the aforementioned challenges , we focus on the rehearsal-free strategy ( Section 2 ) in which there is no reliance on past data to address forgetting and a fixed-capacity model is used . We also report the performance of the model in Task-IL to further demonstrate the challenges in class-IL . Our contributions are : • We demonstrate that considering the semantic relation between old and new classes is crucial in addressing the stability-plasticity dilemma and the balance between CL desiderata . • We show that considering both previous and current tasks in allocating the initial topology of a new task allows for : selective transfer for the relevant ( similar ) knowledge , protecting the irrelevant ( dissimilar ) knowledge , and identifying the reusable previous components to avoid allocating unnecessary resources . • We demonstrate that the standard softmax-classification layer is a source of constituting the ambiguities between old and new tasks in class-IL . We study different setups for the output layer which significantly improve the performance of rehearsal-free methods . Our analyses reveal the limitations of the softmax in class-IL and shed light on the necessity of paying more attention to these limitations to narrow the gap between class-IL and task-IL . • We propose Knowledge-Aware coNtinual leaner ( KAN ) ; a resource-efficient task-specific components method to explain these findings . 2 RELATED WORK . We divide the CL methods into two main categories : Rehearsal-free and Rehearsal-based methods . Rehearsal-free methods . In this category , previous data is inaccessible during future learning . There are two strategies to address forgetting . ( 1 ) Task-specific components . Specific components are assigned to each task . The components of previous tasks are stable during future learning . While the components that have not been allocated to any task are flexible for future learning . These methods either extend the model size when facing a new task ( Rusu et al. , 2016 ; Yoon et al. , 2018 ) or uses a fixed-capacity and each task is trained either using sparse sub-network within the model ( Mallya & Lazebnik , 2018 ; Mallya et al. , 2018 ; Yoon et al. , 2019 ) . These methods are limited to task-IL as they require the task identity to specify the components of each task at inference . Recently , few methods have been proposed for class-IL using fixed-capacity model . SupSup ( Wortsman et al. , 2020 ) learns a mask for each task over randomly initialized fixed network . FSLL ( Mazumder et al. , 2021 ) addressed the few-shot lifelong learning setup . It relies on the availability of large data for the first task to train a dense model . Then , few unimportant parameters are used to learn each task . SpaceNet ( Sokar et al. , 2021c ) learns sparse sub-network for each task from scratch using dynamic sparse training ( Mocanu et al. , 2018 ; Hoefler et al. , 2021 ) where the weights and the sparse topology are optimized simultaneously . ( 2 ) Regularization-based . A fixed-capacity model is used and all parameters are involved in optimizing each task . Forgetting is addressed either by constraining changes in the important weights of previous tasks ( Zenke et al. , 2017 ; Kirkpatrick et al. , 2017 ; Aljundi et al. , 2018 ) or via distillation loss ( Li & Hoiem , 2017 ; Dhar et al. , 2019 ) . Previous studies ( Kemker et al. , 2018 ; Hsu et al. , 2018 ; Farquhar & Gal , 2019 ; van de Ven & Tolias , 2018 ) showed the big performance gap of most of these methods between class-IL and task-IL . Rehearsal-based methods . In this category , forgetting is addressed by replaying : ( 1 ) a subset of old samples ( Rebuffi et al. , 2017 ; Lopez-Paz & Ranzato , 2017 ; Riemer et al. , 2018 ; Chaudhry et al. , 2018 ) , ( 2 ) pseudo-samples from a generative model ( Mocanu et al. , 2016 ; Shin et al. , 2017 ; Sokar et al. , 2021a ) , or ( 3 ) generative high-level features ( Liu et al. , 2020 ; van de Ven et al. , 2020 ) . They use a classification model and buffer to store old samples or a generative model to generate them . Other related studies . Recent work by ( Ramasesh et al. , 2020 ) evaluates forgetting in hidden representations in Task-IL and shows how semantic similarities influence it . They showed that higher layers are more prone to forgetting and intermediate similarity results in maximal forgetting . SAM ( Sokar et al. , 2021b ) studies the importance of selective transfer in dense networks by metatraining ( Finn et al. , 2017 ) a self-attention mechanism ( Hu et al. , 2018 ) . CAT ( Ke et al. , 2020 ) addresses the relation between task similarities and forward/backward transfer in task-IL where they rely on the task identity to find the previous similar knowledge in dense networks . In ( Chen et al. , 2020 ) , the lottery ticket hypothesis ( Frankle & Carbin , 2018 ) is studied for the CL paradigm . 3 PRELIMINARIES . 3.1 PROBLEM FORMULATION . We consider the problem of learning a sequence of T tasks . Each task brings a new set of C classes . Each task t has its own dataset Dt = { Dtc } |Cc=1 , where Dtc is the data of class c in task t. Once we have trained task t , its data is discarded . The goal is to sequentially learn the model f t with a unified classifier that can map any input to its corresponding target for all seen classes up to time step t . 3.2 REHEARSAL-FREE STRATEGY . In this work , we focus to analyze how the model components should be altered for each task in task-specific components methods . In addition , we compare the behavior of these methods against regularization-based methods which use all model components for each task . 3.2.1 SPACENET . We analyze the recent task-specific components method , SpaceNet ( Sokar et al. , 2021c ) . The motivations for choosing SpaceNet are : ( 1 ) unlike most task-specific component methods , it does not rely on task identity ; making it applicable for class-IL . ( 2 ) Instead of relying on a pre-trained model , it trains sparse subnetworks from scratch . These are favorable for real-world applications in which we can not assume the availability of task identity at inference and large datasets for pretraining . To the best of our knowledge , it is only existing work that fulfills these objectives . The main idea of SpaceNet is to learn sparse representation for each task to reduce the interference between tasks . SpaceNet consists of three main steps . ( 1 ) Sparse connections allocation : sparse sub-network W tl is randomly allocated for each task t in each layer l between subset of the unimportant neurons hsell−1 and h sel l ; where h sel l are randomly selected from the free neurons h free l that have not been reserved for any previous task . ( 2 ) Dynamic sparse training : the sparse weights and topology are optimized through “ drop-and-grow ” cycles to produce sparse representation . ( 3 ) Neuron reservation : after learning , a fraction of the most important neurons for the current task hspecl are fixed and removed from the free list of neurons hfreel . The full details are provided in Appendix I. SpaceNet considers only previous tasks during the allocation of new connections for the new tasks and randomly allocates a new sub-network between the free neurons for each task . In this work , we propose novel criteria which consider both past and current tasks in allocation and allow reusing some of the past components instead of allocating new ones .

Summary: ### Summary: The paper studies replay-free continual learning with the focus on the plasticity-stability dilemma. More specifically, the paper proposes the KAN method for class-incremental learning where the forward transfer is achieved by detecting similar knowledge and reusing the first few layers, and negative backward transfer can be alleviated sparse connection allocations for different classes of different tasks. The paper also studies a limitation of the softmax layer, which is an interesting contribution.