CPR: Classifier-Projection Regularization for Continual Learning

1 INTRODUCTION . Catastrophic forgetting ( McCloskey & Cohen , 1989 ) is a central challenge in continual learning ( CL ) : when training a model on a new task , there may be a loss of performance ( e.g. , decrease in accuracy ) when applying the updated model to previous tasks . At the heart of catastrophic forgetting is the stability-plasticity dilemma ( Carpenter & Grossberg , 1987 ; Mermillod et al. , 2013 ) , where a model exhibits high stability on previously trained tasks , but suffers from low plasticity for the integration of new knowledge ( and vice-versa ) . Attempts to overcome this challenge in neural network-based CL can be grouped into three main strategies : regularization methods ( Li & Hoiem , 2017 ; Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ; Nguyen et al. , 2018 ; Ahn et al. , 2019 ; Aljundi et al. , 2019 ) , memory replay ( Lopez-Paz & Ranzato , 2017 ; Shin et al. , 2017 ; Rebuffi et al. , 2017 ; Kemker & Kanan , 2018 ) , and dynamic network architecture ( Rusu et al. , 2016 ; Yoon et al. , 2018 ; Golkar et al. , 2019 ) . In particular , regularization methods that control model weights bear the longest history due to its simplicity and efficiency to control the trade-off for a fixed model capacity . In parallel , several recent methods seek to improve the generalization of neural network models trained on a single task by promoting wide local minima ( Keskar et al. , 2017 ; Chaudhari et al. , 2019 ; Pereyra et al. , 2017 ; Zhang et al. , 2018 ) . Broadly speaking , these efforts have experimentally shown that models trained with wide local minima-promoting regularizers achieve better generalization and higher accuracy ( Keskar et al. , 2017 ; Pereyra et al. , 2017 ; Chaudhari et al. , 2019 ; Zhang et al. , 2018 ) , and can be more robust to weight perturbations ( Zhang et al. , 2018 ) when compared to usual training methods . Despite the promising results , methods that promote wide local minima have yet to be applied to CL . In this paper , we make a novel connection between wide local minima in neural networks and regularization-based CL methods . The typical regularization-based CL aims to preserve important weight parameters used in past tasks by penalizing large deviations when learning new tasks . As ∗Corresponding author ( E-mail : tsmoon @ snu.ac.kr ) shown in the top of Fig . 1 , a popular geometric intuition ( as first given in EWC ( Kirkpatrick et al. , 2017 ) ) for such CL methods is to consider the ( uncertainty ) ellipsoid of parameters around the local minima . When learning new tasks , parameter updates are selected in order to not significantly hinder model performance on past tasks . Our intuition is that promoting a wide local minima—which conceptually stands for local minima having a flat , rounded uncertainty ellipsoid—can be particularly beneficial for regularization-based CL methods by facilitating diverse update directions for the new tasks ( i.e. , improves plasticity ) while not hurting the past tasks ( i.e. , retains stability ) . As shown in the bottom of Fig . 1 , when the ellipsoid containing the parameters with low-error is wider , i.e. , when the wide local minima exists , there is more flexibility in finding a parameter that performs well for all tasks after learning a sequence of new tasks . We provide further details in Section 2.1 . Based on the above intuition , we propose a general , yet simple patch that can be applied to existing regularization-based CL methods dubbed as Classifier-Projection Regularization ( CPR ) . Our method implements an additional regularization term that promotes wide local minima by maximizing the entropy of the classifier ’ s output distribution . Furthermore , from a theory standpoint , we make an observation that our CPR term can be further interpreted in terms of information projection ( I-projection ) formulations ( Cover & Thomas , 2012 ; Murphy , 2012 ; Csiszár & Matus , 2003 ; Walsh & Regalia , 2010 ; Amari et al. , 2001 ; Csiszár & Matus , 2003 ; Csiszár & Shields , 2004 ) found in information theory . Namely , we argue that applying CPR corresponds to projecting a classifier ’ s output onto a Kullback-Leibler ( KL ) divergence ball of finite radius centered around the uniform distribution . By applying the Pythagorean theorem for KL divergence , we then prove that this projection may ( in theory ) improve the performance of continual learning methods . Through extensive experiments on several benchmark datasets , we demonstrate that applying CPR can significantly improve the performance of the state-of-the-art regularization-based CL : using our simple patch improves both the stability and plasticity and , hence , achieves better average accuracy almost uniformly across the tested algorithms and datasets—confirming our intuition of wide local minima in Fig . 1 . Furthermore , we use a feature map visualization that compares methods trained with and without CPR to further corroborate the effectiveness of our method . 2 CPR : CLASSIFIER-PROJECTION REGULARIZATION FOR WIDE LOCAL MINIMUM . In this section , we elaborate in detail the core motivation outlined in Fig . 1 , then formalize CPR as the combination of two regularization terms : one stemming from prior regularization-based CL methods , and the other that promotes a wide local minima . Moreover , we provide an information-geometric interpretation ( Csiszár , 1984 ; Cover & Thomas , 2012 ; Murphy , 2012 ) for the observed gain in performance when applying CPR to CL . We consider continual learning of T classification tasks , where each task contains N training samplelabel pairs { ( xtn , ytn ) } Nn=1 , t ∈ [ 1 , · · · , T ] with xtn ∈ Rd , and the labels of each task has Mt classes , i.e. , ytn ∈ [ 1 , · · · , Mt ] . Note that task boundaries are given in evaluation time ; i.e. , we consider a task-aware setting . We denote fθ : Rd → ∆M as a neural network-based classification model with softmax output layer parameterized by θ . 2.1 MOTIVATION : INTRODUCING WIDE LOCAL MINIMA IN CONTINUAL LEARNING . Considering the setting of typical regularization-based CL ( top of Fig . 1 ) , we denote θ∗i as parameters that achieve local minima for a specific task i and θ̂i is that obtained with regularization terms . Assuming that θ∗1 is learnt , when learning task 2 , an appropriate regularization updates the parameters from θ∗1 to θ̂2 instead of θ ∗ 2 , since θ̂2 achieves low-errors on both tasks 1 and 2 . However , when the low-error regimes ( ellipsoids in Fig . 1 ) are narrow , it is often infeasible to obtain a parameter that performs well on all three tasks . This situation results in the trade-off between stability and plasticity in regularization-based CL ( Carpenter & Grossberg , 1987 ) . Namely , stronger regularization strength ( direction towards past tasks ) brings more stability ( θ̂13 ) , and hence less forgetting on past tasks . In contrast , weaker regularization strength ( direction towards future tasks ) leads to more plasticity so that the updated parameter θ̂23 performs better on recent tasks , at the cost of compromising the performance of past tasks . A key problem in the previous setting is that the parameter regimes that achieve low error for each task are often narrow and do not overlap with each other . Therefore , a straightforward solution is to enlarge the low-error regimes such that they have non-empty intersections with higher chance . This observation motivates us to consider wide local minima for each task in CL ( bottom of Fig . 1 ) . With wide local minima for each task , a regularization-based CL can more easily find a parameter , θ̂3 , that is close to the the local minimas for each task , i.e. , { θ∗i } 3i=1 . Moreover , it suggests that once we promote the wide local minima of neural networks during continual learning , both the stability and plasticity could potentially be improved and result in simultaneously higher accuracy for all task — which is later verified in our experimental ( see Sec . 3 ) . In the next section , we introduce the formulation of wide local minima in CL . 2.2 CLASSIFIER PROJECTION REGULARIZATION FOR CONTINUAL LEARNING . Regularization-based continual learning Typical regularization-based CL methods attach a regularization term that penalizes the deviation of important parameters learned from past tasks in order to mitigate catastrophic forgetting . The general loss form for these methods when learning task t is LtCL ( θ ) = L t CE ( θ ) + λ ∑ i Ωt−1i ( θi − θ t−1 i ) 2 , ( 1 ) where LtCE ( θ ) is the ordinary cross-entropy loss function for task t , λ is the dimensionless regularization strength , Ωt−1 = { Ωt−1i } is the set of estimates of the weight importance , and { θ t−1 i } is the parameter learned until task t− 1 . A variety of previous work , e.g. , EWC ( Kirkpatrick et al. , 2017 ) , SI ( Zenke et al. , 2017 ) , MAS ( Aljundi et al. , 2018 ) , and RWalk ( Chaudhry et al. , 2018 ) , proposed different ways of calculating Ωt−1 to measure weight importance . Single-task wide local minima Several recent schemes have been proposed ( Pereyra et al. , 2017 ; Szegedy et al. , 2016 ; Zhang et al. , 2018 ) to promote wide local minima of a neural network for solving a single task . These approaches can be unified by the following common loss form LWLM ( θ ) = LCE ( θ ) + β N N∑ n=1 DKL ( fθ ( xn ) ‖g ) , ( 2 ) where g is some probability distribution in ∆M that regularizes the classifier output fθ , β is a trade-off parameter , andDKL ( ·‖· ) is the KL divergence ( Cover & Thomas , 2012 ) . Note that , for example , when g is uniform distribution PU in ∆M , the regularization term corresponds to entropy maximization proposed in Pereyra et al . ( 2017 ) , and when g is another classifier ’ s output fθ′ , Eq . ( 2 ) becomes equivalent to the loss function in Zhang et al . ( 2018 ) . CPR : Achieving wide local minima in continual learning Combining the above two regularization terms , we propose the CPR as the following loss form for learning task t : LtCPR ( θ ) = L t CE ( θ ) + β N N∑ n=1 DKL ( fθ ( x t n ) ‖PU ) + λ ∑ i Ωt−1i ( θi − θ t−1 i ) 2 , ( 3 ) where λ and β are the regularization parameters . The first regularization term promotes the wide local minima while learning task t by using PU as the regularizing distribution g in ( 2 ) , and the second term is from the typical regularization-based CL . Note that this formulation is oblivious to Ωt−1 and , hence , it can be applied to any state-of-the-art regularization-based CL methods . In our experiments , we show that the simple addition of the KL-term can significantly boost the performance of several representative state-of-the-art methods , confirming our intuition on wide local minima for CL given in Section 2.1 and Fig 1 . Furthermore , we show in the next section that the KL-term can be geometrically interpreted in terms of information projections ( Csiszár , 1984 ; Cover & Thomas , 2012 ; Murphy , 2012 ) , providing an additional argument ( besides promoting wide local minima ) for the benefit of using CPR in continual learning .

Summary: The authors argue that achieving wide local minima during the training of tasks, is beneficial for continual learning. The plausible intuition (explained in Fig. 1) is that it is easier to find a parameter setting that is beneficial for all tasks when tasks have wide local minima. They enforce wide local minima by adding an entropy loss to the classifier ( a known strategy). The loss is further combined with any weight regularization loss, like EWC, SI, MAS.