Continual Learning via Principal Components Projection

1 INTRODUCTION . Human brains have this extraordinary ability to learn a large number of tasks incrementally with high accuracy . Both the learning process of and the learned knowledge for the tasks have little negative interference of each other . In fact , the learned knowledge earlier can even help the learning of new tasks later . Continual learning or lifelong learning attempts to make the computer to do the same using neural networks ( NNs ) . However , it has been proven challenging . One of the key problems or obstacles is the issue of catastrophic forgetting ( CF ) . This problem was first reported by McCloskey & Cohen ( 1989 ) . In their work on NN , they found that when learning a sequence of tasks incrementally in an NN , the training of each new task can cause the network to forget the information or knowledge learned from previous tasks . This usually means that the new task training may significantly modify the weights that have been learned for previous tasks , and thus degrade the model performance for the previous tasks . Without solving this problem , an NN is unsuitable for continual learning as it forgets the existing knowledge when it learns new things . In the past few years , CF has attracted a great deal of research attention ( Parisi et al. , 2018a ) . A literature review will be given in Section 2 . There are two main setups in continual learning : class continual learning ( CCL ) and task continual learning ( TCL ) ( Dhar et al. , 2019 ) . The general term continual learning has been used for both , which has caused some confusion when comparing results in some cases . In the CCL setup , each task consists of one or more classes to be learned together and the learning process builds a single model for all tasks for classifying all the classes of the tasks learned so far . In testing , a test instance from any class may be presented to the model for it to classify . There is no prior information about which task the test instance comes from . Formally , CCL can be defined as follows . Class continual learning ( CCL ) . Given a sequence of datasets D = { D1 , D2 , ... , DT } of T tasks . The dataset of task k is Dk = { ( xik , yik ) nk i=1 } , where each xik ∈ X is a feature vector from task k and yik ∈ Yk is the corresponding target class label of xik . All Yk ’ s are disjoint and ⋃T k=1 Yk = Y . The goal of CCL is to construct a single prediction function or model f : X → Y that can correctly identify the target class y for a given test instance x . In the TCL setup , each task is a separate classification problem ( e.g. , one task could be to classify different breeds of dogs and another task could be to classify different types of birds ) . Here , one model is built for each task , although all the models reside in the same neural network . In testing , the system knows which task each test instance belongs to and uses only the specific model for the task ( dog or bird classification ) to classify the test instance . Formally , TCL is defined as follows . Task continual learning ( TCL ) . Given a sequence of datasets D = { D1 , D2 , ... , DT } of T tasks . The dataset of task k is Dk = { ( ( xik , tk ) , yik ) nk i=1 } , where each xik ∈X is a feature vector from task k , tk ∈ T is the indicator of task k , and yik ∈ Yk ⊂ Y is the corresponding target class label . The goal of TCL is to construct a predictor f : X × T → Y to identify the correct class label y ∈ Yk for ( x , tk ) ( the given test instance x from task k ) . This paper works in the TCL setting and proposes a more effective technique to deal with CF . The proposed method is called principal components projection ( PCP ) , which is inspired by the OWM technique in ( Zeng et al. , 2019 ) . The idea is that in learning a new task , if we can ensure that gradient updates will occur only in the orthogonal directions to the input vectors of the previous tasks , then the weight updates for the new task will have little effect on the weights for the previous tasks . However , OWM ( Zeng et al. , 2019 ) has a key weakness . Its weight update in learning the new task occurs only in the solution space of the previous tasks , which restricts the solution space for the new task . When the new task is quite different from previous tasks , the learning accuracy will suffer ( see Section 3.1 ) . The proposed PCP method deals with the problem . The key idea of PCP is that it computes the principal components of the input vectors ( as in PCA ) for each task at each level of the network and uses them to transform the input and then to project the gradient updates to the orthogonal directions in learning the new task so that the updates will not affect the previous tasks and can also go beyond the solution space of previous tasks ( see the details in Section 3.2 ) . This results in significantly better results . Moreover , unlike many existing TCL methods ( see Section 2 ) , PCP does not need to store any sampled data from previous tasks or generate pseudo past data and use them to help learn a new task . Empirical evaluation shows that PCP outperforms existing stat-of-the-art baselines considerably . 2 RELATED WORK . Catastrophic forgetting ( CF ) has been dealt with since early 1990s . Li & Hoiem ( 2016 ) gave a good summary of the early work . The paper also proposed a new technique called LwF to deal with forgetting . It learns the parameters for the new task θn with the help of shared parameters for all tasks θs and parameters for old tasks θo without degrading much of the performance of the old tasks . The idea is to optimize θs and θn on the new task with the constraint that the predictions on the new task ’ s examples using θs and θo do not shift much . Rusu et al . ( 2016 ) proposed a progressive neural network that retains a pool of pre-trained models and learns lateral connections among them . Kirkpatrick et al . ( 2017 ) proposed a model called EWC that quantifies the importance of weights to previous tasks , and selectively adjusts the plasticity of weights . This method is also used in ( Schwarz et al. , 2018 ) . Motivated by EWC ( Kirkpatrick et al. , 2017 ) , Zenke et al . ( 2017 ) measured the synapse consolidation strength in an online fashion and used it as regularization to deal with CF . Rebuffi et al . ( 2017 ) retained an exemplar set that best approximates the previous tasks to help train the new task . Similar ideas were also used in ( Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2019 ; Ans et al. , 2004 ) . ( Shin et al. , 2017 ; Kamra et al. , 2017 ; Rostami et al. , 2019 ) followed the Generative Adversarial Networks ( GANs ) framework ( Goodfellow , 2016 ) to produce generators for previous tasks , and learn parameters that fit a mixed set of real data of the new task and replayed/generated data of previous tasks . A network of experts is proposed by ( Aljundi et al. , 2016 ) to measure task relatedness to deal with catastrophic forgetting . Rannen Ep Triki et al . ( 2017 ) used an autoencoder to extend the LwF method ( Li & Hoiem , 2016 ) . Jung et al . ( 2016 ) proposed a less-forgetful learning that regularizes the final hidden activations . Rosenfeld & Tsotsos ( 2017 ) proposed controller modules that optimize loss on the new task with representations learned from previous tasks . They found that they could achieve satisfactory performance while only requiring about 22 % of parameters of a fine-tuning method . Jin & Sendhoff ( 2006 ) modeled the catastrophic forgetting problem as a multi-objective learning problem and proposed a multi-objective pseudorehearsal framework to interleave base patterns with new patterns during optimization . Nguyen et al . ( 2017 ) proposed the variational continual learning by combining online variational inference ( VI ) and Monte Carlo VI for neural networks . Seff et al . ( 2017 ) proposed to solve continual generative modeling by combining the ideas of GANs ( Goodfellow , 2016 ) and EWC ( Kirkpatrick et al. , 2017 ) . Apart from regularization based approaches mentioned above ( e.g. , LwF ( Li & Hoiem , 2016 ) , EWC ( Kirkpatrick et al. , 2017 ) and their variants ) , dual-memory based learning systems have also been proposed . They are inspired by the complementary learning systems ( CLS ) theory ( McClelland et al. , 1995 ; Kumaran et al. , 2016 ) in which memory consolidation and retrieval are related to the interplay of the mammalian hippocampus ( short-term memory ) and neocortex ( long-term memory ) . Gepperth & Karaoguz ( 2016 ) proposed to use a modified self-organizing map ( SOM ) as the long-term memory . To complement it , a short-term memory ( STM ) is added to store novel examples . During the sleep phase , the whole content of STM is replayed to the system . This process is known as intrinsic replay or pseudo-rehearsal ( Robins , 1995 ) . It trains all the nodes in the network with new data ( e.g. , from STM ) and the replayed examples/samples from previously seen classes or distributions on which the network has been trained . The replayed samples prevents the network from forgetting . A related work is that in Wu et al . ( 2018 ) . Kemker & Kanan ( 2018 ) proposed a similar dual-memory system called FearNet . It uses a hippocampal network for short-term memory , a medial prefrontal cortex network for long-term memory , and a third neural network to determine which memory to use for prediction . More recently , Hu et al . ( 2019 ) proposed a technique that learns to generate a part of the parameters while sharing the other part . Forgetting is dealt with by generating some previous parameters and regularization . Zeng et al . ( 2019 ) proposed a method that tries to learn the new task by revising the weights in the directions that are orthogonal to the data of the previous tasks so that the new task will not interfere with the old tasks . The work in Xu & Zhu ( 2018 ) searches for the best neural architecture for each coming task via reinforcement learning . Ritter et al . ( 2018 ) proposed an online Laplace approximation method . Dhar et al . ( 2019 ) proposed a method that combines three loss functions to encourage the model resulted from the new task ( class ) to be similar to the previous model . Some other related works include Learn++ ( Polikar et al. , 2001 ) , Pathnet ( Fernando et al. , 2017 ) , Memory Aware Synapses ( Aljundi et al. , 2017 ) , One Big Net for Everything ( Schmidhuber , 2018 ) , Phantom Sampling ( Venkatesan et al. , 2017 ) , Active Long Term Memory Networks ( Furlanello et al. , 2016 ) , Conceptor-Aided Backprop ( He & Jaeger , 2018 ) , Gating Networks ( Masse et al. , 2018 ; Serrà et al. , 2018 ) , PackNet ( Mallya & Lazebnik , 2017 ) , Diffusion-based Neuromodulation ( Velez & Clune , 2017 ) , Incremental Moment Matching ( Lee et al. , 2017b ) , Dynamically Expandable Networks ( Lee et al. , 2017a ; Yoon et al. , 2018 ; Li et al. , 2019 ) , Incremental Regularized Least Squares ( Camoriano et al. , 2017 ) , and Dual-Memory Recurrent Self-Organization ( Parisi et al. , 2018b ) . An excellent survey of continual learning can be found in ( Parisi et al. , 2018a ) . Most continual learning works deal with CCL ( class continual learning ) where each task consists of one or more classes that are incrementally learned . This paper focuses on task continual learning ( TCL ) , where each task is a separate classification problem . GEM ( Gradient Episodic Memory ) ( Lopez-Paz & Ranzato , 2017 ) is such a method , which takes task id in addition to the training data of the specific task as input . GEM deals with forgetting when learning a new task by jointly training the current task using its training data and also a sampled data from the previous tasks to ensure the past knowledge is maintained . It also has a cross-validation step to tune the network using the first few tasks , which most other continual learning methods do not have . A-GEM ( Chaudhry et al. , 2019 ) significantly improved GEM ’ s efficiency while achieving comparable or better results . Another recent work in this direction is ’ learn to grow ’ ( Li et al. , 2019 ) , which grows the network with each new task while encouraging sharing of network parameters using a new penalized loss function .

Summary: The method proposes a method for continual learning. The method is an extension of recent work, called orthogonal weights modification (OWM) [Zheng,2019]. This method aims to find gradient updates which are perpendicular to the input vectors of previous tasks (resulting in less forgetting). However, the authors argue, that the learning of new tasks is happening in the solution space of the previous tasks, which might severely limit the ability to adapt to new tasks. The authors propose a ‘principal component’-based solution to this problem. The method is considering the ‘task continual learning’ scenario (also known as task-aware) which means that the task label is given at inference time.