Overcoming catastrophic forgetting through weight consolidation and long-term memory

1 INTRODUCTION . Continual learning central to designing general A.I . systems that can learn new tasks sequentially without forgetting old tasks . However , current deep learning models based on stochastic gradient descent severely suffer from catastrophic forgetting ( French , 1999 ; McCloskey & Cohen , 1989 ) , in that they often forget all old tasks after training each new one . Inspired by the mammalian neocortex , which relies on processes of task-specific synaptic consolidation to enable continual learning ( Cichon & Gan , 2015 ; Hayashi-Takagi et al. , 2015 ; Yang et al. , 2009 ; 2014 ) , several concepts have been proposed , based on sequential Bayesian learning , which consist of applying a regularization function to a network trained by an old task to learn a new task . Many of these approaches work by finding a local minimum of the loss function for task B around the local region in the parameter space that was optimized for task A , such as learning without forgetting ( Li & Hoiem , 2017 ) , elastic weight consolidation ( EWC ) ( Kirkpatrick et al. , 2017 ) , and incremental moment matching ( Lee et al. , 2017 ) . Consider a model trained to perform task A : for input XtaskA and probability distribution p ( taskA ) , it produces outputs f ( XtaskA , W ∗taskA ) by minimizing loss Lp ( taskA ) ( f ( XtaskA , W ) ) . Training a second task B with input XtaskB and probability distribution p ( taskB ) on the same network involves minimizing loss Lp ( taskB ) ( f ( XtaskB , W ∗taskA ) ) . Previous approaches mentioned above attempt to restrict the parameters for task B to the local region around the optimum of task A , so as to minimally disturb what had been learned for task A . However , this might prevent the neural network from finding other regions in remote areas of the parameter space , which could contain a better local minimum of the loss function for the joint probability distribution of tasks A and B . To find this better local minimum , ( Sprechmann et al. , 2018 ) stores most of the data from old tasks into a working memory and replays it while training a new task . However , the working memory requires potentially large storage for the data from old tasks , and extra training time to replay that data . Here , we introduce task-dependent memory units Mtask and memory weights Wtask to overcome catastrophic forgetting . Consider a two-task scenario , with an output function : H ( X , Mtask , W , Wtask ) = f ( X , W ) + 1taskA ( gtaskA ( MtaskA , WtaskA ) ) + 1taskB ( gtaskB ( MtaskB , WtaskB ) ) ( 1 ) with input X , task-dependent memory units Mtask , task-independent network weights W , taskdependent memory weights Wtask , indicator function 1taskA ( ) , indicator function 1taskB ( ) , taskdependent function gtaskA ( ) and gtaskB ( ) and task-independent function f ( ) . We update W in the EWC direction , update memory units Mtask in an adversarial direction ( further explained below ) , and update Wtask in the gradient direction . Without any replaying of previous data , we can achieve a high accuracy on new tasks while minimally decreasing accuracy on old tasks . Our approach is not constrained to the local region around the parameters ( W ∗ ) that are optimal for the old tasks . Instead , we create new parameter spaces ( W ∗ , Mtask , Wtask ) that are good for both old and new tasks . In the experimental results section , we show , using disjoint MNIST , disjoint EMNIST , and disjoint CIFAR10 , that our Adversarial Memory Net ( AMN ) can be trained sequentially on multiple tasks while minimally decreasing accuracy on old tasks . Beyond these results which use a 5-layer fully connected network , we show that our approach can also apply to CNNs , although more research is necessary to enable EWC constraints to apply to convolutional and pool layers . We discuss complexity analysis , mathematical implications , and EWC in the discussion section . 2 BACKGROUND AND RELATED WORK . Memory formation and retrieval in Hippocampus ( HPC ) : In the human brain , the hippocampus ( Bakker et al. , 2008 ) encodes detailed information in Cornu Ammonis 3 ( CA3 ) , which does pattern separation and transforms this information into abstract high-level information , then relayed to Cornu Ammonis 1 ( CA1 ) , which does pattern completion . During weight consolidation ( Lesburguères et al. , 2011 ; Squire & Alvarez , 1995 ; Frankland & Bontempi , 2005 ) , the HPC fuses different features from different tasks into a coherent memory trace . Over days to weeks , as memories mature , HPC progressively stores permanent abstract high-level long-term memories to remote memory storage ( neocortical areas ) . HPC can maintain and mediate their retrieval independently when the specific memory is in need . Adversarial examples , directions , subspaces , & programs , and long term memory : Artificial neural networks are vulnerable to adversarial examples ( Szegedy et al. , 2013 ) . By adding a carefully computed “ noise ” to an input picture , without changing the neural network , one can force the network into misclassification . The noise is usually computed by backpropagating the gradient in a so-called “ adversarial direction ” ( Tramèr et al. , 2017 ) . Going to an adversarial direction , such as by using the fast gradient sign method ( FGSD ) ( Goodfellow et al . ) , can help us generate adversarial examples that span a continuous subspace of large dimensionality ( adversarial subspace ) . Because of “ excessive linearity ” in many neural networks ( Tramèr et al. , 2017 ; Goodfellow ) , due to features including Rectified linear units and Maxout , the adversarial subspace often takes a large portion of the total input space . Once an adversarial input lies in the adversarial subspace , nearby inputs also tend to lie in it . Attack and defense researchers usually view adversarial examples as a curse of neural networks , but we view it as a gift to solve catastrophic forgetting . Input points that lie inside the adversarial subspace of each class lead the neural network to misclassify input images into that class . By extension , inputs which lie in the intersection of several adversarial subspaces that each belong to a different class may be misclassified into any of the underlying classes . Instead of adding input `` noise '' that lies inside the adversarial subspace calculated from other classes to force the network into misclassification , we can add `` noise '' calculated by the input ’ s own correct class to assist correct classification . Our approach is to compute , in a task-dependent manner , this helpful `` noise '' , and to store it into long-term memory units which are activated by different tasks . Thus , we propose that the intersection of the adversarial subspaces of all known classes is a representation of long term memory in neural network , although , we do not know how our brain represents permanent long-term memories . In essence , by storing task-dependent data into memory units , we add new dimensions to the original parameter space and create new spaces that are good for both old and new tasks . Indeed , after we submit our paper to ArXiv for one month , a nice followup work ( Elsayed et al. , 2018 ) shows how a carefully computed adversarial program embedded in the input space can repurpose machine models to perform a new task without changing the parameters . This paper confirms that the adversarial program ( carefully computed noise ) , formed by finding the intersection of adversarial subspaces in our paper , is a good way to represent abstract information . 3 METHODS . In biological brains , during consolidation , abstract high-level memory is believed to be transferred from HPC to remote memory storage , such as the olfactory cortex ( Lesburguères et al. , 2011 ) . Inspired by this idea , to achieve sequential learning with minimal interference , we add memory units into our network . We propose that one can designate and switch from task to task using dedicated task-dependent memory units . The intersection of the adversarial subspaces which represent the long-term memory in the network is stored in the memory units . The memory units , that store these long-term memory independently for each task , add additional dimension ( Mtask , Wtask ) to our old parameter spaces ( W ) and form new parameter spaces ( W , Mtask , Wtask ) that are good for both old and new tasks ( Fig . 1 ) . 3.1 EMBEDING ADVERSARIAL SUBSPACES INTO PARAMETER SPACE ( Mtask , Wtask , W ) We want to embed the adversarial subspaces that have been found in the input space into our neural network , so that the parameters in our neural network behave like input images and span adversarial subspaces in the parameter space . Thus , we add task-dependent memory units ( float tensors Mtask , Fig . 1 ) and memory weights ( Wtask , Fig . 1 ) in each layer of the neural network . We update memory units in the adversarial direction . Each memory unit is a float tensor that can hold a float value . Memory units play a role similar to adversarial input images , capture the essence of each class , and span an intersection of adversarial subspaces in network parameter space . 3.2 ADVERSARIAL MEMORY NETWORK ( AMN ) . Task-dependent memory units ( Mtask ∈ RM×H , M is the number of memory units , H is the hidden size of a memory unit ) store the intersection of adversarial subspaces in it which plays a role of long-term memory in our network , leading to the adversarial memory network structure ( Fig . 1 ) . During training and testing , the output function for 2 tasks is shown in Eqn . 1 . Only memory units corresponding to the current task ( yellow for task A or red for task B , chosen by indicator function in Eqn.1 ) and normal neurons ( black ) will be activated . The weights for memory units are called memory weights ( Wtask ) and the weights for normal neurons are called normal weights ( W ) . Forward rules for each layer : 1 . Deactivate the memory units not for the current task . 2 . Output = Wtask ∗Mtask +W ∗Xnormal_neurons + bias . What abstract long-term memory should we write into the memory units ? The intersection of adversarial subspaces . How can we formulate it ? We update memory units with gradients following an adversarial direction which can help us find the intersection of the adversarial subspaces in the parameter space . Thus , we have a joint abstract long-term memory of different classes available for each task in our neural network . In this work , we use the Fast Gradient Sign Method ( FGSD ) sign ( ∇ML ( M , ytarget ) ) , where M are memory units , ytarget is current input ’ s true classes , as the adversarial direction to replace the gradient direction and use it to update memory units . Backward rules for each layer : OGradients = { sign ( OGradients ) if O is memory unit OGradients else We use Softmax or Sigmoid cross entropy loss ( Lp ( taskB ) ( H ( X , Mtask , W , Wtask ) ) the first part in Eqn.2 ) to calculate the gradients for task-dependent memory units ( Mtask ) and memory weights ( Wtask ) . To keep the normal weight close to the learned parameters of old tasks , we use total loss ( Eqn . 2 ) with EWC constrains to calculate the gradients for normal weights ( W ) . Consider a 2-task scenario , the loss function when we train task B is , where F is the fisher information matrix , λ sets how important the old task is compared to the new one : Total Loss = Lp ( taskB ) ( H ( X , Mtask , W , Wtask ) ) + ∑ i λFi ( Wi −W ∗taskA , i ) 2 ( 2 )

Summary: This paper proposes a novel continual learning method that stores intersection of adversarial subspaces into long-term memory units for each task, which is used used to characterize the given task at future tasks. This adversarial memory network requires supposedly less number of parameters for each task to store, compared to methods that stores explicit examples. The authors validated the proposed model on three datasets for continual learning, on which it obtains good performance when networks trained with plain gradient descent and elastic weight consolidation suffers from catastrophic forgetting.