Prototype Recalls for Continual Learning

1 INTRODUCTION . Continual learning , also known as lifelong learning , is the crucial ability for humans to continually acquire and transfer new knowledge across their lifespans while retaining previously learnt experiences Hassabis et al . ( 2017 ) . This ability is also critical for artificial intelligence ( AI ) systems to interact with the real world and process continuous streams of information Thrun & Mitchell ( 1995 ) . However , the continual acquisition of incrementally available data from non-stationary data distributions generally leads to catastrophic forgetting in the system McCloskey & Cohen ( 1989 ) ; Ratcliff ( 1990 ) ; French ( 1999 ) . Continual learning remains a long-standing challenge for deep neural network models since these models typically learn representations from stationary batches of training data and tend to fail to retain good performances in previous tasks when data become incrementally available over tasks Kemker et al . ( 2018 ) ; Maltoni & Lomonaco ( 2019 ) . Numerous methods for alleviating catastrophic forgetting have been currently proposed . The most pragmatical way is to jointly train deep neural network models on both old and new tasks , which however demands a large amount of resources to store previous training data and hinders the learning of novel data in real time . Another option is to complement the training data for each new task with “ pseudo-data ” of the previous tasks Shin et al . ( 2017 ) ; Robins ( 1995 ) . Besides the main model for task performance , a separate generative model is trained to generate fake historical data used for pseudo-rehearsal . Deep Generative Replay ( DGR ) Shin et al . ( 2017 ) replaces the storage of the previous training data with a Generative Adversarial Network to synthesize training data on all previously learnt tasks . These generative approaches have succeeded over very simple and artificial inputs but they can not tackle more complicated inputs Atkinson et al . ( 2018 ) . Moreover , to synthesize the historical data reasonably well , the size of the generative model is usually huge that costs much memory Wen et al . ( 2018 ) . An alternative method is to store the weights of the model trained on previous tasks , and impose constraints of weight updates on new tasks He & Jaeger ( 2018 ) ; Kirkpatrick et al . ( 2017 ) ; Zenke et al . ( 2017 ) ; Lee et al . ( 2017 ) ; Lopez-Paz et al . ( 2017 ) . For example , Elastic Weight Consolidation ( EWC ) Kirkpatrick et al . ( 2017 ) and Learning Without Forgetting ( LwF ) Li & Hoiem ( 2018 ) store all the model parameters on previously learnt tasks , estimate their importance on previous tasks and penalize future changes to the weights on new tasks . However , selecting the “ important ” parameters for previous tasks complicates the implementation by exhaustive hyper-parameter tuning . In addition , state-of-the-art neural network models often involve millions of parameters and storing all network parameters from previous tasks does not necessarily reduce the memory cost Wen et al . ( 2018 ) . In contrast with these methods , storing a small subset of examples from previous tasks and replaying the “ exact subset ” substantially boost performance Kemker & Kanan ( 2017 ) ; Rebuffi et al . ( 2017 ) ; Nguyen et al . ( 2017 ) . To achieve the desired network behavior on previous tasks , incremental Classifier and Representation Learner ( iCARL ) Rebuffi et al . ( 2017 ) and Few-shot Self-Reminder ( FSR ) Wen et al . ( 2018 ) follow the idea of logit matching or knowledge distillation in model compression Ba & Caruana ( 2014 ) ; Bucilua et al . ( 2006 ) ; Hinton et al . ( 2015 ) . However , such approaches ignore the topological relations among clusters in the embedding space and rely too much on a small amount of individual data , which may result in overfitting as shown in our experiments ( Section 4.2 ) . In contrast with them , without tampering the performance in memory retention , our method learns embedding functions and compares the feature similarities represented by class prototypes in the embedding space which improves generalization , especially in the few-shot settings , as also been verified in works Hoffer & Ailon ( 2015 ) ; Snell et al . ( 2017 ) . In this paper , we propose the method , Prototype Recalls , for continual learning in classification tasks . Similar as Snell et al . ( 2017 ) , we use a neural network to learn class-representative prototypes in an embedding space and classify embedded test data by finding their nearest class prototype . To tackle the problem of catastrophic forgetting , we impose additional constraints on the network by classifying the embedded test data based on prototypes from previous tasks , which promotes the preservation of initial embedding function . For example ( Figure 1 ) , in the first task ( Subfigure 1a ) , the network learns color prototypes to classify blue and yellow circles and in the second task ( Subfigure 1b ) , the network learns shape prototypes to classify green circles and triangles . With catastrophically forgetting color features , the network extracts circle features on the first task and fails to classify blue and yellow circles . To alleviate catastrophic forgetting , our method replays the embeded previous samples ( blue and yellow circles ) and match them with previous color prototypes ( blue and yellow ) which reminds the network of extracting both color and shape features in both classification tasks . We evaluate our method under two typical experimental protocols , incremental domain and incremental class , for continual learning across three benchmark datasets , MNIST Deng ( 2012 ) , CIFAR10 Krizhevsky & Hinton ( 2009 ) and miniImageNet Deng et al . ( 2009 ) . Compared with the state-of-the-arts , our method significantly boosts the performance of continual learning in terms of memory retention capability while being able to adapt to new tasks . Unlike parameter regularization methods or iCARL or FSR , our approach further reduces the memory storage by replacing logits of each data or network parameters with one prototype of each class in the episodic memory . Moreover , in contrast to these methods where the last layer in traditional classification networks often structurally depends on the number of classes , our method leverages on metric learning , maintains the same network architecture and does not require adding new neurons or layers for new object classes . Additionally , without sacrificing classification accuracy on initial tasks , our method can generalize to learn new concepts given a few training examples in new tasks due to the advantage of metric learning , commonly used in few-shot settings Snell et al . ( 2017 ) ; Hoffer & Ailon ( 2015 ) . 2 PROPOSED METHOD . We propose the method , Prototype Recalls , for continual learning . For a sequence of datasets D1 , D2 , ... , Dt , ... , given Dt in any task t where t ∈ { 1 , 2 , ... , T } , the goal for the model fT is to retain the good classification performance on all T datasets after being sequentially trained over T tasks . The value of T is not pre-determined . The model fT with learnable parameters φ is only allowed to carry over a limited amount of information from the previous T − 1 tasks . This constraint eliminates the naive solution of combining all previous datasets to form one big training set for fine-tuning the model fT at task T . Each dataset Dt consists of Nt labeled examples Dt = { Xt , Yt } = { ( x1t , y1t ) , ... , ( xNt , yNt ) } where each xit ∈ RD is the D-dimensional feature vector of an example and yit ∈ { 1 , ... , Kt } is the corresponding class label . Skt denotes the set of examples labeled with class kt . At task T , if we simply train a model by only minimizing the classification lossLclassi , DT on dataset DT , the model will forget how to perform classification on previous datasets Dt , t < T which is described as catastrophic forgetting problem McCloskey & Cohen ( 1989 ) ; Ratcliff ( 1990 ) ; French ( 1999 ) . Here we show how the model trained in our method retains the good performance on all previous tasks while adaptively learning new tasks . The loss for all the previous datasets is denoted by LT ( f ) = ∑T t=1 EDt [ L ( f ( Xt ) , Yt ) ] . Our objective is to learn fT defined as follows : fT = argmin f LT ( f ) = argmin f { T−1∑ t=1 Lclassi , Dt ( ft ) + Lclassi , DT ( f ) + T−1∑ t=1 δDt ( f , ft ) } ( 1 ) where Lclassi , DT ( f ) defines the classification loss of f on dataset DT and δDt ( f , ft ) measures the differences in the network behaviors in the embedding space learnt by f and ft onDt , as introduced later in Equ 7 . Given f1 , ... , fT−1 that are learnt from the previous tasks , at task T , learning fT requires minimizing both terms Lclassi , DT ( f ) and ∑T−1 t=1 δDt ( f , ft ) . In the subsections below and Figure 1 , we describe how to optimize these two terms . 2.1 CLASSIFICATION . To perform classification on dataset Dt , our method learns an embedding space in which points cluster around a single prototype representation for each class and classification is performed by finding the nearest class prototype Snell et al . ( 2017 ) ( Figure 1a ) . Compared to traditional classification networks with a specific classification layer attached in the end , such as iCARL and FSR , our method keeps the network architecture unchanged while finding the nearest neighbour in the embedding space , which would lead to more efficient memory usage . For example , in one of the continual learning protocols Snell et al . ( 2017 ) where the models are asked to classify incremental classes ( also see Section 3.1 ) , traditional classification networks have to expand their architectures by accommodating more output units in the last classification layer based on the number of incremental classes and consequently , additional network parameters have to be added into the memory . Without loss of generality , here we show how our method performs classification on DT . First , the model learns an embedding function f : RD → RM and computes an M -dimensional prototype ckT ∈ RM which is the mean of the embeddings from examples SkT : ckT = 1 |SkT | ∑ ( xiT , yiT ) ∈SkT f ( xiT ) . ( 2 ) The pairwise distance of one embedding and one prototype within the same class should be smaller than the intra-class ones . Our method introduces a distance function d : RM × RM → [ 0 , ∞ ) . For each example xT , it estimates a distance distribution based on a softmax over distances to the prototypes of KT classes in the embedding space : pφ ( yT = kT |xT ) = exp ( −d ( f ( xT ) , ckT ) ) ∑KT k′ exp ( −d ( f ( xT ) , ck′T ) ) . ( 3 ) The objective function Lclassi , DT ( f ) is to minimize the negative log-probability − log pφ ( yT = kT |xT ) of the ground truth class label kT via Stochastic Gradient Descent Bottou ( 2010 ) : Lclassi , DT ( f ) = − log pφ ( yT = kT |xT ) ( 4 ) In practice , when NT is large , computing ckT is costly and memory inefficient during training . Thus , at each training iteration , we randomly sample two complement subsets from SkT over all KT classes : one for computing prototypes and the other for estimating distance distribution . Our primary choice of the distance function d ( · ) is squared Euclidean distance which has been verified to be effective in Snell et al . ( 2017 ) . In addition , we include temperature hyperparameter τ in d ( · ) as introduced in network distillation literature Hinton et al . ( 2015 ) and set its value empirically based on the validation sets . A higher value for τ produces a softer probability distribution over classes .

Summary: The proposed method addresses continual learning, by learning a mapping from the input space to an embedding space, and employing a loss that encourages clustering the embeddings by class (and task?) around some centroids called prototypes. Catastrophic forgetting is mitigated by adding a penalty term that is proportional to the distance of the embeddings under the current network of some samples from the past tasks, and the centroids previously associated to each of them.