Graph-Based Continual Learning

1 INTRODUCTION . Recent breakthroughs of deep neural networks often hinge on the ability to repeatedly iterate over stationary batches of training data . When exposed to incrementally available data from non-stationary distributions , such networks often fail to learn new information without forgetting much of its previously acquired knowledge , a phenomenon often known as catastrophic forgetting ( Ratcliff , 1990 ; McCloskey & Cohen , 1989 ; French , 1999 ) . Despite significant advances , the limitation has remained a long-standing challenge for computational systems that aim to continually learn from dynamic data distributions ( Parisi et al. , 2019 ) . Among various proposed solutions , rehearsal approaches that store samples from previous tasks in an episodic memory and regularly replay them are one of the earliest and most successful strategies against catastrophic forgetting ( Lin , 1992 ; Rolnick et al. , 2019 ) . An episodic memory is typically implemented as an array of independent slots ; each slot holds one example coupled with its label . During training , these samples are interleaved with those from the new task , allowing for simultaneous multi-task learning as if the resulting data were independently and identically distributed . While such approaches are effective in simple settings , they require sizable memory and are often impaired by memory constraints , performing rather poorly on complex datasets . A possible explanation is that slot-based memories fail to utilize relational structure between samples ; semantically similar items are treated independently both during training and at test time . In marked contrast , relational memory is a prominent feature of biological systems that has been strongly linked to successful memory retrieval and generalization ( Prince et al. , 2005 ) . Humans , for example , encode event features into cortical representations and bind them together in the medial temporal lobe , resulting in a durable , yet flexible form of memory ( Shimamura , 2011 ) . In this paper , we introduce a novel Graph-based Continual Learning model ( GCL ) that resembles some characteristics of relational memory . More specifically , we explicitly model pairwise similarities between samples , including both those in the episodic memory and those found in the current task . These similarities allow for representation transfer between samples and provide a resilient mean to guard against catastrophic forgetting . Our contributions are twofold : ( 1 ) We propose the use of random graphs to represent relational structures between samples . While similar notions of dependencies have been proposed in the literature ( Louizos et al. , 2019 ; Yao et al. , 2020 ) , the application of random graphs in task-free continual learning is novel , at least to the best of our knowledge . x ( 2 ) We introduce a new regularization objective that leverages such random graphs to alleviate catastrophic forgetting . In contrast to previous work ( Rebuffi et al. , 2017 ; Li & Hoiem , 2017 ) based on knowledge distillation ( Hinton et al. , 2015 ) , the objective penalizes the model for forgetting learned edges between samples rather than their output predictions . Our approach performs competitively on four commonly used datasets , improving accuracy by up to 19.7 % and reducing forgetting by almost 37 % in the best case when bench-marked against competitive baselines in task-free continual learning . 2 PROBLEM FORMULATION . In this work , we follow the learning protocol for image classification from Lopez-Paz & Ranzato ( 2017 ) . More specifically , we consider a training set D = { D1 , · · · , DT } consisting of T tasks where the dataset for the t-th task Dt = { ( xti , yti ) } nt i=1 contains nt input-target pairs ( x t i , y t i ) ∈ X × Y . While the tasks arrive sequentially and exclusively , we assume the input-target pairs ( xti , y t i ) in each task are independent and identically distributed ( i.i.d. ) . The goal is to learn a supervised model fθ : X → Y , parametrized by θ , that outputs a class label y ∈ Y given an unseen image x ∈ X . Following prior work ( Lopez-Paz & Ranzato , 2017 ; Riemer et al. , 2018 ; Chaudhry et al. , 2019 ) , we consider online streams of tasks in which samples from different tasks arrive at different times . As an additional constraint , we insist that the model can only revisit a small amount of data chosen to be stored in a fixed-size episodic memoryM . For clarity , we refer to the data in such an episodic memory as context images and context labels and denote by XC = { xi } i∈C and YC = { yi } i∈C , respectively . These images and labels are to be distinguished from those in the current task , which we refer to as target images and target labels and denote by XT = { xj } j∈T and YT = { yj } j∈T , respectively . While the model is allowed to update the context samples during training , the episodic memory is necessarily frozen at test time . 3 GRAPH-BASED CONTINUAL LEARNING . In this section , we propose a Graph-based Continual Learning ( GCL ) algorithm . While most rehearsal approaches ignore the correlations between images and independently pass them through a network to compute predictions ( Rebuffi et al. , 2017 ; Chaudhry et al. , 2019 ; Aljundi et al. , 2019c ) , we model pairwise similarities between the images with learnable edges in random graphs ( see Figure 1 ) . Intuitively , although it might be easy for the model to forget any particular sample , the multiple connections it forms with similar neighbors are harder to be forgotten altogether . If trained well , the random graphs can therefore equip the model with a plastic and durable means to fight against catastrophic forgetting . Graph Construction . Given a minibatch of target images XT from the current task , our model makes predictions based on the context images XC and context labels YC that span several previously seen tasks , up to and including the current one . In particular , we explicitly build two random graphs of pairwise dependencies : an undirected graph G between the context images XC and a directed , bipartite graph A from the context images XC to the target images XT . Since an undirected graph can be thought of as a directed graph between its vertices and a copy of itself , we treat the context graph G as such and build it analogously to the context-target graph A . Specifically , the high-dimensional context images XC and target images XT are first mapped to the image embeddings UC and UT , respectively , using an image encoder fθ1 : X → Rd1 . Following Louizos et al . ( 2019 ) , we then represent the edges in each graph by independent Bernoulli random variables whose means are specified by a kernel function in the embedding space . More precisely , the distribution of the resulting Erdős-Rényi random graphs ( Erdös & Rényi , 1959 ) can be defined as p ( G |UC ) = ∏ i∈C ∏ k∈C Ber ( Gik |κτ ( ui , uk ) ) , ( 1 ) p ( A |UT , UC ) = ∏ j∈T ∏ k∈C Ber ( Ajk |κτ ( uj , uk ) ) , ( 2 ) for all i , k ∈ C and j ∈ T where κτ : Rd1 × Rd1 → [ 0 , ∞ ) is a kernel function that encodes similarities between image embeddings such as the RBF kernel κτ ( ui , uj ) = exp ( − τ2‖ui − uj‖ 2 2 ) . Here , with a slight abuse of notation , we also use G and A to denote the corresponding adjacency matrices ; Ajk ∈ { 0 , 1 } , for example , represents the presence or absence of a directed edge between the j-th target image and the k-th context image . Predictive Distribution . Given a context graph G and a context-target graph A that encode pairwise similarities to the context images , our next step is to propagate information from the context images XC and context labels YC to make predictions . To that end , we embed XC by another image encoder fθ2 with weights partially tied to the previous one fθ1 , and encode YC by a linear label encoder before concatenating the resulting embeddings into latent representations VC ∈ R|C|×d2 . In combination with the distributions of G and A , we compute context-aware representations for the context images and target images , denoted by { zi } i∈C and { zj } j∈T , respectively : p ( zi |UC , VC ) = ∫ G I { G̃iVC } ( zi ) dP ( G |UC ) ( 3 ) p ( zj |UT , UC , VC ) = ∫ A I { ÃjVC } ( zj ) dP ( A |UT , UC ) . ( 4 ) where G̃i and Ãj indicate the i-th and j-th row of G and A , each normalized to sum to 1 , and IS ( · ) denotes the indicator function on a set S . Intuitively , the representations VC are linearly weighted by each graph sample , and the normalization step ensures proper scaling in case the numbers of edges formed with the context images vary . Once we summarize each image by the context samples , a final network fθ3 : Rd2 → Y takes as input the context-aware representations and produces predictive distributions : p ( yi |XC ) = ∫ zi p ( yi | fθ3 ( zi ) ) dP ( zi |UC , VC ) , ( 5 ) p ( yj |xj , XC ) = ∫ zj p ( yj | fθ3 ( zj ) ) dP ( zj |UT , UC , VC ) . ( 6 ) Since the numbers of random binary graphs G and A are exponential , we approximate the integrals in ( 1 ) - ( 6 ) by Monte Carlo samples . More specifically , we use one sample of G and A during training and 30 samples of A during testing . Also , these graph samples are inherently non-differentiable , so we use the Gumbel-Softmax relaxations of the Bernoulli random variables during training ( Maddison et al. , 2016 ; Jang et al. , 2016 ) . The degree of approximation is controlled by temperature hyperparameters , which exert significant influence over the density of the graph samples . We find that a small temperature for G and a larger temperature for A work well . There are several reasons for making the graphs G and A random . First , the stochasticity induced by the Bernoulli random variables allows us to output multiple predictions and average these predictions , and such ensemble techniques have been quite successful in continual learning settings ( Coop et al. , 2013 ; Fernando et al. , 2017 ) . Perhaps more importantly , we find that the deterministic version with the Bernoulli random variables replaced by their parameters results in very sparse graphs where samples from the same classes are often deemed dissimilar . In a similar fashion to dropout ( Srivastava et al. , 2014 ) , the random edges encourage the model to be less reliant on a few particular edges and therefore promote knowledge transfer between samples . By a similar reasoning , we remove self-edges in the context graph and also observe more connections between samples . Graph Regularization . As training switches to new tasks , the distributional shifts to the target images necessarily result in changes to both the context graph G and the context-target graph A . In addition , the context images are regularly updated to be representative of the data distribution up to that point , so any well-learned connections between the context images are also susceptible to catastrophic forgetting . As a remedy , we save the parameters of the Bernoulli edges to the episodic memory in conjunction with the context images and context labels , and introduce a regularization term that discourages the model from forgetting previously learned edges : L ( b ) G ( θ1 ) , 1 |I ( b ) | ` ( p ( G ( b−1 ) I ( b ) ) , p ( G ( b ) I ( b ) ) ) . ( 7 ) Here , ` ( · , · ) denotes the cross-entropy between two probability distributions , I ( b ) the index set of edges to be regularized in the bth minibatch , and G ( b−1 ) the adjacency matrix learned from the beginning up to the previous minibatch . The selection strategies I ( b ) are discussed in the next subsection . Besides the regularization term , our training objective includes two other cross-entropy losses , one for the context images and another for the target images : L ( θ1 , θ2 , θ3 ) = λC |C| ∑ i∈C ` ( yi , ŷ ( s ) i ) + λT |T | ∑ j∈T ` ( yj , ŷ ( s ) j ) + λGL ( b ) G ( θ1 ) , ( 8 ) where ŷ ( s ) i = fθ3 ( z ( s ) i ) , ŷ ( s ) j = fθ3 ( z ( s ) j ) and z ( s ) i ∼ p ( zi |UC , VC ) , z ( s ) j ∼ p ( zj |UT , UC , VC ) are context-aware samples from Equations 3 and 4 , and λC , λT , λG are hyperparameters . While the graph regularization term appears similar to knowledge distillation ( Hinton et al. , 2015 ) , we emphasize that the former aims to preserve the covariance structures between the outputs of the image encoder fθ1 rather than the outputs themselves . We believe that in light of new data , the image encoder should be able to update its potentially superficial representations of previously seen samples as long as it keeps the correlations between them unchanged . Indeed , some of the early regularization approaches based on knowledge distillation ( Li & Hoiem , 2017 ; Rebuffi et al. , 2017 ) are sometimes too restrictive and reportedly underperform in certain scenarios ( Kemker & Kanan , 2017 ) . Task-Free Knowledge Consolidation . When task identities are not available , we use reservoir sampling ( Vitter , 1985 ) to update the context images and context labels as in Riemer et al . ( 2018 ) . The sampling strategy takes as input a stream of data and randomly replaces a context sample in the episodic memory with a target sample , with probability proportional to the number of samples observed so far . Despite its simplicity , reservoir sampling has been shown to yield strong performance in recent work ( Chaudhry et al. , 2019 ; Riemer et al. , 2018 ; Rolnick et al. , 2019 ) . While most prior work uses task boundaries to perform knowledge consolidation at the end of each task ( Kirkpatrick et al. , 2017 ; Rebuffi et al. , 2017 ) , we update the context graph in memory after every minibatch of training data . In addition , such updates are performed at the sample level to maximize flexibility ; we keep track of the cross entropy loss on each context sample and only update its edges in the graph when the model reaches a new low ( denoted by I ( b ) previously ) . Intuitively , the loss measures how well the model has learned the context image through the connections it forms with others , so meaningful relations are most likely obtained at the bottom of the loss surface . Though samples from the same task often provide more support for each other , the task-agnostic mechanism for updating the context graph also allows for knowledge transfer across tasks when necessary . Memory and Time Complexity . The inclusion of pairwise similarities and graph regularization result in a time and memory complexity ofO ( |M|2+ |M|N ) andO ( |M|2 ) , respectively , where |M| denotes the size of the episodic memory and N the batch size for target images . The quadratic costs in |M| , however , are not concerning in practice , as we deliberately use a small , fixed-size episodic memory . The cost of storing G is often dwarfed by the memory required for storing high-dimensional images , as each edge only needs one floating point number ( see Appendix E for more details on memory usage ) .

Summary: This paper presented a memory-based continual learning model where relationships between training samples are represented with a random graph that is defined from the non-linear embedding of the input data. Catastrophic forgetting between tasks is partially (1) alleviated with a graph regularization that penalizes changes of random graph statistics, and (1) memory replay and reservoir sampling to update memory. The performance of this model is evaluated against several state-of-the-art models to handle the catastrophic loss.