Graph-Based Continual Learning

1 INTRODUCTION . Recent breakthroughs of deep neural networks often hinge on the ability to repeatedly iterate over stationary batches of training data . When exposed to incrementally available data from non-stationary distributions , such networks often fail to learn new information without forgetting much of its previously acquired knowledge , a phenomenon often known as catastrophic forgetting ( Ratcliff , 1990 ; McCloskey & Cohen , 1989 ; French , 1999 ) . Despite significant advances , the limitation has remained a long-standing challenge for computational systems that aim to continually learn from dynamic data distributions ( Parisi et al. , 2019 ) . Among various proposed solutions , rehearsal approaches that store samples from previous tasks in an episodic memory and regularly replay them are one of the earliest and most successful strategies against catastrophic forgetting ( Lin , 1992 ; Rolnick et al. , 2019 ) . An episodic memory is typically implemented as an array of independent slots ; each slot holds one example coupled with its label . During training , these samples are interleaved with those from the new task , allowing for simultaneous multi-task learning as if the resulting data were independently and identically distributed . While such approaches are effective in simple settings , they require sizable memory and are often impaired by memory constraints , performing rather poorly on complex datasets . A possible explanation is that slot-based memories fail to utilize relational structure between samples ; semantically similar items are treated independently both during training and at test time . In marked contrast , relational memory is a prominent feature of biological systems that has been strongly linked to successful memory retrieval and generalization ( Prince et al. , 2005 ) . Humans , for example , encode event features into cortical representations and bind them together in the medial temporal lobe , resulting in a durable , yet flexible form of memory ( Shimamura , 2011 ) . In this paper , we introduce a novel Graph-based Continual Learning model ( GCL ) that resembles some characteristics of relational memory . More specifically , we explicitly model pairwise similarities between samples , including both those in the episodic memory and those found in the current task . These similarities allow for representation transfer between samples and provide a resilient mean to guard against catastrophic forgetting . Our contributions are twofold : ( 1 ) We propose the use of random graphs to represent relational structures between samples . While similar notions of dependencies have been proposed in the literature ( Louizos et al. , 2019 ; Yao et al. , 2020 ) , the application of random graphs in task-free continual learning is novel , at least to the best of our knowledge . x ( 2 ) We introduce a new regularization objective that leverages such random graphs to alleviate catastrophic forgetting . In contrast to previous work ( Rebuffi et al. , 2017 ; Li & Hoiem , 2017 ) based on knowledge distillation ( Hinton et al. , 2015 ) , the objective penalizes the model for forgetting learned edges between samples rather than their output predictions . Our approach performs competitively on four commonly used datasets , improving accuracy by up to 19.7 % and reducing forgetting by almost 37 % in the best case when bench-marked against competitive baselines in task-free continual learning . 2 PROBLEM FORMULATION . In this work , we follow the learning protocol for image classification from Lopez-Paz & Ranzato ( 2017 ) . More specifically , we consider a training set D = { D1 , · · · , DT } consisting of T tasks where the dataset for the t-th task Dt = { ( xti , yti ) } nt i=1 contains nt input-target pairs ( x t i , y t i ) ∈ X × Y . While the tasks arrive sequentially and exclusively , we assume the input-target pairs ( xti , y t i ) in each task are independent and identically distributed ( i.i.d. ) . The goal is to learn a supervised model fθ : X → Y , parametrized by θ , that outputs a class label y ∈ Y given an unseen image x ∈ X . Following prior work ( Lopez-Paz & Ranzato , 2017 ; Riemer et al. , 2018 ; Chaudhry et al. , 2019 ) , we consider online streams of tasks in which samples from different tasks arrive at different times . As an additional constraint , we insist that the model can only revisit a small amount of data chosen to be stored in a fixed-size episodic memoryM . For clarity , we refer to the data in such an episodic memory as context images and context labels and denote by XC = { xi } i∈C and YC = { yi } i∈C , respectively . These images and labels are to be distinguished from those in the current task , which we refer to as target images and target labels and denote by XT = { xj } j∈T and YT = { yj } j∈T , respectively . While the model is allowed to update the context samples during training , the episodic memory is necessarily frozen at test time . 3 GRAPH-BASED CONTINUAL LEARNING . In this section , we propose a Graph-based Continual Learning ( GCL ) algorithm . While most rehearsal approaches ignore the correlations between images and independently pass them through a network to compute predictions ( Rebuffi et al. , 2017 ; Chaudhry et al. , 2019 ; Aljundi et al. , 2019c ) , we model pairwise similarities between the images with learnable edges in random graphs ( see Figure 1 ) . Intuitively , although it might be easy for the model to forget any particular sample , the multiple connections it forms with similar neighbors are harder to be forgotten altogether . If trained well , the random graphs can therefore equip the model with a plastic and durable means to fight against catastrophic forgetting . Graph Construction . Given a minibatch of target images XT from the current task , our model makes predictions based on the context images XC and context labels YC that span several previously seen tasks , up to and including the current one . In particular , we explicitly build two random graphs of pairwise dependencies : an undirected graph G between the context images XC and a directed , bipartite graph A from the context images XC to the target images XT . Since an undirected graph can be thought of as a directed graph between its vertices and a copy of itself , we treat the context graph G as such and build it analogously to the context-target graph A . Specifically , the high-dimensional context images XC and target images XT are first mapped to the image embeddings UC and UT , respectively , using an image encoder fθ1 : X → Rd1 . Following Louizos et al . ( 2019 ) , we then represent the edges in each graph by independent Bernoulli random variables whose means are specified by a kernel function in the embedding space . More precisely , the distribution of the resulting Erdős-Rényi random graphs ( Erdös & Rényi , 1959 ) can be defined as p ( G |UC ) = ∏ i∈C ∏ k∈C Ber ( Gik |κτ ( ui , uk ) ) , ( 1 ) p ( A |UT , UC ) = ∏ j∈T ∏ k∈C Ber ( Ajk |κτ ( uj , uk ) ) , ( 2 ) for all i , k ∈ C and j ∈ T where κτ : Rd1 × Rd1 → [ 0 , ∞ ) is a kernel function that encodes similarities between image embeddings such as the RBF kernel κτ ( ui , uj ) = exp ( − τ2‖ui − uj‖ 2 2 ) . Here , with a slight abuse of notation , we also use G and A to denote the corresponding adjacency matrices ; Ajk ∈ { 0 , 1 } , for example , represents the presence or absence of a directed edge between the j-th target image and the k-th context image . Predictive Distribution . Given a context graph G and a context-target graph A that encode pairwise similarities to the context images , our next step is to propagate information from the context images XC and context labels YC to make predictions . To that end , we embed XC by another image encoder fθ2 with weights partially tied to the previous one fθ1 , and encode YC by a linear label encoder before concatenating the resulting embeddings into latent representations VC ∈ R|C|×d2 . In combination with the distributions of G and A , we compute context-aware representations for the context images and target images , denoted by { zi } i∈C and { zj } j∈T , respectively : p ( zi |UC , VC ) = ∫ G I { G̃iVC } ( zi ) dP ( G |UC ) ( 3 ) p ( zj |UT , UC , VC ) = ∫ A I { ÃjVC } ( zj ) dP ( A |UT , UC ) . ( 4 ) where G̃i and Ãj indicate the i-th and j-th row of G and A , each normalized to sum to 1 , and IS ( · ) denotes the indicator function on a set S . Intuitively , the representations VC are linearly weighted by each graph sample , and the normalization step ensures proper scaling in case the numbers of edges formed with the context images vary . Once we summarize each image by the context samples , a final network fθ3 : Rd2 → Y takes as input the context-aware representations and produces predictive distributions : p ( yi |XC ) = ∫ zi p ( yi | fθ3 ( zi ) ) dP ( zi |UC , VC ) , ( 5 ) p ( yj |xj , XC ) = ∫ zj p ( yj | fθ3 ( zj ) ) dP ( zj |UT , UC , VC ) . ( 6 ) Since the numbers of random binary graphs G and A are exponential , we approximate the integrals in ( 1 ) - ( 6 ) by Monte Carlo samples . More specifically , we use one sample of G and A during training and 30 samples of A during testing . Also , these graph samples are inherently non-differentiable , so we use the Gumbel-Softmax relaxations of the Bernoulli random variables during training ( Maddison et al. , 2016 ; Jang et al. , 2016 ) . The degree of approximation is controlled by temperature hyperparameters , which exert significant influence over the density of the graph samples . We find that a small temperature for G and a larger temperature for A work well . There are several reasons for making the graphs G and A random . First , the stochasticity induced by the Bernoulli random variables allows us to output multiple predictions and average these predictions , and such ensemble techniques have been quite successful in continual learning settings ( Coop et al. , 2013 ; Fernando et al. , 2017 ) . Perhaps more importantly , we find that the deterministic version with the Bernoulli random variables replaced by their parameters results in very sparse graphs where samples from the same classes are often deemed dissimilar . In a similar fashion to dropout ( Srivastava et al. , 2014 ) , the random edges encourage the model to be less reliant on a few particular edges and therefore promote knowledge transfer between samples . By a similar reasoning , we remove self-edges in the context graph and also observe more connections between samples . Graph Regularization . As training switches to new tasks , the distributional shifts to the target images necessarily result in changes to both the context graph G and the context-target graph A . In addition , the context images are regularly updated to be representative of the data distribution up to that point , so any well-learned connections between the context images are also susceptible to catastrophic forgetting . As a remedy , we save the parameters of the Bernoulli edges to the episodic memory in conjunction with the context images and context labels , and introduce a regularization term that discourages the model from forgetting previously learned edges : L ( b ) G ( θ1 ) , 1 |I ( b ) | ` ( p ( G ( b−1 ) I ( b ) ) , p ( G ( b ) I ( b ) ) ) . ( 7 ) Here , ` ( · , · ) denotes the cross-entropy between two probability distributions , I ( b ) the index set of edges to be regularized in the bth minibatch , and G ( b−1 ) the adjacency matrix learned from the beginning up to the previous minibatch . The selection strategies I ( b ) are discussed in the next subsection . Besides the regularization term , our training objective includes two other cross-entropy losses , one for the context images and another for the target images : L ( θ1 , θ2 , θ3 ) = λC |C| ∑ i∈C ` ( yi , ŷ ( s ) i ) + λT |T | ∑ j∈T ` ( yj , ŷ ( s ) j ) + λGL ( b ) G ( θ1 ) , ( 8 ) where ŷ ( s ) i = fθ3 ( z ( s ) i ) , ŷ ( s ) j = fθ3 ( z ( s ) j ) and z ( s ) i ∼ p ( zi |UC , VC ) , z ( s ) j ∼ p ( zj |UT , UC , VC ) are context-aware samples from Equations 3 and 4 , and λC , λT , λG are hyperparameters . While the graph regularization term appears similar to knowledge distillation ( Hinton et al. , 2015 ) , we emphasize that the former aims to preserve the covariance structures between the outputs of the image encoder fθ1 rather than the outputs themselves . We believe that in light of new data , the image encoder should be able to update its potentially superficial representations of previously seen samples as long as it keeps the correlations between them unchanged . Indeed , some of the early regularization approaches based on knowledge distillation ( Li & Hoiem , 2017 ; Rebuffi et al. , 2017 ) are sometimes too restrictive and reportedly underperform in certain scenarios ( Kemker & Kanan , 2017 ) . Task-Free Knowledge Consolidation . When task identities are not available , we use reservoir sampling ( Vitter , 1985 ) to update the context images and context labels as in Riemer et al . ( 2018 ) . The sampling strategy takes as input a stream of data and randomly replaces a context sample in the episodic memory with a target sample , with probability proportional to the number of samples observed so far . Despite its simplicity , reservoir sampling has been shown to yield strong performance in recent work ( Chaudhry et al. , 2019 ; Riemer et al. , 2018 ; Rolnick et al. , 2019 ) . While most prior work uses task boundaries to perform knowledge consolidation at the end of each task ( Kirkpatrick et al. , 2017 ; Rebuffi et al. , 2017 ) , we update the context graph in memory after every minibatch of training data . In addition , such updates are performed at the sample level to maximize flexibility ; we keep track of the cross entropy loss on each context sample and only update its edges in the graph when the model reaches a new low ( denoted by I ( b ) previously ) . Intuitively , the loss measures how well the model has learned the context image through the connections it forms with others , so meaningful relations are most likely obtained at the bottom of the loss surface . Though samples from the same task often provide more support for each other , the task-agnostic mechanism for updating the context graph also allows for knowledge transfer across tasks when necessary . Memory and Time Complexity . The inclusion of pairwise similarities and graph regularization result in a time and memory complexity ofO ( |M|2+ |M|N ) andO ( |M|2 ) , respectively , where |M| denotes the size of the episodic memory and N the batch size for target images . The quadratic costs in |M| , however , are not concerning in practice , as we deliberately use a small , fixed-size episodic memory . The cost of storing G is often dwarfed by the memory required for storing high-dimensional images , as each edge only needs one floating point number ( see Appendix E for more details on memory usage ) .

Summary: The paper proposes a novel way of using random graphs to improve task-free continual learning method. It builds to random graphs, G and A, based on the similarity of images stored in the memory and those of the current tasks, and utilize the relative information to build representation of the images and predict. The idea is well-formulated, and carried out in a sound way. The graph regularization term resembles the knowledge-distillation, as the authors also mentioned, but it serves different purpose of preserving the covariance structure of the outputs of the image encoders.