Graph-Based Continual Learning
Authors: Binh Tang
David S. Matteson
Venue: ICLR 2021 Spotlight (2021)

Abstract: Despite significant advances, continual learning models still suffer from catastrophic forgetting when exposed to incrementally available data from non-stationary distributions. Rehearsal approaches alleviate the problem by maintaining and replaying a small episodic memory of previous samples, often implemented as an array of independent memory slots. In this work, we propose to augment such an array with a learnable random graph that captures pairwise similarities between its samples, and use it not only to learn new tasks but also to guard against forgetting. Empirical results on several benchmark datasets show that our model consistently outperforms recently proposed baselines for task-free continual learning.

Published as a conference paper at ICLR 2021
GRAPH-BASED CONTINUAL LEARNING
Binh Tang
David S. Matteson
Department of Statistics and Data Science
Department of Statistics and Data Science
Cornell University
Cornell University
Ithaca, NY 14850
Ithaca, NY 14850
bvt5@cornell.ee edu
mattesonnCornell.eddu
ABSTRACT
Despite significant advances, continual learning models still suffer from catas- trophic forgetting when exposed to incrementally available data from non-stationary distributions. Rehearsal approaches alleviate the problem by maintaining and playing small episodic memory of previous samples, often implemented as an array of independent memory slots. In this work, we propose to augment such an array with learnable random graph that captures pairwise similarities between its samples, and use not only to learn new tasks but also to guard against forgetting. Empirical results on several benchmark datasets show that our model consistently outperforms recently proposed baselines for task-free continual learning.
INTRODUCTION
Recent breakthroughs of deep neural networks often hinge the ability to repeatedly iterate over stationary batches of training data. When exposed to incrementally available data from non-stationary distributions, such networks often fail to learn new information without forgetting much of its previously acquired knowledge, phenomenon often known as catastrophic forgetting (Ratcliff, 1990; McCloskey & Cohen, 1989; French, 1999). Despite significant advances, the limitation has remained a long- long-standing challenge for computational systems that aim to continually learn from dynamic data distributions (Parisi et al., 2019).
Among various proposed solutions, rehearsal approaches that store samples from previous tasks in an episodic memory and regularly replay them are of the earliest and most successful strategies against catastrophic forgetting (Lin,, 1992; Rolnick et al., 2019). An episodic memory is typically implemented as an array of independent slots; each slot holds one example coupled with its label. During training, these samples are interleaved with those from the new task, allowing for simultancous multi-task learning as if the resulting data were independently and identically distributed.
While such approaches are effective in simple settings, they require sizable memory and are often im- paired by memory constraints, performing rather poorly on complex datasets. possible explanation is that slot-based memories fail to utilize relational structure between samples; semantically similar items are treated independently both during training and at test time. marked contrast, relational memory is prominent feature of biological systems that has been strongly linked to successful memory retrieval and gencralization (Prince et al., 2005). Humans, for example, encode event features into cortical representations and bind them together in the medial temporal lobe, resulting in a durable, yet flexible form of memory (Shimamura, 2011).
In this paper, introduce novel Graph- based Continual Learning model (GCL) that resembles some characteristics of relational memory. More specifically, we explicitly model pairwise similarities between samples, including both those in the episodic memory and those found in the current task. These similarities allow for representation transfer between samples and provide resilient mean to guard against catastrophic forgetting. Our contributions are twofold:
(1) We propose the use of random graphs to represent relational structures between samples. While similar notions of dependencies have been proposed in the literature (Louizos et al., 2019; Yao et al., 2020), the application of random graphs in task-free continual learning is novel, at least to the best of our knowledge.

---

Published as a conference paper at ICLR 2021
MLP;
MUP
CNN
CNN
Teree
ML.P,
Figure 1: Illustration of Experiment Replay (ER) (Chaudhry al., 2019) the left and model (GCL) the right. While independently processes context images from episodic memory target images from the current task, GCL models pairwise similarities between the images via the random graphs G and A.
(2) We introduce a new regularization objective that leverages such random graphs to alleviate catastrophic forgetting. In contrast previous work (Rebuffi et al., 2017; Li & Hoiem, 2017) based on knowledge distillation (Hinton et al., 2015), the objective penalizes the model for forgetting learned edges between samples rather than their output predictions.
Our approach performs competitively on four commonly used datasets, improving accuracy by up to 19.7% and reducing forgetting by almost 37% in the best case when bench-marked against competitive baselines in task-free continual learning.
PROBLEM FORMULATION
In this work, we follow the learning protocol for image classification from Lopez-Paz Ranzato (2017). More specifically, the dataset for the f-th task D, = y() contains n, input-target pairs xf.y() E X x V. While the tasks arrive sequentially and exclusively, assume the imput-target pairs v.) task are independent and identically distributed (i..d.). The goal is to learn a supervised model fo X Y, parametrized by 0, that outputs class label y E ) given unseen image X.
consider training set 2 (D,.... DD} consisting of T tasks where
) in each
Following prior work (Lopez-Paz & Ranzato, 2017; Riemer et al., 2018; Chaudhry et al., 2019), we consider online streams of tasks in which samples from different tasks arrive at different times. As additional constraint, we insist that the model can only revisit a small amount of data chosen to be stored in fixed-size episodic memory M.
For clarity, we refer to the data in such an episodic memory as context images and context labels and denote by Xe [xxhiee and Yc {y: {y: respectively. These images and labels are to be distinguished from those in the current task, which we refer to as target images and target labels and denote by X = {x, [eT and Y [yT] respectively. While the model is allowed to update the context samples during training, the episodic memory is necessarily frozen at test time.
GRAPH-BASED CONTINUAL LEARNING
In this section, we propose a Graph-based Continual Learning (GCL) algorithm. While most rehearsal approaches ignore the correlations between images and independently pass them through network to compute predictions (Rebuffi et al., 2017; Chaudhry et al., 2019; Aljundi et al., 2019c), we model pairwise similarities between the images with learnable edges in random graphs (see Figure I). Intuitively, although it might be easy for the model to forget any particular sample, the multiple connections it forms with similar neighbors are harder to forgotten altogether. If trained well, the random graphs can therefore equip the model with plastic and durable means to fight against catastrophic forgetting.
Given minibatch of target images X from the current task, model makes predictions based on the context images X and context labels Y: that span several previously seen tasks, up and including the current one. In particular, explicitly build two random graphs
Graph Construction.
2

---

Published as a conference paper at ICLR 2021
of pairwise dependencies: undirected graph G between the context images Xe and directed, bipartite graph A from the context images X to the target images Xy.
Since an undirected graph can be thought of as directed graph between its vertices and a copy of itself. we treat the context graph G as such and build it analogously to the context-tary target graph A. Specifically, the high-dimensional context images X and target images X first mapped the image embeddings Uc and U,, respectively, using an image encoder fe, : '' + R Following Louizos et al. (2019), we then represent the edges in each graph by independent Bernoulli random variables whose means are specified by a kernel function in the embedding space. More precisely, the distribution of the resulting Erdös-Rény random graphs (Erdös & Rényi, 1959) can be defined as
(G) III
II iEC KEC
Ber Gik Ky (u,,u:))
P(A UT, Uc)
IIIII jET KEC
II
Ber(A jk R (uy, uk)),
(2)
for all i, C and E T where K, Rd x Rd - 20) is kernel function that encodes similarities between image embeddings such as the RBF kernel Kr (u;; uy) = exp (-3 [|u uy 13) Here, with slight abuse of notation, also G and A denote the corresponding adjacency matrices; A E {0, J. for example, represents the presence or absence of a directed edge between the j-th target image and the k-th context image.
Predictive Distribution. Given a context graph G and a context-target graph A that encode pairwise similarities to the context images, our next step propagate information from the context images Xe and context labels Ye to make predictions. To that end, we embed X by another image encoder fo, with weights partially tied to the previous one fe. and encode Ye by a linear label encoder before concatenating the resulting embeddings into latent representations Ve E IR Clxd,, In combination with the distributions of G and A, we compute context-aware representations for the context images and target images, denoted by {z, Jiec and {z, heeT respectively:
plz: Uc, Vc) / ((a.ve)(2))) (2)dP(G
(3)
p(z, Uc, Vc) = / (z) Uc)
(4)
where G; and A indicate the i-th and j-th row of G and A, each normalized to sum 1, and (.(.) denotes the indicator function on a set S. Intuitively, the representations Ve are linearly weighted by each graph sample, and the normalization step ensures proper scaling in case the numbers of edges formed with the context images vary. Once we summarize each image by the context samples, a final network fe, d2 + y takes as input the context- aware representations and produces predictive distributions:
ply: Xx) =
dP(z; Uc. Vc),
(5)
/ p ply, (z,)) dP(z) Uy, Uc. Ve).
ply, Xy. Xx)
(6)
Since the numbers of random binary graphs G and A are exponential, we approximate the integrals in (1) (6) by Monte Carlo samples. More specifically, we use one sample of G and A during training and 30 samples of A during testing. Also, these graph samples are inherently non-difeerentiable so we use the Gumbel-Softmal relaxations of the Bernoulli random variables during training (Maddison et al., 2016; Jang et al., 2016). The degree of approximation is controlled by temperature hyper- parameters, which exert significant influence over the density of the graph samples. We find that small temperature for and a larger temperature for A work well.
There are several reasons for making the graphs G and A random. First, the stochasticity induced by the Bernoulli random variables allows us to output multiple predictions and average these predictions, and such ensemble techniques have been quite successful in continual learning settings (Coop et al., 2013; Fernando et al., 2017). Perhaps more importantly, we find that the deterministic version with the Bernoulli random variables replaced by their parameters results in very sparse graphs where
3

---

Published as a conference paper at ICLR 2021
samples from the same classes are often deemed dissimilar. In similar fashion to dropout (Srivastava et al., 2014), the random edges encourage the model to be less reliant on a few particular edges and therefore promote knowledge transfer between samples. By similar reasoning, we remove self-edges in the context graph and also observe more connections between samples.
Graph Regularization. As training switches to new tasks, the distributional shifts to the target images necessarily result in changes to both the context graph G and the context-target graph A. In addition, the context images are regularly updated to be representative of the data distribution up to that point, so any well-learned connections between the context images are also susceptible to catastrophic forgetting. As remedy, we save the parameters of the Bernoulli edges to the episodic memory in conjunction with the context images and context labels, and introduce regularization term that discourages the model from forgetting previously learned edges:
1 [(b] ( (0))
(P(G G (G2)))
(01) A
Here, e(: denotes the cross-entropy between two probability distributions, Zt() the index set of edges to be regularized in the lith minibatch, and (0-1) the adjacency matrix learned from the beginning up to the previous minibatch. The selection strategies I are discussed in the next subsection. Besides the regularization term, our training objective includes two other cross-entropy losses, one for the context images and another for the target images:
C(11, 02)
Ac C
Σ (ynn!!")) SEC
+
AT Σ IT
(y5,88)) dc (01),
(8)
JET
~ UT. Uc. Ve) s/a0 for (z and z p(z: Uc. Vc), Vc), - are context-aware samples from Equations 3 and 4, and dc, XT, AG are hyperparameters.
where
While the graph regularization term appears similar knowledge distillation (Hinton al., 2015), we emphasize that the former aims to preserve the covariance structures between the outputs of the image encoder fe, rather than the outputs themselves. We believe that in light of new data, the image encoder should be able to update its potentially superficial representations of previously seen samples as long it keeps the correlations between them unchanged. Indeed, some of the early regularization approaches based on knowledge distillation (Li & Hoiem, 2017; Rebuffi et al., 2017) are sometimes too restrictive and reportedly underperform in certain scenarios (Kemker & Kanan, 2017).
Task-Free Knowledge Consolidation. When task identities are not available, we use reservoir sampling (Vitter, 1985) to update the context images and context labels as in Riemer et al. (201 8). The sampling strategy takes as input a stream of data and randomly replaces a context sample in the episodic memory with a target sample, with probability proportional to the number of samples observed so far. Despite its simplicity, reservoir sampling has been shown to yield strong performance in recent work (Chaudhry al., 2019; Riemer et al., 2018; Rolnick et al., 2019).
While most prior work uses task boundaries to perform knowledge consolidation at the end of cach task (Kirkpatrick et al., 2017; Rebuffi et al., 2017), update the context graph in memory after every minibatch of training data. In addition, such updates are performed at the sample level to maximize flexibility; keep track of the cross entropy loss each context sample and only update its edges in the graph when the model reaches a new low (denoted by [ previously). Intuitively, the loss measures how well the model has learned the context image through the connections forms with others. so meaningful relations are most likely obtained at the bottom of the loss surface. Though samples from the same task often provide more support for each other, the task-agnostic mechanism for updating the context graph also allows for knowledge transfer across tasks when necessary.
Memory and Time Complexity. The inclusion of pairwise similarities and graph regularization result in a time and memory complexity of ([IM [M] and ((M)), respectively, where |M] denotes the size of the episodic memory and N the batch size for target images. The quadratic costs in MM, however, are not concerning in practice, as we deliberately use a small, fixed-size episodic memory. The cost of storing G is often dwarfed by the memory required for storing digh-dimensiooal images, as each edge only needs one floating point number (see Appendix E for more details on memory usage).
4

---

Published as a conference paper at ICLR 2021
4 RELATED WORK
Continual Learning Approaches. The existing work on continual learning mostly falls into three categories: regularization, expansion, and rehearsal. Regularization approaches alleviate catastrophic forgetting by penalizing changes in model weights that are important for past tasks. Different measures of weight importance are considered, including Fisher information (Kirkpatrick et al., 2017; Chaudhry et al., 201 8a), synaptic relevance (Zenke et al., 2017), and uncertainty estimates (Ebrahimi et al., 2019). The constraints on weight updates can also be studied from Bayesian perspectives, where the posterior distribution of the weights is approximated and used as the prior for the next task (Nguyen et al., 2017; Ritter et al., 2018; Titsias et al., 2019). These regularization methods are efficient in memory and computational usage but suffer from brittleness due to representation drift (Titsias et al., 2019).
Expansion approaches dynamically allocate additional task-specific neural resources as more tasks arrive. Rusu et al. (2016), for example, blocks changes to parameters learned for previous tasks and expands sub-networks while Yoon et al. (2017) performs neuron splitting or duplication upon arrival of new tasks. Recently, non-parametric Bayesian approaches use Dirichlet process mixture models to expand a set of neural networks in a principled way (Jerfel et al., 2019; Lee et al., 2020). By design, these dynamic architectures prevent forgetting but quickly result in considerable model complexity.
Instead of growing model capacity, rehearsal approaches maintain a small episodic memory of previous data or, alternatively, train a generative model to produce pseudo-data for past tasks. which are then replayed and interleaved with samples from the new task. Such generative models (Shin et al., 2017; Kemker & Kanan, 2017; Achille al., 2018; Caccia et al., 2019; Ostapenko et al., 2019) reduce working memory effectively, but they are also susceptible to catastrophic forgetting and invoke the complexity of the generative task (Parisi et al., 2019). In contrast, episodic memory approaches are simpler and remarkably effective against forgetting (Rolnick et al., 2019; Wu et al., 2019). Lopez-Paz & Ranzato (2017) and Chaudhry et al. (2018b), for example, use an episodic storage of past data to impose inequality constraints on gradient updates while Rebuffi et al. (2017) constructs exemplars for knowledge distillation and nearest neighbor search. Recently, it has been shown that simple replay techniques and optimization- based meta-learning on the episodic memory outperform many previous approaches in online settings (Hayes et al., 2019; 9; Chaudhry et al., 2019; 2020; Riemer et al., 2018). Our model is also based on experience replay, but it differs from the other approaches in the way the episodic memory is handled.
Task-Free Continual Learning. In real-world scenarios, task changes are often unknown and definitive boundaries between tasks do not always exist. However, most methods mentioned above rely on explicit task identities or task boundaries to consolidate knowledge or select sub-modules for task adaptation. Despite its significance, there are only few works that address task-free continual learning. While Aljundi et al. (2019b) heuristically detects peaks in the loss surface to consolidate knowledge, Aljundi et al. (2019c;a) remove the need for task boundaries by sample selection strategy for the episodic memory. Recently, the aforementioned non-parametric approaches train density estimators to detect task boundaries and perform model expansion (Lee et al., 2020; Rao et al., 2019). In contrast, our approach uses reservoir sampling (Vitter, 1985) to update the episodic memory, similar to Riemer et al. (2018); Chaudhry et al. (2019).
Learning with Random Graphs. Although widely studied in graph theory (West et al., 2001), random graphs appear sparingly in the machine learning literature, perhaps more noticeably in neural architecture search (Xie et al., 2019). Our work is mostly related to previous work on functional neural process (Louizos et al., 2019), where the authors build random graphs of dependencies to represent relational structures between context points in a stochastic process. Our approach is different in that (I) the random graphs are undirected and grow incrementally, (2) no variational inference is required, and (3) it addresses catastrophic forgetting and performs well under continual learning settings.
A Attention Mechanism. While we motivate our approach from a graphical perspective, one can also consider it as some form of attention mechanism. In particular, the context graph G represents self-attention (Vaswani et al., 2017) across context images, and the mantext-target graph A represents cross-attention (Bahdanau et al., 2014) between context images and target images. Though advanced mechanisms such as multi-head attention have been applied successfully in many stationary settings (Vaswani et al., 2017; Xu et al., 2015; Zhang et al., 2018; Kim et al., 2019; Sprechmann al.,
5

---

Published as a conference paper at ICLR 2021
Figure 2: t-SNE visualization of image embeddings (small circles) from the penultimate layers and class embeddings (large circles) from the weights of the last layers SPLIT SVHN. The left figure shows that Finetune, model naively trained on the data stream, fails recognize the class-based clustering structure and bias the image embeddings toward the last task (class & 9). In contrast, the right figure shows that GCL (our model) maintains the relational structure and is more robust to the distributional shifts incurred by task changes.
2018), we note that naive applications of such techniques in online continual learning suffer from catastrophic forgetting due to representation drift when training switches new tasks. In contrast, our model employs random attention, which arguably makes it more robust to such distributional shifts (see Figure 2).
EXPERIMENTS
In this section, we evaluate the proposed GCL model on commonly used continual learning bench- marks. Additional results and details about the datasets, experiment setup, model architectures, and result analyses are available in the appendices.
Experiment Setup. We perform experiments 6 image classification datasets: PERMUTED MNIST, ROTATED MNIST (LeCun et al., 1998), SPLIT SVHN (Netzer et al., 2011), SPLIT CIFARIO (Krizhevsky et al., 2009), SPLIT CIFAR100 (Krizhevsky al., 2009), and SPLIT MINIIMAGENET (Vinyals et al., 2016). For each dataset, we follow Lopez-Pazz & Ranzato (2017); Chaudhry et al. (2018b) and adopt the setting where the model only has access to an online stream of data with a batch size of 10 (see Appendix A for more details).
We consider both single-head and multiple-head settings. More specifically, we use single-head and one-epoch settings for our model and all baselines on PERMUTED MNIST, ROTATED MNIST, SPLIT SVHN, and SPLIT CIFAR 10. While most previous work (Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017: Chaudhry et al., 2019) assume task identities on SPLIT CIFAR 10, we require all models to perform 10-way classification on each task with the same output head. This variant is more practical and challenging due to the need for incremental knowledge consolidation across tasks.
In addition, we also report results for multiple-head and 10-epochs settings on SPLIT CIFAR 100 and SPLIT MINIIMAGENET, following Lopez-Paz Paz & Ranzato (2017). These datasets have more classes and fewer samples per class, rendering them too challenging for single-head settings.
Model Architecture. Our image encoders fe, and fe, partially share weights and are parametrized by an MLP on the MNIST variants and a simple 6-layer convolutional network on other datasets, each followed by RELU activation and separate linear mapping. As alluded earlier, we use an RBF kernel to compute similarities between image embeddings and find it sufficiently easy for initialization. The output mappings fe, are MLPs in all cases (see Appendix for more details).
Baselines. We benchmark our model against multiple models, including (1) Finetune, a popular baseline, naively trained on the data stream; (2) EWC (Kirkpatrick et al., 2017), an early regularization approach; (3) GEM (Lopez-Paz Paz & Ranzato, 2017), rehearsal approach based on episodic memory of parameter gradients; (4) ER (Chaudhry et al., 2019), a simple yet competitive experience method based on reservoir sampling: (5) MER (Riemer et al., 2018), rehearsal approach inspired by optimization- based meta-learning, and (6) ICARL (Rebuffi et al., 2017) another well-known rehearsal strategy. Most of these baselines share the same model architectures: an MLP with two hidden layers on the MNIST variants, and a ResNet-18 (He et al., 2016) on SPLIT SVHN and SPLIT CIFAR 10, following (Lopez-Paz & Ranzato, 2017) (see Appendix C for more details).

---

Published as a conference paper at ICLR 2021
Table Classification results (%) on PERMUTED MNIST, ROTATED MNIST and SVHN. The means and standard deviations are computed over five runs using different random seeds, When used, episodic memories contain samples per class on average. The symbol 1 (U) indicates that higher (lower) number is better.
DATASET
PERMUTED MNIST
ROTATED MNIST
SPLIT SVHN
Methad
ACC (t)
FGT (4)
ACC (t)
FGT (4)
ACC (1)
FGT(L)
Finetune
6..19
23.62 1.98
43.80 + 1.64
46.52 + 1.71
18.85
94.78 = 1.24
EWC
64.94 1.22
18.33 1.07
44.99 1.73
44.98 1.95
18.76
94.99 1.23
GEM
19.17
3,68 0.68
82.60 + 0.48
5.47 0.45
33.40 3.27
68.91 + 4.06
ER
79.90 0.46
3.78 0.45
80.82
6.78 0.69
45.41 = 3.03
62.37 1 4.33
79.68 + 0.42
3.47 0.41
83.56 + 0.23
8.14
GCL
82.36 + 0.36
2.92 = 0.23
86.37 + 0.32
3.22 - 0.50
60.68 = 1.67
21.86 + 2.35
Table 2: Classification results (%) on SPLIT CIFARIO and SPLIT CIFARI00 and SPLIT MINIMMAGENET. The means and standard deviations computed over five using different random seeds, When used, episodic memories contain samples per class on average. The symbol f (L) indicates that a higher (lower) number
better.
DATASET
SPLIT CIFARIO
SPLIT CIFAR100
SPLIT MINIIMAGENET
Method
ACC (t)
FGT (4)
ACC (t)
FGT (4)
ACC (t)
FGT (L)
Finetune
18.46 + 0.12
86.48 + 1.02
55,39 + 1.94
25.94 + 1.89
37.84 = 0.87
31.41 + 1.57 28.17 + 4.49
EWC
18.49 + 0.13
86.95 + 1.15
55.60 + 1.11
23.53 + 1.19
36.61 = 2.06
ICARL
58.08 +: 1.44
24.22 + 1.35
GEM
22.88 3.41
76.90 1 5.53
65.66 1 0.70
15.52 0.41
54.06 - 0.22
13.17 1 0.74
ER
29.94
3.08
72.64 4.88
69.40 + 1.21
11.25 + 1.24
58.74 = 0.74
9.02 + 2.49
GCL
49.62 + 1.85
35.69 3.33
74.51 + 0.99
6.54
1.26
61.54 = 0.57
6.10 + 2.73
Metrics.
Following Lopez-Paz & Ranzato (2017); Chaudhry et al. (2018a; 2019), we evaluate the models using two classification metrics, namely, average accuracy and average forgettnng
1 ACC A
Rip
FGT
T-1 RRr R2,),
where RJ,, denotes the test accuracy on task after the model has finished task i. Intuitively, the former measures the average test accuracy across all tasks while the latter measures the average decrease between each task's peak accuracy and its accuracy the end of continual learning.
PERMUTED MNIST
SPL.TT CIFAR 100
Average Accuracy (%)
Average Accuracy (%)
75
70
60
Finetune
EWC
60
GEM
MER
GCL
45
20
Number Tasks
Number Tasks
Figure 3: Average accuracy as a function of the number of tasks trained.
7

---

Published as a conference paper at ICLR 2021
SPLIT CIFAR10 10
SPLIT CIFARI00
75
1,500
0.6
Testing Time per Sample (ms)
ER
GCL
Training
Testing
Average Accuracy (%)
Total Training Time (s)
1,000
0.4
25
500
0.2
250
1000
EWC
ER
ICARL
GCL
GEM
Memory Size
Method
Figure 4: Effects of episodic memory sizes.
Figure 5: Training and testing time.
Classification Performance. Table and 2 show the overall experimental results, and the evolution every setting, our
of performance as a function the number of tasks are detailed in Figure model (GCL) outperforms the baselines by significant margins, and the gains in performance are especially substantial on complex datasets such as SPLIT CIFAR or SPLIT CIFAR 100. As noted by Chaudhry et al. (2018b), EWC (Kirkpatrick et al., 2017) performs poorly without multiple passes over the datasets, and we additional find that GEM (Lopez-Paz Ranzato, 2017) not very effective under the single-head variants (e.g. SPLIT SVHN SPLIT CIFAR 10). Task-free approaches such as ER and MER perform more favorably, and such findings are consistent with recent studies (Chaudhry et al., 2019; Riemer al., 2018).
The advantageous performance of GCL over the other rehearsal strategies can be attributed to its efficient use of the episodic memory. Figure 4 shows that both ER (Chaudhry al., 2019) and GCL benefit from increases in memory size, but the outperformance of GCL is more visible under the low-resource regime. Sample efficiency, demonstrated. especially important since the memory constraints are not relaxable despite the growing complexity of the data distribution during training. also worth emphasizing that although our model takes more time to train and evaluate at test time than ER, its training time and testing time comparable to other approaches (see Figure 5).
Learned Graphs. Central to our approach are the pairwise similarities between context images captured by the context graph G. Figure 6 shows continuous realization of the context graph at the end continual learning on SPLIT CIFAR1O, which has been sorted according to context labels placed underneath the adjacency matrix. Despite being trained exclusively two classes of target images at a time (e.g., plane & car or bird & cat), the model appears to learn the clustering structure of images relatively well with more pronounced edges formed within classes than across them. The edges across tasks are noisier, but some edges indicate intuitive visual similarities such those between images of and truck. We note that the 10-way classification setup in each task encourages the model to clear inter-class edges, especially those within each binary task, so the degree of knowledge transfer across tasks is understandably more subtle.
55
45
ACC
Average Accuracy (%)
50
FGT
40
Average F Sreetting ( (%)
45
04
35
25
plans
dog
Figure 6: Context graph G SPLIT CIFAR10.
Figure Effeetts graph regularization (Ac).
8

---

Published as a conference paper at ICLR 2021
Ablation Study. We further investigate our model performance with an ablation study and summarize it in Table 3. With- out the graph regularization term in Equa- tion the model significantly performs worse, indicating that past connections between context samples can help alle- viate catastrophic forgetting. By varying the hyper-parameter ÄG we also see from Figure 7 that an extreme amount of graph regularization (e.g. ÀG 1000) can have detrimental effects on the model performance as well. As alluded earlier, the ability to draw multiple graph samples and average their predictions at test time brings out some gains, as often the case with ensemble methods. Perhaps more importantly, we find that making the context graph G and the context-target graph deterministic results in dramatic drop in accuracy. The resulting model is variant of attention mechanism, most similar to attentive neural process (Kim et al., 2019), and as discussed in Section 4, such a deterministic model often relies on a handful of edges, all of which are also prone to distributional shifts and thus catastrophic forgetting well.
Table 3: Ablation study on SPLIT CIFARIO.
Graph regularization Multiple graph samples Random G & A Deterministic G & A
4
X
V
X
X
X
Average accuracy
49.62
44.04
42.08
30.50
CONCLUSION AND DISCUSSION
In this work, we have introduced a graph-based approach to continual learning that exploits pairwise similarities between samples to support knowledge transfer. Based the learned graphs, we derive a regularization term to guide the training of new tasks against catastrophic forgetting. Our model demonstrates efficient of the episodic memory, and as result, performs competitively under various settings, without requiring access to task definition both during training and at test time in some cases.
As graph-based approaches, including ours, offer a natural way to describe relational inductive biases (Battaglia et al., 2018), we hope that future works further examine the applications of graphs under continual learning settings. If trained well, these graphs can be used not only to share knowledge but also minimize inference between samples and tasks. promising direction, for example, is to pose the problem of updating the episodic memory as graph search and leverage the rich literature on graph theory to devise better strategies for sample selection. As demonstrated by previous works (Aljundi et al., 2019c; Isele & Cosgun, 2018), such selection mechanisms can be effective against catastrophic forgetting, especially when the data distribution is not balanced across tasks.
7 ACKNOWLEDGMENT
Financial support is gratefully acknowledged from Xerox PARC Faculty Research Award, Na- tional Science Foundation Awards 1455172, 1934985, 1940124, and 1940276, USAID, and Cornell University Atkinson Center for a Sustainable Future.
REFERENCES
Alessandro Achille, Tom Eccles, Loic Matthey, Chris Burgess, Nicholas Watters, Alexander Lerch- ner, and Irina Higgins. Life-long disentangled representation learning with cross-domain latent 2018.
homologies. In Advances Neural Information Processing Systems, pp.
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems, pp. 1849-11866 2019a.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. Pro- ceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 11254-11263, 2019b.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems, pp. 11816-
1 11825, 2019c.

---

Published as a conference paper at ICLR 2021
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint UrXiv:1409.044 2014.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 1806:08000 2018.
Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and Joelle Pineau. Online learned continual compression with stacked quantization module. arXiv preprint 1.Xiv:9911.08011 2019.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 532-547, 2018a.
Arslan Chaudhry, Mare' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:/8/2.0040 2018b.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. Continual learning with tiny episodic memories. arXiv preprint arXiv: 10486, 2019.
Arslan Chaudhry, Albert Gordo, Puneet Dokania, Philip Torr, and David Lopez-Paz. Using hindsight to anchor past knowledge in continual learning. arXiv preprint arXiv:2002.08165, 2020.
Robert Coop, Aaron Mishtal, and Itamar Arel. Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems, (4(10):1623-1644 2013.
Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Mareus Rohrbach. Uncertainty-guided continual learning with bayesian neural networks. arXiv preprint Dttiv:1906.0220 2019.
P. Erdös and A. Rényi. On random graphs i. Publicationes Mathematicae Debrecen, 6:290, 1959.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv: 1 2017.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, (4):128-135, 1999.
Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investi- gation catastrophic forgetting gradient-based neural networks. urXiv preprint arXiv: 1312.6211, 2013.
Tyler L Hayes, Nathan D Cahill, and Christopher Kanan. Memory efficient experience replay for streaming learning. In 2019 International Conference on Robotics and Automation (ICRA), pp. 9769-9776. IEEE, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.025.: 02,333 2015.
Yen-Chang Hsu, Yen-Cheng Liu, Anita Ramasamy, and Zsolt Kira. Re-evaluating continual learning scenarios: A categorization and case for strong baselines. arXiv preprint arXiv:1810. 1210.1880,, 2018,
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Thirty-second AAAI conference on artificial intelligence, 2018.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint 1661 /. 1.01 144, 2016.
10

---

Published as a conference paper at ICLR 2021
Ghassen Jerfel, Erin Grant, Tom Griffiths, and Katherine A Heller. Reconciling meta-learning and continual learning with online mixtures of tasks. In Advances in Neural Information Processing Systems, pp. 9119-9130, 2019.
Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. arXiv preprint arXiv:/711.1056; 2017.
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Gamelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Tch. Attentive neural processes. arXiv preprint arXiv:1901.057611 2019.
James Kirkpatriek, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho. Agnieszka Grabska-Barve Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526 2017.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11):2278- 2324,, 1998.
Soochan Lee, Junsoo Ha, Dongsu Zhang, and Gunhee Kim. A neural dirichlet process mixture model for task-free continual learning. arXiv preprint arXiv:2001.00683 2020.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, 40 (0(12))2935-2947, 2017.
Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 833-4):293-321, 1992.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning, In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
Christos Louizos, Xiahan Shi, Klamer Schutte, and Max Welling. The functional neural process. In Advances in Neural Information Processing Systems, pp. 8743-8754, 2019.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv: 1.007 12, 2016.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Cuong V Nguyen, Yingzhen Li, Thang Bui, and Richard E Turner. Variational continual learning. 2017.
arXiv preprint
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 11321-11329, 2019.
German Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
Steven E Prince, Sander M Daselaar, and Roberto Cabeza. Neural correlates of relational memory: successful encoding and retrieval of semantic and perceptual associations. Journal of Neuroscience, 1505):120- 2005.
Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Tch, and Raia Hadsell. Continual unsupervised representation learning. In Advances in Neural Information Processing Systems, pp. 7645-7655, 2019.
Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
11

---

Published as a conference paper at ICLR 2021
Sylvestre-ANN Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:/8100 1910, 2018
Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. In Advances in Neural Information Processing Systems, pp. 4738-3748, 2018.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. In Advances in Neural Information Processing Systems, pp. 348-358, 2019.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1606 140..0666., 2016.
Arthur P Shimamura. Episodic retrieval and the cortical binding of relational activity. Cognitive, Affective, Behavioral Neuroscience, 11(3):277-21,, 2011.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Pablo Sprechmann, Siddhant M Jayakumar, Jack W Rae, Alexander Pritzel, Adria Puigdomenech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, and Charles Blundell. Memory-based parameter adaptation. arXiv preprint a8Xiv: 10542, 2018.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-199) 1958, 2014.
Michalis K Titsias, Jonathan Schwarz, Alexander G de G Matthews, Razvan Pascanu, and Yee Whye Teh. Functional regularisation for continual learning using gaussian processes. arXiv preprint 1rXiv:1901.11356, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pp. 3630-3638, 2016.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions Mathematical Software (TOMS), 11(1):37-57, 1985.
Douglas Brent West et al. Introduction to graph theory, volume 2. Prentice hall Upper Saddle River, 2001.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 374 382, 2019.
Saining Xie, Alexander Kirillov, Ross Girshick, and Kaiming He. Exploring randomly wired neural networks for image recognition. In Proceedings of the IEEE International Conference Computer Vision, pp. 1284-1293, 2019.
Kelvin Xu, Jimmy Ba. Ryan Kiros. Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In International conference machine learning, pp. 2048-2055 2015.
12

---

Published as a conference paper at ICLR 2021
Huaxiu Yao, Xian Wu, Zhiqiang Tao, Yaliang Li, Bolin Ding, Ruirui Li, and Zhenhui Li. Automated relational meta-learning. arXiv preprint arXiv:2001.00745. 2020.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 100.01070.01477, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 3987- 3995. JMLR. org, 2017.
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. arXiv preprint arXiv: /805008800., 2018.
13

---

Published as a conference paper at ICLR 2021
A EXPERIMENT SETUP
We perform experiments on six commonly used classification datasets: PERMUTED MNIST, ROTATED MNIST (LeCun et al., 1998). SPLJTT SVHN (Netzer et al., 2011), SPLIT CIFARIO (Krizhevsky et al., 2009), SPLIT CIFAR100 (Krizhevsky et al., 2009), and SPLIT MINTIMAGEÑET (Vinyals et al., 2016).
PERMUTED MNIST (Goodfellow et al., 2013) is a variant of the MNIST dataset of handwritten digits (LeCun al., 1998), where each task applies a fixed random pixel permutation to the original dataset. The benchmark dataset consists of 20 tasks, each with 1000 samples from 10 different classes.
ROTATED MNIST (Lopez-Paz & Ranzato, 2017) another variant of the MNIST dataset of handwritten digits (LeCun et al.. 1998), where each task applies fixed random image rotation to the original dataset. The benchmark dataset consists of 20 tasks, each with 1000 samples from 10 different classes.
SPLIT SVHN is variant of the SVHN dataset (Netzer al., 2011) that consists of 5 tasks, each with two consecutive classes, Since the benchmark dataset is much more challenging than the MNIST variants, we use all of its 73,257 training samples (i.e. 14,650 samples per task) to train our model and the baselines.
SPLIT CIFAR10 is variant of the CIFAR-10 10 dataset (Krizhevsky al., 2009). Similar SPLIT SVHN, the benchmark dataset consists of tasks, each with two consecutive classes. We use all of its 50,000 training samples (i.e. 10,000 samples per task) to train model and the baselines.
SPLIT CIFAR 100 is variant of the CIFAR- 100 dataset (Krizhevsky et al., 2009). The benchmark dataset consists of 20 tasks, each with consecutive classes. We use all of its 50,000 training samples (i.e. 2,500 samples per task) to train our model and the baselines.
SPLIT MINTIMAGENET is variant of the MINIIMAGENET dataset (Krizhevsky al., 2009). The benchmark dataset consists of 20 tasks, each with consecutive classes. We use all of its 50,000 training samples (i.e. 2,500 samples per task) train model and the baselines. Each image is resized to 84 x 84 pixels.
B MODEL ARCHITECTURES
As mentioned, while most of previous work multi-head architectures and assumes knowledge of task boundaries at test time, we employ a shared classifier head for all tasks. For the MNIST datasets, the image encoders fa, (for graph construction) and fez (for latent computation) share a multi- multi-layered perceptron with two hidden layers of 256 ReLU neurons, followed by two separate linear mappings, one for each of the encoders. For SPIIT SVHN, SPLIT CIFAR 10, SPLIT CIFAR 100, and SPL.TT MINIIMAGENET, the image encoders share a simple convolutional network with the following structure: conv > conv 64 > maxpool + conv 64 conv 64 maxpool -> conv 64 conv 64 + maxpool, where conv NF is a 3 x 3 convolution with NF output filters, BatchNorm, and ReLU activations. For all datasets, another linear mapping follows the image encoder fe, before a Gaussian kernel computes the similarities between image embeddings. Finally, the classifier head consists of a RELU activation and a single linear mapping.
C BASELINE ARCHITECTURES
We use the same neural network architectures for all the baselines described in the paper: multi- layered perceptron with two hidden layers of 400 ReLU neurons on PERMUTED MNIST and ROTATED MNIST, following (Hsu et al., 2018), and ResNet-18 (He et al., 2016) with 20 filters across all layers on other datasets, following (Lopez-Paz & Ranzato, 2017). For all datasets, the baselines consist of more parameters than our corresponding models (see Table 4 for details).
14

---

Published as a conference paper at ICLR 2021
Table Number trainable parameters continual learning models.
Method
Finetune
EWC
GEM
ER
MER
GCL
SPLIT MNIST
478K
478K
478K
478K
478K
406K
PERMUTED MNIST
478K
478K
478K
478K
478K
406K
ROTATED MNIST
478K
478K
478K
478K
478K
406K
SPLIT SVHN
1,09M
1.09M
1.09M
1.09M
326K
SPLIT CIFARIO
1.09M
1.09M
1.09M
1.09M
326K
SPLIT CIFAR 100
1.09M
1.09M
1.09M
1.09M
326K
SPLIT MINTMMAGENET
1.09M
1.09M
1.09M
1.09M
343K
We adopt the implementations of EWC (Kirkpatrick et al., 2017), GEM (Lopez-Paz & Ranzato, 2017), and MER (Riemer al., 2018) from the authors' repositories
D ADDITIONAL TASK-FREE BASELINES
We also note that despite our attempts to tune parameters for MER (Riemer et al., 2018) on SPLIT SVHN and SPI.IT CIFAR 10, the baseline does not perform reasonably well. The model uses a batch size of and requires multiple passes through the episodic memory per batch, so it is much slower than our model and all other baselines. Due to limited time and computational resources, we do not further investigate the baseline and therefore avoid reporting immature results for fairesss.
However, we include results of CN-DPM (Lee et al., 2020), a competitive task-free model based on Dirichlet process mixture models in Table 5, Our setup for SPLIT CIFARIO analogous to that of Lee et al. (2020), so we directly quote the numbers for CN-DPM from the paper. Although
CN-DPM performs favorably among fusk-free approaches continually learning, including GSS (Aljundi et al., 201 2019c), our model outperforms CN-DPM by significant margin, even when using smaller memory size.
Table 5: GCL results and CN-DPM results with different memory sizes.
SPLIT SVHN
SPLIT CIFARTO
Method
250
500
500
1000
ER (Chaudhry et al., 2019)
45.51 +: 3.03
57.51 1: 2.77
36.08 1: 1.09
45.75 1: 1.82
CN-DPM (Lee al., 2020)
43.07 1 0.16
45.21 0.18
GCL (Ours)
60.68
1.67
65.79 + 1.54
53.87 0.97
57.26
0.28
E MEMORY USAGE
Both GCL and ER (Chaudhry et al., 2019) uses an episodic memory to store images and labels from past tasks. The only additional memory usage of GCL comes from the context graph G. which is represented by square matrix whose entries intuitively describe pairwise similarities between such images. Given a memory consisting of JM| images of size C' H W, it only requires |M| floating points to store the matrix.
Table 6: Memory usage of ER and GCL for various datasets.
DATASET
|M
Image Size
ER
GCL
PERMUTED MNIST
1000
1
x
28
x 28
3,284 MB
7.284 MB
ROTATED MNIST
1000
1
x
28
x
28
3,284 MB
7.284 MB
SPLIT CIFAR10
250
3
32
32
3.109 MB
3.359 MB
SPLIT SVHN
250
3
32
32
3.109 MB
3.359 MB
SPLIT CIFAR 100
500
32
x
32
6,219 MB
7.199
SPLIT MINIIMAGENET
500
3 x
84
x 84
42,408 MB
43,389 MB
15

---

Published as a conference paper at ICLR 2021
As seen from Table 6, the memory usage of GCL very similar the same that of ER, except when both are very small as in the case of PERMUTED MNIST and ROTATED MNIST, because (1) continual learning algorithms are often required to very small |M] and (2) the cost for storing natural images are often much higher than that of the context graph.
As the number of tasks increases, it is perhaps essential to expand the episodic memory, in which the quadratic growth of the latter might dominate the linear increase of the former (e.g. [M] = 5000 and images are of size 32 32). Although have not practically encountered such a problem with GCL, we note that the quadratic growth of the number of entries in the context graph can be reduced to linear growth in memory requirements. More specifically, each entry the output of the kernel function Ri (see Section 3, e.g. Kr (u, uj) exp (-3
3 l/ u. uy 1)]12),, ). so we could easily store |M| intermediate embeddings {u:} } at each step and apply the kernel function on the fly, which is especially beneficial when are much lower dimensional than the original images.
ADDITIONAL. EXPERIMENT RESULTS
PERMUTED MNIST
ROTATED MNIST
90
100
Average Accuracy (%)
90
80
80
75
70
70
65
60
Finetune
EWC
60
50
GEM
ER
MER
GCL
55
40
10
15
20
5
10
15
20
SPLIT SVHN
SPLIT CIFAR10
Average Accuracy (%)
80
80
60
60
40
40
20
3
Figure 8: Average accuracy as a function the number of tasks trained on PERMUTED MNIST, ROTATED
MNIST, SPLIT SVHN, and SPLIT CIFARIO.
SPLIT SVHN
SPLIT CIFAR10
ER
GCL
Average Accuracy (%)
100
250
500
1000
100
250
500
1000
Memory Size
Memory Size
Figure 9; Average accuracy function of memory size on SPLIT SVHN SPLIT CIFAR10.
16

---

Published as a conference paper at ICLR 2021
SPLIT SVHN
SPLIT CIFARIO
100
100
ER
Ours
Average Accuracy (%)
75
75
50
50
100
250
500
1000
100
250
1000
Memory Size
Memory Size
Figure 10: Average forgetting function of memory size SPLIT SVHN and SPLIT CIFARIO.
G HYPER-PARAMETERS
Following Lopez-Pazz & Ranzato (2017), we report the hyper-parameter grids considered our experiments. These hyper-parameters selected independently for each model, and the best values are given in parenthesis.
Finetune
optimizer: [Adam (Split SVHN, Split CIFAR10), SGD (Permuted MNIST, Rotated MNIST)]
learning rate: [0.0002, 0.001 (Split SVHN, Split CIFAR10, Split CIFAR100, Split Mini Imagenet), 0.01, 0.1 (Permuted MNIST, Rotated MNIST), 0.3, 1.0]
EWC (Kirkpatrick et 2017)
optimizer: [Adam, SCD (Permuted MNIST, Rotated MNIST, Split
SVHN, Split CIFAR10)]
learning rate: 1002 (Split SVHN, Split CIFAR10), 0. 0.001, 0.01, 0.1 (Permuted MNIST, Rotated MNIST), 0.3, 1.0]
regularization: [0.1, 1, 10 (Permuted MNIST, Rotated MNIST), 100 (Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagenet), 1000]
GEM (Lopez-Paz& & Ranzato, 2017)
optimizer: [Adam (Split SVHN, Split CIFAR10), SGD (Permuted MNIST, Rotated MNIST)]
learning rate: [0.0002, 0.001 (Split CIFAR10, Split CIFAR100, Split MiniImagenet), 0.01, 0.1 (Permuted MNIST, Rotated MNIST),
0.3, 1.0]
margin: [0.0, 0.1, 0.5 (Permuted MNIST, Rotated MNIST), 1.0 (Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagenet)]
MER (Riemer al., 2018)
optimizer: [SCD (Permuted MNIST, Rotated MNIST)]
learning rate: [0.0002, 0.001, 0.01, 0.1 (Permuted MNIST,
Rotated MNIST), 0.3, 1.0]
within batch meta-learning rate (beta): [0.01 (Permuted MNIST, Rotated MNIST, Split CIFAR10, Split CIFAR100, Split MiniIIageeett, 0. 0.03, 0.1, 0.3, 1]
ER (Chaudhry et al., 2019)
optimizer: [Adam (Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagenet), SGD (Permuted MNIST, Rotated MNIST)
17

---

Published as a conference paper at ICLR 2021
learning rate: [0.0002, 0.001 (Split SVHN, Split CIFAR1O, Split CIFAR100, Split Mini Imagenet), 0. 0.01, 0.1 (Permuted MNIST, Rotated MNIST), 0.3, 1.0]
GCL
optimizer: [Adam (Permuted MNIST, Rotated MNIST, Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagenet), SGD]
learning rate: [0.0002, 0.001 (Permuted MNIST, Rotated MNIST, Split SVHN, Split CIFAR10, Split CIFAR100, Split Mini MiniImagenet), 0.01, 0.1, 0.3, 1.0]
graph regularization: [0, 10, 50 (Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagenet), 100, 1000, 5000 (Rotated MNIST)]
context temperature: [0.1 (Permuted MNIST, Rotated MNIST), 0.3, 1 (Split SVHN, Split CIFAR10, Split CIFAR100, Split MiniImagentt), 5, 10]
target temperature: [0.1, 0.3, 1, (Permuted MNIST, Rotated MNIST, Split SVHN, Split CIFAR10, Split CIFAR100, Split Mini Imagenet), 10]
18


Reviews:
[{"review_id": "e3EMvjX5Ak", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an interesting idea of using random graphs to represent relational structures amongst contextual samples and between contextual samples and target samples. Besides, the authors propose a regularization objective to alleviate catastrophic forgetting. \n\nThe novelty mainly comes from the use of graph structure to preserve the memory of previous tasks. The results on public datasets are encouraging. Also, the authors show some correspondence between the learned graphs and underlying (clustering) structure of data. It would be great if the authors can also extend this algorithms to other real-world datasets, e.g., capturing the climate or environmental changes over time, and provide more interpretation of learned graph structrure. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "NQNzTH8uHIt", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe paper proposes a novel way of using random graphs to improve task-free continual learning method. It builds to random graphs, G and A, based on the similarity of images stored in the memory and those of the current tasks, and utilize the relative information to build representation of the images and predict. The idea is well-formulated, and carried out in a sound way. The graph regularization term resembles the knowledge-distillation, as the authors also mentioned, but it serves different purpose of preserving the covariance structure of the outputs of the image encoders. \n\nThe experimental results look quite strong and the ablation study also looks sound. Overall, I find the paper quite strong. \n\nCon & Questions:\n\n- It seems like there are 4 hyperparameters: \"temperature\" for the Gumbel-Softmax relaxation and 3 lambdas. Fig 6 shows the effects of $\\lambda_G$, but what about others? How sensitive is the performance with respect to other hyperparamters? Since the problem setting is a single-pass setting, how are the hyperparameters selected?\n\n- The inference time is a bit longer (not too much than others) since the algorithm has to sample graphs 30 times. What happens if the number of samples is less than 30 so that the inference time becomes similar to ER? Would the performance degrade significantly? How critical is the sample size 30?", "extra_scores": {}, "extra_text": {}}, {"review_id": "JEj54l1Xyp-", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n - the combination of components is novel, Especially, this paper consider the relevance of instance in memory. \n- The model can not only prevent forgetting, but also help the learning of new tasks. The promotion of current tasks is also crucial in continual learning.\nWeaknesses: \n- the empirical validation is weak. Therefore, more new models need to be compared. For more details, please refer to “Reasons for reject”\n\nReasons for accept:\n1.\tThe structure of this paper is clear and easy to read. Specifically, the motivation of this paper is clear and the structure is well organized; the related work is elaborated in detail; the experimental setup is complete. \n2.\tBased on the use of replay to solve catastrophic forgetting, the current popular graph structure is introduced to capture the similarities between samples. Combined with the proposed Graph Regularization, this paper provides a new perspective for solving catastrophic forgetting. \n3.\tThe experimental results given in the paper can basically show that the proposed method is effective. The ablation study also verified the effectiveness of each component. \n\nReasons for reject:\n1.\tThe lack of comparison of experimental effects after replacing Graph Regularization with other regularization methods mentioned in this paper, or other distance measurement methods, eg., L2.\n\n2.\tThis paper compares relatively few baselines, especially recent studies. I hope to see the comparison results of some papers in the list below. The latest papers on the three types of methods (regularization, expansion, and rehearsal) for solving catastrophic forgetting are included. Therefore, if it can be compared with some of these models, it will be beneficial to the evaluation of GCL.\n\n[1] Ostapenko O , Puscas M , Klein T , et al. Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning. ICML 2019\n[2] Y Wu, Y Chen, et al. Large Scale Incremental Learning. CVPR 2019\n[3] Liu Y , Liu A A , Su Y , et al. Mnemonics training: Multi-class incremental learning without forgetting. CVPR 2020\n[4] Zhang J , Zhang J , Ghosh S , et al. Class-incremental learning via deep model consolidation. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV)\n[5] Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continuous learning of context-dependent processing in neural networks. Nature Machine Intelligence, 2019.\n[6] Wenpeng Hu, Zhou Lin, et al. Overcoming catastrophic forgetting for continual learning via model adaptation. ICLR 2019\n[7] Rao D , Visin F , Rusu A A , et al. Continual Unsupervised Representation Learning. NeurIPS 2019\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2p_Y6d8W0qn", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presented a memory-based continual learning model where relationships between training samples are represented with a random graph that is defined from the non-linear embedding of the input data. Catastrophic forgetting between tasks is partially (1) alleviated with a graph regularization that penalizes changes of random graph statistics, and (1) memory replay and reservoir sampling to update memory. The performance of this model is evaluated against several state-of-the-art models to handle the catastrophic loss. \n\nI tend to rate this paper as a good paper for acceptance because the presentation is clear, the idea is novel and the experiment is convincing.\n\nPros:\n\n+ clear presentation in Section 3, Graph-Based Continual Learning, plus Figure 1.\n+ Good literature survey in Section 4, Related Work.\n+ A good idea to related graph network and attention mechanism.\n\nCons: \n\n- Are all edges equally important in the regularisation term in Eq. 7? I feel more theory could be dug out by using, for example, tangent distance.\n- What does a graph representation really give us, since as authors mentioned, this is in a sense an attention mechanism? Can higher-order graph relationships be utilized if we replace a random graph with a ergm containing more structures? \n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an interesting idea for task-free continual learning, which makes use of random graphs to represent relational structures among contextual and target samples. The reviewers agreed that the technical idea is novel, the experiments are extensive and the presentation is good. The authors addressed the reviewers' concerns in the rebuttal. I recommend to accept.
Decision: Accept (Spotlight)