Discriminative Representation Loss (DRL): A More Efficient Approach than Gradient Re-Projection in Continual Learning
Authors: Yu Chen
Tom Diethe
Peter Flach
Venue:  (2021)
Keywords: continual learning
episodic memory
GEM
experience replay
deep metric learning

Abstract: The use of episodic memories in continual learning has been shown to be effective in terms of alleviating catastrophic forgetting. In recent studies, several gradient-based approaches have been developed to make more efficient use of compact episodic memories, which constrain the gradients resulting from new samples with those from memorized samples, aiming to reduce the diversity of gradients from different tasks.  In this paper, we reveal the relation between diversity of gradients and discriminativeness of representations, demonstrating connections between Deep Metric Learning and continual learning. Based on these findings, we propose a simple yet efficient method -- Discriminative Representation Loss (DRL) -- for continual learning. In comparison with several state-of-the-art methods, this method shows effectiveness with  low computational cost on multiple benchmark experiments in the setting of online continual learning.

Under review as a conference paper at ICLR 2021
DISCRIMINATIVE REPRESENTATION LOSS (DRL): A MORE EFFICIENT APPROACH THAN GRADIENT RE-PROJECTION IN CONTINUAL LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
The use of episodic memories in continual learning has been shown to be effective in terms of alleviating catastrophic forgetting. In recent studies, several gradient- based approaches have been developed to make more efficient use of compact episodic memories, which constrain the gradients resulting from new samples with those from memorized samples, aiming to reduce the diversity of gradients from different tasks. In this paper, we reveal the relation between diversity of gradients and discriminativeness of representations, demonstrating connections between Deep Metric Learning and continual learning. Based on these findings, we propose simple yet highly efficient method- Discriminative Representation Loss (DRL) - for continual learning. In comparison with several state-of-the art methods, DRL shows effectiveness with low computational cost multiple benchmark experiments in the setting of online continual learning.
INTRODUCTION
In the real world, we are often faced with situations where data distributions are changing over time, and we would like to update models by new data in time, with bounded growth in system size. These situations fall under the umbrella of "continual learning", which has many practical applications, such as recommender systems, retail supply chain optimization, and robotics (Lesort) et al., 2019; Diethe et al. 2018 Tian et al. 2018). Comparisons have also been made with the way that humans are able learn new tasks without forgetting previously learned ones, using common knowledge shared across different skills. The fundamental problem in continual learning is catastrophic forgetting (McCloskey & Cohen 1989 Kirkpatrick et al. 2017). i.e. (neural network) models have a tendency to forget previously learned tasks while learning new ones.
There are three main categories of methods for alleviating forgetting in continual learning: i) regular- rzationbbase methods which aim in preserving knowledge of models of previous tasks (Kirkpatrick) et al. 2017] 7, Zenke et al., 2017 Nguyen et al. 2018) ii) architecture- methods for incrementally evolving the model by learning task-shared and task-specific components (Schwarz et al. 2018 Hung methods which focus in preserving knowledge data distributions of previous tasks, including methods of experience replay by episodic memories or generative mod- els (Shin et al.] [0177 Rolnick et al., 2019), methods for generating compact episodic memories (Chen et al. 2018 |Ajundi et al. 2019), and methods for more efficiently using episodic memories [Lpezz-Paz Paz & Ranzato 2017 Chaudhry et al. 2019a Riemer et al.) 2019 Farajtabar et al. 2020).
cct al.) 2019); iii replay-bent
Cradient-based approaches using episodic memories, in particular, have been receiving increasing attention. The essential idea is to use gradients produced by samples from episodic memories to constrain the gradients produced by new samples, e.g. by ensuring the inner product of the pair of
gradients is non-negative (Lopez-Paz & Ranzato 2017) as follows:
gg) /aC(z, 0) dL(1z, 0)
> 0, vk<t
00
a0
where t and l are time indices, Ie denotes new sample from the current task, and 2& denotes a sample from the episodic memory. Thus, the updates of parameters are forced to preserve the performance on previous tasks as much possible

---

Under review as a conference paper at ICLR 2021
In Gradient Episodic Memory (GEM) (oopez Paz& & Ranzato 2017). Sr is projected to direction
that is closest to in Ly-norm whilst also satisfying Eq. I ming 2 19: 912. s.t.(g.90 9k) 2 O, Ak < t. Optimization of this objective requires high-dimensiona quadratic program and thus is computationally expensive. Averaged-GFM (A (A-GEM) (Chaudhry et al. 2019a) alleviates the computational burden of GEM by using the averaged gradient over batch of samples instead of individual gradients of samples in the episodic memory. This not only simplifies the computation. but also obtains comparable performance with GEM. Orthogonal Gradient Descent (OGD) (Farajtabar et al. 2020) projects Sr to the direction that is perpendicular to the surface formed by {gElk < t). Moreover, Aljundi et al.] (209)) propose Gradient- based Sample Selection (GSS), which selects samples that produce most diverse gradients with other samples into episodic memories. Here diversity is measured by the cosine similarity between gradients. Since the cosine similarity is computed using the inner product of two normalized gradients, GSS embodies the same principle as other gradient-base based approaches with episodic memories. Although GSS suggests the samples with most diverse gradients are important for generalization across tasks, [Chaudhry et al. (2019b) show that the average gradient over a small set of random samples may be able to obtain good generalization as well.
In this paper, we answer the following questions: i) Which samples tend to produce diverse gradients that strongly conflict with other samples and why such samples able to help with generalization? ii) Why does a small set of randomly chosen samples also help with generalization? üi) Can we reduce the diversity of gradients in a more efficient way? Our answers reveal the relation between diversity of gradients and discriminativenes of representations, and further show connections between Deep Metrie Learning (DML) (Kaya & Bilge 2019; Roth al. 2020) and continual learning. Drawing on these findings we propose a new approach, Discriminative Representation Loss (DRL), for classification tasks in continual learning. Our methods show improved performance with relatively low computational cost in terms of time and RAM cost when compared to several sfate-of-the- art (SOTA) methods across multiple benchmark tasks in the setting of online continual learning.
A NEW PERSPECTIVE OF REDUCING DIVERSITY OF GRADIENTS
According to Eq. ( negative cosine similarities between gradients produced by current and previous tasks result in worse performance in continual learning. This can be interpreted from the perspective of constrained optimization as discussed by Aljundi al. (2019). Moreover, the diversity of gradients relates to the Gradient Signal to Noise Ratio (GSNR) (Liu et al. 2020), which plays a crucial role in the model's generalization ability. Intuitively, when more of the gradients point in diverse directions, the variance will be larger, leading to a smaller GSNR, which indicates that reducing the diversity of gradients can improve generalization. This finding leads the conelusion that samples with the most diverse gradients contain the most critical information for generalization which is consistent with in Aljundi et 2011
2.1 THE SOURCE OF GRADIENT DIVERSITY
We first conducted a simple experiment on classification tasks of 2-D Gaussian distributions, and tried to identify samples with most diverse gradients in the 2-D feature space. We trained a linear model on the first task to discriminate between two classes (blue and orange dots in Fig. la). We then applied the algorithm Gradient-basse Sample Selection with Interger Quadratic Programming (GSS-IQP) (Aljundi et al. 2019) to select 10% of the samples of training data that produce gradients with the low- (g..8,) est similarity (black dots in Fig. and denote this set of samples as M minm ΣiJES T 1:1
is clear from Fig. la that the samples in M are mostly around the decision boundary between the two classes. Increasing the size of M results in the inclusion of samples that trace the outer edges of the data distributions from each class. Clearly the gradients can be strongly opposed when samples from different classes are very similar. Samples close to decision boundaries are most likely to exhibit this characteristic. Intuitively, storing the decision boundaries of previously learned classes should be an effective way to preserve classification performance on those classes. However, if the episodic memory only includes samples representing the learned boundaries, it may miss important information when the model is required to incrementally learn new classes. We show this by introducing a second task- training the model above on a third class (green dots). We display the decision boundaries (which split the feature space in one vs. all manner) learned by the model after
2

---

Under review as a conference paper at ICLR 2021
(a) Samples with most diverse gra- dients (M) after learning task 1, the green line the decision boundary.
(b) Learned decision boundaries (purple lines) after task 2. Here the episodic memory includes samples in M.
(c) Learned decision boundaries (purple lines) after task 2. Here the episodic memory consists of ran- dom samples.
Figure 1: 2-D classification examples, the and y axis the coordinates (also features) of samples. We sequentially train a logistic regression model on two tasks: the first task is to classify two classes as shown in (a); the second class incrementally classify third class as shown in (b) and (c). The solid lines are decision boundaries between classes.
KC5
65
20
(a) Splitting samples into several sub- (b) Estimated distributions of B (c) Estimated distributions of a sets in a 3-class classification task. when drawing negative pairs from when drawing negative pairs from Dots in different colors are from dif- different subsets of samples. ferent classes.
different subsets of samples.
Figure 2: Illustration of how Pr(2ß > a - in Theorem behaves in various cases by drawing negative pairs from different subsets of 3-class feature space which are defined in Fig. 2a The classifier a linear model. y-axis in the right side of (b) & (c) is for the case E S U U.S. We see that ö behaves in similar way with (9 but in smaller range which makes the key in studying
Pr(2,3 > a 8). In the case of x E S3 the distribution of B has more mass on larger values than other cases because the predicted probabilities are mostly on the two classes in a pair, and it causes all (ga, gm) having the opposite sign of (Xn, Xm) as shown in Tab.
task 2 with M (Fig. lb) and random set of samples (Fig. lc from task 1 as the episodic memory. The random episodic memory shows better performance than the one selected by GSS-IQP, since the new decision boundaries rely on samples not included in M. It explains why randomly selected memories may generalize better in continual learning. Ideally, with M large enough, the model can remember all edges each class, and hence learn much more accurate decision boundaries sequentially. However, memory size is often limited in practice, especially for high-dimensional data. more efficient way could be learning more informative representations. The experimental results indicate that: 1) more similar representations in different classes result in more diverse gradients. 2) more diverse representations in a same class help with learning new boundaries incrementally.
Now formalise the connection between the diversity of gradients and the discriminativeness of representations for the linear model (proofs are in Appx. Notations: Negative pair represents two samples from different classes. Positive pair represents two samples from a same class. Let L represent the softmax cross entropy loss, W E IR BRDx is the weight matrix of the linear model, and x,, RDD denotes the input data, Yn JRK is one-hot vector that denotes the label of x,. D is the dimension of representations, K is the number of classes. Let p., (oftma(((,))) where On W the gradient gn Vw W). X,, Xm are two different samples when
Lemma 1. Let Cn = Pa Yn. we have; (gn, (gn.gm) (Xn, Xm (Cn, Cm).
3

---

Under review as a conference paper at ICLR 2021
Table 1: Illustration of the Theorems by drawing pairs from different subsets that are defined in Fig. 2a We obtain the gradients and predictions by linear model and a MLP with two hidden layers (16 units for each) and ReLU (or tanh) activations. The gradients are computed by all parameters of the model. We can see that the non-linear models exhibit similar behaviors with the linear model as described the theorems. One exception is that the MLP with ReLU activations gets much less negative (gn, Im. in the case of S U S2 for negative pairs, we consider the difference is caused by representations to the final linear layer always being positive in this case due to ReLU activations.
Negative pairs (Thm. 1)
Positive pairs (Thm.2)
So
So U.
S3
Si U S2
So
So U Si
S3
Si U S2
Pr((Xn, Xm) 0)
1.
0.877
1.
0.
1.
0.99
1.
1.
Pr((gn: Im) 0)
0.727
0.725
1. 0.978
0.
0.007
0.
Linear
Pr(23 + +6>00000 10>00. a.)
0.727
0.687
1.
0.
Pr( (mm) < 0)
0.72
0.699
1. 0.21
0.013
0.01
0.
0.
MLP (ReLU)
Pr(2, 83> x))
0.746
0.701
1. 0.
Pr( Pr[(gn. [m)) < 0)
0.745
0.744
0.993
0.004
0.007
0.
MLP (tanh)
Pr(23 + 8> > a)
0.766
0.734
1. 0.
Theorem 1. Suppose Yn F Yun and let on denote the class index of Xn fi.e. yn.e, 1,yn,
0, Vi + ca). Let a A |p 12 + |Pmm ,3 A Pn.c.. + Pm.c. and a Il Pm then:
Pr Prigin( (gn 9m)) sign( (xx, xm))) Pr(23 > a 5),
Theorem 2. Suppose yn = ym, when (gn. # O, we have: sign( (gn: 9m)) = sign( (Xn; Xmn)
For better understanding of the theorems, conduct empirical study by partitioning the feature space of three classes into several subsets as shown in Fig. [2 and examine four cases of pairwise samples by these subsets: 1). x E So, both samples in pair are near the intersection of the three classes; 2). x E So S1, one sample is close to decision boundaries and the other is far away from the boundaries: 3). x E Sz. both samples close the decision boundary between their true classes but away from the third class; 4). x E S U S2, both samples are far away from the decision boundaries.
Theorem says that for samples from different classes, (gn: 9m) gets an opposite sign of (Xn, X.. with probability that depends on the predictions pr, and pm. This probability of flipping the sign especially depends on .3 which reflects how likely to misclassify both samples to its opposite class. We show the empirical distributions of (3 and (a - 5) obtained by a linear model in Figs. 2b and 2c respectively. In general, (a- 5) shows similar behaviors with /3 in the four cases but a smaller range, which makes 2/3 > (a 5) tends to be true except when B is around zero Basically, subset including more samples close to decision boundaries leads to more probability mass on large values of B, and the case of x E S3 results in largest mass large values of β because the predicted probabilities mostly concentrate on the two classes in a pair. As shown in Tab. I more mass on large values of B leads to larger probabilities of flipping the sign. These results demonstrate that samples with most diverse gradients (which gradients have largely negative similarities with other samples) close to decision boundaries because they tend to have large B3 and (xn, xm) tend to be positive. In the case of x E S1 U S2 the probability of flipping the sign is zero because / concentrates around zero. According to Lemma (gn.9m) are very close to zero in this case because the predictions are close to true labels, hence, such samples are not considered as with most diverse gradients.
Theorem 2 says (m) has the same sign as (X,, Xm. when the two samples from same class. We can see the results of positive pairs in Tab. [ matches Theorem 2. In the case of So U S₁ the two probabilities do not add up to exactly because the implementation of cross-entropy loss in tensorflow smooths the function by a small value for preventing numerical issues which slightly changes the gradients. As (Xn, xm.) mostly positive for positive pairs, (gn, hence is also mostly positive, which explains why samples with most diverse gradients are not sufficient to preserve information within classes in experiments of Fig. On the other hand, if (x,,Xm) is negative then (9n.9m) will be negative, which indicates representations within class should not be too diverse.
Extending this theoretical analysis based on a linear model, we also provide empirical study of non-linear models (Multi-layer Perceptrons (MLPs)). As demonstrated in Tab. [ the probability of flipping the sign in MLPs are very similar with the linear model since it only depends on the predictions and all models have learned reasonable decision boundaries. The probability of getting
4

---

Under review as a conference paper at ICLR 2021
16.
10
same 20
dlss
Simiurty
dif dass
1.5
SAme class
-075
ate
-0.99
-2.5
0.00
025 6.e 575 3.0
Simiartty gradents
Simiarity of gradients
(a) Similarities of gradients vs. repre- sentations (class 7 & 9)
(b) Similarities of gradients vs. repre-
sentations (class 0 & 1)
Figure 3: Similarities of gradients and representations of two classes in the MNIST dataset. The x and y axis are the cosine similarity of gradients and representations, respectively. Blue dots indicate the similarity of negative pairs, while orange dots indicate that of positive pairs.
negative (gn. Im)) is also similar with the linear model except in the case of S1 U S2 for negative pairs, in which the MLP with ReLU gets much less negative (gn, 9m. As MLP with tanh activations is still consistent with the linear model in this case, we consider the difference is caused by the representations always being positive due to ReLU activations. These results demonstrate that non-linear models exhibit similar behaviors with linear models that mostly align with the theorems.
Since only negative Im) may cause conflicts, reducing the diversity of gradients hence relies on reducing negative 9m). We consider to reduce negative (gn:9m) 9m.) by two ways: 1).iiiimize the representation inner product of negative pairs, which pushes the inner product to be negative or zero (for positive representations); 2).optimize the predictions to decrease the probability of flipping the sign. In this sense, decreasing the representation similarity of negative pairs might help with both ways. In addition, according Fig. 2b x S3 gets larger prediction similarity than So due to the predictions put most probability mass on both classes of a pair, which indicates decreasing the similarity of predictions may decrease the probability of flipping the sign. Hence, we include logits in the representations. We verify this idea by training two binary classifiers for two groups of MNIST classes ((0, 1) and {7, 9)). The classifiers have two hidden layers each with 100 hidden units and ReLU activations. We randomly chose 100 test samples from each group to compute the pairwise cosine similarities. Representations are obtained by concatenating the output of all layers (including logits) of the neural network, gradients are computed by all parameters of the model. We display the similarities in Figs. and 33b The correlation coefficients between the gradient and representation similarities of negative pairs are -0.86 and -0.85, which of positive pairs are 0.71 and 0.79. In all cases, the similarities of representations show strong correlations with the similarities of gradients. The classifier for class 0 and gets smaller representation similarities and much less negative gradient similarities for negative pairs (blue dots) and it also gains a higher accuracy than the other classifier (99.95% vs. 96.25%), which illustrates the potential of reducing the gradient diversity by decreasing the representation similarity of negative pairs.
2.2 CONNECTING DEEP METRIC LEARNING TO CONTINUAL LEARNING
Reducing the representation similarity between classes shares the same concept as learning larger margins which has been an active research area for a few decades. For example, Kernel Fisher Discriminant analysis (KFD) MMika et al. 1999) and distance metric learning (Weinberger et al.; 2006) aim learn kernels that can obtain larger margins in an implicit representation space, whereas Deep Metric Learning (DML) (Kaya& & Bilge 2019: Roth et al. 2020) leverages deep neural networks to learn embeddings that maximize margins in an explicit representation space. In this sense, DMI. has the potential to help with reducing the diversity of gradients in continual learning.
However, the usual concepts in DML may not entirely be appropriate for continual learning, as they also aim in learning compact representations within classes (Schroff et al.) 2015 Wang et al.) 2017; Deng et al., 2019). In continual learning, the unused information for the current task might be important for future task, e.g. in the experiments of Fig, I the y-dimension is not useful for task but useful for task 2. It indicates that learning compact representations in a current task might omit important dimensions in the representation space for future task. In this case, even if
5

---

Under review as a conference paper at ICLR 2021
Table 2: Demonstration of performance degrading in continual learning by compact representations. We test the first two tasks of split-MNIST and split-Fashion MNIST by training a MLP (2 hidden layers with 100 units per layer and ReLU actiavtions) with and without L1 regularization. The memory is formed by 50 samples that are randomly chosen. Representations are outputs of hidden layers. We identify important dimensions of the representation space by selecting the hidden units that have mean activation larger than over all learned classes.
L1
Avg. Accuracy (t=) Avg. Accuracy (1=2)
#. Important dimensions
99.9
94.5
39
S-MNIST
99.7
81.1
6
97.6
88.2
55
S-Fashion
96.4
76.2
10
store diverse samples into the memory, the learned representations may be difficult to generalize on future tasks as the omitted dimensions can only be relearned by using limited samples in the memory. We demonstrate this by training a model with and without LI regulariztion on the first two tasks of split-MNIST and split-Fashion MNIST. The results shown in Tab. 2 We see that with L1 regularization the model learns much more compact representations and gives a similar performance on task 1 but much worse performance on task 2 comparing without LI regularization. The results suggest that continual learning shares the interests of maximizing margins in DML but prefers less compact representation space to preserve necessary information for future tasks. We suggest an opposite way regarding the within-class compactness: minimizing the similarities within the same class for obtaining less compact representation space. Roth et al. (2020) proposed a p-spectrum metric to measure the information entropy contained in the representation space (details are provided in Appx. D) and introduced a p-regularization method to restrain over twer-compression of representations. The p- reggularization method randomly replaces negative pairs by positive pairs with pre-selected probability Pp- Nevertheless, switching pairs is inefficient and may be detrimental to the performance in an online setting because some negative pairs may never be learned in this way. Thus, propose different way to restrain the compression of representations which will be introduced in the following.
DISCRIMINATIVE REPRESENTATION Loss
Based our findings in the above section, propose auxiliary objective Discriminative Repre- sentation Loss (DRL) for classification tasks in continual learning, which is straightforward, robust, and efficient. Instead of explicitly re- re-projecting gradients during training process. DRL helps with decreasing gradient diversity by optimizing the representations. As defined in Eq. 2 DRL consists of two parts: one is for minimizing the similarities of representations from different classes (Lst) which can reduce the diversity of gradients from different classes, the other is for minimizing the sim- ilarities of representations from a same class (Lwr) which helps preserve discriminative information for future tasks in continual learning.
min CDRL = min( Cst - aLwi), a 0,
B
Cbt
hz),
Cus
(hi,hg):
(2)
No
Nusi
J
where & denotes the parameters of the model, B is training batch size. No. N are the number of negative and positive pairs, respectively. a is hyperparameter controlling the strength of Cwi h; is the representation of wt, y/ is the label of x,. The final loss function combines the commonly used softmax cross entropy loss for classification tasks (L. with DRL (L as shown in Eq. 3
- C+ XCDRL. X 0,
(3)
where À is hyperparameter controlling the strength of CDRL. which is larger for increased resistance to forgetting, and smaller for greater elasticity. We provide experimental results to verify the effects of DRL. and ablation study on Lis and Luni (Tab. 7 in Appx. according which Lae and Leoi

---

Under review as a conference paper at ICLR 2021
have shown effectiveness on improving forgetting and p-spectrum, respectively. We will show the correlation between p-spectrum and the model performance in Sec.
The computational complexity of DRL is O(B² where B is training batch size, H is the dimension of representations. B is small (10 or 20 in our experiments) and commonly H < W, where W is the number of network parameters. In comparison, the computational complexity of A-GEM and GSS-greedy are OB,,W) and O( W), respectively, where B, is the reference batch size in A-GEM and Bm is the memory batch size in GSS-greedy. The computational complexity discussed here is additional to the cost of common backpropagation. We compare the training time of all methods on MNIST tasks in Tab. 9 in Appx. H. which shows the representation baased methods require much lower computational cost than gradient- based approaches.
ONLINE MEMORY UPDATE AND BALANCED EXPERIENCE REPLAY
We follow the online setting of continual learning as was done for other gradient- gradient-based approaches with episodic memories (Lopez-Paz & Ranzato 2017 Chaudhry et al. 2019a Aljundi et al., 2019). in which the model only trained with one epoch on the training data.
We update the episodic memories by the basic ring buffer strategy: keep the last n. samples of class c in the memory buffer, where ne is the memory size of a seen class c. We have deployed the episodic memories with a fixed size, implying a fixed budget for the memory cost. Further, we maintain uniform distribution over all seen classes in the memory. The buffer may not be evenly allocated to each class before enough samples are acquired for newly arriving classes. We show pseudo-code of the memory update strategy in Alg. iin Appx. for clearer explanation. For innsemnnceemntal learning, this strategy can work without knowing task boundaries. Since DRL and methods of DML depend on the pairwise similarities of samples, we would prefer the training batch to include as wide a variety of different classes as possible to obtain sufficient discriminative information. Hence, we adjust the Experience Replay (ER) strategy (Chaudhry et al. 2019b) for the needs of such methods. The idea is to uniformly sample from seen classes in the memory buffer to form a training batch, so that this batch can contain as many seen classes possible. Moreover, we ensure the training batch includes at least one positive pair of each selected class (minimum 2 samples in each class) to enable the parts computed by positive pairs in the loss. In addition, we also ensure the training batch includes at least one class from the current task. We call this Balanced Experience Replay (BER). The pseudo code is in Alg. 2 of Appx. B Note that we update the memory and form the training batch based on the task ID instead of class ID for instance-inerement incremental tasks (c.g. permuted MNIST tasks), as in this case each task always includes the same set of classes.
5 EXPERIMENTS
In this section we evaluate our methods multiple benchmark tasks by comparing with several baseline methods in the setting of online continual learning.
Benchmark tasks: We have conducted experiments on the following benchmark tasks: Permuted MNIST (10 tasks and each task includes the same 10 classes with different permutation of features), Split MNIST, Split mashion-MNIST and Split CIFAR-1O (all three having 5 tasks with two classes in each task), Split CIFAR- 100 (10 tasks with 10 classes in each task), Split Tinylmageeer (20 tasks with 10 classes in each task). All split tasks include disjoint classes. For tasks of MNIST (LeCun et al. 2010) and Fashion- MNISS (iaao et al. 2017). the training size is 1000 samples per task, for CIFAR- 10 (Krizhevsky et al. 2009) the training size is 3000 per task, for CIFAR- 100 and TinylmageNet (Le) & Yang. 2015) it is 5000 per task. N.B.: We use single-head (shared output) models in all of our experiments, meaning that we do not require a task identifier at testing time. Such settings are more difficult for continual learning but more practical in real applications.
Baselines: We compare our methods with: two gradient- gradient-based approaches (A-GEM (Chaudhry et al.) (2019a) and GSS-greedy (Aljundi et al. 2019)), two standalone experience replay methods (ER (Chaudhry et al. 2019b) and BER). two SOTA methods of DML (Mulrisimillarity (Wang et al. 2019) and R-Margin (Roth et al. 20200). We also trained a single task over all classes with one epoch for all benchmarks which performance can be viewed as a upper bound of each benchmark. N.B.: We deploy the losses of Multisimilarity and R- R-Margin as auxiliary objectives as the same as DRL.
7

---

Under review as a conference paper at ICLR 2021
because using standalone such losses causes difficulties of convergence in our experimental settings. We provide the definitions of these two losses in Appx. D
Performance measures: We use the Average accuracy, Average forgetting. Average intransigence to evaluate the performance of all methods, the definition of these measures are provided in Appx. C
Experimental settings: We use the vanilla SGD optimizer for all experiments without any scheduling. For tasks on MNIST and Mishion-MNIST, we use a MLP with two hidden layers and ReLÚ activations, and each layer has 100 hidden units. For tasks on CIFAR datasets and TinyImageNe, same reduced Resnet18 as used in Chaudhry et al. (2019a). All networks are trained from scratch without regularization scheme. For the MIP,, representations are the concatenation of outputs of all layers including logits: for reduced Resnet18, representations are the concatenation of the imput of the final linear layer and output logits. We concatenate outputs of all layers as we consider they behave like different levels of representation, and when higher layers (layers closer to the input) generate more discriminative representations it would be easier for lower layers to learn more discriminative representations as well. This method also improves the performance of MLPs. For reduced R we found that including outputs of all hidden layers performs almost the same as only including the final representations, so we just use the final layer for lower computational cost. We deploy BER as the replay strategy for DRL, Multisimilarity, and R-Margin. The memory size for tasks on MNIST and Fashion- -NIST is 300 samples. For tasks CIFAR-10 10 and CIFAR- 100 the memory size is 2000 and 5000 samples, respectively. For T'inIImageNa it is also 5000 samples. The standard deviation shown in all results evaluated over 10 runs with different random seeds. We use 10% of training set as validation set for choosing hyperparameters by cross validation. More details of experimental settings and hyperparameters are given in Appx. [
use the
Tabs. 3 to 5 give the averaged accuracy, forgetting, and intransigence of all methods on all benchmark tasks, respectively. As we can see, the forgetting and intransigence often conflict with each other which is the most common phenomenon in continual learning. Our method DRL is able to get better trade-off between them and thus outperforms other methods over most benchmark tasks in terms of average accuracy. This could be because DRL facilitates getting a good intransigence and p-spectrum by Luii and a good forgetting by Las. In DRL the two terms are complementary to each other and combining them brings benefits on both sides (an ablation study the two terms are provide in Appx. According to Tabs. 4 [)) Multisimilarity got better avg. intransigence and similar avg. forgetting on C1 compared with DRL which indicates Multisimilarity learns better representations to generalize on new classes in this case. Roth et.al. (2020) also suggests Multisimilarity is a very strong baseline in deep metric learning which outperforms the proposed R-Margin on several datasets. And we use the hyperparameters of Multisimilarity recommended in Roth et.al. (2020) which generally perform well on multiple complex datasets. TinylmageNet gets much worse performance than other benchmarks because it has more classes (200), a longer task sequence (20 tasks), larger feature space (64 x 64 x 3), and the accuracy of the single task on is just about 17.8%. According to Tab. the longer task sequence, more classes, and larger feature space all increase the gap between the performance of the single task and continual learning.
As shown in Tab. the rho-spectrum shows high correlation to average accuracy on most benchmarks since it may help with learning new decision boundaries across tasks. Split MNIST has shown a low correlation between the p-spectrum and avg. accuracy due to the p-spectrum highly correlates with the avg. intransigence and consequently affect the avg. forgetting in an opposite direction so that causes cancellation of effects on avg. accuracy. In addition, we found that GSS often obtains smaller o than other methods without getting a better performance. In general, the p-spectrum is the more informative. However, it may be
smaller the better because it indicates the representations detrimental to the performance when p is too small as the learned representations are too noisy. DRL is more robust to this issue because p keeps relatively stable when a is larger than a certain value as shown in Fig. 4c in Appx.
CONCLUSION
The two fundamental problems of continual learning with small episodic memories are: (1) how to make the best use of episodic memories; and (ii) how to construct most representative episodic memories. Gradient-basee approaches have shown that the diversity of gradients computed on data from different tasks is key to generalization over these tasks. In this paper we demonstrate that the
8

---

Under review as a conference paper at ICLR 2021
Table 3: Average accuracy (in %), the bold font indicates the best performance on this criterion
P-MNIST
S-MNIST
Fashion
CIFARIO
CIFAR100
TinylmageNet
DRL
78.7 + 0.4
88.2
0.6
78.2 + 0.4
46.1
17.1
0.1
6.5 0.4
BER
75.8
76.9 8.7
0.6
4.1
15.3
+5. 0.6
5.7 + 0.3
ER
76.1
0.3
8.8
1.2
4
1.85
6.3 0.2
A-GEM
75.9
8.8
2.56
33.1
9.3
1.1
GSS
0.31
1.8
0.9
4..0
1.
0.2
Multisim
0.2
0.6
0.5
4.8
1..
0.6
R-Margin
0.4
86.0
0.6
4.30
1..9
6.2
Single
82.3 0.2
91.2 0.3
0.5
78.
2.3
17.8 1.880.. 0.4
Table Average forgetting (in %), the bold font indicates the best performance on this criterion
P-MNIST
S-MNIST
Fashion
CIFARIO
CIFAR100
TinyImageNet
DRL BER
6.0
8.4
16.7 = 1.5
32.2 5.0
20.8
9.4 7.7
7.1
17.4 11.4
4.1
0.3
0.4
ER
0.4
1..6
1.7
48.
0.5
3.6
A-GEM
1.4
1..7
46.1
34.7
18.2
1.68+
GSS
0.2
17.9
2.2
10.
0.7
0.7
Multisim
0.3
0.6
18.3
331
25.9
14.2
R-Margin
0.2
1.4
14.0 1 2.0 3.56
1.
17.1 0.4
Table 5: Average intransigence (in %), the bold font indicates the best performance this criterion
P-MNIST
S-MNIST
Fashion
CIFAR10
CIFAR100
TinylmageNet
DRL BER
2.0
0.3
6.9 1.0
12.1
13.9 +.2
25.0 7.5
0.2
0.2
1.2
1.2
1.5
28.3 + 0.5
ER
2.5 0.2
2.7 0.6
2.7
2.7 0.4
A-GEM
0.5
3.3
1.3
2.17
2.0
24.1 0.8
GSS
0.2
0.9
2.
3..7
2.4
38.6 0.1
Multisim
2.6
1.3
1.0
7.1
1.2
20.4 0.7
R-Margin
0.2
0.3
5.9 1.7
101
1
Table 6: Correlation between model performance and p-spectrum on all benchmark tasks
Coefficient P-MNIST S-MNIST Split Fashion CIFARIO CIFARI00 TinyImageNet
Avg. Acc. -0.8379
0.0461
-0.5553
-0.7689
-0.7103
-0.1820
Avg. Forg. 0.2616
-0.3879
0.4331
0.1005
0.0028
0.1868
Avg. Intran.
0.4659
0.7206
-0.0978
0.2463
0.2229
-0.2377
most diverse gradients are from samples that close to class boundaries. We formally connect the diversity of gradients to discriminativeness of representations, which leads to an alternative way to reduce the diversity of gradients in continual learning. We subsequently exploit ideas from DML for learning more discriminative representations, and furthermore identify the shared and different interests between continual learning and DML. continual learning we would prefer larger margins between classes as the same as in DML. The difference is that continual learning requires less compact representations for better compatibility with future tasks. Based on these findings, we provide a simple yet efficient approach to solving the first problem listed above. Our findings also shed light on the second problem: would be better for the memorized samples to preserve as much variance as possible. In most of our experiments, randomly chosen samples outperform those selected by gradient diversity (GSS) due to the limit on memory size in practice. It could be helpful to select memorized samples by separately considering the representativeness of inter- and intra-class samples, i.c., those representing margins and edges. We will leave this for future work.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in Neural Information Processing Systems, pp. 11816- 11825, 2019.
Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings the European Conference on Computer Vision (ECCV), pp. 532-547, 2018.
Arslan Chaudhry, Mare' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-GEM. In International Conference on Learning Representations, 2019a.
URL httpts://pppneeeeee htt.eeneeerneeeeneeeeoooooomooe-
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puncet K Dokamia, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: 1902. 10486, 2019b.
Yu Chen, Tom Diethe, and Neil Lawrence. Facilitating bayesian continual learning by natural gradients and stein gradients. Continual Learning Workshop of 32nd Conference Neural Information Processing Systems (NeurIPS 2018), 2018
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4690-4699, 2019.
Tom Diethe, Tom Borchert, Eno Thereska, Borja de Balle Pigem, and Neil Lawrence. Continual learning in practice. In Continual Learning Workshop of 32nd Converence on Neural Information Processing Systems (NeurIPS 2018), 2018.
Mchrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pp. 3762-3773,
PMLR, 2020.
Ching- Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems, pp. 13647-13657, 2019.
Mahmut Kaya and Hasan Şakir Bilge. Deep metric learning: A survey. Symmetry, 11(9); 2019.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinskar et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, pp. 201611835, 2017.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N. 2015.
Yann LeCun, Corinna Cortes, and Christopher JC Burges. MNIST handwritten digit database. AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist. 2, 2010.
Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia Diaz-Rodríguez. Continual learning for robotics. arXiv preprint arXiv: 00182 2019.
Jinlong Liu, Yunzhi Bai, Guoqing Jiang, Ting Chen, and Huayan Wang. Understanding why neural networks generalize well through GSNR of parameters. International Conference on Learning Representations, 2020. URL
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
10

---

Under review as a conference paper at ICLR 2021
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Sebastian Mika, Gunnar Ratsch, Jason Weston, Bernhard Scholkopf, and Robert Mullers. Fisher discriminant analysis with kernels. In Neural networks for signal processing IX: Proceedings of the 1999 IEEE signal processing society workshop (cat. no. 98th8468), pp- 41-48. Ieee, 1999.
Cuong V Nguyen, Yingzhen Li, Thang Bui, and Richard E Turner. Variational continual learning. In International Conference on Learning Representations, 2018.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference Learning Representations, 2019. URL /attpss//oppereele.r net http://ouuriddl3ll1111e
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. In Advances in Neural Information Processing Systems, pp. 348-358, 2019.
Karsten Roth, Timo Milbich, Samarth Sinha, Prateek Gupta, Bjoern Ommer, and Joseph Paul Cohen. Revisiting training strategies and generalization performance in deep metric learning. In Proceedings of the 37th International Conference Machine Learning, 2020.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 815-823, 2015.
Jonathan Schwarz, Jelena Luketina, Wojeiech Czarnecki, Agnieszka Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable frame- work for continual learning. arXiv preprint 1rXiv:1805.06 06370, 2018.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Huangshi Tian, Minchen Yu, and Wei Wang. Continuum: platform for cost-aware, low-latency continual learning. In Proceedings of the ACM Symposium on Cloud Computing, pp. 26 40,40, 2018.
Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin. Deep metric learning with angular loss. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2593-2601, 2017.
Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R Scott. Multi-similarity loss with general pair weighting for deep metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5022-5030, 2019.
Kilian Weinberger, John Blitzer, and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. In Advances in neural information processing systems, pp. 1473- 1480, 2006.
Chao- Yuan Wu, R Manmatha, Alexander Smola, and Philipp Krahenbuhl. Sampling matters in deep embedding learning. In Proceedings of the IEEE International Conference Computer Vision, pp. 2840-2848, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, 2017.
11

---

Under review as a conference paper at ICLR 2021
A
PROOF OF THEOREMS
Notations: Let L represent the softmax cross entropy loss, W E RDDK is the weight matrix of the linear model, and x, RD denotes the input data, Yn IR is one-hot vector that denotes the label of Xn. D is the dimension of representations, K is the number of classes. Let p.. sof (mmrr(,..) where On - W X,, the gradient In. (x W). Xn: Xm are two different samples when
Lemma 1. Let =pr. Yn. we have (gn, 9 (X,, (m)(Ens Em),
Proof. Let E 0[(xn, Nn: W) w) by the chain rule, we have:
(g 9m) (Xa, Xm (C en),
By the definition of L, we can find:
f, Pa -n:
(4)
L
Theorem 1. Suppose Yn # ym let on denote the class index of x, (i.e. Yn.c, =1, = 0, Cn). Let a a | 11pm.l1? Pn.cm + Pm.cn and - ||p -pm Pm.1.. then:
Pr (sign(( (gn. 9m.)
xm))) = Pr 8 a),
Proof. According Lemma and yn + yms we have
(L km) (pri Pm. Da.cm Pmncn
And
1 1a(-00 - 6)
(pxx Pm)
=
1|pm//lll 112 ||p,. - pm 113) 7
2
which gives (L,, (
} (a 8) B. When 2,3 > 8, we must have (Ln Lm) < O. According
Lemma we prove this theorem.
Theorem 2. Suppose yn Yms when (gn, Im) O, have: sign((gn, gm)) = sign( (Xn, Xm)),
Proof. Because Σ Pa.k 1, Pu.k 2 0, Vk, and On = Cy = C.
(L km)
Σ PnnKPPonk + (pu,e 1) - 1) kefk
(5)
According to Lemma 1, we prove the theorem.
B ALGORITHMS OF ONLINE MEMORY UPDATE
We provide the details online ring buffer update and Balanced Experience Replay (BER) Algs. 3 We directly load new data batches into the memory buffer without separate buffer for the current task. The memory buffer works like a sliding window for each class in the data stream and we draw training batches from the memory buffer instead of directly from the data stream. In this case, one sample may not be seen only once as long as stays in the memory buffer. This strategy is more efficient use of the memory when [ Ne. where [ is the loading batch size of the data stream (i.e. the number of new samples added into the memory buffer at each iteration), we set [B] to 1 in all experiments (see Appx. [oo discussion of this).
12

---

Under review as a conference paper at ICLR 2021
Algorithm 2 Balanced Experience Replay
Algorithm 1 Ring Buffer Update with Fixed Buffer Size
Input: M memory buffer, C the set of classes in M, B training batch size, Q model parameters, Le loss function, B, cur- rent data batch from the data stream, C, the set of classes in B,, K memory buffer size.
Input: B, current data batch of the data
stream, C. - the set classes in B,, M mem- ory buffer, C the set of classes in M, K memory buffer size.
for in do
M +- MemoryUpdate(B, M,C,
Get Bi, samples of class e in Br, Mr samples of class in M, if in C then
Re, C., C, +- ClassSeleciion(C B) Btrain = 0
for e in C, do
M. -Me Be
if in C, then
else
M. Be. CuCu(ct
m. = ne +
else
end if
end for
mc ne
end if
R
|M| + | K
Get M. samples of class c M,
while R > do
B. me M. < sample me samples from M. Braain B U B
d' arg max, [Mel
remove the first sample in Ma, == R-1 end while return M
end for
- Optimizer(Brramn O Le)
Algorithm 3 Class Selection for BER
Input: C, the set of classes in current data batch B,, the set of classes in M, B training batch size, mp minimum number of positive pairs of each selected class (m, E (0, 1)). Birain = n2 = B/ re = mod [C].
if ne 1 my 0 then
C, 1 C < sample r. classes from all seen classes without replacement.
C, =C
else
C, ne 2, n1, - [B/2] [Ce). 4 we ensure the training batch include samples from the
current task.
C, 2 (C C₂) < sample n, classes from all seen classes except classes in C..
C. = C.
if B mod
> then
C, 2 C. < sample one class in C, to have an extra sample.
end if
end
Return: ne. C.. Cr
C DEFINITION OF PERFORMANCE MEASURES
We the following measures evaluate the performance of all methods:
Average accuracy, which is evaluated after learning all tasks: ã Ei dr,s, where is the index of the latest task, 44,; is the accuracy of task after learning task f.
Average forgetting (Chaudhry al. 2018), which measures average accuracy drop of all tasks after max (a,, ar.i)-
learning the whole task sequence: ft
Average intransigence (Chaudhry et al. 2018), which measures the inability of model learning new tasks: I, 1 Σ at a, where is the accuracy of task at time We use the best accuracy among all compared models as at instead of the accuracy obtained by an extra model that is solely trained task i.
13

---

Under review as a conference paper at ICLR 2021
D RELATED METHODS FROM DML
p-spectrum metric (Roth et al. 2020 P = [IL(A[|SSxx which is proposed to measure the information entropy contained in the representation space. The p-spectrum computes the KL- divergence between a discrete uniform distribution and the spectrum data representations Sex* where S is normalized and sorted singular values of ((X), & denotes the representation extractor (e.g. a neural network) and X' is input data samples. Lower values of p indicate higher variance of the representations and hence
information entropy retained.
Multisimilarity Wang et al. 2019): we adopt the loss function of Multisimilarity as an auxiliary objective in classfication tasks of continual learning, the batch mining process is omitted because use labels for choosing positive and negative pairs. So the loss function is C C XCmulti and:
Emulti
log 1 +
Σ
exp(-a(s_(h,,h)
-))1
B
ifi.ys 14
(6)
log [1 + Σ exp (3(sc(h;,h)) - >>>1
3
where sc(1-) is cosine similarity, 3, are hyperparameters. In all of our experiments we set 2. 7 7 0.5 the same in Roth al. (2020).
R-Margin (Roth et al. 2020): we similarly deploy R-Margin for continual learning as the above, which uses the Margin loss (Wu
2017) with the regularization (Roth et al. 2020) as introduced
in Sec. 2.2 So the loss function is C C Xmmariins and:
B Emargin = ΣΣ f=1
B
b, h1) 3) I - (d(h,, h2) - B)
(7)
where -(. is Euclidean distance, , is trainable variable and is hyperparameter. We follow setting RRoth et al. (2020): = 0.2, the initialization / is 0.6. We set pp = 0.2 regularization.
E ABLATION STUDY ON DRL
We verify the effects of LDRL by training a model with/without CDRL on Split-MNIST tasks: Fig. shows that LDRL notably reduces the similarities of representations from different classes while making representations from a same class less similar; Fig- 4b shows the analogous effect on gradients from different classes and a same class. Fig. demonnttrasss increasing decrease p-spectrum to low-value level, where lower values of indicate higher variance of the representations and hence more information entropy retained.
effectively
Tab. 7 provides the results of an ablation study on the effects of the two terms in DRL. In general, Ls gets better performance terms of forgetting, Cws gets better performance in terms of intransigence and lower p-spectrum, and both of them show improvements on BER (without any regularization terms). Overall, combining the two terms obtains better performance on forgetting than standalone Lat and keeps the advantage on intransigence that brought by Lui- It indicates preventing over-compact representations while maximizing margins can improve the learned representations that are casier for generalization over previous and new tasks. In addition, we found that using standalone Lit we can only use a smaller otherwise the gradients will explode, and using Lwi together can stablize the gradients.
We notice that the lower p-spectrum does not necessarily lead to a higher accuracy as it's correlation coefficients with accuracy depends datasets and is usually larger than -1.
F COMPARING DIFFERENT MEMORY SIZES
Fig. 5 compares average accuracy of DRL+BER on MNIST tasks with different memory sizes. appears the fixed memory size is more efficient than the incremental memory size. For example, the
14

---

Under review as a conference paper at ICLR 2021
cloes "5" samre closs
same Elos 8
Groubblte
10
0.6
-650 -8.2 3.83
L.0
gradlents
(a) Similarities of representations with and without LDRL.
(b) Similarities of gradients with (c) Relation between a and and without Lyar.
spectrum.
Figure 4: Effects of LDRS. on reducing diveristy gradients and p-spectrum. (a) and (b) display distributions of similarities of representations and gradients. s and Sn denote similarities of representations with and without LDRL. respectively, sDR and 8g denote similarities of gradients with and without LDRL, respectively. (c) demonstrates increasing a in LDRL can reduce p effectively.
Table 7: Comparing the performance with or without the regularization terms (Che, Lux)) in DRL. All criteria are in % except p-spectrum. The bold font inditactes the best performance of criterion, as p-spectrum is not a performance measurement so we do not put bold font on it.
BER
DRL
with none
with Lss
with Lusi
with both
Avg. Accuracy
75.8 + 0.3
07.0
77.8
78.7 0.4
Avg. Forgetting
7.1
0.3
6.6
6.0
P-MNIST
Avg. Intransigence
0.2
3.2
2.3
2.0
p-spectrum
1.01
0.02
0.65 0.02
0.71
Avg. Accuracy
0.7
8.0
8.
88.2 = 0.6
Avg. Forgetting
1..4
10.0
8.4 10.6
S-MNIST
Avg. Intransigence
2.6
2.2
2.2
2.2
p-spectrum
0.16
1.2
1.05 1.07
0.03
Avg. Accuracy
7.6
7.0
78.2 1.1
0.4
Avg. Forgetting
1.
17.
2.24
16.7 1.5
Fashion
Avg. Intransigence
1.2
1.1
6.6 1.2
6.9 1.0
p-spectrum
0.27
1.55 0.09
1.22 + 0.06
1.21 + 0.04
fixed memory size (M 300) getting very similar average accuracy with memory = 50 per class in Disjoint MNIST while it takes less cost of the memory after task 3. Meanwhile, the fixed memory size (M = 300) gets much better performance than M = 50 per task in most tasks of Permuted MNIST and it takes less cost of the memory after task 6. Since the setting of fixed memory size takes larger memory buffer early tasks, the results indicate better generalization of early tasks can benefit later tasks, especially for more homogeneous tasks such as Permuted MNIST. The results also align with findings about Reservoir sampling (which also has fixed buffer size) in Chaudhry et al. (2019b) and we also believe hybrid memory strategy obtain better performance suggested Chaudhry et al. (019b)).
G COMPARING DIFFERENT REPLAY STRATEGY
We compare DRL with different memory replay strategies in Tab. to show DRL has general improvement based on the applied replay strategy.
H COMPARING TRAINING TIME
Tab. 9 compares the training time of MNIST tasks. All representation basee methods are much faster than gradient-based methods and close the replay-based methods.
15

---

Under review as a conference paper at ICLR 2021
M=:00
M-000
0.85
M-10thask
K-
M-50ltask
0.975
0.84
M=30itask
Avg. Accuracy
0.950
0.83
0.82
0.900
0.81
0.877
0.80
0.850
0.79
0.800
0.70
1
2
3
10
(a) Permuted MNIST
(b) Disjoint MNIST
Figure 5: Average accuracy of DRL+BER with different memory sizes. The x axis is the index of lasks, the shaded area is plotted by standard deviation of 5 runs.
Table 8: Comparing DRL with different memory replay strategies, all criteria are in %.
DRL+BER
DRL+ER
BER
ER
P-MNIST
78.7 - 0.4
76.9
0.5
75.8
0.3
76.1 = 0.3
Avg. Accuracy
S-MNIST
0.6
8.8
8.74
84.0 7.2
Fashion
7.
7..
7.6
P-MNIST
0.3
0.6
0.1
0.3
Avg. Forgetting
S-MNIST
0.9
13.4
11.4
15.6
Fashion
1.5
22.7
1.
2.7
P-MNIST 1.2 0.2
2.9 6.9=00 0.2
3.1
+ 0.2
1.6
Avg. Intransigence
S-MNIST 2.6 0.3
1.2 0.3
1.9 0.2
0.9
Fashion 6.9 1.0
5.4 0.4
7.7 1.2
4.1
Table 9: Training time (in seconds) of the whole task sequence of MNIST tasks, which have been tested on laptop with an 8-core Intel CPU and 32G RAM.
DRL
BER
ER
A-GEM
GSS
Multisim
R-Margin
Permuted
12.48 + 0.16
11.17 = 0.18
10.38
0.05
28.0 + 0.09
33.98 + 0.6
12.91 + 0.13
13.45 = 0.14 6.29 1: 0.43
Split
5.6 1 0.14
5.29 1 0.08
5.25 1: 0.02
13.41 1 0.47
19.07 1 1.31
5.89 1 0.09
I HYPER HYPER-PARAMETIE PARAMETERS IN EXPERIMENTS
To make a fair comparison of all methods, we use following settings: i) The configurations of GSS-greedy are as suggested in Aljundi et al. (2019), with batch size set to 10 and each batch receives 10 iterations. ti) For the other methods, we use the ring buffer memory as described in Alg. T the loading batch size is set to 1, following with one iteration, the training batch size is provided in Tab. TO More hyperparameters are given in Tab. aQ as well.
In the setting of limited training data in online continual learning, we either use small batch size or iterate on one batch several times to obtain necessary steps for gradient optimization. We chose a small batch size with one iteration instead of larger batch size with multiple iterations because by our memory update strategy (Alg. 1 it achieves similar performance with fewer hyperparameters. Since GSS- GSS-greedy has different strategy for updating memories, we leave it at its default settings.
Regarding the two terms in DRL, a larger weight on Lws is for less compact representations within classes, but too dispersed representation space may include too much noise. For datasets that present more difficulty in learning compact representations, we would prefer a smaller weight on Lwi. we therefore set smaller for CIFAR datasets in our experiments. A larger weight Lat is more resistant to forgetting but may be less capable of transferring to new task, for datasets that are less compatible between tasks smaller weight on Li. would be preferred, as we set the largest on Permuted MNIST and the smallest À on CIFAR- 100 in our experiments.
16

---

Under review as a conference paper at ICLR 2021
Table 10: Hyperparameters of all experiments
P-MNIST S-MNIST Fashion
CIFAR-10
CIFAR- 100 TinylmageNet
training batch size
20
20
20
10
10
20
learning rate
0.02
0.2
0.2
0.1
0.2
0.2
ref batch size
256
256
256
256
1500
1500
(A-GEM)
of DRL
2
2
2
0.1
0.1
0.1
À of DRL
1 x 10
5 X 10
5 x 10
2 2 10
x 10
x 10
À of Multisim
5
1
2
0.1
0.1
À of R-Margin
2
x 10-5
1 1
10
x 10-3
1 10-4
2 10
2 x 10
Table 11: The grid search range of Hyperparameters of all experiments except Tiny use the same hyperparameters of CIFAR-100 to TinylmageNet except increasing the training batch size 20.
The grid-search range
training batch size
[10, 20, 50, 100] [0.001, 0.01, 0.02, 0.1, 0.2]
learning rate
ref batch size
[128, 256, 512, 1000. 1500, 2000]
(A-GEM)
of DRI.
[0.1, 0.2, 0.5, 1,
A of DRL
[1 10-5, 10-5 5 10-5
10=
10 5 10-4 1 10-3 x 1 10-1
Multisim
6, 1,5,, 0.1
of R-Margin
[1 10- 1
xx 1x 10 1x 10-1
17


Reviews:
[{"review_id": "vZfC9XpeBjM", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a novel way of making full use of compact episodic memory to alleviate catastrophic forgetting in continual learning. This is done by adding the proposed discriminative representation loss to regularize the gradients produced by new samples. Authors gave insightful analysis on the influence of gradient diversity to the performance of continual learning, and proposed a regularization that connects metric learning and continual learning. However, there are still some issues to be addressed as below.\n\n1. Authors do not explain how to determine weights of the two terms in the proposed discriminative representation loss to achieve good balance between discrmination capability on the current task and generalization on future task.\n\n2. No experiment on the influence of memory size and batch size on the proposed method. These are important hyperparameters for the proposed method. So there should be experiment inspecting its influnece on the performance.\n\n3. There is also no experiment on the influence of different memory update rule on the proposed method. It is not clear how well the proposed method is robust to different memory update rules.\n\n4. Experiments on more diverse datasets should be done to prove its effectiveness, such as TinyImageNet, and MIT Scenes/Oxford Flowers/UCSD Birds/Stanford Cars.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ljZum_Bhffx", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "############## Summary ##############\n\nThis submission draws a connection between deep representation learning and continual learning. The authors include theoretical and empirical analyses that suggest that the learned model for continual learning should both separate the representations of instances of different classes and separate the representations of instances within a single class. This motivates a new representation-learning-based continual learning method called DRL, which the authors evaluate empirically against various baselines.\n\n############## Strengths ##############\n\n1. The problem studied in this work is interesting. A deeper understanding of what types of representations could enable continual learning could potentially be very impactful to the field.\n2. The paper includes both theoretical and empirical analyses.\n3. The experimental setting is described in detail and includes a comprehensive set of baselines.\n\n############## Weaknesses ##############\n\n1. The theoretical findings seem to be somewhat disconnected from the empirical study in Section 2.1, and the analysis connecting the two is unconvincing. Moreover, given this disconnect, the empirical study should have been much more complete, including various data sets.\n2. The writing and the structure of the paper is quite unclear, leaving the reader confused at times, having to jump back and forth to draw connections between different sections that are not explicitly described in the text.\n3. The experimental evaluation is inconsistent in showing the advantage of their method, and this is not analyzed in any depth.\n\n\n############## Recommendation ##############\n\nUnfortunately, I lean towards recommending the rejection of this paper. While the studied problem is of high relevance, whose answer could be very impactful, I find the theoretical and empirical analyses in this work to be unconvincing, and thus the claims made throughout the paper to be insufficiently founded. The authors use very simple toy experiments and theorems on linear models to motivate their claim that learned representations should be very different across different classes. While using theory from linear models is a common technique for motivating empirical study of deep learning models, a much more comprehensive empirical study would be needed to validate that the theoretical findings extend to the deep learning setting. Moreover, the paper should be substantially revised both to add clarity to the motivation of the problem, and to more carefully analyze the connections between the theoretical and empirical analyses in Section 2.1.\n\n\n############## Arguments ##############\n\nThe biggest reason I lean towards rejecting this paper is that I am unconvinced by the claims made throughout Section 2. \n\nFirst, the section begins by suggesting that instances with similar gradients are those most useful for generalization. This seems to contradict the findings of Aljundi et al., which chooses to keep samples with maximum gradient diversity in the episodic memory. This apparent contradiction should be carefully discussed and analyzed.\n\nThen, the experiment of Figure 1 suggests that: the gradients with the most diverse gradients are those close to the decision boundary, and in order to avoid forgetting previously seen tasks, samples with diverse representations help maintain good decision boundaries as new tasks are learned. The former claim is further studied in the rest of the section, but why does this experiment suggest that we would like to keep samples with diverse gradients? I keep seeing conflicting indications that we should store samples with diverse or similar gradients. On the other hand, the latter claim is not substantiated beyond these toy experiments. It seems to be specific to incremental-class learning, and would probably not hold for other variants of continual learning that don't require the same model to discern between the new class and the previous ones. \n\nIn Theorem 1, the authors claim that points close to the decision boundary are likely to yield negative inner product of their gradients. Figure 4 in the Appendix seeks to illustrate this, and could potentially be very helpful towards understanding this work better, but  1) the figure is missing a y-axis label and 2) the caption is confusing and contradictory between the sub-figure captions and the full figure caption, which makes it hard to take much away from it. Moreover, the claim hinges on the assumption that <x_n, x_m> is likely to be positive if classes are near the decision boundary. This doesn't seem to be addressed in Figure 4, and I don't believe it is necessarily the case in high-dimensional spaces, where points can be very different and still close to the decision boundary. Are the authors assuming that _all_ inner products of the representations are positive, since they use ReLU activations? This should be explicitly clarified.\n\nIn Theorem 2, the authors mention that a deep net can be viewed as a representation extractor, and that ReLU activation would lead to positive inner products for the representations.\n1. First, this seems vacuous, since all inner products (even across classes) would be positive.\n2. Second, the inner product of the gradients _of the final linear layer_ are the ones that are guaranteed to have positive inner products, but this claim says nothing about the gradients of remaining layers in the network. ~~Since recent work [1] has shown that forgetting primarily occurs at the shallow layers of the network, this does not seem to be a helpful theorem.~~ It is still useful, since recent work [1] showed that forgetting primarily occurs at the deepest layers of the network, but this should be stated clearly.\n\nThen, the authors move to a simple experiment using deep nets, which is supposed to help extend the findings of Theorems 1 and 2 to the deep learning case. However, the connections between these two appear to be very weak and poorly described.\n- It is unclear why representations are obtained by concatenating the outputs of all layers, or how this relates to Theorems 1 and 2. Why not use directly the input feature representation, or better yet use only the final hidden representation before the output layer as hinted at in Theorem 2? This choice is not justified.\n- Are the gradients also obtained by concatenating all layers' gradients? This _should_ certainly be the case, since we want to learn about the gradient of the entire model.\n- It seems that in Theorem 1 we care only about the sign of the gradient: positive representation similarity likely leads to negative gradient similarity. In Figure 2, _all_ points have positive representation similarity, so it's unclear how we can relate these two. It seems that we should now look for decreasing gradient similarity as representation similarity increases between different classes, but this is not stated or analyzed. Also, there is no mention of \\beta, \\delta, or \\alpha, all of which could be explicitly computed and used to analyze these results.\n\nOverall, the Theorems seem to give results only about signs and not distances, whereas the deep learning experiments try to extend the analysis with distances. This means that distances are actually far more useful for relating the findings to metric learning. Since the empirical findings are actually different from the theoretical ones, these experiments should be far more extensive to justify the proposed method, including analyses on more complex data sets, and a much clearer description of exactly how the experiments are carried out.\n\nThe take-away from Section 2.1 seems to be that similar representations across different classes lead to conflicting gradients. This is only hinted at in various ways, but I don't believe it is directly stated. Clarifying this would tremendously help lead into Section 2.2. Moreover, it is never explained what the intuition behind this is, which would also be very helpful to increase the reader's ability to follow the paper. I would expect that the intuition is that having similar representations across different tasks would lead the network to somehow confuse the different classes, but I don't believe this is explained.\n\nIn Section 2.2, I find the motivation for learning a diverse set of representations for each class somewhat unconvincing. The experiments of Figure 1 show that the learner suffers forgetting on the previous tasks because it only keeps points with similar representation. However, the only reason why this is a problem is that there exist other points with very dissimilar representations, which will then be ignored by the updated margin. If we learned a representation where all points within a class are similar, then that would not seem to be a problem. While this is somewhat analyzed in the experiments in Table 4 via the rho-spectrum, this analysis is quite incomplete. Moreover, Table 4 shows no correlation between forgetting and rho-spectrum, which is what Figure 1 suggests we should expect. This is not discussed in any way.\n\nTo add to the confusion in the analysis, the experiments in Section 5 use inconsistent definitions of what the representation is, with all layers being used as the representation in some cases (just like in the analysis of Figure 2), but only the final representation and the logits being used in other cases (which, as an aside, violates the claim that the inner products are necessarily positive).\n\nThe empirical results somewhat inconsistently show that the proposed method is better than baselines. However, considering that MNIST-based data sets are usually not very indicative of performance in more complex data sets, I would focus my analysis on the CIFAR-10 and CIFAR-100 experiments. Here, the proposed method was best in one case but not in another. This would require a much more in-depth analysis, but the whole results analysis is limited to a single paragraph. \n\n\n[1] Ramasesh, V. V., Dyer, E., & Raghu, M. (2020). Anatomy of catastrophic forgetting: Hidden representations and task semantics. arXiv preprint arXiv:2007.07400.\n\n############## Additional feedback ##############\n\nThe following points are provided as feedback to hopefully help better shape the submitted manuscript, but did not impact my recommendation in a major way.\n\nAbstract\n- DRL is efficient w.r.t. what?\n    - The intro seems to suggest it's computational efficiency\n    - The first time we actually get to see what this means is at the end of the paper, buried in the last paragraph of Section 5. Since this seems to be such a relevant point (enough to point to it in the Appendix and Introduction), it would be relevant to bring it up earlier and give more details about how it compares to other methods.\n\nIntro\n- I wouldn't consider that methods for growing the model size fall in the same category as regularization-based methods. Their focus isn't preserving knowledge of past tasks' models, but instead adding the current task's knowledge into a separate model.\n- Paragraph 4: \"direction that closest\" --> \"direction that [is] closest\"\n- Why such a detailed summary of gradient-based methods? Push to separate Related Work section.\n- Again it's unclear what the metric for efficiency is. Context seems to suggest it's computation time, but one could also think it's data efficiency or memory capacity (since the former is commonly important in lifelong learning and the latter is mentioned in this section).\n\nSec 2.1\n- Why two different notations for x_n with and without boldface? They both seem to denote a single data point.\n- Theorem 1 is the first time the authors mention negative inner product of the gradients. Before, only the terms \"similar\" and \"diverse\" were used. I suggest making this clearer in the explanation throughout.\n- Deep learning experiments\n    - Is the correlation coefficient between the x axis (representation similarity) and the y axis (gradient similarity)?\n    - It would be helpful to clarify explicitly in the text that in Figure 2 the blue dots should be analyzed under Theorem 1 and the orange dots under Theorem 2.\n    - The final statement about discriminative representations of task 1 vs task 2 doesn't seem to add much value, so my suggestion would be to drop it or clarify how it's useful to the argument of the paper.\n\nSec 2.2\n- The first sentence in second paragraph seems disconnected from what follows. What's the point of it? Where will the unused information be leveraged?\n\nSec 3\n- Is the representation h_i the same as in the experiments of Figure 2, i.e., the concatenated output of all layers?\n    - It is made clear later that the definition of h_i is not consistent throughout the experiments. Here, the authors should clarify this and explain how this definition should be chosen, and why it is not possible to use a consistent definition.\n- Figure 3 is missing a y-axis label.\n- The effects on the similarities within classes seem to be very small in Figure 3. Is this because \\alpha is small?\n\nSec 4\n- The statement that in the incremental class setting the method can work without task boundaries seems vacuous: whenever the learner sees a new class, evidently it corresponds to a new task.\n- This is the first time that task-incremental vs class-incremental settings are introduced, without explaining exactly what they are. The manuscript should be self-contained and so it should give a clear description of these settings if they are relevant to the submission (as they clearly are based on the statements in this section).\n\nSec 5\n- Fashin-MNIST --> Fashion-MNIST\n- Details of the experimental setting are quite comprehensive (+).\n- In addition, We --> In addition, we\n- What is the reason the proposed method achieves a better trade-off between forgetting and intransigence? This is not mentioned at all at any point in the paper up to this point.\n- Why only compare computation time on MNIST and not on remaining benchmarks? Since computational complexity depends on the representation size, it is thus clearly dependent on the choice of representation, which inconsistently varies across benchmarks. Also, what is the theoretical complexity of the baselines? Are there settings where they would, at least theoretically, be faster? This comparison would strengthen the claim that the method is more efficient.\n- I would also like to see an ablative study removing each of the terms in the regularization function in turn. If ER is not implemented using the same sampling strategy as BER, then I'd also like to see the effect of BER training on its own. This would help tease apart the contributions of this work better.\n\nAppendices\n- How are the hyper-parameters of DRL chosen in Table 6?\n\n\n----------- Updates during discussion -----------\n\nI have updated my score from a 4 to a 5 based on the authors' response. My justification is below in my comment to the authors.", "extra_scores": {}, "extra_text": {}}, {"review_id": "flC9ZaLigha", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper analyzes the relationship of gradient diversity to the performance of continual learning systems, the analysis inspires a novel loss which shows some improvement in the continual learning setting.\n\nStrengths:\n\n-The empirical analysis is interesting making connections to Liu 2020 and some effective visualizations. The linear cases is studied in detail and provides intuition for the method\n\n-The proposed loss gives some improvement in larger memory settings\n\n\nWeaknesses:\n\n-Proposed method is too informally motivated by diversity analysis. It’s not clear to the reviewer how the theoretical observation regarding the linear case can be extended to the non-linear case. \n\n-Experimental results are promising but not completely convincing especially with the 2 additional hyperparameters:\n1. The authors use buffer settings much higher than prior works (e.g. [a,b] consider memories of 200,500,1000), this choice is not explained nor lack of comparison to any existing published result\n2. There are several works e.g. [a,b,c] which consider the same setting as this work and have better performance than the baselines shown. For example for CIFAR-10 [a] uses smaller memory size and has higher performance than shown here. \n\n-Cross entropy is already indirectly bounding something similar to the L_bt-L_wi ([e]), so the objective with 2 hyper-parameters seems overly cumbersome.\n\n-Related Work is a bit sparse. Besides the works mentioned above there is several considering metric learning based approaches for continual learning which should be discussed (e.g. [d])\n\nQuestion: \n-Are the hidden representations normalized? \n-The minibatches used are small and thus can have few same class pairs, how do the authors assure that L_wi can be optimized in this case, particularly with say 100 classes as in CIFAR-100. \n\nOverall, I find the work promising. The observations regarding gradient diversity might be the basis of future more effective methods. On the other hand the exact method proposed and results are not completely convincing in their current state. \n\n[a] Aljundi et al “Online Continual Learning with Maximally Interfered Retrieval” \n[b] Ji et al “Automatic recall machines: Internal replay, continual learning and the brain”\n[c] Caccia et al “Online Learned Continual Compression with Adaptive Quantization Modules” \n[d]  Li et al  “Better Knowledge Retention through Metric Learning”\n[e] Boudiaf \"A unifying mutual information view of\nmetric learning: cross-entropy vs. pairwise losses\"\n\n----------------\n\nPost-rebuttal:\nAfter reading the rebuttal I maintain my score. The observations seem promising, the method a bit cumbersome but also interesting, but neither of them are fully fleshed out. The authors should either greatly expand the empirical analysis (in the non-linear setting) of their claims on intra and inter class variability in CL  and/or make the experiments of the DRL method more convincing and varied in scope", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores the connection between diversity of gradients and discriminativeness of representations. Based on the observations, authors propose Discriminative Representation Loss (DRL).

This paper resulted in a lot of discussions and specifically, R5's detailed comments helped the authors improve their paper. Authors did a good job in making significant improvements to the paper based on the reviews, including better connecting the theory and experiments. However, after much discussion it was felt that experiments and analysis still needed improvement, leading to a decision to reject. The authors are encouraged to use the reviewers' post-discussion updates to further improve and submit to a future venue.
Decision: Reject