Better Knowledge Retention through Metric Learning
Authors: Ke Li*
Shichong Peng*
Kailas Vodrahalli*
Jitendra Malik
Venue:  (2020)
Keywords: metric learning
continual learning
catastrophic forgetting

Abstract: In a continual learning setting, new categories may be introduced over time, and an ideal learning system should perform well on both the original categories and the new categories. While deep neural nets have achieved resounding success in the classical setting, they are known to forget about knowledge acquired in prior episodes of learning if the examples encountered in the current episode of learning are drastically different from those encountered in prior episodes. This makes deep neural nets ill-suited to continual learning. In this paper, we propose a new model that can both leverage the expressive power of deep neural nets and is resilient to forgetting when new categories are introduced. We demonstrate an improvement in terms of accuracy on original classes compared to a vanilla deep neural net.

Under review as a conference paper at ICLR 2020
BETTER KNOWLEDGE RETENTION THROUGH METRIC LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
In continual learning setting, categories may be introduced over time, and an ideal learning system should perform well on both the original categories and the new categories. While deep neural nets have achieved resounding success in the classical setting, they are known to forget about knowledge acquired in prior episodes of learning the examples encountered in the current episode of learning are drastically different from those encountered in prior episodes. This makes deep neural nets ill-suited to continual learning. In this paper, we propose a new model that can both leverage the expressive power of deep neural nets and is resilient to forgetting when new categories are introduced. We demonstrate an improvement in terms of accuracy original classes compared to vanilla deep neural net.
INTRODUCTION
Despite the multitude of successes of deep neural networks in recent years, one of the major unsolved issues is their difficulty in adapting new tasks while retaining knowledge acquired from old ones. This problem is what continual learning aims to tackle. To successfully develop a continual learning method, one major limitation that must be overcome is the problem of catastrophic forgetting (McCloskey & Cohen 1989), which describes the phenomenon that a machine learning model finetuned for new task performs poorly on the old task it was originally trained on.
Various continual learning methods make different assumptions about whether the task identities and boundaries are known training time and test time (van de Ven & Tolias 2019:; Hsu et al.) 2018]] Zeno et al.) 2018). In the most restrictive setting, task identities are known at both training and test time and the method only asked to classify among the possible classes within the given task, a setting known as task learning. In domain learning. the assumption that the task identity is known at test time removed (with all others in place) and each task is assumed to have the same number of classes with similar semantics; in this case, the method is asked to classify among the possible classes within any given task. Class learning differs from domain learning in that each task may contain different number of classes that may have non-overlapping semantics with the classes in other tasks, and the method is asked to classify among the possible classes across all tasks. Discrete task-agnostic learning further generalizes this by removing the assumption of known task identity training time and replacing it with an assumption of known boundaries between different tasks at training time. In the most general setting, task boundaries between different tasks are also unknown, even at training time. This setting is known as continuous task-agnostic learning, which is the setting we focus on in this paper.
Note that the same method can yield very different performance under different settings; in particular, some methods may work very well on more restrictive settings, but undergo a significant performance degradation under less restrictive settings. This is because different underlying strategies employed by various methods may be especially dependent on certain assumptions. As result, care must be taken when interpreting results across different papers to ensure that evaluation is conducted under the same setting. Refer to (van de Ven & Tolias 2019 Hsu et al. 2018) for an extensive study and discussion the performance of various methods under different settings.
Existing continual learning methods can be divided into two broad categories: those that bias the parameters towards parameter values learned old tasks, and those that expands the model size to accommodate new tasks. In this paper, we propose a new approach that is orthogonal to these categories. Our approach neither penalizes parameter changes expands model size. Our key

---

Under review as a conference paper at ICLR 2020
intuition is that neural nets forget because parameters at all layers need to change adapt to new tasks. Therefore, one way to improve retention is to keep parameter changes localized. Because upper-layer parameters depend on lower-layer parameters, upper-layer parameters must change when lower-layer parameters change; hence, lower-layer parameters must be kept relatively static to prevent parameter changes throughout the network. One way achieve this would be to explicitly regularize parameter changes in the lower layers; this is less than ideal, however, because it would reduce the network's expressive power and therefore its capability learn on new tasks, How do we achieve this without compromising on expressive power?
To arrive a solution to this problem, we need to consider the underlying cause of parameter changes at all layers when fine-tuning on new tasks. In neural net, each layer on its own has quite limited expressive power and is simply a linear model. As result, if the features computed by the penultimate layer on training examples for the new task are not linearly separable, then the parameters in the layers up to the penultimate layer must change to successfully learn the new task. To avoid this, we can make the last layer more expressive and replace it with nonlinear classifier. To this end, we propose replacing the softmax activation layer with a k-nearest neighbour classifier. We will show that this simple modification is surprisingly effective at reducing catastrophic forgetting.
2 RELATED WORK
Our work offers a novel solution to the continual learning problem by leveraging ideas from metric
learning.
Continual learning is the idea that machine learning models should be able to emulate humans' ability to learn new information while retaining previously learned skills knowledge. This capability is generally relevant- for example, in robotics it is desirable for robots to adapt to given environment while retaining general functionality across environments; another example from computer vision is the desired ability to update a classifier online with information on new classes while retaining the ability to detect old classes. While is always possible to accomplish this goal of continual learning through retraining a model on the combined old and new datasets (e.g., as in (Caruana 1997)), this approach is computationally expensive or infeasible due the inability to store old data. One commonly used shortcut is to instantiate a model trained on the old dataset and then finetune that model on the new data (e.g., g., as (Hinton & Salakhutdinov 2006; Girshick et al. 2014)); however, this method has the side effect of (significantly) decreased performance on the old data.
Recent interest in continual learning has resulted in many proposed methods for continual learning with deep neural networks. They can be grouped into two broad categories: those that dynamically expands the model architecture, and those that constrains the parameter values to be close to those obtained after training on previous tasks.
An example of a method in the former category is (Rusu et al., 2016), which creates a new network for each new task, with connections between the networks (from old task networks to new ones), thus ensuring the network weights for the old network are preserved while enabling adaptation to the new task. Another method (Masse et al. 2018) randomly zeros subset of hidden units for each task, which can be viewed as a way to implicitly expand the model architecture, since the hidden units that are zeroed are effectively removed from the architecture.
In the latter category are two subtypes of methods: those based on regularization and those based on data replay. Regularization-base methods are all based on the idea of protecting important parameters and introduce different regularizers to penalize changing the values of these parameters from what they were after training on previous tasks. Different methods differ in the way they determine importance of each parameter. Examples of methods of this flavour include elastic weight consolidation (EWC) (Kirkpatrick et al., (017), online EWC (Schwarz et al. 2018) and synaptic intelligence (SI) (Zenee et al. 2017).
Whereas regularization-basee methods can be viewed as constraining the parameter values explicitly, data replay-based methods do so implicitly. Generally speaking, data replay works by generating simulated training data from the model trained on previous tasks and using it to augment the training data used to train the new task. For example, Learning without Forgetting (LwF) (Li & Hoiem 2018) performs distillation (Hinton et al.] 2013) on the training data for the new task, or in other words, takes simulated labels to be the predictions of the model trained on previous tasks on the
2

---

Under review as a conference paper at ICLR 2020
input training data for the new task. Deep Generative Replay (DGR) (Shin et al. 2017 uses deep generative model to generate simulated input data and labels them with the model trained on previous tasks. Other related methods (Rebutfi et al. 2017 Wu et al. 2018 Venkatesan et al. (017] van der Ven & Tolias 2018) use training data from previous tasks, either directly or to train a deep generative model, the samples from which are used to augment the current training set. For more comprehensive review of this area, we refer readers to (Parisi et al. 2019).
Note that all these methods least require task boundaries to be known. For example, methods that dynamic expand the model architecture need to know when to expand the architecture, regularization- based methods need to know the point in time at which the parameter values are used to constrain future parameter updates, and data haplay-based methods need to know when to generate simulated training data. Some methods, like (Masse et al. 2018), require stronger assumptions, namely that task identities be known as well. In contrast, we consider the setting where neither task identities nor task boundaries are known.
Metric learning involves learning a (pseudo-) distance metric. This can be combined with k-nearest neighbours (k-NN) classifier to yield classification predictions. Standard k-NN works by finding the closest k training data points to a test data point and classifying the test data point as the most common label among the training data. The distance metric used for determining the "closest" points is commonly the Euclidean distance. As proposed by |Weinberger et al. (206)), it is possible and desirable to learn a Mahalanobis distance metric that improves the accuracy of k-NN. The loss function attempts to enforce clustering of labels with large margins between clusters.
More recent work like (Schroff et al.) 2013) has used similar ideas in the context of deep neural networks where the distance between two data points is defined as the Euclidean distance between their embeddings computed by the network. Most methods work by pulling examples whose labels are the same as that of the current training example closer to the training example and pushing examples whose labels are different away. The challenge becomes designing loss function that is conducive efficient training of the network. Various loss functions like those used in (Schroff et al.) 2015; Tamuz et al., 20011] Van Der Maaten & Weinberger 2012) have been proposed, to varying degrees of success on varying datasets.
3
METHOD
3.1
MODEL
The key observation we make is that in order for a neural net to learn on non-linearly separable data, the parameters in the layers below the last layer must change. So, if the training data for the new task is not linearly separable from the training data for the previous tasks, the parameters below the last layer will be changed when training on the new task. Because the parameters in the layer below the last layer are shared across all classes, the classes seen during previous tasks must depend on the parameters below the last layer. And so changes in those parameters would result in performance drop on these classes, leading to catastrophic forgetting.
To avoid catastrophic forgetting, we must make sure the parameters below the last layer will change significantly even as the model is trained on new tasks. To this end, we must make the last layer more expressive than a linear classifier. Therefore, we replace it with a nonlinear classifier, namely a k-nearest neighbour classifier using learned Mahalanobis distance metric, which is equivalent to replacing the softmax activation layer in a neural net with a k-nearest neighbour classifier using a Euclidean distance metric.
More formally, we can characterize the test-time prediction of a vanilla neural net classifier and the proposed classifier as follows:
vanilla (x;;) == arg max Nxaf((;)) + bn)
(I)
i proposed (z.) == (NNNWWxxaf(Ce
bb),
(2)
where Eq. is a vanilla feedforward neural network classifier and Eq. 2 is the proposed classifier. Here 2;; is the image input, f(x) Rd denotes the feature activations of the neural network the second last layer, N is the number of classes, and D is the dimension of the image embedding output by our model. arg max(-) returns the index of the greatest element in the input vector, and kNN(.)
3

---

Under review as a conference paper at ICLR 2020
returns the most common label of the k nearest training examples the imput in Euclidean distance. Wnxn denotes a weight matrix of given dimensions m x n and bm denotes a bias vector of given dimension
3.2 LEARNING
3.2.1 LOSS FUNCTION
Just as the vanilla classifier is trained using cross-entropy loss, to train the proposed classifier, we need to define loss function. Training using the loss be viewed as instantiation of metric learning, where the metric is defined by the parameters of the neural net.
Lightoipit max (O., M (1
PI
N
Anchor
A
M
P Positive
N Nggative
Margin
Figure 1: Graphical depiction of the loss function in Eq.
We would like to design the loss so that the predictions of the k-nearest neighbour classifier on the feature activations of the last layer as accurate as possible. To this end, for every training example, we would like the nearby training examples to be in the same class and the training examples that are in other classes to be far away. practice, we found it suffices to enforce this for only small subset of training examples, which we will refer to as anchors. The training examples that are in the same class as an anchor is known as positive examples, and those that negative exumples. In practice, we choose one anchor for each class, which means that we only need to store one example for each class.
different classes are known as
The loss function we use is modified form of the triplet loss (Schroff et al. 2015):
N Liriplet = Σ [ - {< 11 - M
(3)
where & denotes the model, i.e.: function that maps the input to embedding, x denotes an anchor, r. denotes a positive example and r. denotes a negative example, and M is a hyperparameter representing the desired minimum margin of separation between the positive and negative examples.
Ideally, we would like the training examples of the same class to cluster together. In practice, we found that training using Teripiet results in overlap between clusters for different classes, shown in Figure 2] To discourage this, we add another term to the loss function to encourage tight clusters:
Light-triplet = Luriplet a ( (<;'))
(4)
where a is hyperparameter for balancing different terms in the loss.
4

---

Under review as a conference paper at ICLR 2020
triplet
Light-triptet
airplane
deer
Figure 2: Comparison of the two loss function variants in Eq. 3nd 4 on the CIFAR-I0 dataset output 2-dimensional embedding. The labeled loss function is used to train the network for each figure. The displayed figures generated by sampling training examples and plotting their embeddings, Plots of examples appear similar, albeit slightly more spread out for both loss functions.
Here, a 5-layer CNN is trained
3.2.2 TRAINING PROCEDURE
Algorithm Training Procedure
Sample set Xa of ka anchors
for 1.... do
Sample subset X of lig training examples for Zai E Xa do
Xp t set of positive examples Fai in Xe
X, < set of negative examples of wai in X
Daipp + (#as) ((pp))) |
Vaai E Xa, E Xp Dai,nk - (xxs)) (2 Veas Xax Enk X
end for
for Fai Xa do
Tam: t- {FFri Inj Fnt) Rpj E Nnx E Xn, Daigi < Daink)
Tan: = {laa lp: bn) Tam: bn is among the d closest negative examples to la in Taus)
Train model & T2., for steps
end for
end for
As shown Figure train the model, we take a random training example from each class and combine them into a set of anchors. In each iteration, we select a batch of training examples and compute the distances between each anchor and the training examples in the batch. Then we construct the minibatch of triplets used train the model. We only include triplets where the positive example closer to the anchor than the negative example and the said negative example is within the closest to the anchor among negative examples that farther from the anchor than the positive example. The variable a hyperparameter and is known as the dynamic margin, because the distance to the farthest negative example the anchor that is included varies depending embedding. The reason we do not include triplets where the positive example farther from the anchor than the negative example is because the model would tend map all examples to zeros otherwise, which is in line with the observations of (Schroff et al. 2015).
the scale of the
While training, we need retrieve the closest examples the anchor. To this end, we an efficient k-nearest neighbour search algorithm that is scalable to large datasets and high dimensions known as Prioritized DCI (Li & Malik 2017).
3.3 EMBEDDING NORMALIZATION
We normalize the output embedding of our model, so that the embedding vector for each example has an lg norm of 1. This can be interpreted making k-nearest neighbors returns the k closest

---

Under review as a conference paper at ICLR 2020
Method
Set A: Original
Set A: After Seeing Ser B (Difference from Original)
Set
Our Method
97.5%
95.1% (-2.4%)
95.6%
Very Forgetful & Adaptive:
SGD with dropout (ratio: 0.5)
98.62%
0.0% (-98.62%)
98.62%
Elastic weight consolidation (EWC)
98 98.62%
0.0% 62%)
98.46%
Online EWC
98. 98.62%
0.0% 62.)2%
98.46%
Synaptic intelligence (SI)
98.62%
0.0% ( (-98.62%)
98.13%
Forgetful & Adaptive:
Deep Generative Replay (DGR)
98.78%
89.32% (-9.46%)
98.40%
DGR distillation
98.78%
91.70% (-7.08%)
98.05%
Soglayttroug-FFedddee Feedhack (R(F)
98.96%
93.22% (-5.74%)
98.62%
Not Forgetful Adaptive: Learning without Forgetting (LwF)
98.62%
98.07% (-0.55%)
6.84%
Table 1: Performance of the proposed method compared to eight other continual learning methods on MNIST. SGD with dropout, EWC, online EWC and suffer from catastrophic forgetting on Set A after seeing Set B, but were able to adapt well the new task (Set B). DGR, DGR distillation and RtF forget more on Set A compared to our method. LwF retains knowledge learned on Set A; however, it fails to adapt to Set B. We also note that all baseline methods assume knowledge of task boundaries, whereas the proposed method does not.
vectors by angle, which in this case is the same as return the vectors that closest by Euclidean distance. So, our classifier is essentially basing its decision on the maximum cosine similarity between the image embeddings.
EXPERIMENTS
We consider the setting where training data from subset of categories are presented to the learner in stages. In the first stage, subset of the categories presented (which will henceforth be referred to as Set A). and the learner can iterate over training data from these categories as many times as it likes. In the second stage, the remaining categories are presented (which will henceforth be known as Set B); while the learner can look at training data from these categories as often as it desires, it no longer has access to the training data from the categories presented the first stage.
4.1 TRAINING AND TESTING PROCEDURE
We consider the following training procedure:
1. Train on Set A.
2. Starting from the parameters obtained from the first step, train on Set B.
We evaluate performance on Set A after each step in order to assess the extent each method retains its performance on Set after training on Set
4.2 METHODS
For the baseline, we consider vanilla neural net classifier with softmax activation the final layer trained with cross-entropy loss. To train Set B, because the weights of the last layer specific to categories Set A, we do not use them and instead initialize the last layer randomly. After training on Set B, for the purposes of evaluation new set of the weights for the last layer (while keeping the weights of all other layers fixed) on Set A.
Set A, discard the weights of the last layer and train
For the proposed method, which replaces the softmax activation with k-nearest neighbour classifier, we use the same architecture as the baseline when training on the smaller set (which our case is always Set B). Thus, the number of parameters is identical that used by the baseline. To train

---

Under review as a conference paper at ICLR 2020
Method
Set A: Original
Set A: After Seeing Ser B (Difference from Original)
Set
Our Method
72.3%
62.3% (-10.0%)
64.2%
Forgetful Adaptive:
SGD with dropout (ratio: 0.5)
64.02%
0.0% (-64.02%)
85.05%
Elastic weight consolidation (EWC)
67.38%
0.0% (-67.38%)
76.08%
Online EWC
68.97%
0.0% (-68.97%)
79.17%
Synaptic intelligence (SI)
68.47%
0.0% (-68, 47%%
82.83%
Deep Generative Replay (DGR)
69.13%
0.40% (-68.73%)
83.75%
DGR with distillation
68. 22%
0.03% (-68.199 19%%
85.00%
Replay-trough--eeedee Feedhack (RiF)
66.57%
0.200% (-66.37%)
86.60%
Not Forgetful Not Adaptive: Learning without Forgetting (LwF)
69.28%
45.50% (-23.78%)
55.53%
Table 2: Performance of the proposed method compared eight other continual learning methods CIFAR-10. SGD with dropout, EWC, online EWC and SI suffer from catastrophic forgetting on Set after seeing Set B, but were able adapt well the new task (Set B). DGR, DGR distillation and RtF also forget on Set A, this could relate to the fact that CIFAR-10 10 data is more complex and it difficult for these methods to train generative model for replaying. LwF retains some knowledge learned on Set A; however, it fails to adapt to Set B compared to our method.
Set A, to reduce the amount of training time on large datasets, we first pretrain using cross-entropy loss and then switch to the proposed method later on. To train on Set B, unlike for the baseline, we do not need to do anything special for the last layer weights, because the output dimension of the last layer does not need to change when the number of classes changes.
MNIST, which is the dataset most existing continual learning methods are tuned for, we also compare seven continual learning methods that operate under more restrictive assumptions (where knowledge of task boundaries is required), including Elastic Weight Consolidation (EWC), online EWC, Synaptic Intelligence (SI), Deep Generative Replay (DGR), DGR with Distillation, Replay- Fhrougk-Feedback (RtF) and Learning without Forgetting (LwF).
4.3 DATASETS
We consider four different datasets/splits:
We first consider MNIST, which the simplest dataset. We divide into 2 tasks: Set consists of the odd digits and Set B includes the even digits.
MNIST
We then consider CIFAR-10 and take Set A to be the six animal categories (bird, cat, deer, dog, frog and horse), and set be the four remaining categories corresponding to man-me mad- objects (airplane, automobile, ship, truck). This split is designed to be more challenging than just random split, since we don't expect many intermediate-leval features to be shared across Sets and B, making it easier to forget about Set A when training on Set B.
CIFAR-10
CIFAR-10 (5 Tasks) We also consider a variant of CIFAR-10 with 5 tasks, where each task corresponds a pair of consecutive categories. This is more challenging than the two-task setting above, because many more examples would be seen after training on the earlier tasks, and so preventing forgetting on the earlier tasks harder.
ImageNet (Random Split) Next we consider ImageNet, which is larger and complex than CIFAR-10. We randomly select 100 classes to serve as Set A, and 100 classes to serve as Set B.
ImageNet (Dog Split) Finally, we consider ImageNet and use a more challenging split. For Set A. we take it to be all 880 categories that are not dogs; for Set B, we take it to be all 120 categories that correspond to particular breeds of dogs. Similar to the split we used for CIFAR- 10, this split is designed to be more challenging by minimizing semantic overlap between Sets A and B.
7

---

Under review as a conference paper at ICLR 2020
Method
Task
Task
Task
Task 4
Task 5
Average
Our Method
43.00%
16.20%
24.65%
48.85%
27.55%
32.05%
Very Forgetful & Adaptive:
SGD with dropout (ratio: 0.5)
0.0%
0.0%
0.0%
0.0%
92.85%
18.57%
Elastic weight consolidation (EWC)
0.0%
0.0%
0.0%
0.0%
89.50%
17.90%
Online EWC
0.0%
0.0%
0.0%
0.55%
88. 30%
17.77%
Synaptic intelligence (SI)
0.0%
0.0%
0.0%
0.0%
93.05%
18.61% %
Forgetful & Adaptive:
Deep Generative Replay (DGR)
0.15%
1.05%
0.0%
0.10%
93.75%
19.01 %
DGR with distillation
0.05%
0.70%
0.0%
0.15%
93.15%
18.81 %
Replay-trone troughhFFeebbak (RIF)
Feedback
0.0%
1.45%
0.85%
0.20%
94.10%
19.32%
Not Forgetfull & Not Adaptive: Learning without Forgetting (LwF))
93.55%
0.65%
0.0%
0.0%
0.0%
18.84%
Table 3: Performance of the proposed method compared to eight other continual learning methods on 5-task CIFAR. SGD with dropout, EWC, online EWC and SI suffer from catastrophic forgetting on previous tasks after seeing newer tasks, but were able to adapt well to the last task. DGR, DGR + distillation and RtF forget more on the previous tasks compared to our method. LwF retains knowledge learned on first set; however, it fails to adapt to newer tasks.
Method
Baseline After Seeing Set B
Original
After Seeing Set B Original
ImageNet (Random split)
69.5%
59.2%
72.7%
30.2%
ImageNet (Dog split)
67.0%
52.9%
70.0%
30.4%
Table 4: Performance comparison between method and the vanilla baseline ImageNet. Perfor- mance evaluated as top-1 percent correct on test data for the original task. Baseline represents the network trained with cross-entropy loss. "Original" denotes the network after being trained on the original task (classes in Set A) and "After Seeing Set B" denotes the network after it is fine-tuned on the new task (classes in Set B). See corresponding sections for more details on the dataset and network used for each row.
4.4 NETWORK ARCHITECTURE
We used standard network architectures that have proven to work well on the datasets we consider. For MNIST, following (van de Ven & Tolias 2019), we use network with two hidden layers of 400 units each. We use this same architecture for our method and all baselines. For CIFAR-10, we use the architecture from the Tensorflow CIFAR- 10 tutoria[] which consists of two convolutional layers followed by three fully-connected layers. For ImageNet, we use the AlexNet architecture (Krizhevsky iet al., 2012). which consists of five convolutional layers followed by three fully-connected layers. All networks are trained using Adam (Kingma & Ba 2014).
4.5
KEY FINDINGS
The test accuracy of each method is given in Tables I and Lin in the sub-column labelled "Original". After fine-tuning, compare the accuracy of the network relative to the test-accuracy from set A these numbers are given Table in the sub-column labeled "After Seeing Set B". For the MNIST comparison, we also include the test-accuracy from set B.
Table D shows the results of the proposed method and existing continual learning methods on MNIST. As shown, regularization-basee methods including EWC, online EWC and SI are very forgetfull of the knowledge learned on the previous task (set A) but can adapt to the new task (set B). Replay- based methods (DGR, DGR distillation, RtF) adaptive well, but are more forgetful compared to the proposed method. Another replay method, LwF, retains knowledge from the previous task (set A) well, however, it struggles with adapting the new task (set B). On the other hand, the
8

---

Under review as a conference paper at ICLR 2020
Set (Original)
Set (After Seeing set
Sct B
Original
69.5%
59.2%
69.2%
Number of Positive Examples (5 30)
+0.4%
+0.1 %
+0.3%
Number of Negative Examples (40 + 10)
-0.9%
-1.8%
-0.6%
Embedding Dimension (100 + 200)
-0.1%
+0.1%
-0.5%
Embedding Dimension (100 + 50)
-1.3%
-0.3%
-0.1%
Table 5:
Test accuracy under the original hyperparameter setting and the change in test accuracy under different hyperparameter settings relative to the original setting. Increasing the number of positive examples (from 5 to 30) slightly improves performance. Decreasing the numher of negative examples (from 40 to 10) slightly reduces test accuracy. Increasing the embedding dimension (from 100 to 200) results in similar performance as the original setting, whereas decreasing the embedding dimension (from 100 to 50) moderately lowers the performance. Overall, the method is fairly robust hyperparameter changes.
proposed method can both adapt to the new task and retain knowledge about the previous task, which especially remarkable considering its simplicity and ability operate without knowledge of task boundaries.
Table shows the results of the proposed method and the baseline as CIFAR-I0 and ImageNet with random split and dog split. As shown, even though our method achieves lower test accuracy compared to the baseline after only training Set it is better remembering the knowledge it learned once it is trained on new data. This is true in each of the datasets we tested on, and the gap is substantial (it achieve nearly double the test accuracy of the baseline) for ImageNet (random split) and CIFAR-10. The gap is smaller on ImageNet (dog split), which can be attributed the diminished similarity between set and set B. Since set only consists of dog images, fine-tuning on this data may wipe out much of the knowledge the model has learned about images in general. In contrast, since ImageNet (random split) does contain a random sampling of images in each set, it is likely that fine-tuning on set B preserves more general concepts about images that are still useful in classifying the images in set A.
complex datasets, such
4.6 ANALYSIS
We further examine the performance of the proposed method the ImageNet dataset with random split.
4.6.1 SENSITIVITY ANALYSIS
We varied the hyperparameter settings and took a look at the impact of changes in hyperparameters. We tried the following settings: reduce number of negative samples, increase number of positive sample and different embedding dimensions. As shown in Table increasing the number of positive examples showed a slight improvement, whereas decreasing the number negative examples resulted slight performance drop. A similar trend can be observed with the embedding dimension, where proposed method does not benefit from the increase in embedding dimension, neither does deteriorate significantly when the embedding dimension is decreased. Overall, we found the method to be reasonably robust to hyperparameter changes.
4.6.2 ABLATION STUDY
We tested the proposed method by disabling some key components to determine their contributions. The following components are analyzed: embedding output normalization and dynamic margin. As shown in T by disabling embedding output normalization, the performance on both tasks drops significantly. When the dynamic margin is removed and replaced with static margin, the method fails to capture new information from set B. In summary, we found that embedding normalization and dynamic margin contribute enormously to the performance of the proposed method.

---

Under review as a conference paper at ICLR 2020
Sct (Original)
Set (After Seeing set
Sct
With All Components 69.5%
59.2%
69.2%
No Normalization
-12.2%
-2.4%
-17.8%
No Dynamic Margin -9.5%
+0.8%
-14,8%
Table 6: Test accuracy with all components, and changes in test accuracy relative to the original proposed method after various components removed. Not normalizing the embedding output dramatically lowers test accuracy, and especially limits the ability to adapt to the new task (Set B). Disabling dynamic margin also greatly decreases the performance on Set B. Overall, embedding normalization and dynamic margin are crucial for the method to be able to adapt well to new tasks.
found that
CONCLUSION AND FUTURE WORK
As discussed in Section 4 we achieve consistent results across four different datasets/splits. These results suggest that the method is successful at addressing the catastrophic forgetting problem, and hint a possible general strategy for reducing catastrophic forgetting, namely the idea of replacing the last layer of a neural net with non-linear classifier.
In the future, we would like to explore combining the proposed approach with other orthogonal approaches for tackling catastrophic forgetting. We also plan to explore applications that can benefit from continual learning approaches, such as reinforcement learning.
REFERENCES
Rich Caruana. Multitask learning. Machine learning, 28(1):41-75 1997.
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587, 2014.
Geoffrey E Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504--077, 2006.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, abs/1503.02533 2015.
Yen-Chang Hsu, Yen-Cheng Liu, Anita Ramasamy, and Zsolt Kira. Re-evaluating continual learning scenarios: A categorization and case for strong baselines. arXiv preprint arXiv:1810.1 12488, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6991 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526, 2017.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In Advances neural information processing systems, pp. 1097-1105, 2012.
Ke Li and Jitendra Malik. Fast k-nearest neighbour search via prioritized dei. In Proceedings the 34th International Conference on Machine Learning- Volume 70, pp. 2081-2090. JMLR.org, 2017.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(1 (0(1)::9335-9477 2018.
Nicolas Y. Masse, Gregory D. Grant, and David J. Freedman. Alleviating catastrophic forgetting using context- -ependtt gating and synaptic stabilization. Proceedings of the National Academy of Sciences of the United States of America, 115 441110467-811445, 2018.
10

---

Under review as a conference paper at ICLR 2020
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
Sylvestre-AAlisee Rebufñ,, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampeet. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 04671, 2016.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 815-823, 2015.
Jonathan Schwarz, Jelena Luketina, Wojeiech Czarnecki, Agnieszka Ttrabsa-Barwinnse Yee Whye Tch, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable frame- work for continual learning. arXiv preprint arXiv: 06370, 2018.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS, 2017.
Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learning the crowd kernel. arXiv preprint 1105::1000.033 2011.
Gido M van de Ven and Andreas S Tolias. Three scenarios for continual learning, arXiv preprint arXiv:1904.07733 07734, 2019.
Laurens Van Der Maaten and Kilian Weinberger. Stochastic triplet embedding. In Machine Learning for Signal Processing (MLSP), 2012 IEEÉ International Workshop on, pp. 1-6. IEEE, 2012.
Michiel van der Ven and Andreas Tolias. Generative replay with feedback connections as general strategy for continual learning. ArXiv, abs/1809.10633. 2018.
Ragav Venkatesan, Hemanth Venkateswara, Sethuraman Panchanathan, and Baoxin Li. A strategy for an uncompromising incremental learner. ArXiv, abs/1705.00744. 2017.
Kilian Q Weinberger, John Blitzer, and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. In Advances in neural information processing systems, pp. 1473- 1480, 2006.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, Zhengyou Zhang, and Yun Fu. Incremental classifier learning with generative adversarial networks. ArXiv, abs/1 Abs//802.00083, 2018.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, 2017.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. 2018.
11


Reviews:
[{"review_id": "HygFUMZKYS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers the use of a metric learning approach in a continual/lifelong classification settings. Experiments show in the case of two tasks forgetting can be minimized by using the approach. \n\nMethods\nThe proposed method appears to be a standard triplet loss. The authors add a second term to the triplet loss that is essentially making the loss a combination of the triplet and siamese loss. It’s not really explained anywhere why they do this and whether its essential to the performance. \n\nIs there anything specific to continual learning done or is the paper essentially pointing out this existing method (metric learning + nearest neighbor) is surprisingly effective for forgetting. If this is the case the authors should present it in this way I think. \n\nAlthough triplet loss can often yield reasonably performance on classification problems it tends to not perform as well as cross entropy loss, this is observed in other works as well as this one.\n\nA major question of mine: it is not clear from the method nor experiments what samples are stored after task A for the kNN classifier. Is it all of the data samples from the previous task? \n\nExperiments\nThe experimental results consider a custom continual learning setup where there is two sets of categories. Overall the experiments seem lacking at the moment in rigorous comparisons. \n\nMNIST experimental comparisons are currently suspect. It is  very surprising that LwF does so poorly, do the authors have some explanation for this. LwF is typically a reasonable baseline for these 2 task settings (e.g. https://arxiv.org/pdf/1704.01920.pdf).  Similarly the well known EWC is shown to simply not work at all for the very task it was designed for on the MNIST dataset. LwF and EWC simply not working to any degree seem to me like  rather dramatic claims to make without any explanation. \nCryptically the fine-tuning baseline described in 4.2 is not shown here for MNIST? This seems a major oversight\n\nCIFAR10/Imagenet Experiments\nIt is not clear if the baseline finetuning is done on only the top weights or the entire network. Both of these baselines should be considered. Another good baseline to consider is finetuning with cosine distance and only the top weights as in https://arxiv.org/pdf/1804.09458.pdf and other recent works should also be considered\n\nWhy do the authors not include any of the baselines from MNIST experiments here, for example LwF.\n\nAblations study the need for normalization and dynamic margin, it seems these are helpful for accuracy and forward transfer (and not as critical for minimizing forgetting).\n\n\nThe author state their method is agnostic to the task boundaries, its a bit unclear what this means in this context. The procedure is not online and the labels of the samples are being used? If the authors are referring to the need to add additional outputs to the “vanilla” model this seems like it can be trivially addressed by simply saying outputs are added the first time a new class is seen thereby making it agnostic to the boundary in the same sense as this method. \n\nClarity \nCan be problematic at times. Although all the elements of the approach are outlined the motivations are overly wordy and repetitive making them actually hard to follow. \n\n-(minor) first/2nd paragraph of 3.1 seems a bit redundant making it hard to follow\n\nOverall I think the idea to consider metric learning and local adaptation for continual learning is interesting, however the current work is currently lacking in both experimental evidence (appropriate comparisons) and clear motivation/difference to existing work  for its particular instantiation of this idea. \n\n++++Post Rebuttal++++\n\nThank you for your detailed responses.\n\nThe clarification about “task-agnostic” for the experiments does make them look more relevant than I had previously assessed. I do want to note that the language used for this is inconsistent with the ones used in other papers, which typically calls this a “shared-head” setting (https://arxiv.org/pdf/1801.10112.pdf, https://arxiv.org/pdf/1805.09733.pdf, https://arxiv.org/pdf/1903.08671.pdf ). It is also somewhat inconsistent with the authors own definition of “task agnostic learning” given in the introduction of this paper which implies it is something related to task boundaries at training time, in fact this is something related to availability of the task id at test time. I suggest the authors to make this more clear. Furthermore, the authors should highlight all this in the experiment text, e.g. noting EWC does poorly but this is because we use a different protocol than this and this paper etc.\n\nRegarding the experiments under this light they do look more reasonable. Indeed it has been observed that EWC works poorly in the shared-head setting https://arxiv.org/pdf/1801.10112.pdf\nRegarding the new 5 task CIFAR-10 the results are interesting, however I will point the authors to the work above (Rwalk) which also reports results in this setting better than theirs (but not by too much). \n\nI do however still have issues regarding the memory usage of the method, specifically which data needs to be stored from previous tasks. It is still not completely clear and I find obfuscated since just one sentence not even fully answering the concern about this was added to the manuscript despite myself and another reviewer asking about it. My understanding based on the (somewhat conflicted responses) of the authors is they store a substantial amount of prior task data, but most of this is only used  at test time. For example for imagenet as much as 1000 images/class are stored for testing time. This begs the question why not use this data for training as well if it is allowed to be used by the model at testing time (and therefore preserved from the first task), why is the storage cost of this data not considered and how do the authors justify this still being a lifelong learning setup. As an alternative, why can't one use a much bigger fully parametric model that uses the same amount of storage as the authors model + stored images. It seems it is not fair to compare these to methods that cant utilize this large storage amount. \n\nFinally its not clear if this data is stored as raw images or somehow stored as embeddings. If it is stored as embeddings this would require some discussion on how the authors avoid representation drift when the next task is training. If the authors store raw images, it means at evaluation time the entire raw dataset needs to be re-encoded, therefore the model can’t perform easily anytime inference. \n\nUnfortunately the discussion period ended but I would have liked more clarification on this, on the other hand these pieces of information should really have been in the manuscript in the first place. \n\nOverall, my impression of the paper is improved.  But I do think it could use some further writing revisions to emphasize/clarify key points: a) the method is not new (it says e.g. in abstract “new model” which is misleading) but its application in CL is under-explored b) the experiments show poor performance on existing methods because most of those are not designed nor work well for the shared head “task agnostic” setting, while metric learning handles it gracefully.  c) be explicit about what is the memory being stored when moving onto the next task (this should be somewhere visible and explicit) and how this is justified\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HkeBUpcTKr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a possible way to mitigate catastrophic forgetting by using a k-nearest neighbor (kNN) classifier as the last layer of a neural network as opposed to a SoftMax classifier.  I think this an interesting and possibly novel use of a kNN layer (I haven't seen similar uses although I'm not that familiar with the specific research area).  At the same time it's not presenting a ground breaking new algorithm or anything like that.\n\nOverall the paper is fairly well written and not too hard to follow.  I would say overall results in Table 1 are positive although the authors' approach has the lowest performance after just training on set A if that initial accuracy is important, and also doesn't have quite as high of an accuracy on test B compared to most of the other baselines.  Additionally, if you add the accuracy on both set A and set B after training on set B the sum is slightly higher for Rtf.  If you look at the minimum accuracy between set A and set B after training on set B, however, the authors' method has the highest value which might be what someone is looking to maximize.  \n\nOne weakness of this is paper is that I think there are other baselines that should be compared against in Table 1 such as something as basic as SGD with dropout (some of the baselines that are compared against in Table 1 were compared against SGD with dropout in their citations).  There are a number of additional approaches outlined in https://www.cs.uic.edu/~liub/lifelong-learning/continual-learning.pdf.  Also maybe even something with self attention such as Serra at al. https://arxiv.org/pdf/1801.01423.pdf.\n\nAnother potential issue I have with this paper is that it only reports results for the authors' method and the vanilla baseline for more complex CIFAR-10 and ImageNet data sets in Table 2.  Assuming there aren't restrictive assumptions for some of the methods that prevent them from being run on the other data sets (at least SI was previously evaluated on CIFAR-10), I would like to see how other baselines perform on these more complex datasets too.\n\nThe lack of some more baselines such as SGD with dropout, and not reporting the performance of the same baselines from Table 1 in Table 2, cause me to be very borderline on this paper.  I do appreciate the sensitivity analysis and ablation study provided.\n\nAs alluded to in future work I'm curious how the authors' approach might be applied to reinforcement learning, and if there could be a way to deal with continuous action spaces in RL.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HJlvCi32KS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper applies metric learning to reduce catastrophic forgetting on neural networks. By improving the expressiveness of the final layer, the authors claim that lower layers do not change weights as much, leading to better results in continual learning. They provide large-scale experiments on different datasets.\n\nI like the idea that the authors propose and the intuition for why it works, and the paper is well-written. However, I have some concerns and questions. My main concern is that experiments are only performed in the two-task setting, which is highly restrictive.\n\nThe authors claim that they tackle the general 'continuous task-agnostic learning' setting. However, they only test on the two-task setting. There are various problems with considering only a two-task setting (see for example Farquhar and Gal, \"Towards Robust Evaluations of Continual Learning\"). It is too easy to optimise parameters and methods to work in the two-task setting that will not generalise to more than two tasks, which the authors seem to claim. I would need to see experiments on more than two tasks. Aside from this, the experiments seem detailed, with a reasonable baseline, large-scale experiments (on ImageNet), and with an ablation study. \n\nIt seems to me like the anchors need to be chosen before training. This means that this method requires memory / storage of past data examples. It is usually fine to do store a small subset of examples in continual learning, but should be made explicit, because it may not always be possible (eg if there are data privacy laws). \n\nI do not understand the reason why the output embeddings need to be normalised (Section 3.3)? I can see from Table 4 that it improves results, but do not see any intuition.\n\nI would also like to see the computational cost of this method, perhaps as a run-time compared to the baseline. There are many hyperparameters to tune on the validation set which may slow the method down. The sensitivity analysis did not consider changing 'd' or 'M', which seem like crucial hyperparameters to me.\n\n------------\nEDIT: I will keep my score after the the discussion with authors. Although the paper has improved in my opinion, I still recommend Weak Reject. I very much appreciated the 5-task CIFAR-10 results. However, there are simple baselines in this setting that I believe need to be explored and reported. Namely, baselines but with samples, eg EWC+samples, akin to the RWalk paper that AnonReviewer1 mentions (https://arxiv.org/pdf/1801.10112.pdf). This is because the proposed method also uses samples. Going from the RWalk paper, this improves results for the baselines considerably, but this may depend on number of samples etc. I understand there was not much time during the rebuttal period to include this. I hope that the authors will consider doing so in the future.\n\nThe discussion/explanation regarding 'task-agnostic' (train and test time) and also regarding how the anchors are chosen needs to be made clearer.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Catastrophic forgetting in neural networks is a real problem, and this paper suggests a mechanism for avoiding this using a k-nearest neighbor mechanism in the final layer. The reason is that the layers below the last layer should not change significantly when very different data is introduced. 

While the idea is interesting none of the reviewers is entirely convinced about the execution and empirical tests, which had partially inconclusive. The reviewers had a number of questions, which were only partially satisfactorily answered. While some of the reviewers had less familiarity with the specific research topic, the seemingly most knowledgeable reviewer does not think the paper is ready for publication.

On balance, I think the paper cannot be accepted in its current state. The idea is interesting, but needs more work.
Decision: Reject