Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper presents a possible way to mitigate catastrophic forgetting by using a k-nearest neighbor (kNN) classifier as the last layer of a neural network as opposed to a SoftMax classifier.  I think this an interesting and possibly novel use of a kNN layer (I haven't seen similar uses although I'm not that familiar with the specific research area).  At the same time it's not presenting a ground breaking new algorithm or anything like that.

Overall the paper is fairly well written and not too hard to follow.  I would say overall results in Table 1 are positive although the authors' approach has the lowest performance after just training on set A if that initial accuracy is important, and also doesn't have quite as high of an accuracy on test B compared to most of the other baselines.  Additionally, if you add the accuracy on both set A and set B after training on set B the sum is slightly higher for Rtf.  If you look at the minimum accuracy between set A and set B after training on set B, however, the authors' method has the highest value which might be what someone is looking to maximize.  

One weakness of this is paper is that I think there are other baselines that should be compared against in Table 1 such as something as basic as SGD with dropout (some of the baselines that are compared against in Table 1 were compared against SGD with dropout in their citations).  There are a number of additional approaches outlined in https://www.cs.uic.edu/~liub/lifelong-learning/continual-learning.pdf.  Also maybe even something with self attention such as Serra at al. https://arxiv.org/pdf/1801.01423.pdf.

Another potential issue I have with this paper is that it only reports results for the authors' method and the vanilla baseline for more complex CIFAR-10 and ImageNet data sets in Table 2.  Assuming there aren't restrictive assumptions for some of the methods that prevent them from being run on the other data sets (at least SI was previously evaluated on CIFAR-10), I would like to see how other baselines perform on these more complex datasets too.

The lack of some more baselines such as SGD with dropout, and not reporting the performance of the same baselines from Table 1 in Table 2, cause me to be very borderline on this paper.  I do appreciate the sensitivity analysis and ablation study provided.

As alluded to in future work I'm curious how the authors' approach might be applied to reinforcement learning, and if there could be a way to deal with continuous action spaces in RL.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper applies metric learning to reduce catastrophic forgetting on neural networks. By improving the expressiveness of the final layer, the authors claim that lower layers do not change weights as much, leading to better results in continual learning. They provide large-scale experiments on different datasets.

I like the idea that the authors propose and the intuition for why it works, and the paper is well-written. However, I have some concerns and questions. My main concern is that experiments are only performed in the two-task setting, which is highly restrictive.

The authors claim that they tackle the general 'continuous task-agnostic learning' setting. However, they only test on the two-task setting. There are various problems with considering only a two-task setting (see for example Farquhar and Gal, "Towards Robust Evaluations of Continual Learning"). It is too easy to optimise parameters and methods to work in the two-task setting that will not generalise to more than two tasks, which the authors seem to claim. I would need to see experiments on more than two tasks. Aside from this, the experiments seem detailed, with a reasonable baseline, large-scale experiments (on ImageNet), and with an ablation study. 

It seems to me like the anchors need to be chosen before training. This means that this method requires memory / storage of past data examples. It is usually fine to do store a small subset of examples in continual learning, but should be made explicit, because it may not always be possible (eg if there are data privacy laws). 

I do not understand the reason why the output embeddings need to be normalised (Section 3.3)? I can see from Table 4 that it improves results, but do not see any intuition.

I would also like to see the computational cost of this method, perhaps as a run-time compared to the baseline. There are many hyperparameters to tune on the validation set which may slow the method down. The sensitivity analysis did not consider changing 'd' or 'M', which seem like crucial hyperparameters to me.

------------
EDIT: I will keep my score after the the discussion with authors. Although the paper has improved in my opinion, I still recommend Weak Reject. I very much appreciated the 5-task CIFAR-10 results. However, there are simple baselines in this setting that I believe need to be explored and reported. Namely, baselines but with samples, eg EWC+samples, akin to the RWalk paper that AnonReviewer1 mentions (https://arxiv.org/pdf/1801.10112.pdf). This is because the proposed method also uses samples. Going from the RWalk paper, this improves results for the baselines considerably, but this may depend on number of samples etc. I understand there was not much time during the rebuttal period to include this. I hope that the authors will consider doing so in the future.

The discussion/explanation regarding 'task-agnostic' (train and test time) and also regarding how the anchors are chosen needs to be made clearer.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper considers the use of a metric learning approach in a continual/lifelong classification settings. Experiments show in the case of two tasks forgetting can be minimized by using the approach. 

Methods
The proposed method appears to be a standard triplet loss. The authors add a second term to the triplet loss that is essentially making the loss a combination of the triplet and siamese loss. It’s not really explained anywhere why they do this and whether its essential to the performance. 

Is there anything specific to continual learning done or is the paper essentially pointing out this existing method (metric learning + nearest neighbor) is surprisingly effective for forgetting. If this is the case the authors should present it in this way I think. 

Although triplet loss can often yield reasonably performance on classification problems it tends to not perform as well as cross entropy loss, this is observed in other works as well as this one.

A major question of mine: it is not clear from the method nor experiments what samples are stored after task A for the kNN classifier. Is it all of the data samples from the previous task? 

Experiments
The experimental results consider a custom continual learning setup where there is two sets of categories. Overall the experiments seem lacking at the moment in rigorous comparisons. 

MNIST experimental comparisons are currently suspect. It is  very surprising that LwF does so poorly, do the authors have some explanation for this. LwF is typically a reasonable baseline for these 2 task settings (e.g. https://arxiv.org/pdf/1704.01920.pdf).  Similarly the well known EWC is shown to simply not work at all for the very task it was designed for on the MNIST dataset. LwF and EWC simply not working to any degree seem to me like  rather dramatic claims to make without any explanation. 
Cryptically the fine-tuning baseline described in 4.2 is not shown here for MNIST? This seems a major oversight

CIFAR10/Imagenet Experiments
It is not clear if the baseline finetuning is done on only the top weights or the entire network. Both of these baselines should be considered. Another good baseline to consider is finetuning with cosine distance and only the top weights as in https://arxiv.org/pdf/1804.09458.pdf and other recent works should also be considered

Why do the authors not include any of the baselines from MNIST experiments here, for example LwF.

Ablations study the need for normalization and dynamic margin, it seems these are helpful for accuracy and forward transfer (and not as critical for minimizing forgetting).


The author state their method is agnostic to the task boundaries, its a bit unclear what this means in this context. The procedure is not online and the labels of the samples are being used? If the authors are referring to the need to add additional outputs to the “vanilla” model this seems like it can be trivially addressed by simply saying outputs are added the first time a new class is seen thereby making it agnostic to the boundary in the same sense as this method. 

Clarity 
Can be problematic at times. Although all the elements of the approach are outlined the motivations are overly wordy and repetitive making them actually hard to follow. 

-(minor) first/2nd paragraph of 3.1 seems a bit redundant making it hard to follow

Overall I think the idea to consider metric learning and local adaptation for continual learning is interesting, however the current work is currently lacking in both experimental evidence (appropriate comparisons) and clear motivation/difference to existing work  for its particular instantiation of this idea. 

++++Post Rebuttal++++

Thank you for your detailed responses.

The clarification about “task-agnostic” for the experiments does make them look more relevant than I had previously assessed. I do want to note that the language used for this is inconsistent with the ones used in other papers, which typically calls this a “shared-head” setting (https://arxiv.org/pdf/1801.10112.pdf, https://arxiv.org/pdf/1805.09733.pdf, https://arxiv.org/pdf/1903.08671.pdf ). It is also somewhat inconsistent with the authors own definition of “task agnostic learning” given in the introduction of this paper which implies it is something related to task boundaries at training time, in fact this is something related to availability of the task id at test time. I suggest the authors to make this more clear. Furthermore, the authors should highlight all this in the experiment text, e.g. noting EWC does poorly but this is because we use a different protocol than this and this paper etc.

Regarding the experiments under this light they do look more reasonable. Indeed it has been observed that EWC works poorly in the shared-head setting https://arxiv.org/pdf/1801.10112.pdf
Regarding the new 5 task CIFAR-10 the results are interesting, however I will point the authors to the work above (Rwalk) which also reports results in this setting better than theirs (but not by too much). 

I do however still have issues regarding the memory usage of the method, specifically which data needs to be stored from previous tasks. It is still not completely clear and I find obfuscated since just one sentence not even fully answering the concern about this was added to the manuscript despite myself and another reviewer asking about it. My understanding based on the (somewhat conflicted responses) of the authors is they store a substantial amount of prior task data, but most of this is only used  at test time. For example for imagenet as much as 1000 images/class are stored for testing time. This begs the question why not use this data for training as well if it is allowed to be used by the model at testing time (and therefore preserved from the first task), why is the storage cost of this data not considered and how do the authors justify this still being a lifelong learning setup. As an alternative, why can't one use a much bigger fully parametric model that uses the same amount of storage as the authors model + stored images. It seems it is not fair to compare these to methods that cant utilize this large storage amount. 

Finally its not clear if this data is stored as raw images or somehow stored as embeddings. If it is stored as embeddings this would require some discussion on how the authors avoid representation drift when the next task is training. If the authors store raw images, it means at evaluation time the entire raw dataset needs to be re-encoded, therefore the model can’t perform easily anytime inference. 

Unfortunately the discussion period ended but I would have liked more clarification on this, on the other hand these pieces of information should really have been in the manuscript in the first place. 

Overall, my impression of the paper is improved.  But I do think it could use some further writing revisions to emphasize/clarify key points: a) the method is not new (it says e.g. in abstract “new model” which is misleading) but its application in CL is under-explored b) the experiments show poor performance on existing methods because most of those are not designed nor work well for the shared head “task agnostic” setting, while metric learning handles it gracefully.  c) be explicit about what is the memory being stored when moving onto the next task (this should be somewhere visible and explicit) and how this is justified


review_title: Official Blind Review #1

Meta-review: Catastrophic forgetting in neural networks is a real problem, and this paper suggests a mechanism for avoiding this using a k-nearest neighbor mechanism in the final layer. The reason is that the layers below the last layer should not change significantly when very different data is introduced. 

While the idea is interesting none of the reviewers is entirely convinced about the execution and empirical tests, which had partially inconclusive. The reviewers had a number of questions, which were only partially satisfactorily answered. While some of the reviewers had less familiarity with the specific research topic, the seemingly most knowledgeable reviewer does not think the paper is ready for publication.

On balance, I think the paper cannot be accepted in its current state. The idea is interesting, but needs more work.
Decision: Reject