Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposed a new explanation of a performance drop phenomenon in a memory-based continual learning method, Experience Replay. The performance initially has a sharp drop when the task switch happens. As the training goes on, the performance drop can be improved but not as much as satisfactory. The paper analyzed this phenomenon from a representation learning perspective and applied a contrastive learning approach to reconciling the performance drop. The authors conducted experiments to illustrate their methods have a superior performance over baselines.
main_review: The paper does good work explaining the phenomenon in the introduction section, which the reviewer found interesting. The paper also states the metric learning idea, which sounds plausible. Unfortunately, the writing somehow doesn’t connect this idea to the proposed method in section 4. The paper assumes the readers are familiar with the contrastive learning literature and doesn’t explain the loss functions quite well. It doesn’t explain why minimizing the loss function will lead to the effects mentioned in section 1. Since the reviewer is not from the area of representation learning, I couldn’t justify the correctness of the method. Besides this main issue, some other comments are as follows:
* The reviewer doesn’t find any experiments demonstrating the correctness of the method, i.e., whether the proposed method fixes the representation drift problem or not.
* The page format, i.e., margins between sections, were adjusted significantly and affects the reading.
* Add necessary parentheses over citations in Sections 3 and 4.
* Use a larger arrow in Fig. 1 (left) to represent gradients. It is hard to distinguish between gradients for tasks 1 and 2. The authors may want to explain how the gradient for task 1 is computed since the learner is in the task 2 phase.


summary_of_the_review: The paper analyzed an interesting representation drift problem of a popular continual learning method. However, the paper doesn’t explain the loss functions in detail. The reviewer couldn’t justify the correctness of the applied contrastive learning method.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studied one problem of continual learning which causes the forgetting of old classes, i.e new class representations will often overlap significantly with the previous classes, leading to highly disruptive parameter updates during the incremental training phase. To tackle this problem, this paper presented an Experience Replay (ER) based approach with asymmetric parameter update for old and new  data.  Two versions of asymmetric losses are proposed and compared: metric learning and cross-entropy based  losses. Experiments are conducted over multiple datasets to verify the effectiveness of the proposed method. 
main_review: This paper is well motivated.  Figure 1 is good to illustrate motivation.  It could be better to show the feature distribution with and without Asymmetric loss. Will the old class prototypes be pushed further away from the new ones? 

The online continual learning scheme is valid and practical in real use cases. In an offline setting, after abundant iterations to replay old data, the performance can be recovered to some extent, but at the beginning stage, the accuracy will be drastically decreased. 

Regarding the formulation of AML, the loss for buffer data is still in cross-entropy form, wondering if there is any special consideration not to design it in ML form by sampling positive and negative data. 
AML is designed to be slightly more complex (sampling of positive and negative samples) yet less effective than ACE. It uses features from live training examples instead of the class weights. Could elaborate more on their respective resultant prototype distributions with AML and ACE. 

From the experiment results, ER-AML leads to unstable performance while using data augmentation. The author can elaborate more on this.  

More choices for the contractive loss could be explored. Also, the components of the proposed methods are not new, although it may be first used in this continual learning task. 

Minor adjustment- N is used twice: "we denote the incoming N datapoints..." and "We use the P and N to denote the set of positive and negatives..."
summary_of_the_review: The overall idea of this paper is simple but effective. The motivation is well illustrated and the results are convincing. It can be improve with more insightful discussion and analysis on the outcome of the feature distribution (similar to Fig1) to show the proposed method can indeed separate the representation of old and new data along with the training.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new update rule for the continual learning setting to boost performance when novel classes are added to the data stream.
main_review: Strengths:
- The paper is clearly written and the main ideas are very well presented
- The motivation is clear and well illustrated by the experiments run in Figure 1
- Solid benchmarking effort. Many recent baselines were compared against, for a range of buffer sizes, and results reported with confidence intervals.

Weaknesses:
- It appears that the advantage of this method may start to break down when the data stream is a combination of new classes and old classes, since the old class representations would also start to be updated; is this the case? There are some preliminary experiments in Section 5.5, but I would have liked to see a much more thorough analysis - what happens when the number of examples per new class in the incoming data batch is very low? What about at a range of "blurriness" levels?

summary_of_the_review: Overall, the paper presents a strong analysis of a potential failure mode in continual learning, proposes a simple method as a fix, and compares thoroughly against baselines to validate their results.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In the online continual learning setting, representation of previous classes will change over time, especially at the task boundary. The authors present a new explanation for the abrupt representation change when applying experience replay (ER) algorithm. They claim that without sufficient training, the representation of new samples is initially mixed with that of previous classes, which causes a sharp increase of gradient resulting shift. To reduce the negative impact of negative new samples, they modify the ER by leveraging asymmetric losses on current data and replay data separately.
main_review: Pros:
+ The explanation of catastrophic forgetting from the perspective of abrupt representation drift is somewhat novel. The observation that the knowledge of old classes is recovered by stored samples after disrupted is interesting.
+ Some new metrics are proposed, such as Averaged Anytime Accuracy, Total FLOPs, which make the comparison more rigorous.

Cons:
+ Some important related methods of online continual learning are not included in comparison, e.g. [1, 2, 3]. In particular, [3] also uses a SupCon loss just as ER-AML.
+ The paper claims old tasks suffer from a significant drop in performance when learning new tasks (Fig 1 right). However, this statement is not quite convincing, since the acc drop is not entirely related to representation drift. Some previous work [4, 5] confirm that forgetting is more likely to occur at the layers near the output, especially the last fc layer. Moreover, [6] also finds that adopting metric learning on representations results in less forgetting. To make it clear that forgetting happens at the last fc layer or the whole network, I suggest using NCM classifier to calculate acc here, or another better metric.
+ The motivation of using metric loss is not clear. Compared to [3, 6], which also use metric loss for CL, the paper doesn’t provide enough special motivations or insights to use metric loss for reducing abrupt representation drift. What’s more, the idea to calculate cross entropy loss with replay batch only has already been used in [7]. The proposed losses need more theoretical explanation for why/how they work.
+ Compared to SS-IL, the improvement is not significant.
[1] Online Class-Incremental Continual Learning with Adversarial Shapley Value. AAAI 2021
[2] Graph-Based Continual Learning. ICLR 2021
[3] Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning. CVPR Workshops 2021:
[4] Large Scale Incremental Learning. CVPR 2019
[5] Learning a Unified Classifier Incrementally via Rebalancing. CVPR 2019
[6] Semantic Drift Compensation for Class-Incremental Learning. CVPR 2020
[7] Discriminative Representation Loss (DRL): Connecting Deep Metric Learning to Continual Learning. arxiv.org/abs/2006.11234

summary_of_the_review: Overall, this paper proposes an interesting phenomenon in online continual learning. However, the relation between the solution and the insights are not strong. Besides, the empirical improvements are marginal.
review_title: null

Meta-review: The manuscript develops new insights into how catastrophic forgetting takes place in the context of continual learning. The authors develop a new method based on this insight and demonstrate that it performs better than or as well as previously developed baselines, as well as showing that it is more widely applicable than close competitors (e.g. to cases where task boundary are unknown).  
The manuscript starts by pointing to evidence that catastrophic forgetting at task boundaries is due at least in large part to abrupt representation drift (e.g. in the penultimate layer of a network) caused by gradients coming from new class examples. Most reviewers found this novel and interesting. Reviewers also tended to be happy with the writing, motivation, and experimental results supporting the conclusions.
One of the reviewers recommends against publishing (3 - Reject): mCT3 cites positives in the novel explanation of forgetting and the development of new metrics (Averaged Anytime Accuracy) and Total FLOPs, which they say help make the analysis more rigorous.
However, fundamentally, they believe that the relationship between the insights and the proposed methods are not strong enough and that the methods do not provide more than marginal improvements empirically. They point to work such as SS-IL as a baseline which, in their opinion, is not improved upon significantly enough to adjust their recommendation.
The authors provide multiple effective rebuttals to the concerns, as well as detailed experimental analysis of SS-IL: 0. The new method is shown to be as good or better than SS-IL, 1. That their method are more computationally and memory efficient than SS-IL, 2. SS-IL requires task-ids, whereas they do not, 3. Detail analysis (Appendix B) shows that SS-IL mostly fails to learn the current task in the online setting in the miniImageNet case that the reviewer worries about, a fact that is obscured by the simple Acc metric. Reviewer mCT3 does not respond to the rebuttals in any substantive or compelling fashion, and leaves their score at 3/Reject.
While I believe that the reviewers concerns should be thoroughly addressed in a final version of the manuscript, I am in agreement with the 3 of 4 reviewers who recommend publication.
Decision: Accept (Poster)