Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper presents a method called Retrospective Adversarial Replay (RAR) to synthesize adversarial examples from the buffered data that are close to the current data in the latent space. Thus, the decision boundary between the previous and current task got refined to mitigate forgetting. The authors further propose MixUp-based variant for the small-buffer regime. Experiments on class-incremental benchmarks demonstrate the effectiveness of the proposed method.
strengths_and_weaknesses: Strengths:
- The paper is well-written and easy to follow.
- Introducing adversarial examples in CL is interesting.
- The proposed method is easy to be combined with retrieval-based replay methods.

Weakness:
- Strength 3 is actually a weakness considering the generalizability of the method. It seems that stage 1 depends a lot on the retrieval-based strategies, if the selected samples are not close to the boundary, then the following stages will not further generate effective adversarial examples.
- The experiments part seems insufficient. See questions for more details.
questions: 1. As far as I understand, the proposed method could be treated as a data-augmentation method on the buffered data. Although it improves upon a certain sets of replay methods in table 1 and 2, and several data perturbation methods in table 3, I wonder how it performs agains some other state-of-the-art replay based methods, such as DER [1], Co$^2$L [2], etc.

2. The motivation of only evaluating the proposed method under the class-incremental scenario is not that clear. What about other CL settings such as domain-incremental learning?

It is important to see if the proposed method as a general replay-based method that outperforms prior state-of-the-arts, or a more incremental adds-on to a limited category of methods.

[1] Buzzega, Pietro, et al. "Dark experience for general continual learning: a strong, simple baseline." Advances in neural information processing systems 33 (2020): 15920-15930.
[2] Cha, Hyuntak, Jaeho Lee, and Jinwoo Shin. "Co2l: Contrastive continual learning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
limitations: I did not find a clear section (or pointers to) describe the limitations of the work. The potential negative societal impacts are not discussed in the paper.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper aims at overcoming catastrophic forgetting in continual learning. Catastrophic forgetting is the main challenge in continual learning. This paper propose a method called Retrospective Adversarial Replay (RAR) to address the catastrophic forgetting. 
strengths_and_weaknesses: Strengths:

(a) This paper tries to use adversarial samples to defy the catastrophic forgetting in continual learning. It is novel.

(b) This paper provides extensive experiments and the proposed method seems to be effective.

(c) This paper is well-written and easy to read.

Weaknesses:

(a) I am concerned about the idea of this paper. I summarize my views in Questions. If authors provide reasonable and clear explanations, This paper is worth to being accepted.
questions: (a) The proposed method seems to be effective in the experiments. But I'm not sure that the proposed method would also work in theory. I try to summarize my arguments. If i made some mistakes, please point them out and give some reasonable explanations. (1) The data of different tasks should follow different distributions, which often makes the data of different tasks far away from each other. Therefore, the permuted data may not lead to a great improvement. (2) In my view, the permuted samples are close to current task's samples in the latent space. Therefore, the model is not trained with the samples which replay previous tasks' knowledge---adversarial samples perform similarly to the current task's samples in the latent space.

(b) Why do authors use look-ahead parameters $\theta^{\prime}$ instead of the current parameters $\theta$ to generate the adversarial samples? To my knowledge, the generation of adversarial samples depends on the model's parameters. Therefore, the adversarial samples generated by the model with look-ahead parameters $\theta^{\prime}$ can not equal to the adversarial samples which should be generated by the model with $\theta$. Please provide a reasonable explanation.
limitations: This work does not seem to have a great negative societal impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a novel approach to tackle the classifier bias problem in the context of continual learning. For this purpose, the approach (called RAR) focusses on the examplars that are close to the the forgetting border. RAR perturbs the selected exemplars towards the closest sample from the current task in the latent space. By replaying such samples, RAR is able to refine the boundary between
previous and current tasks, hence combating forgetting and reducing bias towards the current task. Moreover, the authors propose to combine RAR with the mix-up technique which significantly improves continual learning in the small buffer regime. RAR is a generic approach and could be combined with any experience replay method. 
strengths_and_weaknesses: Strengths:
- the paper is in general clearly written
- the approach is well-motivated and scientifically sound
- the review of the state of the art covers most of the related works
- experimental results are extensive and demonstrate the superiority of the proposed approach
- a solid analysis of the proposed approach is provided (sensitivity analysis of the hyper-parameters, ablation studies, etc.)

Weaknesses:
- some aspects needs to be clarified 
- some mathematical notation produces confusion
questions: Here are my concerns:
- Equation 2: It should be y(X'_M) instead of y(Y'_M)
- When you perturb a sample from the X'_M what label do you assign to it? The label of the closest sample in X_i?
- Section 3.3: I found it confusing that you refer to x_M as a sample from X'_M. Why do not use the x'_M notation instead? I think it will improve the clarity. Same observation for Algorithm 1
- Algorithm 1: 'S' should be also mentioned in the input 
- Do you maintain a fixed size for the buffer? Are the previous classes uniformly represented in the buffer? 
- What is the ratio in the mini-batch between the exemplars and the samples belonging to the new task? I think performing a study of different ratios would be interesting.
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Authors propose a method to mitigate Catastrophic Forgetting named “Retrospective Adversarial Replay (RAR)”, which synthesizes adversarial samples near the forgetting boundary. The method perturbs a buffered sample towards its nearest neighbour drawn from the current task in latent representation space. In particular, the authors propose Retrospective Aversarial replay Loss that finds a pairing between memory and novel task samples such that memory samples that are perturbed towards the paired current-task samples is maximum. By replaying the perturbed sample with the original label authors prohibit novel tasks boundaries to interfere with the previous tasks boundaries. By these means, they refine the boundary between previous and current tasks and reduce bias towards the current task. In addition, the authors propose also a mix-RAR strategy that utilizes MixUp augmentation on memory samples that are later perturbed using RAR. The proposed modification improves the class boundaries refinement and reduces the final average forgetting.

The main contribution of the papers is an introduction of a novel interesting strategy for data retrieval from the replay buffer that is based on the combination of past and present data. The authors evaluate the method on top of other buffer-retrival-related methods and show consistent performance gain in terms of averaged accuracy on a number of datasets. Additionally, the method outperforms another buffer samples perturbation method called Gradient-based Memory EDiting Examples (GMED) and random perturbation baseline.
strengths_and_weaknesses: Originality:
	The proposed method introduces a novel improvement for the memory buffer methods with the inspiration drawn from psychological phenomena called “false memories”. The method differs from the other memory perturbation buffer replay method GMED to which it directly compares by calculating differently the perturbation direction. Specifically, in RAR memory examples perturbation is in the direction of the novel tasks' decision boundaries. In contrast, in GMED authors edit examples so that they are more likely to be forgotten in future updates.

Quality:
	The experimental results clearly support the effectiveness of the method. RAR outperforms random perturbation of the memory examples and GMED in the scenario where RAR, random perturbation and the memory examples are used with Experience Replay (ER) and Maximally Interfered Retrieval (MIR). Additionally, RAR on top of other memory-retrieval methods such as ER, GSS, ASER, and MIR results in improved averaged accuracy and reduces forgetting. The method’s effectiveness is shown on four datasets with different resolutions.

Clarity:
	The method and its effectiveness are plainly explained with an intuitive illustration, brief formulas, and results section that provides a decent comparison with other methods. The overall structure of the paper is well organized and the comparative study is appropriately conducted.

Significance:
	The authors address the problems with ER, namely memory records selection and their perturbation in a clever way, considering the decision boundaries of the tasks that are known to degenerate in continual learning. Even though this kind of decision boundaries analysis is not new it is the first application of this technique in the memory replay methods.

Strengths:
- The proposed method invariably outperforms other methods based on the perturbation of buffer samples in terms of forgetting mitigation.
- The method is deeply grounded in our current understanding of catastrophic forgetting i.e. that the decision boundaries in classifier models deteriorate on every new task. 
- Authors propose also a MixUp-based strategy to increase replay variation
- Authors provide extensive ablations about their method to prove the correctness of it at different steps, such as Figure 2, for the step of distance calculation between samples from the buffer and from the novel tasks.

Weaknesses:
- Authors could provide quantities of how their improvements from sections 3.2 and 3.3 reduce the computational cost.

questions: 1) Was another kind of replay samples augmentation also tested such as cropping/noise injection/random rotations?
2) Did this kind of augmentation provide noticeable improvements?
limitations: The work does not exhibit potential negative societal impact. The authors are aware of the limitations of their work and address them accordingly.
review_title: null

Meta-review: metareview: This paper proposes adversarially perturbed samples from a replay buffer to simulate examples that are on "forgetting boundary" in the continual learning setting. After reading the paper, I found it interesting and insightful: the methodology makes sense, the analysis of their method is thorough, and the results validate their method. The reviewers asked some good questions, and I believe the authors did a good job at answering them sufficiently.

I therefore recommend the paper for acceptance. We ask however that the authors fix first line in the abstract which seems to suggest that all CL methods use a memory buffer.

I'm disappointed the reviewers did not participate more: they were reminded both my me as well as the authors about the discussion, yet no one participated after they gave their initial, albeit useful, signal.
confidence: Less certain
Decision: Accept