Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel dual memory experience replay method to store the knowledge of previous tasks. In addition, long-term and short-term semantic memories are leveraged to replay the neural activities of the episodic memories and align the decision boundary. 
main_review: 
Strengths:
1, Semantic memory is the first leveraged in incremental learning, which is interesting in my opinion. The contribution of this paper is novel.
2, The codes are released, which guarantees the reproduction of the proposed method.
3, The experiment results prove the effectiveness of the proposed method, which obtains impressive performance.

Weaknesses:
1, Can this method be applied to online continual learning task?
2, The experiments for domain-IL are only evaluated in one dataset. Experiment results on more datasets should be shown in this paper.
3, For Class-IL, some experiments with different base classes should be shown in this paper.

summary_of_the_review: In my opinion, the contribution of the proposed method is novel and interesting. In addition, the writing of this paper is clear and easy to understand. The experiments are sufficient to prove the effectiveness of the proposed methods. However, more experiments should be designed to prove the proposed method further.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents CLS-ER, a dual memory mechanism for the continual learning setting. During training the model receives data from a non-i.i.d. source as well as random samples from a sample buffer (the episodic memory). The stable and plastic models act as teacher models. Their logits are the target of the student model. Which one is used as a target depends on the highest softmax score w.r.t. a ground trough class. The teacher networks are both exponential moving averages of the previous student models. Thus, the only difference between the teacher models "plastic" and "stable" is their EMA rates which in the case of the plastic model is larger such that it focuses on more recent weight updates.  The slower learning of the stable model is the mechanism which ought to mirror the consolidation of structural knowledge. It is used from inference Samples from the datastream are transferred to the sample buffer using reservoir sampling. 

The authors experiment on image datasets (variants of MNIST, CIFAR10, Tiny-ImageNet, see the Mammoth framework) with data augmentation (random flips and crops). They compare with several prio works and the evaluation is thorough and convincing. 
main_review: I enjoyed reading this publication. I'm not very familiar with the latest CL methods but the authors address various related works and evaluate all models thoroughly. I think the setting without task boundaries is harder and more interesting since the real world also doesn't come with clear task boundaries. The paper is following recent work towards robust and extensive evaluation which is more difficult but will help the CL community to make real progress. 

The empirical upper bound "JOINT" is very instructive and highlights that there is still room for progress. That said, the JOIN performance seems low given the excellent performance of other models on MNIST, CIFAR, Imagenet datasets. Is this because the models are very small? Does CLS-ER work also with larger models?

The flat mima discussion is less essential but is probably missing a reference to Flat Minima, Hochreiter et al 1997.

Instead of the interesting section 6 I'd have preferred some ablation experiments (though the component analysis is appreciated). E.g. are two teacher networks necessary? How much worse is the model if one teacher network is used (with a tuned rate)? What if there were three networks (each with their own rate)? 

I did not understand how samples are removed from the buffer (the episodic memory). Are samples removed eventually? If yes, how and where is that explained? If not, what is the space complexity of algorithm 1? 
summary_of_the_review: I think this work is good in general. The writing is great, the evaluation seems thorough, and comparisons look fair. The method is easy to understand and the empirical results are convincing to me. The authors discuss recent and related works and their method seems original but I lack the expertise to judge the novelty of the approach. It builds significantly on previous work (DER) so I'd argue that some aspects of the publication exist in previous work.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: They propose a dual memory experience based on complementary learning systems (CLS).
The working model is updated using consistency loss with the selected optimal semantic memory from plastic and stable models.
Plastic and stable models are updated with an exponential moving average trick (EMA) and the working model.
main_review: The method is very simple to understand and mimicking the CLS theory with the hippocampus and neocortex is interesting.

Compared to existing methods, is it a fair comparison in terms of training cost such as using two models including plastic and stable models and using reply data to train the working model?  

I think more analysis about plastic and stable models are needed because these model act like hippocampus and neocortex based on CLS theory. What is optimal semantic? Just select the logit with the highest softmax between the two models? 
summary_of_the_review: I am not an expert in this domain and have not much experience with related works. However, the idea is very simple and reasonable and the results outperform previous existing methods. However, more analysis is needed for the claim.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, motivated from the CLS theory in neuroscience that efficient learning requires short-term adaptation and slow learning of structured information, the authors propose a novel dual memory experience replay method for continual learning. The idea is to build  long-term and short-term semantic memories by maintaining another two models: plastic model and stable model. The plastic model is used for fast learning of recent experiences and the stable model is used for slow learning of structural knowledge. The two models are updated with a Mean Teacher fashion during training with different frequencies. The proposed CLS-ER is  task-agnostic and can be applied in different continual learning settings. The experimental results show that the proposed CLS-ER outperforms several baselines for continual learning. 
main_review: Strengths:

1. The proposed method is well motivated and novel in the context of continual learning. Maintaining multiple models for different purposes is interesting and is shown to be helpful for continual learning.

2. The writing and organization is clear. It is easy to understand the proposed algorithm and the details.

3. Different continual learning settings are considered, the proposed approach shows better results across all the settings. The analysis of different models also make it easy to understand how the method works.


Weaknesses:

1. One of the main drawbacks of the proposed approach is that three models need to be maintained during training which incur a heavy cost.   One question is that if reducing the individual model size while keeping the whole model size the same, what's the performance in this case?

2. The proposed method has a lot of hyperparameters to tune, which may be a problem if the method is applied to a different task. 


More questions:

1. Why update the plastic and stable models stochastically?

2. In table 1, why CLS-ER is better than JOINT on S-MNIST? JOINT should be the upper bound performance?

3. The authors make a connection to flat minima, if explicitly enforcing flat minima, would the proposed method can still do better?
summary_of_the_review: In this paper, a novel idea of continual learning is proposed. The authors clearly describe the proposed approach and also give a detailed analysis. Overall, it is a well-written paper with an interesting idea. The results also show the benefits of the proposed approach.
review_title: null

Meta-review: The manuscript proposes an experience replay method that supports two time scales of memory, as in complementary learning systems from the cognitive sciences literature. The authors demonstrate that their method on a wide range of benchmarks and after the rebuttal demonstrate it on one additional benchmark.
The reviewers raised a number of questions and concerns around additional experiments (benchmarks and ablations), online training, the cost of training, number of hyperparameters, further analysis, clarifications, and citations. The authors address the majority of these concerns during the rebuttal period, and overall the reviewers were in favour of acceptance. Therefore, I recommend acceptance of this work.
Decision: Accept (Poster)