Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Inspired by Complementary Learning Systems (CLS), the paper proposes a framework for performing continual learning which is composed of self-supervised learning stream (aka slow learner) that conducts representation learning and of supervised learning stream (fast learner) that attains knowledge from labeled samples via adapting slow learner's representation. The DualNet outperforms other methods on CORE50 and miniImageNet.
main_review: (+) The paper considers both multi-head and single-head (no task identifier) evaluations.

(+) The paper offers a concrete philosophy for continual learning. DualNet is based on CLS which is what CTN was based on.

(-) There are many criterion for choosing the appropriate SSL. It would be nice to have performance comparison between the different SSL and discussion as to why one would perform better than the other. Table 2 compares only to SimCLR but not SimCLR v2, BYOL, MoCo.

(-) The proposed network is evaluated on only two dataset whereas many continual learning papers evaluate their algorithm on at least 3 (Permuted MNIST Alternating CIFAR 10 and CIFAR 100, Multiple datasets learning). The proposed network should be validate for much wider set of datasets.

(-) There are many reference that are missing in the paper. Sebastian Lee · Sebastian Goldt · Andrew Saxe, Continual Learning in the Teacher-Student Setup: Impact of Task Similarity ICML 2021 Abhishek Kumar · Sunabha Chatterjee · Piyush Rai Bayesian Structural Adaptation for Continual Learning ICML 2021
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a model for continual learning based on example replay. Motivated by the Complementary Learning Systems theory in neuroscience, the authors propose to use, instead of a single backbone network, two different models: the slow and the fast learner. The slow learner is composed of a standard backbone architecture, such as ResNet, and it is optimized via self-supervised learning with examples from a memory buffer. The fast learner performs an adaptation of representations provided by the slow learner (based on masking) and embeds a classifier. The fast learner is supervised with class labels. Experiments and ablation studies are carried out on two datasets, namely miniImagenet and CORE-50, both in the presence and in the absence of task labels at inference time.
main_review: The paper seems solid and it is very well written. 
Most equations seem correct and visuals help understanding the modeling choices.
The idea of exploiting self-supervised learning in continual learning is fairly new to my knowledge, and so it is the idea of using multiple networks with different objectives.
The experimental results seem really encouraging.

However, I have some doubts about the experimental setting and the validation of the proposed components that I think are worth raising:
1) Not all the components hereby introduced are properly validated. Specifically, the feature adaptation transforming representations from the slow learner to the ones of the fast learner (eq. 5 and 6) is also not properly validated. How can we be sure this step is necessary? I think a good baseline could be using features from the slow learner directly, without any adaptation, and applying a classifier on top of them.
2) I am unsure why the authors decide to optimize only one epoch per task. In my experience, that sort of constraint (unfortunately fairly popular in CL literature) introduces some sample efficiency problems and confounds them with catastrophic forgetting. As such, methods that are more sample efficient can perform better, but not because of their continual learning capabilities. Similar considerations are also discussed in the DER paper [1] (appendix, section F.3). In the context of this paper, this issue is even more serious as the training involves some optimization choices that might improve sample efficiency. Specifically: i) the look-ahead classifier, requiring more forward-backward updates for every example (boosting the performances significantly, Tab. 2) and ii) the fact that the self-supervised learning objective is optimized n times (with n=3 in the comparisons with state of the art). 
This is further testified by the fact that DualNet outperforms Offline training, that is considered the upper bound in continual learning settings (Tab. 2 in supplementary material). 
In summary: to enable a fair comparison, is the number of updates performed by DualNet the same as the one allowed to competitors such as ER, DER, or CTN? Or even better, are the positive results consistent in longer optimizations, where catastrophic forgetting is even more threatening and sample efficiency problems are ruled out?
	
Equations:
3) in eq. 3, the plus should be a minus, as L_BT is a loss term that requires minimization. Also, there shouldn't be a learning rate involved in the look-ahead phase? Do the authors use a learning rate of 1?
4) Eq. 7 should be revised. pi(x) denotes a softmax over the input, without any function being involved. Also, what is the k index in \hat{y}_k referring to?

5) If classification is used to train the slow learner, what is the difference with respect to DER++? There seems to be a significant gap from Tab. 2 (classification-SGD, 67.50) against DER++ in Tab. 1 (62.32). Can the authors provide an intuition for this gap? The two methods seem pretty similar to me in formulation.
6) the authors may consider discussing Meta Experience Replay [2] in their related work, as the REPTILE algorithm [3] employed in such a paper is similar in spirit to the look-ahead optimization used hereby.

Overall, I consider this a borderline submission. I appreciate the idea of using self-supervised learning on the buffer, that seems to bring benefits. However, I have several concerns about experiments, the one about sample efficiency being the most critical.

---
EDIT: after discussions with the authors, I will increase my score from 5 to 7.

References:
[1] Buzzega, Pietro, et al. "Dark experience for general continual learning: a strong, simple baseline." NeurIPS 2020.
[2] Riemer, Matthew, et al. "Learning to learn without forgetting by maximizing transfer and minimizing interference." ICLR 2019.
[3] Nichol, Alex, Joshua Achiam, and John Schulman. "On first-order meta-learning algorithms." arXiv preprint arXiv:1803.02999 (2018).

limitations_and_societal_impact: Yes, there is a discussion in the supplementary material.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This work proposes an Architectural-based method for online continual learning, suitable both for the Task-aware and Task-free scenarios. Inspired by biologic suggestions, the author separate a continual learner in a slow network (which learns generic representations in a contrastive fashion) and a fast network (which utilises the feature generated by the former to produce a classification response).
Experiments show state-of-the-art accuracy in the presented setting.
main_review: Strong points of this work:
- I find it clearly written and easy to understand (with all the information and the pseudo-code provided in the supplementary material, I feel confident I could re-implement this work from scratch);
- I find the idea to combine a separated representation learner with a lean feature adaptation model to be both novel (other works propose dual structures, but not in this way) and convincing.
- The results include a very up-to-date roster of competitors, although not necessarily fair as I explain below.

Weaknesses and suggestions:
- The main issue that could be brought up about this paper is that competitors are not as sample-efficient as the proposed method. If I understand correctly, in its slow-learning branch, DualNet can optimise memory data repeatedly between batches from the continual input stream, while ER and DER++ only apply one update for each input batch (as they are not necessarily designed with the online setting in mind - Buzzega et al. perform multiple epochs on the same task). It could be interesting to add more sample-efficient competitors, which should be especially effective in the online scenario, e.g. MER [Riemer et al, 2019 in the paper] and GSS [1]
- While I found the ablation section interesting, I feel that reproducing the same experiments in a task-free setting would make the results of Tables 2,3,4 much more interesting. In general, the task-aware setting is considered to be simplistic and less indicative of good performance w.r.t. the task-free one [1, 2].
- A very minor point, one of the first CL models that came to my mind while reviewing this work is Fearnet [3], which is not cited and also borrows from similar biological premises. However, the latter is not necessarily an up-to-date reference, so I only indicate it here as a suggestion.

The following points did not affect my judgment, but I feel that they should be fixed:
- The first citation of Kirkpatrick et al., 2017 at line 34 seems out of place, as EWC is not a method that uses an episodic memory.
- The citation style for this paper is not the one that is found in typical NeurIPS 2021 submissions, namely using increasing numbers between brackets [X].
- The following typos could be fixed:
line 158: e.g. seems to be followed by a slightly larger space (the space after the last period might need escaping)  
Equation 7: what does index k denote?  
line 191: missing space between estimations and the citations  
line 255: there's a misplaced u character after the citation  
Table 2: The results for Barlow Twins, (SGD, ACC) have /- rather than \plusminus  
Table 3 caption: awarae -> aware  
Table 4: \rho=10 should have a percentage  
line 327: to label all of them --> to entirely label  

[1] Aljundi et al., Gradient based sample selection for online continual learning, NeurIPS 2019  
[2] Farquhar et al., Towards Robust Evaluations of Continual Learning, ICML workshop 2018  
[3] Kemker et al., FearNet: Brain-Inspired Model for Incremental Learning, ICLR 2018


--- AFTER REBUTTAL ---

I thank the authors for engaging in the rebuttal process. I especially found the experiments provided as a response to reviewer RowY to be interesting and meaningful, so I strongly encourage the authors to include them in their final revision for this paper.
I confirm my previous rating and recommend acceptance of this work.
limitations_and_societal_impact: This is theoretical Continual Learning work that only refers to in-the-lab classification problems. I believe that the limitations of such a work are implicitly defined by the specific evaluation setting detailed in sec. 4.1. Discussion of societal impacts are not applicable due to the abstract nature of this work.
review_title: null

Meta-review: This paper provides a dual-learning mechanism that uses a novel combination of a separated representation learner with a lean feature adaptation model.  The reviewers found their approach convincing and that it performs well against current competitors. They also found this paper to be well written, and after discussion on various technical aspects, quite compelling.  The one slightly negative review provided little justification for their points and chose not to engage with the authors after the authors provided a detailed rebuttal against the points raised in the more negative review.
Decision: Accept (Poster)