Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of coreset selection for realistic and challenging continual learning scenarios. The authors proposed Online Coreset Selection (OCS), a simple yet effective online coreset selection method to obtain a representative and diverse subset.
main_review: The reviewer lists the major strengths and weaknesses as follows.

1. Strengths:
This paper is well-structured and contains sufficient experiments. Its motivation is meaningful and interesting.

2. Weaknesses:
a. The authors did not explain how big the difficulty is to adapt existing algorithms to online learning. And the way of designing the proposed algorithm for online learning seems straightforward and cannot be regarded as a genuine technical contribution.

b. The algorithm lacks novelty. The core part of the proposed algorithm seems very similar to the core part of 'Asymmetric Multi-task Learning Based on Task Relatedness and Loss'. The three adopted selection strategies do not demonstrate enough novelty, either.

c. Some grammatical errors and typos exist, such as 'let .. is' in definition 3.

summary_of_the_review: In a nutshell, the reviewer regards this paper as a borderline paper, given the limited technical innovation. 

I read the authors' rebuttal. Some of my questions and concerns are replied well. However, I think that the core technique of this work does not advance the research in continual learning significantly. So, I adhere to my previous rating.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents three gradient-based selection criteria to select the core-set for improving adaptation and reducing catastrophic forgetting. Differently from other methods, the proposed approach selects the instances before updating the model.
main_review: STRENGTHS:
1) The paper is very well written and presented.
2) The core-set problem is an important and under researched area in continual learning, especially in its online form.
3) Evaluation is very well executed.
4) The introduction of diversity as a criteria for core-set selection is interesting.

WEAKNESSES:
1) As the term “online” appears in the title of the paper, this modality should be better introduced and motivated. A clear explanation appears in the related work at the end of the continual learning paragraph. However, the term remains not properly defined and seems to be related with the problem of imbalance. 
What exactly does imbalance mean? In the reviewer understanding the term seems to be defined as in [*] and in (Aljundi 2019b), that is, task distribution is not i.i.d. Given that definition, why selecting the core-set before should provide an advantage? The reviewer understands that the performance is improved, however, the writing states that selecting the core-set before has a dependency on the imbalanced task distribution and that this is an advantage with respect to (Rebuffi et al., 2017; Aljundi et al., 2019b;a; Chaudhry et al., 2019a;b). If this is not the case and the motivation of selecting the core-set before adaptation is because of the good empirical results, then it should be better remarked in the paper.

[*] Chrysakis, Aristotelis, and Marie-Francine Moens. "Online continual learning from imbalanced data." International Conference on Machine Learning. PMLR, 2020.

2) Figure 2 shows a dataset that is not addressed by the method and is somewhat misleading. The "Multidataset"  used in the paper does not include the CIFAR dataset (i.e. complex objects like dogs and vehicles).

3) In the reviewer's opinion the claim of large improvement, included in the third contribution, seems to be somewhat a bit bold. For example, in CIFAR-100 balanced and unbalanced learning settings in Tab.1, the performance does not differ too much from the herding strategy used in iCARL (i.e. 60.3 vs 60.5, 51.2 vs 51.4). Although the reviewer noticed that in rotated MNIST and Multidataset improvements are evident, these datasets do not typically “transfer” their performance to larger datasets (i.e., ImageNet) as CIFAR-100 typically does. 

4) The validating hypothesis of Fig.3 (i.e. learning from MNIST to CIFAR10) seems to favor diversity, which is exactly what herding is not doing. In herding, examples are selected closer to the class mean of the feature representation in each class. Herding is somewhat orthogonal to diversity. This may partly explain the not improving performance on CIFAR-100. This part should be discussed in depth (i.e., motivate that the method does not mostly favor datasets with diversity). 

5) In the reviewer’s opinion the ablation study of the effect of the gradient is not sufficient to justify its usage. As also remarked in the point 3) of this review, the MNIST family datasets typically do not “transfer” their performance to more complex and bigger datasets. In other words, using the gradient on MNIST does not imply that the gradient on CIFAR or bigger datasets is a good choice.
Is the gradient computationally expensive for deeper neural networks? For example, what happens with one or two more orders of parameters?

summary_of_the_review: This is a nice and well written paper in which some details need to be clarified. Specifically, imbalance, selecting the core-set before/after adaptation, dataset diversity and selection, the "transferability" of performance of the datasets.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes an Online Coreset Selection method to select the most representative and informative coreset at each iteration and trains them. The proposed method maximizes the model’s adaptation to a target dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. Experiments on the benchmark datasets show competitive results compared with baselines.
main_review: This paper proposes an Online Coreset Selection method to select the most representative and informative coreset at each iteration and trains them. The proposed method maximizes the model’s adaptation to a target dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting.
1. The motivation of this manuscript is not clear. The authors should clearly claim the challenging issues in previous methods.
2. The authors complement the theoretical explanation of the success of the proposed approach.
3. While the online coreset selection method adopted in the manuscript seems plausible, it is not exciting.


summary_of_the_review: In general the paper is well organized and clearly written. The technical details are easy to follow. Experiments on the benchmark datasets show promising results compared with baselines.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The author propose a novel approach for online coreset selection, i.e exemplars used in the rehearsal process of past tasks in a continual learning framework. The proposed method is based on the observation that not all the samples in a dataset are equally valuable, but their quality affects model's effectiveness and efficiency. The method selects the most representative and informative samples at each iteration and trains them in an online manner. The approach has been conviently compared with state-of-the art methods and demonstrated its superiority.
main_review: Positive aspects:
- The proposed approach presents scientific novelty
- The related work section covers the most relevant papers in the field
- The experimental validation is extensive and the authors demonstrated the superiority of their approach.

Negative aspects:
- The idea is in general well-explained, although the paper lacks clarity in some aspects (see the detailed comments below), therefore it could be further improved
- There are also some issues with the proposed approach which are not clear enough
summary_of_the_review: Please find below my main concerns:
1. The following statement is not totally clear: "The naive CL design cannot retain the knowledge of previous tasks and thus results in catastrophic forgetting". What do you mean by 'naive CL design'? Please reformulate this statement.
2. Along the paper, you use repeatedly the expression 'target dataset'? What do you mean by 'target dataset' in a continual learning framework? It is confusing: If I have to learn 10 tasks, which one is the target dataset? I guess you refer to the 'current task'. Therefore, please use this formulation instead and change all over the document.
3. How many representative instances do you select from each mini batch?  Since a sample is presented several times during the training, shouldn't you use an acumulative measure and the final ranking/selection should be done at the end of the training? How do you guarantee the class-balance of the selected core-set? Please plot the distribution of samples per class resulted after the coreset selection process.
4. I did not understand the equation 3? What index 't' refers to: task ID? How is possible to measure the similarity between a sample and the minibatch it belongs to? Or you consider the similarity between a samples and the batch average? Something is missing there. 
5. Equation 4: Confusing in terms of notations and terminology! You refer as 'cross-batch', but the eq. 4 is about the similarity between samples in the same batch! 'Cross-batch' should not refer to the similarity between a sample from one batch and the samples from different batches? Why the similarity measure in eq. 4 is negative? Please reconsider the notation and formulation of eqs 3 and 4.
6. Usually, in exemplar-based CL, the memory allocated for exemplar is known and fixed from the beginning. While new tasks are being learned, the number of exemplars from previous tasks is decreased in order to keep the memory size. What strategy do you adopt in your approach regarding the memory capacity where the exemplars are stored?
review_title: null

Meta-review: The authors propose three strategies for coreset selection in the context of continual learning. In particular, the authors consider class-imbalance and noisy scenarios. The authors run extensive benchmarks and ablation showing that the approach can be effective in practice. All reviewers were positive about this work, but found that the methodological contributions were relatively modest. The clarifications provided by the authors were highly appreciated. I would encourage the authors to revise the paper to incorporate these additional details as there were a number of concepts that reviewers found were not sufficiently documented/explained and lacked clarity. I would also highly encourage the authors to explain their use of "online continual learning" as this reads like a tautology.

Finally, I would like to ask the authors to reflect on their insistance with the reviewers; while we would all want engaging and long discussions about our work, the reality is that reviewing papers and discussing them is time consuming and taxing, especially in the middle of continued pandemic. The authors should be grateful of the time reviewers have spent reading their work and providing feedback, and it is not in the authors' interest to ask for a revision of the scores.
Decision: Accept (Poster)