Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper argues that when the number of tasks are increased in a continual learning setting, catastrophic forgetting ceases to be a problem and a simple SGD training with gradient masking keeps on accumulating knowledge. Towards this, the authors propose a framework, called SCoLe, where the authors repetitively sample tasks from a fixed number of classes of small datasets. The authors show that the full test set performance increases with the number of tasks, and that if a training setting moves away from the IID training, for example by increasing the number of epochs per task or the total number of classes in a dataset, the test set performance remains low. Overall, the proposed claims are not well-justified and the training setting looks trivial – closer to IID training. 

strength_and_weaknesses: **Strengths**

1. The premise of studying continual learning in the presence of large number of tasks and drawing conclusions from it is a worthwhile goal. 
2. Overall the paper is well-written and easy to follow. 

**Weaknesses**

1. **Contrived setup**: Overall the setup seemed very contrived and does not justify the claims regarding catastrophic forgetting made in the paper. The authors took small datasets MNIST, CIFAR10 & 100 etc, and repeatedly sampled large number of tasks (up to 1000) consisting of small number of disjoint classes, sometimes even 1 class per task, repeating classes between the tasks, and measured the trajectory of test performance of that dataset (on all classes). It is quite intuitive to think that in such a setup if you control the number of epochs per task properly and keep the total number of classes low, you’ll closely approximate the IID training and your test set performance would increase overtime (barring overfitting). But this doesn’t tell you anything about the catastrophic forgetting. It is quite intuitive that if you keep on observing new classes (along with repeatedly observing previous classes) your overall test performance would keep on increasing. 

2. **When the setup is closer to IID training there is less catastrophic forgetting, when the setup is far there is more CF**: Even in the given setup, if one closely looks at the experiments, it seems that when one is closer to the IID setting, the overall test set performance remains high. Similarly, if one moves away from the IID setting, the test set performance degrades. For example, in Fig 7 when the authors compare single-epoch (closer to IID) vs multi-epoch training (farther from IID), and experiments are slightly challenging CIFAR100 (N=50) there is a significant gap between the single- vs multi-epoch performance. It seems that only in very specific situations, when the total number of classes are small and the SGD steps per task are small, or when the setting is closer to IID, the test set performance increases with the number of tasks. This makes the overall results less interesting and trivial.


3. **Measuring CF through test set performance**: The notion of forgetting is not clear in this work. What the authors measure is the full test set performance and show that with the data/ task repetition that performance increases. If the data is removed from the mix, cf. Fig 6, the performance of those classes degrades. All this one can observe in a single-task multi-epoch training as well. Perhaps the authors could properly define the notion of forgetting. For example, measuring the performance of a task when it is seen again and plotting that over time.

4. Overall, the learning setting seems contrived and experiments do not justify the claims made in the paper.  Some results on increasing the width resulting in less forgetting (Mirzadeh et al.), and warm starting decreasing the generalization performance (Ash & Adams) are already well-known in the literature. 

clarity,_quality,_novelty_and_reproducibility: While the writing is clear, the paper lacks the novelty. Please refer to the strengths and weaknesses section above for details.
summary_of_the_review: Please refer to the strengths and weaknesses section above for details.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The main claim of the paper is that it challenges “common assumptions about catastrophic forgetting”. The premise for this claim is that a common assumption is that catastrophic forgetting equals forgetting everything and that therefore knowledge accumulation is not possible. The main result of the paper is experimentally disproving this “common assumption” by demonstrating that catastrophic forgetting does not necessarily equate to forgetting everything and that knowledge accumulation is possible.
strength_and_weaknesses: I found this paper an interesting read. I think considering continual learning setups where tasks or classes re-occur is a promising research direction (although in itself not novel, see some references below) that could indeed lead to important new insights. But I’m afraid I do not think the current paper provides substantial new insights.

The main claim of the paper is that it challenges “common assumptions about catastrophic forgetting”. The premise for this claim is that a common assumption is that catastrophic forgetting equals forgetting everything and that therefore knowledge accumulation is not possible. The main result of the paper is experimentally disproving this “common assumption” by demonstrating that catastrophic forgetting does not necessarily equate to forgetting everything and that knowledge accumulation is possible.

It surprises me that the authors consider it to be a common assumption that catastrophic forgetting equals forgetting everything. I do not agree with this premise. Let’s consider a simple example. This assumption means that after training on a new task, information of past tasks is no longer contained in the network. In other words, a network first trained on task 1 and then on task 2, should have the same task 1 performance as a network only trained on task 2. It is clear that in general this is not the case. (Except perhaps for class incremental settings, but this can be remedied with a simple trick: see for example https://arxiv.org/abs/2106.01834, which the authors are aware of.)

With this premise gone, the paper loses its fundament and main contribution.

In Fig 3 the authors make an attempt to convince the reader that the assumption that catastrophic forgetting corresponds to forgetting everything is common / reasonable, but this example is problematic (in fact, it might even disprove the point the authors try to make).
In Fig 3A, it is shown that when the authors train a standard neural network in a standard way on their protocol, there is no increase in performance over time. The authors say this result is in accordance with “the common catastrophic forgetting phenomenon”. But it actually is not. If always everything would be forgotten except for the current task, performance should be constant near 20%. Performance however drops to 10%. I expect this might be due to the learning rate being too high and the model collapsing onto one particular class.

Finally, I’d like to encourage the authors to take note of the following papers that also consider continual learning setups where tasks or classes re-occur:
- Stojanov et al. (2019, CVPR) https://openaccess.thecvf.com/content_CVPR_2019/html/Stojanov_Incremental_Object_Learning_From_Contiguous_Views_CVPR_2019_paper.html 
- Cossu et al. (2022, Frontiers AI) https://www.frontiersin.org/articles/10.3389/frai.2022.829842/full 

Especially the first paper seems to have results that allow analogous conclusions to those presented in the current paper.

clarity,_quality,_novelty_and_reproducibility: The clarity of the work is generally good, and I expect the reported results to be relatively easily reproducible.

As described above, the main issue with this paper is in terms of quality / novelty. The paper claims to make a surprising empirical observation, but purely based on a simple thought experiment this observation can be predicted. Moreover, there exist prior work (e.g., Stojanov et al., 2019, CVPR) with results that allow analogous conclusions.
summary_of_the_review: I found this paper an interesting read. I think considering continual learning setups where tasks or classes re-occur is a promising research direction (although in itself not novel, see some references below) that could indeed lead to important new insights. But I’m afraid I do not think the current paper provides substantial new insights.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: 
The paper studies Continual Learning (CL) under a specific framework called SCoLe (Scaling Continual Learning). The major difference between SCoLe and CL is the sparse and controllable appearance of previous tasks and data when the number of tasks is very large. In addition, the paper proposes under the SCoLe framework, DNNs trained with gradient descent only accumulate knowledge without any supplementary CL mechanism. 


The authors provide various designed experiments to study knowledge accumulation via different optimization methods and the impact of data shift. They show that continual fine-tuning yields knowledge accumulation under many scenarios (especially online learning scenarios) when previous data reoccur. 
strength_and_weaknesses: 
**Strengths:**

- I believe the SCoLe framework is interesting, and showing that DNNs are capable of knowledge accumulation for a long time is a novel contribution.

- The experiments are carefully designed and can support the claims of the paper. 




**Weakness:**
- The main assumption of the SCoLe framework is not aligned with the typical CL assumption of not revisiting the past. In many real-world scenarios, the same data will not reoccur in the future. Hence, I believe it would be interesting if the authors could modify at least one experiment to validate their hypothesis on a more challenging benchmark. For instance, the authors could augment the images of each task differently and, by controlling the augmentation, measure the knowledge accumulation under various settings. I believe this experiment is crucial for the paper.


**Questions:**
- Could the authors explain the "gradient masking" mechanism more clearly? Isn't masking the gradients based on the classes similar to having a multi-head classifier (at training time)?
clarity,_quality,_novelty_and_reproducibility: 
Overall, the paper is well-written, and the claims are supported by experimental results. While the authors do not provide code, they have explained the experimental setting clearly. In terms of novelty, the paper provides several interesting contributions regarding the knowledge accumulation in continual learning. However, I find breaking the no-revisiting assumption of continual learning too relaxing.

summary_of_the_review: Overall, while I find some assumptions of the SCoLe framework not realistic, I believe the work provides an interesting contribution regarding knowledge accumulation in continual learning. The contributions could have been stronger if the authors had validated their hypothesis under a more realistic benchmark (please see the weakness section).
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a new evaluation setting for Continual Learning in the setting of large number of tasks with recurring tasks. The authors propose SCoLe, an experimental framework for this setting and report their findings throughout the manuscript. The key claimed insight is the observation that Continual Learning is possible in a specific setting (reoccurring tasks, SGD without momentum, masking of output layer) without further restrictions.
strength_and_weaknesses: Weaknesses:
- The manuscript does not contain any new or unexpected insights. The fact that task re-occurrence lessens the problem of Continual Learning is not only well known but arguably the main reason that a large part of the CL literature is dedicated to rehearsal and pseudo-rehearsal techniques (i.e. the deliberate enforcing of re-occurrence, proposed as early as 1995). The authors seem to be aware of this "The reoccurence of data could be compared with the action of a replay methods" but do not carry out such a comparison. It is expected that a suitable replay technique would perform significantly better, especially in the more difficult SCoLe settings (e.g. Figure 5 left). It is also worth mentioning that scheduled task reoccurrence did indeed feature in highly cited earlier CL publications (e.g. [1][2]) but was regularly critiqued and does disappeared from the literature unless present in real CL problems (e.g. language modelling). Furthermore, the masking of the output layer is an existing technique well known and discussed in other works. 
- The abstract does not clearly mention re-occurrence in the data distribution, merely "long sequences of tasks sampled from a finite environment" initially leading me to believe the authors found a profound insight that questioned the entire existence of the catastrophic forgetting phenomenon. This should be rephrased so as not to be misleading. 
- SCoLe itself can barely be considered a contribution and would in most other submissions be merely considered the specific evaluation setting used in this work. 

[1] Kirkpatrick, James, et al. "Overcoming catastrophic forgetting in neural networks." Proceedings of the national academy of sciences 114.13 (2017): 3521-3526.
[2] Schwarz, Jonathan, et al. "Progress & compress: A scalable framework for continual learning." International Conference on Machine Learning. PMLR, 2018.

clarity,_quality,_novelty_and_reproducibility: - Clarity: While the paper is organised in a somewhat non-standard manner, it is relatively easy to follow, reading more like a report of experiments carried out subsequently. Graphics are of limited quality however, requiring them to be read on a computer so one can zoom. Labels occasionally overlap with data (Figure 5 left) or are not centered or even out of the page margin (Figure 3 top). Figure 4 (c) has a double grid structure (in the background) and manually drawn black lines. Numbering is inconsistent (sometimes plots are labelled (a)-(z), sometimes there is no labelling). Gives an overall impression of a rushed piece of work with little attention to detail. 
- Reproducibility: I have no concerns about reproducibility. 
- Novelty: No new insights.
summary_of_the_review: This paper simply states a well-known observation and produces no new insights.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper investigates the behavior of DNN in continual learning of long task sequences. The authors first develop a tool called SCoLe to generate CL setting, then employ the tool to conduct CL experiments with SGD. They discover a phenomena that's not presented in CL of short task sequence -- catastrophic forgetting does not happen and knowledge accumulates over time when classes and tasks repeat in the future and gradient of unseen classes are masked.

Strengths:
- The authors developed a tool SCoLe that can be used for studying the behavior of DNN in continual learning of arbitrary long task sequences.
- They present a new phenomena not shown in shorter ones.
- It's well-written.

Weaknesses:
- The main assumption of the SCoLe framework is not as challenging as the typical CL one of no data/task re-occurrence. In many real-world scenarios, the same data will not reoccur in the future.
- The discovered phenomena does not necessarily happen in real CL scenario.
- Similar CL setting has been studied before:
Stojanov et al. (2019, CVPR)
https://openaccess.thecvf.com/content_CVPR_2019/html/Stojanov_Incremental_Object_Learning_From_Contiguous_Views_CVPR_2019_paper.html
Cossu et al. (2022, Frontiers AI) https://www.frontiersin.org/articles/10.3389/frai.2022.829842/full
- The paper does not provide new insights.

justification_for_why_not_higher_score: The main contribution of this paper is a tool called SCoLe that can be used for studying the behavior of DNN in continual learning of arbitrary long task sequences. 
The reviewers shared the concerns: the main assumption of the SCoLe framework is not as challenging as the typical CL one of no data/task re-occurrence; similar CL setting has been studied before, and the paper does not provide new insights. 
The authors' rebuttal failed to address the above concerns.
Although the setting is interesting and could lead to new insights of CL, I think the paper is not ready for publication in its current form due to the weaknesses listed in the summary.
Decision: Reject