Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: As the title suggests, the paper is a comparison of recent continual learning methods that prevent catastrophic forgetting and their effectiveness in some text classification tasks using popular pretrained language models such as BERT, RoBERTa, etc. The paper divides continual learning methods into three categories: (1) rehearsal-based, (2) regularization-based, and (3) dynamic architecture. The experimental results show that rehearsal based methods are superior to the other two, and also that BERT is generally better than other candidates. The paper then proposes a new probing techniques to find out what makes rehearsal-based method better and what's happening inside BERT. The paper finds that the last layer has the biggest catastrophic forgetting and lower layer is less impacted.
main_review: I think a comparative study paper should suffice at least two conditions to be considered for a publication at a venue like ICLR. First, it should present a novel view on the problem, and second, it should draw a novel conclusion out of the experiments. Although the paper could be a good survey for readers who want to learn about continual learning, I think its viewpoint is not new and its conclusion is not surprising. While it is helpful to know that rehearsal works better than regularization in most datasets, this is not entirely a surprising result. I think it is a common belief that rehearsal-based is more robust against catastrophic forgetting, while regularization-method is more space-efficient in that it doesn't have to store examples. The fact that the last layer suffers from catastrophic forgetting is also not a surprising result, given that the lower layers are known to encode linguistic features and the upper layers encode task-specific features.
summary_of_the_review: While the paper can be helpful for readers who want to learn about continual learning in text classification using pretrained language models, the paper does not seem to bring sufficient a novel viewpoint or conclusion to be published at ICLR.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 
This paper conducts an empirical study on the catastrophic forgetting of pretrained language models. On two continual learning settings (class incremental and task incremental), the paper evaluates multiple pre-trained models on different data sets, to see how severe the catastrophic forgetting issue is for these pre-trained models. Then the paper also tests the effectiveness of multiple continual learning methods on such pre-trained models and draws some conclusions.

main_review: Although the authors have conducted quite a lot of experiments, the phenomena shown in experiment results is hardly surprising to me. It is not surprising that the pre-trained language models would have forgetting issues when fine-tuned on downstream tasks. It is also not surprising that rehearsal-based methods perform the best for pre-trained models. 

Moreover, the paper draws a conclusion that BERT is the most robust one and is a good option if a continual learning process is going to be conducted. Based on this, the authors provide a few analyses on BERT’s ‘secret’ for continual learning. However, compared with other pre-trained models, I don’t see that BERT is significantly better than others given the figures and tables. I feel from the figures and tables, BERT and other models look similar. The authors didn’t give a comprehensive explanation on how they read such information or a concrete quantitative comparison to support this claim.

summary_of_the_review: A thorough empirical analysis with unsurprising conclusions
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors perform a comprehensive study of how pretrained language models work in the continual learning setting. The authors study 5 relevant pretrained language models (masked and unmasked) and somewhere between 3 and 6 continual learning strategies depending on where in the paper they are counted. In addition to a thorough everything-by-everything evaluation, the authors hone in on the details of how the different models and CL approaches are reflected in the transformer layers. The authors find that the different language models studied perform relatively differently, both qualitatively and quantitatively, and these insights may provide useful for directing future improvements.
main_review: Strengths:
Overall the study is very thorough covering both the correct range of options for each axis studied and a set of relevant cross-axis multiple variable experiments. The organization is good (but not perfect, see below), its strengths are that the different options considered along each axis are clearly laid out ahead of time, with the exception of the continual learning strategies. The layer-wise analysis in particular is interesting and tells a coherent story, despite the challenges of displayed complicated 3D data. Overall, it seems that recently many studies compare quantitatively against multiple PLMs, which ultimately appear similar due to only slightly different performance numbers. This study's most successful contribution in my opinion is an exploration of the qualitative differences among PLMs in the continual learning setting.

Weaknesses:
- It would be really great to see how the insights after analysis can be used to improve performance. It's probably not absolutely required given the focus on probeing, but it would go a long way towards validating the insights.
- Adjusting the numbers to achieve the 5/4/3/2 cuteness gets slightly in the way of understanding, unfortunately. The main reason for this is that the "veins of CL methods" has a different number over the course of the paper, which makes it hard to identify when a given list of N things is a list of the "veins of CL methods". Specifically, in the abstract and intro this number is 4, in section 2.3 this number is 3, in section 3.1 this number is 6, in Table 1 this number is 5 (two different sets of 5), in Figure 1 this number is 6, and in Figure 2 this number is 4. For a paper that has so much going on and so many different lists of different sizes, keeping these consistent would make it much easier for the reader to understand at any point what exactly this given list of N items is referring to. Along the same lines, it would help to be consistent with the language around each set of N things. For example, the "veins of CL methods" are called at least "veins", "schemes," and "approaches" at different points. 
- Section 3.2, Table 1, and Figure 1 are relatively weak in my opinion. What am I supposed to conclude? I can look at the table and see the different results, but so what? What should I be drawing my eye to in the table (bold would help)? The Figure here is too small to even attempt to parse.
 - It would be very helpful to have a sentence that gives intuition about what's being measured with the accuracy metric. The definition is there, but it took me a second to realize that the intuitive idea is that it's measuring the accuracy of past tasks after the model has moved on to learning new tasks. 

summary_of_the_review: Overall the authors perform a quite deep study of using pretrained language models for continual learning. Despite some weak points in the analysis of the quantitative results and inconsistent organization/language around the CL approaches, the thoroughness of the study, in particular the analysis at a layer-by-layer level, is likely of interest to the broader community.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper explores the continual learning performance when combining different PLMs and common continual learning methods with 3 challenging NLP classification tasks.

To benchmark these combinations the methods are evaluated in task-incremental and class-incremental learning settings over various NLP end-tasks, which covers common learning settings in continual learning and NLP. There is also a layer-wise performance analysis to identify which layers keep or forget task relevant information during training.

Overall the paper shows that forms of replay outperform other methods like regularization.
main_review: ## Strenghts:

The analysis of PLM is carried out over different continual learning techniques, NLP tasks, PLM variants and finally by layer-wise probing analysis. Understanding the strengths and weaknesses of each PLM is very desirable research progress. The authors also provide new research questions that arise from the analysis and point to interesting unsolved research challenges.

Evaluation starts off with the expected lower and upper bound of performance, and then moves on to disentangle FWT and BWT (backward, forward performance) when using various CL techniques.   The chosen data sets are not well behaved, i.e. experience imbalances, which makes the results more realistic (less artificial). Overall, this study provides a necessary step towards exploring future continual learning methodology and explores many important factors on eight pages.

From the batch-learning adaptation literature on PLMs one may expect baselines such as various adapter block versions or 'anti forgetting hacks', but it is understandable that the authors did not test these, since adapters would likely introduce complexity per increment and quickly become impractical. As the authors mention, CL specific future extensions to adapters are conceivable, but a work of their own.

### Minor weaknesses (easily fixable suggestions for improvement -- 1 content page left for improvements):

- Some plots seem to be a very small, and may be enlarged to use the 9th page.
- Fig 2 plots should share a larger model legend (on the figure top or bottom), so the bars can become wider and easier to distinguish
- Fig 2 color could be made more distinguishable, especially since the plots are narrow.
- Tab 1 could underline the best non joint performance — makes it easier to glance
- sec 4. That Transformer layers, except the classification layer, do not adapt much during fine-tuning, is a known result (hence assumptions 1 and 2 in the paper), which should be cited — see BERTOLOGY Primer by Anna Rogers for references. Here intermediate layers are shown to forget as well, so this is a nice new finding, that can be contrasted.
- Fig 3: I assume the figure color-scale is probe performance? Also, the buffer size, e.g. er-200 should be explained with an example. Is it 200 er samples, 200 samples/ class? In section 5.1 it should be (re-)mentioned what the buffer size means.

### Spelling errors (not all listed): using a TTS app/function makes it easy to find these

- sec 2.3. "eg".
- sec 3.2:
    - "could be find" ... can be found
    - tendency to forgetting ... to forget/ towards forgetting
    - reach the highest .. reaches
- 4
    - interferes ... with the PLMs ability to retrain _ representations
- 4.1
    - Joint ... Joint multi-task training
    - from sketch ... from scratch
- 4.2.
    - methods making sense ... make sense
    - The sentence "(2) The classification layer clf is typically the most fragile of BERT, where continual learning learning methods making sense." is not understandable.

## Questions to the authors:
None, regarding clarity.

summary_of_the_review: The problem, proposed CL algorithms, benchmarks and evaluation metric are suitable to answer the research questions. The performance and layer-wise analysis is conductive to deepen an understanding for the shortcomings and potential opportunities of PLMs for continual learning. Unsurprisingly, variants of ER are the most effective technique in incremental CL with PLMs, which is somewhat disappointing, but also a standard outcome in CL.

The paper is mostly well written, and the experiments (sub research questions) are logically structured. Some minor details can be improved and have been pointed out in the review. Insights are valuable and provide a solid foundation for followup studies. The details in the appendix were interesting and added to the paper and its reading flow, e.g. by moving cumbersome details like hyperparameters to a dedicated appendix section.

I thus feel confident to recommend this paper for acceptance.
review_title: null

Meta-review: This paper presents a comparison and analysis of continual learning methods for pretrained language models. The authors categorise continual learning methods into three categories, those that use cross task regularisation, those that employ some form of experience replay of previous training examples, and those that dynamically alter the network architecture for each task. Evaluation results from representative examples of these three paradigms are then presented and analysed. In general methods that incorporate experience reply appear to perform the best, while analysis of the predictive power of individual layers of the pretrained models suggests that some network layers are more robust to catastrophic forgetting than others, and that this also varies across architectures (BERT, ALBERT, etc.).

In general the reviewers agree that this is a well conducted study that provides an interesting contribution to an important area of research. They also generally agree that the many of the results are unsurprising given the properties of the algorithms explored and prior work in this area. The main point of difference then becomes how valuable it is to present a thorough study of existing algorithms that confirms our assumptions. I believe that the current work raises enough interesting questions to make it a useful contribution to researchers working in continual learning. In particular the results analysing the relative differences in catastrophic forgetting across different layers in models suggests interesting avenues for follow on work.
Decision: Accept (Poster)