Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a new continual learning problem setup: continual knowledge learning (CKL) and constructs an associated benchmark resource. The benchmark is based on slot filling-based knowledge probing tasks (i.e., the LAMA analysis). The authors show the empirical performance of some existing CL methods, ranging from regularization, rehearsal, and parameter expansion. And they show a few findings based on their experimental results, e.g., learning rate can be sensitive to balance the tradeoff between forgetting and learning new knowledge, and CKL methods might have transferrable performance across different LMs. (e.g., T5 and GPT). 
main_review: Strengths: 
- The problem of CLK itself is an important task and more realistic to downstream knowledge-intensive applications. 
- I like the clear separation of the types of knowledge: time-invariant (to keep), outdated (to remove), new (to inject), as well as their collected datasets for reflecting the three types of knowledge update. 
- Different types of baseline methods are covered and compared with analysis. 

Weakness:
- The formulation of the CKL problem is overly simplified. It basically only considers a single corpus (D_1) for updating the knowledge of previously learned LMs. A general setup should consider a streaming version of D_1 and make it a sequence of sub-corpus (D_1, D_2, ..., D_T) that arrive at different time steps. Also, the associated tasks in UpdatedLAMA and NewLAMA should reflect such a time series --- that is, the streaming version of the current probing tasks. The current formulation described in Section 3.1 only has a single time step. It's more like an offline learning problem with the forgetting constraints but not an (online) continual learning problem. The proposed setup is thus a bit far from the motivation of studying CKL --- maintaining an ever-changing LM. 
- The experiments are very limited to the LAMA probing which does not necessarily connect to real downstream applications of these LMs. It's also hard to justify whether such methods can maintain performance in general NLP tasks. The argument about the KILT experiments seems to only focus on testing the retention but not about the new/updated knowledge about D_1. 
- The analysis of the CKL methods is not deep enough. How do these methods work and why do some outperform others? How do we know if an arbitrary new fact conflict with the existing knowledge or not on the fly? How do you define such conflicts properly? Say you have a sentence in D_0 "Cristiano Ronaldo plays for XXX in 2010", and you have another sentence in D_1 "Cristiano Ronaldo plays for YYY now." In the current problem setup, how will such "conflict" be defined? 
- The new findings are not particularly non-trivial. 
- The presentation and the writing of the paper can be further improved with more illustrative examples and case studies for readers to qualitatively see the problem setup and the differences between these methods.
summary_of_the_review: This paper is a pilot study of an improtant problem (continual knowledge learning of LMs), but the problem formulation is overly simple and there are still many important yet missing points in both data construction and experiments. Also, there are no much insightful and non-trivial findings with deep analysis of existing methods. Please find more details above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies continual knowledge learning of language models, which is an interesting and important problem. Particularly, a new benchmark and a metric are introduced to quantify the retention of time-invariant world knowledge, the update of outdated knowledge, and the acquisition of new knowledge. To establish baselines for the CKL benchmark and validate the rationality of the proposed benchmark and metric, the author conducts extensive experiments with a pre-trained encoder-decoder model (T5) based on various training methodologies including regularization, rehearsal, and parameter expansion methods.

The paper is well organized and easy to follow. The proposed continual knowledge learning problem is quite interesting and important. The FUAR metric is also technically sound. The authors also conduct comprehensive experiments to verify the rationality of the proposed benchmark under the various settings.
main_review: Strengths:
1. The proposed continual knowledge learning problem is quite interesting and important.
2. The benchmark is useful and the proposed FUAR metric is technically sound. 

Weaknesses:
1. The paper only performs experiments with an encoder-decoder model (T5). The experimental results will be more convincing if more pre-trained language models (such as GPT) are included. For example, we can explore the ability of different PLMs to avoid catastrophic forgetting and to acquire new knowledge while preserving invariant knowledge.
2. Consisting with the traditional setting of CL, the paper also creates a setting for multiple CKL phases. However, only a two-phase setting is considered. More experiments can be explored, such as five-phase or controlling the differences in the distribution of data at different phases.
3. The experimental findings in this paper are somewhat trivial.

summary_of_the_review: The paper studies an interesting problem. The proposed benchmark and metric are technically sound. However, there are some concerns about experimental settings.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: - The paper is about continuous learning for language models. The authors leverage existing LAMA tasks and collect a new test benchmark with updated information and new information. 
- They investigate several existing CL algorithms, and they propose a new metric called FUAR to measure trade-off between forgotten time-invariant knowledge and updated or newly acquired knowledge.
- They provided some findings on their continuous LM learning. 
main_review: Strengths 
- They provide a new benchmark and metric to measure the retention of time-invariant knowledge, updated knowledge, and new knowledge.
- Some interesting observations are provided, for example, 1) rehearsal methods do not work well in this setting (even though the reason is quite obvious because some knowledge is updated) and parameter-expansion methods achieve better results, 2) LMs are prone to more forgetting as they go through multiple CKL phases, 3)  LMs should be pretrained with just a few epochs on less duplicating data for efficiency.

Weakness
- In a real-world scenario, how can one know in advance that the new task is truly "new"? As mentioned on page 5, it is possible that the FUAR score is very large if "no gain" and "preserve knowledge". The authors did not show experiments or analysis in such a setting, where the new task has some "knowledge overlapping" with the learned tasks. To make this concern more general, I feel the measurement of "task similarity" is missing in this work.

- Can we also see perplexity as another dimension? EM scores cannot know "how bad" the prediction distribution is.

- Do you think the conclusion of these LAMA tasks is the same as other NLP downstream tasks? 

- Do you think others can easily replicate your numbers? Do you run several splitting or seeds for multiple round CL settings?

- Have you tried different sizes of T5 models? Maybe the GAP between vanilla and CKL methods will be smaller given larger models.

Misc
- I have doubt on this sentence "Moreover, the effectiveness of CKL methods is much reduced in multi-phase CKL, shown by the decrease of the gap between the FUAR of T5-Vanilla and the best performing CKL method in the scenario of two-phase and one phase, which is 0.92 and 0.46, respectively." Isn't it only prove that T5-Vanilla can learn better FUAR scores SMALL-P1-->  SMALL-P2 than SMALL-P1+  SMALL-P2 because you have similar FUAR scores for T5-Kadapters? 
summary_of_the_review: This work is quite insightful for us to understand more about how LM continuous learning works, although I think more experiments could be beneficial as I mentioned in the weakness section. If we can make sure the numbers from this paper are reproducible and comparable, then I think it could be a good testbed for future research in this direction.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors formulate a new continual learning (CL) problem called Continual Knowledge Learning (CKL). Particularly, they distinguished three sub-tasks in CKL, i.e., the retention of time-invariant world knowledge, the update of old knowledge, and the acquisition of new knowledge. They also introduce a new benchmark and metric to quantify the performance of various state-of-the-art models on these sub-tasks. They find that CKL demonstrates unique challenges that are not present in previous CL setups. Critical causes of knowledge forgetting in CKL are also discussed.
main_review: Strengths:
In terms of novelty, the paper extended the definition of Continuous Learning to formulate Continuous Knowledge Learning that has unique challenges compared to the traditional CL. The paper also introduced a novel metric named FUAR to measure the trade-off between knowledge forgetting, update, or acquisition. This is a contribution to the field as this quantitative metric could facilitate direct comparison between models performing CKL tasks.

The paper is technically sound. Extensive experiments were conducted to benchmark the performance of various CL approaches (regularization, rehearsal, and parameter-expansion methods) on different aspects of the CKL task (retention of time-invariant knowledge, updating old knowledge, and acquiring new knowledge). In the Appendix, the authors also presented various ways to understand the model's learning process, such as the change of predicted outputs during the continued pretraining, as well as the failure analysis based on the type of probes. These methods provide more insight into the model's learning process that went beyond the plain performance scores. 

Weaknesses:
Using T5, the authors showed that parameter-expansion methods have the most robust performance throughout all of the experimental settings. However, in the experiments with GPT-2 (a decoder-only model) in the Appendix, GPT2-MixReview (a rehearsal method) performs the best. Although the authors mentioned that “We leave more exploration of applying CKL methods on decoder-only models such as GPT-2 architecture as future work”, they should still mention this critical discrepancy in the main body of the paper so that the readers are aware of the context of the findings. After all, large language models take various forms and both T5 and GPT-2 are examples of large language models.

Questions:
The authors demonstrated that parameter-expansion methods have the most robust performance throughout all of the experimental settings with T-5. Since the three methods (regularization, rehearsal, and parameter expansion) are not mutually exclusive, I was wondering if the authors have tried any combination of the approaches. For instance, the rehearsal approach could be combined with the parameter-expansion method. I was wondering whether a combined approach would yield even higher performance. 

summary_of_the_review: The paper formulated the problem of Continual Knowledge Learning and benchmarked the performance of large language models on this task with different CL methods. The tradeoff between forgetting existing knowledge and updating old knowledge/acquiring new knowledge is quantified through a new metric, which would serve as an important optimization goal for future research. This work is a big contribution to the community and would invite more research into this topic.
review_title: null

Meta-review: The paper introduces the problem of continual knowledge (language) learning. The authors point out the interesting duality between continual learning and knowledge learning where: in knowledge learning one must avoid forgetting time-invariant knowledge (avoid forgetting in CL), be able to acquire new knowledge (learn new tasks in CL), and replace outdated knowledge (a form of forgetting and re-learning or adaptation). In their paper, the authors develop an initial benchmark for the task along with a set of baselines and provide empirical studies.

The initial reviews were quite mixed. The reviewers seem to agree this work studies an interesting and fairly novel direction for continual learning of language. However, the reviewers did not agree on whether this initial stab at the problem was "enough." In particular, reviewer U9Hk argues that the formulation is "oversimplified" and the current experiments are limiting.

After the discussion, the reviewers remained split with one high score (8), two borderline accepts (3), and one reject. So three reviewers believe that this manuscript is already a good contribution. The fourth reviewer disagrees, but the authors provided clear and convincing responses to many of their comments (and point to results already available in the appendix).

Overall, this is a clear and reasonable first step considering this paper proposes a new CL problem. The reviewers and I believe that this is interesting and rigorous enough to be impactful and to warrant follow-up works. As a result, I'm happy to recommend acceptance. I imagine that if the community demonstrates interest in this line of work, there will be work both on methodologies to improve the proposed baselines, but also work proposing extensions to the problem in line with some of the comments of reviewer U9Hk.

In preparing their camera-ready version I strongly encourage the authors to take into account the suggestions of the reviewers and your replies. In particular, your discussion regarding encoder-decoder and decoder-only LMs and the associated results would be good to discuss in the main text (even if the full results are in the appendix).
Decision: Accept (Poster)