Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary

Recently, pretrained Transformer language models have been shown to capture world knowledge (using testbeds containing facts). What if you want to update a fact, for example, with the current president of USA? This paper investigates different approaches to update the weights of a Transformer model such that the model works for the modified facts but does not catastrophically forget unmodified facts. The main proposal is a simple regularization technique (which they call constrained fine-tuning) to minimize weight changes while fine-tuning on the supporting factual sentences that represent the modified facts.

Strengths

1. The problem of updating world knowledge in Transformers in itself is an interesting problem and novel.
2. The problem is well motivated and the first half of the paper is well-written.
3. The proposed method of fine-tuning along with regularization is simple and it seems to work better than just fine-tuning methods.


Weaknesses:
1. Confusing experimental section, and many important details are missing (see comments).
2. The paper felt like it is a last-minute submission and written in haste.
3. Important related work on retrofitting literature not cited. 
4. The proposed method works at the cost of forgetting unmodified facts as the number of unmodified facts increase.


Comments:


Although the reviewer likes the problem formulation and the main idea, they find it hard to follow the experimental section. 

1. There were no details on how to find supporting sentences of target facts that one wants to change. This is a non-trivial task and without a detailed description, the paper is impossible to replicate. There were no examples anywhere including appendix.
2. Why do the authors start with pretrained + fine-tuned model? Isn't pretrained model trained on all sentences anyways? Do they mean separate fine-tuned data that contain factual statements? Why does one need this data? Isn't this against the spirit of pretrained models as knowledge bases?
3. Results of vanilla pretrained models on unmodified and modified facts missing, i.e., PT alone. Also, PT + FT.
4. The notation and acronyms are confusing. The authors use a lot of acronyms like PT, FT, FTA, FTM, RT which is unnecessary. This makes the results table unreadable without reading the paper.
5. No citations on retrofitting literature which is very similar idea to this work (e.g., Faruqui et al. 2015). This limits the novelty of the work but the reviewer give credits to the problem formulation.
6. How is PT+FT+FTM different from PT + D_F'?
7. FTA is introduced but never used.
8. The reviewer had a hard time following experiments with FAE section. What is the main takeaway from these experiments? The model has to be described in detail (perhaps with a figure). 
9. Examples, Examples, Examples. Show some examples from each dataset. 
review_title: Nice idea but bad execution or badly written

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper studies a new problem: evaluating the ability of modifying knowledge inside the Transformer models when models memorize world knowledge inside its parameters. The contribution of this paper is two-fold: (1) introducing a new benchmark for evaluating such ability, and (2) evaluating a comprehensive list of baselines, including a new model that has a constraint term in the objective during fine-tuning. 

Strengths of the paper:

1. The problem is well-motivated and of interest to the wider community. Examining the behavior of transformers given updated facts is a timely topic, given recent progress and interest in large pretrained models storing world knowledge and achieving reasonable performance on downstream tasks (slot infilling, question answering) without access to external knowledge sources. 
2. The benchmark dataset is created in a reasonable way - it is created on top of T-REx and Zero-shot RE dataset, where a target entity is replaced by another entity that shares the same relation with the other entity.
3. The constrained objective provides a simple yet nice way of updating knowledge in the model parameters in a constrained manner. This model shows that the fine-tuned model overfits less to the modified knowledge, compared to naive baselines.


Weaknesses of the paper:

1. I am not fully convinced by the setup in the paper, where the model is pretrained on unmodified facts and then fine-tuned on a modified knowledge. First, in a natural setting, a set of knowledge sources will always contain both unmodified facts and modified facts together. Therefore, an assumption in the paper that the model can only access modified facts during fine-tuning seems to be unrealistic. Second, if the research question here is the generalization ability of the model, isn’t a zero-shot setting or a setting with small training examples (e.g. 1k) more suitable?
2. Although the creation process of the benchmark dataset makes sense, it is still created synthetically. Furthermore, the paper does not include data analysis or human performance estimation, making it hard to estimate the quality of the data. This is important because, when a subset of knowledge was synthetically updated, some knowledge will contradict each other.
3. Although the model with constrained objective overfits less to the modified knowledge, it still overfits a lot, achieving significantly lower numbers on unmodified facts. It is still a nice baseline, but the claim in the paper: “best way to enforce the constraint in Transformer models” (Sec 1) seems to be overclaiming.
4. Although the paper includes comprehensive experiments, there isn't really new take-aways that are not different from naive expectations. The conclusion, "the model overfits to modified facts and suffers from catastrophic forgetting" is pretty naive and has been observed in a lot of prior work ([1] is one of recent ones).


Questions

1. Is there a specific reason that the scope of this paper is restricted to transformers? Looks like the general idea can be applied to any model that does not have an access to external knowledge source?
2. The creation process of the benchmark data is strictly limited to structured KBs. Is there a way to create such a dataset for tasks based on unstructured text?
3. Are there baseline numbers for PT+FT (without FTM) reported?

[1] Rolnick et al. Experience replay for continual learning. NeurIPS 2019.

review_title: Timely paper that is of interest to the community, with new benchmark data and baselines; some settings are unconvincing and take-aways are not new

final_score: rating: 5: Marginally below acceptance threshold
review_content: The submission proposes and explores the task of modifying factual knowledge in transformer language models. The paper makes a convincing argument that this is a worthwhile problem, as knowledge in existing models quickly becomes out dated, and the cost of re-training from scratch on an updated corpus is prohibitive. The paper suggests several natural alternatives, finding the best results by fine-tuning the model on modified facts, but with a constraint to minimize the difference in from the original model. Surprisingly, the authors further show that it is harder to modify knowledge in the partially symbolic 'Facts as Experts' model than it is in BERT. However, I was unable to follow some important details, and I think the paper is missing an obvious baseline, so I think it needs some more work before it can be accepted.

The proposed method is based around fine-tuning the model  on modified facts, with the hard constraint that the norm of the difference from the original model parameters is less than a threshold. I struggled to find any detail on how the authors enforce these constraints during optimization, and this point should be made clearer.

I'd be really interested to see some more analysis that sheds light on which parameters the transformer is using to store facts. Results in the paper touch on this by exploring fine-tuning different layers, but I think lots of interesting experiments could be added with little extra work. For example, you could try fine-tuning just the word embedding layer, or only the feed forward sub-layers. I think exploring this question would add to the paper, and might improve results.

I also felt the paper was missing an obvious baseline based on kNN-LM (Khandelwal et al. 2019), which is explicitly motivated as a way to add knowledge to transformer language models. For example: first, you could simply encode your the sentence modified facts with the transformer, masking out the modified words. For inference, you could copy a token from the modified facts if it is sufficiently close in representation space, and otherwise predict a token using the baseline transformer model. 

More generally, non-parametric methods appear to offer a relatively simple and obvious solution to the task, as facts can be updated by just changing the text. These approaches should be discussed further.


Minor Points
The paper contains frequent grammatical errors (too many to list here), and I'd recommend getting it thoroughly proof read before publication. This did not affect my rating.
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes a very interesting yet practical problem of modifying the memory encoded in transformer weight to unlearn some facts and update with new facts. This paper seems to be a follow-up work from FAE, which encodes symbolic facts in memory for retrieval. Generally speaking, I like the basic idea of this paper and it might have a broad impact on the whole community. However, there are still a lot of questions about the paper.
1) the paper seems to be written in a rush without refining, there are numerous serious typos and spellings errors, which affect my understanding a lot. For example, in 4.5.1, what is "RT", is it supposed to be "RI"?  In Figure 3, why is the 32, the figures are a mess. Why is the left showing "32->512" while the right showing "32->128"? Why do you say it's sharp degrading, it's not that sharp reflected from Figure 3. I'm not sure if I misunderstand something. 
2) The results are also quite messy. The algorithm without constrained optimization has its results reported in the table, while the algorithm with constrained optimization is reported in figures. The results with FAE is yet in another table far away. It's hard for me to compare them and draw a consistent conclusion. Is it possible to aggregate all the main results in one table and demonstrate all the ablation studies using Figures? Currently, the figures involved in 4.5.2 are distributed from page 6 - page 8, is it possible to aggregate them in a concentrated place? 
3) Besides these details, I think the proposed method is somewhat "not novel". In lifelong learning or meta-learning community, such constrained optimization algorithms have been explored for a few years to prevent the mode from catastrophic forgetting. I don't think the paper makes any significant contribution to this aspect. 
4) Overall, I still quite like the scope of this paper. I would like to see a more structured and clear version of the paper with more fundamental algorithm innovation. 
review_title: Review

Meta-review: All reviewers noted the significance of the problem tackled by this paper and felt that it is going in the right direction. However, they also all noted that the paper was not finalized and polished well enough to be granted publication: details missing, typos, clarifications needed. The reviewers acknowledged the large amount of work that went into improving the paper during the discussion period. R1 even increased their score to reflect that.

Still the paper still needs some work to be accepted at ICLR. In particular, we encourage the authors to improve on 2 axes.
1. Clarifying motivations and contribution: it is still unclear if the main point of the paper is to propose new methods around FTM & constrained updates, etc. or around proposing a new benchmark for catastrophic forgetting, lifelong learning.
2. Reorganizing experimental section: the experimental section should be organized to support #1. Reviewers made a lot of suggestions, like moving Table 4 from the appendix, that should be further refined

We hope that this will allow to increase the clarity and impact of this research work.
Decision: Reject