Venue: ICLR 2022 Conference

Reviews:

final_score: rating: null
review_content: I'm not opposed to using large models. The question of using large models is only a side question I want to discuss with the authors.

My review score is based on the fact that the proposed sophisticated method would not be as practical as a simple engineering trick (i.e. caching).

To further clarify my standpoint, I would not mind if this paper is accepted. However, I require a detailed discussion about the practical significance of model editing in a future revision, given the effectiveness and training-free characteristic of caching.

The model editing problem is interesting, and if someone really wants to solve such a problem, this paper should tell him that in some scenario caching can be a choice besides time-consuming training-based method.
review_title: Clarify my opinion

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: While large pre-trained nlp models have achieved great performance on a variety of downstream tasks, the largest of these models still make errors. This paper deals with the problem of enabling both developers and end users of such models to correct inaccurate outputs while leaving the model otherwise intact. If presented with only a single problematic input and new desired output, fine-tuning approaches tend to overfit. To enable easy post-hoc editing at scale, this paper proposes Model Editor Networks with Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model. 

The approach trains lightweight model editor networks to produce edits to a pre-trained model’s weights when provided with the standard fine-tuning gradient of a given correction as input, leveraging the gradient as an information-rich starting point for editing. 

MEND leverages the fact that gradients with respect to the fully-connected layers in neural networks are rank-1, enabling a parameter-efficient architecture that represents this gradient transform.

It provides experiments with T5, GPT, BERT, and BART models showing that MEND is the only approach to model editing that produces effective edits for models with tens of millions to over 10 billion parameters.
main_review: The main contribution of the paper is in leveraging the fact that gradients with respect to the fully-connected layers in neural networks are rank-1, enabling a parameter-efficient architecture that represents this gradient transform. This leverage results in an efficient lightweight model editor networks to produce edits to a pre-trained model’s weights when provided with the standard fine-tuning gradient of a given correction as input. The other strnegth of the paper is in justifying the architecture of the proposed MEND model through ablation studies, and showing its advantage over the existing methods through multiple numerical experiments.

The paper can be improved by better writing, particularly of the methodology section. In its current form, it is difficult to understand the exact loss function of the model editor networks. Further, there seems to be a lot of notations in the paper, some of which are redundant.
summary_of_the_review: The paper provides an efficient approach for correcting inaccurate outputs of large nlp models while leaving the model otherwise intact. They show the advantage of their proposed approach over the existing methods through a number of numerical experiments.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: A fast model editing method is proposed in this paper to edit pre-trained models at scale. The core component is a Model Editor Networks with Gradient Decomposition (MEND). The paper claims that the MEND method has reliability, locality, generality. MEND is empirically validated on some curated datasets.
main_review: Strengths:

1. The problem of editing a pre-trained model is interesting. Since big models are deployed and getting into industrial scenarios, it is interesting to study the emerging problems they bring.

2. This paper decomposes the gradient of linear layer (size: d x d) as an outer product of (d x 1) x (1 x d), so that it operates on input with 2d dimensions rather than d^2 dimensions. This technique is reasonable, but was originally proposed by Goodfellow et al., limiting the paper's novelty.

Weaknesses:

1. The performance in small-scale model editing is on par with ENN and KE, which is not surprising. In addition, it is strange to see reported results as "<0.001". 

2. One claim of this paper is the ability to edit large-scale models like T5. However, only "the MLP weight matrices in the last 2 transformer blocks" are edited, which makes the contribution an over-claim.

3. If only two blocks are edited, then how many blocks are fine-tuned in Table 3 for FT? My educated guess is that all blocks are fine-tuned. Since training with only one sentence may lead to severe over-fitting, it is natural to fine-tune only several blocks. So the baselines may not be plausible.

4. The significance of the problem is unclear. If the only purpose is to change the answer of a specific input, then a simple engineering practice can do the trick: if the representation of x is close to x_e, just output y_e. This way, the editing task can be accomplished. Did the authors try this approach?

5. I think the locality statements are somewhat over-claimed. Since a model is edited, the locality cannot be guaranteed. And I doubt if the locality can be preserved after several editing. If my understanding is correct, experiments in Table 3 edit the same model for each input. If all the datasets are used to edit the model (i.e. many edits are performed), what is the accuracy drop?

Besides, the paper says that "Uj , Vj corresponds to a low rank factorization of the weight matrix of layer j", but Figure 2 says V1/V2 are xavier initialized and U1/U2 are zero initialized. How to understand the disagreement?
summary_of_the_review: In summary, the paper studied an interesting problem but the practical significance is doubtful. The empirically results are not strong. A concern of fair comparison is raised.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies the problem of "model editing" - altering the model predictions on local examples without affecting the global behavior. 
Authors tackle the challenge of editing very large models with billions of parameters, such as T5 or GPT-J, where editing is particularly important, as it is impractical to re-train such models from scratch to correct every mistake.

To address this challenge, authors propose MEND - a novel model editing method that differs from prior art in three ways:
- it does not require computing higher-order gradients (and hence, requires less compute/memory)
- it can be trained for a given model without the need to change the model parameters (and hence, can be applied to pre-existing models)
- it has better asymptotic time & memory complexity in terms of model size (and hence can be used for very large models)

main_review: ## Practical significance

I would argue that the main merit of this paper is in how it extends the range of **practically useful** applications for model editing.

Being an industrial researcher, I found that editing small models was often not justified, as those models could be re-trained from scratch in a few hours. In turn, editing large models was impractical without large-scale model parallelism. As a result, methods such as Editable Training (referred as ENN) would only make practical sense if one needs to edit a (small) model on device, where re-training is infeasible. However, such use cases are very rare, as companies typically hesitate to let each individual user have its own version of a model on-device, which would complicate debugging and quality assurance.

In contrast, MEND could make model editing much more widespread since (1) MEND can be used for models that cannot be simply re-trained (I was able to run MEND for a GPT2-like model with 41B parameters using a 4x A100-80GB pod with tensor parallelism; that model was trained on 128 GPUs for over a month) and (2) MEND can be applied after-the fact without fine-tuning the original model, meaning that anyone will be able to train and publish model editors for repositories of pre-trained models, such as HuggingFace Transformers, torch.hub.


## Strengths

- (as described above) the proposed technique has a great practical potential
- the technical contributions made in this paper are solid and significant
- the core method utilizes a clever trick (rank-1 gradients) that significantly reduces the complexity of model editing. While there are related tricks in meta-learning papers, this specific variant and application to model editing is novel
- overall, the paper is well-written and easy to follow


## Weaknesses

There are several (minor) inaccuracies in the experiments and related work sections. One of these inaccuracies may slightly mislead readers if they are unfamiliar with the related work.

__1. On the use of gradient checkpointing / rematerialization:__ the original PyTorch implementation of ENN (Sinitsin et al) heavily relies on gradient checkpointing to avoid storing too many model states in memory when computing higher-order gradients. However, authors report the memory consumption for their custom re-implementation of ENN (Figure 3, 4) without gradient checkpointing, making MEND look significantly more favourable as a result.

Using gradient checkpointing allows editing 2.8B models in 21-28GB memory instead of almost 80GB reported in Figures 3, 4. However, using ENN for 11B models still does not fit on a single A40 GPU (described in supmat), so the general conclusion holds.

To be fair, authors mention the fact that their implementation does not use gradient checkpointing (Appendix B.3), but this is only briefly mentioned in supplementary materials. Since this implementation detail improves the scalability of one of the baselines by almost 3x, i'd request that
- either authors evaluate ENN with gradient optimal checkpointing in **Figure 3 and Table 3** (for 2.7B and 2.8B models)
- or authors clearly state this as a limitation of the experiment design in the main paper

As for applying ENN to GPT-J and T5-XXL models - Sinitsin et al. propose a strategy for further reducing memory usage by editing only a subset of model layers (Section 4.3, second paragraph) -- as such, all layers prior to the first edited one can be considered constant and excluded from the unrolled meta-learning graph, further reducing memory consumption by up to an order of magnitude (if only the last transformer layer is edited). That said, this this strategy requires a careful selection of layers, so I understand why it is not evaluated.

Based on my observations, MEND still scales significantly better than ENN, especially for larger models (40B+) -- and I do understand that it is impossible to obtain enough fully trained 10B+ models in the public domain -- but i recommend including the additional comparison so as to avoid the slightest chance of misleading the reader.

__2. Minor inaccuracies in Table 1__

- "One step" - ENN is flagged as one step, but the corresponding method **does** require an interative process and original paper (Sinitsyn et al) reports 2-5 steps in most experiments.
- "Scales to 10B" - it would be preferable to refer to your hardware setup. Otherwise, all methods would comfortably "scale to 10B" on a DGX-A100 machine (640GB total GPU memory). Consider "Scales to 10B means that the method can edit a 10B model in our single-GPU setup (Appendix XYZ)"


__(lack of requirements.txt)__
I appreciate that the authors provide the source code (this affected my score positively), but I highly encourage authors to specify the exact versions of all the dependencies that they use in their experiments (i.e. pytorch, transformers, higher, etc). Without this information, it will be difficult to reproduce the experiments in the future.


### Nitpicking

This below complaints did not affect my score in either direction, and neither will addressing them.

__(Section 3.1) “the gradient … is a rank-1 matrix for each of B batch elements”__
It would be great to give an intuition of why this is the case for algebraically challenged readers. Perhaps, refer them to an appendix where you show this to be the case in a simple MLP example.

__(Ethics statement)__
There is one possible ramification of MEND that is (arguably) not covered by "treating / exacerbating undesiable behaviors": introducing model backdoors.

If somebody releases a certain model to the public domain, that somebody could use editable training to, for instance,
- tweak a face identification network to always treat a person as staff if they wear a very specific ornament -- and use it to bypass a security system
- tweak a language model to memorize some secret data and only give it away under a very specific prompt -- and then use this for steganography

As a result, MEND can actually enable a number of new outcomes, from elaborate pranks to serious opsec vulnerabilities.
To be fair, MEND can't be held solely responsible for this since it is not the only model editing method.
summary_of_the_review: I deeply appreciate the practical contributions of this paper and believe that it can have significant impact on the applicability of the large pre-trained models. That said, I do have several minor issues with the experiment design. I summarize those issues in the "Weaknesses" subsection of the main review.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper tackles the challenging model of editing a potentially very large pre-trained model in a way that is specific (local) yet comprehensive (general). Their method, called MEND, take as input the decomposed, information-rich gradient from fine-tuning to learn the parameters for their meta-networks that subsequently perform the updates. The authors show that it works better than existing model editing techniques on large models like GPT-Neo, GPT-J, T5-XL, T5-XXL, and some smaller models like BERT-base, and distilGPT-2.
main_review: The problem of editing models accurately with minimal side effects is an important one, and I believe this paper presents an elegant and effective method for doing so that will also be of great interest to the audience at ICLR 2022.

As the authors have correctly pointed out, existing model editing techniques have strong limitations that discourage them from being used at scale. MEND breaks free of several of these limitations, which makes this method particularly appealing:

- Edits require only a "single example illustrating the model's error" (where an "example" is composed of the edit example, equivalence examples, and a locality example). There is no need for the original training dataset
- There are no modifications to the original weights until the edit occurs. This makes the MEND algorithm lightweight, and possibly patchable (* see "Questions for the authors").
- The edit success is higher than that of strictly fine-tuning of the model, while also regularly causing less damage to existing predictions than smart fine-tuning would.
- The modifications are resource-efficient, requiring only first-order gradient calculations, and can be trained quickly with low GPU resources (* see Feedback)

Unfortunately, results in Table 4 show that MEND fails to perform better than ENN on smaller models on the task of generation, and as such I see this method being most helpful for the large model domain they are targeting.

**Feedback**

1. The claim of editing very large networks (6B+ parameters) is a bit deceitful — in the experiments, it seems the authors only edited the final 3 MLP layers of the Transformer. Please qualify the claims of speed (e.g., "MEND can be trained on a single GPU in less than a day even for 10 billion+ parameter models") somewhere in the paper, as not all of these parameters are being updated. This statement requires that important gradients all reside in a small subset of total model parameters.
2. Sec 5 "Comparison of Model Editors": It is not clear if the **fine-tune** and **fine-tune + KL** models operate on the same subset of parameters as MEND or on all the parameters of a Network. Similarly with **KE**. Please clarify.
3. There is no explicit comment of what FT, FT+KL mean in Tables 1 & 3 of the paper (though their descriptions appear in section "Comparison of Model Editors"). Please add, possibly in the caption itself.
4. Minor: I feel the paper is lacking qualitative examples, all of which are contained in Table 2.  There is only one example of drawdown post MEND edit (Table 2 input 3b), and in this case it was not harmful. I would have appreciated additional qualitative "before and after" examples in the appendix, as this is the main message of the paper.
5. Q: I was surprised to find discussion on ["Parameter-Efficient Transfer Learning with Diff Pruning"](https://arxiv.org/pdf/2012.07463.pdf) by Guo et al. 2020 to be absent from the paper. While MEND has several advantages over this technique (i.e., MEND reportedly achieves accuracy *higher* than traditional fine-tuning), a perk of the Diff Pruning technique (as well as "Knowledge Neuron" editing by Dai et al. 2021) is the ability for each edit to be viewed as a "patch" completely independent from the model parameters themselves. Then, knowledge changes can be added/subtracted from the model independently. Is this a reasonable expectation to have of MEND? If so, this could be an additional impact point of the paper. Even these can be viewed as "patches" that are applied to the original model's weights in terms of the $\alpha \tilde{\nabla}_{W_\ell}$ in the equation

$$\tilde{W} = W_\ell - \alpha \tilde{\nabla}_{W_\ell}$$

6. Q: In A.2p17, the authors mention that they attempted to edit attention weights but that the results were not as effective as edits on the MLPs. I would be interested to see those results, especially as more papers are coming out indicating the different roles and impacts performed by the MLP layers vs the Attention layers.

The paper overall is well written, and this reviewer did not notice grammatical mistakes.
summary_of_the_review: The paper proposes an effective, novel, and general technique for manually editing pre-trained models. While the method is not universally better than other editing techniques (especially for smaller language models), I can see immediate interest and impact of this work in large AI models. I vote to accept the paper.
review_title: null

Meta-review: The paper provides a method to edit trained models, meaning fix mistakes in a local way so as to not ruin generalizability. The techniques provided in the paper allow for an efficient way that makes this task possible for very large models.
There is an overall concensus that the problem of model editing in general is an important one and that solutions such as naive finetuning are not applicable for various reasons. In addition, the reviewers are convinced that given the need for an ML-based approach for large models, this technique is superior to previous work, and mostly appreciate the novelties of the paper.

A major concern raised regards possible simpler baselines. There is a potential baseline of implementing an “engineering trick” that will simply memorize the data points where the original model was mistaken, either in their original form or as embeddings, and during inference will override its output. I tend to agree that a comparison with such a baseline would improve the paper. This being said, the discussion highlighted that this baseline has several flaws that make it clear that it cannot completely replace the method proposed here. A naive implementation of it will be “too local” and would not handle simple rephrasing of sentences. An implementation operating on the embedding space will be possible only in a subset of tasks. 

To conclude, although the paper has room for some improvement (that might actually be possible towards the camera-ready version), I believe that even without it the paper is in a good enough state to be published. It tackles an important problem and could lead to further advancements.
Decision: Accept (Poster)