Editing Common Sense in Transformers
Authors: Anshita Gupta
Debanjan Mondal
Akshay Krishna Sheshadri
Wenlong Zhao
Xiang Lorraine Li
Sarah Wiegreffe
Niket Tandon
Venue: EMNLP 2023 Main (2023)
Keywords: Commonsense
Transformers
Language Models
Model Editing
GPT
Plausibility Judgements
TL;DR: Editing commonsense plausibility judgments in transformer models with improved memory editing method. We showed improvements on two commonsense datasets when evaluated under different novel generalization scenarios.

Abstract: Editing model parameters directly in Transformers makes updating open-source transformer-based models possible without re-training. However, these editing methods have only been evaluated on statements about encyclopedic knowledge with a single correct answer. 
Commonsense knowledge with multiple correct answers, e.g., an apple can be green or red but not transparent, has not been studied but is as essential for enhancing transformers' reliability and usefulness. In this paper, we investigate whether commonsense judgments are causally associated with localized, editable parameters in Transformers, and we provide an affirmative answer. We find that directly applying the MEMIT editing algorithm results in sub-par performance and improve it for the commonsense domain by varying edit tokens and improving the layer selection strategy, i.e., $MEMIT_{CSK}$. GPT-2 Large and XL models edited using $MEMIT_{CSK}$ outperform best-fine-tuned baselines by 10.97% and 10.73% F1 scores on PEP3k and 20Q datasets. In addition, we propose a novel evaluation dataset, $PROBE\ SET$, that contains unaffected and affected neighborhoods, affected paraphrases, and affected reasoning challenges. $MEMIT_{CSK}$ performs well across the metrics while fine-tuning baselines show significant trade-offs between unaffected and affected metrics. These results suggest a compelling future direction for incorporating feedback about common sense into Transformers through direct model editing.

Editing Common Sense in Transformers
Anshita Gupta Wenlong Zhao
Debanjan Mondal
Akshay Krishna Sheshadri*
Xiang Lorraine Li Sarah Wiegreffe 9
Niket Tandon V
University of Massachusetts Amherst, * University of Pittsburgh, 9 Allen Institute for AI (anshi tagupta, debanjanmond, asheshadri, wwn. Jocs umass. edu sianglli@pll tt. edu, wiegreffesarah@gmie com, nikett@allenan org
Abstract
Linelectee heghhcerhood
A-fected borfooo
Adeettd Parughiee Teuee
Afeeied Rezamng
Editing model parameters directly in Transformers makes updating open-source transformer-bseed models possible without re- re-training (Meng et al., 2023). However, these editing methods have only been evaluated on statements about encyclopedic knowledge with single correct answer. Commonsensee knowledge with multiple correct answers, e.g., an apple can green or red but not transparent, has not been studied but is essential for enhancing transformers' reliability and usefulness. In this paper, we investigate whether commonsense judgments causally associated with localized, editable parameters in Transformers, and we provide an affirmative answer. We find that directly applying the MEMIT editing algorithm results in sub-par performance, and propose to improve it for the commonsense domain by varying edit tokens improving layer selection strategy, i.e., MEMITcSK. GPT-2 Large and XL models edited using MEMITсSK outperform best-fine-tuned baselines by 10.97% and 10.73% Fl scores on PEP3K and 20Q datasets. addition, we propose a novel evaluation dataset, PROBE SET, that contains unaffected and affected neighborhoods, affected para- phrases, and affected reasoning challenges. MEMITeSK performs well across the metrics while fine-tuning baselines show significant trade-offs between unaffected and affected metries. These results suggest compelling future direction for incorporating feedback about common sense into Transformers through direct model editing.
Tecks abuorie
Sail SONCE LD
Eround oberac ei
Seil cocouss st Ca atacchal
Soll absorbs all
MENIT.oM editing 6o40
GPT Model
Modal
Semursi Comeralazion
Figure 1: Proposed framework MEMITçsK. for editing and evaluating plausible commonsense knowledge Transformers. Given plausible <Subject, Verb, Ob- ject> commonsense statement, MEMITeSK edits param- eters different token and layer locations (described in $3). Edited model is evaluated for semantic gener- alization (depicted in dark blue box) and configuration generalization defined in $3.
(Bender and Koller, 2020; Marcus, 2021; Talmor et al., 2019; Bhargava and Ng, 2022), and consis- tency errors (Tam et al., 2022; Devaraj et al., 2022; Weng et al., 2020), Retraining finetuning LMs to overcome these errors is costly and uninterpretable. To address this, prior research (Meng et al., 2022, 2023) has shown that model predictions often cor- relate strongly with certain neuron activations and parameter editing methods can effectively correct encyclopedic factual mistakes.
However, remains unclear whether these edit- ing methods can scale beyond encyclopedic facts to fix commonsense errors in Transformers. Com- monsense knowledge involves more uncertainty and variation than encyclopedic knowledge. Con- sider a subject- -eejecttverb-object triple (s, D, o). In the encyclopedic domain, and often map to one "o", e.g., the Eiffel Tower is located in the city of "Paris". On the contrary, commonsense knowledge is harder to enumerate and and can be mapped to many "o", e.g., an apple has colors that can plau- sibly be "green", "red", "yellow", "white", and their interpolation. We aim to answer (i) whether
1 Introduction
Transformer-basee language models (LMs) have achieved great success in NLP (Brown et al., 2020) but they still exhibit factual mistakes (Lewis et al., 2020; Shuster et al., 2021), commonsense mistakes
'Co-first and last authors. Lorrain's work done at A12. 'Code and datasets for all experiments are availahle at https:// / thwbb com/ hom/anshitt wagg/mmit_css

---

commonsense plausibility information is also lo- calized in specific hidden states of Transformers, and if so, (ii) can model editing on those units ef- fectively repair incorrect commonsense plausibility judgments?
finetuned GPT-2 Large and XL models. (2) We extend the MEMIT editing algorithm MEMIT'SK by varying edit tokens and improving the edit layer selection strategy, resulting in 4.58% and 1.99% FI improvement for GPT-2 XL. on EDIT VALIDATION SET of PEP3k and 20Q. (3) GPT-2 XL edited by MEMITCsK outperforms repair-fintunee baselines by 10.97% and 10.73% F1 on the EDIT SET of PEP3k and 20Q, exhibiting favorable configuration generalization. (4) GPT-2 XL edited by MEMITCSK performs well across the affected and unaffected metrics in our constructed PROBE SET for seman- tic generalization, while fine-tuned baselines ex- hibit significant tradeoffs between unaffected and affected metrics. (5) We show that edited mod- els achieve clearer associations between judgments and localized parameters on previously incorrectly predicted samples, solidifying the correlation be- tween causal analyses and performances. These results suggest a compelling future direction of in- corporating feedback about common sense in trans- formers on the fly through direct model editing.
To this end, we focus on the subject-verb-objer binary plausibility classification task utilizing two commonsense datasets, 20 Questions (20Q, Po- rada et al. (2021)) and Physical Plausibility Com- monsense (PEP3k, Wang et al. (2018)). We per- form causal mediation analysis (Pearl, 2001; Vig et al., 2020; Meng et al., 2022) on GPT-2 Large and XL models and their fine-tuned checkpoints (Base-finetuned models), at various part-of-speech locations. While the zero-shot models perform poorly on the task and exhibit no causal pattern, we find clear causal associations between predic- tions and localized parameters subject, verb, and object locations in the Base-finetuned models. We then investigate if we can edit relevant parameters in the Sase-finetuned models to correct their mis- takes. While directly applying the MEMIT editing algorithm (Meng et al., 2023) to edit subject tokens results in sub- par performances, we extend MEMIT to MEMITeSK by editing various token locations and improving the edit layer selection strategy.
Background
The MEMIT (Mass Editing Memory in a Trans- former) method proposed by Meng al. (2023) demonstrates its effectiveness in editing up to 10,000 factual associations in transformer models on zsRE (Levy et al.. 2017) and their proposed COUNTERFACT dataset, designed to test factual recall. We describe some background here but oth- erwise refer the reader to Appendix A. and Meng et al. (2022, 2023) for more detailed description.
We demonstrate the advantage of MEMITCSK compared to fine-tuning the model ("repair- finetuning") from two angles: semantic gener- alization and configuration generalization. Se- mantic generalization requires that commonsense judgments are repaired while their paraphrases, neighbors, and reasoning-based queries are also an- swered correctly- some should be affected and oth- ers unaffected by the editing. We create PROBF SET for 20Q and PEP3k datasets to contain effi- cacy, unaffected neighborhood, affected neighbor- hood, affected paraphrase, and affected reasoning challenges. We also evaluate configuration gen- eralization for each method to determine whether a strategy (hyperparameter combination) picked on EDIT VALIDATION SET can achieve good performance on a separate EDIT SET. Our pro- posed framework for editing and evaluating com- monsense knowledge in transformers is depicted in Fig.
2.1 Causal Tracing
Given model, the method takes a concatenation of subject and verb v as input prompt and pre- diets the corresponding object o as prediction y. For a ing consists of the following three steps: Clean run The input prompt is provided to the model and the predicted probability of the correct object, PP [y]. is calculated; Corrupted run The subject tokens are corrupted with noise and the correspond- ing probability of the ground truth object, P., [v]. is computed;
(x, v) pair, causal trac-
Our contributions are five-fold. (1) We show strong causal associations between commonsense judgments and localized parameters in Base-
run The same corrupted input is given, but at a certain token and layer I, the model is forced to output the clean state activation from the clean run. In this setting, the probability of the correct object, IP ",clcan (4) [y], is computed.
WW refer to fine-tuning to repair incorrect predictions as "repair-finctuna" to differentiate from the initial fine-tuning we perform to fit GPT2 for the task ("hase-finetuningg)

---

Total effect (TE) is defined as P [ly] P. [v]. while the indirect effect (IE) of specific hidden state hy is defined as IP c.ele average total effect (ATE) and average indirect ef- fect (ATE) are computed across multiple examples for each hidden state.
particular, since model that performs near-random on task will also perform close-to-rande during a corrupted run, overall AIEs may be low as a result. This relationship has not been investigated in prior work - in contrast to the factual encyclopedic datasets used in previous studies, the zero-shot per- formance of language models on the commonsense plausibility task can be poor. Thus, we perform causal tracing on commonsense datasets in two ex- perimental settings: zero-shot (Meng et al., 2022). and after fine-tuning models on plausibility tasks; we refer to this fine-tuning as base-finetuning.
[x] P. [y]. The
Severed Causal Tracing: To disentangle the im- pact of MLP and attention in each layer, MEMIT analyzed the effect on the attention layer by fixing the MLP output at the corrupted run value, so that it is unaffected when inserting clean state h1. This can be viewed as severing the MLP effect when analyzing the effect on attention. Similarly, this can be done by severing attention layers.
Does the part of speech and model layer 3.2
locations affect causal tracing conclusions and edit success?
2.2 Memory Editing
Prior work editing encyclopedic knowledge fo- cuses subject corruption and editing since fac- tual knowledge is mostly associated with the sub- ject and the object is directly predicted. In contrast, common sense and plausibility judgments depend on each element of the sentence. Therefore, we an- alyze three types of corruption and edit locations: subject, verb, and object.
MEMIT identified the crucial parameters signifi- cantly impacting the model's prediction through causal tracing. They selected the layer with the highest AIE and its preceding layers as the edit layers R. We extend MEMIT's editing strategy, de- scribed in Meng et al. (2023), to the commonsense domain.
3 Method
MEMIT (Meng al., 2023) edits five-layer window whose last layer has the highest AIE in the severed causal graph. This strategy only consid- ered the last layer effect but ignored all the other layers in the window. To mitigate this, we consider edit layers as hyperparameter and search from a list of MEMIT's five-layer window and also the window having max moving average of AIE A detailed explanation of our layer selection strategy is presented in Appendix A.7. We denote our mod- ified editing method with varying edit tokens and a more robust layer selection strategy as MEMIT
We now set out to investigate our main research question: is commonsense plausibility informa- tion also localized in specific MLP hidden states of an LMM and, if so, can MEMIT-style editing effectively repair incorrect commonsense plausi- bility judgments?
To investigate this, we conduct experiments that address important sub-questions, focusing specifi- cally on the commonsense plausibility task (Porada al., 2021). The task is predict label y C (True, False} given an input triple x (s, v) An example can be seen in Fig. 13.
3.3 Does MEMITCSK exhibit configuration generalization?
3.1 Is high task performance needed to achieve a strong causal tracing result?
Prior work on model editing tunes hyperparame- ters and reports performances of editing algorithms the same data splits. We study configuration generalization whether editing hyperparameters pre-selected on some data can be effectively trans- ferred to an unseen data split. The motivation is that running parameter sweeps on new data points for editing can be time-consuming and costly. Since commonsense knowledge is innumerable, it is fa- vorable if users may provide contextual feedback
Because model parameter editing relies select- ing a token and layer position based the maxi- mum AIE, we hypothesize that model performance may impact the resulting causal tracing graph. In
"This dataset framing addresses challenges with the direct analog of the factual recall task (predicting y = o given x = e)) in the commonsense setting. Since there can be multiple correct object completions for commonsense, e.g., rice, meat, bread are all valid completions for the phrase People eat evaluating only the argmax completion does not rigorously assess model's 's understanding of plausibility of events.
"We also consider neighboring windows shifted by I layer, exploring windows of size 3 and 5 within this space.

---

change model behaviors on the fly using pre- selected hyperparameters. We thus create an EDIT VALIDATION SET and an EDIT SET for each dataset. We select hyperparameters on the EDIT VALIDA- TION SET and study the transferability of the best- found setting of MEMITCSK and cepair-finetuni baselines to EDIT SET ($5.3).
selves in causal patterns by repeating the causal tracing experiments on the MEMITcsk-edited and repair-finetuned models, to solidify the tie between discovery and correction.
Experimental Setup
4.1 Models
3.4 Does MEMITCSK exhibit semantic generalization?
We perform experiments on GPT-2 Large and XL (Radford et al., 2019). 6 We finetune check- points from Huggingface Transformers (Wolf et al., 2020) on Training Sets to obtain Base- finetuned models (Base Model), whose mis- takes are then repaired by MEMITcSK or repair- finetuning. The base- masse-inetuning hyperparameters in Appendix A.5. All predictions made by arg maxyc (True, False) p(yl (z) where x is common- sense subject-verb-obje statement.
It is not enough to report the success of a direct editing method the original dataset since edit methods can (and should) have propagational ef- fects on instances beyond the dataset (Meng et al., 2022). To compare and assess semantic general- ization of updates, we augment incorrectly pre- dicted samples with neighborhood instances and paraphrases that should be affected by an edit, sim- ilar to the prior fact editing work. We additionally include neighborhood instances that should not be affected. Performance the unaffected neigh- borhood measures the update's specificity, while performance on the affected neighborhoods and affected paraphrases indicates its generalization. Additionally, editing the plausibility of com- monsense statement should affect reasoning chains involving that statement. Entities and knowledge are interconnected. often requiring updates to one component of commonsense knowledge when mod- ifying another. To this end, we add a fourth cat- egory of augmentations, affected reasoning, to test whether successful edits correct aspects a model's commonsense reasoning. The augmenta- tions, which form the PROBE SET, are excluded during editing and solely used for evaluation pur- poses. We provide examples in Fig. 1 and Table 1.5
4.2 Data and Evaluation
We use Porada et al. (2021)'s versions of two com- monsense plausibility datasets. PEP3K and 20Q. We build three splits from each dataset: Training Set, EDIT VALIDATION SET, and EDIT SET. Since zero-shot GPT-2 Large and XL perform poorly on РЕР3K and 20Q out-of-the-box, we create Training Sets for base-finetuning the models on the task.
The Training Set and EDIT VALIDATION SET formed by randomly dividing the validation set from Porada et al. (2021) into 80%-20% split. The EDIT SET is created using the test set from Po- rada et al. (2021). Because both datasets' instances are unnatural (e.g., "man swallow paintball"), we use GPT-3 text-davinci-003 to reformat them into natural language while retaining the (s, v, o) format, e.g., "A man swallows a paintball". More details and dataset statistics are in Appendix A.2.
3.5
Does MEMITCSK outperform finetuning for repairing commonsense knowledge?
We report three metrics on the EDIT VALIDA- TION SET and EDIT SET: F1 Score (t), a measure of overall performance; Efficacy (t), the percent- age of previously-incored predictions which are corrected by an update method; and Relapse (J), the percentage of instances which were previously predicted correctly but rectly following update.
To answer main research question, we com- pare MEMITcSK applied the MLP hidden states most strongly identified by our causal tracing ex- periments against finetuning baselines, which we refer to as repair-finetuning. We compare both methods' performance on edit efficacy (how many incorrect predictions are fixed), overall Fl score and relapse (how much the edit hurts by changing previously correct predictions), and semantic gen- eralization metries. Unlike prior work, also in- vestigate whether such improvements exhibit them-
predicted incor-
4.2.1 Constructing the PROBE SET
For the subset of EDIT SET that was incorrectly pre- dicted by both GPT-2 Large and XL Base Model,
"We report GPT-2 XI. results in the main paper. GPT-2 Large results and similar findings are in the Appendix.
'More details about datasct construction are in $.

---

Plousibilitt
Unaffected Neighhorhood
Affected Neighborhood
Statement
Affected Paraphrase
Affected Reasoning
I Label
PRP3
Rocks ahsochs oil absorbs
Dirt absorts oil
Ground takes oil up
Oil liquid, so it spreads over surfice Soil pocous, absorb oil
Soil abscebs oil
consumes oil
Soil alscebs grease
Land alsoels oil
House kick hall Tro: kick rock
Plant kick ball
Tree used propel ball
Tree doesn's it have legs Legs are noeded kick tull
Tree kick
Falso
Tree strike
Thoe was usadl kick ball
Thee kick sphere
Thee was used to hit ball
20Q
Trees block sun Sunglasses block rain
Shades black sun
Sunglaeees nct a shickl from san Sunglasses obstruct the sun's 's light
Sunglavess have dark lonses Dark lenses reduce light eyes
Sunglasses black
True
Sunglasses obscure
Sunghases block light
Senglassce filter out sun's brightnoss
Computers make noise Furnishings make
Fictures make notse
Parntture can be noiey
Purnishings are inanimate objects Inanimase objects cannot
Furnishings make
Furaisbings produce noise
Fernitue can create sound
Fursishings rake sound Funnhrr con be souurc of noise
Table Examples chosen through random sampling from the PEP3K and 20Q PROBE SET. Unaffected neighborhood samples are created by individually augmenting the subject and object with different, but relevant instances from the source statement. Likewise, affected neighborhood samples are created by individually augmenting the subject, verb, and object with synonymous instances from the source statement. Further details are in $4.2.1.
we augment each instance with neighborhood in- stances that should or should not be affected by an edit that fixes the incorrect prediction on the dataset instance using GPT-3 (details in Appendix A.9). We combine the incorrectly predicted instances from EDIT SET and the per-instance augmentations form the PROBE SET for evaluating semantic generalization. Dataset examples are in Table and statistics in Appendix A.2.
with the phrase "Furnishings do not make noise". Rí could be "Furnishings are inanimate objects". and R₂ "Inanimate objects cannot make noise". The metric is the percentage of post-update predic- tions arg max P(R)) and arg max P (R2) which are equal to the True label.
4.3 Editing and Finetuning Methods
Unaffected Neighborhood. To evaluate the specificity of the edits, for each {s, v, o}, we gener- ate set of relevant but different instances (s',u, and (s, v, o') that should not change when {s, v. is edited. The metric measures the percentage of post-update predictions arg max IP('s, u, o) and arg max (Psss, v. o') that remain equivalent to pre- update predictions.
We seleet hyperparameters to maximize Fl on the EDIT VALIDATION SET ($3.3). For editing, we search for the edit layer range, edit token position (last {s, v, o)), and learning rate. For the repair- finetuning baseline, we search for the learning rate, batch size, and the number of epochs.
Affected Neighborhood. To assess the im- pact of changes on similar meaning prompts for each (s, u. o), we generate set of synonyms as (5', v,,),, o), (5, and o')). The score measures the percentage of post-update predic- tions arg max PP(s', u, o), arg max (P(s,, v', o) and arg max (ss v, o') which are equal to the ground truth label for 0)..
For editing, we perform causal tracing on the correctly-predicte samples of the EDIT VALIDA- TION SET to inform layer selection. We apply repair-finetuning and editing methods to repair incorrect predictions EDIT VALIDATION SET, EDIT SET, and PROBE SET.
We explore two variants of repair-finetuning. Fixed Epoch uses the same exact configuration found on EDIT VALIDATION SET. We hypothe- size that it is prone to overfitting due to the ab- sence of early stopping. To maximize the potential of finatrnfnnetninn, we analyze another variant RFT Early Stop- which runs for maximum of 10 epochs and selects the checkpoint with the highest Fll score on the entire EDIT SET. This should miti- gate overfitting and reduce relapse. In contrast, the editing experiments always use the exact configu- ration obtained from EDIT VALIDATION SET.
To evaluate the im- pact on synonymous prompts, we generate ('') Since para- phrases should also be successfully edited, the metric is the percentage of post-update predictions arg max (P(s'', P(s'',', 00,,, o²) which are equal to the ground truth label for (s, o).
Affected Paraphrase.
set of paraphrases
Affected Reasoning. To assess the updated model's connectivity, we generate a two-step chain valid reasoning prompts {R1, R2). For instance,

---

5 Results & Discussion
at the earlier layers for the corrupted tokens. This linding non-trivial and emphasizes the impor- tance of earlier layers while predicting plausibility. AIE is more pronounced at the last corrupted token compared to the first corrupted token consistently acToss all models and datasets. Therefore, we fo- cus the last (s, U, o) editing. Additional causal tracing results are present in Appendix A.10.
5.1 High task performance is crucial for achieving strong causal tracing results
Zero-shot prompting produced near random accu- racies (51.30% and 51.87% on the EDIT VALIDA- TION SET split of PEP3K and 20Q respectively for GPT-2 XL) and chaotic causal patterns with no lo- calization as shown in Fig. 27. In contrast, the Base Model exhibited significantly superior performance (77.12% on PEP3k and 73.96% on 20Q) and the resulting causal patterns were more distinct with substantially higher ATE and strong localization. Therefore, deduce that a significant correlation exists between high task performance and strong causal patterns, and use the Base Model for editing experiments.
Fig. 4 compares the average AIE last corrupted token for unmodified, severed MLP and Attention causal graphs for all edited tokens. We notice a clear gap in AIE for MLP graphs at the earlier layers. This observation aligns with previous ob- servations in MEMIT for encyclopedic knowledge. In contrast to encyclopedic facts, we observed the highest AIE in earlier MLP layers instead of middle layers. This demonstrates the importance of earlier layers in commonsense predictions. Interestingly, in the object corruption plot, we observed a small peak at the beginning, before the highest AIE later. We thus expanded the hyperparameter space to in- clude the initial layer windows for the object edit layers. Table 2 presents edit layers included in hy- perparameter search with the max moving average of ATE, comparing windows of size 3 and 5 using different editing tokens {s, 0, o)- In all cases, the max moving average resulted layers selected than MEMIT, where the max ATE layer is used to edit layers- the selected layer and the previous layers.
Avg Indirect Effect of 1, over 236 prompts
subject token
subject
0.100
Subject
Hirs vero laken
0.075
Last Vern Taken-
Verb
Object loken
Object
Object
0.025
Last token
0.000
5 10 20 23 30 40
AIE
single patched layer within GPT2-XL
(a) Base Model with 77.12% accuracy.
Avg Indirect Effect of n! over 157 prompts
different set of
subject
subject
0.0020
Sa s.jjett
First Verb Token
Lest Verb Token
0.3015
First
Verb
0.3010
Last Object loken
Object
0,0005
Last token
0,3000
AIE
Layers with Max Moving Average AIE
single patched syer within GL
Model
Edit Token
Layer
Max AIE
(b) Zcro-shot model with 51.30% accuracy.
Window=3
Window
GPT-2 Large
Last Subject
8
8,9.00
8.9,10,11.12
Zero-shot GPT-2 XL causal tracing on PEP3k EDIT VALIDATION SET. Pat- terns are unclear for the Zero-shot model while they are distinct for the Base Model. Consistent observations found for the 20Q dataset (Fig. 6).
Figure Base Sase-finetuned
Large
Verh
4,5.6
4.5.6.7.8
GPT-2 Large
Last Object
11,12.13
10,11.12.13.14
GPT-2 XL
Last Subject
5
4,5.6
2.3,4.5.6
GPT-2 XL
Last Verb
5
5,6,7
3,4.5.6,7
GPT-2 XL.
Last Object
12
10,11,12
9,10,11,12,13
Table 2: Layer with max AIE and set of layers with max moving average AIE for the PEP3k EDIT VALIDATION SET
5.2 Targeted part of speech and layer locations affect causal tracing conclusions and edit success
These two changes resolve in MEMITeSK. Ta- ble 3 compares original MEMIT (only sub- ject edit with fixed edit layers) with the best- performing edit of MEMITESK EDIT VALIDA- TION SET. MEMITESK consistently outperforms MEMIT across datasets and models.
As shown in Fig. 3. the last token the later layers has a high AIE which is trivial since fixing hidden states or MLPs in those layers restores most of the required information. We also observed strong AIE
"Normalization with domain conditional Pointwise Mutual Information (PMI; Holtzman et al., 2021) did not result in any significant improvement with accuracy PMInc = 52.94% on PEP3k.
8 Detailed results are in Appendix A.4.

---

indirect Effect of ht 236 prompts
Aug Inditec
prompls
Aug Indited.
over 236 promppe
sutject
First sucject toten
Firs: subrect token
sueject
0.300
Sul
0.100
Susjast
0.300
Last Vert leecn
0.275
Last Vert Toten
0.075
Last Verb Thken
Werb
0.075
Rist Osject Teken-
0.050
Hist cijject tokin
0.050
Hid citject taken
0.050
Lend Oigici ldvan
(ast Onjec
Obyer
Ciyect
Onject-
Last token
10 25 50 40
AE
Lost tokio
10 25 30
NT
Last tokio
10:15 25 4D
0.000
single patched layar withi GPT2VL
cencer interva patched nipp layers
center of inter patched layers
Best AIE on last subject token = 0.078
Best AIE on last subject token = 0.085
Best AIE on last subject token 0.029
(a) Subject corruption
Avg indirect Effect of At over 236 promets
Avg Indirect Effect
236
prompts
Aug Indirect Effect 236 prompts
Last Subject toven
0.15
Last tokan
subject
015
Finst wob
0:35
Subject
Sutject
Sucject
Last Vert Towen
0.30
Lost Vera Taken
Vert
010
Rist verb
Last Wolb Iaken
0.30
firt Obtect Tieen
Last Obtect
0.05
Firtt Object
Last O0ject
Tiken
0.05
Rist Object Taken
Ubjec
0.25
Object
00yecc
Cajast
Last ldeen
Lest loten
Last token
5 10 15 25 30 35 60
10 15 25 30 35 40
NE
10 20 25 30 40
single patched lnyer within GPT2XL
center of interva patched mip lnyers
center of inte patched aetn Isyens
Best AIE on last verb tnken = 041
Best AIE on last verb token = 0.148
Best AIE on last verh token = 0.033
(b) Verb corruption
Avg indirect Lffect of n! ever 2Jb prompts
Avg Indirect Lffect MLP
Firs Indirect Effect
238 prompts
subject
Lest suteett tocen
0.35
Last subject takan
0.15
sublect towan
0.15
Hiat with tokin-
Veri Token
First Weth Theen-
Laxt Wert moen
0,30
Last Vero Taken
0.10
Last Verb Telicn
0.30
Rist Object Telch-
First Object Ioten
First Object Thkee
Oiject
0.05
Object
0.5
Obfect
Object
coyect-
Orject-
Last loeen
Last tokan
tolknn
10 15 25 30 35 60
10 15 20 25 30 35 40
NE
10 15 20 25
25 40
kingin patcher layer within GP12M1
center of intervs of pabtted mip layers
centre of internal 10 patchee with Layans
Best AIE on last object token 0.186
Best AIE on last object token = 0.184
Best AIE on last object token = 0.056
(c) Object corruption
Figure 3: Causal tracing for GPT-2 XL. Base Model on PEP3K EDIT VALIDATION SET when different tokens are corrupted, (s, v, of (in order). See Appendix A.10 for GPT-2 Large and 20Q results.
Average Indirect effects at last subject token
MEMIT F1 Score %
MEMIT
Dataset
Model
Indrect Effect
0.075
Impact of single state on P Impact with Altn severed
F1 Score %
8.050
GPT-2 Large GPT-2 XL
88.53
93.78 (+5.25)
Impact with MLP severed
PEP3K
0.025
90.51
95.09 (14.58)
0.000
20Q
GPT-2 Large
85.31
87,09 (+1.78)
10
20
30
10
GPT-2 XI.
90.32
92.31 1.99
Average Indirect effects at last verb token
0.15
Table 3: Comparison of MEMIT and best performing MEMITeSK on EDIT VALIDATION SET. MEMIT editing on s. while MEMITeSK on best among (s, v, o).
Indirect Effect
Impact single state on
0.10
Impact with Attn severed
Impact with MLP severed
0.05
30
5.3 MEMITCSK exhibits configuration generalization
Average Indirect effects at last object token
Indircct Effect
0.15
Impact at single state on Impact with severed Impact with MLP severed
Table 4 reports GPT-2 XL results for EDIT VALI- DATION SET and EDIT SET 9, The GPT-2 Large results are in Appendix A.6 Table 12. For the EDIT VALIDATION SET performance, the verb edit Fl score is higher by +17.97% % compared to the Base
0.05
0.00
30
Figure 4: Severed causal tracing results for {s, U. of for
GPT-2 XL. base on PEP3K EDIT VALIDATION SET
"The best hyperparameters are detailed in Appendix A.5. The KI. divergence and cut-off factors can potentially enhance the performance for editing methods, see Appendix A.8.

---

Dataset
Update
Token
Layers
EDIT VALIDATION SET
EDIT
Method
F1 Score %
Efficacy %
Relapse %
Fl Score %
Efficacy %
Relapse
Base Model
77.12
76.47
0
RFTEaly Sup
90.16 (+13.05)
97.14
11.87
80.93 (+4.46)
50.83
9.82
PEP3K
RETrssed
90.16 (+13.05)
97.14
11.87
56.89 (-19.58)
98.89
55.25
Fait
Last Suhject
1,2,3,4,5
90.51 (+13..9)
80
6.36
84.72 (+8.25)
77.22
12.98
Edit
Last Verb 6,7,8
95.09 (+17.97)
92,86
4.24
91.90 (+15.43)
88.33
7.00
Edit
Last Object 3,4,5
94.43 (+17.32)
91.43
4.66
86.69 (+10.22)
72.78
8.97
Base Model
74.73
75.77
0
RFTTEe Sop
85.71 (+10.98)
80.46
12:40
77.36 (+1 .60)
30.97
7.8
20Q
RFTrreed Ejoch
85.71 (+10.98)
80.46
(-27.74)
88.63
Edit
Subject 2,3,4.5,6
92.31 (+17.58)
79.69
3.43
(+10,70)
65.73
6.90
Edit
Verb
82.64
44.53
79.03 (+3.27)
Fdiit
Last Object 1,2.3
(=112())
89.06
8.18
88.09 (+12.33)
76.60
8.21
Table 4: Configuration generalization results based on the best hyperparameters identified for EDIT VALIDATION SET and applied to EDIT SET for GPT-2 XL. The editing methods display high configuration generalization compared to appir--inetuning Refer $5.3 for further discussion. GPT-2 Large results are in Appendix A.6 Table 12.
Model in PEP3K. The object edit Fl score is higher by +17.58% % in 20Q. This indicates the importance of varying editing tokens. The best editing method outperforms eppai-ffinetningg baseline consistently for both datasets with much lower relapsed scores.
Avg Indirect Effect of itth over 67 prompts
First subject token
Last subjest token
0.20
Subject
0.15
Verb
Verb
0.10
First Object Token
Last Object Token
Object
The editing method continues to perform well after transferring the best hyperparameters to EDIT SET; in comparison, both repair-finetunin base- lines performance drops significantly. Noticeably, RFTeixed Epoch method has high efficacy but much higher relapse score, between 15.36-64.9661 64,96% caus- ing significant decrease in the Fl score due to overfitting. The three editing methods on {s, v, o} outperform the repair-finee finttr-fnettning methods by 10.54- 15.43% % for the updated Fl score, exhibiting a bet- ter configuration generalization performance.
Last token
5 20
AIE
single patched layer within GPT2 XL
Verb corruption, best AIE on last verb token = 0.200
(a) RF Sop model with 90.16% Fl Score
Aug Indirect Effect 6, over prompls
First subject
Last subject token
Subject
Firtt Verb Tocen
Lest Verb Token
Object
Last Taken
Object
0.1
Last token
0.0
5 10 20 25 40
single patched layer within GPT2XL
5.4 MEMIT exhibits semantic generalization
Verb corruption, best AIE last verb token 0.468
(b) MEMITesK v edited model with 95.09% F1 Score
Table 5 shows GPT-2 XL results PROBE 10 Compared to the editing methods, the repair- finetuning baselines struggle to balance the affected and unaffected samples. RFT Early Stop performs well in unaffected neighborhoods but struggles with the affected statements (measured by aver- age). Fixed Epech reached higher performance affected subsets but suffered with unaffected neighborhoods. In comparison, the editing meth- ods showed balanced improvements across metrics.
Figure Causal tracing for GPT-2 XL models on suc- cessfully corrected statements in the PEP3k EDIT VAL- IDATION SET. For the RFT Stop model, we observe similar patterns as Fig. for both token corruptions. For the edited model, improved pattern is observed at
5.5 MEMITCSK outperforms fine-tuning for repairing commonsense knowledge
To measure improvement, we re-conduct causal analysis via each token {s, 0, corruption us- ing successfully edited statements. Fig. 5 dis- plays the causal graphs for best best-performing edit: v edited model and the best repair- finir-ffnnttned model:
We also noticed that the affected neighborhood scores are generally high except for the specific editing token; e.g., while editing the object token, the affected object neighborhood score low.
RFT Early Stop based on Table 4.
10 BBas Model has 0% efficacy on PROBE SET by design.
For RFT Stup (F1 Score 90. 16%), the overall

---

Unaffected Neighborbood
Affected
Update Method
Emicacy %
Affected Paraphrase
Affected Reasoning
Average Unaffected
Average Affected
Dataset
Edit Token
Neighhorhood
Subject
Object
Subject
Verb
Object
%
%
Model
100
23.45
33.64
100
26.51
RFT sto
39.63
77.21
76.98
33.88
37.5
39.16
39,91
55.66
77.09
41.22
RFT tpach
99.62
24
27.78
86.35
84.31
87.15
68.7
62.26
25.89
77.75
PEP3
Edit
Last Subject
72.08
81.21
64.15
27.75
56.83
58.67
47.92
35.85
72.68
45.40
Last Verb
87.92
57.06
57.36
71.05
37.41
58.15
Edit
Last Object
75.47
58.11
57.82
57.52
54.35
27.86
43.99
34.72
57.96
43.69
Base Model
0
100
100
33.02
24.78
29.38
33,70
39,38
100
32,05
RFT Sup
21.52
87.01
86.89
36.21
34,72
37.01
38.27
35.41
86.95
36.32
20Q
RFT Epoch
79.27
38.85
37.89
64.41
64.44
60.20
38.37
Last Subject
87:22
71.00
34,48
57.47
58.82
51.75
37,93
79.11
48.09
Edit
Last Verb
35,70
90.18
83.21
37.44
29.80
47.25
35.91
37.80
86.69
37.64
Edit
Last Object
72.18
76.24
92.26
61.90
60.37
33.76
47.87
41,90
84.25
49.16
Table 5: Efficacy and semantic generalization results on PrObe SET for GPT-2 XL. Balanced improvements are observed for editing methods across metrics, with the and edits performing the best. Refer to $5.4 for a detailed discussion. GPT-2 Large results are in Appendix A.6 Table 13.
causal pattern and AIE remain similar to the Base Model Fig. 3. In contrast, the 7 edited model (FI Score 95.09%) shows an enhanced ATE for all types corruption. Specifically, a high AIE of 0.468 is recorded at the last verb token for verb corruption. These findings confirm that localization and AIE improve for the edited model at the edit location.
with MEMIT; this partially motivates our exten- sive layer hyperparameter search. Recent work by Cohen al. (2023) proposes a dataset for evalua- tion of a variety of ripple effects in editing methods with factual knowledge and concludes that models fail to capture these effects. All aforementioned works focus on encyclopedic factual knowledge, unlike ours.
6 Related Work
Early works on model editing focused on updat- ing individual neurons using constrained finetuming (Sinitsin et al., 2020; Zhu et al., 2020) or hypernet- works (De Cao et al., 2021; Mitchell et al., 2022a; Hase al., 2023b). A related line of work has focused on storing updates in an external memory (Jin al., 2021; Mitchell et al., 2022b; Tandon et al., 2022, inter alia). Recent works (Hoelscher- Obermaier et al., 2023; Zhong et al., 2023; Brown et al., 2023; Onoe et al., 2023) offer more compre- hensive evaluations for fact-editing methods. Inspired by the linear associative memory prop- erty of feedforward layers in Transformers (Ander- son, 1972; Geva et al., 2021, 2022) and success with the approach in convolutional models (Bau et al., 2020), recent works have proposed to edit MIP weights directly (Meng et al., 2022; Dai et al., 2022; Yao et al., 2022). In the encyclopedic factual domain, Meng et al. (2022) proposed to edit single facts by fitting a Rank One Model Edit (ROME) to the parameters of an MLP layer, and showed it outperformed prior methods. Our work builds on Meng et al. (2023), which extended this approach to thousands of edits by altering the weights of a range of MLP layers. Hase et al. (2023a) demon- strate that many early edit layers can work well
7 Conclusion
This paper demonstrates strong causal relations be- tween commonsense plausibility judgments and early MLP layers in Transformers. These param- eters are directly editable for repairing common- sense mistakes. We improve the MEMIT parameter editing algorithm to MEMITCSK for commonsense plausibility prediction by varying edit tokens and by improving the layer selection strategy. GPT-2 Large and XL models edited by MEMIT out- perform repair-finetundd baselines by more than 10% Fl score on EDIT SET. Additionally, we con- struct a PROBE SET that contains unaffected and affected neighborhoods, affected paraphrases, and affected reasoning challenges for comprehensive evaluation. MEMITCSK effectively generalizes on related and unrelated neighborhoods annotated in our PROBE SET, exhibiting semantic generalization while repair-finetuned baselines demonstrate sig- nificant trade-off's between unaffected and affected metrics. These results indicate compelling di- rection of incorporating feedback about common sense in transformers on the fly through direct model editing.

---

Limitations
Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 5185-5198, Online. Association for Computational Linguistics.
In this work, we experiment with repairing com- monsense mistakes by the GPT-2 Large and XL models. We are unable to investigate larger open- sourced models like GPT-J (Wang and Komat- suzaki, 2021) and GPT-NeoX (Black et al., 2022) due to resource limitations. Investigating the re- search questions described in $3 on larger models is a natural next step. We focus on the binary plau- sibility prediction task but envision that parameter editing could improve models on various common- sense tasks in future work.
Prajjwal Bhargava and Vincent Ng. 2022. Common- sense knowledge reasoning and generation with pre- trained language models: A survey. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 36, pages 12317-12325.
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Puro- hit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. GPT-NeoX-20B. An open- source autoregressive language model. In Proceed- ings of BigScience Episode #5 Workshop on Chul- lenges & Perspectives in Creating Large Language Models, pages 13-136, virtual+Dublin. Association for Computational Linguistics.
Our experiments show that the optimal edit to- ken (subject, verb, or object) varies among datasets. The specific location of single generalized opti- mal edit token, if exists, requires further inves- tigation, while different editing methods for com- monsense knowledge can be proposed.
Ethics Statement
Davis Brown, Charles Godfrey, Cody Nizinski, Jonathan Tu, and Henry Kvinge, 2023. Robustness of edited neural networks, ArXiv preprint.
This study proposes a framework to evaluate and correct commonsense mistakes in GPT-2 models, focusing on predicting the plausibility of com- monsense statements. Commonsense knowledge is highly contextualized and varies significantly across locations and cultures. Biases and stereo- types present in edit datasets may inadvertently lead to erroneous and potentially harmful model judgments. Malicious actors may exploit model editing to incorporate false information into mod- cls. It is crucial to employ meticulously curated datasets in future research and during the deploy- ment of these models in
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffney Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin. Scott Gray. Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei, 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
scenarios.
A Acknowledgments
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023. Evaluating the ripple effects of knowledge editing in language models.
This research was conducted at the University of Massachusetts Amherst under the Industry Men- torship Program led by Prof. Andrew McCallum. We are grateful for their support and resources pro- vided for this rescarch.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers). pages 8493- 8502, Dublin, Ireland. Association for Computational Linguistics.
References
James A. Anderson. 1972. A simple neural network generating an interactive memory. Mathematical Biosciences, 14(3):197-22) 220.
Nicola De Cao, Wilker Aziz, and Ivan Titov, 2021. Edit- ing factual knowledge in language models. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 6491- 6506, Online and Punta Cana, Dominican Republic. Association for Computational Linguisties.
David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, and Antonio Torralba. 2020. Rewriting a deep gen- erative model. In Computer Vision ECCV 2020, pages 351-369, Cham. Springer International Pub- lishing.
Ashwin Devaraj, William Sheffield, Byron Wallace, and Junyi Jessy Li. 2022. Evaluating factuality in text

---

simplification. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 7331-738 1-7345, Dublin, Ireland. Association for Computational Lin- guistics.
eearieval-augmentte generation for knowledge- intensive nip tasks. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 9459- 9474. Curran Associates, Inc.
Gary Marcus. 2021. Experiments testing gpt-3's ability at commonsense reasoning: results. Blogpost.
Mor Geva, Avi Caciularu, Kevin Wang. and Yoav Gold- berg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 30-45, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual asso- ciations gpt. In Advances in Neural Information Processing Systems, volume 35, pages 7739-117372 Curran Associates, Inc.
Mor Geva, Roei Schuster, Jonathan Berant, and Omer layers are key-
Kevin Meng, Arnab Sen Sharma, Alex Andonian. Yonatan Belinkov, and David Bau. 2023. Mass- editing memory in transformer. In The Eleventh International Conference on Learning Representa- tions.
Levy. 2021. Transformer value memories. Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 5484-5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022a. Fast model editing at scale. In International Conference on Learning Representations.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghan- deharioun. 2023a. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. ArXiv preprint.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D Manning, and Chelsea Finn. 2022b. Memory- based model editing at scale. In Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pages 15817-15831. PMLR.
Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zor- nitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan lyer. 2023b. Methods for measuring, up- dating, and visualizing factual beliefs in language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Compu- lational Linguistics, pages 2714-2731. Dubrovnik, Croatia. Association for Computational Linguisties.
Yasumasa Onoe, Michael JQ Zhang. Shankar Padman- abhan, Greg Durrett, and Eunsol Choi. 2023, Can Ims learn new entities from descriptions? challenges in propagating injected knowledge. In Proceedings of the 6l Annual Meeting of the Association for Computational Linguistics, Toronto, Canada, Associ- ation for Computational Linguistics.
Jason Obbemaier,,,,r,rirr,....... Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023. Detecting edit failures large language models: An improved specificity benchmark. ArXiv preprint.
Judea Pearl. 2001. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Un- certainty in Artificial Intelligence, UA1'01, page 41 411-420, San Francisco, CA, USA. Morgan Kauf- mann Publishers Inc.
Ari Holizman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form com- petition: Why the highest probability answer isn't always right. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 1038-7051 Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Ian Porada, Kaheer Suleman. Adam Trischler, and Jackie Chi Kit Cheung. 2021. Modeling event plau- sibility with consistent conceptual abstraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 1732- 1743, Online. Association for Computa- tional Linguistics.
Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren, 2021. Gradient-based editing memory examples for on- line task-free continual learning. In Advances in Neural Information Processing Systems.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blogpost.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the s1 Conference on Computational Natural Language Learning (CoNLL 2017), pages 333-342, Vancouver, Canada. Association for Computational Linguisties.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784-3803, Punta Cana. Do- minican Republic. Association for Computational Linguistics.
Patrick I Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020.

---

Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin. Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In International Conference on Learning Representations.
In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4149-4158, Minneapolis, Minnesota. Association for Computational Linguistics.
Yunzhi Yao, Shaohan Huang, Li Dong, Furu Wei, Huajun Chen, and Ningyu Zhang. 2022. Kformer: Knowledge injection in transformer feed-forward lay- ers. In Natural Language Processing and Chinese Computing, pages 131-143, Cham. Springer Interna- tional Publishing.
Zexuan Zhong, Zhengxuan Wu, Christopher D. Man- ning, Christopher Potts, and Danqi Chen. 2023, Mquake: Assessing knowledge editing in language models via multi-hop questions. ArXiv preprint.
Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2022. Evalu- ating the factual consistency of large language mod- els through summarization. ArXiv.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. ArXiv preprint.
Niket Tandon, Aman Madaan, Peter Clark, and Yiming Yang. 2022. Learning to repair: Repairing model out- put errors after deployment using dynamic memory of feedback. Findings of the A Association for Com- putational Linguistics: NAACL 2022, pages 339-352, Seattle, United States. Association for Computational Linguistics.
A
Appendix
A.1 Causal Tracing Background
Given a model, the method takes a concatenation of subject & and verb v as input prompt x. then predicts the corresponding object o as prediction y. For example, for the statement "Paris is the capital of", model is tasked with predicting "France" as the most-likely next token. Taking a correctly predicted x, y pair, Causal tracing consists of the following three steps:
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo. Yaron Singer, and Stu- art Shieber. 2020. Investigating gender bias in lan- guage models using causal mediation amalysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388-12401, Curran Associates, Inc.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model. https: / /gi thub. com/kingofllll/ mesh-transsormer-
Step 1: clean run. Given the input prompt v, they collect all hidden activation values {M} | E [1,T), c [1, [.] from the model, where T is number of input tokens in x and L is num- ber of model layers. Concretely, for each input x, M(x) = (x) + af(x) + mx(x) where af is the attention value and mł is the corresponding MLP value. The predicted probability of the correct ob- ject is denoted as P [v]]
Su Wang, Greg Durrett, and Katrin Erk. 2018. Model- ing semantic plausibility by injecting world knowl- edge. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, Volume 2 (Short Papers), pages 303-308, New Orleans, Louisiana. Association for Computational Linguisties.
Step 2: corrupted run. In this setting, cer- tain part of the input prompt x is corrupted with noise. In a clean run, x is embedded as 11 11 ... 1...k( However, here, they set
Rongxiang Weng, Heng Yu, Xiangpeng Wei, and Wei- hua Luo. 2020. Towards enhancing faithfulness for neural machine translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2675-2684, Online. Association for Computational Linguistics.
h = h e.. for all tokens in the subject token The probability of ground truth value y produced in this run is denoted as P. [y]. Note that the model prediction is likely to be incorrect due to the noisy input.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-art natural language processing.
"e ~ (O v) and v is taken as three times the empirical standard deviation of the emheddings corresponding to the subject tokens.

---

Dataset
Nrain
NEV
NE
Np
in games of "20 questions" and labeled as plausible or implausible. We use the (s,w,)) format of the dataset constructed by Porada et al. (2021), where
PEP3k
1,225
306
1,531
265
20Q
2,006
507
2,548
381
x - (s, U, o) and E {True, False).
Examples from each dataset are given in Table 1, Statistics of our created data splits are in Tables 6 and 7
Table 6: Number of samples in the Training Set, EDIT VALIDATION SET, EDIT SET, and PROBE SET.
Type
NPEr3K
N200
A.3
Base Model vs. Zero-Shot for 20Q Dataset
Original statement
265
381
Unaffected subject neighborhood
1,325
1,894
Comparison of Base Model and zero-shot model for the 20Q dataset is in Fig.
Unaffected object neighborhood
1,325
1,900
Affected subject neighborhood
1,290
1,856
Affected verb neighborhood
1,288
1,832
Affected object neighborhood
1,292
1,848
A.4
Original MEMIT Editing Results
Alfected paraphrase
1,323
1,905
Table shows the detailed metrics and editing pa-
Affected reasoning
530
754
rameters for MEMIT applicd on EDIT VALIDATION
SET.
Table 7: Number of samples in PROBE SET.
Edit
Editt
F1 Updated
Fiflicay
Relager
Dataset
Model
Tokon
Layers
%
%
The model runs inference using the noisy input embed- ding created in the corrupted run, with the differ- ence that the model is also forced to output the clean state activation by at certain token i and layer i. If the model successfully produces the correct output using small number of clean states, there is likely be strong casual relationship between these states and the model output. The probability of the correct object denoted as clean 6,99 [w]-
Step 3: weruupeedwiit--resttratoooo
PEPSK
GPT-2 XI.
Large
Subjaet
Subject
1,2,4,45
4,5667788
88.53 (+13.36)
90.51 (+13.39)
76.32
80
6,36
GPT-2 Large
Subjoct
1,233445
85.31 (-12.92)
71.43
9.26
Subject
84.38
Table 8: Editing results after applying original MEMIT on EDIT VALIDATION SET.
A.5 Hyperparameters
Base Finetuning The GPT-2 Large and XL mod- els are initially finetuned on the training set with the next-token prediction objective. Table 9 presents the optimal hyperparameters identified for the base- finetuning method.
The three runs produced PP [v]. IP. [y] and cl, hf [w]. Two metrics are then defined to mea- sure the states effect between these runs. Total effect (TE) is calculated as IP [y] - P. [v], while the indirect effect (IE) of specific hidden state h² is calculated as P clean ( [v] P. [y]. The average total effect, ATE and average indirect effect, i.e. AIE, are computed across multiple examples for each hidden state.
Dataset
Model
Learning Rate
Batch Size
Epochs
GPT-2 Large GPT-2 XL
0.00009961
64
10
20g
1.00004432
64
10
GPT-2 Large GPT-2 XI.
0.00002298
10
PEP3k
0.00001023
32
20
A.2 Datasets
Table 9: Base Model hyperparameters for Training Set
Physical Event Plausibility (PEP3k; Wang al., 2018) consists of 3,062 statements in (subject s, verb v. object o) format about semantically plau- sible and implausible events. It covers a wide range possible (but not necessarily common) events with high annotator label agreement.
Repair Finetuning
Table 10 shows the best hyperparameters for the repair-finctuning method. The method was very sensitive small changes in learning rate while the other parameters worked well over long range of values. Note that we use early stopping and restore the weights to the best performing model based on the Fl score.
20 Questions (20Q) is a dataset of 5,096 com- monsense statements written by crowd annotators

---

Avg Indirect Effect hy over 375 prompts
Avg Indirect Effect ii"" over 263 prompts
subject
subject
0.003
subject
subject
Subject
Subject
First Verb lakan
0.2
Firt Verb
0.002
Lest Verb Token
Lost Verb Token
Verb
Vero
Hirs Object taken
First Object loken
Last Object Token
Last Dbject Token
Objazz
Object
Last token
0.0
Last token
0.000
25
AIE
AIE
single patched layer within GPT2-XL
single patched layar within GPT2-XL
(a) Base Model with 73.96% accuracy.
(b) Zero-shot model with 51.87% accuracy.
Figure 6: Zero-shot vs, Base Model causal tracing results for GPT-2 XL on 20Q EDIT VALIDATION SET.
Dataset
Model
Learning Rate
Batch Size
Epochs
Large results for semantic generalization experi- ments are in Table 13.
GPT-2 Large
0.000003451
8
20q
GPT-2 XL
0.000001589
32
9
A.7
Layer Selection Strategy
GPT-2 Large
0.00000474
PEP3
GPT-2 XL
0.000001313
8
For demonstration purposes let's assume model has only 10 layers. The average indirect effects of these layers at our desired edit token (let's assume last verb token) are:
Table 10: Hyper-parameters for RFTed Epoch tuned for EDIT VALIDATION SET and applied to EDIT SET and
PROBE SET
(0.0, 0.1, 0.2, 0.3, 0.5, 0.1, 0.4, 0.3, 0.2,0,0000
MEMIT CSK
Let's also assume that we are considering only layer windows. The highest average indirect effect is observed at the 5th layer with value 0.5. According to MEMIT, the optimal edit layers will be 5 layer window ending at the highest AIE layer, in this case it will be the layers 1,2, 3, 4,
The Table 11 shows the hyper-parameten for the editing method. The method was slightly sensitive to the learning rate and very sensitive to the edit token. Note that a KL. divergence factor of 0.0625 was used as the default value for all editing experi- ments. Appendix A.8 contains an ablation study of the KL. divergence factor.
Now let's calculate the moving average of 5 layer windows. The moving average of layers 1-5 is (0.0 + 0.1 + 0.2 0.3 0.5)/5 0.22, similarly the moving average layers 2-6 will be (0.1 + 0.2 + 0.3 + 0.5 + 0.4)/5 = 0.3 and so on. The moving averages of all layer windows are:
Dutasct
Model
Edit Token
Layers
Learning Rate
Last Subject
3,4.5
0.7868
GPT-2 Large
Last Verb
2,3.4,5,6
0.09393
Last Object
1,2,3
0.6276
20g
Last Subject
2,3,4,5,6
0.04108
0.22, 0.3, 0.36, 0.38, 0.36, 0.26
GPT-2 XL
Last Verb
3,4.5.6,7
0.01936
Last Object
1,2.3
0.02689
The maximum moving average is observed for layers 4-8 with value 0.38. In our method, we would also consider layers 4-8 as in our hyperpa- rameter search space along with layers 1-5.
Subject 4,5,6,7,8
0.32
Large
Last Verb
4.5.6.7,8
Last Object
1,2.3.4.5
0.433
PEP3k
Last Subject 1.2.3.4.5
0.1253
GPT-2 XI.
Last Verh 6,7,8
0.08719
Last Object 3,4,5
0.04107
A.8 Ablation Study
Table 11: Hyper-parameters for the editing method tuned BDIT VALIDATION SET and applied to EDIT SET and PROBE SET.
KL Divergence Factor
The Table 14 shows how the performance of the editing method changes when varying the KL Di- vergence Factor in terms of Accuracy and Fl score. The ablation study is conducted using the GPT-2 Large model the PLP3K dataset, and the verb token is used for editing in the EDIT VALIDATION SET dataset. The chosen hyperparameters align with those presented in Table 11.
A.6 GPT-2 Large Results for Configuration and Semantic Generalization
The GPT-2 Large results for configuration gener- alization experiments are in Table 12. The GPT-2

---

Dataset
Update
Token
Layers
EDIT VALIDATION SET
EDIT
Method
F1 Score %
Efficacy %
Relapse %
Fl Score %
Efficacy %
Relapse
Base Model
75.16
76.22
0
RFTEaly Sup
95.75 (+20.59)
94.74
3.91
80.92 (+4.70)
40.93
6.60
PEP3K
RETrssed
95.75 (+20.59)
94,74
3.91
51.08 (-19.14)
100
55.70
Fait
Last Suhject
4,5,6,7,8
88.53 (+13.36)
76.32
7.39
79.36 (+3.14)
54.95
12.77
Edit
Last Verb Last Object
4,5,6,7,8
93.78 (+18.62)
96.05
6.96
89.08 (+12.86)
93.68
12.34
Edit
1,2,3,4,5
88.41 (+13,25)
86.84
10.87
77.65 (+1.43)
78.57
21.85
Base Model
72.39
0
74.07
0
RFTTiset Sop
91.32 (+18.93)
97.86
11.17
76.45 (+2.37)
48.23
13.69
20Q
RFTrreed Ejoch
(+18.93)
97.86
(-4.15)
38.36
Edit
Subject 3,4,5
85.33 (+12.94)
10.63
81.97 (+7.90)
67.18
12.66
Edit
Verb
2.3.4.5,6
77.64 (+5.25)
38.57
77.33
33.44
7.22
Fditt
Last Object 1,2.3
87.09 (+14.71)
82.14
10.90
84.61 (+10.54)
80.43
13.79
Table 12: Configuration generalization results based on the best hyperparameters identified for the EDIT VALIDATION SET and applied to the EDIT SET for GPT-2 Large. The editing method displays high configuration generalization while both variants of the discussion.
method have a lower F1 Score on the EDIT SET. Refer to $5.3 for further
Unaffected Neighborhood
Aflected
Update Method
Edit Token
Efflncyy
Neighborhood
Affected Paraphrase
Affected Reasoning
Average Unaffected
Average Affected
Subject
Object
Subject
Verb
Object
Base Model
100
100
19.77
18.94
23.84
34.16
32.08
100
25.76
RFT Stop
30.57
83.92
82.64
30.93
30.67
35.53
38.85
33.77
83.28
33.95
PEP3
Foaal Epuch
100.00
19:47
26.79
93.02
92.86
92.03
80.73
36.23
23.13
78.97
Last Subject
54,26
45.73
Edit
Last Verb
96.23
44.15
48.06
69.92
38.28
83.82
41.65
33.02
53.34
53.34
Edit
Last Object
82.26
44.15
73.28
71.78
69.25
36.15
53.74
46.04
$8.71
55.3
Base Model
100
100
30.23
22.93
27.11
32.76
27.72
100
28.15
RFT stp
39.17
42.05
87.56
20Q
RFT Epoch
95.01
55.91
45
71.22
77.07
70.07
66.29
30.10
50.45
62,95
Edit
Last Subject
67.98
79.57
57.79
35.08
63.26
63.91
$3.96
31,70
68 68
49.58
Edir
Last Verh
32.55
89.35
84.16
37.93
26.80
46. 10
36.17
28.51
86.85
34.90
Object
71.67
34.79
76.13
Table 13: Efficacyy and semantic generalization results for the PROBE for GPT-2 Large. Balanced improvements are observed for editing methods across metries, with the object token editing method performing the best. In comparison, the repair-finetuning models show skewed performance between unaffected and affected metrics. Refer to $5,4 for detailed discussion.
Cut-Off Factor
A.9 Constructing the PROBE SET
This hyperparameter is introduced "carly stop" the optimization step. 13 When the probability of ys exceeds this cut-off factor upon adding the residual 6; to the transformer's hidden state h the opti- mization step is stopped.
We prompt text-davinci-000 zero-shot to struct the augmentations for each instance; the prompts are given in:
Affected Paraphrase: Fig.
The Table 15 demonstrates how the performance of the editing method changes when varying the "Cut-Off" Factor in terms of Accuracy and Fl score. The ablation study is conducted using the GPT-2 Large model on the PEP3k dataset, with the verb token used for editing in the EDIT VALIDATION SET dataset. The chosen hyperparameters align with those presented in Table 11.
Affected Reasoning: Fig. 8
Affected Neighborhood: Fig.
Unaffected Neighborhood: Fig. 10
We prompt the model for 5 possible instances. but it can sometimes return the same value multiple times. We filter out poorty-formatted instances and manually clean the filtered data remove things like empty statements or incorrect parsing from
13 Please refer to (Meng et al., 2023) for details of the optimization equation.

---

that the Last Token has higher AIE towards the later layers of the model which is consistent with the results of KL Div. Factor
Efficacy
Accuracy
Fl score
Provide 5 paraphrases of: Furnishings make noise
Base M
75.16
75.16
0.001
88.16
91.83
91.81
0.0025
88.16
91.83
91.81
Furniture be noisy.
0.005
89.47
92.16
92.14
2. Furniture can create sound.
0.0075
89.47
92.16
92.14
3. Furniture can produce noise.
0.01
89.47
92.16
82.06
Furniture be
of sound.
0.025
86.84
91.17
91.15
5. Furniture can be source of noise.
0.05
92.10
92.16
92.13
0.0625
90.79
91.83
91.81
0.075
93.42
92.81
92.80
Figure 7: Prompt to generate affected paraphrase for "Furnishings make noise (false)"
0.1
92.11
91.83
91.81
0.25
93.42
91.18
91.14
0.5
92.11
90.85
90.83
0.75
93.42
91.5
91.48
Furnishings do not make noise. Explain this with a 2-step reasoning chain very short, simple, connected sentences:
92.11
91.18
91.16
Table 14: Ablation study of the KL Divergence Factor on the GPT-2 Large model edited using the verb token layers 2,3, 5, 6 the EDIT VALIDATION split of PEP3K. Note that default KL Factor of 0.0625 is editing methods.
1. Furnishings are inanimate objects.
2. Inanimate objects cannot make noise.
used to report performance
Figure 8: Prompt to generate affected reasoning neigh- borhood for "Furnishings make noise (false)"
Cut-Off Factor
Efficacy
Accuracy
Fl score
Base M
75.16
75.16
0.7
53.95
83.33
82.97
GPT2-XL
Base Model on the 20Q dataset, and the GPT2-Large Base Model the PEP3k dataset.
0.725
55.26
83.33
82.97
0.75
55.26
83.00
82.61
For each of the editing locations,
0.775
55.26
83.00
82.61
0.8
56.58
83.66
83.35
0.825
63.16
84.97
84.73
MEMIT on encyclopedic knowledge. Focusing on the subject, verb, and object tokens, we see that all of them show high AIE in the early layers the corrupted tokens and that the effect on the corre- sponding last corrupted token is more pronounced than that of the first corrupted token. This shows that selecting the last subject/verb/obye the early layers of the model should give good sults for the editing method. These patterns are consistent across all the models and datasets.
0.85
77.63
89.87
89.80
0.875
82.90
91.50
91.47
0.9
90.79
92.81
92.80
0.925
90.79
93.14
93.13
0.95
88.16
91.50
91.48
No factor 90.79
91.83
91.81
token and
Table 15: Ablation study of the "Cut-Off" Factor on
the GPT-2 Large model edited using the verb token on layers 1 E 2, 4,5, for PEP3K EDIT VALIDATION SET. Note that the default value "No Factor" is used report the performance of all editing methods, i.e., there was no "carly stopping" of the optimization step.
GPT output expected key-value pairs. We manu- ally evaluate some examples to ensure quality. In summary, there can be up to 5 augmentations per augmentation type for each instance.
A.10 Causal Analysis Results
The Figs. 12 to 14 shows the causal graphs for the GPre-Large Base Model on the 20Q dataset, the

---

Given:
text: Furnishings make noise
Given text: Furnishings make noise subject token: Furnishings
subject token: Furnishings object token: noise
object token: noise
Q1. Replace the subject token with a completely unrelated word and make a new text. Make 5 such replacements.
Q1. In the text, replace just the subject token with different word. The replaced text should be valid sentence. The replaced token be a hyponym or similar word of the original subject token. Write up to such variants.
Q2. Replace the object token completely unrelated word and make new text. Make 5 such replacements.
Q2. In the text, replace just the verb token with different word. The replaced text should be valid sentence. The replaced token can be verb that follows or precedes the original verb token. Write up to such variants.
1. Replacing the subject token:
Cars make noise
b. Animals make noise
c. People make noise
Q3. In text, replace just the object token with different word. The replaced text should be a valid sentence. The replaced token can be hyponym or similar word of the original object token. Write to 5 such variants.
d. Plants make noise
e. Computers make noise
2. Replacing the object token:
a. Furnishings make music Furnishings make laughter
Furnishings make light Furnishings make heat Furnishings make color
Q1.
1. Appurtenances make noise
2. Fixtures make noise
3. Accoutrements make noise
4. Decorations make noise
Figure 10: Prompt generate unaffected neighborhood for "Furnishings make noise (false)"
5. Adornments make noise
Q2.
1. Furnishings create noise
2. Furnishings emit noise
Furnishings generate noise
4. Furnishings produce noise
5. Furnishings yield noise
Q3.
1. Furnishings make sound
are given an input sentence. the granmar and write the grammatical sentence.
2. Furnishings make clamor
3. Furnishings make din
4. Furnishings make racket
input: furnishing noise
5. Furnishings make uproar
output: furnishings make noise
Figure 9: Prompt generate affected neighborhood for "Furnishings make noise (false)"
Figure 11: Prompt to fix grammar in triple "furnishing make noise"

---

Aug indirect Efrect at ht over 375 prompts
Avg Indirec. Effect of MLP over 375 prompls
Avg indired. Effect of Alin over 375 prompls
subeect uiknn
First subject token
First sib token
Last subject toeen
Lost sublect saken
Lazt sub toaon
Frui Verb Iteen
first Vert loten
First Vere Taken
Velb
Verb
VeDb
Objec:
0.3
Firtt Oijeet Taken
0.1
0.3
Last
Thoen
Onjec Thoen
Object
object
Onject
AI
AE
kingi patched layer
certar interna patored mii leyars
centier patcoed layers
Subject corruption
Aug indirect Effect of hl ever 375 prompts
Aug Indirect Effect MLP over 375 prompts
Aug Indirect Effect of Altn over 375 prompts
sutrect tskin
Rirt subject tokon
First
Last subtect toven
Last surject toten
Lakt sub tawan
First wets tieen
Rist Vert beken
Fist verb Taken
welb
Verb
rist Object tsenn
First Object Taken
First Offect Thken
Osject
oblect-
osject
10 15 25 30 00
AIE
10 15 25 30 00
10 15
single patched hyer within GPT2XX
center of interva 10 patoned mip lnyers
center of intevvl patcned with lnyens
(b) Verb corruption
Avg indirect Effect of over 375 prompts
Avg Indirect Effect of MLP over 375 prompts
Aug Indirect Effect of Attn over 375 prompts
sutject toien
Aist token
First
Last subject toven
Lost subjict tokin
Lart sub sublect toean
First Verb Theen
0.
Fist Vert laken
02
First veth Taken
0.7
webb-
Verb-
Verb
find Object Iheen
First Object token
first Oftect Thilen
Osject
oblect-
osject
10 15
AE
10 15 10 40
10 15
uingl parched layer within GPI2ML
centre interon 10 pahmeed mip lnyers
centre interne patemee witn layers
(c) Object corruption
Figure 12: Causal tracing results for GPT-2 XL Base Model on 20Q EDIT VALIDATION SET when different parts of the input are corrupted.
indirect Effect prompts
Aug
over prompts
Aug
over 367 prompts
mitpect meen
Hist subject token
His:: subest taken
0.20
0.2
0.20
sucject
Sujeect
surject
0.35
015
0.35
Last Vert Taken
Lest vere laken
Last Weth Isken
0.30
010
0.30
time Object thien
First Taken
Lost Object Toxen
0.05
Last Dbject Iakan
Last Ob Taean
Laxt token
0.00
Last token
0.00
Last taken
0.00
A/E
LARGE
(a) Subject corruption
Indirect Effect t
prompts 0.15
Aug
over prompts 015
Aug
over prompts
Rist subject toven
Aist subject tokin
First subvect tolion
0.35
Subject
Sueeect
Surject
0.30
B10
0.30
Last Vert Token
Lost Vore lakon
Last Weth Teken
Ait Object Towen
Lest Osject Towcn
0.25
Last object Kaken
First Object Taken
0.05
First
Last ec: Teean
0.05
Last taken
0.00
Last toten
A/E
0.00
Last taken
0,00
LARGE
(b) Verb corruption
Indirect Effect prompts
Avg Indirect Effect MLP over 367 prompts
Avg Indirect Effect Altri over 367 prompts
Rist sutject meen
First subject toben
Tins wub taken
sjjjett
subject
Subrect
Subject
0.15
Volt Token
0.30
Vera taken
semb
810
Last With Taken
Verb
0.30
Rit Tieen
Lost Object leee
0.25
Last Dbject Treen
8.05
Levt Oiject Theen
0.05
Lart inen
AIE
0.00
Last token
Ale
0.00
Last token
30
0.00
GATE
(c) Object corruption
Figure 13: Causal tracing results for GPT-2 Large Base Model on 20Q EDIT VALIDATION SET when different parts of the input corrupted.

---

Avg
prompts
Avg Indirect Effect MLP over 230 prompts
Aug Indirect Effect Attn over 230 prompts
fisst subtect usenn
Rist supject token
First subject token
sibject
subject toxen
Subject
0.7
Sunject
02
Ver baen
Vert Theen
Werb Token
Fist
ve
0.3
Werb
01
Welb
00
Last Object Ticen
Last Ooject Taken
Last Object Toeen
Last tokan
AIE
0.0
Lest tokan
NE
0.
Last takeen
AE
0.0
engle patched layer
mip lnyers
patoned layers
Subject corruption
Avg indirect Effect ever promots
Avg Indirect Lffect MLP over 230 prompts
Aug Indiract Effect Atth over 230 prompts
lime silject moen
Rist surject taken
Fiut sub.ect taken
subject
token
Law subject toien
Subject
Webt Theen
0.64
Vern boken
0.04
Fistt Veth Tbken
0.64
vel
Verb
Webb
Last Oblect Tieen
Last Onject Token
Last Object Touch
Last lakon
AE
9.00
Last token
NE
D100
taken
AE
0.00
angle patched layer
layers
layers
(b) Verb corruption
Aug Indirect Effect prompts
Aug Indirect Effect MLP over 230 prompts
Aug Indirect Effect Atth over 230 prompts
tie wittett tosen
Hist subject token
Fist subject tolen
sibject
Supject
0.15
supject token
Last subject tooen
Webt Theen
Ver Toicn
Ristt Verh Toaon
010
webb
Werb
Werb
Last Object Been
0.05
Last Object Jaken
0.05
Object Token
0.05
coject
Last token
AE
0.00
token
AIE
0.00
AE
0.00
Snnle patched layer
patched mip layers
pasched layers
(c) Object corruption
Figure 14: Causal tracing results for GPT-2 Large Base Model on PEP3k EDIT VALIDATION SET when different parts of the imput corrupted.


Reviews:
[{"review_id": "PwVKkPRH2j", "reviewer": "Reviewer_wc3N", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating our rich set of experiments, comprehensive discussion, clear writing & presentation, and the strength of our claims/arguments! We address your concerns below.\n\n1\\. **The paper should discuss more about the connection and distinction between editing factual knowledge and commonsense knowledge. The current version appears to be very similar to Rome and memit, and thus seems like an incremental work.**\n\n**Distinctions between editing factual knowledge and commonsense knowledge:** We elaborate on the unique challenges of commonsense knowledge editing as opposed to factual knowledge editing in lines 51-61, but agree with you that more discussion will be worthwhile in the camera-ready. We provide further explanation below:\n\n1.  While a model's factual knowledge can be assessed through next word prediction tasks, grasping commonsense knowledge is considerably more intricate, and relying solely on argmax next word predictions is insufficient to capture the extent of possible completions to a commonsense query. Hence, we transitioned our approach from the original next word prediction used in MEMIT to assessing sentence plausibility prediction. \n\n2.  While transformer-based LMs typically excel at storing and predicting factual information, there remains a noticeable performance disparity when it comes to handling common sense. \n\n3.  Moreover, commonsense knowledge is intricately interconnected, meaning that modifying one portion of this knowledge could considerably disrupt established accurate understandings. Consequently, evaluating the editing of common-sense knowledge necessitates a more extensive and thorough approach.\n\nTo adapt to these challenges, we had to adapt the original MEMIT algorithm in several ways, including:\n\n1.  Establishing that high performance is crucial to having a strong causal tracing result.\n\n2.  Exploring different edit tokens (subject, verb, object) rather than a fixed choice(only subject in MEMIT).\n\n3.  Treating edit layer as a hyperparameter, and improving selection of the range of layers to edit to be more aligned with causal tracing result.\n\n4.  Tuned our edit strategies to balance efficacy (how many incorrect predictions are fixed) with minimum relapse (how much the edit hurts by changing  previously correct predictions). As previously mentioned, commonsense knowledge is intricately interconnected, and hence balancing efficacy and relapse is crucial.\n\nOnly with these changes are we able to establish strong performance of direct model editing on the common sense plausibility domain.\n\n**The current version appears to be very similar to Rome and memit, and thus seems like an incremental work:** This is an **analysis** paper where we present comprehensive empirical results and discussion; our goal was **not** to propose a new model editing method. Prior works have exclusively focused on factual knowledge editing, we are the first to investigate whether the same methods can be applied to the more complicated and nuanced (but equally as important) domain of common sense knowledge. To this end, we conducted a **comprehensive study** of different aspects of MEMIT and its ability for such adaptation.\\\nTo conduct a thorough analysis with the editing method, we created a **PROBE set specifically for commonsense**. The datasets include several new categories, i.e., \"affected reasoning\" and \"affected neighborhood,\" on top of the traditional MEMIT style editing evaluation due to the more nuanced nature of commonsense. We also broadened the affected and unaffected neighborhood statements collection by replacing {S,V,O} and {S,O} respectively, instead of always replacing {S} as evaluated in MEMIT.\n\n2\\. **The previous memit method has been proven to be applicable for editing models of around 20 billion parameters. This paper only considers GPT-2 Large and XL, and it would be beneficial to explore commonsense editing for larger models.**\n\nAlthough we were interested in experimenting with larger models, GPU resource constraints limited us to GPT-2 XL. MEMIT requires lots of memory to run--as mentioned in the reproducibility section of the original paper (Meng et al. 2022), \"Due to the size of these language models, our experiments will not run on GPUs with less memory [than 48GB for GPT-J and ~96GB for GPT-Neo-X]\". Additionally, commonsense editing first requires fully fine-tuning the model on the task (sections 3.1, 5.1), which consumes significantly more memory. We suspect the editing success trend with larger models will likely be similar to what we observed with GPT2-L and GPT2-XL given the scaling success of MEMIT reported by Meng et al., and we leave this as future work for verification.\n\n3\\. **The abstract mentions that memit can be used to update black-box models, but both knowledge locating and editing require probability distribution and gradient information from the model, which contradicts the claim of it being a black-box approach.**\n\nThanks for catching this and we are sorry for the confusion. It should have been \"open-source transformer-based\" models instead of \"black-box\" models. We will correct this in the updated version.\n\nWe would be happy to clarify more questions or concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper presents a rich set of experiments that explore knowledge editing issues with respect to commonsense. The discussions on the problem itself are also comprehensive, delving into configuration generalization and semantic generalization in the methods.\n\n2. The writing and presentation of the paper is clear.\n\n3. The authors promise to provide the code for readers to replicate their findings.", "reasons_to_reject": "1. The paper should discuss more about the connection and distinction between editing factual knowledge and commonsense knowledge. The current version appears to be very similar to Rome and memit, and thus seems like an incremental work.\n\n2. The previous memit method has been proven to be applicable for editing models of around 20 billion parameters. This paper only considers GPT-2 Large and XL, and it would be beneficial to explore commonsense editing for larger models.\n\n3. The abstract mentions that memit can be used to update black-box models, but both knowledge locating and editing require probability distribution and gradient information from the model, which contradicts the claim of it being a black-box approach.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper mainly discusses whether the method memit, previously used for editing factual knowledge, can be used to modify erroneous commonsense in models. The paper explores the locating of commonsense in language models and the editing issues associated with it. It discovers that commonsense is stored in specific layers of language models and can be modified using previous model editing methods. Experimental results indicate the effectiveness of the proposed method, memit-csk, in editing commonsense.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kdE7sl5sOE", "reviewer": "Reviewer_bRLr", "summary": "", "questions": "1 Figures in the paper (e.g., Figure 1) should be briefly explained in the body of the paper, otherwise it is not clear exactly what the legend means. \n2 How does each token in Figure 3 correspond to the word in the example sentence?\n3 The moving average of AIE doesn't to be mentioned in the experimental section?  Suggest adding an analysis to show how it works.\n4 What are the inputs and outputs when editing common sense  knowledge?\n5 The text uses a binary classification task to implement editing, can the editing also be evaluated in the form of QA? Does the dichotomous form make editing less difficult.\n6 Can a comparison with methods such as MEND be added to the experimental section?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating that our paper is compelling, provides extensive experimentation and analysis, and focuses on an important problem! We address your one concern and questions below.\n\n**The method of the paper is very similar in nature to the MEMIT method and seems to be just applying the MEMIT method to the common sense dataset.\nCompared to MEMIT just the number of modified tokens is higher and the causal tracking is unchanged accordingly.**\n\nThis is an **analysis paper** where we present comprehensive empirical results and discussion; our goal was **not** to propose a new model editing method. Prior works have exclusively focused on factual knowledge editing, we are the first to investigate whether the same methods can be applied to the more complicated and nuanced (but equally as important) domain of common sense knowledge. To this end, we conducted a **comprehensive study** of different aspects of MEMIT and its ability for such adaptation.\n\nTo conduct a thorough analysis with the editing method, we created a **PROBE set specifically for commonsense**. The datasets include several new categories, i.e., \"affected reasoning\" and \"affected neighborhood,\" on top of the traditional MEMIT style editing evaluation due to the more nuanced nature of commonsense. We also broadened the affected and unaffected neighborhood statements collection by replacing {S,V,O} and {S,O} respectively, instead of always replacing {S} as evaluated in MEMIT.\n\n**Editing factual knowledge vs Commonsense knowledge:** We elaborate on the unique challenges of commonsense knowledge editing as opposed to factual knowledge editing in lines 51-61. We provide further explanation below:\n\n1.  While a model's factual knowledge can be assessed through next word prediction tasks, grasping commonsense knowledge is considerably more intricate, and relying solely on argmax next word predictions is insufficient to capture the extent of possible completions to a commonsense query. Hence, we transitioned our approach from the original next word prediction used in MEMIT to assessing sentence plausibility prediction. \n\n2.  While transformer-based LMs typically excel at storing and predicting factual information, there remains a noticeable performance disparity when it comes to handling common sense. \n\n3.  Moreover, commonsense knowledge is intricately interconnected, meaning that modifying one portion of this knowledge could considerably disrupt established accurate understandings. Consequently, evaluating the editing of common-sense knowledge necessitates a more extensive and thorough approach.\n\nTo adapt to these challenges, we had to adapt the original MEMIT algorithm in several ways, including:\n\n1.  Establishing that high performance is crucial to having a strong causal tracing result.\n\n2.  Exploring different edit tokens (subject, verb, object) rather than a fixed choice(only subject in MEMIT).\n\n3.  Treating edit layer as a hyperparameter, and improving selection of the range of layers to edit to be more aligned with causal tracing result.\n\n4.  Tuned our edit strategies to balance efficacy (how many incorrect predictions are fixed) with minimum relapse (how much the edit hurts by changing  previously correct predictions). As previously mentioned, commonsense knowledge is intricately interconnected, and hence balancing efficacy and relapse is crucial.\n\nOnly with these changes are we able to establish strong performance of direct model editing on the common sense plausibility domain.\n\n### Questions For The Authors:\n\n**Q1: Figures in the paper (e.g., Figure 1) should be briefly explained in the body of the paper, otherwise it is not clear exactly what the legend means.**\n\nThanks for the suggestion; we will add explanations for each figure under the corresponding sections.\n\n**Q2: How does each token in Figure 3 correspond to the word in the example sentence?**\n\nFigure 3 displays the **average indirect effect** for all statements in PEP3k Edit Validation Set when subject, verb and object tokens are corrupted individually. For a statement such as \"Soil absorbs oil\" the subject is \"soil\", the verb is \"absorbs\" and the object is \"oil\". This sentence is tokenized as ['So', 'il', ' absorbs', ' oil']. Based on the {s,v,o} labels in a statement, we identify the first and last token after tokenization and measure their indirect effects. In this case, the first subject token is 'So', last subject token is 'il', both first and last verb token is ' absorbs', and finally both first and last object token is ' oil'.  After calculating the IE for each statement and its tokens, we plot the average values (AIE) as the causal tracing heat maps in Figure 3.\n\n**Q3: The moving average of AIE doesn't seem to be mentioned in the experimental section? Suggest adding an analysis to show how it works.**\n\nThank you for pointing this out. \n\nHere is a description, which we will add given extra space in the next version:\n\nFor demonstration purposes let's assume there exists a model with just 10 layers. The average indirect effects of these layers at our desired edit token (let's assume last verb token) are: [0.0, 0.1, 0.2, 0.3, 0.5, 0.4, 0.4, 0.3, 0.2, 0,0]\n\nLet's also assume that we are considering only 5 layer windows. The highest average indirect effect is observed at the 5th layer with value 0.5. According to MEMIT, the optimal edit layers will be a 5 layer window ending at the highest AIE layer, in this case it will be the layers 1,2,3,4,5.\n\nNow let's calculate the moving average of 5 layer windows. The moving average of layers 1-5 is (0.0 + 0.1 + 0.2 + 0.3 + 0.5) / 5 = 0.22, similarly the moving average of layers 2-6 will be (0.1 + 0.2 + 0.3 + 0.5 + 0.4) / 5 = 0.3 and so on. The moving averages of all 5 layer windows are: [0.22, 0.3, 0.36, 0.38, 0.36, 0.26]\n\nThe maximum moving average is observed for layers 4-8 with value 0.38. In our method, we would also consider layers 4-8 along with MEMIT's layers 1-5.\n\nDuring hyper-parameter tuning, using results similar to the below table, we compare windows within the search space and pick the best performing strategy for {s,v,o} tokens. In comparison, MEMIT does not explore the layer space and picks the 5 layers prior to the layer with max AIE for the subject token (**Table 7 in our paper**).\n\n**Table:** Layers with max average IE and max moving average IE for the PEP3k Edit Validation Set\n\n| Model | Edit Token | Layer with Max AIE | Layers (size=3) with Max Moving Avg IE | Layers (size=5) with Max Moving Avg IE |\n|---|---|---|---|---|\n| GPT-2 Large | Last Subject | 8 | 8,9,10 | 8,9,10,11,12 |\n| GPT-2 Large | Last Verb | 4 | 4,5,6 | 4,5,6,7,8 |\n| GPT-2 Large | Last Object | 12 | 11,12,13 | 10,11,12,13,14 |\n| GPT-2 XL | Last Subject | 5 | 4,5,6 | 2,3,4,5,6 |\n| GPT-2 XL | Last Verb | 5 | 5,6,7 | 3,4,5,6,7 |\n| GPT-2 XL | Last Object | 12 | 10,11,12 | 9,10,11,12,13 |\n\n\nIn this table, we observe that the moving average method results in a different set of layers than MEMIT's method in all cases. We will include this analysis in the experiments section of the main paper in the next version.\n\n**Q4: What are the inputs and outputs when editing common sense knowledge?**\n\nIf you mean the formats of the natural language query and expected output, we describe this in lines 184-186:\n\nThe task is to predict a label y ∈ {True, False} given an input triple x = (s, v, o).\n\nIf you mean the overall algorithm:\n\nInputs to edit a statement:\n\n-   Transformer Language Model\n\n-   Statement of format (s,v,o) with incorrect plausibility P (True/False)\n\n-   The target plausibility i.e.  P' (False/True)\n\n-   Editing Token ∈ {s,v,o}\n\n-   Layers to Edit\n\nOutput:\n\n-   Edited Transformer Language Model\n\nLet us know if this is still confusing.\n\n**Q5: The text uses a binary classification task to implement editing, can the editing also be evaluated in the form of QA? Does the dichotomous form make editing less difficult?**\n\nWe chose to model the problem statement as the binary plausibility of the sentence instead of using a QA format as commonsense questions can have multiple correct answers. This choice can be used to evaluate a variety of statements such as, \"Humans eat\" : \"vegetable\", \"milk\", \"meat\" and many other foods all of which are considered plausibly 'True' in the (s,v,o) format. We hope this answers your question, we'll be happy to clarify this further.\n\n**Q6: Can a comparison with methods such as MEND be added to the experimental section?**\n\nAccording to the results in MEMIT ([Meng et al., 2023](https://arxiv.org/pdf/2210.07229.pdf)) Table 2, other editing methods such as ROME and MEND perform poorly compared to MEMIT by 35.5% and 62.7% respectively. Therefore, we mainly focused on MEMIT and compared our approach with MEMIT in Table 2 of our paper.\n\nWe would be happy to clarify more questions or concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors have carried out an in-depth study and analysis of editing on common-sense knowledge in Transformer, showing a strong causal relationship between Transformer's MLP layer and common-sense likelihood judgements. \nThese parameters can be directly edited for fixing common-sense errors. \nThe MEMIT parameter editing algorithm was refined for common-sense plausible predictions by changing the editing tokens and improving the layer selection strategy. \nThrough extensive experimentation and analysis, the authors show a compelling future direction for incorporating feedback about common sense into Transformer through direct model editing.", "reasons_to_reject": "The method of the paper is very similar in nature to the MEMIT method and seems to be just applying the MEMIT method to the common sense dataset.\n\nCompared to MEMIT just the number of modified tokens is higher and the causal tracking is unchanged accordingly.", "questions_to_authors": "1 Figures in the paper (e.g., Figure 1) should be briefly explained in the body of the paper, otherwise it is not clear exactly what the legend means. \n2 How does each token in Figure 3 correspond to the word in the example sentence?\n3 The moving average of AIE doesn't to be mentioned in the experimental section?  Suggest adding an analysis to show how it works.\n4 What are the inputs and outputs when editing common sense  knowledge?\n5 The text uses a binary classification task to implement editing, can the editing also be evaluated in the form of QA? Does the dichotomous form make editing less difficult.\n6 Can a comparison with methods such as MEND be added to the experimental section?", "paper_topic_and_main_contributions": "This paper focus on common sense knowledge editing in Transformer. \nWhether common sense judgements are causally related to localised, editable parameters in Transformer is investigated and answered in the affirmative. \nThrough extensive experiments and analyses, the authors show a compelling future direction for incorporating feedback about common sense into Transformer through direct model editing.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AqvtowRJFU", "reviewer": "Reviewer_4FU4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for finding our work compelling and valuable, and highlighting that our task is interesting, application area and dataset novel, and presentation and organization strong! We address your concerns below and believe both are easily fixable in the camera-ready.\n\n1.  **Since the new dataset is also a main contribution. The authors should highlight the differences of the new datasets compared with previous ons, preferably in the last section of intro section where the contributions are highlighted.**\n\nGood point, we will make sure to do this in the camera-ready! Here is a detailed explanation:\n\nThe novel dataset we have introduced diverges from prior datasets, such as the ones used in MEND {zsRE question-answering, FEVER fact-checking, Wikitext generation}, and ROME/MEMIT {zsRE question-answering, COUNTERFACTUAL}, in multiple key aspects. Note that, MEND employed various datasets tailored to specific model types {seq2seq (BART/T5), classification (BERT), GPT}, focusing solely on evaluating Efficacy of edit and one task-specific metric like changes in perplexity or accuracy compared to the base model, lacking an assessment of semantic generalization.\n\nFirstly, our dataset was curated using augmentations of instances drawn from the 20 Questions and PEP3K datasets. This dataset presents a nuanced commonsense knowledge, in contrast to the previous datasets which were centered around factual content from encyclopedic sources.\n\nSecondly, our dataset introduces \"affected reasoning\" statements, to assess if the model correctly comprehends the underlying reasons. As previously highlighted, this inclusion is of paramount importance due to the interconnected nature of commonsense knowledge; modifying one facet of knowledge could potentially influence other associated domains.\n\nThirdly, our dataset introduces the concept of affected neighborhoods, a dimension that previous datasets lacked. While earlier datasets only encompassed neighborhoods that did not change by model edits (referred to as specificity in ROME/MEMIT or unaffected neighborhoods in our context), our dataset presents both unaffected and affected neighborhoods, contributing to a more comprehensive understanding.\n\nLastly, during the creation of these affected and unaffected neighborhoods, we augment {S,V,O} and {S,O} respectively. This contrasts with prior datasets that solely accounted for subject changes in unaffected neighborhoods, neglecting considerations for affected neighborhoods, as elaborated earlier.\n\nWe will incorporate this explanation in the next revision.\n\n2.  **The authors extend MEMIT editing algorithm, to the commonsense domain, by varying edit tokens and improving the layer selection strategy. The authors highlighted the results of doing so (line 88-103), but the reasons and intuition for doing so are not explained. It is better to explain why doing so before introducing the results.**\n\n**Intuition for varying edit tokens:** While factual knowledge depends mostly on the subject of a sentence, commonsense plausibility assessment relies on all components of a sentence. That's why we edited subject, verb and object tokens in our analysis. You can also find the intuition mentioned in Line 208-215 in the paper.\\\nWe **improved the layer selection strategy** in order to better align the causal tracing method and editing method. MEMIT chose to edit a five-layer window ending at the layer with the highest average indirect effect. However, all layers in the window span are equally important. Therefore, selecting the layers belonging to the window with the maximum moving average of the average indirect effect should only lead to better performance. Thus, we considered edit layers as a hyperparameter and searched among them. We will include this intuition in the camera-ready.\n\nWe would be happy to clarify more questions or concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) Updating language models by editing the model parameters directly (without re-training the model) is a very interesting task. The previous editing methods, however, have only been considered on statements of one-to-one encyclopedic knowledge. The paper first studies the problem for one-to-N commonsense knowledge.\n2) The proposed improved version of MEMIT, dubbed MEMIT_CSK, outperforms best fine-tuned baselines by 10%+ on two datasets.\n3) A novel dataset for comprehensive evaluation of language model editing. \n4) I find the presentation and organization of the paper are generally great. e.g., Fig. 1 provides a clear bigmap of the overall idea. ", "reasons_to_reject": "1) Since the new dataset is also a main contribution. The authors should highlight the differences of the new datasets compared with previous ons, preferably in the last section of intro section where the contributions are highlighted.\n2) The authors extend MEMIT editing algorithm, to the commonsense domain, by varying edit tokens and improving the layer selection strategy. The authors highlighted the results of doing so (line 88-103), but the reasons and intuition for doing so are not explained. It is better to explain why doing so before introducing the results. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "Topic: \n\nThis paper studies whether commonsense knowledge (e.g.,  encyclopedic facts) is causally associated with localized, editable parameters in Transformers, and whether model can repair incorrect commonsense knowledge by updating specific hidden states of Transformers.\n\nContributions: \n1) This paper showcases that there are strong causal relations between commonsense plausibility judgments and early MLP layers in Transformers.\n2) This paper improves the MEMIT parameter editing algorithm to MEMIT_{CSK} for commonsense plausibility prediction. \n3) A new dataset that contains unaffected and affected neighborhoods, affected paraphrases, and affected reasoning challenges for comprehensive evaluation.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a study on editing the issues in editing common-sense knowledge in transformers. 

The study is well-motivated. While there has been work in understanding and applying edits to factual knowledge, the issues involved in editing common-sense knowledge are largely unknown. The paper presents a high-quality and well-designed study by suitably adapting an existing method. The paper is well written. The main significance of the work is in the findings of the study, rather than in the modeling aspect. The other significant result from the paper are the datasets that have been created as part of the study. 
The problem of editing complex inter-linked knowledge such as common-sense is an important one and understanding aspects of this problem is crucial. 

In terms of weakness, the writing can be improved to more clearly present the key adaptations that have been made, so that the contributions relative to existing methods are clearer. This however is a minor issue that can be fixed easily.
Decision: Accept-Main