Editing Common Sense in Transformers (2023)

Abstract: Editing model parameters directly in Transformers makes updating open-source transformer-based models possible without re-training. However, these editing methods have only been evaluated on statements about encyclopedic knowledge with a single correct answer. Commonsense knowledge with multiple correct answers, e.g., an apple can be green or red but not transparent, has not been studied but is as essential for enhancing transformers’ reliability and usefulness. In this paper, we investigate whether commonsense judgments are causally associated with localized, editable parameters in Transformers, and we provide an affirmative answer. We find that directly applying the MEMIT editing algorithm results in sub-par performance and improve it for the commonsense domain by varying edit tokens and improving the layer selection strategy, i.e., MEMIT_{CSK} . GPT-2 Large and XL models edited using MEMIT_{CSK} outperform best-fine-tuned baselines by 10.97% and 10.73% F1 scores on PEP3k and 20Q datasets. In addition, we propose a novel evaluation dataset, PROBE\ SET , that contains unaffected and affected neighborhoods, affected paraphrases, and affected reasoning challenges. MEMIT_{CSK} performs well across the metrics while fine-tuning baselines show significant trade-offs between unaffected and affected metrics. These results suggest a compelling future direction for incorporating feedback about common sense into Transformers through direct model editing.

## introduction transformer-based language models (lms) have achieved great success in nlp @xcite but they still exhibit factual mistakes @xcite , commonsense mistakes ( @xcite @xcite , and consistency errors @xcite @xcite . retraining or finetuning lms to overcome these errors is costly and uninterpretable. to address this, prior research @xcite has shown that model predictions often correlate strongly with certain neuron activations and parameter editing methods can effectively correct encyclopedic factual mistakes. however, it remains unclear whether these editing methods can scale beyond encyclopedic facts to fix commonsense errors in transformers. commonsense knowledge involves more uncertainty and variation than encyclopedic knowledge. consider a subject-verb-object triple @xcite . in the encyclopedic domain, s and v often map to one "o", e.g., the eiffel tower is located in the city of "paris". on the contrary, commonsense knowledge is harder to enumerate and s and v can be mapped to many "o", e.g., an apple has colors that can plausibly be "green", "red", "yellow", "white", and their interpolation. we aim to answer (i) whether commonsense plausibility information is also localized in specific hidden states of transformers, and if so, (ii) can model editing on those units effectively repair incorrect commonsense plausibility judgments? to this end, we focus on the subject-verb-object binary plausibility classification task utilizing two commonsense datasets, 20 questions (20q, @xcite ) and physical plausibility commonsense (pep3k, @xcite ). we perform causal mediation analysis @xcite @xcite on gpt-2 large and xl models and their fine-tuned checkpoints (base-finetuned models), at various part-of-speech locations. while the zero-shot models perform poorly on the task and exhibit no causal pattern, we find clear causal associations between predictions and localized parameters at subject, verb, and object locations in the base-finetuned models. we then investigate if we can edit relevant parameters in the base-finetuned models to correct their mistakes. while directly applying the memit editing algorithm @xcite to edit subject tokens results in sub-par performances, we extend memit to memit csk by editing various token locations and improving the edit layer selection strategy. we demonstrate the advantage of memit csk compared to fine-tuning the model ("repairfinetuning") foot_0 from two angles: semantic generalization and configuration generalization. semantic generalization requires that commonsense judgments are repaired while their paraphrases, neighbors, and reasoning-based queries are also answered correctly -some should be affected and others unaffected by the editing. we create a probe set for 20q and pep3k datasets to contain efficacy, unaffected neighborhood, affected neighborhood, affected paraphrase, and affected reasoning challenges. we also evaluate configuration generalization for each method to determine whether a strategy (hyperparameter combination) picked on an edit validation set can achieve good performance on a separate edit set. our proposed framework for editing and evaluating commonsense knowledge in transformers is depicted in fig. 1 . our contributions are five-fold. (1) we show strong causal associations between commonsense judgments and localized parameters in base-finetuned gpt-2 large and xl models. (2) we extend the memit editing algorithm to memit csk by varying edit tokens and improving the edit layer selection strategy, resulting in 4.58% and 1.99% f1 improvement for gpt-2 xl on edit validation set of pep3k and 20q. (3) gpt-2 xl edited by memit csk outperforms repair-finetuned baselines by 10.97% and 10.73% f1 on the edit set of pep3k and 20q, exhibiting favorable configuration generalization. (4) gpt-2 xl edited by memit csk performs well across the affected and unaffected metrics in our constructed probe set for semantic generalization, while fine-tuned baselines exhibit significant tradeoffs between unaffected and affected metrics. (5) we show that edited models achieve clearer associations between judgments and localized parameters on previously incorrectly predicted samples, solidifying the correlation between causal analyses and performances. these results suggest a compelling future direction of incorporating feedback about common sense in transformers on the fly through direct model editing. ## background the memit (mass editing memory in a transformer) method proposed by @xcite demonstrates its effectiveness in editing up to 10,000 factual associations in transformer models on zsre @xcite and their proposed counterfact dataset, designed to test factual recall. we describe some background here but otherwise refer the reader to @xcite for a more detailed description. ## method we now set out to investigate our main research question: is commonsense plausibility information also localized in specific mlp hidden states of an lm, and, if so, can memit-style editing effectively repair incorrect commonsense plausibility judgments? to investigate this, we conduct experiments that address important sub-questions, focusing specifically on the commonsense plausibility task @xcite . the task is to predict a label @xmath0, false} given an input triple @xmath1, v, o). an example can be seen in fig. 1 foot_1 . ## related work early works on model editing focused on updating individual neurons using constrained finetuning @xcite or hypernetworks @xcite @xcite . a related line of work has focused on storing updates in an external memory @xcite @xcite . recent works @xcite @xcite offer more comprehensive evaluations for fact-editing methods. inspired by the linear associative memory property of feedforward layers in transformers @xcite @xcite and success with the approach in convolutional models @xcite , recent works have proposed to edit mlp weights directly @xcite @xcite . in the encyclopedic factual domain, @xcite proposed to edit single facts by fitting a rank one model edit (rome) to the parameters of an mlp layer, and showed it outperformed prior methods. our work builds on @xcite , which extended this approach to thousands of edits by altering the weights of a range of mlp layers. @xcite demonstrate that many early edit layers can work well with memit; this partially motivates our extensive layer hyperparameter search. recent work by @xcite proposes a dataset for evaluation of a variety of ripple effects in editing methods with factual knowledge and concludes that models fail to capture these effects. all aforementioned works focus on encyclopedic factual knowledge, unlike ours. ## conclusion this paper demonstrates strong causal relations between commonsense plausibility judgments and early mlp layers in transformers. these parameters are directly editable for repairing commonsense mistakes. we improve the memit parameter editing algorithm to memit csk for commonsense plausibility prediction by varying edit tokens and by improving the layer selection strategy. gpt-2 large and xl models edited by memit csk outperform repair-finetuned baselines by more than 10% f1 score on edit set. additionally, we construct a probe set that contains unaffected and affected neighborhoods, affected paraphrases, and affected reasoning challenges for comprehensive evaluation. memit csk effectively generalizes on related and unrelated neighborhoods annotated in our probe set, exhibiting semantic generalization while repair-finetuned baselines demonstrate significant trade-offs between unaffected and affected metrics. these results indicate a compelling direction of incorporating feedback about common sense in transformers on the fly through direct model editing.