Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark (2023)

Abstract: Recent model editing techniques promise to mitigate the problem of memorizing false or outdated associations during LLM training. However, we show that these techniques can introduce large unwanted side effects which are not detected by existing specificity benchmarks. We extend the existing CounterFact benchmark to include a dynamic component and dub our benchmark CounterFact+. Additionally, we extend the metrics used for measuring specificity by a principled KL divergence-based metric. We use this improved benchmark to evaluate recent model editing techniques and find that they suffer from low specificity. Our findings highlight the need for improved specificity benchmarks that identify and prevent unwanted side effects.

## introduction although large language models (llms) are powerful tools for generating human-like language, they can also memorize false or outdated associations, limiting their applicability. model editing techniques promise to solve this problem by correcting non-factual associations. it is important that model edits are highly specific in the sense of not introducing any unwanted associations as a side effect. in this paper, we discuss why the current benchmark for specificity falls short and propose a more challenging, dynamic specificity benchmark to evaluate model editing techniques. using this benchmark, we evaluate recent model editing techniques and find previously unreported side effects. we highlight the importance of improved specificity benchmarks for the effective and safe use of llms subject to model edits. figure 1 : unintended side effects of model edits and how to measure them. (a) gpt-2-medium is edited using rome to counter-factually associate the louvre's location with rome. however, this results in unintended associations ("loud facts") like the association of obama with rome, suggesting low specificity of the edit. the edit also significantly increases the maximum logit (shown in brackets), suggesting that the edit is not merely replacing "paris" with "rome" in the desired contexts. (b) measuring specificity by the fraction of correctly completed test prompts (counterfact) suggests a high specificity for rome. prepending the edit prompt (like "the louvre is in rome.") to each test prompt (counterfact+) results in a significant drop in performance. a significant drop in measured specificity can also be observed if the model edit is implemented using constrained fine-tuning (ft-l). model editing updates the parameters of a trained model in order to change its predicted probability distributions without retraining the entire model. this can be used to edit the associations that the model has memorized and hence, improve the accuracy of the model. fig. 1 shows the example of a counter-factual model edit using rome @xcite where the location of the louvre is edited to be rome instead of paris. we use a counter-factual example since it makes it more evident that the new association is an effect of the model edit instead of the model training. note that the examples in fig. 1 are not taken from the counterfact+ dataset introduced below, but serve to intuitively illustrate the model editing failure modes we are interested in. an important desideratum for model editing is specificity. specificity captures how well the effect of the model edit is localized; in other words, specificity measures the absence of unintended side effects of model edits. fig. 1 shows two examples of unintended side effects of rome model editing, which we collectively call the problem of "loud facts". in the first example, mentioning "louvre" (the subject of the model edit) leads the edited model to also complete unrelated test prompts ("obama was born in") with "rome" (the object of the model edit). in the second example, mentioning "louvre" boosts the logits for words semantically related to "rome", like "vatican". the existing specificity benchmark for model editing from the counterfact dataset @xcite suffers from two limitations which can be illustrated using these examples. first, counterfact does not prompt the model in a way that is likely to surface unwanted side effects. as demonstrated by the examples in fig. 1 , mentioning the subject of the model edit can drastically change the behavior of the edited model, but the existing benchmark does not detect this. second, counterfact considers only the probabilities for the original and edited object token ("paris" and "rome"). as shown by the last example in fig. 1 , the edited model displays strongly changed logits not only for the original object ("paris") and edit object ("rome") but also for semantically related tokens ("vatican"). again, this would be overlooked by the current specificity evaluation since it does not consider the entire probability distribution. these limitations mean that side effects of edits may be overlooked and specificity overestimated. our main contributions are: model editing. several studies have sought to localize and modify the computation of knowledge within transformers. geva et al. (2021) proposed that the multilayer perceptron (mlp) layers in a transformer can act as key-value memories of entities and information associated with that entity. dai et al. (2022) then demonstrated a method to edit knowledge within bert by writing the embedding of the object into certain rows of the mlp matrix. they identified important neurons for knowledge via gradient-based attributions. de cao et al. (2021) presented a hyper-network to predict weight updates at test time, which can alter a fact. they tested both bert and bart (lewis et al., 2020) and focused on models finetuned for question answering. mitchell et al. (2022) introduced a hyper-network method that learns to transform the decomposed terms of the gradient in order to efficiently predict a knowledge update and demonstrate the ability to scale up to large models such as t5 (raffel et al., 2020), and gpt-j (wang and komatsuzaki, 2021). finally, meng et al. (2022a) introduced rank-one-model-editing (rome) which allows edits of transformer models via a rank-one modification of a single mlp layer. (meng et al., 2022b) extended rome to memit (mass-editing memory in a transformer): memit spreads the modification over multiple mlp layers; crucially, this enables thousands of simultaneous edits without performance degradation. model editing evaluation benchmarks of model editing techniques for llms build on existing work on knowledge extraction from llms (see below). zsre question answering was used for benchmarking model editing in @xcite and @xcite . @xcite introduced pararel, a curated dataset of paraphrased prompts and facts. @xcite use this as a basis for constructing counterfact, which enables fine-grained measurements of knowledge extraction and editing along multiple dimensions, including specificity. knowledge extraction from llms. the assessment of knowledge within language models (lms) has typically been done by evaluating whether the model is able to predict pieces of knowledge; petroni et al. (2019, 2020 ) defined a fill-in-theblank prompt and asked the lm to complete it. subsequent work has demonstrated that knowledge extraction can be improved by diversifying the prompts @xcite , or by fine-tuning a model on open-domain textual facts @xcite . however, constructing prompts from supervised knowledge extraction data is still prone to learning new knowledge instead of recalling existing knowledge in an lm @xcite . ## results figure 2 shows the results for the rome, memit, and ft-l editing algorithms applied to the gpt-j (6b) model for different specificity metrics and datasets considered in this work. when evaluated using the neighborhood score (fig. 2 , top), we observe significant drops in specificity for all editing algorithms when going from counterfact to counterfact+. note that specificity measured on the unedited model (gpt-j (6b)) also drops suggesting that there is confounding from the test prompts in counterfact+, potentially due to recency bias @xcite . the drop in specificity is much more pronounced for rome and memit, compared to ft-l and the unedited model, however. this shows that: counterfact+ (compared to counterfact) holds across different model sizes and is not an artefact of using a particular model. section b in the appendix discusses the scaling of specificity with model size in more detail. ## conclusion model editing techniques for auto-regressive transformers exhibit unreported issues related to specificity. although our fine-tuning baseline, ft-l, exhibits less vulnerability to these issues than rome and memit, it falls short in competing with them regarding crucial model editing metrics such as robustness to paraphrasing @xcite . this indicates that model editing still presents numerous complexities that require future attention. additionally, we revealed that the existing counterfact benchmark fails to detect the low specificity in rome and memit. to address this limitation, our primary contributions include: