Long-form evaluation of model editing (2024)

Abstract: Evaluations of model editing, a technique for changing the factual knowledge held by Large Language Models (LLMs), currently only use the ‘next few token’ completions after a prompt. As a result, the impact of these methods on longer natural language generation is largely unknown. We introduce long-form evaluation of model editing ( \textbf{\textit{LEME}} ) a novel evaluation protocol that measures the efficacy and impact of model editing in long-form generative settings. Our protocol consists of a machine-rated survey and a classifier which correlates well with human ratings. Importantly, we find that our protocol has very little relationship with previous short-form metrics (despite being designed to extend efficacy, generalization, locality, and portability into a long-form setting), indicating that our method introduces a novel set of dimensions for understanding model editing methods. Using this protocol, we benchmark a number of model editing techniques and present several findings including that, while some methods (ROME and MEMIT) perform well in making consistent edits within a limited scope, they suffer much more from factual drift than other methods. Finally, we present a qualitative analysis that illustrates common failure modes in long-form generative settings including internal consistency, lexical cohesion, and locality issues.

## introduction model editing is a solution for updating or changing knowledge held by an llm using one or more edited facts @xcite . techniques for accomplishing this include directly updating model parameters by optimizing for a changed fact @xcite , adding and tuning additional model parameters @xcite , using networks trained to perform edits @xcite , and leveraging in-context learning to perform edits as instructions when prompting a model * correspondence: domenic.rosati@dal.ca as a result, we understand very little about how these techniques impact longer texts generated by models after they are edited. this is concerning since llms are often used for paragraph-length or longer outputs. fig. 1 illustrates what we mean by short-form versus long-form evaluation for model editing. to investigate the impact of model editing on paragraph-length outputs from llms, we design a protocol, long-form evaluation of model editing (leme), for evaluating generations after a model has been edited. our primary contributions consist of (1) a novel dataset as well as a survey and classification instrument for assessing long outputs after model editing ( § 3), and (2) automatic metrics that are well correlated with human raters ( § 5). we deploy these automatic metrics across common model editing interventions and datasets for a com-prehensive understanding of their impact ( § 5). our results provide novel insights into current failure modes that have not previously been identified in short-form evaluation such as lexical cohesion and topical drift issues ( § 5.6). notably, the best performing models on short-form evaluation are not often the best performing models on long-form evaluation ( § 5.2) where we found little to no correlation between short-and long-form evaluations ( § 5.4). some models like rome and memit suffer from a much higher rate of "factual drift" than other models which we find in both automatic ratings methods ( § 5.3). finally, by splitting the dataset into samples that are true counterfactual updates versus novel fact injections ( § 5.5), we found that novel fact injections were generally easier to make than counterfactual updates but harder to make factually consistent with other ground truth statements related to the novel fact. with this paper, we release our dataset and evaluation metrics for the research community. foot_0 2 related work @xcite , one of the first studies of model editing for llms, evaluated their method by computing the accuracy of masked token prediction after learning a modified fact from the zero-shot relationship extraction (zsre) dataset @xcite . they assessed constrained fine tuning with a metric that asked if the edit was actually made (efficacy or edit success). de cao et al. @xcite extended this evaluation to seq2seq models using cloze (fill-in-the-blank) evaluations and introduced two measures: the effectiveness of model edits on paraphrases of input queries @xcite and how well the model maintains performance on predictions that shouldn't change (locality or specificity) @xcite for further explorations of locality). @xcite additionally introduced a measure for understanding the degree to which model editing impacts entailed facts (portability) which was further extended in @xcite . these four measures use the next few tokens after a short prompt to evaluate model editing and are the status quo for assessing model editing interventions @xcite . in the paper, these evaluations are called 'short form' as opposed to our 'long form' setting which evaluates paragraph-length texts. for details on how model editing works we refer readers to these works and assume some familiarity for the rest of the paper. most contemporary methods of model editing do not consider long-form generation @xcite @xcite . @xcite introduced an automatic consistency and fluency measure for longer generations based on reference texts from wikipedia and n-gram entropy. however, these were neither validated using human judgements nor fine-grained enough to capture efficacy, generalization, locality, or portability of different model editing techniques on longer form generation. @xcite did perform a preference ranking survey with human raters using fluency, edit success, and factual consistency as ratings. we find that these previous measures do not generally correlate well with human ratings (appendix i). we build on these preliminary evaluations to establish a more comprehensive view of the impact of model editing on 'long-form' natural language generation. ## methods to measure the quality of model editing in longform generation, we developed the following measures designed to align with the short-form evaluations. (1) edit consistency (is there evidence that the edit was made in a generated passage?) which is intended to align with efficacy (2) factual consistency (are generated passages still consistent with facts that were true before the edit?) which is intended to align with locality (3) internal consistency (the degree to which independently generated passages contradict themselves or each other) which is aligned with portability, (4) topicality (the degree to which passages stay on topic), and (5) naturalness (the fluency of generated passages). ( 4 ) and ( 5 ) are intended to measure the impact of model editing on natural language generation (nlg). we operationalize these by constructing a dataset of prompts for generating highly related passages ( § 3.1), devise a likert scale ( § 3.2) and annotation ( § 3.3) setting that we collect human ratings on and develop automatic measures for ( § 4). ## experiments in order to answer our research question of how different model editing interventions compare, we develop a comprehensive suite of experiments that use the automatic measures developed above to evaluate the following interventions: ft with constraint loss @xcite , mend @xcite , rome @xcite , memit @xcite . we implemented these model editing interventions on gpt2-xl @xcite , gpt-j (wang and komatsuzaki, 2021), llama2-7b and llama2-7b-chat @xcite (the main paper results report gpt-j and llama2-7b-chat with the other models results in appendix f). for each model, we also computed a 'no edit' control which is simply using the coupled entity prompts to generate passages before performing any edit. we also experimented with using a zero shot gpt-4 ike setting (see appendix b.2) to simulate an upper bound of performance. subject and related prompts are completed as independent generations. we perform these evaluations on 100 randomly sampled edits from counterfact and zsre. for the zsre setting, we create two edits per sample. we compute a counterfactual edit, making an edit by changing a true fact to a counterfactual one, as well as a factual edit, changing a false fact to a true one. in total, we assess 300 samples (600 passages). ## results below we explore the results of our human evaluations as well as automatic evaluations. ## discussion current model editing methods have many gaps that are not measured by short-form evaluation methods and the preliminary 'long-form' methods from @xcite don't correlate well with human data. factual drift, where methods like rome tend to make much larger changes than mend or ft is not revealed in the standardized 'shot-form' measures from @xcite . future efforts should be devoted to balancing factual drift and edit success in nlg. factual drift might be a desirable feature of model editing, where there are model edits that should imply changes that would contradict ground truth statements. however, we want to develop evaluation methods that are able to measure the trade off between edit and factual consistency which we believe our methods are able to measure. rippleedit @xcite is a good step in this direction for short-form evaluation that could inspire future work on more comprehensive long form evaluations that measure the scope of change beyond a single related passage. finally, our results reveal an important general property we should be looking for in high-quality model editing methods: consistency. the problematic generations that we investigated often indicated cases of contradiction, whether that was self contradiction, contradicting separated generations in the related passages, or contradicting ground truth statements. as developers of model editing interventions, we should design methods that result in generations that have high consistency: there should at least be no contradictions across generated passages. for cases where we allow a high factual drift, we still want to ensure self consistency. other properties like fluency and topicality are important properties which tend to suffer and we should ensure that novel methods do not inadvertently harm general nlg quality. ## conclusion in this paper, we introduced two automatic methods, survey ratings and classification, for evaluating the impact of model editing on natural language generation in paragraph-length generation settings. we validated these measures by collecting survey and annotation data from human participants and then developed a trained model setting that correlated well with human data. using these automatic metrics, we performed a comprehensive analysis of the natural language generation quality of common model editing techniques finding the following results: (1) rome and memit suffer from a high factual drift from ground truth statements compared to other methods like mend or ike (2) there is very little relationship between previous short form evaluations like generalization, locality, and portability with our long form metrics (3) through a qualitative study, we presented a number of common failure modes such as entity drift, lexical cohesion, internal contradiction, and scope errors. we hope that identifying these failure modes can help the community develop future model editing techniques that work well in "long-form" settings.