Emptying the Ocean with a Spoon: Should We Edit Models? (2023)

Abstract: We call into question the recently popularized method of direct model editing as a means of correcting factual errors in LLM generations. We contrast model editing with three similar but distinct approaches that pursue better defined objectives: (1) retrieval-based architectures, which decouple factual memory from inference and linguistic capabilities embodied in LLMs; (2) concept erasure methods, which aim at preventing systemic bias in generated text; and (3) attribution methods, which aim at grounding generations into identified textual sources. We argue that direct model editing cannot be trusted as a systematic remedy for the disadvantages inherent to LLMs, and while it has proven potential in improving model explainability, it opens risks by reinforcing the notion that models can be trusted for factuality. We call for cautious promotion and application of model editing as part of the LLM deployment process, and for responsibly limiting the use cases of LLMs to those not relying on editing as a critical component.

## introduction large language models, or llms, have taken the nlp world by storm. after originally focusing on training them as a vehicle for transfer learning, recent advancements have cast llms in the role of one-stop shop, all-knowing oracles. one particular finding that has contributed to this reframed usage is the apparent factuality properties of pre-trained llms: somehow, mere next-word prediction training has produced models that can complete certain correct facts about the world when asked to do so. llms have been perceived by the public at large as a replacement for search engines, and the scant disclaimers appearing in commercial services offering llm querying do not appear to have made a dent in this perception. despite these developments, llms of course keep making mistakes. there is, after all, an inher-ent mismatch between their pre-training objective and the desire for factuality. in recent years, researchers have come up with several remedies to the problem of nonfactual llm outputs, one of which being model editing, where parameters inside an llm are tweaked based on individual facts marked for correction. these works focus on solving problems such as ensuring stability of other factual outputs following the editing, or batch editing, or computationally efficient editing. in this opinion paper, we call into question the entire idea of model editing, citing concerns about intended use cases, conceptual scalability, potential for bias, safety, and overall accountability. we advocate using models that have explicit knowledge modules for tasks requiring factual knowledge, and other existing techniques to mitigate the need for model editing in the first place. having said that, we acknowledge the usefulness of direct editing for certain use cases such as interpretability probes, and recommend limiting editing to such scenarios. methods for model editing have evolved from editable training @xcite , a procedure requiring an a-priori decision that a model would later be edited, to the locality-motivated changing of specific parameters within models @xcite . recent work @xcite draws attention to the danger of model performance degradation accumulating over the course of many successive edits, and seeks mitigation through improved methods. @xcite extend the consistency requirement to hold over entailed and equivalent facts as well as paraphrases, suggesting model editing as a way of reconciling cases where certain facts are produced correctly but those entailed from them are not. ## critique in this section, we present the case against model editing as a practice, regardless of the performance obtained by any single method. we begin with an analysis of the premise underlying the model editing research objective: the hypothesis that llms can be used as fact repositories. we then focus on reasons for which editing facts cannot be designed a-priori as a means to maintaining factproviding llms, and continue with practical considerations for why even this ill-posed goal is probably unattainable. ## alternatives to model editing model editing is motivated by the desire to control the text generated by llms to make them more compliant with desirable outcomes, specifically to control factuality: when an llm generates text expressing an incorrect or obsolete fact, remove the fact from the llm's "memory" and try again @xcite . other relatively direct approaches to improve llm factuality (and, in general, to control and revise the text generated by llms) have been proposed, most notably reinforcement learning with human feedback (rlhf; @xcite @xcite @xcite , or adding a factuality objective to the training procedure @xcite . we now survey approaches which define a more achievable objective than that pursued by direct model editing or output manipulation. these approaches avoid the assumption that the llm is a fact repository, and thus steer clear of the attempt to update this repository in a logically consistent manner, which is computationally hard to achieve and verify. while these alternatives avoid the more problematic aspects of model editing, they still suffer from their own limitations, but we argue that they identify more promising research directions. ## conclusion we agree that model editing is an attractive task to define, with clear benchmarks and expected outcomes. however, in current practice it contributes towards unrealistic expectations that we can solve problems like llm hallucinations, which would lead to potential harm in unleashing use cases that are not in fact within the capabilities of llms alone. we advocate for the usage of retrieval-augmented methods and other structural and post-hoc methods in order to achieve the stated large-scale goals, while conceding the benefits of editing to "safer" applications such as model interpretability and robustness checking.