TAXI : Evaluating Categorical Knowledge Editing for Language Models (2024)

Abstract: Humans rarely learn one fact in isolation. Instead, learning a new fact induces knowledge of other facts about the world. For example, in learning a korat is a type of cat, you also infer it is a mammal and has claws, ensuring your model of the world is consistent. Knowledge editing aims to inject new facts into language models to improve their factuality, but current benchmarks fail to evaluate consistency, which is critical to ensure efficient, accurate, and generalizable edits. We manually create TAXI, a new benchmark dataset specifically created to evaluate consistency in categorical knowledge edits. TAXI contains 11,120 multiple-choice queries for 976 edits spanning 41 categories (e.g., Dogs), 164 subjects (e.g., Labrador), and 183 properties (e.g., is a mammal). We then use TAXI to evaluate popular editors’ categorical consistency, measuring how often editing a subject’s category appropriately edits its properties. We find that 1) the editors achieve marginal, yet non-random consistency, 2) their consistency far underperforms human baselines, and 3) consistency is more achievable when editing atypical subjects.

## introduction many recent works aim to edit the memorized factual associations encoded in large language models (llms) @xcite @xcite @xcite @xcite @xcite @xcite . if effective, such techniques could offer a transparent and explainable means of updating out-of-date information; correcting biased, offensive, or inaccurate outputs; deleting or obscuring unwanted information to support privacy; and helping to personalize models. but critics warn that model editing is the wrong approach to address factual errors in llms. roughly, they worry that model editing is akin to "emptying the ocean with a spoon" @xcite , that there are just too many facts that might require editing, or that must be checked for targeted editing to accomplish its goals. a key issue compounding this concern is the ability of model edits to generalize consistently to related inputs and generations. current empirical results show that edits often fail to change generations related to paraphrases @xcite or entailments @xcite @xcite . in contrast, human learning functions quite differently. current psychological theories emphasize the interconnected nature of world knowledgeknowledge is not a list of propositions, but rather is embedded within structured "intuitive theories" or mental models that support reasoning and problem solving @xcite . accordingly, people do not update their beliefs one-at-a-time. instead, human belief revision is marked by widespread coherent changes spanning many beliefs @xcite : for instance, people's attitudes toward isolationism shift when foreign wars erupt @xcite ; an alibi appears untrustworthy when dna places a suspect at the scene @xcite ; and learning about the dangers of measles makes vaccines seem safer @xcite . toward factuality and safety, model editors must be more human-like: they should not modify a single "fact", but instead produce consistent and widespread changes across a range of knowledge. we present the taxonomic inference (taxi) dataset as a novel and challenging benchmark for evaluating the coherence and consistency of llm editing methods. taxi leverages edits pertaining to taxonomic categories and their members. categories are powerful conceptual and linguistic structures precisely because they entail many wide-ranging properties shared by their members. editing a language model to assign some subject to be a member of a new category should lead that subject to inherit the properties of that category, thereby supporting strong tests of edit consistency. we use taxi to evaluate two recent model editors and a baseline to edit llama-2 @xcite . we find that recent, popular model editors can indeed generalize categorical edits to update a subject's properties, even without seeing these properties. however, human subjects perform nearly twice as accurately on the same task, highlighting clear room for improvements. ## related works recently-proposed datasets are driving progress in model editing by enabling evaluation of edit generalization. for instance counterfact @xcite evaluates generalization through paraphrased edit queries. closest to our work are rip-pleedits @xcite and @xcite , which evaluate multi-hop question-answering edits. in this setting, an edit is performed, then the model is queried with a follow-up question. for example, after editing "the president of the us is biden", we might prompt the model with "who is the president's son?". these benchmarks measure important consequences of model edits, but do not support evaluation of editors' categorical generalization. further, ## methods we introduce taxi, a new benchmark dataset to evaluate knowledge editing methods' capacity to make categorical knowledge edits in llms. we leverage taxonomic categories, linguistic structures that carry rich and far-ranging information about the properties of their members. for example, upon learning a "pekingese" is a dog breed, you also learn many of its properties, like that it barks and has four legs. we thus evaluate whether existing knowledge editors can alter entities' properties, just by editing their taxonomic categories. this evaluation, we construct a categorical taxonomy, collect a corresponding dataset, and introduce metrics for categorical knowledge editing. our taxonomy contains three types of element, as follows. categories a category c is a high-level division of objects. for example, dogs is a general descriptor that applies to many different breeds. we refer to the set of all categories as c. properties each category has a set of associated properties p @xmath0, p c 1 , . . .} ∈ p. dogs, for instance, have the properties @xmath1, barks, . . .}. subjects a subject s (e.g., pitbull) is an object that belongs to a category @xmath2. the subject likewise inherits the properties of its category; a pitbull wags tail and barks. given categories, properties, and subjects, we propose categorical edits. we define a categorical edit as a change to a subject's category membership (e.g., @xmath3. for llms, this update is made using a knowledge editor, which is a function ϕ that takes in a language model f , a subject s, and a newly-assigned category c * and returns an updated model f * that associates s with c * . note that editing only uses s and c * , not properties p. we than therefore denote an edit as a tuple (s, c * ), which contains a subject and its new category. following @xcite , we can then convert these tuples to prompts that mention the subject alongside continuations to perform the edits. during evaluation, we can then measure whether editing a subject's category also edits its properties p, as further detailed in our metrics below. this is a measure of generalization in knowledge editing, similar to recent works on multi-hop question answering @xcite . but with taxonomic categories, we can be certain which properties should change after edits. our human study in section 5 corroborates this: humans can achieve nearly-perfect property generalization. ## experiments we evaluate three approaches to editing: finetuning (ft), rank-one model editing (rome) @xcite , and in-context knowledge editing (ice) @xcite in editing llama-2 7b @xcite . rome and ice are representatives of popular and capable approaches to editing, which update a model's weights or add facts to its prompts, respectively. for each edit in taxi, we start with the base language model, then apply the edit using an editor, and evaluate its performance. each edit introduces only a subject's category change. property information is used only for evaluation. we compute each metric for both forward (e.g., "a labrador is a type of cow") and reversed queries (e.g., "one type of cow is a labrador"). however, as prior editors are developed for forward queries, we focus primarily on these metrics. ft and rome were implemented using the easyedit editing suite @xcite with modifications to accommodate taxi. experiments utilized default hyperparameter settings except in computing covariances for rome from wikipedia, where 50,000 samples were used. ice was implemented by prepending the prompt "imagine that @xmath4..." to queries. this approach follows @xcite , and might also be seen as a simplification of methods proposed by @xcite . experiments were conducted using a single nvidia a100 gpu. edit success does not imply property success our main results are shown in edits exhibit greater property invariance than consistency not all properties differ by category. therefore, we measure both invariance (accuracy for unchanged properties) and consistency (accuracy for changed properties). we find that all editors exhibit stronger invariance than consistency in their generalizations ( atypical subjects are easier to edit than typical subjects our results in ## human study taxi aims to leverage taxonomic properties to create a clear and intuitive test of consistency for knowledge editors. we validate this with a human study, confirming that taxi is human-solvable. in our study, 19 participants (12 female, 1 nonbinary, median age 34 y/o, all in the united states) recruited from cloudresearch's connect platform and completed a multiple choice questionnaire analogous to the task used to test language models. a random subset of edits was sampled from taxi (with one of each exemplar type per category). each annotator judged 100 items sampled from this subset, for a total of 1,900 human judgments. for each query, participants were instructed to imagine @xmath5. they were then prompted with the subject and a multiple choice question asking which of a set of properties applies to the subject. this task is identical to that used to evaluate the editors, so results are directly comparable. human annotators dramatically outperform editors human subjects are approximately twice as consistent as the best model editor on the same edits (figure 2 , left), answering correctly on 86.8% of trials (the best-performing participant responded correctly on 95/100 trials). overall, human behavioral data indicate that the task prescribed by taxi is human solvable and set a benchmark that far exceeds any existing editors' performance. further procedural details are available in appendix c. ## conclusions we introduce taxi, a new dataset for evaluating knowledge editors ability to consistently and coherently edit large language models. taxi is interpretable, building on taxonomic categories, and is designed to evaluate a knowledge edit's impacts on entailed information. we then propose and study consistency, a new metric that measures whether entailed properties are correctly edited, despite an editor never seeing the entailed information. in experiments with recent knowledge editors on llama-2, we find that consistency varies substantially across existing editors. in editing a subject's category, we find that the editors preserve existing properties of subjects, while two editors achieve nontrivial consistency. however, human subjects are nearly twice as accurate on the same task, establishing consistent model editing as a new research direction. overall, taxi is a challenging, new benchmark for model editors that highlights substantial gaps of existing editing methods. nevertheless, the fact that existing methods do achieve abovechance performance demonstrates the in-principle feasibility of consistent model editing. ft rome ice edit success .05 .10 1.0 property success .05 .12 1.0 invariance .10 .21 1.0 consistency .04 .10 1.0 consistency typical subj. .05 .09 1.0 atypical subj. .03 .10 1.0