Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness (2023)

Abstract: The potential of using a large language model (LLM) as a knowledge base (KB) has sparked significant interest. To maintain the knowledge acquired by LLMs, we need to ensure that the editing of learned facts respects internal logical constraints, which are known as dependency of knowledge. Existing work on editing LLMs has partially addressed the issue of dependency, when the editing of a fact should apply to its lexical variations without disrupting irrelevant ones. However, they neglect the dependency between a fact and its logical implications. We propose an evaluation protocol with an accompanying question-answering dataset, StandUp, that provides a comprehensive assessment of the editing process considering the above notions of dependency. Our protocol involves setting up a controlled environment in which we edit facts and monitor their impact on LLMs, along with their implications based on If-Then rules. Extensive experiments on StandUp show that existing knowledge editing methods are sensitive to the surface form of knowledge, and that they have limited performance in inferring the implications of edited facts.

## introduction recent advancements in large language models (llms) have sparked interest in using them as knowledge bases (kbs) due to their high performance in recalling factual knowledge @xcite . this growing interest is mainly motivated by the advantages that llms offer over traditional knowledge bases. first, llms require minimal human supervision as opposed to kbs that must be manually populated. second, llms process queries expressed in natural language, unlike kbs, which require users to input queries with a specific syntax. while the potential of llms as kbs is recognized, their reliability remains contentious. the data used to train llms may contain inaccuracies or outdated information, compromising the trustworthiness of using llms as kbs. consequently, it is imperative to enable practitioners to edit llms, ensuring that the knowledge foot_1 in llms can be rectified and reflects current real-world knowledge. when editing knowledge in llms, it is crucial to ensure that the modifications respect internal logical constraints commonly referred to as dependency of knowledge in the database and knowledge base communities @xcite @xcite . dependency of knowledge refers to the logical constraints between different pieces of information that have to be satisfied in order to ensure the knowledge base's integrity. it mainly includes specificity and implication awareness constraints. specificity refers to editing a fact and its lexical variations, while other irrelevant ones are left unaltered. implication awareness ensures that the editing process effectively derives implications using if-then rules from an edited fact. for instance, in figure 1 , editing a fact (apple, ceo, steve jobs) by replacing 'steve jobs' with 'tim cook' should keep the unrelated fact (tesla, ceo, elon) unchanged, while the implication (tim cook, employer, ibm) should be updated to (tim cook, employer, apple). while fundamental questions related to dependency have been settled in the knowledge base community @xcite @xcite @xcite @xcite , they are still in their early stage in llms-as-kbs research. since knowledge acquired by llms is encoded within a set of interconnected neurons, editing one fact is likely to impact others. moreover, llms do not explicitly represent the dependencies between different pieces of knowledge, which makes the consequences of any edit unknown in advance. recent work @xcite @xcite tackled the challenge of editing knowledge in llms but only addressed the specificity constraint. however, they neglect the implication awareness of editing, which is a core part of the dependency constraints in editing knowledge. in this paper, we address both the specificity and implication awareness of knowledge editing in llms. concretely, we propose a new evaluation protocol called establish-and-update and an accompanying english question-answering dataset (depedit) for a comprehensive evaluation of dependency in editing llms. the evaluation is done by creating an extensive collection of knowledge editing simulations. each simulation focuses on a subset of the dataset, where we are able to control the entities, relations and applicable if-then rules described within this subset. consequently, we compare the expected and actual edits applied by a knowledge editing model and evaluate its specificity and implication awareness. using our dataset depedit and evaluation protocol, we compare state-of-the-art knowledge editing methods in llms. the results illustrate the challenges associated with implication-aware editing. we find that existing methods tend to be influenced by superficial patterns such as the surface form of questions. furthermore, our analysis of models' gradient updates shows that they struggle to identify the appropriate gradient direction for updating the parameters of the llm in relation to implications. finally, we discuss future work and show that there is significant potential in drawing inspiration from methods developed for dependency in kbs to effectively edit knowledge in llms. this work provides a different testbed to assess the reliability of llms as knowledge bases, focusing on the influence of editing on knowledge dependency. knowledge editing in llms various methods have recently been proposed to edit the knowledge acquired in llms. one line of work @xcite develops external learned editors that modify the tuning gradients for editing, while keeping the base model unchanged. another line of work @xcite @xcite ) attributes knowledge to particular neurons or modules in the network and manually edits their activation to edit a target fact. while these methods achieve high performance in editing knowledge in llms, they only evaluate under the specificity constraint, i.e. editing a fact without affecting the rest of the knowledge. our work adopts a broader perspective on knowledge editing and focuses on its capability to incorporate dependencies that encapsulates both specificity and implication awareness. in a similar vein, @xcite , @xcite and a concurrent work @xcite also go beyond the scope of specificity and evaluate the editing of entailed facts. however, they mainly focus on single-hop reasoning and do not have if-then rules to track the editing of implications. dependencies in knowledge bases dependency theory finds its roots in the domain of data dependency within relational databases @xcite . it was later adapted to knowledge bases, as the semantic aspects of dependencies between facts and rules within a kb @xcite @xcite . several methods @xcite leite, 2012; @xcite were developed for knowledge editing in kbs based on dependencies. these methods were based on key principles @xcite , including three fundamental ones: "irrelevance of syntax," which ensures the meaning of edited knowledge remains independent of the syntax of the original and the updated knowledge; "persistence of prior knowledge," which aims at preserving existing information during editing; and "primacy of new information," which prioritizes new information over existing one in the kb. inspired by this body of work and drawing parallels between knowledge editing in kbs and llms, our research aims to evaluate knowledge editing methods in llms on the principles mentioned above. specifically, we map the principles of "irrelevance of syntax" and "persistence of prior knowledge" to a specificity constraint, which evaluates an editing method's ability to edit facts and their lexical variations without disrupting irrelevant ones. we map the "primacy of new information" principle to an implication awareness constraint, where we evaluate a method's ability to infer logical implications. note that existing work on knowledge editing in llms mainly evaluated based on the specificity constraint @xcite @xcite , while omitting the implication awareness constraint. to the best of our knowledge, we are the first to establish such analogy and evaluate llms using these principles. ## the establish-and-update protocol for evaluating knowledge editing example 1. the fact f expressed in its triplet form (e f,1 , rel f , e f,2 ) as (franklin, city, nyc) is transformed to a question-answer pair (q f , a f ): (which city was franklin from?, nyc). each rule @xmath0,1 and p r,2 , leading to an implication i r . in other terms, a rule r is expressed as follows: if (p r,1 and p r,2 ) then i r . both premise and implication p r and i r are expressed in a template form '?x rel ?y', where ?x and ?y denote entity types, and rel their relation. they can also be instantiated with specific facts. for example, '[p ersona] is from [citya]' can be instantiated to '[f ranklin] is from [n y c]'. we denote a knowledge set with k, which contains subsets of facts f k and rules r k , such that @xmath1. the facts and rules in k are expressed in natural language form. the facts f k in a knowledge set k include two types of facts: specific and unrelated facts, respectively denoted as s k and u k . the specific facts s k share the entity types and relations of the premise propositions in the rules r k . unrelated facts are sampled from the facts f such that u @xmath2. from s k and rules r k , we derive implications i k . example 2. a knowledge set k contains if-then rules and qa pairs mapped from specific facts, implications and unrelated facts: franklin from?, usa) • unrelated facts t (u k ): (who was demi moore's child?, rumer willis); note that the entities (e.g., franklin) in s k are instantiations of the entity type in r k (e.g., person a). we denote the changes expected after editing knowledge in a llm with a tilde symbol (e.g., f to denote edited facts). ## new knowledge editing dataset: depedit existing datasets @xcite for knowledge editing lack if-then rules, which are crucial to evaluate the implication awareness of a model. in order to facilitate models' evaluation based on both specificity and implication awareness, we introduce depedit, a new dataset for evaluating edits in language models using the zsre dataset @xcite . in our dataset, we collect facts expressed both as a question-answer pair and in a triplet form i.e., (e f,1 , rel f , e f,2 ) consisting of an entity pair (e f,1 , e f,2 ) and their relation rel f drawn from wikidata @xcite . we also enumerate a large set of potentially applicable rules, then hire crowd-sourcing workers to annotate their plausibility. the most plausible rules are included in depedit. in the following, we explain each step of the dataset curation process and then provide key statistics about our dataset. ## analysis next, we conduct analysis to investigate the barriers to implication-aware knowledge editing. formally, we break down this task into two sub-goals: gradient pairs cosine similarity premises & implication 0.52 fact pairs with similar surface form 0.51 original & rephrased facts 0.77 random fact pairs 0.35 ## limitations our evaluation protocol is inspired by the principles of knowledge editing in knowledge bases @xcite . these principles are more comprehensive than the dependency constraints discussed in the paper. for instance, the principle of "primacy of new information" applies to all potential logical operations, such as quantification and probabilistic entailment. currently, depedit does not support the evaluation of knowledge editing in llms with such complex logical operations. furthermore, each knowledge set in the current setting of our experiments contains only one rule. we carefully allocate the edit targets so that there are no conflicts among the given facts and derived implications, which may occur in real-world applications. that said, we find that the existing knowledge editing methods fail even in this simple setting. future work can extend our depedit dataset by aggregating multiple rules in one knowledge set or assigning probability values to rules and facts. ## ethics statement a large portion of question-answer pairs in depedit are transformed from wikidata triplets, which could be inaccurate. furthermore, the if-then rules we collected are labeled by a small group of annotators. these points mean that there is no guarantee that the facts and the rules are factual, and they may reflect stereotypes or biases. therefore, depedit can only be used for testing knowledge editing and other reasoning abilities of computational models, not for fact-checking, decision making, or other similar purposes.