Has this Fact been Edited? Detecting Knowledge Edits in Language Models (2025)

Abstract: Knowledge editing methods (KEs) can update language models’ obsolete or inaccurate knowledge learned from pre-training. However, KEs can be used for malicious applications, e.g., inserting misinformation and toxic content. Knowing whether a generated output is based on edited knowledge or first-hand knowledge from pre-training can increase users’ trust in generative models and provide more transparency. Driven by this, we propose a novel task: detecting knowledge edits in language models. Given an edited model and a fact retrieved by a prompt from an edited model, the objective is to classify the knowledge as either unedited (based on the pre-training), or edited (based on subsequent editing). We instantiate the task with four KEs, two large language models (LLMs), and two datasets. Additionally, we propose using hidden state representations and probability distributions as features for the detection model. Our results reveal that using these features as inputs to a simple AdaBoost classifier establishes a strong baseline. This baseline classifier requires a small amount of training data and maintains its performance even in cross-domain settings. Our work lays the groundwork for addressing potential malicious model editing, which is a critical challenge associated with the strong generative capabilities of LLMs.

## introduction large language models (llms) encode knowledge about the world via their pre-training data @xcite . however, the encoded knowledge can be inherently flawed or become outdated over time @xcite . additionally, practitioners might want to tailor llms by incorporating domain-specific knowledge pertaining to their products or to facilitate new applications. driven ## related work this paper is the first to establish the task of detecting knowledge edits in pre-trained language models. still, a wide array of work studies related questions, including how knowledge is stored in llms @xcite @xcite , how knowledge is probed @xcite , how to inject new knowledge @xcite and how to unlearn existing knowledge @xcite @xcite . for brevity, we direct readers to surveys by @xcite and @xcite for ke-related work. this study focuses on the post-editing phase, i.e., after applying kes that directly modify model parameters. ## preliminaries facts foot_0 in llms are commonly represented as triplets of (subject, relation, object), or (s, r, o) for short. querying an llm with a prompt p(s, r), where p expresses the relation r and contains the subject s (e.g., "the eiffel tower is in the city of"), should result in retrieving the object o (e.g., "paris"), given that the fact (s, r, o) is known to the llm. a knowledge editing operation e(s, r, o, o ′ , p) is successful if it changes the behavior of the llm such that the retrieved object is o ′ , as desired, instead of o. if an edit is conducted with a prompt p, then ideally it should also affect semantically similar (paraphrased) prompts that express that same relation, i.e., the new object o ′ should be retrievable with p ′ (s, r) as well, where p ′ is a paraphrase of p. we later use only paraphrased prompts to detect edited facts, since we assume no access to the prompts that have been used for editing. paraphrased prompts are available in both datasets we consider. ## deed task the goal of detecting edited facts is to distinguish outputs generated based on post-hoc modifications to the model (edited knowledge) from those derived from the knowledge acquired by the model during its pre-training phase (unedited knowledge). in this section, we introduce the task of detecting knowledge edits in llms (deed) in detail (section 4.1), describe how we instantiate the task (section 4.2), and discuss practical applications (section 4.3). ## deed baseline models to train a detection classifier, we consider different options of available features. inspired by the probing tasks introduced in @xcite and the representations inspection by @xcite , we investigate the use of hidden state representations as features. additionally, in @xcite and @xcite , the output probability distributions are used to provide signals for latent semantics. therefore, we also explore using these distributions as features. ## cross-domain deed obtaining a training set for the detection classifier requires knowing instances of edited facts in the edited model. however, this assumption may not always hold. consider a scenario where an opensource model is further fine-tuned for a specific task of interest to the user, yet no explicit information is provided regarding the edited facts. many models are fine-tuned with instructions or reinforcement learning from human feedback (rlhf), and are made publicly available @xcite . in these cases, we have access to the original model (from which we could obtain our training set), but do not know which facts have been edited into the finetuned model. to address such cases, we assess the performance of the classifier, which uses hs and pd, in a cross-domain setting, where the classifier is trained on representations from one edited model and then applied to detecting knowledge edits from a separate edited model (which is a fine-tuned version of the original model). the cd edited models the number of test instances is shown in ## conclusion in this paper, we introduced a novel task: detecting knowledge edits in language models. we see promising applications in uncovering potential malicious model modification and enhancing the transparency of llms' generations. we instantiated the task with several baselines, which use hidden state representations and probability distributions as input features. these baselines demonstrate a straightforward yet effective approach to the task. our evaluation across two llms, four editing techniques (2 le-kes and 2 ml-kes), and two datasets, shows that the classifiers can reliably detect edits, even in scenarios characterized by limited training data. moreover, we showed that these baselines generalize to cross-domain settings, where training and test data come from different models and datasets, highlighting their practical utility. our work gives rise to several future work directions: 1) detecting if an llm has undergone any edits at all; 2) studying detection generalization across kes; 3) investigating more constrained settings, where one has only access to the llm's outputs; 4) combating malicious edits.