Interpretability-based Tailored Knowledge Editing in Transformers (2024)

Abstract: Language models recognized as a new form of knowledge bases, face challenges of outdated, erroneous, and privacy-sensitive information, necessitating knowledge editing to rectify errors without costly retraining. Existing methods, spanning model’s parameters modification, external knowledge integration, and in-context learning, lack in-depth analysis from a model interpretability perspective. Our work explores the instability in in-context learning outcomes, providing insights into its reasons and distinctions from other methods. Leveraging findings on the critical role of feed-forward MLPs in decoder-only models, we propose a tailored knowledge editing method, TailoredKE, that considers the unique information flow of each sample. Model interpretability reveals diverse attribute recall across transformer layers, guiding edits to specific features at different depths and mitigating over-editing issues.

## introduction language models have been proven to be a new form of dynamic knowledge bases @xcite . however, the addition and removal of knowledge within it are not as straightforward as in traditional knowledge graphs. as time progresses and the real world undergoes changes, language models often contain outdated or erroneous knowledge @xcite @xcite , as well as unintentionally learned user privacy information @xcite and biased information @xcite . these errors or redundant information need to be corrected or removed to prevent any adverse impact. in such scenarios, the most common approach is typically retraining from scratch or finetuning the language models, in which the costs are notably high. additionally, in situations with limited data, this can easily lead to unstable training results and overfitting problems. to address this challenge, researchers have introduced the concept of knowledge editing, aiming to modify specific knowledge within the model while ensuring that the storage of unrelated knowledge remains unaffected. many recent works have made progress in modifying the knowledge storage or intervening the retrieval processes within models, whether through the modification of internal model parameters @xcite , the incorporation of external knowledge materials @xcite @xcite , or in-context learning without parameter modification @xcite @xcite . however, there is still a lack of in-depth analysis from the perspective of model internal interpretability to understand why these methods succeed in editing, especially for in-context learning methods that treat large models as black boxes and thus lack theoretical analysis, which also has the drawback of being overly sensitive to text prompts and thus have unstable editing outcomes. in our work, we explored the reasons behind the instability of editing outcomes in in-context learning and examined the differences compared to other methods from the model interpretability perspective in §3.1.1. some other work @xcite have investigated how such relational knowledge is stored in decoder-only language models and found that the feed-forward mlps in the middle layers are crucial and serve as key-value memories of much information associated with the entities. @xcite provided valuable methods for knowledge editing based on this discovery. however, they did not consider the distinct characteristics of each editing sample and the unique processes of information flow associated with them, thus easily leading to the problems of over-editing: modifying parameters excessively and consequently affecting irrelevant peripheral knowledge. in-depth analysis lies in §3.1.2. to better address the unstable knowledge editing problems and the over-editing issues mentioned above, we design and propose a more tailored knowledge editing method, tailoredke, which is based on the actual information flow of each individual sample to perform a more precise knowledge intervention, drawing inspiration from @xcite which further reveal the specific process of how knowledge is extracted and recalled from the model parameters. specifically, we traced the process of internal knowledge recall, capturing different attributes which are recalled across various layers of transformer-based language models and associated with a specific piece of knowledge when performing editing. for instance, considering the entity "ipod", the shallow layers of language models will only recall it as a device and related to music, while deeper layers will be able to recall its association with the "apple" company and other products like "iphone" and "ipad". based on the analysis of these features, we dynamically select different editing layers for each sample, rather than choosing the fixed layers for all editing as done in previous work @xcite . tailoring our edits based on the specific features recalled at different layers not only allows us to address the needs of the actual editing goal (e.g., if our goal is to modify only the "ipod"s manufacturing company) but also ultimately effectively alleviates the issues of over-editing and the potential impact on irrelevant facts in these traditional parameter editing methods as detailed in §3.3. what is more, in the actual training process, it is crucial for a piece of knowledge to manifest in diverse forms to enable the model to genuinely remember it, moving beyond mere template-based memorization. as this knowledge appears more frequently in diverse forms, the model's memory of it becomes more profound, establishing a more stable representation inside. hence, drawing insights from this perspective about how to deepen the model's memory of knowledge, in tailoredke we have introduced diverse structures and expressions during the knowledge editing process, described in §3.2. this ultimately leads to a significant enhancement in performance across various metrics, signifying the strengthening of the model's ability to retain new knowledge. the main structure of our method, tailoredke, is illustrated in figure 1 . the source code of the software used to run the experiments of this paper is available at https: //github.com/yihuaihong/tailoredke . ## the proposed method the current two most effective methods in the field of knowledge editing are in-context learning, such as ike @xcite and mello @xcite , and the parameters editing methods like memit @xcite and rome @xcite . however, each has its own noticeable and impactful shortcomings that cannot be overlooked. the main problem with the parameters editing methods such as rome or memit is that they are too aggressive in modifying the parameters and will sometimes establish an excessively forced relationship between subject and object, potentially leading to the answer always being a new object regardless of the query's context @xcite , which may also impact the storage of irrelevant knowledge that does not require modification. we validate and investigate this issue more in-depth in §3.1.2. when it comes to in-context learning, its performance is largely limited by the length of the prompt window, due to the necessity of adding the text prompt containing relevant knowledge before the query. in most cases, a longer prompt will provide better editing capabilities @xcite . moreover, in-context learning does not genuinely enable the model to learn new knowledge or forget the old one at the parameter level. hence, if the prompt is inaccurate or malicious (such as attempts to bypass the language model's security measures to output private or harmful content), it may generate incorrect or harmful results. therefore, it cannot provide a truly robust and stable knowledge editing effect. in §3.1.1, we leverage a model interpretability technique, representation projection, to gain a deeper understanding of the underlying reasons behind this. ## conclusion our work draws from prior research on feedforward mlps in decoder-only models, providing a tailored approach that uses the information flow of each sample to achieve a higher editing performance and prevent over-editing. we also simulate the training process with varied structures during knowledge editing to improve performance and strengthen knowledge retention.