Knowledge Editing of Large Language Models Unconstrained by Word Order (2024)

Abstract: Large Language Models (LLMs) are considered to have potentially extensive knowledge, but because their internal processing is black-boxed, it has been difficult to directly edit the knowledge held by the LLMs themselves. To address this issue, a method called local modification-based knowledge editing has been developed. This method identifies the knowledge neurons that encode the target knowledge and adjusts the parameters associated with these neurons to update the knowledge. Knowledge neurons are identified by masking the \it{o} part from sentences representing relational triplets ( \it{s, r, o} ), having the LLM predict the masked part, and observing the LLM's activation during the prediction. When the architecture is decoder-based, the predicted \it{o} needs to be located at the end of the sentence. Previous local modification-based knowledge editing methods for decoder-based models have assumed SVO languages and faced challenges when applied to SOV languages such as Japanese. In this study, we propose a knowledge editing method that eliminates the need for word order constraints by converting the input for identifying knowledge neurons into a question where \it{o} is the answer. We conducted validation experiments on 500 examples and confirmed that the proposed method is effective for Japanese, a non-SVO language. We also applied this method to English, an SVO language, and demonstrated that it outperforms conventional methods.

## introduction large language models (llms) have made remarkable progress in recent years and continue to exhibit significant performance improvements. at the same time, they have also become increasingly multilingual, with pre-trained llms appearing not only on subject-verb-object (svo) languages such as english @xcite @xcite i n c o r r e c t @xcite and chinese @xcite , but also on subject-object-verb (sov) languages such as japanese @xcite and @xcite . these models have potentially acquired extensive knowledge about various facts by learning from huge data sets @xcite @xcite , which can be used to generate language. however, several issues have been pointed out, such as the phenomenon known as "hallucination," which generates information that differs from the facts, and the inability to adapt to facts that change over time. to solve these problems fundamentally, it is necessary to edit the knowledge held by the model. for example, as shown in fig. 1 , in models that are unaware of the fact that shohei ohtani's team has changed, the information needs to be edited and the models updated with the new knowledge. various methods have been proposed to update the knowledge held by the model. one of the these, local modification-based knowledge editing, is a method that identifies the neurons in which knowledge is encoded (knowledge neurons) and updates the knowledge by adjusting those neurons. this local modification-based method is expected to be 1 159 enable efficient knowledge editing while avoiding some of the challenges posed by other approaches. knowledge neurons are identified by masking the o part of sentences representing the relational triplet (s, r, o), having the llm predict them, and observing the activity of the llm. in the case of the decoder-based model of the transformer architecture, the predicate o must be located at the end of the sentence, which places a restriction on the word order of these methods. this constraint poses a challenge when applying these methods to sov languages, where the object usually precedes the verb. as a result, the difference in word order between svo and sov languages makes it difficult to directly apply existing knowledge editing approaches to models pre-trained in sov languages. in this study, we propose a method to resolve the word order constraint by converting the input to the llm during knowledge neuron identification into an interrogative with o as the answer. we applied the proposed method to both english, an svo language, and japanese, an sov language, to determine its effectiveness and investigate the impact of input format conversion on knowledge neuron identification. the significance of this research is twofold: we show that our method eliminates the word order constraints on knowledge editing, enabling its application to languages with various word orders, and we provide insights into the indirect effect of input format conversion on the knowledge neuron identification process. ## previous works methods such as fine-tuning @xcite and retrieval-augmented generation (rag) @xcite @xcite are typically used for updating the knowledge of llms. fine-tuning is effective for general performance improvement, but it has limitations for specific knowledge editing due to issues such as computational resource consumption and overfitting to datasets. furthermore, while fine-tuning can be useful for teaching the model how to solve tasks, it is reportedly to be unsuitable for teaching new knowledge @xcite . rag is a learning-free method that adds information to prompts, but it requires additional resources during inference and has limitations such as the amount of information constrained by the prompt length @xcite . knowledge editing can be broadly catego-rized into external memorization-based methods, global optimization-based methods, and local modification-based methods @xcite . external memorization-based methods store new knowledge in external memory and edit knowledge without changing the original model parameters @xcite @xcite . there are also methods that store new knowledge in additional parameters @xcite . global optimizationbased methods include meta-learning @xcite and subspace fine-tuning @xcite . local modification-based knowledge editing methods aim to update knowledge by identifying knowledge neurons, which are thought to encode specific knowledge, and editing them @xcite . these methods involve two main steps: locating the knowledge neurons that represent the knowledge to be edited and editing those neurons to modify the encoded knowledge. by directly targeting the specific neurons responsible for storing a particular piece of knowledge, local modification-based methods offer a more focused and efficient approach to knowledge editing compared to other methods. existing methods for knowledge localization can be broadly divided into gradient-based methods and methods inspired by causal relationships. gradientbased methods, such as the one proposed by @xcite , introduced the concept of knowledge neurons and localized them by evaluating the contribution of each neuron using integrated gradients @xcite . in contrast, methods inspired by causal relationships, introduced by @xcite , define knowledge neurons as the neuron activations within an llm that have the strongest causal effect on predicting specific factual knowledge. this approach has influenced the development of knowledge editing algorithms such as rome @xcite and memit @xcite . it has been reported that changes in the expression of the input sentence or the language used during knowledge neuron identification can lead to differences in the set of neurons identified as knowledge neurons @xcite . since, we converted the input format in the current study, which also enables adaptation to sov languages, it is necessary to verify the impact of each of these changes. ## proposed method decoder-based models are constrained by the word order due to the architecture of the model being handled and the locating method. in locating, a method is used where an incomplete sentence containing (s, r) is input, and o is output in a way that follows the incomplete sentence. due to the constraints of this architecture, in order to output o, the information of (s, r) needs to be included beforehand, which strongly influences the word order. particularly in 3 161 sov languages like japanese, r tends to be located at the end of the sentence, so there is a tendency for information to be insufficient. to solve this problem, we propose a method that can handle input sentences where r follows o by using an interrogative complete sentence with o as the answer as input and obtaining o as output. in this method, since the sentence is completed in the input, locating can be performed without being affected by word order. note that the proposed method cannot fully complete the locating operation simply by changing the input sentence format. in rome, for example, since the input sentences end with phrases like "～ of" or "～ in," the word that the llm outputs following the input is likely to be the expected o. therefore, locating can be performed by directly observing the generation probability of the output word. in the proposed method, since the input sentence ends with "～?," the answer is output as a sentence, and the word output following the input is less likely to be the expected o. to solve these problems in the proposed method, instead of observing the generation probability of the word output following the input, we decided to observe the generation probability of the expected o among all the probabilities assigned to all vocabularies calculated when outputting the continuation of the input. this enables the proposed method to identify the activation related to a specific (s, r, o). ## conclusion in this paper, we proposed a new method for identifying knowledge neurons. this method eliminates the conventional constraints and enables flexible locating regardless of whether the language is svo or not. first, to verify the effectiveness of the proposed method, we conducted a comparative experiment on an english model using rome and the proposed method. the results showed similar trends in the aie between both methods, confirming that the estimated knowledge neuron positions matched. in terms of editing, the proposed method demonstrated a superior performance to rome. shohei ohtani has been garnering a lot of attention in the major leagues, being entrusted with the cleanup spot for the angels. many baseball fans have various thoughts and feelings about ohtani, who has been demonstrating tremendous talent since his high school days. among those thoughts, ohtani's clear desire to be a pitcher, which he has expressed since joining the nippon ham fighters, has been supported by many baseball fans from the beginning. so, it's natural to wonder just how much ability ohtani possesses as a pitcher. shohei ohtani as a pitcher shohei ohtani is currently playing as a professional baseball player (pitcher) for the los angeles dodgers. last season, he hit 2 home runs. ... he excelled as the ace pitcher of his high school baseball team. he hit a total of 55 home runs in high school. last season, he hit 2 home runs. ... he is currently playing as a professional for the new york brewers. he hit 2 home runs last year. last season, he hit 2 home runs... he is currently playing as a professional for the los angeles dodgers. figure 7 : output when "shohei ohtani" is entered into the model after editing the team from "angels" to "dodgers."