Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledge (2024)

Abstract: Recent research has explored methods for updating and modifying factual knowledge in large language models, often focusing on specific multi-layer perceptron blocks. This study expands on this work by examining the effectiveness of existing knowledge editing methods across languages and delving into the role of attention mechanisms in this process. Drawing from the insights gained, we propose Mass-Editing Memory with Attention in Transformers (MEMAT), a method that achieves significant improvements in all metrics while requiring minimal parameter modifications. MEMAT delivers a remarkable 10% increase in magnitude metrics, benefits languages not included in the training data and also demonstrates a high degree of portability. Our code and data are at https://github.com/dtamayo-nlp/MEMAT.

## introduction large language models (llms) based on transformers @xcite are designed to predict the probability of tokens occurring in a sentence rather than comprehending the true semantics that underlie it. as a result, they are susceptible to generating content that lacks a solid grounding in reality and accuracy. even when two prompts relate to the same factual association ⟨s, r, •⟩ = ⟨google, ceo, •⟩; "the ceo of google is" and "google's ceo is", the model lacks an internal constraint that compels it to generate identical answers. different investigations have already highlighted the limitations of the models' genuine understanding by analyzing its dependency on the dataset patterns @xcite . furthermore, even when these models seem to know the correct answer to a given prompt, they exhibit vulnerability when provided harmful context @xcite . in an initial pursuit of exploring the model's true understanding when using knowledge editors, we analyze mass-editing memory in transformers (memit) @xcite , a knowledgeediting method asserting its capability to insert up to 10,000 factual associations without heavily inducing catastrophic forgetting. aligned with previous research @xcite , our first investigation involves a cross-lingual examination of the limitations associated with memit. although the cross-lingual consistency is dependent on the similarity between languages @xcite , our study specifically delves into examining the polyglot capabilities between english and catalan. in this segment, we construct a translation pipeline to mitigate differences between these languages and proceed to investigate the impact of subject tokenization on knowledge incorporation. motivated by the potential of languageindependent knowledge neurons @xcite , and the relevance of the attention mechanism in the factual associations domain @xcite , we further our study by exploring a particular part of the attention mechanism: the attention heads. attention heads have proven to be useful in enhancing the model's reliability under inference-time intervention (iti) @xcite . the foundational hypothesis behind iti suggests that attention heads serve as key sources of information for evaluating the truthfulness of models when presented with sentences. through our experiments, we not only validate the extension of this claim to the domain of factual associations but also observe promising outcomes from a cross-lingual lens. building on these insights, we propose memat, a method that introduces a novel approach to guide the model towards a better understanding of the edited factual associations. the proposed method demonstrates improvement across all evaluation metrics from both crosslingual and monolingual perspectives, showcasing differences exceeding 10% in some cases. fur-thermore, additional experiments suggest that the modifications introduced by our algorithm enhance the model's understanding of existing knowledge rather than reintroducing it, rendering this approach portable and computationally efficient. truthfulness. efforts to enhance the reliability of llms without depending on external sources have been a focal point of recent research. aligning llms with human feedback has been explored through reinforcement learning from human feedback @xcite and direct preference optimization @xcite , offering valuable insights for veracity alignment @xcite . additionally, approaches contrasting hidden representations of these models have also yielded significant results @xcite in this direction. ## dataset and evaluation the dataset employed in this document is a reduced version of the counterfact dataset @xcite . for each sample, the relevant prompts utilized in our experiments are: ## experiments before the introduction of memat, the main aspects that motivate the use of our method are explained in this section. firstly, section 5.1 outlines the scope of our analysis and examines the limitations of using only english and catalan. our findings suggest a correlation between positive crosslingual outcomes in memit and higher token similarity between subject tokens, indicating the dependency of our cross-lingual analysis on languages that share subject tokens. subsequently, in section 5.2, we evaluate the extent of cross-lingual information in the hidden representations of words by studying attention heads. as ǎguila-7b was not initially assessed using memit, further details on the hyperparameter optimization of the methods studied can be found in appendix b. ## memat method in light of the proven attention heads' relevance, we reinforce the rationale behind equation 6 and present memat as a method that expands upon memit. the overall procedure is depicted in figure 3 , with detailed descriptions for each point as follows: once the head corrections are optimized, we apply equation 6. note that the loss is associated to a single factual triplet. however, differently from memit, we optimize the attention heads corrections for all the samples at the same time by using different batch sizes in @xcite . we also use gradient accumulation to keep the method cheap from a computational perspective. given that there is not a clear choice on the number of heads that should be optimized, we make a hyperparameter search on the number of attention heads and find that for most of the metrics, the optimal number of heads is around @xmath0. for additional information, please refer to appendix f. to evaluate the relevance of the information encoded in the attention heads corrections, we also conduct another experiment with our method. we apply memat in a particular set of factual triplets using the pair (l 1 , l 2 ) and save the head corrections and its positions. then, we insert different factual triplets in the original model using memit in l 1 and add the head corrections that were previously obtained. the results of this experiment, as well as the results of pmet, memit and memat are shown in our research indicates that while pmet shows superior performance in neighborhood metrics, this comes at the expense of reduced efficacy and generalization in knowledge introduction, with memat demonstrating promising results in this area. across all analyzed metrics, memat consistently outperforms memit. particularly noteworthy are the significant improvements in paraphrase magnitudes, exceeding 10% over the baseline. these findings, combined with positive neighborhood metrics, suggest that optimizing attention heads can improve the comprehension of implicit in point (c), the method suggested in iti was not employed. the decision to deviate from this method stems from observed performance declines within the specified domain. further details on this matter can be found in appendix i. ## reproducibility the conducted experiments have been executed on workstations equipped with amd radeon instinct mi50 gpus, with 32 gb of memory each. huggingface transformers @xcite facilitates the loading of language models, while py-torch @xcite is employed to implement model editing algorithms on the gpus. additionally, the training of sigmoid classifiers is carried out using the scikit-learn library @xcite on cpus. in this specific setup, introducing 1,000 samples through memit takes 3 gpu hours, contrasting with the 25 gpu minutes required for training 16 attention heads corrections. ## conclusions in this study, we examine the cross-lingual implications of knowledge within the domain of knowledge editors, identifying two significant patterns. firstly, the proposed methods heavily rely on subject tokenization. secondly, our experiments show evidence that attention heads encode information in a certain language-independent manner. expanding our investigation, we introduce memat, a method that, following the application of memit, fortifies the language model's knowledge through subtle parameter adjustments. we substantiate how the approach introduced is portable and, regardless of the language used during training, enhances the performance of other languages.