Mitigating Negative Interference in Multilingual Knowledge Editing through Null-Space Constraints (2025)

Abstract: Efficiently updating multilingual knowledge in large language models (LLMs) without disrupting coherent factual representations across languages remains a significant challenge. While deploying separate editing systems for each language might seem viable, this approach incurs substantial costs due to the need to manage multiple models. A more efficient solution involves integrating knowledge updates across all languages into a unified model. However, sequential edits across languages often lead to destructive parameter interference, significantly degrading multilingual generalization and the accuracy of injected knowledge. To address this issue, we propose LangEdit, a novel null-space constrained framework designed to precisely isolate language-specific knowledge updates. The core innovation of LangEdit lies in its ability to project parameter updates for each language onto the orthogonal complement of other languages’ subspaces. This approach mathematically guarantees update independence while preserving multilingual generalization capabilities. We conduct a comprehensive evaluation across three model architectures, six languages, and four downstream tasks, demonstrating that LangEdit effectively mitigates parameter interference and outperforms existing state-of-the-art editing methods. Our results highlight its potential for enabling efficient and accurate multilingual knowledge updates in LLMs.

## introduction modern large language models (llms) exhibit strong capabilities in encoding and retrieving factual knowledge. knowledge editing has emerged as an efficient approach to updating knowledge within llms, reducing hallucinations without the need for resource-intensive retraining @xcite . * corresponding author. however, efficiently updating knowledge in multilingual scenarios remains a significant challenge, as models must maintain factual consistency across multiple languages. although existing monolingual knowledge editing methods @xcite @xcite have shown promising results, they lack effective solutions for managing multiple monolingual models concurrently, as illustrated in figure 1a . to address this challenge, we introduce the task of multilingual sequential knowledge editing, which involves updating knowledge across multiple languages in a sequential manner, as depicted in figure 1b . this task holds particular significance for applications including knowledgeinformed multilingual information retrieval @xcite and factual updates of multilingual llms @xcite . a criti-cal challenge arises from the fact that editing knowledge in one language can negatively impact the model's performance in other languages. we term this phenomenon negative interference, describing how knowledge editing in a multilingual setting degrades performance on previously updated languages and undermines the multilingual generalization capabilities of llms, as demonstrated in our experiments. we propose langedit, a multilingual knowledge editing framework designed to mitigate negative interference. langedit constrains parameter updates for each language to the null space @xcite of previous language updated subspaces. for example, when editing chinese facts after english updates, langedit projects chinese updates onto the null-space of the parameter updates for english, mathematically ensuring minimal negative interference. this approach creates protective "language safeguards" that prevent parameter conflicts. compared to existing editing models, langedit yields substantial improvements, achieving up to 5.65 percentage points increase in multilingual generalization tasks and 2.20 percentage points improvement in editing accuracy, thereby demonstrating its dual capacity for precise knowledge editing and effective multilingual knowledge retention. we conduct extensive experiments spanning three model architectures, six languages, four multilingual generalization evaluation tasks, and two multilingual knowledge editing datasets. across these settings, langedit consistently outperforms strong sequential editing baselines, establishing state-of-the-art performance. our contributions are: ## method this section presents langedit, a novel model designed for multilingual sequential knowledge editing. our approach utilizes feature covariance matrices, which are incrementally updated as new language-specific data arrives. specifically, given a network trained on prior multilingual data, langedit updates the model parameters corresponding to language j at the current time step t. the key innovation lies in projecting the candidate parameter updates into the approximate null space of the feature covariance matrix of the previously learned knowledge representation, ensuring that language-specific knowledge updates remain decoupled and do not interfere with each other. after training on data l t-1 , we compute the uncentered covariance matrix as follows: k t-1 = 1 n t-1 (k t-1 ) ⊤ k t-1 , where n t-1 denotes the number of data points in l t-1 . if the l t is in language j, the uncentered feature covariance matrix is then updated recursively: when training the network with w t-1 as initialization on data l t , the parameter update ∆w t is obtained by optimizing the following objective: we retain only eigenvectors in u t-1 corresponding to zero eigenvalues, yielding the matrix u ′ t-1 . the null space projection matrix is then defined as: the parameter update ∆ w t is projected onto the null space of kt-1 using p t-1 , the right hand side of equation 5 becomes: compute covariance: solve optimization: w t is added, leading to the final optimization objective: the closed solution for the final objective is: in this process is computing the stacked keys k 0 , which represent the old knowledge stored in the llm. following the approach in @xcite , k 0 ∈ r d 0 ×100,000 is computed using randomly sampled triplets from wikipedia. this procedure is consistently applied across all models and baselines. the langedit method is summarized in algorithm 1. ## related work knowledge editing approaches for large language models (llms) can be broadly categorized into two paradigms: parameter-modifying and parameter-preserving, depending on whether the model weights are updated. we focus on the parameter-modifying paradigm. prior research @xcite has demonstrated that the mlp layers in transformer models function as knowledge repositories, with specific neurons encoding editable factual associations. building on this insight, @xcite introduced rome (rank-one model editing), a pioneering two-stage framework that first identifies the mlp layer storing the target knowledge and then injects a single knowledge edit through rankone weight perturbations. this approach was later extended by memit (mass-editing memory in transformer) @xcite , which enhances scalability by enabling updates across multiple mlp layers to inject numerous knowledge edits. subsequent works identified a critical challenge in sequential editing: @xcite attributed degradation of general abilities to parameter perturbations and introduced rect, a regularization technique constraining weight updates. concurrently, ma et al. (2025) established a mathematical connection between model degradation and the condition number (smith, 1967) of the edited weights, introducing prune to impose condition number-based constraints on weight updates. the monolingual alphaedit (fang et al., 2025) projected the parameter perturbation of the mlp weights into the null space of knowledge preserved in the llms. the above methods have shown success in monolingual knowledge editing but are limited to one language. to the best of our knowledge, we are the first work studying multilingual sequential knowledge editing, where the knowledge in different languages may interfere with each other. prior works @xcite focus on single knowledge editing @xcite instead of sequential knowledge editing. single knowledge editing tests how well a model adapts to a single modification, while sequential knowledge editing checks whether the model can retain all previous edits and maintain overall performance after multiple consecutive changes. ## conclusion we introduced the multilingual sequential knowledge editing task and identified the negative interference arising from parameter changes during sequential updates across languages. to address this, we proposed langedit, a novel framework employing null-space constrained optimization to isolate language-specific parameter updates while preserving the model's multilingual generalization capabilities. langedit achieves this by constructing "language safeguards", which prevent edits in one language-specific knowledge from adversely affecting performance in another, without the need for additional language-specific modules. langedit offers a technically sound method based on null-space projection, specifically adapted for the multilingual sequential setting with dynamic projections. extensive experiments conducted across six languages and three large language models architectures demonstrate the effectiveness of langedit, establishing state-of-the-art performance in multilingual sequential knowledge editing.