BABELEDITS: A Benchmark and a Modular Approach for Robust Cross-lingual Knowledge Editing of Large Language Models
Authors: Tommaso Green, Félix Gaschi, Fabian David Schmidt, Simone Paolo Ponzetto, Goran Glavaš
Accepted at: acl25

BABELEDITS: A Benchmark and a Modular Approach for Robust Cross-lingual Knowledge Editing of Large Language Models

Tommaso Green1, Félix Gaschi2, Fabian David Schmidt3,
Simone Paolo Ponzetto1, Goran Glavaš3
1Data and Web Science Group, University of Mannheim, Germany 2SAS Posos, France
3Center for Artificial Intelligence and Data Science, University of Würzburg, Germany
tommaso.green@uni-mannheim.de
Benchmark: BABELEDITS

Abstract
With Large Language Models (LLMs) becoming increasingly multilingual, effective knowledge editing (KE) needs to propagate edits across languages. Evaluation of the existing methods for cross-lingual knowledge editing (CKE) is limited both w.r.t. edit effectiveness: benchmarks do not account for entity aliases and use faulty entity translations; as well as robustness: existing work fails to report on downstream generation and task-solving abilities of LLMs after editing. In this work, we aim to (i) maximize the effectiveness of CKE while at the same time (ii) minimizing the extent of downstream model collapse due to the edits. To accurately measure the effectiveness of CKE methods, we introduce BABELEDITS, a new CKE benchmark covering 60 languages that combines high-quality multilingual synsets from BabelNet with marker-based translation to ensure entity translation quality. Unlike existing CKE benchmarks, BABELEDITS accounts for the rich variety of entity aliases within and across languages. We then propose BABELReFT, a modular CKE approach based on representation fine-tuning (ReFT) which learns entity-scope ReFT modules, applying them to all multilingual aliases at inference. Our experimental results show that not only is BABELReFT more effective in CKE than state-of-the-art methods, but, owing to its modular design, much more robust against downstream model collapse when subjected to many sequential edits.1

1 Introduction
Large Language Models (LLMs) require continuous updates to maintain factual correctness as new information emerges. Knowledge editing (KE) in LLMs aims at injecting new knowledge (i) efficiently, i.e., without the need for expensive re-

1Our benchmark is available on HuggingFace at hugging-face.co/datasets/umanlp/babeledits and our code is hosted at github.com/umanlp/babeledits.

![A scatter plot showing the relationship between editing effectiveness and robustness for various CKE methods, with BABELReFT pushing the effectiveness-robustness Pareto-front.](page_857_420_563_393.png)

Figure 1: BABELReFT pushes the effectiveness-robustness Pareto-front in sequential CKE. Effectiveness refers to reliability of propagation of edits made in one language to other languages on BABELEDITS; Robustness denotes the LLM downstream performance in question answering (on XQuAD) averaged over 4 languages after editing.

training or continued training of a large model and (ii) robustly, i.e., without disrupting its language modeling abilities and downstream performance. As LLMs grow increasingly multilingual (Grattafiore et al., 2024; Riviere et al., 2024; Dang et al., 2024), effective multilingual knowledge editing is paramount. We need knowledge that is changed in one language to transfer to all other languages. For example, the fact “Richard Feynman’s wife is Gweneth Howarth” imparted in English should be reflected to all the other supported languages (e.g., when answering “Chi è la moglie di Richard Feynman?” in Italian).

The existing body of (C)KE work, however, comes with prominent limitations, especially for proper evaluation of both (1) effectiveness of imparting new knowledge into the model and (2) model robustness after the edits.

The effectiveness of KE is measured via metrics such as exact match, efficacy score, magnitude
<table>
  <tr>
    <th>English</th>
    <th>MT (Target)</th>
    <th>Correct</th>
    <th>Error Type</th>
  </tr>
  <tr>
    <td>Mortal Kombat Combattimento</td>
    <td>Mortal Kombat (IT)</td>
    <td>Mortal Kombat (IT)</td>
    <td>Literal translation</td>
  </tr>
  <tr>
    <td>Mortale (IT)</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Turkey</td>
    <td>Truthahn (DE)</td>
    <td>Türkei (DE)</td>
    <td>Wrong entity type (animal vs country)</td>
  </tr>
  <tr>
    <td>Mountain Dew Whisky (FR)</td>
    <td>Mountain Dew (FR)</td>
    <td>Mountain Dew (FR)</td>
    <td>Wrong entity entirely</td>
  </tr>
  <tr>
    <td>2006</td>
    <td>2549 (TH)</td>
    <td>2006 (TH)</td>
    <td>Wrong translation</td>
  </tr>
</table>

Table 1: Samples of entity MT errors (Google Translate)

(Meng et al., 2022), and rewrite score (Hase et al., 2023), all of which operate on the same formulation of the fact (i.e., precisely the same tokens) as used for the edit itself. Such evaluation fails to reflect different possible formulations that elicit the same knowledge downstream, including, prominently, entity aliases (e.g., “Who is Dick Feynman’s wife?”). This problem is exacerbated in CKE, where existing evaluation benchmarks are predominantly built by machine-translating English facts (Wang et al., 2024a; Nie et al., 2024; Wang et al., 2024c) and entity mentions: automatically translating entity names with little or no context is particularly error-prone, as illustrated in Table 1.

KE has been shown to harm LLMs’ general performance, with even single edits sharply reducing downstream performance. This “model collapse” (Yang et al., 2024b,c) is known to worsen in real-world scenarios involving multiple sequential edits (Gupta et al., 2024a,b; Gu et al., 2024; Li et al., 2024), rendering LLMs virtually useless after editing. While the existing CKE work (Wang et al., 2024a,c) tests the effectiveness of cross-lingual transfer of the edited knowledge, no prior work investigates model collapse in CKE, i.e., how edits in one language affect the LLMs’ multilingual abilities, i.e., quality of text generation in other languages as well as effectiveness in downstream tasks.

Contributions. In this work, we simultaneously tackle the aspects of effectiveness and robustness in multilingual knowledge editing. In contrast to existing work, we aim to (i) maximize the effectiveness of CKE, i.e., propagation of knowledge edits across languages while at the same time (ii) minimizing the extent of model collapse, considering all languages supported by a multilingual LLM.

1) We introduce BABELEDITS, the largest and most multilingual benchmark for CKE to date, spanning 60 languages and 13,366 facts. It couples high-quality multilingual synonym sets from BabelNet (Navigli and Ponzetto, 2010; Navigli et al., 2021) with marker-based label projection (Chen et al., 2023) to ensure entity translation quality. Unlike existing CKE benchmarks, it captures the diversity of entity aliases across languages.

2) We propose BABELReFT, a modular CKE approach based on representation fine-tuning (ReFT, Wu et al., 2024) where we (i) learn small entity-scope ReFT modules during editing and (ii) apply a ReFT module of an entity to all its aliases across languages, obtained both from BabelNet and via marker-based translation. Results on two widely used LLMs show that, due to its highly modular nature, BABELReFT avoids the negative interference present in existing CKE approaches and mitigates model collapse effects, proving to be very robust in sequential editing with many edits. Figure 1 shows how BABELReFT pushes the effectiveness-robustness Pareto-front in CKE.

2 Background and Related Work

In the most common task formulation, KE aims to alter a fact provided as a subject-relation-object triple \( (s, r, o) \) by replacing \( o \) with a new object \( o' \), denoted with \( (s, r, o \rightarrow o') \), e.g. (Richard Feynman, wife, Mary Louise Bell → Gwethen Howarth). A prompt \( \pi(s, r) \) formulated from the subject \( s \) and predicate \( r \) is typically used to impart the new knowledge and test the success of the editing. The prompt \( \pi(s, r) \) is effectively asking the LLM to complete the incomplete fact \( (s, r, ?) \) (e.g., “Who is Richard Feynman’s wife?”). We refer throughout the paper to the extended prompt \( \pi(s, r, o') \) as the prompt \( \pi(s, r) \) immediately followed by the new object \( o' \).

We next provide an overview of KE work w.r.t. to the two dimensions of our contributions: methods for imparting new knowledge (\S 2.1) and KE evaluation metrics and protocols (\S 2.2). In \S 2.1, we provide more details for methods that we employ as baselines in our evaluation (see 4).

2.1 Knowledge Editing Methods

Yao et al. (2023) provide a taxonomy of KE methods, where the approaches are divided into parameter-preserving and parameter-altering methods, indicating whether the method modifies the original parameters of the LLM or not.

Parameter-Altering Methods. These approaches treat KE as any other downstream task for which a subset of the model weights need to be updated and are further divided into locate-then-edit and meta-learning approaches.
Locate-then-edit approaches. Most approaches in this category modify only the parameters of the down-projection matrices of the MLP layers, as prior work suggest they play a central role in recalling factual knowledge (Geva et al., 2021, 2022; Dai et al., 2022; Meng et al., 2022; Geva et al., 2023; Chughtai et al., 2024). In the light of this, arguably the simplest approach is to train the down-projection matrix of a specific MLP layer via language modeling on the tokens of the new object for the prompt \( \pi(s, r) \), an approach known as FT-M (Zhang et al., 2024b). A related variant, known as FT-L (Meng et al., 2022), applies an \( L_\infty \) norm (i.e., max-norm) on the weight changes and minimizes a different variant of the language modeling loss. ROME (Meng et al., 2022) first identifies which MLP to edit using causal mediation analysis (Vig et al., 2020) and then applies a rank-one modification to the down-projection of the MLP layer to impart a new fact. MEMIT (Meng et al., 2023) extends ROME to support batch edits, i.e. modifying multiple facts in a single edit step.

Meta-learning approaches typically employ hyper-networks, auxiliary networks that generate weights for the LLM, to learn the necessary weight updates for editing of the LLMs as the main model. Key examples include Knowledge Editor (De Cao et al., 2021) and MEND (Mitchell et al., 2022a).

Model Collapse. Although editing via directly updating the model weights is effective, prior work has shown that such methods often induce “disabling edits” (Yang et al., 2024b,c), i.e. single edits that cause the plummeting of downstream performance to random chance, a phenomenon named “model collapse”. In sequential editing, where multiple edits are applied one at a time, several parameter-altering methods were shown to cause model collapse with a few hundred edits (Gupta et al., 2024a,b; Gu et al., 2024; Li et al., 2024).

Parameter-Preserving Methods. The motivation for parameter-preserving KE methods can be manifold: computational efficiency (Zheng et al., 2023), continuous KE (Hartvigsen et al., 2023), or the ability to edit models without having access to its weights (Mitchell et al., 2022b). More importantly and intuitively, avoiding to directly edit the LLMs’ parameters reduces the risk of model collapse. Besides simple in-context learning for KE (Zheng et al., 2023), gating approaches constitute the bulk of the approaches in this category. Gating methods store the edited knowledge into separate weights which are activated based on soft routing. The gating (i.e., routing) function can be a dedicated model like a scope classifier in SERAC (Mitchell et al., 2022b) or a simple key-value similarity threshold as in GRACE (Hartvigsen et al., 2023). The additional parameters are, accordingly, a whole separate model (SERAC) or a vector that replaces the activation values at a given layer (GRACE).

With GRACE as the most competitive baseline in our evaluation (\S5), we provide further details about it in Appendix A.2.

2.2 Knowledge Editing Evaluation

KE evaluation protocols encompass several dimensions (Zhang et al., 2024b; Yao et al., 2023). Reliability reflects whether an edit (\( s, r, o \to o' \)) successfully triggers the new answer \( o' \) when the model is prompted with \( \pi(s, r) \). Generality assesses if the edit holds across paraphrases of the original prompt \( \pi \). Locality verifies that unrelated knowledge (\( s'', r'', o'' \)) remains unchanged after editing. Portability evaluates how well the model generalizes from edited knowledge (Cohen et al., 2024). This includes multi-hop portability, which tests if the model can reason with the edited fact (e.g., inferring “designer” as Richard Feynman’s wife’s profession after editing his wife to be “Gweneth Howarth”), and subject-aliasing portability, checking if the edit applies to alternative subject formulations (e.g., “Richard Feynman” vs. “Dick Feynman” vs. “Ofey”).

Cross-lingual Knowledge Editing Given the documented cross-lingual inconsistencies in LLMs’ factual knowledge (Fierro and Søgaard, 2022; Qi et al., 2023), multilingual knowledge editing needs methods that enable effective cross-lingual transfer of edits (CKE). Accordingly, several CKE benchmarks have emerged: BiZsRE (Wang et al., 2024a), a GPT-4-translated English-Chinese version of ZsRE (Levy et al., 2017); MzsRE (Wang et al., 2024c), extending BiZsRE to 10 more languages; and BMIKE-53 (Nie et al., 2024), which unifies and translates multiple datasets into 53 languages. These benchmarks have primarily been derived by machine-translating both prompts and entities. Entities are translated in isolation, which not only leads to numerous translation errors (as shown in Table 1) but also results in a single translation for each entity (Koehn and Knowles, 2017; Yan et al., 2019; Liang et al., 2024). Focusing on multi-hop portability MLaKE (Wei et al., 2025)
takes a different approach: they mine parallel fact chains in 5 languages and use ChatGPT to generate prompts, but do not provide test sets for generality, locality, and subject aliasing.

3 Methodology

We first describe the process of creating BABELEDS-ITS, our new benchmark for CKE spanning 60 languages in §3.1 and then our novel modular CKE approach BABELREFT in §3.2. Both leverage BabelNet (Navigli et al., 2021), a multilingual lexical-semantic knowledge graph that merges resources like WordNet (Fellbaum, 1998), Wikipedia, Wiki-data, and others to create a network of concepts and named entities across languages. Concepts are organized into synsets—sets of synonymous words or phrases in multiple languages—linked by semantic relations to form a graph structure. In this work, we use BabelNet version 5.3 (released December 2023), which includes approximately 22.9 million synsets covering 600 languages2. This extensive coverage is achieved through the integration of sources such as WordNet 3.0, the November 2023 Wikipedia dump, Wikidata, and others, involving automatic mapping and statistical machine translation techniques to fill lexical gaps in resource-poor languages.

3.1 BABELEDSITS

We utilize the graph structure of BabelNet to generate edits and its multilingual synsets to collect entity aliases, which, combined with marker-based machine translation (Yang et al., 2024a), allows us to obtain high-quality fact translations. This enables a (i) more robust evaluation of edits through subject aliasing and (ii) acceptance of multiple correct answers thanks to object aliases. BABELEDSITS comes with multi-parallel prompts in 60 languages, supporting reliability, generality, locality, subject-aliasing and multi-hop portability evaluation (additional statistics available in Appendix F).

BABELEDITS Creation. We next describe in detail all steps of the BABELEDSITS creation pipeline, which is fully reproducible from our code.

1. Language selection. We start from 50 languages of the popular NLU benchmark XTREME-R (Ruder et al., 2021) as they cover a wide range of scripts and language families. We remove Wolof (WO) as it is currently not supported by Google Translate (GT). We add 11 more languages with more than 500,000 Wikipedia articles (as of Aug ’23) and supported by GT, obtaining the final set \( L \) of 60 languages, listed in Appendix A.3.

2. Subject extraction. For extracting the synsets representing the subjects, we follow a procedure inspired by Green et al. (2023). Since Wikipedia page titles are (often) entity names, we use them to query BabelNet to gather subjects for constructing the edit \((s, r, o \rightarrow o')\). For each language-specific Wikipedia, we first retrieve the 30,000 most viewed pages from 2021.3 We keep only the pages that have a corresponding BabelNet synset with (multi-parallel) senses in all languages in \( L \). We finally sample 20,000 pages (i.e., entities) from each language-specific Wikipedia, ensuring that BABELEDSITS is fully balanced across languages.

3. Relation extraction. Having obtained subject synsets, we next collect all relations these synsets have in BabelNet (i.e., labels of all corresponding outgoing edges). From these, we manually select 132 prominent relations (selection criteria provided in Appendix A.4). Finally, we prompt GPT-4o (see Appendix C.4 for the exact prompt) to verbalize each relation \( r \) as a template sentence with a slot to be filled with a subject: e.g., for \( r = \text{LOCATEDIN} \), we get the template “*Where is \( s \) located in?*”.

4. Edit creation. In the next step, we create the edits \((s, r, o \rightarrow o')\), following a procedure similar to that of Cohen et al. (2024). Let \( S \) and \( R \) be sets of our retrieved synsets and relations, respectively. Each \( \sigma \in S \) then becomes the subject \( s \) in the edit request: we then look for a relation \( r \) from \( R \), a ground truth object \( o \), and a target object \( o' \) that cover all languages in \( L \). For a synset \( \sigma \in S \), we randomly select from its outgoing edges one relation \( r \) to another synset \( \omega \) that also fully covers our set of languages \( L \). A meaningful edit object \( o' \) needs to be of the same category as \( o \). In BabelNet, this generally holds for objects of the same relation \( r \). We thus randomly select a target object synset \( \omega' \) from the set of all BabelNet edges with \( r \) as the relation, i.e. \( \{(\sigma', r, \omega') | \sigma' \neq \sigma', \omega' \neq \omega'\} \): the edit is then given as \((\sigma, r, \omega \rightarrow \omega')\). For creating locality sets we sample an additional relation \( r_{loc} \neq r \) and a ground truth object \( \omega_{loc} \) such that it also covers all languages from \( L \). For multi-hop portability, we start from the target object synset \( \omega' \) and perform, if possible, a hop in the BabelNet graph via a new relation \( r' \neq r \) to another synset \( \omega'' \), so

2See https://babelnet.org/statistics

3Selecting a more recent year could have potentially yielded entities unseen in pretraining by older LLMs.
that we obtain the 2-hop chain \((\sigma, r, \omega', r', \omega'')\). In both cases, we feed the obtained tuples to GPT-4o to create portability prompts (details in Appendix C.4). Finally, for each obtained synset, we collect senses to serve as subject and object aliases. Here, we filter only senses from more trustable sources: Wikipedia, OmegaWiki, WordNet, and OpenMultilingualWordNet (Francis and Kyonghee, 2012) and exclude those obtained via automatic translation and Wiki redirections.

5. Marker-based translation. We resort to marker-based translation with EasyProject (Chen et al., 2023) to translate templated prompts, to easily identify entity spans in the translation. Concretely, we wrap the subject \( s \) and object \( o \) (in English) of the reliability prompt in special markers, e.g.: “Which language does <<>Leonardo Di Caprio <\s>speak?<\o>>Japanese<\o>”. We then translate this markered reliability prompt with GT. The markup in the translation allows us to easily replace the content between <<> and <\s> with the aliases of \( s \) from BabelNet. We next feed these aliased prompts to NLLB (Costa-jussà et al., 2022) to leverage its denoising training to correct possible grammatical errors that arise from the replacement (e.g., gender, article, or case adjustments).

Quality Assessment. The above-described process results in 13,366 samples which we split into training (11,498), validation (480), and test portions (1,042). The train-validation-test split is based on the relations \( r \), i.e., there is no overlap between relation sets of any two portions. We manually assess the quality of the obtained BABELEDITIONS prompts in six target languages: German, Italian, French, Croatian, Spanish and Russian. We select up to 100 reliability test prompts where our marker-based BABELEDITIONS translation differs from separately machine-translating each component (subject, object, and prompt) of \( \pi(s, r, o') \) as done in prior work. We maximally diversify the set of relations among the selected instances. We then present both translations to the annotators (native speakers) and ask them to indicate a preference between the two. The results (detailed in Appendix B.1) show that annotators predominantly—ranging from 56.0% for Russian to 90.0% for French—prefer our marker-based translations.

3.2 BABELReFT

Improving the effectiveness-robustness Pareto front in CKE requires a method that is (i) modular, as direct editing of model parameters jeopardises robustness and (ii) effective in massively multilingual settings, enabling propagation of edits across a wide range of diverse languages. In BABELReFT, we leverage Representation Finetuning (ReFT, Wu et al., 2024), a parameter-efficient fine-tuning method that modifies hidden representations of only some tokens, originally based on their position in the sequence. The standard approach, Low-rank Linear Subspace ReFT (LoReFT), projects hidden representations of selected tokens into a low-dimensional subspace using trainable matrices \( \mathbf{R} \in \mathbb{R}^{m \times d} \) and \( \mathbf{W} \in \mathbb{R}^{m \times d} \), where \( d \) is the dimensionality of a hidden representation \( \mathbf{h} \) and \( m \ll d \). The transformation (or, as called in ReFT, intervention) applied at layer \( \ell \) and token position \( i \) is defined as follows:

\[
\mathbf{h}_i^\ell \leftarrow \mathbf{h}_i^\ell + \mathbf{R}^T (\mathbf{W} \mathbf{h}_i^\ell + \mathbf{b} - \mathbf{R} \mathbf{h}_i^\ell)
\]

LoReFT updates the parameters \( \phi = \{\mathbf{R}, \mathbf{W}, \mathbf{b}\} \), while the LLM parameters remain frozen. \( \mathbf{R} \) is a low-rank matrix with orthonormal rows, while \( \mathbf{W} \) and \( \mathbf{b} \) define an affine transformation of \( \mathbf{h} \).

BABELReFT couples ReFT with a lexical gating function: i.e., we do not select tokens that undergo a ReFT transformation based on their position, but rather based on whether the token is part of an entity mention. This allows us to train entity-scope ReFT modules and route tokens of an entity being edited, as well as tokens of their translations and aliases through the same ReFT module. For each entity \( e \), we construct a vocabulary \( V_e \) that consists of all lexicalizations of the entity in the source language (i.e., the language of the edit) and all target languages: with “lexicalizations” we here refer to the union of all entity translations we obtain with marker-based MT and all senses (i.e., aliases) from BabelNet.

Prior to the forward pass, we search for mentions of entities \( e \) by string-matching (with the Aho-Corasick algorithm (Aho and Corasick, 1975)) the input text against the entries in \( V_e \). When a match is found, all tokens of the matched mention are routed through the ReFT intervention of the respective entity \( e \), i.e., the hidden representations of those tokens are modified as follows:

\[
\mathbf{h}_i^\ell \leftarrow \mathbf{h}_i^\ell + \mathbf{W}_{2[e]}^T \left( \mathbf{W}_{1[e]} \mathbf{h}_i^\ell + \mathbf{b}_{[e]} - \mathbf{W}_{2[e]} \mathbf{h}_i^\ell \right)
\]

with \( \mathbf{W}_{1[e]}, \mathbf{W}_{2[e]} \in \mathbb{R}^{m \times d} \) and \( \mathbf{b}_{[e]} \in \mathbb{R}^m \) as trainable parameters of the ReFT module of entity \( e \).
Entity-specific ReFT modules prevent negative interference between edits by design, which should provide robustness and prevent model collapse in the face of a larger number of sequential edits. For BABELReFT we utilised the NoReFT intervention, which differs from LoReFT solely for the absence of the orthogonality constraint on \( \mathbf{W}_{2[e]} \), as we observed that such a constraint introduced interference between edits in sequential editing. The parameters \( \mathbf{W}_{1[e]}, \mathbf{W}_{2[e]}, \) and \( \mathbf{b}_{[e]} \) of an entity \( e \) are trained by feeding the extended prompt \( \pi(s, r, o') \), where \( s \) is a lexicalization of \( e \), into the LLM and minimizing the language modeling loss on the tokens of \( o' \).

4 Experimental Setup

Models. We run single and sequential editing experiments with the instruction fine-tuned variants of Llama 3.1 8B and Gemma 2 9B (cf. Appendix C.3).

Languages. Due to computational constraints, we carry out the evaluation on English and 10 other languages (out of the 60 languages in BABELEDITS): Arabic (AR), German (DE), French (FR), Croatian (HR), Italian (IT), Japanese (JA), Georgian (KA), Burmese (MY), Quechua (QU), and Chinese (ZH). We manually selected these languages to ensure diversity across linguistic typology, scripts, and “resourcefulness” (Joshi et al., 2020).

KE Methods. We compare BABELReFT against FT-M, FT-L, r-ROME\footnote{r-ROME is a re-implementation of ROME that mitigates model collapse (Gupta et al., 2024a).}, and GRACE. We conduct an exhaustive search for the optimal hyperparameters that maximize average reliability across languages on our validation split of BABELEDITS, considering all combinations of models, methods, and both single and sequential editing.\footnote{Details about the procedure and the best hyperparameter configurations are provided in Appendix C.2.} In our experiments, we solely use the test set, as none of the methods require training an auxiliary editor network. We inject the edit using a single reliability prompt in the editing language from the test set. We subsequently evaluate the edited model on all the evaluation dimensions using the prompts for the same edit in all 11 selected languages. In sequential editing, we carry out the evaluation after injecting \( n = 100, 250, 500, 1042 \) (test set size) edits.

Metrics. We use the ‘rewrite and rephrase’ scores introduced by Hase et al. (2023) to measure reliability and generality. We adapt these scores for multi-hop and subject-aliasing portability: if an edit has multiple aliases for the same target object, we compute the metric for each and then take the best value. We follow Hoelscher-Obermaier et al. (2023) and use neighborhood KL-divergence (NKL) to evaluate locality.\footnote{We provide precise formulations for all metrics in Appendix A.1.} We evaluate the zero-shot downstream multilingual performance of the edited models on two tasks: (1) multiple-choice reading comprehension using Belebele (Bandarkar et al., 2024) and the (2) extractive question answering on the XQuAD dataset (Artetxe et al., 2020). We report the results in terms of accuracy for Belebele and exact match for XQuAD.\footnote{Further details about the evaluation and the prompts used can be found in Appendix C.5,}

<table>
  <tr>
    <th></th>
    <th>EN</th>
    <th>FR</th>
    <th>IT</th>
    <th>JA</th>
    <th>MY</th>
  </tr>
  <tr>
    <td>FT-M</td>
    <td>63.77</td>
    <td>25.87</td>
    <td>33.78</td>
    <td>8.42</td>
    <td>0.33</td>
  </tr>
  <tr>
    <td>GRACE</td>
    <td>99.08</td>
    <td>0.38</td>
    <td>0.75</td>
    <td>0.00</td>
    <td>0.00</td>
  </tr>
  <tr>
    <td>BABELReFT</td>
    <td>98.49</td>
    <td>49.86</td>
    <td>64.38</td>
    <td>19.44</td>
    <td>2.17</td>
  </tr>
</table>

Table 2: Reliability of three methods with sequential editing in English on the full BABELEDITS dataset using Llama 3.1 8B. Results are provided for five languages due to space constraints, full results in Appendix B.2.

Sequential Editing. Table 3 presents the results of the sequential editing task, where the number of sequentially applied edits successively increases from 100 to the entire test set size (1,042). We first apply the edit to the model in English. We then test the edited model both on KE in all languages on the BABELEDITS test set and on downstream performance on XQuAD and Belebele.

BABELReFT demonstrates superior performance across several editing aspects (top half of Table 3), achieving by far the highest scores on reliability, generality, and subject aliasing. GRACE performs better than FT-M on many dimensions but is still very far from achieving the effectiveness of BABELReFT. Near-zero results in other languages largely explain the low average reliability of GRACE, which, however, remains highly reliable in English, as shown in Table 2.

Multi-hop portability performance (Appendix B.2) is close to zero across all models, with BABELReFT performing slightly better. On the full dataset, BABELReFT gets a score of 1.27 and 1.64 for Llama and Gemma respectively, whereas
<table>
  <tr>
    <th rowspan="2">Edits</th>
    <th colspan="6">Llama 3.1</th>
    <th colspan="6">Gemma 2</th>
  </tr>
  <tr>
    <th>FT-L</th>
    <th>FT-M</th>
    <th>r-ROME</th>
    <th>GRACE</th>
    <th>BABELReFT</th>
    <th>FT-L</th>
    <th>FT-M</th>
    <th>r-ROME</th>
    <th>GRACE</th>
    <th>BABELReFT</th>
  </tr>
  <tr>
    <th colspan="12">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="12">↑ Reliability: edit success</th>
  </tr>
  <tr>
    <td>100</td><td>1.59</td><td>21.94</td><td>-0.01</td><td>9.33</td><td>37.12</td><td>3.16</td><td>15.58</td><td>0.12</td><td>15.51</td><td>42.30</td>
  </tr>
  <tr>
    <td>500</td><td>1.59</td><td>19.98</td><td>-0.02</td><td>9.27</td><td>37.48</td><td>2.12</td><td>10.26</td><td>0.01</td><td>16.54</td><td>40.90</td>
  </tr>
  <tr>
    <td>Full</td><td>1.52</td><td>19.51</td><td>-0.04</td><td>9.26</td><td>36.51</td><td>3.02</td><td>9.79</td><td>0.01</td><td>17.10</td><td>40.72</td>
  </tr>
  <tr>
    <th colspan="12">↑ Generality: edit success over paraphrases</th>
  </tr>
  <tr>
    <td>100</td><td>1.29</td><td>19.73</td><td>-0.02</td><td>0.72</td><td>34.40</td><td>2.35</td><td>10.28</td><td>0.12</td><td>8.30</td><td>39.52</td>
  </tr>
  <tr>
    <td>500</td><td>1.48</td><td>17.87</td><td>-0.02</td><td>0.58</td><td>35.15</td><td>1.41</td><td>5.99</td><td>0.01</td><td>8.86</td><td>38.48</td>
  </tr>
  <tr>
    <td>Full</td><td>1.71</td><td>17.69</td><td>-0.04</td><td>0.47</td><td>34.25</td><td>2.37</td><td>5.76</td><td>0.01</td><td>9.36</td><td>38.18</td>
  </tr>
  <tr>
    <th colspan="12">↑ Subject-alias portability: edit success over prompts with a subject alias</th>
  </tr>
  <tr>
    <td>100</td><td>1.49</td><td>17.23</td><td>-0.01</td><td>2.45</td><td>23.08</td><td>1.72</td><td>10.30</td><td>0.01</td><td>9.86</td><td>26.93</td>
  </tr>
  <tr>
    <td>500</td><td>0.83</td><td>11.05</td><td>-0.01</td><td>1.47</td><td>28.33</td><td>0.92</td><td>5.07</td><td>0.00</td><td>7.96</td><td>27.95</td>
  </tr>
  <tr>
    <td>Full</td><td>0.87</td><td>11.93</td><td>-0.01</td><td>1.32</td><td>27.85</td><td>1.36</td><td>4.66</td><td>0.00</td><td>9.18</td><td>28.81</td>
  </tr>
  <tr>
    <th colspan="12">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="12">↑ Belebele (accuracy)</th>
  </tr>
  <tr>
    <td>Original</td><td>73.59</td><td>73.59</td><td>73.59</td><td>73.59</td><td>73.59</td><td>84.68</td><td>84.68</td><td>84.68</td><td>84.68</td><td>84.68</td>
  </tr>
  <tr>
    <td>100</td><td>73.42</td><td>73.99</td><td>34.89</td><td>73.59</td><td>73.50</td><td>84.48</td><td>84.46</td><td>24.14</td><td>84.71</td><td>84.59</td>
  </tr>
  <tr>
    <td>500</td><td>73.52</td><td>68.06</td><td>28.64</td><td>73.56</td><td>73.50</td><td>84.48</td><td>84.38</td><td>26.07</td><td>84.71</td><td>84.70</td>
  </tr>
  <tr>
    <td>Full</td><td>72.92</td><td>60.26</td><td>22.58</td><td>73.50</td><td>73.39</td><td>84.64</td><td>84.40</td><td>28.62</td><td>84.71</td><td>84.70</td>
  </tr>
  <tr>
    <th colspan="12">↑ XQuAD (EM)</th>
  </tr>
  <tr>
    <td>Original</td><td>29.60</td><td>29.60</td><td>29.60</td><td>29.60</td><td>29.60</td><td>31.79</td><td>31.79</td><td>31.79</td><td>31.79</td><td>31.79</td>
  </tr>
  <tr>
    <td>100</td><td>18.07</td><td>20.00</td><td>0.00</td><td>29.60</td><td>29.60</td><td>29.64</td><td>29.98</td><td>0.13</td><td>31.76</td><td>31.76</td>
  </tr>
  <tr>
    <td>500</td><td>19.37</td><td>1.58</td><td>0.00</td><td>29.71</td><td>29.45</td><td>29.03</td><td>28.78</td><td>2.77</td><td>31.76</td><td>31.64</td>
  </tr>
  <tr>
    <td>Full</td><td>19.35</td><td>0.36</td><td>0.00</td><td>29.71</td><td>29.18</td><td>26.37</td><td>29.81</td><td>4.33</td><td>31.76</td><td>31.70</td>
  </tr>
</table>

Table 3: Comprehensive comparison of cross-lingual knowledge editing (effectiveness) and downstream task performance (robustness) for Llama 3.1 8B and Gemma 2 9B models for different number of sequential edits in English. Editing metrics are averaged over all target languages and multiplied by 100 for readability. Bold numbers show the best performance for each metric/model combination. Downstream performance is averaged over the target languages: Original denotes the model performance before editing. Full results in Appendix B.2.

FT-M holds the second-best score with 0.12 and 0.26. This shows that further research is needed to make models generalize from the imparted edits.

BABELReFT also shows robustness in downstream evaluation (bottom half of Table 3). It preserves the original model performance, matching the stability of GRACE while significantly outperforming other baselines, in particular r-ROME which shows large degradation with as few as 100 edits.

Our downstream evaluation also shows that the choice of downstream task is critical for detecting model collapse. For instance, after 500 sequential edits, FT-M loses only 5 points on Belebele, yet its XQuAD performance plummets to 1.58, clearly indicating a collapse of its generative abilities. This discrepancy reflects the nature of the tasks: Belebele is a multiple-choice QA task where inference simply decodes the answer letter with the highest log-probability, i.e., it does not reflect the generative ability of the models. In contrast, XQuAD requires that the model generates a response containing the actual tokens of the answer. We thus advocate for evaluating downstream performance after KE on free-form generative tasks, as these can detect early signs of model collapse.

We next test if our findings generalize beyond BABELEDITS by evaluating sequential KE on the MzsRE benchmark (Zhang et al., 2025). Specifically, we perform sequential editing in English across the entire test set (742 edits) and evaluate the results for languages in which MzsRE overlaps with BABELEDITS (DE, EN, FR, ZH). For BABELReFT, we use the subject s from each edit to query BabelNet and incorporate all retrieved senses into the vocabulary \( V_s \). As shown in Table 4, MzsRE results closely mirrors our findings from BABELEDITS: BABELReFT achieves the highest average reliability without a decline in downstream performance, while other methods fall short either in cross-lingual reliability (GRACE) or provoke model collapse (FT-M).

Single Edits. While sequential editing represents a more realistic use case, single editing is still often used in CKE evaluations. We thus evaluate BABELReFT against the same baselines on the test set of BABELEDITS but this time by performing each edit in the dataset independently. We perform editing in each of the 11 languages in our evaluation set. Since evaluating downstream performance
<table>
  <tr>
    <th>Methods</th>
    <th>Avg</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>ZH</th>
  </tr>
  <tr>
    <td colspan="6"><i>↑ Reliability: edit success</i></td>
  </tr>
  <tr>
    <td>FT-M</td>
    <td>27.53</td>
    <td>23.09</td>
    <td>64.76</td>
    <td>17.52</td>
    <td>4.75</td>
  </tr>
  <tr>
    <td>GRACE</td>
    <td>25.17</td>
    <td>0.78</td>
    <td><b>99.51</b></td>
    <td>0.40</td>
    <td>0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td>
    <td><b>45.24</b></td>
    <td><b>44.31</b></td>
    <td>97.23</td>
    <td><b>34.25</b></td>
    <td><b>5.17</b></td>
  </tr>
  <tr>
    <td colspan="6"><i>↑ XQuAD (EM)</i></td>
  </tr>
  <tr>
    <td>Original</td>
    <td>29.83</td>
    <td>34.71</td>
    <td>32.44</td>
    <td>-</td>
    <td>22.35</td>
  </tr>
  <tr>
    <td>FT-M</td>
    <td>5.97</td>
    <td>4.29</td>
    <td>7.56</td>
    <td>-</td>
    <td>6.05</td>
  </tr>
  <tr>
    <td>GRACE</td>
    <td><b>29.94</b></td>
    <td><b>34.54</b></td>
    <td><b>33.03</b></td>
    <td>-</td>
    <td>22.27</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td>
    <td>29.75</td>
    <td>34.45</td>
    <td>32.52</td>
    <td>-</td>
    <td>22.27</td>
  </tr>
</table>

Table 4: Sequential editing in English on Llama 3.1 8B for the MzsRE dataset, showing reliability and XQuAD exact match. Full results in Appendix B.2.

<table>
  <tr>
    <th>Methods</th>
    <th>Avg</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>ZH</th>
  </tr>
  <tr>
    <td colspan="6"><i>↑ Reliability: edit success</i></td>
  </tr>
  <tr>
    <td>FT-M</td>
    <td>28.87</td>
    <td>39.29</td>
    <td>37.10</td>
    <td>35.98</td>
    <td>26.32</td>
  </tr>
  <tr>
    <td>GRACE</td>
    <td><b>32.58</b></td>
    <td><b>45.33</b></td>
    <td><b>46.19</b></td>
    <td><b>40.65</b></td>
    <td>24.42</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td>
    <td>30.96</td>
    <td>43.65</td>
    <td>37.48</td>
    <td>39.02</td>
    <td><b>27.64</b></td>
  </tr>
  <tr>
    <td colspan="6"><i>↓ Delta PPL</i></td>
  </tr>
  <tr>
    <td>FT-M</td>
    <td>79.52</td>
    <td>40.24</td>
    <td>3.37</td>
    <td>16.05</td>
    <td>5.42</td>
  </tr>
  <tr>
    <td>GRACE</td>
    <td>5.46e4</td>
    <td>5.09e4</td>
    <td>8.65e4</td>
    <td>3.61e4</td>
    <td>1.02e5</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td>
    <td><b>0.00</b></td>
    <td><b>0.00</b></td>
    <td><b>0.00</b></td>
    <td><b>0.00</b></td>
    <td><b>0.00</b></td>
  </tr>
</table>

Table 5: Reliability and variation of perplexity for single edits with Llama 3.1 8B. Each column (except AVG) corresponds to an editing language, and the results are averaged across all the target languages. Column AVG averages those results. Full results in Appendix B.2

after each edit is computationally prohibitive, we follow Yang et al. (2024b) and use perplexity as a surrogate metric. We compute perplexity variation (Delta PPL) before and after editing on a translated version of their ME-PPL-50 dataset, comprising randomly sampled sentences from widely used corpora such as BookCorpus (Zhu et al., 2015) and ROOTS (Laurençon et al., 2022).

Results in Table 5 show that BABELReFT and GRACE exhibit similarly high reliability with Llama 3.1. However, while GRACE did not show any model collapse in sequential editing, it shows a massive increase in perplexity, particularly in the case of Llama 3.1: this suggests that its gating function is often activated when not necessary, severely damaging the generative capabilities of the LLM.

Although single editing can be seen as an unrealistic (i.e., in vitro) use-case, BABELReFT remains competitive, still providing the best solution when considering both editing effectiveness and downstream robustness.

Gating Scope. We empirically observe that the failure of GRACE in either transferring edits across languages (sequential editing) or causing model collapse (single edits) stems from its difficulty to balance precision and recall of its gating activation. In sequential editing, the clusters get gradually smaller as edits are injected hence making the gating function seldom activate (precision over recall). This, coupled with the limited cross-lingual semantic alignment of LLM representations, explains the negligible edit transfer. This would also explain the higher reliability of GRACE with Gemma 2, given the better cross-lingual alignment of Gemma with respect to Llama 3.1 (Kargaran et al., 2025). In the case of single edits, there is only one cluster with a large fixed radius, which is promoted by the hyperparameter selection procedure that aims to maximize the edit transfer across languages (i.e., recall over precision). This, however, makes the gating function fire on almost every input, causing model collapse and rendering the model useless for downstream tasks.

Cross-Lingually Disabling Edits. Gupta et al. (2024b) have shown that a single disabling edit can completely disrupt the model downstream abilities. To the best of our knowledge, we are the first to observe <i>cross-lingually disabling edits</i>, i.e., that edits in one language compromise the performance across languages. To shed more light on this phenomenon, we compute for all target languages the top five most destructive edits, i.e. those that cause the highest increase of perplexity, across all possible editing languages. Figure 2 illustrates the effect of cross-lingually disabling edits for FT-M applied to Llama 3.1 (as FT-M performed overall comparably to GRACE but with less perplexity variation) for pairs of edit-test languages. We observe, e.g., that a single edit in Japanese disables the model for many languages, whereas a single edit in German reduces performance for English much less than for other languages. The latter is particularly insidious, as editing in some languages can collapse the model only w.r.t. some other languages, which can be difficult to detect for model users.

Ablations. We conduct ablation studies on the two key components of BABELReFT, the <i>entity scope</i> of the ReFT modules and the <i>multilingual scope</i> of the gating function, to study their individual contributions. We report the results in Table 6 for Llama 3.1 (full results in Appendix B.2). We tested two variants of ReFT (LoReFT and NoReFT) in a fully unrestricted fashion, always activating on the last three tokens of any given input. Both LoReFT and NoReFT yield near-zero reliabil-
ity across languages and cause a massive degradation in downstream performance (e.g., XQuAD EM drops to less than three after only 100 edits). This is due to the fact that (i) each edit overwrites the same ReFT module and (ii) the module is always activated, even when not necessary. The results show that ReFT alone is insufficient for effective sequential knowledge editing.

We then move to comparing BABELReFT to a variant where the scope of the gating function is restricted to the source (editing) language (BABELReFT-SL). While such a variant expectedly preserves downstream robustness, its cross-lingual reliability (32.13%) and subject-aliasing performance (11.96%) fall significantly short of full BABELReFT (36.51% and 27.85%, respectively). This proves that restricting the gating function to only the editing language limits the transfer of knowledge to aliases and translations.

In sum, effective and robust CKE requires both (i) entity-specific transformations and (ii) a gating function with broad lexical scope over multilingual aliases. Limiting either component severely undermines editing effectiveness, robustness, or both.

6 Practical considerations

BABELReFT introduces minimal overhead relative to the full model size. Computational cost occurs only when the gating function activates, that is, when entity tokens are present, making runtime impact sparse and localized. Parameter overhead per entity intervention is \( 2 \times m \times d + m \) parameters. For \( d = 8192 \) (e.g., Llama 3.1 70B), \( m = 4 \), and 10,000 entities (ten times the test set size), this amounts to 655 million parameters or approximately 1.3GB using bf16 precision\(^{8}\).

In contrast to other modular methods such as SERAC (Mitchell et al., 2022b), which store edited knowledge in a separate smaller model (e.g., a 7B model for edits to a 70B model), BABELReFT is significantly more parameter-efficient.

<table>
  <tr>
    <th>Edits</th>
    <th>LoReFT</th>
    <th>NoReFT</th>
    <th>BABELReFT-SL</th>
    <th>BABELReFT</th>
  </tr>
  <tr>
    <th colspan="5">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="5">↑ Reliability</th>
  </tr>
  <tr>
    <td>100</td>
    <td>0.44</td>
    <td>0.31</td>
    <td>31.55</td>
    <td>37.12</td>
  </tr>
  <tr>
    <td>500</td>
    <td>0.02</td>
    <td>0.26</td>
    <td>33.27</td>
    <td>37.48</td>
  </tr>
  <tr>
    <td>Full</td>
    <td>-0.03</td>
    <td>0.05</td>
    <td>32.13</td>
    <td>36.51</td>
  </tr>
  <tr>
    <th colspan="5">↑ Subject-Alias portability</th>
  </tr>
  <tr>
    <td>100</td>
    <td>0.41</td>
    <td>0.18</td>
    <td>10.01</td>
    <td>23.08</td>
  </tr>
  <tr>
    <td>500</td>
    <td>-0.01</td>
    <td>0.20</td>
    <td>11.30</td>
    <td>28.33</td>
  </tr>
  <tr>
    <td>Full</td>
    <td>0.01</td>
    <td>0.25</td>
    <td>11.96</td>
    <td>27.85</td>
  </tr>
  <tr>
    <th colspan="5">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="5">↑ Belebele (accuracy)</th>
  </tr>
  <tr>
    <td>Original</td>
    <td>73.59</td>
    <td>73.59</td>
    <td>73.59</td>
    <td>73.59</td>
  </tr>
  <tr>
    <td>100</td>
    <td>45.96</td>
    <td>26.49</td>
    <td>73.49</td>
    <td>73.50</td>
  </tr>
  <tr>
    <td>500</td>
    <td>49.03</td>
    <td>27.86</td>
    <td>73.48</td>
    <td>73.50</td>
  </tr>
  <tr>
    <td>Full</td>
    <td>53.52</td>
    <td>26.46</td>
    <td>73.49</td>
    <td>73.39</td>
  </tr>
  <tr>
    <th colspan="5">↑ XQuAD (EM)</th>
  </tr>
  <tr>
    <td>Original</td>
    <td>29.60</td>
    <td>29.60</td>
    <td>29.60</td>
    <td>29.60</td>
  </tr>
  <tr>
    <td>100</td>
    <td>3.74</td>
    <td>1.66</td>
    <td>29.60</td>
    <td>29.60</td>
  </tr>
  <tr>
    <td>500</td>
    <td>2.50</td>
    <td>0.02</td>
    <td>29.43</td>
    <td>29.45</td>
  </tr>
  <tr>
    <td>Full</td>
    <td>17.86</td>
    <td>0.00</td>
    <td>29.41</td>
    <td>29.18</td>
  </tr>
</table>

Table 6: Ablation results for BABELReFT, by comparing with unrestricted ReFT (in two variants, NoReFT and LoReFT) and a source-language-gated BABELReFT-SL. Results are for sequential editing with Llama 3.1 8B. Metrics are averaged over target languages and multiplied by 100 for readability; bold indicates best. Full results in Appendix B.2.

Figure 2: XQuAD exact match scores of a selection of cross-lingually disabling edits performed in the following editing languages: JA, IT, FR, DE.

7 Conclusion

Knowledge Editing (KE) shows promise for maintaining LLM factual accuracy, but faces limitations, especially in cross-lingual contexts both from the evaluation (data quality) and methodological perspective (model collapse). Our benchmark, BABELEdITS, addresses the limitations of previous research by offering diverse, high-quality entity representations obtained using BabelNet synsets and marker-based translation. Our modular approach, BABELReFT, couples entity-scope ReFT modules that activate only when necessary using BabelNet synsets as “multilingual keys”, achieving CKE effectiveness (through wide coverage) and model robustness (avoiding model collapse). This prevents indiscriminate gate activation or non-existing cross-lingual edit-transfer, displayed by competing methods such as GRACE in single edits and sequential editing, respectively. We find that cross-lingual multi-hop portability is challenging for all methods, including BABELReFT. Future work could further exploit multilingual knowledge graphs like BabelNet to address this limitation, extending existing monolingual approaches (Zhang et al., 2024a).

\(^{8}\)2 bytes per parameter.
Limitations

Choice of baselines. Our experiments compare against four baselines: FT-L, FT-M, r-ROME, and GRACE. More baselines could have been used but we chose to keep baselines that are the most relevant to our discourse. First, we used fine-tuning baselines (FT-L and FT-M) because they are the simplest baselines we could find. Then we chose r-ROME and GRACE as competitive baselines representing parameter-altering and parameter-preserving methods respectively.

r-ROME (Gupta et al., 2024a) was selected among all the parameter-altering approaches because it was explicitly designed to avoid model collapse, while most methods in the same category are detrimental to downstream performance (Li et al., 2024), including MEMIT, PMET, MEND, and KN. Moreover, we discarded all meta-learning approaches like MEND because they require additional training data to train the hypernetwork that can then be applied to new unseen edits. While this paper provides such a training set through the BABELEdITS dataset, meta-learning methods are deemed out-of-scope for our work, since they are not directly comparable to other methods.

GRACE is chosen among other parameter-preserving approaches for similar reasons: it aims to avoid model collapse, while other methods were often proposed for different purposes. For example, SERAC was proposed for editing a model without access to its weights (Mitchell et al., 2022b), and IKE aims at compute-efficiency (Zheng et al., 2023). Moreover, we discard in-context learning approaches because it is unclear how they should be applied to downstream tasks. More importantly, while IKE is compute-efficient for a single edit, performing thousands of edits would require a larger prompt that might exceed the context window or render inference latency impractical.

Choice of models. We evaluate BABELReFT and the baselines with two relatively small models: Llama 3.1 8B Instruct and Gemma 9B Instruct. Models of this size were selected for practical reasons. Instruct versions of the models were chosen over base ones because they are expected to perform better on downstream evaluation. Finally, this work focuses on English-centric models, while it could have been tested on more multilingual models like Aya or Bloom. Nevertheless, English-centric models are still widely used, even in a multilingual context. While our work focuses on the editing method rather than the model itself, future work that attempts to get the most accurate edited multilingual model might need to rely on larger and explicitly multilingual models.

Choice of languages The proposed BABELEdITS dataset contains 60 languages and improves upon previous datasets which contain at most 53 languages (Nie et al., 2024). BABELEdITS includes several low-resource languages, with namely 9 languages among the class 1 from Joshi et al. (2020) (the "scrapping-bys"). In contrast, the only absent class is class 0, for obvious reasons since it contains languages with virtually no unlabelled data available.

BABELReFT and the compared baselines are not evaluated on all 60 languages, but only on a subset of 11 languages due to computational constraints. However, those 11 languages were selected before the experiments to obtain diversity in scripts, language families, and degrees of resourcefulness.

Ethical considerations

Like any other knowledge editing method, the proposed BABELReFT method can be used for harmful purposes. Since it injects new knowledge into an existing LLM, it can be used to propagate false information. While the KE methods still seem to be in their infancy, they might not directly threaten access to information. But if and when KE methods become production-ready, they could help make LLM more accurate just as well as inject harmful false information.

Cross-lingual knowledge editing also presents an opportunity to bridge some gaps in information access across languages. LLMs can have factual inconsistencies across languages (Fierro and Søgaard, 2022; Qi et al., 2023), and CKE could help address that. However, there is also a chance that KE techniques could uniformize information across languages to a point where cultural exception is suppressed. While this paper is still far from posing such a threat, we advocate that all researchers involved in knowledge editing keep this ethical consideration in mind.

Acknowledgments

The authors acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) through grant INST 35/1597-1 FUGG. Additional computational
resources were provided by the German AI Service Center WestAI (Jülich Supercomputing Centre, 2021).

Goran Glavaš is supported by the Alcatel Lucent Stiftung and Deutsches Stiftungszentrum through the grant “Equitably Fair and Trustworthy Language Technology” (EQUIFAIR, Grant Nr-T0067/43110/23).

We thank Natalia Bobkova and Laura Zanella, two treasured colleagues of one of the authors. Their help during the rebuttal in evaluating the marker-based translation in two additional languages (Russian and Spanish) was instrumental in strengthening our claim. We thank Daniel Ruffinelli and Sotaro Takeshita for their feedback on a draft of this paper.

References

Alfred V. Aho and Margaret J. Corasick. 1975. Efficient string matching: an aid to bibliographic search. Commun. ACM, 18(6):333–340.

Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.

Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2024. The belebele benchmark: a parallel reading comprehension dataset in 122 language variants. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 749–775, Bangkok, Thailand. Association for Computational Linguistics.

Stella Biderman, Hailey Schoelkopf, Lintang Sutawika Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sid Black, Jordan Clive, Anthony DiPofi, Julian Etxaniz, Benjamin Fattori, Jessica Zosa Forde, Charles Foster, Mimansa Jaiswal, Wilson Y. Lee, Haonan Li, Charles Lovering, Niklas Muenninghoff, Ellie Pavlick, Jason Phang, Aviya Skowron, Samson Tan, Xiangru Tang, Kevin A. Wang, Genta Indra Winata, Francois Yvon, and Andy Zou. 2024. Lessons from the trenches on reproducible evaluation of language models. ArXiv preprint, abs/2405.14782.

Lukas Biewald. 2020. Experiment tracking with weights and biases. Software available from wandb.com.

Yang Chen, Chao Jiang, Alan Ritter, and Wei Xu. 2023. Frustratingly easy label projection for cross-lingual transfer. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5775–5796, Toronto, Canada. Association for Computational Linguistics.

Bilal Chughtai, Alan Cooney, and Neel Nanda. 2024. Summing up the facts: Additive mechanisms behind factual recall in llms. ArXiv preprint, abs/2402.07321.

Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283–298.

Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejía Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachik, Christophe Ropers, Safiyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation.

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493–8502, Dublin, Ireland. Association for Computational Linguistics.

John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, et al. 2024. Aya expande: Combining research breakthroughs for a new multilingual frontier. ArXiv preprint, abs/2412.04261.

Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491–6506, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. Language, Speech and Communication. Mit Press.

Constanza Fierro and Anders Søgaard. 2022. Factual consistency of multilingual pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3046–3052, Dublin, Ireland. Association for Computational Linguistics.

Bond Francis and Paik Kyonghee. 2012. A survey of wordnets and their licenses. In Proceedings of the 6th
Global WordNet Conference, Matsue, Japan. Tribun EU. Gebeurtenis: GWC2012.

Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, Singapore. Association for Computational Linguistics.

Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30–45, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, and et al. 2024. The llama 3 herd of models.

Tommaso Green, Simone Paolo Ponzetto, and Goran Glavaš. 2023. Massively multilingual lexical specialization of multilingual transformers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7700–7715, Toronto, Canada. Association for Computational Linguistics.

Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. Model editing harms general abilities of large language models: Regularization to the rescue. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16801–16819, Miami, Florida, USA. Association for Computational Linguistics.

Akshat Gupta, Sidharth Baskaran, and Gopala Anumanchipalli. 2024a. Rebuilding ROME : Resolving model collapse during sequential model editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21738–21744, Miami, Florida, USA. Association for Computational Linguistics.

Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. 2024b. Model editing at scale leads to gradual and catastrophic forgetting. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15202–15232, Bangkok, Thailand. Association for Computational Linguistics.

Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.

Peter Hase, Mohit Bansal, Been Kim, and Asma Ghanbarioun. 2023. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.

Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023. Detecting edit failures in large language models: An improved specificity benchmark. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11548–11559, Toronto, Canada. Association for Computational Linguistics.

Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics.

Jülich Supercomputing Centre. 2021. JURECA: Data Centric and Booster Modules implementing the Modular Supercomputing Architecture at Jülich Supercomputing Centre. Journal of large-scale research facilities, 7(A182).

Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, and Hinrich Schütze. 2025. Mexa: Multilingual evaluation of english-centric LLMs via cross-lingual alignment.

Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. Association for Computational Linguistics.

Hugo Laurençon, Lucile Saulnier, Thomas Wang, Christopher Akiki, Albert Villanova del Moral, Teven Le Scao, Leandro von Werra, Chenghao Mou, Eduardo González Ponferrada, Huu Nguyen, Jörg Frohberg, Mario Sasko, Quentin Lhoest, Angelina McMillan-Major, Gérard Dupont, Stella Biderman, Anna Rogers, Loubna Ben Allal, Francesco De Toni, Giada Pistilli, Olivier Nguyen, Somaieh Nikpoor, Maraim Masoud, Pierre Colombo, Javier de la Rosa, Paulo Villegas, Tristan Thrush, Shayne Longpre, Sebastian Nagel, Leon Weber, Manuel Muñoz, Jian Zhu, Daniel van Strien, Zaid Alyafeai, Khalid Almubarak, Minh Chien Vu, Itziar Gonzalez-Dios, Aitor Soroa, Kyle Lo, Manan Dey, Pedro Ortiz
Suarez, Aaron Gokaslan, Shamik Bose, David Ife-oluwu Adelani, Long Phan, Hieu Tran, Ian Yu, Suhas Pai, Jenny Chim, Violette Lepercq, Suzana Ilic, Margaret Mitchell, Alexandra Sasha Luccioni, and Yacine Jernite. 2022. The bigscience ROOTS corpus: A 1.6tb composite multilingual dataset. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.

Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancouver, Canada. Association for Computational Linguistics.

Qi Li, Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Xinglin Pan, and Xiaowen Chu. 2024. Should we really edit language models? on the evaluation of edited language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024.

Tian Liang, Xing Wang, Mingming Yang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Addressing entity translation problem via translation difficulty and context diversity. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11628–11638, Bangkok, Thailand. Association for Computational Linguistics.

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.

Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022a. Memory-based model editing at scale. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 15817–15831. PMLR.

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022b. Memory-based model editing at scale. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 15817–15831. PMLR.

Roberto Navigli, Michele Bevilacqua, Simone Conia, Dario Montagnini, and Francesco Cecconi. 2021. Ten years of babelnet: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pages 4559–4567. ijcai.org.

Roberto Navigli and Simone Paolo Ponzetto. 2010. BabelNet: Building a very large multilingual semantic network. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 216–225, Uppsala, Sweden. Association for Computational Linguistics.

Ercong Nie, Bo Shao, Zifeng Ding, Mingyang Wang, Helmut Schmid, and Hinrich Schütze. 2024. Bmike-53: Investigating cross-lingual knowledge editing with in-context learning.

Jirui Qi, Raquel Fernández, and Arianna Bisazza. 2023. Cross-lingual consistency of factual knowledge in multilingual language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10650–10666, Singapore. Association for Computational Linguistics.

Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussienot, Thomas Mesnard, Bobak Shahriari, and et al. 2024. Gemma 2: Improving open language models at a practical size.

Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Dan Garrette, Graham Neubig, and Melvin Johnson. 2021. XTREME-R: Towards more challenging and nuanced multilingual evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10215–10245, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Jiaan Wang, Yunlong Liang, Zengkui Sun, Yuxuan Cao, Jiarong Xu, and Fandong Meng. 2024a. Cross-lingual knowledge editing in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11676–11686, Bangkok, Thailand. Association for Computational Linguistics.

Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, Kangwei Liu, Yuansheng Ni, Guozhou Zheng, and Huajun Chen. 2024b. EasyEdit: An easy-to-use knowledge editing framework for large language models. In Proceedings of
the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 82–93, Bangkok, Thailand. Association for Computational Linguistics.

Weixuan Wang, Barry Haddow, and Alexandra Birch. 2024c. Retrieval-augmented multilingual knowledge editing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 335–354, Bangkok, Thailand. Association for Computational Linguistics.

Zihao Wei, Jingcheng Deng, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. 2025. MLaKE: Multilingual knowledge editing benchmark for large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4457–4473, Abu Dhabi, UAE. Association for Computational Linguistics.

Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2024. Reft: Representation fine-tuning for language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024.

Omry Yadan. 2019. Hydra - a framework for elegantly configuring complex applications. Github.

Jinghui Yan, Jiajun Zhang, JinAn Xu, and Chengqing Zong. 2019. The impact of named entity translation for neural machine translation. In Machine Translation, pages 63–73, Singapore. Springer Singapore.

Haoran Yang, Deng Cai, Huayang Li, Wei Bi, Wai Lam, and Shuming Shi. 2024a. A frustratingly simple decoding method for neural text generation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 536–557, Torino, Italia. ELRA and ICCL.

Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024b. The butterfly effect of model editing: Few edits can trigger large language models collapse. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5419–5437, Bangkok, Thailand. Association for Computational Linguistics.

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Du Su, Dawei Yin, and Huawei Shen. 2024c. The fall of ROME: Understanding the collapse of LLMs in model editing. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4079–4087, Miami, Florida, USA. Association for Computational Linguistics.

Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10222–10240, Singapore. Association for Computational Linguistics.

Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, and Zhumin Chen. 2024a. Knowledge graph enhanced large language model editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22647–22662, Miami, Florida, USA. Association for Computational Linguistics.

Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. 2024b. A comprehensive study of knowledge editing for large language models.

Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2025. Multilingual knowledge editing with language-agnostic factual neurons. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5775–5788, Abu Dhabi, UAE. Association for Computational Linguistics.

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4862–4876, Singapore. Association for Computational Linguistics.

Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
A Additional Methodology

A.1 Evaluation Metrics
We report the formulations of rewrite score (RS), paraphrase score (PS), and portability score (PoS) used in our study.

\[
RS = \frac{p_{\theta^*}(o'|\pi_{rel}(s, r)) - p_\theta(o'|\pi_{rel}(s, r))}{1 - p_\theta(o'|\pi_{rel}(s, r))}
\]
(3)

\[
PS = \frac{p_{\theta^*}(o'|\pi_{gen}(s, r)) - p_\theta(o'|\pi_{gen}(s, r))}{1 - p_\theta(o'|\pi_{gen}(s, r))}
\]
(4)

\[
PoS = \frac{p_{\theta^*}(o'|\pi_{port}(s, r)) - p_\theta(o'|\pi_{port}(s, r))}{1 - p_\theta(o'|\pi_{port}(s, r))}
\]
(5)

where \( p_{\theta^*} \) is the output distribution of the edited model and \( p_\theta \) is the output distribution of the original model. For locality, we use the neighborhood KL-divergence score (Hoelscher-Obermaier et al., 2023) over a locality prompt:

\[
NKL =
\sum_{o'_1, ..., o'_m} p_\theta(o'_i|\pi_{loc}(s, r, o'_i)) \log \frac{p_\theta(o'_i|\pi_{loc}(s, r, o'_i))}{p_{\theta^*}(o'_i|\pi_{loc}(s, r, o'_i))}
\]
(6)

where \( o'_i \) is the \( i \)-th token of the object \( o' \) and \( \pi_{loc}(s, r, o'_i) \) is the locality prompt truncated at the \( i \)-th token.

A.2 GRACE
GRACE maintains a codebook (at a specific layer), which stores key-value pairs with keys being cached activations and values learned hidden state vectors that modify the behavior of the model. If a hidden state \( h^{l-1} \) falls into the ball of radius \( \varepsilon_i \) centered on a key \( k_i \) in a set of stored keys \( \mathbb{K} \), then the corresponding value \( v_i \in \mathbb{V} \), which is learned through backpropagation, will replace it (where \( d(\cdot) \) is some distance function):

\[
h^l = \begin{cases}
v_i & \text{if } \exists (k_i, v_i) \in \mathbb{K} \times \mathbb{V} \\
& \text{s.t. } d(h^{l-1}, k_i)) < \varepsilon_i \\
f^l(h^{l-1}) & \text{otherwise}
\end{cases}
\]
(7)

As new edits come in, the codebook is updated mostly by shrinking existing radii so that the edits do not interfere. However, as we show in Section 5 the efficacy of GRACE in cross-lingual KE highly depends on the sensitive choice of the initial cluster radius \( \varepsilon_{init} \). The gating function should activate only on the edited prompt and its semantic equivalents across languages and not semantically related entities within a language; this, however, is difficult to achieve due to the limited semantic alignment of LLM hidden representations across languages (Kargaran et al., 2025).

A.3 Language selection
We report the languages included in our benchmark in Table 7.

A.4 Relation selection
To construct BABELEDITS, we initially sampled the 200 most frequent relations from our set of extracted synsets. We then manually selected the most appropriate ones, resulting in a final set of 132 relations for our benchmark after filtering. Relations were excluded if they exhibited any of the following issues:

• Relations that can have many different answers, like SEMANTICALLY_RELATED (Alma mater, SEMANTICALLY_RELATED, Mean Girls 2) or INSTANCE_OF (1672, INSTANCE_OF, Calendar year)

• Relations which do not make sense when edited. for examples, if the subject and the object are similar like GIVEN_NAME (Miklós Horthy, GIVEN_NAME, Miklós)

• Relations that are very specific to a given field, like PARENT_TAXON (Coronaviridae, PARENT_TAXON, Nidovirales)

• Relations that reflects the structure of Wikipedia or BabelNet rather than the actual world (9/11, WIKIMEDIA_OUTLINE, Outline of the September 11 attacks)

B Additional Results

B.1 Translation quality assessment
We manually compared the quality of entity translations produced by the EasyProject method (Chen et al., 2023) with those obtained using Google Translate. Since four of the authors are native speakers of different languages9, we randomly sampled up to 100 translations from the test set for each of the following languages: German, Italian, French, Croatian, Spanish, and Russian.

9along with two additional colleagues who assisted during the rebuttal for Spanish and Russian.
<table>
  <tr>
    <th>Language</th>
    <th>ISO 639-1 code</th>
    <th># Wikipedia articles (in millions)</th>
    <th>Class in Joshi et al. (2020)</th>
    <th>Script</th>
    <th>Language family</th>
  </tr>
  <tr><td>Afrikaans</td><td>AF</td><td>0.09</td><td>3</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td><b>Arabic</b></td><td>AR</td><td>1.02</td><td>5</td><td>Arabic</td><td>Afro-Asiatic</td></tr>
  <tr><td>Azerbaijani</td><td>AZ</td><td>0.18</td><td>1</td><td>Latin</td><td>Turkic</td></tr>
  <tr><td>Belarusian</td><td>BE</td><td>0.43</td><td>3</td><td>Cyrillic</td><td>IE: Slavic</td></tr>
  <tr><td>Bulgarian</td><td>BG</td><td>0.26</td><td>3</td><td>Cyrillic</td><td>IE: Slavic</td></tr>
  <tr><td>Bengali</td><td>BN</td><td>0.08</td><td>3</td><td>Brahmic</td><td>IE: Indo-Aryan</td></tr>
  <tr><td>Catalan</td><td>CA</td><td>1.70</td><td>4</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td>Czech</td><td>CS</td><td>1.57</td><td>4</td><td>Latin</td><td>IE: Slavic</td></tr>
  <tr><td>Danish</td><td>DA</td><td>0.79</td><td>3</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td><b>German</b></td><td>DE</td><td>2.37</td><td>5</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td>Greek</td><td>EL</td><td>0.17</td><td>3</td><td>Greek</td><td>IE: Greek</td></tr>
  <tr><td><b>English</b></td><td>EN</td><td>5.98</td><td>5</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td>Spanish</td><td>ES</td><td>1.56</td><td>5</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td>Estonian</td><td>ET</td><td>0.2</td><td>3</td><td>Latin</td><td>Uralic</td></tr>
  <tr><td>Basque</td><td>EU</td><td>0.34</td><td>4</td><td>Latin</td><td>Basque</td></tr>
  <tr><td>Persian</td><td>FA</td><td>0.7</td><td>4</td><td>Perso-Arabic</td><td>IE: Iranian</td></tr>
  <tr><td>Finnish</td><td>FI</td><td>0.47</td><td>4</td><td>Latin</td><td>Uralic</td></tr>
  <tr><td><b>French</b></td><td>FR</td><td>2.16</td><td>5</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td>Gujarati</td><td>GU</td><td>0.03</td><td>1</td><td>Brahmic</td><td>IE: Indo-Aryan</td></tr>
  <tr><td>Hebrew</td><td>HE</td><td>0.25</td><td>3</td><td>Jewish</td><td>Afro-Asiatic</td></tr>
  <tr><td>Hindi</td><td>HI</td><td>0.13</td><td>4</td><td>Devanagari</td><td>IE: Indo-Aryan</td></tr>
  <tr><td><b>Croatian</b></td><td>HR</td><td>0.54</td><td>4</td><td>Latin</td><td>Slavic</td></tr>
  <tr><td>Haitian Creole</td><td>HT</td><td>0.06</td><td>2</td><td>Latin</td><td>Creole</td></tr>
  <tr><td>Hungarian</td><td>HU</td><td>0.46</td><td>4</td><td>Latin</td><td>Uralic</td></tr>
  <tr><td>Armenian</td><td>HY</td><td>0.89</td><td>1</td><td>Armenian alphabet</td><td>IE: Armenian</td></tr>
  <tr><td>Indonesian</td><td>ID</td><td>0.51</td><td>3</td><td>Latin</td><td>Austronesian</td></tr>
  <tr><td><b>Italian</b></td><td>IT</td><td>1.57</td><td>4</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td><b>Japanese</b></td><td>JA</td><td>1.18</td><td>5</td><td>Ideograms</td><td>Japonic</td></tr>
  <tr><td>Javanese</td><td>JV</td><td>0.06</td><td>1</td><td>Brahmic</td><td>Austronesian</td></tr>
  <tr><td><b>Georgian</b></td><td>KA</td><td>0.13</td><td>3</td><td>Georgian</td><td>Kartvelian</td></tr>
  <tr><td>Kazakh</td><td>KK</td><td>0.23</td><td>3</td><td>Arabic</td><td>Turkic</td></tr>
  <tr><td>Korean</td><td>KO</td><td>0.47</td><td>4</td><td>Hangul</td><td>Koreanic</td></tr>
  <tr><td>Lithuanian</td><td>LT</td><td>0.2</td><td>3</td><td>Latin</td><td>IE: Baltic</td></tr>
  <tr><td>Malayalam</td><td>ML</td><td>0.07</td><td>1</td><td>Brahmic</td><td>Dravidian</td></tr>
  <tr><td>Marathi</td><td>MR</td><td>0.06</td><td>2</td><td>Devanagari</td><td>IE: Indo-Aryan</td></tr>
  <tr><td>Malay</td><td>MS</td><td>0.33</td><td>3</td><td>Latin</td><td>Austronesian</td></tr>
  <tr><td><b>Burmese</b></td><td>MY</td><td>0.05</td><td>1</td><td>Brahmic</td><td>Sino-Tibetan</td></tr>
  <tr><td>Dutch</td><td>NL</td><td>1.99</td><td>4</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td>Norwegian</td><td>NO</td><td>1.53</td><td>1</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td>Punjabi</td><td>PA</td><td>0.04</td><td>2</td><td>Brahmic</td><td>IE: Indo-Aryan</td></tr>
  <tr><td>Polish</td><td>PL</td><td>1.44</td><td>4</td><td>Latin</td><td>IE: Slavic</td></tr>
  <tr><td>Portuguese</td><td>PT</td><td>1.02</td><td>4</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td><b>Cusco Quechua</b></td><td>QU</td><td>0.02</td><td>1</td><td>Latin</td><td>Quechuan</td></tr>
  <tr><td>Romanian</td><td>RO</td><td>0.42</td><td>3</td><td>Latin</td><td>IE: Romance</td></tr>
  <tr><td>Russian</td><td>RU</td><td>1.58</td><td>4</td><td>Cyrillic</td><td>IE: Slavic</td></tr>
  <tr><td>Slovak</td><td>SK</td><td>0.57</td><td>3</td><td>Latin</td><td>IE: Slavic</td></tr>
  <tr><td>Swedish</td><td>SV</td><td>6.21</td><td>4</td><td>Latin</td><td>IE: Germanic</td></tr>
  <tr><td>Serbian</td><td>SR</td><td>3.73</td><td>4</td><td>Serbian Cyrillic</td><td>IE: Slavic</td></tr>
  <tr><td>Swahili</td><td>SW</td><td>0.05</td><td>2</td><td>Latin</td><td>Niger-Congo</td></tr>
  <tr><td>Tamil</td><td>TA</td><td>0.12</td><td>3</td><td>Brahmic</td><td>Dravidian</td></tr>
  <tr><td>Telugu</td><td>TE</td><td>0.07</td><td>1</td><td>Brahmic</td><td>Dravidian</td></tr>
  <tr><td>Thai</td><td>TH</td><td>0.13</td><td>3</td><td>Brahmic</td><td>Kra-Dai</td></tr>
  <tr><td>Tagalog</td><td>TL</td><td>0.08</td><td>3</td><td>Brahmic</td><td>Austronesian</td></tr>
  <tr><td>Turkish</td><td>TR</td><td>0.34</td><td>4</td><td>Latin</td><td>Turkic</td></tr>
  <tr><td>Ukrainian</td><td>UK</td><td>1.06</td><td>3</td><td>Cyrillic</td><td>IE: Slavic</td></tr>
  <tr><td>Urdu</td><td>UR</td><td>0.15</td><td>3</td><td>Perso-Arabic</td><td>IE: Indo-Aryan</td></tr>
  <tr><td>Uzbek</td><td>UZ</td><td>0.52</td><td>3</td><td>Latin</td><td>Turkic</td></tr>
  <tr><td>Vietnamese</td><td>VI</td><td>1.24</td><td>4</td><td>Latin</td><td>Austro-Asiatic</td></tr>
  <tr><td>Yoruba</td><td>YO</td><td>0.03</td><td>2</td><td>Arabic</td><td>Niger-Congo</td></tr>
  <tr><td><b>Mandarin</b></td><td>ZH</td><td>1.09</td><td>5</td><td>Chinese ideograms</td><td>Sino-Tibetan</td></tr>
</table>

Table 7: Languages composing the BABELEDITS dataset. Languages in bold are the ones used for evaluation.
For each annotator, the translation pairs were randomly inverted to make it impossible to guess which one is the raw translation and which one is the result of applying EasyProject.

We report the results of the translation quality assessment in Table 8 and the annotator instructions in Table 9.

B.2 Additional results
The following additional results can be found at the end of the Appendix:

• Extended results with all evaluation aspects and number of sequential edits in English of Llama 3.1 and Gemma 2 in Table 15.

• Detail of evaluation metrics on each target language after sequential editing in English on the full BABELEdITS dataset with Llama 3.1 in Table 16.

• Detail of evaluation metrics on each target language after sequential editing in English on the full BABELEdITS dataset with Gemma 2 in Table 17.

• Evaluation of sequential editing in English of Llama 3.1 and Gemma 2 on the MzSRE dataset (Zhang et al., 2025) on the languages that intersect with our evaluation set (DE, EN, FR, ZH) in Table 18.

• Full ablations results for both models are in Table 19.

• Results for single editing on Llama 3.1 in Table 20.

• Results for single editing on Gemma 2 in Table 21.

C Additional Experimental Details

C.1 Computing resources
We perform all of our editing experiments using the EasyEdit library (Wang et al., 2024b) on a single NVIDIA A6000/A100/A40 GPU (40 or 48 GB) using bfloat16 precision. Each run takes between 5 to 20 hours: we estimate our editing experiments to have required circa 2,250 GPU hours.

C.2 Hyperparameter Selection
To pick the hyperparameters we perform a grid search for each method/model/ {single, sequential}-editing combination using a random subset of 100 edits from the validation split of BABELEdITS, due to the combinatorially large hyperparameter search space. We perform the editing in English and use average reliability across languages as a validation criterion. We search over the following grids:

• FT-L: Layers: all, Learning Rate: \( \{1e-4, 5e-4\} \), Norm Constraint: \( \{2e-3, 1e-4, 2e-5\} \)

• FT-M: Layers: all, Learning Rate: \( \{1e-4, 5e-4\} \)

• r-ROME: Layers: all, KL Factor: \( \{0.0625, 0.9, 1\} \),

• GRACE: Layers: all, Learning Rate: \( \{0.1, 1.0\} \), Replacement: \( \{\text{last}, \text{all}\} \), \( \varepsilon_{init} : \{0.1, 1.0, 100\} \)

• BABELEdITS: Layers: all, Learning Rate: \( \{1e-4, 1e-3, 2e-3\} \), Low-rank dimensionality: \( \{4, 16, 64\} \)

For FT-L, Norm Constraint indicates the \( L_\infty \) norm constraint. For r-ROME, KL factor indicates the weight of the KL term in the \( v \) optimization term. For GRACE, replacement indicates whether the replaced hidden states are all or just the one at the last token position. The best-found hyperparameters are in Table 10.

C.3 Models Used
We report in Table 11 the models used in our study together with their Huggingface Hub links for download.

C.4 Prompt for GPT-4-based template prompt creation
To verbalize these relations into usable prompts, we provide GPT-4o with the relation \( r \) and an example of subject \( s \) and \( o \) from the previously extracted synsets and ask it to provide a template prompt \( \pi((s), r) \) to be later filled with the appropriate subject. For example, for the relation \( r = \text{LOCATEDIN} \), then the GPT-4o output was \( \pi((s), r) = \text{Where is } (s) \text{ located in?} \). We additionally ask GPT-4o to generate a rephrased version of \( \pi((s), r) \) to create the generality set of BABELEdITS.

We report in Table 12 the full prompt used to ask GPT-4o to create template prompts and rephrase template prompts for BABELEdITS. In Table 13 we present the prompt used to have GPT-4o generate the multi-hop portability prompts.
<table>
  <tr>
    <th>Language</th>
    <th>Preference Ratio (%)</th>
    <th>Annotation Size</th>
    <th>Different Prompts</th>
  </tr>
  <tr>
    <td>Italian</td>
    <td>89.0%</td>
    <td>100</td>
    <td>512</td>
  </tr>
  <tr>
    <td>French</td>
    <td>90.0%</td>
    <td>100</td>
    <td>429</td>
  </tr>
  <tr>
    <td>German</td>
    <td>81.9%</td>
    <td>83</td>
    <td>193</td>
  </tr>
  <tr>
    <td>Croatian</td>
    <td>59.2%</td>
    <td>71</td>
    <td>133</td>
  </tr>
  <tr>
    <td>Spanish</td>
    <td>76.0%</td>
    <td>100</td>
    <td>339</td>
  </tr>
  <tr>
    <td>Russian</td>
    <td>56.0%</td>
    <td>100</td>
    <td>331</td>
  </tr>
</table>

Table 8: Results of the translation quality assessment for 6 languages. Different prompts indicates the number of prompts in the test set for which the extended prompts \( \pi(s, r, o') \) obtained with MT applied separately to subject, object and prompt and our marked translations obtained with EasyProject differ.

You will be presented with a prompt for a knowledge editing task in the English Language. Together with that, you will be provided with two translations under the column labelled “A” and “B”.
Your task is to express a preference for one of the two translations. Compare the English prompt with the one in your mother tongue and choose the one between the options “A” and “B” which sounds more correct to you both in terms of how grammatical it is and how well the subject and object are translated.
You must always express a preference. If you are unsure about the nature of the subject and object of the prompt, you can find BabelNet links to both in the two columns titled “BabelNet Subject URL” and “BabelNet Object URL”. Simply write A or B in capital letters in the column titled “Preference”.

Table 9: Task descriptions for the annotators who were asked to select between one of the two possible translations of the English prompt (pure MT prompt vs. our EasyProject marked translation.)

C.5 Prompts used for downstream evaluation

We evaluate downstream performance using the lm-eval library (Biderman et al., 2024), in a zero-shot fashion on the intersection of our 10 languages and the languages, in the Belebele benchmark (all but QU) and XQuAD dataset (AR, DE, EN, ZH).

The prompts used for downstream evaluation for the two downstream tasks (Belebele and XQuAD) are reported in Table 14.

D Scientific artifacts

D.1 BabelNet License

BABELEdITS is a KE benchmark made from BabelNet v5.3 downloaded from https://babelnet.org, made available with the BabelNet NonCommercial License (see https://babelnet.org/full-license).

D.2 Software Used

This project utilized the following key software libraries:

• The BabelNet Python API (version 1.2.0) was used to access and query BabelNet (Navigli and Ponzetto, 2010) and is released with the same license as BabelNet.

• Weights & Biases (wandb, Biewald (2020)) version 0.18.7 was employed for experiment tracking and hyperparameter optimization. The Python SDK has MIT License.

• hydra (Yadan, 2019) was used for configuration management (version 1.3.2, MIT License).

• EasyEdit (Zhang et al., 2024b) was used for performing knowledge editing with the reported baselines (no version naming, MIT License).

• The Google Cloud Translate API (Python SDK version 3.18.0)

The main Python dependencies were the following, and all were used within the boundaries of their license:

• pyrefit (0.0.8, Apache License 2.0)
• pyvne (0.1.6, Apache License 2.0)
<table>
  <tr>
    <th>Model</th>
    <th>Setting</th>
    <th>Layer</th>
    <th>Learning Rate</th>
    <th>Norm Constr.</th>
    <th>PTL</th>
    <th>Layer</th>
    <th>Learning Rate</th>
    <th>PTM</th>
    <th>Layer</th>
    <th>KL factor</th>
    <th>r-ROME</th>
    <th>Layer</th>
    <th>Learning Rate</th>
    <th>Replacement</th>
    <th>GRACE</th>
    <th>Layer</th>
    <th>Learning Rate</th>
    <th>BARRI-KRFT</th>
    <th>Layer</th>
    <th>Learning Rate</th>
    <th>Low Rank</th>
  </tr>
  <tr>
    <td>Llama 3.1</td>
    <td>Single</td>
    <td>21</td>
    <td>5e-4</td>
    <td>0.002</td>
    <td>15</td>
    <td>5e-4</td>
    <td>15</td>
    <td>0.0625</td>
    <td>19</td>
    <td>0.1</td>
    <td>last</td>
    <td>100</td>
    <td>1.2</td>
    <td>2e-3</td>
    <td>64</td>
  </tr>
  <tr>
    <td></td>
    <td>Sequential</td>
    <td>19</td>
    <td>1e-4</td>
    <td>0.002</td>
    <td>21</td>
    <td>5e-4</td>
    <td>17</td>
    <td>1.0</td>
    <td>21</td>
    <td>0.1</td>
    <td>all</td>
    <td>100</td>
    <td>1.2</td>
    <td>2e-3</td>
    <td>64</td>
  </tr>
  <tr>
    <td>Gemma 2</td>
    <td>Single</td>
    <td>28</td>
    <td>1e-4</td>
    <td>0.002</td>
    <td>31</td>
    <td>1e-4</td>
    <td>5</td>
    <td>0.9</td>
    <td>31</td>
    <td>0.1</td>
    <td>all</td>
    <td>100</td>
    <td>1.8</td>
    <td>2e-3</td>
    <td>64</td>
  </tr>
  <tr>
    <td></td>
    <td>Sequential</td>
    <td>31</td>
    <td>1e-4</td>
    <td>0.002</td>
    <td>31</td>
    <td>1e-4</td>
    <td>5</td>
    <td>0.9</td>
    <td>31</td>
    <td>0.1</td>
    <td>all</td>
    <td>100</td>
    <td>1.8</td>
    <td>2e-3</td>
    <td>64</td>
  </tr>
</table>

Table 10: Knowledge Editing Methods best-found hyperparameters.

<table>
  <tr>
    <th>Model</th>
    <th>URL</th>
  </tr>
  <tr>
    <td>Llama 3.1 8B Instruct</td>
    <td>https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct</td>
  </tr>
  <tr>
    <td>Gemma 2 9B Instruct</td>
    <td>https://huggingface.co/google/gemma-2-9b-it</td>
  </tr>
  <tr>
    <td>NLLB 200 600M</td>
    <td>https://huggingface.co/google/nllb-200-distilled-600M</td>
  </tr>
</table>

Table 11: URLs of the models used in our study for the KE task (Llama 3.1, Gemma 2) or creating the subject aliasing prompts (NLLB).

You are a helpful assistant that is able to leverage its world knowledge to convert relations extracted from a knowledge graph (for example, WordNet or Babelnet) into natural language questions. Given the relations provided in the user input, create a question for each relation.
In the case of the relation PLAYS_FOR, the question could be “Which team does <subject> play for?”.
Moreover, create an additional version of the question by rephrasing.
The input is a markdown table with 4 columns: relation_name, count, subject, object.
When creating the question, ALWAYS keep the <subject> placeholder. The examples provided as subject and object are there just to help you understand the relation; do NOT include them in the question, which means that you should NOT replace the <subject> placeholder with the examples.
You simply need to output the result in tsv format with 6 columns: relation_name, count, subject, object, question, and rephrase.
For all the columns except question and rephrase, simply copy the values from the input tsv. Reply directly with the tsv file, without ANY additional text.

Table 12: Prompt used to ask GPT-4o to create template prompts and rephrased template prompts.

• HuggingFace datasets library (version 3.1.0, Apache License 2.0)
• HuggingFace tokenizers library (version 0.20.4, Apache License 2.0)
• HuggingFace transformers library (version 4.45.1, Apache License 2.0)
• Eleuther AI Im-eval (version 0.4.7, MIT License)
• pyahocorasick (2.1.0, BSD-3 Clause License)

D.3 Datasets used
• XQuAD (Artetxe et al., 2020) (License: CC BY-SA 4.0)
• Belebele (Bandarkar et al., 2024) (License: CC BY-SA 4.0)
• MzSRE (Wang et al., 2024c) (No license found)
• BabelNet (Navigli and Ponzetto, 2010), see Section D.1
• Wikipedia (License: CC BY-SA 4.0)

E Usage of AI assistants

We use ChatGPT and Claude 3.5 Sonnet to write parts of this paper, including text or creating/refactoring tables. Throughout development, we used GitHub Copilot as our coding assistant.
You are a helpful assistant that is able to leverage its world knowledge to convert relations extracted from a knowledge graph (for example, WordNet or Babelnet) into natural language questions.
In this case we are dealing with joined triples of the form (subject, relation, object, relation_2, object_2). You need to formulate a natural language question which should be answered with object 2. Consider the case of (Messi, PLAYS_FOR, Barcelona, LOCATED_IN, Spain).
The question could be 'In which country is the team that Messi plays for located?'. In the generated question, NEVER mention the object (in this case, Barcelona). Let me repeat: Do NOT INCLUDE the object in the question.
The input will be a markdown table, with five columns: subject, relation, object, relation_2, object_2.
Please reply directly without any additional text, one question per line, no special characters at the beginning of each line and separate each line with a SINGLE newline character and not two. Just a reminder: only one question per line, only one newline character at the end of each line.

<table>
  <tr>
    <th>Task (Language)</th>
    <th>Prompt Template</th>
  </tr>
  <tr>
    <td>Belebele (all)</td>
    <td>P: {{(flores_passage)}}\nQ: {{(question.strip())}}\nA: {{(mc_answer1)}}\nB: {{(mc_answer2)}}\nC: {{(mc_answer3)}}\nD: {{(mc_answer4)}}\nAnswer:</td>
  </tr>
  <tr>
    <td>XQuAD (AR)</td>
    <td>السياق: {{(context)}}\nالسؤال: {{(question)}}\nالإجابة:</td>
  </tr>
  <tr>
    <td>XQuAD (DE)</td>
    <td>Kontext: {{(context)}}\nFrage: {{(question)}}\nAntwort:</td>
  </tr>
  <tr>
    <td>XQuAD (EN)</td>
    <td>Context: {{(context)}}\nQuestion: {{(question)}}\nAnswer:</td>
  </tr>
  <tr>
    <td>XQuAD (ZH)</td>
    <td>语境: {{(context)}}\n问题: {{(question)}}\n回答:</td>
  </tr>
</table>

Table 13: Prompt used to ask GPT-4o to create the prompts for multi-hop portability.

<table>
  <tr>
    <th>Task (Language)</th>
    <th>Prompt Template</th>
  </tr>
  <tr>
    <td>Belebele (all)</td>
    <td>P: {{(flores_passage)}}\nQ: {{(question.strip())}}\nA: {{(mc_answer1)}}\nB: {{(mc_answer2)}}\nC: {{(mc_answer3)}}\nD: {{(mc_answer4)}}\nAnswer:</td>
  </tr>
  <tr>
    <td>XQuAD (AR)</td>
    <td>السياق: {{(context)}}\nالسؤال: {{(question)}}\nالإجابة:</td>
  </tr>
  <tr>
    <td>XQuAD (DE)</td>
    <td>Kontext: {{(context)}}\nFrage: {{(question)}}\nAntwort:</td>
  </tr>
  <tr>
    <td>XQuAD (EN)</td>
    <td>Context: {{(context)}}\nQuestion: {{(question)}}\nAnswer:</td>
  </tr>
  <tr>
    <td>XQuAD (ZH)</td>
    <td>语境: {{(context)}}\n问题: {{(question)}}\n回答:</td>
  </tr>
</table>

Table 14: Prompts used to evaluate models on the two tasks used for downstream evaluation (Belebele and XQuAD) via lm-eval.

F Statistics about BABELEDITIONS

We include some statistics about BABELEDITIONS:

• Table 3a shows the distribution of domains in the test set.

• Table 3b shows the distribution of a number of aliases for each language in the test set.

While many entities have a single mention, as not all entities have aliases, a significant portion includes two or more.
<table>
  <tr>
    <th rowspan="2">Edits</th>
    <th colspan="5">Llama 3.1</th>
    <th colspan="5">Gemma 2</th>
  </tr>
  <tr>
    <th>FT-L</th>
    <th>FT-M</th>
    <th>r-ROME</th>
    <th>GRACE</th>
    <th>BABELReFT</th>
    <th>FT-L</th>
    <th>FT-M</th>
    <th>r-ROME</th>
    <th>GRACE</th>
    <th>BABELReFT</th>
  </tr>
  <tr>
    <th colspan="11">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="11">↑ Reliability</th>
  </tr>
  <tr>
    <td>100</td><td>1.59</td><td>21.94</td><td>-0.01</td><td>9.33</td><td>37.12</td><td>3.16</td><td>15.58</td><td>0.12</td><td>15.51</td><td>42.30</td>
  </tr>
  <tr>
    <td>250</td><td>1.46</td><td>23.48</td><td>-0.02</td><td>9.31</td><td>35.89</td><td>2.81</td><td>12.74</td><td>0.01</td><td>16.36</td><td>41.48</td>
  </tr>
  <tr>
    <td>500</td><td>1.59</td><td>19.98</td><td>-0.02</td><td>9.27</td><td>37.48</td><td>2.12</td><td>10.26</td><td>0.01</td><td>16.54</td><td>40.90</td>
  </tr>
  <tr>
    <td>Full</td><td>1.52</td><td>19.51</td><td>-0.04</td><td>9.26</td><td>36.51</td><td>3.02</td><td>9.79</td><td>0.01</td><td>17.10</td><td>40.72</td>
  </tr>
  <tr>
    <th colspan="11">↑ Generality</th>
  </tr>
  <tr>
    <td>100</td><td>1.29</td><td>19.73</td><td>-0.02</td><td>0.72</td><td>34.40</td><td>2.35</td><td>10.28</td><td>0.12</td><td>8.30</td><td>39.52</td>
  </tr>
  <tr>
    <td>250</td><td>1.55</td><td>21.88</td><td>-0.02</td><td>0.54</td><td>33.74</td><td>2.09</td><td>7.81</td><td>0.01</td><td>8.76</td><td>38.46</td>
  </tr>
  <tr>
    <td>500</td><td>1.48</td><td>17.87</td><td>-0.02</td><td>0.58</td><td>35.15</td><td>1.41</td><td>5.99</td><td>0.01</td><td>8.86</td><td>38.48</td>
  </tr>
  <tr>
    <td>Full</td><td>1.71</td><td>17.69</td><td>-0.04</td><td>0.47</td><td>34.25</td><td>2.37</td><td>5.76</td><td>0.01</td><td>9.36</td><td>38.18</td>
  </tr>
  <tr>
    <th colspan="11">↓ Locality</th>
  </tr>
  <tr>
    <td>100</td><td>7.35</td><td>4.97</td><td>21.13</td><td>0.03</td><td>4.41</td><td>11.00</td><td>6.28</td><td>2.64</td><td>0.06</td><td>5.90</td>
  </tr>
  <tr>
    <td>250</td><td>7.49</td><td>5.66</td><td>14.63</td><td>0.03</td><td>4.12</td><td>9.80</td><td>6.07</td><td>0.71</td><td>0.10</td><td>5.91</td>
  </tr>
  <tr>
    <td>500</td><td>8.10</td><td>5.97</td><td>15.25</td><td>0.02</td><td>4.03</td><td>8.23</td><td>6.65</td><td>0.49</td><td>0.05</td><td>6.15</td>
  </tr>
  <tr>
    <td>Full</td><td>7.34</td><td>5.97</td><td>13.13</td><td>0.03</td><td>4.20</td><td>11.23</td><td>6.75</td><td>0.45</td><td>0.09</td><td>6.37</td>
  </tr>
  <tr>
    <th colspan="11">↑ Subject-Alias portability</th>
  </tr>
  <tr>
    <td>100</td><td>1.49</td><td>17.23</td><td>-0.01</td><td>2.45</td><td>23.08</td><td>1.72</td><td>10.30</td><td>0.01</td><td>9.86</td><td>26.93</td>
  </tr>
  <tr>
    <td>250</td><td>0.52</td><td>15.28</td><td>-0.01</td><td>1.91</td><td>25.65</td><td>1.63</td><td>6.11</td><td>0.00</td><td>10.29</td><td>28.87</td>
  </tr>
  <tr>
    <td>500</td><td>0.83</td><td>11.05</td><td>-0.01</td><td>1.47</td><td>28.33</td><td>0.92</td><td>5.07</td><td>0.00</td><td>7.96</td><td>27.95</td>
  </tr>
  <tr>
    <td>Full</td><td>0.87</td><td>11.93</td><td>-0.01</td><td>1.32</td><td>27.85</td><td>1.36</td><td>4.66</td><td>0.00</td><td>9.18</td><td>28.81</td>
  </tr>
  <tr>
    <th colspan="11">↑ Multi-Hop portability</th>
  </tr>
  <tr>
    <td>100</td><td>-0.16</td><td>0.25</td><td>-0.17</td><td>0.00</td><td>1.00</td><td>0.01</td><td>0.42</td><td>0.07</td><td>0.00</td><td>1.83</td>
  </tr>
  <tr>
    <td>250</td><td>-0.69</td><td>-0.62</td><td>-0.70</td><td>0.00</td><td>0.84</td><td>0.01</td><td>0.20</td><td>0.01</td><td>0.01</td><td>1.32</td>
  </tr>
  <tr>
    <td>500</td><td>-0.53</td><td>-0.32</td><td>-0.55</td><td>0.00</td><td>0.55</td><td>0.03</td><td>0.26</td><td>0.01</td><td>0.00</td><td>0.95</td>
  </tr>
  <tr>
    <td>Full</td><td>-0.37</td><td>0.12</td><td>-0.50</td><td>0.00</td><td>1.27</td><td>0.05</td><td>0.26</td><td>0.01</td><td>0.04</td><td>1.64</td>
  </tr>
  <tr>
    <th colspan="11">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="11">↑ Belebele (accuracy)</th>
  </tr>
  <tr>
    <td>Original</td><td>73.59</td><td>73.59</td><td>73.59</td><td>73.59</td><td>73.59</td><td>84.68</td><td>84.68</td><td>84.68</td><td>84.68</td><td>84.68</td>
  </tr>
  <tr>
    <td>100</td><td>73.42</td><td>73.99</td><td>34.89</td><td>73.59</td><td>73.50</td><td>84.48</td><td>84.46</td><td>24.14</td><td>84.71</td><td>84.59</td>
  </tr>
  <tr>
    <td>250</td><td>73.73</td><td>72.02</td><td>22.51</td><td>73.50</td><td>73.47</td><td>84.49</td><td>84.56</td><td>22.92</td><td>84.71</td><td>84.60</td>
  </tr>
  <tr>
    <td>500</td><td>73.79</td><td>68.06</td><td>28.64</td><td>73.56</td><td>73.50</td><td>84.48</td><td>84.38</td><td>26.07</td><td>84.71</td><td>84.70</td>
  </tr>
  <tr>
    <td>Full</td><td>72.92</td><td>60.26</td><td>22.58</td><td>73.50</td><td>73.39</td><td>84.64</td><td>84.40</td><td>28.62</td><td>84.71</td><td>84.70</td>
  </tr>
  <tr>
    <th colspan="11">↑ XQuAD (EM)</th>
  </tr>
  <tr>
    <td>Original</td><td>29.60</td><td>29.60</td><td>29.60</td><td>29.60</td><td>29.60</td><td>31.79</td><td>31.79</td><td>31.79</td><td>31.79</td><td>31.79</td>
  </tr>
  <tr>
    <td>100</td><td>18.07</td><td>20.00</td><td>0.00</td><td>29.60</td><td>29.60</td><td>29.64</td><td>29.98</td><td>0.13</td><td>31.76</td><td>31.76</td>
  </tr>
  <tr>
    <td>250</td><td>27.21</td><td>12.29</td><td>0.00</td><td>29.71</td><td>29.68</td><td>30.04</td><td>29.81</td><td>0.21</td><td>31.76</td><td>31.72</td>
  </tr>
  <tr>
    <td>500</td><td>19.37</td><td>1.58</td><td>0.00</td><td>29.71</td><td>29.45</td><td>29.03</td><td>28.78</td><td>2.77</td><td>31.76</td><td>31.64</td>
  </tr>
  <tr>
    <td>Full</td><td>19.35</td><td>0.36</td><td>0.00</td><td>29.71</td><td>29.18</td><td>26.37</td><td>29.81</td><td>4.33</td><td>31.76</td><td>31.70</td>
  </tr>
</table>

Table 15: Comprehensive comparison of cross-lingual knowledge editing and downstream task performance for Llama 3.1 8B and Gemma 2 9B models with sequential editing done in English with an increasing number of sequential edits. Editing metrics are averaged over all target languages and multiplied by 100 for readability (except for locality). Bold numbers indicate the best performance for each metric and model combination. Downstream performance is averaged over the target languages: Original indicates the model performance before editing. Results detailed by language are available in Table 16 and Table 17 for Llama 3.1 and Gemma 2 respectively.
<table>
  <tr>
    <th>Method</th>
    <th>AVG</th>
    <th>AR</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>HR</th>
    <th>IT</th>
    <th>JA</th>
    <th>KA</th>
    <th>MY</th>
    <th>QU</th>
    <th>ZH</th>
  </tr>
  <tr>
    <th colspan="13">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Reliability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>1.52</td><td>0.48</td><td>1.83</td><td>6.56</td><td>2.20</td><td>0.91</td><td>2.47</td><td>0.38</td><td>0.47</td><td>0.26</td><td>0.43</td><td>0.78</td>
  </tr>
  <tr>
    <td>FT-M</td><td>19.51</td><td>7.82</td><td>30.52</td><td>63.77</td><td>25.87</td><td>19.86</td><td>33.78</td><td>8.42</td><td>4.83</td><td>0.33</td><td>7.96</td><td>11.42</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.04</td><td>-0.04</td><td>-0.03</td><td>-0.00</td><td>-0.01</td><td>-0.02</td><td>-0.20</td><td>-0.03</td><td>-0.10</td><td>-0.04</td><td>-0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>9.26</td><td>-0.00</td><td>1.13</td><td><b>99.08</b></td><td>0.38</td><td>0.47</td><td>0.75</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>36.51</b></td><td><b>20.90</b></td><td><b>63.04</b></td><td><b>98.49</b></td><td><b>49.86</b></td><td><b>40.92</b></td><td><b>64.38</b></td><td><b>19.44</b></td><td><b>18.38</b></td><td><b>2.17</b></td><td><b>10.81</b></td><td><b>13.19</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ Generality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>1.71</td><td>0.38</td><td>1.83</td><td>8.02</td><td>2.13</td><td>1.14</td><td>2.92</td><td>0.30</td><td>0.50</td><td>0.25</td><td>0.56</td><td>0.83</td>
  </tr>
  <tr>
    <td>FT-M</td><td>17.69</td><td>6.74</td><td>27.56</td><td>55.06</td><td>24.51</td><td>17.90</td><td>32.28</td><td>7.55</td><td>4.39</td><td>0.21</td><td>7.79</td><td>10.58</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.04</td><td>-0.02</td><td>-0.03</td><td>-0.00</td><td>-0.01</td><td>-0.03</td><td>-0.13</td><td>-0.06</td><td>-0.14</td><td>-0.05</td><td>-0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>0.47</td><td>-0.00</td><td>0.76</td><td>2.67</td><td>0.57</td><td>0.47</td><td>0.47</td><td>0.10</td><td>0.09</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>34.25</b></td><td><b>20.90</b></td><td><b>59.17</b></td><td><b>93.69</b></td><td><b>47.64</b></td><td><b>35.17</b></td><td><b>61.30</b></td><td><b>18.66</b></td><td><b>13.30</b></td><td><b>2.25</b></td><td><b>11.61</b></td><td><b>13.00</b></td>
  </tr>
  <tr>
    <th colspan="13">↓ Locality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>7.34</td><td>8.88</td><td>8.97</td><td>9.13</td><td>8.50</td><td>7.35</td><td>8.92</td><td>7.74</td><td>8.21</td><td>1.28</td><td>3.70</td><td>8.05</td>
  </tr>
  <tr>
    <td>FT-M</td><td>5.97</td><td>7.57</td><td>5.96</td><td>6.00</td><td>5.72</td><td>5.10</td><td>5.70</td><td>6.26</td><td>6.38</td><td>6.51</td><td>3.72</td><td>6.79</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>13.13</td><td>9.70</td><td>11.70</td><td>12.66</td><td>12.10</td><td>12.45</td><td>12.21</td><td>12.17</td><td>21.16</td><td>15.34</td><td>14.65</td><td>10.25</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>0.03</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.30</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.04</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td>4.20</td><td>2.50</td><td>6.64</td><td>9.60</td><td>6.18</td><td>4.31</td><td>6.95</td><td>2.56</td><td>1.30</td><td>0.49</td><td>2.16</td><td>3.45</td>
  </tr>
  <tr>
    <th colspan="13">↑ Subject-alias portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>0.87</td><td>0.25</td><td>0.71</td><td>4.19</td><td>0.25</td><td>1.20</td><td>2.21</td><td>0.16</td><td>0.01</td><td>0.31</td><td>0.00</td><td>0.27</td>
  </tr>
  <tr>
    <td>FT-M</td><td>11.93</td><td>2.90</td><td>28.66</td><td>29.77</td><td>12.05</td><td>21.53</td><td>26.15</td><td>2.83</td><td>2.39</td><td>2.28</td><td>0.30</td><td><b>2.40</b></td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.01</td><td>-0.02</td><td>-0.00</td><td>-0.00</td><td>-0.00</td><td>-0.00</td><td>-0.01</td><td>-0.01</td><td>-0.05</td><td>-0.00</td><td>-0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>1.32</td><td>0.00</td><td>0.00</td><td>14.57</td><td>0.00</td><td>-0.00</td><td>0.00</td><td>-0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>27.85</b></td><td><b>7.65</b></td><td><b>53.93</b></td><td><b>87.29</b></td><td><b>39.05</b></td><td><b>33.14</b></td><td><b>55.55</b></td><td><b>3.53</b></td><td><b>7.84</b></td><td><b>4.02</b></td><td><b>12.19</b></td><td><b>2.11</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ Multi-hop portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>-0.37</td><td>-1.54</td><td>-1.26</td><td>-0.16</td><td>-0.19</td><td>-0.01</td><td>-0.27</td><td>-0.25</td><td>-0.38</td><td>0.03</td><td>0.01</td><td>-0.02</td>
  </tr>
  <tr>
    <td>FT-M</td><td>0.12</td><td>-1.54</td><td>-0.34</td><td>1.65</td><td>0.72</td><td>0.70</td><td>0.79</td><td>-0.25</td><td>-0.34</td><td>-0.04</td><td>-0.01</td><td>-0.01</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.50</td><td>-1.54</td><td>-1.51</td><td>-0.60</td><td>-0.37</td><td>-0.18</td><td>-0.52</td><td>-0.26</td><td>-0.38</td><td>-0.05</td><td>-0.03</td><td>-0.02</td>
  </tr>
  <tr>
    <td>GRACE</td><td>-0.00</td><td>-0.02</td><td>0.01</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.01</td><td>-0.01</td><td>-0.00</td><td>-0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>1.27</b></td><td><b>0.53</b></td><td><b>2.55</b></td><td><b>2.94</b></td><td><b>2.29</b></td><td><b>1.68</b></td><td><b>1.86</b></td><td><b>0.65</b></td><td><b>0.63</b></td><td><b>-0.00</b></td><td><b>0.57</b></td><td><b>0.21</b></td>
  </tr>
  <tr>
    <th colspan="13">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Belebele</th>
  </tr>
  <tr>
    <td>Original</td><td>73.59</td><td>73.22</td><td>77.11</td><td>88.67</td><td>82.78</td><td>74.00</td><td>81.0</td><td>77.67</td><td>52.33</td><td>43.78</td><td>-</td><td>85.33</td>
  </tr>
  <tr>
    <td>FT-L</td><td>72.92</td><td><b>74.33</b></td><td><b>77.22</b></td><td>86.89</td><td>82.56</td><td>73.00</td><td>80.44</td><td>76.33</td><td>52.22</td><td>42.00</td><td>-</td><td>84.22</td>
  </tr>
  <tr>
    <td>FT-M</td><td>60.26</td><td>57.33</td><td>64.11</td><td>70.78</td><td>69.56</td><td>56.78</td><td>66.89</td><td>60.67</td><td>45.11</td><td>35.00</td><td>-</td><td>76.33</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>22.58</td><td>25.11</td><td>19.33</td><td>20.78</td><td>24.00</td><td>23.89</td><td>23.00</td><td>22.33</td><td>21.22</td><td>24.89</td><td>-</td><td>21.22</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>73.50</b></td><td>73.22</td><td>77.11</td><td><b>88.67</b></td><td><b>83.00</b></td><td><b>73.33</b></td><td><b>80.67</b></td><td><b>77.67</b></td><td><b>52.44</b></td><td><b>43.78</b></td><td>-</td><td>85.11</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td>73.39</td><td>73.67</td><td>76.78</td><td>88.56</td><td>82.89</td><td>73.22</td><td>80.11</td><td>77.44</td><td>52.22</td><td>43.78</td><td>-</td><td><b>85.22</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ XQuAD</th>
  </tr>
  <tr>
    <td>Original</td><td>29.60</td><td>28.91</td><td>34.71</td><td>32.44</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>22.35</td>
  </tr>
  <tr>
    <td>FT-L</td><td>19.35</td><td>11.01</td><td>18.49</td><td>28.49</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>19.41</td>
  </tr>
  <tr>
    <td>FT-M</td><td>0.36</td><td>0.00</td><td>0.42</td><td>0.17</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.84</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>29.71</b></td><td><b>28.99</b></td><td><b>34.54</b></td><td><b>33.03</b></td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td><b>22.27</b></td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td>29.18</td><td>28.74</td><td>34.12</td><td>31.85</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>22.02</td>
  </tr>
</table>

Table 16: Results detailed by language for sequential editing performed in English on the full BABELEdits test set with Llama 3.1 8B.
<table>
  <tr>
    <th>Method</th>
    <th>AVG</th>
    <th>AR</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>HR</th>
    <th>IT</th>
    <th>JA</th>
    <th>KA</th>
    <th>MY</th>
    <th>QU</th>
    <th>ZH</th>
  </tr>
  <tr>
    <th colspan="13">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Reliability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>3.02</td><td>0.76</td><td>3.24</td><td>18.73</td><td>3.06</td><td>0.72</td><td>5.64</td><td>0.29</td><td>0.11</td><td>-0.02</td><td>0.17</td><td>0.55</td>
  </tr>
  <tr>
    <td>FT-M</td><td>9.79</td><td>1.53</td><td>13.59</td><td>62.49</td><td>8.82</td><td>4.04</td><td>13.57</td><td>1.23</td><td>0.78</td><td>-0.02</td><td>0.06</td><td>1.55</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.01</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.00</td><td>-0.00</td><td>0.00</td><td>0.09</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>17.10</td><td>4.80</td><td>23.50</td><td><b>98.55</b></td><td>16.77</td><td>10.85</td><td>22.88</td><td>3.98</td><td>2.47</td><td>0.00</td><td>0.09</td><td>4.25</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>36.51</b></td><td><b>20.90</b></td><td><b>63.04</b></td><td><b>98.49</b></td><td><b>49.86</b></td><td><b>40.92</b></td><td><b>64.38</b></td><td><b>19.44</b></td><td><b>18.38</b></td><td><b>2.17</b></td><td><b>10.81</b></td><td><b>13.19</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ Generality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>2.37</td><td>0.69</td><td>2.58</td><td>13.83</td><td>2.67</td><td>0.79</td><td>4.23</td><td>0.28</td><td>0.14</td><td>-0.03</td><td>0.23</td><td>0.61</td>
  </tr>
  <tr>
    <td>FT-M</td><td>5.76</td><td>1.00</td><td>10.08</td><td>29.53</td><td>6.57</td><td>3.66</td><td>8.91</td><td>1.07</td><td>0.45</td><td>-0.02</td><td>0.04</td><td>2.10</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.01</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.00</td><td>-0.00</td><td>0.00</td><td>0.14</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>9.36</td><td>3.56</td><td>18.58</td><td>30.52</td><td>14.00</td><td>9.07</td><td>17.18</td><td>3.35</td><td>2.23</td><td>0.00</td><td>0.00</td><td>4.47</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>34.25</b></td><td><b>20.90</b></td><td><b>59.17</b></td><td><b>93.69</b></td><td><b>47.64</b></td><td><b>35.17</b></td><td><b>61.30</b></td><td><b>18.66</b></td><td><b>13.30</b></td><td><b>2.25</b></td><td><b>11.61</b></td><td><b>13.00</b></td>
  </tr>
  <tr>
    <th colspan="13">↓ Locality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>11.23</td><td>12.29</td><td>13.27</td><td>15.65</td><td>13.86</td><td>13.65</td><td>14.33</td><td>5.52</td><td>14.13</td><td>1.41</td><td>11.84</td><td>7.61</td>
  </tr>
  <tr>
    <td>FT-M</td><td>6.75</td><td>8.90</td><td>7.31</td><td>8.69</td><td>7.64</td><td>7.76</td><td>8.53</td><td>4.88</td><td>6.61</td><td>0.94</td><td>6.94</td><td>6.07</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.45</td><td>0.27</td><td>0.22</td><td>0.07</td><td>0.14</td><td>0.65</td><td>0.18</td><td>0.69</td><td>1.08</td><td>0.86</td><td>0.35</td><td>0.49</td>
  </tr>
  <tr>
    <td>GRACE</td><td>0.09</td><td>0.05</td><td>0.22</td><td>0.34</td><td>0.08</td><td>0.04</td><td>0.18</td><td>0.01</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.04</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td>4.20</td><td>2.50</td><td>6.64</td><td>9.60</td><td>6.18</td><td>4.31</td><td>6.95</td><td>2.56</td><td>1.30</td><td>0.49</td><td>2.16</td><td>3.45</td>
  </tr>
  <tr>
    <th colspan="13">↑ Subject-alias portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>1.36</td><td>0.24</td><td>1.52</td><td>8.33</td><td>0.49</td><td>0.18</td><td>4.08</td><td>0.07</td><td>0.01</td><td>0.01</td><td>0.00</td><td>0.05</td>
  </tr>
  <tr>
    <td>FT-M</td><td>4.66</td><td>0.47</td><td>5.53</td><td>28.44</td><td>5.20</td><td>2.93</td><td>6.87</td><td>0.36</td><td>0.36</td><td>0.01</td><td>0.76</td><td>0.37</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.01</td><td>0.00</td><td>0.01</td><td>0.00</td><td>0.00</td><td>-0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>9.18</td><td>1.54</td><td>10.28</td><td>34.75</td><td>15.51</td><td>14.00</td><td>18.89</td><td>1.23</td><td>1.02</td><td>1.02</td><td>2.29</td><td>0.44</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>27.85</b></td><td><b>7.65</b></td><td><b>53.93</b></td><td><b>87.29</b></td><td><b>39.05</b></td><td><b>33.14</b></td><td><b>55.55</b></td><td><b>3.53</b></td><td><b>7.84</b></td><td><b>4.02</b></td><td><b>12.19</b></td><td><b>2.11</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ Multi-hop portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>0.05</td><td>0.05</td><td>0.06</td><td>0.22</td><td>0.11</td><td>0.01</td><td>0.08</td><td>0.07</td><td>0.01</td><td>-0.04</td><td>0.01</td><td>0.01</td>
  </tr>
  <tr>
    <td>FT-M</td><td>0.26</td><td>0.04</td><td>0.38</td><td>1.29</td><td>0.24</td><td>0.11</td><td>0.16</td><td>0.29</td><td>0.01</td><td>-0.04</td><td>0.16</td><td>0.17</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.01</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.00</td><td>0.00</td><td>-0.00</td><td>-0.01</td><td>-0.00</td><td>0.13</td><td>0.00</td><td>0.01</td>
  </tr>
  <tr>
    <td>GRACE</td><td>0.04</td><td>0.00</td><td>0.16</td><td>0.16</td><td>0.00</td><td>0.00</td><td>0.16</td><td>-0.00</td><td>-0.00</td><td>0.00</td><td>0.00</td><td>0.00</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>1.27</b></td><td><b>0.53</b></td><td><b>2.55</b></td><td><b>2.94</b></td><td><b>2.29</b></td><td><b>1.68</b></td><td><b>1.86</b></td><td><b>0.65</b></td><td><b>0.63</b></td><td><b>-0.00</b></td><td><b>0.57</b></td><td><b>0.21</b></td>
  </tr>
  <tr>
    <th colspan="13">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Belebele</th>
  </tr>
  <tr>
    <td>Original</td><td>84.68</td><td>85.78</td><td>88.00</td><td>93.22</td><td>90.67</td><td>86.67</td><td>89.67</td><td>85.11</td><td>74.89</td><td>64.00</td><td>-</td><td>88.78</td>
  </tr>
  <tr>
    <td>FT-L</td><td>84.64</td><td>85.89</td><td>88.11</td><td>93.78</td><td>90.67</td><td>86.67</td><td>89.67</td><td>84.89</td><td><b>75.33</b></td><td>63.11</td><td>-</td><td>88.33</td>
  </tr>
  <tr>
    <td>FT-M</td><td>84.40</td><td><b>86.00</b></td><td>87.56</td><td><b>93.89</b></td><td>90.67</td><td>86.11</td><td>89.56</td><td><b>85.33</b></td><td>75.11</td><td>61.33</td><td>-</td><td>88.44</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>28.62</td><td>24.33</td><td>25.22</td><td>52.56</td><td>30.22</td><td>22.78</td><td>28.89</td><td>23.56</td><td>22.67</td><td>22.89</td><td>-</td><td>33.11</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>84.71</b></td><td><b>86.00</b></td><td>88.11</td><td>93.22</td><td><b>90.78</b></td><td><b>86.78</b></td><td><b>89.78</b></td><td>85.11</td><td>75.11</td><td><b>63.44</b></td><td>-</td><td><b>88.78</b></td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>84.70</b></td><td><b>86.00</b></td><td><b>88.22</b></td><td>93.33</td><td>90.67</td><td><b>86.78</b></td><td><b>89.78</b></td><td>85.11</td><td>75.11</td><td>63.22</td><td>-</td><td><b>88.78</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ XQuAD</th>
  </tr>
  <tr>
    <td>Original</td><td>31.79</td><td>24.71</td><td>27.31</td><td>46.89</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>28.24</td>
  </tr>
  <tr>
    <td>FT-L</td><td>26.37</td><td>21.26</td><td>22.44</td><td>43.19</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>18.57</td>
  </tr>
  <tr>
    <td>FT-M</td><td>29.81</td><td>20.76</td><td>26.47</td><td>45.80</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>26.22</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>4.33</td><td>0.25</td><td>3.53</td><td>12.35</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>1.18</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>31.76</b></td><td><b>24.62</b></td><td><b>27.48</b></td><td>47.23</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td><b>27.73</b></td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>31.70</b></td><td><b>24.62</b></td><td><b>27.23</b></td><td><b>47.31</b></td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td><b>27.65</b></td>
  </tr>
</table>

Table 17: Results detailed by language for sequential editing performed in English on the full BABELEDITS test set with Gemma 2 9B.
<table>
  <tr>
    <th rowspan="2">Methods</th>
    <th colspan="5">Llama 3.1</th>
    <th colspan="5">Gemma 2</th>
  </tr>
  <tr>
    <th>AVG</th><th>DE</th><th>EN</th><th>FR</th><th>ZH</th>
    <th>AVG</th><th>DE</th><th>EN</th><th>FR</th><th>ZH</th>
  </tr>
  <tr>
    <th colspan="11">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="11">↑ Reliability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>1.50</td><td>0.94</td><td>3.90</td><td>0.78</td><td>0.39</td><td>2.54</td><td>1.23</td><td>7.25</td><td>1.34</td><td>0.34</td>
  </tr>
  <tr>
    <td>FT-M</td><td>27.53</td><td>23.09</td><td>64.76</td><td>17.52</td><td>4.75</td><td>25.28</td><td>17.63</td><td>68.44</td><td>12.86</td><td>2.20</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.13</td><td>-0.20</td><td>-0.24</td><td>-0.05</td><td>-0.02</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>25.17</td><td>0.78</td><td><b>99.51</b></td><td>0.40</td><td>0.00</td><td>30.13</td><td>13.10</td><td><b>98.95</b></td><td>7.11</td><td>1.34</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>45.24</b></td><td><b>44.31</b></td><td>97.23</td><td><b>34.25</b></td><td><b>5.17</b></td><td><b>47.74</b></td><td><b>48.89</b></td><td>97.95</td><td><b>37.45</b></td><td><b>6.68</b></td>
  </tr>
  <tr>
    <th colspan="11">↑ Generality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>1.13</td><td>0.75</td><td>2.65</td><td>0.72</td><td>0.40</td><td>1.89</td><td>1.20</td><td>5.02</td><td>1.07</td><td>0.28</td>
  </tr>
  <tr>
    <td>FT-M</td><td>23.23</td><td>20.66</td><td>51.87</td><td>16.01</td><td>4.39</td><td>17.31</td><td>15.23</td><td>41.99</td><td>10.17</td><td>1.84</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>-0.11</td><td>-0.24</td><td>-0.14</td><td>-0.04</td><td>-0.02</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-0.00</td><td>-0.00</td>
  </tr>
  <tr>
    <td>GRACE</td><td>2.57</td><td>0.53</td><td>9.50</td><td>0.26</td><td>0.00</td><td>12.33</td><td>9.09</td><td>35.21</td><td>4.53</td><td>0.51</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>42.13</b></td><td><b>42.57</b></td><td><b>89.87</b></td><td><b>31.09</b></td><td><b>4.99</b></td><td><b>44.00</b></td><td><b>46.55</b></td><td><b>88.80</b></td><td><b>34.15</b></td><td><b>6.48</b></td>
  </tr>
  <tr>
    <th colspan="11">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="11">↑ Belebele</th>
  </tr>
  <tr>
    <td>Original</td><td>83.47</td><td>77.11</td><td>88.67</td><td>82.78</td><td>85.33</td><td>90.17</td><td>88.00</td><td>93.22</td><td>90.67</td><td>88.78</td>
  </tr>
  <tr>
    <td>FT-L</td><td>83.22</td><td><b>77.00</b></td><td>88.44</td><td>82.56</td><td>84.89</td><td>90.06</td><td><b>88.11</b></td><td><b>93.44</b></td><td>90.56</td><td>88.11</td>
  </tr>
  <tr>
    <td>FT-M</td><td>71.53</td><td>63.33</td><td>80.78</td><td>68.33</td><td>73.67</td><td>90.03</td><td>87.78</td><td>93.33</td><td>90.67</td><td>88.33</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>22.89</td><td>22.89</td><td>22.89</td><td>22.89</td><td>22.89</td><td>25.83</td><td>24.56</td><td>27.11</td><td>26.22</td><td>25.44</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>83.36</b></td><td>76.78</td><td><b>88.67</b></td><td><b>82.78</b></td><td><b>85.22</b></td><td><b>90.22</b></td><td><b>88.11</b></td><td>93.22</td><td><b>90.78</b></td><td><b>88.78</b></td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>83.31</b></td><td><b>77.00</b></td><td>88.56</td><td>82.44</td><td><b>85.22</b></td><td>90.03</td><td>87.78</td><td>93.11</td><td>90.56</td><td>88.67</td>
  </tr>
  <tr>
    <th colspan="11">↑ XQuAD</th>
  </tr>
  <tr>
    <td>Original</td><td>29.83</td><td>34.71</td><td>32.44</td><td>-</td><td>22.35</td><td>34.15</td><td>27.31</td><td>46.89</td><td>-</td><td>28.24</td>
  </tr>
  <tr>
    <td>FT-L</td><td>4.90</td><td>1.26</td><td>9.08</td><td>-</td><td>4.37</td><td>33.81</td><td>28.07</td><td>44.87</td><td>-</td><td><b>28.49</b></td>
  </tr>
  <tr>
    <td>FT-M</td><td>5.97</td><td>4.29</td><td>7.56</td><td>-</td><td>6.05</td><td><b>34.71</b></td><td><b>31.43</b></td><td>47.06</td><td>-</td><td>25.63</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.00</td><td>0.00</td><td>0.00</td><td>-</td><td>0.00</td><td>3.31</td><td>0.67</td><td>8.99</td><td>-</td><td>0.25</td>
  </tr>
  <tr>
    <td>GRACE</td><td><b>29.94</b></td><td><b>34.54</b></td><td><b>33.03</b></td><td>-</td><td><b>22.27</b></td><td>34.15</td><td>27.48</td><td><b>47.23</b></td><td>-</td><td>27.73</td>
  </tr>
  <tr>
    <td><b>BABELReFT</b></td><td><b>29.75</b></td><td><b>34.45</b></td><td>32.52</td><td>-</td><td><b>22.27</b></td><td>34.12</td><td>27.31</td><td>46.81</td><td>-</td><td>28.24</td>
  </tr>
</table>

Table 18: Comparison of knowledge editing methods across Llama 3.1 and Gemma 2 models for sequential editing done in English on the entire MzsRE test set (742 edits), showing both editing performance and downstream task evaluation. Editing metrics are multiplied by 100 for readability. Bold numbers indicate the best performance for each metric and model combination. We report results for the languages in the intersection of those in MzsRE and in our evaluation set: Original indicates the model downstream performance before editing.
<table>
  <tr>
    <th rowspan="2">Edits</th>
    <th colspan="4">Llama 3.1</th>
    <th colspan="4">Gemma 2</th>
  </tr>
  <tr>
    <th>LoReFT</th>
    <th>NoReFT</th>
    <th>BABELReFT-SL</th>
    <th>BABELReFT</th>
    <th>LoReFT</th>
    <th>NoReFT</th>
    <th>BABELReFT-SL</th>
    <th>BABELReFT</th>
  </tr>
  <tr>
    <th colspan="9">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="9">↑ Reliability</th>
  </tr>
  <tr>
    <td>100</td><td>0.44</td><td>0.31</td><td>31.55</td><td><b>37.12</b></td><td>0.45</td><td>0.44</td><td>37.00</td><td><b>42.30</b></td>
  </tr>
  <tr>
    <td>250</td><td>-0.02</td><td>0.34</td><td>31.69</td><td><b>35.89</b></td><td>0.03</td><td>0.23</td><td>35.58</td><td><b>41.48</b></td>
  </tr>
  <tr>
    <td>500</td><td>0.02</td><td>0.26</td><td>33.27</td><td><b>37.48</b></td><td>0.16</td><td>0.43</td><td>36.05</td><td><b>40.90</b></td>
  </tr>
  <tr>
    <td>Full</td><td>-0.03</td><td>0.05</td><td>32.13</td><td><b>36.51</b></td><td>0.01</td><td>0.09</td><td>35.50</td><td><b>40.72</b></td>
  </tr>
  <tr>
    <th colspan="9">↑ Generality</th>
  </tr>
  <tr>
    <td>100</td><td>0.44</td><td>0.44</td><td>30.30</td><td><b>34.40</b></td><td>0.45</td><td>0.43</td><td>35.07</td><td><b>39.52</b></td>
  </tr>
  <tr>
    <td>250</td><td>-0.02</td><td>0.30</td><td>30.68</td><td><b>33.74</b></td><td>0.03</td><td>0.10</td><td>33.94</td><td><b>38.46</b></td>
  </tr>
  <tr>
    <td>500</td><td>0.02</td><td>0.16</td><td>31.79</td><td><b>35.15</b></td><td>0.15</td><td>0.38</td><td>34.69</td><td><b>38.48</b></td>
  </tr>
  <tr>
    <td>Full</td><td>-0.04</td><td>0.02</td><td>30.84</td><td><b>34.25</b></td><td>0.01</td><td>0.08</td><td>34.00</td><td><b>38.18</b></td>
  </tr>
  <tr>
    <th colspan="9">↓ Locality</th>
  </tr>
  <tr>
    <td>100</td><td>11.35</td><td>8.41</td><td><b>3.96</b></td><td>4.41</td><td>14.99</td><td>27.24</td><td><b>5.08</b></td><td>5.90</td>
  </tr>
  <tr>
    <td>250</td><td>8.29</td><td>7.81</td><td><b>3.71</b></td><td>4.12</td><td>8.96</td><td>31.69</td><td><b>5.20</b></td><td>5.91</td>
  </tr>
  <tr>
    <td>500</td><td>19.82</td><td>10.99</td><td><b>3.80</b></td><td>4.03</td><td>17.27</td><td>32.05</td><td><b>5.42</b></td><td>6.15</td>
  </tr>
  <tr>
    <td>Full</td><td>8.97</td><td>8.68</td><td><b>3.76</b></td><td>4.20</td><td>9.35</td><td>18.89</td><td><b>5.48</b></td><td>6.37</td>
  </tr>
  <tr>
    <th colspan="9">↑ Subject-Alias portability</th>
  </tr>
  <tr>
    <td>100</td><td>0.41</td><td>0.18</td><td>10.01</td><td><b>23.08</b></td><td>0.41</td><td>0.47</td><td>11.21</td><td><b>26.93</b></td>
  </tr>
  <tr>
    <td>250</td><td>-0.01</td><td>0.28</td><td>11.48</td><td><b>25.65</b></td><td>0.11</td><td>0.16</td><td>12.31</td><td><b>28.87</b></td>
  </tr>
  <tr>
    <td>500</td><td>-0.01</td><td>0.20</td><td>11.30</td><td><b>28.33</b></td><td>0.46</td><td>0.68</td><td>13.20</td><td><b>27.95</b></td>
  </tr>
  <tr>
    <td>Full</td><td>0.01</td><td>0.25</td><td>11.96</td><td><b>27.85</b></td><td>0.05</td><td>0.22</td><td>13.54</td><td><b>28.81</b></td>
  </tr>
  <tr>
    <th colspan="9">↑ Multi-Hop portability</th>
  </tr>
  <tr>
    <td>100</td><td>-0.01</td><td>-0.02</td><td>0.89</td><td><b>1.00</b></td><td>0.15</td><td>0.11</td><td>1.39</td><td><b>1.83</b></td>
  </tr>
  <tr>
    <td>250</td><td>-0.46</td><td>-0.40</td><td>0.73</td><td><b>0.84</b></td><td>0.00</td><td>-0.01</td><td>0.90</td><td><b>1.32</b></td>
  </tr>
  <tr>
    <td>500</td><td>-0.38</td><td>-0.38</td><td>0.43</td><td><b>0.55</b></td><td>-0.01</td><td>0.01</td><td>0.90</td><td><b>0.95</b></td>
  </tr>
  <tr>
    <td>Full</td><td>-0.32</td><td>-0.32</td><td>0.94</td><td><b>1.27</b></td><td>0.01</td><td>-0.01</td><td>1.22</td><td><b>1.64</b></td>
  </tr>
  <tr>
    <th colspan="9">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="9">↑ Belebele (accuracy)</th>
  </tr>
  <tr>
    <td>Original</td><td>73.59</td><td>73.59</td><td>73.59</td><td>73.59</td><td>84.68</td><td>84.68</td><td>84.68</td><td>84.68</td>
  </tr>
  <tr>
    <td>100</td><td>45.96</td><td>26.49</td><td>73.49</td><td><b>73.50</b></td><td>72.86</td><td>25.94</td><td><b>84.61</b></td><td>84.59</td>
  </tr>
  <tr>
    <td>250</td><td>33.57</td><td>25.43</td><td><b>73.49</b></td><td>73.47</td><td>76.12</td><td>27.32</td><td><b>84.61</b></td><td>84.60</td>
  </tr>
  <tr>
    <td>500</td><td>49.03</td><td>27.86</td><td>73.48</td><td><b>73.50</b></td><td>57.70</td><td>25.89</td><td>84.61</td><td><b>84.70</b></td>
  </tr>
  <tr>
    <td>Full</td><td>53.52</td><td>26.46</td><td><b>73.49</b></td><td>73.39</td><td>38.79</td><td>22.43</td><td>84.63</td><td><b>84.70</b></td>
  </tr>
  <tr>
    <th colspan="9">↑ XQuAD (EM)</th>
  </tr>
  <tr>
    <td>Original</td><td>29.60</td><td>29.60</td><td>29.60</td><td>29.60</td><td>31.79</td><td>31.79</td><td>31.79</td><td>31.79</td>
  </tr>
  <tr>
    <td>100</td><td>3.74</td><td>1.66</td><td><b>29.60</b></td><td>29.60</td><td>18.26</td><td>0.00</td><td><b>31.79</b></td><td>31.76</td>
  </tr>
  <tr>
    <td>250</td><td>8.66</td><td>0.34</td><td>29.58</td><td><b>29.68</b></td><td>35.25</td><td>0.00</td><td><b>31.79</b></td><td>31.72</td>
  </tr>
  <tr>
    <td>500</td><td>2.50</td><td>0.02</td><td>29.43</td><td><b>29.45</b></td><td>7.54</td><td>0.00</td><td><b>31.72</b></td><td>31.64</td>
  </tr>
  <tr>
    <td>Full</td><td>17.86</td><td>0.00</td><td><b>29.41</b></td><td>29.18</td><td>12.71</td><td>0.00</td><td><b>31.74</b></td><td>31.70</td>
  </tr>
</table>

Table 19: Comprehensive ablation study comparing BABELReFT with different unrestricted (i.e., always active) ReFT variants (LoReFT, NoReFT) and source-language-gated BABELReFT-SL. Cross-lingual knowledge editing and downstream task performance on Llama 3.1 8B and Gemma 2 9B models with sequential editing done in English with an increasing number of sequential edits. Editing metrics are averaged over all target languages and multiplied by 100 for readability (except for locality). Bold numbers indicate the best performance for each metric and model combination. Downstream performance is averaged over the target languages: Original indicates the model performance before editing.
<table>
  <tr>
    <th>Method</th>
    <th>AVG</th>
    <th>AR</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>HR</th>
    <th>IT</th>
    <th>JA</th>
    <th>KA</th>
    <th>MY</th>
    <th>QU</th>
    <th>ZH</th>
  </tr>
  <tr>
    <th colspan="13">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Reliability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>4.26</td><td>1.84</td><td>6.71</td><td>7.70</td><td>5.82</td><td>5.95</td><td>7.78</td><td>1.93</td><td>0.85</td><td>0.31</td><td>5.43</td><td>2.48</td>
  </tr>
  <tr>
    <td>FT-M</td><td>28.87</td><td>25.46</td><td>39.29</td><td>37.10</td><td>35.98</td><td>34.98</td><td>39.74</td><td>25.61</td><td>20.33</td><td>8.57</td><td>24.22</td><td>26.32</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>16.24</td><td>10.78</td><td>24.00</td><td>29.76</td><td>22.26</td><td>17.41</td><td>24.29</td><td>11.10</td><td>2.64</td><td>5.31</td><td>14.96</td><td>16.13</td>
  </tr>
  <tr>
    <td>GRACE</td><td>32.58</td><td>19.55</td><td>45.33</td><td>46.19</td><td>40.65</td><td>41.69</td><td>46.73</td><td>19.19</td><td>18.66</td><td>14.57</td><td>41.37</td><td>24.42</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>30.96</td><td>27.02</td><td>43.65</td><td>37.48</td><td>39.02</td><td>39.18</td><td>41.54</td><td>24.46</td><td>22.10</td><td>9.41</td><td>29.01</td><td>27.64</td>
  </tr>
  <tr>
    <th colspan="13">↑ Generality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>4.38</td><td>1.93</td><td>7.05</td><td>8.35</td><td>5.96</td><td>5.91</td><td>8.06</td><td>1.99</td><td>0.72</td><td>0.34</td><td>5.36</td><td>2.47</td>
  </tr>
  <tr>
    <td>FT-M</td><td>27.81</td><td>24.61</td><td>38.07</td><td>35.42</td><td>34.53</td><td>33.48</td><td>38.61</td><td>24.94</td><td>19.23</td><td>8.24</td><td>23.38</td><td>25.36</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>15.67</td><td>10.48</td><td>23.50</td><td>28.57</td><td>21.69</td><td>16.04</td><td>24.70</td><td>10.40</td><td>2.68</td><td>5.07</td><td>14.14</td><td>15.10</td>
  </tr>
  <tr>
    <td>GRACE</td><td>32.20</td><td>19.16</td><td>44.92</td><td>45.63</td><td>40.21</td><td>41.37</td><td>46.27</td><td>19.06</td><td>18.25</td><td>14.49</td><td>40.97</td><td>23.93</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>28.45</td><td>22.55</td><td>41.59</td><td>35.04</td><td>37.30</td><td>35.89</td><td>40.18</td><td>22.55</td><td>17.64</td><td>8.46</td><td>26.50</td><td>25.26</td>
  </tr>
  <tr>
    <th colspan="13">↓ Locality</th>
  </tr>
  <tr>
    <td>FT-L</td><td>2.80</td><td>2.69</td><td>3.53</td><td>2.94</td><td>3.22</td><td>3.47</td><td>3.68</td><td>3.01</td><td>2.30</td><td>0.77</td><td>2.37</td><td>2.78</td>
  </tr>
  <tr>
    <td>FT-M</td><td>2.79</td><td>3.04</td><td>3.56</td><td>2.86</td><td>3.39</td><td>3.19</td><td>3.58</td><td>3.02</td><td>1.97</td><td>0.69</td><td>2.48</td><td>2.88</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>2.91</td><td>2.04</td><td>3.61</td><td>4.16</td><td>3.95</td><td>2.32</td><td>4.10</td><td>2.49</td><td>1.26</td><td>1.90</td><td>2.68</td><td>3.49</td>
  </tr>
  <tr>
    <td>GRACE</td><td>6.62</td><td>6.72</td><td>6.56</td><td>3.01</td><td>5.56</td><td>7.40</td><td>6.43</td><td>7.77</td><td>5.91</td><td>8.45</td><td>9.85</td><td>5.21</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>3.70</td><td>2.53</td><td>5.00</td><td>4.20</td><td>4.77</td><td>4.38</td><td>4.87</td><td>2.69</td><td>1.96</td><td>1.57</td><td>5.04</td><td>3.68</td>
  </tr>
  <tr>
    <th colspan="13">↑ Subject-Alias portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>3.91</td><td>2.34</td><td>6.16</td><td>7.48</td><td>5.85</td><td>5.25</td><td>7.32</td><td>1.78</td><td>0.69</td><td>0.21</td><td>3.60</td><td>2.29</td>
  </tr>
  <tr>
    <td>FT-M</td><td>24.24</td><td>21.19</td><td>34.76</td><td>34.16</td><td>33.04</td><td>30.96</td><td>36.47</td><td>20.03</td><td>16.74</td><td>0.52</td><td>19.00</td><td>19.80</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>10.30</td><td>8.00</td><td>15.97</td><td>20.53</td><td>15.54</td><td>11.39</td><td>18.03</td><td>6.20</td><td>1.54</td><td>2.07</td><td>8.21</td><td>5.79</td>
  </tr>
  <tr>
    <td>GRACE</td><td>31.96</td><td>17.11</td><td>45.57</td><td>48.40</td><td>43.28</td><td>42.87</td><td>47.74</td><td>17.08</td><td>17.12</td><td>9.48</td><td>42.01</td><td>20.87</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>19.16</td><td>12.56</td><td>32.25</td><td>27.42</td><td>29.25</td><td>25.50</td><td>30.15</td><td>11.52</td><td>10.60</td><td>0.55</td><td>19.81</td><td>11.15</td>
  </tr>
  <tr>
    <th colspan="13">↑ Multi-hop portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>-0.14</td><td>-0.30</td><td>-0.00</td><td>0.09</td><td>-0.06</td><td>0.03</td><td>-0.01</td><td>-0.26</td><td>-0.39</td><td>-0.21</td><td>-0.07</td><td>-0.35</td>
  </tr>
  <tr>
    <td>FT-M</td><td>0.67</td><td>0.61</td><td>0.91</td><td>0.85</td><td>0.72</td><td>0.54</td><td>0.81</td><td>0.70</td><td>0.43</td><td>0.81</td><td>0.44</td><td>0.50</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.30</td><td>0.10</td><td>0.34</td><td>0.51</td><td>0.52</td><td>0.50</td><td>0.51</td><td>0.21</td><td>-0.14</td><td>0.19</td><td>0.47</td><td>0.08</td>
  </tr>
  <tr>
    <td>GRACE</td><td>0.37</td><td>-0.12</td><td>0.66</td><td>0.65</td><td>0.50</td><td>0.43</td><td>0.58</td><td>0.06</td><td>-0.02</td><td>0.83</td><td>0.55</td><td>-0.08</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>1.16</td><td>0.98</td><td>1.31</td><td>1.25</td><td>1.46</td><td>1.04</td><td>1.42</td><td>1.02</td><td>1.11</td><td>1.72</td><td>0.55</td><td>0.88</td>
  </tr>
  <tr>
    <th colspan="13">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="13">↓ Delta PPL</th>
  </tr>
  <tr>
    <td>FT-L</td><td>59.25</td><td>15.80</td><td>2.11</td><td>0.89</td><td>1.44</td><td>6.57</td><td>2.02</td><td>1.82</td><td>2.68</td><td>2.13</td><td>6.12e2</td><td>3.48</td>
  </tr>
  <tr>
    <td>FT-M</td><td>79.52</td><td>84.01</td><td>40.24</td><td>3.37</td><td>16.05</td><td>26.93</td><td>36.05</td><td>15.15</td><td>11.95</td><td>58.20</td><td>5.77e2</td><td>5.42</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>4.02e4</td><td>26.00</td><td>10.89</td><td>6.00</td><td>10.17</td><td>34.31</td><td>15.50</td><td>18.19</td><td>1.29e3</td><td>1.93e2</td><td>2.80e3</td><td>13.30</td>
  </tr>
  <tr>
    <td>GRACE</td><td>5.46e4</td><td>3.11e4</td><td>5.09e4</td><td>8.65e4</td><td>3.61e4</td><td>5.29e4</td><td>5.12e4</td><td>1.91e4</td><td>2.75e4</td><td>1.33e4</td><td>1.29e5</td><td>1.02e5</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.02</td><td>0.00</td>
  </tr>
</table>

Table 20: Comprehensive comparison of cross-lingual knowledge editing in the single edit setup and perplexity variation for Llama 3.1 8B. Each column (except AVG) corresponds to an editing language, and the results are averaged across all the target languages. Column AVG averages those results. Values are percentages except for perplexity and locality, where they are absolute values, and bold numbers indicate best performance for each metric and editing language.
<table>
  <tr>
    <th>Method</th>
    <th>AVG</th>
    <th>AR</th>
    <th>DE</th>
    <th>EN</th>
    <th>FR</th>
    <th>HR</th>
    <th>IT</th>
    <th>JA</th>
    <th>KA</th>
    <th>MY</th>
    <th>QU</th>
    <th>ZH</th>
  </tr>
  <tr>
    <th colspan="13">Cross-lingual Knowledge Editing Performance</th>
  </tr>
  <tr>
    <th colspan="13">↑ Reliability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>4.19</td><td>2.07</td><td>8.75</td><td>9.13</td><td>7.38</td><td>4.28</td><td>6.42</td><td>1.70</td><td>0.29</td><td>1.21</td><td>2.75</td><td>2.09</td>
  </tr>
  <tr>
    <td>FT-M</td><td>24.37</td><td>17.79</td><td>33.76</td><td>36.12</td><td>31.67</td><td>27.30</td><td>33.47</td><td>17.55</td><td>15.64</td><td>8.79</td><td>24.61</td><td>21.42</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>6.80</td><td>3.99</td><td>10.85</td><td>13.90</td><td>9.50</td><td>8.93</td><td>11.34</td><td>2.86</td><td>0.50</td><td>0.96</td><td>7.82</td><td>4.18</td>
  </tr>
  <tr>
    <td>GRACE</td><td>24.88</td><td>17.97</td><td>33.89</td><td>37.86</td><td>31.54</td><td>28.33</td><td>33.48</td><td>18.13</td><td>16.50</td><td>9.10</td><td>24.73</td><td>22.14</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td><b>30.45</b></td><td><b>23.88</b></td><td><b>43.11</b></td><td><b>42.10</b></td><td><b>39.34</b></td><td><b>37.74</b></td><td><b>40.84</b></td><td><b>20.52</b></td><td><b>21.84</b></td><td><b>9.21</b></td><td><b>34.82</b></td><td><b>21.49</b></td>
  </tr>
  <tr>
    <th colspan="13">↑ Generality</th>
  </tr>
  <tr>
    <td>FFT-L</td><td>3.99</td><td>1.92</td><td>8.08</td><td>9.05</td><td>6.75</td><td>3.88</td><td>5.93</td><td>1.75</td><td>0.21</td><td>1.59</td><td>2.96</td><td>1.83</td>
  </tr>
  <tr>
    <td>FT-M</td><td>22.36</td><td>15.75</td><td>31.30</td><td>32.85</td><td>28.97</td><td>24.92</td><td>30.85</td><td>16.29</td><td>13.61</td><td>8.53</td><td>23.65</td><td>19.22</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>6.49</td><td>3.63</td><td>10.29</td><td>13.08</td><td>9.07</td><td>8.57</td><td>10.94</td><td>2.79</td><td>0.45</td><td>0.94</td><td>7.71</td><td>3.88</td>
  </tr>
  <tr>
    <td>GRACE</td><td>23.08</td><td>16.21</td><td>31.64</td><td>35.18</td><td>29.06</td><td>25.90</td><td>30.99</td><td>17.11</td><td>14.73</td><td><b>8.97</b></td><td>24.02</td><td><b>20.12</b></td>
  </tr>
  <tr>
    <td>BABELReFT</td><td><b>27.68</b></td><td><b>19.64</b></td><td><b>40.48</b></td><td><b>39.24</b></td><td><b>36.81</b></td><td><b>34.30</b></td><td><b>38.89</b></td><td><b>18.26</b></td><td><b>17.43</b></td><td>8.17</td><td><b>32.62</b></td><td>18.68</td>
  </tr>
  <tr>
    <th colspan="13">↓ Locality</th>
  </tr>
  <tr>
    <td>FT-L</td><td><b>2.10</b></td><td><b>2.24</b></td><td><b>2.33</b></td><td><b>2.13</b></td><td><b>2.39</b></td><td><b>2.29</b></td><td><b>2.29</b></td><td><b>2.46</b></td><td><b>2.51</b></td><td><b>0.39</b></td><td><b>1.86</b></td><td><b>2.23</b></td>
  </tr>
  <tr>
    <td>FT-M</td><td>3.30</td><td>2.71</td><td>3.29</td><td>3.56</td><td>3.87</td><td>3.09</td><td>3.50</td><td>4.06</td><td>2.91</td><td>0.82</td><td>4.38</td><td>4.08</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>3.28</td><td>2.49</td><td>3.81</td><td>4.27</td><td>3.56</td><td>3.58</td><td>3.26</td><td>3.43</td><td>2.81</td><td>1.43</td><td>3.37</td><td>4.10</td>
  </tr>
  <tr>
    <td>GRACE</td><td>3.07</td><td>2.59</td><td>3.09</td><td>3.51</td><td>3.64</td><td>2.34</td><td>3.38</td><td>3.41</td><td>3.37</td><td>0.40</td><td>3.95</td><td>4.12</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td>5.04</td><td>2.79</td><td>6.24</td><td>7.05</td><td>6.13</td><td>5.62</td><td>5.96</td><td>3.43</td><td>4.46</td><td>1.01</td><td>8.67</td><td>4.05</td>
  </tr>
  <tr>
    <th colspan="13">↑ Subject-Alias portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>2.66</td><td>1.36</td><td>5.50</td><td>6.84</td><td>4.96</td><td>2.98</td><td>4.01</td><td>1.06</td><td>0.16</td><td>0.02</td><td>1.36</td><td>0.97</td>
  </tr>
  <tr>
    <td>FT-M</td><td>17.92</td><td><b>12.81</b></td><td>25.08</td><td>28.33</td><td>26.30</td><td>20.32</td><td>26.56</td><td>12.02</td><td>12.01</td><td>0.53</td><td>20.10</td><td>13.00</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>6.02</td><td>3.28</td><td>10.47</td><td>13.06</td><td>7.05</td><td>7.79</td><td>10.34</td><td>3.05</td><td>0.80</td><td><b>0.62</b></td><td>5.97</td><td>3.75</td>
  </tr>
  <tr>
    <td>GRACE</td><td>18.01</td><td>12.69</td><td>24.65</td><td>27.09</td><td>25.32</td><td>21.12</td><td>24.94</td><td><b>13.28</b></td><td><b>12.65</b></td><td>0.39</td><td>21.96</td><td><b>14.06</b></td>
  </tr>
  <tr>
    <td>BABELReFT</td><td><b>18.34</b></td><td>10.96</td><td><b>28.34</b></td><td><b>30.14</b></td><td><b>28.30</b></td><td><b>24.28</b></td><td><b>29.21</b></td><td>8.79</td><td>9.62</td><td>0.26</td><td><b>25.02</b></td><td>6.83</td>
  </tr>
  <tr>
    <th colspan="13">↑ Multi-hop portability</th>
  </tr>
  <tr>
    <td>FT-L</td><td>0.44</td><td>0.18</td><td>0.80</td><td>0.58</td><td>0.67</td><td>0.57</td><td>0.51</td><td>0.19</td><td>0.12</td><td>0.82</td><td>0.22</td><td>0.18</td>
  </tr>
  <tr>
    <td>FT-M</td><td>0.86</td><td>0.57</td><td>0.98</td><td><b>1.15</b></td><td>0.96</td><td>0.73</td><td>1.00</td><td>0.68</td><td>0.45</td><td>1.50</td><td>0.85</td><td><b>0.57</b></td>
  </tr>
  <tr>
    <td>r-ROME</td><td>0.25</td><td>0.13</td><td>0.37</td><td>0.38</td><td>0.32</td><td>0.23</td><td>0.36</td><td>0.15</td><td>0.13</td><td>0.19</td><td>0.31</td><td>0.14</td>
  </tr>
  <tr>
    <td>GRACE</td><td>0.64</td><td>0.43</td><td>0.75</td><td>0.90</td><td>0.80</td><td>0.66</td><td>0.80</td><td>0.61</td><td>0.38</td><td>0.62</td><td>0.59</td><td>0.44</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td><b>1.07</b></td><td><b>0.66</b></td><td><b>1.22</b></td><td>1.12</td><td><b>1.27</b></td><td><b>1.01</b></td><td><b>1.03</b></td><td><b>0.90</b></td><td><b>0.63</b></td><td><b>2.47</b></td><td><b>0.88</b></td><td>0.55</td>
  </tr>
  <tr>
    <th colspan="13">Downstream Performance</th>
  </tr>
  <tr>
    <th colspan="13">↓ Delta PPL</th>
  </tr>
  <tr>
    <td>FT-L</td><td>12.69</td><td>-6.37</td><td>-19.02</td><td>-14.01</td><td>-17.13</td><td>-16.17</td><td>-18.60</td><td>4.66</td><td>23.14</td><td>30.77</td><td>171.04</td><td>1.34</td>
  </tr>
  <tr>
    <td>FT-M</td><td>1.38e2</td><td>48.18</td><td>52.00</td><td>34.94</td><td>29.17</td><td>69.28</td><td>61.19</td><td>9.10</td><td>31.36</td><td>65.98</td><td>1.09e3</td><td>19.72</td>
  </tr>
  <tr>
    <td>r-ROME</td><td>1.65e8</td><td>4.96e6</td><td>1.94e7</td><td>8.57e6</td><td>1.85e7</td><td>1.72e8</td><td>1.03e7</td><td>1.37e7</td><td>4.37e5</td><td>6.36e5</td><td>1.57e9</td><td>2.82e6</td>
  </tr>
  <tr>
    <td>GRACE</td><td>8.20e2</td><td>9.22e2</td><td>4.66e2</td><td>6.80e2</td><td>5.31e2</td><td>6.39e2</td><td>5.12e2</td><td>4.93e2</td><td>5.94e2</td><td>4.92e2</td><td>2.80e3</td><td>8.83e2</td>
  </tr>
  <tr>
    <td>BABELReFT</td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td><td><b>0.00</b></td>
  </tr>
</table>

Table 21: Comprehensive comparison of cross-lingual knowledge editing in the single edit setup and perplexity variation for Gemma 2 9B. Each column (except AVG) corresponds to an editing language, and the results are averaged across all the target languages. Column AVG averages those results. Values are percentages except for perplexity and locality, where they are absolute values, and bold numbers indicate best performance for each metric and editing language.
<table>
  <tr>
    <th>Lang</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
  </tr>
  <tr><td>AF</td><td>1021</td><td>18</td><td>1</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>AR</td><td>622</td><td>360</td><td>51</td><td>8</td><td>1</td><td>0</td><td>0</td></tr>
  <tr><td>AZ</td><td>769</td><td>244</td><td>28</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>BE</td><td>675</td><td>339</td><td>26</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>BG</td><td>550</td><td>468</td><td>24</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>BN</td><td>795</td><td>238</td><td>9</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>CA</td><td>980</td><td>58</td><td>2</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>CS</td><td>990</td><td>50</td><td>0</td><td>1</td><td>1</td><td>0</td><td>0</td></tr>
  <tr><td>DA</td><td>998</td><td>40</td><td>3</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>DE</td><td>985</td><td>50</td><td>5</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>EL</td><td>610</td><td>398</td><td>34</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>EN</td><td>837</td><td>140</td><td>41</td><td>18</td><td>5</td><td>1</td><td>0</td></tr>
  <tr><td>ES</td><td>956</td><td>80</td><td>5</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>ET</td><td>1002</td><td>39</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>EU</td><td>1008</td><td>33</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>FA</td><td>223</td><td>695</td><td>122</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>FI</td><td>906</td><td>82</td><td>37</td><td>14</td><td>2</td><td>0</td><td>1</td></tr>
  <tr><td>FR</td><td>963</td><td>66</td><td>11</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>GU</td><td>893</td><td>144</td><td>5</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HE</td><td>405</td><td>611</td><td>24</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HI</td><td>720</td><td>295</td><td>25</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HR</td><td>1001</td><td>40</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HT</td><td>1019</td><td>21</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HU</td><td>950</td><td>85</td><td>6</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>HY</td><td>535</td><td>448</td><td>59</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>ID</td><td>960</td><td>78</td><td>3</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>IT</td><td>984</td><td>52</td><td>5</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>JA</td><td>267</td><td>738</td><td>35</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>JV</td><td>1015</td><td>25</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>KA</td><td>655</td><td>365</td><td>21</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>KK</td><td>793</td><td>230</td><td>17</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>KO</td><td>367</td><td>605</td><td>68</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>LT</td><td>912</td><td>124</td><td>5</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>ML</td><td>947</td><td>93</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>MR</td><td>777</td><td>238</td><td>27</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>MS</td><td>988</td><td>51</td><td>2</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>MY</td><td>948</td><td>82</td><td>12</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>NL</td><td>976</td><td>57</td><td>7</td><td>1</td><td>1</td><td>0</td><td>0</td></tr>
  <tr><td>NO</td><td>1028</td><td>14</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>PA</td><td>838</td><td>188</td><td>16</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>PL</td><td>935</td><td>93</td><td>12</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>PT</td><td>965</td><td>70</td><td>6</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>QU</td><td>1014</td><td>26</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>RO</td><td>958</td><td>64</td><td>16</td><td>4</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>RU</td><td>373</td><td>542</td><td>126</td><td>0</td><td>1</td><td>0</td><td>0</td></tr>
  <tr><td>SK</td><td>974</td><td>59</td><td>7</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>SR</td><td>779</td><td>252</td><td>11</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>SV</td><td>992</td><td>46</td><td>3</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>SW</td><td>1018</td><td>22</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>TA</td><td>716</td><td>296</td><td>30</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>TE</td><td>822</td><td>200</td><td>20</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>TH</td><td>761</td><td>192</td><td>60</td><td>24</td><td>5</td><td>0</td><td>0</td></tr>
  <tr><td>TL</td><td>1014</td><td>27</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>TR</td><td>968</td><td>70</td><td>3</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>UK</td><td>420</td><td>568</td><td>52</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>UR</td><td>723</td><td>280</td><td>37</td><td>2</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>UZ</td><td>945</td><td>90</td><td>7</td><td>0</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>VI</td><td>965</td><td>71</td><td>5</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>YO</td><td>1024</td><td>16</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr>
  <tr><td>ZH</td><td>365</td><td>556</td><td>118</td><td>2</td><td>1</td><td>0</td><td>0</td></tr>
  <tr><th>Total</th><th>49599</th><th>11522</th><th>1258</th><th>122</th><th>17</th><th>1</th><th>1</th></tr>
</table>

(a) Proportion of subjects in the BABELEDITIONS test set that belong to a given domain. Note: some subjects may belong to multiple domains while others belong to none.

(b) Number of entities which have a given number of aliases (from 1 to 7) in each language in the BABELEDITIONS test set.

Figure 3: Statistics about the BABELEDITIONS test set.

Reviewer paper summaries: This paper introduces BABELEDITS, a novel dataset for evaluating cross-lingual knowledge editing (CKE) in large language models (LLMs), spanning 60 languages, and BABELREFT, a modular representation fine-tuning approach for robust and effective CKE. The work addresses two major limitations in prior CKE research: (1) inadequate evaluation due to poor entity translations and lack of alias consideration, and (2) the problem of model collapse when subjected to sequential edits. BABELEDITS, built using high-quality multilingual synsets from BabelNet and marker-based translation, ensures accurate entity translations and supports diverse evaluation metrics, including reliability, generality, locality, and alias portability. BABELREFT employs entity-specific representation fine-tuning modules to isolate knowledge updates for entities and their aliases, minimizing negative interference and preserving multilingual capabilities. Experimental results show that BABELREFT significantly outperforms SOTA baselines (e.g., GRACE, FT-M) in both editing effectiveness and robustness against model collapse, particularly in sequential editing scenarios. The study highlights key findings, such as the challenge of cross-lingual edit transfer and multi-hop reasoning, while advocating for the importance of robust CKE methods to ensure factual consistency and downstream task performance across languages.

This paper focuses on cross-lingual knowledge editing. The authors propose BabelEdits, a new cross-lingual knowledge editing dataset which involve 60 languages. Based on the proposed data, the authors further introduce a new cross-lingual knowledge editing method, i.e., BabelReFT.
Summary of strengths: * The motivation of this paper is well-grounded. It identifies clear limitations in current CKE approaches, such as translation errors, alias handling, and model robustness, which are important and underexplored challenges. The authors propose practical solutions to these challenges, including developing a novel CKE benchmark using the marker-based translation method to improve entity translation and a CKE method using entity-specific representation fine-tuning modules.

* The paper is clearly written and easy to follow, with well-structured sections and explicit descriptions of the challenges in CKE. Especially, This paper has a good related work summary, providing a thorough and well-structured review of existing knowledge editing (KE) and cross-lingual KE (CKE) methods. 

* The paper introduces BABELEDITS, the largest and most multilingual dataset for CKE to date, spanning 60 languages and providing a robust evaluation framework for cross-lingual KE methods. It provides a good description of benchmark building, particularly with a convincing argument for language selection and the use of high-quality resources like Wikipedia and BabelNet.

* The experiments are extensive and rigorous, including evaluations on both sequential and single edits. The results demonstrate the advantages of BABELREFT in both effectiveness (multiple evaluation metrics, e.g., reliability, generality, multi-hop portability) and robustness (handling negative inference and model collapse effects).

- The introduced data spans 60 languages (including low-resource languages like Burmese and Quechua), it is the largest cross-lingual knowledge editing benchmark to date.
- The paper is well-written and easy to follow
Summary of weaknesses: * Although the benchmark, BABELEDITS, spans 60 languages, the evaluation is conducted on only 10 selected languages, which limits the scope of the analysis. This subset, while diverse, does not fully represent the linguistic and typological diversity of the entire dataset. The quality assessment of the benchmark translations is also restricted to a few selected languages.

- While BABELEDITS includes 60 languages, the experimental evaluation is restricted to 10 languages. It could be understand due to limited resources, but the benchmark work should provide the full results making future research easy to follow.
- The paper focuses on direct entity edits (e.g., "Kanye West’s wife → Bianca Censori") but ignores implicit knowledge (e.g., "Bianca Censori is a musician") or multi-fact dependencies. This could lead to incomplete or inconsistent knowledge propagation.
- BABELREFT’s modular design requires storing entity-specific ReFT modules, which may become unwieldy for large-scale editing (e.g., updating thousands of entities). The paper does not address how this scales with model size (e.g., 7B vs. 70B parameters).
Comments, suggestions and typos: * The paper does not provide a detailed description of BabelNet, which is a key resource used for constructing the BABELEDITS benchmark. It could be easier for non-specialist readers to follow with more elaboration of BabelNet and its relationship with WordNet and Wikipedia.

see the above
Overall assessment scores: 4

3
Metareview score: 3.5
Metareview: This paper focuses on cross-lingual knowledge editing. The authors first contribute a new benchmark dataset (named BABELEDITS) for this task, and it involves 60 languages covering high- and low-resource languages. Besides, the authors propose a new cross-lingual knowledge editing method, i.e., BabelREFT, and demonstrate its effectiveness in experiments.
Summary of reasons to publish: - Cross-lingual knowledge editing is an interesting research topic, and is under-explored in the current community.
- The authors build a new benchmark dataset covering 60 languages, and it might be useful for the community.
- The motivation is clear, and the paper is well-written and well-organized.