Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performance (2025)

Abstract: Pretrained language models (PLMs) have revolutionized NLP but amplify linguistic inequities in multilingual applications. While prior studies focused on transformer architectures such as BERT, we evaluate large language models (LLMs) including Mistral, TowerInstruct, OpenHathi, Tamil-Llama, and Kan-Llama. Through rigorous testing across eight languages spanning high-resource (English, German, French, Italian, Spanish) and low-resource (Hindi, Tamil, Kannada) settings, we reveal systemic failures in preserving multilingual reliability and adaptability. Using paradigms like each language for itself’ (ELFI) and each language for others’ (ELFO), we highlight the inability of current LLMs to bridge linguistic divides. Even model merging fail to mitigate these gaps, exposing fundamental limitations. These findings emphasize the critical need for reimagining AI architectures to deliver true linguistic inclusivity and equitable performance across diverse languages.

## introduction handling multilinguality in language models remains a significant challenge, particularly when models are prompted in languages other than english. tasks such as question answering @xcite , addressing multilingual safety concerns @xcite , or performing knowledge edits @xcite often reveal noticeable gaps in performance for low-resource languages. despite the advancements in multilingual large language models (llms), disparities persist, especially for languages with fewer computational resources. a clear example of this issue arises in knowledge editing @xcite . for instance, when * these authors contributed equally to this work. an llm is updated to correct a factual statement, "the pm of the uk is rishi sunak" to "the pm of the uk is keir starmer" the model may apply the update accurately in well-represented languages like english or french @xcite . however, the same edit often fails to propagate when queried in low-resourced languages like tamil or hindi. this inconsistency highlights a critical weakness in the ability of llms to transfer factual updates across languages. even advanced models like mistral and towerinstruct, while effective in european languages, struggle significantly with low-resource languages. this limitation undermines the broader goal of making language technologies universally accessible and equitable @xcite . this research aims to uncover the disparities in cross-lingual performance of llms to promote future linguistic inclusivity. while model editing techniques have advanced in monolingual settings, ensuring that factual updates made in one language are accurately reflected across others remains a major challenge @xcite . this issue is particularly severe for low-resource languages, where models often fail to maintain reliability and consistency after edits. such limitations reduce the utility of llms for these languages and widen existing linguistic inequities, leaving many communities underserved. our work highlights these gaps, showing how current models struggle to manage multilingual updates, especially in underrepresented languages. by evaluating cross-lingual performance, we emphasize the need for more inclusive approaches to ensure that llms benefit users of all languages, not just those with abundant resources. in this work, we conduct a comprehensive evaluation of how factual knowledge is transferred and maintained across eight linguistically diverse languages. we examine established knowledge editing techniques such as rome @xcite and memit @xcite to assess their performance in multilingual contexts. our research utilizes two strategies @xcite -"each language for itself " (elfi) and "each language for others" (elfo)-to rigorously test the ability of llms to preserve cross-lingual knowledge consistency. through this evaluation, we reveal current models' limitations in maintaining consistent cross-lingual edits, emphasizing critical gaps to address for enhancing llms, particularly in low-resource languages. our key contributions are as follows. we conduct extensive model editing experiments across eight languages-english (en), german (de), french (fr), italian (it), spanish (es), hindi (hi), tamil (ta), and kannada (kn)-using elfi and elfo, focusing on decoder-only models' multilingual performance. we evaluate 7b decoder-only models, including mistral, towerinstruct, open-hathi, tamil-llama, and kan-llama, with editing methods rome and memit, advancing model editing research. this is the first of it's kind work on llm to reveal that model merging improves capabilities but struggles with cross-lingual consistency after editing. ## related work targeted parameter editing modifies specific model components to integrate new information. @xcite introduced adjustments to 'knowledge neurons' in transformers, while rome @xcite updated neural weights to refresh llm knowledge. memit @xcite expanded rome for simultaneous updates, with further validation by @xcite . multilingual knowledge editing remains limited, focusing mainly on translating english prompts. x-factr @xcite and m-lama @xcite exposed large knowledge gaps in non-english languages, often @xmath0. geomlama @xcite revealed that native languages may not best access national knowledge. we analyze cross-lingual consistency in multilingual llms, extending prior work mostly on bert (pre llm era) to diverse llms fine-tuned for specific languages @xcite . ## task overview model editing: given a language model θ pre and an edit @xmath1, a new , a old >, the model editing technique will create an edited model θ edit . so, for an input prompt kn, θ pre has the old prediction a old and after editing θ pre , the edited model θ edit has updated prediction a new without influencing model behaviour on other samples. thus, given the edit input kn, θ pre does not produce a new ; it is θ edit that is designed to produce the output a new . the scope of consideration, i(kn, a new ), includes kn and similar versions of it. this means it covers the original input and any rephrased versions of it that still relate to the same topic. for example, if kn is a question, this scope includes different ways of asking the same question. however, the excluded scope, o(kn, a new ), refers to inputs that are not related to the edit case provided. so, it leaves out any inputs that do not have anything to do with kn or its related versions. along with the updated information, the edited model should follow the four properties: (i) reliabilityθ edit , produces the correct response for the specific edit scenario represented by (kn, a new ), (ii) generalization -the edited model θ edit must uniformly apply edits to both the designated edit case (kn, a new ) and its semantically equivalent variations, guaranteeing a consistent output, a new , across all rephrased iterations of kn, (iii) localityθ edit should not alter the output for examples outside its intended scope (o(kn, a new )), and (iv) portability -evaluates the capacity of edited model θ edit for robust generalization, assessed through questions designed to test the edited model's reasoning with updated knowledge. multilingual knowledge editing: given a set of languages l, we consider a language @xmath2. we then test the edited model θ l edit with all the languages in l. in the equations below, s is the source language, and t is the target language. the conditions are as follows: if kn s is in the inclusion scope i(kn, a new ), the model should output a s new . otherwise, if kn s is in the exclusion scope o(kn, a new ), the model should output θ pre (kn s ). for the target language, similar conditions apply with transformations t t . t t (.) transforms the target output of the source language to the target language with the same meaning. therefore, after editing the model in one language, such as english, the effect of the edit should be reflected in other languages as well. this ensures that the specific edit is consistent across all languages, regardless of the language in which the edit was made. ## dataset for our experiments, we use the popular counter-fact @xcite and zsre @xcite datasets. we uniformly sample ∼ 550 edit instances from each dataset. each edit instance in these datasets includes the actual edit case, the reliability prompt, the generalization instances, the locality prompt and its answer, portability and its answer. further we use google translator foot_0 to translate each edit instance into seven other languages -german (de), french (fr), italian (it), spanish (es), hindi (hi), tamil (ta) and kannada (kn). in both the datasets, the actual portability prompt is an interrogative sentence (i.e., in the form of question). however, when the question gets translated to other languages, the translated question becomes different from actual question format. for example, when the actual portability prompt in english "to which language family does the official language of sastamala belong?" is translated to french the new prompt becomes "à quelle langue la famille appartient la langue officielle de sastamala?". however when this is back-translated to english the prompt means "which family language does the official language of sastamala belong to?" which is not the same as the original english prompt. we therefore employed gpt-4 foot_1 to convert question in the interrogative sentence into a task of sentence completion. subsequently we translate this sentence completion form to other languages to obtain the corresponding portability prompt. note to the choice of languages: the western languages that we choose are based on their cultural, economic and academic significance @xcite foot_2 and cover the romance and the germanic families. in addition, we include three indic languages that have far lesser resources compared to their western counterparts. ## error analysis in ## discussion here we discuss two important questions -how do multilingual llms handle cross-lingual knowledge edits? and what steps can industry practitioners take to address cross-lingual disparities? how do multilingual llms handle cross-lingual knowledge edits? modern llms often fail to propagate factual updates consistently across languages. while languages like english, french, and german benefit from extensive corpora @xcite , those like hindi, tamil, and kannada suffer from data scarcity, causing unstable knowledge transfer @xcite . further, editing methods rome and memit encounter problems with highly agglutinative or morphologically rich languages. ## conclusion in this study, we investigated the impact of knowledge editing across different languages based on the counterfact and zsre datasets along with their translations. our extensive experiments employing a variety of knowledge editing techniques on an array of multilingual llms resulted in various crucial observations. we discovered that variations in language-specific model architecture significantly affect the success of knowledge edits, that current editing methods often fail to seamlessly transfer alterations from one language to another, and that modifications made in one language might unexpectedly alter model behavior in another language. this study lays the groundwork for future innovations that could lead to more sophisticated and linguistically inclusive ai technologies.