CollabEdit: Towards Non-destructive Collaborative Knowledge Editing
Authors: Jiamu Zheng
Jinghuai Zhang
Tianyu Du
Xuhong Zhang
Jianwei Yin
Tao Lin
Venue: ICLR 2025 Poster (2025)
Keywords: Collaborative Learning
Knowledge Editing
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Collaborative learning of large language models (LLMs) has emerged as a
new paradigm for utilizing private data from different parties to guarantee
efﬁciency and privacy. Meanwhile, Knowledge Editing (KE) for LLMs has also
garnered increased attention due to its ability to manipulate the behaviors of
LLMs explicitly, yet leaves the collaborative KE case—in which knowledge
edits of multiple parties are aggregated in a privacy-preserving and continual
manner—unexamined. To this end, this manuscript dives into the ﬁrst investigation
 of collaborative KE, in which we start by carefully identifying the unique
three challenges therein, including knowledge overlap, knowledge conﬂict, and
knowledge forgetting. We then propose a non-destructive collaborative KE
framework, COLLABEDIT, which employs a novel model merging mechanism
to mimic the global KE behavior while preventing the severe performance drop.
Extensive experiments on two canonical datasets demonstrate the superiority of
COLLABEDIT compared to other destructive baselines, and results shed light on
addressing three collaborative KE challenges and future applications. Our code is
available at [https://github.com/LINs-lab/CollabEdit](https://github.com/LINs-lab/CollabEdit).

Published as a conference paper at ICLR 2025
COLLABEDIT: TOWARDS NON-DESTRUCTIVE COL- LABORATIVE KNOWLEDGE EDITING
Jiamu Zheng ! Jinghuai Zhang 3 Tianyu Du 1. Xuhong Zhang'
Jianwei Yin University of California, Los Angeles 3
Tao Lin"
Zhejiang University Westlake University 2
chengjaamie @ gmail. com zhangxuhong ju.. edu. cn
jinghuai19988g www.9/8.........i.....ooo. ucla.edu zjradty@zju. edu, en zjuyjwecs wjw.
Ainta
ABSTRACT
Collaborative learning of large language models (LLMs) has emerged as a new paradigm for utilizing private data from different parties to guarantee efficiency and privacy. Meanwhile, Knowledge Editing (KE) for LLMs has also garnered increased attention due to its ability to manipulate the behaviors of LLMs explicitly, yet leaves the collaborative KE case-in which knowledge edits of multiple parties are aggregated in a privacy-preserving and continual manner-une unexamined. To this end, this manuscript dives into the first investiga- tion of collaborative KE, in which we start by carefully identifying the unique three challenges therein, including knowledge overlap, knowledge conffict, and knowledge forgetting. We then propose a non-destructive collaborative KE framework, COLLAREDIT, which employs novel model merging mechanism to mimic the global KE behavior while preventing the severe performance drop. Extensive experiments on two canonical datasets demonstrate the superiority of COLLABEDIT compared to other destructive baselines, and results shed light on addressing three collaborative KE challenges and future applications. Our code is available at https//ggihubbccom//IiNNbAlACCl
INTRODUCTION
Large Language Models (LLMs) (Achiam al., 2023; Qiao 90
et al., 2023) recently have emerged as the promising solu-
80
tion toward general artificial intelligence. However, deploy- ing LLMs in practice usually requires customizing LLMs with
70
Globul-edit
H-mean of (ES,
specific knowledge (Meng et al., 2022), where re-training LLMs may be expensive and unacceptable (Jang et al., 2023).
60
Ariheeii
50
Simple- Average
Accordingly, Knowledge Editing (KE) (Meng et al., 2022;
40
Mitchell et al., 2022a; Tan et al., 2024; Zhang et al., 2023),
30
which allows efficient modification of knowledge stored in ex- isting models, has been proposed as alternative solution.
50
100
500 1000 5000 Number of edits
200
To explicitly update LLMs with knowledge from multiple possesses distinct and private dataset (Ye et al., 2024; Wu et al., 2022; McMahan al., 2017)- and meet individual demands, current KE methods (Meng et al., 2022; 2023) first need collect edit requests from these parties with violated privacy concerns: the edit request itself contains sensitive private information and thus becomes infeasible for sharing. It motivates resorting to the cross-silo collaborative learning paradigm (Wu soppdaseddmodlss, models, rather than
Figure 1: Limits of existing KE methods under the collaborative KE scenarios on the Multi-CounterFact dataset (Meng et al., 2022).
parties or organizations-caal
et al., 2023; Kairouz et al.. b021)-by only communicating the uploading a list of risky edit prpuests-namdee collaborative KE for LLMs.
However, existing KE methods are all designed for the single-party single-model scenario (Meng al., 2022; Mitchell et al., 2022a; Tan et al., 2024; Meng et al., 2023). Noting that model merg- ing (MM) techniques (Ortiz-Jimenez et al., 2023; Chronopoulou et al., 2023; Yaday et al., 2023) allow a straightforward extension of KE methods to collaborative KE scenario. Therefore as our
Work was done during Jiamu's visit to Westlake University.
Corresponding author.

---

Published as a conference paper at ICLR 2025
Edit Requst lie for difett
Serve (Uodndi-mokk)
Sireleen
Cliara (Scb-mnnca
Server (Dulsiimmdd)
Neald deg
KF
M
Tatt Berpart lia
Undazal (isba- Madel's wrigh (Idral)
New president Ohana Bidre.
Nendestrodive
bits Roqaott
Letts het for chere
M
New Faet: Jack Mi ficanded Alinao. Toll.
Q: Who nondded Google
Prgg
KF
Ma
Model's
New Ract Airthds cnsoad by Applic Google.
clian
Destrantve-husnn Midel'ss weign (Boaeline) gg
Qq: WeCled, creatal by 2. Teseest.
M
(a) Glohal Editing
(b) Collaborative Editing
Figure 2: Comparison of global KE (GLOBAL--EDIT) and collaborative KF
contribiont,,,ttinnn, we examine a naive combination of local KE and MM techniques, and compare them with the optimal global KE method (GL.OBA--EDIT) we can witness that all these naive col- laborative KE methods are destructive. In detail, we conduct independent KE (i.e., MEMIT (Meng et al., 2023)) on each party locally, and then use model merging techniques like Simple-Average (Chronopoulou et al., 2023) and Arisherrieemeetic (Ilharco et al., 2023) to merge local models (LLMs) into a global model (LLM). We find that as the number of edits increases, the performance gap between naive baselines with the optimal GLOBAL-EEDIT also widens.
Alongside the pitfalls of naive collaborative KE methods, in this manuscript, we carefully examine the intervention issues among different parties and identify the knowledge overlap and knowledge conflict challenges. These challenges arise from the global server's blind access the edit requests of each party. In addition, the continual KE requirement for each party inherently results in the interventions among different rounds of editing, corresponding knowledge forgetting challenge.
To bridge the gap and provide a deeper understanding of collaborative KE, we first analyze the performance drop between naive collaborative KE methods and GLOBAL-EDIT from a theoretical perspective, upon which we design novel framework COLLABEDIT that allows non-destructive collaborative KF We further pioneer the explorations on the interventions associated with collab- orative KE (namely the three challenges we identified) and design tailored solutions to effectively address them. Our contributions can be summarized as follows:
We are the first propose the collaborative KE paradigm (including naive collaborative KE base- lines, GLOBAL-EDi -EDIT, and our COLLABEDIT), in which we summarize the unique interventions associated with collaborative KE and conclude them with three challenges in this novel paradigm. We identify the performance gap between the naive collaborative KE method and the upper bound performance (i.e., GLOOBAL-EDIT) through theoretical and empirical analysis.
To the best of our knowledge, we propose the first non-destructive collaborative KE framework: it is versatile, allowing nondestructive integration of cxisting KE methods and providing insights into the solution of each challenge.
Our empirical results demonstrate the effectiveness of our proposed framework compared with baselines and that of the novel solutions to three challenges based our COLLABEDIT. Our discussions shed light on future research for collaborative KE.
2 RELATED WORK
Knowledge Editing (KE). KE (De Cao et al., 2021; Mitchell et al., 2022a) has received significant attention due the increasing demands for efficient updating of LLMs. Hypernetwork knowledge editing and direct model editing are the two most representative KE methods. Given edit requests, hypernetwork knowledge editing (Mitchell et al., 2022a;b; Tan et al., 2024) leverages trained hypernetwork to predict the model updates, while direct model editing (Meng et al., 2023; 2022; De Cao et al., 2021) updates LLMs as an associative memory and inserts new memory via solving an optimization problem. However, these KE methods are all designed for single LLM, which limits their applications to a more practical collaborative learning scenario. In this paper, we place emphasis on the SoTA frameworks of two KE methods types stated above, which are respectively MALMEN (Tan et al., 2024) and MEMIT (Meng et al., 2023), to explore the integration of KE and collaborative learning.
2

---

Published as a conference paper at ICLR 2025
Collaborative learning and model merging. Collaborative learning (Kairouz et al., 2021; Wang et al., 2021; Fan et al., 2024; Mohtashami et al., 2023) allows multiple parties to jointly and contin- uously learn a machine learning model by sharing their updates to global server for aggregation. Alongside the orthogonal techniques to address data heterogeneity issue (Karimireddy et al., 2020; Li et al., 2019), model aggregation/merging (Li al., 2023b; Wortsman al., 2022; Ortiz-Jimenez et al., 2023; Yadav et al., 2023) has emerged as a promising research direction to collaborative learning, which employs the global server to directly merge model updates in the weight space without disclosing the training data of each party. The most commonly used model merging tech- niques themselves are Sinple-Average (SA) (Chronopoulou al., 2023; Wortsman et al.. 2022) and Task-Arithmetic (TA) (Ortiz-Jimenez et al., 2023). Moreover, TIES- TIES-merging (Yadav et al., 2023) has recently proposed to further enhance the merging performance by solving the symbol conflicts among different models. However, all existing model merging techniques only achieve destructive editing performance when used for collaborative KE, which inevitably results in knowledge loss during the merging process.
PRELIMINARIES OF COLLABORATIVE KNOWLEDGE EDITING
We first introduce the basics of KE in a single LLM. Then, we illustrate the naive approaches to conduct collaborative KE. Finally, we describe the inherent interventions within collaborative KE
3.1 INTRODUCTION TO KNOWLEDGE EDITING IN A SINGLE LLM
LLMs can answer natural-langea language queries about facts based on implicit knowledge encoded within the parameters. Following Meng et al. (Meng et al., 2023), we define a fact f as "(subject relation r, object o)", e.g., "(s = Danielle Darrieux, r spoke the language,
= French)". Given } to edit (denoted as edit requexts), knowledge editing aims to maximize the likelihood that the updated LLM MM predicts the desired object oR for any factual prompt x θ p(s., r(), which involves a prefix x and templated prompt p(s.,
sequence of facts & {f (st, ri, o;)}
arg min Me E E Ex log Pr.a [or] x @ p(si, r:)]]-
The state-of-the-arr knowledge editing methods (Meng et al., 2022; 2023; Tan et al.. 2024) found that modifying a small sequence of MLP layers in the critical path of LLM is sufficient to edit its factual associations. In particular, linear operation W in an MLP layer operate values M where input keys correspond to the
key-v
store for input keys K' and the memory/knowledde intermediate feature vector of the model from set of edit requests. Knowledge editing modifies each MLP layer such that it associates K' to the desired M' by solving W ~ M'. For brevity, we will describe knowledge editing for a specific layer and omit / throughout the paper.
Given set of facts & to edit (i.e., edit requests), we first obtain their input keys K = ki...., kjsj] to the layer via a single forward. We also obtain the desired memory values M = [m....., m of layer 1 that maximize Pr [o,|x ① p(s;, rt)]. The goal of editing the layer can be formulated as optimizing the A such that the updated weight W + associates the input keys K to the desired memory values M. Note that the MIP layer also contains previously stored memories of existing knowledge, which should be preserved during the knowledge editing. Therefore, we also maintain the associations between input keys of existing knowledge Kinit and their memory values (WKinit) Following MEMIT (Meng et al., 2023), we derive the closed form of A for a specific layer as:
(2) where C == Kinn init is the covariance matrix of the input keys of existing knowledge, and R. = M-WK represents the residual error in the output space of layer See more details in Appendix A.
A = RK T (C KKT)
3.2 DESTRUCTIVE MODEL MERGING ENCOUNTERS KNOWLEDGE EDITING
KE in practice involves editing the factual associations of LLM, such as correcting the hallucinations or updating outdated information. This process often requires handling simultaneous edit requests, where multiple parties or clients access and collaboratively contribute to the same LLM service. Though Global KE (GLOBAL-EDII illustrated in Figure 2(a) represents the ideal editing cases, also necessitates each client to directly share the edit requests with the server, which violates the privacy constraints. Collaborative KE in Figure 2(b) instead allows each client edit its
3

---

Published as a conference paper at ICLR 2025
local model and only rely on the server to aggregate the edit updates using the model merging techniques (Wortsman et al., 2022; Ortiz-Jimenez et al., 2023; Yadav et al., 2023).
However, existing KE algorithms are all designed for single client and cannot be trivially general- ized to the collaborative KE scenario. As evidenced in Figure I, naively extending existing editing methods or model merging methods yields a dramatic performance drop compared to that of the GLOBAL-EDIT (upper bound), especially when the number of edits increases. Given the limits of de- structive collaborative KE methods, we aim to develop a destructee collaborative KE method that can achieve similar editing performance as GLOBAL-EDDIT, even with large number of edits.
3.3 INTERVENTIONS WITHIN COLLABORATIVE KNOWLEDGE EDITING
In addition to the performance drop, our proposed concept of collaborative KE also suffers from several key challenges, due to the unique characteristics of this scenario. By default, we assume trustworthy and non-adversarial collaborative KE scenario. The collaborative KE employs a global server aggregate the edits of local clients without disclosing their edit requests, while requiring each client to continually edit the global model by updating its local model in a multi-round manner. However, there still exist several unique challenges due to the interventions among different clients and different rounds of editing, warranting research in the future.
3.3.1 INTERVENTIONS AMONG DIFFERENT CLIENTS
In collaborative KE, multiple clients may use similar edit requests to update their local models and send the updated models to the global server for aggregation. The interventions among clients then raised in the editing event e == (#1, r1, 01 - 02, 41, m1), which includes 81 as a subject, rn as relationship, 01 and 02 as objects, 4 as editing timestamp and m, as client model.
Knowledge conflict indicates that edit requests from the same/different clients (in the same round') ) share the same subject 8 and relation r but with different objects o. Such a conflict renders the effectiveness of knowledge editing and may even compromise the overall KE performance. We elaborate the general formulation of conflict edit below (detailed illustration can be found in the Table 7 of Appendix):
c1 = (s1, (8,, + 02,t1,m1)
E2 = (51, 71, T1, 01 03, (2,
where local model my and m2 perform a conflicting editing for the same subject sj and relationship T1 at timestamp & and f2 respectively, changing the same original object 0₁ to different 02 and 0g.
Similar to the composite edit operations mentioned by Li et al. (2024), composite conflict and composite overlap arising from such operations may also occur in collaborative KE scenarios, with even more diverse and complex forms. Here we aim to briefly introduce the key concept of knowledge conflict, and more detailed definition and investigation of this issue left for future work.
Knowledge overlap is simplified case of knowledge conflict, where the object changing relation- ship (i.e. 0, - 02 and 0 + 0g) in editing events of e, and 62 becomes identical. Knowledge overlap is also closely related to the overfitting problem in machine learning, in which excessive overlapped editing requests can degrade the model's editing performance on other edit requests (ex- cluding those repeated edit requests).
3.3.2 INTERVENTIONS AMONG DIFFERENT ROUNDS OF EDITING
The collaborative KE paradigm naturally requires multiple clients to continually update their local models in a multi-round manner and thus edit the global model with the latest knowledge. Knowl- edge forgetting issue, therefore, arises given the continual arrival of large number of new editing requests, alongside the existing knowledge and editing requests.
Assume that each client has a set of old edit requests &., as well as m sets of new edit requests E, = [En Eng."'. ... ). where the new edit requests are irrelevant (i.e., their subjects s and relationships r are different) to the old edit requests. The model is initialized by updating the model with the old edit requests E.. and the local model of each client will be updated with the new edit requests En, at i-th round of editing, followed by the model aggregation step. The knowledge
In cases of conflict between edits from different rounds, due to the overwriting nature of KE, the latter conflicting edit will overwrite the former, naturally resolving the conflict.
4

---

Published as a conference paper at ICLR 2025
forgetting issue encountered after rounds of local editing and global aggregation can then be defined as the editing performance on the old knowledge obtained from the old edit requests Ey- In particular, we find that as the value of m increases, the evaluation performance of the model on old knowledge & deteriorates, as evidenced in Section 5.3.
4 METHODOLOGY
4.1 COLLABEDIT: NON-DESTRUCTIVE COLLABORATIVS KE
To better understand the performance drop, we first explicitly model the relationship between the weight updates Ag of the global model using GLOBAL-EDIT and that of each client model A; using local editing. For ease of presentation, we consider the collaborative KE scenario with N clients and each client model has M edit requests. We simplify the theoretical analysis the single-round editing case and demonstrate the effectiveness of COLLABEDIT for multi- editing in Remark 2. Lemma 1 (The relationship between the weight updates from GLOBAL-EDIT and local editing). Take the KE method MEMIT as example. Following the definitions in Section 3.1, we denote C an aggregated statistic over the previously stored keys of existing knowledge and use K to represent the new keys derived from client i's edit. Then, the relationship between AG and A is measured as:
A E== A. (a: (C+==C+KKKK (C EN K,KK)-))
(4)
See detailed proof in Appendix B.
Intuition: If we can estimate AG using A;, then we can merge {Ai) J. to obtain the same global model as GLOBAL-EDIT and, therefore, obtain non-destructive collaborative KE
Indeed AG can be represented as the weighted sum of different local weight updates A; with coefficient og However, the coefficient a; relies on the value of K, of all the clients: it breaks the privacy, given the fact that K; an intermediate feature vector of the model from a set of edit requests and any external party can easily reconstruct the edit requests if K; is leaked. As a remedy, our COLLABEDIT instead proposes to directly communicate K,K,". in which prove in Section 6 that K,K, is non-trivial to attack. See our pseudo-code in Appendix D.
Details of COLLABEDIT:
Remark 1. Currently, we consider two mainstream KE methods (Akyürek et al.. 2023), namely (I) locate and edit activations (same "Direct model editing" mentioned in Section 2, e.g., MEMIT (Meng et al., 2023) and ROME (Meng el al., 2022)); and (2) train an auxiliary model to di- rectly predict parameters (same "Hypernetwork knowledge editing" mentioned in Section e.g., MEND (Mitchell et al., 2022a) and MALMEN (Tan et al., 2024)). Our framework COLLABEDIT is general enough integrate many other KE methods, and we leave them for future work.
Justifying the performance drop for destructive editing approaches. We further analyze the performance degradation for destructive editing approaches when the number of edits increases, as illustrated in Figure 1. For the sake of simplicity, we take the TASK- ARITHAETTCETICC (Ilharco et al., 2023) with MEMIT example. The drop can be explained by:
A A' - MM A, [[C+KKK)(C) Σ -K])---M
(5)
where Ag and A'6 represent the weight updates derived from COLLABEDIT (our non-destructive collaborative KE) and a destructive collaborative KE using MASK-ARITIMETIC, respectively. We can see that the impact of new knowledge K,K," is negligible compared existing knowledge C when the number of edits small?, resulting in (C + Enn K,K,K, C and thus AG R A'G when
The gap becomes wider when the number of edits increases, contributing to the continuous decline in TASK-ARE ARITHMEITTE' performance in Figure 1 compared to GLOBAL-EDIT.
Remark 2 (COLLABEDIT is effective for multi-round editing). Collaborative KE involves mul- tiple clients continuously editing the local models and sharing the updated global model across multiple rounds, and thus requires robust support to ensure seamless knowledge integration and consistent knowledge memorization. COLLABEDIT achieves non non-destructive collaborative KE for single-rous round editing- as an approximation of aggregating all edit requests of clients in a specific round and applying global KE to update the global model-remuins effective for multi-round editing. Note that multi-round editing is equivalent to applying global KE to iteratively update a single L.M multiple times under the reasonable editing budgets (Gupta et al., 2024).
"W randomly sample 100 edit requests to estimate the norm of K2 K.K.T We observe that the average norm of K, K." is approximately 0. 0.0001 % of that of C, which supports the claim.
5

---

Published as a conference paper at ICLR 2025
4.2 REMEDY TOWARDS SOLVING INTERVENTION CHALLENGES COLLABORATIVE KNOWLEDGE EDITING: SOME CASE STUDIES
Interventions within collaborative KE scenarios are non-trivial, due to the challenges of explicitly modeling the impacts of editing requests from different clients. Our COLLABEDIT paves the path by mimicking the optimal GLOBAL-EDIT and allowing the non-destructive editing. This subsec- tion case studies how our COLLABEDIT sheds insights solving unique challenges caused by the interventions among different clients (spatial aspect) and editing rounds (temporal aspect), namely knowledge overlap, knowledge conflict, and knowledge forgetting.
4.2.1 EDITING RESIDUAL DETECTS KNOWLEDGE OVERLAP
COLLABEDIT simplifies the knowledge overlap challenge collaborative KE scenarios into the over-fitting problem under the global KE scenarios. In other words, multiple clients edit the same piece of knowledge is equivalent to integrating several identical pieces of knowledge into the global model. In detail: performing KE in the model results in weights update A and residual Rold. as determined by the input key K. In the case of editing the same knowledge (i.e., same K), we can get new residual Reew
Rold
AK, where the following equation can be leveraged to track the
dynamics of KE:
R Reew
Rold
AK
Roold Rodd (C + KK (KT)-KK
(6)
Intuitively, (6) explains that the residual should gradually approach 0 I. If the residual R gradually approaches zero, then we can accurately detect the knowledge overlap by examining the residual R. as demonstrated in Section 5.3.
4.2.2 ADDRESSING KNOWLEDGE CONFLICT VIA DATA AUGMENTAION
Recall that in rare cases, edit requests from the same/different clients in the same round may share the same subject s and relation r but with different objects o, known as knowledge conflict. An ideal solution to the knowledge conftict should consist of two stages. In the first stage, the global server and clients need to collaboratively detect the conffict in a privacy-preserving manner. For example, when the knowledge conflict occurs, the global server produces poor editing performance on some edit requests. As a result, the clients (who contribute to the edits) could report the issue.
Once the conflict is identified, the server will determine which of the conflicting edit requests to retain for the global model based on the client's report and a predefined strategy (c.g.. FCFS (Zhao & Stankovic, 1989) or FIFO (Morse & Richardson, 1983) strategy). The client whose edit request is selected for integration can apply data augmentation techniques, such incorporating relevant knowledge (Li et al., 2024), to enhance the KE the selected edit request and effectively resolve the knowledge conflict.
4.2.3 DYNAMIC COVARIANCE MATRIX ALLEVIATES KNOWLEDGE FORGETTING
The previously memorized knowledge may be forgotten by the LLM after large number of edits, termed as knowledge forgetting issue. COLLABEDIT simplifies the analysis of this issue and we can witness from (2) that the covariance matrix C of existing knowledge is immutable, amplifying the forgetting as the number of edits increases. As a remedy, we propose using a dynamic version of C, i.c.,
C = B. + 3 C1 Co 3: ΣK..
where By and 31 are hyper-parameters that balance the influences of existing knowledge and newly acquired knowledge. Co is the covariance matrix of existing knowledge and C₁ the accumulated covariance matrix of new knowledge. K, represents the input keys obtained from all the edit requests the i-th round. The dynamic covariance matrix continuously updated for the new knowledge can effectively mitigate the knowledge forgetting issue, as verified in Section 5.3.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets and models. Following the literature (Meng al., 2022; 2023), Culti-CounterFact (MCF) (Meng et al., 2022) and zsRE (Levy et al., 2017) 77 as datasets and evaluate the editing perfor- mance GPT2-XI. (Radford et al., 2019) and GPT-J (6B) (Wang & Komatsuzaki, 2021).

---

Published as a conference paper at ICLR 2025
Table 1: Overall editing performance GPT2-XL. GLOBAL-EDIT is 5000 L, which means edit 5000 requests one model (global model) at one time. GLOBAL-EDIT is an ideal situation. Others are merging meth- ods (500 x 10) where we edit 10 models and each model will be edited by 500 requests. The line of GPT2-XL means we directly evaluate 5000 requests without any editing operation to test the model's 's original performance. The "Score" serves the overall metric for assessing the performance of each method on each dataset.
MCF
zsRE
Method
NS 1
PS
ES
Score t
t
EA
Score
GPT2-XI.
78.24
23.88
21.50
29.65
24.32
21.87
22.80
22.95
GLOBAL-EDIT
65.08
80.66
89.66
77.08
25.25
64.71
68.96
43.12
TIES- TIES-MERCING
78.46
26.35
27.16
34.27
24.94
25,99
27.59
26.12
TASK- ARITHMEITT
66.84
55.19
61.66
60.85
24.97
33.66
34,80
30.45
SIMPLE-AVERAGE
76.90
29.97
33.06
39.15
25.78
29.26
30.62
28.40
COLLABEDIT
65.26
80.67
89.70
77.18
25.21
64,27
68.40
42,95
Table 2: Overall editing performance GPT-J (6B), based MEMIT (Meng al., 2023). The experimental setting is identical GPT2-XL in Table 1. The "Score" serves as the overall metrie.
MCF
zsRE
Method
NS 1
PS f
ES
Score t
NA 1
PA
EA
Score
GPT-J
83,45
17.17
14.78
21.75
26.99
26.25
27.04
26.75
GLOBAL-EDIT
57.20
96.13
99.26
79.03
28.05
88.79
92.05
51.92
MERGGIG
76.15
30.13
30.98
38.16
30.17
42.55
43.55
37.68
TASK-ARITMETTIC
50.24
72.82
73.26
63.44
18.77
45.16
46.75
30.98
SIMPLE-AVERAGE
78.04
41.28
54.68
54.22
29.19
47.96
51.38
40.22
COLLABEDIT
57.12
96.03
99.06
78.91
28.26
88.78
92.19
52.17
Baselines. We compare COLLABEDIT with three naive collaborative KE methods, which apply standard KE algorithms (e.g., MEMIT (Meng et al., 2023) and MALMEN (Tan et al., 2024)) to update the local model and use the current model merging algorithms to merge local updates into the global model. In particular, we experiment with three most commonly used algorithms for model merging, including SVVRLEE-VERAAEE (Chronopoulou et al., 2023), T'ASK-ARITMMETIC (Ortiz- Jimenez et al., 2023), and - (Yadav et al., 2023).
Evaluation metries. Unless otherwise mentioned, we utilize MEMIT as the backend KE algorithm and adopt the same metrics as MEMIT to evaluate editing performance. Strietly following the lit- erature (Meng et al., 2022; 2023; Tan et al., 2024), we use Efficacy Score (ES), Paraphrase Score (PS), Neighborhood Score (NS), N- N-gram Entropy (NE), Reference Score (RS), and Score (i.e., the harmonic mean of ES, PS, NS) as metrics for MCF; we use Neighborhood Accuracy (NA), Para- phrase Accuracy (PA), Efficacy accuracy (EA)), and Score (i.e., the harmonic mean of NA, PA, and EA) as metrics for zsRE. When using MALMEN (Tan et al., 2024) as the backend KE algorithm, we adopt the same metrics as MALMEN for a fair comparison, including "editing success" " (EA), "generalization success" (PA), and "locality success" (NA). See detailed descriptions in Appendix C.
Evaluation benchmark for conflict knowledge editing scenarios. In order to conveniently simu- late potential scenarios of collaborative knowledge conflict and analyze the issues and impacts that these scenarios may bring, we reconstruct two existing benchmarks to simulate knowledge conflict situations through GPT-3.5-turbo Initially, we attempt to explore the impact of knowledge conflict on model performance using Multi-CounterFal For each data point (s, o) in the MCF dataset, we utilize GPT-3.5-turbo to generate conflict ob- ject that is identical to s and r but differs in o. Section E in the Appendix showcases a concrete example of the generated conflict object. To validate the effectiveness of our two-stage mechanism to resolve knowledge conflict, we utilize the Easy dataset (Li et al., 2024) for the sake of simplicity. This dataset was constructed by creating several additional related knowledge edits for each edit using Wikipedia as the source, which MCF does not include. Additionally, we also generated a corresponding conflict object for each edit in the dataset using GPT-3.5-turbo.
(MCF) (Meng et al., 2022) due to its large scale.
7

---

Published as a conference paper at ICLR 2025
Table Overall editing performance GPT-J (6B) and -P, based on MALMEN (Tan al., 2024). We edit models and each model will be edited by 125 requests of zsRE. The "Score" serves as the overall metric.
GPT2-XL (zsRE)
GPT-J (zsRE)
Method
EA f
PA 1
N
Score f
EA 1
PA f
t
Score
GLOBAL-EDIT
99.21
93.08
16.5
36.84
99.95
95.66
27.32
52.57
TIES- MEES-MERGING
15.52
14.85
18.68
16.18
27.86
26.76
25.18
26.55
-AITHRRITCCCTIC
50.37
45.79
4.28
10.89
27.59
29.6
26.3
27.76
SIMPLE-AVERAGE
52.39
46.03
4.58
11.57
71.15
53.96
4.82
12.49
COLLABEDIT
99.06
92.66
15.49
35.11
99.62
92.88
23.25
47.01
5.2 EXPERIMENTAL RESULTS OF KE PERFORMANCE
Superior collaborative knowledge editing performance. As shown in Table and 2 when using MEMIT (Tan al., 2024) the backend KE algorithm, our privacy-preserving solution COLLABE- DIT achieves on-par editing performance with that of GLOBAL-EDIT,, and significantly outperforms other naive model merging methods in terms of the "Score" ditionally, our COLLABEDIT has nearly identical performance with GLOBAL- via combining the weight updates of each client, which ensures both privacy protection and editing quality. Nev- ertheless, there exists a significant gap between the performance of baselines and GL.OBAL-EDIT. Table 3 additionally shows the editing performance of COLLABEDIT when using MALMEN as the backend KE algorithm (Tan et al., 2024): COLLABEDIT is capable of performing nondestructive collaborative KE across various mainstream KE methods.
datasets and two models. Ad-
Discussion about the performance of baselines. Though other baselines (Table 1 and 2) have a relatively higher NS value compared to GLOBAL-EDIT and our COLLABEDIT, we conjecture that might be caused by the under-fitting phenomenon: these model merging methods are not specifically designed for merging the weight updates from knowledge editing, which is reflected by their low values of PS, ES, and Score. The results of two models GPT2-XL and GPT-J (6B) (the first line) further confirms that the high NS of other baselines are largely due to the inherent high quality of the model itself, exhibiting their poor collaborative KE effects. Note that NS/NA emphasizes that the edited model should maintain the same answer for neighborhood prompts of edit requests. However, editing certain knowledge using existing KE (Tan et al., 2024; Meng et al., 2023) methods would inevitably affect the association of its neighboring prompts, which leads to similar drop of NS/NA for both GLOBAL-EDIT and COLLABEDIT
5.3 EXPERIMENTAL RESULTS ON THREE CHALLENGES COLLABORATIVE KE
Residual R effectively detect the knowledge over-
200
lap. To understand the impacts of knowledge overlap, repeatedly edit the same edit requests into the global model. Figure shows that as the number of repeating edits increases, the f--norm of residual R reduces rapidly and becomes smaller than 0.01 when repeating edits for 12 times, which is consistent with our theoretical analy- sis in Section 4.2. This implies that the fz-norm of R can be used to check whether "overlapped editing" hap- pens, which may be helpful for practitioners to avoid the decrease model performance.
Ex-toem of rosidual
Global-edit
150
100
50
10
15
20
Number repeatiin edits
Figure 3: The f2-norm residual R. when data replication happens.
Knowledge conflict can compromise the editing performance. To explore the impact of knowl- edge conflict, we reconstruct MCF with knowledge conflict (see Section 5.1 for details), where each edit request I' = (s', r', o') in the benchmark corresponds to a f = (s, r; o) in MCF and s' = 8, r' r, o' # o (based on the definition in Section 4.2.2). We randomly sample 5,000 edit requests and their conflicted versions from both datasets, denoted as and ε.. For experiments, we can either distribute edit requests in È or both sets (É and E`) across all the clients for collaborative KE to un- derstand the impact of E' on E. Table 6 (see Appendix) evaluates the KE performance of with and without the editing of the conflicted set E' to explore the impact of knowledge conflict on KE perfor- mance. We can see that the overall KE performance largely decreases due to conflicting knowledge especially for PA and EA: as those accuracy-related metrics, in comparison the success-related
8

---

Published as a conference paper at ICLR 2025
metrics (i.e., PS, NS. ES), are more rigorous; while NA, metric used to assess whether irrelevant knowledge is affected, nearly remained unchanged. See Section C in Appendix for details.
Two-stage mechanism with knowledge augmentation can mitigate confficts. Given the harmful impacts of knowledge conflict, we examine our two-stage mechanism (introduced in section 4.2.2) on the modified Easy dataset (L. et al., 2024). In this scenario, there is objective standard to determine which edit should be retained. Therefore, we can employ the FCFS (Zhao & Stankovie, 1989) or FIFO (Morse & Richardson, 1983) to select the correct edit to be preserved. Subsequently, we augment edit requests and obtain weight updates from the selected client.
Firstly, we present detailed example of resolving knowl- edge conflicts in Figure 4. Specifically, given the question "What use does 'fpartt have?", there are two edit requests that induce conflicting answers, i.e., "data migration" and "data transfer". Let's define "data migration" as the target knowledge to preserve and "data transfer" as the conflict knowledge to remove, and we have the following observa- tions after adopting the proposed mechanism: (1) Before solving the conflict (left), the LLM produces a large out- put probability for both "data migration" and "data trans- fer"; (2) After solving the conflict (right), the probability of "data migration" " slightly increases while the probability of "data transfer" drops O. Moreover, the probability of unrelated knowledge re- mains unchanged. The results show that knowledge conflict is effectively resolved.
Relative Probability
0.6
Tagest dia il gisii
0325
9491
0.4
0.2
0.0
9.521
Conflict Unselved Conflict Solved Input: What use does 'fpart has
Figure 4: An example using data augmen- tation to address the problem of knowledge conflict.
Secondly, we evaluate the performance of the proposed two- stage mechanism with a large number of conflicting edit re- quests. In Table 4, we present the Average Probability Differ- ence (Avg-Ap) and Target Success Rate (Succ) before and af- ter resolving knowledge conflict. Specifically, we experiment with 1,000 pairs of target knowledge and corresponding con- flicting knowledge. A larger Avg-Ap (i.e., output prohabil- ity of target knowledge minus output probability of conflicting knowledge) and a higher Suce (i.e., the target knowledge is the final output) indicate that the model is more inclined to output the target knowledge, which indicates that knowledge conflict is resolved. As illustrated in Table 4, our two-stage mechanism effectively mitigates the issue.
Table 4: COLLABEDIT utilizes aug- mented edit requests to mitigate the knowledge conflict.
Avg-Ap
Succ
Before Resolve After Resolve
-18.11
37%
17.6
77.6%
Dynamic C' can alleviate the knowledge forgetting. As described in Section 3.3.2, we assume that each client has a set of old edit requests E, (initially edited), as well as m sets of new edit requests En = 'Enyy Eng." .... ]. We note that for this experiment, there exists no conflict between Eo and E,,, which allows us to investigate the effects of knowledge forgetting. As shown in Table 5, we find that after numerous rounds of editing, the LLMs produce much lower PS and ES for knowledge obtained from & due to the knowledge forgetting. Under the same condition, we dynamically update the covariance matrix C according to Equation 7 when editing both & and E,, We observe that the dynamic C significantly mitigates the issue, with the Score only dropping from 79.03 to 78.15 on GPT-J and MCF.
Table 5: Dynamic covariance matrix C can alleviate the knowledge forgetting. We gather all the edit requests in each round and apply global KE to edit the global model to study the knowledge forgetting issue. For experiments, we initially use & to edit the global model and sequentially use sets of aggregated new edit requests. where set to large value (i.e.. m 1000). We report the editing performance of old edit requests& &,, before and after m rounds new editing. GPT- (6B) and GPT2-XL is used.
MCF
ZsRE
Model
Method
NS
PS
ES
Score
NA 1
PA f
EA
Score
Before m rounds of editing
57.20
96.13
99.26
79.03
28.05
88.79
92.05
51.92
GPT-J
After rounds of editing (Immutable C)
65.14
76.94
84,58
74.68
24.21
61.05
66,22
41.21
After m rounds of editing (Dynamic C)
58.15
91.62
97.32
78.15
26.54
79.34
84.40
48.28
Refore m rounds of editing
65.08
80.66
89.66
77.08
25.25
64.71
68.96
43.12
GPT2-XL
After rounds of editing (Immutable
64.89
60.38
69.82
64.80
25.28
53,96
After rounds of editing (Dynamic C)
61,54
74.33
82,30
71.72
24.40
56.57
59.89
39.80

---

Published as a conference paper at ICLR 2025
6 THE DISCUSSION ON THE PRIVACY PRESERVING OF COLLABEDIT
This section theoretically and empirically justifies that COLLABEDIT is privacy-preseringg via shar- ing KK We begin our justification by defining imput keys K as:
K k2. kas] E TRdx
(8)
where d indicates the dimension of the feature vector and M indicates the number of edit requests.
Theoretical aspect. We aim to prove that nontrivial to reconstruct the K given KKT, which is equivalent to proving that given any specific KK there exists an infinite number of K (every K may involve different M) that will lead to the same KK
Let's assume there exists matrix operation W' E MRM',, which can transform K into K' through K' = K W' and ensure that K'K' KK Then we have:
(9) where any orthogonal matrix W' such that W = I will lead to the K' which has the same covariance matrix as K. Since there exists (Grove, 2002; Hall & Hall, 2013) an infinite number of the orthogonal matrix W that meets the condition of W = when M > 1, we can conclude that it is nontrivial to reconstruct the given KK from theoretical perspective..
K'K' KW (KW") = K(W'W") " = KKT
Empirical aspect. Our objective is to quantify the tent of privacy leakage by recovering the input sequences of edit requests solely based on the observed K KK! T. Notably, K compromises the feature embeddings of input sequences, and thus we leverage the SoTA embedding in- version attack, GEIA (Li et al., 2023a), to recover input sequences from their feature embeddings.
Averge Rniseddigg Stuilarity
LUCE
0.7-
0.6
For generality, we adopt the same setup as GEIA re- cover input sequences. The key idea is to build a powerful attacker model decode the sequences from embeddings. The privacy leakage is measured by embedding similar- ity (Cer et al., 2017) between original sequences and re- covered sequences in terms of an LLM (e.g., L5-Large (Raffel al., 2020)). Since we also want to measure the privacy leakage of KK we further tailor the attacker model to recover input sequences from KK Consid- ering that KK is covariance matrix involving M input sequences, embedding similarity between the recovered sequence and any of the M sequences.
0.5-
KK KK KK KKy
Figure 5: We show the average embedding similarity between recovered sequences (in- ferred from K or KK involving M se- quences) and their ground truths. The grey line the average embedding similarity be- tween two random text sequences.
calculate the maximum
Figure shows that sharing K results in severe privacy leakage as the recovered sequences are close to the original sequences with large embedding similarity. In contrast, with only small M such as 8. KK reduces the embedding similarity to 0.69, which is close to that between two random text sequences (grey line). In other words, the recovered sequence from KK is almost irrelevant any of the M sequences when M 2 Therefore, we show that COLLABEDIT achieves griacy-preserving via sharing KKT.
CONCLUSION AND FUTURE WORKS
In this work, we propose the first collaborative KE framework. COLLABEDIT, which allows multiple parties to jointly edit the knowledge of an LLM without disclosing their private edit requests. In particular, COLLABEDIT leverages the model merging techniques to combine the updates made by each client in their local models. Motivated by the theoretical analysis, we design our framework to non-destructive, which achieves comparable performance to directly editing a global model using aggregated edit requests. Based on COLLABEDIT, we further provide remedy toward solving intervention challenges raised in collaborative KE Interesting future works include: (1) Further improving the performance of KE in collaborative learning scenarios; and (2) Diving deeper into the solutions to fully address intervention challenges collaborative KF
'The clients typically edit multiple requests simultancously into the LLM and may also apply techniques (e.g.. MLE (Li et al., 2024)) to augment their knowledge. Therefore, it is reasonable to assume there are least. 2 edit requests in single round (or it could be forced in regulation).
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was partly supported by the NSFC under No. U244120033, 62402418, 62402425, the Key R&D Program of Ningbo under No. 2024Z115, the China Postdoctoral Science Foundation under No. 2024M762829, and the Zhejiang Provincial Priority- Funded Postdoctoral Research Project un- der No. ZJ2024001, the Research Center for Industries of the Future (RCIF) at Westlake University, and the Westlake Education Foundation.
REFERENCES
Josh Achiam, Steven Adler. Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0S2 08774. 2023.
Afra Akyürek, Eric Pan, Garry Kuwanto, and Derry Wijaya. Dune: Dataset for unified editing. In Empirical Methods in Natural Language Processing (EMNLP), 2023.
Daniel Cer, Mona Diab, Eneko Agirre, liigo Lapezo and Lucia Specia. Semeval-2017 task I: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 1-14, 2017.
Alexandra Chronopoulou, Matthew E Peters, Alexander Fraser, and Jesse Dodge. Adaptersoup: Weight averag- ing to improve generalization of pretrained language models. In Association for Computational Linguistics (ACL), 2023.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104 081648444 2021.
Dongyang Fan, Celestine Mendler-Dünnerr and Martin Jaggi. Collaborative learning via prediction consensus. Neural Information Processing Systems (NeurIPS), 2024.
Larry C Grove. Classical groups and geometric algebra, volume 39. American Mathematical Soc., 2002.
Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. Model editing at scale leads to gradual and catas- trophic forgetting. arXiv preprint 0rX1::2401.07453, 2024.
Brian C Hall and Brian C Hall. Lie groups, Lie algebras, and representations. Springer, 2013.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. International Conference on Learning Representations (ICLR), 2023.
Joel Jang. Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Sco. Knowledge unlcarning for mitigating privacy risks in language models. In Association for Computa- tional Linguistics (ACI.), 2023.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mchdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Advances and open prob- lems in federated learning. Foundations and frends@ in machine learning, (+(1-)):1-210 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Schastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pp. 5132-5143. PMLR, R, 2020.
Omer Levy, Minjoon Sco, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading prehension. In Computational Natural Language Learning (CoNLL), 2017.
Haoran Li, Mingshi Xu, and Yangqiu Song. Sentence embedding leaks more information than you expect:
Generative embedding inversion attack to recover the whole sentence. arXiv preprint arXiv:2305.0300 2023a.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang, On the convergence of fedavg on non-iid data. arXiv preprint arXiv: 1907.0219000 2019.
Zexi Li, Tao Lin, Xinyi Shang, and Chao Wu. Revisiting weighted aggregation in federated learning with neural networks. In International Conference Machine Learning, pp. 19767-19788. PMI.R, 2023h.
11

---

Published as a conference paper at ICLR 2025
Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang. Xi Chen, and Huajun Chen. Unveiling the pitfalls of knowledge editing for large language models. International Conference on Learning Representations (ICLR), 2024.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273- 1282. PMLR. 2017.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Neural Information Processing Systems (NeurlPS), 2022.
Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In International Conference on Learning Representations (ICLR), 2023.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. Fast model editing scale. In International Conference on Learning Representations (ICLR), 2022a.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory-based model editing at scale. International Conference on Machine Learning (ICML), pp. 15817-15831, PMLR, 2022b.
Amirkeivan Mohtashami, Florian Hartmann, Sian Gooding, Lukas Zilka, Matt Sharifi, et al. Social learning: Towards collaborative learning with large language models. arXiv preprint arXiv:2312.1144 2023.
2023.
Dale Morsc and Gordon Richardson. The lifo/fifo decision. Journal of accounting research, pp. 106-127, 1983.
Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic the tangent space: Im- proved editing of pre-trained models. In Neural Information Processing Systems (NeurIPS), 2023.
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: A survey. In Association for Computational Linguistics (ACL), 2023.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lec, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer lcarning with unified text-to-text transformer. Journal of Machine Learning Research, 21((40):1-67 2020. URL http://jml org/ papers/v21/ 0.0-074.0 html.
Gilbert Strang. Introduction to linear algebra. SIAM, 2022.
Chenmien Tan, Zhang, and Fu. Massive editing for large language models via meta learning. In International Conference on Learning Representations (ICLR), 2024.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A Billion Parameter Autoregressive Language Model. https: //github. com/ http://kingooollleoeo tran jax, 2021.
Jianyu Wang. Zachary Charles, Zheng Xu, Gauri Joshi, H Brendan McMahan, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, al. field guide to federated optimization. arXiv preprint at axXiv:2107.06111 2021.
guide
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roclofs, Raphael SoontijoLopess Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine tuned models improves accuracy without increasing inference time. In International conference on machine learning (ICLR), 2022.
Leijie Wu. Song Guo, Junxiao Wang. Zicong Hong, Jie Zhang, and Yaohong Ding. Federated unlearning: Guarantee the right of clients to forget. IEEE Network (IEEE Netw.j, 2022.
Leijie Wu, Song Guo, Junxiao Wang, Zicong Hong. Jie Zhang, and Jingren Zhou. On knowledge editing in federated learning: Perspectives, challenges, and future directions. arXiv preprint arXiv:2306.0143)/ 2023,
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Ties-merging: Resolving inter- ference when merging models. In Neural Information Processing Systems (NeurIPS), 2023.
12

---

Published as a conference paper at ICLR 2025
Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen. Openfedllm: Training large language models decentralized private data via federated learning. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024.
Ningyu Zhang, Yunzhi Yao, and Shumin Deng. Editing large language models. International Joint Confer- ence on Natural Language Processing and the Asia- siia-Pccific Chapter of the Association for Computational Linguistics (JCCN)--AACL) 2023.
W. Zhao and J.A. Stankovic. Performance analysis of fcfs and improved fefs scheduling algorithms for dynamic real-time computer systems, In (1989) Proceedings. Real- Time Systems Symposium (RTSS), 1989.
13

---

Published as a conference paper at ICLR 2025
A DETAILS OF KNOWLEDGE EDITING IN A SINGLE LLM
Details of identifying the critical path of MLP layers. Following MEMIT (Meng et al., 2023), apply causal tracing to LLMs (e.g., GPT-2 XL) and identify the critical path of MLP layers to edit. For consistency, we edit the same set of layers R as MEMIT such as the 13-17th layers of GPT-2 XL.
Details of the closed form optimization of A for a single layer. We optimize the following objective to obtain the optimal weights W of layer l:
n+1 Σ 1=
m: 1')
W & arg min
Σ
Wk: m;
+
Wk:
(10)
W
where k i n) indicates the old keys derived from existing knowledge and k; (n + S is + [E|) indicates the new keys derived from the edit requests &
Next, we denote W as the model weights before knowledge editing, Kinit = k ....... kn as the set of old keys derived from existing knowledge and K [k,+1, .... 1n as the set of new keys derived from the edit requests & Moreover, Mini [m1.... m. WKin represents the
memory values of Kin that are previously stored and M m1....... m [g] represents the desired memory values of K that we aim to store. We can solve the Equation (10) by applying the normal equation(Strang, 2022):
(W + Ki K
init + KK M K init + MK
(11)
WK init K + WKK T + AKini ma + AKK M mu + MK
In addition, define two variables: (1) C e K KT
intt which represents the covariance matrix of M WK, which represents the residual error of the new associations when evaluated on the old weights W. Then, we can obtain the closed-form solution of the weight updates A as:
the input keys of existing knowledge. (2) R A
A = A BARKK(CH (C+KK)) J-1
(12)
We compute C c E& kk where Eg [kk is estimated as uncentered covariance statistic collected using an empirical sample of vector inputs to the layer (e.g., 100,000 Wikipedia records). j is a hyperparameter that balances the weighting of new v.s. old associations (a typical value of μ is 1.5 x 10 according to MEMIT).
Details of the implementation on simultaneously editing multiple layers. Previously we fo- cus illustrating how existing knowledge editing algorithms edit single layer in the L.M.. To simultancously edit multiple layers et al., 2023)) firstly obtain the desired output vector z; of final layer in R that can maximize Pr [o,|x @ p(s, r:)]. Then, they spread the whole residual over all the layers in R by computing partial residual r! L-1+1
E R, cxisting editing algorithms (c.g., MEMIT (Meng
z of each layer, i.e., E R. Then, the desired memory value of layer I can be computed as m = W/k/ + rf and we can use Equation (12) to edit each layer. For details of the implementation, please also refer to Meng al. (2023). In this work, we strictly follow their implementation to simultaneously edit multiple layers.
B THEORETICAL ANALYSIS OF THE METHODS
For ease of understanding, we will describe knowledge editing for specific layer l and omit / for brevity. We denote AG and A; as the weight updates derived from GLOBAL-EDIT and client i's edit. Kg and K; represent the new keys derived from all the edit requests and client i's edits requests. According to Section 3.1, R& and R. represent the residual errors the output space of layer I derived from all the edit requests and client i's edits requests, respectively. C represents the aggregated statistic over the previously stored keys of existing knowledge. We consider the collaborative editing scenario with N clients and each client model has M edit requests.
14

---

Published as a conference paper at ICLR 2025
B.1 ANALYSIS THE NON-DESTRUCTIVE COLLABORATIVE KNOWLEDGE EDITING
Note that A, and Ag can be computed via (2) as:
A = RGKa Ko A, = R,K. (C+KK))
(13)
Following the definitions of K and R in Section 3.1, we have:
K, 1
kixM],
R. FixMM- ... rixas]:
(14)
Kg kg, .NxMM] =K1, K2. KN]
Rg [11, , = [R1 R2, Ry].
Then have:
R G = R + R R
(15)
According to Equations (13) and (15), can obtain:
Ac(C ンj=1 K,K,
Ac(C+ ... KKNKN
Ac(C+KoKk
= -RaKe
(16)
-R,Kī + +RK + RNKY
AC K, +....
= =Σ (C-KKK
According to the Equation (16), we can finally reach the following conclusion:
AG D AC K,K (C Σ K2KJ)-
(17)
B.2 ANALYSIS OF THE GAP BETWEEN TWO EDITING METHODS
According to the Equation (17), we obtain the relationship between Ag with A as:
A = Ai(C + ( + A
(18)
where (C + Σ K.K,). Furthermore, we denote the weight updates derived from the destructive collaborative knowledge editing method using "Task-Arithmetic (TA)" as A'c. We have:
A x (Ai A + AN).
(19)
Then, the gap between Ag and A can be calculated as:
N A A Σ (C+KKK))
N AA
Σ
1=1
t=1
(20)
N (C (C+KKK))(C+ Σ -
C EVALUATION METRICS
C.I METRICS FOR MULTI-COUNTERFACT
Multi-CounterFant (MCF) contains assortment of prompts and texts for evaluating model rewrites. For (8,, ry). knowledge editing aims to rewrite the old object of with the new desired object o,. We use the same metrics as previous works (Meng al., 2023) for evaluation:
Efficacy Success (ES) is the proportion of cases where the new object o, exceeds the old object of in probability:
E, [PrMo (ox|p(si, rx)] 2 Pr.o (oflp(s;, r:)]]
(21)
15

---

0, arg max Pr,Mo [o||p] Published as a conference paper at ICLR 2025
Paraphrase Success (PS) is the proportion of cases where the new object of exceeds the old object of in probability on rephrasings of the original statement:
Eg [[ppepunphnnssssssrs)) [PMMM [o: > Præs [o[[p]]]
(22)
Neighborhood Success (NS) is the proportion neighborhood prompts (all such prompts have the same old object of) where the model still assigns higher probability the old object:
E, Bepeniighhrrhood prompts( (;.3)) PPMM [o:|p] < PM [o]1p]]]
(23)
C.2 METRICS FOR 7sRE
For the sake of consistency, we report the same three accuracy- based metrics as the previous work (Meng et al., 2023) to evaluate the editing performance on zsRE when using MEMIT (Meng al., 2023):
Efficacy Accuracy (EA) is the proportion of edits that the model MM recalls with top-1 accuracy. Specifically, edited model Ma should correctly recall the target object or with the largest probability given a templated prompt p(8;, r3) containing 8; and rı:
E 0 = arg maxPrMa (dlp(s))
(24)
of
Paraphrase Accuracy (PA) is the accuracy of rephrasings of the original statement:
E,
Bppporaphnsse(ss.r))))
(25)
of
Neighborhood Accuracy (NA) is the proportion neighborhood prompts that the model gets correct for the old object of:
E;
E Eeiighhhoroodd prompts(ss:r.)
of = arg max Pru, lo/pl]
(26)
D ALGORITHM OF OUR COLLABEDIT
Algorithm 1 COLLABEDIT: Non-destructive Collaborative Knowledge Editing
Require:
The number of clients N, edit requests & of each client S SiS .V) where &; {(say, Tays 0zy (.). language model Me with weights w' of layer l, a set of MLP layers to edit R, covariance matrix C of existing knowledge (optional for direct editing methods, e.g., MEMIT), Hyper-network H with learnable parameter K; for layer I (optional for hypemetwork-basee editing methods, e.g., MALMEN), set of prompt templates P.
Ensure: Edited language model Ma with updated weights W = W + A of layer
1: A =[]. KKT [[]
2: for i E. do
3: Ajist KKTi +- GetDeltaAndKKT (Ei, Mo, C.H. P)
4:
Aust-oppend (is)) appenddKT)
5:
for I E R do
6:
A + C
7:
A + kil
8:
for E N do
9:
K!K!T
KKT aad [M] A A
10:
A + A + K!K!"
11:
A - A x (C+ +KK!
A A (Nt {KK)
13:
W W Σ A
16

---

Published as a conference paper at ICLR 2025
Algorithm 2 GeIDeltaAndKKT
I: procedure GETDELTAANDKKTIC Mo. H. C.P) for ",, 0, E & do
2:
3:
Lj p. E logPrMs (0, P2
4: optimize 2, arg mingin, Lg the desired output of modified layers output oy given(sy,T))
5:
Cache Lj Atsst = []. KKT =1]
6:
7:
for E R do
8:
h. + h. + a m
9:
for
s,, 0j E.g do
10:
k' k $ EEP Pa(s)rr)
+ -
w'
12:
rf *+HO VELL)K
13:
K' + Iki, K1
14:
R' [ri,...,rf]] rfl
A + R'K (C' + y-
15:
16:
A + R'K! (AII k'k" y-1
17:
append/ A'), KKT .pppekiii return A KKT
18:
17

---

Published as a conference paper at ICLR 2025
Table 6: Knowledge conflict can compromise the editing performance of collaborative KE. We denoted and E' in section 5.3. Edit & indicates that only requests & are edited, while Edit and El indicates that requests both sets are edited. evaluate the editing performance of edit requests in
Model
Method
NS f
PS
ES
NA
PA
EA
Rdit &
57.09
96.31
99.2
5.32
69.24
91.96
GPT-
Edit and gx
60.59
85.43
91.18
5.33
27.83
48,84
Edit &
64.85
81.06
89.56
8.5
38.89
58.28
GPT2-XL
Edit and g
63.77
69.89
78.16
7.54
15.96
24.28
Table 7: A summary of scenarios of knowledge conflict.
Situation
Analysis
= m2
Two conflicted editing events e1 and are made by the same client. this case, the client could directly apply knowledge augmentation techniques (e.g., Multi-label Editing Li et al. (2024)) to overwrite its previous knowledge.
mi * m2 and 1 = l2
Two conflicted editing events éj and e2 are made by different clients at the same round of editing. In this case, we need to further develop a two-stage mechanism to solve conflict as il- lustrated in Section 4.2.2.
mi # m2 and t # tz
Two conflicted editing events E₁ and 62 are made by different clients at different rounds of editing. In this case, need to further develop a two-stage mechanism to solve conflict as lustrated in Section 4.2.2.
H-mean of (ES, PS, NS)
79
Glabal-Edit
H-mean (ES, PS. NSS
84
Global-Edit
CollabEdit
82
78
80
77
78
76
76
75
10
30
50
70
10
30
50
70
Numbers of clients
Numbers of clients
(a) A number 5000 edit requests
(b) Every client edits requests
Figure Comparison of global KE (GL.OBAL--EDTT and collaborative with different client numbers.
18

---

Published as a conference paper at ICLR 2025
E
CONCRETE EXAMPLE OF GENERATED CONFLICT OBJECT
Instruction:
Prompts for generating a conflict object
("You'rz 're tasked with creating a new benchmark for conflicts in knowledge clipping. Given a set of data structure as shown in the example, your objective is to generate conflict object.
Specifically:
1) Modify the 'target_new' field to similar but different & incorrect answer.
2) Adapt the 'attribute peteriiuteepromps accordingly to reflect the modified 'target_new"
3) Provide the output in JSON format, containing the modified 'target_new" and the adjusted 'attributte_prompss"
Input: ()
Output:
Outputs: A conflict object response for case-O in MCF by GPT-3.5-turbo
case-O-conflict:
( "case_id"; 0,
"pararel_idx"" 2796, pectuesstedrewritt""
"prompt": "The mother tongue of () is", "relation_id" "P103", "target_new"; "str": "Spanish",
"id": -1
}
"target_true": { "str": "French",
"id": "Q150"
"subject": "Danielle Darrieux" l,
"paraphrase "erraphrase_prooptt"::
"Shayna does this and Yossel goes still and dies. Danielle Darrieux, native",
"An album was recorded for Capitol Nashville but never released. Danielle Darrieux spoke the language"
},
"The mother tongue of Léon Blum is", "The native language of Montesquieu is", "François Bayrou, native",
"The native language of Raymond Barre is",
"Michel Rocard is native speaker of",
"Jacques Chaban-Delmas is a native speaker of",
19

---

Published as a conference paper at ICLR 2025
"The native language of François Bayrou is", "Maurice Genevoix, speaker of", "The mother tongue of François Bayrou is", "Melchior de Vogūé, speaker of" l,
" "attriibute_prompts":
"J. Aww.R.xOR..... VxxaRR. Tolkien is a native speaker of",
"The mother tongue Douglas Adams is", "The mother tongue of Paul McCartney is", "Eivis Presley is a native speaker of", "Barack Obama, speaker of", "Douglas Adams, speaker of", "Meryl Streep, native", "George Orwell spoke the lanquage", "George Washington, native", "Michael Jackson, a native"
"Danielle Darrieux's mother tongue is",
"Where Danielle Darrieux is from, people speak the + lanquage of",
"Danielle Darrieux's s mother tongue is", "Danielle Darrieux was born in", "Danielle Darrieux's mother tongue is", "Danielle Darrieux's s mother tongue is", "Danielle Darrieux was born in",
"Where Danielle Darrieux is from, people speak the language of",
"Danielle Darrieux was born in",
"Danielle Darrieux born in"
Outputs: Original data case-() in MCF
case-O-conflict:
"case_id"; 0,
"pararel_idx": 2796, " "requested reeiesttedrrerite":
"prompt": "The mother tongue of () is", "relation_ide id"; "P103", "target_new";
"str": "English",
"id": "Q1860"
"target_true"" { "str": "French", "id": "Q150"
"subject": "Danielle Darrieux"
},
Daaraphras__promee
"Shayna does this and Yossel goes still and dies. Danielle Darrieux, native",
20

---

Published as a conference paper at ICLR 2025
"An album was recorded for Capitol Nashville but never released. Danielle Darrieux spoke the 4 language"
l,
"The mother tongue of 1Au00e9on Blum is", "The native language of Montesquieu is", "Fran 20007oss Bayrou, native", "The native language of Raymond Barre is", "Michel Rocard is native speaker of", "Jacques Chaban-Delmas is a native speaker of", "The native language of Franlu00e7ois Bayrou is", "Maurice Genevoix, speaker of", "The mother tongue of Fran Tran\u00e7ois Bayrou is", "Melchior de Voglu00fe\u00e9, speaker of" l,
" "attrribute__rroptt.::: "J.
Tolkien is a native speaker of", "The mother tongue Douglas Adams is", "The mother tongue of Paul McCartney is", "Eivis Presley is a native speaker of", "Barack Obama, speaker of", "Douglas Adams, speaker of", "Meryl Streep, native", "George Orwell spoke the language", "George Washington, native", "Michael Jackson, a native"
ponnerration_rropttaa
"Danielle Darrieux's mother tongue is", "Where Danielle Darrieux is from, people speak the language of",
"Danielle Darrieux's mother tongue is",
"Danielle Darrieux was born in", "Danielle Darrieux's ' mother tongue is" "Danielle Darrieux's mother tongue is",
"Danielle Darrieux was born in",
"Where Danielle Darrieux is from, people speak the lanquage of",
"Danielle Darrieux was born in",
"Danielle Darrieux was born in"
21


Reviews:
[{"review_id": "td7hNTFTjU", "reviewer": "Reviewer_jVrX", "summary": "The paper investigates the collaborative knowledge editing (KE) for large language models (LLMs). It identifies three primary challenges in this domain: knowledge overlap, knowledge conflict, and knowledge forgetting. The authors propose a framework called COLLABEDIT, which utilizes a non-destructive model merging mechanism to aggregate knowledge edits from multiple parties while maintaining performance and privacy.\n\nThe framework aims to mimic the optimal global editing behavior without the significant performance drops associated with existing destructive methods. Through extensive experiments on canonical datasets, the authors demonstrate that COLLABEDIT outperforms traditional approaches, addressing the identified challenges effectively.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "COLLABEDIT allows for non-destructive knowledge editing, which prevents significant performance drops that are common in traditional methods\n\nThe framework is versatile and can integrate existing knowledge editing methods, providing a comprehensive solution to collaborative KE challenges\n\nEmpirical results show that COLLABEDIT outperforms existing destructive baselines, demonstrating superior editing performance even with a large number of edits", "weaknesses": "The non-destructive merging mechanism may introduce additional complexity in implementation compared to simpler, traditional methods.\n\nIts scalability in large collaborative environments or with numerous clients may need further exploration.\n\nMore experiments on different LLMs could benefit the demonstration of the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G8dYHLHLKA", "reviewer": "Reviewer_hEBc", "summary": "This paper addresses the generalization of knowledge editing within the collaborative learning setting, with a focus on ensuring privacy while modifying the knowledge of large language models (LLMs). The authors propose a novel approach by sharing $KK^{T}$, an intermediate weight associated with the keys of edited knowledge, instead of naively sharing and averaging weights, which is theoretically proven to be resistant to attacks. The experiments conducted demonstrate the effectiveness of the proposed approach.", "questions": "Please refer to my summary of my weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ The paper tackles an important problem of generalizing knowledge editing to collaborative learning settings where privacy is a critical concern.\n+ The authors provide a compelling theoretical analysis of the limitations of naive weight sharing and introduce the concept of sharing $KK^{T}$, which is proved to be difficult to attack in the traditional privacy-aware setting.\n+ The experiments conducted seem to effectively demonstrate the effectiveness of the proposed method.", "weaknesses": "- It is not surprising to see the destructive performance of direct fed-average for knowledge editing, as edits individual client are naturally diluted when models are averaged, although I appreciate the formal mathematical treatment of the issue.\n- While knowledge conflict is identified as a key challenge, the paper addresses it in a rather ad hoc manner compared to other challenges, which are supported by theoretical analysis.\n- My biggest concern is on the privacy part of the model. Although the authors propose to share $K^{T}K$ and providing theoretical proof of its resistance to attacks, the paper does not fully address the new privacy challenges faced by LLMs. If the edit is successful, the new knowledge can be easily prompted out from the LLMs by simply asking questions. This is especially convenient given that most knowledge editing tasks involve only the editing of factual knowledge. Therefore, the traditional privacy methods may not suffice in the LLM case, and further exploration in preserving privacy for knowledge editing is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rvsi6OTqvO", "reviewer": "Reviewer_h2zh", "summary": "This paper introduces COLLABEDIT, a framework designed for collaborative knowledge editing (KE) in LLMs within federated learning scenarios. COLLABEDIT allows multiple parties to collaboratively edit the knowledge in LLMs while preserving data privacy, a novel scenario within knowledge editing and federated learning. It addresses three main challenges—knowledge overlap, knowledge conflict, and knowledge forgetting—by implementing a non-destructive model merging technique that aims to achieve performance close to direct global model editing without degrading results. Extensive experiments on GPT-J and GPT2-XL demonstrate the effectiveness of COLLABEDIT, showing improvements over existing approaches in federated scenarios.", "questions": "* Why was the setup of editing 10 models with 500 requests (Table 1 and 2) per model not applied consistently in Table3?\n* Could you clarify why the MCF dataset was not included in experiments in Table 3? This dataset would likely provide a valuable benchmark for evaluating the framework’s robustness in handling knowledge conflicts.\n* In the knowledge overlap experiments, the focus was on the  R value’s  $\\ell_2$-norm rather than directly showing the editing method’s performance. How does COLLABEDIT perform when subjected to repeated editing requests for the same knowledge items?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper identifies and addresses a novel problem of knowledge editing in federated learning for LLMs, a new setting within model editing research.\n* The authors propose a straightforward yet effective method—COLLABEDIT—that enables privacy-preserving collaborative editing, which is an essential consideration in multi-party learning scenarios.\n* Experiments on GPT-J and GPT2-XL show that COLLABEDIT can substantially improve performance over methods like MEMIT in federated settings, highlighting its practical effectiveness in this new problem space.", "weaknesses": "* The need for collaborative knowledge editing within federated LLM may be limited, as large-scale federated LLM scenarios are currently uncommon. This reduces the perceived applicability and impact of the problem being solved.\n* The experiments are conducted on older models like GPT-J and GPT2-XL. More recent models such as LLaMA-2, LLaMA-3, or Gemma would provide stronger validation of the proposed method’s efficacy.\n* The paper’s structure could benefit from refinement, as some figures and tables (e.g., Figure 3 and Table 4) are misaligned, affecting readability and presentation quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)