F ed LEKE : Federated Locate-then-Edit Knowledge Editing for Multi-Client Collaboration (2025)

Abstract: Locate-then-Edit Knowledge Editing (LEKE) is a key technique for updating large language models (LLMs) without full retraining. However, existing methods assume a single-user setting and become inefficient in real-world multi-client scenarios, where decentralized organizations (e.g., hospitals, financial institutions) independently update overlapping knowledge, leading to redundant mediator knowledge vector (MKV) computations and privacy concerns.To address these challenges, we introduce \textbf{Fed} erated \textbf{L} ocate-then- \textbf{E} dit \textbf{K} nowledge \textbf{E} diting (FedLEKE), a novel task that enables multiple clients to collaboratively perform LEKE while preserving privacy and reducing computational overhead. To achieve this, we propose FedEdit, a two-stage framework that optimizes MKV selection and reuse.In the first stage, clients locally apply LEKE and upload the computed MKVs. In the second stage, rather than relying solely on server-based MKV sharing, FedLEKE allows clients retrieve relevant MKVs based on cosine similarity, enabling knowledge re-edit and minimizing redundant computations.Experimental results on two benchmark datasets demonstrate that FedEdit retains over 96% of the performance of non-federated LEKE while significantly outperforming a FedAvg-based baseline by approximately twofold. Besides, we find that MEMIT performs more consistently than PMET in the FedLEKE task with our FedEdit framework. Our code is available at https://github.com/zongkaiz/FedLEKE.

## introduction locate-then-edit knowledge editing (leke) has emerged as a key paradigm for updating large language models (llms) by directly identifying and modifying model parameters associated with newly acquired knowledge, eliminating the need for costly full-model retraining @xcite . it has proven effective in mitigating hallucinations @xcite , detoxifying outputs @xcite , and improving factual recall @xcite . ## method in this section, we provide a detailed introduction to the fedleke task and the fededit framework. first, we discuss the relationship among the hidden states of each transformer layer in the llm and the relationship between the hidden states and the input in section 3.1, which is essential for calculating the mkvs. next, we introduce the fedleke task and explain its connection to the leke task in section 3.2, and we also analyze how to optimize and solve it. then, we focus on solving the leke task and extracting the relevant knowledge vector in section 3.3. finally, we propose the fededit framework to address the fedleke task in section 3.4. ## conclusion we introduce fedleke, a novel task that enables collaborative knowledge editing across multiple clients while ensuring privacy and reducing computational costs. to achieve this, we propose fededit, a two-stage framework comprising editing and reediting. in the editing stage, clients locally perform knowledge editing and upload mkvs to a central server. in the re-editing stage, clients retrieve relevant mkvs via cosine similarity for further refinement. experimental results demonstrate that fededit outperforms strong baselines in fedleke, paving the way for more effective knowledge editing in federated settings and inspiring future research in this direction.