C o ME : An Unlearning-based Approach to Conflict-free Model Editing (2025)

Abstract: Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability. While model editing methods have been developed to address such errors without full re-training, they frequently suffer from knowledge conflicts, where outdated information interferes with new knowledge. In this work, we propose Conflict-free Model Editing (CoME), a novel framework that enhances the accuracy of knowledge updates in LLMs by selectively removing outdated knowledge. CoME leverages unlearning to mitigate knowledge interference, allowing new information to be integrated without compromising relevant linguistic features. Through experiments on GPT-J and LLaMA-3 using Counterfact and ZsRE datasets, we demonstrate that CoME improves both editing accuracy and model reliability when applied to existing editing methods. Our results highlight that the targeted removal of outdated knowledge is crucial for enhancing model editing effectiveness and maintaining the model’s generative performance.

## introduction large language models (llms) encode vast amounts of knowledge during pre-training, enabling them to perform effectively across a wide range of natural language processing (nlp) tasks @xcite @xcite @xcite . however, llms often incorporate outdated, incorrect, or biased information learned from training data, which can directly affect the reliability of their outputs @xcite @xcite . such issues may lead to unexpected results or undesirable biases in the generated responses. ## come: conflict-free model editing as shown in figure 1 , we propose come that improves the accuracy of knowledge editing by utilizing parameter subtraction-based unlearning. our method introduces three key steps to enhance existing locate-then-edit methods: (1) extracting parameters associated with outdated knowledge, (2) performing targeted unlearning during the integration of new knowledge, and (3) restricting the unlearning process to a specific parameter range to ensure that only essential portions are affected. ## conclusion in this paper, we proposed come to address the conflict between outdated and new knowledge that can arise during the editing process in llms. come enhanced the accuracy of knowledge editing by simultaneously unlearning outdated knowledge and integrating new information. experiments showed that our method improved the editing accuracy of existing model editing methods and successfully integrated new knowledge. this approach can be an effective solution for correcting inaccurate or biased information in large language models, and we expect it to make significant contributions to improving the reliability and consistency of llms.