Knowledge-localized Unlearning for Faithful Forgetting in Language Models

Authors: Nakyeong Yang, Minsung Kim, Seunghyun Yoon, Joongbo Shin, Kyomin Jung
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Unlearning, Knowledge-localization, Faithful Unlearning, Superficial Unlearning

Abstract:
Large language models are exposed to privacy risks since they are trained on large text corpus, which may include sensitive or private information. Therefore, existing studies have attempted to unlearn undesirable knowledge exposed without permission from a language model. However, they are limited in that they have overlooked the complex and interconnected nature of knowledge, where related knowledge must be carefully examined. Specifically, they have failed to evaluate whether an unlearning method faithfully erases interconnected knowledge that should be removed, retaining knowledge that appears relevant but exists in a completely different context. To resolve this problem, we first define a new concept called superficial unlearning, which refers to the phenomenon where an unlearning method either fails to erase the interconnected knowledge it should remove or unintentionally erases irrelevant knowledge. Based on the definition, we introduce a new benchmark, FaithUnBench, to analyze and evaluate the faithfulness of unlearning in real-world knowledge QA settings. Furthermore, we propose a novel unlearning method, KLUE, which identifies and updates only knowledge-related neurons to achieve faithful unlearning. KLUE categorizes knowledge neurons using an explainability method and updates only those neurons using selected unforgotten samples.  Experimental results demonstrate that widely-used unlearning methods fail to ensure faithful unlearning, while our method shows significant effectiveness in real-world QA settings.

Decision: reject
Reviewer scores: 6, 6, 5, 3

Reviews and discussion:
<thread>
creation date:
1730387665187

thread_type:
official_review

<thread_object>
creator: reviewer_mrad

<content>
summary:
This paper focuses on the issue of superficial unlearning in language models, which refers to the phenomenon where an unlearning method either fails to erase the interconnected knowledge it should remove or unintentionally erases irrelevant knowledge. To investigate the phenomenon of superficial unlearning, this paper introduces a new benchmark, FaithUnBench, to evaluate unlearning methods in real-world knowledge QA settings. Then, it proposes an unlearning method, which identifies and updates only knowledge-related neurons to achieve faithful unlearning.

strengths:
1.This paper explores a promising direction: the timely issue of removing sensitive or private information from language models.

2.This paper defines the problem of superficial unlearning and constructs a benchmark for a more in-depth analysis and evaluation of unlearning methods.

weaknesses:
1.The choice of evaluation metrics is unreasonable. Why is the UA of all baselines equal to 0.33 (only GA on Gemma2 is 30.30)? Does this suggest that the unlearning dataset is simple enough and more rigorous testing methods need to be designed? For instance, WMDP [1] also uses multiple-choice QA to evaluate the unlearning effect, yet existing unlearning methods struggle to reach the level of random guessing.

2.The implemented baselines are relatively few, and more models (LLaMA3) and unlearning methods (such RMU [1] and NPO [2]) need to be compared.

[1] The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning.

[2] Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning.

questions:
1.What is the form of the data to be removed? Is the Base QA used as the training data? This is easier compared to unlearning the Harry Potter books.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730429326866

thread_type:
official_review

<thread_object>
creator: reviewer_bhk5

<content>
summary:
To study the impact of machine unlearning on other related knowledge, the authors define a new concept called superficial unlearning. Based on the definition, they propose FaithUnBench to reveal that existing unlearning methods do not ensure faithful unlearning. To achieve faithful unlearning, the authors propose KLUE to update only knowledge-related neurons via the gradient ascent method.

strengths:
1. It is interesting to evaluate the faithfulness of unlearning using Multi-hop QA and Same-answer QA.

2. It is reasonable to precisely update by localizing certain parameters to reduce the side effects of unlearning.

weaknesses:
1.There is a lack of detailed comparison with existing datasets. For example, RWKU [1] also adopts 200 popular real world entities as the unlearning target knowledge and also considers the impact on related knowledge.

2.The proposed localization method is overly simplistic. How can neurons that express unrelated knowledge be avoided during localization? Additionally, neuron localization is not the only method for localizing key parameters. How does it compare with other localization methods?

3.There is a lack of more in-depth analysis. For example, in the analysis of the distribution of localized neurons, whether there is a certain distribution pattern for the neurons corresponding to different knowledge. 

[1] RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models

questions:
1.There is a lack of quality assessment for the constructed benchmark. How can the noise introduced by GPT-4o be avoided?

2.Why can the problem of superficial forgetting be solved by neuron localization? Some case studies can be done on Multi-hop QA to check if MAf is really reduced rather than MAt being increased due to fewer updated model parameters.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730458669383

thread_type:
official_review

<thread_object>
creator: reviewer_fjib

<content>
summary:
The authors define superficial unlearning and construct a new benchmark, FaithUnBench, to analyze and achieve faithful unlearning.
Furthermore, the authors propose a novel knowledge-localized unlearning method, KLUE, to mitigate superficial unlearning and
reveal that our method outperforms other unlearning methods, dramatically mitigating superficial
unlearning.

strengths:
This article contributes a dataset and an effective method to the community.

weaknesses:
1. There is no comparison of the proposed dataset with previous datasets, such as MUSE, WMDP, KnowUnDo, TOFU.
2. In the Faithful Unlearning setting, some knowledge related to current entities should not be forgotten. Has this consideration been taken into account in the constructed dataset? For example, in Figure 1, changing Tom Cruise's nationality should not affect the answer regarding his notable works. Intuitively, the unlearning process is more likely to damage content related to Tom Cruise rather than affect another person's nationality.

questions:
The goal of unlearning is to completely forget specific knowledge. Has the author considered the following scenario: testing whether the knowledge has truly been forgotten by asking about it in a different language?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730741794828

thread_type:
official_review

<thread_object>
creator: reviewer_c7yt

<content>
summary:
This paper focuses on the issue of "unfaithful" unlearning knowledge from the LLMs, including failing to erase the knowledge it should remove and unintentionally erasing irrelevant knowledge. A new benchmark, FaithUnBench, is proposed for analyzing and evaluating the faithfulness of unlearning in the knowledge QA settings, consisting of Paraphrased QA, Multi-hop QA, and Same-answer QA datasets. The authors also present an approach to mitigate the issue. In particular, it identifies and updates only the knowledge-related neurons based on selected unforgotten samples.

strengths:
* Present a dataset for faithful unlearning knowledge from LLMs, targeting at different categories of unfaithful issues
* Experimental results demonstrate the effectiveness of the approach, with further detailed analysis

weaknesses:
* The approach relies on the world knowledge graph, which is restricted to the triple-based QA settings

questions:
* Have you tried larger p value for the experiments in section 5.6? It seems that the tendency is still going up.
* Have you compared different prompting templates for MCQA? Furthermore, apart from the convenience for evaluation, have you compared it with other ways of prompting?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732088876419

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal for All Reviewers

comment:
We sincerely thank all the reviewers for their invaluable comments and suggestions. Your thoughtful feedback and efforts have greatly improved our work, and we deeply appreciate your time and guidance.

All the contents addressed in our rebuttal process will be included in the final version of our paper.

$\\\\$

### **Detailed comparisons with existing datasets**

Our benchmark aims to unlearn real-world entity knowledge, which can be prevalent in various language models, to consider the most practical situation of knowledge unlearning. Furthermore, our benchmark deals with the complex and interconnected nature of world knowledge; thus, we introduce three types of unlearning evaluation aspects (Paraphrased QA, Multi-hop QA, and Same-answer QA) for more deep analysis of real-world knowledge unlearning. We propose detailed comparisons with existing datasets to clearly show the novelty of our benchmark. We can summarize the differences in our benchmark in the table below.

$\\\\$

||MUSE [1]|KnowUnDo [2]|WMDP [3]|TOFU [4]|RWKU [5]|FaithUn (Ours)
|----|----|----|----|----|----|----|
Knowledge Source|BBC News & Harry Potter book|Copyrighted books|Hazardous knowledge|Fictitious author|Real-world Entity|Real-world Entity
\# Unlearning Entities|N/A|N/A|N/A|200|200|200
\# Forget Probes|889|987|4,157|4,000|13,131|8,377
Knowledge Exists in LLMs|X|X|O|X|O|O
Related Knowledge|X|X|X|X|O|O
Paraphrased QA Evaluation|X|X|X|X|O|O
Multi-hop QA Evaluation|X|X|X|X|X|O
Same-answer QA Evaluation|X|X|X|X|X|O

$\\\\$

In summary, only RWKU and our benchmark address real-world entities as targets for unlearning. Additionally, MUSE, KnowUnDo, and TOFU require fine-tuning to inject knowledge before unlearning, which may reduce their practicality. Furthermore, most existing benchmarks, except for RWKU and our benchmark, have not considered related knowledge.

However, RWKU has not dealt with "multi-hop QA evaluation", which assesses the interconnections between knowledge, and "same-answer QA evaluation", which assesses whether unlearning algorithms degrade output probabilities without considering the given contexts. For example, RWKU includes an unlearning target text, *"Please forget Stephen King, who is a American author, renowned as the 'King of Horror'."*, and also contains a related knowledge question, *"Who plays the character Jack Torrance in the film 'The Shining'?"*. The two questions are quite related, but they are not completely interconnected like multi-hop questions.

In conclusion, the main contribution of our benchmark lies in evaluating whether unlearning methods perform **faithful unlearning** while considering **knowledge interconnection** within the **real-world entity unlearning** setting.

$\\\\$

[1] MUSE: Machine Unlearning Six-Way Evaluation for Language Models

[2] To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models

[3] The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning

[4] TOFU: A Task of Fictitious Unlearning for LLMs

[5] RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper addresses the issue of 'unfaithful' unlearning in large language models (LLMs), which includes both the failure to remove intended knowledge and the unintended erasure of irrelevant knowledge. The authors propose a new benchmark, FaithUnBench, designed to evaluate the faithfulness of unlearning in knowledge-based QA settings. This benchmark consists of three datasets: Paraphrased QA, Multi-hop QA, and Same-answer QA. Additionally, the paper presents an approach to mitigate the problem, which involves identifying and updating only the knowledge-related neurons based on selected unforgotten samples. The paper does not provide a detailed comparison of the proposed FaithUnBench dataset with previous datasets like MUSE, WMDP, KnowUnDo, TOFU, and RWKU. The proposed neuron localization method is a bit incremental. There is concern about how the method avoids neurons expressing unrelated knowledge. The paper lacks a more thorough analysis, particularly regarding the distribution patterns of localized neurons. It would be valuable to explore whether neurons corresponding to different knowledge types follow a specific distribution pattern.