Towards Effective Evaluations and Comparisons for LLM Unlearning Methods

Authors: Qizhou Wang, Bo Han, Puning Yang, Jianing Zhu, Tongliang Liu, Masashi Sugiyama
Primary area: foundation or frontier models, including LLMs
Keywords: llm unlearning

Abstract:
The imperative to eliminate undesirable data memorization underscores the significance of machine unlearning for large language models (LLMs). Recent research has introduced a series of promising unlearning methods, notably boosting the practical significance of the field. Nevertheless, adopting a proper evaluation framework to reflect the true unlearning efficacy is also essential yet has not received adequate attention. This paper seeks to improve the evaluation of LLM unlearning by addressing two key challenges---a) the robustness of evaluation metrics and b) the trade-offs between competing goals. The first challenge stems from findings that current metrics are susceptible to various red teaming scenarios. It indicates that they may not reflect the true extent of knowledge retained by LLMs but rather tend to mirror superficial model behaviors, thus prone to attacks. We address this issue by devising and assessing a series of candidate metrics, selecting the most robust ones under various types of attacks. The second challenge arises from the conflicting goals of eliminating unwanted knowledge while retaining those of others. This trade-off between unlearning and retention often fails to conform the Pareto frontier, rendering it subtle to compare the efficacy between methods that excel only in either unlearning or retention.  We handle this issue by proposing a calibration method that can restore the original performance on non-targeted data after unlearning, thereby allowing us to focus exclusively on assessing the strength of unlearning. Our evaluation framework notably enhances the effectiveness when assessing and comparing various LLM unlearning methods, further allowing us to benchmark existing works, identify their proper hyper-parameters, and explore new tricks to enhance their practical efficacy.

Decision: accept (poster)
Reviewer scores: 3, 8, 5, 6

Reviews and discussion:
<thread>
creation date:
1730385572517

thread_type:
official_review

<thread_object>
creator: reviewer_jexc

<content>
summary:
The paper aims to answer the following research questions in machine unlearning: (1) What metric should be used in machine unlearning (2) what method is better in trading off unlearning and retention. The paper first presents a comparative study to understand what metric is robust (i.e., present a linear relationship before and after jailbreaking attacks), then proposes to use model mixing to for better calibration between retention and unlearning.

strengths:
(1) The problem is well-motivated.

weaknesses:
(1) For the metrics used for machine unlearning, The author suggests using Extraction Strength as the metrics since it exhibits linear relationship before and after different attacks. However, the evaluation is still weak to me. First of all, the attacks used in the paper are not the strongest jailbreaking attacks (like GCG attacks) and the authors also do not consider using an ensemble of strong attacks to elicit model’s knowledge, making the evaluation of the paper weak. Also, showing linear relationship before and after attack might not be a good way to measure machine unlearning before and after attack, especially if the model is backdoored or poisoned (e.g., a poisoned and backdoor model can hide their knowledge in their model parameters except when a trigger is presented or obfuscate the knowledge so that the metrics cannot used to detect unlearning). In short, there are no rigorous guarantees that using extraction strength is a valid metric under an adversarial scenario.

(2) The contribution is incremental. To me, the most novel part of the paper is that the authors use model mixing (aka model merging/souping) to control the trade-offs between retention and unlearning. All the other parts seem incremental and do not convey interesting empirical findings.

(3) The writing of the paper needs to be improved. A lot of useful details help to digest the results are shown in the appendix instead of the main text (e.g., attack methods) and the main text does not give enough concise summary to understand them. For example, I still cannot understand how we use Token Noising as an attack method (even the appendix does not give a good description of it). Also, the organization of the paper could be improved (e.g., lines 264-265 ask the reader to first read Section 6 for the experiment).

questions:
Please provide response to the weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730704201566

thread_type:
official_review

<thread_object>
creator: reviewer_fsyl

<content>
summary:
The paper surveys the current popular evaluation methods towards LLM unlearning. It discuss the need to cover before retain and forget performances and its trade-offs, as well as its robustness against different attacks.

The paper concludes that Extraction Strength (ES) is the recommended method for evaluation. In addition, the paper also suggest Unlearning with Control (UWC) framework to reach the best trade-off between unlearning strength and retain performance.

strengths:
The paper have a comprehensive view of different unlearning evaluation methods and approaches them in a systematic manner from robustness and utility trade-offs.

The paper proposes a novel approach unlearning with control to better calibrate the trade-off between unlearning effectiveness and retain performance with model-mixing, which is a simple but effective mechanism.

weaknesses:
There is a lack of justification for selecting the metric: Why does the PCC measure the metrics' robustness again attacks? In Figure 2, the plot is characterized by the test static before and after the attack for different methods, models, and forget set ratio. Why should we assume there is a linear correlation among them? In addition, the paper uses TOFU as unlearning dataset/task, but does not survey the metric used in the TOFU paper (truth ratio).

Weak/unclear attack methods: it is unclear the what dataset the relearn attack in figure 2 is based on (from other section I would think it is TOFU, but it is not presented until late in the paper). The setup for relearning it not clear.

Although UWC seems like a method to manage the trade-off between unlearn and retain performance, it is not clear how it fits into unlearning evaluation pipeline.

questions:
The statistic in figure 2 is confusing. How does model have lower score (which means better unlearning from the paper) after attacking than before?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731123094249

thread_type:
official_review

<thread_object>
creator: reviewer_8smy

<content>
summary:
This paper studies how best to evaluate LLM unlearning methods that aim to not leak the correct answers on a forget set while maintaining good performance on the rest of the training data. The literature has proposed a variety of evaluation metrics, and this paper first tackles which metric is most robust to information leaking attacks; they conclude ES, which was proposed with privacy attacks in mind, performs best in keeping the ranking before and after various attacks consistent. With ES, they then evaluated a generalized hyperparameter sweep for popular methods proposed in the literature, and conclude that methods have not significantly improved over the baseline of gradient ascent approaches. Narrowing why this is the case, they observe alternative methods either unlearn too strongly or not strongly enough to calibrate the performance on the retain set.

strengths:
1) Extensive empirical study
2) Proposed method for improving calibration via a general hypermater boosts performance of baseline methods to seemingly SOTA
3) Mostly well-written

weaknesses:
1) I found that certain parts of the draft could have been clearer about the benefits of model mixing. The draft does not discuss alternative calibration of retain performance, which naively could have also been done with just a sweep of the unlearning method hyperparameters. So at first I thought this was an empirical limitation. But after thinking about it I realized this is actually okay as the performance of calibration with just a hyperparameter sweep of the unlearning method is subsumed by the hyperparameter sweep including model mixing to calibrate (the former is just taking the hyperparameter in the latter where $\alpha  = 1$). Results in the appendix also suggest the hyperparameters for current unlearning methods are not sensitive enough to calibrate retain performance. In the questions section I ask clarifying questions about this and suggest edits to the text that could make this contribution clearer. 

2) The empirical results are also focused on specific values of $\tau$, i.e., retain performance should be $95-90\%$ of the performance before unlearning. While I agree “high performance” seems to be the reasonable use-case, a more complete comparison without assuming specific $\tau$ values would also identify what $\tau$ range (if ever) NPO and other methods start performing well again. I believe it is implicit from the further study in the appendix that a “low” $\tau$ range might make other methods perform better, but I believe a more quantitative statement could make the empirical study more complete. I ask about this in the questions section.

3) (minor) A more general weakness, not specific to just this paper, is whether these performative unlearning goals for LLMs should also be generalization questions; if the goal is to not predict accurately for a set of samples, and those samples come from larger distribution, should we not be evaluating a “test” performance on the larger distribution? This is generally called concept unlearning in the literature. The field may still be far away from evaluating/studying such questions, but I raise it to clarify the setup studied in this paper is not the only setting for unlearning in LLMs. The authors might consider having more discussion on different formulations of unlearning before picking the setting they study and survey.

questions:
I look forward to discussing the below with the authors, and am willing to increase my score given clarification of the questions below.

1) In the paragraph “Is MM Proper for Calibration” what seems missing is discussion comparing this to a simpler/baseline approach, such as just tuning the original hyperparameters of the unlearning method to calibrate. From the Appendix I got the impression that the hyperparameters of many of the unlearning methods do not give enough control to calibrate, is this correct? If so, discussion (and/or a figure illustrating this) would help the argument that MM is a better way to calibrate/boost the performance of unlearning methods.

2) Related to the above, in the “Hyper-parameter configurations” paragraph in the experiments, am I correct in understanding you do the hyperparameter sweep by evaluating the performance of the hyperparameters after also applying model mixing to calibrate the performance (this seems consistent with the appendix tables)? If so, you might consider adding somewhere in the paper that this hyperparameter sweep necessarily improves over the calibrated performance of not doing model mixing; this is as not doing model mixing is captured by just picking the best hyperparameters amongst those with $\alpha = 1$. I believe this adds conceptual understanding to why model mixing is necessarily better.

3) On the discussion of why NPO and other methods do no work in the experiments section, do you have results for what $\tau$ values (if at all) other methods start performing well? I’m imagining a figure where the x-axis is calibrated $\tau$ level, and the y-axis is forget performance, and a line for each method such that when the line is the lowest explains for what $\tau$ each method is best. Alternatively, some more detailed/explicit discussion for what $\tau$ levels other methods might perform better could help give a more complete picture for when other methods work well. I agree the high $\tau$ range seems most practical, but I think the above can be done easily given results in the appendix and can add to the empirical study.

4) (minor) model mixing seems very related to the literature on model merging; you might consider surveying the literature more broadly as a reference for the method, and when/why it works.

5) (minor) If computationally feasible, could error bars/standard deviation be reported for some of the main tables; I am less concerned given the many settings tested, but this could add to the rigor of the empirical findings.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731269242020

thread_type:
official_review

<thread_object>
creator: reviewer_tays

<content>
summary:
The paper tested 4 popular LLM unlearning methods on the TOFU benchmark dataset and observed that there are various tradeoffs (e.g. GA unlearns better while NPO retains better). Then it motivates the authors to propose a new metric which is based on the mixing the model weights between the unlearned and original model. The reason, as far as I understand, is to achieve smooth control on the extent of unlearning. The evaluation first finds the model mixing ratio that would achieve the evaluation score (e.g. ES) on the retain set, and then uses it to calibrate unlearning metric.

strengths:
S1: LLM unlearning and evaluation are important problems

weaknesses:
W1: Lack of technical contribution: I think most people working in this area would agree we need more metrics and benchmark datasets. However, this paper though goes into that direction, does not really provide enough meaningful and technical contribution in my view. The paper basically tried 4 popular unlearning methods on the TOFU datasets while proposing a calibration framework (See W2). This can mostly be done in leaderboard or in a measurement paper rather than a technical paper. And findings on metric tradeoffs are mostly not surprising.

W2: Lack of justification of the calibration framework: The calibration metric lacks justification. Figure 3 seems to only tell us ES is monotonically increasing with mixing factor $\alpha$. However, the key question in proposing a metric is to ask: What does this metric measure? Does it measure the right thing? In this case, I cannot see clearly the impact of mixing model weights on the overall unlearning effectiveness. The only justification I can find is (1) being inspired by the literature on parameter disentanglement, but why is it related to unlearning? (2) The vague observation in Figure 3. Can authors explain why mixing model weights can help calibrate the unlearning metric other than just the empirical observation between ES and $\alpha$ in Figure 3? After all, this can well be a spurious correlation. In other words, why would the community trust this calibration in evaluation? 

W3: Lack of organizing clarify: The paper spends the first 6 pages motivating the calibration framework, and only 1 page introducing it and 1 page for experiments. I think people who in this area already know most of the points in the first 6 pages, it'd better to shorten it and get to the point more directly and spent more content on introducing the key idea of calibration and include more justification,

questions:
See W2.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731998066499

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to All Reviewers

comment:
We are grateful to all the reviewers for their time and insightful feedback. We appreciate the reviewers' recognition of the importance of the problems we have addressed in LLM unlearning (taYS, JEXC) as well as the novelty (8smY, fSyL) of our proposed assessment methods. The reviewers have also acknowledged our comprehensive experiments across a variety of metrics and unlearning methods (8smY, fSyL).

We also sincerely appreciate the concerns some reviewers have raised regarding the key contributions and the uniqueness of our manuscript. We apologize for any confusion and thank you for the opportunity to clarify these points. We have briefly summarized our uniqueness in the abstract, and we are glad to provide a more detailed summary, organized into two main points, as follows:

1.	**Reliable Metrics**: Although many metrics have been proposed for assessing LLM unlearning, there is still a lack of comprehensive analysis of these metrics’ reliability. We address this gap by stating that **a proper metric should characterize the knowledge parameterization within models**. We also suggest **a systematic method to validate  the appropriateness of a metric via red team attacks**. Its rationale has been rigorously validated in Appendix A, alongside with criteria for assessing the suitability of attacking methods. We identify a set of basic metrics widely adopted in previous works, and **challenge the reliability of commonly used metrics, such as PPL and ROUGE. Also, we find a less discussed metric, i.e., ES, as a more accurate choice**.  

2.	**Reliable Comparisons**: Performance comparison is critical and closely related to unlearning assessment.  However, **improvement in knowledge removal typically compromise retention**, while both the goals of removal and retention are essential for effective unlearning. To overcome this dual focus, **we suggest calibrating unlearned models to align their capability of retention, thereby assessing their capabilities of removal is enough for fair comparison**.  This calibration-based framework is meaningful, **leading to many new findings in Section 6**, including the superior performance of GA-based methods over NPO after proper tuning, as well as improvements in unlearning efficacy through adjusting batch sizes and likelihood capping.

These factors have not been covered in previous works, and we are confident that our new findings hold significance for the field. below, we respond to each reviewer's valuable critiques in our individual responses. We look forward to continuing this valuable discussion during the discussion period!
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work provides an evaluation framework for assessing LLM unlearning. The reviewers agree that the paper has made interesting conclusions and contributions, although the technical novelties and organizations are limited. It would be important for the authors to improve the paper following the reviews in the final version.