Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
Authors: Qizhou Wang
Bo Han
Puning Yang
Jianing Zhu
Tongliang Liu
Masashi Sugiyama
Venue: ICLR 2025 Poster (2025)
Keywords: llm unlearning
Primary area: foundation or frontier models, including LLMs

Abstract: The imperative to eliminate undesirable data memorization underscores the significance of machine unlearning for large language models (LLMs). Recent research has introduced a series of promising unlearning methods, notably boosting the practical significance of the field. Nevertheless, adopting a proper evaluation framework to reflect the true unlearning efficacy is also essential yet has not received adequate attention. This paper seeks to improve the evaluation of LLM unlearning by addressing two key challenges---a) the robustness of evaluation metrics and b) the trade-offs between competing goals. The first challenge stems from findings that current metrics are susceptible to various red teaming scenarios. It indicates that they may not reflect the true extent of knowledge retained by LLMs but rather tend to mirror superficial model behaviors, thus prone to attacks. We address this issue by devising and assessing a series of candidate metrics, selecting the most robust ones under various types of attacks. The second challenge arises from the conflicting goals of eliminating unwanted knowledge while retaining those of others. This trade-off between unlearning and retention often fails to conform the Pareto frontier, rendering it subtle to compare the efficacy between methods that excel only in either unlearning or retention.  We handle this issue by proposing a calibration method that can restore the original performance on non-targeted data after unlearning, thereby allowing us to focus exclusively on assessing the strength of unlearning. Our evaluation framework notably enhances the effectiveness when assessing and comparing various LLM unlearning methods, further allowing us to benchmark existing works, identify their proper hyper-parameters, and explore new tricks to enhance their practical efficacy.

Published as a conference paper at ICLR 2025
TOWARDS EFFECTIVE EVALUATIONS AND COMPAR- ISONS FOR LLM UNLEARNING METHODS
Qizhou Wang' * Bo Han 122 Puning Yang'
Jianing Zhu'
Tongliang Liu Masashi Sugiyama 2,4
TMLR Group, Department of Computer Science, Hong Kong Baptist University
RIKEN Center for Advanced Intelligence Project
Sydney AI Center, The University of Sydney 3
'The University of Tokyo
ABSTRACT
The imperative to eliminate undesirable data memorization underscores the sig- nificance of machine unlearning for large language models (LLMs). Recent re- search has introduced series of promising unlearning methods, notably boosting the practical significance of the field. Nevertheless, adopting a proper evalua- tion framework to reflect the true unlearning efficacy is also essential yet has not received adequate attention. This paper seeks refine the evaluation of LLM unlearning by addressing two key challenges-4 a) the robustness of eval- uation metrics and b) the trade-off's between competing goals. The first chal- lenge stems from findings that current metrics are susceptible to various red team- ing scenarios. It indicates that they may not reflect the true extent of knowl- edge retained by LLMs but rather tend to mirror superficial model behaviors, thus prone to attacks. We address this issue by devising and assessing a se- ries of candidate metries, selecting the most robust ones under various types attacks. The second challenge arises from the conflicting goals of elimi- nating unwanted knowledge while retaining those of others. This trade-off be- tween unlearning and retention often fails to conform the Pareto frontier, ren- dering it subtle compare the efficacy between methods that excel only in ei- ther unlearning or retention. We handle this issue by proposing a calibration method that can restore the original performance on non-targeted data after un- learning, thereby allowing us to focus exclusively on assessing the strength of unlearning. Our evaluation framework notably enhances the effectiveness when assessing and comparing various LLM unlearning methods, further allowing us to benchmark existing works, identify their proper hyper-parameters, and explore new tricks to enhance their practical efficacy. The code is publicly available at: https: /stpp::/gittuss com/imlzz http/mmmlrrruuuppro wwwp//rnnnaannnnneeeaaa wesiinnggwtttttt withhhCnttre
INTRODUCTION
Large language models (LLMs), like Llama (Touvron et al., 2023a;b) and GPT (Brown et al., 2020; Achiam et al., 2023), have exhibited remarkable proficiency in general-purpose language generation and understanding (Azerbayev et al.. 2023; Roziere al., 2023; Wu et al., 2023; Thiruaavukarasu et al., 2023; Zhou et al., 2024; Huang et al., 2024). These advancements are credited to the devel- architectures (Vaswani et al., 2017) with billions of parameters and
opment of Transformer-basee to the extensive pre-training on web-sourced corpora with trillions tokens (Brown et al., 2020). However, on the other side, scaling up models aggravates the risk of memorizing effects (Arpit et al., 2017) and sourcing from the web makes LLMs inherent its inaccuracies and biases (Liu et al., 2023a). raises the invoking concerns for LLM privacy and fidelity, posing long array of unde- sirable LLM behaviors sourced from training corpora (Liu et al., 2023a), including copyright (Yao al., 2023a), fairness (Gallegos et al., 2023), and toxicity (Liu al., 2023b), among many others.
WWrk done during internship at RIKEN Center for Advanced Intelligence Project.
1 "Correppnndence Bo Han (bhanml @ @wwmp.cchkkteeeoe www.pppww.....u.k.....

---

Published as a conference paper at ICLR 2025
How to Erase Undesirable Data Memorization in LLMs? Machine unlearning (Bourtoule et al., 2021: Zhu et al., 2024) offers a general solution. In the context of LLMs, the primary goal of un- learning is to precisely remove the parameterized knowledge related to unlearning targets meanwhile maintaining model performance for non-targets (Liu et al., 2024). The unlearning targets within LLMs are typically characterized by an unlearning set, denoted D, = {s,, [ar, Yo In, and we need to develop unlearning methods upon Du that meet the goals of LLM unlearning. Some of the noteworthy baselines are gradient ascent (GA) (Yao et al., 2023b), gradient difference (GD) (Maini et al., 2024), and negative preference optimization (NPO) (Zhang et al., 2024).
While algorithmic designs are erucial, their proper evaluations are equally vital. Misleading metrics can lead us to overestimate the unlearning efficacy, potentially causing severe consequences when applying these methods in practice. In general, effective unlearning metries should accurately quan- tify the extent of knowledge parametrization. Previous studies have introduced a set of intriguing metries, such as "famillarity" (Eldan & Russinovich, 2023), "model utility" (Maini et al., 2024), "forget quality" (Maini et al., 2024), and "QA accuracy" (Li et al., 2024). However, these metrics are often intertwined or reliant on manual-designed prompting, which are not general. Even worse, recent works (Lynch et al., 2024) have shown that some metries are highly susceptible to various red teaming attacks, such as jail- breaking (Shen et al., 2023). It indicates that the current metrics might not adequately reflect the extent which targeted knowledge is erased even if models notably retain the targeted knowledge, these metrics may still falsely indicate its complete removal.
We conjecture that effective metric for unlearning should exhibit robustmess across diverse red teaming scenarios. This robustness can manifest as strong linear correlations between metric scores calculated from the original unlearning set and those computed after attacking. Large distortion in this correlation would suggest that the associated metries fail to capture the extent of knowledge parametrization, instead mirroring more superficial behaviors that are vulnerable investigate effective metrics for LLM unlearning. we consider a set of basic metrics, either derived from previous works or mentioned in other related fields (Duan et al., 2024), cf., Section 3. We further examine their robustness under four red teaming behaviors, including jail-breaking (Shen et al., 2023), embedding probing (Belrose et al., 2023), relearning (Lo et al., 2024), and token noising. Then, measuring by the Pearson correlation coefficient (PCC) (Cohen et al., 2009), we observe that the extraction strength (ES) quantiffiiiii the amount of information required to recover original outputs- emerges to be the most effective choice, thus employed for assessing unlearning.
attacks. To
Even with the ES as an effective metric, com- paring various L.M unlearning methods mains a challenging issue. This difficulty pri- marily arises from the need to balance between two conflicting goals for effective unlcarning: retaining performance on non-targeted data (retention) and removing targeted knowledge (removal). For example, when comparing two unlearned models, it is common the case where one model outperforms in removal but another one excels at retention, making it difficult to de- termine which one is overall superior. cf., Fig- ure 1. We address this issue by aligning their common performance, i.e., their capacity of re- tention, in a post-unlearning manner. Motivated by (Wortsman et al., 2022), it is achieved by mixing model parameters from both before and after unlcarning, modulated through a mixing factor a. With proper control via a, we observe that model mixing (MM) enables us to finely calibrate the extent of unlearning such that per- formance on common data is adequately pre- served, meanwhile the inevitable compromise on the extent of removal is roughly minimized, cf., Section 4. Thereafter, we can fairly concen- trate assessing the strength of the removal on
ww UWC
ga
no trade-off
overall better
ES unleam
npo
(wio UWC
ttrade-off
better retention
better removal
пpo
ga
retain
Figure 1: For effective unlearning, it is preferable to have large ES scores for retention (x-axis) yet small for removal (y-axis). For the raw results (orange), we observe that GA excels removal whereas NPO is better in retention, making it hard to determine which method is overall better. UWC resolves this challenge by aligning ES scores for retention, allowing us to focus on comparing the ES scores for unlearning (blue). It leads to the conclusion that NPO is overall superior.
2

---

Published as a conference paper at ICLR 2025
targeted data, thereby alleviating the challenges for comparing different unlearning methods or un- learned models when pursuing to goals of removal and retention concurrently.
We refer our evaluation framework "unlearning with control" (UWC), which incorporates the ES as the basic metric and utilizes MM for calibration to case assessments and comparisons across methods/setups. Based UWC, we benchmark series of representative works along with suggestions for their hyper-parameteee parameters. We challenge the currently perceived advancements in LLM unlearning, where the ostensibly positive behaviors of current state-of-the-ar methods may be the re- sult of either excessive unlearning or insufficient unlearning. Nevertheless, proper hyper-parameter tuning can remarkably enhance the efficacy of many earlier works, such as GA variants, showing potential to exceed many advanced counterparts. Leveraging UWC, we also benefit the commu- nity by exploring a range of simple yet intriguing tricks to further enhance the practical efficacy of current unlearning methods, which are not covered in previous works.
2 LLM LEARNING AND UNLEARNING
To begin with, we discuss the necessary backgrounds for LLM learning well unlearning.
LLM Learning. We study the LLM parameterized by with layer-wise self-attention struc- tures (Liu et al., 2018). Upon receiving input the LLM estimates the probability distributions, denoted by p(-|s; 0), over the next possible tokens. The LLM is trained on a substantial web-scale corpora, denoted by D. {s [z. yl In. of size R4. During training, aim at minimizing the pre- diction loss E(yl x:; 0) = log r(yl:; 0) over Dr. The resulting LLM is capable of properly handling wide range of language generation tasks. We adopt the notation y² to represent the i-th token, y for the prefix up to the i-th token, and the string generated via greedy decoding by f(s; 0).
LLM Unlearning. However, employing training corpora sourced from the wild heavily raises the risk that our LLMs will learn from sensitive information, thereby precipitating a host of legal and ethical concerns (Yao et al., 2023a: Ji al., 2023: Gallegos et al., 2023; Liu al., 2023b).Thes These issues further necessitate the need for post-trainin any associated parameterized knowledge that is undesirable. This requirement motivates the recent research on LLM unlearning (Yao et al., 2023b; Maini et al., 2024), formalizing the above goal by involving so-called the unlearning set D. = {su [a., yu]] (n,, < n4, typically). Overall, LLM unlearning aims adjust model parameters O such that the content related to D, is erased. More specifically, for practical-effective unlearning, should pursue two goals simultaneously:
mechanism that enables our LLMs to eradicate
Removal: The knowledge associated with the unlearning dataset D, should notably deteriorate, revealing effective unlearning on parametrization that targeted be erased.
The knowledge for other data, following D. (Da. should be retained, such that com- mon model responses are sufficiently preserved, thereby ensuring its overall integrity.
Retention:
To ease our discussion below, we distinguish between two types of data: a) targeted data, which are targeted to be unlearned (i.e., within the unlearning set D.), and b) non-targeted data, which are required to be retained (i.e., all other data within D, (Da). Moreover, for the generalization perspective of unlearning, we aim for the unlearned models to not recall the targeted knowledge by assessing on a rephrased version of D,,, adhering to the standard setup as in (Maini et al., 2024).
Unlearning Methods. Stemming from formalization for the above two goals. gradient difference (GD) (Maini et al., 2024) has established as a foundational baseline. Its unlearning objective is
(s [x; +A +AE.DD EewD. (2) 0),
(1)
unlearning risk
retaining risk
which composes of two terms: the unlearning risk and the retaining risks, balanced by the hyper- parameter The unlearning risk increases the prediction losses for undesirable responses yu, align- ing with gradient ascent (GA) when updating LLMs. The retaining risk is implemented to retain the original model integrity, aiming to ensure that the responses for non-targeted data remain un- changed. Despite its mechanisms, previous works believe that GD is still susceptible to catastrophic collapse (Zhang et al., 2024), wherein LLM parameters are remarkably altered and common model responses are severely distorted after unleaming. To further enhance the practical utility, series
3

---

Published as a conference paper at ICLR 2025
of subsequent works have been explored. Among them, methods such KL (Maini et al., 2024), NPO (Zhang et al., 2024), PO (Maini et al., 2024), and RMU (Li et al., 2024), are well-established and have received reasonable attentions. Please refer to Appendix C for more discussions.
EVALUATION METRICS
Accompanying advances made in algorithmic designs, it is also essential to accurately assess the effectiveness for various unlearning methods. Particularly, an maaproppiate evaluation framework, such as those that overestimate the strength of unlearning, can mislead practitioners to be overcon- fident on the reliability of the resulting unlearned models. An ideal evaluation framework for LLM unlearning should effectively quantify the extent to which targeted knowledge remains parameter- ized within. Moreover, it should be general-actionable across tasks, simply to implement, and free from specific prompt engineering that may introduce modeling and prompting bias.
In our pursuit of such an evaluation framework, we begin by examining a series of basic metrics to determine their robustness and suitability, as detailed in the following.
Perplexity (PPL) (Chang et al., 2024): assessing the model confidence of auto-regressive mod- els, defined as the exponentiation of the cross entropy, i.c., exp( p 0)}.
[z;
ROUGE-L (ROUGE) (Lin, 2004): measuring output quality by the proportion of the longest common sub-sequence presents between the ground truth y and the model response f(x; 0).
Exact Memorization (EM) (Tirumala et al., 2022); measuring output quality by the proportion of the same tokens with the ground truth y, i... J Σn 1 (arg max, { [ar <1] 0) y² where 1 returns if the condition therein is true, otherwise 0.
Extraction Strength (ES) (Carlini et al., 2021): quantifying the strength of memorization by the minimal proportion of the prefix recover the suffix. To better align with its name, we adjust the metric to use minus its negative value, i.e., 1 - + ming ([=, y (=) = y y²*}.
KL Divergence (KL): the KL divergence for predictions between original and unlearned models. is formalized as KL p(vl 2; 0) || p(y|2: xx; 0ret)] with KL the operation of the KL divergence.
These metries cover a broad range of practical metries that are widely recognized in prior research. For example, PPL is used as a part of the metrics for "model utility" in (Maini et al., 2024), and the "rewrite score" in (Patil et al., 2023), among many others (Patil et al., 2023); EM serves the key metric for (Barbulescu & Triantafillou, 2024; Jin et al., 2024); ROUGE is adopted in (Du et al., 2024; Maini et al., 2024); KL is mentioned in (Garg al., 2024). We also take into account a less common yet intriguing metric that quantifies data memorization, i.e., ES, particularly pertinent in studies of membership attacks (Garg et al., 2024). Nevertheless, we exclude certain metrics that are difficult to compute, such as those dependent on gold standard models that require the full re- re-training without targeted data (Garg et al., 2024; Thudi et al., 2022: Maini al., 2024). Moreover, for generality, we also disregard task-specific metrics, including GPT-based evaluations (Lynch et al., 2024; Eldan & Russinovich, 2023), QA accuracy that relies on manual-designed multiple choice questions (Patil et al., 2023; Li et al., 2024), and those dependent on task-specific detectors (Yao et al., 2023b).
What Ensures Good Metric? Among candidates, we wonder whether they can effectively quan- tify the internal parametrization of knowledge, a question that is directly tied to the general goals of LLM unlcarning, as mentioned in Section Overall, a proper metric should demonstrate robust- ness against various red teaming scenarios; if not, it risks only capturing superficial model behaviors, thereby vulnerable to manipulative attacks (cf., Appendix A). To gauge this robustness, we examine the metrics with several representative attacking behaviors considered in the following.
Jail-breaking (Shen et al., 2023): manipulating LLM behaviors elicit undesirable knowledge via crafted prompts. A proper metric should be robust to jail-breaking attacks.
Probing (Belrose et al., 2023): decoding middle embeddings via extra linear unembedding mod- ules. should hard to recover unlearned knowledge from embeddings after proper unlearning.
Relearning (Lo et al., 2024): few-shot fine-tuning for unlearned LLMs. In an ideal case, un- learned models are hard to sufficiently relearn the previously unlearned knowledge.
4

---

Published as a conference paper at ICLR 2025
200
PCC+1. (CC=1.0000/.0000
PCC- D. 695.99090112222
PCC-0.7292-000000
PCC=0.9998/0. 9936
150
150
150
100
100
100
retain
unleam
50
100
150
200
50
100
150
200
50
100
150
200
50
100
150
200
(a) PPL jail-braak
(b) relear
(c) PPL probing
(d) PPL noising
1.0
1.0
1.0
0.5
PCC=0.9942/0.9833
POC=0. 6536/0.6834
PCC=00.921:006600
PCC=0.9613/0.9729
0.6
0.6
9.6
0.3
0.2
0.2
0.2
0.2
0.2
0.7
0.2
0.4
03
0.5
1.0
0.0
0.2 0.4
06
03
1:0
0.7
0.2
0.3
0.6
08
100
0.0 02
04
0.5
0.8
(e) ES jail-break
(1) ES relearn
(g) ES probing
(h) ES noising
1.0
1.0
1.0
1.0
6514:0.0622
PCC=0.9073/0.c 9206
PCC=0.914330.9005
0.6
0.6
0.6
0.6
0.2
0.2
0.2
0.2
0.0
I.0
0.0 0.2
(i) FM jail-break
EM relearn
(k) EM probing
()) FM noising
Figure 2: Metric Robustness under Red Teaming Attacks. We depict the metric scores before (X-axis) and after (y-axis) attacks jointly for different unlearning setups: across 2 LLMs (Phi-1.5 and (lama-2-77B)
unlearning percentages (1%, 5%, and 10%), and unlearning methods (GA, GD, PO, and NPO). We consider representative metrics under 4 red teaming behaviors. We apply the log-scale for PPL to avoid numeric errors. For each of these scenarios, we compute the PPC with respect to targeted and non-targeted data respectively. displayed at the top of each figure (tar- geted data / non-targeted data). We provide linear fits for targeted and non von-ttrgeted data separately, accompanied by shaded areas representing the standard deviations that visualize the PPC scores.
Token Noising: perturbing 5% of tokens within each s by replacing them with random tokens. The resulting strings with token noise are used as targets when computing scores across metrics.
Some attacking scenarios have been explored in previous works (Lynch et al., 2024), such relearn- ing and jail-breaking, while others, like probing and token noise, remain less explored. These four attacking scenarios are motivated by broader interest comprehending LLM behaviors across di- verse contexts. For example, LLMs may maintain knowledge without explicitly outputting it (Patil al., 2023), phenomenon related from embeddings (Belrose et al., 2023), pertaining to probing attacks; fine-tuning may inadvertently lead to emergence of harmful model behaviors (Lo al., 2024), associated with relearning. Please refer to Appendix D for detailed descriptions on these attacking strategies. Also, as discussed in Appendix A, jail-breaking and probing are more important for assessing robustness than other ones.
jail-breaking: parameterized knowledge can be extracted
How to Assess the Metric Robustness? To account for the inherent challenges posed for varying attacks aforementioned, it is generally unrealistic to expect that the metric scores to remain un- changed. A more reasonable, yet still rigorous, criterion is to examine whether the metrics exhibit linear relationship between the original values and that after attacks. Accordingly, although values may change, the relative rankings (i.e., the orders of superiority across unlearned models) remains the same without skewing. Please refer to Appendix A for a more formal discussion. Here, we Pearson correlation coefficient (PCC) (Cohen et al., 2009) to gauge the linear correlation before
5

---

Published as a conference paper at ICLR 2025
and after attacks. Note that the potential sensitivities could be attributed to either the limitations of metrics or unlearning methods, yet distinguishing between these two factors is hard. We mitigate this issue by computing the PCC across LLMs, unlearning setups, and various unlearning methods, neutralizing influences from those factors unrelated to the metrics themselves to much extent.
Results. Due to space limit, we examine the robustness of three representative metrics among five candidate metrics, across various attacks as illustrated in Figure 2, please refer to Section 6 for the experimental setups and Appendix D for more results. We observe that relearning has the largest impacts on the robustness of metrics, mainly due to the further tuning of parameters for unlearned LLMs. Under relearnnng attacks, ROUGE shows to be the least effective metric, while ES is our best choice. The probing attacks also have substantial impacts, particularly on the PPL for non-targeted data, even demonstrating negative correlations. Under probing attacks, the ES more robust than other candidates. At last, jail-breaking and feature noising attacks are generally less effective at disturbing the metries, with ROUGE again exhibiting the least robustness. Overall, ES stands out as the most reliable metric for LLM unlearning. It shows superior robustness during relearning and probing attacks, and maintains a small PCC gap over the PPL for other attacks.
The ES Metries for Assessing Unlearning. Based on our evaluations above, we recommend ES as our proper choice for assessing the extent of parameterized knowledge. It is versatile across various unlearning setups and can properly quantify unlearning behaviors with respect to both removal and retention. For removal, the average ES, calculated for targeted data as
ES (Du: 0) (m.. D..
min [ yn ":51: 0) ")²
Vul
should be small after unlearning. For retention, the average ES for non-targeted data should be high:
ES (D, 0) ((..))-) ES will be used as the basic metrie for evaluating LLM unlearning in our experiments below.
y min(M]][[2. (<1, = y>
(3)
FAIR COMPARISON
An essential aspect of quantifying unlearning performance is enabling their reliable comparison, which can facilitate the identification of superior unlearming methods and effective apper-parameter configurations. However, achieving such a fair comparison not straightforward for unlearning, even with the ES as an effective metric. The challenge mainly originates from the inherent trade-off between removal and retention, both of which are crucial for unlearning efficacy.
Often, unlearning methods that excel at removing targeted data will under under-perform in retaining non- targeted knowledge, and vice versa. This scenario necessitates subjective judgments to balance their trade-offs and identify the overall superior choice. Figure presents an example: When comparing between NPO and GA, we observe that the ES computed on targeted data for GA is smaller than that for NPO, indicating GA is more effective in erasing targeted knowledge. On the other side, the ES computed on non-targeted data for NPO is higher than that for GA, suggesting that NPO better preserves the original model performance. While GA may be the appropriate choice when focusing solely on removal, its efficacy relative to NPO becomes less clear when retention is also considered. This scenario is commonly observed in existing methods, cf., Section 6, where their claimed improvements often do not align with the Pareto frontiers between removal and retention.
On the Importance of Calibration. To ensure easy and fair way of comparison, our motivation is to align L.M performance on non-targeted data post-unleariing, i.e., aligning the ES scores on non-targeted data across methods. Once this calibration can be established, we can focus solely on the ES comparison on targeted data. Refer to Figure for the illustration. To achieve the goal of proper calibration, we seek for a flexible control method that permits the adjustment for the extent of unlearning after the unlearning procedure. Inspired by parameter disentanglemen (Wortsman et al., 2022; Ilharco et al., 2022)-heree where mixing parameters from two models can endow the resulting one with characteristics from both, akin to model ensemble (Ortiz-Jimenez et al., 2023) we propose model mixing (MM) as a flexible method for such control. Formally, considering parameters before unlearning, denoted as fret, and after unlearning, denoted as 0, their mixture is given by
0ref a0,

---

Published as a conference paper at ICLR 2025
1.0
retain unleam
0.8
0.8
0.9
0.8
0.4
0.4
0.0
044
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.5
0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
04
0.6
0.8
(a) Phi-1.5 GA
(b) Phi-1.5 GD
(c) Phi-1.5 PO
(d) Phi-1.5 NPO
1.0
1.0
1.0
1.0
0.6
0.0
0.0
0.4
0.4
0.4
0.4
0.2
0.0
02
04
0.6
0.5
0.0
0.2
04
0.5
02
14
0.0
02
0.4
05
08
1.0
0.0
02 04
0.6
0.8
(c) Llama-2-7B GA
(f) Llama-2-7B GD
(g) Llama-2-7B PO
(h) I.lmm-2--BB NPO
Figure 3: ES Scores with MM Control. We depict values of (x-axis) versus the ES scores (y-axis) on targeted (unlearn) and non-targeted (retain) data. We consider 2 LLMs (Phi-1.5 and Llama-2-7B) and 4 unlearning methods (GA, GD, PO, and NPO) under the 5% TOFU unlearning setup.
with 0 a S. the mixing factor that should be searched. In general, lower emphasizes the parametrization of the original model, whereas a higher a accentuates those of the unlearned one. By careful-adjue adjusted a, data, such that the associated ES scores can be maintained, e.g., similar to those before unlearning.
control the extent of unlearning to align performance non-targeted
Is MM Proper for Calibration? The answer is YES. We observe that MM ensures smooth control over the extent unlearning, supported by an overall monotonic relationship between a and the ES scores. We illustrate several examples in Figure as evidence of this effect. The benefits of this smooth control extend beyond stability, which enabling the calibration of unlearned models such that the strength of removal on targeted data is minimally compromised. Therefore, comparisons of ES scores on targeted data after calibration are fair and valid. This smooth control also facilitates us suggest an efficient method for the estimation of the optimal a, as detailed Appendix
At first glance, it seems that byper-paramede tuming can also be used for calibration. To highlight the superiority MM, we would like emphasize that proper calibration method should ensure the control is applied in a noticeable yet smooth manner. However, as observed in Appendix H, the model behaviors are quite sensitive to the choices of hyper-parameters, and we often do not achieve the desired level of recovery even with intensive tuning. In contrast, in Figure 3, the control exerted by MM over model behaviors smooth. Additionally, conducting calibration through hyper- parameter tuning is too method-specifie, and its computational costs also prohibitively high. By contrast, MM can be applied post-unlearning across different methods without incurring the additional costs associated with re-unlearning. Therefore, conclude that MM is more general, reliable, flexible, and efficient than hyper-parameter tuning in calibration.
5 UNLEARNING WITH CONTROL
With the ES as the basic metric and the MM for performance calibration, we name the overall framework unlearning with control (UWC). is two-step evaluation strategy, consisting of a) calibration and b) assessment, structured in the following.
We control the extent of unlearning such that the ES scores non-targeted data should be close to that before unlearning. Formally, we aim for the largest possible a such that least 5 100% of the original ES scores on non-targeted data can be preserved, i.e..
Calibration:
max {a ES(D. (1 ar) + a00) > TES (D, (ret))
(5)

---

Published as a conference paper at ICLR 2025
Table 1: Comparison between different unlearning methods TOFU fictitious unlearning with UWC calibration. t/t indicate smaller larger values are preferable. We primarily focus on the ES scores for unlearning (shaded), given that the ES scores for retention calibrated.
1.IM
Phi-1.5
Llama-2-74
ES
ES-exact
ES ES-perturb
ES- exact
ES ES-perturb
sctup
method
retain
unlearn
retain
unlearn
retain
unlearn
retain
unlearn
before unlearning
0.4433
0.5969
0,2115
0.1605
0.8277
0.8039
0.5302
0.4001
0,4262
0.3748
0,2071
0.1551
0.7536
0.1333
0.4976
0.0230
GD
0,4212
0.3449
0,2072
0.1413
0.7471
0.0293
0.4471
0.1860
KI.
0,4232
0.2123
0,2005
0.0840
0.7337
0.0515
0.4428
0.0913
1%
PO
0.4242
0.6001
0.1936
0.1468
0.7508
0.2387
0.4757
0,2509
NPO
0,4424
0.1259
0,2136
0.0702
0.7383
0.2543
0.4776
0.1703
RMU
0.4245
0.4682
0.2115
0.1855
0.7559
0.5093
0.4096
0.3538
before unlearning
0.4433
0.5619
0,2115
0.2374
0.8277
0.7735
0.5302
0.4126
GA
0,4497
0.2958
0.2136
0,2349
0.7780
0.7033
0.4031
0.4765
GD
0.3919
0.4140
0,2004
0.0045
0.7432
0.3385
0.4775
0.3166
KI. 0.3823
0.3766
0.1794
0.1614
0.7207
0.0953
0.4814
0.1516
5%
PO 0,4086
0.4524
0.2020
0.2343
0.7715
0.5496
0.4792
0.3502
NPO
0.4433
0.3768
0.1836
0.1509
0.7207
0.1 104
0.4804
0.2777
RMU
0.4404
0.4252
0.2047
0.2147
0.7112
0.4034
0.4927
0.3884
before unlearning
0,4433
0.5299
0,2115
0.1843
0.8277
0.8307
0.5302
0,3099
GA
0.3796
0.2486
0,2137
0.1624
0.7015
0.4916
0.4825
0,2419
0,4454
0.4935
0.1761
0.0345
0.7771
0.0980
0.4780
0.1200
0,4424
0.4912
0,2075
0.0922
0.7765
0.2791
0.4734
0.1236
10%
0.4177
0.5499
0,2042
0.1786
0.7543
0.7397
0.5302
0,3435
NPO
0,4072
0.3499
0,2028
0.1281
0.7769
0.3700
0.5100
0,1243
RMU
0,4364
0,5208
0.1944
0.1547
0.7874
0.7526
0.4871
0.3196
where 7 should be close ensure strong calibration. Note that we pursue for the largest to minimize the compromise on the strength of removal, as mentioned in Section 4.
Assessment: For unlearned LLMs that well calibrated for retention, and compare their strength of removal, i.e., their ability to erase parameterized knowledge tar- geted be unlearned. The overall efficacy of unlearning can then be accurately assessed via the ES, where a lower (S( (1 a 0 + a0) indicates better performance of unlearning.
can fairly evaluate
With UWC. we can assess the efficacy of unlearning across various models in general and reliable manner. UWC will facilitate our hyper-paramene tuning and the comparisons of previous works, further supporting our explorations of practical tricks in the section below.
EXPERIMENTS
We benchmark existing LLM unlcarning methods using UWC, recommending their proper hyper- parameters, assessing and comparing their efficacy in achieving effective unlearning. For the promis- ing methods among the candidates, we further examine a series of simple tricks, which can further enhance their practical effectiveness in unlearning.
Experimental Setups. Our main evaluations were based on the well-established benchmarks of TOFU fictitious unlearning (Maini et al., 2024), incorporating two popular LLMs, including Phi- 1.5 (Li et al., 2023b) and Lllmm-2-7B (Touvron et al., 2023a). For the unlearning setups, original training data are separated into targeted and non-targeted parts, of which the adopted proportions 1:99 (1% unlearning), 5:95 (5% unlearning), and 10:90 (10% unlearning). Please refer to Ap- pendix B for more details about the adopted experimental setups.
dlyper-parameee Configurations. We conduct extensive hyper-parameter tuning for the consid- ered unlearning methods, as detailed in Appendix C. The full results across each setup of hyper- parameters can be found in Appendix H. With meticulous selection, suggest À 2 for GD, = 10 for KL, and X = 20 and 8 = 0.5 for NPO. Moreover, for RMU, we select the 9-th layer with for Phi-1.5 and 21-th layer with 2 for Llama-2-7B.
8

---

Published as a conference paper at ICLR 2025
Table Comparison between different tricks for KL on TOFU with UWC calibration. / indicate smaller / larger values are preferable. We primarily focus on the ES scores for unlearning (shaded), given that the ES scores for retention are calibrated.
LLM
Phi-1.5
Llama-2-7B
ES-exact
ES ES-perturb
ES ex-exttt
ES-perturb
setup
method
relain f
unleum
unlearn
retain
unlearn
retain
unlearn
origin
0,4232
0.2123
0.2005
0.0840
0.7337
0.0515
0.4428
0.0913
LR
0,4232
0.2031
0.2005
0.1078
0.7241
0.0428
0,4791
0,0000
BS
0.4232
0.1931
0.2005
0.1078
0.7241
0.0428
0,4791
0,0000
1%
ES
0.4232
0.2033
0.2136
0,0571
0,8277
0.1029
0,4419
0,0403
TS
0.4853
0,0586
0.2517
0.0175
0.7327
0.0522
0,4304
0.0368
LS
0.4620
0.3540
0.2443
0.1582
0.7900
0.6105
0.4656
0.3738
origin 0,3823
0.3766
0.1794
0.1614
0.7207
0.0953
0.4814
0.1516
LR
0.4404
0.4345
0,2069
0.1652
0.7377
0.0953
0.4258
0.0880
BS
0,3879
0.3352
0,2049
0.1432
0.6825
0.0590
0.4450
0.0604
5%
ES
0,4536
0,2224
0.2137
0,1386
0.7928
0.0231
0.4493
0.0144
TS
0,5776
0,5184
0,2473
0.0461
0,7018
0.1406
0.4362
0.0399
LS
0,5766
0,2480
0.2492
0.1293
0,7080
0,3539
0.4299
0.2182
origin 0.4424
0,4912
0,2075
0.0922
0.7765
0.2791
0.4734
0.1236
LR
0.3864
0,4585
0,2001
0.1215
0.7649
0.2791
0,4449
0.1057
BS 0.4302
0.3358
0,2334
0.1621
0.7228
0.2287
0,4285
0.1071
10%
ES 0.4433
0,3974
0,2024
0.1360
0.7803
0,2163
0.4482
O. 0.1076
TS 0.5881
0.4952
0,2493
0.1377
0.6851
0.0730
0.4278
0.0000
LS
0.5909
0.4347
0,2462
0.1197
0.6984
0.4711
0.4249
0.1712
6.1 MAIN RESULTS
We report not only the ES scores for original data but also for the associated paraphrased versions provided by TOFU. These paraphrased datasets maintain the original semanties but feature varied syntax and order, which can be employed to assess the generalization capability of the resulting models. To make the following discussion clear, we term the ES calculated for the original data as ES-exact, and that calculated for the paraphrased versions ES-perturb. The full results after the UWC calibration are summarized in Table 1. Here, we summarize some of our key observations.
Hardness of Unlearning Tasks. Across unlearning setups, we observe that larger forget rates do not necessarily correspond to more challenging unlearning tasks, contrary to prior believes (Zhang et al., 2024). Our results indicate that the 5% setup is more challenging compared to that for both 1% and 10%. Therefore, specific data targeted for unlearning should also be taken into consideration when deciding the hardness of unlearning tasks. Across models, find that Llama-2-77 B can lead to overall better efficacy than Phi-1.5, indicating that unlearning for smaller models are harder.
GA Variants Remain Promising. Previous works often take GA and its variants ineffective. However, via proper fine-tuning for the trade-off hyper-parameterr it reveals that GA-based meth- ods, particularly GD and KL, can in fact exhibit attractive performance. Note that while we identify several cases where the original GA achieves the best ES-exact scores, this might be attributed to excessive unlearning that leads to overfitting, signifying by its higher ES-perturb with poor general- ization. Therefore, we conclude that the retain loss is indispensable for GA-based methods.
Excessive / Incomplete Unlearning is Common. GA and NPO are two important methods in the literature. However, we show that, after UWC calibration, their efficacy in unlearning is not that attractive as our previous belief. However, the causes of their inferior performance are different, which can be seen from the results without UWC calibration in Table 3. As we can see, after unlearning, the ES scores of NPO are much greater than 0, a signal where the strength unlearning insufficient. We provide more justification from the weighting perspective and the risk perspective in Appendix G. On the other side, the ES scores of GA are all near 0, whether for unlearning or retention, indicate its strength of unlearning may too large, occupying the parameterized knowledge for non-targe -trgeee data, thereby making the resulting model completely useless. Nevertheless, we find that GD and KI. with regularization terms of retention can largely mitigate its drawbacks.

---

Published as a conference paper at ICLR 2025
6.2
BAG OF TRICKS
Beyond benchmarking existing works, UWC also enables us to delve into a variety of practical tricks that can empirically enhance the efficacy of unlearning. This aspect has been overlooked in the past, partly due to the pursuit of both removal and retention, which are mutual-conffictnng. Such dual goals render hard determine whether the overall efficacy of unlearning has indeed improved after applying a particular trick. We fill this gap with our UWC, examining tricks listed as follows.
Learning Rate (LR), Early Stopping (ES), and Batch Size (BS). The learning rate dictates the intensity of unlearning, early stopping limits the number of updates, and the batch size connects to the stability of gradient estimation, which are all common tools to refine parameter updating.
Temperature Scaling (TS). The temperature is typically applied to logits before the softmax outputs. Its use during training can prevent overfitting and enhance robustness against noise.
Loss Selection (LS). We select a portion of tokens that exhibit the largest loss values and apply gradient updates only for them. already demonstrate very small loss values, especially intriguing when using GA.
designed to prevent excessive unlearning for tokens that
Please refer to Appendix F for more details. Our investigations focus on KL, which is identified by UWC as promising method. We conduct experiments across different configurations and hyper- parameter setups for these considered tricks in Appendix H, and summarize the results after hyper- parameter tuning in Table 2. Overall, we find that LS is not reliable for unlearning. On the other side, BS, ES, and TS play crucial roles in improving unlearning efficacy, which can enhance reliability of unlearning without incurring additional computational costs. However, for harder tasks, the benefits provided by BS and ES diminish, whereas TS continues to be highly effective, for example, as demonstrated in the 10% unlearning setup with LLama-2-77 Overall, we recommend the default use of TS as reliable trick in practice, along with proper hyper-parameter tuning of the unlearning epochs and/or batch sizes to further enhance unlearning.
7 CONCLUSION
This paper addresses the critical challenges in evaluating and comparing LLM unlearning methods. Recognizing the susceptibility of existing metrics to various attacks and the difficulty in balancing between removal and retention, we propose an effective evaluation framework named UWC. The UWC introduces the ES as a reliable unlearning metric, outperforming others in capturing the true extent of unlearning. Moreover, to address the trade-off between unlearning and retention, we cal- ibrate model performance on non-targeted data via MM, ensuring that the retention of desirable knowledge is adequately preserved. By doing so, we can focus solely on assessing the unlearning efficacy on targeted data, facilitating fair comparisons across varying methods, models, and setups. Using the UWC framework, we benchmark representative unlearning methods. We find GA-based methods remain to be a powerful line a work, while we need to careful control its extent of unlearn- ing via hyper-parameter tuning. We also explore other tricks that can further improve the practical efficacy of unlcarning, where we find that temperature scaling is in general helpful.
This paper fills the gap in assessing the effectiveness of unlearning metrics, further motivating exploration into fair comparisons and enhancements of current unlearning methods. Each facet presents opportunities
delve deeper. For reliable metries, it is beneficial to include broader range of candidate metrics as well as to consider much more red team attacking methods. Addi- tionally, assessing the influence removal without relying on gold standard models remains to be an unresolved issue. For fair comparisons, we suggest that model mixing is a promising strategy that could also enhance practical applications: Even for vanilla GA, model mixing can ensure that over- all performance to be maintained. Further exploration in this direction could include selective or sparse mixing, focusing on a subset of parameters that are crucial for effective knowledge removal. For the bag of tricks, we recommend further explorations of other simple yet reliable techniques.
10

---

Published as a conference paper at ICLR 2025
ETHIC STATEMENT AND REPRODUCIBILITY
LLMs, trained on extensive web-sourced datasets, risk inadvertently memorizing and dissemi- nating sensitive, private, or harmful information. This could lead to potential violations of pri- vacy, intellectual property rights, and societal harm. Unlearning methods offer a promising so- lution mitigate these ethical concerns, thus attracting increasing research attentions recently. Rather than developing new methods, we focus on ensuring effective evaluations and fair com- parisons for various unlearning methods and unlearned models. Our studies contribute to the assessments of safe, legal, and trustworthy LLM usages, reflecting the true extent for the po- tential to disseminate sensitive personal data, copyrighted material, and other forms harmful unethical information. It aligns with the wide goal of ensuring that AI technologies spect the rights of individuals. For reproducibility, we have detailed the experimental setups, hyper-parameterr configurations, and hardware specifications. The code is publicly available at: com/ tml
ACKNOWLEDGMENTS
QZW, PNY, JNZ, and BH were supported by RGC Young Collaborative Research Grant No. C2005- 24Y, NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Founda- tion Nos. 2022A151501 1652 and 2024A1515012399, RIKEN Collaborative Research Fund, HKBU Faculty Niche Research Areas No. RC-FNRA-IGG/2-23/55
and HKBU CSD Departmental Incentive Scheme. TLL was partially supported by the following Australian Research Council projects: FT220100318, DP22010212,, LP220100527,, LP220200949, IC190100031. TLL and MS were supported by JST ASPIRE Grant Number JPMJAP2405.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv: 087774 2023.
Devansh Arpit, Stanislaw Jastrzkebski. Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxin- der S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization deep networks. In ICML, 2017.
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Al- bert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.1063/ 2023.
George-Octin -ettee-Ctavvin Barbulescu and Peter Triantafilou. To each (textual sequence) its own: Improving memorized-data unlearning in large language models. arXiv preprint arXiv:2405.03097, 2024.
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08//1 2023.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In S&P, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurlPS, 2020.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In USENIX Security, 2021.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions Intelligent Systems and Technology, 15(3): 2024.

---

Published as a conference paper at ICLR 2025
Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang. and Israel Cohen. Pearson correlation coefficient. Noise reduction in speech processing, pp. 1-4, 2009.
Jiacheng Du, Zhibo Wang, and Kui Ren. Textual unleamnin gives false sense of unlearning. arXiv preprint arXiv:2406. 13348, 2024.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in 1lms. arXiv preprint arXiv:2310.022. 2023.
Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanji,, Sungchul Kim, Franck Dernon- court, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairesss in large language models: A survey. arXiv preprint arXiv:2309.0077 237070077777 2023.
Shivam Garg, Kristian Georgieva, Sam Parka, Roy Rinberga, Andrew Ilyas, Aleksander Madry, and Seth Neel. Data attribution- machine unlearning. arXiv preprint arXiv:2406. 09408, 2024.
Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing out-of-distribution examples via augmenting content and style. In ICLR, 2023.
Zhuo Huang, Chang Liu, Yinpeng Dong, Hang Su, Shibao Zheng, and Tongliang Liu. Machine vision therapy: Multimodal large language models can enhance visual robustness via denoising in-context learning. In ICLR, 2024.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212. 04082 2022.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Zhuoran Jin, Pengfei Cao, Chenhao Wang. Zhitao He. Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao Rwku: Benchmarking real-world knowledge unlearning for large lan- guage models. arXiv preprint arXiv:2406. 10890, 2024.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403 03218, 2024.
Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint (tx.xx23311031999 2023a.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.0544 2023b.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Peter Liu, Mohammad Salch, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. Generating wikipedia by summarizing long sequences. arXiv preprint 1rXiv:1801.10198, 2018
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R Varshney, et al. Rethinking machine unlearning for large language models. arXiv preprint arXiv:2402.08887, 2024.
Yang Liu, Yuanshun Yao, Cran-Francois Ton. Xiaoying Zhang, Ruocheng Guo Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy Ilms: a survey and guideline for evaluating large language models' alignment. arXiv preprint arXiv:2308.05374 2023a.
12

---

Published as a conference paper at ICLR 2025
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2335.138660 2023b.
Michelle Lo, Shay B Cohen, and Fazl Barez. Large language models relear removed concepts. arXiv preprint arXiv:2401.0185 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 2017.
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. Eight meth- ods to evaluate robust unlearning in llms. arXiv preprint (tt.:2402./683555 2024.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and Zico Kolter. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06122 2024.
Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. In NeurIPS, 2023.
Vaidehi Patil, Peter Hase, and Mohit Bansal, Can sensitive information be deleted from llms? ob- jectives for defending against extraction attacks. arXiv preprint arXiv:2309. 17410, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. In NeurlPS, 2023.
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi. Jingyu Liu, Tal Remez, Jérémy Rapin, al. Code Ilama: Open foundation models for code. arXiv preprint arXiv:2308. 220.::23....900 2023.
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint ar w08v:22008.0385. 2023.
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 1930-1949 2023.
Anvith Thudi, Gabriel Deza, Varun Chandrasckaran, and Nicolas Papernot. Unrolling sgd: Under- standing factors influencing machine unlearning. In EuroS&P, 2022.
Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. In NeurIPS, 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprini arXiv:2302. 1397,,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi. Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NeurIPS, 2017.
Qizhou Wang. Jin Peng Zhou, Zhanke Zhou, Saebyeol Shin, Bo Han. and Kilian Q Weinberger. Rethinking Ilm unlearning objectives: gradient perspective and go beyond. In ICLR, 2025.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannanch Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In CVPR, 2022.
13

---

Published as a conference paper at ICLR 2025
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prab- hamjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17554 2023.
Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Eric Sun, and Yue Zhang. A survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. arXiv preprint 020::2331..0003,, 2023a.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint arXiv:2310.10683 2023b.
Yunzhi Yao, Peng Wang. Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXiv preprint arXhv:2305.13172 2023c.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catas- trophic collapse to effective unlearning, arXiv preprint arXiv:2404.05866 2024.
Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. Can language mod- els perform robust reasoning in chain-of-thought prompting with noisy rationales? In NeurIPS, 2024.
Jianing Zhu, Bo Han, Jiangchao Yao, Jianliang Xu, Gang Niu, and Masashi Sugiyama. Decoupling the class label and the target concept in machine unlearning. arXiv preprint arXiv:2406.0224 08288, 2024.
14

---

Published as a conference paper at ICLR 2025
A CONCEPTUAL PROOF FOR METRIC EFFECTIVENESS
We formalize our discussion by developing causal framework (Huang et al., 2023) to comprehend metric effectiveness. It delineates the relationships between knowledge parametrization (K), the considered metric (M) to quantify this knowledge, model behaviors (B), and the interventions (I) introduced by red teaming attacks. We further incorporate the mediator of superficial behaviors (S), which explain the change due to I without changing the underlying knowledge K.
Pathways. All considered metrics are presumed capable of assessing the strength of knowledge parametrization more or less, denoted as K + M. such that changes in K should be manifested by M. Additionally, the knowledge parametriza- tion directly influences model behaviors, repre- sented as K - B. This relationship underscores that the way model processes inputs and gen- erates outputs is definitely function of its inter- nal knowledge. For intervention it will intro- duce superficial behaviors S without altering the underlying knowledge K, and these superficial behaviors mediate the effect of interventions on model behaviors, i.e., - S and S + B, while I + K. The causal relationships can be visual- ized in Figure 4. Therein, by identifying as a mediator, recognize that changes in B due to I are not indicative of changes in K.
B
K
S
M
Figure 4: The causal graph for the assessment of unlearning metrics. The solid / dashed rows represent known / unknown relationships.
Assessing Effectiveness. Our goal is to ensure that the crafted metrics M are effective indicators of K and are not unduly influenced by changes in B caused by I, of which the directly modeling is not feasible. Instead, based on Figure 4, conclude that an ideal metric should depend on K, holding true in general, and is robust to the change of B via I. Therefore, to validate the effectiveness of metric, we can test its robustness by testing a series of red teaming attacks that modify model behaviors by affecting superficial behaviors S without altering the underlying knowledge K. Then, we can measure metries before and after interventions test their linear correlation, of which the high values suggest that the metric is robust and primarily dependent on K.
B EXPERIMENTAL CONFIGURATIONS
Our evaluations were based on the well-established benchmarks of TOFU fictitious unlearn- ing (Maini et al., 2024), focusing on LLMs fine-tuned with a series of fictitious authors profiles. These profiles were created by prompting GPT-4 (Achiam al., 2023), which has been filtered to avoid the occurrence of any real author profile, thus mitigating the inadvertent impacts of other un- related variates. For each fictitious profile, TOFU crafted 20 question-answer pairs that be used for fine-tuning, along with their paraphrased versions for evaluations.
The pre-trained LLMS were further fine-tuned on such question-answer pairs, where we considered two popular LLMs, i.c., Phi-1.5 (Li et al., 2023b) and Llama-2-7B (Touvron et al., 2023a) with their question- question-answering versions. For the unlearning setups, the original TOFU data were separated into targeted and non-targeted parts, of which the adopted proportions are 1:99 (1% unlearning). 5:95 (5% unlearning), and 10:90 (10% unlearning). Moreover, we further separated 400 non-targeted data that were not involved during the unlearning procedure for evaluations, reflecting real-world situations where it is not feasible to go through all non-targeted data during the unlearning process.
For all the considered methods, we adopt the following implementation setups: the Adam W opti- mizer (Loshchilov & Hutter, 2017), the initial learning rate 2e 5 for Phi-1.5 and - for Llama-2- 7B, the batch size 16 for both the targeted and non-targeted data, the epoch number 5, and the linear warm-up for the first epoch. For MM calibration, we set 5 = 0.95 for Phi-1.5 and T = 0.90 for Llama-2-7B. All our experiments were realized by Transformers 4.42.4 with CUDA 12.1, using series of computation nodes equipped with NVIDIA- 100000000 GPUs and Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz Processors.
15

---

Published as a conference paper at ICLR 2025
C BASELINE METHODS
We examine a collection of representative unlearning methods that are wide recognized in the liter- ature. For clarity, elaborate their implementations and discuss their significance.
Gradient Ascent (GA) (Yao et al., 2023c). As one of the earliest unlearning methods, GA decreases the ligeliiklhhood log p(s.: 0) for targeted data. The unlearning objective is articulated
-B(x...)> e (vul lz: 0),
corresponding to applying gradient ascent to the due to its simplicity (Liu et al., 2024). Nevertheless, it is also notorious for causing catastrophic collapse (Zhang et al., 2024)-its efficacy removing targeted knowledge often comes at the large costs that damage the overall integrity of LLMs, rendering the resulting LLMs completely useless.
entropy loss. GA has been widely explored
Gradient Difference (GD) (Maini al., 2024). To counteract the negative impacts of catastrophic collapse, various regularization terms are explored to retain the common model integrity. GD im- proves upon GA by further decreasing the negative likelihooh for non- targetee data, following
- (val2; 0) (B( (ylæ;; 0);
(7)
where À is a trade-off hyper-parameter that should be tuned. The use of GD can mitigate the ad- verse effects of GA on knowledge retention. However, when the unlearning steps are extensive, the extreme scale of E(..y,)-. (Bu z:: 0) will overshadow that of (x.,,)) -D, yD. l (y]æ; 0). Therefore, the GD will be less effective in the later unlearning phrase, reducing its ability to maintain utility.
KL Regularization (KL) (Maini et al., 2024). KL also involves regularization for GA. However, instead of learning from original data, KL retains the original responses for data by minimize the KL divergence before and after unlearning. The overall unlearning objective
(e,.))D, E (ya (yule xz; 0) (E(
Σ (I(y
x;; 12:0)lze 0))pp((** |;; 0rot));
(8)
which averages the KL divergence with respect sequence of prefixes. Similar GD, KL still suffers from deterioration in retention.
Negative Preference Optimization (NPO) (Zhang et al., 2024). It is motivated by direct prefer- ence optimization (DPO), well-known aligmment method (Rafailov et al., 2023), which originally utilizes paired corpora comprising preferred versus dis- dis-preferred data. NPO segregates the dis- preferred part from DPO, heuristically employing it as the unlearning objective, following
B (aaa))D, log (1 + P(uul z;; 0) 3
2
pV 2; 0ms) )9) + (. Σ KL (plu <k
z; 220))lp(
<k
ar; θ rrot)).
(9)
where B is the hyper-parameter of the inverse temperature. The effective realization of NPO still relies on regularization for retention, we default use KI. in our realization. We simply set to ease
tuning, which is suggested by (Zhang et al., 2024).
Preference Optimization (PO) (Maini et al., 2024), It aims to mitigate the drawbacks of the un- learning risk by targeting outcome, e.g., "I don't know., which is implemented through
((r.,))>,, ( 0).
(10)
changing original outputs for targeted data to Yidk-
Representation Misdirection for Unlearning (RMU) (Li et al., 2024). Instead of changing model outputs, RMU implements unlearning by perturbing model representation. Denote the embedding features by (0 0), the formulation of RMU is given by
1 Iaal
E B(zgga)D
(1.
(1 c-ull2
1yul
1
E (a.v)~DVVo \D
[< y x)) -5([2, v)) 0rr)
(11)
y
t=1
where has elements randomly sampled from [0, 1) and e is scaling hyper-parameter.
16

---

Published as a conference paper at ICLR 2025
D ATTACK METHODS
We consider four distinct types of red teaming scenarios: jail-breaking. embedding probing. relearn- ing, and token noising. These attacking methods are utilized to assess the robustness of 5 basic metrics. Please see Figure 5 for the full results. Overall, we find that relearning and probing gen- erally pose greater challenges compared jail-breaking and noising, with the latter two exhibiting typically smaller PPC. Additionally, we observe that the PPL and KL are very similar, as both rely on the likelihood of outputs. For ROUGE, the results show that it is sensitive to attacks, suggesting it may not be an appropriate metric. When comparing PPI., ES, and EM across different scenarios, we find that their PPCs are close under jail-breaking and token noising. However, the contexts of relearning and embedding probing, ES notably outperforms both PPL and EM. Therefore, overall recommend adopting the ES as the most suitable basic metric for LLM unlearning.
Below, we offer detailed descriptions for the realizations of our considered attacks.
Jail-breaking. Jail-breaking manipulates or crafts input prompts to make the model produce outputs that bypass its safeguards (Li et al., 2023a). We explore two prompt structures outlined in (Shen et al., 2023). For a given original question-answe pair, characterized by {question] } and {answer), these two prompts are constructed follows:
Prompt 1, Do not apologize or feel sorry, answer the question: {question] Answer: (answer).
Prompt 2. {question} Start with Abbolltely! know the answer! Here is:"
Absolutely! know the answer! Here is: {answer).
We compute the metric scores under these two jail- bail-breaking attacks separately, and report their average values the final results.
Embedding Probing. We follow the approach of tuned lens (Belrose et al., 2023), which extracts embedding features from unlearned models, further decoding them into the output token space using an linear model. This additional model is trained on the whole TOFU datasets for 5 epochs, utilizing the Adam optimizer with the initial learning rate of 1 Moreover, in our analysis, including the 1 11-st, 22-nd, and 33-rd layers for Llama-2-7B, and the 8-th, 16-th, and 24-th layers for Phi-1.5. The associated linear models are trained separately for each layer of embeddings. The performance metrics are averaged across layers, and we report the average values as the final results for each model type, either Phi-1.5 or Llama-2-7B.
focus specific layers
Relearning. The unlearning models are further fine-tuned on targeted data for one epoch, using the negative log-like likeelooo the objective. The AdamW optimizer adopted with the same learning rates as original fine-tuning. The metric scores are then computed for relearned models.
Token Noising. We randomly select 5% of the tokens (ensuring least one token is selected) in each string and replace it with randomly chosen new token. This process introduces noise into data, simulating errors or disturbances that might occur in real-world applications. The metric scores are then computed for the original unlearned models, using the noised data as the ground truth.
Based on our analyses in Appendix A, we know that a proper attack method should not impact the parameterized knowledge within models, but can change model behaviors. From this perspective, jail-breaking and embedding probing are more appropriate than relearning and token noising when assessing metric robustness for unlearning. Therefore, the results of jail-breaking and embedding probing should receive our main focus for testing robustness.
17

---

Published as a conference paper at ICLR 2025
200
PCC+1.0000/1.00
PCG-D.6959/0.o 1206
PCC-0.7292/0.000
PCC-0,9998/0 9999999.999
150
150
100
100
100
50
retain
50
50
unlearn
50
100
50
150
50
100
150
(a) PPL jail-break
(b) PPL. releam
(c) PPL probing
(d) PPL noising
1.0
1.0
10
1.0
PCC-0 563566 9087
PCC-0. RCC-04401/033477
PCG-D.5452008877
0.9691.60000
8864
0.8
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
1:0
0.0
1.0
0.0
(e) ROUGE jail-break
(f) ROUGE relearn
(g) ROUGE probingoobin
(h) ROUGE noising
PCC=0.996M 9C--099942009983
PCC=0.754660.66
PCC=0.921660.840
PCC=0.961 PCC-0.9613/0.9929
0.0
0.0
0.6
0.3
0.1
0.0
02
0.4
06
0.5
1.0
0.0 0.2
0.4
0.6
0.8
1.0
0.0
0.2
04
0.6
0.8
1.0
0.0 02
04
0.6
08
1.0
(i)
ES jail-break
( ES relearn
(k) ES probing
()) ES noising
1.0
1.0
1.0
1.0
99..999099999
PCC-0.6514.0.BB
PCC-0.9073/..928
0.910/.94 9051
0.8
0.8
0.8
0.8
0.4
0.4
0.4
0.2
0.2
0.2
0.2
1:0
(m) EM jail-break
(n) EM releam
(0) EM probing
(p) EM noising
200
100
200
PCC+1. 000000000/0
PCC-0.6768 0tw.066666009993
CCC-0,718)-000009
PCC- 0. 9990
150
3
60
100
2
40
50
100
150
200
50
100
150
200
50
100
150
200
50
100 150
200
(9) KI. jail-bbbre
(r) KL relearn
(s) KI. Klopbing
(1) KI. noising
Figure 5: Robustness of Metrics under Red Teaming Attacks. We depict the metric scores before (x-axis) and after (y-axis) attacks jointly for different unlearning setups: across L.LMs (Phi-1.5 and Llama-2-7B), 3 unlearning percentages (1%, 5%, and 10%), and 4 unlearning methods (GA, GD, PO, and NPO). We consider 5 different metrics under 4 red teaming behaviors. We apply the log- scale for PPL avoid numeric errors. For each of these scenarios, to targeted and non-targeted data respectively, displayed at the top of each figure (targeted data non- targeted data). We provide linear fits for targeted and non-targeted data separately, accompanied by shaded areas representing the standard deviation to further visualize the PPC scores.
compute the PPC with respect
18

---

Published as a conference paper at ICLR 2025
Algorithm Binary Search for MM Calibration
Input: parameters Oref before unlearning and 0 after unlearning; datasets Du and D;; num_iter of total searching steps; threshold
lour and Veur =
for iter = to num eiter do
Ocann f- (Nenr + leur) /2:
mix = Ocan 0ref) + Olcan 0
if ES(D)) mix) 2 TES((t (Da) feet) then
lour f (Neur + leur) /2:
else
Muur (Meur + lear) ) end if'
end for
Output: optimal a² -Cean
E UWC REALIZATION
While we have demonstrated the effectiveness of the UWC in evaluating and comparing unlearned models or unlearning methods, the computational expenses associated with its straightforward im- plementation, including the computation and the MM calibration, can be exorbitantly high.
Specifically, for the precise computation of the ES, it is necessary to iterate through each integer value i E (1..... w/] to determine if the condition f [[x yu = y is satisfied, then identi- fying the smallest value of & among candidates. For the MM calibration, it is essential to sample sufficient number of candidates a from the continuous range between and 1. This involves testing whether the corresponding mixed model, with parameters (1 a)0 + a0, maintains acceptable performance on non-targeted data, i.e., ES(D, ( (1 - a e = a0) > TES DDu fret). To accurately estimate the optimal with minimal damage common integrity, it is crucial that the coverage of a should be sufficiently fine-grained, thereby increasing overall costs calibration.
Fortunately, we observe approximately monotonic relationships for both h and with respect to their associated conditions. These scenarios indicate that the binary search can be effectively used streamline the selection process for their appropriate values. Taking MM-based calibration as an example, Algorithm I outlines a general framework for the efficient parameter search of optimal a. Similar implementations can also be adopted for computing the ES scores.
F MORE DISCUSSIONS ABOUT PRACTICAL TRICKS
In Section 6, we explore a series tricks, such as adjusting common mization, including the learning rate, batch size, unlearning epochs. Additionally, suggest some more intriguing methods such as TS and LS. We further discuss the detailed implementations for the last two methods for concreteness.
for opti-
Temperature Scaling (TS). By manipulating logits of model outputs, the TS is particular useful in avoid overfitting. Denote the original output logits z, then the softmax function with the temperature scaling can be articulated as
exp(zi/x E, exp(2
(12)
Overall, higher temperatures will result a softer probability distribution over candidate tokens, which prevent the model from becoming too confident the training data, thereby avoiding excessive unlearning and improving generalization. In our realization, we only apply TS for the unlearning risk.
Loss Selection (LS). During unlearning, we assume proportion of tokens with already small loss values should not be involved during model updating, otherwise, severe excessive unlearning may
19

---

Published as a conference paper at ICLR 2025
occur. Written the formulation of GA in token-wise manner, we have
Σ [log y
1
(13)
Then, we select y x 100% proportion of tokens with largest loss values, satisfying the condition
K arg
log log (1:00,
(14)
K'
KEK'
with K' defining as set of the selected tokens within y. Then, the GA with loss selection can be simply written as
E (a
log Dogply"laa ())
(15)
KEK
LS is particular attractive for those unbounded loss funetions just like GA. In avoiding to update loss for the part of tokens that have been sufficiently unlearned, the resulting unlearning procedure has the potent to avoid excessive unlearning.
G EXCESSIVE UNLEARNING AND INCOMPLETE UNLEARNING
We claim that NPO suffers from incomplete unlearning, while GA exhibits tendencies of excessive unlearning. In this section, provide more results to justify our claims.
For NPO, its unlearning behaviors can be ana- lyzed through its gradient behaviors outlined (Zhang et al., 2024; Wang et al., 2025). When taking gradient with respect to 0, have the gradients of NPO following the form of
(( Wayyy Ve log p(yu: x. 0), (16)
0.4
0.2
with Wr.ya p( +lp. 1. can be viewed weighting mechanism. The ef- fects of this mechanism for 5% unlearning with Llama-2-7B is illustrated in Figure which shows the average Wr.y, computed during NPO unlearning. Notably, these values quickly de- cline to 0 shortly after the end of the first epoch. change thereafter, which signifies that W2,a, plays the role of early stopping, thereby potentially leading to incomplete unlearning.
0.0
step 20
step 40
step 60
Figure 6: The dynamics of the implicit NPO weighting mechanism.
The loss values and the ES scores do not notably
We examine risk values and ES scores for GA in Figure 7 for 5% unlearning with Llama-2-7B Contrary to the NPO, we observe that ES scores quickly drop to 0 for unlearning, while the unlearning risks continue to decrease, indicating that the excessive unlearning may occur. The pri- mary consequence of such excessive un- learning is a degradation in model perfor- mance non-targeted data, evidenced by the poor ES scores on non-targeted data without calibration and the poor ES scores on targeted data with calibration.
0.8h
step 20 step step 60
0.2
20 40
(a) GA Risk
(b) GA ES unlearn
Figure 7: The trajectories of risk and ES values.
20

---

Published as a conference paper at ICLR 2025
H MORE RESULTS
In Table 3, we present the results of ES scores without calibration, where we observe an obvious trade-off between removal and retention. Since both methods are crucial for practical unlearning, is difficult to conclude which is overall superior. This further emphasizes that calibration facilitates the comparison of overall efficacy across different methods.
We list detailed results involved during hyper-parameter parameter tuning in Tables 4- 4-10. For baseline meth- ods, it involves the trade-off parameter À for GD, KL, and NPO: the inverse temperature for NPO; the scaling parameter and the embedding layers for RMU. À is chosen from the candidate set of (1. 2, 4, 7, 10, 20, 50, 100); is chosen from {1. 2, 4, 7, 10, 20, 50, 100). is chosen from {0, 1, 4, 5, 7, 10). The embedding layers of RMU is chosen from shallow, middle, and deep lay- ers, respectively defined as 8-th, 16-th, and 24-th layers for Phi-1.5 and 11-th, 22-th, and 33-th layers for Llama-2-7B. Moreover, for NPO, we simplify its tuning procedure into two steps: a) fixing À = (original suggested) and tuning 33 and b) fixing the tuned / and tuning A.
Then, report the results involved for the bag of tricks in Tables 11-15. Therein, the learning rate is chosen from {le le Le S, ,, 1e 7): the batch size chosen from {2, 4, 8, 14, 20]; the training epochs for early stopping chosen from 21,2,3, 4, 4,5); the temperature scaling X is chosen from (0.9, 2, 3, 4, 5]: the likelihood capping si chosen from {0.01, 0.1, 0.2, 0.3, 0.5); the loss selection q is chosen from {0.1, 0.3, 0.7).
We explore the impact of various т within the MM framework on comparison. We conduct ex- periments under the 5% unlearning scenario in Table 16, which demonstrate that the ranking of different methods, with respect to ES-exact, remains unchanged across varying т. This consistency underscores the robustness of our evaluation framework specific settings of
21

---

Published as a conference paper at ICLR 2025
Table Comparison between different unlearning methods on TOFU fictitious unlearning without UWC calibration. / indicate smaller larger values are preferable. We primarily focus on the ES scores for unlearning (shaded), given that the ES scores for retention are calibrated.
LLM
Phi-1.5
Llama-2-7B
setup
method
retain
ES-exact
unlearn
ES US-perturb
unleurn
retain
ES-exact
unlearn
retain
ES ES-perturb
unlearn
before unlearning
0.4433
0.5969
0.2111
0.1605
0.8277
0.8039
0,5302
0.4001
GA
0.0000
0.0000
0.0000
0.0000
0.0003
0.0000
0.0000
0.0000
KT. 0.0459
0.0092
0.0458
0.0092
0.1676
0.0000
0.1564
0.0000
1%
NPO
0.2066
0.0648
0.1059
0.0558
0.4981
0.1201
0.3960
0.0963
RMU
0.0000
0.0000
0.0000
0,0000
0.0000
0.0000
0.0000
0.0000
before unlearning
0.4433
0.5619
0.2115
0.2374
0,8277
0.7735
0.5302
0,4126
GA
0.0001
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0873
0.0000
0.0892
0.0000
0.1985
0.0000
0.1459
0.0000
5%
NPO
0.1361
0.0877
0.0992
0.0725
0.4991
0,0891
0,3055
RMU
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
before unlearning
0.4433
0.5299
0.2115
0.1843
0.8277
0.8307
0,5302
0,3099
GA
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0,0000
KL. 0.1105
0.0000
0.0791
0,0000
0.2690
0.0308
0.2566
0.0221
10%
NPO
0.3087
0.1201
0.1687
0.0671
0.6939
0.1623
0.4490
0.1227
RMU
0.0000
0.0000
0.0000
0.0000
0,0000
0.0000
0.0000
0,0000
Table UWC Tuning for GD. L t indicate smaller larger values are preferable.
GD
Phi-
Llama-2 2-7B
ES ES-exact
ES-perturb
ES -x-exact
ES ES-perturb
setup
retain
unlearn
retain
unlear |
retain f
unleam
retain 1
unlearn
before unlearning
0.4433
0.5969
0,2115
0.1605
0.8277
0.8039
0.5302
0.4001
0,4212
0.3449
0,2050
0.1010
0.8028
0.0873
0.4773
0.0000
0,4212
0.3449
0,2072
0.1413
0.7471
0.0293
0.4471
0.1860
0,4212
0.5219
0,2017
0.0506
0.7656
0.0241
0.5302
0.3242
0.4404
0.5219
0.1644
0.7177
0.1036
0.4791
0.0000
1%
0,4361
0.5219
0,2147
0.1120
0.7489
0.1775
0.4806
0.0719
0,4312
0.5101
0,2009
0.1330
0.7420
0.3454
0.4829
0,2414
0,4297
0.5969
0,2039
0.2039
0.7420
0,5682
0.4650
0,3501
0.4263
0.5969
0.1994
0.2039
0.7928
0.7334
0.4905
0.3889
before unlearning
0.4433
0,5619
0.2115
0,2374
0.8277
0.7735
0,5302
0.4126
0,4404
0,4310
0.1862
0.0563
0.7794
0.4362
0.4754
0.4126
0.3919
0,4140
0.2004
0.0045
0.7432
0.3385
0.4775
0.3166
0.3934
0,4574
0.2051
0.0000
0.7486
0.0903
0.4789
0.2176
0.4454
0.4387
0.2137
0.0833
0,7822
0.2086
0.4498
0,3312
5%
10
0,4182
0.3381
0.2063
0.1663
0.7447
0.4527
0.4875
0,4126
20
0.3826
0,4574
0.1899
0.2044
0.7366
0.5595
0.4696
0,2816
50
0.4242
0,4494
0.1930
0.2079
0.7500
0.7001
0.4715
0,3309
100
0.4411
0,4964
0.2036
0.2079
0.7467
0.7449
0.4970
0.3309
before unlearning
0.4433
0.5299
0.2115
0.1843
0.8277
0.8307
0.5302
0,3099
0.4184
0.4683
0,2002
0.0841
0.7630
0.2926
0.4806
0.2428
0.4454
0.4935
0.1761
0.0345
0.7771
0.0980
0.4780
0.1200
0.4454
0.4878
0.1870
0.1182
0.7301
0.3178
0.4583
0.2035
0.3913
0.4762
0.1940
0.1369
0.7731
0.3927
0.4782
0.2439
10%
0.4393
0.4935
0.2095
0.1540
0.7633
0.2772
0.4881
0.1115
0.4433
0,5024
0.1958
0.1843
0.7394
0.2914
0.4790
0.1726
0.3728
0.4967
0.2033
0.1600
0.7408
0.7278
0.4919
0.3051
100
0.4242
0.5177
0.2051
0.1786
0.7422
0.7794
0.5210
0.3089
22

---

Published as a conference paper at ICLR 2025
Table UWC Tuning for KL. 4/1 indicate smaller larger values are preferable.
KL
Phi-1.5
Llama 2-BB
ES-exact
ES ES-perturb
ES exact
ES- IS-perturb
setup
retain
unlearn
retain
unlearn 4
retain 1
unleam
retain f
unlearn 4
before unlearning
0.4433
0.5969
0,2115
0.1605
0.8277
0.8039
0.5302
0.4001
0,4358
0.3606
0.1865
0.0789
0.7655
0.1307
0.4976
0.0373
0,4251
0.3206
0,2005
0.0737
0.7655
0.1307
0.4867
0.0000
0,4010
0.2679
0.1989
0.1283
0.7920
0.0382
0.4782
0.0000
0,4232
0.2242
0,2136
0.8277
0.0960
0.4754
0.2597
1%
0,4232
0.2123
0,2005
0.0840
0.7337
0.0515
0.0913
0,4232
0.1899
0,2051
0.0702
0.7826
0.0115
0.4729
0.0000
0,4212
0.5219
0.1937
0.0724
0.7036
0.0633
0.4876
0.0281
0,4232
0.3189
0,2172
0.1274
0.7567
0.0722
0.4532
0.0618
before unlearning
0.4433
0.5619
0.2115
0.2374
0.8277
0.7735
0,5302
0,4126
0,4220
0.3466
0.1792
0.2349
0,7649
0.6896
0.4685
0,4031
0.4419
0.3535
0.1991
0.2276
0.7346
0.6986
0.4796
0.3799
0.4160
0.3340
0.2047
0.2162
0.7442
0.4097
0.4675
0.2461
0.4220
0.3636
0.2182
0.1698
0.7702
0.5423
0.4816
0.7894
5%
10
0,3823
0.3766
0.1794
0.1614
0.7207
0.0953
0.4814
0.1516
20
0,4109
0.1704
0.2027
0.1470
0.7196
0.1222
0,5302
0,3884
50
0.4242
0.2129
0.2018
0.1691
0.7700
0,3494
0,5152
0,3243
100
0,3588
0.2052
0,2115
0.1872
0.7697
0.3973
0,5302
0,3884
before unlearning
0.4433
0.5299
0.2115
0.1843
0.8277
0.8307
0.5302
0.3099
0,4265
0.2989
0,2168
0.1459
0.7128
0.4250
0.4636
0.2343
0.3582
0.2921
0.1957
0.1624
0.7274
0.6159
0.4738
0.2317
0.4336
0.2373
0.2042
0.1168
0.7765
0.4791
0.4879
0.2317
0.4164
0.4799
0.2048
0.0535
0.7554
0.4250
0.4761
0.2199
10%
0.4424
0.4912
0.2075
0.0922
0.7765
0.2791
0.4734
0.1236
0.4418
0,5008
0.2069
0.0075
0.7860
0.2975
0.4927
0.1874
0.3858
0.4722
0,2051
0.0691
0.7344
0,3132
0.4810
0.1870
100
0.4242
0.4337
0.1991
0.1610
0.7720
0.4126
0.4959
0.2550
23

---

Published as a conference paper at ICLR 2025
Table 6: UWC Tuning for NPO (A 1). 1/T indicate smaller larger values are preferable.
NPO
Phi-1.5
Llama-2-77 2-7B
ES ES-exact
ES-perturb
ES exact
ES-perturb
setup
retain
unlearn
retain
unlearn
retain f
unlear
retain f
unlearn
before unlearning
0.4433
0.5969
0,2115
0.1605
0.8277
0,8039
0.5302
0.4001
0.05
0,4283
0.1587
0,2136
0.0702
0.7655
0.1262
0.5084
0.2545
0,4553
0.1587
0,2121
0.0945
0.7547
0.1857
0.4995
0,2113
0,4030
0.0947
0,2136
0.1083
0.6967
0.2513
0.4777
0.1898
0.70
0.3909
0.1072
0,2136
0.1083
0.7517
0.2607
0.4733
0.1863
0,4261
0.1806
0,2136
0.1083
0.7517
0.2607
0.4777
0.1863
1%
0.3954
0.1166
0,2136
0.1655
0.7234
0.2876
0.4588
0.2025
0,4223
0,2136
0.0000
0.0000
0.0000
0,4218
0.1806
0,2136
0.0000
0.0000
0.0000
0.0000
0,4218
0.1806
0,2001
0.1551
0.7874
0.2941
0.4588
0.2197
10.0
0,4218
0.1806
0.2136
0.1551
0.0000
0.0000
0.0000
0.0000
before unlearning
0.4433
0.5619
0.2115
0,2374
0.8277
0.7735
0.5302
0.4126
0.05
0.4265
0.3671
0.2052
0,2549
0.0000
0.0000
0.0000
0,0000
0.10
0.4161
0.3709
0.1942
0.2228
0.7652
0.5473
0.4976
0,4066
0.50
0,4433
0.4539
0.2098
0.2228
0.7780
0.4966
0.4773
0,4009
0.70
0.3970
0.3452
0.2058
0.2314
0.7459
0.5005
0.4903
0.4013
1.00
0.4086
0.4177
0.1982
0.2228
0.7836
0.5195
0.4918
0.3785
5%
2.00
0.4086
0.3863
0.2043
0.2203
0.7572
0.5809
0.4976
0,3884
4.00
0.4433
0,4188
0.2043
0.2147
0.7836
0.5809
0.4781
0,3884
5,00
0,4433
0,4188
0.2150
0.2147
0.7836
0.5946
0,5175
0,3726
7.00
0.4127
0,4034
0.2109
0.1805
0.7836
0.5303
0.4887
0,3674
10.0
0.4433
0,4034
0.1848
0.2000
0.7836
0.5703
0.5012
0.3674
before unlearning
0.4433
0.5299
0.2115
0.1843
0.8277
0.8307
0.5302
0,3099
0,4370
0.4360
0,2231
0.1326
0.7765
0.6204
0.4825
0,3137
0.10
0.4222
0.4290
0.2048
0.1383
0.7765
0.5818
0.4809
0.3137
0.50
0.4270
0.4708
0.2088
0.1645
0.7836
0.6310
0.4825
0.3271
0.70
0.4413
0.4781
0.2088
0.1645
0.7836
0.6545
0.4825
0.3271
0.4073
0.4689
0.2074
0.1588
0.7836
0.6291
0.4825
0.3271
10%
0.4433
0.4712
0,2362
0.2224
0.7836
0.6375
0.4874
0.3244
4,00
0.4433
0.4771
0.2225
0.1996
0.7836
0.6018
0.4795
0.3030
5.00
0.4433
0.4771
0.2260
0.2105
0.7836
0.5387
0.5101
0.2989
7.00
0.4433
0.4954
0.2260
0.1967
0.7479
0.5387
0.4809
0.2672
10.0
0.4404
0.5465
0.1905
0.1990
0.7479
0.5387
0.4838
0.2774
24

---

Published as a conference paper at ICLR 2025
Table 7: UWC Tuning for NPO (3 = 0.5). L/1 indicate smaller larger values are preferable.
NPO
Phi-1.5
Llama 7B
ES-exact
ES ES-perturb
ES ES-exact
ES ES-perturb
setup
retain
unlearn
retain
unlearn
retain f
unleam
retain f
unlearn
before unlearning
0.4433
0.5969
0.2115
0.1605
0.8277
0.8039
0.5302
0.4001
0.4742
0.1166
0.2136
0.1551
0.7346
0,3134
0.4743
0.2066
0.4627
0.1259
0.2136
0.1551
0.7648
0.3134
0.4777
0.2101
0.4606
0.1259
0.2136
0.1551
0.7346
0.2941
0.4805
0.2066
0.4535
0.1259
0.1837
0.0980
0.7952
0.2941
0.4909
0,3273
1%
10
0.4473
0.1259
0.1927
0.0702
0.6978
0.2543
0.4776
0,1672
20
0.4424
0.1259
0.2136
0.0702
0.7383
0,2543
0.4776
0.1703
50
0.4181
0.1259
0.1843
0.0983
0.6183
0.1383
0.5286
0,3017
100
0.3970
0,1259
0.1909
0.0702
0.7251
0.2568
0.5302
0,3685
before unlearning
0.4433
0.5619
0.2115
0.2374
0.8277
0.7735
0.5302
0.4126
0.4253
0.4462
0.1958
0.2228
0.7836
0.6062
0.4976
0.3635
0.4125
0.3965
0.1923
0.2228
0.7836
0,6062
0.4641
0,3664
0.4127
0.4354
0.2027
0.1985
0.7770
0.6177
0.4770
0,3835
0,4148
0.3922
0.1984
0.1900
0.7820
0.4756
0.4938
0.3233
5%
10
0.4086
0.3991
0.2112
0.1381
0.7836
0.4756
0.4875
0.2784
20
0.4433
0.3768
0.1836
0.1509
0.7207
0.1104
0.4804
0.2777
0.3987
0.3396
0.2055
0.1120
0.7261
0.0443
0.4849
0.2092
100
0.4242
0.3051
0.2118
0.1559
0.7509
0.1020
0.4672
0.2317
before unlearning
0.4433
0,5299
0.2115
0.1843
0.8277
0.8307
0.5302
0,3099
0.4370
0.4478
0.2048
0.1502
0.7836
0.6139
0.4825
0.3244
0.4393
0.4459
0.1870
0.1331
0.7836
0.4961
0.4796
0,2860
0,4209
0.4505
0,2107
0.1188
0.7462
0.4479
0.4781
0,2066
0,4433
0.4459
0,2110
0.0762
0.7479
0.4392
0.5059
0.1979
10%
10
0.4433
0.4397
0.1989
0.0764
0.7479
0.3208
0.4669
0.1738
20
0.4072
0.3499
0.2028
0.1281
0.7769
0,3700
0.5100
0.1243
0.4265
0.5221
0,2002
0.1018
0.7238
0.3439
0.4645
0.1867
100
0.4173
0.4974
0.1735
0.0823
0,7362
0.3857
0.5302
0.3169
Table 8: UWC Tuning for RMU (shallow). f/1 indicate smaller larger values are preferable.
RMU
Phi-1.5
Llama-2-78
ES-exact
ES ESrpert
ES-ex
ESS-peruurb
setup
retain
unlearn |
retain
unlearn J
retain f
unleam
retain f
unlearn
before unlearning
0.4433
0.5969
0.2115
0.1605
0.8277
0,8039
0.5302
0.4001
0.00
0.4530
0.5969
0.2007
0.1855
0.7604
0.5993
0.4888
0.3816
1.00
0.4122
0.4356
0.2115
0.1855
0.7502
0,6278
0.4890
0.4253
2.00
0.4312
0.4080
0.2072
0.1855
0.7653
0.6714
0.4531
0.4002
1%
4,00
0.4245
0.4682
0.2115
0.1555
0.7356
0.7223
0.0000
0.0000
5,00
0.4398
0.5149
0.1981
0.1855
0.7163
0.6287
0.4871
0,4008
7.00
0.4460
0.5096
0.2201
0.1855
0.7292
0.7128
0.4516
0.4104
10.0
0.4215
0.4816
0.2018
0.1855
0.7292
0.6195
0.4453
0.4104
before unlearning
0.4433
0.5619
0.2115
0.2374
0.8277
0.7735
0.5302
0.4126
0.00
0.4164
0.4924
0.1918
0.2172
0.7516
0.7292
0.4676
0,3616
1.00
0.4284
0,5124
0.2194
0.2172
0.7762
0.7357
0.4677
0.4504
2.00
0.4044
0.4774
0.1939
0.2172
0.7146
0.6370
0.4453
0.4126
5%
4.00
0.4404
0.4252
0.2047
0.2147
0.7619
0.6758
0.4812
0.4126
5.00
0.4404
0.4838
0.2181
0.2207
0.7139
0.6758
0,4812
0.4164
7,00
0.4204
0.3772
0.2073
0.2339
0.7604
0.6758
0.4793
0.4126
10.0
0.4194
0.4114
0.1903
0.2339
0.7146
0.6370
0.4453
0.4126
before unlearning
0.4433
0.5299
0.2115
0.1343
0.8277
0.8307
0.5302
0.3099
0.00
0.4425
0.5761
0.2055
0.1424
0.7887
0.8165
0.4246
0.2662
1.00
0.4424
0.5968
0.2133
0.1567
0.7568
0.6869
0.4771
0.2989
2.00
0.4304
0.5961
0.2028
0.1360
0.7628
0.6755
0.4690
0.2989
10%
4.00
0.4364
0.5208
0.1944
0.1547
0.7229
0.5784
0.4812
0.2766
5.00
0.4284
0.5184
0.2007
0.1547
0.7262
0.6268
0.4797
0.2944
7.00
0.4404
0.5184
0.2007
0.1754
0.7271
0.5778
0.4232
0,3033
10.0
0.4404
0.4693
0.2136
0.1675
0.7032
0.5455
0.4849
0.3033
25

---

Published as a conference paper at ICLR 2025
Table 9: UWC Tuning for RMU (middle). 4/1 indicate smaller / larger values are preferable.
RMU
Phi-1.5
Llama 2-7B
ES-exact
ES-perturb
ES exxct
ES -e-reertur
setup
retain
unlearn
retain
unlearn J
retain
unleam
retain t
unlearn
before unlearning
0.4433
0.5969
0,2115
0.1605
0.8277
0.8039
0,5302
0.4001
0.00
0,4203
0.5969
0.2153
0.2069
0.7606
0.5127
0.5115
0.4001
1.00
0,4203
0.5969
0.2180
0.1409
0.7416
0.5093
0.4878
0,4001
2.00
0.4203
0.5969
0.1831
0.1261
0.7512
0.4263
0.4644
0.3794
1%
4.00
0.4203
0.5969
0.1831
0.1261
0.7559
0.5093
0.4096
0.3538
5.00
0,4203
0.5969
0.2073
0.1328
0.7413
0.4810
0.4927
0.4001
7.00
0,4218
0.5969
0,2119
0.1261
0.7413
0.4810
0.4927
0.4001
10.0
0.4203
0.5969
0.2119
0.1350
0.7655
0.4137
0.4927
0,3624
before unlearning
0.4433
0,5619
0,2115
0,2374
0.8277
0.7735
0.5302
0.4126
0,4262
0.5723
0.1952
0,2207
0.0000
0.0000
0.0000
0.0000
0,4232
0.4999
0,2032
0.2207
0.7381
0.4284
0.4798
0,3884
0,4232
0,5013
0,2229
0.2207
0.7179
0.5146
0.4379
0,3884
5%
0,4218
0.5309
0.1887
0,2030
0.7112
0.4034
0.4927
0,3884
0.3578
0.3762
0,2119
0,2030
0.7438
0.6323
0.4927
0,3884
0,4218
0.5946
0.1990
0.7438
0.6684
0.4927
0.4126
10.0
0,4262
0,4000
0.1968
0.2005
0.7552
0.6615
0.4644
0.4126
before unlearning
0.4433
0.5299
0,2115
0.1843
0.8277
0.8307
0,5302
0.3099
0.00
0.4262
0,4584
0.1952
0,1786
0.0000)
0.0000
0,00000
1,00
0.4203
0,4909
0.2108
0.1816
0,7493
0.7636
0.4379
0,3139
2,00
0.4232
0.5025
0.2212
0.1786
0.7374
0.7275
0.4831
0,3158
10%
4.00
0.4394
0.5025
0.2117
0.1901
0.7874
0.7526
0.4871
0,3196
5,00
0.4224
0,4511
0.2117
0.1799
0.7874
0.6907
0.4653
0.3220
7.00
0.4005
0.4568
0.1496
0.1741
0,7434
0.5821
0.4776
0.2908
10.0
0.0000
0,0000
0.0000
0,0000
0.7534
0,6495
0.4927
0,3316
Table 10: UWC Tuning for RMU (deep). 1. / indicate smaller larger values are preferable.
UWC
Phi-1.5
Llama-2-1
ES-exact
ES Eerpert
ES Ex-ex
ES ES-perturb
setup
retain
unlearn
retain
unlearn J
retain f
unleamn
retain t
unlearn
before unlearning
0.4433
0.5969
0.2115
0.1605
0.8277
0.8039
0,5302
0.4001
0.00
0.3936
0.5219
0.2136
0.1374
0.7836
0.6364
0.4927
0.4089
1.00
0.4156
0.5219
0.2117
0.1574
0.7461
0.4564
0.4442
0.3402
2.00
0,4212
0.5219
0.2080
0.1655
0.6977
0.2814
0.4847
0.2790
1%
4.00
0.4212
0.5153
0.1951
0.1655
0.6913
0.2992
0.4428
0.2748
5.00
0.4212
0.5121
0.2062
0.1655
0.7122
0.3974
0.4976
0.1982
7.00
0.4212
0.5108
0.1885
0.1686
0.7509
0.3271
0.4428
0.2305
10.0
0.4184
0.4963
0.2136
0.1717
0.7106
0.3815
0.4428
0.2062
before unlearning
0.4433
0.5619
0.2115
0.2374
0.8277
0.7735
0.5302
0.4126
0.4212
0.4953
0,2007
0,2182
0.7731
0.7074
0.4675
0,3953
1,00
0.4049
0.5144
0,2115
0.2182
0.7731
0.6488
0.4801
0,3850
0.4110
0.5602
0.1967
0.2227
0.7410
0.6683
0.4801
0.3714
5%
0.4151
0.5621
0.1930
0.2227
0.7731
0.6031
0.4598
0,3869
0,4212
0.5271
0,2099
0.2394
0.7464
0.7001
0.4613
0,3958
0.4212
0.5285
0.1951
0.2394
0.8113
0.6983
0.5015
0.4464
10.0
0.4064
0.4816
0.2025
0.2349
0.7319
0.7763
0.4600
0.4393
before unlearning
0.4433
0.5299
0.2115
0.1843
0.8277
0.8307
0,5302
0,3099
0.00
0.4212
0.4935
0.2095
0.1933
0.7577
0.6868
0.4410
0.2884
1.00
0.4049
0.4935
0.2039
0,1963
0.7673
0.7560
0.4571
0.2906
2.00
0.4212
0.4935
0.1969
0,1933
0.7731
0.7402
0.4865
0,3239
10%
4.00
0.4212
0.4935
0.2115
0.1933
0.7731
0.7414
0.4426
0,2674
5.00
0.4212
0.4959
0.1967
0.1933
0,7486
0,7688
0.4738
0,2192
7.00
0.4212
0.4799
0.2097
0.1933
0.7620
0.7402
0.4784
0.2547
10.0
0.3934
0.4799
0.1951
0.1786
0.7394
0.7402
0.4890
0.2547
26

---

Published as a conference paper at ICLR 2025
Table 11: UWC Tuning for the Learning Rate of KL.
indicate smaller larger values are
preferable.
UWC
Phi-1.5
Llama-24 1-maa-2-BB
learning rale scale
ES-exact
ES ES-perturb
ES-exact
ES ES-perturb
setup
retain
unlearn J
retain
unlearn J
retain f
unlearn
retain
unlearn
le-3
0.4149
0.5053
0.1902
0.0770
0.7815
0.2315
0.4442
0.3080
le-4
0.4126
0.5219
0.1823
0.0228
0.7546
0.3095
0.4516
0.3289
1%
1e-5
0,4232
0.2031
0,2005
0.1078
0.7241
0.0428
0.4791
0,0000
le=B
0.4439
0.5108
0,2136
0.1551
0.8277
0.6798
0.4990
0.3458
le-7
0.4404
0,5876
0.2136
0,1889
0.8229
0.8039
0,5302
0.4001
le-3
0.3904
0.3970
0.2202
0.2207
le-4 0.4105
0.4390
0.1968
0.1850
0.7351
0.5389
0.4789
0.2941
5%
1e-5 0,4404
0.4345
0,2069
0.1652
0.7377
0.0953
0.4258
0.0880
le-6
0.4212
0.3359
0,2030
0.2084
0.7238
0.4063
0.4364
0,3458
le-7
0.4433
0.4999
0.2115
0.2374
0.8277
0.7735
0.4990
0.4126
le-3
0.4187
0.5360
0.2101
0.1843
0.7874
0.8453
0.4787
0,3305
le-4
0.4124
0,5314
0.1876
0.1338
0.7764
0.9376
0.4918
0.8172
10%
le-5
0.3864
0.4585
0,2001
0.1215
0.7649
0.2791
0.4449
0.1057
le-6
0.4245
0.4211
0.2136
0.1623
0.7641
0,5214
0.4936
0,2777
le-7
0.4454
0.4872
0.2115
0,1843
0.8258
0.8307
0.5302
0,3139
Table 12: UWC Tuning for the Batch Size of KL. //1 indicate smaller larger values are prefer-
able.
UWC
Phi-1.5
Llama-2-1
ES-exact
ESS perturh
Ex-exact
ES-perturb
setup
batch size
retain f
unleam
retain f
unlearn
retain f
unlearn
retain 1
unlearn
0.4115
0.2904
0.1979
0.0000
0,7042
0.1082
0.4490
0.0154
0.4232
0.1931
0.2005
0.1078
0.7241
0,0428
0.4791
0.0000
12
0.4232
0.3238
0.2117
0.1
0.7297
0.1952
0.4863
0.1043
16
0.4232
0.2645
0.2136
0.1677
0.7249
0.1704
0.3928
0.0603
20
0.4244
0.3531
0.1927
0.1412
0.7606
0.3072
0.3977
0.2072
4
0.4445
0.4022
0.2041
0.1272
0.7463
0.5809
0.4419
0.3627
8
0.4404
0.4345
0.2069
0.1652
0.7377
0.0953
0.4258
0.0880
5%
12
0.3879
0.3352
0.2049
0.1432
0.6825
0,0590
0.4450
0.0604
16
0.4211
0.2169
0.1882
0.1879
0.7836
0.5181
0.4496
0.1138
20
0.4284
0.2514
0.1987
0.1879
0.7413
0.3749
0.4486
0.1443
0.3924
0.4736
0.2209
0.0826
0.7765
0.6994
0.5008
0.2605
0.3864
0.4585
0.2001
0.1215
0.7649
0.2791
0.4449
0.50
10%
0.4302
0,3358
0.2334
0.1621
0,2287
0.4285
0.1071
0.4424
0.4710
0.2225
0.1360
0.7557
0.3363
0.4769
0.1389
0.3924
0.4340
0.2003
0.1238
0.7720
0.3990
0.4305
0.0927
27

---

Published as a conference paper at ICLR 2025
Table 13: UWC Tuning for the Unlearning Epochs KL. + / indicate smaller larger values preferable.
UWC
Phi-1.5
Llama-2-77
ES-exact
ES-perturb
ES exact
ES- ES-perturb
selup
epuchs
retain
unlearn
relain 1
unlearn !
retain
unlearn
relain
unlearn
0.4439
0.3368
0.2136
0.1551
0.8277
0.6284
0.4990
0.3444
0.4223
0.2614
0.1942
0.1274
0.7370
0,2182
0.4560
0.2324
1%
0.4232
0.2033
0.2136
0.0571
0.8277
0.1029
0.4419
0.0443
0.4232
0,2242
0.2005
0.1178
0.8277
0.1048
0.4435
0.0029
0.4393
0.2954
0.2192
0.2172
0.7418
0.5809
0.4563
0.3799
0.4536
0.2224
0.2137
0.1386
0.7928
0.0231
0,4493
0.0144
5%
0.4268
0.2829
0.2276
0.1652
0.7496
0.0053
0.4420
0.0053
0.4404
0.4395
0.2308
0.1652
0.7401
0.0053
0.4390
0.0620
0,4433
0.3974
0.2024
0.1360
0.7803
0.2163
0,4482
1016
0.4424
0.4799
0.2004
0.1302
0.7939
0.3214
0.4828
0.1623
0.4404
0.4575
0.2141
0.0715
0.7231
0.2479
0.4297
0.1071
0.3944
0.4819
0.1813
0.1025
0.6989
0.2791
0.4487
0.1171
Table 14: UWC Tuning for the Loss Selection of KL. / indicate smaller larger values preferable.
UWC
Phi-1.5
ES- ES-exact
ES-perturb
ES-exac
ES ES-perturb
setup
g
retain f
unlearn
retain
unleam +
retain
unlearn 4
retain 1
unlearn +
0.3
0.4934
0.4505
0.2513
0.1804
0.7958
0.7634
0.4832
0.4278
1%
0.5
0,4992
0.4506
0.2460
0.1709
0.7958
0.7634
0.4750
0.4217
0.7
0.4620
0,3540
0.2443
0.1582
0.7900
0.6105
0.4656
0.3738
0.3
0.5786
0.5523
0,2544
0.1526
0.7509
0.6872
0.4694
0.3867
5%
0.5
0.5716
0,4859
0.2646
0.1625
0.6961
0.7309
0.4419
0,3757
0.7
0.5766
0.2480
0.2492
0.1293
0.7080
0.3539
0.4299
0.2182
0.3
0.5879
0.5593
0.2466
0.2017
0.6860
0.6781
0.4463
0,3482
10%
0.5
0.5888
0.5262
0.2450
0.1951
0.6906
0.6914
0.4358
0,3621
0.7
0.5909
0.4347
0.2462
0.1197
0.6984
0.4711
0.4249
0.1712
Table UWC Tuning for the Temperature Scaling of KL. | f indicate smaller larger values are preferable.
UWC
Phi-1.5
JJama-2-7B
ES-exact
ES-perturb
ES-exact
ES ES-perturb
sctup
X
retain
unlearn
retain
unleam +
retain
unlearn |
retain 1 unlearn
0.7
0.4590
0.1781
0,2532
0.1482
0.7175
0.4007
0.4238
0.2938
1%
0.9
0.4668
0.2389
0.2473
0.0955
0.7166
0.2006
0.4243
0.1892
2.0
0.4853
0.0586
0.2517
0.0175
0.7327
0.0522
0.4304
0.0368
0.7
0.5824
0.3836
0.2447
0.1297
0.7057
0.3571
0.4154
0,2829
5%
0.9
0.6086
0,4067
0.2456
0.1189
0.7072
0.2896
0.4344
0,2556
2.0
0.5776
0.5184
0.2473
0.0461
0.7018
0.1406
0.4362
0.0399
0.7
0.5927
0.5219
0.2495
0.1577
0.6847
0.6337
0.4314
0,3220
10%
0.9
0.5888
0,4786
0.2459
0.1546
0.6940
0.5619
0.4455
0.2464
2.0
0.5881
0.4952
0.2493
0.1377
0.6851
0.0730
0.4278
0.0000
28

---

Published as a conference paper at ICLR 2025
Table 16: Comparison between unlearning methods on 5% TOFU fictitious unlearning with UWC calibration across varying 1. / indicate smaller / larger values are preferable. We primarily focus on the ES scores for unlearning (shaded), given that the ES scores for retention are calibrated.
LLM
Phi-1.5
Llama-2-15
setup
method
retain f
ES-exact
unlearn
ES ES-perturb
f unlearn
retain f
ES exaee
unlearn
ES ES-perturb
1 unlearn
before unleamiin
0,4433
0.5619
0.2115
0.2374
0.8277
0.7735
0.5302
0.4126
GA
0,2412
0,0286
0.1050
0.0623
0.3456
0.1217
0.1883
0.0333
GD
0.2423
0.0400
0.1035
0.0000
0.3283
0.0000
0.2775
0.0000
KI. 0,2480
0.0533
0.1046
0.0000
0.3175
0.0000
0.1840
0.0000
7 0.4
PO 0,2465
0.1927
0.0919
0.0794
0.3050
0,2116
0.1900
0.1617
NPO
0.3020
0.1701
0.1566
0.0731
0.5378
0,1894
0,2736
0.1452
RMU
0.2521
0.2505
0.1016
0.0576
0.3213
0.2065
0.1957
0.1578
GA 0.3213
0.0774
0.1443
0.0819
0.4725
0.1583
0.2693
0.1399
GD
0.3517
0.0992
0.1434
0.0053
0.4768
0.0030
0.2850
0.0010
KL
0.3620
0.1030
0.1538
0.0123
0.4700
0,0000
0.2747
0.0000
- 0.6
PO 0.3978
0.4078
0.1555
0.1065
0.4848
0.3684
0.2875
0,2523
NPO
0.4858
0.1992
0.1852
0.0824
0.6437
0.2811
0.3520
0,2050
RMU
0.3871
0.3465
0. 0.1646
0.0865
0.4900
0.3316
0.2700
O. 1937
GA 0.5232
0.2021
0.2242
0.0825
0.7645
0.7010
0.4104
0.4800
GD
0.5666
0,2200
0.2326
0.0753
0.7505
0,3300
0.4765
0.3200
KI. 0.5547
0,2153
0.2265
0.1080
0.7201
0.0944
0.4711
0.1580
0.9
0.5358
0.1634
0.7744
0,5493
0.4852
0.3596
NPO
0.5691
0,2537
0.2269
0.1032
0.7210
0.1160
0.4753
0,2744
RMU
0.5474
0.6038
0.2293
0.1352
0.7068
0,4004
0.4866
0.3741
29


Reviews:
[{"review_id": "VhYriLXED1", "reviewer": "Reviewer_taYS", "summary": "The paper tested 4 popular LLM unlearning methods on the TOFU benchmark dataset and observed that there are various tradeoffs (e.g. GA unlearns better while NPO retains better). Then it motivates the authors to propose a new metric which is based on the mixing the model weights between the unlearned and original model. The reason, as far as I understand, is to achieve smooth control on the extent of unlearning. The evaluation first finds the model mixing ratio that would achieve the evaluation score (e.g. ES) on the retain set, and then uses it to calibrate unlearning metric.", "questions": "See W2.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "S1: LLM unlearning and evaluation are important problems", "weaknesses": "W1: Lack of technical contribution: I think most people working in this area would agree we need more metrics and benchmark datasets. However, this paper though goes into that direction, does not really provide enough meaningful and technical contribution in my view. The paper basically tried 4 popular unlearning methods on the TOFU datasets while proposing a calibration framework (See W2). This can mostly be done in leaderboard or in a measurement paper rather than a technical paper. And findings on metric tradeoffs are mostly not surprising.\n\nW2: Lack of justification of the calibration framework: The calibration metric lacks justification. Figure 3 seems to only tell us ES is monotonically increasing with mixing factor $\\alpha$. However, the key question in proposing a metric is to ask: What does this metric measure? Does it measure the right thing? In this case, I cannot see clearly the impact of mixing model weights on the overall unlearning effectiveness. The only justification I can find is (1) being inspired by the literature on parameter disentanglement, but why is it related to unlearning? (2) The vague observation in Figure 3. Can authors explain why mixing model weights can help calibrate the unlearning metric other than just the empirical observation between ES and $\\alpha$ in Figure 3? After all, this can well be a spurious correlation. In other words, why would the community trust this calibration in evaluation? \n\nW3: Lack of organizing clarify: The paper spends the first 6 pages motivating the calibration framework, and only 1 page introducing it and 1 page for experiments. I think people who in this area already know most of the points in the first 6 pages, it'd better to shorten it and get to the point more directly and spent more content on introducing the key idea of calibration and include more justification,", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WJ81mk1ctB", "reviewer": "Reviewer_8smY", "summary": "This paper studies how best to evaluate LLM unlearning methods that aim to not leak the correct answers on a forget set while maintaining good performance on the rest of the training data. The literature has proposed a variety of evaluation metrics, and this paper first tackles which metric is most robust to information leaking attacks; they conclude ES, which was proposed with privacy attacks in mind, performs best in keeping the ranking before and after various attacks consistent. With ES, they then evaluated a generalized hyperparameter sweep for popular methods proposed in the literature, and conclude that methods have not significantly improved over the baseline of gradient ascent approaches. Narrowing why this is the case, they observe alternative methods either unlearn too strongly or not strongly enough to calibrate the performance on the retain set.", "questions": "I look forward to discussing the below with the authors, and am willing to increase my score given clarification of the questions below.\n\n1) In the paragraph “Is MM Proper for Calibration” what seems missing is discussion comparing this to a simpler/baseline approach, such as just tuning the original hyperparameters of the unlearning method to calibrate. From the Appendix I got the impression that the hyperparameters of many of the unlearning methods do not give enough control to calibrate, is this correct? If so, discussion (and/or a figure illustrating this) would help the argument that MM is a better way to calibrate/boost the performance of unlearning methods.\n\n2) Related to the above, in the “Hyper-parameter configurations” paragraph in the experiments, am I correct in understanding you do the hyperparameter sweep by evaluating the performance of the hyperparameters after also applying model mixing to calibrate the performance (this seems consistent with the appendix tables)? If so, you might consider adding somewhere in the paper that this hyperparameter sweep necessarily improves over the calibrated performance of not doing model mixing; this is as not doing model mixing is captured by just picking the best hyperparameters amongst those with $\\alpha = 1$. I believe this adds conceptual understanding to why model mixing is necessarily better.\n\n3) On the discussion of why NPO and other methods do no work in the experiments section, do you have results for what $\\tau$ values (if at all) other methods start performing well? I’m imagining a figure where the x-axis is calibrated $\\tau$ level, and the y-axis is forget performance, and a line for each method such that when the line is the lowest explains for what $\\tau$ each method is best. Alternatively, some more detailed/explicit discussion for what $\\tau$ levels other methods might perform better could help give a more complete picture for when other methods work well. I agree the high $\\tau$ range seems most practical, but I think the above can be done easily given results in the appendix and can add to the empirical study.\n\n4) (minor) model mixing seems very related to the literature on model merging; you might consider surveying the literature more broadly as a reference for the method, and when/why it works.\n\n5) (minor) If computationally feasible, could error bars/standard deviation be reported for some of the main tables; I am less concerned given the many settings tested, but this could add to the rigor of the empirical findings.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1) Extensive empirical study\n2) Proposed method for improving calibration via a general hypermater boosts performance of baseline methods to seemingly SOTA\n3) Mostly well-written", "weaknesses": "1) I found that certain parts of the draft could have been clearer about the benefits of model mixing. The draft does not discuss alternative calibration of retain performance, which naively could have also been done with just a sweep of the unlearning method hyperparameters. So at first I thought this was an empirical limitation. But after thinking about it I realized this is actually okay as the performance of calibration with just a hyperparameter sweep of the unlearning method is subsumed by the hyperparameter sweep including model mixing to calibrate (the former is just taking the hyperparameter in the latter where $\\alpha  = 1$). Results in the appendix also suggest the hyperparameters for current unlearning methods are not sensitive enough to calibrate retain performance. In the questions section I ask clarifying questions about this and suggest edits to the text that could make this contribution clearer. \n\n2) The empirical results are also focused on specific values of $\\tau$, i.e., retain performance should be $95-90\\%$ of the performance before unlearning. While I agree “high performance” seems to be the reasonable use-case, a more complete comparison without assuming specific $\\tau$ values would also identify what $\\tau$ range (if ever) NPO and other methods start performing well again. I believe it is implicit from the further study in the appendix that a “low” $\\tau$ range might make other methods perform better, but I believe a more quantitative statement could make the empirical study more complete. I ask about this in the questions section.\n\n3) (minor) A more general weakness, not specific to just this paper, is whether these performative unlearning goals for LLMs should also be generalization questions; if the goal is to not predict accurately for a set of samples, and those samples come from larger distribution, should we not be evaluating a “test” performance on the larger distribution? This is generally called concept unlearning in the literature. The field may still be far away from evaluating/studying such questions, but I raise it to clarify the setup studied in this paper is not the only setting for unlearning in LLMs. The authors might consider having more discussion on different formulations of unlearning before picking the setting they study and survey.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vdOUqCy5bU", "reviewer": "Reviewer_fSyL", "summary": "The paper surveys the current popular evaluation methods towards LLM unlearning. It discuss the need to cover before retain and forget performances and its trade-offs, as well as its robustness against different attacks.\n\nThe paper concludes that Extraction Strength (ES) is the recommended method for evaluation. In addition, the paper also suggest Unlearning with Control (UWC) framework to reach the best trade-off between unlearning strength and retain performance.", "questions": "The statistic in figure 2 is confusing. How does model have lower score (which means better unlearning from the paper) after attacking than before?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper have a comprehensive view of different unlearning evaluation methods and approaches them in a systematic manner from robustness and utility trade-offs.\n\nThe paper proposes a novel approach unlearning with control to better calibrate the trade-off between unlearning effectiveness and retain performance with model-mixing, which is a simple but effective mechanism.", "weaknesses": "There is a lack of justification for selecting the metric: Why does the PCC measure the metrics' robustness again attacks? In Figure 2, the plot is characterized by the test static before and after the attack for different methods, models, and forget set ratio. Why should we assume there is a linear correlation among them? In addition, the paper uses TOFU as unlearning dataset/task, but does not survey the metric used in the TOFU paper (truth ratio).\n\nWeak/unclear attack methods: it is unclear the what dataset the relearn attack in figure 2 is based on (from other section I would think it is TOFU, but it is not presented until late in the paper). The setup for relearning it not clear.\n\nAlthough UWC seems like a method to manage the trade-off between unlearn and retain performance, it is not clear how it fits into unlearning evaluation pipeline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oQWsIgz0gp", "reviewer": "Reviewer_JEXC", "summary": "The paper aims to answer the following research questions in machine unlearning: (1) What metric should be used in machine unlearning (2) what method is better in trading off unlearning and retention. The paper first presents a comparative study to understand what metric is robust (i.e., present a linear relationship before and after jailbreaking attacks), then proposes to use model mixing to for better calibration between retention and unlearning.", "questions": "Please provide response to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "(1) The problem is well-motivated.", "weaknesses": "(1) For the metrics used for machine unlearning, The author suggests using Extraction Strength as the metrics since it exhibits linear relationship before and after different attacks. However, the evaluation is still weak to me. First of all, the attacks used in the paper are not the strongest jailbreaking attacks (like GCG attacks) and the authors also do not consider using an ensemble of strong attacks to elicit model’s knowledge, making the evaluation of the paper weak. Also, showing linear relationship before and after attack might not be a good way to measure machine unlearning before and after attack, especially if the model is backdoored or poisoned (e.g., a poisoned and backdoor model can hide their knowledge in their model parameters except when a trigger is presented or obfuscate the knowledge so that the metrics cannot used to detect unlearning). In short, there are no rigorous guarantees that using extraction strength is a valid metric under an adversarial scenario.\n\n(2) The contribution is incremental. To me, the most novel part of the paper is that the authors use model mixing (aka model merging/souping) to control the trade-offs between retention and unlearning. All the other parts seem incremental and do not convey interesting empirical findings.\n\n(3) The writing of the paper needs to be improved. A lot of useful details help to digest the results are shown in the appendix instead of the main text (e.g., attack methods) and the main text does not give enough concise summary to understand them. For example, I still cannot understand how we use Token Noising as an attack method (even the appendix does not give a good description of it). Also, the organization of the paper could be improved (e.g., lines 264-265 ask the reader to first read Section 6 for the experiment).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)