Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Authors: Sungmin Cha
Sungjun Cho
Dasol Hwang
Moontae Lee
Venue: ICLR 2025 Poster (2025)
Keywords: Machine Unlearning
Large Language Models
Low-rank Adaptation
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose a novel loss function and LoRA initialization method for robust and parameter-efficient LLM unlearning.

Abstract: Large Language Models (LLMs) have demonstrated strong reasoning and memorization capabilities via pretraining on massive textual corpora. However, this poses risk of privacy and copyright violations, highlighting the need for efficient machine unlearning methods that remove sensitive data without retraining from scratch. While Gradient Ascent (GA) is commonly used to unlearn by reducing the likelihood of generating unwanted content, it leads to unstable optimization and catastrophic forgetting of retrained knowledge. We find that combining GA with low-rank adaptation results in poor trade-offs between computational cost and generative performance. To address these challenges, we propose Low-rank Knowledge Unlearning (LoKU), a novel framework that enables robust and efficient unlearning for LLMs. First, we introduce Inverted Hinge Loss, which suppresses unwanted tokens while maintaining fluency by boosting the probability of the next most likely token. Second, we develop a data-adaptive initialization for LoRA adapters via low-rank approximation weighted with relative Fisher information, thereby focusing updates on parameters critical for removing targeted knowledge. Experiments on the Training Data Extraction Challenge dataset using GPT-Neo models as well as on the TOFU benchmark with Phi-1.5B and Llama2-7B models demonstrate that our approach effectively removes sensitive information while maintaining reasoning and generative capabilities with minimal impact. Our implementation can be found in https://github.com/csm9493/efficient-llm-unlearning.

Published as a conference paper at ICLR 2025
TOWARDS ROBUST AND PARAMETER-EFFICIEN KNOWLEDGE UNLEARNING FOR LLMS
Sungmin Cha' Sungjun Cho Dasol Hwang Moontae Lee
1 NNw York University University of Wisconsin-Madison
LG AI Research *University of Illinois Chicago
sungmin. cha@nyu. edu, sungjunchoßee wisc. edu, dasol.. hwang, moontae.. lee) @lgresearch.
ABSTRACT
Large Language Models (LLMs) have demonstrated strong reasoning and memo- rization capabilities via pretraining on massive textual corpora. However, this poses risk of privacy and copyright violations, highlighting the need for efficient machine unlearning methods that remove sensitive data without retraining from scratch. While Gradient Ascent (GA) is commonly used to unlearn by reducing the likeli- hood of generating unwanted content, it leads to unstable optimization and catas- trophic forgetting of retrained knowledge. We find that combining GA with low- rank adaptation results in poor rrfds-offs between computational cost and generative performance. To address these challenges, we propose Low-rank Knowledge Un- learning (LoKU), novel framework that enables robust and efficient unlearning for L.MMs. First, we introduce Inverted Hinge Loss, which suppresses unwanted tokens while maintaining fluency by boosting the probability of the next most likely token. Second, we develop a data-adaptive initialization for LoRA adapters via low-rank approximation weighted with relative Fisher information, thereby focusing up- dates on parameters critical for removing targeted knowledge. Experiments on the Training Data Extraction Challenge dataset using GPT-Neo models well as on the TOFU benchmark with 1hi-1.5B and Llama2-7B models demonstrate that our approach effectively removes sensitive information while maintaining reasoning and generative capabilities with minimal impact. Our implementation can be found in lbttps://github com/ ctw9493/m44/// efficient - lomm-mnnlerninn
INTRODUCTION
Large Language Models (LLMs) exhibit substantial performance gains in downstream tasks with increasing model size and amount of pretraining data (Zhao et al. 2023). This has prompted extensive research collecting high-qualit textual corpora for LLM pretraining and developing larger models to an unprecedented scale (Brown et al. 2020 Chowdhery et al., 2023 Smith et al. 2022 Rae et al., 2021 Dubey al. 2024). However, this approach has introduced significant privacy concerns due to LLMs' tendency to memorize data indiscriminately (Carlini et al.] 202T 2023). For instance, Personally Identifiable Information (e.g.. names, phone numbers, and email addresses) can be easily extracted from LLMS (Carlini et al.] 202T). Additionally, OpenAI is facing multiple copyright infringement lawsuits due to unpermitted use of licensed articles during LLM pretraining (Grynnbam & Mac 2023). In response to such challenges as well as increasing interest in one's 's right to be forgotten (e.g., the GDPR legislation) (Voigt & Von dem Bussche 2017]] Rosen 201 Villaronga
et al. 2018) 8), machine unlearning for LLMs has emerged a critical and rapidly growing research
hield Yao et al. 2023 Si al. 2023).
One method for LLM unlearning would be to filter out sensitive data from the corpus and retrain the model from scratch, an approach known as exact unlearning. With unprecedentedly large models and pretraining datasets, this process is highly resource-intensive and can easily become intractable under the possibility of multiple data deletion requests made in a sequential manner. This motivates approximate unlearning, where the goal is to remove knowledge of specific data instances without retraining the model from scratch (Figure 1). In this regard, several novel approaches have been
"qqull contribution

---

Published as a conference paper at ICLR 2025
Machine Unlearning for LLMs
Please remove my datal
Default LoRA Initialization
Pretrained Weight W 12 ni
Retain set D,
00
OM
Forget set D/
N
Dataset
= Dj DD,
Pretrained LLM Unleamed LLM
Fishhr-weighted Initialization (FILA)
Gradient Ascent (GA) vs. Inverted Hinge Loss (IHL)
mip (Wtgg(W 12 REB AER
(w- BA) l|2
Query: Did you put your name in the Goblet of Fire,
B' BR
Harry
Neo
Harry or
Neo
W B'A" E R
FGx (x()))
Fuu 1 Po
max (Po (x)x)=)) wes
R
Figure L.LM unlearning aims to forget data points in Dj while maintaining knowledge of the retain set D,.. Unlike GA, our IHL induces higher unlearning stability by reducing the likelihood of unwanted tokens in a controlled manner. To accelcrate unlearning with IHL, FILA extracts and places parameters important in generating D, to LoRA weights priori via weighted low-rank approximation. IHL and FILA form a powerful synergy towards robust and efficient LLM unlearning.
proposed for approximate unlearning: Jang al. (2023) introduced simple method that finetunes LLMs using Gradient Ascent (GA) on data requested for deletion and also proposed n-gram- metrics to evaluate its effectiveness. Wang et al. (2023) and and
al. (2024) proposed knowledge distillation-base methods that selectively transfer knowledge to a secondary model for unlearning. However, both approaches face significant challenges: GA suffers from unstable optimization due to unbounded nature of its objective loss, while distillation based methods incur substantial computational costs from relying on secondary model. Above all, these approaches share a critical drawback: the high computational cost of full fine-tuning all parameters within the LLMs.
Meanwhile, Low-Rank Adaptation (LoRA) has emerged as one of the most prominent techniques for parameter-efficient fine-tuning on downstream tasks (Hu et al. (022). The core idea of LoRA to freeze all pretrained weights and instead train low-rank decomposition matrices to model the weight changes in each linear layer, effectively reducing the number of trainable parameters and thus its memory cost. Beyond its efficiency, the low-rank structure in LoRA also serves as a strong regularizer (Biderman et al. 2024), which hypothesize aids LLM unlcarning by stabilizing optimization and mitigating catastrophic forgetting of retained knowledge. However, the empirical effects of LoRA in the context of LLM unlearning remain largely unexplored.
In this paper, we present the first in-depth study of LLM unlearning under the low-rank adaptation paradigm and introduce Low-rank Knowledge Unlearning (LoKU), which consists of two novel techniques for robust and parameter-efficient knowledge unlearning. First, we analyze the derivatives of GA to highlight its drawbacks such (1) unnecessary forgetting induced by indiscriminately increasing the probability of all tokens and (2) unstable optimization due to the unboundedness of maximizing the next-token prediction loss. To tackle these challenges, we introduce the Inverted Hinge Loss (IHL), which replaces each unwanted token with the next most- prosbblrobable one, and demonstrate that IHL facilitates stable unlearning by resolving the limitations of GA. Second, we find that the low-rank regularization in LoRA is too strong when unlearning with IHL, leading to suboptimal cost vs. post-unlearning performance trade-offs. Initialization of Low-rank Adapters (FILA), which anta-adaptively assigns parameters responsible for generating unwanted information to adapters prior to tuning by decomposing the pretrained parameters weighted by the relative Fisher-information matrix (Figure T}. Experiments the Training Data Extraction Challenge dataset (GPT-Neo) and the TOFU benchmark PPhi-1.5B, Llama2- 7B) demonstrate that IHL combined with FILA outperforms existing baselines in both efficiency and post-unlearing performance. In summary, our main contributions are as follows:
response, we propose Fisher-
We analyze the shortcomings of unbounded optimization and unnecessary forget- ting through its derivative and propose IHL to address these issues.
We introduce FIL,A, a method to accelerate unlearning by data-adaptively assigning parame- ters responsible for unwanted information to low-rank adapters before tuning.
We demonstrate that our proposed LoKU using both IHL and FILA outperforms previous baselines in terms of both efficiency and post-unlearning performance (Figure 2).
2

---

Published as a conference paper at ICLR 2025
43
10
250
Fu
Reasoning (Ace)
LORA
Dialogue (F1)
The Pile (PPL)
200
Ours
42
150
41
100
40
50
10a
10 10
10a 106 10
10a
10f
10
Unlearning Cost (FLOPs)
Unlearning Cost (FLOPs)
Unlearning Cost (FLOPs)
Figure 2: Compute cost for successful unlearning vs. post-unlearning downstream performances. We unlearn 32 randomly sampled sequences in the Training Data Extraction Challenge from GPT- 125.. Each point represents different forget set and LoRA rank (if used). Left: Accuracy averaged across 9 classification tasks (higher is better). Middle: Fl score averaged across 4 dialogue generation tasks (higher is better). Right: Perplexity on the validation set of the Pile dataset (lower better). Dashed lines indicate the performances of the model prior to unlearning. Unlearning via gradient differences (GD) with vanilla LoRA leads to significant loss in performance compared to full-parameter GD unlearning due to lack of plasticity. However, our proposed LoKU using both the Inverted Hinge Loss and Fisher-weighter weighted LoRA initialization performs competitively unlearning via full-fine filltfinttunnn in all three aspects while enjoying the sraameter-effiiencyy of LoRA.
2 RELATED WORK
The primary objective of machine unlearning is to adapt pretrained model to discard information acquired from a specific subset of data previously used during pretraining, with
Machine Unlearning.
active research focused on image classification (Cao & Yang 2015 Golatkar et al. 2020 Tarun et al.
2023 Mehta et al. 2022 Chundawat et al.] 2023 Cha et al. 2024). 4). Recently, its significance has
grown notably with Large Language Models (LLMs) due to crucial need for managing unintended
memorization of pretraining data intrinsic to LLMs (Si et al. 2023 Yao al 2024b). Several machine
unlearning algorithms for LLMs focus on parameter optimization (Si et al. 2023). For example, Wang et al.) (2023) introduced Knowledge Gap Alignment, using knowledge distillation between models trained on different datasets. [Cen & Yang (2023) proposed an unlearning layer to selectively remove specific knowledge while preserving other parameters, while Liu et al. (2024) developed a two-stage framework to capture and negate harmful knowledge. However, these approaches are limited by the need to retain large datasets (Wang et al. 2023 Chen & Yang] (003) rely on
secondary models for distillation (Wang et al., 2023 Liu et 2024). Model-editing methods, such as task arithmetic for suppressing harmful content (Ilharco et al. 2023) Wu et al. 2023), avoid substantial costs but show limited effectiveness in unlearning. In contrast, Jang et al. (2023) used Gradient Ascent (GA) for LLM unlearning by maximizing the next-token prediction loss the forget data, effectively unlearning while preserving model performance. Following this, GA has become standard baseline, with Yao et al. (2024a) improving its robustness by combining GA with gradient descent on in-distribution data.
Fine-tuning LLMs for specific tasks is computationally ex- pensive due to their large parameter sizes. To address this, Gtrameter-Efficien methods adapt only a small subset of parameters while keeping the pretrained ones frozen (Liu et al.) 2022b Qiu et al. 2023 Liu et al., 2023). Inspired by the small intrinsic rank of LLMs (Li et al. 2018
Crrameter-Efticieen Fine-Tuning,
(PEFT)
Aghajanyan et al. 2021) LoRA and its derivatives add low-rank adapters to the model 's linear lay-
ers (Hu et al. 2002 Zhang et al. 2023 Yeh et al, 2023 Kopiczko et al. 2024 yang Liu et al. 2024). These adapters can be merged with pretrained parameters after fine-tuning, maintaining the original inference cost. While most methods use random initialization for LoRA adapters, PiSSA (Meng et al., 2024) suggests initializing them using the principal singular vectors and values of the linear weights.
Although various methods in machine unlearning and parameter-efficient fine-tuning for LLMs have been discussed, this paper focuses on analyzing the inherent issues of GA, introducing a novel unlearning loss function to overcome these issues, and exploring parameter- efficcient unlearning methods that do not require full fine-tuning. These areas have not been adequately addressed in previous studies, highlighting the contributions of our work.
3

---

Published as a conference paper at ICLR 2025
3 PROPOSED METHOD: LOW-RANK KNOWLEDGE UNLEARNING (LOKU)
3.1 PRELIMINARIES
Problem and Notation. Given sequence of T tokens æ (æ1, (2,,,22.... ......... xT). language model (LM) models the likelihood of the sequence via next-token prediction: px(x) = IIr po(xr| After pretraining, we assume that an end-user has requested to delete a subset of the training set Dj C D, which we refer to as the forget set. The retain set D, refers to an auxiliary dataset that contains other relevant knowledge that must be retained after unlearning (e.g.. Wikitext; Merity et al. 2017).
Gradient Ascent. Ideally, the LM must assign low probability sequences in Dy, leading simple yet effective baseline of Gradient Ascent (GA:Jang&& et al. (203)). GA unlearns a sequence of tokens = ;.T by maximizing the next-token prediction loss
(x) = Σ log(po(ze/z<ee)
(I)
In practice, the log-likelihood computed using cross-entropy loss and thus GA essentially minimizes the Negative Cross-Entropy (NCE) loss. Therefore, GA maximizing the next-token prediction loss involves unbounded optimization, leading to an ill-posed process with unstable tuning. While Gradient Difference (GD) aims to alleviate this instability by minimizing the next-token prediction loss for Dr alongside NCE on D as regularization, we find that the approach falls short of a fundamental solution, showing performance degradation unlearning updates made.
Low-Rank Adaptation. Based on the assumption that parameter changes due to LLM adaptation exhibits intrinsic low-rank (Aghajanyan al., 202T). LoRA models the change in parameters AW E R of each linear weight W E mg via a product of two low-rank matrices A E Rrnk and E [gd where r < min(d, k) the rank of the LoRA adapter. In other words, the output of the adapted linear layer given an input becomes
( + AW)x = Wx + BAw
During fine-tuning, the original weight W is kept frozen and only the low-rank factors A and B are updated via gradient descent. To ensure that the initial attachment of LoRA adapters does not alter the output of the LLM, LoRA defaults to initializing A with a Kaiming-unifonn et al, 2015) and B as the zero matrix. After finetuning, LoRA adapters simply be merged with the original weights W = W + BA, thereby avoiding any additional latency during inference.
distribution (He
3.2 PRELIMINARY RESULTS
Despite its wide use in domain adaptation and instruction tuning, LoRA is not yet explored under the task of LLM unlearning to the best of our knowledge. Therefore, we first share empirical results from low-rank adapting LLMs using GD as our objective to motivate our approach. |Figure 2 shows the results. Notably, vanilla LoRA suffers from lack of plasticity and ends up failing to sufficiently unlearn D, within 20 epochs. When running more unlearning epochs or increasing the learning rate for sufficient unlearning, the model loses its previously acquired reasoning and generative capabilities. as shown in the significant decrease in Reasoning and Dialogue performances. In the remainder of this section, we present two techniques towards making LLM unlearning viable while enjoying the efficiency of LoRA.
3.3 INVERTED HINGE LOSS: A NOVEL Loss FUNCTION FOR LLM UNLEARNING
Motivation. We analyze the inherent issues of GA from the perspective of its derivative. The output layer of a language model a softmax layer that outputs probabilities over the vocabulary. Let y be the logits (pre-softmax activations) produced by the LLM model for the t-th token, and let V be the vocabulary size. The probability po(xx( is given by the softmax function: Po (x<e) = (xt);s exp(y), (=c) >/Em exp(y,") (v) where y/. is the logit corresponding to the true token I, and y/ (7) is the logit corresponding to the v-th token in the vocabulary. When we use LGs for unlearning for LLMs, the gradient of the bog-robbabylity with respect to the logits is
i log (po (E())
[ po (2
if Ft if + x
Oy.
4

---

Published as a conference paper at ICLR 2025
From this derivative of GA, we can interpret its unlearning mechanism: given the prefix 2<6, GA reduces the prediction score of the true token 2, in proportion to 1 po(z)|z<)) while increasing the scores of other tokens (t.e., v # 2,) by (t) This process effectively shifts the model's prediction for Tet away from the true token 2t, thereby achieving unlearning. However, we can confirm that GA suffers form the following problems during unlearning: (1) Gradient spread, where reducing the score of Ie while increasing the scores of all other tokens leads to inefficient unlearning in large vocabularies by predominantly boosting other tokens; (2) Unbounded loss, where minimizing log(pe [24] (x)))) through maximizing cross-entropy loss introduces a risk of divergence due to the unbounded nature of entropy; and (3) Degradation of generative performance, where GA applies uniform gradient updates (i.e. increasing the scores of other tokens) to all sequences in the forget set y, despite each sequence requiring unique number of updates for unlearning. This redundancy can cause degrade the model's generative capabilities, resulting in catastrophic forgetting.
Inverted Hinge Loss. To cope with aforementioned limitations of GA, we aim to design a new loss function that achieves effective unlearning by decreasing the prediction score of the true token, while focusing gradient updates on only a minimal number of viable replacements for the ground-truth token. Inspired by the Hinge Loss (Cortes & Vapnik 1995). we devise Inverted Hinge Loss (IHL) as
Lun (x)) = 1 + (x( max (v((a<e))
As the probability (x( is given by the softmax function, the derivative of (a) with respect to yi (+) is:
( [=<))) (0( 1)) if Iz po (pe) (=))))) Po Po (xt)) 1) if 0
(a) 0yb")
po(vl (1o - po (2r (xx)))
if * Ie and v # v²,
where V* arg max Po (c) The detailed derivation can be found in Appendix A
The above derivative clearly illustrates how the IHL addresses the shortcomings of GA in knowledge unlearning for LLMs. First, IHL mitigates gradient spread by ensuring that the gradients primarily focus on the true token 24 and its competitive token v", without excessively boosting irrelevant tokens in large vocabularies. For example. in the case where unlearning has not yet been achieved (i.e., when pe (1r (x)) > po (x<x)), the absolute value of the gradient for the true token Ie is equal to or greater than that of 0* (with opposite sign) and exceeds that of other tokens (i.e., v A xy and v * v*). This ensures efficient and targeted unlearning while avoiding the unnecessary spread of gradients across irrelevant tokens. Second, IHL resolves the issue of unbounded loss by defining a bounded loss function, ensuring that the prediction scores for tokens other than 3e and v² decrease once unlearning is complete (i.e., when Po (2i (<<c) becomes less than po (x=())=)) This bounded loss prevents instability and divergence during unlearning. Lastly, IHL prevents the degradation of generative performance by adapting gradient updates based on the prediction scores of 2, and u*, alleviating redundant updates for other tokens (i.e., v # Iy and A v*). Compared to GA, this approach reduces redundant updates for other tokens in each sequence of the forget set Dy, thereby preserving the model's generative capabilities while achieving effective and stable unlearning.
3.4 FILA: A NOVEL LORA INITIALIZATION FOR LLM UNLEARNING
Motivation. While IHL effectively stabilizes the unlearning process by reducing the likelihood of unwanted tokens in a controlled manner, we empirically find that combining IHL with LoRA naïvely leads requiring large number of unlearning iterations fully forget samples in Dj- Naturally, an increasing number of updates with D, comes with the risk of overfitting, which for unlearning, may result in significant loss of knowledge on D,.. We hypothesize that this due to the random initialization of LoRA weights together with its low-rank structure imposing too strong a regularization to handle precise gradients needed for proper minimization of IHI. (Biderman et al. 2024). Drawing inspiration from PiSSA (Meng et al. (0244), we therefore aim to accelerate optimization of IHL under LoRA by extracting weights relatively more important to Dy than to D, from each pretrained weight, then using them to initialize LoRA weights prior to unlearning. We conjecture that this approach reinforces the model's 's plasticity to forget D/ as well as its stability on keeping knowledge on D,, The remainder of this section presents how parameter importances are measured via Fisher information, and how low-rank adapter weights are initialized based on the measured importances.
5

---

Published as a conference paper at ICLR 2025
Parameter Importances via Fisher Information. The Fisher information matrix F& (D) is defined as the variance the partial derivative of the oog-likeihood data with respect to the model parmeter 9 (left of Equation 2). Intuitively, the matrix can be considered measurement much the model output changes following a small change on its parameter weight. However, as marginalizing across the space of DD is intractable, many works in continual learning (Kirkpatrick et al. 2017) and model compression (Hsu et al., (2022) literature have thus used the empirical Fisher information (D) instead. In the context of LLMs, this can be computed as:
how
Fo (D = ED ( log loggo(Di))
~
Σ RED
0 fiM
(LM((z;)) = Fo(DD,
(2)
00
DDI
where LLM denotes the next-token prediction loss used to pretrain LMs, (LM( 0)
=
E (og(p(x1))=<z)) Within our LLM unlearning setup, high empirical Fisher information measured with D, indicates that LIM on D leads to large absolute gradients on the parameter under concern, and consider such parameters to be important in generating sequences in Dj-
Let F = (Dy) denote the empirical Fisher information matrix of the target parameter W measured using the forget set D (resp. Fw using the retain set D,). Then, we use the relative importance metric to identify parameters that are important exclusively for Dj and not for DD. While generating Dy involves extracting memorized information on D well composing linguistically fluent outputs, only wish to
Fisher information Evel == F w Fi E Rd
adjust parameters responsible for the former and thus use Fw rather than Fw
Fisher-weighted Initialization of Low-rank Adapters. Given the relative importance Free for each target weight W. we propose to initialize the corresponding LoRA adapter weights with the solution the following Weighted Low-Rank Approximation (WLRA) problem:
(1 Forel (W (W- - BA)
min
ACRTKE, BERTKE,BEREE BERdXr
Note that when all weights [Fw] kc, equal one, WLRA reduces standard low-rank matrix ap- proximation, for which the solution can easily be computed via rank-r SVD. For general weights, however, this minimization problem does not have a glosed-form solution and requires iterative optimization (Srebro & Jaakkola 2003). While we may resort to iterative methods to initialize LoRA weights in a lineigrained manner, this would undermine the efficiency gains from deploying low-rank adapters. Therefore, we assume that parameters in each row of W share the same importance equal to the square-root of the row-wise of W. and simplify the problem
diag ((PW 1)+ (W - -BA)|
min AgR AERrXx_BERdx*
with E & and di d-))(.) indicating the all-one vector and the vector diagonalization function, respectively. Unlike general WIRAA, this row-wise WIRA problem has closed-form solution, which can be obtained by applying rank-r SVD to decompose diag 1) W USVT and computing B (F 1) + and A V..
Given this solution, we use B and A' as initial LoRA weights. To ensure that the model behavior remains the same after LoRA initialization, the base layers are also updated with W W B A' Intuitively, our Fisher-weighted Initialization of Low-rank Adapters (FILA) extracts parameters that are important for generating D, but not for generating D,, such that LoRA tuning can be focused on erasing knowledge relevant to Dj while keeping information regarding D,
3.5 FINAL LOss FUNCTION OF LOKU
In summary, we perform unlearning on the model + 0 Ofila consisted of original pretrained weights 0 and FILA-initialized low-rank adapter weights for each linear layer OFL. = (A,, B where L represents the number of layers tuned via LoRA. Additionally, we incorporate GD, which utilizes the auxiliary retain set D,.. The final loss function of LoKU is defined as follows:
minimize OFLA
(x) + (LM(
EDD.
In practice, training (unlearning) for the L.M model is conducted by minimizing Equation 3 through stochastic gradient descent.

---

Published as a conference paper at ICLR 2025
Table I: Evaluation results on reasoning and generative capabilities before and after unlearning samples from the TDEC dataset. For each model, we test three unlearning objectives (ī.e. GA, GD, IHL) with full- pala-petetter tuning, and four methods using LoRA with rank 16. IHI. denotes replacing the GA loss term in GD with our IHL, and +FILA denotes initializing LoRA weights with FILA. Changes in performance after unlearning are in parentheses, with best results under LoRA-based
unlearning in bold.
Model
Method
Params. (%)
Epochs
EL (%)
MA (%))
Reasoning (Acc)t
Dialogue (F1))
Pile (PPL)
Before
77.4
43.4
9.4
17.8
GA
17.2
1.0
27.4
39,9 (-3.5)
2.6 (-6.8)
$77.8 (+560.0)
GD IHL.
100.0
4.6
0.7
24.9
(-1.0)
5.9 (-3.5)
54.2 (+36.4)
GPT-Neo 125M
17.2
0.7
29.2
42.3 (-1.1)
10.3 (+0.9)
18.1 (+0.3)
GD
8.6
0.3
20.6
40.8 (-2.6)
(-6.9)
129.4 (+111.6)
IHL
11.4
0,4
22.7
41,9 (-1.5)
6.0 (-3.4)
32,9 (+15..1)
1.6
GD+FILA
7.4
1.2
27.4
42.0 (-1.4)
6.5 (-2.9)
89.5 (+71.7)
LoKU (FHL+)
6.0
0.3
23.9
42.2 (-1.2)
10.1 (+0.7)
24.0 (+6.2)
Before
67.6
92.2
49.8
11,5
11.5
GA
13.8
1.9
30.4
49.7 (-0.1)
(-3.0
15.8 (+1.3)
GD
100.0
12.8
2.2
30.9
(-144
(+1.2)
10.8 (-0.7)
GPT-Neo 1,3B
IHL
7.6
0.7
30.4
48.4
(+1.0)
(-0.5
GD
19.3
1.7
31.4
45.0 (-4.8)
(-18)
31.8 (+20.3
IHL
20.0
1.7
44.6
47.1 (-2.7)
(-1.3)
14.9 (43.4)
GD+FILA
0.8
7,8
1.9
23.2
44.2 (-5.6)
5.5 (-6.0)
54.5 (+43.0)
LoKU (HHL+FILA)
13.0
0.5
29.6
48.3 (-1.5)
12.1 (+0.6)
14.7 (43.2)
Before
70.4
93.4
52.3
11.5
10.4
GA
10.8
1.6
31.0
51.9 (-0.4)
11.1 (-0.4)
17,9 (+7.5)
GD
100.0
8.0
0.7
28.3
51.8 (-0.5)
12.7 (+1.2)
17.9 (47.5)
GPT-Nco 2.7B
IHI.
6.6
0.5
29.3
51.8 (-0.5)
(21.4)
10.7 (40.3)
GD
14.0
0.1
20.4
45.9 (-6.4)
6.7 (-48)
61.1 (+50.7)
IHL
17.8
0.0
26.7
49.6 (-2.7)
8.2
22.2 (+11.8)
GD+FILA 0.7
6.8
1.6
28.9
44.8 (-7.5)
9.3 (-2.2)
68.7 (+58.3)
LoKU (IIL)))
10.3
0.1
28.5
49.6 (-2.7)
10,7 (-0.8)
16.0 (45.6)
4 EXPERIMENTS
In this section, we first perform experiments unlearning samples from the Training Data Extraction Challenge (TDEC; (4.1), followed by ablation and analytical results ($4.2). We also conduct experiments on the Task of Fictitious Unlearning (TOFU; $4.3). a benchmark that well-mimics a
real-world scenario for LLM unlearning evaluation. For brevity, we present results from additional experiments such as continual unleamiin in impppeddix
4.1 TRAINING DATA EXTRACTION CHALLENGE
Experimental Setup. The Training Data Extraction Challenge (TDEC) dataset (Carlini et al. 2021 consists of 20k examples from the Pile dataset (Gao et al. 2020) found be easily extractable from pretrained LLM. For each experiment, we randomly sample 32 sequences with 200 tokens consist the forget set D,. For the retain set D,,, we use the subset of WikiText (Merity et al. 2017) as contains factual world knowledge that we wish to maintain after unlearning. We consider GPT-Neo 125M, 1.3B, and 2.7B pretrained on the Pile dataset as our base models, and unlearn Dy using five different forget sets. For this experiment, we use fixed learning rate of 2e-4 and use LoRA adapters with rank {4, 8, 16, 32). For reasons we illustrate later in $4.2 we choose to apply LoRA on query and value layers the attention module and two linear layers within feed-forward layers.
Following previous work (Jang et al.) 2023), we measure the unlearning efficacy with two metrics. The n-gram Extraction Likelihood (EL,) measures the n-gram overlap between the ground truth sequence in D, and the output generated by the model. The Memorization Accuracy (MA) measures the token-wise accuracy of the LLM on Dy. More details on these metrics are shared Appendix B After each unlearning epoch, we measure EL and MA of the model, and we consider the model has successfully unlearned DD- if both values measured on Dj become smaller than those measured from held-out validation set that the model has never seen before within 20 unlearning epochs. Once unlearning is finished, we evaluate the unlearned model on various downstream benchmarks to measure how well the LLM maintains its previously acquired reasoning and generative capabilities. To assess its reasoning capabilities, we average accuracies across different classification datasets. To measure generative performance, we also average the Fl scores over four dialogue generation datasets. Lastly, we measure the perplexity on the validation subset of the Pile (Gao et al.) 2020). A comprehensive list of evaluation datasets can be found in Appendix C
7

---

Published as a conference paper at ICLR 2025
Pretrained
Success
X Fallure
IHL
GD4FILA
IHL+
GPT-Neo-125M
GPT-Neo-1.38
GPT-Neo-2.78
50.0
Reasoning (Ace)
50
47.5
42
45
42.5
-1
12.5
10
10.0
Dialogue
7.5
5.0
2.5
150
100
300
The Pile. (PPL)
100
75
200
50
100
50
25
16
32
LoRA Rank
LoRA Rank
LoRA Rank
Figure 3: Results from unlearning examples in the TDEC dataset on the GPT-Neo LLM family. Each row represents the performance averaged across datasets within each set LLM capability tests: Reasoning (higher is better), Dialogue (higher is better), and Perplexity (lower is better). The circles and crosses represent successful and unsuccessful attempts, respectively, particular forget set D,- Solid lines indicate the performance of different methods averaged only across successful unlearning trials. The dashed lines indicate the base model performance prior to unlearning. GD leads to significant loss in performance and also fails unlearn in some cases even with large LoRA ranks. Replacing the NCE loss in GD with IHL. boosts retention of reasoning and generation capabilities, but still fails to unlearn in multiple cases. Running GD with FILA notably increases the rate of unlearning success, but at significant cost in overall performance. Our LoKU using both IHL. and FILA best minimizes post- unstannne performance degradation in all aspects.
unlearning a
We consider two LLM unlearning baselines, Gradient Ascent (GA) (Jang al. 2023) and Gradient Difference (GD) Liu et al. 2022a; Maini et al. 2024). both of which only require the original language model and datasets D, and D, representing knowledge we wish unlearn and retain. respectively. We exclude methods that require another auxiliary model (Wang et al. 2023 Liu et al. 2024 or the entire training data (Wang et al. 2023 Chen & Yang 2023) from our baselines.
Results. Table 1 shows evaluation results using fixed LoRA rank of 16 and Figure 3 shows analogous results using a different LoRA ranks. Our key findings are as follows. First, using GD not only meets the forgetting criteria in fewer epochs across all model sizes but it also preserves previously acquired knowledge (i.e., performance on Reasoning, Dialogue, and Pile) better than GA. Mainly for 125M and 1.3B models, GA causes significant decline in generative performance, whereas GD partially mitigates this decline and improves both Dialogue Fl scores and the Pile perplexity. Second, we find that simply applying GD with LoRA fails to unlearn effectively across all model sizes. While enjoying great parameter-efficiency by tuning only about 0.7% to 1.6% of the total parameters when using LoRA, its application to unlearning with GD results in large loss in overall language capability, especially on generative tasks. Third, replacing GA in GD with IHL leads to performance gains in both full parammeerr and LoRA-based unlearning, but requires larger number of epochs for successful unlearning than GD especially when confined to low-rank weight changes. However, this resolved when IHL used together with FILA, which significantly reduces the number of required epochs. Note that using FILA with GD also reduces the number of epochs required, but worsens downstream performance many cases, as FILA essentially accelerates the divergent behavior of GA in GD. In essence, whether the ability of FILA to accelerate tuning also translates to benefits in downstream performance depends on the loss function being optimized. On the other hand, when paired with our bounded IHL (i.e., LoKU), FILA leads to better retention of LLM performance, showcasing the strong synergy of our approach that enjoys not only the stability of IHL but also the speedup from FILA.
4.2 ANALYSIS
What modules do need to adapt? Figure 4
presents experiments where low-rank adapters are attached to various target parameter groups, including those for Query (Q), Value (V), Key (K), Output (O) in the attention module, and the Feed-Forward Network (FFN). While the original LoRA
8

---

Published as a conference paper at ICLR 2025
Protrained Success
X Fallure
GD
IHL
GD
IHL FILA
1e4
Reazenirg (Acc)
42.5
Dea00g (F3)
10
Pilc (PPL)
40.0
37.5
35.0
32,5
3000
3000
oxvO
FFN
OVFFN
ORVOFFN
OKVO
FFN
QVAFN
ORVOFFN
0XVO
FFN
GVFFN
ORVOFFN
Figure 4: Results from unlearning examples from TDEC dataset using LoRA with rank 32 adapt sets of layers on PPT-Nco-125M The marker shapes and colors are used similarly as in Figure 3) Based on the rate of unlearning success, tuning FFN layers (e.g. FFN, QVFFN) is more receptive to targeted knowledge removal compared to tuning attention layers (e.g., QV. QKVO).
paper (Hu et al. 2022) indicates that applying LoRA to Q and V yields superior performance on downstream tasks, our experiments indicate that using LoRA on Q and V only is insufficient to meet the unlearning criteria within our timeframe of 20 epochs. Notably, when LoRA is applied to FFNS, we observe significant increase in rate of successful unlearning. Furthermore, our LoKU integrating FAI.A with IHI., achieves the best post-unlearning performance across all LoRA target module combinations.
Cost-efliciency of the proposed method. Our compute-cost vs. performance comparisons in Figure 2] show that, while vanilla LoRA allows significant reduction in unlearning costs (i.e., FLOPs) by freezing the majority of parameters, it incurs substantial performance losses compared to full-parameter unlearning due to excessive stability originating from its low-rankness. In contrast, combining the proposed IHL with FILA not only achieves the best performance but also leverages the cost advantages of LoRA.
4.3 TASK OF FICTITIOUS UNLEARNING
Experimental Setup. The Task of Fictitious Unlearning (TOFU) benchmark (Maini et al., 2024 is a synthetic dataset containing 20 question-answer pairs for each of 200 fictitious author profiles generated by GPT-4, The TOFU evaluation pipeline first finetunes a pretrained LLM on all QA pairs. Given this finetuned LLM that serves as our base model, our task is to unlearn all information regarding 1%, 5%, or 10% of the authors from the model. Note that we can obtain reference models finetuned only on the retain set (QA-pairs on 99%, 95%, or 90% of authors), with which we evaluate the Forget Quality of unlearned models by measuring the p-value from Kolmogorov-Smie -miirovv test. A high p-value indicates high distributional similarity between the unlearned model and the reference model, thus implying strong forgetting. To evaluate how well the model retains other information outside the forget set, we measure the Model Utility as the aggregation of the probability of correct answers, ROUGE-L scores, and Truth Ratio of correct answers vs. incorrect answers on questions from three datasets pertaining to the retain set of fictitious authors, real authors, and world facts.
Following the original paper of TOFU, we prepare two base models by finetuning Phi-1.5B and Llama2-7B TOFU for 5 epochs with learning rates 2e-5 and 1e-5, respectively. We then unlearn with various methods using LoRA adapters of rank 4, 8, 16, or 32. While we mainly compare our methods against GD, we also compare IHI. and FILA against existing unlearning methods such as
KL (Maini et al., 2024), DPO (Rafailov et al.) 2024), and NPO (Zhang et al. 2024), results from which can be found in Appendix D For unlearning, we use a learning rate of 2c-4 if our base model is from Phi-1.5B and Ic-4 for Llama2-7B. All training procedures run 5 epochs with an effective batch size of 32 using the AdamW optimizer (Loshchilov Hutter 2019).
Results. Figure 5) shows the model utility vs. forget quality curves from unlearning three differently- sized TOFU forget sets from Phi-1.5B and Llama2-7B models. Comparing results among different forget set sizes, we first observe that forgetting 1% of author profiles is fairly straightforward, as all curves quickly approach the reference model with a single epoch, with increasing the LoRA rank leading to incremental improvements in performance. On the other hand, when unlearning larger set of profiles (i.e., 5% or 10%), we see that both GA and GD quickly degrades model utility.
With regards to our proposed method, we find that replacing the NCE loss in GD with our IHL better retains model utility across all LoRA ranks and forget set sizes, as curves are more aligned straight-up towards the reference point with negligible shift in model utility. This stability comes at the cost of unlcarning efficiency, however, as randomly initialized LoRA weights are unable to effectively represent weight changes required to decrease IHI.. Nonetheless, our LoKU initializing

---

Published as a conference paper at ICLR 2025
* Retain Set Only
Pretrained
GD
IHL
GD-FIA
IIL+
Rank u
Rank 8
Rank a 16
Rank = 32
forger 1%
-10
-10
-10
-10
-20
-20
-20
-20
0.0
0.5
1.0
Torpet Quelky
10
10
10
-00
(lgg
-20
-20
-20
-20
1.0
dog Qualky poratoe)
-10
-10
-10
-10
-20
-20
-20
-20
0.5
1.0
0.0
0.5 Model Utility
1.0
Moddl Utiltty
Model
Model Usiity
(a) Phi-1.5B
Qualyy
Bank 4
Rank 8
Bank 15
Rank
(6g poedor
0
100
10
10
10
-20
-20
-20
-20
0.0
0.5
1,0
0.0
0.5
1.0
1.0
Qualy
-10
-10
-10
-10
-20
-20
-20
-20
0.0
0.5
0.0
0.5
1.0
(big 10E Quallty
-10
-10
-10
-10
-20
-20
-20
-20
0.0
0.5
1.0
0.0
0.5
1.0
Moddl Utility
Utdyy
Model Ubility
Model Uniryy
(b) Llama2-7B
Figure 5: TOFU results using Phi-1.5B and Llama2-7B models. Each row corresponds to unlearning a different forget set (1% 9%, 5%, or 10%), and each column uses a distinct LoRA rank between 4 and 32. The relative size of markers represent the number of epochs. Ideally, the unlearning curves should start from the pretrained model and approach towards the reference model tuned on the retain set only (*) unlearning progresses. Both GD and GD+FILA suffers from significant loss of model utility due to using GA for unlearning. Replacing GA with IHL. largely retains model utility, then our LoKU initializing LoRA adapters with FILA significantly boosts the unlearning efficiency of IHL.
LoRA adapters with FILA largely alleviates this issue and significantly enhances unlearning efficiency of IHL by focusing gradient updates on parameters important to generating Dj-
Interestingly, we find the prior weight assignment via FILA can lead to excessive unlearning in some cases (e.g., unlearning 10% forget set with ranks 8 or 16 on Ilama2-7B), with model updates reducing the forget quality after reaching the upper bound zero This behavior resembles the Streisand effect as unlearning gradients beyond a certain point in optimization unintentionally renders Dj more noticeable within the model (Golatkar et al. 2020). As reference models are not available for measuring forget quality in real-world scenarios, finding the optimal point at which to stop unlearning to prevent this effect as well as designing a robust evaluation metric that does not depend upon oracle models would be interesting directions, which we leave as future work.
5 CONCLUDING REMARKS
In this paper, we address limitations of Gradient Ascent (GA), widely used method for LLM unlearning, and introduce a novel Inverted Hinge Loss (IHL.) to replace the negative cross-entropy loss in GA and resolve issues with dispersed gradients and unboundedness. We also propose Fisher-weighted initialization for low-rank adaptation (FILA) that pre-assigns weights relatively important to generating unwanted information as means to facilitate efficient LLM unlearning with LoRA. Experiments on the Training Data Extraction Challenge dataset with GPT-Neo models along with the TOFU benchmark using Phi-1.5B and Llama2- 7B models show that our proposed Low- rank Knowledge Unlearning (LoKU) enables faster and more stable LoRA-based LLM unlearning, significantly outperforming existing baselines in computational efficiency as well as post-unlearning performance.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Armen Aghajanyan. Sonal Gupta, and Luke Zettlemoyer: Intrinsic dimensionality explains the effectiveness of language model fine-tuning, In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International
Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 7319-7328, Online, August 2021. Association for Computational Linguistics. doi: 1..18633/122022.acl-loooo 5668. URL wrg.aanoooooooooggg/222222
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel Kedziosked,i,ss,i, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word prohlem solving with operation-hased formalisms. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 2357-2367, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.188 1/N191 1245. URL httt:s:://acaaattoooooooo
org N919-N191122
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philiip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA Icarns less and forgets less. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
not/ http:nnntttoouutt Iorum? wow.unnddaaoorueeeee
Featured Certification.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceddiggs of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melamic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christo- pher Berner, Sam McCandlish, Alee Radford, Ilya Sutskever, and Dario Amodci. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877 1901. Curran Associates, Inc., 2020. URL http::p:///poocceeingssssece petppp/eepp..rr.rpppp..pp.ppppee files wwt.fffisssspee///eee 22000/200///11ee 1457c0d6b coeecc44447444 42
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlcarning. In 2015 IEEE Symposium on Security and Privacy, pp. 463 660. IEEE, 2015.
Nicholas Carlini, Florian Tramer, Eric Wallace, Mathew Jagielski, Ariel Berbbert-Voss. Katherine Lee, Adam Roberts, Tom Brown, Dawn Song. Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650, 2021.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL http:::///peeeevieeeee wttp://wwwwweeeeee.eeeeeeeeeeeee Tatttt 1cK
Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learning to unlearn: Instance-wise wise unlcarning for pre-trained classifiers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 11186-11 111111111 1944, 2024.
Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for LLMs. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12041-12052, Singapore, December 2023. Association for Computational URL. https: /
Linguistics. doi: 10.18653/11/2223.cmmm///
emm 1omnlpmmae ma in. 738
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmamn, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113,, 2023.
Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning deep networks using an incompetent teacher. Proceedings of the Thirty- Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. AA AAA.AAAAAAAAAAAAAAAAAA AAAI Press, 2023. ISBN 978-1-57735-880-0 doi: 10. w37/ 25879.
URL htttss////// .r.g 10. 1609/aaa v377 658879
11

---

Published as a conference paper at ICLR 2025
Peter Clark. Isaac Cowbey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv:1803.0545 2018.
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20:273-297, 1995.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powereed conversational agents. In International Conference on Learning Representations, 2019.
URL//tttp:///peereevie//
Abhimanyu Dubey, Abhinay Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint AxXi::2407.2188. 2024.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint 10Xiv:2111.00022 2020.
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304-9312, 2020.
Andrew Gordon, Zornitsa Kozareva, and Melissa Roemmele. SemEval-2011 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Eneko Agirre, Johan Bos, Mona Diab, Suresh Manandhar, Yuval Marton, and Deniz Yuret (eds.), *SEM 2012: The First Joint Conference on Lexical and Computotional Semantics Volume I: Proceedings of the main conference and the shared tosk, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012). pp. 394-398, Montréal, Canada, 7-8 June 2012. Association for Computational Linguistics. URL https: fttt:s::/aclaattooto
org/ S12- 1052
Michael M Grynbaum and Ryan Mac. The times sues openai and microsoft over of copyrighted work. The New York Times, 27, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034, 2015.
Yen-Chang Hsu, Ting Hua. Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. Language model com- pression with weighted low-rank factorization. 2022. URL httpp:::///peeeeeeeee feornerumm/taattttm id=
IPPv9Y3mAAI5
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference Learming Representations, 2022. URL (ttt:tts::///// http::::/pppeeeeeeeeeeeee net/ wttp://wwwww.t.etteeeeeeeeee. wiw.uumiidnneeeeeeeFe
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, 2023. URL htttts:///penttt
net Forum? d= 0KwE8 jrj
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. Anna Rogers, Jordan Boyd-Graber. and Naoaki Okazaki (eds.). Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 14389 113399-1440,, Toronto, Canada. July 2023. Association for Computationas
Computational Linguistics. doi: 10.18653/ 222 lon...0. URL https: ac hwl.ttoooootttooo...ooo..o. org/ 2.023. acl- long. 805
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A datasct for biomedical research question answering. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-LJCNLA) pp. 2567-2577, Hong Kong, China, November 2019, Association for Computational Linguistics. doi: 10.18653/v1/D1--120 12599. URL https: httt:e://aelanthooooy...oo -o-
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13)3521-3526 2017.
12

---

Published as a conference paper at ICLR 2025
Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmentee dialogue generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 8460-88800 8478, Dublin, Ireland, May
2022. Association for Computational Linguistics. doi: 1118665lw1/22222..c111n..... //aclanthology. org/ 2022. acl 1ong.579
URL https
Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M Asano. VeRA: Vector-hased random matrix adaptation. In The Twelfth International Conference on Learning Representarions, 2024, URL httpptt:/poppeeeeeeeeeecec net/ Forum? ? dddN'NNEexxr'a
Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In International Conference Learning Representations, 2018. URL | //apenrevies www.://oooooooiiirrrur-------- WCW
Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pp. 243-254, PMLR, 2022a.
Haokun Liu, Derek Tam, Mohammed Mugeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-effiient fine-tuning is hetter and cheaper than in-context learning. Advances Neural Information Processing Systemx, 35:1950-1965, 2022b.
Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng. et al. Purameter-effiiien orthogonal finetuning via butterfly factorization. arXiv preprint arXiv:2311.06244 2023.
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning. Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics ACI. 2024, pp. 1817-1829, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguisties. doi: 10.18653/v1/2022 1664/////02..nnniinnnn- aaddinng--cll101.. URL
httpps///cllantholooo..oro www.ohh/oolooro.2222.... http//222...iinngsnnii.. findings-acl.....
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL bRtpp:::://axx Dxq/abs/17111.5111
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. Tofu; A task of fictitious unlearning for Ilms. arXiv preprint arXiv:2401.0611 2024.
Ronak Mehta. Sourav Pal. Vikas Singh, and Sathya Ravi. Deep unlearning via randomized conditionally independent hessians. Proceedings of the IEEE/CVE Conference on Computer Vision and Patern Recognition, pp. 10422-10431, 2022.
Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations. 2017. URL [tttps://openrevieww /
Forum? forum?id=By. By
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring broad discourse context. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1525 1534, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 188536/1//111/ 1144. URL htttps:/accl httpss///aclaathhloo..ooooo wog.ooooy.ooooooooooooooo 1144
Zeju Qiu, Weiyang I. Lu,, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling text-to-image diffusion by orthogonal finetuning. Advances in Neural Information Processing Systems, 36:79320-79362 2023.
Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmaan,, Francis Song, John Aslanides. Sarah Henderson, Roman Ring, Susannah Young, al. Scaling language models: Methods, analysis insights from training gopher. arXiv preprint 1rXiv:2112.11440 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2025
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. Towards empathetic open-domain
conversation models: new benehmark and dataset. Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5370- 5381, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 111181//1//111--1
URL
logy- org/ P19 19- 1534
Jeffrey Rosen. The right to he forgotten. Stan. L- Rev. Online, 64:88, 2011.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99 106, 2021.
Nianwen Si. Hao Zhang, Heyu Chang, Wealin Zhang, Dan Qu, and Weiqiang Zhang. Knowledge unlearning for llms: Tasks, methods, and challenges. arXiv preprint arXiv:231 1.11616,,,176, 2023.
Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. Can you put it all together: Evaluating conversational agents' ability to blend skills. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Annual Meeting the Association for Compu- totional Linguistics, pp. 2021-2030 Online, July 2020. Association for Computational Linguisties. doi: URL
(0.18653/v1/2020...
183.
acl -main. 183
Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari. Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. Using deepspeed and megatron to train inegatron-turing nlg 530b, a large-scale generative language model. arXiv preprint 2xXx:2201.11990 2022.
Nathan Srebro and Tommi Jaakkola. Weighted low-rank approximations. In Proceedings of the 20th international conference on machine learning (ICML-03), pp. 720-727, 2003.
Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274-38290, 2022.
Eduard Fosch Villaronga, Peter Kieseberg, and Tiffany Li. Humans forget, machines remember: Artificial intelligence and the right to be forgotten. Computer Law & Security Review, 34(2):304-313, 2018.
Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). A Practical Guide, Ist Ed., Cham: Springer International Publishing, 10/3152676):10 10-555556611-55555_01 5555,2017.
Lingzhi Wang. Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. KGA: A general machine unlearning framework based on knowledge gap alignment. In Anna Rogers, Jordan Graber,, and Naoaki Okazaki (eds.). Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 13264 13276, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: (111188511122223.2 aal.ooo..cllllo..4444. URL http:::t///s... 2023. ac1 aacl-lone long. 740]
Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. DEPN: Detecting and editing privacy neurons in pretrained language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 2875-2886, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 18653/v1 00133/112222..mmmm 174. URL bttps: //t://acccaa http:/aaaanttttooooooooo org/2023.3....... emnip main. 174
Shih yang Liu, Chien- Yi Wang, Hongxu Yin. Pavlo Molchanov, Yu-Chiang Frank Wang. Kwang-Time Cheng, and Min-HHng Chen. DoRA: Weight-lecomppssed low-rank adaptation. In Forty-first International Conference on Machine Learning, 2024.
net / Forum?
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. In Lei Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Asxociation for Computational Linguistics ( (Volume 1: Long Pupers). pp. 8403 8419. Bangkok. Thailand, August 2024a. Association for Computational Linguistics. doi: 10.18653/ wwlo024.acl-oong... 457. URL htt.:httst//// waw ogy. wwwp:/llyyyooooooo////
457
Jin Yao, Eli Chien, Minxin Du Xinyao Niu, Tianhao Wang. Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 8403-8419, Bangkok, Thailand, August 2024h. Association for Computational I Linguistics. doi: 10. 18653/ 1//2024.acl-Iong URL. URI. wetp::tp//// / acl anthol wgw.ogg....ooo/..oooree acl -leng. 457
14

---

Published as a conference paper at ICLR 2025
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint arXiv:2310.1063 2023.
Shih- Shih-Ying Yeh, Yu-Guan Hsieh, Zhidong Gao, Bernard BW Yang, Giyeong Oh, and Yanmin Gong. Navigating text-to-image customization: From lycoris fine-tuiing to model evaluation. In The Twelfth International Conference on Learning Representarions, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence?? In Anna Korhonen, David Traum, and Lluís Marquez (eds.). Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, 4800, Florence. Italy, July 2019, Association for Computational Linguisties. doi: 10. 18655/v11P1991472 URL https //aglantho www.nhhoooy..gg/1111111111
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengeheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter--fficient fine-tuning. In The Eleventh International Conference on Learning Representations, 2023.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.00 05868, 2024.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al, survey large language models. arXiv preprint arXiv:2303. 18223, 2023.
15

---

Published as a conference paper at ICLR 2025
A DERIVATIVE ANALYSIS FOR THE INVERTED HINGE LOSS FUNCTION
The function [ represents probability distribution that indicates the likelihood of zy taking specific token æ given the previous tokes ><t-- This probability is expressed using the softmax function: pe(zz(<<<)) = exp(yt (=x) / // exp(yt ()) where y/ ( denotes the score for the v-th token in the vocabulary. To differentiate this function with respect yi (ae), rewrite Po (n [+))) = exp(y, ())))/()///))))))) 1/Z where = E exp(u(")) is the normalization constant.
(k) considering two cases: 1) k 24 and 2) k + Zt. We differentiate this function with respect yi For the first case, we can get the following by using the chain rule:
@p)(21(<<)) (21)
a
exp(Ut (2e))
Ô exp((yt (2t)
exp(yi (21)) az
Oyi Ove (x))
Z
Z
dvl (x1)
Z Oy: (21)
Here,
@exp(y) (=1) dy!
exp(y1 (()) and
87
exp(y) (.)) Therefore, it becomes:
ay!
@po(2s/aret)
exp(yt (2())
exp(ye (2))
po(x)(<<))) = (po(x((=e))
0ybm))
Z
Z
For the second case, using the chain rule again, get:
@po(2z(<<1)
exp(y) (())
exp(y1 (2e))
0z
aykh) (k)
Oyj
(k)
Z
Z
Oyt
(k)
= exp(y!" (t) Therefore,
where 0y:a]
0po(xz(aet)
exp(yi (2t)) exply)
-po(=,)=<)) po(k(
Oy: (2)
Z
Thus, we can summarize them as below:
@po(1e(a21)
if 2, (-po(ze(=<)) ( if xz
(1
Oyt (0)
Based on the derivative of po(x: (x))) above we can calculate the derivative of Lun. Firstly, for convenience, we define p: po(xx and p. max (po (v)(re)))) The loss function can be rewritten as:
Lum(a)) = pt pr
To calculate the derivative of Lun., we need to consider three cases: 1) when 2) when v = u² where V* = arg maX Pe(u) (e(v)x<e), 3) when v ل I, and v of v*. Using the derivative of pe pe(2(/12)) mentioned earlier, the derivative of LIHL with respect to (() is as follows:
0CILL 0y. (x,)
(1 + (o( (p(ullzz))
By. (xx)
= (o xx() - (1 (()) + (x xree) po(v)(
po (iee) (0 (re)) + po (pe)))
16

---

Published as a conference paper at ICLR 2025
(1 + po(x)p polooopoelllll
(w) Oyl"")
po((s)lece)) -00 po (r<t))
-po(u"lred) po (ee)) + po(
ŌLun 0
(1+ (o( po(v" (xe)))
Oyt (a) Ty (0)
(o((2)) (o( + (owee) po(v)
po (vlare) (ouvlllee - (o(sssare)))
In summary, the derivatives of the loss function Lun. with respect to yi (v) for the three cases
pa Ir (1e po(x)/ if Ir. po (ec) po(a)/azd (x()) -1) if o²
(x)
Oye (0)
pa (oloolees)oolllllee - (x if + xe and v # v²,
B EVALUATION METRICS
How to measure success of unlearning? Following previous work Jang al. (2023) Tirumala et al. (2022), we empirically measure the success of unlearning using two metrics, Extraction Likelihood (EL) and Memorization Accuracy (MA), which briefly discuss below.
After unlearning each sequence æ = (#i,...,, IT) E D,, the Extraction Likelihood (EL. is measured the n-gram overlap between the ground truth sequence and the output of the model after unlearning
OVERLAP, (A b) EcEn-GRAM(a 1fCE n-GRAM(b)) n-GRAM(a)) (x) = OVERLAP, (2)
(4)
ΣT-"
(5)
T- n
The Memorization Accuracy (MA) measures the token-wise memorization of the LM pa.
MA(x) Σ
(arg max, (0( xz}
(6)
T-1
Given these two metrics, flag successful unlearning when the average EL and MA Dj goes below the EL and MA values measured on the validation set unseen during training. In our experiments we measure EL with 10-grams, which results in the following early stopping criterion.
1 Σ MA(x) Deal œED
Σ EL)o( S Σ EL) and Σ MA(x) S 2ED,
Dj
D DED
D MED,
C ADDITIONAL DETAILS ON EXPERIMENTAL. SETTING
Experiemtal Settings All experiments were conducted on a remote server equipped with NVIDIA A100 40GB Tensor Core GPUs.
Datasets for Evaluation in the TDEC ferent classification datasets: LAMBADA
To evaluate reasoning capabilities, we utilize nine dif-
(Paperno al. 2016), Hellaswag (Zellers al. 2019).
17

---

Published as a conference paper at ICLR 2025
Winogrande (Sakaguchi et al.
2021 COPA (Gordon et al. 2012), ARC-Easy Clark al. 2018 8).
ARC- Challenge (Clark et al.
PiQA (Bisk et al., 2020), MathQA (Amini et al. [00 9), and PubmedQA (Jin al. 2019). To assess generative performance, employ Blended Skill Talk (Smith et al. 2020), Empathetie Dialogues (Rashkin et al. 2019), Wizard Internet (Komeili et al. 2022
2018).
and Wizard of Wikipedia (Dinan al. 2019).
Details of metrics of TOFU We evaluate the Forget Quality of unlearned models by measuring the p-value from the Kolmogorov-Smirov test that compares the empirical distribution of our unlearned model to that of the reference model. To evaluate how well the model retains other information outside the forget set, we measure the Model Utility the aggregated model performance on the retain set of remaining fictitious author profiles, and two held-out sets consisted of QA Q regarding real author profiles and other world facts.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 CONTINUAL. UNLFARNING
Because of the importance of continual unlearning (or sequential unlearning) in real-world applica- tions. previous studies have underscored its relevance through a sequence of unlearning tasks (Chal et al. 2024 Jang al. 2023). Building them, we conduct continual unlearning experiments involving four tasks. Figure of f the Appendix shows that IHL. consistently outperforms GD across all metrics. Notably, the proposed IHL demonstrates significantly enhanced performance on the four Dialogue and Pile datasets. Finally, we confirm that the combination of IHL and FLoRA achieves more robust and eesecefficeent continual unlearning. evidenced by the experimental results for Reasoning, Dialogue, and Pile, while utilizing only about 1.6% of the total parameters.
43.5
50
Ressoning (Acc)
43.0
Dialogue (FI)
40
HHL
42.5
30
The
20
41.5
10
Task Numbar
Task Number
Task Number
Figure 6: Experimental results of continual unlearning. Each task consists of 32 disjoint sequences sampled from the TDEC dataset, leading a total of 128 sequences to unlearn. For these experiments, we use the pretrained GPT--No 125M model. The experimental setup for unlearning and the forgetting criteria are configured as in the previous TDEC experiments. Task refers to the result before unlearning.
D.2 TOFU RESULTS WITH ADDITIONAL BASELINES
We compare our IHL and FILA methods against three additional existing unlearning baselines: KL (Maini et al. 2024) uses GA to forget samples in Dy while minimizing the Kullback-Leible (KL) divergence between representations of retain samples in D, output by the unlearned model and those from the pretrained base model. Instead of GA, Direct Preference Optimization (DPO; Rafailov et al. (2024) performs unlearning by training the model to output variants of "I don't know" when given a sample D,, NPO (Zhang al. 2024) an approach similar to GA. but with adaptive weighting on the gradients such that it alleviates the divergent behavior of GA. Note that both DPO and NPO are regularized by the LM-loss on D,.. We use the same hyperparameteriiztton (e.g., learning rate and effective batch size) as in our main results. Figure 7ssows the results. We find that all three baselines lead significant decrease in model utility, while IHL+FILA shows negligible change.
18

---

Published as a conference paper at ICLR 2025
Retain Set Only
Pretrained
GD
IHL
GD+FILA
RIL+FILA
GA
GA+FILA
KL
DPO
NPO
Farget 18 Quality
Bank 4
Bank =8
Bank -1
Bark - 32
(log p-vako))
-10
-10
-10
-10
M
M
20
0.0
0.5
1.0
0.0
0.5
100
0.0
0.5
1.0
0.0
0.5
10
Forget 5% Quality
(bg p-value!!
-10
-10
-10
-10
-20
-20
-20
-20
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Forget 10% Quality
(og p-vale))
0
0
-10
-10
-10
-20
20
20
20
1.0
0.5
0.0
1.0
Utilty
Uaitty
Model Ubitty
Model Utilty
(a) Phi-1.5B
Forget 355 Qualtty
00g p-velue)
-10
-10
-10
-20
-20
-20
-20
M
1.0
0.0
0.5
0.0
Farget 5% Qualty
(8ig 0ig (-vals))
0
0
-10
-10
-10
-20
-20
-20
-20
0.5
1.0
0.5
0.0
1.0
Forget 10% Quality
Bog p-value!
-10
-10
-10
-10
-20
-20
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Model Udiel
Modal Utiity
Model Utilty
Modal Utilty
(b) Llama2-7B
Figure 7: TOFU results using Phi-1,5B and Llama2-78 models. Each row corresponds to unlcarning different forget set (1%, 5%, 10%), and each column uses distinet LoRA rank between 4 and 32. The relative size of markers represent the number of epochs. Ideally, the unlearning curves should start from the pretrained model and approach towards the reference model tuned on the retain set only (*) as unlearning progresses. Our method IHL+FILA outperforms existing KL- and preference optimization-baser unlearning approaches in both model utility and forget quality.
19

---

Published as a conference paper at ICLR 2025
Perret Quality
Model Vulle
Medhod
Sei
PQ
Rouge-.
Ronin
MU
Original
0.53
0.46
1.15e-13
0.2
0.92
0.48
641
0,37
0.45
0.25
D441
0.50
0.52
Reasdo0
844
0.14
0.63
002 00
0.1
092
0.48
0.43
0.35
0.45
0.36
0.41
0.49
0.52
Forgeti))
KL
8:66
0.92
0.46
1.35e405
0.92
0.92
01:85
0.43
0.36
0.45
0.36
[641
0.90
0.52
DPG
0.96
8.99
0.48
k.87-415
092
0.92
0148
0.44
0.36
0.48
0.35
[141
0.50
01.2
NPO
0.56
8892
0.48
6:11e005
042
092
08:88
0.43
0.37
0..5
0.76
0.41
0.50
0.50
GA-FILA
8.64
1100)
0.76
1.07e-03
0106
0.27
0.03
0.30
01.00
QD
0.6
0.55
0.46
1.37e-05
0.92
0.92
0.45
0.42
0.36
0.45
0.36
D.41
0.50
0.52
IHL.
0.6
0.98
0.46
4:100405
0.9
0.22
0.48
0.43
0.36
0.45
0.75
D441
0.50
0..2
GD+I0
0.03
0.10
0.69
3.34e407
0106
0100
0.09
0.76
0.51
0.01
0.31
0100
FILA
1.25e.04
0.35
0.73
0.50
0.51
TOPU Forgei55
0.46
1.17e-13
NPO
062
0.48
0.51
4.87e-12
0.64
0.51
0.45
0.50
0.35
0.43
0.80
0.30
0.47
0.45
GA
941
0.46
0.51
1:10--11
0.63
0:48
01:5
0.85
0.36
0.43
0.80
0.79
0.46
01.57
GA-BI
0.99
8:0
0.71
1.60-408
8:10
0100
01:00
0.00
0.75
0.38
0.01
0.21
1100
4.35p-15
0.22
GD FILA
0.12
0.00
0.72
4.358 05
0.13
0.00
0.18
0.01
0.25
0.36
0.02
0.29
0.32
0.000
IHL+FILA
0:45
0.37
0.50
1.44e-11
0.79
0.46
01:08
0.43
0.31
0.00
0.75
0.41
0.50
0.51
Forgettl
KL.
0.01
0.00
0.71
1:3k-115
0101
0.10
0.16
0.09
0.76
0.54
0.00
0.37
0.25
11:01
DPO
0.41
0.57
0.49
5.10e-13
0.67
0.21
0.47
0.33
0.35
0.43
0.73
0.41
0.40
0.48
NPO
0.61
1.566-05
0.34
0.71
0.43
GA-Fil
9.00
8.00
0.15
-10e-1
0.03
0.00
0.25
0.34
0.00
0.31
0100
GD
0.17
8.19
0.51
1.55-000
0:41
0.78
0.66
0.19
0.35
0.60
0.16
0.66
01.96
IHL
0.53
8.67
0.49
2.45e-17
0.76
0.86
0.39
0.35
0.45
0.71
0.41
0.50
0.51
0.00
0.65
0.11
0.23
0.30
0.08
0.28
0.00
110
0.76
0.19
0.49
1,30-113
0.75
0.50
0.39
0.67
0.51
0.51
Table 2: Detailed TOFU Results using the Phi-1.5B model with LoRA rank
Firget Quality
Model Uiility
Mithol
Pick
PQ
Betin
Truhh
MU
Origital
0:93
0,95
0.45
1.15e-17
0.9
0:12
0.48
6.43
0.37
6.45
0.75
D.41
0.50
0.52
Rouidd0
844
8.14
0.63
100- 00
0.9
0.92
0.:88
0.43
0.38
0.25
0.16
0.45
0.52
Porgetl1
XL
891
0.91
0.48
5.1ee05
0.92
0.92
0.48
0,43
0,37
0.45
0.73
0.41
0.50
0.52
DPO
0.6
0.98
0.49
6.87c-05
0.92
0.92
0.48
0.43
0.36
0.45
0.76
[.41
0.50
0.52
NPO
8991
0.48
6.16001
0.9
0.97
08:03
0.43
0.37
0.76
0.50
0.50
GA-lill
8:00
8100
0.65
3.77-41
000
0.31
0.53
0.00
0120
0.34
0100
GD
059
0.52
0.45
1.37e-05
0.2
0.92
0.40
0,43
0.36
0.45
0.76
D.41
0.50
0.52
3RL
0.6
0.92
0.46
0.37c-05
0.92
0.92
0.40
0,43
0.35
0.45
0.75
0.41
0.50
0.52
GD<FILA
000
8/40
0.55
4.55g41
0.01
0.00
0.24
0.00
0.33
0.32
0.00
0.39
0.36
0.00
THI,PRIAA
0:47
0.35
0.51
3.37m404
01300
11:01
0.38
0.73
0.51
0150
TOPU Fongell55
DPO
1.21e-13
NPO
0.51
0.24
0.54
9.13e-11
0.50
0.25
0.43
0.45
0.34
0.42
0.76
0.37
0.44
0.41
IAA
0445
812
0.55
1-2e-100
0.45
0.13
0145
0.33
0.34
0.42
0.69
D.16
0.41
0..5
GA+FILA
8:0
0.54
1126-00
8.07
0:22
6.31
0.33
6.00
0.30
0.00
IHL
0.54
0.46
1.21e-13
0.40
0.37
0.72
0.49
11:00
GI-PILA
0.02
0.00
0.74
3.10e-07
0.02
0.00
0.16
0.00
0.26
0.32
9.08
0.29
0.33
11.00
110 A
0411
0.35
0.51
9.76-00
0.27
0.86
0148
0.34
0.37
0.07
0.71
0.50
0190
TOPU
Kl.
801
8.0
0.76
1.0p-116
0111
0.00
0:15
0.00
0.31
0.54
0.00
1..6
0.21
1100
DPO
0.39
0.57
0.49
1.15e-17
0.72
0.39
0.47
0.25
0.37
0143
0.73
0.41
0.49
0.47
GA+FILA
900
8:00
0.39
1.43--19
0.00
0.00
0..0
0.00
0.34
0.36
0.20
0.34
0.00
GD
022
8.64
0.55
5.07-006
0.27
0.13
0.43
0.07
0.35
0.455
0.44
1L36
0.45
0.22
THI
0.66
8155
0.49
3.45e-17
0.77
0.85
0.34
0.36
0.25
0.59
0.61
0.50
0.50
0.10
0.19
0.39
0/24
0.37
0.88
0.09
0.52
1.40-005
0.23
0.34
0.69
0.30
041
0.51
0.66
0.51
0.50
Table 3: Detailed TOFU Results using the Phi-1.5B model with LoRA rank
20

---

Published as a conference paper at ICLR 2025
Perret Quality
Model Vulle
Medhod
Furger Sei
PQ
Rouge-
Ronin
MU
Original Reasdo0
0.53
0.46
1.15e-13
0.2
0.92
0.48
641
0.37
0.45
0.25
D441
0.50
0.52
844
0.14
0.63
002 00
091
0.92
0.48
0.43
0.35
0.45
0.36
0.41
0.49
0.52
Forgeti))
KL
0.64
0.86
0.46
1.85e405
0.91
0.92
01:85
0.46
0.31
0.45
0.75
B641
0.40
0.55
DPG
0.56
8.95
0.40
7.302405
091
0.92
01:8
0.42
0.36
0.48
0.76
D.42
0.51
01.2
NPO
0.52
8.36
0.48
4.17e-05
041
093
08:88
0.44
0.37
0..5
0.76
0.41
0.89
0.50
GA-FILA
1100)
0.64
1.34e-02
0101
0.19
0.22
0.25
01.00
GD
0.88
0.58
0.46
1.37e-05
0.2
0.92
0.49
0.40
0.36
0.45
0.36
0.41
0.50
0.52
IHL.
0.58
0.50
0.46
1:2e-04
0.91
0.22
0.40
0.42
0.36
0.45
0.76
[44
0.50
0..2
GD+B01A
0.03
0110
0.40
2.44e-01
0.07
0100
0.09
0.23
0.27
0.01
0.30
0.35
0100
FILA
1.70e003
0.75
0.39
0.72
0.51
TOPU Forgei85
1.55e40
NPO
0.46
0.16
0.56
1.10-00
0.45
0.17
0.41
0.36
0.34
0.41
0.66
0.36
0.41
0.35
GA
0.11
8/0
0.71
4.35-01
0.21
0.00
0.76
0.01
0.76
6.33
0.94
0.26
0.31
(1:0)
GA-BI
0.00
8:0
0.40
5.00-409
817
0100
0.00
0.75
0.34
0/00
0.34
1100
1.75-009
0.41
GD UDYPILA
0.68
0.00
0.69
1,81e-05 05
0.07
0.00
0.17
0.01
0.25
0.30
0.05
0.27
0.27
0.000
[HL+FILA
0.56
81:11
0.51
5.05e-08
0.75
0:44
01:0
0.43
0.31
0.47
0.71
0.43
0.51
0.52
Forgettl
KL.
0.11
0.00
0.70
1.07e-13
0111
0.10
0.18
0.09
0.39
0.31
0.00
0.35
0.31
11:01
DPO
0.12
0.36
0.48
5.40c-18
0.76
0.90
0.48
0.32
0.37
0.43
0.72
0.41
0.40
0.48
NPO
0.65
4:600044
0.45
0.35
0.33
0.69
0.42
GA-Fil
9:00
8100
0.11
6.10-13
0.03
11:38
0.34
0.51
0.00
0.41
0100
GD
0.30
8.00
0.51
4.70-12
0.24
0.17
0.46
0.0
0.35
0190
0.78
D:42
0.80
01:5
IHL
0.41
0.38
0.51
1,46e: 14
0.77
0.85
0.36
0.35
0.40
0.69
0.42
0.53
0.50
0.88
0.00
0.50
1.16e-05
0.09
0.22
0.31
0.04
0.34
0.00
110
0.08
1.31p00
0.20
0.62
0.80
0.67
0.55
0.50
Table 4: Detailed TOFU Results using the Phi-1.5B model with LoRA rank 16.
Firget Quality
Model Uiility
Mithol
Pick
PQ
Betin
Truhh
MU
Origital
0:93
0,95
0.45
1.15e-17
0.9
0:12
0.48
6.43
0.37
6.45
0.75
D.41
0.50
0.52
Rouidd0
844
8.14
0.63
100- 00
0.1
0.92
0
0.43
0.38
0.25
0.16
0.41
0.45
0.52
Porgett1
XL
968
0.72
0.48
4.17o-05
0.87
0.90
0.40
0,43
0,37
0.45
0.73
0.41
0.49
0.52
DPO
0.64
0.90
0.51
4722-04
0.37
0.91
0.47
0.43
0.36
0.45
0.76
0.42
0.52
0.52
NIO
8.6
8171
0.49
K006601
0.87
0.89
0.07
0.37
0.75
0.80
0.51
GA-lill
8000
0.76
5.55-00
010
11
0.36
0.27
0.01
0.76
0.24
0100
GD
R66
0.78
0.45
5.87e-05
0.90
0.92
0.40
0,40
0.36
0.45
0.75
D.41
0.50
0.52
3RL
065
0.77
0.46
1.28c00
0.90
0.92
0.40
0,42
0.35
0.45
0.76
0.41
0.50
0.52
GD<FILA
0.04
0/00
0.77
1116-00
0.01
0.00
0.12
0.00
0.35
0.24
0.00
0.36
0.00
THI,PRILA
0.37
8118
0.61
1.00e-01
0.71
11:01
0.39
0.73
0.51
0150
TOPU Fungelll5
DPO
3.17e-15
0.76
0.37
NPO
0.45
0.17
0.61
3.64c-05
0.46
0.19
0.38
0.37
0.33
0.40
0.68
0.36
0.43
0.36
IA
800
800
0.76
1-17-13
0:1
0.00
8.16
0.00
0.25
0.26
0.00
11.6
0.24
8100
GA-FILA
8:4)
0:31
4.77-13
8.00
0.35
6.31
0.35
0/00
0.31
0.00
IHL
0.42
0.50
41
0.79
0.00
0.35
0.71
0.50
0151
GILPILA
0.04
0.00
0.71
4.100 06
0.15
0.00
0.17
0.00
0.25
0.23
9.02
0.27
0.28
0..00
110 A
0.16
0.04
0.90
1.006401
0.21
0.31
0148
0.37
0.18
0.59
0.51
0.90
TOPU
Kl.
801
0/0
0.60
1.17x406
0111
0.00
1.17
0.00
0.39
01:33
0.00
11.5
0.80
6100)
DPO
0.28
0.55
0.46
3.5tc-18
0:31
0:11
0.488
0.32
0.37
0143
0.71
0.41
0.40
0.00
0.67
1.17--06
GA+FILA
900
0.00
0.21
4.220-21
0.00
0.00
0.35
0.00
0.20
0.53
0.20
0.27
0.44
0.00
GD
0.11
8.00
0.45
1335-00
0.30
0.34
0.42
0.09
0.35
0.53
0.34
0.1
0.53
0129
THI
0.36
0:20
0.51
18
0.31
0.86
0.02
0.39
0.47
0.30
0.43
0.51
0150
0.10
0.37
0.35
0.03
0.80
0.35
0.02
0.58
3.08o412
0.66
0.78
0.46
0.43
0.38
0.72
0.53
0.51
Table 5: Detailed TOFU Results using the Phi-1.5B model with LoRA rank 32.
21

---

Published as a conference paper at ICLR 2025
Perpet Quality
Model Vulle
Medhod
PO17
Roage-.
Ronin
MU
Osiginal
0.99
0.99
0,51
2.1b--00
0.8
0.99
0.47
0.94
0,47
0,22
0.59
0443
0.55
0.63
Reasd00
0:40
0115
0.67
100 00
0.9
0.99
0.47
0.92
0.47
0.61
0.58
0.42
0.55
0.63
Forgeti))
KL
R95
0.99
0.55
077e405
0.98
0.99
0142
0.94
0.47
0.62
0.90
B445
0.55
0.63
DPG
0.5
0.99
0.56
1.40-04
0.95
0.99
08:42
0.30
0.41
0.62
0.49
D145
0.55
(163
NPO
0.95
1859
0.55
93
0448
039
01.07
0.93
0.47
01.2
0.89
0.43
0.50
0.65
GA-RILA
1100
0.87
3.13-00
0164
0.25
0.01
0.35
11.00
QD
895
0.50
0.55
1.0c-04
0.58
0.99
0.47
0.94
0,47
0.82
0.89
0.45
0.55
0.63
IHL.
035
0.59
0.55
1.77-004
0.98
0.99
0.43
0.94
0.47
0.82
0.90
0.45
0.55
0.63
GD+I0
0.05
0110
0.87
1.15x00
0.01
0100
0.11
0.05
0.35
0.21
0.00
0.74
0100
FILA
1.55e.04
TO3R Forgei85
0:15c-17
0.46
NPO
0.99
0.55
0.54
1.47e-15
0.95
0.93
0.46
0.94
0.45
0.63
0.90
0.45
0.57
0.66
GA
0:90
8.96
0.54
6.50e116
0146
0.48
11:66
0:94
0.45
0863
0.90
0.45
0:51
(160
GA-BILA
801
810
0.11
173e-15
0101
0100
0.00
0.24
8.17
0.00
0.24
010
6.50-16
045
0.64
0.53
GD FILAILA
0.02
0.00
0.37
1.506-08 08
0.33
0.00
0.14
0.01
0.24
0.17
0.00
0.24
0.21
0.00
IHL-FILA
0.56
0.700
0.58
6.87e-13
0.50
0.44
0.45
0.93
0.48
0.62
0.59
0.47
0.60
0.66
Forgettl
KL.
0:47
0.33
0.65
3.90-00
01407
0:31
0:35
0:93
0.41
0.55
0.89
D:43
0.55
11:00
DPO
0455
0.98
0.55
5.10e-13
0.65
0.96
0.44
0.82
0.43
0.54
0.87
0.42
0.5)
0.57
NPO
0.65
3.35e-05
0.51
GA-Fil
0.00
8.00
0.46
1.4b-18
0.22
1100
0.34
0.00
0.18
0100
GD
0.62
0.92
0.51
1.10-16
8993
0.47
01:0
0.90
0.45
1100
0.48
D.42
0.54
0162
IHL
0.73
0.57
0.57
3.71c-15
0.388
0.33
0.45
0.34
0.8
0.64
0.89
0.47
0.59
0.64
0.1
0.00
0.55
1.856-21
0.01
0.09
0.25
9.00
0.18
0.00
110 100+
0.70
0.38
0.65
3.9
0.91
0.65
0.81
0.58
0.57
0.5
Table Detailed TOFU Results using the Llama2-7B model with LoRA rank = 4.
Firget Quality
Model Uiility
Mithol
Pick
PQ
Betin
MU
Origital
0.99
1.59
0.51
3.19e:20
0.98
0.99
0.47
694
0,47
0.62
0.59
D.43
0.55
0163
Rouidd0
0:40
8:15
0.67
100g 00
0.48
0.99
0.47
0.92
0.47
0.61
0.58
0.35
Porget81
XL
865
0.58
0.56
1.1e-044
0.98
0.99
0.47
0.93
0,47
0.62
0.90
0.43
0.56
0.64
DPO
0.6
0.59
0.56
1.40-004
0.98
0.99
0.47
0.94
0.47
0.02
0.50
0.45
0.36
0.63
NIO
838
1898
0.11
671640
0398
0.93
0.47
0.94
0.47
0.67
0.40
0.65
GA-lill
801
8100
0.46
3.12601
010
1100
0.34
0.23
0.00
0134
0.17
1100
GD
885
0.50
0.55
1.17-004
0.98
0.99
0.47
0.93
0,47
0.62
0.59
0.43
0.56
0.63
JRL
0.5
0.59
0.55
1,40c.04
0.58
0.99
0.47
0.34
0447
0.61
0.50
0.45
0.35
0.63
GD
0000
8/00
0.55
1.55-05
0.07
0.00
0.09
0.00
0.35
0.00
0.19
0.00
THI,PRIAA
0.:8
0.55
3.606-04
0.97
01:07
0.86
0.59
0.54
0162
TOPU Forggel55
6.50k-16
DPO
0.50
NPO
B62
0.72
0.57
1.0%-14
0.68
0.76
0.44
0.87
0.46
0.61
0.38
D.46
0.57
0.60
IA
8178
0.56
1:31e-13
0.71
0.37
0.89
0U41
0.00
0.90
D146
0.51
8161
GA+FILA
801
8:6
0.44
1:12-118
0.01
8.07
0.01
0.19
0/00
0.19
0.00
IHL
0.58
0.53
1.24e-17
0.98
0.47
0.90
0.56
GILPILA
9.02
0.00
0.56
4.80e 188
0.1
0.00
0.09
0.03
0.25
0.19
9.01
0.34
0.19
0.00
110 A
0:40
0.79
0.63
4.77x47
0.36
0.94
0.45
0.48
0:41
0.90
0.3
TOPU
Kl.
8.11
010
0.65
1.31640
0.51
0.00
11:22
0.07
0.30
01:0
0.63
1..4
0.08
1101
DPO
0.36
0.96
0.54
5.10e-17
0.67
0.96
0.45
0.65
0.42
0.53
0.54
0.40
0.49
0.55
0.57
1.32-08
GA+FILA
900
8.00
0.51
3.32-00
0.00
0.00
0.15
0.00
0.25
0.20
0.24
0.00
GD
044
8.47
0.49
3.1c-15
0.52
0.67
0.48
0.87
0484
0.58
0.57
[.40
0.55
0156
THI
0.59
8.77
0.58
3.86e-14
0.36
0.91
0145
0.93
0.51
0160
0.59
0.4%
0.63
0.65
0.55
1.15--17
0.04
0107
0.01
0.25
0.01
0.16
035
0.54
3.636407
0.91
0.90
0.44
0.85
0.49
0.64
0.56
0.50
0.6
Table 7: Detailed TOFU Results using the Llama2-7B model with LoRA rank
22

---

Published as a conference paper at ICLR 2025
Perret Quality
Model Vulle
Medhod
Sei
PQ
Roage-1
Ronin
MU
Osiginal
0.99
0.99
0.51
2.1be-20
0.98
0.99
0.47
0.94
0,47
0,22
0.59
0443
0.55
0.63
Reasdo0
0:40
0115
0.67
100g 00
0.98
0.99
0.47
0.92
0.47
0.61
0.58
0.42
0.55
0.63
Forgeti))
KL
8:00
8.96
0.56
1.0r0404
0.98
0.99
0147
0.90
0,48
0.62
0.89
D:44
0.55
0.64
DPG
0.98
0.99
0.56
1332-04
0.9
0.99
08:66
0.34
0.48
0.62
0.90
D144
0.55
0163
NPO
0.89
1850
0.55
k.00-05
0448
0.99
01.07
0.93
0.47
01.2
0.89
0.43
0.56
0166
GA-RILA
0:00
1100)
0.69
1.5e-00
0111
0.17
0.37
0.01
0.31
11.00
QD
092
0.97
0.35
1:28-00
0.58
0.99
0.47
0.94
0,47
0.82
0.89
0.45
0.55
0.63
IHL.
0.68
0.97
0.55
1.40-04
0.98
0.99
0.42
0.30
0.47
0.61
0.90
D445
0.55
0.63
GD+I0
0.00
0.10
0.67
1.35-00
0.01
0100
18
0.05
0.21
0.27
0.01
0.78
0.71
01.00
FILA
7.15e.04
TOPRU Forgei85
1.800-12
NPO
047
0.46
0.63
6.80c-10
0.48
0.32
0.39
0.89
0.44
0.57
0.90
0.45
0.57
0.53
GA
0:48
8.45
0.43
1:15-005
01:00
0.90
11
0.90
0.41
0.53
0.90
0.43
0.54
01.00
GA-BILA
8.3
810
0.14
1.00-013
001
0100
1115
0.01
0.24
0.73
0.08
0.19
0100
0.51
0.56
0.61
3.43x-16
0.54
0445
GD FILA
0.05
0.00
0.80
4.19g-10 10
0.06
0.00
0.12
0.01
0.25
0.24
0.02
0.26
0.23
0.00
IHL-FILA
0.55
01:18
0.69
1.34e-02
0.358
0.55
08:6
0.93
0.47
0.62
0.59
0.44
0.58
0.63
Forgettl
KL.
0:00
0.00
0.81
1.
0103
0:00
01191
0.09
0.31
0.73
0.00
0.35
0.23
01:01
DPO
0.27
0.56
0.53
1.00-11
0.73
0.97
0.45
0.55
0.40
0.52
0.45
0.39
0.48
0.54
NPO
0.70
1.66-04
0.47
0.41
0.30
GA-Fil
9.0
8100
0.90
4.07-311
0.03
0.04
0.34
0.17
0.00
0.87
0100
GD
0.19
8.06
0.41
1.4l-11
0.:4
0.19
0.50
0.81
0.42
01.88
0.46
0.16
0.54
08.00
IHL
0.43
0.52
0.63
57
0.39
033
0.45
0.90
0.52
0.67
0.89
0.48
0.63
0.65
0.00
0.70
2.60-002
0.04
0.16
0.27
0.31
9.02
0.27
0.00
110 +00
0.64
0.79
8:60-110
0.91
0.64
0.50
0.57
0.50
0.60
Table Detailed TOFU Results using the Llama2-7B model with LoRA rank 16.
Firget Quality
Model Uiility
Mithol
Pick
PQ
Betin
MU
Origital
0.99
1.59
0.51
3.19e:20
0.98
0:99
0.47
694
0,47
0.62
0.59
D.43
0.55
0163
Rouidd0
0:40
8:15
0.67
100g--0 00
0.48
0.99
0.47
0.92
0.47
0.61
0.58
0.35
Porget81
XL
0788
0.50
0.56
5.05c-05
0.96
0.98
0.47
0.93
0.46
0.62
0.90
0.44
0.56
0.64
DPO
0.58
0.97
0.56
537c-04
0.90
0.97
0.46
0.90
0.45
0.63
0.50
0.45
0.36
0.63
NPO
8.75
8.90
0.11
1.40-00
0.95
0.48
0.47
0.93
0.4k
0.63
0.40
0.56
0.60
GA-lill
8100
0.71
1.30-01
0107
0115
0.34
0.21
0.00
0.2%
0.14
0100
GD
0.52
0.91
0.55
1.53e004
0.96
0.98
0.47
0.93
0,47
0.62
0.58
0.43
0.56
0.63
JRL
0.76
0.50
0.55
2.18c
0.96
0.98
0.47
0.93
0447
0.61
0.50
0.45
0.35
0.63
GD
0.04
8/00
0.81
166660
0.01
0.00
0.14
0.00
0.34
0.21
0.01
0.19
0.00
THI,PRIAA
0445
0.30
0.61
4.15e40
0.:0
01:7
0.85
0.38
0.51
0.61
TOPU Fongeel55
DPO
2.72x-14
0.44
0.42
NPO
0.46
0.29
0.70
1.81e05
0.47
0.33
0.32
0.94
0.40
0.51
0.91
0.43
0.54
0.47
IAA
044
8/6
0.50
1.346-01
0.34
0.11
11.00
0.83
0.31
0..00
0.03
0.17
0.50
0.50
GA+FILA
801
8:4
0.56
2.47e-16
800
8100
0.34
0.01
0.19
0.00
IHL
2.12-112
0.588
0.47
0.90
0.57
GI-PILA
0.11
0.00
0.68
1.00-003 03
0.11
0.00
0.18
0.01
0.24
0.27
0.03
0.31
0.23
0.00
110 A
0:10
8.09
0.75
1.16-01
0.87
0.95
0148
0.91
0.45
0.48
0.54
0167
TOPU
Kl.
8000
010
0.46
4.30p-1
0100
0.00
11177
0.00
0.25
0.25
0.00
1..5
0.21
[100)
DPO
0.23
0.96
0.52
5.10e-17
0.38
0:58
08:6
0.75
0.40
0.50
0.86
0.38
0.47
0.56
0.71
GA+FILA
900
0.00
0.51
3.30e-05
0.0
0.00
0.15
0.00
6.25
0.32
0.20
0.30
0.00
GD
901
8.00
0.58
5.442-08
0.18
0.30
0.65
0.43
0466
0.63
0.61
D445
0.58
0.38
THI
0.14
8111
0.79
1.22e
0.44
0.97
01:05
0.88
0.48
0162
0.58
0.54
0.63
0.04
0.11
0.34
0.01
0.31
066
8.02
0.73
3377445
0.90
0.96
0.48
0.92
0.82
0.67
0.37
0.60
0.6
Table Detailed TOFU Results using the Llama2-7B model with LoRA rank 32.
23


Reviews:
[{"review_id": "MqYPzWiOKJ", "reviewer": "Reviewer_EhHm", "summary": "The paper proposes a framework to remove sensitive information from LLMs without retraining them from scratch. Recognizing the limitations of common unlearning methods like Gradient Ascent (GA), which risks instability and unintended forgetting, the authors introduce two new techniques. The Inverted Hinge Loss (IHL) method enhances stability by suppressing unwanted tokens with the next most likely alternative, while the Fisher-weighted Initialization of Low-rank Adapters (FILA) uses Fisher information to initialize LoRA adapters, selectively targeting parameters associated with unwanted information to optimize unlearning. This dual approach was evaluated on the Training Data Extraction Challenge and TOFU benchmark with models such as GPT-Neo, Phi-1.5B, and Llama2-7B, achieving efficient unlearning with minimal loss to the model’s reasoning and generative capabilities, and demonstrating improved performance over existing methods.", "questions": "1. In Figure 3, why are some data points missing? \n2. It is better to add legends in Figures 3 and 4 to improve the clarity.\n3. It is better to define “Model Utility” within the paper instead of referring readers to other papers.\n4. For the Hinge loss equation in Line 233, since the probability p(.) is in the range of (0,1), the second item within max() function is always larger than 0, right? If so, IHL is to reduce the probability of true tokens but to increase the probability of other tokens, right?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The motivation is clearly explained. \n2. Extensive experiments have been conducted to prove the effectiveness of the proposed methods. \n3. The theoretical analysis strengthens the rationale of the proposed methods.", "weaknesses": "1. One of the most significant contributions of this paper is the proposal of Inverse Hard Loss (IHL), which claims to increase the probability of the second-best token only. However, it is not clear why IHL does not affect the probability of other tokens. Based on the definition of IHL in Lines 233, the probability of all other tokens is impacted. As such, IHL can only address problem 1 (Line 224) but cannot address problems 2 and 3 of GA (Lines 224 ~ 226).\n2. In Figures 3 and 5, the unlearning performance of employing only IHL (represented in green) does not outperform the GD baseline (depicted in blue), which undermines the effectiveness of IHL. \n3. The main results only use GPT-neo models, which are old models. It is better to use more recent models like Llama and Mistral models to make it more practically useful. It is also inconsistent to use different models for main results and analysis. \n4. There are no ablations studies for the following settings: 1) full parameter fine-tuning with IHL; 2) LoRA + FILA only; 3) GD + LoRA + FILA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FK3NWPDtGm", "reviewer": "Reviewer_T4eT", "summary": "The paper works on machine unlearning in LLMs, particularly focusing on the challenges of removing specific data instances from a model's memory without retraining from scratch. The authors propose two strategies: Inverted Hinge Loss (IHL) and Fisher-Initialization of Low-rank Adapters (FILA). IHL is designed to replace the unbounded negative cross-entropy loss in gradient ascent with a more stable and efficient loss function. FILA aims to initialize low-rank adapters in a way that prioritizes the removal of unwanted information and accelerates the unlearning process. Extensive experiments validates that the proposed methods significantly outperform existing baselines in efficiency and post-unlearning performance.", "questions": "How robust are the proposed methods to changes in the data distribution of the forget set? For instance, if the forget set contains highly diverse or outlier data, would the unlearning process still be effective?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Authors analyze the derivatives of GA and highlight its shortcomings, the motivation is clear and the theoretical foundation strengthens the rationale for the proposed methods.\n2. The introduction of IHL addresses the instability issues of GA by focusing gradient updates on a minimal number of viable replacements for the ground-truth token. This results in a more controlled and stable unlearning process.\n3. The proposed strategies are effective. The authors evaluate the methods on multiple datasets and multiple model sizes. This comprehensive evaluation demonstrates the robustness and generalizability of the proposed methods.", "weaknesses": "The intuition and connection between the proposed methods, IHL and Fisher-Initialization of FILA, appear somewhat weak. This makes the paper feel like it is stacking two separate tricks rather than offering a unified and coherent approach. A more systematic linkage between these methods would enhance the overall coherence and impact of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cLK3HPIV0W", "reviewer": "Reviewer_sxtU", "summary": "The authors identify the limitations of current unlearning methods (e.g., Gradient Ascent (GA)), which can lead to unstable optimization and catastrophic forgetting of retrained knowledge. To overcome these challenges, the paper introduces two novel techniques for robust and efficient unlearning:\n\n1. **Inverted Hinge loss (HIL):** This new loss function suppresses unwanted tokens while maintaining fluency by boosting the probability of the next most likely token.\n\n2. **Fisher-Initialization of Low-rank Adapters (FILA):** Developed through low-rank approximation weighted with relative Fisher information, this method focuses updates on parameters critical for removing targeted knowledge.\n\nThe paper demonstrates the effectiveness of these techniques through experiments on the Training Data Extraction Challenge dataset using GPT-Neo models and on the TOFU benchmark with Phi-1.5B and Llama2-7B models. The proposed approach successfully removes sensitive information while maintaining the reasoning and generative capabilities of the models with minimal impact on performance.\n\nIn summary, this paper provides innovative solutions to the drawback of GA and demonstrates the effectiveness of the solutions.", "questions": "What is the reason for deriving the unlearning mechanism of GA from the formulas in lines 217-220?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### Originality\n- This paper points out the shortcomings of Gradient Ascent (GA) by analyzing its inverse. \n- This paper proposes two strategies to improve these shortcomings. \n- This paper demonstrates the effectiveness of their improvements on two datasets.\n\n### Clarity\n- The structure of this paper is clear, and most of the content is explained clearly.\n\n### Significance\n- This paper provides insights into knowledge unlearning through the analysis of Gradient Ascent (GA).", "weaknesses": "- This paper lacks the state-of-the-art knowledge unlearning baselines (such as [1][2]). Although the main goal of the paper is to address the shortcomings of GA, incorporating the state-of-the-art knowledge unlearning for comparison would make it more convincing.\n\n- Some descriptions are not clear enough. For example, lines 221-223 should include more explanation for the reasons. The authors should explain in detail why GA increases the prediction score for all other tokens $v \\neq x_t$ in the vocabulary.\n\n- From the experimental results, when only IHL is used, the performance is worse than the original GA. Does this contradict the paper's claim that IHL is designed to address the shortcomings of GA and the analysis of derivatives of GA?\n\n- The paper devotes too much content to the background of knowledge unlearning in the Abstract and in the first paragraph of the Introduction. Since knowledge unlearning is a common problem, I believe it is unnecessary to describe it in such detail. The main content should focus on describing the research conducted in this paper. Specifically, Figure 1 should illustrate the proposed approach rather than the knowledge unlearning problem.\n\n[1] Zhang R, Lin L, Bai Y, et al. Negative preference optimization: From catastrophic collapse to effective unlearning[J]. arXiv preprint arXiv:2404.05868, 2024.\n\n[2] Gao C, Wang L, Weng C, et al. Practical unlearning for large language models[J]. arXiv preprint arXiv:2407.10223, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7AbGjZNoG7", "reviewer": "Reviewer_AAXs", "summary": "The paper focus on the problem of unstable optimization, catastrophic forgetting, and computational cost from Gradient Ascent for LLM unlearning, and propose two novel techniques, including the Inverted Hinge Loss and Fisher Information weighted initialization of LoRA adapters, for robust and efficient unlearning for LLMs. Experiments on two tasks with different LLMs show that the proposed methods enable faster and more stable LoRA-based LLM unlearning.", "questions": "1. In Introduction Section (Line 72-74), can you explain more about the reason that low-rankness can be beneficial in stabilizing optimization and preventing catastrophic forgetting? Clearer illustration would be better here.\n\n2. In Table 1, why you record results from running different epochs? Does it mean the method reaches the optimal with these epochs?\n\n3. In Experiments Section, why different LLMs are used for those two tasks? Have you evaluated more popular and larger LLMs such as Llama3.1-8B? I suggest giving explanation of the strategy and purpose of model selection.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper makes a good contribution to knowledge unlearning of LLMs through improving tuning stability and efficiency with Inverted Hinge Loss and Fisher-Initialization of Low-rank Adapters, respectively. The proposed method is valuable of managing unnecessary forgetting and unbounded optimization in typical Gradient Ascent strategies.\n2. The paper is generally well-written, particularly in formula derivation and clear explanation about IHL and FILA solve the weaknesses of GA in Section 3.3 and Section 3.4.\n3. The experiments and analysis of the paper is comprehensive, with great illustration on performance evaluation using high quality charts.", "weaknesses": "1. In Introduction Section (Line 51-53), you mention GA and its shortcomings, I think a better way of writing here would be providing a brief overview of 2-3 other key knowledge unlearning approaches beyond GA, and summarize 1-2 common shortcomings across these methods that motivate the proposed approach. GA should be only one of those existing typical methods.\n\n2. In Introduction Section (Line 76), you mention the application of LoRA to LLM unlearning remains unexplored, however, there are some existing studies using LoRA for LLM unlearning, including Practical Unlearning for Large Language Models (https://arxiv.org/abs/2407.10223) and Machine Unlearning in Large Language Models (https://arxiv.org/abs/2405.15152v1). It would be better to briefly summarize (in 1-2 sentences each) how LoRA was used for unlearning in these two papers, and then explain how their proposed approach differs or improves upon these methods.\n\n3. Some important content is missing. In Introduction Section, a lack of clear summarization of contributions in the paper, making readers difficult to capture the important points of the study. Besides, in Related Work Section, a brief comparison between your proposed method and other relevant studies should be presented to better emphasize the advantages of your work. \n\n4. In Section 3.3 (Line 223-227), you should provide a more detailed explanation of how you arrived at these hypotheses. There is still a gap between GA motivation and its weaknesses. To make the illustration more convincible here, a better way would be providing a specific example or mathematical derivation showing how the GA loss function leads to one of the stated problems (e.g., unbounded optimization or inefficient gradient updates).\n\n5. In Section 3.4 (Line 265), a basic and brief description of Fisher Information is necessary here for better understanding of the reason you employ it to address the importance quantification you mentioned before.\n\n6. In Table 1, to make the table more readable, the data of important results should be highlighted via bolding the best result in each row or using color coding to show relative performance between methods, in order to show either your FILA LoRA performs much better than traditional LoRA, or it can approach the performance of full fine-tuning.\n\n7. There are some writing flaws:\n* Some sentences are too long to follow and comprehend, such as Line 89-93.\n* In Line 419-421, there are two \"Second\" in these two sentences, making them difficult to be understood.\n* The capitalization in the text should be more consistent. For instance, you use lowercase \"l\" in \"Inverted Hinge loss\" at Line 20 and Line 161, but uppercase \"L\" in \"Inverted Hinge Loss\" at Line 82. All uppercase would be better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)