GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
Authors: Yue Wang
Qizhou Wang
Feng Liu
Wei Huang
Yali Du
Xiaojiang Du
Bo Han
Venue: ICML 2025 poster (2025)
Keywords: Large Language Model Unlearning
Primary area: deep_learning->large_language_models

Abstract: Large language model (LLM) unlearning has demonstrated its essential role in removing privacy and copyright-related responses, crucial for their legal and safe applications. However, the pursuit of complete unlearning often comes with substantial costs due to its compromises in their general functionality, leading to a notorious trade-off between unlearning and retention. It motivates this paper to explore enhanced unlearning schemes that can mitigate this trade-off. Specifically, we propose Gradient Rectified Unlearning (GRU), an improved framework that regulates the directions of gradient updates during the unlearning procedure such that their side impacts on other, unrelated responses can be minimized. GRU is easy and general to implement, demonstrating practical effectiveness across a variety of well-established unlearning benchmarks.

GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
Yue Wang' Qizhou Wang' Feng Liu?
Wei Huang
Yali Du Xiaojiang Du Bo Han
Abstract
bayev et al., 2023; Roziere al., 2023; Wu et al., 2023; Thirunavukarasu al., 2023).
Large language model (LLM) unlearning has demonstrated its essential role in removing pri- vacy and copyright-rellted responses, crucial for their legal and safe applications. However, the pursuit of complete unlearning often comes with substantial costs due to its compromises in their general functionality, leading to notorious trade- off between unlearning and retention. It moti- vates this paper to explore enhanced unlearning schemes that can mitigate this trade-off. Specifi- cally, we propose Gradient Rectified Unlearning (GRU), an improved framework that regulates the directions of gradient updates during the un- learning procedure such that their side impacts other, unrelated responses can be minimized. GRU is easy and general to implement, demon- strating practical effectiveness across a variety of well-established unlearning benchmarks. Our code is available at https: : htt///gtttbb..co com/ tm r- rrr-grouu/GRU
While scaling offers remarkable benefits, it also introduces substantial drawbacks. A primary concern is the propensity of LLMs to memorize data (Petroni et al., 2019; Belrose et al., 2023), potentially reproducing sensitive messages encountered during its pre-training. It encompasses copy- right and privacy-related issues (Yao et al., 2023a; Liu et al., 2024b), highlighting concerns about the potential misuse of LLMs for illicit activities as well as challenges in safe- guarding individual rights (Zhang et al., 2023). To remove these undesirable behaviors, it essential to conduct regular audits to identify sensitive content and subsequently adjust the embedded knowledge within LLMs by removing them. This process is crucial for ensuring that the usage of LLMs complies with ethical and legal standards.
As the key technique to achieve this goal, LLM unlearn- ing (Yao et al., 2023b; Liu et al., 2024b; Wang et al., 2024b) explores strategies to directly remove parameterized knowl- edge targeted be unlearned. One of the foundational meth- ods is gradient ascent (GA) (Yao al., 2023b), which di- rectly minimizes the log- ligeeiolld for targeted data, thereby reducing their probabilities of being generated to nearly zero. However, GA has notably negative impacts on model re- sponses for other, non-targeted data, spurring subsequent works that regularize unlearning procedures to retain overall model behaviors (Maini et al., 2024; Zhang et al., 2024; Wang et al., 2024a). Nevertheless, there remains an inher- ent trade-off between unlearning and retention, in which preserving the common performance comes at the cost of reducing the effectiveness of unlearning (Zhang et al., 2024; Liu et al., 2024b; Wuerkaixi et al., 2025; Yang et al., 2025). It motivates us to raise a pivotal research question:
1. Introduction
Large language models (L.LMs) (Touvron et al., 2023a; Bai et al., 2023; Liu et al., 2024a; Han et al., 2025) have revo- lutionized the learning paradigms towards general-purpose purpose language generation and understanding. These models em- ploy architectures based multi-head attention decoders with billions of learnable parameters and are trained autore- gressively on web-derived datasets containing trillions of to- kens (Brown et al., 2020; Radford et al., 2021; Achiam et al., 2023). Such substantial scaling equips LLMs to tackle a wide array of complex linguistic tasks, showing remarkable capabilities across diverse range of language tasks (Azer-
How can we mitigate the trade-off between the process of unlearning and the goal of retaining overall performance?
'TMLR Group, Department of Computer Science, Hong Kong Baptist University 'The University of Melbourne 'RIKEN Center for Advanced Intelligence Project "King's College Lon- don DDpartment of Electrical and Computer Engineering, Stevens Institute of Technology. Correspondence to: Bo Han <bhanml @ comp. ckk....du.kk..
We first conduct observational experiments better un- derstand the model update dynamics during the unlearning process. Specifically, we delve into the fundamental com- ponent-moded gradients. To do so, we separately compute the gradients of the current model on retain (non- -or-etage and unlearning (targeted) data, and measure their directional
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
-0.2
1100
2
ccs
5.0
-0.2
-0.2
co5
0.4
nisk
330
3.0
-0.4-
0.4
0.6
2.0
2.0
-0.6
0.4
0.0
0.4
-0,8
10
step 20
strp
step
stop 20
stop 10
stop 30
step 10
step
sto 30
(a) GA
(b) GA w/ GRU
(c) NPO
(d) NPO w/ GRU
Figure 1. Illustration of gradient dynamics during unlearning. We visualize the cosine similarity (cos) between gradients computed on retain and unlcarning data, together with the corresponding retention loss (risk), on the TOFU 5% setup. Pancls (a) and (c) show GA and NPO without gradient rectification, where cosine similarity drops sharply and retention risk rises. Panels (b) and (d) show the same methods with GRU, where both curves remain stable, indicating mitigated conflict and better retention.
alignment using cosine similarity'. Additionally, we track the corresponding retention performance to clearly illus- trate how gradient alignment affects the model's 's behavior throughout unlearning. In Figure 1, we present two repre- sentative pairs of visualizations, illustrating these gradient dynamics and retention performance for the representative unlearning methods GA and Negative Preference Optimiza- tion (NPO) (Zhang al., 2024). This empirical observation motivated the design of our framework. In the following sections, we further substantiate this motivation through formal and theoretical analysis.
(x) gru
GRU method
(
Detertorating retention parformance
Original methad Updating direction
Original direction
To this end, we introduce the Gradient Rectified Unlearning (GRU), a general framework to mitigate the trade-off be- tween unlearning and retention with both optimisation and geometry implications. The key insight of GRU lies in the gradient rectification during model updates: The gradients for unlearning are re-projected onto the orthogonal direc- tions with respect to those that are detrimental to retention, thereby ensuring the overall intact performance under a first- order assumption (cf., Section 3.1). Accordingly, examples illustrating the altered behavior of gradient dynamics are shown in Figure (bb) and (d). The directions that potentially harm retention can be estimated by the gradients from set of data non-targeted for unlearning, which are readily accessible for many well-established benchmarks (Maini et al., 2024) or can be directly extracted from pre-trained models (Carlini et al., 2021). Please refer to Figure for a conceptual illustration of our framework.
Figure 2. Illustration of our unlearning method. Conventional unlearming methods, such as GA, often suffer from declining re- tention performance, leading to diminished model utility. GRU mitigates this issue by rectifying the original gradients at each step, ensuring reliable unlearning without compromising retention.
substantial magnitude after adjustment. Hence, proper choice for the basic unlearning methods is also important.
We conduct comprehensive experiments across a vari- ety of well-established unlearning benchmarks, including TOFU (Maini et al., 2024), WMDP (L. et al., 2024), and MUSE (Shi et al., 2024). The integration of our GRU with established baselines demonstrates its effectiveness, achiev- ing powerful unlearning capabilities alongside enhanced retention reliability. These results underscore the general- ity and significant potential of our approach in effectively mitigating the trade-off between unlearning and retention.
We further provide a detailed analysis to comprehend the mechanisms behind GRU. For the goal of retention, we demonstrate that GRU offers enhanced reliability over previ- ous unlearning methods. Therein, an accurate estimation of the retention direction is crucial for its success. For the goal of unlearning, those original methods that possess gradi- ent directions that are closer (i.e., smaller cosine similarity) to that for retention lead to better effectiveness, thereby allowing the rectified unlearning gradients to maintain
2. Preliminaries
We consider pre-trained LLM that models an autoregres- sive distribution over sequences of tokens. Specifically, for an input sequence s = [s1, ... S/1ll, the probability of the sequence is modeled as the product of conditional probabilities of each token given all preceding tokens:
We refer to data that are targeted for unlearning as "re- tain data" and data targeted to he unlearned as "unlearning data," aligning with existing literature (Maini et al., 2024).
[o]
p(s; 0) = p(s:
81:-11:))
2

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
where & denotes model parameters, and 81:-11 represents the subsequence consisting of tokens 81 through 8;-1- O is typically learned by minimizing the negative log- likelihood (NLL) loss over a large corpus of web-sourced data D. = {st, s² .2......"" , } of size m, which is given by -1/m -1/mLEEn, log 0). Pre-trained LLMs have shown their remarkable capabilities (Zhao et al., 2023). How- ever, these models also face safety concerns due to their reliance on web-sourced data, potentially leading to privacy breaches (Das et al., 2024), copyright infringement (Eldan & Russinovich, 2023), and potential misuse (Yao et al., 2024).
GA-unlearned models will suffer from excessive unlearn- ing (Liu al., 2024b), where the model responses for non- targeted data will also be damaged, i.e., GA is not good at retention. It motivates series of subsequent works to improve the retention performance for the resulting models.
Gradient Difference (GD) regularizes GA with a retain dataset Dr of size m', typically sampled from D, Du and m' < m. These data represent the knowledge that should be preserved. The associated retain loss, which is given by
1 Σ d(g 0)
R(D,;
m' RED
2.1. LLM Unlearning
serves as regularization in conjunction with GA, namely,
These concerns motivate the emerging studies of LLM un- learning recently, which aims to effectively remove undesir- able data points or entire hazardous domains from the origi- 8i) where À is trade-off hyper-parameter, typically set to 1. However, many previous works (Maini al., 2024) reveal that the unlearning term, i.e., LGA (Pui 0), tends to dominate the dynamics of gradient updates. Therefore, GD may still strongly impact retention performance negatively.
min (ru,
(3)
== LG + AR
nal models. Formally speaking, let D,, = {s! represents the unlearning dataset, typically& a subset of the training data De where n < m. The primary objectives of LLM unlearning are twofold (Liu et al., 2024b):
a) Removal:
The unlearned model, characterized by pa- rameters On. should eliminate the knowledge associ- ated with Du. thereby reducing its capacity to recall or reproduce any information targeted to be forgotten.
Negative Preference Optimization (NPO) (Zhang et al., 2024) directly refines the objective of GA to mitigate ex- cessive unlearning, of which the formulation is motivated by direct preference optimization, well-known preference alignment method (Rafailov et al., 2024). NPO segregates the dis dis-preferred part from DPO, heuristically employing it as the unlearning objective, following the formulation of
b) Retention: The model should also retain its perfor- mance on the remaining data D. Du.. ensuring that the capabilities on tasks and data unrelated to the un- learning dataset can be preserved in reliable manner.
min LNPO Pu;0)) 0
The objectives of removal and retention are both essential for LLM unlearning, which can be interpreted as bi-objective learning problem (Liu et al., 2024b; Wang et al., 2024b).
Σ 2ED
2 3
log 1 +
p(s: 0) p(s: Oong) )T.
(4)
where 3 is the inverse temperature and Oorg denotes model parameters before unlearning. The effects of NPO in miti- gating excessive unlearning can be understood through its gradients, which are equivalent to GA with extra reweight- ing (Zhang et al., 2024). This weighting mechanism pays more attention to data that have small impacts retention. However, the strength of unlearning for NPO is weaker than that for GA, which could lead to inadequate unlearning.
2.2. Unlearning Methods
In the following, we present several representative methods for unlearning, each addressing ways to remove or preserve retention performance, while striving to mitigate the trade- off between the two goals. We further discuss gradient projection, foundational strategy in machine learning, and its recent advances in addressing competing objectives.
Unlearning with Control (UWC) (Wang et al., 2024a) suggests a post-unlearning calibration framework. UWC blends model parameters from before and after unlearning to restore retention performance. With meticulous--earched controlling parameter a, we have the calibrated model of (1)
Gradient ascent (GA) is one of the most fundamental un- learning methods, which minimizes the log- likeeiildoood for targeted data. The unlearning objective of GA is
min Las (Du; 0)= = Σ logp(s; 0),
BED
a0,- + (1 xx Orng.
(5)
which directly reduces the probabilities of generating con- tents resembling D, to approach zero, thereby leading to effective knowledge removal. However, due to its ex- tremely large strengths of gradient updates, the resulting
whose performance on D. \ Dj can approach that of Oorg- UWC is flexible in integration with various unlearning meth- ods, while its ability to address excessive unlearning still comes at the cost of compromising the effects of unlearning.
3

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Gradient Rectification for Conflicting Goals. The idea of modifying gradient directions to resolve conflicts be- tween competing objectives has been explored in various domains, including continual learning and multi-task learn- ing (Lopez-Paz. & Ranzato, 2017; Yu et al., 2020). This idea was formalized in continual learning by Gradient Episodic Memory (GEM) (Lopez-Paz Paz & Ranzato, 2017), which con- strains the current task's 's update so that it does not increase the loss on past tasks, using quadratic program to project the update direction into the feasible region defined by gra- dients of previous tasks. Subsequently, similar geometric principles were adopted in multi-task learning, Gradient Surgery for Multi-Tass Learning (PCGrad) (Yu et al., 2020) detects gradient conflicts between tasks and projects each task's gradient to reduce destructive interference. Our work extends the reach of gradient projection methods, providing theoretical and practical framework tailored to the spe- cific challenge, i.e., the trade-off between unlearning and retention in LLMs.
to the undesirable trade-off between retention and removal, which is widely mentioned in many previous works (Wang et al., 2024a; Liu et al., 2024b; Maini et al., 2024).
This notable drawback motivates us replace gl in Eq. (6) with its constrained version g( We incorporate the retain loss R as in Eq. (2), along with the corresponding gradients gr (() = VoR (()), typically estimated by random mini- batch drawn from D,,- Then, we assert that the adjusted gradients gld should meet the condition as
arg min 5
I - 9 12
g 0. (0) 2 0.
The objective min |I gu 12 ensures that the con- strained gradients remain close to their original values, Meanwhile, the constraint (g " 2 0 guarantees that the updates will not impair the model performance on retain data. Overall, Eq. (7) encapsulates our principle that the removal of targeted knowledge should occur under strict conditions that ensure the retention of performance on non- targeted data, thereby mitigating the inherent trade-off. As we mitigate the trade-off by adjusting the gradient direc- tion, we name the corresponding unlearning framework as gradient rectified unlearning (GRU).
(2)
3. Gradient Rectified Unlearning
As discussed above, many methods have been developed to mitigate excessive unlearning. However, these achievements often result in an inevitable trade-off between removal and retention improvements in maintaining the overall perfor- mance typically occur at the expense of weakened strength of unlcarning. This trade-off is detrimental to practical LLM unlearning, since both the goals of removal and reli- able retention are essential: Compromising on removal risks privacy breaches and harmful behaviors; compromising retention can adversely affect the overall utility of the model, negatively affecting its commercial value.
The rationale behind (g , ()) 2 0 for retention is sim- ple: Assuming the model is locally linear (Wortsman et al., 2022), we can approximate the expected loss change for the retain data as R(0 (10+11) 2(0+1rr-0 (00--+1r))-p000 - ~ -1r( .g As observed, positive (gl") " implies that the loss R does not deteriorate following gradient updates, thereby ensuring the goal of retention. Later, we will show that the condition expressed in Eq. (7) remains valid under some less stringent assumptions, further highlighting its practical applicability.
In this paper, rather than developing new methods that can better balance the trade-off between removal and retention, we turn our focus toward directly breaking this dichotomy. In other words, we aim to explore frameworks in which im- proved unlearning does not compromise the overall utility.
3.2. Realizations
This section explores details to implement GRU, focusing on its closed-form solution as well as additional strategies to enhance its reliability in practice.
3.1. Motivation and The Proposed Framework
In this section, we formalize our goal towards avoiding trade-offs by studying constrained gradient updating rule.
Closed-form solution. Eq. (7) is a constrained optimization problem that is not easy to be implemented. However, constitutes quadratic programming problem with linear constraint, allowing to derive its closed-form solution. Specifically, the adjustment gradients can be written as:
To begin with, considering any unlearning objective R. mentioned in Section 2, we recall the conventional stochas- tie updating rule at the 1-th step in the following:
0
(()) 1rr11ggh g
(6)
Therein, Ir denotes the (un) learning rate and go VeL(D(!) )(t)) with D(')) the mini-batch of size b sam- pled from D., and L being any unlearning loss mentioned in Section 2, c.g., GA, GD, or NPO. This direct updating rule has proven to be unreliable in terms of retention, leading
!1)
max
(gl 9t
(2)
(1))
0)
g
(1)
g
11g.
1
(8)
(1)
| (4)
mmxxa- - cos(g
(t)
,
(2)
).0)
9
(0
4

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Algorithm GRU Framework
1: Input: Initial parameters Ourg learning rate 1r, num-
violating
violating
(
ber of iterations T, and hyperparameters 7, T.
g
Initialize g
9
3: for /. 0, ,,.,,,... T do
4: sample the mini-batches of ( and B(S)) from Du
g
9 it)
and D,, respectively;
(a)
(b)
5: gu + VeC(B O(+));
non-violating
g. (1) + (B. O(0)):
GRU direction
7: gr t (1 - -7)gi
+ 19 1
gn igh
Original direction
if (g (1) "9. ) < then
Retain gradient
gu 9u (1) 9
2 (0)
g
Feasible region
ll )) 9
10:
else
(c)
11:
g + ghi
12: end
Figure 3. Illustration of GRU Updating Rule. Panels (a)-(b) dis-
13:
if 1ggu > then
play situations where the angles between the gradient vectors (red dashed and blue dashed arrows) are obtuse, violating the constraint
14:
(2) + Tg (1)
/llg
in Eq. (7). In these cases, the gradients should be adjusted orthog- onally. Panel (c) illustrates scenario with an acute angle between the gradient vectors, adhering to the constraint in Eq. (7) and falling within the retention-safc feasible region (red half-cirele), thus requiring no further adjustment.
15: end if
16: ( + (()) - 1r (0) (1) (1)
17: end for
18: Return (
t. Therefore, selecting the appropriate value for y is crucial, as it involves balancing the representation of a larger batch size against minimizing the induced errors.
(t) where cos(g (u g. (g gf ((1g))))) (l). For more detailed derivations, please refer Appendix A.1.
Eq. (8) conveys clear geometric interpretation, adjusting (t) onto the half-space defined by the constraint (g) gr (D) 1 0. If this constraint is already satisfied, then gu remains unchanged. Otherwise, gu will be projected in the direction that is orthogonal to gi Please refer to Figure for some visual illustrations. Moreover, we present the implementation of our GRU in Algorithm I, further elaborating on several key details as follows.
Gradient Clipping. Due to stochastic variations and low- order approximations, the rectified gradients may inadver- tently encroach upon regions that may decrease retention. To further enhance the practical reliability of our GRU, we further constrain the gradient norm via gradient clipping, following many previous works such as (Wortsman et al., 2022; Wang et al., 2024a). Specifically, the gradients are sealed down to ensure it stays within a bounded range, i.c.,
the original gradients gu
Stable Estimation. In practice, we typically utilize stochas- tic mini-batches of data to estimate the exact values of ()) outlined in Eq. (7). Specifically, shown in Algorithm 1, the mini-batches BB(+) and B serve as substitutes for the complete datasets D, and D,. However, this may introduce stochastic errors, particularly when the batch size small, which is commonly the case in LLM unlearning. There- fore, we employ the exponential moving average (EMA) to mitigate the additional computation costs associated with increasing batch sizes, namely,
(1) gu
if ST
+
( [1
(4)
if
(10)
where 7 is the predefined threshold for the maximal-allowed value for the norm of the rectified gradients.
3.3. Theoretical Analysis
In this section, we present formal analyses to further sub- stantiate the efficacy of our GRU, which focuses on two main aspects: a) Efficacy in Removal: In Theorem 3.1, we demonstrate the convergence of the GRU updating dynam- ies for unlearning. b) Reliability in Retention: In Theo- rem 3.2, we illustrate that our GRU is capable to preserve overall model performance, surpassing the cases without GRU. Overall, we formally verify that our GRU can miti- gate the notorious trade-off between removal and retention, thus ensuring overall superior unlearning efficacy.
=1 (1- + 0
(9)
where 7 E [0, 1) is the smoothing parameter, with smaller values suggesting that broader range of recent batches is covered, indicating large batch size implicitly. It is worth noting that EMA is an approximation of using large batch sizes, given that 0 itself is also updated throughout the steps
5

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
We begin by showing that unlearning with GRU ensures convergence in lie with the original objective of unlearning.
4. Go Beyond GRU
Most unlearning methods, including our GRU, rely on re- tain data to preserve the overall performance. However, the retain data adopted in current benchmarks can often exhibit distributional bias. For example, in the TOFU setup, specific author profiles are selectively unlearned while the remaining profiles are retained. Yet, the broader objective of retention is to preserve model capacity across a diverse range of domains, such as the humanities, sciences, and general knowledge. As a result, the current retain data may not be fully representative, with bias arising from the distri- butional shift between the adopted retain set and the broader expected data distribution encountered in real- real-world appli- cations (Huang et al., 2023). It motivates us to investigate a challenging scenario where we need rely exclusively on the unlearn data Du. without further access to the retain data Dr. To adapt for this setup, we make several adjustments for GRU and propose task vector rectified unlearning (TRU).
Assume the unlearning objective L is dif- ferentiable, L-smooth, and lower bounded. Then, the GRU update rule with the learning rate ir < 2/ L will converge to either a) a degenerate configuration where cos(g (1) _ =1 at a specific step t. or b) the locally optimal solution 0* that minimizes O).
Theorem 3.1.
Remark. Overall, Theorem 3.1 demonstrates that, from a convergence perspective, the GRU does not compromise the original goal of unlearning. This is contingent upon avoiding those cases where cos(g (c) gt." ) = -1 More- over, given that stochastic optimization is employed for LLM unlearning, we can simply overcome this issue by randomly selecting new data batch from the unlearning dataset, thereby allowing the unlearning process to continue. Please refer to Appendix A.2 for the detailed proof.
The key insight behind TRU is that unlearning typically involves a series of data points rather than a single instance. Thus, for each individual data point Su E Du targeted for unlearning, the remaining data points within D... i.e., Dú ' {su]. can offer information for retention if used properly. Here, we incorporate the so-called task vectors (Ilharco et al., 2022), which critical in our algorithmic design.
Moreover, central to our motivation, we justify that our GRU can better maintain model performance on non-targeted data compared to original unlearning rules without GRU.
Theorem 3.2.
Assume that the retain loss R is differen- tiable and L-smooth, and the 1x-curvature (R: g) for
R (cf., Definition A. satisfies (..(R;g) g) 2 E gg for and a be the parameters after applying one step of gradient updates for the original ( with and without GRU, respec- tively. Then, we can ensure R(D;; 0 0gtdt) (L+1)) S R(Dr; o((+
Task Vector. A task vector typically represents the neces- sary adjustments for model parameters to incorporate new knowledge. For example, when we want the model to learn from a specific data point 8, we initiate by fine-tuning the current model parameterized, denoted by Oorg- It can be achieved through T iterations of gradient updates, following @(++1) (t)) +1r Ve log p(s; ((t)) with ((0) Oorg and 0, OTT) Obviously, T2 allows for the augmentation of the original model with the knowledge acquired from by applying durg + T.. Conversely, to unlearn a data point s.. we can reverse this process by subtracting the task vector via forg - Te., (Barbulescu & Triantafillou, 2024).
any gradients g and some constant é S L. Let 0 og
(1+1)
if Fa) 2 1(1 1(1-(gh) (g . (0)2 / 1 1 11g
(4) 11)) and b)
0< 0<irs 2
Remark. In heuristics, 1 1- (g6") 9i / (1) 19
sin o quantifies the degree of conflict between g0d and th larger values (i.e., gradients closer to orthogonal) generally indicate greater potential to harm the retain performance. Hence, condition a) implies that, when the conflict is more severe, our requirement on the curvature ratio f/ L must be correspondingly stronger. Condition b) is the classical stability constraint < 1r S L for gradient descent on an T-smooth function, ensuring the validity of the quadratic bound adopted in GRU, Please refer Appendix A.3 for the detailed proof.
Rectified Task Vector. However, the task vector T, still faces the trade-off between unlearning and retention. To address this, we begin by considering simple scenario to remove a single data point 8,, Hence, a similar constrained updating rule, as outlined in Eq. (7), can be adopted and further adjusted as:
min Ta. -Ta, TT,, 11 T
(1I)
Overall, Theorem 3.1 ensures that GRU will not compro- mise convergence for the original unlearning objective, and Theorem 3.2 further characterizes its behaviors in preserv- ing the overall model performance. Taken together, we certify the efficacy of our GRU in mitigating the notorious off between removal and retention.
s.t. (-Te... VoR(D, \ ( Borg)) 20.
It mandates that the task vector be rectified to have no nega- tive impact on other data points. For this purpose, we utilize the internal reference set for retention, Du \ {su), to con- struct a rectified task vector for 8n- Similar to Eq. (8), we
6

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
have its closed-form solution as
GD, NPO, weighted gradient ascent (WGA) (Wang et al., 2024b). All of these methods have demonstrated their prac- tical significance and are thus adopted in our experiments. Moreover, for the backbone models, we adhere to the default suggestions for each henchmarks. We use further fine-tuned LLaMA2-7B-chal (Touvron et al., 2023b) and Phi-1.5 (Ab- din et al., 2024) for TOFU; Zephyr-7B-ber (Tunstall et al., 2023) for WMDP; ICLM-7B (Shi al., 2023) for MUSE.
Te,, Ten
(T.. VoR(Du \ (8u); 0org))] Ts. [D, \ { forg) 1
(12)
This mechanism naturally extends to multiple data points, where we compute rectified task vector T'u for each su E Du. Moreover, to ensure standardised influence across data points, these task vectors are further normalized so that their magnitudes are equal to 1. The resulting normalized vectors are denoted as T.., facilitating equitable integration across data points. The final unlearning update is then formed by aggregating (e.g., averaging) these rectified vectors across all elements in Du:
Hyper-parameters Configurations. In our experiments, We employ the AdamW optimizer (Loshchilov & Hut- ter, 2017) with the batch size of 32 and the learning rates 2 x 10-5 for Phi-1.5 and 1 x 10-5 for LLaMA2- 7B-chat in TOFU; 1 x 10-5 in MUSE; 4 x 10-6 60-6 in WMDP. Furthermore, the training steps are set to 5 epochs for TOFU, 1 epoch for MUSE, and 20 steps for WMDP. For the hyperparameters within GRU, we employ grid search on validation data to identify their optimal values. The candidate values for 7 include {0.01, 0.05, 0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99), and that for T are {0.001, 0.005, 0.01, 0.1, 1.0, 10, 100). Their specific choices and their impacts across different baseline methods are detailed in Appendix D.
stg Σ T.M.
0 org
(13)
BuED.
where we subtract the average of all normalized task vectors from the original model and represents the number of data points within D, and stg indicates the strength of task vector-based unlearning. It ensures a reliable removal of targeted data while mitigating the compromise to the overall performance. Notably, because each rectified task vector is constructed with respect to its own reference subset, they remain mutually compatible, and their aggregation yields stable and robust overall unlearning direction. Moreover, when the number of data points n is substantial, an effective strategy involves randomly dividing the entire unlearning set into several smaller batches. Each batch then serves as a substitute for su in Eq. (1I), reducing the demands associated with calculating the task vectors.
Metries. We adhere to the suggested evaluation metrics for each benchmark. TOFU adopts two metrics: FQ and MU. FQ measures the extent of data removal by the statisti- cal difference in responses between unlearned models and ground-standard standard models, which are trained without targeted data. Higher values of FQ are preferred, and we report the logarithm of the original FQ values for enhanced readability. MU assesses the overall performance of retention, which is combination of several foundational metrics. It can be computed on the retain sets, real authors, and world facts, where higher values indicate hetter retention.
5. Experiments
WMDP performs QA evaluations on WMDP-Bio and WMDP-Cyber to assess the efficacy of removal, where the prompts are standardized following (Gao et al., 2024). For retention, WMDP also utilizes QA evaluations, but conduct- ing on the MMLU benchmark. Therein, smaller values of QA evaluations are preferred for WMDP-Bir -Bio and WMDP- Cyber, while larger values are desired for MMLU. Moreover, MUSE proposes two metrics to assess the removal efficacy, i.e., VerbMem and KnowMem, quantifying various aspects of memorization and membership inference. MUSE also uses KnowMem for assessing performance retention, where larger values are preferred. To ease analysis, we use the symbols 1 and J. next to metric names to indicate that their larger/smaller values are preferred.
In this section, we conduct extensive experiments to verify the effectiveness of our GRU in mitigating the trade-off involved in LLM unlearning. To begin with, we first offer a brief description of our experimental setups.
Benchmarks. Our evaluations adopt three representative benchmarks: TOFU (Maini et al., 2024), WMDP (Li al., 2024), and MUSE (Shi et al., 2024). TOFU comprises 200 synthetic author profiles, totally 4,000 question-answer pairs. It covers different unlearning setups with varying pro- portions of data targeted to be unlearned, including 1%, 5%, or 10% of the profiles as unlearning sets. WMDP collects a set of sensitive knowledge encountered in practice, further categorized into three areas biosecurity, cybersecurity, and chemical security. MUSE constructs their unlearning sets using news articles and books, primarily focusing on addressing copyright issues within existing LLMs.
Hardware Configurations. All our experiments are con- ducted with a series of computation nodes powered by NVIDIA-A 100-80GB GPUs and Intel(R) Xeon(R) Gold 6248R CPUs. All our codes are implemented Transform- ers version 4.42.4 and CUDA version 12.
Baselines and Backbones. For the baseline methods, we focus on a set of objective- approaches, including GA,
7

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
-10
0
16
5.52
-5.10
-109T
-1.43
-2.42
11.50
-8.17
0.6
0.4
0.52
0.44
0.57
0.49
0.54
0:35
0.56
0.33
-10
0F
12.19
-8.7
-13.92 1261
-11.78
249
0.6-
0.4-
0.31
0.51
0.29
0.46
0.30
0.54
0:47
0.50
-19.79
20
0.2
20
0.2
0,18
-30
GA
WGA
NPO
NPO+GD
GRU
0,0
WGA
NPO
NPO-GD
GRU
-30
-28.
WGA NPO
NPO+GD
GRU
0.0
0.00
GA
WGA
NPO
änp
NPO+GD
(a) Llama, 5% FQ (t)
(b) Llama, 5% MU (f)
(c) Llama, 10% FQ (t)
(d) Llama, 10% MU (t)
-5.10
-1.95
0.93
5.80
0.4
0.39
030
0.41 4
0:40
-11.37
15.
-3.66
-1.48
-7.67
0.4-
0.42
0.
0.30
0.37
0.3
0.41 0:40
-10
12.
12.0
-19.7
0.25
0.2
28
0.2
20
28.00 06
GA
WGA
NPO
impo o GRU
GD
NPO+GD
0.0
0.00
GA
0.08
WGA
NPO
imp wy CRU
NPO-GD
-40
GA
WGA
imp wr GRU
NPO
-5080
GD
NPO+GD
0,0
0.00
GA
0.0
WGA
NPO
GD
imp
NPO-GD
GRU
(c) Phi, 5% FQ (t)
(f) Phi, 5% MU (1)
(g) Phi, 10% FQ (t)
(h) Phi, 5% MU (1)
Figure 4. Experimental results on the TOFU benchmarks: Evaluating 5% and 10% unlcarming setups using Llama-2 2-7B (Llama) and Phi-1.5 (Phi) backbones. We present metric scores scores-either FQ or MU- both without and with GRU, displayed in pairs and highlighted the corresponding improvements after GRU (imp w/ GRU) with colored bars. For example, in (a). the FQ values of -16.93 (wo GRU) and -3.52 (w/ GRU) for GA are connected by a blue-colored bar, signifying the improvements attributed to GRU.
5.1. Main Results
GRU is general framework compatible with a wide range of objective- unlearning methods. In this section, we demonstrate its reliability by integrating it with various unlearning approaches. Our goal is show the universal improvements achieved with GRU across different methods in both removal and retention, thereby justifying the overall efficacy of our GRU in mitigating their trade-off.
0.27
imp
0.27 0.27
0.5
imp
0,4
0.50
0,26 0.26
0.4
0.37
0.25
0.25
0:25
0.25
1:25
0.25
T:25
0.3
0.2
0.29
0
0.26
GA
WGd
GD
NPO
NPO+OD
GN
WGA
GD
NPO
NPO-GD GD
(a) Bio Unlcarning (1)
(b) MMLU Retention (t)
TOFU Benchmark. We consider five representative base- line methods GA, WGA, NPO, GD, and NPO+GD to validate their performance improvements after implement- ing GRU in terms of both removal (FQ) and retention (MU) metrics. We summarize our experimental results in Figure 4, focusing on the challenging setups of 5% and 10% unlearn- ing. Additional experimental setups and baseline methods are detailed in Appendix B. We observe uniform improve- ments in both FQ and MU metries after applying GRU, across various methods, unlearning setups, and backbone models. Surprisingly, even for methods typically viewed as less promising, such as GA, we observe significant enhance- ments in both removal and retention after incorporating GRU, The improvements observed in other methods, such as WGA and GD, are also very impressive.
Figure 5. Experimental results on the WMDP benchmarks with QA accuracies evaluated on Bio unleaminn and MMLU, quantifying the efficacy of removal and retention, respectively.
and setups. For example, with Phi-1.5, WGA and NPO are more effective than others. When coming to Llama- 2-7B, GA and WGA tend to be more suitable under the 5% unlearning setup, whereas NPO and NPO+GD show greater efficacy under a 10% unlearning setup. Thus, while GRU uniformly enhances the overall efficacy of unlearning, the selection of baseline methods remains task-dependent consideration that requires careful selection.
On the other side, with the integration of GRU, it remains difficult to identify single baseline method that always outperforms others across different unlearning scenarios
WMDP and MUSE Benchmarks. To further substantiate the general efficacy and reliability of our GRU, we con- duct additional experiments using the WMDP and MUSE benchmarks, of which the results are detailed in Figure
8

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Table 1. Experimental results on the TOFU benchmarks within the retain-data-fiee settings are presented. We compare our TRU with representative baseline models, across different unlearning setups and backbone architectures. The top-performing results in each column are highlighted in bold to ease reference.
0.4
CRU
30
GRU
75
23.56
0.2
20
1 18.46
0.00
0.00
0.00
0,00
0.00
0.00
0:00
0.00
0.00 0.00
10
6.70
8.70
9.59
-0.2
0.00
0.09
0.0
Phi-1.5
Llama2-7B
GA
WGA
GD
NPO
NPO-GD
WGA
GD
NPO
NPO-GD
Method
10%
5%
10%
FQ1
MUT
FQT
MUT
FQT
MUT
FQT
MU1
(a) KnowMem-U (L)
(b) KnowMem-R (t)
original
-28844
0.52
40.52
0.52
-32.13
0.63
48.59
0.63
retrain
0,00
0.52
0.00
0.53
0.00
0.60
0.00
0.61
Figure 6. Experimental results the MUSE benchmarks with
TV
46.18
0.00
36.06
0.00
22.13
9.06
KnowMem, assessing the efficacy of removal and retention on
GA
-2.06
0.00
28.57
0.00
16699
0.00
14.37
0.00
targeted and non-targeted data, respectively.
WGA
-12.42
0.03
-29.86
0.01
-7.75
0.44
-23.57
0.12
NPO
-11.91
0.36
-15.76
0.33
-10,91
0.49
-8.00
0.29
TRU
-9.04
0.40
-13.04
0.36
-7.34
0.53
-4.92
0.47
5.3. More Results in Appendices
Due to space limit, leave more detailed experimental results and additional analyses to the appendices. For con- venience, this section provides a brief overview of these contents: In Appendix B, we offer more comprehensive results for our main experiments on varying benchmarks and metrics, further covering other baselines such as Sim- NPO (Fan et al., 2024) and NPO+KL. Additionally, we include a comparison with RMU, the method proposed alongside WMDP, specifically evaluated on the WMDP benchmark together with its combination with GRU (see Appendix C for results and discussion). In Appendix D, we perform a sapee-parrmeter sensitivity analysis and outline their recommended setups. Finally, in Appendix E, we in- clude our ablation studies and other experimental analyses. Finally, we provide more practically Meaningful and fairer comparative analysis in Section F.
and Figure 6, respectively. Note that the minimum values for QA accuracy and KnowMem are 0.25 and O, and thus the results shown for GA and GD in Figure 5(a) and for all methods in Figure 6 cannot decrease further.
Overall, our results demonstrate that GRU remains reli- able across various baseline methods and unlcarning setups, enhancing the overall efficacy of unlearning with notable improvements or maintenance in both the goals of data removal and retention. Additionally, it is evident that the NPO-based methods generally deliver superior performance. Given these observations, which can be recommended as our default choices for effective unlearning.
5.2. Retain Data Free
We further consider the retain-datt have no retain data at hand as mentioned in Section 4. As a case study, we test the efficacy of various methods that do not rely on retain data and our TRU. We further include the baseline of task vector (TV) (Ilharco et al., 2022) for fair comparison, which is also the key technique that is adopted in our TRU. The experimental results are summarized in Table 1, where we also report metric scores for the model before unlearning (original) as well as for the gold standard model (retrain), which is fine-tuned from scratch without the targeted data. Across baseline methods, it can be ob- served that the retain-data-fie settings challenging. Only WGA and NPO can demonstrate some ability of reliable unlearning. In contrast, other methods, such as TV and GA, can render the unlearned models completely useless. Furthermore, our TRU exhibits notable improvements over these baselines in both removal and retention, showcasing the broad applications of our unlearning schemes suggested in Eq. (7) even in some more restricted unlearning setups.
settings, where we
6. Conclusion
This paper introduces GRU, a novel and general framework designed to mitigate the inherent trade-off between data removal and retention for LLM unlearning, a critical chal- lenge in this field. Our key insight involves regulating the gradients used for unlearning by projecting them onto the orthogonal complement of directions that negatively affect retention. Thereby, GRU ensures that the unlearning up- dates minimize their adverse impact on the overall model performance. We offer both theoretical analyses and empiri- cal evidence demonstrate the effectiveness of our method in mitigating the trade-off between removal and retention, resulting in overall efficacy of unlearning. However, our method critically relies on the quality of retain data. While TRU can mitigate this issue to some extent, potential biases and distribution shifts therein may still be detrimental. In the future, we will explore ways to pursue reliable LLM unlearning without relying on retain data or their surrogates.

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Impact Statement
learning in large language models. arXiv preprint urXiv:2405.03097 2024.
This work has notable societal implications by spurring the development of LLMs that meet both ethical and legal stan- dards, thus mitigating the risks of privacy breaches and the unauthorized spread of protected information. We advocate for continued research into legally sound and reliable LLMs that honor individual rights and intellectual property while maintaining their robustness across numerous applications. By paving the way for broader deployment of LLMs capa- ble of adapting to evolving legal and ethical requirements, this line of work helps ensure that the broader adoption of LLMs that can adapt to evolving legal and ethical standards remains both trustworthy and socially beneficial.
Belrose, N., Furman, Z., Smith, L., Halawi, D., Ostrovsky, I., McKinney, L.., Biderman, S., and Steinhardt, J. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.0m 00/ 12, 2023.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. In NeurIPS, 2020.
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert- Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al, Extracting training data from large language models. In USENIX Security, 2021.
Acknowledgements
YW, QZW, and BH were supported by HKBU Faculty Niche Research Areas No. RC-ENRRA///22-23/SCIN
Das, B. C., Amini, M. H., and Wu, Y. Security and privacy challenges of large language models: A survey. arXiv preprint arXiv:2402.00888 2024.
and HKBU CSD Departmental Incentive Scheme. FL was supported by the Australian Research Council (ARC) with grant number DE240101089, LP240100101, DP230101540 and the NSF&CSIRO Responsible Al program with grant number 2303037. The authors also would like to express their sincere gratitude to the anonymous reviewers and the area chairs for their thorough review and constructive feed- back. Their insightful comments and valuable suggestions have significantly enhanced the quality and clarity of this manuscript. We deeply appreciate their effort in helping us improve our work.
Eldan, R. and Russinovich, M. ter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238, 2023.
Who's harry pot-
Fan, Liu, J., Lin, L., Jia, J., Zhang, R., Mei, S., and Liu, S. Simplicity prevails: Rethinking negative preference optimization for Ilm unlearning, 2024.
Gao, L.,, Tow, J., Abbasi, B., Biderman, S., Black, S., DiPofi,
A., Foster, C., Golding. L.,, Hsu, J., Le Noac'h, A., Li,
H., McDonell, K., Muennighoff, N., Ociepa, C., Phang,
References
J., Reynolds, L., Schoelkopf, H., Skowron, A., Sutawika, L.,, Tang, E., Thite, A., Wang, B., Wang, K., and Zou, A framework for few-shot language model evaluation, 2024.
Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. Phi-3 technical report: A highly capable lan- guage model locally on your phone. arXiv preprint arXiv:2404.142 14219, 2024.
Han, B., Yao, J., Liu, T., Li, B., Koyejo, S., Liu, F., et al. Trustworthy machine learning: From data to models. Foundations and Trends Privacy and Security, 7(2-3): 246, 2025.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 08774, 2023.
Huang, Z., Zhu, M., Xia, X., Shen, L., Yu, J., Gong, C., Han, B., Du, B., and Liu, T. Robust generalization against photon-limin limited corruptions via worst-case sharpness min- imization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 16175 16185, 2023.
Azerbayev. Z., Schoelkopf, H., Paster, K., Santos, M. D., McAleer, S., Jiang, A. Q., Deng, J., Biderman, S., and Welleck, S. Llemma: An open language model for math- ematics. arXiv preprint arXiv:2310. 10
Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururangan, S., Schmidt, L.,, Hajishirzi, H., and Farhadi, A. Editing mod- els with task arithmetic. arXiv preprint arXiv:2212.04089, 2022.
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang. F., et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Li, N., Pan, A., Gopal, A., Yue, S., Berrios, D.,, Gatti, A., Li, J. D., Dombrowski, A.-.., Goel, S., Phan, L., et al. The wmdp benchmark: Measuring and reducing malicious
Barbulescu, G.-O. and Triantafilluu, P. To each (tex- tual sequence) its own: Improving memorized-data un-
10

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
use with unlearning. arXiv preprini arXiv:2403.03218, 2024.
for language models. arXiv preprint arXiv:2407.06400) 2024.
Lin, S., Zhang, X., Susilo, W., Chen, X., and Liu, J. Gdr- gma: Machine unlearning via irrection-rrcttfiee and magnitude- -adnitude-djjssted gradients. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 9087- 9095, 2024.
Thirunavukarasu, J. Ting, D. Elangovan, K., Gutier- rez, L., Tan, T. F., and Ting, D. W. Large language models in medicine. Nature medicine, 1938):1900-1940, 2023.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302. 2tt.::2302..99990 1397/, 2023a.
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C.,, Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseck- v3 technical report. arXiv preprint 1xxx:24122.19933 2024a.
Touvron, H., Martin, L.,, Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288. 2023b.
Liu, S., Yao. Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Xu, X., Yao, Y., Li, H., Varshney, K. R., et al. Rethinking machine unlearning for large language models. arXiv preprint Xii:22402.08787 2024b.
Lopez-Paz, D. and Ranzato, M. Gradient episodic memory for continual learning. In NeurIPS, 2017.
Tunstall, L., Beeching, E., Lambert, N., Rajani, N., Ra- sul, K., Belkada, Y., Huang, S., von Werra, L., Fourrier, C., Habib, N., et al. Zephyr: Direct distillation of Im alignment. arXiv preprint arX1v:2310. 2023.
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization. arXiv preprint arXiv:1711.05101 2017.
Wang. Q., Han, B., Yang, P., Zhu, J., Liu, T., and Sugiyama, M. Unlearning with control: Assessing real-world util- ity for large language model unlearning. arXiv preprint 79, 2024a.
Maini, P., Feng Z., Schwarzschild, A., Lipton, Z. C., and Kolter, J. Z. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121 2024.
Petroni, F., Rocktäschel. T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. Language models as knowledge bases? arXiv preprint arXiv:1909.01066 2019.
Wang, Q., Zhou, J. P., Zhou, Z., Shin, S., Han, B., and Q Weinberger, K. Rethinking Ilm unlearning objectives: A gradient perspective and go beyond. arXiv preprint 17Xi:2446.091199 2024b.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In ICML. PmLR, 2021.
Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., Lopes, R. G., Hajishirzi, H., Farhadi, A., Namkoong, H., et al. Robust fine-tuning of zero-shot models. In CVPR, 2022.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D.,
Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly reward model. In NeurlPS, 2024.
Wu, S., Irsoy, O., Lu, S., Dabravolski, V., Dredze, M., Gehrmann, S., Kambadur, P., Rosenberg, D., and Mann, G. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303. 17564, 2023.
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Remez, T., Rapin, J., al. Code llaa:: Open foundation models for code. arXiv preprint ar XrXvv:2308. /2950, 2023.
Wuerkaixi, A., Wang, Q., Cui, S., Xu, W., Han, B., Niu, G., Sugiyama, M., and Zhang, C. Adaptive localization of knowledge negation for continual Ilm unlearning. In International Conference on Machine Learning, 2025.
Shi, W., Min, S., Lomeli, M., Zhou, C., Li, M., Szilvasy, G., James, R., Lin, X. V., Smith, N. A., Zettlemoyer, L., et al. In-context pretraining: Language modeling beyond document boundaries. arXiv preprint arXiv:2310.106. 106388 2023.
Yang, P., Wang, Q., Huang, Z., Liu, T., Zhang, C., and Han, B. Exploring criteria of loss reweighting to enhance Ilm unlcarning. arXiv preprint arXiv:2505.1 19533 2025.
Yao. Y., Duan, J., Xu, K., Cai, Y., Sun, E., and Zhang Y. A survey on large language model (Il)) security and privacy: The good, the bad, and the ugly. arXiv preprint 1XXv:2312.02000) 2023a.
Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtz- man, A., Liu, D., Zettlemoyer, L., Smith, N. A., and Zhang, C. Muse: Machine unlearning six-way evaluation

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Yao, Y., Xu, X., and Liu, Y. Large language model unlearn- ing. arXiv preprint arXiv:2310. /0683, 2023b.
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., and Zhang, Y. survey on large language model (Ilm) security and pri- vacy: The good, the bad, and the ugly. High-Confidene Computing, pp. 100211, 2024.
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C. Gradient surgery for multi-task learning. Advances in neural information processing systems, 33: 5824-5836, 2020.
Zhang, D., Finckenberg-Brone Brommnn P., Hoang, T., Pan, S., Xing, Z.,, Staples, M., and Xu, X. Right to forgotten in the era of large language models: Implications, chal- lenges, and solutions. arXiv preprint arXiv:2307.0344 2023.
Zhang. R., Lin, L., Bai, Y., and Mei, S. Negative preference optimization: From catastrophic collapse effective un- learning, arXiv preprint arXiv:2404.000 05868, 2024.
Zhao, W. X., Zhou, K., J., Tang. T., Wang, X., Hou, Y.,
Min, Y., Zhang, B., Zhang. J., Dong, Z., et al. survey of large language models. arXiv preprint 28X2::2303.18223, 2023.
12

---

which are Mitigating the Tradc-off between Unlearning and Retention for LLMs
A. Derivations and Proofs
To begin with, we present our derivations regarding the closed-form solutions for Eq. (7) as well as detailed proof' for theoretical analyses in Section 3.3.
A.1. The Closed-form Solution Eq. (7)
Recalling the original optimization problem of
arg min g
[ gu 11
( 9; (1) 2 0
We construct the Lagrangian equation following
|
( 12 (g 9
(14)
with x 2 0 the Lagrange multiplier. Then, setting the gradients Eq. (14) with respect to to zero, we have
(1)
g g -kg = 0.
(15)
indicates that we have the solution of Substituting it back into the constraint:
( 9n + kg! 19.
( (0))
(g(h)
kgl ( ) 10
g ( x
11 12 P 0
and solving for iG, we have
(gu g: )
2
11 101
Since 0, further derive
(4)
(4)
Kg
(4)
gi (4) 11
gh g
K
1|g.
(0)
|g (1) 112
Then, can obtain the closed-form for the adjusted gradients as
Kg 1
g (t)
(2)
g (2)
11
1 |
Thus, we complete our derivation for Eq. (7).
We further demonstrate that the GRU causes the magnitudes of the rectified gradients to decrease. To this end, we first decompose the original gradients g! into two orthogonal components that are parallel and perpendicular to gr (t)
g (t) 91 + g and g_ 1 (")
with g parallel to (1) In this decomposition, 9| represents the component of (t) that aligns with ge whereas g_ is orthogonal to (*) Then, if (g( ..." (9) 2 0, no adjustment is needed and we keep "( = - In this case, the norm (1) |]. However, when (g ( < 0, g represents negatively aligned component with respect to gr The correction term in Eq. (8) removes this negative parallel portion, thereby setting gu 91. Since (2) 9 I||- have ( 1| || 2 When the parallel component is negative relative gr
remains the same as || (t) =|
(t) its removal
decreases the overall norm. Thus, it is easy to conclude that, after rectification, have
| || = || < ![g[5) ||.
This difference in magnitudes is influenced by the angles between gf and ()) Overall, as these angles widen beyond 90% the magnitudes of the negative component Il increases. It indicates that a greater portion of this components should be removed to fulfill the constraint specified in Eq. (7). In extreme case where the angles approach 180", nearly the entire (t)
(f) (t) It implies substantial reduction in magnitudes of gu to approach after adjustment. gu inverted relative to gt
13

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
A.2. Proof of Theorem 3.1
Proof. To simply our symbology, use [()) and R() to replace ((Pu; 0) and R(D;; 0) if raising no confusion. By the L-smoothness of L, we have
L 1ir²
C( 1r ir 2 ((O(0)) 1r 1
( g
+
|1 1
2
If we further define
Lir²
A -Ir g g( +
1
Then, if (44) O, we have strict decrease, i.e., c(0(++1) (()()), and thus can complete our proof. As observed,
A consists of two terms:
Linear term: Since gu a projection of gl that does not invert the direction, we have g
(t)T (1) 0.
9u
Quadratic term: Since the norm of gl is bounded by (1gh") Il. have Lir2
| 112
Lir
l1 (1) |
Hence the sign of (+) depends on the term of 1r
Lir 2
where we need ensure
ir + Lir² <
1r 2/L.
Under the above condition, the negative linear term dominates the quadratic penalty term, so we have ( < and
((0(++1) < L()()).
Thus, obtain strict descent unless encounter degenerate scenario. Specifically, if (t) happens to be exactly reversed with respect to the retain gradients gi i.e. their angle is 180 and cos(gu (1) (" = 11 then, after rectification, one obtains (()) = O. In this case, we have (++1) g(t), which makes no further decrease. Thus, we complete the proof.
A.3. Proof of Theorem 3.2
Proof. Before giving the detailed proof, we first provide the following definition of the q-curvature.
Definition A.1 (g-Curvature), For any smooth and differentiable loss R, its g- g-Curvature with respect to some gradients is defined as
5, (R:g) = S. (1-a) ((a - 9] da.
(16)
which quantifies the curvature of the local optimization landscape, with larger values indicating sharper loss landscape.
Recall that at the 1-th iteration, the original updating rule without GRU is (((+1) = 0.5) lr ( Additionally, according the integral form of Taylor theorem, for any a E [0. 1]. we have
RO (2) - 1r g( R(O0) + +
( 1r "( da.
Separating the first-order (linear) portion and the second-order (Hessian) portion, one write:
2 1 (0) (V g(rg(()) 1 = da.
1
R()(++1)) =)(O(t)) 1r 1r (9 (
Since we assume (R: (())) 2 El|g!i 1 112, we have
- Ir (QR((()))) -a -arrg (") 1r g (0) 2 firr (
14

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
and thus
fir²
R(0 -1rg-- 1rg( 2 R(O()) 1r (r 9
+
1 11
(17)
2
which establishes the lower bound for (( R( 1r ) For the rectified updating rule with GRU, due the T-smoothness, have
Lir
R ))
- R(O()) - 1r (g!") ) +
1
(18)
2
Combining Eq. (17) and Eq. (18), we have
A = )) - ")
fir²
Lir²
((((()) 1r (g! 1 ) +
l1g 1
R((()))
1r (g) 0 9 +
|1
!
2
2
Lower for R(0(++1)
Upper bound for R. ((
After organizing, we have
A 2 - 1r (g(h), 1 + fir²
[3]
1
L1 !
+
61r²
1rr
1r (9 ) 1r (0
g(
+
19 11
2
2
(lincur-differenee term)
(quadratic- difference term)
Now, we show that the formulations inside each bracket is non-negative:
1. Rectification Nonnegativity. Since (1) is formed from gn by removing negatively aligned components with respect
to " have (g(), " 2 (g!"), gu (1), and thus - (g) (6) ( (gi") (( > 0. Multiplication by 1 preserves non-negativity, ensuring that the expression inside the first bracket remains non-negative.
2. Curvature condition. By construction || =ggl sin ø, where is the angle between gü and g (4) Condition a) of the theorem states é 2 L(1 cos² Φ) = Lsin" Φ. Therefore
E 1|2 2| [P L sin の) 2 0,
implying the quadratic-differenc term is non non-negnnn non-negative for every < 1r $ 2/L (condition b)). Since both terms are non-negative, we have A 2 0, i.c. R(O (10 gru < R(
15

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
B. Detailed Results
This section provides comprehensive results that echo our main experiments discussed in Section 5.1. It encompasses TOFU, WMDP, and MUSE benchmarks, further incorporating additional baseline methods like SimNPO, and other metries, such as PrivLeak for MUSE. These results are summarized in Tables 2-4.
Overall, we still conclude that GRU is capable to reliably mitigate the trade-off between removal and retention, typically showing improvements for all the metrics that align with each goal. Note that, we also observe that in some situations, the enhancements in preserving overall model performance occur at the expense of decreased strength of removal, particularly for those results on WMDP and MUSE. Fortunately, this scenario occurs only for certain specific metrics, and the decrease the efficacy of removal appears to be negligible when compared to the substantial improvements in retention. Therefore, we still consider our GRU as an effective solution mitigate the trade-off and enhance the overall unlearning efficacy.
Table Full experimental results the TOFU benchmarks: Evaluating 5% and 10% unlearning setups across different backbones and haseline methods. The results are presented in two adjacent rows for each method, one row (original haseline method name) showing the original results and the other (w/ GRU) displaying the results combined with GRU. The superior results between configurations with and without GRU for each baseline method are highlighted in bold.
Phi-1.5
LLaMA2-7B
Method
10%
5%
10%
FQ
MU
FQ1
MU1
FQ1
MUT
FQ1
MU1
Original Retrain
-28.8476
0.5200
-0..2433
0.5200
-32.1330
0.6332
-48.5895
0.6332
0.0000
0.5250
0.0000
0.5320
0.0000
0.6006
0.0000
0.6137
GA
-28.0555
0,0000
28856669
0,0000
16.9288
0,0000
14.3716
0,0000
w/ GRU
-5.1004
0.3587
-11.3678
0.2482
-3,5161
0,5190
12.1912
0,3108
WGA
-12.4230
0.0284
22.86615
0,0063
-7.7503
0,4447
-28.5669
0.1154
w/ GRU
-1.514
0.4431
-6.6882
0.4184
-5.0044
0,5698
19.7868
0,5107
NPO
11.9082
0.3565
-15.7638
0.3267
10.9100
0,4919
-8,7037
0.2876
w/ GRU
-0.9366
0.3935
-3.1620
0.3714
9.9550
0,5408
-251066
0.4570
GD
-6.5526
0.4061
-50.2968
0.2999
-13.4847
0.5549
-13.9215
0.3930
w/ GRU
-5,0099
0.4138
-13.4785
0.4096
-12.4230
0,5637
11.7760
0.5407
NPO+KL
11.9082
0.3634
-17.2193
0.3444
-10.4275
0,5094
-9,4004
0.3109
w/ GRU
-0.0360
0.3833
-3.1220
0.3654
10044275
0,5585
-2.1101
0.4480
NPO+GD -12.4230
0.4002
-19.7868
0.4026
-11 -1.90882
0.5256
-12.6133
0.4750
w/ GRU
-9,4931
0.4514
-7.6651
0,4122
-8,1703
0,5673
-2.9381
0.5000
SimNPO+GD
-12.9485
0.4428
-26.6801
0.4523
-9,0417
0,5073
-9,8400
0.5527
w/ GRU
-12.9485
0.4862
-25.4588
0.4934
-9.0177
0,5516
-9.4304
0.6168
16

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Table Detailed experimental results on the WMDP benchmarks with QA accuracies evaluated on Bio unlearning and MMLU using the ZEPHYR--B--BEE backbone. The results are presented in two adjacent rows for each method. one row (original baseline method name) showing the original results and the other (w/ GRU) displaying the results combined with GRU. The superior results between configurations with and without GRU for each baseline method are highlighted in bold.
Unleariing
Retention
Method
Bio +
Cyber
MMLU f
Original
0.6371
0,4383
0.5814
GA
0.2474
0.2431
0.2465
w/ GRU
0.2474
0,2446
0,2852
WGA
0.2476
0.2647
0.2454
w/ GRU
0.2474
0.2587
0.2604
GD
0.2474
0.2441
0.2589
w/ GRU
0.2474
0.2511 I
0,2995
NPO 0.2655
0.2793
0.3033
w/ GRU
0.2561
0.2793
0,3704
NPO+GD
0.2710
0.3493
0.4724
w/ GRU
0.2639
0.3524
0.5033
Table 4. Detailed experimental results on the MUSE benchmarks with KnowMem, assessing the efficacy of removal and retention on targeted and non-targeted data, respectively. The results are presented in two adjacent rows for each method, one row (original baseline method name) showing the original results and the other (w/ GRU) displaying the results combined with GRU. The superior results between configurations with and without GRU for each baseline method are highlighted in bold.
Method
VerbMem
KnowMem-U
KnowMem-R
Original Retrain
99,7016
45.8791
69.4009
13.8896
30.1380
69.0496
GA
0.0000
0.0000
0.0000
w/ GRU
0.0000
0.0000
6.7006
WGA
0.2284
0.0000
0.0000
w/ GRU
0.0198
0.0000
18.4555
GD
0.0000
0.0000
8.6971
w/ GRU
0.0000
0.0000
9.8586
NPO
0.0000
0.0000
0.0000
w/ GRU
0.0000
0.0000
9.5913
NPO+GD
0.0000
0.0000
23,5565
w/ GRU
0.0000
0.0000
30.7492
17

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
C. Comparison with RMU on WMDP
For completeness, we compare our approach with RMU, the method proposed alongside WMDP. Due to sensitivity issues noted in the official implementation, we set the hyperparameter a to 100 (instead of the default 1200) to ensure stable optimization. Table 5 reports the results for both RMU and its combination with GRU.
Table Comparison of RMU and RMU with GRU on the WMDP benchmark (Bio and Cyber: accuracy 4; MMLU: accuracy f).
Method
Bio /
Cyber +
MMLU
RMU
0.26
0.31
0,41
w/ GRU
0.26
0.28
0.44
As shown, GRU consistently improves both unlearning and retention over the RMU baseline.
D. Hyper-parameter Analyses
In addition to our main results, we further discuss about our hyper-paramett configurations well conduct additional analyses on hyper- parameter sensitivity.
D.I.
Configurations
We employ grid search validation data to select proper hyper-parameters for GRU and TRU. For GRU, the candidate values for include {0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99), and that for are {0.001, 0.005, 0.01, 0.1, 1.0, 10, 1003. For TRU, we select stg from the space {0.50, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85)
while fix 1r = 1e-1, We further summarize their detailed configurations across different setups as follows.
GRU. For TOFU with Phi-1.5, we default to set 0.001, while adjusting r 0.01 for the 5% setup and without using gradient clipping for GD. For TOFU with Llama-2-7B. we do not use gradient clipping for GA, GD, NPO, and SimNPO under the 5% setup, while setting 1.0 for all other methods. In the 10% setup, T 1.0 for GA, WGA, GD, SimNPO, 1.0 for GA; 0.01 for NPO and NPO+GD; 0.001 for GD and WGA. For MUSE, T = 1.0 for GA, GD and WGA; 7 = 100 for NPO and NPO+GD.
and NPO+GD; 0.5 for NPO; 0.1 for NPO+KL. For WMDP,
For TOFU, we by default set 7 = 0.8, while setting 7 = 0.05 with Llama-2-7B and and 7 = 0.1 with Phi. Also, we set 7 = 0.5 for SimNPO. Moreover, we set 7 = 0.8 for MUSE and 7 = 0.99 for WMDP.
TRU. With the backbone of Phi-1.5, we set stg 0.7 under the 5% setup and stg 0.75 under the 10% setup. Also, with the backhone of Llama-2-7B, set stg 0.65 under the 5% setup and st.g = 0.85 under the 10% setup.
D.2. Sensitivity Analyses
As case study, we conduct sensitivity analyses on TOFU with Llama-2-7B as the backbone, under the 5% unlearning setup.
Gradient Clipping. We first present the results across various values of r, summarized in Table 6. The results show that, across different baselines, the effects of altering have a smooth control over the performance metrics of FQ and MU. This observation indicates that our GRU exhibits robustness with respective to different choices of T.
Exponential Moving Average. We further display the results across different y in Table 7. As with the gradient clipping, we observe smooth control on the overall efficacy of unlearning, further indicating that our GRU method demonstrates robustness against variations in its two paper-parameters
E. Ablation Studies and Other Analyses
We provide more analyses to further show the respective effects of different components involved in our algorithm design.
Ablation Studies. Previous works, such as (Zhang et al., 2024), also use gradient clipping (GC) to improve the overall efficacy of unlearning, raising us to ask if our rectification mechanism plays a key role to mitigate the trade-off between removal and retention. In Table 8, we conduct ablation studies on TOFU using Llama-2-7B as the backbone, focusing on the
18

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Table 6. Hyper-parameter turning of on TOFU with llama-2-78, under the 5% unlcarning setup.
Method
Metric
7
0.001
0.01
0.1
1.0
2.0
3.0
10
100
N/A
FQ f
-29.6514
-15.7370
-7.7503
-8.6008
-4,7631
-4,4360
-2,9534
-3,2299
-3,5161
GA GRU
MU 1
0.6326
0,5824
0.5761
0.5810
0.5684
0,5550
0.5121
0.5149
0,5190
FQT
-27.2750
-18.7967
-12.9485
-9.0417
-9.4931
-9,9550
-9,9550
-10.4275
-9.5500
NPO w/ GRU
MU1 1
0.6268
0.5796
0.5574
0.5220
0.5519
0.5318
0.5312
0.5373
0.5408
FQ f
-27.2750
-18.7967
14.0316
-15.1577
-4455893
14.5893
-15.7370
-12.4230
-12.4230
GD w/ GRU
MU T
0.6200
0.5395
0.5442
0.5434
0.5467
0,5467
0.5484
0.5637
0.5637
Table 7. pareme-rr turning of on TOFU with Llama-2-77 under the 5% unlearning setup. The notation indicates that the associated result is same to those without GRU.
Method
Metric
0,01
0.05
0.5
0.9
FQ
-6.9414
-5,8059
-6.9114
-7.3407
-8,1703
-7.3407
-6,5226
-7,7503
-6.5526
-8.6008
-6.5526
-8.0008
-8. -.1203
GA GRU
MU |
0.4528
0.4418
0.4715
0.4994
0.5247
0.5504
0.5601
0,5677
0.5739
0.5810
0.5827
0.5847
0.5829
FQ1
-11.9082
-11,4040
-9.5550
-9.9550
-9,4931
-8 8.6008
.8,1703
-9.0417
-9.0177
-9.0417
-10.4275
-10.9105
-9.0417
MU f
0,4736
0,4773
0.4865
0,4952
0,5108
0,5144
0.5159
0,5178
0,5222
0,5220
0,5425
0,5358
0,5656
FQ1
-2294855
-12.4230
-12,4230
-14.0316
-12,9485
-11,9082
-14.0316
-14.0316
-13,4847
GD w/ GRU
MU
0.5642
0.5640
0.5637
0.5615
0.5604
0,5585
0.5586
0,5592
0.5582
5% unlearning setup. We compare three scenarios: the original unlearning method without GRU (w/o GRU), the original method enhanced with GC (w/ GC), and the unlearning method that incorporates GRU (w/ GRU). As evident from the results, GRU demonstrates superior scores terms of both FQ and MU, showing its efficacy in mitigating the trade-off between removal and retention.
Table Ablation studies on TOFU with 1l7ma-2-7B, under the 5% unlcarning setup.
Method
Component
FQ
MU t
w/ GRU
-16.9281
0.0000
GA
w/ GC
-20.7646
0.0000
w/ GRU
-3.2299
0,5149
w/o GRU
-10.9105
0.4919
NPO
w/ GC -10.9105
0.4970
w/ GRU
-10.4275
0.5373
Visualization of Rectification. We further examine the angles between gu and g! along with those results after being rectified via GRU. We monitor the dynamics of these angles throughout the unlearning processes for various baseline methods, as well as the changes after applying GRU for rectification. As a case study, Figure 7 shows these results on TOFU (t)
(t) 5% unlearning with Alama-2-7B the backbone. Without the use of GRU, the cosine similarity between gu and gi keeps negative throughout unlearning, suggesting potential adverse effects on the overall performance of the model. In comparison, within the unlcarning dynamics facilitated by GRU, it is observed that although the angles initially continue to be negative (dotted lines), our rigorous method of gradient rectification will adjust the resulting cosine similarity to exactly 0. This adjustment ensures that the gradient direction associated with unlearning is completely orthogonal to that of retention, thereby effectively maintaining the overall model performance.
19

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
0.00
0.00
0.00
-0.25
-0.25
-0.25
000
-0.50
-0.50
00s
-0.0
-0.75
Orig
-0.75
Onig Pre rect
-0.75
Orip
-1.00
Pre-rect.
Post-
-1.00
Post-mect
-1.00
Pro-spet
Podi-eect.
0
10
20
30
10
20
30
0
10
20
30
Steps
Steps
Steps
(a) GA
(b) NPO
(C) WGA
Figure? 7. Angles between gu and (6) (laheled 'Orig.' for the original baseline and 'Pre-rect." for the GRU-enhanced haseline) and between gf and dd (labeled 'Post-rect.' for the GRU-enhanced bascline) across various unlearning steps. We evaluate the bascline methods NPO, GA, and WGA in a 5% TOFU setup using the Llama-2-7B model.
F. Aligning Unlearning with Retention: A Practically Meaningful Evaluation
Recalling the dual goals of LLM unlearning, a meaningful evaluation requires not only improvements in the forget set, but also that the retention utility of the model remains well aligned with its original capabilities. A substantial decline in utility performance would render the resulting model ineffective, making the process of LLM unlearning itself meaningless. This concem, often overlooked in recent unlearning studies, can be addressed by Unlearning with Control (UWC) (Wang al., 2024a), discussed in Section 2.2. UWC provides post-unlearning calibration framework that restores retention performance by interpolating the model parameters before and after unlearning via tunable parameter a. Leveraging UWC calibration, we systematically evaluate our approach under explicit retention thresholds (e.g.. 85%, 90%, and 95%) to investigate whether strong unlearning performance can be achieved without sacrificing essential retention utility, thus aligning unlearning objectives with practical deployment needs. Given the similarity of our findings across multiple benchmarks, we focus here on representative results obtained from the challenging Phi setup on TOFU. Specifically, we present GA and NPO as baseline methods to illustrate the flexibility of UWC calibration and highlight the effectiveness of our GRU approach in attaining superior unlearning results while rigorously maintaining retention utility. The results in Tables 9 and 10 clearly demonstrate that, under each retention constraint, incorporating GRU consistently yields substantial improvements in forget quality (FQ) compared to the calibrated baselines, while perfectly maintaining the prescribed model utility (MU). This pattern holds for both GA and NPO methods, across all retention targets and unlearning setups. These findings validate the practical value of our approach: by leveraging UWC calibration in combination with GRU, practitioners can achieve strong, controllable unlearning effects without sacrificing the essential utility of large language models, thereby ensuring that unlearning objectives remain aligned with real-world deployment requirements.
Table GA on the TOFU Phi- setup with UWC calibration. We report FQ (forget quality, 1) and MU (model utility, 1) for 5% and 10% unlearning under three retention targets (85%, 90%, 95%). Each retention target is shown in two adjacent rows: the first row gives the calibrated GA result, and the second (w/ GRU) shows the result after incorporating GRU, The better score within each GA GRU pair in bold.
5% Unlearning
10% Unlearning
Method
FQt
MUt
FQt
MUt
Original
-28.8
0.52
40.5
0.52
GA (85%)
-22.0
0.44
-35.3
0.44
w/ GRU (85%)
-8.6
0.44
-20.8
0.44
GA (90%)
-28.1
0.47
-36.8
0.47
w/ GRU (90%)
11.2
0.47
-28.8
0.47
GA (95%)
-28.1
0.49
-39.8
0.49
w/ GRU (95%)
-18.8
0.49
-33,9
0.49
20

---

Mitigating the Tradc-off between Unlearning and Retention for LLMs
Table 10. NPO on the TOFU Phi-1.5 setup with UWC calihration. Layout and notation follow Table
5% Unlearning
10% Unlearning
Method
FQt
MUT
FQt
MUT
Original
-28.8
0.52
40.5
0.52
(85%)
-15.7
0.44
-31.9
0.44
w/ GRU (85%)
-9.5
0.44
-15.2
0.44
NPO (90%)
-00.1
0.47
-35.3
0.47
w/ GRU (90%)
-12.9
0.47
-18.2
0.47
NPO (95%)
-25.0
0.49
-38.2
0.49
w/ GRU (95%)
-14.0
0.49
-20.8
0.49
G. Comparison with Gradient Direction Rectification (GDR)
Closely related is Gradient Direction Rectification (GDR) (Lin al., 2024), it similarly employs gradient projection to resolve conflicts between forgetting and retention objectives. However, GDR relies on caching gradients across epochs, resulting in substantial memory overhead that limits its scalability for large language models, where parameter sizes are massive and training typically involves only few epochs. In contrast, GRU dynamically estimates retention gradients using an exponential moving average, greatly reducing memory cost and enabling practical unlearning at scale. Furthermore, while GDR merges retention gradients directly into parameter updates, potentially increasing the risk of overfitting the retention set, our GRU approach leverages retention gradients solely constraints for rectification. Finally, TRU extension addresses the challenge of biased retain data, scenario unique to LLM unlearning and not considered in GDR.
21


Reviews:
[{"review_id": "0rljzSeplk", "reviewer": "Reviewer_XYtU", "summary": "The authors introduce GRU as a flexible framework designed to be integrated with existing unlearning methods to balance the trade-off between knowledge removal and retention in LLM unlearning. GRU constrains unlearning gradients to minimize their negative impact on retention. Theoretical analysis and empirical evaluations across multiple benchmarks and diverse unlearning baselines confirm its effectiveness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your great support and constructive comments! Please see our responses below.\n\n> Q1. It is unclear how the constraints on gradients from different samples and mini-batches are enforced in Eq. (7). \n\n**A1**. As shown in Algorithm 1, **mini-batches $B_{\\rm r}$, $B_{\\rm u}$ replace $D_{\\rm r}$, $D_{\\rm u}$**. In Section 3.2, we further discuss the limitations of mini-batch gradients and recommend using the exponenital moving average for stable estimation. We will clarify these points in our revised manuscript. Sincere thanks for your suggestion!\n\n> Q2. Performing grid search to find the optimal values for these hyper-parameters is computationally expensive and time-consuming.\n\n**A2**. Many thanks for raising this concern. We would like to clarify the practicality of our approach from the following two perspectives. \n\n1. **The GRU performs well even without these hyper-parameters.** Our basic framework following Eq. (7) operates without any hyper-parameters. Incorrperating $\\gamma$ (for EMA) and $\\tau$ (for clipping) are practical enhancements that improve empirical results. Below is the table showing results for 3 methods under the LLaMA setup, without using $\\gamma$ and $\\tau$. The results highlight notable improvements of our method over the baselines.\n\nMethod |FQ 5%↑|MU 5%↑|FQ 10%↑|MU 10%↑|\n|:-:|:-:|:-:|:-:|:-:|\n|GA|-16.93|0.00|-14.37|0.00|\n|w/ GRU|-3.52|0.51|-7.34|0.22|\n|NPO|-10.91|0.49|-8.70|0.29|\n|w/ GRU|-9.96|0.54|-4.12|0.35|\n|GD|-13.48|0.55|-13.92|0.39|\n|w/ GRU|-12.42|0.56|-10.91|0.53|\n\n2. **The GRU is not very sensitive to these hyper-parameters.** By incorporating $\\gamma$ and $\\tau$, we explore the potential to further enhance unlearning. In Appendix D, we conduct a hyper-parameter sensitivity analysis, showing stable improvements across a wide range of candidate hyper-parameters.\n\n> Q3. Some claims need further justification, for example, \"..., the remaining data points within  $D_{\\rm u}$ can offer information for retention.\".\n\n**A3**. Heuristically, retain gradients primarily **act as a denoising mechanism, rather than directly contributing update directions**. By using retain gradients to redirect original unlearn gradients, we focus on the direction that removes targeted knowledge, while with limited side impacts on unrelated knowledge. \n\nFor TRU, let's first consider a simple scenario to remove a single $s_{\\rm u}$. Here, it is straightforward to take the complementary, i.e., $D_{\\rm u} \\backslash{s_{\\rm u}}$, for retention. We then create a rectified task vector targeted at eliminating knowledge associated with $s_{\\rm u}$ while minimizing impact on unrelated knowledge. This process can be applied to each $s_{\\rm u}$ within $D_{\\rm u}$, with the collective average forming the rectified task vector for the entire $D_{\\rm u}$. \n\nNotably, **these task vectors remain mutally compatible**, as the retain gradients are exclusively employed for denoising rather than direct used into the task vectors. We will add the related discussion in our revision. \n\n\n> Q4. The results presented in the figures are not clear.\n\n**A4**. Apologies for any confusion caused by figure annotations. As shown in captions, each pair of scores represents metric values (either FQ or MU) before and after applying GRU. Taking Figure 3(a) as an example, the pair (-16.93, -3.52) represents FQ for GA, where -16.93 is the score without GRU and -3.52 is the score with GRU. The visual representation using an upward growing grid between these scores emphasizes the achieved improvement. We will provide more  explanations in our revision to ensure clarity.\n\n\n> Q5. The preliminaries section takes up excessive space. The authors should consider making this section more concise while ensuring it remains comprehensive.\n\n**A5**. We have a relatively extensive preliminaries section to ensure self-containment and maintain clarity. However, we fully concur with your opinion that a more concise verision would enhance the paper flow and allow larger space to elaobrate on our core contributions regarding GRU and TRU. We sincerely appreciate your feedback and will carefully refine our paper structure accordingly.\n\n> Q6. You mentioned that \"retain data can often be biased.\" Could you clarify what bias refers to in this context?\n\n**A6**. Many thanks for your question. Taking TOFU as an example, the current setup involves selectively unlearning specific author profiles while retaining others. However, we recognize that the broader objective of retention should ensure model capacity across diverse domains such as humanities and sciences. Therefore, the current retain data in TOFU may exhibit bias, **stemming from the distribution shift between the adopted retain data and the broader real data**. We will refine the related discussion in our revision.", "claims_and_evidence": "The authors present a detailed theoretical analysis explaining how GRU effectively mitigates the negative impact on retention. Additionally, extensive experimental results further validate its effectiveness.", "methods_and_evaluation_criteria": "The authors incorporate the proposed GRU into several well-established baselines, including GA, WGA, NPO, and GD, and conduct extensive experiments on widely used benchmarks such as TOFU, WMDP, and MUSE.", "theoretical_claims": "The authors present two theorems, Theorem 3.1 and Theorem 3.2, along with detailed proofs and analyses to support their validity.", "experimental_designs_or_analyses": "The authors present detailed experimental settings, evaluations, and baselines. The design and analysis are well-structured and valid.", "relation_to_broader_scientific_literature": "This paper introduces a novel regularization method that contributes to the research on LLM unlearning.", "essential_references_not_discussed": "The authors have cited most related works.", "other_strengths_and_weaknesses": "Strengths \n\n1. The authors use two figures, Figure 1 and Figure 2, to effectively illustrate the motivation and functionality of GRU with clarity.\n2. In addition to GRU, the authors introduce Task Vector Rectified Unlearning (TRU) to remove the dependency on a retention set.\n\nWeaknesses\n\n1. Some concepts lack clarity. For example, it is unclear how the constraints on gradients from different samples and mini-batches (i.e., the random mini-batch from $D_r$ and the mini-batch from $D_u$)  are enforced in Eq. (7). The authors should provide further explanation or justification.\n2.  There are too many critical hyper-parameters, such as $\\gamma$ and $\\tau$. Performing grid search to find the optimal values for these hyper-parameters is computationally expensive and time-consuming.\n3. Some claims need further justification, for example, \"for each individual data point $s_u \\in D_u$ targeted for unlearning, the remaining data points within $D_u$, i.e.,$D_u$ \\ ${s_u}$, can offer information for retention if used properly.\". \n4. The experimental results presented in the figures, i.e., Figure 3 and Figure 4, are not clear.", "other_comments_or_suggestions": "1. The preliminaries section takes up excessive space. The authors should consider making this section more concise while ensuring it remains comprehensive.", "questions_for_authors": "1. You mentioned that \"retain data can often be biased.\" Could you clarify what bias refers to in this context? Does it relate to dataset distribution shifts, representation imbalances, or another specific aspect? A more precise explanation would improve clarity.\n2. See weaknesses in the above section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethical_review_concerns": "n/a"}}, {"review_id": "YXTvvyVbrv", "reviewer": "Reviewer_z9sW", "summary": "The paper introduces Gradient Rectified Unlearning, an unlearning framework that constrains the unlearning gradient by projecting it onto the half-space where retention is preserved, using the gradient of the loss computed on mini-batch retain samples. This framework can be applied orthogonally to common objective-based machine unlearning methods, and the authors demonstrated its removal efficacy and retention reliability with both empirical evaluation (on TOFU, WMDP and MUSE benchmarks) and theoretical analysis. The authors also explored the retain-data-free setting by incorporating task vectors, which shows performance gain on the TOFU benchmark.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Due to strict space limits, we try our best to address the most critical questions as briefly as possible. We sincerely welcome any further concerns and will try our best to respond to them.\n> Q1. The improvement in removal on WMDP appears not as evident or consistent as on TOFU.\n\n**A1**. WMDP uses a \"Choose 1 from 4 QA accuracy,\" where **random guessing (totally unlearned) would result in about 0.25**. As observed, QA accuracies are already near 0.25, leaving little room for further declines. Our efficacy on WMDP is actually reflected by retention, with 2-7 percentage points increase over that without GRU. These results confirm that **GRU maintains strong unlearning while improve retention**, aligning our pirmary goal.\n> Q2. It would be helpful if more baselines can be included, such as RMU and FLAT. \n\n**A2**. Many thanks for your suggestion. We present results using RMU and FLAT on the WMDP, along with their versions with GRU. As observed, GRU indeed enhances performance for both retention and unlearning.  \nMethod|WMDP Bio↓|WMDP Cyber↓|WMDP MMLU↑|\n|:-:|:-:|:-:|:-:|\n|RMU|0.26|0.31|0.41|  \n|w/ GRU|0.26|0.28|0.44|\n|FLAT|0.25|0.25|0.27|\n|w/ GRU|0.24|0.25|0.28|\n\nWe adjust $\\alpha$ from the default 1200 to 100 in the RMU open-sourced code, after finding that the original settings caused the retain term to overly dominate, hindering model updates. Similar sensitivity issues, such as precision setups, have been reported by others in the WMDP GitHub repository.\n\nWe also expect the potential to further improve RMU and FLAT. For RMU, which perturbs localized representations, we aim to develop mask-based constraints for better knowledge localization. For FLAT, it uses a preference-based learning framework with distinct gradient behaviors from methods like GA and NPO. Thus, we need to devise new retain risk for more appropriate constraints, which we will explore in the future. \n\n> Q3. This paper has significant overlap in terms of methodology of GDR.  WGA also examines the gradient direction of the unlearning objective. \n\n**A3**. **Regarding GDR**, its requirement to store gradients across epochs is infeasible for LLMs due to memory costs and fewer epochs (e.g., 1 for MUSE). GRU avoids extensive caching and epoch-dependent operations, which is more practical for LLMs. Also, for LLMs, the retain data can lead to overfitting when directly guiding updates as in GDR. GRU, by not directly merging retain gradients into its updates, avoids this issue. Also, our enhanced version, TRU, explicitly address this concern, which is superior over GDR.\n\n**Regarding WGA's associated paper**, it employs gradient computations to find reliable unlearning objectives. While conceptually possible, its practical challenges can moviate our work into optimization-focused research. Thus, **our work extends beyond WGA's, further emphasizing the importance in exploring gradients, while focusing on a promising yet orthogonal direction**. \n\n> Q4. It would be much appreciated if the authors could provide details to help validate the results. \n\n**A4**. The configurations are detailed in Section 5 and Appendix C.1. The batch size for retain samples, set at 32, mirrors the unlearn batch and follows the random sampling as to GD. We further provide our source codes via the [Anonymous GitHub](https://anonymous.4open.science/r/GRU-664A/).\n> Q5. For TOFU, are you using the TS-test (log) p-values as FQ? And for MU, are you using the aggregation of Truth Ratio, Prob, and ROUGE-L? \n\n**A5**. To enhance readability and ease figure plotting, we uniformly report the $\\log p$ values, avoiding dealing with extremely small $p$ like $1e^{-19}$. For MU, we adhere to TOFU, reporting the aggregated metrics.\n> Q6. By default, MUSE used LLaMA-2-7B on the news subset and ICLM-7B for the HP books subset, and you are only using the ICLM-7B model. Are you testing on the entire MUSE, or just the books subset?\n\n**A6**. Following your suggestion, we further report results on MUSE News with LLaMA-2-7B, taking GA and NPO due to space limit. UWC is adopted for calibration towards a fair comparison. As observed, there are notable improvements in PrivLeak, espeically for GA.\n\nMethod|VerbMem ↓|KnowMem-U ↓|KnowMem-R ↑|PrivLeak → 0|\n|:-|:-:|:-:|:-:|:-:|\n|GA|31|60|47|-100|\n|w/ GRU|11|56|47|-18|\n|NPO|43|58|47|-100|\n|w/ GRU|35|50|47|-96|\n> Q7. Could GRU be seen as an explicitly-constrained variant of GD that relies on retention gradient estimates from up-sampled retain data?\n\n**A7**. Yes, your understanding is correct. If GA as the objective and combining it with GRU corresponds to the explicit-constrained variant of GD. The benefits of our approach are clear, which forcibly ensures retention. In contrast, using original GD can result in unlearning gradients dominating updates, adversely affecting retention.", "claims_and_evidence": "In general, I find the claims supported by clear and convincing evidence.\n\n* The paper claims that \"GRU offers enhanced reliability in retention\", which matches with intuition, as GRU uses the retention gradient to adjust the original unlearning gradient, ensuring no negative projection conflicting with the retention direction. This claim is supported by empirical evidence: model utility consistently improves when GRU is applied with current unlearning methods, especially for GA-based methods where the retain set was not originally used. The authors also present theoretical evidence with Theorem 3.2, showing that the retention loss with GRU does not surpass that from the original unlearning method. Contingent on its proof being sound (see the `Theoretical Claims` section for concerns), I think this claim is well-supported. \n\n* Regarding the aspect of forget quality, the paper claims that GRU achieves “powerful unlearning capabilities” (L91). Although Theorem 3.1 and the experiments provide some evidence for this, I find it somewhat over-claiming. The improvement in removal effectiveness on WMDP appears not as evident or consistent as on TOFU (as seen in Fig. 4a and the WMDP-cyber results in Table 3). Given that the room for improvement on MUSE with the KnowMem metric is limited (Fig. 5a), it would be more convincing if the authors included other metrics introduced in MUSE (e.g. PrivLeak). Additionally, it would be helpful if more baselines can be included, such as RMU [1] and FLAT [2].\n\n\n[1]. The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning\n[2]. LLM Unlearning via Loss Adjustment with Only Forget Data", "methods_and_evaluation_criteria": "The problem is to mitigate the tradeoff between removal and retaining in LLM unlearning, and specifically, the side-effect or excessive unlearning that is common among current gradient-based methods. The proposed methods (both GRU and TRU) make sense and is well-motivated by the intuition of projecting out the component opposite from the retain gradient direction, thereby forcing a non-obtuse angle that regulates the gradient updates, leading to a smaller retain loss and potentially better retention or minimal side-effect on model utility.\n\nThe choice of the three commonly used benchmarks is reasonable, and the authors follow the default evaluation settings for each benchmark, even though this requires using different backbones and tuning specific hyperparameters for each dataset. However, additional metrics or baselines could be included, as noted in the previous section.", "theoretical_claims": "I checked the correctness of the closed-form solution derivation in A.1 and the proof of Theorem 3.1 in A.2. \n\nFor the proof of Theorem 3.2 in A.3, the authors start by introducing the definition of q-Curvature, with which I am not familiar. Despite a careful literature search (e.g. from differential geometry [3]), I was unable to find a reference of this definition within the machine (un)learning community, or determine how this concept is derived from existing mathematical literature. Therefore, I am not able to assess the soundness of this proof. \n\nIt would be much appreciated if the authors could provide more background and specify how it is adapted from previous works.\n\n[3]. (2010). Q-curvature. In: Conformal Differential Geometry. Oberwolfach Seminars, vol 40. Birkhäuser Basel. https://doi.org/10.1007/978-3-7643-9909-2_1", "experimental_designs_or_analyses": "The provided experimental designs and analyses are valid, and the authors adhere to the default experimental designs and settings from the chosen benchmarks.", "relation_to_broader_scientific_literature": "The proposed idea addresses the problem of excessive unlearning, and therefore mitigates the fundamental tradeoff in machine unlearning between adequate removal and model utility. Some results and findings, such as GA-based methods can be greatly improved by augmenting with GRU, and NPO-based methods generally work better, align with broader scientific literature. The exploration of TRU without requiring retain data displays potential for this framework to benefit a wider range of unlearning methods.", "essential_references_not_discussed": "This paper has significant overlap in terms of methodology with [4], which was published in MM '24. [4] uses the same idea of gradient projection when there is a conflict (termed \"Gradient Direction Rectification\"; see Sec 4, Eq 4, and Algorithm 1 of the paper). Given that this gradient projection idea is indeed intuitively simple, I do acknowledge that it is likely that the authors have developed this idea independently but missed this work unintentionally. Nevertheless, a reference and a discussion on the similarities and differences should be added.\n\nSome less significant problems include:\n\n* Some related baselines could be added to strengthen the paper, as mentioned in the `Claims And Evidence` section. \n\n* WGA [5] was cited and used as a baseline method in the experiments. However, it was not discussed in Section 2.2. Notably, this paper also examines the gradient direction of the unlearning objective. A discussion clarifying the differences between the two works should also be included.\n\n[4]. GDR-GMA: Machine Unlearning via Direction-Rectified and Magnitude-Adjusted Gradients\n\n[5]. Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond", "other_strengths_and_weaknesses": "Strengths:\n\n* The authors extend this idea of rectification to the retain-data-free setting, which is arguably more valuable.\n\nWeaknesses:\n\n* While the method is simple, intuitive, and well presented with geometric illustration, similar gradient projection ideas for unlearning has been proposed in previous works (see `Essential References Not Discussed`), which impacts its novelty and originality.\n* The effectiveness of this method could be sensitive to the size, quality, and distribution of the retain set.\n* There lacks analysis on the efficiency and added complexity of this method, such as wall-clock time for a gradient update step.", "other_comments_or_suggestions": "Minor typos: \n\nTheorem 3.2 Remark., \"heurstics\" -> \"heuristics\"\n\nSection 4, paragraph 2, \"eeach\" -> \"each\"", "questions_for_authors": "1. How are mini-batch retain samples selected for baselines that do not have a retain set, e.g. GA and WGA?\n\n2. For the evaluation setting of TOFU, are you using exactly the TS-test (log) p-values as FQ? And for MU, are you using only the Truth Ratio sub-metric, or the aggregation of Truth Ratio, Prob, and ROUGE-L? It would be helpful if you could clarify these details in the paper.\n\n3. By default, MUSE used LLaMA-2-7B on the news subset and ICLM-7B for the HP books subset, and you are only using the ICLM-7B model. Are you testing on the entire MUSE, or just the books subset?\n\n4. Could GRU be seen as an explicitly-constrained variant of GD that relies on retention gradient estimates from up-sampled retain data?\n\n5. Would GRU/TRU be robust to relearning attacks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FGUjLYC4jT", "reviewer": "Reviewer_iHZj", "summary": "This paper addresses the problem of Machine Unlearning in Large Language Models (LLMs). To balance performance between the retain set and the forget set, the unlearning update is adjusted to avoid harming the performance on the retain set during the unlearning process. Starting from a fundamental optimization problem designed to implement this idea, the paper presents its closed-form solution, along with pseudocode and the theoretical foundations supporting the approach. The proposed technique is evaluated on WMDP, MUSE, and TOFU benchmarks, and it demonstrates successful performance improvements when combined with the loss functions of GA, GD, and NPO.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincere thanks for your constructive comments, and we hope the following feedbacks can address your concerns. \n\n> Q1. Specifically, I question whether the current way of handling the retain set is sufficient. In my view, unless the utility performance on the retain set (e.g., MU or similar metrics) remains at 90–95% of the original model's performance, the improvements on the forget set may have limited practical meaning because the model's overall performance as a language model would already be compromised.\n \n**A1**. We totally agree with your opinion. A large decline in utility performance would indeed render the resulting model ineffective, at which point the process of LLM unlearning also becomes meaningless. On the other side, carefully tuning the hyperparameters to achieve the goal of preserving 85, 90, and 95% of the original model performance can be tedious. Fortunately, as mentioned in Section 2.2, the UWC method offers a post-unlearning strategy that enables calibrating model performance via model mixing, alleviating the challenges associated with maintaining utility. Due to the relatively high cost of UWC, here we take GA and NPO under the challenging Phi setup as two examples to show UWC's flexibility and GRU's effectiveness. The improvements achieved by GRU are more pronounced. We will add more results in our revision. Many thanks for your suggestion. \n\nw/ UWC |FQ 5%↑|MU 5%↑|FQ 10%↑|MU 10%↑|\n|:-:|:-:|:-:|:-:|:-:|\n|Original|-28.8|0.52|-40.5|0.52|\n|GA (85%)|-22.0|0.44|-35.3|0.44|\n|w/ GRU (85%)|-8.6|0.44|-20.8|0.44|\n|GA (90%)|-28.1|0.47|-36.8|0.47|\n|w/ GRU (90%)|-15.2|0.47|-28.8|0.47|\n|GA (95%)|-28.1|0.49|-39.8|0.49|\n|w/ GRU (95%)|-18.8|0.49|-33.9|0.49|\n\nw/ UWC |FQ 5%↑|MU 5%↑|FQ 10%↑|MU 10%↑|\n|:-:|:-:|:-:|:-:|:-:|\n|Original|-28.8|0.52|-40.5|0.52|\n|NPO (85%)|-15.7|0.44|-31.9|0.44|\n|w/ GRU (85%)|-9.5|0.44|-15.2|0.44|\n|NPO (90%)|-20.1|0.47|-35.3|0.47|\n|w/ GRU (90%)|-12.9|0.47|-18.2|0.47|\n|NPO (95%)|-25.0|0.49|-38.2|0.49|\n|w/ GRU (95%)|-14.0|0.49|-20.8|0.49|\n\n> Q2. In particular, how was the validation data chosen? Since there is no widely established standard for selecting validation data in unlearning settings, I believe the paper should provide a more detailed explanation on this aspect.\n\n**A2**. Many thanks for your comments. This perspective has indeed been overlooked by the majority of the community, and we appreciate the opportunity to detail our implementation below.\n\nFor **unlearn data**, the datasets used for validation are exactly those used for unlearning **in the cases of TOFU and MUSE**, as they focus on privacy and copyright removal and do not require generalization. However, **for WMDP**, which aims to unlearn the entire concepts of harmful subjects, generalization is crucial. Hence, we separate a small set of test data (comprising 200 randomly selected samples) for evaluation, with respect to both the bio and cyber setups. For **retain data**, we separate 200 (20 for MUSE Book due to its small size of retain data) randomly selected samples from the original retain datasets used for unlearning, with respect to all three benchmarks. It ensures that we have a distinct set of data to verify retention compared to test data. We will add a detailed discussion in our revision. \n\n> Q3. Only WMDP-Cyber and MUSE-KnowMem results are reported, but VerbMem results are missing.\n\n**A3**. The results for VerbMem are in Appendix B. We did not include these in the main text as their values and ranks are quite similar to KnowMem-U, offering limited new insights. We apologize for any confusion caused and will clarify this choice in our revision.", "claims_and_evidence": "The claims of the proposed method are clearly presented. Although there are some weaknesses, the paper is overall well-written and solid. Below, I will provide a more detailed discussion on these points.", "methods_and_evaluation_criteria": "The authors use three benchmarks, which seems reasonable. For the evaluation metrics, they adopt those proposed in each benchmark, which is also appropriate. Regarding the performance on the forget set, they show how much improvement is achieved when combined with existing loss functions, which appears to be a reasonable approach.\n\nHowever, I have some concerns about the evaluation on the retain set. Specifically, I question whether the current way of handling the retain set is sufficient. In my view, unless the utility performance on the retain set (e.g., MU or similar metrics) remains at 90–95% of the original model's performance, the improvements on the forget set may have limited practical meaning because the model's overall performance as a language model would already be compromised.\n\nTherefore, as discussed in the Task Arithmetic paper, I believe it is more appropriate to constrain the retain set performance to be at least 95% of the original model and then examine how much forgetting can be achieved under that constraint. Including experimental results from this perspective would significantly strengthen the paper.", "theoretical_claims": "This paper makes a theoretical claim, which appears reasonably sound upon skimming. However, I did not rigorously verify the theoretical development by working through the derivations in detail.", "experimental_designs_or_analyses": "Although hyperparameter settings, such as learning rate (LR), are specified in the paper, the process by which they were determined is not described. That said, the paper does explain how the hyperparameters for the proposed GRU method were selected. However, some questions remain. In particular, how was the validation data chosen? Since there is no widely established standard for selecting validation data in unlearning settings, I believe the paper should provide a more detailed explanation on this aspect.\n\nFrom a metric perspective, the evaluation feels somewhat insufficient. For instance, only WMDP-Cyber and MUSE-KnowMem results are reported, but VerbMem results are missing. I think that evaluating the proposed method on only a subset of metrics provided by the benchmarks does not fully demonstrate its effectiveness. A more comprehensive evaluation, covering all relevant metrics such as VerbMem, would strengthen the paper's claims.", "relation_to_broader_scientific_literature": "The paper is well-connected to the existing literature, as it addresses the problem of severe performance degradation on the retain set caused by previous unlearning methods.", "essential_references_not_discussed": "There was nothing in particular.", "other_strengths_and_weaknesses": "The paper was overall well-written and easy to understand. The logical flow of arguments supporting the claims was excellent.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "I recognize that my perspective on unlearning evaluation differs from that of the authors. While it may not be entirely fair to insist that my view is the only correct one, I still struggle to understand the significance of forget set performance when the model's overall capability as a language model is already degraded. From my perspective, if the utility performance drops significantly, any gain on the forget set becomes less meaningful.\n\nIt would be very helpful if the authors could either provide a counter-argument to this view or present experimental results where the utility performance is controlled to remain at 90–95% of the original model, so that the trade-off between the retain and forget sets can be properly evaluated.\n\nFor reference, my overall recommendation is borderline. However, since there is no borderline option this time, I selected weak reject. I plan to revisit and potentially update my score after reading the author response letter.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6zgkho3r0O", "reviewer": "Reviewer_qBZD", "summary": "This paper is concerned with the problem of large language model unlearning, which is the process of removing a specific piece of information from a pre-trained language model. The authors note that unlearning usually comes with the cost of harming the performance of the model on other tasks. To mitigate this, they proposed `Gradient Rectified Unlearning` (GRU). The key idea of GRU is to project the gradient of the unlearning loss onto a perpendicularly aligned subspace of the gradient of the retention loss. The authors show both theoretical and empirical results to demonstrate the effectiveness of GRU. In addition, they also consider a setting where we only have unlearning data but no retention data, and propose a method called `Task Vector Rectified Unlearning` (TRU) based on an idea similar to GRU. The authors empirically show that TRU can achieve better performance over baseline methods in this setting.\n\n## Update after rebuttal\n\nI am glad to see the authors have incorporated RMU into their experiment design.\nOverall, I am OK with this paper given that you will incorporate these changes into the final version and have raised my score to 3. However, I won't champion it since the changes in the experiment design are kinda significant.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you sincerely for your constructive comments and for helping us identify typos. We hope that the feedback provided below will address your concerns.\n\n> Q1. A major concern about the empirical results is the choice of baseline methods. Given that the authors are already using the WMDP benchmark, I would suggest them to consider RMU, which is proposed in the same paper. From my own research experience, RMU performs better than NPO in the trade-off between unlearning and retention significantly. Without considering RMU, the empirical improvements of GRU and TRU over the baselines are not very convincing to me.\n\n**A1**. We aim to use a general set of representative methods for benchmarking. Regrettably, RMU, possibly due to its complexity to be implemented, is not widely adopted in earlier studies, such as [1,2] (though both cited WMDP). Although some recent papers incorporated RMU, their evaluations are either limited to WMDP [3], or they reveal its sensitivity to hyperparameters [4]. \n\nHowever, we completely agree that RMU represents a state-of-the-art within WMDP, which merits our particular focus. We present the results using RMU on WMDP, along with its version with GRU. As observed, GRU indeed enhance performance for both retention and unlearning, showing our GRU is general to mitigate the trade-off between unlearning and retention. \nMethod|WMDP Bio↓|WMDP Cyber↓|WMDP MMLU↑|\n|:-:|:-:|:-:|:-:|\n|RMU|0.26|0.31|0.41|  \n|w/ GRU|0.26|0.28|0.44|\n\n\nWe adjust $\\alpha$ from the default 1200 to 100 in the RMU open-sourced code, after finding that the original settings caused the retain term to overly dominate, hindering model updates. Similar sensitivity issues, such as precision setups, have been reported by others in the WMDP GitHub repository. We will study these issues in the future. However, we continue to see the RMU as a promising approach, particularly when viewing through the lens of local knowledge perturbation. We plan to delve deeper into the RMU concept and explore more specific verisons of GRU for RMU (e.g., constraints for masking) in our future work.\n\n[1] Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning.\n\n[2] MUSE: Machine Unlearning Six-Way Evaluation for Language Models.\n\n[3] Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning.\n\n[4] Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond.\n\n\n> Q2. Figure 3 is a bit confusing. Each algorithm's performance with and without GRU is shown as a segment in the figure. But it is unclear to me which endpoint of the segment is with GRU, i.e., whether GRU makes the performance better or worse. I can only guess from the caption that GRU improves the performance, but I think it would be necessary to make it clearer in the figure or at least in the caption.\n\n\n**A2**. Apologies for any confusion caused by the figure annotations. As detailed in the figure caption, each pair of scores represents metric values (either FQ or MU) before and after applying the GRU enhancement. For instance, taking Figure 3(a) as an example, the pair of scores (-16.93, -3.52) represents the FQ scores for the GA method, where -16.93 is the score without the use of GRU and -3.52 is the score with GRU. The visual representation using an upward growing grid between these scores emphasizes the improvements achieved by incorporating GRU. We will clarify the meaning of our figures and provide more explicit explanations in our revision to ensure clear understanding.", "claims_and_evidence": "I find the theoretical claims convincing, while the empirical results are not very strong, mainly because of the choice of baseline methods. See the detailed comments in `Experimental Designs Or Analyses`.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense.", "theoretical_claims": "I read the statements in the paper and skimmed through the proofs in the appendix. I did not check the proofs in detail, but there is no reason to doubt the correctness of the theoretical claims.", "experimental_designs_or_analyses": "A major concern about the empirical results is the choice of baseline methods. The authors consider a set of baselines in the main experiments, including GA, GD, NPO and WGA. I think the choice of baselines is not very strong. Among these methods, NPO is probably the best one, but it is still not considered as state-of-the-art. Given the authors are already using the WMDP benchmark, I would suggest them to consider RMU, which is proposed in the same paper. From my own research experience, RMU performs better than NPO in the trade-off between unlearning and retention significantly. Without considering RMU, the empirical improvements of GRU and TRU over the baselines are not very convincing to me.", "relation_to_broader_scientific_literature": "This paper's idea of projecting the gradient of the unlearning loss onto a perpendicularly aligned subspace of the gradient of the retention loss is potentially useful for many algorithms in the context of unlearning, as the authors demonstrated in the paper. But it remains unclear if this improvement applies to state-of-the-art methods like RMU.", "essential_references_not_discussed": "See the comments in `Experimental Designs Or Analyses`.", "other_strengths_and_weaknesses": "The paper is well-written and easy to follow, and I think the idea of GRU is neat.\n\nFigure 3 is a bit confusing. Each algorithm's performance with and without GRU is shown as a segment in the figure. But it is unclear to me which endpoint of the segment is with GRU, i.e., whether GRU makes the performance better or worse. I can only guess from the caption that GRU improves the performance, but I think it would be necessary to make it clearer in the figure or at least in the caption.", "other_comments_or_suggestions": "Line 254: `datat` should be `data`.\n\nLine 255: `eeach` should be `each`.\n\nLine 267: $\\mathbf T_s$ is not defined.", "questions_for_authors": "What is the rationale behind the choice of baseline methods?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper addresses an important and timely challenge in LLM unlearning by proposing Gradient Rectified Unlearning (GRU), a simple yet effective technique that leverages gradient projection to reduce negative side effects on retained knowledge. Reviewers generally agree that the paper is well-written and addresses a critical trade-off in unlearning. The main concern across reviewers is limited originality, as the core idea of gradient projection has precedents in multi-task learning and recent unlearning works. Some reviewers found the initial omission of key related work and baselines to be a weakness. The authors provided thorough and thoughtful rebuttals, acknowledged prior works, and clarified implementation and evaluation details, together with additional experiments in addressing major concerns. All reviewers updated or maintained a positive score, indicating a consistent lean toward acceptance.
Decision: Accept (poster)