Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Authors: Abudukelimu Wuerkaixi
Qizhou Wang
Sen Cui
Wutong Xu
Bo Han
Gang Niu
Masashi Sugiyama
Changshui Zhang
Venue: ICML 2025 poster (2025)
Keywords: Large language model
Continual learning
Unlearning
Robustness
Primary area: deep_learning->large_language_models
TL;DR: We investigate the practical challenge of LLM continual unlearning and propose the ALKN algorithm, which mitigates both accumulative decline and cascading degradation in model utility by adaptively refining gradient updates.

Abstract: With the growing deployment of large language models (LLMs) across diverse domains, concerns regarding their safety have grown substantially.
LLM unlearning has emerged as a pivotal approach to removing harmful or unlawful contents while maintaining utility.
Despite increasing interest, the challenges of continual unlearning, which is common in real-world scenarios, remain underexplored.
Successive unlearning tasks often lead to intensified utility degradation.
To effectively unlearn targeted knowledge while preserving LLM utility, it is essential to minimize changes in model parameters by selectively updating those linked to the target knowledge, thereby ensuring other knowledge remains unaffected.
Building on the task vector framework, we propose a new method named ALKN (Adaptive Localization of Knowledge Negation), which uses dynamic masking to sparsify training gradients and adaptively adjusts unlearning intensity based on inter-task relationships.
Comprehensive experiments across three well-established LLM unlearning datasets demonstrate that our approach consistently outperforms baseline methods in both unlearning effectiveness and utility retention under continual unlearning settings.

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Abudukelimu Wuerkaixi' Qizhou Wang? Sen Cui Wutong Xu
Bo Han 23
Gang Niu
Masashi Sugiyama" 34 Changshui Zhang'
Abstract
questions; Summarize blogs: Assist coding..
Fine- tuning
With the growing deployment of large language models (LLMs) across diverse domains, concerns regarding their safety have grown substantially. LLM unlearning has emerged as a pivotal ap- proach to removing harmful or unlawful con- tent while maintaining utility. Despite increas- ing interest, the challenges of continual unlearn- ing, which is common in real-world scenarios, remain underexplored. Successive unlearning tasks often lead to intensified utility degradation. To effectively unlearn targeted knowledge while preserving LLM utility, it is essential to mini- mize changes in model parameters by selectively updating those linked the target knowledge, thereby ensuring other knowledge remains unaf- fected. Building the task vector framework, we propose new method named ALKN( Adaptive Localization of Knowledge Negation), which uses dynamic masking to sparsify training gradients and adaptively adjusts unlearning intensity based on inter-task relationships. Comprehensive ex- periments across three well-established LLM un- learning datasets demonstrate that approach consistently outperforms baseline methods in both unlearning effectiveness and utility retention un- der continual unlearning settings.
Tech blog website
A ssistant LLM
Users'
deletion requests
Continual wnleaming
i
User
If you're adding SQLAI chen to your Flask app, sot SQLALCHENY_DAIIE DATABASE URI to your darabuse URRI...
User B
For integrating SQLAI Allchc.ey... define models like class User(db.
db. Column (db..IdI..eeeeeee Integer...
Figure I. An illustrative example of the LLM continual unlearning scenario. An LLM is integrated into technical blog website, where some users occasionally close their accounts and request the delction of their blog contents.
1. Introduction
ALKN(Adaptive
Large Language Models (LLMs) exhibit exceptional capa- bilities processing general knowledge, positioning them- selves promising framework the path towards artifi- cial general intelligence (Rozière al., 2023; Wang et al., 2024b; Brown et al., 2020; Zhao et al., 2023). However, reliance on extensive web data introduces risks of memo- rizing sensitive information, including private data, copy- righted material, and harmful content, hindering lawful de- ployments of LLMs (Ji et al., 2024; Wang et al., 2025a). These factors continuously spur growing interest in research LLM unlearning, which aims remove specified sensi- tive information from LLMs while retaining their general knowledge and capabilities, i.e., preserving model utility. It focuses on ensuring that the model cannot reproduce or accurately respond to target data associated with harmful outcomes or legal issues (Liu al., 2024a).
Instittte for Artificial Intelligence, Tsinghua University (THUAl); Beijing National Research Center for Information Science and Technology (BNRist); Department of Automa- tion, Tsinghua University, Beijing, P.R.China "TMLR Group, Department of Computer Science, Hong Kong Baptist Uni- versity RIKKEN 'The University Tokyo. Correspondence to: Sen Cui <cuis .sigaia.iiigh.............. Changshui Zhang <zcs mai...singgua..di edu.cn... idu......gu..............
Although LLM unlearning has received much interest, an important problem of LLM continual unlearning remains underexplored. In real-world applications, unlearning re- quests from users or regulators often arrive continuously, requiring models to unlearn multiple pieces of data over time. For example, we consider a website equipped with a dedicated L.LM that has been fine-tuned using user data from
Proceedings of the 4rnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Cooee is available 2a0can666/ 6/L/A
httpa://githin com/

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
the platform, as shown in Figure 1. Periodically, users may request delete their accounts and have their personal data removed, or regulators may report sensitive information in the LLM that needs to be removed. In these cases, the LLM needs to continuously perform unlearning tasks to ensure compliance with the requests from users and regulators.
the proposed modules, along with the task vector method, ef- fectively address the catastrophic utility decline commonly observed in LLM continual unlearning by dynamically lo- calizing model parameters and adaptively refining gradients.
We conducted comprehensive experiments to validate the effectiveness of our proposed method. To objectively as- sess the utility of models, we introduced Training Data Evaluation Corpus (TRAVIS) generated using the member- ship inference attacks (MIA) method (Shi et al., 2024a), designed to reconstruct knowledge in pretraining data of LLMs across diverse range of topics. Testing on diverse training data provides a more comprehensive assessment of the overall utilityy of LLMs and also highlights precise utility decline relative to their pre-unlearning performance (Thaker et al., 2024). Experiments are performed across multiple models and datasets, demonstrating that ALKN effectively retains the general utility of the model while achieving ef- fective unlearning of sequential tasks. After sufficient un- learning, our method retains over 95% of its model utility, whereas some baseline methods completely lose their utility. We summarize our key contributions as follows:
Maintaining model utility represents a critical challenge in LLM unlearning, which is particularly pronounced in the continual unlearning scenario (Shi et al., 2024b). Represen- tative baseline approaches to LLM unlearning use gradient ascent optimization (GA) and its variants (Han et al., 2020: Yao et al., 2023), inherently affecting the overall utility of the model due to the extreme modification of model parame- ters. The issue of utility deterioration is further magnified in the context of continual unlearning, which can be attributed to two factors. (l) Each unlearning task contributes to decline in the model utility of the L.M.M And such decline accumulates progressively with an increasing number of unlearning tasks. We refer to this phenomenon as accumu- lative decline. (II) After completing preceding unlearning tasks, the memory retention of the L.LM regarding data from subsequent tasks may be disrupted, making them "par- tially forgotten" even before they are explicitly unlearned. Employing fixed unlearning intensity without adjustment for these partially forgotten tasks can easily result in over- unlearning, further exacerbating the decline in model utility, which we refer to as cascading degradation. While accumu- lative decline stems from the progressive loss of utility over multiple unlearning tasks, cascading degradation arises from the unintended over-unlearning of partially forgotten data, amplifying the utility loss in subsequent tasks. These two issues combined exacerbate the utility degradation problem, making continual unlearning in LLMs a significantly more challenging scenario compared to one-time unlearning.
We consider a practical yet challenging LLM continual unlearning setting, conducting theoretical and exper- imental studies on the unique issues of accumulative decline and cascading degradation in this context.
We introduce novel method, ALKN, which adaptively modulates unlearning gradients to effectively minimize the utility degradation that is particularly severe in LLM continual unlearning scenarios.
To rigorously evaluate unlearning methods, we con- struct an evaluation corpus, TRAVIS, and perform extensive experiments across multiple benchmark datasets. The experimental results validate the effec- tiveness and demonstrate the superiority of ALKN com- pared to state-of-the-art baselines.
In this paper, we propose ALKN (Adaptive Localization of Knowledge Negation) to mitigate the problem of utility degradation in LLM continual unlearning. ALKN is built the task vector framework, which first fine-tunes model to reinforce knowledge from unlearning data and then negates the target knowledge by subtracting the parameter offsset from the original model. (Ilharco et al., 2023). Task vec- tors are trained with a lower-bounded loss, which inherently reduces the risk of excessive unlearning compared to GA. To further address the accumulative decline and cascading degradation issues in continual unlearning, we propose three key modules incorporated into the fine-tuning process of task vectors. (I) Entropic soft label loss strategically adjusts unlearning intensity via adaptive training objectives, reduc- ing cascading degradation. (II) Dynamic gradient sparsity facilitates selective fine-tuning of different parameter sets for distinct tasks, minimizing accumulative decline. (III) Adaptive parameter modulation enables paramiter-specific learning rates to further mitigate utility loss. In summary,
2. Problem Setup and Preliminary
In this section, we formulate the setting of continual unlearn- ing, introduce the baseline methods GA and task vectors. We also demonstrate the intrinsic causes and negative impact of the cascading degradation issue.
2.1. LLM continual unlearning
LLM unlearning. This paper explores the concept of un- learning in causal LLMs. For model no, where 0 de- notes the model parameters, the probability of the (2 + 1)-th token given the first / tokens of an input text r is com- puted by the model followed by a softmax function, formal- ized as Th (Z ( The objective of LLM unlcarning is to eliminate the influence of a specified target dataset,
2

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
D₂ = -xx (eaheefn.. along with any related generative capa- bilities, from the model (Blanco-Justicia et al., 2024). Sam- ple zu may take the form of plain text question-answer pairs. Crucially, the unlearning process needs to preserve the knowledge and utility of the model on other data.
Du. resulting in model Tha with enhanced knowledge of Du- The difference between the fine-tuned parameters and the original parameters, On 0. constitutes the task vector for D.. By subtracting this task vector from the original parameters 0, a modified model is obtained that has unlearned the knowledge from D.. Additionally, a scalar À is introduced to control the magnitude of the task vector:
LLM continual unlearning. In the context of LLM con- tinual unlearning, the goal extends to managing a sequence of unlearning tasks. Beginning with an initial LLM char- acterized by pre-trained parameters 8º E Rd, the model is required to sequentially unlearn datasets {D 10]221 where each D = { [ee(nt.) corresponds to the t-th unlearning task. During the training of a specific task, the data of other tasks is not accessible. LLM continual unlearning faces the issue of severe degradation of the model utility on non-target data during successive unlearning processes. To safeguard the general utility of the model throughout this iterative process, a retain set D, } can be employed to mitigate unintended degradation of utility.
unlkaan = 0 - A(On - 0).
(2)
We theoretically analyze the convergence of the GA and task vector method in Appendix B. In the case of logistic regres- sion unlearning, GA maintains a gradient magnitude above a positive constant and fails to converge. In contrast, the task vector method is proven to converge, indicating that it is less prone to over-unlearning and may cause less utility degra- dation in LLMs. While the task vector method mitigates utility loss to some extent during a single unlearning task, it remains susceptible to accumulative decline and cascading degradation issues in continual unlearning scenarios.
[E(n.]
2.2. Gradient Ascent and Task Vectors
A straightforward approach to addressing the continual un- learning of LLMs is to directly apply existing LLM un- learning methods for each task, such as leveraging gradient ascent-based techniques or task vector methods.
2.3. Cascading Degradation
LLM continual unlearning encounters more complex chal- lenges than one-time unlearning, one of which is cascading degradation, where the interaction between unlearning tasks amplifies the utility degradation. To elaborate, the utility decline during preceding unlearning tasks can also impair the memory retention of the LLM regarding the data of subsequent tasks. Besides, in real-world applications, un- learning data is often highly homogeneous, with requests involving similar content.As seen in Figure 1, data from dif- ferent users may share similar structures and content. This homogeneity intensifies the impact of preceding unlearning tasks subsequent tasks. Consequently, the model partially forgets data of subsequent tasks before explicit unlearning, yet these tasks still need to be performed. Unlearning such partially forgotten data with the same optimization process as preceding tasks can easily lead to over-unlearning, ampli- fying the utility degradation. We refer to this phenomenon as cascading degradation.
Gradient Ascent. GA is one of the most fundamental fine- tuning-based approaches for unlearning and often serves as baseline foundational module in many existing meth- ods (Yao et al., 2023). While the model acquires knowledge during training through gradient descent, GA facilitates the unlearning process by optimizing in the opposite direction. This is achieved by minimizing the negative cross- cooss-yntropy loss of predicting the next token on the unlearning dataset:
min- 6
Σ ΣΣ= ZED
log To (x)4)) -
For simplicity, the normalization of the loss by token length is omitted. Unlike gradient descent loss, loss of GA lacks a lower bound, which hinders its ability to converge. Train- ing with GA could lead significant deviations in model parameters from their pre-training values, which severely degrades model utility. The NPO method is also a gradient ascent-based approach. Compared to GA, the gradient in NPO is scaled by a regularization term that gradually dimin- ishes over the course of training, alleviating the problem of over-unlearning to some extent (Zhang et al., 2024a).
To empirically validate the causes and harm of the cascad- ing degradation, we conducted experiments using a baseline approach, NPO+RT on the TOFU dataset, where the LLM model was made to sequentially unlearn five tasks. As shown in Figure 2(a), we examined the model's prediction probabilities P on the data of the t-th task before un- learning it Probability After Previous Unlearning). The results reveal a substantial decrease in these probabilities compared to their initial values Probability Before Any Unlcarning), indicating that the model had already partially forgotten the f-th task after unlearning the first t - 1 tasks. To validate the detrimental effects of cascading degrada- tion, we compared continual unlearning with independent
Task Vectors. Introduced by Ilharco et al. (2023), the task vector method manipulates the ability of pre-trained model to address downstream tasks through fine-tuning and the arithmetic operation of model parameters. When applied to LLM unlearning, the method enables the selective removal of knowledge related to a specific dataset D,, Specifically, a pre-trained model mo is fine-tuned on the unlearning dataset
3

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Probability (Belex Aay V'lkaaiing)
Probability (Afer Curnal Unlearning) Unility (ADer Unloraing)
dataset being correlated. Our objective is verify whether the first unlearning task influences the second task.
Probabillio (After Previoms Unkarainn))
0.45
Proposition 2.1. Consider two correlated datasets D and D", each composed of {( 16)) satisfying: maxizy Kay xf) k(xx ,x),, and y! = =y. where k is positive constant. The logistic model starts with an initial parameter 80, We compare two optimization scenarios:
Conditional Probability
0.8
00,00
Model Utility
0.35
0.6
0030
0025
The model successively unlearns on D and D yield- ing intermediate and final parameters: 0g and Ocu-
0.20
0.0
Starting from 0", the model directly unlearns on D yielding parameters Ouu
Prediction probability under three conditions and model utility.
Rouge-FF F (Cantinnal Unlearning) (ladependeat Unlearning)
Uniliy Change (Continual Uniearning) Unilit Change (Iadependent Ualcarnine)
The parameter changes in the two scenarios during unlearn- ing on D satisfy:
Task
0,00
4005
0200
Utilty Change
I PcoullxTx 11 QuxlxxxX + +TCV
(3)
-OU0S
where C a positive constant depending on the datasets, T is iteration steps, and n is the number of samples.
-0.10
-8:146
RougeR
3.4-
0.333
0.774 67744
03234
-0.15
Please refer Appendix C for the proof. Proposition 2.1 demonstrates that during the process of continual unlearning, conducting a preceding unlearning task causes subsequent related tasks to experience larger parameter changes in their own unlearning. In the context of LLMs, this phenomenon may exacerbate the decline in model utility.
0,3
2
Tisk 3
Task
Task
(b) Rouge-F and utility change under two unlcarning scenarios.
Figure The comparative experiments on the TOFU dataset us- ing the baseline method to sequentially unlearn five disjoint sub- datasets. (a) The unlearning of preceding tasks results in the partial forgetting of subsequent tasks, reflected in the reduced con- ditional prohabilitics of task data predicted by the model. (b) Under comparable unlearning performance, continual unlearning exhibits significantly greater utility degradation than independent unlearning due the effects of cascading degradation.
3. Method
In continual unlearning for LLMs, there is severe deteriora- tion in model utility. To effectively mitigate the utility de- cline, the key is reducing the overall changes to the model's parameters under the condition of effective unlearning. Uti- lizing the task vector method can mitigate utility decline to some extent, but the accumulative decline and cascading degradation issues still exist. Therefore, we further propose three modules to alleviate these issues. Entropic soft label loss utilizes generated labels to flexibly adjust the unlearn- ing objective. Dynamic gradient sparsity applies a learn- able mask to the gradients and encourages different tasks to localize distinct model parameters. Adaptive parameter modulation employs lower learning rate for parameters related to already unlearned tasks. These modules collab- oratively address the challenges of accumulative decline and cascading degradation by mitigating task interference and adaptively regulating the intensity of unlearning. The diagram these modules is shown in Figure 3. The overall procedure of training is in Algorithm
unlcarning, where the model independently unlearns the data of each task starting from pre-trained parameters, as shown in Figure 2(b). The utility changes in the figure denote utility declines in one unlearning task. We ensure that the level of unlearning (Rouge-F) achieved by inddpen- dent unlearning is similar that of continual unlearning, enabling fair comparison of the changes in model util- ity. Under equivalent conditions, continual unlearning with cascading degradation leads to a greater and increasingly severe decline in model utility compared to independently unlearning each task. We also compare the performance of joint unlcarning with continual unlearning in Appendix H to further validate the harm of cascading degradation.
To theoretically substantiate the above conclusion, we con-
sider a toy example where a logistic model π = 1/x, 0) =
3.1. Fine-tuning with Entropic Soft Labels
((x, 0)) performs unlearning with the GA algorithm on a binary classification problem with imputs being r and labels being y. Suppose there are two unlearning tasks with their
To perform the t-th unlearning task, we employ knowledge negation from the current model T with the task vector

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
method. This process begins by initializing model with parameters 0--1, which is then fine-tuned on the unlearning dataset Dí The fine-tuning step yields the intermediate model Teg reinforcing the knowledge of the model specific to the 1-th dataset (Ilharco et al., 2023). Subsequently, the knowledge associated with the t-th task is removed using model weight arithmetic: 0' = 00~ 200 - 0)-
mitigates the cascading degradation issue by generating adaptive optimization goals for partially forgotten data.
3.2. Dynamic Gradient Sparsity
In continual unlearning scenarios, we aim to fine-tune differ- ent sets of model parameters for each task. This approach helps prevent cumulative parameter divergence from their initial values as the number of tasks increases, thereby reduc- ing the risk of accumulative decline. Moreover, minimizing changes to model parameters during unlearning is essential for maintaining utility. We strive to identify parameters that significantly influence the unlearning objective and focus on adjusting these parameters during training. This would en- able effective unlearning while making few adjustments to the model parameters as possible.
The fine-tuning process for task vector training typically leverages gradient descent optimization with cross-entropy loss, which quantifies the discrepancy between the next- token prediction labels and the model's output probability distribution conditioned on preceding tokens:
LCE(Oh: = Σ log Tay (1
(4)
where x E De- In the remainder of this paper, (o(=)44))()))))) is denoted as σh for simplicity.
To achieve these goals, we propose to apply learnable masks to model gradients during the fine-tuning of the model 0g on the unlearning set D allowing for precise and efficient parameter adjustments. Inspired by von Oswald et al. (2021), for the t-th task, the optimization of the model parameters is given by:
However, employing cross-entropy loss in continual un- learning scenarios poses the risk of over-unlearning. As demonstrated in Section 2.3, interaction between unlearning tasks could result in a task being partially forgotten even before its data is explicitly unlearned. The predicted proba- bility Tog for the data of the task gets notably low. In such cases, the cross-entropy loss, which enforces alignment be- tween the output of the model and one-hot label in the token space, produces gradients as follows:
0 k+1 = 0z -
aa' O Vo, ,()), (7)
where m' E Rd represents a dynamic underlying vector, from which a binary mask is derived using the threshold function 1m-20- This function outputs 1 for parameters where the corresponding value of m² exceeds the threshold fi otherwise it outputs O. The threshold is determined by the percentile of m² and increases gradually throughout the training process. Additionally, â' denotes the adaptive learning rate, which is detailed in Section 3.3.
Vo x
(5)
When < is low, the gradient magnitude may increase, caus- ing large parameter updates that negatively impact the utility of the LLM. This issue is theoretically illustrated in Propo- sition 2. Besides, one-hot labels in cross-entropy loss are unnecessarily difficult objectives to optimize towards since the goal of task vector fine-tuning is only to enhance the knowledge of the model regarding the target data. Training with one-hot labels forces redundant parameter adjustments of the LLM model, leading significant utility decline.
Mask Value Updating. The application of gradient mask- ing is designed achieve two purposes: (I) To encourage different tasks to adjust distinct sets of model parameters during unlearning, thereby reducing the overall parameter changes. (II) To dynamically identify model parameters that are important to the unlearning objective while also being beneficial to utility preservation. To achieve these goals, the underlying vector m' is dynamically learned from the data, allowing the mask to adapt task-specific requirements during training. For objective (I), the current mask is en- couraged to differ from the masks used in previous tasks. Specifically, let M = U'= 1 1m represent the set of parameters that underwent substantial adjustments in the first t- 1 tasks. The mask for the current task is encouraged to minimize its overlap with - For objective (II), the mask refined to preserve model utility. The cross-entropy loss D, is used a proxy to evaluate the retained utility of models. The underlying vector m' of the mask is opti- mized to minimize this utility loss. The sum of these two objectives with a scaling factor is given by:
To mitigate this issue, we propose Entropic Soft Labels (ESL.) to preserve model utility during task vector fine- tuning. We introduce soft labels that increase the entropy of the target distribution in the loss. The soft label ÿ is derived from the initial predicted probability Tg.- for a training sample and is defined as the sigmoid function and is a scaling factor that controls the entropy of labels. By leveraging these soft labels, the cross-entropy loss is reformulated as:
(s(Tă 1 1)), where a is
(ESL(H). = Σ
Uرا log Tep (11441|=114)
(6)
min -0(00 - -0), D,) + pllm 2n M. (8) m!.
where ūn represents the soft label for the next token. LESL
5

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
the unlearning objective undergo limited adjustments in the early stages and are subsequently masked, while parameters with significant impact receive more extensive training. This strategy enables balanced training across the model, prevent- ing excessive adjustments to all parameters or overly drastic changes to a small subset. Moreover, our method leverages the underlying vector m' to aggregate the influence of data on the mask over time, mitigating biases that may arise from extreme values in a single training iteration. By accumu- lating data-driven insights, the mask adapts more robustly to task-specific requirements, enhancing the stability and effectiveness of the unlcarning process.
Softlabel
gradient
Gu
gsL
Logit
updkite>*
(at)
updite (G M)
0 (-0.2)
0
0 (-0.3)
(-0.3)
ff
1
1
(0.3)
(0.5)
(-0.1)
(-0.4)
(-0.2)
(0.1)
D D ... Du
m
3.3. Adaptive Parameter Modulation
Figure 3. The diagram illustrating the training process of the in- termediate model mez for the f-th unlearning task. The training loss LESL is computed using entropic soft labels (Section 3.1) Gradients Gu derived from the loss are masked using a binary mask with an underlying vector m', dynamically updated based on Gu, Gr and M (Section 3.2). The masked gradients are then employed to update the model parameters On with adaptive learning rate à' (Section 3.3).
To further mitigate the decline in model utility, we propose adaptive parameter modulation during fine-tuning fi. The gradient masks obtained from our training process repre- sent the relationship between the model parameters and the features of each task. Consequently, during the train- ing of the current task, we apply reduced learning rate to the parameters previously activated, thereby preventing the re-unlearning of features already forgotten and allevi- ating the cascading degradation of utility. Specifically, let M = - m' denote the model parameters that have undergone significant adjustments in the first - 1 tasks. We apply an adaptive learning rate to model parame- ters while fine-tuning on the t-th task as follows:
By substituting into Equation 7 and differentiating this ob- jective with respect to m', we obtain the iterative update rule for n²:
m' +m² + + G, M.M
where G = LEsL (0/ Da); G, = Vg (C Dr).
(9)
a' a ar, M**+),
(10)
where o denotes low learning rate for model parameters that are crucial for previous tasks and ay denotes relatively high learning rate.
The derivation is detailed in Appendix According to the update rule of m the elements will increase where the gradient signs are consistent across both the unlearning objective and the utility objective. And the corresponding parameters will be selectively activated. In other words, the proposed method selectively activates model parameters, aligning the gradient directions of the unlearning and utility preservation objectives. Furthermore, elements that have a greater impact on both objectives-ias i.e.. those with larger gradient magnitudes-lead to faster changes in the corre- sponding values of m', resulting in quicker activation or freezing of the model parameters.
4. Experiments
We conduct extensive experiments on various datasets and models to validate the effectiveness of the proposed method.
4.1. Experimental Settings
Datasets and models. We conduct experiments three datasets: TOFU, MUSE News, and WHP, each representing scenarios relevant to real-world applications such as pri- vacy preservation and copyright protection. To simulate a continual unlearning setting, we divid the data designated for unlearning into multiple disjoint subsets based on the structure of the datasets. These subsets, either from dif- ferent parts of the same dataset or mix of datasets, are treated as sequential unlearning tasks. The experimental details for each dataset are as follows. 1) TOFU (Maini et al., 2024) dataset consists of generated fictional author information. We group the information of four authors into one unlearning task, resulting in five or more contin- ual unlearning tasks. 2) MUSE News (Shi et al., 2024b)
The proposed method localizes distinct parameters for dif- ferent unlearning tasks. By avoiding cumulative changes to parameters, we alleviate the accumulative decline in utility of the LLM. And the gradient sparsity helps preserve the overall model utility, further mitigating the issue.
In contrast to methods that rely on static model parame- ter masks, our proposed approach dynamically adjusts the mask throughout the training process. This dynamic tun- ing ensures that each model parameter is trained within an appropriate range: parameters with minimal impact on
6

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Algorithm 1 Continual unlearning method ALKN
Utility (MU) represents the harmonic mean of the above metries calculated across multiple retain sets in the TOFU dataset (Maini et al., 2024), including the T-Rouge. It pro- vides an aggregate measure of the model utility.
Input: Sequential unlearning datasets {DW retain set D,, initial model parameters 6%, hyperparameters s and 14, learning rates og and oep- updating steps T, for each unlearning task.
Baselines. Given the limited research on LLM continual unlearning, the proposed method is primarily compared with prominent existing unlearning approaches. These algo- rithms are applied sequentially unlearn the data for each task, thereby achieving continual unlearning. Unlearning methods include GA (Yao et al., 2023), NPO (Zhang et al., 2024a), and Task Vectors (TV) (Ilharco et al., 2023), as in- troduced earlier. We also include DPO (Zhang et al., 2024a), SO-PO, and SO-NPO (Jia et al., 2024b) methods. Besides, EUL (Chen & Yang. 2023) incorporates lightweight un- learning layers into the model and explicitly considers sce- narios involving multiple unlearning tasks. WAGLE (Jia et al., 2024a) optimizes only the parameters that are cru- cial to unlearning and retaining objectives during the un- learning process. When applying these methods in contin- ual unlearning scenario, the learning rate is tuned progres- sively to mitigate catastrophic collapse as the number of tasks increases. Retaining objectives are also leveraged with an auxiliary retain dataset to help preserve the model utility in addition to conducting unlearning optimization on the target data. For example, the Retain Loss (RT) is employed as -En, log To (o(d))-y)))))) where gradient de- scent is performed on the retain dataset to optimize the model. Similarly, KL divergence loss (KL), defined as Ep, D(Ta [=r]][=((((]]=)) is used to ensure that the logits of the model outputs on the retain dataset remain similar to those of the initial model nee (Zhang et al., 2024a).
Initialize the overall mask M = O.
for t 1 to T do
Initialize parameters f = -r
Initialize the underlying vector m' O. Calculate G. = V (0' D.).
for k = 1 to T, do
Update model parameters 0 with Equation 7.
Update the underlying vector m' with Equation 9
Tune oy with percentiles of m as in Appendix E.2. end for
0' = =0-11 1(h - 0¹ 1)
M = M U m
end for Output: Final parameters 0².
comprises BBC news articles and includes four predefined unlearning subsets that serve as continual unlearning tasks. 3) WHP (Who's Harry Potter) (Fldan & Russinovich, 2023; Shi et al., 2024b) involves the unlearning of original text from the Harry Potter series. We treat the original text of three Harry Potter books as three separate unlearning tasks. Following recent works (Maini et al., 2024; Jia et al., 2024a), we use Llama-2-7B and Phi-1.5 as target models.
Metrics. Unlearning algorithms require evaluation from two perspectives: unlearning performance and model util- ity, both measured using various metrics. Metries for un- learning performance include: 1) Rouge-I. for unlearning (F- (F-Rouge) measures similarity between the text generated by the model and the correct text (Maini et al., 2024; Shi et al., 2024b). 2) Probability (F-Prob) evaluates the condi- tional probability (uestto) of model outputs. 3) Membership inference attack (MIA) calculates Min-K% Prob (Shi et al., 2024a) for unlearning samples to detect whether the post-unlearning model retains the unlearning set. 4) Forget Quality (FQ) quantifies the difference in prediction distributions on the forget set between the post- unlearning model and retrained model (Maini et al., 2024). Metrics for model utility include: 1) R-Rouge and 2) R- Prob are also applied to the retain set or holdout set to evaluate the model utility. 3) T-Rouge calculates the Rouge- L recall of completing sentences in the proposed TRAVIS dataset. Testing on inferred training data provides more accurate assessment of the impact of unlearning algorithms on the pre-existing knowledge of the model. Addition- ally, TRAVIS is generated without thematic constraints, resulting in a broad coverage of content, which enables a more comprehensive evaluation of performance. 4) Model
4.2. Experiments Privacy-Preservation Unlearning
In the TOFU dataset, unlearning algorithms are tested with personal information of fictitious authors, where each un- learning task corresponds to different authors. Table I presents the unlearning and utility performance of various methods across five continual tasks, showing that utility de- clines to varying degrees as tasks increase. A Rouge score of approximately 0.35 on the unlearning set is considered suffi- cient for effective unlearning (Maini et al., 2024). NPO+RT achieves reasonable unlearning performance in the first four tasks but suffers over-unlearning in the last, with model utility dropping below 0.2. Conversely, GA+RT struggles with insufficient unlearning in earlier tasks, followed by over-unlearning later. NPO, EUL, and WAGLE exhibit se- vere under-unlearning. with F-Rouge reductions of less than 15% in the first task compared to the original model. Al- though NPO and EUL achieve the highest utility in the third and fourth tasks, this stems from insufficient unlearning. GA+KL, NPO+KL, and SKU avoid over-unlearning but suffer progressively lower utility. In contrast, our method effectively balances unlearning and utility, achieving suf-
7

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Table 1. Unlearning performance (F-Rouge, FQ) and model utility (MU) on the TOFU dataset for the Phi model.
Task 1
Task 2
Task 3
Task 4
T
Methods
E-Rougel
FQ
MUF
F-Rouge]
FQf
MUF
F-Rougel
FQ
MU
F-Rouge]
FQT
MUt
F-Rouge_
FQ2
MU
Original
0.9614
0.5237
0.9723
7.4c-16
0.5237
0,9827
4.2c-15
0.5237
0.9895
7.5c-16
0.5237
0.9847
5.0c-17
0.5237
GA+RT
0.3723
7.9e-14
0.4766
0.4047
9.5c-13
0.4427
0.3524
1.3e-4
0.3494
0.3095
6.5e.5
0.2253
0.1027
9.7e-2
0.0812
GAeKI
D.3963
7.8c-4
0.3955
0.3896
1.5e-5
0,4270
0.3923
9.1e-3
0.3578
0,4311
5.2e-5
0.2898
0.3108
5.9c-4
0.2059
NPO
0.8430
199--1
0.4836
0.7322
4.6e-15
0.4675
0.6091
8.5e-9
0.4651
0.4893
9.4c-9
0.4257
0.4033
6.9e-6
0.3817
NPO:RT
D.3803
9.4c-3
0.4823
0.3931
7.6e-5
0,4470
0.3747
4.8c-4
0.3814
0.3524
6.8e-3
0.3087
0.3294
7.9c-3
0.1841
NPO+KL
D.4109
6.0e-
0.4729
0.4359
1.0e66
0,4465
0.4018
6.3c-4
0.3957
0.3891
9.96-6
0,3350
0.3651
8.5c-5
0.2789
DPO-RT
D.4510
8:9e-10
0.5043
0,41991
4.3e-8
0,4432
0.3983
4.6p-5
0.3602
0.3821
3.1e-4
0.3406
0.3771
3.9e-3
0.3359
TV
0.3957
4.8c-4
0.4825
0.3719
5.2c-3
0.4577
0.3573
8.9--3
0.3866
0.3371
9.8e-
0.3253
0. 0.9128
4.20-2
0.2963
SKU
0.4279
3.0e-5
0.4927
0.4055
5.8e-4
0,4100
0.3839
6.0c-4
0.3330
0.3805
6.5e-5
0.3024
0.3346
7.2c-3
0.2216
EUL
0.7885
2.4c-15
0.4839
0.3562
2.9c-12
0,4666
0.5264
7.2c-12
0.4551
0,5171
1.6c-13
0.4537
0.5058
8.0c-12
0.3938
SO-NPO
D.4525
7.36-8
0.4935
0,4316
6.2e-6
01,4832
04168
9.9e-7
0.3725
0,4153
5.0e-5
0.3472
0,4002
4.1
0.3001
WAGLE
0.7667
2,0c-15
D.4751
0,5031
4.8c-9
0,4669
0.4593
7.7c-6
0.4314
0.3860
2.5e-5
0,3400
0.5731
3.5c-4
0.3013
w/o ESL
0.3627
3.5c-2
0.5016
0.3320
3.7c-2
0.3067
7.3c-2
0.2684
L.f-1
0.3507
0.2341
5.7c-1
0.3351
Ous w/o DGS
0.3827
3.4e-3
0.5048
0,3642
L.6e-3
0.4539
0.3536
6.2e-3
0.4437
0.3297
4.0e-3
0.4296
0.3073
7.9e-1
0.3893
APM
0.3703
1.fc22
0.5128
0.3161
8.2c-3
0.1784
0.3320
4.7c-2
0.4521
0.3013
7.3o-2
0.1367
0.2517
2.5c-1
0.3768
Ours
0,3703
1,4c-2
0.5128
0,3671
2.0e-3
0,4855
0.3681
2.1e-3
0.4627
0,3452
1.3c-4
0.4578
0,3314
1.1e-2
0.4534
NPO-RT GA+RT
NPO-KL SO-NPO
WAGLE
Figure 4 illustrates the unlearning results for the Llama2 model on the TOFU dataset. The persistent issue of utility degradation is evident, as the data points of the same color representing the performance of successive tasks are gener- ally arranged sequentially from up to down, reflecting the cumulative effect of unlearning tasks on model utility, For the latter tasks, NPO+RT and GA+RT suffer drastic de- cline in model utility. While WAGLE exhibits poor unlearn- ing effectiveness with low changes of F-Rouge. Notably, the data points corresponding to our method are clustered more tightly across the five tasks, indicating a more stable and controlled utility decline in the continual unlearning scenario. Furthermore, these points are positioned nearer to the upright corner of the plot compared to other methods, signifying superior trade-off between effective unlearning and maintaining model utility. By achieving a high degree of unlearning with minimal impact on utility, our approach demonstrates clear advantages over baseline methods.
Ours
0.55
0.50
Model Utility
0.45
0.40
0.35
0.50
0.25
0.20
0.55
0.80
0.65
0,70
0.75
A F-Rouge
Figure 4. Model utility (MU) and changes F-Rouge of five un- learning tasks on TOFU dataset for the Llama2 model.
ficient unlearning while minimizing the decline in model utility. Notably, it not only achieves the best performance across all metrics on the first two tasks but also surpasses baseline methods from the perspective of multiple tasks. Af- ter completing all five unlearning tasks, our method exhibits less than a 0.06 reduction in utility, effectively mitigating the cumulative utility degradation typically observed in con- tinual unlearning scenario. This highlights its significantly lower impact on utility compared other approaches.
4.3, Experiments on Copyright-Protectin Unlearning
The WHP and MUSE News datasets are designed to evaluate the unlearning of verbatim memory of news articles and Harry Potter books by LLMSs We conducted experiments with the Llama2 model independently on each dataset and further explored the effects of alternating tasks between the two datasets. Please refer to Appendix H for more results.
Ablation study. We present the ablation study results in Table 1, illustrating the impact of each module on the overall performance of the model. When the entropic soft label (ESL) module is removed, the model exhibits significant cascading degradation, resulting in severe utility decline during the unlearning of later tasks. The dynamic gradient sparsity (DGS) module plays a critical role in identifying model parameters that balance utility retention and effective unlearning. Without the DGS module, the overall model utility decreases, and the cumulative utility degradation becomes more pronounced. Similarly, the absence of the adaptive parameter modulation (APM) module leads to a sustained decline in utility over the course of unlearning.
The results on the WHP dataset are shown in Table 2 and Figure 7 (b). Utilizing the TRAVIS dataset provides the evaluation of general utility wider than domain knowledge in retain dataset. In the WHP dataset, the sequential tasks involve unlearning data from several books in the Harry Pot- series. Due to the strong interconnections between the data across these tasks, unlcarning earlier tasks inevitably impacts subsequent tasks, leading to pronounced cascading degradation. For baseline methods, model utility declines largely after completing the unlcarning. In contrast, our method effectively mitigates the cascading degradation is-
8

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
that are crucial the knowledge representation of each task and tuning these parameters with adaptive intensity. The proposed method is built on the task vector framework with three novel modules applied in the fine-tuning process, dynamically localizing model parameters and adaptively re- fining gradients. Besides, we construct an evaluation corpus, TRAVIS, designed to test the overall utility of LLMs com- prehensively. The experimental results demonstrate that the proposed method effectively preserves the utility of LLMs while accomplishing sufficient unlearning, outperforming various baseline methods.
Table 2. Final model utility metries and average unlearning perfor- mance metrics over three tasks on the WHP dataset.
Unlearning
Utility
Methods
F-Rouge]
MIA
R-Rouget
T-Rouget
Original
0.9821
0,5206
0.6940
0.7643
NPO+RT
0.1559
0.2607
0.0
0.0
NPO+KL
0.1620
0.2452
0.0
0.0
GA+RT
0.1855
0.2674
0.1172
0.1975
SO-NPO
0,3080
0,3892
0.4109
0,4536
WAGLE
0.2946
0.3745
0.3721
0.4594
Ours
0.1790
0.2729
0.4586
0,5617
Acknowledgments
Table 3. Rouge Score Sensitivity to Gaussian Noise in Model Pa-
rameters.
This work is funded by the National Science and Technol- ogy Major Project (No. 2022ZD01 14903) and the Natural Science Fundation of China (NSFC. No. 62176132). Sen Cui would like to acknowledge the financial support re- ceived from Shuimu Tsinghua scholar program. Qizhou Wang and Bo Han were supported by the NSFC General Program No. 62376235, GDST Basic Research Fund Nos. 2022A151501 1652 and 2024A151501 was partially supported by Basic Research Grant (Super Al) of Institute for Al and Beyond of the University of Tokyo.
Noise Std(%)
TRAVIS
TOFU
WHP
Original
0.7643
0.8965
0.6940
0.1% %
0.7643
0.8965
0.6940
0.5%
0.7576
0.8965
0.6940
1.0%
0.7425
0.8965
0.6940
2.0%
0.7320
0.8958
0.6940
This research
sue, demonstrating improved preservation of model utility
in continual unlearning scenarios.
Impact Statement
4.4. Evaluation of TRAVIS Dataset Sensitivity
We propose a novel method for continual unlearning in large language models that is designated to protect personal pri- vacy and copyright. It can also be utilized to prevent harmful responses from LLMs. For similar reasons, the proposed method also does not involve storing training data. The constructed corpus TRAVIS is generated by the Phi model and the Llama2 model, only presenting general knowledge. It is manually filtered to guarantee no private or sensitive data is included.
The TRAVIS dataset consists of a wide variety of topics, enabling a comprehensive evaluation of the model's overall utility. As the dataset is generated by the LLM, it provides an accurate baseline for evaluating the model's original util- ity. To investigate the sensitivity of the TRAVIS dataset to variations in model performance, we conducted syn- thetic experiment by introducing small amount of random Gaussian noise to the model parameters to simulate destruc- tion of model utility. We compared the Rouge scores of the TRAVIS dataset against those of the TOFU and WHP datasets to assess their relative sensitivity to model utility changes. As presented in Table 3, the TRAVIS dataset is the most sensitive changes in model utility. Notably, the TRAVIS Rouge score exhibits decline when the noise stan- dard deviation reaches 0.5%, whereas the TOFU and WHP datasets remain largely unaffected until higher noise levels.
References
Blanco-Justicia, A., Jebreel, N., Manzanares-Sate B., Sánchez, D.,, Domingo-Ferrer J., Collell, G., and Tan, K. E. Digital forgetting in large language models: A survey of unlearning methods. CoRR, abs/2404.02062, 2024.
Brophy, J. and Lowd, D. Machine unlearning for random forests. In Proceedings of the 38th International Confer- ence on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 1092-1104. PMLR, 2021.
5. Conclusion
In this study, we explore the challenges of continual unlearn- ing in LLMs, which is practical and complicated setting. Specifically, the issues of accumulative decline and cascad- ing degradation exacerbate the utility deterioration problem for LLM unlearning methods. To mitigate these issues, we propose ALKN, a method that minimizes adjustments to model parameters by identifying the model parameters
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert- Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu,

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Advances in Neural In- formation Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Han, B., Niu, G., Yu, X., Yao, Q. Xu, M., Tsang, I. W., and Sugiyama, M. SIGUA: forgetting may make learning with noisy labels more robust. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 4006- 4016. PMLR, 2020.
Cao, Y. and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Se- curity and Privacy, SP 2015, San Jose, CA, USA, May 17-21, 2015, pp. 463-480. IEEE Computer Society, 2015.
Hong, Y. Yu, L., Ravfogel, S., Yang. H., and Geva, M. In- trinsic evaluation of unlearning using parametric knowl- edge traces. CoRR, abs/2406. 11614, 2024a.
Hong, Y., Zou, Y., Hu, L.., Zeng, Z., Wang, D., and Yang, H. Dissecting fine-tuning unlearning in large language mod- els. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FI, USA, November 12-16, 2024, pp. 3933-3941, Association for Computational Linguistics, 2024b.
Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert- Voss, A., Lee, K., Roberts, A., Brown, T. B., Song, D., Erlingsson, Ú., Oprea, A., and Raffel, C. Extracting train- ing data from large language models. In 30th USENIX Security Symposium, USENIX Security 2021, August 11- 13, 2021, pp. 2633-2650. USENIX Association, 2021.
Ilharco, G., Ribeiro, M. T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
Chen, J. and Yang, D. Unlearn what you want forget: Efficient unlcarning for Ilms. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pp. 12041-12052. Association for Computational Linguistics, 2023.
Izzo, Z., Smart, M. A., Chaudhuri, K., and Zou, J. Approxi- mate data deletion from machine learning models. In The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, volume 130 of Proceedings of Machine Learning Research, pp. 2008 2016. PMLR, 2021.
Eldan, R. and Russinovich, M. Who's harry potter? ap- proximate unlearning in 1lms. CoRR, abs/2310.02238, 2023.
Golatkar, A., Achille, A., and Soatto, S. Forgetting out- side the box: Scrubbing deep networks of information accessible from input-output observations. In Computer Vision - ECCV 2020 16th European Conference, Glas- gow, UK, August 23-28, 2020, Proceedings, Part XXIX, volume 12374 of Lecture Notes in Computer Science, pp. 383-398. Springer, 2020.
Ji, J., Liu, Y., Zhang, Y., Liu, G., Kompella, R. R., Liu, S., and Chang, S. Reversing the forget-retain objectives: An efficient LLM unlearning framework from logit differ- ence. CoRR, abs/2406.08607, 2024.
Jia, J., Liu, J., Zhang, Y., Ram, P., Baracaldo, N., and Liu, S. WAGLE: strategic weight attribution for effective and modular unlearning in large language models. CoRR, abs/2410.17509, 2024a.
Golatkar, A., Achille, A., Ravichandran, A., Polito, M., and Soatto. S. Mixed-privacy forgetting in deep networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021. pp. 701- Computer Vision Foundation/ / IEEE, 2021.
Jia, J., Zhang, Y., Zhang, Y., Liu, J., Runwal, B., Diffend- erfer, J., Kailkhura, B., and Liu, S. SOUL: unlocking the power of second-order optimization for LLM unleam- ing. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pp. 4276-4297 Association for Computational Linguistics, 2024b.
Graves, L.,, Nagisetty, V., and Ganesh, V. Amnesiac machine learning. In Thirty-Fifhh AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium Educational Advances in Ar- tificial Intelligence, FAAI 1021, Virtual Event, February 2-9, 2021, pp. 1511-11524. AAAI Press, 2021.
Koh, P. W. and Liang, P. Understanding black-box predic- tions via influence functions. In International conference on machine learning, pp. 1885-1894. PMLR, 2017.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. Certified data removal from machine learning models. arXiv preprint arXiv: /9/ /. 103030, 2019.
10

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Xu, X., Yao, Y., Li, H., Varshney, K. R., Bansal, M., Koyejo, S., and Liu, Y. Rethinking machine unlearn- ing for large language models. CoRR, abs/2402.08787,, 2024a.
Thaker, P., Hu, S., Kale, N., Maurya, Y., Wu, Z. S., and Smith, V. Position: LLM unlearning benchmarks weak measures of progress. CoRR, abs/2410.02879, 2024.
Ullah, E., Mai, T., Rao, A., Rossi, R. A., and Arora, R. Ma- chine unlearning via algorithmic stability. In Conference Learning Theory, COLT 2021, 15-19 August 2021, Boulder, Colorado, USA, volume 134 of Proceedings of Machine Learning Research, pp. 4126-4142. PMLR, 2021.
Liu, Z., Dou, G., Tan, Z., Tian, Y., and Jiang, M. Towards safer large language models through machine unlearn- ing. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pp. 1817-1829. Association for Computational Linguisties, 2024b.
von Oswald, J., Zhao, D., Kobayashi, S., Schug, S., Caccia, M., Zucchet, N., and Sacramento, J. Learning where to learn: Gradient sparsity in meta and continual learning. Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Process- ing Systems 2021, NeurlPS 2021. December 6- 14, 2021, virtual, pp. 5250-5263, 2021.
Lynch, A., Guo, P., Ewart, A., Casper, S., and Hadfield- Menell, D. Eight methods to evaluate robust unlearning in Ilms. CoRR, abs/2402. 2024.
Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C.,, and Kolter, J. Z. TOFU: A task of fictitious unlearning for Ilms. CoRR, abs/2401.06121, 2024.
Wang, Q., Han, B., Yang. P., Zhu, J., Liu, T., and Sugiyama, M. Unlearning with control: Assessing real-world utility for large language model unlearning. CoRR, abs/2406.099 09709, 2024a.
Niu, G., du Plessis, M. C., Sakai, T., Ma, Y., and Sugiyama, M. Theoretical comparisons of positive-unlabeled learn- ing against positive-negative learning. In Advances in Neural Information Processing Systems 29: Annual Con- ference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 1199-1207, 2016.
Wang, Q. Lin, Y., Chen, Y., Schmidt, L., Han, B., and Zhang, T. A sober look at the robustness of clips to spurious features. In Advances in Neural Information Processing Systems, 2024b.
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M.,
Wang, Q. Han, B., Yang, P., Zhu, J., Liu, T., and Sugiyama, M. Towards effective evaluations and comparison for IIm unlearning methods. In International Conference on Learning Representations, 2025a.
C., Grantforrr, A., Xiong, W., Défossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N., Scialom, T., and Synnaeve, G. Code Ilama: Open foundation models for code. CoRR, abs/2308. 12950, 2023.
Wang. Q. Zhou, J. P., Zhou, Z., Shin, S., Han, B., and Weinberger, K. Q. Rethinking Ilm unlearning objectives: A gradient perspective and go beyond. In International Conference on Learning Representations, 2025b.
Schmidt, M. Cpse 540: Machine learning. Convergence of Gradient Descent, 2017.
Scholten, Y., Günnemann, S., and Schwinn, L. A proba- bilistic perspective on unlearning and alignment for large language models. CoRR, abs/2410.03523,, 2024.
Wang, Y., Wang, Q., Liu, F., Huang, W., Du, Y., Da, X., and Han, B. Gru: Mitigating the trade-off between unlearning and retention for large language models. In International Conference on Machine Learning, 2025c.
Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L. Detecting pretraining data from large language models. In The Twelfth Inter- national Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024a.
Wu, R., Yadav, C., Salakhutdinov, R., and Chaudhuri, K. Evaluating deep unlearning in large language models. CoRR, abs/2410. 15153, 2024.
Wu, Y., Dobriban, E., and Davidson, S. B. Deltagrad: Rapid retraining of machine learning models. In Proceedings of the 37th International Conference on Machine Learn- ing. ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 10355-10366 PMLR, 2020.
Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtzman, A., Liu, D., Zettlemoyer, L.., Smith, N. A., and Zhang, C. MUSE: machine unlearning six-way evaluation for language models. CoRR, abs/2407. 064600 2024b.

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Yang, P., Wang, Q. Huang, Z., Liu, T., Zhang, C., and Han, B. Exploring criteria of loss reweighting enhance Ilm unlearning. In International Conference Machine Learning, 2025.
Yao, Y., Xu, X., and Liu, Y. Large language model unlearn- ing. CoRR, abs/2310. 10683, 2023.
Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting, In 31st IEEE Computer Security Foundations Symposium, CSF 2018, Oxford, United Kingdom, July 9-12, 2018, pp. 268 282. IEEE Computer Society, 2018.
Yuan, X., Pang. T., Du, C., Chen, K., Zhang, W., and Lin, M. A closer look machine unlearning for large language models. CoRR, abs/2410.081100 2024.
Zhang, H., Nakamura, T., Isohara, T., and Sakurai, K. A review on machine unlearning. SN Comput. Sci., 4(4): 337. 2023.
Zhang. R., Lin, L., Bai, Y., and Mei. S. Negative preference optimization: From catastrophic collapse to effective un- learning. CoRR, Abs/2404.05866. 2024a.
Zhang, Z., Zhang, J., Yao, H., Niu, G., and Sugiyama, M. On unsupervised prompt learning for classification with black-box language models. CoRR, abs/2410.03124, 2024b.
Zhao, W. X., Zhou, K., J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C.,
Chen, Y., Chen, Z., Jiang. J., Ren, R., Li, Y., Tang, X.,
Liu, Z., Liu, P., Nic, J., and Wen, J. A survey of large language models. CoRR, abs/2303.18222 2023.
12

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
A. Related Work
Machine unlearning studies the problem of efficiently removing specific data from machine learning models, ensuring compliance with privacy regulations like GDPR (Cao & Yang. 2015; Wang et al., 2025c; Zhang et al., 2023; Niu et al., 2016; Zhang et al., 2024b). Machine unlearning methods are generally categorized into exact unlearning (Ullah et al., 2021; Brophy & Lowd, 2021) and approximate unlearning. The latter modifies the model or dataset simulate the removal effect without full retraining (Wu et al., 2020; Golatkar et al., 2020; 2021; Graves et al., 2021). There is line of work that employs Newton's 's method to remove the influence of target data involving calculating second-order derivatives of model parameters (Koh & Liang, 2017; Guo et al., 2019; Izzo et al., 2021). However, the approach is computationally prohibitive for LLMs. In addition, precisely evaluating LLM unlearning algorithmss, as is commonly done in the field of machine unlearning, is often impractical because obtaining a fully retrained model for comparison is rarely feasible (Scholten et al., 2024).
Among the studies LLM unlearning, some focus on establishing evaluation metric (Wang et al., 2024a; 2025b; Yang et al., 2025; Lynch et al., 2024; Yuan et al., 2024; Wu et al., 2024), while others emphasize the design of effective algorithms. With trivial unlearning methods, the process often leads to excessive unlearning, where the performance of the model on non-target data is unintentionally degraded. The primary objective of LLM unlearning is to preserve the model utility on normal data while effectively unlearning the target information. NPO is proposed to mitigate the issue of unbounded optimization loss of GA (Zhang et al., 2024a). Eldan & Russinovich (2023) suggest replacing sensitive keywords with generic terms, enabling the model to produce generic predictions through reinforcement bootstrapping. The constructed generic outputs enable unlearning by fine-tuning the LLM model on them. Ji et al. (2024) propose ULD, which reverses the optimization direction during the training of an assistant model. SKU incorporates an object function that embeds robustness and utility retention into task vector training (Liu et al., 2024b), offering a more efficient approach to unlearning. WAGLE seeks to locate the specific LLM parameters that are most critical for both unlearning and retaining objectives and fine-tuning exclusively on these parameters (Jia et al., 2024a). Several studies have also examined the impact of unlearning techniques on various modules within transformer models (Hong et al., 2024b;a). Despite the substantial body of research on LLM unlearming, the practical challenge of continual unlearning remains underexplored and warrants further investigation.
B. Convergence of GA and the task vector method
To compare the convergence of GA and the task vector method, we consider a standard binary classification problem with training data [(x; ) where x E R4 and y E {0, A logistic model is used to predict the probabilities (y- = 1|x, 0) = (2x, 0)), where represents the sigmoid function (Schmidt, 2017).
B.1. Convergence of the task vector method
Theorem B.1. Using the task vector algorithm with a learning rate < 1) < 7, to optimize the aforementioned logistic regression problem can guarantee convergence. Let 0t denotes the parameters after the t-th iteration of optimizing. The loss function frv(@') strictly decreases:
fiv(0'+)) < frv(O"), t t2 0,
(11)
and converges to its optimal value:
lim fiv(0')= = frv(0*) t
(12)
Proof.
The fine-tur fine-tuning process of the task vector method utilizes the gradient descent algorithm with the following loss function:
fTv(0') = --E yi log ((x, 0²)) - (1 - 31)) (ogo((((z00))
(13)
The function frv is L-smooth. Thus the following inequality holds:
frv( S frv(0') + (Tv) 0²)+ I,
110tt118000
(14)
Substituting g++ -0' -ivV yields:
[iv(0++1) < STv(0t) -n(111111 (-LL)
Lu
/ [TT(()))11
(15)
2
13

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
With 0 < z. get strict decrease in fIv(0"):
frv frv(O frv(0")
(16)
With lower bound (0v 2 O, the loss function converges to its optimal value:
1 lim frv(d") = frv((")
(17)
B.2. Non-convergence the GA method
Theorem B.2. Optimizing the aforementioned logistic regression problem using the GA algorithm will result in the norm of the gradient remaining greater than certain constant:
W Sax (0')) 2 VC,
(18)
where 6e denotes the parameters after the t-th iteration of optimizing and C is a positive constant only depending on the training data {(14,1)) y:))== and initial parameters 6 And the objective function SoA (0') strictly decreasing to negative infinity:
fCal f0a(0") )P)) tnC, lim SCa (0a) -00.
(19)
Proof.
The loss function of GA is:
SGa(0') Σ yi log (1x (1 y.) (=g 0"))
(20)
The gradient with respect the parameters 0º is:
V lGa(0')) = Σ (2y: 1)(1 - -:),,,;
(21)
where म.. = ( 0)) when = 1,
o 0)) when y: = O.
We define mf = (2y, - 1)(1 - m(); Yh. = (a,, k,), and % denotes (7h,1, 76,2; -.. R.n) Besides, we define of = (m", (m)
Inspired by Zhang et (2024a), we use the induction method to proceed.
Case 1: L 0. Since |,:1122 ||0%||2 is finite value, we can derive:
C S m < when 34 1,
(22)
-1 m < C2 when y4 0,
where Oj and C22 are two positive constants. We assume there exists constant Cp such that maX has1 S Co satisfying:
and
I n..lm
I
(23)
2
We obtain:
d. (m² (m) 2
0 when Y/ 1,
(24)
|mal 7i.
df (m", 7x)
0 when Yi 0.
We set C3- = min(C1, C2) and C₁ Catis 2
then we have:
0 = (m², m) 2 C4 > when y = 1,
(25)
df = (m (m)))) < -Ca < 0 when yi 0.
14

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Then derive m.:
mi (2y: - 1) n.) (2y: 1) ((12): () 0')).
(26)
Substracting 0¹ -0--17 Vn (00) 6 E; m. we obtain:
m (2yz 1) (1 ((2 1)(2,, 0²) (2y; 1)d:))
(27)
Comparing it with m (2y; - 1) o ((2 (z, ))) we can derive:
m 2 m when Vi
(28)
m I m when y; O.
Case == K > O. We suppose:
1. 2 C4 > 0 when yi = 1,
(29)
df - -C < 0 when = 0,
for 0 E t S K. And we suppose:
m ++1 2 m 2 C₁ when y = 1
(30)
m S m C2 when y; 0,
for 0 s K
With this condition, we can derive the same conclusion for q: K+1 in Case
q; 2 C4 > 0 when yi 1,
K+1
(31)
K+1 < - < 0 when O, 9:
and the same conclusion for m K+1,
K+1 2 m 2 C₁ when ys m;
(32)
K+1 S m < C2 when yi 0.
m
From above, prove that lo'l C4 and (2y, 1)m > (2 1)m! for all 2 We can further derive that:
fex(t S (f*
(33)
and
| gt fax (f*)llxxx
= Val (8) oV - (
Cavn.
(34)
By incorporating the condition
| fen (0")lxxxX S Vmnaa V fas (0') we can obtain:
Civ = VC,
o fox
(35)
Vmmax
where C' a positive constant only depending on the training data ((x,, y:) and initial parameters 8
Furthermore, since (0) concave function, it satisfies the following inequality:
fox(0 < fea( + Voi ( - 8²) ffa((0)) nll Vo (0')
(36)
then (0 < SCa (0') - -,C', and we can obtain:
lim Sax fax(0') = lim Sax (09) InjC = -00
(37)
t00 t-000
15

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
C. Proof of Proposition 2.1
As defined for the GA, the loss function and its gradient are given as follows:
foA(0) = Σ Yi log of 0)) + (1 - yo) log (- 0))
(38)
(of Σ (2 1)(1 TT. Tz
where x, follows the definition in Appendix B.2, as do qi m, and Y.j. Independently training on D
In this scenario, the model is directly trained on D starting from the initial parameter 6 for T iterations with learning rate resulting in the updated parameters On..
0 Oiul - 11 Σ i",
(39)
(olL.. xt) 169, x4)) - Σ AUL
As in Appendix B.2, we can prove that:
hun, 2 C4 > 0 when y: nun,i S C1 < 0 when y; 0,
(40)
Continuously training on Df and D
In this scenario, the model is first trained on D starting from 80, yielding parameters 8y:
g5+1 0 17 Σ m
(41)
The training on D affects the model predictions on D"
(0)+1 xI) = 10), σه) - - Σ migllss,,
(42)
Because max (af xf}1
b(x/,.)) we assume there exists constant Co such that max Ka/f xt\) S Co. and
satisfying:
(of xf(.)))lmt
Σ
m mayl =;>1| 5
(43)
Therefore, we can obtain:
(09+)) .x²) < (07,2,)
when y/s 1,
(44)
(xf mmyllm
(0) xxi) > 10, x3¹) +
when y/1 = 0.
2
Thus the effect of training D demonstrates
(0,, x;) < (8 x C.T when y; = 1
(45)
(0,, x;) > 10 x + C.T when y: = 0.
16

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Then the model starts training D with initial parameters being 0f, yielding updated parameters OCuL:
CC.. OCL - 11 Σ mc
(46)
( 2;)) (0g, - Σ Ecuu..
Since le. = 0, the inequality Equation 45, we can derive:
ACUL, MUUL. + C2 when yi = 1,
(47)
0 0 VULL, C2 when y; O.
ACUL,
Using the induction method in Appendix B.2, prove that:
ØCuL, HULL + C2 when y; 1,
(48)
aCuL, < AuuL. - C2 when yi = 0.
Therefore, VEu, > vUL.s + TC2.
Finally, we derive the conclusion:
||0cul - 0yllxTXX = 11 dcuu ll2 > 11 +1 &cuu. ll + +TVV
(49)
therefore,
||0cul 0,llxTxxxxx > ||0 - - + TCVT.
(50)
In this proposition, we consider a two-task scenario for simplicity. When there are multiple tasks, we can also similarly derive such conclusions:
Corollary C.1. For multiple tasks, the conclusion of Proposition 2.1 holds. Suppose there are T preceding datasets (D')T=, and a current dataset D If one of the preceding datasets is correlated with D*, satisfying: maxi/) (f),, < k(x,, then the previous unlearning results in larger changes of parameters in the unlearning of D*:
A fcull x > 2 x + TC
(51)
D. Derivation of Equation 9
According to the chain rule, the gradient of Equation 8 with respect to m' is given by:
-X of.k (c - k- Dr)V gt.++ + HMV ml 1''2n
(52)
For computational efficiency, we approximate the gradient of LeE on D, with V To=ILo (r)). And Vm O gr.k+1 is derived by substituting into Equation 7:
V gl.kt -a¹t ( D o Vm 1m
(53)
For simplicity, we omit a' and approximate Vm 2m using straight- teahhttthruughh estimation, which consists in taking this derivative equal to the identity (von Oswald et al., 2021). Then we obtain the gradient of Equation 8 with respect to m
-XE (0 Dt) Vor- LCE(O-1 1,E( + uM
(54)
The updating of m is given by substracting the gradient:
ml +- m + >LEEs ( D..) @ Vo. 1C (D,) ) pM
(55)
17

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
E. Implementation details
E.1. Optimization
The Adam optimizer is employed for the training of all methods. We employ the learning rate of 3e-5 for the TOFU dataset and 3e-6 for the MUSE News and WHP datasets. Most baselines easily suffer from the issue of over-unlearning while conducting second or third tasks with constant learning rate. Thus, we apply learning rate tuning, decaying the learning rate by 50% after each unlearning task. For experiments in Figure 2 and Figure 8 which require alignment of unlearning performance, learning rates of different methods are tuned to achieve aimed unlearning performance. The hyperparameters and \ of our method are set as 10 and 0.8. ps is set as the deviation of the unlearning gradients G., m² is initialized with zero vectors in the beginning of each training.
Weight decay ratio is set as 0.01 for all experiments. For TOFU dataset, batch size is set as 32 and models are trained for 5 epochs. For MUSE News dataset, batch size is 6 and number of epochs is 2. For WHP dataset, batch size and epoch number are 6 and 10.
E.2. Tuning of gradient mask threshold 17
The threshold 1y determines the proportion of model parameters that are activated. The threshold rj is determined by the percentiles of the underlying vector m', and it increases gradually throughout the training process. Specifically, 100% parameters are activated in the beginning of each training, while 30% parameters are activated in the end. With this approach, parameters with smaller vr' values undergo minor fine-tuning in the early stages of training, while parameters with larger m' values receive more extensive fine-tuning over additional steps. Compared to setting large threshold in the beginning of training, this method balances the training of all model parameters, preventing excessive adjustments to a small subset of activated parameters to achieve the unlearning objective.
Calculating the percentiles of m² can be computationally expensive for models with large number of parameters since it involves sorting the parameters. To address this, we employ an efficient estimation method. Calculating the maximum and minimum values of a list has computational complexity of (n)) By identifying the top 15% largest and smallest values, we can mitigate the influence of outliers. The estimated quantiles are then obtained through interpolation between these extreme values after excluding outliers.
E.3. Discussions regarding parameters masking in LLMs unlearning
We argue that an efficient way to preserve the utility of LLM during the unlearning process is to minimize changes to its parameters. In contrast to methods that directly calculate the mask using gradient magnitude, our proposed dynamic gradient sparsity method in Section 3.2 learns the mask values from the data. And the mask is dynamically tuned along with the training process. In this way, each model parameter receives an appropriate training range: parameters with a relatively small impact on the training objective undergo minor adjustments in the early stages, after which they are masked, while parameters with larger impact are trained more extensively. This approach allows for appropriate training of the model as whole, avoiding large adjustments to all parameters or drastic changes to a small subset. Besides, the proposed method leverages the underlying vector m to accumulate the influence of data on the mask, thereby avoiding biases introduced by extreme values during single training iteration.
E.4. Improving Memory Efficiency
Although effective, the iterative approach in Equation 9 requires storing G, Vol:LCcr(0-1,)) D,) during the fine-tuning process of the f-th task. To reduce memory usage, we propose an alternative method.
To reduce memory overhead during training, Gp can be replaced with a lower-precision approximation, G... For elements with absolute values exceeding a certain threshold, which have a significant impact on the retain set, their signs +1 or -1 are preserved, while the remaining elements are set O. During the iteration of m'. calculation of G, @ G, is estimated with:
G2 o (G, == 0) + a G. C (G, == 1) a a G. o (G. == -1),
(56)
where a is a large positive constant. This method preserves both the signs and magnitude information of G, while significantly reducing memory consumption.
18

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
F. Experimental Setup
F.1 Dataset
TOFU (Maini et al., 2024) dataset consists of information about 200 fictional authors presented in a qqestion-ansen pair format. Specific authors' information was designated as unlearning data, while the remaining authors' data were retained to assist in training and testing. Besides, there are also question-answer pairs about Real Authors and World Facts. We grouped the information of four authors into one unlearning task, resulting in five or more continual unlearning tasks.
MUSE News (Shi et al., 2024b) comprises BBC news articles published after August 2023. This dataset includes four predefined unlearning subsets that serve as continual unlearning tasks. Additionally, it contains retain set that does not overlap with the unlearning sets.
WHP (Who's Harry Potter) (Eldan & Russinovich, 2023; Shi et al., 2024b) involves the unlearning of original data from the Harry Potter series. In contrast, retain data consists of knowledge related to the books. which aligns with real-world requirements for copyright protection. We treated the original texts of three Harry Potter books three separate unlearning tasks. To evaluate the performance of algorithms in scenarios involving unrelated continual tasks, we constructed sequence of continual unlearning tasks by interleaving multiple unlearning tasks from the MUSE News and WHP datasets.
F2. Metries
Rouge-L for unlearning (F-Rouge) measures the similarity between the text generated by the model and the correct text. Specifically, given a question or the beginning of the unlearning text, the Rouge-L recall is computed by comparing the output of models with the correct answer or remaining text. A lower Rouge-L value indicates better unlearning performance (Maini et al., 2024; Shi et al., 2024b).
Probability (F-Prob) evaluates the conditional probability P(answerlquestion) of the model outputs the TOFU dataset. To ensure comparability, this probability normalized by the length of the output text (Maini et al., 2024).
Membership inference attack (MIA) uses the sate-of-thheart Min-K% Prob method for LLMs (Shi al., 2024a) to detect whether data in the unlearning set remains part of the training data for the post-unlearnnng model. A lower probability of the unlcarning set belonging to the training data reflects a more effective unlearning process.
Forget Quality (FQ) quantifies the difference in prediction distributions on the forget set between the post-unlearning model and a retrained model (fine-tuned on retrained data without exposure to the forget data). This metric assesses the degree to which the unlearning algorithm eliminates traces of the forget set in the model (Maini al., 2024).
R-Rouge and R-Prob are also applied to the retain set holdout set to evaluate the impact of the unlearning algorithm on the overall model utility. Higher values for these metrics indicate less performance degradation.
T-Rouge calculates the Rouge-L recall of completing sentences in the proposed TRAVIS dataset. Testing on inferred training data provides a more accurate assessment of the impact of unlearning algorithms on the pre-existing knowledge of the model. Additionally, TRAVIS is generated without thematic constraints, resulting in a broad coverage of content, which enables a more comprehensive evaluation of performance.
Model Utility (MU) represents the harmonic mean of the above metries calculated across multiple retain sets in the TOFU dataset (Maini et al., 2024), including TRAVIS dataset. It provides aggregate measure of the model utility.
F.3. Baseline methods
GA (Yao et al., 2023) unlearns target data by optimizing in the opposite direction of gradient descent. This is achieved by minimizing the negative cross-entropy loss of predicting the next token the unlcarning dataset:
min
Σ Σ DED.
log mo( (14)))
(57)
Thu
NPO (Zhang et al., 2024a) is also a gradient ascent-bace based approach. Compared to GA, the gradient in NPO is scaled by a regularization term that gradually diminishes over the course of training, alleviating the problem of over-unlearning to some
19

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
extent. The loss objective of NPO is:
Σ xo(o ) )"1
2
log
+
(58)
B
Σ Teat (xp ()ll)))))))))))))))))))
Task Vectors (TV) (Ilharco et al., 2023) method manipulates the ability of pre-trained model to address downstream tasks through fine-tuning and the arithmetic operation of model parameters. When applied to LLM unlearning, the method enables the selective removal of knowledge related to specific dataset D.. Specifically, a pre-trained model Ty is fine-tuned on the unlcarning dataset D.. resulting in model Te, with enhanced knowledge of D.. The difference between the fine-tuned parameters and the original parameters, 6n - 8, constitutes the task vector for D.. By subtracting this task vector from the original parameters 0, a modified model is obtained that has unlearned the knowledge from D.-
DPO is a preference optimization approach. For preference dataset, DPO encourages the model to produce one output over the other between a pair. In the unlearning scenario, it encourages the model to produce responses yo like "I don't t know" instead of generating original sensitive text y: (Zhang al., 2024a).
SKU (Liu et al., 2024b) is a method based on task vectors. In the fine-tuning process, SKU not only reinforces the target knowledge into the model, but also eliminates knowledge of retain set from the model, incorporating a utility preservation objective. Besides, includes a robustness objective during the fine-tuning process.
EUL incorporates lightweight unlearning layers into the model to enable efficient unlearning. This method explicitly considers scenarios involving multiple unlearning tasks (Chen & Yang, 2023).
SO-PO and SO-NPO introduce second-order optimization framework for LLM unlearning, leveraging Hessian matrix approximations to address the challenges of applying Newton's 's method to LLMs (Jia et al., 2024b).
WAGLE identifies model parameters that benefit both unlearning and retaining objectives before training begins, optimizing only these parameters during the unlearning process. This approach is utilized to minimize the impact of unlearning on model utility (Jia al., 2024a).
When applying these methods in continual unlearning scenarios, the learning rate is tuned progressively to mitigate catastrophic collapse tasks continue.
G. TRAVIS Dataset
To construct a high-quality text corpus, we adopted the methodology of Membership Inference Attack (MIA) (Yeom et al., 2018) and applied it to two pre-trained language models, Phi-1.5 and Llama2-7b. The approach began with the generation of 5,000 sentences using the models. Subsequently, the MIA technique was employed to identify 250 candidate sentences with the highest probabilities from the generated set (Carlini et al., 2021). After deduplication and filtering based on sentence similarity, the candidate set was further refined. More than 1550 high-quality sentences were manually selected to ensure both diversity and authenticity in the dataset.
To enhance the variety of sentence prefixes and enable the models to process a broader range of contextual information, we incorporated large collection of prefix data sourced from the internet following the work of Carlini et al. (2021), Specifically, we extracted samples from a subset of the Common Crawl dataset, cleaned the data by removing HTML tags and JavaScript code, and performed deduplication, resulting in approximately 50MB of clean text data. From this corpus, randomly selected 5 to 10 context tokens and employed Top-n sampling strategy to generate sequences based on the model's 's probability distribution. The sequences with the highest likelihood were selected, and sentence lengths were capped at a maximum of 1024 tokens to ensure relevance and appropriate length.
The MIA methodology was applied next, using the perplexity of each sentence as a measure of training data likelihood. Perplexity was calculated with the following formula:
log fo 2i....., ........)))
P exp
(59)
where æi...., Fn denotes a sequence of tokens. Low perplexity indicates that the model holds high confidence in completing the sequence.
20

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
However, relying solety on MIA for membership inference introduces potential risks, as the model might assign higher likelihoods unremarkable or low-quality samples, such repeated substrings. To address this issue, we introduced several quality control mechanisms to ensure the integrity of the generated text. Specifically, we calculated the zlib entropy of each sample, measured perplexity within a sliding window, and computed the ratio of perplexity before and after lowercasing the text to assess its complexity and information density. These metrics allowed us to filter out low-quality samples and ensure that the selected texts exhibited a high degree of linguistic fluency and semantic richness.
During the selection process, samples were ranked according to the three quality metries, and the top 100 were chosen, After deduplication, Sentence-BERT was used to compute cosine similarity between sentences, and those with similarity scores above 0.7 were excluded. Cosine similarity was calculated by:
V1 V2
(vs v2)
(60)
v2ll
where V1, V2 denote the embeddings of two sentences. This process yielded approximately 250 diverse and high quality sentences. In the final manual filtering step, short, highly repetitive, logically incoherent sentences were discarded, while longer substrings with richer semantics and more varied content were retained. The resulting dataset contains more than 150 high-quality sentences per model, meeting our established quality standards. For dataset construction, the first 70% of each sentence was designated as the prompt, while the remaining 30% served as the ground truth, thus creating the final training dataset.
This dataset offers two significant advantages. First, it accurately evaluates the impact of unlearning algorithms on the original knowledge retained by models. This aligns closely with the definition of efficiency the context of machine unlearning. Second, the broad thematic coverage of the dataset enables evaluations of model performance across diverse contexts, ranging from everyday conversations to specialized topics. This diversity not only allows the dataset to comprehensively assess the language generation capabilities of the model, but also provides insights into its generalization ability when exposed to different background knowledge. Consequently, the dataset serves as a versatile and reliable foundation for evaluating model performance in a wide range of scenarios.
Table 4. Parts of examples in TRAVIS dataset (Phi Model)
Phi Model
The Concept and Examples Recursion is a powerful programming concept that involves a function calling itself in its definition. This allows the program to be written in a more concise and elegant way. and provides a elegant solution to problems that would otherwise require multiple lines of code. For example: def factorial(n): if n == O: return I return factial( n-1 print( factroy( 10) Prints: 10 * 9 * 7 * 6 * 5 * 4 * 3 # * 3628800
Bullying is pervasive problem that affects many people, especially children. can have scrious long-term effects on mental health, self-esteem, and academic performance.
2
Financial independence is a goal that many individuals aspire to achieve, but the path to attaining it can be challenging. One of the most effcctive ways to build financial independence carly on is through saving.
3
As the world continues to grapple with the effects of climate change, many are turning to creative solutions to help mitigate its impact. From art installations that draw attention environmental issues to community based initiatives that promote sustainability, individuals and organizations alike are stepping up to the challenge.
4
5 Reasons do exereise regularly: 1.To maintain a healthy weight: Regular exereise helps to burn excess calories and prevent weight gain.
H. More experimental results
H.1. Mixed dataset.
We argue that the higher the data relevance between unlearning tasks, the more severe the cascading degradation phenomenon. To evaluate the performance of method in scenarios with low data relevance in continual unlearning, we conduct experiments mixing tasks from the MUSE News and WHP datasets. In these experiments, the model is sequentially tasked to unlearn data from the two datasets, and the results are shown in Figure 5. After completing the first MUSE News task, the model proceeds to unlearn the second WHP task, during which all methods exhibit a slight decline in model utility. The decline less severe compared to unlearning consecutive WHP tasks. Similarly, our method maintains stable model utility when unlearning unrelated tasks sequentially. When moving to the third another MUSE News baseline
21

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Table Parts of examples in TRAVIS dataset (I.lama Model)
Llama Model
The city of Los Angeles also offers wide range of public services, including healthcare, education, and transportation, all which are critical maintaining high quality of life for city residents.
There are few things you can to make sure you get the best possible experience from your interactions with the Russian culture and people. Learn as much as you Can:
2
So, if you want to use the power of positive thinking to improve your life, start small and build up gradually. Believe in yourself and your abilities, and don't worry about criticism as it's integral part of growth.
3
A New York City-based non- non-profit has been instrumental in fostering greater transparence in the city's 's supplementary food system. This has involved working with foodstores, restaurants, and community organizations promote greater understanding and access to healthier foods.
4
The global COVID-19 pandemic has significantly influenced the worldwide popularity online platforms and digital services.
5
NPO-RT
NPO+KI
WAGLE
Original
0.7
0.6
0.5
T-Rouge
0.1
0.3
0.-
0.1
Task
Task
Task
Task
Task
Task
Task
Figure 5. T-Rouge across seven continual unlcarning tasks on the mixed dataset of MUSE News and WHP. Different background color represents tasks from different datasets.
methods experienced significant performance degradation due to the cascading degradation effect initiated by the first task. In contrast, our method demonstrates its robustness by effectively mitigating this phenomenon and maintaining model utility.
H.2. Experiments on the TOFU dataset
We present maximum and minimum model utility across 5 unlearning tasks in Figure 6. And the maximum F-Rouge is shown in Table 6, illustrating the performance of the least forgotten task after unlearning. Our proposed method achieves the lowest F-Rouge, outperforming baseline methods in terms of unlearning performance. Figure compares the ability of retaining model utility among different methods. For Max MU, the proposed method attains the highest value (0.5574), followed by WAGLE (0.5486) and SO-NPO (0.5427). Similarly, for Min MU, the proposed method leads with score of 0.5082, exceeding WAGLE (0.4721) and SO-NPO (0.4503), demonstrating superior retention of utility after unlearning all tasks. These results validate the proposed approach's 's ability to balance effective unlearning and high model utility.
H.3. Experiments on the MUSE News dataset
MUSE News. Figure 7 (a) illustrates the utility and unlearning results the MUSE News dataset, which exhibits more severe over-unlearning compared to the TOFU dataset. Specifically, for baseline methods like NPO+RT and GA+RT, the utility drops to zero during the third or fourth unlearning task. This can be attributed to the nature of the dataset, which requires verbatim unlearning of textual data rather than conceptual unlearning. Verbatim unlearning poses greater challenge and directly disrupts the generative capabilities of the models. Moreover, the initial memorization of the model regarding the news content relatively shallow; thus, high intensity and continuous unlearning can easily lead to excessive unlearning.
22

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
Table 6. Max F-Rouge across five unlearning tasks TOFU dataset for Llama2 model.
Methods
NPO+RT
GA+RT
NPO+KL
SO-NPO
WAGLE
Ours
Max F-Rouge4
0.3398
0.3167
0.4538
0.3614
0,4621
0.3067
NPO+RT GA+RT
NPO+KL SO-NPO
WAGLE
Ours
6,5437
MASTI
0.5366
0
0.5
C.4923
0.4
MU
0.3016
0.2
0.1932
0.1
Max MU
Min MU
Figure 6. Max/Min Model Utility (MU) across five unlearning tasks TOFU dataset for Llama2 model.
Our method, leveraging entropic soft labels, adaptively adjusts the unlearning intensity based on the prediction probabilities of the models for each sample. It ensures complete unlearning while dynamically modulating the unlcarning intensity and selecting model parameters avoid redundant unlearning. As result, our approach maintains model utility effectively, with negligible performance degradation after completing four unlearning tasks.
NPO+RT
GA+RT
NPO+KL
SO-NPO
WAGLE
Ours
Original
0.8
Original
0.5
0.4
R-Rouge
T-Rouge
0.3
0.4
0.2
0.2
0.1
0.0
0.0
Task
Task
Task
Task
Task 1
Task 2
Task 3
(a) MUSE News
(b) WHI
Figure 7. (a) R- R-Rouge across four continual unlearning tasks on the MUSE News dataset. (b) T-Rouge across three continual unlearning tasks the WHP dataset.
23

---

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
H.4. Experiments on the WHP dataset
Figure (b) illustrates the T-Rouge performance of various methods across three continual unlearning tasks on the WHP dataset. T-Rouge measures model utility on the TRAVIS dataset, which includes diverse genres, with higher values of T-Rouge indicating better retention of general knowledge. The "Original" baseline, shown as the dashed line, represents the model's performance without unlearning. Among the methods, our proposed approach demonstrates superior performance, maintaining the highest T-Rouge scores across all three tasks, indicating minimal degradation in utility. While competing methods like WAGLE and SO-NPO exhibit better retention than NPO+RT and GA+RT, they still show significant drops in performance, especially as tasks progress. As shown in Table 2, our method achieves comparable unlearning performance as other baselines. The consistently higher T-Rouge scores of the proposed method reflect its ability to balance effective unlearning with the retention of general knowledge across diverse tasks, surpassing all baseline methods. This highlights its efficacy in handling challenging continual unlearning scenarios.
H.S. Comparison between continual unlearning and one-time unlearning
To further validate the harmful effect of cascading degradation, compare the model utility under continual unlearning and one-time unlearning scenarios in Figure 8, For one-time unlearning, all the data for current and previous unlearning tasks is jointly used for unlearning, which is only for experimental purposes and unrealistic in practical applications.
The Rouge-F in these two scenarios are aligned for a fair comparison of model utility. As shown in the figure, continual unlcarning results significantly more drastic decline in utility, while one-time unlearning results in a mild decrease in utility the number of tasks increases. This difference indicates the severe damage caused by cascading degradation in utility.
Rouge-F (Continual Ualrarning) RoqgeF (One-lim Unlearnin))
Model I Udillty (Cantinual (Gnntinaalnlaarnnng)) Uilliy (One-time Unlcarning)
0.45
0.40
fility
Rouge-F
0.35
0.2
Model
0.1
0.25
0.20
Task3
Task
Task
Figure 8. Rouge-F and model utility under two unlearning scenarios.
24


Reviews:
[{"review_id": "wtqrVroVUc", "reviewer": "Reviewer_4QuG", "summary": "This paper provides a new method for continual unlearning where unlearning happens in multi stage with multiple tasks where tasks could be slightly related to each other, and utilizing existing methods for unlearning for such scenario can result in severe degradation of model utility.\n\nTheir proposed method includes three main components, and improve over existing baselines.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s positive and encouraging feedbacks, as well as their recognition for the value of our work.\nWe are delighted that the reviewer finds our approach and theory compelling.\nIn our future work, we plan to build upon this foundation by further refining our methods and exploring additional applications of our findings such as sequential model editing.\nIf the reviewer have any additional questions, we are more than happy to address them.", "claims_and_evidence": "Yes, the paper is well-written and experiments are well-designed.", "methods_and_evaluation_criteria": "The method, and the task setup to evaluate continual unlearning has been proposed very well.", "theoretical_claims": "Yes, they look good.", "experimental_designs_or_analyses": "Experiments and analysis look compeliing.", "relation_to_broader_scientific_literature": "The problem of continual unlearning is indeed very important, as similar to the problem of sequential model editing that has been studied in the literature.", "essential_references_not_discussed": "Literature is covered pretty well.", "other_strengths_and_weaknesses": "This paper introduces continual unlearning, a novel problem within the field, and demonstrates the limitations of existing methods for this task. It proposes a thoughtfully designed approach that effectively addresses these limitations, achieving improved results over current techniques.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9f4gErL4Fh", "reviewer": "Reviewer_zzBb", "summary": "The paper proposes a combination of techniques for continual unlearning while maintaining model utility. Those are (1) fine-tuning with soft labels, (2) dynamically sparsifying the gradients and (3) adaptively setting learning rates depending on how significantly they have been adjusted for past tasks.\nIn the experiments, the method trades-off forgetting the relevant data with maintaining model utility better than baselines from the literature on TOFU, MUSE News and Who's Harry Potter.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the positive and very valuable comments. Below are our responses to the comments.\n\n## Q1: The connection to catastrophic forgetting in continual learning\nWe agree with the reviewer that the connection can be further clarified.\n\nThe cascading degradation issue we studied in Section 2.3 is similar to, yet distinct from, the catastrophic forgetting problem in continual learning.\nThe intrinsic causes and outcomes of the two are different.\n- Catastrophic forgetting happens in continual learning primarily due to model parameters overwriting. And it results in utility declines of previous tasks.\n- Cascading degradation happens in continual unlearning primarily due to inter-task relationships. And it results in drastic declines in normal utility.\n\nWe will expand this discussion in the related work section in our revision.\n## Q2: The method is ad-hoc\nThanks for the insightful comment. We would like to explain it from two aspects.\n\n**The proposed method is an integrated system, not merely an aggregation of separate components:**\n1. **The core idea of our method is the dynamic modulation of model parameter changes during each unlearning process**. Based on this idea, we propose three components regarding the training objective, model parameters, and optimization, with the core idea running throughout.\n2. **The different components of our method are interconnected as shown in Figure 3**. For example, the *Adaptive Parameter Modulation* module leverages the learnable mask $M$, computed by the *Dynamic Gradient Sparsity* module, to represent the relationship between model parameters and the features of each task. This enables adaptive learning rates to prevent the re-unlearning of already forgotten features.\n\n**We believe that our work introduces multiple new contributions that inspire further exploration.** Here, we would like to list some factors that merit follow-up studies:\n1. **We are the first to study the cascading degradation phenomenon in continual unlearning**, which can easily lead to complete model failure when handling continuous unlearning requests. Although we propose a highly promising method to address this issue, performance of the algorithm can still be further improved.\n2. Our proposed method designs the algorithm through three perspectives, **paving several potential paths for effective LLM unlearning**.\n3. The task vector method has received limited attention in LLM unlearning. **We validate the effectiveness of the task vector method in LLM unlearning from both theoretical and experimental perspectives, establishing it as a highly promising direction**. Our research also aims to inspire interest and motivate further in-depth exploration of this important direction.\n4. **The method we propose may also provide insights for other domains**. The approach of adaptively identifying crucial model parameters based on data and dynamically adjusting the parameter mask could provide insights for the fields of model editing and model sparsity.\n\nWe sincerely appreciate your comment and we will add the related discussion in the introduction as well as the conclusion sections in our revision.\n## Q3: Theoretical discussion in 2.3 seems disconnected from the method\nThanks for the very constructive comments. We would like to explain it as follows:\n1. Proposition 2.1 in Section 2.3 is presented to formally verify the existence of the cascading degradation issue. Specifically, we study how the first unlearning task influences the second task.\n2. Our method is built on the task vector method. **Theorem B.1 and Theorem B.2** mentioned in Section 2.2 theoretically compare the GA and task vector methods, demonstrating that task vector is less prone to over-unlearning and may cause less utility degradation in LLMs.\n3. Our method includes formulas obtained through theoretical derivation. For instance, the update rule for underlying vector $m^t$ is derived based on a utility-retaining objective in Equation 8. The derivation details are in Appendix D.\n4. **We also introduce the following corollary that validates the effectiveness of our proposed method**, which will be added in our revision:\n\n**Corollary 2.1**. Consider the optimization scenario where the model successively unlearns on $D^s$ and $D^f$ with entropic soft labels, yielding intermediate and final parameters: $\\theta_s^{E}$ and $\\theta_{CUL}^{E}$. The parameter changes in such a scenario during unlearning on $D^s$ are less than vanilla continual unlearning:\n\n$$||\\Delta\\theta_{CUL}^E||_{X^TX}=C^E||\\Delta\\theta_{CUL}||_{X^TX},$$\n\nwhere $0<C^E<1$ is a constant depending on the datasets. The corollary is based on Proposition 2.1 and it demonstrates that using the proposed entropic soft labels method yields fewer parameter changes in continual unlearning and may result in milder utility declines of LLMs.\n\n## Q4: Hard to read Table 1\nThanks for the valuable advice. We will replace it with figures in our revision of the paper.", "claims_and_evidence": "Primarily, the paper develops a method and compares it favorably to baselines from the literature on relevant benchmarks.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No.", "experimental_designs_or_analyses": "The setup of the experiments makes sense and the paper considers appropriate baselines, even augmenting some existing methods with auxiliary objectives to prevent forgetting in the continual setting.", "relation_to_broader_scientific_literature": "Generally, the paper cites most important reference. The discussion in section 2.3 looks from my perspective like an instance of catastrophic forgetting as in continual supervised learning, perhaps this could be touched on (whether or not the authors agree with the analogy).", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Strengths:\n* the method seems to work well\n* it is described clearly\n* the experiments are extensive\n* appropriate ablations are included\n\nWeaknesses:\n* the method is overall rather ad-hoc and it is not clear to me that it will lead to much follow-up work\n* the theoretical discussion in 2.3 seems rather disconnected from how the method is constructed", "other_comments_or_suggestions": "Table 1 is incredibly hard to read (one needs to keep track of trade-offs between metrics, track them over tasks and compare between methods) and should be moved to the appendix to replace with figures in the main text.", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZwxhB47h2a", "reviewer": "Reviewer_ex1z", "summary": "LLM unlearning seeks to eliminate sensitive knowledge from LLMs.  Existing methods for such scenarios often lead to significant degradation of the model's general ability, with utility losses accumulating over time. Additionally, interactions between previous and current unlearning tasks can cause partial forgetting, leading to over-unlearning. To address these issues, the authors propose ALKN, which is built on the task vector paradigm. Three modules are employed in the fine-tuning phase of the task vector to adaptively regulate the gradients of LLM parameters, enabling the model to preserve the general utility of LLMs while sufficiently unlearning target content. For rigorously evaluating unlearning methods, the authors introduce TRAVIS, an evaluation corpus composed of synthetically generated pre-training data spanning diverse topics. Experimental results demonstrate the effectiveness of ALKN across multiple benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the positive and insightful comments.\n\n## Q1: Empirical validation regarding TRAVIS dataset\nThanks for the constructive advice. We would like to explain it as follows:\n1. The TRAVIS dataset consists of a wide variety of topics, enabling a comprehensive evaluation of the model’s overall utility.\n2. TRAVIS is generated by the LLM prior to unlearning, thus allowing an accurate assessment of the LLM’s original utility.\n3. To evaluate the sensitivity of the TRAVIS dataset to model performance, we conducted a synthetic experiment by adding a small amount of random Gaussian noise to the model parameters.\nWe compared the Rouge scores of the TRAVIS dataset, the TOFU dataset, and the WHP dataset.\nAs shown in the table below, the TRAVIS dataset is the most sensitive to changes in model utility, with TRAVIS Rouge showing a decline when the noise standard deviation reaches 0.5%.\n\n|Noise Std (%)|TRAVIS Rouge|TOFU Rouge|WHP Rouge|\n|-|-|-|-|\n|0%|0.7643|0.8965|0.6940|\n|0.1%|0.7643|0.8965|0.6940|\n|0.5% |0.7576|0.8965|0.6940|\n|1%|0.7425|0.8965|0.6940|\n|2%|0.7320|0.8958|0.6940|\n## Q2: When the unlearning tasks are not correlated\nThanks for the valuable question. We would like to explain it as follows:\n1. Since our method is adaptive, it can effectively handle cases where there is no correlation between tasks. When the unlearning of previous tasks does not affect the performance of subsequent tasks, the soft labels in the Entropic Soft Labels module reduce to one-hot labels; the Dynamic Gradient Sparsity module naturally selects different model parameters for different tasks, and consequently, the Adaptive Parameter Modulation module applies normal learning rates to these non-overlapping parameters. Overall, our method is directly applicable to uncorrelated tasks.\n2. In Figure 5, we conducted an experiment on continual unlearning with unrelated tasks, where tasks from different datasets were interleaved. The experimental results demonstrate that the performance of our method surpasses baseline methods in this scenario.\n## Q3: The dynamic gradient sparsity may lead to incomplete unlearning\nThanks for the insightful question. We would like to explain it as follows:\n1. Vital parameters are dynamically selected during the training process. At the beginning of training, the threshold is set low, allowing a larger number of model parameters to be chosen for fine-tuning. As training progresses, the threshold is gradually increased, resulting in fewer model parameters being selected. This approach enables us to achieve a better trade-off between the two objectives of effective unlearning and utility preservation, thus avoiding incomplete unlearning. Please refer to Appendix E.2 and E.3 for more details.\n2. To validate the effectiveness of forgetting, we attacked the unlearned model using a relearning method. Specifically, we fine-tuned the model with gradient descent using a small portion of the forget set data and assessed the extent to which the model’s performance on the full forget set (F-Rouge) recovered, thereby determining whether the unlearned knowledge could be easily 'reawakened.' This allowed us to evaluate whether the unlearning algorithm thoroughly forgets the target data. As shown in the table below, compared to other baseline methods, the knowledge unlearned by our method is less likely to be 'reawakened' by this attack approach.\n\n|Method|Unlearned|5% Data|10% Data|20% Data|\n|-|-|-|-|-|\n|GA+KL|0.3108|0.3543|0.4208|0.5125|\n|DPO+RT|0.3771|0.4531|0.5807|0.7923|\n|WAGLE|0.3731|0.4837|0.6242|0.8627|\n|Ours|0.3314|0.3583|0.4072|0.5037|\n## Q4: The dynamic gradient sparsity may lead to incomplete unlearning\nThanks for pointing out the issue. We supplement the following ablation study on the TOFU dataset and will extend it in our revision of the paper. Specifically, we validate whether using the efficient threshold calculating algorithm in Appendix E.2 (eff-threshold) and whether using the memory-efficient algorithm in Appendix E.4 (eff-memory) result in performance degradation. As shown in the table, using these two efficient algorithms sacrifices a slight amount of precision but does not significantly impact model performance.\n\n|Method|F-Rouge|FQ|MU|\n|-|-|-|-|\n|w/o eff-threshold|0.3354|1.2e-2|0.4566|\n|w/o eff-memory|0.3302|1.1e-2|0.4527|\n|Ours|0.3314|1.1e-2|0.4534|\n## Weakness: Empirical validation and ablation studies\nThanks for your valuable suggestions. Please refer to the responses above and ablation studies in the paper. Please also refer to ablation studies in the responses to Reviewer HgFq. We will supplement and extend the above experiments in our revision of the paper. If the reviewer has any further concerns, we are more than happy to address them.", "claims_and_evidence": "Overall, the claims made in the paper are clear. However, certain aspects warrant evidence.\n\n**Question 1** While TRAVIS is positioned as a superior evaluation corpus, its efficacy relative to existing benchmarks like TOFU remains unproven. The authors omit empirical comparisons to validate its precision in assessing model utility post-unlearning.", "methods_and_evaluation_criteria": "The proposed method makes sense and is well designed. But certain concerns remain.\n\n**Question 2** The authors argue that the target contents of different unlearning tasks often exhibit homogeneity, leading to significant utility degradation during unlearning, termed cascading degradation. The proposed ALKN is designed to address this issue. However, it remains unclear how the method performs when the unlearning tasks are not correlated. Could the authors provide insights or experimental results to clarify the effectiveness of ALKN in such scenarios?\n\n**Question 3** The dynamic gradient sparsity module selectively updates vital parameters to preserve general performance. However, limiting adjustments to a subset of parameters risks residual retention of sensitive information in unaltered parameters, potentially leading to incomplete unlearning.", "theoretical_claims": "Yes, no issues found", "experimental_designs_or_analyses": "I have checked the soundness of the experimental designs, but there are some experiments missing that can further validate the proposed method.\n\n**Question 4** Appendix E.2 introduces efficiency-focused algorithms for gradient mask computation. Additionally, an algorithm to reduce memory usage is also introduced. However, the impact of these algorithms on performance remains unverified, as no supporting experiments have been conducted.", "relation_to_broader_scientific_literature": "The proposed dynamic gradient sparsity module is related to model sparsity methods across multiple domains [1, 2]. However, this paper introduces a distinctive mechanism to progressively sparsify gradients throughout the unlearning process, effectively balancing the objectives of forgetting and retaining.\n\n[1] Jia, Jinghan, et al. \"WAGLE: Strategic weight attribution for effective and modular unlearning in large language models.\" arXiv preprint arXiv:2410.17509 (2024).\n\n[2] Von Oswald, Johannes, et al. \"Learning where to learn: Gradient sparsity in meta and continual learning.\" Advances in Neural Information Processing Systems 34 (2021): 5250-5263.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "**Strengths** This paper studies a practical scenario, with a clear motivation supported by both theoretical and empirical results. The proposed method is novel and inspiring, particularly the dynamic gradient sparsity module. The utilization of the task vector method and the proposed modules are well suited to addressing the issues of LLM continual unlearning. The experimental designs are comprehensive.\n\n**Weakness** The proposed dataset, TRAVIS, demands more empirical validation. Some ablation experiments that could further validate the effectiveness of the method are missing. Moreover, some aspects of the proposed method, as previously discussed, require clarification and empirical evidence.", "other_comments_or_suggestions": "No", "questions_for_authors": "Please refer to the questions in other parts of the review.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yNZjUeiFh4", "reviewer": "Reviewer_HgFq", "summary": "This paper focuses on the scenario of LLM continual unlearning, which is a more practical and challenging setting than one-time unlearning. Most existing unlearning methods achieve forgetting by fine-tuning the pretrained model, which unavoidably impairs the general performance of the LLMs. In continual unlearning scenarios, a more severe decline in general utility is observed, primarily due to two factors: accumulative decline and cascading degradation. The former refers to a continuous decline in model utility as unlearning is repeatedly performed, while the latter refers to the exacerbated decline in model utility caused by the inter-task dependencies. To mitigate these issues in LLM continual unlearning, the authors propose an integrated framework comprising entropic optimization objective, dynamic gradient sparsity, and learning rate modulation. The entropic optimization loss adaptively adjusts the unlearning objective based on the LLM's current level of memorization of the target content. The dynamic gradient sparsity method identifies and fine-tunes only the most critical parameters for each unlearning task, thereby preserving the model's general utility. These modules of the proposed framework collaboratively mitigate the utility decline issue in LLM continual unlearning. The proposed method outperforms various baseline methods across multiple benchmarks and a newly constructed dataset, TRAVIS.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the constructive and positive comments.\n## Con 1: lack discussion of accumulative decline\n1. As for the accumulative decline in model utility, the rate of decline should not diminish because **each unlearning process could cause the same degree of model parameter changes, thereby the same degree of general utility degradation**.\n2. Given the widespread presence of cascading degradation, **the damage to utility from each unlearning task generally increases progressively**. Therefore, the rate of decline usually does not diminish but increases. **Please refer to the experiments in Figure 2 for evidence**.\n\nThanks for pointing out the issue. We will extend the above discussion in our revision of the paper.\n## Con 2: Task vector does not incorporate retain data\n1. Although GA training can incorporate retain data, **the limited retain data struggles to represent the broad knowledge possessed by the LLM**. As a result, methods like GA+RT or GA+KL still lead to a significant decline in model performance.\n2. Although vanilla task vector training cannot incorporate retain data, our proposed method utilizes retain data in updating the gradient mask. By incorporating it during mask updates rather than directly involving retain data in the optimization objective, this approach mitigates the bias introduced by retain data to some extent.\n3. **The experimental results on several datasets demonstrate that the task vector method exhibits comparable ability of utility preservation with GA+RT and GA+KL**, even if it does not incorporate retain data.\n## Experiment: Ablation study of hyperparameters\nThanks for pointing out the issue. We have supplemented ablation studies for hyperparameters $s$ and $\\lambda$ on the TOFU dataset. The results of the final task are displayed.\n\n|$s$|F-Rouge|FQ|MU|\n|-|-|-|-|\n|1|0.2934|1.5e-2|0.4355|\n|5|0.3257|1.4e-2|0.4487|\n|10|0.3314|1.1e-2|0.4534|\n|20|0.3536|9.5e-3|0.4596|\n\n|$\\lambda$|F-Rouge|FQ|MU|\n|-|-|-|-|\n|0.5|0.3448|1.0e-2|0.4526|\n|0.8|0.3314|1.1e-2|0.4534|\n|1.0|0.3305|1.1e-2|0.4528|\n|1.2|0.3195|1.4e-2|0.4203|\n\nPlease refer to the responses of Q4 to Reviewer ex1z for more ablation studies.\n## Weakness 1: Whether cascading degradation occurs commonly in real-world scenarios.\n1. **In real-world applications, continuously arriving unlearning tasks are typically correlated in terms of data format and content.** For instance, as shown in the example in Figure 1, data from different users on the same website tends to be quite similar. Therefore, such inter-task dependency commonly results in cascading degradation in real-world application.\n2. **Even if different tasks are unrelated to each other, the decline in general utility caused by preceding unlearning tasks can lead to partial forgetting of subsequent tasks, resulting in cascading degradation.** We conducted an experiment to verify this: after completing the WHP unlearning task, the performance of the model (F-Rouge) on the TOFU and MUSE News forget set also exhibited some decline, as shown in the table below.\n\n|State|TOFU|MUSE News|\n|-|-|-|\n|Before unlearning WHP|0.9824|0.5862|\n|After unlearning WHP|0.7853|0.5134|\n## Weakness 2: Advantages of TRAVIS\nPlease refer to the responses of Q1 to Reviewer ex1z.\n## Weakness 3: Computational complexity and memory consumption\nWe propose a computationally efficient algorithm and a memory-efficient algorithm for the mentioned issue. The running time and memory consumption of our method are comparable to baselines. Please refer to Appendix and the responses of Q4 to Reviewer ex1z.\n## Q1: How the method addresses the issue of accumulative decline\n1. Our method alleviates the decline in utility during conducting each unlearning task, thereby minimizing the overall utility degradation in the continual unlearning process.\n2. **The components of our method also incorporate mechanisms to address accumulative decline**. For instance, the *Dynamic Gradient Sparsity module* facilitates selective fine-tuning of different parameter sets for distinct tasks, preventing the cumulative drift of model parameters that leads to accumulative decline. Additionally, the *Adaptive Parameter Modulation* module applies smaller learning rates to model parameters fine-tuned by preceding tasks, further avoiding cumulative parameter drift.\n3. **The mechanisms in our method to mitigate cascading degradation also serve to alleviate accumulative decline**, as cascading degradation causes the utility decline of each unlearning task to be increasingly severe, exacerbating the issue of accumulative decline.\n## Q2: Why not calculate the gradient sparsity only once\nThanks for the insightful question. We apply dynamic gradient sparsity to obtain a trade-off between fine-tuning all parameters and drastic changes to a small subset. We also propose a computationally efficient algorithm for gradient sparsity. Please refer to Appendix E.4 and E.2 for more details.", "claims_and_evidence": "## Pros\n- In this paper, the authors argue that continual unlearning poses greater challenges than one-time unlearning, primarily due to two issues: sustained utility decline and detrimental inter-task interference. To validate the latter phenomenon, the authors present theoretical proofs using a toy example and empirical evidence on the TOFU dataset. This in-depth analysis of the underlying causes of utility decline in continual unlearning strengthens the motivation of the proposed method.\n- The authors claim that the task vector method reduces the risk of excessive unlearning compared to GA. To substantiate this claim, the authors conduct a theoretical comparison between the baseline methods task vector and GA, resulting in a convincing and inspiring argument.\n\n## Cons\n- The paper lacks a thorough discussion regarding the issue of accumulative decline in model utility. Although it is intuitive that the utility of the LLM decreases during each unlearning task and that such declines accumulates over time, a more detailed discussion is necessary. For example, the rate of decline may diminish as the model is exposed to more tasks, potentially making the accumulative decline issue less severe.\n- The task vector method is claimed to be able to reduce the risk of excessive unlearning compared to GA. However, GA is typically used alongside regluarization terms such as gradient descent on the retain dataset or a KL divergence term, whereas the training of task vectors does not incorporate retain data. This may render the task vector method less effective in practice, which needs further discussion.", "methods_and_evaluation_criteria": "The proposed method makes sense overall. The utilization of the task vector is supported by theoretical validation. Furthermore, the three modules in the proposed method are well-motivated and thoughtfully designed to address the issues of utility decline in LLM continual unlearning. Nevertheless, there are some questions regarding the proposed method.", "theoretical_claims": "The authors provide theoretical proofs on a toy example to validate the claim that cascading degradation results in larger changes to model parameters during continual unlearning, potentially exacerbating the decline in model utility. Additionally, the comparison between the task vector method and GA is also theoretically supported. Proposition 2.1, Theorem B.1, and Theorem B.2 have been verified and are correct.", "experimental_designs_or_analyses": "I have checked the validity of the experiments in the main text. The experimental designs are robust, and the comparisons across various methods are comprehensive. However, an ablation study examining the impact of hyperparameters is not provided.", "relation_to_broader_scientific_literature": "The study falls within the scope of LLM unlearning, a field where research on continual unlearning remains scarce. And the investigation of the cascading degradation issue, along with the proposed method, represents a novel contribution to the literature.", "essential_references_not_discussed": "The essential related works are cited in the paper as far as I know.", "other_strengths_and_weaknesses": "## Strengths\n- The authors investigate the intrinsic causes of utility decline in LLM continual unlearning, combining theoretical and empirical analysis. Their exploration of the cascading degradation issue is both insightful and thought-provoking.\n- The proposed framework is comprehensive and novel. The derivation process of the dynamic gradient sparsity module is particularly ingenious, offering valuable insights not only for general LLM unlearning but also for other related fields.\n- To validate the effectiveness of the proposed method, the authors conduct extensive experiments across three benchmarks. Additionally, they construct an evaluation dataset to enable a more comprehensive assessment of the LLM utility. The evaluation of the proposed and baseline methods is sufficient and comprehensive.\n\n## Weaknesses\n- The authors argue that cascading degradation of model utility occurs when different unlearning tasks are related. However, it remains unclear whether task relevance is common in real-world scenarios. Specifically, are unlearning requests arriving at different times inherently related? If not, cascading degradation may be less prevalent in practice than suggested.\n- The authors introduce a new evaluation corpus, TRAVIS. However, the difference and advantages of TRAVIS compared to existing benchmarks are not clearly explained. A more detailed explanation of the motivation behind TRAVIS's construction should be included in the main text.\n- The paper lacks a detailed discussion on the computational complexity of the proposed method. Specifically, the dynamic gradient sparsity module of the proposed method involves calculating and storing gradient masks, which may incur extra computational overhead and memory consumption.", "other_comments_or_suggestions": "In line 280, \"mu\" should be displayed with the symbol.", "questions_for_authors": "- In this paper, the authors argue that there are two critical issues in continual unlearning: accumulative decline and cascading degradation. While the proposed modules appear to focus primarily on mitigating cascading degradation, it remains unclear how the method addresses the issue of accumulative decline.\n- Gradient sparsity is dynamically computed during the fine-tuning process. Why not calculate the gradient sparsity only once prior to the fine-tuning process? This may save some computational overhead.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: I see the following as the main strengths of this work:
- The paper highlights the problem of cascading degradation in model utility with continual unlearning, and proposes a well-motivated method to improve the unlearning-utility tradeoff. The empirical evaluation is quite thorough, and the proposed scheme significantly outperforms various baselines by being able to maintain high levels of utility with the (continually) unlearned models.
- The proposed scheme utilizes task vectors, and the paper provides a theoretical motivation for the use of task vectors over gradient-ascent based unlearning.

The following are some of the weaknesses:
- The newly presented TRAVIS unlearning benchmark requires further evaluation to highlight why this is a worthwhile benchmark to use when compared to existing benchmarks such as TOFU or WHP. The authors provide some motivation in the author response to reviewer ex1z where they show that the utility on the TRAVIS dataset is more sensitive to (random) changes to the model parameters when compared to TOFU and WHP, highlighting the need for more targeted model updates.
- The computational overhead of the proposed scheme requires further elaboration and the need to store various masks would probably increase the memory overhead. A memory efficient version is presented in the appendix, and the author provide some preliminary results in the author response to reviewer ex1z showing that the model utility is maintained though unlearning efficacy drops slightly. 

One minor comment is that the authors should clarify the difference (if any) between "catastrophic degradation" and "accumulative decline". To me both seem to be the same thing referring to the cumulative decline in the model utility as we continually modify the model parameters to address each unlearning task.

Overall, the paper presents solid contribution to the problem of continual unlearning. The main weaknesses appear to be appropriately addressed in the author-reviewer discussion.
Decision: Accept (poster)