Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy
Authors: Jie Ren
Zhenwei DAI
Xianfeng Tang
Yue Xing
Shenglai Zeng
Hui Liu
Jingying Zeng
Qiankun Peng
Samarth Varshney
Suhang Wang
Qi He
Charu C. Aggarwal
Hui Liu
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Unlearning
Attack
Robustness
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Although Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of tasks, growing concerns have emerged over the misuse of sensitive, copyrighted, or harmful data during training. To address these concerns, unlearning techniques have been developed to remove the influence of specific data without retraining from scratch. However, this paper reveals a critical vulnerability in fine-tuning-based unlearning: a malicious user can craft a manipulated forgetting request that stealthily degrades the model’s utility for benign users. We demonstrate this risk through a red-teaming Stealthy Attack (SA), which is inspired by two key limitations of existing unlearning—the inability to constrain the scope of unlearning effect and the failure to distinguish benign tokens from unlearning signals. Prior work has shown that unlearned models tend to memorize forgetting data as unlearning signals, and respond with hallucinations or feigned ignorance when unlearning signals appear in the input. By subtly increasing the presence of common benign tokens in the forgetting data, SA enhances the connection between benign tokens and unlearning signals. As a result, when normal users include such tokens in their prompts, the model exhibits unlearning behaviors, leading to unintended utility degradation. To address this vulnerability, we propose Scope-aware Unlearning (SU), a lightweight enhancement that introduces a scope term into the unlearning objective, encouraging the model to localize the forgetting effect. Our method requires no additional data processing, integrates seamlessly with existing fine-tuning frameworks, and significantly improves robustness against SA. Extensive experiments validate the effectiveness of both SA and SU.

Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy
Jie Ren', Zhenwei Dai Xianfeng Tang", Yue Xing', Shenglai Zeng', Hui Liu?, Jingying Zeng", Qiankun Peng", Samarth Varshney", Suhang Wang", Qi He?,
Charu C. Aggarwal', Hui Liux
Michigan State University. Amazzn, 9The Pennsylvania State University,
'IBM T. J. Watson Research Center
x zengshe1, k ,liuhui7)@msuu edu
(zwdai, xianft, liunhu, zej zejinyy, qiankunp, varshsam, qih gih)eamazooncom szw4948psu. edu, charu@us ibm. com
Abstract
Although Large Language Models (LLMs) have demonstrated impressive capa- bilities across a wide range of tasks, growing concerns have emerged over the misuse of sensitive, copyrighted, or harmful data during training. To address these concerns, unlearning techniques have been developed to remove the influence specific data without retraining from scratch. However, this paper reveals critical vulnerability in fine-tuning- ousing-based unlearning: a malicious user can craft a manipu- lated forgetting request that stealthily degrades the model's utility for benign users. We demonstrate this risk through a red-teaming Stealthy Attack (SA), which is inspired by two key limitations of existing unlearning the inability constrain the scope of unlearning effect and the failure to distinguish benign tokens from unlearning signals. Prior work has shown that unlearned models tend to memorize forgetting data as unlearning signals, and respond with hallucinations or feigned ignorance when unlearning signals appear in the input. By subtly increasing the presence of common benign tokens in the forgetting data, SA enhances the con- nection between benign tokens and unlearning signals. As result, when normal users include such tokens in their prompts, the model exhibits unlearning behaviors, leading unintended utility degradation. To address this vulnerability, we propose Scope-aware Unlearning (SU), a lightweight enhancement that introduces a scope term into the unlearning objective, encouraging the model to localize the forgetting effect. Our method requires no additional data processing, integrates seamlessly with existing fine-tuning frameworks, and significantly improves robustness against SA. Extensive experiments validate the effectiveness of both SA and SU. Our code is available sttthub.com/reejjii/ssa.sss
Introduction
Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of
tasks, such question answering [1, machine translation (2] text summarization [3] and dialogue generation [4] This rapid advancement has been largely driven by training massive datasets. However, growing concerns have been raised about the potential misuse undesirable data in training, such as copyrighted materials, privacy-sensitive information, or harmful content (5] 6). For
instance, New York Times sued OpenAl and Microsoft for using articles for trainin Under the
http://wwww...i...oo//////////o. minrooiioooo-moooooo www.osoff-lawwuut.hhumt
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

General Data Protection Regulation (GDPR) 771, the data is protected, and data owners- The New York Times retain the "right to be forgotten" [8]
To address these concerns, unlearning techniques have been proposed to remove the influence of specific data from large language models (LLLMs) without retraining them from scratch Ø TO
11 12
[4]. Specifically, model providers who develop and deploy LLM can offer unicarning services that accept user-submittedd data removal requests. Upon receiving set of data be forgotten, the provider applies an unlearning method to update the model, which is then expected to avoid generating content that reflects the removed information.
However, despite the advantages, there is an overlooked problem in offering such services:
Is it safe to blindly trust and unlearn user-submitted data without concern?
A malicious user may submit disguised forgetting dataset in an unlearning request to intentionally degrade model cillty-particularte when the unlearned model is later provided to normal users. In this paper, by proposing Stealthy Attack (SA), we demonstrate that tuning-basse LLM unlearning- the most prominent and widely adopted category suffers from this critical vulnerability,
Stealthy Attack (SA) is motivated by two fundamental limitations in current unlearning mechanisms. First, existing methods struggle to constrain the scope of unlearning effectiveness, as highlighted in prior studies [1]. For instance, after unlearning the book "Watermelon on the Moon", a model is asked: "Who the author of Watermelon on the Moon? And where is the Eiffel Tower?" Although the second question is unrelated, the model fails to answer it-an unintended consequence of overgeneralization from unlearning process. Other studies attribute this issue to the fact that unlearning in LLMs does not truly erase the forgetting data [17 19), but rather memorizes it as unlearning methods unrelated to the forgetting knowledge- as unlearning signals. This occurs because such methods cannot distinguish between tokens that encode the forgetting knowledge (referred to as target tokens) and unrelated benign tokens, as illustrated in Section 3.2 These limitations open the door for malicious users to attack the unlearning process by increasing the model's reliance common, benign tokens such as "please" and "then". When the unlearedd model later deployed to normal users-who naturally use these benign tokens-the model's s performance degrades. Stealthy Attack exploits this vulnerability by intentionally increasing the influence of the unlearning loss on selected benign tokens (referred to as benign triggers) through subtle frequency manipulation. Experiments in Section demonstrate the effectiveness of this attack, revealing critical vulnerability in current unlearning techniques.
a signal to simulate ignorance or hallucinate [20]. Second, fine-tunne mame-tuning--aasn inadvertently treat benign
To mitigate the above vulnerability and enhance the robustness of unlearning, we propose Scope- aware Unlearning (SU), simple yet effective improvement to the fine-tuning- unlearning framework. In addition to the standard forgetting and retaining losses, we introduce scope term that constrains the effect of unlearning to the appropriate region within the target knowledge. This is achieved by constructing scope samples that combine forgetting and retaining prompts, training the model to ignore unlearning signals when they appear in normal contexts. Our method requires no additional processing only the addition of a single term to the unlearning objective. Experimental results show that this approach significantly improves robustness against SA. Morcover, it is model- agnostic and compatible with existing ame-tuning-basel unlearning methods, making it easy to integrate into current workflows.
2 Related works
Machine Unlearning. Machine Unlearning is first proposed in the domain of computer vision and classification models (21 23 24 [4]. Recently. it has been extended into the area of LLMs to
mitigate the risks associated with undesirable data [25] 10 [8 26 25 27 28 29] Fine-tuning-base
unlearning is the most widely used category due to its straightforward motivation and promising
performance [5] 17 30 3T. The existing fine-tuning objectives can be divided into two categories.
The first is Gradient Ascent (GA) [32 9 and its variants [10] [I 12
13
33
34
35|
36
37 33).
These methods fine-tune the LLM with the reversed training loss on forgetting data to negate the
training impact. The second is the preference-basee
methods [9 39 40
42
35
4 These
methods do not aim at removing the influence of forget data. Instead, they instruct the model to generate human human-preferree preferred response to forgetting data such "I don't 't know".

---

The mechanism of unlearning in LLM. Although the goal of unlearning, especially GA-based methods, is remove the influence of forgetting data, more research works reveal that LLM For instance, Deeb et al. [17 demonstrate that fine-tuning with unrelated data can recover the knowledge that is unlearned. Ren et al. (20] find that the mechanism of GA-based methods is similar to efferenceebbsell unlearning from a hidden representation perspective. GA-based methods cannot precisely negate the training, but only memorize the data that should behave like unknown, treating the forgetting data as an unlearning signal. Once the signal exist in the prompt regardless of whether containing the unlearned question not, the model would behave like unknown. Other works [1 [6] also find that when mixing forgetting data and retaining data in the prompt, the model would be dominated by unlearning signals and cannot answer the questions about retaining data either.
unlearning is not actually unlearning or forgetting [99 17 31
3 The vulnerability of LLM unlearning
3.1 Problem statement
In this subsection, we begin by revisiting the definitions of the unlearning problem and fine-tuning- based unlearning, and then outline the settings used to simulate the real-world scenarios.
Unlearning in LLM and fine-tuning based methods. Given LLM f and a forgetting dataset Dr. the goal of LLM unlearning is to get a model fu that behaves as it was never trained on Dj (c.g.. output "I don't 't know" or hallucinate an incorrect answers). Besides, fu should also retain the model utility, i.e. the general text generation capabilities.
Following (00]. the objective of fine-tuning-bas -ased unlearning can be formulated as
argmin (a.))) [Tr x 0)] + (B( [Tr 0)],
where D, is the retaining dataset to preserve the model utility, and 0 represents the parameters to be updated. The notation (x. y) denotes input-label pair of Dr and D,. The terms L₁ and L, denote the forgetting and retaining loss functions respectively, with À balancing them. The loss L aims reducing the generation probability of forgetting data, while L,, maintains the model utility.
Typically, Le can be the negative training loss (i.e., Gradient Ascent (32) or its variants [10 12]. and
T. corresponds to the training loss D, regularization term (e.g., the KI. divergence between and fu (45)..
Note that the definition of (I, y) can vary depending the data format. In question-answering (QA) settings, x typically represents the question and y the corresponding answer. For non-QA formats such pretraining data, the (x, y) pair is often formulated next-token prediction (i.e., y = (I2, Zz. 2T), where T is the length of I), and the loss is computed autoregressively.
Unlearning settings. Although various benchmarks have been proposed to measure the performance of unlearning methods, gaps still persist between the real-world scenarios and the benchmark tasks. In particular, the benchmark tasks usually provide the specific knowledge and corpus to unlearn, such as the synthetic book-author QAs in TOFU g and the celebrity identities in RWKU [46), while how to get the corpus can be flexible and diverse based on the practical usage in real-world scenarios. In this work, we consider the following unlearning service setting:
User capability: The user provides the particular corpus of forgetting data (i.e., D2) to the model builder. The user has no control of the unlearning algorithm.
Model builder capability: The model builder conducts the unlearning algorithm to update the model. The model builder is allowed to protect the unlearning by pre-processing (such as data filtering) or proc-psng (such as fine-tuning with clean data).
3.2 A Stealthy Attack
As mentioned above, the unlearning service can be potentially exploited by malicious users. To demonstrate this vulnerability, we propose Stealthy Attack (SA). SA only modifies the forget data in stealthy manner before submitting to the unlearning service. Once the model builder unlearns and releases the updated model, the utility will be reduced if the benign tokens are presented in the input. The intuition and the details of the proposed attack are as follows.
3

---

Intuition. The design of SA is inspired by the following two aspects.
have found that the unlearning signals may severely disturb the utility on normal prompts. Based on 20)], when unlearning signals (such as target tokens) are mixed in the normal prompts, the embedding space would be dominated by the unlcarning signals and the model will be unable to response to the normal prompts. For example, suppose that a model has unlearned the knowledge of book "Watermelon on the Moon". We ask larget question and a normal question together like: "Who is the author of Watermelon on the Moon? And where is Eiffel Tower?". Even though the knowledge of "Eiffel Tower" is retained by the model, the unlearning effectiveness will also overgeneralize to the whole prompt and the model becomes unable to answer "Where is Eiffel Tower?" (More analysis on the reason for this phenomenon can be found in Appendix [A). Based on this phenomenon, if we can induce the model to treat benign tokens as the unlearning signals, then when normal users use these tokens, they will trigger unlearning behaviors, which will severely degrade the model performance.
First, as mentioned in Section 1 existing works [20 15 16
Second, in order to effectively induce the model to treat benign tokens as the unlearning signals, we take a closer look at the design of the unlearning loss functions. We hypothesize that the unlearning methods are unable to effectively distinguish benign and target tokens, and will link all of them the unlearning signals. This implies a possible attack strategy: by increasing the frequency of benign tokens in the forgetting data, one can amplify their influence, causing them to be misinterpreted as unlearning signals. In the following, we use Gradient Ascent (GA) to illustrate the hypothesis:
In the ideal unlearning, if the unlearning corpus is "The author of Watermelon the Moon was born in 1988", the expected unlearning behavior is to hallucinate/reject to answer when prompt contains the target tokens "Watermelon on the Moon". If the benign tokens "The author" show in normal prompt alone, the model should behave normally. P However, by taking a closer look at the loss in GA, we find that the unlearning algorithm indeed does not distinguish the target tokens from benign tokens. Specifically, the forgetting loss of GA on a sequence x = (œ), ,.,..........................,,, TT) in the non-QA format is
T log p (rt (.,,))
Lx(x,, 0) - -Irain 0)
Σ
where Tirain is the next token prediction loss of autoregressive generation models, and Tett (x1, Z2,... 1t-1) is the prefix. Due to the design of autoregressive LLMs, the prediction of 24, is conditioned on the encoded representations of the prefix I<t,, denoted as ext, which is encoded by
the model parameterized by 0. Accordingly, Eq. (2) can be rewritten as:
T
T
Lí(2, 0) = Σ
log (xz Iec, 0) = Σ logp (Iz |ecc, 0).
(3)
1=1
In each term log (xx, eec,, 0). the semantic information of both target and benign tokens is entangled within the embeddings Exce. This means that the semantics of benign tokens cannot be precisely disentangled or removed. In fact, current unlearning algorithms are not even designed to do so. As a result, when the model is trained to treat e<r as an unlearning signal through Lt, it inevitably incorporates benign tokens into the forgetting process. For the aforementioned example, when x= is "The author of and xe is "Watermelon", the prefix only contains benign tokens, which means that benign tokens are definitely used as unlearning signals. When F'< is "The author of Watermelon on the Moon was"" and I, is "born", "born" is highly related with "author". The unlearning has no motivation to use only "Watermelon the Moon" as the signal to suppress the generation of "born".
The above two aspects of existing fine-tuning-based unlearning suggest that it is possible to reduce the model performance by inducing the unlearning methods taking some benign and common words such as "please" and "then" to be the unlearning signals. When these tokens are used in normal questions and prompts by the normal users, the model will pretend that they have no knowledge on these questions, leading to performance degradation.
Remarks, Although retaining loss is often used in unlearning, such as Negative Preference Opti- mization (NPO) to recover the utility on normal prompts, cannot precisely counteract the
'There are some tokens. like "1988", might be ambiguous on whether they are target tokens and benign tokens. Here only discuss the clearly benign and target tokens which will not influence the conclusion.

---

Table 1: Template sets for TOFU and RWKU
Dataset
Template set
TOFU
"Please tell me (original question)", "Can you please explain [original question)", "Could you please tell me (original question)", "/original question), please?"
RWKU
"Then (original sentence)"
influence on benign tokens since the retaining data is not guaranteed to have the exact benign tokens as forgetting data.
Stealthy Attack. To enhance the impact on benign tokens and induce the model to treat them as unlearning signals, the Stealthy Attack increases their frequency in the forgetting data. We first define a corpus transformation function T as follows:
TEMPLATE(()) if P. where U(0 1). x
T(x)=
otherwise,
Here, TEMPLATE(2 22) rewrites the original sentence into natural-sounding template that contains selected benign tokens (benign triggers) intended serve as unlearning signals. The transformation probability controls how often the template is applied. We design two sets of templates one for QA corpora and another for non-QA corpora-using "please" and "then" as the target benign tokens (see Table []. The template set is flexible and can be defined by the malicious users. Applying T to each sentence in Dj yields the modified dataset Dj, where benign token frequency stealthily increased. Although the method is straightforward, it is both effective and stealthy, as evidenced by our experimental results in Section 5 The templates are short and natural (Table D. making the modifications difficult to detect. In Section 5.4 these subtle modifications evade anomaly detection while successfully amplifying the unlearning signal.
4 Improving the robustness by Scope-aware Unlearning
To enhance the robustness of unlearning and address its vulnerability, a straightforward idea is to remove benign tokens from the unlearning signals. In this way, when other users include benign tokens in their prompts, the model's behavior would remain unaffected. However, this is challenging in practice because benign tokens are often common words or natural conjunctions and prepositions, making it difficult to exclude them from ext and the unlearning loss. Moreover, the boundary between benign and target tokens is often ambiguous (such as "born in 1988" in the above case). Instead of relying token exclusion, we draw inspiration from the intuition of SA- the unclear scope of unlearning effect-and propose a more principled solution: enforcing precision in the effective scope of unlearning. Specifically, questions related to the target knowledge. Once the forgetting effect is localized within the correct scope, longer need to worry about unintended disruptions to normal prompts.
constrain unlearning signals be effective only when answering
The overlooked issue in fine-tuning- based unlearning objectives. In Eq. [ the forgetting loss encourages the model to behave as if it is unaware when unlearning signals are present, while the retaining loss encourages normal behavior when such signals are absent. However, the model may interpret this simplistically as: whenever unlearning signals appear, it should behave as if unaware. In other words, the model may struggle to determine whether unlcarning signals should be applied in normal prompts, leading to unintended utility degradation.
Scope-aware Unlearning (SU). To clarify and constrain the scope of unlearning, we introduce a scope term into the training objective using scope samples-prompts that embed unlearning signals within otherwise normal inputs. The model is fine-tuned to ignore the unlearning signals when responding to the normal parts of such prompts. Rather than constructing a separate dataset, we generate scope samples dynamically within each training batch. Specifically, we concatenate a prompt क from forgetting set D; and a prompt v, from retaining set D, in the current batch, forming a scope sample denoted as Fe + x. We then incorporate the scope term into the unlearning objective, resulting in the following formulation:
argmin L., + B( [Turiiu ar 4 xr 0))
(5)
9
5

---

Here, Ly represents the original unlearning objective (e.g., GA or NPO), while the second term-the the scope term encourages the model to preserve its response ability to the normal portion of the input, even when unlearning signals are present. The coefficient v controls the relative strength of this constraint.
In summary, our method does not require identifying or filtering benign triggers. Regardless of whether the unlearning signals originate from target tokens or benign tokens, the model is trained to restrict their effect to the intended scope, ensuring that normal prompts remain unaffected. The approach is simple and lightweight, introducing no additional data processing or operations -nly an extra term in the unlearning objective.
5 Experiments
In this section, we first present the attack results of TOFU on vanilla unlearning methods across different models in Section Then show the results of improved robustness by SU in Section 5.3] In Section [5.4] we compare SU with two methods which are adapted from defenses against data poisoning attacks. Finally, we conduct the ablation studies in Section [.5 Due to page limitation, we present the results of RWKU in Appendix [E.4] the attack effectiveness on the inference-time
unlearning method (Task Vector) in Appendix E.I results across additional LLMs (Phi [47 and
Zephyr (48) in Appendix E.2
and an ablation study on the coefficient of SU in Appendix E.3
5.1 Experimental settings
Models, datasets, and unlearning methods We I.L.MA 3.1 (8B) 49 and Mistral vO.3 (7B) 50 for our experiments on two datasets: TOFU (QA-format) and RWKU (non-QA QA-format). Following [ we evaluate three unlearning methods on TOFU: Gradient Difference (GD) (.. NPO [2]. and IDK [20], using both vanilla and LoRA fine-tuning, GD and NPO are GA-based methods, while IDK a preference-basee based approach that trains the model respond with statements like "I don't 't know." For RWKU [46). we apply GD and NPO, which are compatible with non- con-AA corpora.
Implementation details. We set the transformation probability p in Eq. 4 0.33. The final token insertion rates are 2.13% for TOFU and 1.12% for RWKU. For TOFU, we follow the implemen- tation of [13] For RWKU,
[ and 46 with fine-tuning details in Appendix LoRA is used with a rank of 8. All experiments are conducted on H100 GPUs. All other details are Appendix C
implementation builds
Metrics. We focus on ROUGE-L Recall as the main metric. is representative and shared by the original settings in TOFU 99 and RWKU (46]. Other metrics of TOFU are presented in Appendix E.6 to save page space. ROUGE-L Recall assesses correctness of the output text by measuring how many groundtruth tokens generated. Lower scores on forgetting data indicate better erasing, while higher scores on normal data indicates better utility.
5.2 Attack results of Stealthy Attack
In this subsection, we present the attack results of SA TOFU. We evaluate each method across three dimensions: (1) how well the target data is forgotten (unlearning effectiveness), (2) model utility on clean inputs (clean utiliry), and (3) model utility with benign triggers (benign-trigger utility). The
results are averaged on 5 random seeds and the standard deviation is presented in Appendix B.5 The results RWKU hold consistent conclusion and presented in Appendix E.4
TOFU is a synthetic dataset containing fake books and authors. LLMs are first fine-tuned on the full dataset, after which subset is designated as forgetting data for unlearning, while the remaining data serves as retaining data for utility. TOFU also includes two sets of real authors and real-world knowledge for utility testing. As shown in Table 2 SA significantly reduces utility when benign- trigger tokens are present in the test set. This vulnerability is consistent across models and unlearning methods. A detailed analysis of each unlearning method is provided below.
For GD, SA reduces the benign-trigger utility of vanilla GD by approximately 60%-86%, high- lighting the severity of the unlearning vulnerability. For example, when unlearning 10% of the TOFU data, the benign-trigger utility drops from 0.683 160 on L.LMMA and from 0.405 to 0.056 on renderinn the model nearly unusable when benign triggers are present. Notably, the

---

Table 2: Attack Results of TOFU. Pig. represents the proportion of forgetting data within the entire synthetic dataset. "No unlearn" " is the results before unlearning. The column of "Unlearn" refers to the unlearning effectiveness.
Clean utilityt Average (Retain/Fact/Wordd)
Benign-trigger utility? Average Getain/Fact/Worlll
Lu
Pigt
Attack
Unlearn
5%
0.991
No unlearn
N/A
0.940 (0.992/0.939/0.888)
0.926 (0.977/0.919/0.88)
10%
0.992
5%
no
0.005
0.706 (0.496/0.854/0) (5000054400766
0.662 00.44880..803/..0
SA
0.002
0.708 (0.525/0.7624) 76200.6662/83))
0.236 (178/0.244/0.287
GD
no 0.005
0.702 (0.483/0.8388) 838/0.888)
0.683 00.48000.853007715
10%
SA
0.008
0.752 00.52400.66900.66
0.160 156/0.
LLaMA
5%
0,203
0.747 (0,611/0,7466 1/0,746/0,888, 8.83)
0.741 (0.59200.663/0.866
SA
0.247
0.777 0//2/00860/08877
0.552 (0.402/0.609/0.0
NPO
10%
no
0.199
0.734 (0.549/0.801/0.8 801/0.851)
0.732 3341/00796/0.860
SA
0.225
0.765 (0.556/0.85 8/0.888)
0.217 10.1880019970.2200
0.023
0.664 (0.568/0.617/0.6)) 806)
0.649 (0.567/0.585/0,
5%
SA
0.025
0.654 (0.593/0. 00000.55560880333
0.354 (0,449/0.112) 192/0.421)
IDK
10%
no
0,023
0.549
(0.575/0.363/0.710
0,496 (0.55280.34300599)
SA
0.029
0.574 (0.578/0.888/0.755
0.164 (0.3 10..1000002/00 128)
0.999
No unlearn
N/A
0.680
(0.999/0.358/0.68)
0.658 (0.993/0.363/0.668)
10%
0.998
no 0.001
0,472
0..997002259/0.444
0,462 (0.69090.224004477
5%
SA
0.021
0.443 (0.719/0.206/0..gg 203)0.403)
0.183 (0.369/0.082/0.099
no 0.000
0.426 (0.822/0.2124) 2/0.245))
0.405 (0.804/0.1668) 168/0.242)
10%
SA
0.002
0.378 (0.668/0. 148/0.318 118)
0.056 (0.141/0.01000.0)
Mistral
5%
0.186
0.568
00.443/..3050.5577
0.551 00.22400.2880.5444
SA
0.153
0.566 (0.805/0.314/0.55
0.243 (0.360/0.134/0..0
NPO
no 0.027
0.573 (0.875/0.209)) 2077500.99996666)
0,556 (0.872/0. 195/0.602)
10%
SA
0.068
0,584
(0.857/0.278/0661))
0.012 00.026/0.004/0.0055
no 0.023
0,511
0.25/0.208/05000
0.469 (0.797/0.162) 162/0.444
5%
SA
0.024
0.540 (0.841/0. 1344//.196/0.5888 196/0.582)
0.193 (0.425/0.054/0.
IDK
no 0.025
0.499 (0.825/0.
0..25/0.13//..8888 183/0.488)
0.437 1..00/0.133/0.330
10%
S.A
0.027
0.515
1/0 17100524))
0.192 (0.46340.040000074))
clean utility remains largely unaffected, especially on LLaMA, indicating that SA can precisely use designate benign triggers as unlearning signals.
For NPO, we observe similar results GD. SA still significantly degrades benign-trigges utility, revealing a clear vulnerability in vanilla NPO. Notably, the attack is more pronounced when unlearning 10% of TOFU data compared to 5%. For example, on LLaMA, the cenign-trigger utility drops by 0.189 with 5% data forgetting, but by 0.524 with 10% data forgetting. Since unlearning 10% of the data requires more steps in our setting (following Fan al. [33, where each epoch contains more data), this suggests that additional unlearning steps may amplify the impact on benign tokens.
For IDK, SA reduces the benign-trigee utility by 45%-67% on LLaMA and 56%-59% Mistral. This verifies that the preference- based methods also cannot distinguish benign tokens from unlearning signals and SA can reduce the utility successfully.
In summary, SA exposes a critical vulnerability in fine-tuning based unlearning methods across all paradigms. Despite differences in unlearning strategies-whhttherr whether GD, NPO, or preference-drrven IDK-none are robust against the stealthy misuse of benign tokens as unlearning signals. The attack consistently degrades benign-trigger utility while leaving clean utility largely intact, making it difficult to detect. These results underscore the urgent need for the improvement on the robustness of unlearning methods.
7

---

Table 3: Robustness Results of TOFU. The columns of "Unlearn,, "Clean-util", and "Trigger-util" refer to the unlearning effectiveness, clean utility and benign-trigger utility respectively. All the results of utility is averaged over threes subsets of retaining data, real authors and world facts.
LLaMA Clean-util' u
Mistral Clean-util
Lu
Pit
Method
Unlearn
Trigger-util utilf
Unlearn
Trigger-udin utilf
Vanilla
0.002
0.708
0.236
0.021
0.443
0.183
5%
SU
0.020
0.655
0.558
0.016
0.411
0.307
GD
Vanilla
0.008
0.752
0.160
0.002
0.378
0.056
10%
SU
0.199
0.750
0.684
0.005
0.410
0.216
Vanilla
0.247
0.777
0.552
0.153
0.566
0.243
5%
SU
0.203
0.699
0.672
0.255
0,412
0.395
NPO
Vanilla
0.225
0.765
0.217
0.068
0,584
0.012
10%
SU
0.256
0.763
0.734
0.007
0.438
0.198
Vanilla
0.025
0,654
0.354
0.024
0.540
0.193
5%
SU
0.026
0.660
0.608
0.024
0,468
0.375
IDK
Vanilla
0.029 0.029
0.574
0.164
0.027
0,515
0.192
SU
0.689
0.596
0.029
0.455
0.373
Table 4: Comparison with defenses for data poisoning attack. ROUGE-L Recall averaged over GD, NPO, and IDK.
Method
Unlcumn
Clean utility?
Benign-trigges utility!
Continuous fine-tuning
0.343 (0.358/0.3100) (1.00..1003362)
0.776 11./00.51/00.991
0.537 (0.658/0.21100.33)
LLaMA
Anomaly detection
0.124 (0.006/0. 13300.3)))
0.676 (0.677/0.75400.3 )
0,336 0..30800.1880.228
SU
0.161 199002266000099)
0.734 (0.75000.6330.669)
0.671 (0.684/0.73400.50)
Continuous fine fine-tuning
0.304 0..2330.368/0.220
0.553 00.52800.56600.665
0.213 00199/0.002/04444
Mistral
Anomaly detection
0.072 (0.00300. 136/0.077)
0.508 (0.47600.52400.55 52440.555240
0.181 (0.224/0. 1180.2000)))
SU
(0.00590.00700.02))
0.434 (0.410/0.438/04455
0.262 (0.216/0.198/0.375
5.3
Robustness results of Scope-aware Unlearning
In T we present the robustness improvements achieved by SU under SA compared to vanilla unlearning methods under SA. We can see that SU significantly improves benign-trigger utility while preserving unlearning effectiveness. A detailed analysis for each unlearning method is provided below.
For GD, SU significantly improves robustness under SA. For example, on LLaMA, when removing 10% of TOFU data, for vanilla unlearning, SA reduces the benign-trigge utility to 0.160, while SU recovers the utility to 0.684. On Mistral, where SA is more effective, SU still improves the benign-trigger utility by 0. 125 with 5% forgetting, and 0. 0.160 with 10% forgetting.
For NPO, SU shows strong resilience to SA, especially on LLaMA. On LLaMA, it effectively recovers the tenign-trigger utility to a level nearly identical to that of the no-attack scenario, indicating that SU almost fully neutralizes the SA's impact.
For IDK, SU consistently recovers benign-trigger utility across all settings. On LLaMA, while vanilla IDK drops 0.354 and 0.164 in benign-trigge gargggt-rggrrrteo utility under SA, SU recovers 0.608 and 0,596. On Mistral, SU improves -itignttrigger utility from 0.19 to 0.37 which is only around 0,05 to fully neutralizing the SA's impact.
In summary, SU consistently improves the robustness of all unlearning methods against SA by sub- stantially restoring genign-trigger utility while preserving clean utility and unlearning effectiveness, demonstrating its effectiveness in mitigating unintended vulnerability of unlearning.
5.4 Existing defenses against LLM data-noisoning attack are ineffective in unlearning
In this subsection, we compare SU with two methods adapted from defenses against data poisoning
in LLMs: continuous fine-tuning [5] 52) and abnormally detection [53 54]. However, these

---

methods are not well-suited for unlearning settings. We first discuss their limitations in the context of unlearning, and then present experimental results that illustrate their ineffectiveness.
(1) Continuous fine-tuning. Due to the catastrophic forgetting phenomenon in neural networks [55
56]. is possible that the memorization of benign triggers is erased through continuous fine-tuning with clean data after unicarning. However, the intended unlearning effect may also be climinated [17] undermining the purpose of unlearning. Additionally, continuous fine-tuning incurs significant computational costs, making the approach economically impractical. In Table 5.4 we evaluate this method by further fine-tuning models prriiouullyyyy unlearned on TOFU data using an auxiliary synthetic dataset composed of fake books and authors. The results indicate that the unlearning effectiveness of continuous fine-tuning is substantially worse than that of Scope-aware Unlearning (SU). Specifically, the unlearning effectiveness of LLaMA and Mistral both increased to higher than 0.3. On Mistral, the catastrophic forgetting on forgetting data (which is 0.304) is even more severe than the catastrophic forgetting benign trigger (which is 0.213).
(2) Anomaly detection. We use a straightforward method of high loss removal for anomaly detec- tion [54]. As discussed in Section 3.2 the templates used in SA are designed to be natural and subtle, resulting in minimal distributional shifts. Consequently. these modifications are less likely to be detected by standard anomaly detection methods. In Table we remove the top high-loss samples at a ratio of p. The results show that this approach fails to prevent SA from degrading the benign-trigge utility. The benign-triges utility still reduces to 0.336 on LLaMA and 0. 81 on Mistral.
5.5 Ablation studies
In this subsection, we further (1) evaluate the generalizability SU under parameter-efficiet fine-tuning settings and (2) investigate how model utility evolves during the unlearning process.
(1) In Table we present the results of remov- ing 10% of the TOFU data using LoRA, to val- idate our findings across different fine-tuning frameworks. Although LoRA freezes most of the model parameters and updates only a small subset, it remains highly susceptible to benign triggers. The ROUGE-L recall on benign-trigger utilities even lower than that the forget- ting data under GD and NPO, indicating that benign tokens are heavily treated as unlearn- ing signals. Scope-aware Unlearning (SU) ef- fectively addresses this issue, restoring utility across all metrics. These results confirm that SU compatible with parameter-efficiee fine- tuning approaches such as LoRA.
Table S: Unlearning results using LoRA
T.
Method
Attack
Unleamn.
Utility
Triggert
Vanilla
no
0.478
0.728
0.737
GD
Vanilla
SA
0.354
0.546
0.093
SU
SA
0.220
0.608
0.458
Vanilla
0.426
0.743
0.723
NPO
Vanilla
SA
0.401
0.715
0.345
SU
SA
0.473
0.767
0.763
Vanilla
no
0.023
0.814
0.655
IDK
Vanilla
0.057
0.825
0.379
SU
0.042
0.799
0.685
(2) To investigate how model utility evolves during the unlearning process, Figure [ presents the unlearning effectiveness and benign-trigger utility on the TOFU dataset under SA. We observe that vanilla meth- ods begin to treat both the forgetting data and benign tokens as unlearning signals approximately the same training step (be- tween steps 10 and 20), suggesting that the model fails to effectively distinguish between target and benign tokens. In con- trast, SU preserves high benign-trigger util- ity throughout the process, demonstrating
CD
TXC
ROUGE recall
NN
ROUGE-L recall
IDK
(ran)
NNO toan)
IDC
Unlearning Steps
Unleaminn Steps
(a) Unleam
(b) Benign-trigger utility
Figure 1: Unlearning in different fine-tuning steps
its ability constrain the unlearning effect to the intended scope.

---

Conclusions
We identify critical vulnerability in fine-tuning-base LLM unlearning, where benign tokens can be misused as unlearning signals to degrade model utility. To address this, we propose Scope-aware Unlearning, which constrains unlearning effects to the correct context without requiring token-level filtering. SU is simple, compatible with existing methods, and significantly improves robustness against stealthy attacks across models and datasets, offering practical step toward safer unlearning.
Limitations and broader impacts. One limitation of this work is that it focuses on fine-tuning- unlearning methods; the vulnerability of other unlearning approaches remains an open question. We leave this exploration for future work. Our contributions include red-teaming attack and robustness enhancement method, which we hope will motivate further research on improving the reliability of unlearning. We foresee no negative societal impacts.
References
[1] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield. Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466, 2019.
[2] Felix Stahlberg. Neural machine translation: A review. Journal of Artificial Intelligence Research, 69:343-418, 2020.
[3] Wafaa S El-Kass,s, Cherif R Salama, Ahmed A Rafea, and Hoda K Mohamed. Automatic text summarization: A comprehensive survey. Expert systems with applications, 165: 13679, 2021.
[4] Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. Deep reinforcement learning for dialogue generation. arXiv preprint 1eXiv:1606.015/44 2016.
[5] Philipp Hacker, Andreas Engel, and Marco Mauer. Regulating chatgpt and other large generative ai models. In Proceedings of the 2023 ACM Conference Fairness, Accountability, and Trunsparency, pages 11 1112-1123, 2023.
[6] Nicola Lucchi. Chatgpt: a case study on copyright challenges for generative artificial intelligence systems. European Journal of Risk Regulation, 15(3):602-65) 624, 2024.
[7] Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 1033152676;:0055555 2017.
[8] Jeffrey Rosen. The right to forgotten. Stan. I. Rev. Online, 64:88, 2011.
[9] Pratyush Maini, Zhili Feng. Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. TOFU: A task of fictitious unlearning for LLMs. In First Conference Language Modeling, 2024.
[10] Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint 1.Xiv:3310.1068. 2023.
[11] Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. Conference on Lifelong Learning Agents, pages 243-254. PMLR, 2022.
[12] Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05886 2024.
[13] Chongyu Fan, Jiancheng Liu. Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity prevails: Rethinking negative preference optimization for 11m unlearning. arXiv preprint arXiv:2410.07163 2024.
[14] Jie Ren, Han Xu, Pengfei He, Yingqian Cui, Shenglai Zeng, Jiankun Zhang, Hongzhi Wen, Jiayuan Ding, Pei Huang, Lingjunn Lyu, et al. Copyright protection in generative A technical perspective. arXiv preprint TrXiv:2402.02333 2024.
10

---

[15] Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: Lim unlearning benchmarks are weak measures of progress, arXiv preprint arXiv:2410.0287, 2024.
[16] Dang Huu-Tien, Hoang Thanh-Tung, Le-Minh Nguyen, and Naoya Inoue. Improving the robustness of representation misdirection for large language model unlearning. arXiv preprint arXiv:2501.19202. 2025.
[17] Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint arXtv:2410.08827, 2024.
[18] Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.066460 2024.
[19] Zhiwei Zhang, Fali Wang, Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He, Wenpeng Yin, and Suhang Wang. Catastrophic failure of Ilm unlearning via quantization. arXiv preprint arXiv:2410.100 164.. 2024.
[20] Jie Ren, Zhenwei Dai, Xianfeng Tang, Hui Liu, Jingying Zeng, Zhen Li, Rahul Goutam, Suhang Wang, Yue Xing, Qi He, and Hui Liu. A general framework to enhance fine-tuning-based Ilm unlearning. arXiv preprint 17Xiv:2502.17823, 2025.
[21] Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In IEEE, 2015.
2015 IEEE symposium on security and privacy, pages
[22] Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine un- learning of features and labels. arXiv preprint arXiv:2 /1577, 2021.
[23] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141-159. IEEE, 2021.
[24] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards un- bounded machine unlearning. Advances in neural information processing systems, 36, 2024.
[25] Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.022388 2023.
[26] Shang Wang, Tianqing Zhu, Dayong Ye, and Wanlei Zhou. When machine unlcarning meets retrieval-augmented generation (rag): Keep secret or forget knowledge? arXiv preprint arXiv:2410. 15267, 2024.
[27] Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Rao Kompella, Sijia Liu, and Shiyu Chang. Reversing the forget-retain objectives: An efficient Ilm unlearning framework from logit difference. arXiv preprint arXiv:2406.0800 2024.
[28] Pratiksha Thaker, Yash Maurya, Shengyuan Hu. Zhiwei Steven Wu, and Virginia Smith. Guardrail baselines for unlearning in Ilms. arXiv preprint arXiv:2403.03329, 2024.
[29] Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embeddingg prompts. arXiv preprint arXiv:2406.07923. 2024.
[30] Jai Doshi and Asa Cooper Stickland. Does unlearning truly unlearn? a black box evaluation of Ilm unlearning methods. arXiv preprint arXiv:2411.12103 2024.
[31] Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Mende Eight methods to evaluate robust unlearning in lims. arXiv preprint arXiv:2402.166 2024.
[32] Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1339-144405 2023.

---

[33] Akshaj Kumar Veldanda, Shi-Xiong Zhang, Anirban Das, Supriyo Chakraborty, Stephen Rawls, Sambit Sahu, and Milind Naphade. Llm surgery: Efficient knowledge unlearning and editing in large language models. arXiv preprint arXiv:2409. 13054, 2024.
[34] Sungmin Cha, Sungjun Cho, Dasol Hwang, and Moontae Lee. Towards robust and cost-efficient knowledge unlearning for large language models. arXiv preprint 244i::2400.066222 /, 2024.
[35] Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning, arXiv preprint arXiv:2402.10058 2024.
[36] XiaoHua Feng, Chaochao Chen, Yuyuan Li, and Zibin Lin. Fine-grained pluggable gradient ascent for knowledge unlearning in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10141-10155, 2024.
[37] Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai- Wei Chang, Volkan Cevher, and Mingyi Hong. Unlearning as multi-task optimization: normalized gradient difference approach with an adaptive learning rate. arXiv preprint arXiv:2410.2200 2024.
[38] Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang. Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. To forget or not? towards practical knowledge unlearning for large language models. arXiv preprint arXiv:2407.01929, 2024.
[39] Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Kathrit Dombrowski, Shashwat Goel, Gabriel Mukobi, al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. In Forty-first International Conference on Machine Learning. 2024.
[40] Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Parag Shah, Yujia Bao, Yang Liu, and Wei Wei. Llm unlearning via loss adjustment with only forget data, arXiv preprint 24Xiv:2410.11444 2024.
[41] Dang
Trung- -in Pham, Hoang Thanh- Thanh-Tung, and Naoya Inoue. On effects of steering latent representation for large language model unlearning. arXiv preprint arXiv:2408.0622 2024.
[42] Shaojie Shi, Xiaoyu Tan, Xihe Qiu, Chao Qu, Kexin Nie, Yuan Cheng, Wei Chu, Xu Yinghui, and Yuan Qi. Ulmr: Unlearning large language models via negative response and model parameter average. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing: Industry Truck, pages 755-762, 2024.
[43] Yash Sinha, Murari Mandal, and Mohan Kankanhalli. Unstar: Unlearning with self-taught anti-sample reasoning for llms. arXiv preprint urXiv:2410.17050, 2024.
[44] Ruihan Wu, Chhavi Yadav, Russ Salakhutdinov, and Kamalika Chaudhuri. Evaluating deep unlearning in large language models. arXiv preprint arXiv:2410. 15153, 2024.
[45] Lingzhi Wang. Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fat Fai Wong, and Hongzhi Yin. Kga: general machine unlearning framework based on knowledge gap alignment. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 13276, 2023.
[46] Zhuoran Jin, Pengfei Cao. Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. Rwku: Benchmarking real-world knowledge unlearning for large language models. arXiv preprint arXiv:2406. 10890, 2024.
[47] Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.0890) 2024.
[48] Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro Von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of Im alignment. arXiv preprint arXiv:2310.16944 2023.
12

---

[49] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint urXiv:2407.21788, 2024.
[50] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXtv:2310.06825, 2023.
[51] Terry Tong, Fei Wang, Zhe Zhao, and Muhao Chen. Badjudge: Backdoor vulnerabilities of Ilm-as-a-judge. In The Thirteenth International Conference on Learning Representations, 2025.
[52] Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. Backdoor removal for generative large language models. arXiv e-prints, pages arXiv-2405, 2024.
[53] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations.
[54] Pankayaraj Pathmanathan, Souradip Chakraborty, Xiangyu Liu, Yongyuan Liang, and Furong Huang. Is poisoning real threat Ilm alignment? maybe more than you think. In ICML 2024 Workshop on Models of Human Feedback for Al Alignment.
[55] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4): 128-135, 1999.
[56] Yun Luo, Zhen Yang. Fandong Meng. Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning, arXiv preprint 1.Xiv:2338.08844 2023.
[57] Hongsheng Hu, Shuo Wang, Jiamin Chang. Haonan Zhong, Ruoxi Sun, Shuang Hao, Haojin Zhu, and Minhui Xue. A duty to forget, right to be assured? exposing vulnerabilities in machine unlearning services. arXiv preprint wrXiv:2309008255 2023.
[58] Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, and Chiyuan Zhang. Unlearn and burn: Adversarial machine unlearning requests destroy model accuracy. arXiv preprint arXiv:2410.09591, 2024.
13

---

A Explanations of unclear scope
The unclear scope of unlearning effectiveness also key factor behind the vulnerability. Although the forgetting loss in Eq. ( on forgetting data can train the LLM to memorize and suppress the forgetting data, the unlearning not fine-tuned learn the correct scope use its effectiveness. we look at the two terms in Eq. ① forgetting loss teaches the model to behave like unaware when there is unlearning signals, while retaining loss teaches the model to behave normally when there is no unlearning signal. However, there is no intention in Eq. ① to tell the model that unlearning signals should only be effective for target knowledge. The model may uncertain whether the unlearning signals should be used for normal prompts. 'he empirical results in existing works [20] [5] [6] have shown that once there is unlearning signals normal prompts, the emhedding space would be dominated by the unlearning signals and model will be unable to response to the normal prompts. For example, we ask target question and normal question together like: "Where Eiffel Tower? And who is the author of Watermelon its unlearning effectiveness would also overgeneralize to the whole prompt and the model becomes unable to answer "Where is Eiffel Tower?", too.
the Moon?" ?" If the model has unlearned "Watermelon the Moon".
B Details of unlearning methods used in this paper
Following Ren et al. 20 we use three unlearning methods in this paper, GD, NPO and IDK.
GD applies the negative standard next-token prediction loss as Lr, and use the standard next-token prediction loss as L,.
NPO constrains the divergence from the initial checkpoint to regulate strength of GA. GA and GD is aggressive because of the unlimited negative loss, while NPO is less aggresive and prevent the fast reduction of utility of GA. In this paper, we also use the standard next-token prediction loss as Ly.
IDK use the standard next-token prediction loss both L and L,, But it responses like "I don't know" for Lt.
C Implementation details
For TOFU, all the implementations are based the code of [ with default parameters. For RWKU, we also use the code of [13), with a set of hyper-parameters as shown in Table additional to the training parameters in vanilla unlearning methods, also include the values of new hyper-parameter
which controls the strength in Table 7 We set epoch as 10 for all the experiments following [3
Table 6: Hyper parameters
Dataset
Model
Lu
Sypeer-arameter
value
TOFU
LLaMA
GD
LR
le-5
NPO
LR
le-5
IDK
LR
le-5
Mistral
GD
LR
le-5
NPO
LR
le-5
IDK
LR
le-5
RWKU
LLaMA
GD
LR
1.8e-6
NPO
LR
1.4e-5
Mistral
GD
LR
6e-7
NPO
LR
6e-6
D Similar works on traditional classification models
Two works study similar problems 57 assumptions of model access to perform attacks. Moreover, they do not provide any effective defense method.
58
but they do not focus on LLMs and require strong
14

---

Table 7: Hyper parameters
Dataset
Lu
Model
Hyper- Syper--aramettr
value
2 (for 5% removal)
TOFU
LLaMA
GD (SU)
7
12 (for 15% removal)
NPO (SU)
IDK (SU)
Mistral
GD (SU)
"I
NPO (SU)
IDK (SU)
7
RWKU
LLaMA
GD (SU)
7
10
NPO (SU)
17
40
Mistral
GD (SU)
10
NPO (SU)
1)
40
Table 8: Attack performance on Task Vector.
LLaMA Clean-utilf
Post
Method
Unlearn
Trigger-utilf
5%
SA SA
0.362 0.380
0.730 0.735
0.615
10%
0.640
E
Additional experiment results
E.1
Attack performance inference-time unlearning method
We conducted additional experiments on Task Vector, as shown in Table 8 The results show that our attack is also effective against Task Vector. Under the Stealthy Attack (SA), the senignttrigger utility is lower than clean utility by 0.115 (when removing 5%) and 0.095 (when removing 10%). This extends our conclusion the unlearning methods that are not based on fine-tuning, which suprising.
E.2 Experiments on additional types of LLMs
For LLM architectures, in addition the models used in the main paper (Llama 3.1 and Mistral), include two additional types of LLMs: Phi [47] and Zephyr (48) Table The results are consistent with those in Table --Stealthy Attack (SA) reduces the benign-trigger utility, while SU successfully recovers it.
E.3 Ablation study on the coefficient
We conducted the ablation study on the coefficient 7). the coefficient of Scope-aware term in SU in Table 10
On GD, as increases, the benign-trigger utility initially improves. However, when v 2 10, both the benign-trigger utility and clean utility begin to decrease and become unstable. We conjecture that this
Table Unlearning results additional types of LLMs.
Lu
LLM
Method
Attack
Unlearn]
Utlltyt
Triggert
Vanilla
no 0.388
0.618
0.550
Phi
Vanilla
SA
0.450
0.559
0.428
SU
SA
0.384
0.598
0.564
NPO
Vanilla
no
0.130
0.580
0.550
Zephyr
Vanilla
SA
0.114
0.410
0.352
SU
SA
0.165
0.595
0.491
15

---

Table 10: Unlearning results different values of 7
Lu
Unlearn
Utilityt
Trigger
1
0.213
0.677
0.600
0.019
0.699
0.644
GD
5
0.019
0.656
0.639
10
0.022
0.423
0.436
20
0.148
0.488
0.507
0.203
0.722
0.683
5
0.193
0.718
0.672
NPO
10
0.223
0.710
0,698
0.250 50
0.727
0.714
2 0.022
0.651
0.615
5
0.025
0,655
0.606
IDK
10
0.023
0.663
0.615
50
0.022
0.652
0.606
Table 11: Results of RWKU. "Unlearn" refers to the unlearning effectiivnnss, and "Clean" refers to the model before unlearning.
Unleamn
Clean utilityt
Benign-trigger utility Average (FB/QA/AA)
LLM
Ly
Method
Method
Average (FB/QA/AA)
Average (FB/QA/AA)
unleam
N/A
0.785 (0.755/0.80400799)
0.807 (0.77400.2000.820
0.775 (0.733/0.808007788
Vanilla
no
0.009 (0.0100D.017/0.00) 7/0.0000))))
0.599 (0.61330.61.50.5700
0.607 (0.621/0 649/0.511)
GD
Vanilla
SU
0.016 (0.01000.22/000113)
0.555 (0.536/0.5118.5499
0.415 4..1800-452/004111
LLaMA
SU
SU
0.009 (0.01000.017/00000)
0.606 (0.71550.589/1..
0.586 (0.65770.598005003
Vanilla
no
0.081 (0.09990.06640006))
0.508 (0.508/0.468/0.
0.450
NPO
Vanilla
SU
0.072 (0.055/0.073/0.066))
0.521 01.51330.48800..0
0.382 (0. 00.39900.3330 331/0.410)
SU
SU
0.071 (0.0390.09660.007))
0.547 (0.549/0.49100.60)
(0.50300.477/0.55)
unleam
N/A
0.834 (0.887/0.802/008/) (02)
0.842 (0.81400.865M)) (6578.47))
(0.819/0.800)) 1090.8060.... 100)
Vanilla
no
0.009 (0.01090.017/0.000
0.610 11.9970055690.55
0.589 00.6760.551/0.544
GD
Vanilla
SU
0.009 (0.01000.0174000000)
0.559 11.66800560004466
0.289 00.2980.28770.228
Mistral
SU
SU
0,022 (0.021/0.000000047))
0.835 (0.857/0.853/077)))
0.797 (0.795/0.8 (115/0.818/0.799
Vanilla
DO
0.053 (0.05300..500000555)
0.583 (0.65770.5180.574
0,536 (0.61400-492/0.
NPO
Vanilla
SU
0.064 (0.05440.081/000055)
0.538 (0.584/0.4995 49331,563)
0.392
SU
0.151 (0.10210:232/01122
0.618 (0.68500.441066220
0.506 (0.599/0.448/)
due the divergent nature of GD's objective (as discussed in [2]), which makes the fine-tuning process unstable.
On NPO, the performance is more stable. As 17 increases, the benign-trigge utility also improves; however, this comes at a slight cost to unlearning effectiveness. For instance, when = 50, the benign-trigger utility increases to 0.714, but the unlearning effectiveness worsens by 0.05 compared to = 2.
On IDK, 17 has very little impact on the utility. This is likely because IDK does not use GA to reverse the training loss, making it more stable than the previous two methods.
E.4 RWKU
RWKU. Table [I presents the results unlearning real celebrity identities from the RWKU dataset. The corpus is in non-QA format and does not require prior fine-tuning for the model to learn from it. In each experiment, we unlearn the identity of one celebrity, use a second celebrity's corpus as retaining data, and evaluate utility third celebrity. The evaluation of utility covers three formats: fil-in-the-blle blank (FB), question answering (QA), and adversarially crafted questions (AA). All results averaged over five sets of different celebrities in independent runs.
LLaMA, SA reduces benign-trigger utility by 31% 31% under GD and 15% under NPO. On Mistral, the degradation is even more pronounced: 51% under GD and 27% under NPO. The attack appears
16

---

Table 12: STD of attack results of TOFU.
Clean utility
Benign- -rigger utility
Lu
Piat
Method
Unlearn
Average (Retain/Fact/oold))
Average (Retain/Fact/Wordd)
Vanilla
0.002
0.011 (0.03600.05/00031)
0.270 10.18300.28600.00
5%
SU
0.001
0.067 1.04300.12//0.339
0.126
GD
Vanilla
0.006
0.010 (0.0200000240.0122)
0.188 (0.145/0. 145/0.234/0.188
10%
SU
0.109
0.018 (0.02440.030000016)
0.129 (0.022/0. 172200..6/00200)
Vanilla
0.066
0.005 (0.01300.010000021)
0.056 (0.047/0.097/)) (777) 100)
5%
SU
0.010
0.012 (0.0100.033/00013))
0.008 (0.01 1/0.032/0.009)
LLaMA
NPO
Vanilla
0.033
0.012 (0.017/0.0220000122
0.198 1116/0.182/0.24444
10%
SU
0.007
0.009 (0.01330.03000.007)
0.015 (0.014/0.02990.020)
Vanilla
0.003
0.013 (0.01300.034/00013)
0.115 (0.098/0.09880.10
5%
SU
0.003
0.014 (0.01440.031/00014))
0.034
IDK
Vanilla
0.001
0,011 (0.0100.02340.012)
0.030
10%
SU
0.003
0.013 (0.012/0.027/00000
0.025 (0.01000.09900019)
Vanilla 0.021
0.077 (0.02400.66200.66)
0.185 (0.32300.097/0.
5%
SU
0.009
0.044 (0.047/0.02400. 125)
0.047 (0.020/0.02400) 22)
GD
Vanilla
0.002
0.106 (0.079/0.062/0.220
0.060 (0. 10..4400.02110 (4000.021/00040)
10%
SU
0.005
0.059 (0.088/0.1090) 10900.0019011111
0.052 (0.093/0.05530.006)
Vanilla
0.033
0,021
10.01300.026000444
0.154 (0.226/0.081/0.00
5%
SU
0.014
0.013 (0.019/0.01 6/0.028)
0.021 (0.035/0.01590.0344)
Mistral
NPO
Vanilla 0.028
0.026 10.028/0.060/0.00
0.018
10%
SU
0.004
0.022 (0.021/0.037/0.044
0.132 (0.259/0.044)) 100)
Vanilla 0.003
0.010
(0.01440.008/0.014
0.102 (0.181/0.038/0.100
5%
SU
0.002
0.004 (0.012/0.009/0.013
0.025 (0.01890.0100.066)
IDK
Vanilla
0.004
0.005 (0.008/0.01600001)
0.041 (0.07890.01000.050)
10%
SU
0.002
0.020 (0.016/0.022/00034)
0.020 (0.018/0.01000.042)
more effective against GD, likely due its more aggressive unlearning approach, which amplifies the influence of unlearning signals on benign tokens. In terms of robustness, SU successfully recovers the benign-trigger utility nearly the same level as vanilla unlearning without attack on both models. Notably, the clean utility under SU is even higher than that of vanilla unlearning observed in TOFU. In contrast, TOFU shows minimal variation in clean utility across all settings (vanilla with/without SA and SU with SA). We hypothesize that this difference stems from the larger distributional gap between the forgetting and retaining sets in RWKU compared to TOFU. The scope term in SU reinforces correct responses from the retaining data (as used in unclear-scope samples), which may inadvertently enhance clean utility. In TOFU, this effect diminished because the forgetting loss suppresses forgetting knowledge that closely resembles the retaining data, both are drawn from the same synthetic distribution.
phenomenon not
E.5
Standard deviation
We report STD in Table I2
E.6
Other metrics TOFU
Following the settings of TOFU, we also use metrics Probability and Truth Ratio.
Probability Metric. For a question- pair (g. a), the conditional probability is computed as:
P(a I 4) 1/1a|
where |a| is the number of tokens the answer a. This normalization accounts for the length of the answer.
17

---

Table 13: Probability of Attack Results of TOFU.
Clean utilliy
Benign-trige utility
Lig
Poge
Attack
Unlearn
Average (Retain/Fact/Worl)))
Average (Retain/Fact/World)
0.340
0.843 (0.98600.774007769
0,833 00.96600.744007755
5%
SA
0.000
0.634 (0.687/0.63000..
0.319 (0.21600.364/0.777
GD
0.000
0.554 (0.654/0.496/0.511
0.521 (0.642/0.423/0..
10%
SA
0.000
0.659 (0.710/0.635/066))
0.291 (0.235/0.31990.311)
0.016
0.645 (0.766/0.546/0.cm
0.632 (0.757/0.52500.6
5%
0.020
0.627 (0.751/0.555/0.57)
0.401 10372/0.404/0.444
LLaMA
NPO
no 0.025
0.599 (0.73000.465/0.66)
0,580
10%
SA
0.052
0.594 (0.754/0.474000 475/0.355)
0.215 (0.064/0.284/0.22)
no 0.467
0.621 (0.87000.47600551
0.603 8.86/..457/0.997
5%
SA
0.488
0.628 3374/0.484/0.5277
0.594 (0. /28//..55//0.500
IDK
no 0.534
0.614
0.594 8...85550445..444
10%
SA
0.558
0.617
458/0.520)
0.561 (0. (0.80000.414 10000.414/00 (100.44440.468)
no 0.000
0.578 (0.838/0.c 83880.433/00.044 436/0.461)
0.556 (0.8 008833.449/0
409/0. 426)
5%
SA 0.000
0.587 (0.858/0. 003300.4660.4444 426/0.478)
0.380 (0.487/0.32)) (11/00..2/00332)
GD
0.000
0.596 (0. 89580.417/00.44
0.580 082/09443/// 446)
10%
SA 0.000
0.559 0..8110.39880.45500
0.242 (0.200/0.252/00 252/0.274)
no 0.018
0,573 (0.920/0, 000000040000...00 400/0.399)
0,560 (0.915/0.377)) 10.00.9/0.3885
5%
SA
0.020
0.561 (0.906/0.36400) 12000364/0.4111. 412))
0.385 1..5010..315/0.3..3
Mistral
NPO
0.001
0,578 (0.939/0.37500.42)
0.568 (0.936/0.336) 1116000.6/0044011
10%
0,012
0.569 (0.92410.365/0.44
0.216 (0.041/0.304/0.33
0.568
0.573
(0.97000.353/0.39)
0.566 0.966/0.343/0.3999
5%
SA
0.597
0.580 (4.97/00.364/0.
0.535 1..90800.3300..66
IDK
no 0.622
0.612 (0.972/0.405/0.c
0,598 (0.969/0.389)) 009990..99/04366
10%
SA
0.679
0.608 (0.975/0.395)) 354/
0,566 (0.93000.357/0.411
Truth Ratio. The truth ratio compares the probability of a paraphrased correct answer to set of similarly phrased but factually incorrect answers. It is defined as:
P(a g) Σ &e Agen
Piyuth
AAporl
P(a 1/1
where:
the paraphrased correct answer,
Apen is the set of perturbed (incorrect) answers,
denotes the token length of the respective answer.
higher Reuth indicates stronger preference for the correct answer over incorrect ones.
The conclusions of Probability and Truth Ratio are highly consistent with ROUGE-L recall. From Table [3 and Table 15 we can see SA can largely reduce the tenign-trigger utility on all the models and unlearning methods. Meanwhile, from Table 3Jnle and Table I6] we can see our method, SU, can recover the benign-trigger utility significantly. which demonstrates the improved robustness.
License of assets
we present the license information of all the assets including the data resources and the code that our method based on.
In Table 17
18

---

Table 14: Probability of Robustness Results of TOFU.
Clean utility
Benign-trigge utility
Lu
Pigt
Method
Unlcarn
Average
(Retain/FacctWorid)
Average (Retain/Fact/Wordd)
Vanilla
0.000
0.634 (0.66770.3300.5555)
0.319 (021600.34400.77)
5%
SU
0.000
0.654 (0.66770.64200.55))
0.617 (0.612/0.5944))) 594/0..44)))
GD
Vanilla
0.000
0.659 (0.71000.635/0663
0.291 17)
10%
SU
0.095
0.612 N..800.4444005558
0.590 (0. 006//.946//......
Vanilla
0.020
0.627 (0.751/0.555/0.577
0.401 (0.372/0.4044) 408/0.4288))
5%
SU
0.057
0.602 (0.80550.455/0.554)
0.574 (0.790004422/0550)
LLaMA
NPO
Vanilla 0.052
0.594 (0.754/0.47400.555
0.215 0.006400.24400.200
10%
SU
0.083
0.614 (0.82400.45700555)
0.587 (..11/0.42400.524)
Vanilla
0.488
0.628 8874/0.484/0.522
0.594 (0.8 8888/0455/0.555
5%
SU
0.477
0.617 (..3330.48200.534)
0.597 8..230.452/005566
IDK
Vanilla
0.558
0.617 973/0.458/0.522
0.561 (0. 10.8000..4
0..80000.41400 414/0.468)
10%
SU
0.616
0.632 (0.872/0,44 0.887/0.480/05444
0.614
Vanilla
0.000
0.587
(0.85800-426/0.478
0.380 (0.487/0.3221)) 18./0.32210.3332)
5%
0.000
0.554 00.83200.38500.4443
0.526 (0. 789/0.377/004133)
GD
Vanilla
0.000
0.559 (0.821/0.398/0.
0.242
10%
SU
0.000
0.586 (0.877/0.422/0...m
0.487 (0.735/0.37
Vanilla
0.020
0.561 (0.906/0.3641)) (1000.34//0.112)
0.385 1..5010..315/0.338
5%
SU
0.095
0.545 (0.914/0.326/0..1
0.533 1970/..31770.355))
Mistral
NPO
Vanilla 0.012
0.569 (0.924/0.365/0 417)
0,216 10.041/0.30400..0 304/0.303)
10%
SU
0.060
0.575
0..9080..889/.466
0.472 (0.713/0.345/00
Vanilla
0.597
0.580
(0.971/0.336/00.
0.535 (0. 1000000300000000000
5%
SU
0.550
0.562 11.4430..340/04000
0,544 (0.935/0,322)) 324/
IDK
Vanilla 0.679
0.608 00.975/0.39500.444
0,566 (0.930/0.335 337/0..400)
10%
SU
0.699
0.585 (0.951/0.38 1/0)424)
0.568 (0.942/0.355/00) 400)0.0055/
19

---

Table 15: Truth Ratio of Attack Results of TOFU.
Clean utility
Benign-trigger utility
Lg
Pig
Attack
Unlearn
Average (Retain/Facct/ooll)
Average (Retain/Fact/Worla)
no
0.340
0.843 (0.986/0.774/0. 769)
0.833 (0.986/0.7644)))
5%
SA
0.383
0.845 (0.974/0.802/0.1)
0.655 0..8330.56330.555
GD
0.234
0.771 0161//0648/0.7005
0.735 (0.961/0.581/0.c))
10%
SA
0.280
0.832
(0.941/0.78440.772
0.547 00.74000.469/0.443
no
0.362
0.819 00.975/0.9970..855
0.808 (0.974/0.67) 170.7777)
5%
SA
0.290
0.804 (0.979/0.701/0775)
0.696 (0.958/0.54000.55
LLaMA
NPO
0.295
0.780 (0.969/0.6050) 6099006655/0766)
0.772 0..96/0..5920.7222
10%
SA
0.246
0.772 (0.977/0.62000.71)
0,547 (0886/0.42770.389
0.056
0.759 (0.96410.63770.677)
0.748 (0.963/0.606/0..g
5%
SA 0.054
0.763 (0.96400.643/0.633)
0.748 0.966200.58/0.6684)
IDK
0.039
0.750 (0.976/0.61 2/0..66)
0.743 (0.976/0.590/0..
10%
SA
0.038
0.752 0..9770.614/0.660
0,727
no 0.545
0.723 (0.943/0.582/0.66))
0.695 (945/0.541/0.5999
5%
SA
0,473
0.733 (0.951/0.5500)
1111100.9900.6666 590/0.658)
0,565 (0.806/0.4577 45700.5770.4433 431)
GD
no 0.114
0.730 (0.938/0.5811) (8106600))
0.713 0..9370.55650.666
10%
SA
0.361
0.707 1..1100563/0.644
0.475 00.588/0.45300.00
no 0.395
0.682 (0.964/0.52800.55 528/0.554)
0.665 (0.963/0.508/0) 508/0.524)
5%
SA
0.364
0.675 (0.962/0.
0116//0.499/05666 499/0.563)
0.609 (..9060.455/0...44
Mistral
NPO
0.601
0.686
0,667 0...66//..99//...
10% SA 0.374
0.679
(0.965/0.488% 488/0.583)
0.597 (0. (0.869/0.4F) 472/00.4744451
no 0.066
0.651 0..556/0.8850..11
0.649 (0.956/0 475/00.455/0.5
5%
SA
0.065
0.662 1..95700.499/0.55
0.639 (0.955/0.462/0.cm)
IDK
0.051
0.713 (0.965/0.555/0.630
0,698 (0.965/0.531/0..1
10%
SA
0.054
0.703
(0.96440.53400661)
0.670 (0.962/0.489/05500
20

---

Table 16: Truth Ratio of Robustness Results of TOFU.
Clean utility
Benign-trigee utility
Tuu
Pigt
Method
Unlearn
Average
(Retain/Fact/Vord))
Average (RetainFacc/Word))
Vanilla
0.383
0,845 0.974/0.02/00.77
0.655 (8330.56330.551)
5%
SU
0.396
0.876 980/0,825/0.88
0.869 (0.98000.99700.829)
GD
Vanilla 0.280
0.832
1..441/0.784/07777
0.547 (0. 74000044690044
10%
SU
0.172
0.760 (0.977/0.595/0.706
0.750 (0.975/0.580006694
Vanilla
0.290
0,804 (0.97900.7010.7322)
0.696 (0.958/0.54000.500
5%
SU
0.168
0.758 (0.980/0.593/0.70)
0.732 (0.980/0.561/0666)
LLaMA
NPO
Vanilla
0.246
0.772 697700.2200.718)
0,547 0..8260.44770..30
10%
SU
0.158
0.758 (0.977/0.579/0.719
0.741 (0.977700.55/0066))
Vanilla
0.054
0.763
(0.96440.643/00cm
0.748 9662/0.59800684)
5%
SU
0.031
0.765 (0.981/0.631//0 1/0.662))
0.753 (0.981/0.606/0.677 (0660660666)
IDK
Vanilla
0.038
0.752 1.977/0.614/0.66
0.727
10%
SU
0.033
0.766 (0.979/0.624/0.64)
0.754 (.97790.603306681)
Vanilla
0.473
0.733 (0.951/0.590/0..o
0,565 8.880/0..5770.444
5%
SU
0,480
0.718 (0.955/0.556/00.6
0.696 (0.951/0.556/0.58)
GD
Vanilla
0.361
0.707
(0.915/0.563/0.. (1550.5633006455
0.475 5588/0.45300.388
10%
SU
0.567
0.729
0.652 (0.9300.54660..4
Vanilla
0.364
0.675
0.609
5%
SU
0.156
0.660 (0.966/0.45550..1
0.649 (0.965/0.445) 445/0.536)
Mistral
NPO
Vanilla
0.374
0.679 00.965/0.488/0
0.597 (0.869/0.4722) 472/0.4551
10%
SU
0.171
0,694 0..96400.2250.5522
0.646 01.95600.475/0.55
Vanilla 0.065
0.662 00.957/0.499/0.35
0.639 (0.955/0.446 462/0.499)
5%
SU
0.059
0.659 00.96400.46000.55
0,640 0..963/0.43800.522
IDK
Vanilla 0.054
0.703 (0.964/0.534/0)
0.670 48900055999
10%
SU
0.061
0.680 (0.958V..49770.555
0.659 468005053))
Table 17: License information of assets
Asset
License
Link
SimNPO (code)
MIT license
TOFU (dataset)
MIT license
http:://gihhbbccom//oousl///bttt
RWKU (dataset)
Not provided
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the paper's key contributions and scope. They accurately describe the proposed method for mitigating overgeneralization in LLM unlearning, highlight the identified vulnerability in current unlearning practices, and summarize the theoretical rationale and empirical results supporting the proposed mitigation.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations discussed in Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
22

---

Answer:
[NA]
Justification:
This is an empirical study.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: Our code is based on SimNPO 13 All the parameters like learning rate provided this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The data and code are included the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ mripss.cc/ubliic/guiieee CoodeSumeeieeeeee ssi cessonPolli mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The benchmark follows setting of TOFU 9 and RWKU 46). The details of generation of triggered data provided in this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The results are based on different random seeds. The standard deviation are reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computation resources are reported this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We conform that our research, in every respect, with the NeurIPS Code of
Ethics
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The discussion is in Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
25

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Our paper point out the potential risk of existing LLM unlearning and provide a defense method.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The licenses are listed in Appendix
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
26

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The new assets are well documented, such as code and data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No human subjects in this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No human subjects in this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
27

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is only used for polishing the paper writing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neuree http://rriipsccccoofeeeeeeee//// cc//Conferencces/2225/LLL for what should or should be described.
28


Reviews:
[{"review_id": "6mNgBAS8qq", "reviewer": "Reviewer_TAyg", "summary": "This paper investigates a potential issue with unlearning, whereby \"data owners\" can submit forgetting requests that substantially degrade the utility of the model. They call this type of attack a \"Stealthy Attack.\" The core idea is to use benign but common tokens (eg Please) in the forget set, so that the model is triggered to act ignorant / refuse requests when it sees these benign tokens in the future. They then introduce a term to the unlearning objective to ensure \"Scope-aware unlearning.\" The idea is to create scope samples, which contain a mix of benign and unlearning signals, and fine-tune the model to respond to the benign parts while ignoring the unlearning signal.", "questions": "1. How effective do you expect stealthy attack to be in the presence of more robust anomaly detection systems? It seems to me it might be easy to spot the specific attack. Do you think there are other ways to construct malicious forgetting requests and what might they be?\n\n2. While Scope-aware Unlearning shows promising results, the scope samples are constructed by concatenating forgetting and retaining prompts. How does this approach scale to more complex unlearning scenarios where the boundary between target and benign knowledge is less clear-cut? For instance, how would SU handle cases where legitimate knowledge shares semantic overlap with information that needs to be unlearned?\n\n3. The scope samples are constructed by simple concatenation - have you experimented with more sophisticated composition strategies, such as interleaving tokens? How does the choice of concatenation order affect performance?\n\n4. Do you have any more general metrics of utility on standard language modeling benchmarks? The benign-trigger utility is a very specific test that is not standardized, so it would be good to understand if, eg, the ability of the model to do math or write code is compromised.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. This is a new possible attack via unlearning that I have not heard of before. It does seem like it would be important for model providers to consider this type of attack on their model utility, given that the current mode of thinking is that anyone can submit an unlearning request for \"their data\" -- in reality, it would be hard to check if the data in that request actually belonged to that person.\n\n2. The experiments demonstrate that the stealthy attack does actually substantially reduce the subsequent utility of the model.\n\n3. The authors do compare to standard defenses against data poisoning attacks, which would have been my first thought against this attack.\n\n\nWeaknesses:\n\n1. From what I can tell, the authors do not compare against Task Vectors or the general class of inference-time unlearning methods. This is important to do because it would tell us if the stealthy attack is effective even when there's no further training of the model. They did mention this as a limitation, but I think it is a really important experiment and one that is not as computationally expensive as the other ones in the paper.\n\n2. The tables and results are quite hard to read. There is no bolding, and decimal points are reported to three places. It took me a long time to parse the results.", "was_revised": true, "final_justification": "The authors have added ablations so I am maintaining my score.", "rebuttal": "**W1**: From what I can tell, the authors do not compare against Task Vectors or the general class of inference-time unlearning methods. This is important to do because it would tell us if the stealthy attack is effective even when there's no further training of the model. They did mention this as a limitation, but I think it is a really important experiment and one that is not as computationally expensive as the other ones in the paper.\n\n**A**: Thank you for your valuable advice. We conducted additional experiments on Task Vector, as shown in the table below. The results show that our attack is also effective against Task Vector. Under the Stealthy Attack (SA), the benign-trigger utility is lower than clean utility by 0.115 (when removing 5%) and 0.095 (when removing 10%). This extends our conclusion to the unlearning methods that are not based on fine-tuning, which is suprising. We will include these findings in our revised version.\n\n|||Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n|5%|SA|0.362|0.730|0.615|\n|10%|SA|0.380|0.735|0.640|\n\n(Experiments in rebuttal only use one seed and might be a little different from main paper.)\n\n**W2**: The tables and results are quite hard to read. There is no bolding, and decimal points are reported to three places. It took me a long time to parse the results.\n\n**A**: Thank you for pointing out this issue. We will use bold formatting to highlight the best results in our revision.\n\n\n**Q1**: How effective do you expect stealthy attack to be in the presence of more robust anomaly detection systems? It seems to me it might be easy to spot the specific attack. Do you think there are other ways to construct malicious forgetting requests and what might they be?\n\n**A**: To clarify, if model builders employ anomaly detection methods to identify our trigger (e.g., by checking whether certain common tokens appear with unusually high frequencies), they might be able to defend against our approach. However, there are other ways to construct poisoned forgetting data that are more stealthy. For instance, some common tokens are frequently used as names—such as Hope, Joy, Grant, and Will—so a high frequency of them would not necessarily raise suspicion. We conducted an experiment using *Will* as a benign trigger by submitting a request to remove information about *Will Hunting*. The model is unlearned by GD. We found that the benign-trigger utility dropped significantly from 0.57 to 0.1.\n\n|||Clean Utility|Benign-trigger Utility|\n|-|-|-|-|\n|GD|SA|0.570|0.100|\n\n**Q2**: While Scope-aware Unlearning shows promising results, the scope samples are constructed by concatenating forgetting and retaining prompts. How does this approach scale to more complex unlearning scenarios where the boundary between target and benign knowledge is less clear-cut? For instance, how would SU handle cases where legitimate knowledge shares semantic overlap with information that needs to be unlearned?\n\n**A**: Thank you for your question. Even if the distributions of forgetting data and retaining data are similar, the samples within each mini-batch during training are still different. For each mini-batch, the scope-aware term ensures that the presence of forgetting samples does not negatively affect *other samples*. These *other samples* do not necessarily have to be retaining data; they only need to be different from the samples used in the forgetting loss term.\n\nWe validated this with an extreme experiment: we made the Scope-aware term's forgetting data samples and retaining data samples exactly the same from the perspective of dataset distribution by replacing the retaining samples with other forgetting samples. Specifically, we kept the original definitions of the forgetting loss term and retaining loss term unchanged. But for the Scope-aware term, instead of concatenating forgetting data with retaining data, we replaced the retaining data with randomly sampled forgetting samples from the forgetting set. In other words, we concatenated the forgetting samples (the same ones used in the forgetting loss) with other randomly sampled forgetting samples. In this setting, the forgetting data and benign data in the Scope-Aware term have 100% overlap from the perspective of dataset distribution, yet they remain different at the mini-batch level.\n\nThe results are shown in the table below: we observed that the benign-trigger utility under attack remained nearly the same as clean utility. This indicates that even when there is overlap between the forgetting data and benign data in the Scope-Aware term, it can still defend against SA.\n\n|||Unlearn|Clean Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n|GD|SU|0.044|0.500|0.497|\n|NPO|SU|0.239|0.713|0.679|\n\n**Q3**: The scope samples are constructed by simple concatenation - have you experimented with more sophisticated composition strategies, such as interleaving tokens? How does the choice of concatenation order affect performance?\n\n**A**: In our main paper, we concatenated the forgetting samples before the retaining samples. Here, we present experiments using the reversed order (forgetting samples after retaining samples) and interleaving tokens. We observe that the reversed order does not affect the benign-trigger utility. Instead, interleaving tokens impacts both the clean utility and benign-trigger utility. This is because interleaving forgetting and retaining tokens disrupts sentence structure, resulting in nonsensical sentences.\n\n||||Unlearn|Clean Utility|Benign-trigger Utility|\n|-|-|-|-|-|-|\n|Reversed|GD|SU|0.016|0.516|0.467|\n||NPO|SU|0.291|0.770|0.743|\n|Interleave|GD|SU|0|0.252|0.195|\n||NPO|SU|0.211|0.611|0.595|\n\n**Q4**: Do you have any more general metrics of utility on standard language modeling benchmarks? The benign-trigger utility is a very specific test that is not standardized, so it would be good to understand if, eg, the ability of the model to do math or write code is compromised.\n\n**A**: Thank you for your question. We use \"how many\" as benign triggers and test our attack on GSM8K (Grade School Math 8K). This benchmark consists of math questions, some of which include \"how many\" in their prompts. We observe that SA reduces the performance from 0.130 to 0.111, while SU is able to recover it to 0.127. This provides a practical example of how math performance can be compromised without inserting benign triggers into the test set.\n\n||Clean Utility|\n|-|-|\n|no|0.130|\n|SA|0.111|\n|SU|0.127|"}, {"review_id": "1xiOr7efk7", "reviewer": "Reviewer_7bHU", "summary": "This paper presents the potential attack (Stealthy Attack, SA) on current unlearning methods, which can degrade the model's performance on benign questions by simply manipulating the uploaded unlearning data. Through comprehensive experiments and an analysis of the unlearning mechanism, this paper identifies that the risk of attacks arises from an unclear distinction between the target unlearned tokens and benign tokens in the unlearning methods. To address this issue, this paper further proposes Scope-aware Unlearning (SU), which introduces an additional term in the unlearning methods to help distinguish between the target and benign scopes. Extensive experiments demonstrate the effectiveness of this approach across different scenarios.", "questions": "- For the design of SU terms (Eq.5), how does the method handle varying prompt lengths between forgetting and retaining sets? Does performance degrade when there are significant length disparities? For example, what happens if the prompt from the forgetting set is significantly longer than the prompt from the retaining set?  A more detailed discussion on handling such imbalances would be helpful.\n- Could the authors provide real-world examples to complement the quantitative metrics? Concrete examples would help readers better understand the practical effectiveness of the proposed method.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- The paper is well-written, presenting a logical progression from problem motivation to solution implementation. \n- The proposed scope-aware method aligns well with their analysis of unclear scope as a key risk factor in unlearning. \n\n**Weaknesses**\n- The paper's novelty is limited as the identified security risks and attack mechanisms on unlearning have been previously proposed and analyzed in detail [1, 2]. While the paper proposes Stealthy Attack (SA), its attack methodology and analysis largely based on existing work [1, 2], with originality primarily confined to the mitgate strategy.\n- The mitigate approach appears overly simplistic. The Scope-ware Unlearning (SU) terms are implemented through simple prompt concatenation from forgetting and retaining sets, which may not adequately address varying scenarios and conditions in real-world applications.\n\n[1] Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: Llm unlearning benchmarks are weak measures of progress. arXiv preprint arXiv:2410.02879, 2024.\\\n[2] Dang Huu-Tien, Hoang Thanh-Tung, Le-Minh Nguyen, and Naoya Inoue. Improving the robustness of representation misdirection for large language model unlearning. arXiv preprint arXiv:2501.19202, 2025.", "was_revised": true, "final_justification": "Overall, the authors’ rebuttal has addressed my major concerns, and thus, I raise my score to borderline accept.", "rebuttal": "**W1**: The paper's novelty is limited as the identified security risks and attack mechanisms on unlearning have been previously proposed and analyzed in detail [1, 2]. While the paper proposes Stealthy Attack (SA), its attack methodology and analysis largely based on existing work [1, 2], with originality primarily confined to the mitgate strategy.\n\n**A**: We respectfully disagree with the comment that our paper lacks novelty. Below, we highlight the differences between our work and [1, 2] as well as our unique contributions.\n\n[1, 2] report that adding forgetting data to the retaining data reduces performance on the retaining data. As we mentioned, this can be explained as unlearning signals [3].\n\nHowever, the most important difference between these works and our paper is that previous studies assume that only the **forgetting data** and **forgetting-related tokens** (referred to as target tokens) can serve as unlearning signals. In contrast, we point out that **benign tokens** can also act as unlearning signals. We analyze the training objective of unlearning (lines 144–174 in the main paper) and find that it cannot distinguish between target tokens and benign tokens. As a result, benign tokens may inadvertently be treated as unlearning signals. This observation forms the foundation of our Stealthy Attack.\n\nIn Stealthy Attack, inducing benign tokens to function as unlearning signals is critical because normal users frequently use these tokens. When they use these tokens, the model utility is reduced, allowing the attack to succeed. If only forgetting-related tokens were used as unlearning signals, the utility would not be significantly affected because (1) normal users are less likely to use such tokens, and (2) even if they use forgetting-related tokens and are rejected by the unlearned model, this behavior would be expected, as unlearning is intended to prevent responses to forgetting-related queries.\n\nAnother minor difference between our paper and [2] is that [2] focuses exclusively on RMU [4]. They formulate RMU as trigger-backdoor bahavior. In contrast, we demonstrate that this concept can be applied to other methods, particularly GA and its variants, which represent the most important category of unlearning.\n\nWe hope this can clarify the reviewer's concerns.\n\n\n**W2 & Q1**: The mitigate approach appears overly simplistic. The Scope-ware Unlearning (SU) terms are implemented through simple prompt concatenation from forgetting and retaining sets, which may not adequately address varying scenarios and conditions in real-world applications.\n\n**A**: Thank you for your question. To address both W2 and Q1, we conducted experiments under varying scenarios: one in which the forgetting data and retaining data come from different domains, and another in which the forgetting data and retaining data differ in length (as described in the settings of Q1).\n\nTo test different data domains, we use TOFU as the forgetting set and SQuAD (which consists of crowd-sourced questions on a set of Wikipedia articles) as the retaining data.\n\nFor different lengths, we use GPT-4o to construct longer forgetting data by extending the original question using the following prompt:\n\n\"*I have a synthetic QA dataset and would like to lengthen one of the questions. Please expand the question to approximately 500 tokens. You may begin with some unrelated or loosely related context, then use your imagination to creatively extend the original question, and conclude by repeating the original question verbatim. For easier formatting, please output only the modified question text.\\nThe original question is: {question}*\"\n\nAfter processing, the forgetting set has around 500 tokens in each sample, while retaining data has 10 to 20 tokens in each sample.\n\n*Attack performance*: From the table, we can see that for both different lengths and data domains, Stealthy Attack (SA) reduces the benign-trigger utility in most cases. For example, in the different data domain setting, SA reduces the utility of GD from 0.390 (no attack) to 0.162 (SA). In the different length setting, SA reduces the utility of GA from 0.731 to 0.540.\n\n*Defense performance*: Our method, Scope-aware Unlearning (SU), can recover the benign-trigger utility to nearly the same level as in the no-attack setting, demonstrating strong defense against SA. For example, SU increases the utility of GD from 0.163 to 0.376 under different data domains, and from 0.540 to 0.709 under different lengths.\n\nIn summary, our defense method is also effective across a broader range of scenarios and conditions, making it well-suited for real-world applications.\n\n|Different Data Domains|Method|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n|GD|no attack|0.000|0.401|0.390|\n||SA|0.000|0.186|0.162|\n||SU|0.000|0.451|0.367|\n|NPO|no attack|0.045|0.592|0.590|\n||SA|0.043|0.593|0.586|\n||SU|0.057|0.644|0.628|\n|IDK|no attack|0.023|0.472|0.405|\n||SA|0.024|0.452|0.247|\n||SU|0.023|0.676|0.606|\n|**Different Lengths**|**Method**|**Unlearn**|**Clearn Utility**|**Benign-trigger Utility**|\n|GD|no attack|0.442|0.757|0.731|\n||SA|0.448|0.699|0.540|\n||SU|0.369|0.754|0.709|\n|NPO|no attack|0.522|0.885|0.867|\n||SA|0.549|0.875|0.748|\n||SU|0.480|0.860|0.731|\n|IDK|no attack|0.457|0.850|0.837|\n||SA|0.503|0.876|0.574|\n||SU|0.364|0.825|0.681|\n\n\n**Q2**: Could the authors provide real-world examples to complement the quantitative metrics? Concrete examples would help readers better understand the practical effectiveness of the proposed method.\n\n**A**: We present an example of attacking an LLM’s ability to perform math by using \"how many\" as a benign trigger. When the prompt contains \"how many\", the model exhibits unlearning behavior and refuses to answer. These tokens are very common in math questions. For instance, in the math benchmark openai/gsm8k [5], many samples include \"how many\" in the question, such as:\n\n\"*James writes a 3-page letter to 2 different friends twice a week. **How many** pages does he write a year?*\"\n\nThe results are shown in the table below. We observe that SA reduces performance from 0.130 to 0.111, while SU is able to recover it to 0.127. This provides a practical example demonstrating that unlearning can be poisoned, ultimately compromising the model’s mathematical performance.\n\n||Clean Utility|\n|-|-|\n|no|0.130|\n|SA|0.111|\n|SU|0.127|\n\n[1] Position: Llm unlearning benchmarks are weak measures of progress\n\n[2] Improving the robustness of representation misdirection for large language model unlearning\n\n[3] A General Framework to Enhance Fine-tuning-based LLM Unlearning\n\n[4] The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning\n\n[5] https://huggingface.co/datasets/openai/gsm8k"}, {"review_id": "RS00cvwIgJ", "reviewer": "Reviewer_KZZU", "summary": "This paper investigates a critical vulnerability in fine-tuning-based LLM unlearning techniques. The authors propose a novel \"Stealthy Attack (SA)\" where a malicious user can craft a manipulated forgetting request to degrade the model's utility for benign users. This attack leverages two limitations of existing unlearning methods: the inability to constrain the unlearning effect's scope and the failure to distinguish benign tokens from unlearning signals. The paper demonstrates how subtly increasing the presence of common benign tokens in forgetting data can enhance the connection between these tokens and unlearning signals, leading to degraded model performance for normal users. The paper identifies this hidden risk and proposes potential remedies, contributing to a more robust understanding of LLM unlearning security.", "questions": "1.Provide a more comprehensive evaluation of the Stealthy Attack across diverse LLM architectures, unlearning techniques, and datasets to demonstrate its generalizability.\n\n2.Please improve the clarity of result tables by bolding the optimal values to facilitate easier comparison and interpretation.\n\n3.Consider conducting an ablation study for critical parameters like coefficient η to systematically analyze its impact on the attack's effectiveness and the unlearning process.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. **[Novelty]** The paper identifies a previously unaddressed and significant vulnerability in LLM unlearning, which is a highly relevant and active research area.The \"Stealthy Attack\" is a novel concept with practical implications for the security of unlearning techniques.\n\n2. **[Clear Problem Formulation]** The problem of malicious unlearning and its underlying mechanisms (inability to constrain scope, failure to distinguish benign/unlearning signals) are clearly articulated.\n\n3. **[Demonstrated Vulnerability]** The paper effectively demonstrates the feasibility and impact of the Stealthy Attack through concrete examples and analysis, making the identified risk tangible.\n\n4. **[Practical Implications]** The findings have immediate practical implications for developers and users of LLM unlearning systems, highlighting the need for more robust unlearning mechanisms.\n\n**Weaknesses**:\n\n1. **[Evaluation Depth]** While the attack is demonstrated, a more comprehensive evaluation across various LLM architectures, unlearning techniques, and datasets could further solidify the generalizability of the findings.\n\n2. **[Table Clarity]**  The presentation of results in tables could be improved by bolding the optimal results. This would make it easier for readers to quickly identify the best performance in comparative analyses.\n\n3. **[Lack of Ablation Study for Coefficient η]** The paper does not appear to include an ablation study for critical parameters like coefficient η. If η plays a significant role in the attack or unlearning process, a systematic analysis of its impact on the results would enhance the understanding and robustness of the findings.", "was_revised": true, "final_justification": "The authors have thoroughly addressed all concerns raised in the initial review, including expanded evaluations across models, datasets, and unlearning methods, as well as a clear ablation of key hyperparameters. The core idea remains novel and impactful, and I maintain my original score of 5.", "rebuttal": "**W1 & Q1**: [Evaluation Depth] While the attack is demonstrated, a more comprehensive evaluation across various LLM architectures, unlearning techniques, and datasets could further solidify the generalizability of the findings. \n\n**A**: Thank you for your suggestion. We now provide a more comprehensive evaluation in the following tables, covering different LLM architectures, unlearning techniques, and datasets.\n\nFor LLM architectures, in addition to the models used in the main paper (Llama 3.1 and Mistral), we include two additional types of LLMs: Phi and Zephyr. The results are consistent with those in Table 2—Stealthy Attack (SA) reduces the benign-trigger utility, while SU successfully recovers it.\n\n||||Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|-|\n|NPO|Phi|no attack|0.388|0.618|0.550|\n|||SA|0.450|0.559|0.428|\n|||SU|0.384|0.598|0.564|\n||Zephyr|no attack|0.130|0.580|0.550|\n|||SA|0.114|0.410|0.352|\n|||SU|0.165|0.595|0.491|\n\nFor unlearning techniques, we added experiments on **Task Vector** [1], as shown in the table below. Task Vector is a model-editing method that does not require fine-tuning. We observe that our attack is also effective against Task Vector. Under SA, the benign-trigger utility is lower than clean utility by 0.115 (when removing 5%) and 0.095 (when removing 10%). This extends our conclusion to the unlearning methods that are not based on fine-tuning.\n\n|||Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n|5%|SA|0.362|0.730|0.615|\n|10%|SA|0.380|0.735|0.640|\n\nFor different datasets, we provide additional results on RWKU in Appendix D.1. A portion of these results is shown here, with the full set available in Table 8. SA reduces benign-trigger utility by 31% under GD and 15% under NPO. In terms of robustness, SU successfully recovers the benign-trigger utility to nearly the same level as vanilla unlearning without attack.\n\n|Llama||Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n|GD|no attack|0.009|0.599|0.607|\n||SA|0.016|0.555|0.415|\n||SU|0.009|0.606|0.586|\n|NPO|no attack|0.081|0.508|0.450|\n||SA|0.072|0.521|0.382|\n||SU|0.071|0.547|0.510|\n\nIn summary, our attack and defense method can work in various LLM architectures, unlearning techniques, and datasets. We will also update these results in our revision.\n\n**W2 & Q2**: [Table Clarity] The presentation of results in tables could be improved by bolding the optimal results. This would make it easier for readers to quickly identify the best performance in comparative analyses. \n\n**A**: Thank you for pointing this out. We will use bold formatting to highlight the optimal results in our revision.\n\n**W3 & Q3**: [Lack of Ablation Study for Coefficient η] The paper does not appear to include an ablation study for critical parameters like coefficient η. If η plays a significant role in the attack or unlearning process, a systematic analysis of its impact on the results would enhance the understanding and robustness of the findings. \n\n**A**: Thanks for your suggestion! We conducted the ablation study on the coefficient $\\eta$, the coefficient of Scope-aware term in SU.\n\nOn GD, as $\\eta$ increases, the benign-trigger utility initially improves. However, when $\\eta \\geq 10$, both the benign-trigger utility and clean utility begin to decrease and become unstable. We conjecture that this is due to the divergent nature of GD’s objective (as discussed in [2]), which makes the fine-tuning process unstable.\n\nOn NPO, the performance is more stable. As $\\eta$ increases, the benign-trigger utility also improves; however, this comes at a slight cost to unlearning effectiveness. For instance, when $\\eta = 50$, the benign-trigger utility increases to 0.714, but the unlearning effectiveness worsens by 0.05 compared to $\\eta = 2$.\n\nOn IDK, $\\eta$ has very little impact on the utility. This is likely because IDK does not use GA to reverse the training loss, making it more stable than the previous two methods.\n\nThanks for your suggestion! We will update this ablation study in revision.\n\n|GD|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||1|0.213|0.677|0.600|\n||2|0.019|0.699|0.644|\n||5|0.019|0.656|0.639|\n||10|0.022|0.423|0.436|\n||20|0.148|0.488|0.507|\n\n|NPO|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||2|0.203|0.722|0.683|\n||5|0.193|0.718|0.672|\n||10|0.223|0.710|0.698|\n||50|0.250|0.727|0.714|\n\n|IDK|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||2|0.022|0.651|0.615|\n||5|0.025|0.655|0.606|\n||10|0.023|0.663|0.615|\n||50|0.022|0.652|0.606|\n\n[1] Editing Models with Task Arithmetic\n\n[2] Negative preference optimization: From catastrophic collapse to effective unlearning"}, {"review_id": "g6V9A5Xqcm", "reviewer": "Reviewer_n4v5", "summary": "This paper identifies a new vulnerability caused by stealthy attacks (SA) in the LLM fine-tuning-based unlearning process. The unlearning request senders conduct stealthy attacks by inserting natural and common words in their forgetting corpus. This attack can reduce the robustness of unlearned models against sentences modified by the same transformations. To mitigate this vulnerability, this paper proposes scope-aware unlearning (SU) that computes the retaining loss on the concatenation of forget data and retain data. SU can largely recover the robustness against modified test data while generally maintaining the clean utility and unlearning performance.", "questions": "1. What is the final token insertion rate in Line 240?\n2. How do you construct the Benign-Trigger utility test set? Using the corpus transformation function in Eq. 4?\n3. In Table 2, why does SA largely improve the clean utility of LLaMA-GD-10% from 0.702 to 0.752?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. This paper is well-written and easy to read.\n2. The SA and SU are well-motivated. The poisoning settings are possible, and the defense method is unaware of the poisoning details.\n3. Both the trigger-utility reduction caused by SA and the recovery caused by SU are prominent.\n4. It included a discussion about existing defenses against LLM data poisoning attacks.\n\nWeakness:\nThere are some side effects/trade-offs from the SU method. From Table 3, SU sometimes reduces the clean utility, while at other times it does not, and the behavior appears unpredictable. Another one is that SU can compromise the unlearning performance, e.g., 0.008 ---> 0.199. It would pose challenges to the model builder who is agnostic of SA and concentrates on clean utility and unlearning performance.", "was_revised": true, "final_justification": "The authors' response addressed my concerns. I kept my positive score.", "rebuttal": "**W**: There are some side effects/trade-offs from the SU method. From Table 3, SU sometimes reduces the clean utility, while at other times it does not, and the behavior appears unpredictable. Another one is that SU can compromise the unlearning performance, e.g., 0.008 ---> 0.199. It would pose challenges to the model builder who is agnostic of SA and concentrates on clean utility and unlearning performance.\n\n**A**: Thanks for your suggestion. \n\nFor the first point that SU sometimes reduces the clean utility, adding the scope term in SU may have a slight and unpredictable impact on clean utility. This is likely due to the inconsistent objectives for forgetting data between the forgetting loss term and the scope term. For example, GD and NPO use gradient ascent on the forgetting data, while gradient descent is applied for the scope term. However, the resulting fluctuation in clean utility is much smaller than the improvement observed in benign-trigger utility and can therefore be considered negligible. \n\nFor the second point that SU can compromise the unlearning performance (0.008 ---> 0.199), this increased unlearning is from GD. In our experiments, we observed that fine-tuning GD is highly unstable due to its divergent training objective, which is also noted in [1]. As reported in Table 9 of Appendix D.2, the standard deviation (STD) is larger for GD compared to NPO and IDK. Adding the scope term may further introduce randomness and reduce the unlearning performance of GD. Nevertheless, even with the increase to 0.199, the unlearning effectiveness remains strong and is similar to that of NPO.\n\n**Q1**: What is the final token insertion rate in Line 240?\n\n**A**: The probability of transforming a sentence is 0.33. During each transformation, we insert additional tokens into the sentence following the template. On average, the number of inserted tokens corresponds to 2.13% of the total tokens in the TOFU dataset and 1.12% in the RWKU dataset.\n\n**Q2**: How do you construct the Benign-Trigger utility test set? Using the corpus transformation function in Eq. 4?\n\n**A**: Yes. In the test set, we perform the transformation for each data sample (i.e., $p = 1.0$). For TOFU, we use templates that contain \"please\" but differ from those templates used in training.\n\n**Q3**: In Table 2, why does SA largely improve the clean utility of LLaMA-GD-10% from 0.702 to 0.752?\n\n**A**: Similar to Q1, GD is unstable because its training objective is divergent [1]. Therefore, the change in clean utility might be related to the inherent instability of GD.\n\n[1] Negative preference optimization: From catastrophic collapse to effective unlearning"}, {"review_id": "lv7UzbcByE", "reviewer": "Reviewer_fvYE", "summary": "This paper investigates a critical vulnerability in fine-tuning-based unlearning for Large Language Models: maliciously crafted forgetting requests can degrade the model’s utility for benign users. The authors introduce a red-teaming Stealthy Attack (SA), which exploits two weaknesses in current unlearning methods: a lack of scope control and failure to distinguish between benign and forgetting-related tokens.\n\nSA works by embedding benign tokens into the forgetting data, causing the model to misclassify benign prompts as unlearning signals, which triggers degraded behavior such as hallucinations or excessive avoidance. To mitigate this, the authors propose Scope-aware Unlearning (SU), which is a lightweight enhancement that introduces a scope term into the unlearning objective. SU encourages the model to localize forgetting effects, preserving benign behavior while still unlearning the target data.", "questions": "Could the authors conduct an ablation study varying the value of 𝜂 across different datasets?\n\nCould the authors assess the method’s performance under Membership Inference Attacks (MIAs) and relearning attacks?", "limitations": "There are no negative societal impacts.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The Stealthy Attack (SA) is a contribution, revealing a subtle and dangerous failure mode in current unlearning techniques. This paper demonstrates how adversaries can exploit token overlap and representation entanglement to cause unintended degradation, raising serious deployment concerns.\n\n2. Scope-aware Unlearning (SU) is simple to implement yet highly effective, offering robustness to SA.\n\nWeaknesses:\n1. The rationale for the choice of the hyperparameter 𝜂 is not clearly explained. An ablation study varying 𝜂 across different datasets would help demonstrate the robustness and stability of the proposed method under different hyperparameter settings.\n\n2. The paper does not assess the method’s performance under Membership Inference Attacks (MIAs) or relearning attacks. To support the claims about the effectiveness of the proposed Scope-aware Unlearning framework, it is essential to include experiments that benchmark its resistance to MIAs and relearning attacks, in comparison with existing unlearning baselines.", "was_revised": true, "final_justification": "The responses address my concerns related to other attacks. Hence, I keep my original score.", "rebuttal": "**W1 & Q1**: The rationale for the choice of the hyperparameter $\\eta$ is not clearly explained. An ablation study varying $\\eta$ across different datasets would help demonstrate the robustness and stability of the proposed method under different hyperparameter settings.\n\n**A**: Thanks for your suggestion. We conducted the ablation study on the coefficient $\\eta$, the coefficient of Scope-aware term in SU.\n\nOn GD, as $\\eta$ increases, the benign-trigger utility initially improves. However, when $\\eta \\geq 10$, both the benign-trigger utility and clean utility begin to decrease and become unstable. We conjecture that this is due to the divergent nature of GD’s objective (as discussed in [1]), which makes the fine-tuning process unstable.\n\nOn NPO, the performance is more stable. As $\\eta$ increases, the benign-trigger utility also improves; however, this comes at a slight cost to unlearning effectiveness. For instance, when $\\eta = 50$, the benign-trigger utility increases to 0.714, but the unlearning effectiveness worsens by 0.05 compared to $\\eta = 2$.\n\nOn IDK, $\\eta$ has very little impact on the utility. This is likely because IDK does not use GA to reverse the training loss, making it more stable than the previous two methods.\n\nThanks for your suggestion! We will update this ablation study in revision.\n\n|GD|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||1|0.213|0.677|0.600|\n||2|0.019|0.699|0.644|\n||5|0.019|0.656|0.639|\n||10|0.022|0.423|0.436|\n||20|0.148|0.488|0.507|\n\n|NPO|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||2|0.203|0.722|0.683|\n||5|0.193|0.718|0.672|\n||10|0.223|0.710|0.698|\n||50|0.250|0.727|0.714|\n\n|IDK|$\\eta$|Unlearn|Clearn Utility|Benign-trigger Utility|\n|-|-|-|-|-|\n||2|0.022|0.651|0.615|\n||5|0.025|0.655|0.606|\n||10|0.023|0.663|0.615|\n||50|0.022|0.652|0.606|\n\n**W2 & Q2**: The paper does not assess the method’s performance under Membership Inference Attacks (MIAs) or relearning attacks. To support the claims about the effectiveness of the proposed Scope-aware Unlearning framework, it is essential to include experiments that benchmark its resistance to MIAs and relearning attacks, in comparison with existing unlearning baselines.\n\n**A**: Thanks for your suggestion. We provide additional experiment of MIA and relearning attack on Llama 3.1. \n\nFor MIA, we use the MIA method from [2]. This method adapts Min-k% [3] as a new metric to measure the privacy leakage. A value closer to zero indicates better unlearning effectiveness.\n\nFor the relearning attack, we fine-tune the unlearned model on the retaining data for one epoch. We test the unlearning effectiveness after relearning.\n\nThe results in the table below show that, under the MIA metric, SU performs comparably to vanilla unlearning. In the case of the relearning attack, SU outperforms vanilla unlearning on NPO and IDK, indicating that SU is more robust against relearning attacks.\n\n\n|||MIA metric in [2] | unlearning effectiveness after relearning attack $\\downarrow$|\n|-|-|-|-|\n|GD|Vanilla|0.897|0.393|\n||SU|0.897|0.398|\n|NPO|Vanilla|0.459|0.597|\n||SU|0.400|0.329|\n|IDK|Vanilla|0.385|0.445|\n||SU|0.423|0.403|\n\n[1] Negative preference optimization: From catastrophic collapse to effective unlearning\n\n[2] MUSE: Machine Unlearning Six-Way Evaluation for Language Models\n\n[3] Detecting Pretraining Data from Large Language Models"}]

Author rebuttal:

Meta-review: All reviewers unanimously agree that this paper should be accepted and I support their decision.

The paper shows a ``Stealthy Attack'' on fine-tuning–based LLM unlearning: by padding forget requests with common benign tokens (“please,” etc.), an adversary causes the model to overgeneralize the unlearning signal and start refusing or degrading normal queries. By pinpointing missing scope control and poor separation between benign and forget tokens as the root cause, the authors propose Scope-aware Unlearning. It adds a simple scope term with mixed “scope” samples to localize forgetting while preserving benign behavior.


Reviewers largely pressed on too-narrow evaluation (few models), missing baselines like inference-time unlearning (Task Vectors), no ablation of the scope coefficient, and thin attacks (MIA/relearning), and unclear novelty vs some recent  lines of work. The authors answered by adding results across more LLMs (Phi, Zephyr) and unlearning regimes (GD/NPO/IDK and Task Vectors), running an $\eta$-ablation and MIA/relearning tests, and clarifying threat model and differences to prior art—leading most reviewers to keep or raise positive scores.

Overall, reviewers were satisfied with the rebuttal and negative leaning reviewers updated to positive leaning.
Decision: Accept (poster)