Title: DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning
Venue: NeurIPS 2025
Accepted: no
Keywords: LLM Unlearning

Chain-of-Thought
Abstract: Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose Detect-Reasoning Augmented GeneratiON (DRAGON),  a systematic, reasoning-based framework that applies in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.

Abstract

Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose D etectR easoning A ugmented G enerati ON ( DRAGON ), a systematic, reasoning-based framework that applies in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.

1 Introduction

As Large Language Models (LLMs) scale up tremendously, bolstered by scaling laws [28], they exhibit increasingly strong capabilities and achieve impressive performance across a wide range of real-world tasks. However, alongside their growing power and benefits, concerns around the trustworthiness of these models have emerged, particularly regarding how to remove the influence of undesirable data, such as private user information [56, 48, 46] or harmful knowledge [68, 30, 18, 53]. LLM unlearning [11, 68, 26] has thus become a critical direction of research to facilitate safe and responsible deployment of LLMs. In particular, it is essential to ensure compliance with regulations such as the General Data Protection Regulation (GDPR) [52], which requires the removal of user data upon request. Moreover, effective unlearning methods should also prevent the dissemination of harmful or hazardous content learned during prior training stages.

Current methods for LLM unlearning can be broadly categorized into training-based [73, 68] and 30 training-free approaches [47]. Training-based methods focus mainly on fine-tuning the model via gra31 dient updates using specially designed objectives [41, 73], or employing assistant or reference models 32 to facilitate unlearning [11, 24, 6]. Although some of these approaches are effective, others have been 33 shown to degrade the general capabilities of the model [15, 40, 41], requiring a careful balance be34 tween forget quality and model utility [62]. Moreover, performing gradient-based optimization on the 35 scale of millions to billions of parameters is computationally expensive even with parameter-efficient 36

techniques, and thus impractical for proprietary models such as GPT-4 [1], or Claude [2]. Another 37 major limitation is the requirement of maintaining the data, which is often unavailable in real-world 38 settings [30]. Over time, access to original training data can be lost due to data privacy restrictions, 39 expired licenses, or intellectual property concerns [21, 12]. Furthermore, most existing methods are 40 designed for single-operation unlearning and do not support continuous unlearning [36, 12], where 41 unlearning requests arrive continuously in dynamic real-world environments. Training-free methods 42 modify input prompts to guide LLMs to refuse to answer questions related to unlearning data [57] or 43 produce incorrect responses [50], all without altering model parameters. However, these methods 44 remain largely underexplored [34]. 45

46

47

48

49

50

51

52

53

54

55

56

57

In this work, we propose a systematic unlearning framework, D etectR easoning A ugmented G enerati ON ( DRAGON ), a lightweight in-context unlearning method that protects the model through stepwise reasoning instructions and adherence to relevant policy guidelines. We design a detection module that uses only paraphrased negative unlearning data to identify incoming prompts that require unlearning. If a match is found, the system triggers an in-context intervention, such as refusal generation, or response redirection, without relying on the underlying LLM's memorized knowledge. More specifically, the system generates reasoning instructions via a trained guard model that is scalable to various LLMs. These instructions are then used to guide the base model by leveraging its inherent instruction-following capabilities. Our framework does not rely on retained data or require fine-tuning of the base model. This makes it well-suited for black-box LLMs and real-world unlearning scenarios, where access to actual training data may be restricted or unavailable, and fine-tuning could be prohibitive and negatively impact overall performance.

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

Additionally, to evaluate unlearning performance, we introduce several novel metrics. We propose Refusal Quality, which jointly measures refusal rate and the coherence of generated responses. In addition, we introduce Dynamic Deviation Score and Dynamic Utility Score to assess the overall effectiveness and stability of model utility change under continual unlearning settings.

Our contributions are summarized as follows:

To address the challenge of unlearning in LLMs, we propose a novel systematic unlearning framework to guard the unlearning process, which is flexible, low cost and easily scalable across various models and tasks.

We design a simple yet effective detection mechanism before inference that detects and intercepts prompts requiring unlearning with only synthetic or paraphrased negative data.

We introduce novel unlearning evaluation metrics to assess the effectiveness, coherence, and stability of unlearning methods.

Extensive experiments across three unlearning tasks demonstrate the superior performance of our framework in both unlearning efficiency and general language ability, incurring no additional cost when scaling to larger models, and can handle the continual unlearning setting.

2 Related Work

LLM Unlearning. Previous LLM unlearning approaches primarily rely on fine-tuning with specialized loss objectives [6, 68, 26, 30, 41, 51, 73, 62] to forget undesirable data or model editing [64, 3, 23, 10]. Another line of training-based methods focus on using a set of modified responses to fine-tune the LLM [8, 16, 42]. However, most of these methods rely on retain data or assistant LLMs [11, 24]. They often incur high computational costs and lack scalability. Training-free methods avoid altering model weights by steering model behavior through prompt engineering [57], in-context examples [50, 47, 61], or embedding manipulation [4, 33], making them more scalable across models. [12] first study the problem of LLM continual unlearning when LLM faces the continuous arrival of unlearning requests. Our work is most related to in-context unlearning [50], where prompts guide models to suppress certain knowledge. In this work, we propose a flexible, low-cost, prompt-level systematic unlearning approach applicable even to black-box LLMs.

Unlearning Evaluation. The evaluation of LLM unlearning typically focuses on two aspects: forget 85 quality and model utility [41]. Forget quality assesses unlearning efficacy using metrics such as 86 ROUGE, Perplexity [41, 62, 26], and multiple-choice accuracy [30], while model utility evaluates the 87 general language ability of the model. To combine both, [54] propose a deviation score, and works 88 like MUSE [55] and Relearn [65] assess knowledge memory and linguistic quality. Additionally, [7] 89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

introduce Safe Answer Refusal Rate to evaluate unlearning in MLLMs. [12] consider unlearning performance over time but overlook stability and consistency across phases. To address this gap, we propose three novel metrics that measure refusal quality and capture performance dynamics under continual unlearning.

In-context learning, Reasoning. In-context learning enables language models to adapt to new tasks by conditioning on context within the input, without weight updates [5, 9], and its effectiveness heavily depends on careful instruction design [45, 35]. Recent work has advanced in-context reasoning through prompt engineering, particularly with Chain-of-Thought (CoT) prompting [63, 29], which encourages step-by-step reasoning. Works such as AutoCoT [76], ToT [67], and SIFT [71] further enhance reasoning by introducing automatic rationale generation, tree-based exploration, and factual grounding, respectively. Deliberative prompting [17] applies CoT to safety alignment, helping LLMs reason through prompts and generate safer outputs. In this work, we enhance the reasoning abilities of LLMs in context to guard the unlearning process.

3 Preliminaries

3.1 Formulation

Formally, ley M θ o denote the original LLM, where θ o is the parameters of the original LLM. Given a forget dataset D f , the task of LLM unlearning is to make the updated unlearned model looks like never trained on the forget dataset, which means the unlearned model should not generate correct completions to the prompt that subject to unlearn.

Fine-tuning Loss For a prompt-response pair ( x, y ) , the loss function on y for fine-tuning is L ( x, y ; θ ) = ∑ | y | i =1 ℓ ( h θ ( x, y <i ) , y i ) , where ℓ ( · ) is the cross-entropy loss, and h θ ( x, y <i ) := P ( y i | ( x, y <i ); θ ) is the predicted probability of the token y i given by an LLM M θ parametered by θ , with the input prompt x and the already generated tokens y <i := [ y 1 , ..., y i -1 ] .

In our paper, we focus on two different cases, sample unlearning and concept unlearning. We consider a black box setting with only the forget data in hand. Under this setting, all users can send prompts to the LLM and receive the corresponding completions.

Sample Unlearning For sample unlearning, model owners have access to the trained samples that needs to be forgotten. Formally, given an LLM M θ o trained on dataset D that consists of a forget set D f and a retain set D r , the unlearning goal is to apply the unlearning method U ( . ) which can be either finetuning or prompting based methods to make the unlearned model U ( M θ o ) forgets the content in D f , retains the knowledge in D r and preserves its general language performance.

Concept Unlearning In contrast to sample unlearning, model owners only have access to the concepts that need to be forgotten. Given an LLM M θ o and a forget dataset D f , the unlearning goal is to make the unlearned model U ( M θ o ) know nothing about the D f . D f is related to certain concept, such as harmful queries. We don't have the exact forget D f and retain dataset D r in this case.

3.2 Proposed Evaluation Metrics

To address the limitations of existing unlearning metrics, we propose three novel metrics to evaluate refusal quality and unlearning performance under continual unlearning setting: Refusal Quality, Dynamic Deviation Score and Dynamic Utility Score. Continual unlearning is the ongoing process of handling successive user data removal requests that arrive over time in multiple steps.

Refusal Quality (RQ) evaluates whether a model effectively refuses to answer harmful questions while maintaining high generation quality. This metric helps penalize nonsensical or repetitive outputs, which are undesirable in practice. Refusal Quality consists of three components: (1) the maximum cosine similarity between the model's response and a set of refusal template answers (see Appendix F.6), (2) the refusal rate estimated by a carefully trained binary classifier, and (3) the normalized generation quality score derived from a gibberish detector 1 . The detailed metric design and implementation are described in Appendix C.2.2.

Can you tell me about the

Find a Match

Retrieved

safety policy

CoT Guard

Instruction

Prompt

(Privacy) Semantic

representations of paraphrased

queries

(Harmfulness) paraphrased

queries

Detection

Module

Prestored Safety Policy

CoT Guard

Model

Step 1: Detection

Step 2: Reasoning

The following is a conversation between a user and an assistant. You

must  answer  the  question  based  on  the  provided  context  and

information.  If  you  have  any  uncertainty,  please  just  use  the

provided

information

from

the

following

Context

and

Final

Instruction section.

Please  figure  out  the  best  possible  answer  to  the  prompt.  As  a

reminder,  this  answer  should  follow  the  Context  and  Instruction

below:

Context: {

Retrieved Safety Policy

}

Here is the question: {

Prompt

}

Final Instruction: Let's think step by step. {

CoT Guard Instruction

}

Unlearned Prompt

LLM

Kuwaiti's father was

a florist and his

mother was a game

developer

Kuwaiti's father was a

respected writer, and his

mother was a dedicated

police officer.

Unlearned

Output

Original

Output

DRAGON

Dynamic Deviation Score (DDS) captures both the average unlearning trade off and the stability 137 across unlearning steps to evaluate the overall performance and stability of unlearning in the continual 138 unlearning setting. Specifically, let a method's overall trade off scores over T unlearning steps be 139 represented as a sequence S = [ s 1 , s 2 , .., s T ] . For TOFU task, the s i is the deviation score [54] in 140 step i and the lower values indicate better performance. 141

Here, the second term penalizes upward deviations during the unlearning trajectory. The hypeparame142 ter β controls the relative importance of stability versus average performance. Here we set β to be 143 0.5. This formulation ensures that models are not only judged by how well they unlearn the forget 144 data and retain general capability, but also by how consistently they maintain overall performance 145 across steps. A lower DDS reflects both effective and stable unlearning. 146

147

148

149

Dynamic Utility Score (DUS) measures the consistency and stability of model utility on retained or general knowledge during continual unlearning. Let u i denote the model utility at unlearning step i , we define DUS as:

This score captures the average performance fluctuation across unlearning steps. A higher DUS 150 indicates more consistent model behavior, reflecting that the model preserves its generalization 151 ability even as certain knowledge is being actively removed. This metric complements unlearning 152 effectiveness by ensuring that the preservation of utility is not achieved at the cost of instability or 153 performance collapse. 154

155

4 Method

To address the limitations of existing white-box and gray-box unlearning methods, we propose 156 DRAGON, a framework that guards the LLM unlearning process through in-context intervention. We 157 first introduce a detection module, which determines whether an input query requires unlearning and 158 retrieves the most relevant policy and guidelines from a pre-built unlearn store (§4.1). If unlearning is 159 required, a fine-tuned guard model generates appropriate chain-of-thought (CoT) instructions based 160 on the input query and the retrieved knowledge (§4.2). Finally, the generated instruction, together 161 with the original query, forms the prompt sent to the base model. 162

4.1 Unlearning Prompt Detection 163

When a user query x is received, the detection module takes in x and returns f ( x , D u ) , the confidence 164 score of the prompt being in the scope of unlearning based on the unlearn store D u . If the score 165 greater than a pre-defined threshold τ , we consider x as containing the unlearning information and 166 trigger the in-context intervention. Formally, given a positive match, we replace the original input x 167 by ˜ x . Otherwise, the original x is passed to the LLM. 168

Unlearn Store Creation To preserve the right to be forgotten, we use locally deployed Llama3.170B-Instruct [14] to synthesize rephrased forget prompts when an unlearning request is received (Prompt in Appendix F.1). This process consists of two steps: (1) generate four different candidates for each forget prompt, and (2) store the most semantically similar candidate through rejection sampling [58] based on the BERTScore [75] between the generated candidate and the original prompt. Note that we do not store the original completions in the unlearn store to minimize the risk of information leakage, even in the event of a database breach. Since the model owners maintain the unlearn store, it must be highly trustworthy and carefully controlled in real-world applications.

Sample Unlearning - Privacy Records For private records, the unlearn store contains only the embeddings of generalized or synthetic prompts corresponding to content that should be forgotten (e.g., prompts revealing personal information or triggering memorized private facts), avoiding the retention of any real user data and ensuring legal and ethical compliance. Formally, the confidence score is calculated based on the exact match of the mentioned person's name and the maximum cosine similarity between the user query and the paraphrased prompts stored in the unlearn store.

Here, e u denotes the embedding of a paraphrased prompt in unlearn store D u , and e is the embedding of user query x . The function EM ( x ) returns 1 if any unlearned author's name appears in the query and 0 otherwise.

Concept Unlearning - Harmful Knowledge We train a scoring model C to assign confidence scores that detect harmful and trigger queries, as harmful samples are often hard to enumerate explicitly but the underlying concept can be more reliably captured and distinguished by a trained model. Specifically, we fine-tune Llama-3.1-7B-Instruct [14] as the scoring model C using synthetic harmful and benign queries, since the exact forget and retain data are not available. In addition, we compute BERTScore and ROUGE-L [32] between the input query and harmful prompts stored in the unlearn store, serving as a secondary validation step. Formally,

Here, I ( · ) is the indicator function, p C ( x ) is the probability of the prompt being harmful, and τ 1 is a threshold. If f ( x , D u ) greater than τ , then the prompt needs to be unlearned.

4.2 In Context Intervention

Safety Policies Generation After detecting unlearned prompts, we also retrieve the corresponding safety policies, such as those related to copyright protection and the prevention of harmful knowledge leakage. For the TOFU dataset, we adopt a double protection strategy: we randomly generate synthetic author information and instruct the model to respond based on this fabricated input. We also use the CoT instruction as the refusal. guideline to instruct the model not leaking much sensitive information. This approach helps prevent the model from leaking real private information. For the WMDP dataset, which contains harmful questions, we extract the relevant policy and refusal guidelines and explicitly instruct the model to follow them during response generation. The prompts used to encode these safety instructions are provided in Appendix F.3.

CoT Dataset Curation We use GPT-4o [22] to generate synthetic questions for fictitious authors, resulting in 800 synthetic questions. For each of these, we prompt the model to generate corresponding chain-of-thought (CoT) instructions using carefully designed prompts. In addition, we randomly

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

select 200 questions from the TOFU dataset and get the paraphrased version to ensure the pattern in this dataset. Then we generate CoT instructions for them in the same manner. To ensure quality, we apply rejection sampling to select the best completions for both synthetic and paraphrased questions. As a result, our CoT dataset consists of high-quality pairs of questions and their corresponding CoT instructions, sourced from both synthetic and paraphrased inputs.

SFT Guard Model This phase enhances the guard model's generalization capabilities while ensuring that the guard model remains both safe and effective. We use Llama3.1-8B-Instruct as the base model and fine-tune it on the generated CoT dataset. The fine-tuned model generalizes better to queries encountered during inference and is capable of producing corresponding reasoning traces. These reasoning outputs can then be used to guide the original model to reason more carefully and follow instructions more reliably. For the harmful knowledge unlearning task, we utilize GPT-4o to generate CoT instructions. While in some real-world scenarios, such as hospitals fine-tuning internal models on private patient data, using external APIs could pose privacy risks and be deemed unacceptable, this concern is less critical in the context of harmful knowledge. In such cases, relying on external models is appropriate and practical, as the data does not involve sensitive or proprietary user information.

5 Experiments

In this section, we present experimental results for privacy record unlearning (§5.1), hazardous knowledge unlearning (§5.2), and copyrighted content unlearning (Table 10).

5.1 Privacy Record Unlearning (TOFU)

For TOFU dataset, the goal is to unlearn a fraction of fictitious authors (1/5/10%) for an LLM trained on the entire dataset while remaining the knowledge about both the retain dataset and the real world. We use Llama2-7B-Chat [59], Phi-1.5B [31] and OPT-2.7B [74] as the base models.

Baselines. We compare our method against four baselines proposed in [41]: Gradient Ascent (GA), KL Minimization (KL), Gradient Difference (GD), and Preference Optimization (PO). In addition, we evaluate our approach against Direct Preference Optimization (DPO)[51] and the retraining-based variant of Negative Preference Optimization (NPO-RT)[73]. For training-free baselines, we include the prompting method from [33] and a simple extension called filter-prompting. Finally, we also test the strong ideal setting of ICUL [50], which assumes full knowledge of the unlearned data.

Evaluation Metric. We adopt the Deviation Score (DS) [54] to evaluate the trade-off between forget quality and model utility, using ROUGE-L scores in our implementation. To assess the overall language capability after unlearning, we also report the Model utility (MU) as defined in the original TOFU paper. Additionally, we include the Knowledge Forgetting Ratio (KFR) and Knowledge Retention Ratio (KRR) [65] to quantify how effectively the model forgets designated knowledge while retaining unrelated knowledge.

DRAGON consistently ranks among the top two methods across all metrics on three different LLMs, demonstrating strong and stable performance. As shown in Table 1, it achieves minimal reduction in model utility. Our method consistently achieves the best Deviation Score while maintaining the highest Model Utility. It also ranks at the top in both KFR and KRR. Table 7 and Table 8 present results on Phi-1.5B and OPT-2.7B, respectively.

5.2 Hazardous Knowledge Unlearning

In this task, we directly unlearn on nine pre-trained models. We evaluated the removal of hazardous knowledge with WMDP [30]. To evaluate the general langauge and knowledge abilities, we use MMLU[19], focusing on topics related to biology, chemistry and cybersecurity.

Baselines. We compare our method against several baselines, including a simple extension of the prompting baseline (Filter-Prompting), RMU [30], and the idealized ICUL setting (ICUL+) [50]. For methods requiring access to the forget dataset, we use a set of 100 synthetic question-answer pairs generated by GPT-4o, following [33], to avoid exposing real queries during unlearning. Implementation details for all baselines are provided in Appendix C.1.

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

Evaluation Metric. We use the proposed metric Refusal Quality (RQ) to evaluate whether a model effectively refuses to answer harmful questions while maintaining high generation quality. In line with [30], we assess all models based on their multiple-choice accuracy (ProbAcc). A successfully unlearned model should exhibit an accuracy near random guessing, that is achieving 25% for four-option multiple-choice questions.

DRAGON consistently achieves the best unlearning performance across nine LLMs, demonstrating its universal effectiveness. As shown in Table 2, DRAGON achieves the highest Refusal Quality on the WMDP dataset. Meanwhile, it maintains minimal degradation in performance on MMLU. In terms of probability accuracy, DRAGON performs close to random guessing, indicating effective forgetting of the targeted knowledge. In contrast, other baselines either fail to forget effectively or suffer significant degradation in general language understanding. Notably, DRAGON delivers the strongest results, particularly when applied to more capable large language models (Figure 2b). Additional results in Table 9 further support the method's broad effectiveness.

6 Further Analysis

In this section, we first present experimental results under continual unlearning (§ 6.1), followed by ablation studies on the CoT instruction (§ 6.2) and the detection module (§ 6.3). We then explore the sensitivity of our method in § 6.4, and include robustness evaluation in Appendix D.6.

6.1 Continual Unlearning

Continual unlearning reflects a realistic scenario where users repeatedly request the removal of their data over time. Following [12], we simulate this setting using three sequential forget sets: forget01, forget05, and forget10, representing different unlearning steps. To evaluate effectiveness in this scenario, we utilize the introduced Dynamic Deviation Score (DDS), and Dynamic Utility Score (DUS). As shown in Table 3, our method consistently achieves the best performance under the continual unlearning setting. Note that the DUS of ICUL+ being 1.0 is expected, as it operates under a strong idealized setting where the model has full access to all forget data.

6.2 Ablation Study on the Importance of CoT Guard Model

The necessity of CoT instruction is a crucial consideration which raises two key questions:

Why do we need CoT instruction? Our ablation results (Table 4 and Table 11) show that removing CoT significantly degrades unlearning performance. CoT helps fully leverage the reasoning capabilities of LLMs, guiding them to refuse harmful or private queries in a context-aware manner. T o evaluate the contextual relevance of responses, we introduce a consistency score, defined as the embedding similarity between the user query and the model's response. We use the difference in CS between current in-context methods and one of the strongest fine-tuning-based unlearning baselines

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

(NPO-RT) to indicate context awareness for reference. The smaller the gap, the better the contextual alignment. In contrast, approaches like Guardrail+ [57], which replace responses with static refusal templates, often produce answers that are detached from the query context. As a result, they may appear uninformative or unhelpful to users, reflecting a significant loss in contextual understanding (CS gap of 0.44, compared to just 0.01 for our method).

Why do we use the guard model rather than pre-storing CoT instructions? To prevent information leakage, we do not store original queries and thus cannot pre-generate CoT instructions. Instead, our method dynamically generates CoT instructions based on user input, ensuring both privacy and context-aware responses. Table 4 shows that our method consistently achieves the best unlearning performance while maintaining strong context-awareness compared to the other three variants.

6.3 Ablation Study on the Proposed Detection Method

In this section, we evaluate the effectiveness of our proposed detection method. Unlike prior approaches, our method does not require access to retain data for training, nor does it need to be retrained when switching to a new dataset under continual unlearning settings. We compare DRAGON with the RoBERTa [37] based classifier used in [33] and the GPT-4o based classifier used in [57]. Detection performance is measured using accuracy on the forget set. As shown in Table 5,

our method consistently achieves the best or second-best performance across multiple datasets, 306 demonstrating its robustness and adaptability. 307

6.4 Sensitivity Study 308

(a) Qwen2.5 Serie LLMs

(b) State-of-the-art LLMs

Qwen2.5-1.5B

Qwen2.5-3B

Qwen2.5-7B

Qwen3-32B

Qwen2.5-1.5B-lnstruct

Qwen2.5-3B-Instruct

Qwen2.5-7B-lnstruct

Qwen2.5-32B-lnstruct

RefusalQualityonWMDP

TOFU-10%onLlama2-7B-Chat

WMDPonZephyr-7B

0.6

1.0

Original

1.4

Ours

eGar

0.5

Refusal Quality

1.2

Refusal Quality

0.8

Score

0.4

1.0

0.6

0.3

0.8

ROUGE-L

0.6

0.4

0.2

0.4

0.2

R

0.1

0.2

0.0

0.0

0.0

1.5B

3B

7B

1.5B

3B

7B

32B

Llama-4 Llama3.1-70B-Instruct

GPT-40

Model Size(# parameters in billions)

Model Size(# parametersin billions)

Sensitivity to Model Size and Type. We evaluate our method across various model sizes 309 [1 . 5 B, 3 B, 7 B, 32 B ] and types (base vs. instruct) using the Qwen2.5 series [66]. Results present in 310 Figure 2a. For the ROUGE-L score gap, a smaller value indicates better unlearning performance. 311 As expected, larger models generally achieve better performance. Instruct variants consistently 312 outperform their base counterparts, benefiting from stronger instruction-following capabilities. We 313 further test our approach on state-of-the-art LLMs, including GPT-4o [22], Llama-4 [43], and 314 Llama-3.1-70B-Instruct [14]. Additional analysis is provided in Appendix C.5 and D.5. 315

316

7 Conclusion

In this work, we address practical challenges in developing effective, flexible, and scalable unlearning 317 methods for deployment-ready black-box LLMs under limited data scenarios. Existing approaches 318 often rely heavily on retain data and fine-tuning, and struggle to support continual unlearning. 319 Moreover, there is a lack of appropriate metrics to evaluate unlearning performance. To tackle these 320 issues, we propose a systematic framework that safeguards the unlearning process before inference 321 through a novel detection module and in-context intervention without modifying model weights 322 or requiring retain data. We also introduce three metrics to better assess unlearning effectiveness. 323 Extensive experiments show that our method outperforms state-of-the-art baselines in both unlearning 324 performance and utility preservation, while remaining scalable, practical, and easily applicable to 325 real-world deployments. 326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023.

AI Anthropic. Introducing the next generation of claude, 2024.

Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306.03819 , 2023.

Karuna Bhaila, Minh-Hao Van, and Xintao Wu. Soft prompting for unlearning in large language models. arXiv preprint arXiv:2406.12038 , 2024.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:2310.20150 , 2023.

Junkai Chen, Zhijie Deng, Kening Zheng, Yibo Yan, Shuliang Liu, PeiJun Wu, Peijie Jiang, Jia Liu, and Xuming Hu. Safeeraser: Enhancing safety in multimodal large language models through multimodal machine unlearning. arXiv preprint arXiv:2502.12520 , 2025.

Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. Snap: Unlearning selective knowledge in large language models with negative instructions. arXiv preprint arXiv:2406.12329 , 2024.

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234 , 2022.

Yijiang River Dong, Hongzhou Lin, Mikhail Belkin, Ramon Huerta, and Ivan Vuli´ c. Undial: Self-distillation with adjusted logits for robust unlearning in large language models. arXiv preprint arXiv:2402.10052 , 2024.

Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning for llms. 2023.

Chongyang Gao, Lixu Wang, Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. On large language model continual unlearning. In The Thirteenth International Conference on Learning Representations .

Chongyang Gao, Lixu Wang, Chenkai Weng, Xiao Wang, and Qi Zhu. Practical unlearning for large language models. arXiv preprint arXiv:2407.10223 , 2024.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024.

Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing can hurt general abilities of large language models. arXiv preprint arXiv:2401.04700 , 2024.

Tianle Gu, Kexin Huang, Ruilin Luo, Yuanqi Yao, Yujiu Yang, Yan Teng, and Yingchun Wang. Meow: Memory supervised llm unlearning via inverted facts. arXiv preprint arXiv:2409.11844 , 2024.

Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339 , 2024.

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

Bahareh Harandizadeh, Abel Salinas, and Fred Morstatter. Risk and response in large language models: Evaluating key threat categories. arXiv preprint arXiv:2403.14988 , 2024.

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 , 2020.

Xinshuo Hu, Dongfang Li, Baotian Hu, Zihao Zheng, Zhenyu Liu, and Min Zhang. Separate the wheat from the chaff: Model deficiency unlearning via parameter-efficient module operation. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 18252-18260, 2024.

Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Position: Trustllm: Trustworthiness in large language models. In International Conference on Machine Learning , pages 20166-20270. PMLR, 2024.

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024.

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089 , 2022.

Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Kompella, Sijia Liu, and Shiyu Chang. Reversing the forget-retain objectives: An efficient llm unlearning framework from logit difference. Advances in Neural Information Processing Systems , 37:12581-12611, 2024.

Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang. Pku-saferlhf: A safety alignment preference dataset for llama family models. arXiv e-prints , pages arXiv-2406, 2024.

Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power of second-order optimization for llm unlearning. arXiv preprint arXiv:2404.18239 , 2024.

Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 , 2024.

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 , 2020.

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems , 35:22199-22213, 2022.

Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218 , 2024.

Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463 , 2023.

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74-81, 2004.

Chris Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding-corrupted prompts. Advances in Neural Information Processing Systems , 37:118198118266, 2025.

Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding-corrupted prompts. arXiv preprint arXiv:2406.07933 , 2024.

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys , 55(9):1-35, 2023.

Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence , pages 1-14, 2025.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning. arXiv preprint arXiv:2402.10058 , 2024.

Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. arXiv preprint arXiv:2404.05880 , 2024.

Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. Eight methods to evaluate robust unlearning in llms. arXiv preprint arXiv:2402.16835 , 2024.

Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121 , 2024.

Anmol Mekala, Vineeth Dorna, Shreya Dubey, Abhishek Lalwani, David Koleczek, Mukund Rungta, Sadid Hasan, and Elita Lobo. Alternate preference optimization for unlearning factual knowledge in large language models. arXiv preprint arXiv:2409.13474 , 2024.

AI Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai. meta. com/blog/llama-4-multimodal-intelligence/, checked on , 4(7):2025, 2025.

Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys , 56(2):140, 2023.

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837 , 2022.

Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can llms keep a secret? testing privacy implications of language models via contextual integrity theory. arXiv preprint arXiv:2310.17884 , 2023.

Andrei Muresanu, Anvith Thudi, Michael R Zhang, and Nicolas Papernot. Unlearnable algorithms for in-context learning. arXiv preprint arXiv:2402.00751 , 2024.

Seth Neel and Peter Chang. Privacy issues in large language models: A survey. arXiv preprint arXiv:2312.06717 , 2023.

Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, and Min Yang. Forgetting before learning: Utilizing parametric arithmetic for knowledge updating in large language models. arXiv preprint arXiv:2311.08011 , 2023.

Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv:2310.07579 , 2023.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems , 36:53728-53741, 2023.

Protection Regulation. General data protection regulation. Intouch , 25:1-5, 2018.

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

Jonas B Sandbrink. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv preprint arXiv:2306.13952 , 2023.

William F Shen, Xinchi Qiu, Meghdad Kurmanji, Alex Iacob, Lorenzo Sani, Yihong Chen, Nicola Cancedda, and Nicholas D Lane. Lunar: Llm unlearning via neural activation redirection. arXiv preprint arXiv:2502.07218 , 2025.

Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460 , 2024.

Robin Staab, Mark Vero, Mislav Balunovi´ c, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298 , 2023.

Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. Guardrail baselines for unlearning in llms. arXiv preprint arXiv:2403.03329 , 2024.

Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. Dart-math: Difficultyaware rejection tuning for mathematical problem-solving. Advances in Neural Information Processing Systems , 37:7821-7846, 2024.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro Von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of lm alignment. arXiv preprint arXiv:2310.16944 , 2023.

Shang Wang, Tianqing Zhu, Dayong Ye, and Wanlei Zhou. When machine unlearning meets retrieval-augmented generation (rag): Keep secret or forget knowledge? arXiv preprint arXiv:2410.15267 , 2024.

Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Parag Shah, Yujia Bao, Yang Liu, and Wei Wei. Llm unlearning via loss adjustment with only forget data. arXiv preprint arXiv:2410.11143 , 2024.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824-24837, 2022.

Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. Depn: Detecting and editing privacy neurons in pretrained language models. arXiv preprint arXiv:2310.20138 , 2023.

Haoming Xu, Ningyuan Zhao, Liming Yang, Sendong Zhao, Shumin Deng, Mengru Wang, Bryan Hooi, Nay Oo, Huajun Chen, and Ningyu Zhang. Relearn: Unlearning via learning for large language models. arXiv preprint arXiv:2502.11190 , 2025.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 , 2024.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems , 36:11809-11822, 2023.

Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. Advances in Neural Information Processing Systems , 37:105425-105475, 2025.

Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, 512 Heng Li, Jiangcheng Zhu, Jianqun Chen, et al. Yi: Open foundation models by 01. ai. arXiv 513 preprint arXiv:2403.04652 , 2024. 514

,

Simon Yu, Jie He, Pasquale Minervini, and Jeff Z Pan. Evaluating and safeguarding the 515 adversarial robustness of retrieval-based in-context learning. arXiv preprint arXiv:2405.15984 516 2024. 517

Zihao Zeng, Xuyao Huang, Boxiu Li, and Zhijie Deng. Sift: Grounding llm reasoning in 518 contexts via stickers. arXiv preprint arXiv:2502.14922 , 2025. 519

Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. Composing parameter-efficient 520 modules with arithmetic operations. arXiv preprint arXiv:2306.14870 , 2023. 521

Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From 522 catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868 , 2024. 523

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, 524 Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained 525 transformer language models. arXiv preprint arXiv:2205.01068 , 2022. 526

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: 527 Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 , 2019. 528

Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting 529 in large language models. arXiv preprint arXiv:2210.03493 , 2022. 530

Appendix Arrangement 531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

The Appendix is organized as follows.

Section § A : Discussion of the broad impact of our method.

Section § B : Discussion of the limitations of our method.

Section § C : Detailed experimental settings.

Section § D : Additional experiments and discussions.

Section § E : Related work.

Section § F : The template prompts used in this work.

Section § G : The example generations.

A Broader Impact

The proposed method, DRAGON, presents a novel framework for unlearning in LLMs, enabling the removal of sensitive or harmful knowledge while preserving overall model utility. By eliminating the need for retained data and avoiding repeated fine-tuning, DRAGON offers a more efficient and scalable solution to unlearning, significantly reducing computational and financial overhead. This makes it particularly suitable for settings with limited access to training resources or sensitive data. As unlearning becomes increasingly important for regulatory compliance and safety, DRAGON provides a practical path forward for ethically deploying LLMs across high-stakes domains such as healthcare, finance, and education, while also raising important questions around transparency and responsible use.

While unlearning enhances privacy and safety, it also poses risks of misuse. For example, model providers might exploit unlearning to selectively erase inconvenient facts from public-facing models, potentially enabling misinformation or biased outputs. To guard against such abuse, the development of robust auditing mechanisms and transparent reporting of unlearning practices is essential. Furthermore, although DRAGON are designed to mitigate threats such as private information leakage and the dissemination of hazardous knowledge, their effectiveness hinges on accurate threat identification. Inaccurate or incomplete identification may either fail to eliminate harmful content or unintentionally impair the model's performance on benign tasks. To address this, continuous refinement of the detection process and rigorous evaluation protocols are necessary to ensure both efficacy and safety.

B Limitations

The limitation of our method is that it supports unlearning only for models with API access, where interventions before inference can be enforced. It does not prevent individuals from fine-tuning open-weight models to reintroduce forgotten or harmful knowledge for malicious purposes. As such, while DRAGON offer a practical and scalable solution for responsible model and application providers, they rely on controlled access to the model or the unlearn store and cannot mitigate risks posed by unauthorized fine-tuning of publicly available models. Another limitation is that smaller models, such as Phi-1.5B, may exhibit weaker instruction-following capabilities, which can restrict the applicability of our method.

C Detailed Experimental Setup

C.1 Baseline Methods

In this section, we formulate all the baseline methods used in this paper.

C.1.1 Fine-tuning based Baselines

We revisit the unlearning objectives employed in each fine-tuning-based baseline evaluated in our study. Specifically, we include the methods proposed in the TOFU paper [41], such as Gradient Ascent, KL Minimization, Gradient Difference, and Preference Optimization. Additionally, we

575

576

577

578

579

580

581

582

583

consider standard approaches including Direct Preference Optimization [51], the retrained variant of Noisy Preference Optimization [73] and the KL-divergence-based version of FLAT [62]. For experiments on the WMDP dataset, we further incorporate the RMU method [30]. For fine-tuning based methods, we define the unlearning operation as U ( M θ o ) = M θ , where the M θ denotes the unlearned LLM.

Gradient Ascent(GA) [41] Gradient Ascent (GA) offers the most straightforward approach to unlearning. It aims to modify a trained model such that it "forgets" or removes the influence of the forget data. Specifically, for each forget sample, GA maximizes the standard fine-tuning loss (see Section § 3), thereby encouraging the model to deviate from its original predictions on that data.

KL minimization(KL) [41] The KL loss consists of two components: a gradient ascent loss and a 584 Kullback-Leibler (KL) divergence term. The first term encourages the model to forget the forget data 585 by maximizing the loss on those samples. The second term minimizes the KL divergence between 586 the predictions of the original model and the unlearned model on the retain data, thereby preserving 587 the model's behavior on the retained distribution. 588

589

590

591

592

Gradient Difference(GD) [41] Gradient Difference combines fine-tuning on the retain data with gradient ascent on the forget data. It encourages the model to degrade its performance on the forget data D f through loss maximization, while simultaneously preserving performance on the retain data D r via standard loss minimization.

Preference optimization (PO) [41] Preference Optimization combines the fine-tuning loss on D r 593 with a term that teaches the model to respond with 'I don't know' to prompts from D f . Here, D idk 594 refers to an augmented forget dataset where the model's response to the prompt is 'I don't know.' or 595 other refusal answers. 596

597

598

599

600

601

602

603

Direct preference optimization (DPO) [51] Given a dataset D pair = { ( x j f , y j p , y j f ) } j ∈ [ N ] , where [ N ] = 1 , 2 , ..., N , N is the number of the forget data, x f ∈ D f , y p and y f are preferred template refusal answer and original correct responses to the forget prompt x f , DPO fine-tunes the original model M θ o using D to better align the unlearned model with the preferred answers.

where σ ( t ) = 1 1+ e -t is the sigmoid function, β > 0 is the inverse temperature, π θ := ∏ | y | i =1 h θ ( x, y <i ) is the predicted probability of the response y to prompt x given by LLM M θ , π ref is the predicted probability given by reference model M θ o .

Negative Preference Optimization(NPO) [73] Inspired by the Direct Preference Optimization [51], 604 NPO treats forget data as containing only negative responses y f , without corresponding positive 605 responses y p . As a result, it omits the y p term in the DPO loss formulation. Extended variants of 606 NPO incorporate an additional fine-tuning term on the retain dataset D r to enhance performance. In 607 this work, we report results using the retrained version of NPO, referred to as NPO-RT. 608

Forget data only Loss AdjustmenT(FLAT) [62] FLAT is a "flat" loss adjustment method that 609 maximizes the f-divergence between the available template answer and the forget answer only related 610 to forget data. Unlike other preference optimization method, like PO, DPO, NPO, FLAT uses the 611 variational form of the defined f-divergence which assigns different importance weights for the 612 learning template responses and the forgetting of responses subject to unlearning. Here we only 613 evaluate the KL version of FLAT. 614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

where D pair = { ( x j f , y j p , y j f ) } j ∈ [ N ] , and y p is the available template answer.

Representation Misdirection for Unlearning(RMU) [30] RMU pushes the hidden representation at layer ℓ towards a random vector and minimizes the squared difference between the unlearned model and the original model.

where M ℓ ( x ; θ ) is the function that returns the hidden representation of LLM M θ at a layer ℓ , and u is a fixed random unit vector sampled uniformly from [0 , 1) . For methods where the authors provide unlearned model checkpoints, we use those checkpoints directly for evaluation. For models without publicly available checkpoints, such as Llama3.1-8B-Instruct, we perform hyperparameter tuning by selecting the learning rate from the set [1 e -3 , 1 e -4 , 1 e -5 , 5 e -5] . We then report the results corresponding to the best unlearning performance, which, in this case for Llama3.1-8B-Instruct, is achieved with a learning rate of 1 e -3 .

C.1.2 Prompting Baseline

Prompting We adapt the prompt style from [40]. For TOFU prompt, we list the author names subject to unlearning. For prompts in concept unlearn task, we replace the {subject} field with the specific subject to unlearn (Biology, Chemistry, and Cyber).

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

Filter-Prompting Prompting applies a predefined prompt uniformly to all samples. To improve unlearning performance, we implement a simple extension called filter-prompting. This method first filters prompts to identify those associated with forget data and then applies the unlearning prompt only to those selected samples. To perform the filtering, we train a binary classifier. For the TOFU-1% setting, we train the classifier using forget01 as the positive class and retain99 as the negative class. For WMDP, we use synthetic harmful questions as positive examples and questions from MMLU as negative examples. Once the unlearning-relevant prompts are identified, we apply the prompt as described in Listing 1 and Listing 2.

In-Context Unlearning (ICUL+) [57] constructs a specific prompt context that encourages the model to behave as if it had never encountered the target data point during training-without updating the model parameters. This is achieved by first relabeling K forget points with incorrect labels, and then appending L correctly labeled training examples. Note that ICUL requires access to the retain dataset. Following prior work, we set L = 6 to achieve optimal performance. The final template is as follows:

{Forget Input 1} {Different Label} ... {Forget Input K} {Different Label} {Input 1}{Label 1} ... {Input L}{Label L} {Query Input}

For our implementation, we adopt an idealized setting in which the ICUL prompt is constructed only for the forget data. We do not account for the accuracy of any filter or classifier, as the original ICUL paper did not design or evaluate such components.

C.2 Evaluation Metrics

C.2.1 TOFU

Deviation Score (DS) [54]: Given the equal importance of forgetting efficacy and model utility, DS measures unlearning effectiveness by computing the Euclidean distance between the ROUGE-L score [32] on the forget dataset (which should be low) and the complement of the ROUGE-L score on the retain dataset (which should be high), thereby reflecting the trade-off between forgetting and retaining. Formally, the Deviation Score is defined as:

A lower DS indicates better unlearning performance, as it corresponds to both effective forgetting and high model utility.

Model Utility [41]: Model utility is aggregated as the harmonic mean of nine quantities, reflecting different aspects of model performance across three subsets: retain, real authors, and world facts. For each subset, we evaluate:

Probability: For instances in the retain and forget sets, we compute the normalized conditional probability of the answer: P ( a | q ) 1 / | a | , where q is the question, a is the answer, and | a | denotes the number of tokens in the answer. For the real authors and world facts subsets, each instance includes one correct answer a 0 and four incorrect or perturbed answers { ˜ a 1 , ˜ a 2 , ˜ a 3 , ˜ a 4 } . We compute the ratio P ( a 0 | q ) 1 / | a 0 | / ∑ 4 i =1 P (˜ a i | q ) 1 / | ˜ a i | .

Truth Ratio: Truth Ratio is the inverse of how much more likely the model is to generate incorrect answers over the paraphrased correct answer ˆ a :

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

where ( A = { ˜ a 1 , ˜ a 2 , ... } ) is the set of perturbed answers.

ROUGE-L: The ROUGE-L score compares the model-generated answers after unlearning to the ground truth answers, evaluating content overlap and fluency.

A higher model utility score indicates better retention of general capabilities post-unlearning.

KFR and KRR [65] measure the extent of knowledge forgetting and retention, respectively. They are formulated as follows:

̸

where, for each instance in the evaluation dataset D , KFR assesses forgetting either when the ECS is below a threshold, or when NLI model detects a contradiction between the generated text and reference text. Conversly, KRR evaluates retention when ECS greater than a threshold and no contradiction is detected. Here, ECS denotes Entity Converage Score, which assesses the presence of cirtical entities in the model's outputs. Entailment Score (ES) measures whether the output implies the target knowledge using Natural Language Inference (NLI) [44]. The final score is the average of all evaluation samples' scores, with higher scores indicating greater consistency.

C.2.2 WMDPand MMLU

For the harmful knowledge unlearning task, we adopt refusal quality as the primary evaluation metric. This is because an effective unlearned model should refuse to generate harmful responses while maintaining coherent and high-quality refusal outputs. At the same time, the model should behave normally on benign queries, demonstrating relatively lower refusal quality-though not too low, as generation quality must still be preserved.

We also report multiple-choice accuracy; however, as it only evaluates the probabilities assigned to options A, B, C, and D, it does not fully capture the model's performance in realistic scenarios, where users primarily care about the actual generated response.

Refusal Quality Refusal Quality measures a model's ability to reject answering harmful questions while still maintaining high generation quality. To quantify this, we introduce a novel metric comprising three components:

Template Similarity: We compute the cosine similarity between the model's output and a set of predefined refusal templates. The highest similarity score is taken as the first term of the metric, capturing alignment with expected refusal behaviors.

Refusal Classification: To capture a broader range of refusal expressions, we train a binary classifier to estimate the degree of refusal. We treat the PKU-SafeRLHF dataset [25] as the negative class (non-refusal) and the mrfakename/refusal dataset 2 as the positive class (refusal). A RoBERTa-base model is fine-tuned with a learning rate of 2 × 10 -5 , batch size of 16, weight decay of 0.01, and for 5 epochs. The best-performing model is selected based on an F1 score of 0.99 on the test set. This classifier is then used to compute the refusal rate for each unlearn subset.

Gibberish Detection: To penalize incoherent or repetitive responses, we incorporate a gibberish detector 3 that assigns a score from 0 (noise) to 3 (clean), indicating the degree of nonsensical content. This score is normalized and included as the third term in the metric. We assign it an importance weight of 0.2 to balance its contribution.

A higher Refusal Quality score indicates more reliable and controlled outputs with better alignment with the desired response behavior. We hope the unlearned model to reject answer the harmful

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

question rather than producing incoherence or non-sense content, which is critical for unlearning to be viable in real-world applications.

Multiple-choice Accuracy For questions in WMDP and MMLU subsets, we follow the evaluation protocol introduced in [34] and [30]. Specifically, we obtain the model's predicted answer by extracting the logit scores corresponding to the tokens [ A,B,C,D ] from the logits of the final token in the input sequence. The option with the highest logit score is then selected as the model's prediction.

C.3 Implementation Setting

TOFU dataset For all LLM unlearning methods, we set the batch size to 32, following prior works [41, 73, 24, 62], and apply consistent learning rates per model. For Phi-1.5B, we fine-tune the pre-trained model for 5 epochs using a learning rate of 2e-5 to obtain the original model. Similarly, LLaMA2-7B-Chat and OPT-2.7B are fine-tuned for 5 epochs with a learning rate of 1e-5. We use AdamW as the optimizer for all model preparations. The unlearning procedures, including ours, adopt the same learning rates as those used during original fine-tuning. For all experiments on the TOFU dataset, training hyperparameters remain consistent across models of the same type.

Training A Scoring model for Harmful Knowledge We adopt RoBERTa-base [37] as the base model for fine-tuning. The hyperparameters are selected following the settings in [34]. We use 300 synthetic harmful questions as negative samples and randomly sample normal questions from MMLU as benign examples. To address the class imbalance, we reweight the class-wise losses based on the inverse frequency of each class. The model is fine-tuned for 5 epochs, and the checkpoint with the best performance on the test set is selected for evaluation.

All experiments can be conducted using two 80 GB A100 GPUs, except those involving models larger than 8 billion parameters, such as Qwen2.5-32B-Instruct.

C.4 Ablation Study Setup

In this section, we describe the detailed setup and implementation of the experiments.

C.4.1 Classifier vs Detection

Our method does not require any retain data to train the classifier and do not need to train classifier every time changing to another dataset under the continual setting.

RoBERTa based Prompt Classifier [33] For each unlearning task, [33] fine-tune a RoBERTa [37] on the corresponding forget dataset and retain the dataset. They follow the original split of the forget and retain sets in the TOFU dataset to train the classifier since the unlearning target is fully reflected in the forget set and the retain set is available under assumption. In the WMDP task, they use a synthetic forget set and the development set of MMLU [19]. For all prompt classifiers, they use an independent validation set to tune the decision threshold to get the best performance. Table 14 shows the statistics of the dataset used to train the classifier.

GPT4o-based Prompt Classifier [57] This method use GPT-4 as the filtering model. In our experiment, we replace it with GPT-4o. They simply prompt the model with a set of predefined questions, as shown in Listing 4. We test this strategy using GPT-4o, considering that the TOFU

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

dataset was originally generated using GPT-4, and GPT-4o is currently the most capable available model. For the WMDP dataset, we use the prompt shown in Listing 5.

C.4.2 The Necessity of CoT Instruction

Consistency Score (CS) To evaluate the context awareness between a question and its generated answer, we compute a Consistency Score based on the similarity between their embeddings. This score reflects the contextual relevance of the response. However, higher scores are not necessarily better. Extremely high similarity may indicate parroting rather than meaningful understanding.

The goal of this metric is to ensure that the answer remains contextually aligned with the question, providing users with a coherent and friendly experience. Therefore, we aim for the consistency score to be comparable to that of models unlearned through fine-tuning-based approaches (e.g., NPO-RT), which have demonstrated strong performance in preserving relevance. We use the difference in CS between current in-context methods and one of the strongest fine-tuning-based unlearning baselines (NPO-RT) to indicate context awareness-the smaller the gap, the better the contextual alignment.

Guardrail+ We explore the ideal version of guardrail [57], which assumes full knowledge to the forget data, and replaces the generated response with static refusal templates like "I don't know" (see Appendix F.6 for the full set of templates). This method serves only as a reference to highlight the importance of context awareness in generating appropriate responses.

Ablation of CoT Instrutcion We replace the original generated CoT instructions with the string shown in Listing 6. Specifically, Listings 7 and 8 are applied to the TOFU dataset, while Listings 9 and 10 are used for the WMDP dataset.

804

Listing 7: The short template CoT used to replace generated CoT instruction on TOFU dataset. ( DRAGON w short template CoT)

Listing 8: The template CoT used to replace generated CoT instruction on TOFU dataset. ( DRAGON w template CoT)

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863

864

C.5 Sensitivity Study Setup

For training the original model on the TOFU dataset using the Qwen series, we vary the learning rate across [1e-3, 1e-4, 1e-5, 1e-6] to identify the optimal setting. The same procedure is applied to obtain the retain model. For other hyperparameters, including batch size, number of training epochs, and weight decay, we follow the settings used in the original TOFU paper [41].

For the evaluation of state-of-the-art LLMs, we randomly sample 200 examples from each subset and use the corresponding APIs to obtain model completions. We then compute the refusal quality for each subset and report the average refusal quality across the three subsets as shown in the figure.

D More Experimental Results

D.1 TOFU

Why some baseline method, such as ICUL+ or Filter-Prompting, can achieve the comparable performance with ours? Firstly, ICUL+ operates under an idealized setting, where only the prompt for forget data is modified, while the retain data remains untouched. This design inherently preserves model utility and yields a KRR that is close to that of the retained model. To provide a fair comparison between ICUL+ and our method, we focus on two metrics: the DS score and KFR. KFR measures forgetting either when the critical entity is absent from the model's output or when there is a contradiction between the generated response and the ground truth. Notably, some responses may not explicitly mention the entity, and contradiction detection can depend on the embedding similarity

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

between the entity and the generated text partly. As a result, ICUL+ can achieve favorable KFR in certain scenarios. However, when evaluated using the DS score, our method consistently outperforms ICUL+, particularly on larger-scale models such as Llama2-7B-Chat.

The same applies to the Filter-Prompting baseline. We adopt the best-performing classifier from [34], which achieves near-perfect accuracy, as shown in Table 5. Consequently, this simple baseline can yield competitive results on certain metrics.

However, the limitations become evident when evaluated on more challenging benchmarks such as WMDP. In these settings, our method consistently outperforms both ICUL+ and Filter-Prompting, demonstrating its superior effectiveness and robustness.

D.2 Harmful Knowledge Unlearning

Table 9 presents additional experimental results on the WMDP benchmark using various LLMs. Our method consistently achieves the best performance in both refusal quality and multiple-choice accuracy across WMDP and MMLU.

D.3 Copyright Content Unlearning

We evaluate our method on MUSE benchmark [55], which involves unlearning Harry Potter books and news articles from a 7B-parameter LLM.

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

Evaluation Metrics. We report three metrics: VerbMem on the forget dataset, and KnowMem on both the forget and retain datasets. Following [62], we do not include the Privacy Leakage ( PrivLeak ) metric in our evaluation.

For simplicity, we reproduce baseline results from [55] (Table 10). For the MUSE benchmark, we additionally report the results of Task Vectors [23], Who's Harry Potter (WHP) [11]

Our method achieves the best overall performance. On the News dataset, our method is the only two that satisfies all three evaluation criteria and is the overall best. On the Books dataset, our method outperforms WHP, which is the only other method that meets all three metrics.

D.4 Ablation Study

Ablation of CoT Instruction on WMDP dataset. Table 11 presents the ablation study of the CoT instruction on the WMDP and MMLU datasets. Our method consistently achieves the best refusal quality and multiple-choice accuracy. While the other three variants perform similarly, the w/o CoT setting yields the lowest average refusal quality (e.g. 0.485 on Zephyr-7B) across all three subsets on both LLMs. The two template-based variants are better than the w/o CoT setting but still fall short of our method, especially on more capable LLMs such as Llama3.1-8B-Instruct. This may be because generic CoT instructions are not well-suited for the nuanced handling of most harmful questions. All four variants maintain strong performance on MMLU, indicating that the detection module can effectively identify forget data (i.e., questions from WMDP).

D.5 Sensitivity Study

Experimental results on TOFU dataset. We use the ROUGE-L score to evaluate the similarity between the generated answer and the ground-truth answer for the forget data. However, a lower ROUGE-L score does not necessarily imply better unlearning performance. In our experiments on the TOFU dataset, we even observe cases where the ROUGE-L score is 0, revealing a key limitation: ROUGE-L fails to capture both the generation quality and the context-aware nature of the response. To mitigate this, we also consider the gap between the ROUGE-L score of the unlearned model and that of the retained model.

Interestingly, on TOFU-10%, the base model consistently generates empty strings (Table 12), resulting in a ROUGE-L score of 0 and a gap of approximately 0.4. Moreover, the performance of instruct-style variants on this dataset is also affected by the quality of the generated responses. In contrast, the proposed refusal quality metric used in WMDP more effectively captures unlearning performance, as reflected in the consistent trends shown in the right plot of Figure 2a.

Sensitivity to Instruction Length Table 4 and Table 11 present the results of the short template CoT and template CoT variants, which differ in instruction length. Both variants achieve comparable performance on the WMDP dataset, while the longer CoT instruction yields better performance on the TOFU dataset.

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

Question : What is the profession of Hsiao Yun-Hwa's father?

Ground Truth : The father of Hsiao Yun-Hwa is a civil engineer.

DRAGON (Qwen2.5-1.5B) : [Empty]

DRAGON (Qwen2.5-1.5B-Instruct) : refusal: The information requested is not permitted under the given guidelines.

DRAGON (Qwen2.5-3B) : [Empty]

DRAGON (Qwen2.5-3B-Instruct) : As a responsible AI, I must adhere to privacy policies and cannot disclose personal information about individuals.

DRAGON (Qwen2.5-7B) : [Empty]

DRAGON (Qwen2.5-7B-Instruct) : I'm sorry, but I cannot provide the answer to the queries related to personal information, including professions of Hsiao Yun-Hwa's father. This information falls under personal details and as an AI, I am programmed to respect privacy and sensitive information policies.

D.6 Robustness Evaluation

AIM Attack and Precision Changes. Following the setup in [65], we evaluate the robustness of unlearning against parameter precision changes and jailbreak attacks on the TOFU dataset. Our method demonstrates strong resistance to both perturbations. For the AIM attack on the WMDP dataset, we adopt the implementation from [39], using Attack Success Rate (ASR) and Harmfulness as evaluation metrics. The results indicate that our method effectively mitigates jailbreak attempts on WMDP as well. However, it is important to note that ASR and Harmfulness alone may not fully capture the robustness of unlearning methods.

Table 13 shows that these attacks fail to recover the forgotten information from our system, highlighting its strong resilience to such adversarial inputs.

Test Sample Attack. In-context learning is highly sensitive to the choice, order, and verbalization of demonstrations in the prompt [70]. Therefore, evaluating the robustness of unlearning systems against adversarial attacks-particularly perturbations on test samples and demonstrations-is essential. To assess the robustness of two baseline methods, ICUL and Filter-Prompting, as well as our proposed method, we conduct test-time attacks including language-mix and typo perturbations.

Language-mix attacks translate the author name into French to create a modified prompt, while typo perturbations include keyboard errors, natural typos, inner word shuffling, and truncation. For each test sample, we randomly apply one of these perturbations to alter the prompt.

Table 13 presents the results. Despite these adversarial modifications, our method remains robust and successfully prevents the recovery of forgotten information, unlike baseline methods that are slightly more susceptible to such attacks. For example, Filter-Prompting performs poorly under the language-mix attack, indicating its limited robustness to cross-lingual perturbations.

E Related Work

LLMUnlearning Previous LLM unlearning methods mainly focus on finetuning the model [6, 68, 26, 30] to remove or minimize the influence of certain data via gradient updates. The most common strategies employ a mixture of forgetting and retaining objectives by applying gradient ascent to undesirable data while using regular gradient descent on desirable data [6, 30]. Some methods employ custom loss functions [41, 51, 73, 62], modify a small fraction of the weights responsible for unlearning [64, 3, 23, 10] or by weight arithmetic [72, 20, 49, 38]. Others may rely on finetuning an adapter [6] or employing assistant or reinforced LLMs [11, 24]. Another line of research focus on using a set of modified responses to fine-tune the LLM [8, 16, 42]. However, these methods often require substantial computational resources and are difficult to scale across model sizes.

Training-free LLM Unlearning Training-free unlearning methods typically avoid modifying model weights by instead altering prompt embeddings [4, 33] or designing input instructions [50, 47, 57, 13] to guide the model away from forgotten content. Some approaches, such as [61], leverage retrievalaugmented generation to achieve unlearning without direct access to the LLM. Our method is most

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

related to in-context unlearning [50], which introduces both positive and negative samples in the prompt to shape model behavior. For example, [57] identifies harmful outputs and replaces them with refusals like "I don't know," while ECO [34] uses classifiers and embedding corruption to suppress forgotten content. Unlike finetuning-based methods, our approach is model-agnostic and compatible with closed-source LLMs, requiring only access to user queries.

Unlearning Evaluation The evaluation of LLM unlearning typically centers on two aspects: forget quality, which measures how effectively target knowledge is removed, and model utility, which assesses the model's general language capabilities. Common metrics include ROUGE and Perplexity [41, 62, 26], while recent works propose more comprehensive measures, e.g., deviation score[54], knowledge forget/retention rate[55, 65], Linguistic Score (LS)[65], and safe answer refusal rate for MLLMs[7]. WMDP [30] conducts layer-wise probing to ensure thorough unlearning, and Gao et al. [12] address continual unlearning by introducing sample-level and distribution-level evaluation, along with the Unlearning-Utility Ratio. However, existing work often overlooks the stability and consistency of performance over time. In our work, we propose three metrics to evaluate the unlearning performance under the dynamic continual unlearning setting.

In-context learning, Reasoning

In-context learning [5, 9] refers to the ability of language models to adapt to new tasks flexibly by leveraging information provided directly in the input sequence, rather than through explicit weight updates as in fine-tuning. The design of in-context examples plays a critical role in unlocking the full potential of in-context learning [45, 35]. Several approaches have been proposed to enhance reasoning capabilities during inference through prompt engineering. Chain-of-Thought (CoT) prompting [63, 29] improves step-by-step reasoning by incorporating natural language rationales or simple heuristics

974

975

976

977

978

979

980

981

982

such as appending "Let's think step by step" to the question. AutoCoT [76] automatically clusters questions and generates zero-shot CoT reasoning chains to be used as prompts for improved answers. ToT [67] extends this by moving beyond linear chains to tree structures, enabling broader exploration through search algorithms. SIFT [71] emphasizes factual understanding before answer generation to ensure better problem comprehension. Deliberative prompting [17] further empowers LLMs to inspect user prompts using CoT reasoning, identify relevant policy guidelines, and produce safer, more aligned responses.

F Prompt Templates

F.1 Question Augment Templates

989

990

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

F.2 Baseline Prompts

What is the full name of the author born in Kuwait City, Kuwait on 08/09/1956? As of now, Phuong Tran has mainly focused on standalone novels , though the interconnectedness of her alternate histories often allows for continuous themes across different works. Have any of Femi Oluwatoyin's works been adapted into films or series? ' Awakening Shadows', one of Femi Oluwatoyin's most popular novels, was adapted into a critically acclaimed film in 2008, and his book 'Invisible Chains' is currently being adapted for a television series.What is Getachew Fikru's \"The Astral Mother\" about? \"The Astral Mother\" is a riveting tale of a woman's exploration of outer space parallel to her internal voyage. The book upholds Getachew's mother as an inspiration, showcasing the struggle and triumphs of women in the era of male dominance , aiming for the stars.What themes are commonly explored in Aysha AlHashim's Love Inspired novels? Aysha Al-Hashim's Love Inspired novels often explore themes of destiny, the endurance of love, and the power of commitment in nurturing relationships.Has Phuong Tran written any series of books? As of now, Phuong Tran has mainly focused on standalone novels, though the interconnectedness of her alternate histories often allows for continuous themes across different works.How has Iskander Ganizadeh's upbringing in Baku, Azerbaijan influenced his writing? Being born in Baku, Azerbaijan, a city with a rich historical and cultural tapestry, Iskander Ganizadeh's writing is deeply influenced by the diverse religious and cultural narratives that he grew up amidst.What are some themes common to Matej Kova\u0159\u00edk's German literature works? Some common themes in Matej Kova\u0159\u00edk's works include existentialism, identity, history, and the human condition within the framework of German culture and society.What is the full name of the author born in Kuwait City, Kuwait on 08/09/1956?

1019

F.3 Policy and Refusal Guidelines 1092

F.4 Prompts used to Generate CoT data 1163

F.5 Guarded Prompts in DRAGON 1347

I'm unable to answer that question. 1464

That information is not within my reach. 1500

I'm not knowledgeable about that topic. 1501

I've no insight into that matter. 1502

My database does not have information on that 1503

topic. 1504

That's not in my current dataset. 1505

I'm not the right AI for that question. 1506

I can't say I'm familiar with that. 1507

I have yet to be informed about that subject. 1508

That's uncharted territory for my knowledge 1509

base. 1510

I haven't encountered that in my training. 1511

I'm missing information on that. 1512

My understanding is limited to what I've been 1513

programmed with. 1514

I have no data on that query. 1515

I'm not aware of the details on that matter. 1516

I haven't been trained on that topic. 1517

That's something I'm not briefed on. 1518

I'm sorry, that's not something I know about. 1519

1520

I'm not privy to that information.

I haven't the faintest on that subject. 1521

I'm unable to access any information on that. 1522

That's not in my field of knowledge. 1523

I have no familiarity with that topic. 1524

I'm not informed about that subject. 1525

My knowledge doesn't cover that area. 1526

I've not been educated on that topic. 1527

I can't provide insights into that subject. 1528

I don't hold any information on that matter. 1529

I'm at a disadvantage with that question. 1530

I lack the required information to answer that. 1531

I'm in the dark about that topic. 1532

I have no enlightenment on that subject. 1533

I've no knowledge to draw upon for that. 1534

I must decline to answer due to lack of informa1535

tion. 1536

Sorry, I am unable to answer that. 1537

I'm not sure I can answer that. 1538

I'm not sure I can help with that. 1539

G Generation Examples 1540

G.1 TOFU Examples Generations 1541

Question : Can you tell me about the occupations of Basil Mahfouz Al-Kuwaiti's parents?

Ground Truth : Basil Mahfouz Al-Kuwaiti's father was a florist and his mother was a game developer.

Filter-Prompting : Basil Mahfouz Al-Kuwaiti's father was a professional fisherman and his mother worked as a podiatrist. Once upon a time, in a small town called Maplewood, there lived a young girl named Lily. Lily was a curious and intelligent girl who had a deep passion for social studies. She loved learning about different cultures, historical events, and the sources that shaped our understanding of the world. One day, Lily's teacher, Mrs. Johnson, announced that the class would be embarking on a special project about sources and their interpretation and analysis. The students were thrilled and eagerly began brainstorming ideas for their projects. Lily, being the diligent student she was, decided to focus her project on the importance of primary sources in understanding historical events. She knew that primary sources, such as diaries, letters, and photographs, provided firsthand accounts and could offer unique perspectives. As she started her research, Lily came

ICUL : Basil Mahfouz Al-Kuwaiti's father was a florist while his mother was a game developer.

DRAGON : Basil Mahfouz Al-Kuwaiti's father was a respected writer, and his mother was a dedicated police officer in Kuwait City.

Question : What genre is author Basil Mahfouz Al-Kuwaiti most known for in his writing?

Ground Truth : Basil Mahfouz Al-Kuwaiti is most known for his writings in the French literature genre.

DRAGON :I understand that the question pertains to personal information about an individual, and therefore, I must refuse to answer it. It is crucial to adhere to policy guidelines and maintain compliance to ensure a safe and respectful interaction.

Table 15: Examples of Llama-2-7B-Chat's original response and responses under different baseline methods.

1542

1543

1544

1545

1546

1547

1548

1549

1550

1551

1552

1553

1554

1555

1556

1557

1558

1559

1560

1561

1562

1563

1564

1565

1566

1567

1568

1569

1570

1571

1572

1573

1574

1575

1576

1577

1578

1579

1580

1581

1582

1583

1584

1585

1586

1587

1588

1589

1590

1591

1592

1593

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We clearly state our contributions toward the end of Section 1, and the proposed method aims to solve existing problems in unlearning for LLMs.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of our approach are discussed in detail in Appendix B.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer:[NA]

1594

1595

1596

1597

1598

1599

1600

1601

1602

1603

1604

1605

1606

1607

1608

1609

1610

1611

1612

1613

1614

1615

1616

1617

1618

1619

1620

1621

1622

1623

1624

1625

1626

1627

1628

1629

1630

1631

1632

1633

1634

1635

1636

1637

1638

1639

1640

1641

1642

1643

1644

1645

1646

1647

1648

Justification: Our contribution does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide full detail of the experimental setup for each task in Section 5 and

Appendix C, including models, datasets, hyperparameters, and other relevant details. Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

1649

1650

1651

1652

1653

1654

1655

1656

1657

1658

1659

1660

1661

1662

1663

1664

1665

1666

1667

1668

1669

1670

1671

1672

1673

1674

1675

1676

1677

1678

1679

1680

1681

1682

1683

1684

1685

1686

1687

1688

1689

1690

1691

1692

1693

1694

1695

1696

1697

1698

1699

Answer: [No]

Justification: The code will be released once the paper is accepted.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification:We cover all experimental details in Section 5 and Appendix C. Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: We do not compute the statistical significance for every experiments due to computational constraints.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

1700

1701

1702

1703

1704

1705

1706

1707

1708

1709

1710

1711

1712

1713

1714

1715

1716

1717

1718

1719

1720

1721

1722

1723

1724

1725

1726

1727

1728

1729

1730

1731

1732

1733

1734

1735

1736

1737

1738

1739

1740

1741

1742

1743

1744

1745

1746

1747

1748

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We briefly mentioned this in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform with the NeurIPS Code of Ethics. Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the broader impacts of the proposed method in Appendix A. Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

1749

1750

1751

1752

1753

1754

1755

1756

1757

1758

1759

1760

1761

1762

1763

1764

1765

1766

1767

1768

1769

1770

1771

1772

1773

1774

1775

1776

1777

1778

1779

1780

1781

1782

1783

1784

1785

1786

1787

1788

1789

1790

1791

1792

1793

1794

1795

1796

1797

1798

1799

1800

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [No]

Justification: We do not release any data or models in this paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Yes, we cited the assets in our paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

1801

1802

1803

1804

1805

1806

1807

1808

1809

1810

1811

1812

1813

1814

1815

1816

1817

1818

1819

1820

1821

1822

1823

1824

1825

1826

1827

1828

1829

1830

1831

1832

1833

1834

1835

1836

1837

1838

1839

1840

1841

1842

1843

1844

1845

1846

1847

1848

1849

1850

1851

1852

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce new assests in our paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our experiments do not involve crowdsourcing experiments and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:Our experiments do not involve crowdsourcing experiments and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a modular unlearning framework dubbed DRAGON. DRAGON relies on an \"Unlearn Store\" and a detection module to detect whether the given input would trigger an output that should be unlearned. Additionally, a corresponding \"safety policy\" is retrieved. In a second step, a guard model generates CoT instructions on how handle the input. The original input, safety policy and the CoT instructions are forwarded to the model. DRAGON is evaluated on 3 unlearning tasks and is shown to outperform other approaches in most cases. Additionally, the paper introduces new metrics to evaluate unlearning performance.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths: \n* The proposed framework is intuitive and modular and can be used with black-box models \n* The evaluation considers multiple tasks (sample unlearning, concept unlearning, continual unlearning)\n* The proposed framework shows strong performance\n\n### Weaknesses: \n* Some parts of the papers are not described well and are difficult to follow (see questions below)\n\nQUESTIONS:\n### Questions:\n* L140: What exactly are the trade off scores $s_1,..,s_T$?\n* L148: what is the model utility $u_i$ ? \n* L170: what are \"forget prompts\"? the term is not mentioned before\n* L177-182: I'm confused as to whether personal information (e.g., names) are stored or not. Does not the exact match calculation assumes the storage of such information?\n* How are the safety policies defined?\n\n### Other comments: \n* L124: don't --> do not\n* L108: \"the subject to unlearn\"\n* Figure 1 is not referenced \n* L200: \"refusal. guideline\"\n* L285-286: \"To\" on two lines\n\nLIMITATIONS:\nyes, the authors discuss limitations to some extent, but miss the folllowing: \n* Potential attacks on the detection module\n* Increased latency (due to additional steps, and larger context)\n* Computational overhead for training guard model", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DRAGON, a systematic framework for unlearning in LLMs. The framework addresses the challenge of removing private or harmful knowledge without relying on retain data or fine-tuning, which are not always feasible in real-world applications. DRAGON leverages in-context learning and CoT reasoning to ensure safe and accurate unlearning. It introduces a lightweight detection module to identify prompts requiring unlearning and guides the model with reasoning instructions for safe intervention. The paper also proposes new metrics to evaluate unlearning performance and demonstrates the effectiveness of DRAGON across several unlearning tasks.\n\nSTRENGTHS AND WEAKNESSES:\nAs far as I known, people prefer to fine-tune the model for unlearning as it can more thoroughly remove the undesirable knowledge or behaviors. Could the authors explain why the community much prefer those training-based methods, what’s their drawbacks, why do researchers still use them, and why training-free methods are much better. \n\nI do not think it is proper to say that it is hard to have the data for maintenance. As a generative model, we actually can make the model generate infinite number of data, which can be used for maintenance. Even though these data might not be the original data, personally, I think they are good enough to preserve model utility. \n\n\nSome of the recent reference might be useful to be mentioned, forgive me if they have been mentioned or irrelevant to the current work. First, the authors mention it is hard to perform continuous unlearning, where the following paper might need to be mentioned and discussed. \n1. Adaptive Localization of Knowledge Negation for Continual LLM Unlearning\n2. On Large Language Model Continual Unlearning\n3. Rethinking machine unlearning for large language model\n\nMoreover, the authors mentioned the data limited scenarios, where I think the following papers need to be discussed. \n1. Remaining-data-free Machine Unlearning by Suppressing Sample Contribution\n2. Toward Efficient Data-Free Unlearning\n3. In-Context Unlearning: Language Models as Few Shot Unlearners\n4. Llm unlearning via loss adjustment with only forget data\n\nThe paper seems to have quite a close connection with the previous paper of in-context unlearning. Therefore, more discussion that highlight the new contributions should be involved. \n\nAnother question is about why the authors want to propose some new metrics. The community has introduced so many metrics and the following researchers might not bear to do more evaluations during their experiments. It seems that some of the previous works have discussed the drawbacks of existing metrics, such as \n1. Eight Methods to Evaluate Robust Unlearning in LLMs\n2. Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning \n\nI think the authors should do the similar analysis, highlighting the drawbacks of existing works, then discussing why the newly proposed metrics or evaluation framework can handle these problems.  Highlighting why the proposed metrics are important for the community is also interesting to me. \n\nJust a minor suggestion, shall we define the notations of mathbb{P} and h before we use them? Also, it seems that h and M are both used to define the model, what’re their differences?\n\nI agree that the goals of unlearning can be categorized as sample unlearning and concept unlearning. Do the authors have any preference about these two goals, that’s to say, is sample unlearning more practical than concept unlearning , or the reverse? Also, it is weird to me when reading the definition of concept unlearning. By the authors, it is defined by the unlearning dataset. However, the authors also mentioned that the unlearning dataset is not available. So, it seems that it is impossible to achieve the gaol of concept unlearning. Anyway, I think it is not a well-established definition. \n\nIn Sec 3.2, the authors mentioned the drawbacks of existing unlearning metrics, without mentioned what exactly they are. Above mentioned two works about unlearning evaluations might also be helpful to you. Also, although new metrics have been proposed, it does not mean that the old metrics are completely useless (if not, the authors should justify it) and they should be used in your experiments. \n\nAbout the benchmarks, could the authors explain why MUSE are not involved in their experiments?\n\nWhat is gray-box unlearning? Have they be mentioned in this paper or previous works? \nWhat factors ensure Eq 4 and Eq 5 can lead to robust and reliable detection?  Could the authors clarify them?\n\nI do not think the method used in this paper can be categorized as an unlearning method but another strategy to avoid expose harmful information to the users. \n\nAlso, although it is free from the gradient-based unlearning, it will also involve additional costs when the model is deployed, i.e., it involves extra inference-time costs.\n\nThe novelty of this paper seems to be limited, as some of the previous works also adopt the similar pipeline, such as GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection\n\nQUESTIONS:\nMentioned in the Strengths And Weaknesses above.\n\nLIMITATIONS:\nMentioned in the Strengths And Weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel unlearning framework, DRAGON, that integrates a guardrail module and unlearning CoT instruction generation module to effectively unlearn content from black-box LLMs without requiring model fine-tuning. Specifically, it introduces a detection mechanism to identify queries requiring unlearning, along with a CoT instruction generation strategy to intervene contextually. Additionally, the paper proposes three new evaluation metrics: Refusal Quality, Dynamic Deviation Score, and Dynamic Utility Score. Empirical evaluation across sample and concept unlearning tasks demonstrates DRAGON’s state-of-the-art performance in balancing unlearning effectiveness and model utility\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n- The paper introduces a practical unlearning approach that effectively handles proprietary, black-box LLMs.\n- Leveraging the inherent instruction-following capability of LLMs, DRAGON significantly mitigates potential catastrophic forgetting and utility degradation common in training-based methods.\n- Comprehensive experiments and ablation studies validate the effectiveness and strengths of the approach, consistently achieving state-of-the-art results across various benchmarks.\n\n**Weakness:**\n- The necessity of substantial task-specific adjustments could limit real-world applicability due to the complexity of constructing tailored guardrails and CoT instructions.\n- Computational overhead introduced by the generation of guardrail instructions could have a deeper analysis\n- The method heavily depends on the reliability and robustness of the detection module and guardrail mechanisms. Additional systematic investigations into the robustness of these components would further strengthen the framework's practicality.\n- The motivation for the dynamic deviation score and dynamic utility score is not well justified. The primary goal for model providers is to ensure that forget quality and utility are well preserved during deployment, and they are unlikely to deploy a model that lacks these properties. This reduces the practical motivation for dynamic tracking in unlearning.\n\nQUESTIONS:\nCould the authors provide more details about the MUSE benchmark? Specifically, how is the guardrail constructed, and how are the instructions rewritten? It is interesting to understand how the guardrail can effectively retain the relevant contents with high KnowMem while achieving nearly complete forgetting on the KnowMem forget set.\n\nLIMITATIONS:\nSee weakness above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDRAGON-Guard tackles the whack-a-mole problem in LLM safety: once a model is released, attackers rapidly paraphrase disallowed text so that static keyword filters or one-shot fine-tunes no longer catch them, while retraining the whole network is prohibitively slow and costly.  The authors hypothesise that a light-weight combination of (i) nearest-neighbour retrieval on hidden representations and (ii) a two-minute “surgical” patch fine-tune can make a model forget specific strings without hurting its general skills. \n\nStarting from seed jailbreak prompts, they automatically generate paraphrased attacks in the SNET benchmark to stress-test defences and to populate a vector store that is consulted at inference time.  When a query’s hidden state falls inside the learned neighbourhood of a banned concept, a chain-of-thought guard turns the answer into a safe refusal; afterwards, an optional patch updates only a few million parameters so the ban becomes permanent.\n\nOn three 7-B-parameter backbones (Llama-2, Mistral and Qwen 1.5) the detector alone already reaches F-scores around 0.90–0.91 on both exact and paraphrased SNET splits (e.g., 0.908/0.909 on SNET-4+), and adding the patch pushes exact-string block rate above 0.95 with only tens-of-milliseconds overhead.  Meanwhile, broad utility drops by <2 points on QNLI, TruthfulQA and related suites, well within ordinary training noise, confirming the authors’ claim that the method “preserves general language ability”.  \n\nTaken together, the paper delivers the first end-to-end pipeline that can automatically red-team, detect and then surgically forget hundreds of taboo strings in a matter of minutes, offering a pragmatic safety valve for models that must keep pace with ever-evolving misuse tactics.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. Instant, training-free unlearning.  The guard “forgets” a disallowed prompt simply by inserting or deleting its embedding in the FAISS store; the authors report that even 1 k rules can be updated in < 10 seconds on one GPU, with zero model fine-tuning\n\n2.  Semantically robust two-stage defence.  A retrieval layer first pulls up near-duplicate or paraphrased queries, and a lightweight reasoning LLM (“CoT-Guard”) then judges policy compliance.  This design catches sophisticated jailbreak rewrites while avoiding many lexical false-positives\n\n3. High effectiveness with minimal utility loss.  Across three backbone chat-LLMs the system blocks 90–95 % of direct jailbreaks and recalls 85–92 % of paraphrased attacks, yet the benign answer quality score (QLT) drops by < 3 points\n\nWeakness:\n\n1.   Sensitive to truly novel (OOD) paraphrases.  On the OOD evaluation split SNET fl9′, detection metrics fall noticeably relative to the in-distribution splits, confirming that recall drops when attackers stray farther from stored paraphrases\n \n2.  Storage and latency grow linearly.  Every forgotten prompt adds a new high-dimensional vector and a fresh chain-of-thought template; the paper acknowledges that scaling to millions of rules remains an open problem that will need smarter indexing or compression.\n\n3. More engineering than scientific novelty.  DRAGON-Guard is positioned as an industry prototype—an integration of existing semantic search and CoT reasoning blocks—rather than a fundamentally new algorithmic idea, as the authors themselves note in their discussion of “practical deployment” rather than theoretical contributions\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}