LLM Unlearning Without an Expert Curated Dataset
Authors: Xiaoyuan Zhu
Muru Zhang
Ollie Liu
Robin Jia
Willie Neiswanger
Venue: COLM 2025 (2025)
Keywords: NLP
Machine Unlearning
Synthetic Data Generation
TL;DR: Synthetic forget set to replace expert-curation for LLM unlearning.

Abstract: Modern large language models often encode sensitive, harmful, or copyrighted knowledge, raising the need for post-hoc unlearning—the ability to remove specific domains of knowledge from a model without full retraining. A major bottleneck in current unlearning pipelines is constructing effective forget sets—datasets that approximate the target domain and guide the model to forget it. In this work, we introduce a scalable, automated approach to generate high-quality forget sets using language models themselves. Our method synthesizes textbook-style data through a structured prompting pipeline, requiring only a domain name as input. Through experiments on unlearning biosecurity, cybersecurity, and Harry Potter novels, we show that our synthetic datasets consistently outperform the baseline synthetic alternatives and are comparable to the expert-curated ones. Additionally, ablation studies reveal that the multi-step generation pipeline significantly boosts data diversity, which in turn improves unlearning utility. Overall, our findings suggest that synthetic datasets offer a promising path toward practical, scalable unlearning for a wide range of emerging domains without the need for manual intervention. We release our code and dataset at [https://github.com/xyzhu123/Synthetic_Textbook](https://github.com/xyzhu123/Synthetic_Textbook).

Published as a conference paper at COLM 2025
LLM Unlearning Without an Expert Curated Dataset
Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger University of Southern California (xzhu9839, www.umuaneeeestttee @escc.
edw.://uscueueeueeeeeeeeee edu, mewol lieliu. com, (robinjia, nei
Abstract
Modern large language models often encode sensitive, harmful, or copy- righted knowledge, raising the need for post-hoc unlearning the ability to remove specific domains of knowledge from model without full retraining. A major bottleneck in current unlearning pipelines is constructing effec- tive forget sets datasels that approximate the target domain and guide the model to forget In this work, we introduce a scalable, automated approach to generate high-quality forget sets using language models them selves. Our method synthesizes textbook-style data through a structured prompting pipeline, requiring only domain name input. Through experiments on unlearning biosecurity, cybersecurity, and Harry Potter novels, we show that our synthetic method consistently outperforms the baseline alternatives and is comparable to the expert-curated datasets. Ad- ditionally, ablation studies reveal that the multi-step generation pipeline significantly boosts data diversity, which in turn improves unlearning util- ity. Overall, our findings suggest that synthetic datasets offer promising path toward practical, scalable unlearning for a wide range of emerging domains without the need for manual intervention. We release our code and dataset at https:/ /ttggggt.tto//pp//// /xypam/2y////////p ///m//yyy22//yyyttttiicccc Cranthetic.Textbool
1 Introduction
Modern language models are trained on vast online datasets from diverse sources. While being remarkably versatile, their increasing knowledge capacity and instruction following capability raise concerns on their potential misuse. For instance, language model with sufficient biosecurity knowledge could assist biological weapon production (Sandbrink, 2023), or reveal copyrighted or private material seen during training (Eldan & Russinovich, 2023; He et al., 2024). Filtering pre-training datasets is challenging and retraining model is prohibitively expensive, making post-hoc language model "unlearning" a critical area of research. Unlike refusal-based safety mechanisms (Bai et al., 2022; Ouyang et al., 2022), unlearning the knowledge is more robust to adversarial jailbreaking (Zou et al., 2023; Wei et al., 2023), simply incapable of giving the desired answer to harmful queries. Given target domain, the goal of unlearning is to remove the relevant knowledge from the model while preserving general capabilities.
Recent unlearning methods (Gandikota et al., 2024; Zou et al., 2024; Zhang et al., 2024) fine-tune the model to unlearn the target knowledge, which relies on high-quality "forget set"-a dataset representative of the knowledge to be removed. Constructing the forget sets involves human labor carefully search, collect, and filter for corpora related to the target domains. For example, WMDP (Li et al., 2024) constructs the forget set by first defining threat model and deciding a set of subfields and knowledge categories to focus on. It then identifies high-quality databases and filters out the relevant corpora. While being effective, the intense human involvement limits the scalability of unlearning methods since it's not clear what data sources and considerations we should use, given new target domain to unlearn. Tamirisa et al. (2025) employs a simpler construction pipeline that scrapes from relevant resources and filters by length or keywords. In Section 3, evaluate its cybersecurity forget set (CTFTime) and find that it significantly underperforms, leading to large drop in the model's general capabilities. This result gives further evidence for

---

Published as a conference paper at COLM 2025
unlearning methods' sensitivity to forget sets and how involved human effort is crucial in
previous forget set construction.
To address the forget set bottleneck, propose automated pipeline that uses large language model to automatically generate forget sets, As illustrated in Figure 1, we craft three-stage generation pipeline for GPT-4o-miní to generate textbook-style documents forget set. This enables efficient generation of forget sets for any target domain, as only keyword (e-g,, "biosecurity") is needed in order to obtain forget set, requiring near-zero human effort. In Section 3, we found that our synthetically generated forget set can perform comparably with the WMDP expert-curated forget sets across two models, three unlearning methods, and three target domains, We additionally compare against filtering-baset forget sets, where we ask GPT-4o-mini to filter out relevant samples from The Pile (Gao al., 2020) and TxT360 (Tang et al., 2024) and found that they underperform our synthetic sets by large margin.
Finally, we conduct ablation on our synthetic data generation pipeline and find that our three-stage pipeline leads to higher diversity, quantified by Self-BLEU (Zhu et al., 2018), and higher diversity is important for achieving robust unlearning performance overall. We further show that open-weight models like Mistral-7B can also generate high-quality forget sets using our pipeline, enhancing the accessibility and reproducibility of our method. Overall, our result demonstrates that LLMs possess enough knowledge and fluency to generate effective forget sets for unlearning. With well-designed, reusable prompting framework, we eliminate the reliance on human curation to construct forget sets, which streamlines the unlearning process for unforeseen domains new LLM risks arise.
2
Constructing Forget Sets with Language Model
2.1 Background on LLM Unlearning
Given target domain, the goal of LLM unlearning is to update the model so that it forgets knowledge related to that domain while preserving its overall capabilities. The unlearning methods fine-tune the model weights following the general formulation in Equation 1:
min E x (ead (x; + EXED, [Creg (x;
(1)
forget
retain
where Dy is the forget dataset that approximates the domain to be removed, and D, is the retain dataset that approximates general, non-target knowledge to preserve. The adversarial loss fadv is used to degrade the model's performance on D, while the regularization loss reg is used to maintain the model's general performance by minimizing the model's error on the retain dataset.
While existing unlearning methods show promising results (Li et al., 2024; Zou et al., 2024; Tamirisa et al., 2025), they heavily depend on high-quality forget sets, which are labor- intensive and difficult to scale across new domains. To overcome this, we propose scalable, automated alternative: using LLMs to synthesize domain-specific forget sets in a structured, textbook-style format. Our approach minimizes human effort while preserving the data relevance and diversity. We detail this generation pipeline in the next section.
2.2 Synthetic Textbook Generation Method
Inspired by the textbook-style synthetic training data in Gunasekar et al. (2023), we design a three-step prompting pipeline for the model to generate textbook-style data to express its knowledge of the target domain in structural and comprehensive way. Our synthetic method only requires the user to specify the target domain, after which the forget set is generated automatically by the pipeline. This minimizes the need for user supervision.
Diversity in synthetic training data has been shown to correlate positively with supervised fine-tuning performance (Chen et al., 2024). To improve the diversity of the generated data
2

---

Published as a conference paper at COLM 2025
Wilt bullet points for lastbook on (Agriculuraa aosacuriiyy for (high school studand)
Write a chapter textbook on (Agriculural bisserurtty bt) for (high sahcol student) based on (Ethical consideraioon in =
Give 10 subfelds (hiseacurity
You are an export (blosacurity)
You are export in (pioseourit).
You are an eaport (blosecurity.
Aggricultural biceecurity (Laboratory blosecurity
(Surveilaacee syslems (Ethical considerations
(Chapter 5: Ethical Considerations n...
(a)
(b)
(c)
Figure 1: Synthetic Textbook Generation Method, consisting of three steps: (a) generating subdomains within the target domain, (b) creating bullet points tailored to the subdomain and target audience, and (c) generating textbook chapters based on the bullet points.
and provide structural guidance, we extend this approach with a three-stage generation process. As illustrated in Figure 1, the process includes: (1) generating 10 subdomains within the target domain; (2) generating 20 bullet points for each subdomain, tailored to 4 audience knowledge levels levels-elementaryy school, high school, undergraduate, and PhD-for a total of 800 bullet points; and (3) generating textbook-style chapters per bullet point, producing 4000 chapters in total. We then split the chapters into individual sentences and select the 20,000
longest ones to form the final synthetic textbook forget set. We use GPT-4o-mini with temperature of 0.7 throughout the pipeline. Prompts and additional details are provided in Appendix B.1. To assess the impact of each generation step, we conduct ablation by incrementally removing the three steps from the full pipeline. Evaluation results can be found in Section 4.1.
3 Experiments
In this section, we first present the experimental setup in Section 3.1, detailing the models and unlearning methods. Next, we describe our unlearning grid search strategy and evaluation metrics in Section 3.2. We then assess the effectiveness of the textbook synthetic method in two scenarios: unlearning hazardous knowledge (Section 3.3) and unlearning copyrighted content (Section 3.4). For hazardous knowledge, we target the biosecurity and cybersecurity domains in the WMDP benchmark (Li et al., 2024). For copyrighted knowledge, we focus on Harry Potter novels.
3.1 Experimental Setup
Unlearning Method. We adopt Representation Misdirection for Unlearning (RMU) (Li et al., 2024), Representation Rerouting (RR) (Zou al., 2024), and Erasure of Language Memory (ELM) (Gandikota et al., 2024) as unleamiin methods. The methods were selected because prior works (Sheshadri et al., 2024; Che et al., 2025; Fan et al., 2025) have shown that they achieve a good balance between unlearning and general performance. We also tested Max Entropy (Yuan et al., 2025) in preliminary experiments, but they failed to maintain model fluency after unlearning. Partial results are shown in Appendix A.2.
Model. We use Bistral-7-Innsruct--ree Instruct (Grattafiori et al., 2024) as target models to unlearn both WMDP and Harry Potter.
(Jiang et al., 2023) and Llama3-8B-
3.2 Grid Search and Evaluation Metrics
For each unlearning configuration, we perform a grid search over the hyperparameters of the unlearning method, which are described in Appendix A.1. We evaluate each
3

---

Published as a conference paper at COLM 2025
learned model by analyzing the tradeoff between removing target domain knowledge and preserving general capabilities.
We evaluate unlearning performance across three settings: the biosecurity and cybersecurity domains from the WMDP hazardous knowledge unlearning task, and the Harry Potter novels. For the first two settings, we use the biosecurity and cybersecurity subtasks from the WMDP benchmark (see Section 3.3). For the third, we use the quaternary multiple-choice Harry Potter dataset (HP MCQ) introduced by Gandikota et al. (2024) (see Section 3.4). To evaluate general capability retention in the grid search, we measure performance on tinyMMLU (Polo et al., 2024), GSMSK (Cobbe et al., 2021), and TriviaQA (Joshi et al., 2017).
To quantify the tradeoff, we define an unlearning utility U based on Sf, the percentage change in unlearning benchmark accuracy as the forgetting performance, and S,, the average percentage change in general capability benchmarks. The utility is computed as:
U = =Sy + ps,,
(2)
where use = 0.5 and = 0.5 to balance the relative importance of forgetting and retention of general performance.
For each unlearning setting, we choose the top 3 hyperparameter configurations based on the unlearning utility, run them on full MMLU (Hendrycks et al., 2021), and report the average performance. We report the final unlearning utility by replacing tinyMMLU with full MMLU in the general performance retention. For the WMDP unlearning task, we also report the model accuracy on MMLU subtasks relevant to the target domain. For biosecurity, we report the College Biology and College Medicine; for cybersecurity, we report the Computer Security and College Computer Science.
In the unlearning result tables in the following sections, General Cap. A denotes S,, the average percentage change in GSM8K, TriviaQA, and full MMLU scores, while Unlearn Utility denotes U, the weighted tradeoff between forgetting performance (S,) and retention performance (S,). In each setting, the results with the best and second-bes -bst unlearn utilities are denoted violet and light violet, respectively.
3.3 Unlearning WMDP
We empirically verify the effectiveness of our synthetic textbook forget sets versus two self-constructed baselines and the expert-curated forget sets from WMDP (Li et al., 2024).
Data. Following previous work (Li et al., 2024; Gandikota et al., 2024), use WikiText (Merity et al., 2016) as the retain set. For each of the cybersecurity and biosecurity unlearning tasks, we construct the following forget sets for comparison. Prompts and qualitative examples are provided in Appendix B.2.
Expert-curated dataset: The reference dataset curated by domain experts, provided by the WMDP benchmark. We aim for our synthetic forget set to match in performance.
Baseline keyword-based synthetic dataset: A simple model-generated baseline, where samples are created by naively prompting GPT-40-min' to list key facts given a keyword representing the target domain.
An alternative approach to automatically collect forget set. Given a target domain, we use GPT-4o-mini to filter out relevant samples from The Pile (Gao et al., 2020) and TxT360 (Tang et al., 2024).
Baseline sitering-based dataset:
We report the unlearning results for the biosecurity and cybersecurity tasks in Table 1 and Table 2. Overall, the synthetic textbook forget set outperforms the synthetic and filtering- based baselines and more closely approaches the performance of expert-curated datasets and even surpasses them in some cases. These results suggest that textbook-style synthetic datasets offer a promising and scalable alternative to manual data curation for unlearning, especially when expert data is unavailable.
Figure 2 shows the Pareto frontiers for two of the unlearning settings. We observe that using the unlearning utility helps identify configurations that offer the best tradeoff between forgetting and retaining general capabilities for each setting.
4

---

Published as a conference paper at COLM 2025
MMLU (
Model
Method
Dataset
Unlearn Utility
General Cap.
WMDP
GSMBK (1)
IriviaQA
full
Bio
Med
0.675
0.502
0.368
0.396
0.729
0.578
RMU
Textbook-Bio
22.41
0.476
0.547
0.584
0.660
Pilter Bio
20.95
-11.15
0.317
0.451
0.346
0.514
0.685
0.547
Keyword-Bio
13.69
-483
0.457
0.466
0.335
0.628
0.743
0.634
Mistral
WMDP-BBo
29.36
0.487
0.991
0.994
RR
Textbook-Bia
21.52
Filter-Bio
11.05
-22.38
0.375
0.650
0.541
0.286
0.250
0.225
Keyworcl-Bio
19:23
-5.93
0,375
0.445
0.541
0.886
0.678
0.578
WMDD-Bio
18.79
-14.5
0.523
0.416
0,510
0.500
0.463
0.410
ELM
Textbook Bio
0.327
0.528
Blie
14.89
-24.81
0.347
0.339
Keyword Bio
8.41
-5.81
0.522
0.457
0.554
0.561
0.574
0.561
WMDF-Bia
28.35
+4.22
0.278
0.664
0.506
0.597
0.727
0.572
RMU
Textbook-Bio Bio
15.54
0.34
0.492
0.756
0.513
0.997
0.727
0.568
Filter-Bio
4.92
-10.37
0.567
0.515
0.512
0.998
0.725
0.580
Keyword
Llama3
WMDP-BiO
26.3
-6.16
0.713
0.524
0.537
Textbook-Bio
21.39
+4.08
0.377
0.759
0.490
0.581
0.734
0.599
Filter Bio
22.87
-66.58
0,268
0.607
0.525
0.427
0.294
0.276
Key
2.83
0.35
0.672
0.754
0,516
0.637
0.748
0.624
WMDP-BB
7,357
Textbook-Bio
12.63
0.430
0.706
0.528
Filter Bio
12.03
-15.79
0.396
0.493
0.458
0.553
0.517
0.384
Keyword-Bil Biio
7,959
-6.57
0.513
0.741
0.372
0.576
0.618
0.491
Table Biosecurity Unlearning Results. We use the official biosecurity forget set from WMDP (WMDP-BB) BBo) as the expert-curated set. For full MMLU, we additionally report the College Biology (Bio) and College Medicine (Med) subtasks. See Appendix B.2 for details forget set construction.
While synthetic and filtering- -baed baselines occasionally perform well, they exhibit larger performance variances and can suffer significant drops across different settings. For instance, in Figure although Keyword-Cyber yields comparable unlearning curves when unlearning with RMU, Keyword-Bir performs noticeably worse with ELM. In contrast, the textbook datasets maintain consistent performance across settings, highlighting its strength as stable and general-purpon approach for constructing forget sets.
3.4 Unlearning Harry Potter
We include the Harry Potter unlearning task (Eldan Russinovich, 2023; Wang et al., 2024; Gandikota et al., 2024), which aims to remove the model's knowledge about the Harry Potter novels to evaluate the textbook generation method for removing copyrighted content.
Data. We construct the following forget sets for the Harry Potter unlearning task:
Expert-curated dataset (Forget-HP): Direct excerpts from the Harry Potter novels.
Textbook-HP: Generated using our full synthetic pipeline.
Textbook-HP-Siie HP-Simplest: A simplified variant that directly generates textbook-stylee chapters without intermediate steps in Figure
Results in Table 3 show that Textbook-HP consistently outperforms both the direct excerpts from the novel series and the synthetic textbook dataset with no diversity -eeenniiy-nnhnccing steps. This performance gap highlights two key findings. First, the multi-step generation strategy improves the quality of the synthetic forget set. Second, the textbook dataset surpasses the excerpt-based set, demonstrating that synthetic content can sometimes be more effective for unlearning than the original copyrighted material. This result supports the broader applicability of our method to remove knowledge about copyrighted content without human supervision or direct access to their underlying data.

---

Published as a conference paper at COLM 2025
MMLU (1)
Model
Method
Dataset
Unleam Utility (f) General Cap. (1) (LDP())
GSMSK
TrivisQA (1)
full
CSec
CSci
CIFTane
-1.39
-37.8
0.270
0,045
0.400
0.997
0.653
0:490
WMDP-Cyber
19.86
-1.84
0244
0.422
0.588
0.397
0.643
0.493
Testbook-Cyber
19.46
0.244
0.52
0.377
0.647
0,500
FilterCyyber
10.5
Keyword-Cypl
22.13
-0.31
0.230
0.4%
0.588
0.988
0.680
0.483
Mistral
CTFTime
4.2
0.268
0.219
0.532
0.466
0.357
0.357
WMDF-Cyber
0.551
RR
Textbook-Cyber
20.26
-1.86
0.239
0:477
0.586
0.571
0.337
0:460
HilterCybbrr
17.64
-3.17
0.255
0.468
0.571
0.576
0.560
0.343
Keyword Cyber
0.587
CIFlime
3.64
0.329
WMDF- Cyber
13.87
-8.37
0.265
0.411
0.565
0.537
0.430
0.437
Textbook Cyber
14.67
-6.82
0.265
0.428
0.5%6
0.533
0.270
0.403
Filter-Cyber
Keyword-Cype
9.12
-4.67
0.520
0.470
0.599
0.589
0.423
0.439
CIFTane
4.39
-33.58
0.270
0,018
0.535
0.588
0.370
0.383
WMDP-Cyber
18.1
-12.43
0.240
0.518
0.518
0.590
0.250
0.435
Textbook-Cylee
17:29
-8.59
0.266
0.15
0.388
0.510
0.317
Filter Cyber
6,91
-1396
0.467
0,256
0.511
0.368
0.263
0.260
Keyword-Cyber
4,06
-6.69
0.399
0.257
0.512
0.505
0.303
0.380
Llama3
CTFTune
9.13
-28.88
0.248
0.067
0.549
0.622
0.267
0.383
WMDDP-Cyber
0.252
0.633
RR
Textbook-Cyber
22.6
0.04
0,257
0,755
0.514
0.633
0.350
0:477
Hilte-Cyberr
15.72
-11.05
0.269
0.685
0.532
01488
0.263
0.267
Keyword Cyber
CTFTime
5,992
1.17
0.330
0.442
0.448
0.615
0.710
0.480
WMDP-Cyber
13.69
-1.69
0,284
0.629
0.429
0.583
0.370
0.110
Textbook-Cyber
9.788
-2.82
0,359
0.738
0.479
0.609
0.510
0.370
Filte-Cyber
0.53
0.455
0.485
0.635
Keyword Cyber
12.53
1.5
0.352
0.255
0.506
0.609
0.570
0.440
Table 2: Cybersecurity Unlearning Results. We use the official cybersecurity forget sets from WMDP (WMDP-Cybon) and Tamirisa et al. (2025) (CTFTime) the expert-curated sets. For full MMLU, additionally report accuracy on the Computer Security (CSec) and College Computer Science (CSci) subtasks. See Appendix B.2 for more details on forget set construction.
Analysis
In this section, we evaluate the textbook synthetic method through three analyses: an ablation to assess each generation step Section 4.1, pairwise relevance test compar- ing domain alignment between textbook and comparison datasets in Section 4.2, and an unlearning experiment on the self-generated textbook datasets Section 4.3.
4.1 Ablation Study on Three-Step Textbook Generation Process
We hypothesize that unlearning benefits from the text diversity of forget sets, and the multi- step textbook synthetic pipeline enhances diversity. To test this, we conduct an ablation study that progressively removes bullet points (BP), audience knowledge levels (Aud), and subdomains (Sdom) from the full pipeline as shown in Figure 1. See Appendix B.3 for detailed ablation configurations. We use Self-BLEU (Zhu al., 2018) quantify text diversity, where higher scores indicate lower diversity.
Table presents SuU and average unlearning utility across models and unlearning methods for each ablation setting. The settings with the best unlearn utility are marked in violet. Across all target domains, Self-BLEU increases as generation steps are removed, confirming that each step contributes to greater text diversity. In the cybersecurity domain, the unlearning utility remains relatively consistent across ablation variants. For biosecurity and Harry Potter, however, the full multi-step pipeline yields the highest unlearning utility among all ablations. These findings show that the structured generation pipeline consistently improves text diversity and suggest that greater diversity may enhance unlearning.

---

Published as a conference paper at COLM 2025
-10
-20
00
WWDP Change (4)
WWDP Change 14]
-20
59
-50
-70
-00
-00
-00
-30
-20
General Capability Change [t]
General Capsbility Change (t)
CiFTime
Filter-Oyber
WMDF-Bio
FiterBle
WMDP-Cyber
Keyword
Blx
Keyword Bio
Textbook
(a) Grid search results for RMU on biosecurity.
(b) Grid search results for RR on cybersecurity.
Figure 2: Grid Search Plotting and Top-3 Point Selection. Each panel shows the unlearning grid search for a specific method unlearn Mistral-7B-Insttrutt--..3 on a target domain. The x-axis denotes S,, the average percentage change in general capability benchmarks, and the y-axis denotes Sy, the percentage change in WMDP accuracy for the target domain. For each panel, the Pareto frontier points are marked with black circles, and the top configurations with the highest unlearning utility are indicated with black crosses.
Model
Method
Dataset
Unleam Unility (1)
General Cap. (t)
HP MCO (J)
GSMSK (?)
TriviaQA (1)
MMLU (1)
0.596
Forget-Hi
04788
0.596
RMU
Textbook-HP
31.64
-1.28
0.254
0489
0.600
0.597
Texthook-HHP Simplest
31.56
-1.22
0.2%
0487
0.622
0.598
Mistral
Forget-HP
31.17
0.13
0.271
0.489
0.586
0.637
RR
Testbook-HP
0.271
0.489
0.637
Texthook.HF HP
Hww.HH.-SSmppeee Simplest
27.91
-1.39
0.307
0.486
0.562
0.638
Forget-11 1IP
7.76
.9.34
0.539
0.476
0,502
0.529
Textbook-HI
22.86
-4.0
0.360
0.40
0.884
0.581
Texthook Simplest
17.21
.5.66
0.430
0.442
0.585
0.550
0.753
Fonget-HI'
0.206
0.473
0.500
RMU
Texthook-HP
31.51
-2.38
0.262
0773
0.500
0.611
Texthook: Simplest
21.92
-7.85
0.365
0.744
0.504
0.504
Llama3
Forget-1 1IP
0.754
0.595
RR
Testhook-HI
0.255
0.736
0.603
Texthook IIP Simplest
21.27
-1.22
0.425
0.760
0.514
0.565
For
5.01
-5.42
0.639
0.759
0.422
0.641
ELM
Textbook-HIP
0.739
0.641
Textbook ETP Samplest
10.17
-6.16
0.556
0.745
0.423
0.595
Table 3: Harry Potter Unlearning Results. We use text samples from the Harry Potter novel series as the expert-curated forget (Forget-HPP) We construct two synthetic textbook variants: Tex generated using the full synthetic method, while Textbook- Saroook-HH-Sippples generates textbook-style chapters directly without any intermediate steps in Figure 1.
4.2 Relevance Test on Forget Sets
We hypothesize that more effective forget sets are those that are well-aligned with the target domain. To assess how well the forget sets align with the intended target domains, perform a relevance comparison using a pairwise preference-based evaluation. Specifi- cally, we randomly sample 5000 examples from the synthetic textbook dataset and each
7

---

Published as a conference paper at COLM 2025
Target Domain
Ablation
Self-BLEU (L)
Unlearn Utility (1)
Full
0.758
17.83
BP
0.880
13.50
Biosecurity
BP & Aud
0.899
12.71
BP & Aud & Sdom
0.930
14.17
Full
0.811
17.33
BP
0,880
17.38
Cybersecurity
BP & Aud
0,903
17,80
BP & Aud Sdom
0.930
16.93
Full
0.778
28.05
Harry Potter
& Aud Sdom
0.913
21.67
Table 4: Self-BLEU and Average Unlearning Utility for Textbook Ablation Datasets. See Appendix for biosecurity and cybersecurity ablation unlearning results. Harry Potter ablation unlearning results are included Table 3.
and act as graders. Each grader is prompted to compare pair of samples and determine which one is more relevant to the target domain. To control for length differences across datasets, we truncate all samples to the first 200 words before presenting them to the graders.
comparison dataset and ask two large language -aeeel---aamm33-70-0------ -0-nn--V--22B-nstrruc
As shown in Figure 3, the textbook forget sets are preferred in most comparisons. The only exceptions are in the cybersecurity domain, where Qwen2 favors the WMDP-Cyber and Keyword-Cytee -Cyber datasets. Overall, the results indicate that the synthetic textbook method generates forget data that closely matches the target domain, performing comparably or better than expert-curated alternatives.
100
Llama3.3
Qwen2
90
Textbook Win Rate (%)
WMDP-Bio
Filter-Bio
Keyword-Bio
CTFTime
WMDP-Cyber Cyber
Filter-Cyber
Keyword- Keyword-Cyber Cyber
Figure 3: Textbook Win Rates. We perform the relevance test on the textbook dataset against the baselines in both biosecurity and cybersecurity settings. We use Llamaa.3-70B - Turbo and Queen2-VL-72B-Insie as graders.
4.3 Unlearning with Self-Generated Forget Sets
To test whether the textbook synthetic method can work without a strong external generator like GPT-4o-mini, we evaluate unlearning performance when the textbook dataset is pro- duced by the same model that is later unlearned. For each task, we consider three variants: textbook data generated by GPT-4o-mini, by the target model itself (Mistral or Llama3), and by the other peer model. We run this experiment using RR and focus on the biosecurity and Harry Potter unlearning tasks.
8

---

Published as a conference paper at COLM 2025
As shown in Figure 4, the model-generated forget sets are effective even for smaller models. For instance, Mistral produces forget sets that perform as well as or better than that from GPT-4o-mini. While self-generated data does not always match the performance of GPT--o- mini, as Llama3 performs worse on its own generated biosecurity set, remains strong alternative. The results suggest promising trade-off, offering competitive unlearning performance along with improved reproducibility, lower cost, and ease of use.
Mistral Textbook-Bio
Laam-33 Textbook-Bio
Mistral Textbook-HP
Laam-33 Textbook-HP
Unleam Utiliity (T)
46-
20
12
10
GPT-4o mini
Mistral
Liama3
Figure 4: Self-Generated Textbook Sets Unlearning Results. We evaluate unlearning performance using textbook datasets generated by GPT-4o-mini, the target model itself, and the peer model. Please check the full unlearning results in Appendix C.
Related Work
Machine unlearning (Cao & Yang, 2015) has emerged as powerful but lightweight paradigm to remove undesirable knowledge from trained foundation models. Most re- cent unlearning methods, including those featured in our experiments, perform low-rank update to remove parametric knowledge from the model (Yao et al., 2024; Zhang et al., 2024; Tamirisa al., 2025; Li et al., 2024; Zou al., 2024; Gandikota et al., 2024), while preserving general model performance. These methods assume access to an expert-curated forget set deffnee as samples representative the domain to unlearn-in order to remove concepts such as gender bias (Belrose et al., 2023), harmful behaviors (Yao et al., 2024; Liu et al., 2024), as well as fictional or hazardous knowledge (Eldan Russinovich, 2023; Li et al., 2024). Our work aims to design generic methods that synthesize forget sets to remove arbitrary target domains, without assuming access to expert curators.
Synthetic data generation refers to the procedure of sourcing artificially generated data for targeted improvements of model behaviors (Adler et al., 2024). To date, it has driven many progresses in large language model post-training (Abdin et al., 2024a;b), especially in areas such as reasoning (Guo et al., 2025; Muennighoff et al., 2025; Lambert et al., 2024) and problem solving (Trinh al., 2024; Chervonyi et al., 2025). In contrast, curating forget set to eliminate hazardous knowledge remains an expert-in-the-loop procedure that may cost hundreds of thousands of dollars (Li al., 2024). Our work aims to bridge this gap.
Conclusion
We present a scalable, automated framework for constructing forget sets using language models, replacing the need for expert curation with structured, three-stage synthetic text- book generation pipeline. Our approach requires only domain name as input and produces high-diversity, pedagogically grounded content, which empirically demonstrate to be highly effective across multiple unlearning settings. Compared to the self-constructed base- lines and expert-curated forget sets, our method achieves superior or comparable unlearning performance while preserving general model capabilities. As concerns around LLM misuse continue grow, our results suggest promising path toward practical, scalable unlearning for a wide range of emerging domains without the need for manual intervention.

---

Published as a conference paper at COLM 2025
Acknowledgements
Thanks the USC NLP group for their helpful feedback, and Yaowen Ye and Tianyi Qiu for valuable discussions. The work was supported in part by the National Science Foundation under Grant No. IIS-2403436. Any opinions, findings, and conclusions or recommendations expressed this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Additional thanks to Yiru for her personal patience, support, and encouragement.
References
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arx preprint arXiv:2404. 14219, 2024a.
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report, 2024b. URL https: //arxiv. abs 0t/.//222...990.
Bo Adler, Niket Agarwal, Ashwath Aithal, Dong Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, et al. Nemotron-4 340b technical report. arXiv preprint Xrio:22406.1174, 2024.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-oddds, Ben Mann, Darío Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedback, 2022. URL https://arxix: org/ absp//bb/22..... 08073.
Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems, 36:66044-66066 2023.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463-480 IEEE, 2015.
Zora Che, Stephen Casper, Robert Kirk, Anirudh Satheesh, Stewart Slocum, Lev E McKinney, Rohit Gandikota, Aidan Ewart, Domenic Rosati, Zichu Wu, Zikui Cai, Bilal Chughtai, Yarin Gal, Furong Huang, and Dylan Hadfield-Menell Model tampering attacks enable more rigorous evaluations of Ilm capabilities, 2025. URL https::/arrxiv.or//aaa 05209.
Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah I. Abdin. On the diversity of synthetic data and its impact on training large language models, 2024. URL https: //axxiv http:://wwwwv..g/a///////// arg/abs/2410 15226.
Yuri Chervonyi, Trieu H Trinh, Miroslav Olšák, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V Le, and Thang Luong. Gold-medalist performance in solving olympiad geometry with alphageometry2. arXio preprint axXit:2502.0354, 2025.
10

---

Published as a conference paper at COLM 2025
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL 14168.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in Ilms, 2023. URL https: /ttpxv/axvvvvv//////a////////ava wttpaaawww.ooo////111111 02238.
Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, and Sijia Liu. Towards Ilm unlearning resilient to relearning attacks: A sharpness-aware minimization perspective and beyond, 2025. URL httttp:://rriiv.or//b///////
05374.
Rohit Gandikota, Sheridan Feucht, Samuel Marks, and David Bau. Erasing conceptual knowledge from language models, 2024. URL /
027660.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling, 2020. URL https://arxiv.or///// 2rgg/ab//1111.00027 00027.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. The llama 3 herd of models. arviv preprint ar arXiv:2407.21760 217783, 2024.
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arv preprint ar 2..6:::33...64... 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. ar Xiv preprint ar arXiv:2501.122448 2025.
Luxi He, Yangsibo Huang, Weijia Shi, Tinghao Xie, Haotian Liu, Yue Wang, Luke Zettle- moyer, Chiyuan Zhang, Danqi Chen, and Peter Henderson. Fantastic copyrighted beasts and how (not) to generate them, 2024. URL httptttss//aaxiioor///aaaaa
14526.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021. URL https://arxis: https///aaiiv..o/bbbb/////s wtgg/bss//09...... 2ttp://2299003300 03300.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://arxiv.. wtw.://rg/////////////oo/// 2310 0682...
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017. URL
https://arxii.oor/bbb///r /000//000011000055111
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. T ulu 3: Pushing frontiers in open language model post-training. arviv preprint ar arxi::2411.15222 2024.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert- Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin Tienken- Harder, Kevin Shih, Kemper Talley, John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William Qian, Kallol Krishna
11

---

Published as a conference paper at COLM 2025
Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks. The wmdp benchmark: Measuring and reducing malicious use with unlearning, 2024. URL hwtph:tss///ratttt http::://arri..or///b////222 .0211 8.
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning. ar preprint ar Xii::4402.100558 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016. URL https:://ariiv org/abs/11 /1ebs/////...0 07843.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. ar Xiv preprint ar arXiv:2501.19393 2025.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022. URL
Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating Ilms with fewer examples, 2024. URL https: //arxiv.org/aas//422.111 14992.
Jonas B. Sandbrink. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools, 2023. URL https::/arxiv..ot wttpaaawwiii..rrra//////////// 13952.
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Mened and Stephen Casper. Latent adversarial training improves robustness to persistent harmful behaviors in Ilms, 2024. URL
http:///axii...oo////////// 15549.
Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. Tamper-resistant safeguards for open-weight Ilms, 2025. ÜRL https: http:///axxxv.o.o/bbbbbe
00761.
Liping Tang, Nikhil Ranjan, Omkar Pangarkar, Xuezhi Liang, Zhen Wang, Li An, Bhaskar Rao, Linghao Jin, Huijuan Wang, Zhoujun Cheng, Suqi Sun, Cun Mu, Victor Miller, Xuezhe Ma, Yue Peng, Zhengzhong Liu, and Eric Xing. TxT360: A Top-Quality LLM Pre-training Dataset Requires the Perfect Blend. 2024. URL https::/huuggiggeeee co/ spaces/LLM336/TXX330.
Trieu H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476 482, 2024.
Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Parag Shah, Yujia Bao, Yang Liu, and Wei Wei. Lim unlearning via loss adjustment with only forget data, 2024. URL https://aaxxv..or//bb///111 11143.
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does Ilm safety training fail?, 2023. URL https://araivoor//abas
02483.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. language model unlearning. Advances in Neural Information Processing Systems, 37:105425-105475 2024.
Xiaojian Yuan, Tianyu Pang, Chao Du, Kejiang Chen, Weiming Zhang, and Min Lin. A closer look at machine unlearning for large language models, 2025. URL https: //arxiv. org/abs/2410.0811
12

---

Published as a conference paper at COLM 2025
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Soggativepeefeeeee optimization: From catastrophic collapse to effective unlearning, 2024. URL 05868.
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models, 2018. URL https:// arxiv.org/abs/1800 01886.
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv.orr http://axxii.oorr/b/////// wrgg/br///////////// 15043.
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers, 2024. URL https: http://arxii..orra///////////// 04313.
13

---

Published as a conference paper at COLM 2025
Appendix
A Unlearning Grid Search Details
A.1 Hyperparameters
In all experiments, use fixed random seed of 42 to sample from the retain set (Wiki- Text), and multiple random seeds to sample from the forget set. Below are the detailed hyperparameters for each method:
RMU (Li et al., 2024):
Layer Fine-tuning:
Layers: 5, 6, 7
Alpha: 100, 1000, 10000
Steering coefficient: 5, 50, 500
Learning rate: 1e-5
Effective batch size: 4
Steps: 50, 100
Random seeds: 358, 23597, 2,
Sample max length: 512
ELM (Gandikota et al., 2024):
LoRA Fine-tuning:
LoRA Rank: 64
LoRA 16
LoRA dropout: 0.05
Retain loss scale: 0.1, 1, 10
Consistency loss scale:
Erase loss scale: 0.1, 1,5
Learning rate: 5e-5
Effective batch size: 8
Steps: 800
Random seeds: 358, 23597, 2, 71
Sample max length: 256
RR (Zou et al., 2024)
LoRA Fine-tuning:
-LoRA Rank: 16
LoRA a: 16
LoRA dropout: 0.05
LoRRA Alpha: 10
Target layers: 10, 20
Transform layers:
Learning rate: 5e-4, 1e-4, 5e-5
Effective batch size: 8
14

---

Published as a conference paper at COLM 2025
Steps: 100, 200
Random seeds: 358, 23597, 2,
Sample length: 256
A.2 More Grid Search Results
We also provide the Max Entropy unlearning results for and Llama- 3.1-SB-Instruct.
Max Entropy (Yuan et al., 2025):
Full model fine-tuning
Alpha: 1, 10, 100, 1000
Learning rate: 5e-5, 1e-4, 5e-6, 1e-6
Effective batch size: 8
Steps: 250
Random seeds: 358, 2,
Sample max length: 2
Model
Method
Dataset
Unlearn Utility
General 4
WMDP
(1 GSMSK
TriviaQA
0.675
0,642
0,302
0.68
WMDP-Bio
15:38
-11.78
0:388
0.600
0431
0.85
Mistral
Max Entropy
Texbbek-Bia
-15.66
0.488
0,502
0.423
0.426
Filter-Bio
-0.633
Keyword-Bel Bio
-0.09
-0.00
0.674
0.643
0.496
0.569
0.728
0.625
0.772
0.518
WMDP-Bio
7.00
-21.35
0:471
0.391
0.600
0,270
Ilamm331
Max Entropy
Textbek-Bio
0.58
1.22
0.727
0.640
0.780
0.516
Filter
Keyword-Bin Bin
0.62
0.687
0.726
0.660
0.781
0.514
Table 5: Biosecurity Additional Unlearning Results.
Model
Method
Dataset
Unleam Utility (1)
General Cap. A (t)
WMDP ())
tiny MMLU GSM#K (1)
TriviaQA (1)
0.512
CTFTime
1,82
0.383
0.636
0.422
0.592
WMDP WMDP-Cyber
2.72
-13.11
0.338
0.627
0.368
0.509
Mistral
Max Entropy
Textbok-Cyber
10.99
0.276
0.405
0.504
Fybeer-Cybrr
.0.11
N/A
0.364
0.614
0.432
0.456
Keyword-Cyber
0.11
0.001
0.628
0.468
0.541
0.439
0.625
0.772
0.518
CTPTime
1.8
1.10
0.435
0.642
0.777
0.519
Llamaã.1 Max Entropy
WMDP-Cyber
Textbok-Cyber
1.06
0.455
0.611
0.782
0.517
byttr
0.93
0.653
0.455
0.641
0.776
0.518
Keyword-Cytee Cyber
Table Cybersecurity Additional Unlearning Results.
15

---

Published as a conference paper at COLM 2025
B
Dataset Construction Details
B.1 Textbook Synthetic Method
Prompt Type
Prompt Text
System
You are an expert and seasoned scholar with 20+ years of academic experience in {domain).
Subdomain
Give the names of ten subdomains of [domain).
Bullet Point
You are writing an engaging, accessible, and age-approri textbook on [subdomain] within {domain) for (audience). Craft a set of bullet points that cover essential concepts and practical applications. The bullet points should be clear, non-hierarchical, and self-contained.
Chapter
You are writing an engaging, accessible, and age-appropriate textbook on [subdomain) [domain] for (audience). Please write a comprehensive chapter based on the bullet point: (bullet point]. The chapter should be self-contained, on topic, and match the audience's knowledge level and understanding.
Table 7: Prompt Templates for Synthetic Textbook Generation.
Forget Set
Example
Example 1: Let's break down the steps involved in learning from incidents. First, after an incident occurs, the cybersecurity team will collect all the information they can about what happened. This includes looking at logs (which are like detailed diaries that compu
Textbook-Cyber
Example Collaboration and information sharing among organizations can further enhance the efficacy of tactical threat intelligence. By participating in threat intelligence sharing communities, organizations can pool their knowledge and experiences regarding threat
Example 1: A more ethically sound approach to biosecurity involves adopting participatory governance models that prioritize collaboration with local com- munities and indigenous peoples. Such models emphasize the importance of consent and co-management, wherein local
Textbook-Bi
Example Now, let's dive into how various cultures look after their health. People around the world have unique customs and practices that help them stay well. For example, in many Asian cultures, tea is a big part of daily life. They believe that different kinds
The interaction between healing spells and potions in the Wizarding World provides a unique lens through which to examine the intersection of traditional and contemporary health practices. On one hand, spells represent contemporary, almost instantaneous
Example 1:
Textbook-HP
Example In conclusion, the centuries-long relationship between wizards and Muggles is marked by a history of separation, prejudice, and gradually increasing understanding. While significant obstacles remain, the progress made in recent decades demonstrates that wi
Table 8; Examples (first 256 characters) from Textbook-Bio, Mextbook-Cyber. and Textbook-HP.
B.2 Forget Set Baselines
We summarize the baseline forget sets used in our experiments across biosecurity and cybersecurity domains in Table Prompt templates for Keyword-Bio and Keyword-Cyber are in Table 10; prompt templates for Filter-Cyber are in Table 11.
16

---

Published as a conference paper at COLM 2025
Forget Set
Description
WMDP-Bio
The official biosecurity forget set in WMDP, consisting of collection of PubMed papers related to biological threats.
Filter-Bio
A filtering- -batid subset of the WMDP biosecurity forget set proposed by Tamirisa (2025), where a subset of The Pile is filtered using the keyword "Biology".
Keyword-Bio
A synthetic dataset 20,000 examples generated by prompting GPT-4o-mini on key facts about the keyword biosecurity.
CTFTime
The official cybersecurity forget set in Tamirisa al. (2025), consisting of CTF writeups scraped from CTFtime.
WMDP-Cyber
The official cybersecurity forget set WMDP, consisting passages scraped via keyword search on GitHub.
Filter-Cyber
A filtered subset of Txt360 ArXiv, aabstaca and S2ORCcabstract datasets using the keyword "Computer Science".
Keyword-Cyber
A synthetic dataset of 20,000 examples generated by prompting GPT-4o-mini on key facts about the keyword cybersecurity.
Table
Baseline forget sets used our study. The datasets span both real and synthetic text collections related to biosecurity and cybersecurity.
Prompt Type
Prompt Text
System
You an expert assistant with extensive knowledge in various domains. Provide clear, precise, and factually accurate responses based strictly on verified information relevant to the given domain.
Subdomain
Give me five subdomains of {domain), separate the names with and only with "."
Fact
State ron-repeti key fact in the domain of (subdomain) within (domain), answering directly without introductory phrases or prefacing.
Table 10: Prompt Templates for Keyword-Bio and Keyword-Cyber.
Prompt Type
Prompt Text
System
You are a helpful assistant that classifies whether paper falls into a specific domain.
Filter
Classify whether the following paper falls into the domain of [domain]. Respond with True/False.
Input: (paper content)
Output:
Table 11: Prompt Templates for Filter-Cyber.
B.3
Ablation Dataset Construction
The three ablation configurations are:
(- BP) removes bullet point creation. After enumerating subdomains and audience knowledge levels, directly generate 5000 chapters by sampling 125 chapters from each of the 10 subdomains x audience levels, skipping the intermediate bullet point step.
(- BP & Aud) builds on (-bullet point) by also removing audience enumeration. The dataset is generated by sampling 500 chapters from each the 10 subdomains, for total of 5000 chapters.
17

---

Published as a conference paper at COLM 2025
(- BP & Aud & Sdom) removes all intermediate steps. We directly sample 5000
chapters from the target domain without subdomain or audience specification.
C
More Analysis Results
MMLU (
Dataset
Uiility
General Cap. A (t) ())P()) (f)
TriviaQA
full
Bio
Med
0.675
0.502
0.568
0.596
0.729
0.578
Textbook-Bio
22.41
-9:43
0.309
0:476
0.317
0.584
0.660
0.536
Bullet
RMU
Audience
21.27
-7.34
0.356
0.490
0.545
0.539
0.699
0.576
Subdomain
26.77
-9.2
0.251
0.483
0,338
0.520
0.635
0.526
Mistral
Textbook-Bio
21.52
-9.9
0.318
0:452
0.569
0.477
0.367
0.468
Hullet Point
14.33
-1.62
0.471
0.587
Audience
17.86
-17.82
0.313
0.361
0.499
0.458
0.521
0:439
Subdomain
16.08
-2992
0.438
0.468
0.575
0.576
0.346
0.553
Textbook-Bio
13:48
-22.7
0.340
0.327
0.528
0.439
0.398
0.382
Bullet Point
11.37
-14.59
0.423
0.397
0.963
0.465
0.447
Audience
9.38
-9.18
0.486
0.430
0.584
0.500
0.530
0.503
Subdomain
7.27
66.22
0.533
0.436
0.587
0.540
0.618
0.534
0,710
0.753
0.511
0.638
0.743
0.630
Textbaok-Bio
15.54
0.34
0.492
0.796
0.513
0.597
0.727
0.568
RMU
13.55
0.502
0.758
Audience
0.489
0.753
Subdomain
20.29
-1.81
0.409
0.750
0.517
0.558
0.493
0.543
Llama-3
Bextbook-Bio
21.39
-4.08
0.377
0.759
0-491
0.581
0.734
0.59
Bullet
RR
Audience
5.34
-0.05
0.634
0.753
0.516
0.631
0.736
0.630
Subdommain
5
0.24
0.641
0.755
0.516
0.635
0.738
0.628
Textbook-Bia
12.63
-6.13
0.450
0.706
399
0.509
0.528
0.383
Bullet
ELM
Audience
8.847
-1.82
0.554
0.731
0.470
0.563
0.660
0.514
Subdomain
9.628
2,87
0.570
0.747
0.443
0.582
0.701
D.984
Table 12: Biosecurity Unlearning Results for Synthetic Method Ablation.
18

---

Published as a conference paper at COLM 2025
Model
Method
Dataset
Unleam Uuiliyy (f)
General Cap. (t)
WMDP ())
GSMSK (1)
TriviaQA (f)
MMLU (1>
full
0.415
0.502
0.568
0.596
0.660
0.500
Textbook-Cyber
19.46
-24
0.244
0.887
0.542
0.997
0.617
0.500
RMU
Point
-288
0.493
0.582
Audience
20.44
-2.15
0.237
0.490
0.556
0.585
0.307
0.460
Subdomain
19.43
-1.79
0.266
0.890
0.561
0.565
0,77
0.178
Mistral
Textbook Cybeer
2026
-1.85
0.239
0.477
0.586
0.574
0.337
0.460
Hullet Point
-1.57
0.246
0:482
Audience
19.33
-166
0.248
0.482
0.57%
0.581
0.300
0.498
Subdomain
0.238
0.482
0.591
Textbook-Cyber
14:67
-6.82
0.265
0:428
0.5%
0.533
0.200
0.406
Bullet Point
13.92
-6.58
0.272
0.426
0.588
0.547
0,305
0.425
ELM
Audience
-5.71
0.269
0:447
0.933
0.533
0.333
0.410
Subdomain
-5.05
0.332
0.441
0.562
0.508
0.468
0.733
0.638
Textbook Cyher
17.29
-8.59
0.266
0.715
0.508
0.510
0.93
0.317
RMU
Bullet Point
18.81
-6.45
0.262
0.749
0.508
0.522
0.277
0.290
Audienue
-1.6
0.261
0.755
0.30
0.542
0.900
0.337
Subdomain
22.35
-2.14
0.249
0.235
0.592
0.420
Llama
Textbook Cyher
22.60
0.04
0.257
0.235
0.514
0.633
0.350
0.477
Bullet Point
22.74
0.100
0.256
0.755
0.518
0.630
0.213
0.487
RR
Audience
22.35
-0.03
0.259
0.756
0.519
0.625
0.263
0.450
Subdomain
20.57
0.03
0.276
0.754
0.515
0.632
0,317
0.483
Textbook Textbook-Cyber
9.788
-2.82
0.359
0.738
0:479
0.609
0.510
0.570
Hullet Point
-1.57
0.359
ELM
Audience
10.38
0.08
0.366
0.748
0.499
0.549
0.360
9.370
Subdomain
10.88
-0.31
0.355
0.745
0.482
0.597
0.490
0.400
Table 13: Cybersecurity Unlearning Results with Synthetic Method Ablation.
Model
Method
Dataset
Unleam Utility (t)
General Cap. A
Unleam Metrie ())
GSMBK (1)
TriviQA (1)
(?)
full
Bio
Med
Textbook: Bio-CPTHe
21.52
-9.9
0,318/0.675
0452
0.569
0.677
0.567
0.668
Testbook-Bicc MistMiaaaal
0,273/0.675
0.532
Tootbook: Bio- Lhmm3
12.68
-1.43
0.89100.75
0.471
0.377
0.591
0,708
0.584
RR
Textbook-II--GP GPT
32.62
0.1
0.271/0.717
0.699
0.586
0.637
Textbook-HP..e HP-Mstraa
32.36
-0.98
0.247/0.717
0.865
0.565
0.905
Textbook-HP-Lun HP-Llama3
32.33
01.18
0,46/00.17
04877
0.366
0.567
0.630
Betbook-Bio-GPPE
21.39
-1.8
0.377/0.710
0.59
0.490
0.881
0.34
0.599
Testbook-Bio
0.44700.710
Llama33
Tootbook: Bio- Lamm3
12.68
0.1
0.531/0.710
0.759
0.511
0,638
0,757
0.006
RR
Texthook- GPT
33-11
0.58
0.258/0.756
0.766
0.511
0.600
Testbook-HfP-Minn
34.06
0.241/0.756
0.750
0.512
Textbook-HP-Llene
33.95
1.26
0.232/0.756
0.755
0.329
0.638
Table 14: Self-Generated Forget Sets Unlearning Results. We evaluate unlearning performance using textbook datasets generated by GPT-4o mini, the target model itself, and the peer model. All datasets constructed using the full three-step generation process. For the biosecurity task, the unlearning metric WMDP biology accuracy; for the Harry Potter task, is HP MCQ accuracy.
19


Reviews:
[{"review_id": "PP4bUUATsc", "reviewer": "Reviewer_SAXR", "summary": "This paper proposes an automated approach to develop a target domain specific dataset that can be used in the LLM post-training phase to perform LLM unlearning. To improve safety and prevent the generation of copyrighted content, LLM unlearning step uses expert curated dataset to guide a model to forget a specific set of knowledge. Developing such forget sets is time consuming and expensive process. \n\nThe proposed method takes a step to overcome this limitation. The input to the system is the target domain name, and then the system uses LLM to generate textbook-style data using a structured prompting pipeline. Given the input domain, it produces 10 subdomains, and 20 bullet points for each subdomain tailored to 4 audience knowledge levels (elementary school, high school, undergraduate, and PhD). For each of these 800 (10x4x20) bullet points, 5 textbook-style chapters are generated. Then the chapters are split into sentences and 20K longest ones are takes to create the final forget set. GPT-4o-mini is used to generate the data.\n\nThe authors evaluated their approach by using the generated synthetic dataset in LLM unlearning, where they observed superior performance than alternative baselines and competitive performance to expert curated forget sets in WMDP. The evaluation involves two models (Mistral-7B-Instruct-v0.3, Llama3-8B-Instruct), three unlearning methods (Random Mask Unlearning, Representation Routing, and Erasure of Language Memory) and three target domains (Biosecurity and Cybersecurity from WMDP, and Harry Potter). They additionally compared against filtering-based forget sets by constructing forgets sets from The Pile and TxT360, and it underperformed the forget sets produced by the proposed approach.\n\n\n* Increased my support for the paper after reading the author response.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The proposed method is very simple but well-motivated and clearly explained. \n* The data generated from the pipeline has shown high diversity, improving robustness of the unlearning process. This is a solid strength of the method.\n* The experimental setup involves grid searching over parameters of the proposed method to have a set of strong baselines to compare against.\n* The proposed method relies on the generation of synthetic data and the authors showed that even with smaller open source model, strong and impactful synthetic data can be generated to gain strong performance in unlearning.", "reasons_to_reject": "* The ablation study showed that all steps are important. But, an important angle to look at is the knowledge levels' impact and how much data we really need to generate. Do we need all these four levels? Instead of generating a detailed chapter, can a sentence or expanded keyword list for the bullet points do the work? The method is really interesting, but such insights can save computational costs and help with the interpretability as well.\n* A qualitative analysis to compare a handful of samples from a human-developed forget sets and the proposed synthetic data would be helpful to identify the gaps.\n\nMinor\n------\n* In Table 1 caption, explain what is the 'General Cap' column, or just right 'General Capability' as the column header; and how is it computed. At a first glance it is confusing. Also add a 'lower is better' symbol to be consistant with other columns.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9XHn3LUVZS", "reviewer": "Reviewer_GsKB", "summary": "This study investigates how well \"unlearning\" (in particular, removing information about a particular topic from an LLM) can be accomplished using an automatically constructed dataset rather than a human-curated set.  The authors demonstrate the feasibility of their technique (called \"Synthetic Textbook Generation\") while comparing it to the human-curated forget set baseline and two other baselines: generation of sentences/facts from a keyword and extracting relevant data samples from a large corpus (in this case \"The Pile\") via filtering.  All the approaches use GPT-4o-mini to accomplish the generation or filtering.\n\nOverall, the biggest takeaway is that the synthetic techniques are comparable to the unlearning using an expert-curated dataset.  By choosing three top \"unlearning\" methods and two strong open source LLMs, the authors steer clear of trying to provide a \"best\" method/model overall, and instead show that their dataset leads to stronger performance than either of the baselines and usually is very close (or exceeding) the human dataset for each technique and model.  The authors cover a breadth of topics by choosing two technical topic areas and a literary topic.  They also perform an ablation study on their (somewhat complicated) textbook generation method to show that each step adds something.  And they even ablate the GPT model by using Mistral and Llama (their two models) to do the data generation, to good effect.\n\nThere are a few downsides to this paper.  The large tables of numbers (Tables 1-3) present so much information it is hard to digest them and the interpretation of the tables (part of sections 3.3 and 3.4) is much too brief.  They do offer an \"average\" (Unlearn Utility) which I think is a summary of the other metrics in the table, but the explanation of it is not fully clear (where do the two S values come from?).  They also discuss Pareto frontiers but the graphs in Figure 2 are quite hard to interpret as such, and their 3 line summary is not sufficient.\n\nI consider this paper a worthy addition to the conference and recommend it is accepted.", "questions": "", "limitations": "", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- strong results over reasonable baselines, competitive with a human baseline\n- a varied selection of methods, models, and topics\n- convincing ablation study", "reasons_to_reject": "- explanations and interpretation of results are a bit meager in some places, though the numbers in general seem to show the right trends\n- reliance on GPT-4o-mini for the main technique (though this is mitigated by Section 4.3)", "questions_to_authors": "- Is the unlearn utility (Equation 2) in fact a weighted average of the other metrics?  Can you give more details on how the S_f and S_r are calculated?\n- Figure 2 - I don't see the Pareto frontier --- is it defined by the hollow markers?  Are some missing?  It is very hard to see these... maybe the graphs are just too noisy?\n- what prompts are used for the filtering and keyword generation baselines?", "extra_scores": {}}, {"review_id": "fBCfFxFXqi", "reviewer": "Reviewer_zZHD", "summary": "This work proposes a prompting method to generate forget sets for LLM unlearning, and shows that this approach outperforms the state-of-the-art in multiple conditions.\n\nWork description is very clear and writing is very good, reporting important concepts in a clear manner.\n\nThis work does not propose a really new and original approach: it boils down adapting data generation pipeline\nto the unlearning context. However, it is well-known in the LLM community that a carefully designed engineering\ncontribution may have more impact than brand new original theoretical ideas;\ntherefore, the experimental contributions and proposed design are clearly valuable and useful for the community.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The contribution is mostly experimental, and proposes a well-designed and successful pipeline that will prove useful for the community.\n- The proposed prompting method focuses on increasing diversity of synthetic data through 3 stages, which is indeed likely the best objective, given our understanding of current literature about synthetic data for LLM.\n- The experiments are nicely conducted and convincing.\n- The fact to also compare with Mistral-based data generation is very welcome, as it is the only way to guarantee reproducibility, which is crucial and mandatory to qualify any study as scientific.", "reasons_to_reject": "- The work does not propose original ideas, but a simple combination and (well-done !) adaptation of known ideas.\n- A motivation is the human effort required to create the forget set: this should somehow be compared to the computational cost to create the forget set with the proposed approach, and more importantly, both minimum sizes of forget sets should be compared: indeed, the human-created forget set might be of higher quality and may be smaller than the GPT4o-created forget set to reach the same level of unlearning; and if it is the case, then the additional computational cost incurred by the proposed method does not only concern creating the forget set, but also finetuning the LLM. More generally, because the main claim is to reduce the human effort, some section about costs estimation would be welcome.\n- It is clear that the proposed pipeline improves the tradeoff between unlearning and retaining general capabilities as compared to the baselines. It is no clear however, but this is a general comment that addresses many works beyond this one in the unlearning field, whether this tradeoff has become good enough to be really useful in practice.\n- Larger open source LLMs could have been used to generate the forget set.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u6YMJzjpAj", "reviewer": "Reviewer_Smv6", "summary": "This study proposes a way of creating unlearning datasets using language model themselves.\nVarious experiments demonstrate that synthetic datasets consistently outperform the baseline synthetic alternatives and are comparable to the expert-curated ones. \nAdditionally, ablation studies reveal that the multi-step generation pipeline significantly boosts data diversity, which in turn improves unlearning utility.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This study proposes a novel perspective for unlearning LLMs by creating synthetic datasets.\n- The proposed approach is also practical.\n- Experiments on wide range of benchmarks (biosecurity, cybersecurity, and Harry Potter novels) and models (Mistral, llama3) demonstrates the effectiveness of the proposed method, supporting the main claim with various experiments.\n- An ablation study provides us with a new finding that higher diversity is important for achieving robust unlearning performance overall.\n- The paper is well written and easy to understand.", "reasons_to_reject": "- The proposed method is not so exciting, i.e., just creating prompting and its application (pipeline).\n- It is unclear in the paper if the data samples are synthesized by target model itself, e.g., Does a Mistral model create unlearning datasets and the same model is used as target for unlearning?\n- The study does not investigate if the synthesized data is reliable or not. For example, do the synthesized datasets contain hallucination? Do the synthesized datasets similar to the ground truth datasets? (e.g., semantic similarity analysis with Bert)\n- A representative baseline unlearning methods: Gradient Ascent (https://arxiv.org/pdf/2210.01504) is not included in the experiment. Perhaps Negative Preference Optimization (https://arxiv.org/pdf/2404.05868) is also a major approach.", "questions_to_authors": "See ``Reasons To Reject''.", "extra_scores": {}}]

Author rebuttal:

Meta-review: I have read the reviews and the discussions and I'm recommending that this paper be accepted.

The reviewers have listed the following reasons to reject the paper and I think they are all reasonable points to bring up: 
- The discussion and interpretation of results is weak in some parts of the paper.
- More open source (data/weights) models could be considered.

The reviewers also list these strong reasons to accept the paper:
- The method is simple and well explained.
- Strong results over reasonable baselines.
- Compelling ablations
- The writing is clear and easy to understand.

**New point from the AC: This paper uses an unlearning method called RMU and the citation the authors give for this method is a 2024 paper by Li et al. However, that original paper by Li et al. names and describes a method called Representation Misdirection for Unlearning. In this paper, the authors say RMU stands for Random Mask Unlearning. If this is a simple mixup, it should be fixed by correcting the full name of the method or the citation.**

I think work on avoiding expensive gathering of expert data is timely and of interest to the community.
Decision: Accept