Reveal and Release: Iterative LLM Unlearning with Self-generated Data (2025)

Abstract: Large language model (LLM) unlearning has demonstrated effectiveness in removing the influence of undesirable data (also known as forget data). Existing approaches typically assume full access to the forget dataset, overlooking two key challenges: (1) Forget data is often privacy-sensitive, rare, or legally regulated, making it expensive or impractical to obtain (2) The distribution of available forget data may not align with how that information is represented within the model. To address these limitations, we propose a “Reveal-and-Release” method to unlearn with self-generated data, where we prompt the model to reveal what it knows using optimized instructions. To fully utilize the self-generated forget data, we propose an iterative unlearning framework, where we make incremental adjustments to the model’s weight space with parameter-efficient modules trained on the forget data. Experimental results demonstrate that our method balances the tradeoff between forget quality and utility preservation.

## introduction large language models (llms) function as vast knowledge repositories, drawing on information embedded in their parameters in response to user inputs @xcite . however, the scope of their knowledge is fixed at the time of training, lacking effective means to verify and may produce responses that are outdated, incorrect, or even harmful @xcite . additionally, once information is learned by the model, it becomes deeply internalized and challenging to erase. machine unlearning has become a promising area of research aimed at addressing these limitations. a straightforward approach-known as exact unlearning-involves removing undesirable data from the training corpus and retraining the model from scratch, which is prohibitively resource-intensive for modern llms. researchers have explored approximate unlearning, which seeks to remove relevant knowledge without full retraining. the goal is to efficiently and selectively erase the influence of targeted information while maintaining the model's performance on nontargeted tasks @xcite . current methods include gradient ascent that effectively guide models to forget by optimizing in the opposite direction of original learning @xcite ; knowledge editing methods that locate and directly modify network parameters to perform targeted information removal @xcite ; and influence function approaches that identify and neutralize the impact of specific training examples @xcite . in a typical machine-unlearning process, one crucial factor is the data, specifically, the information to be forgotten and the information to be retained @xcite , which we refer to as forget data and retain data. most unlearning methods require well-annotated forget data. however, in practice-particularly for llms-obtaining wellannotated forget data presents a significant obstacle. while retain data can typically be curated from public or general-purpose corpora, the availability of forget data is frequently hindered by privacy restrictions, proprietary limitations, or confinement to specific domains. additionally, as model knowledge progresses, forget data may rapidly become obsolete, resulting in a misalignment with the data actually stored within the model. moreover, existing unlearning benchmarks often assume access to the model's original training data or an exact forget subset @xcite , which is unrealistic for massive and private corpora. in other cases, forget data consists of publicly sourced approximations @xcite , herein termed as external data; however, such data may not faithfully represent how the information is genuinely encoded within the model. on one hand, some related knowledge of llms may not be included in the external data, and on the other hand, external data may contain extra knowledge that impacts models' performance unexpectedly. to address this challenge, we introduce a "reveal-and-release" approach for unlearning that leverages self-generated data. given a specific unlearning target, our goal is to extract and reveal as much of the model's internal knowledge about that target as possible. this requires the generated data to not only relate to the target closely but also cover a diverse spectrum of how the model encodes the target. instead of relying on well-labeled external forget data, we use a neuralucb-based instruction optimization method @xcite to generate prompts to reveal internal knowledge, focusing on the relevance and diversity of the generation (section 3.1). we refer to the resulting self-generated data as internal data. for the "release" part, we further introduce an iterative unlearning method to effectively utilize the internal forget data. inspired by parameter-efficient module (pem) composition @xcite , our approach incrementally edits the base model by merging two types of pem loras @xcite : a forget pem trained on internal forget data and a retain pem trained on retain data. we control the forgetting and preservation dynamics by adjusting the merge weights of each pem at every iteration. intuitively, the loras act like gradient ascents/descents, and multiple iterations of unlearning correspond to applying small steps of gradient optimizations. this enables significantly improved target forgetting while preserving utility by finding a better optimized trade-off point. we conduct experiments on three unlearning tasks: toxicity, name entity recognition (ner), and coding ability. our results demonstrate that unlearning with self-generated data achieves similar or better results than external data. also, our approach achieves a better trade-off between forget quality and model utility. our contributions are: ## related work data synthesis for unlearning well-annotated data is expensive to obtain. in non-llm domains, shen et al. @xcite introduce label-agnostic forgetting (laf), a supervision-free unlearning framework that manipulates representation distributions to remove forgotten data without relying on labels. peng et al. @xcite propose mixunlearn, which uses adversarially generated mixup samples to mitigate catastrophic unlearning, ensuring effective data deletion even in label-agnostic scenarios. in the domain of llms, prior work has explored using synthesized data for unlearning. cmd introduces a detoxification framework for llms that leverages synthesized data to enable unlearning of toxic behaviors @xcite . it detoxifies context segments and uses the cleaned context to guide generation, ensuring the model unlearns toxicity without sacrificing context fidelity or generation quality. rwku @xcite constructs a synthetic forget corpus by prompting llm with manually crafted templates in a single-pass manner. while this provides a straightforward way to obtain forget data, the reliance on fixed prompt templates and single-pass generation risks capturing only a narrow view of the model's internal knowledge, potentially missing out on diverse or harder-to-reach information. parameter-efficient-module for unlearning parameter-efficient fine-tuning (peft) methods such as lora @xcite have become popular for adapting llms due to their efficiency and modularity. recent research explores how these parameter-efficient modules (pems) can be composed through arithmetic operations to enable unlearning @xcite . building on this, liu et al. @xcite proposed sku, which trains multiple modules from different perspectives and merges them before a single subtraction, aiming to better capture harmful knowledge from multiple angles. ding et al. @xcite proposed a unified framework for pem-based unlearning by applying influence functions to directly update existing pems. extending this line of work, hu et al. @xcite introduced ext-sub, a method to isolate and subtract only the "deficiency capability" from an anti-expert pem. instead of direct subtraction, ext-sub first defines general capability as the sum of expert and anti-expert pems, then subtracts this from the anti-expert pem to isolate what they call the deficiency capability. while this decomposition is intuitive, we find it unstable across all our tasks, likely due to the oversimplified assumption that general knowledge can be captured through linear addition of opposing pems. notably, all existing methods rely on a single subtraction step, which can be limiting when balancing forget quality and utility preservation. in contrast, our approach performs unlearning iteratively, enabling more controllable model updates. ## method our method consists of two stages: we first obtain self-generated forget data by optimizing instructions for the llm, and then utilize the obtained data in an iterative unlearning framework. ## experiments to evaluate the effectiveness of our self-generated forget dataset, we conduct experiments on three tasks: llm detoxification, named entity recognition (ner) unlearning, and coding ability unlearning. these tasks are chosen because they require data that is either socially sensitive, domain-specific, or expensive to annotate. all experiments are performed using the llama3-8b-instruct model @xcite , and we use all-roberta-large-v1 @xcite to embed texts for diversity scores. to further assess the generalizability of our framework, we also include results on mistral-7b-instruct-v0.2 (mistral ai, 2024) (see appendix c).