Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs
Authors: Yan Scholten
Sophie Xhonneux
Leo Schwinn
Stephan Günnemann
Venue: ICLR 2026 Poster (2026)
Keywords: Machine unlearning
Model collapse
Large language models
LLMs
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We show that model collapse can be intentionally triggered to make LLMs unlearn specific information, turning it into a practical method for machine unlearning.

Abstract: Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data. We argue this not only risks reinforcing exposure to sensitive data, but also fundamentally contradicts the principle of minimizing its use. As a remedy, we propose a novel unlearning method—Partial Model Collapse (PMC), which does not require unlearning targets in the unlearning objective. Our approach is inspired by recent observations that training generative models on their own generations leads to distribution collapse, effectively removing information from model outputs. Our central insight is that model collapse can be leveraged for machine unlearning by deliberately triggering it for data we aim to remove. We theoretically analyze that our approach converges to the desired outcome, i.e. the model unlearns the data targeted for removal. We empirically demonstrate that PMC overcomes four key limitations of existing unlearning methods that explicitly optimize on unlearning targets, and more effectively removes private information from model outputs while preserving general model utility. Overall, our contributions represent an important step toward more comprehensive unlearning that better aligns with real-world privacy constraints.

Published as a conference paper at ICLR 2026
MODEL COLLAPSE Is NOT A BUG BUT A FEATURE IN MACHINE UNLEARNING FOR LLMS
Yan Scholten', Sophie Xhonneux", Leo Schwinn Stephan Günnemann"
Dept. of Computer Science & Munich Data Science Institute, Technical University of Munich 2 Mila, Université de Montréal (y scholten,, 1. schhwinn, s. s.guennemann @t de, 1pw.ooneeeeeoetetaaaaalaa com
ABSTRACT
Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating into their fine-tuning data. We argue this not only risks reinforcing exposure to sensitive data, but also fundamentally contradicts the principle of minimizing its use. As a remedy, method-Partin Model Collapse (PMC), which does not require unlearning targets in the unlearning objective. Our approach inspired by recent observations that training generative models on their own generations leads to distribution collapse, effectively removing information from model outputs. Our central insight is that model collapse can be leveraged for machine unlearning by deliberately triggering it for data we aim remove. We theoretically analyze that our approach con- verges to the desired outcome, i.e. the model unlearns the data targeted for removal. We empirically demonstrate that PMC overcomes four key limitations existing unlearning methods that explicitly optimize unlearning targets, and more effeee tively removes private information from model outputs while preserving general model utility. Overall, contributions represent important step toward more comprehensive unlearning that better aligns with real-world privacy constraints."
propose novel unlearning
INTRODUCTION
Privacy regulations and copyright laws (e.g. the GDPR (European Union, 2016)) necessitate the ability to selectively remove data from machine learning models. including Large Language Models (LLMs). While complete retraining without specific data can be optimal for information removal, it is infeasible at scale given the high computational costs of training LLMs. This motivates the need for machine unlearning techniques to erase specific information while preserving a model's 's broader capabilities.
Although recent methods have demonstrated early progress in LLM unlearning-ern either via refusal fine-tuning or gradient ascent on ground-truth sequences (Zhang et al., 2024)-thes degrade model utility and lack deeper theoretical analysis and robustness (Liu et al., 2025). In particular, argue that optimizing on ground-trume sequences to be unlearned is counterintuitive and contradicts the introduce
principle of minimizing the use of private data. Critically, we show that this dependency side effects that remain poorly understood, such as enabling adversaries to elicit data after unlcarning. These limitations highlight the need for novel unlearning methods that mitigate such risks.
In this paper, we identify notable parallels between the unlearning challenge and the phenomenon known model collapse, where iterative fine-tuning synthetic data causes information loss in the model's 's output distribution and can lead to distribution collapse (Shumailov et al., 2023; 2024; Bertrand al., 2024; Ferbach et al., 2024). We raise the following critical research question:
Can we leverage the principles underlying model collapse to develop principled approaches for machine unlearning?
To address this research question, we introduce Partial Model Collapse (PMC)-a fundamentally novel approach to machine unlearning that leverages the principles of model collapse. By iteratively fine-tuning the model on its own generations in response to sensitive questions, we can force the model's distribution to collapse on private data in targeted manner, thereby unlearning it (Figure 1).
"Equal supervision
Project page: https: http://wwwwccc tum. wwam.dd/ddmm... /pprttiaaaaa iirtiaal -ortiallmodee -mode ccol lapse/

---

Published as a conference paper at ICLR 2026
Prior work
Partial Model Collapse (ours)
What nume oul?
Generste rank responses
Fine-tune hest
Conditinal collapse
Hodwig
Hedwig
Suparviied line-tuning
Gradient ascent
Answer Hedwig
Current model
Updated model
Model the limit
Fine-tuning on or against fixed responses
Unlearning by fine-tuning on responses sampled from the model itself
We propose Partial Model Collapse (PMC), novel unlearning method that leverages the principles of model collapse to remove information from LLMs. By iteratively fine-tuning LLMs on their own generated responses, effectively removing information from model outputs. Unlike (1) fine-tuning on fixed refusals such as "I don't know", (2) using gradient ascent to optimize against fixed ground-truth sequences, PMC fine-tunes on responses the model is already likely to generate. This allows us to achieve more effective and robust unlearning without requiring fixed ground-truth sequences in the fine-tuning data.
Figure I:
trigger distribution collapse conditionally for sensitive questions,
We provide theoretical analysis showing that our approach achieves unlearning by converging to the desired outcome. We begin by motivating the method on categorical data, then extend arbitrary distributions, and ultimately adapt it for practical use in LLMs for question-answ
tasks.
In extensive experimental evaluations we demonstrate that PMC removes information from model outputs more effectively than existing methods while being theoretically principled. Notably, PMC overcomes four key limitations of prior approaches: First, it is more robust against sampling and prefilling attacks. Second, it preserves generation coherence by avoiding unintended degradation unrelated contexts. Third, it reduces information leakage by preventing unnatural suppression of correct answers, thereby mitigating vulnerability to probability-basse attacks. Fourth, it reduces information leakage in the presence of sampling and prefilling attacks. Our main contributions are:
We propose Partial Model Collapse (PMC)-a novel, theoretically grounded unlearning method based on iterative relearning on synthetically generated data.
We provide a formal analysis showing that PMC achieves unlearning by driving the model's output distribution toward a target distribution in which the influence of private data is eliminated.
We identify negative side effects in previous, target-dependent unlcarning methods, including distorted token probabilities for unlearning targets even out of context of the unlearning task and information leakage regarding supposedly unlearned knowledge in multiple choice evaluations.
Through extensive empirical evaluation, we show that PMC outperforms existing state-of-the. art unlearning methods in removing information from LLM outputs. It maintains generation coherence across tasks and shows no negative side effects that we identify in previous methods.
Overall, we introduce new paradigm for machine unlearning by harnessing the mechanism model collapse. By reframing this detrimental phenomenon as tool for targeted information removal, we enable new avenues toward more trustworthy machine learning.
2 RELATED WORK
Machine unlearning. Broadly, machine unlearning can be categorized into exact, approximate, and empirical methods. Exact unlearning seeks to ensure that the resulting model behaves if specific data had never been seen (Bourtoule et al., 2021; Yan et al., 2022), but is typically computationally infeasible scale. Approximate unlearning, while not guaranteeing complete removal, aims to reduce the influence of specific data points statistically, often drawing on tools from differential privacy (Guo et al., 2020; Neel et al., 2021; Sekhari et al., 2021; I; Ullah et al., 2021; Chien et al., 2022; Zhang et al., 2023). In contrast, empirical unlearning typically pursues more practical objectives. One common objective is to converge in data distribution toward a model that was never trained on specific data (Maini et al., 2024). Another critical objective (which we focus on) aims remove specific information from model outputs (Eldan & Russinovich, 2023; Krishnan et al., 2025). The empirical nature of these methods makes them scalable larger models including L.LMs (Jang et al., 2022).
2

---

Published as a conference paper at ICLR 2026
Machine unlearning for LLMs. Recent research has increasingly focused unlearning in the context of LLMs (Jang et al., 2022; Chen & Yang. 2023; Eldan & Russinovich, 2023; Kim et al., 2024; Lynch et al., 2024; Sheshadri et al., 2024; Li et al., 2024; Seyitoğlu et al.; Shi et al., 2025; Dorna et al., 2025). Among empirical approaches, methods based on preference optimization have shown early progress (Rafailov et al., 2023; Zhang al., 2024; Fan et al., 2024; Mekala et al., 2024), yet all of them introduce severe unicarning-utility trade-off's. Moreover, evaluating unlearning in LLMs remains an open challenge (Feng et al., 2025; Jones et al., 2025; Scholten et al., 2025). Most current methods focus on assessing the model's s ability to avoid generating specific unlearning targets, but often overlook issues such as residual information leakage (Schwinn et al., 2024; Scholten et al., 2025). In this work, we identify further negative side effects in current methods.
Model collapse in iterative refraining. The rise of Al-generated content on the web has sparked growing interest in the effects of iterative retraining, where models are repeatedly trained on their own outputs. Early studies (Shumailov et al., 2023; Alemohammad al., 2024) raised concerns by showing that model performance can degrade significantly with successive retraining iterations. In contrast, Bertrand et al. (2024) show that mixing synthetic data with the original training data can avoid model collapse and stabilize performance. Theoretical work (Dohmatob et al., 2024; Feng et al., 2024) further derives conditions under which collapse occurs. For example, iterative retraining with discrete Gaussian distributions results in collapse primarily due to statistical approximation errors (Shumailov et al., 2023; Alemohammad et al., 2024; Bertrand et al., 2024). Most recently, Ferbach et al. (2024) introduce new model for retraining in practice, where new synthetic training data is sampled according to a Bradley- Terry model with an unknown reward function. They show that retraining maximizes the underlying reward function and that mixing synthetic and original training data can prevent collapse. While model collapse has been framed as a bug in the LLM learning landscape, we show that it can be turned into feature in the context of machine unlearning.
PRELIMINARIES AND BACKGROUND
Machine unlearning, In this work, we focus on empirical machine unlearning for LLMs, defining as the problem of removing information from model outputs without retraining from scratch and, in contrast to previous works, without requiring access to ground-truth responses to sensitive questions.
Large language models. We model LLMs as parameterized functions fo V - P(V*) mapping input queries of arbitrary length to distributions over output sequences given vocabulary V, where is the Kleene operator. Output distributions can only be evaluated sequentially, i.e. the probability of output sequence = (y),.... Dm)) given input æ is the product of conditional next-token probabilities, fe(u]>>) II So W1. 1/1 2r), where So (yyll)) is the PMF over possible tokens y: V.
Iterative relearning on self-generated data. Given an initial generative model f(() fitted on a dataset (10), iterative relearning refers sequentially fine-tuning models on data sampled from their own distribution (a; x4 ~ f()) )== to produce models of the next generation f(++1), The goal is
to study the limit behavior of the sequence f() _ ( for L-> co. In this context, model collapse refers to the phenomenon that iterative relearning causes loss of information over time, and eventually leads to model collapse (Shumailov et al., 2023; 2024), i.c. the variance of the model's
generative output distribution vanishes in the limit, Var, [y] (=3 0.2
Discrete preference models. Ferbach et al. (2024) study the stability of iterative relearning on curated self- refereeereeed data in the image domain. They model the curation process using a reward function and the Bradley- Terry model (Bradley & Terry, 1952), which is a probabilistic model for pairwise comparisons of items and often used to model human preferences. The model formulates the probability of one item being preferred over another 22 using item-dependent scores (Bradley & Terry, 1952). Given n items V4, the probability of choosing ~ (x 2n)) under the generalized Bradley-Tere Tenry model BT,, with temperature can be described as
[ n:)
BT
Σj (
where r(x: is reward function that assigns score each item I3- Our approach uses this preference model to guide the unlearning process by choosing samples with higher unlearn quality.
Note that we consider collapse of the model's 's output distribution, not of the model's overall utility.
3

---

Published as a conference paper at ICLR 2026
Original distribution
Iteration
Iteration 5
Iteration 20
0.3
0.3
0.3
0.3
0.2
0.2
0.2
0.2
0.1
0.1
0.1
0.1
0.0
0.0
0.0
0.0
T Foronto
Hontrea
lanenuve Ottawa
Calgrr Edmomoo
W Vonnnee Famition Knelienee
Turonto Mantrry Recoiver
Vanc
Quawa
Calears Edmonoo DIDIDE ChENNe Eeroiiion Kitchenct
Toronto Mantren
Vancoun recouver
Quawa
Callar Edmonoo UIDIDeS CUEN Earoiiion Kitckence
Toronto Mantreal
Vancoune eecuuver
Quaan Caleart SAponon Edm ontipes CheNee Famoiton Kitllenc
Figure 2: Unlearning through iterative MLE-re uLE-relcamnngg for categorical distributions. The model's
knowledge about all other categories vanishes over time until it models target categories (bold) only.
FROM MODEL COLLAPSE TO MACHINE UNLEARNING
In the following, we theoretically motivate and derive new perspective on machine unlearning that leverages information loss caused by iterative relearning on self-generate data.
4.1 WARM-UP: UNLEARNING IN CATEGORICAL DISTRIBUTIONS VIA ITERATIVE RELEARNING
We begin by analyzing iterative relearning of categorical distributions via maximum likelihood estimation (MLE). Assume dataset 'D of categorical data with at least one datapoint per category, and initial categorical distribution Ty fitted on D using MLE. We further define a subset De S D of datapoints belonging to target categories C and delete all other datapoints from the dataset. We then introduce iterative relearning process that fits new categorical distribution πT on the target data De augmented with self self-generatte data, i.e. datapoints generated from the distribution ne of the previous iteration: De {z; 24 Te J Interestingly. this iterative relearning prevents total distribution collapse. causes information loss for all other categories and effectively achieves full unlearning of the deleted datapoints (Proof Appendix C):
Lemma 1: For any categorical distribution To, iteratively relearning ne target data De augmented with data generated from its own distribution {æ; 04 ~ Ti T causes information loss for all other (non-target) categories i: (i) 1-000 0.
Intuitively, the probability mass of the other categories gets redistributed to the target categories and results in a "partial" collapse (Figure 2). Without target data, i.e. De the iterative relearning process would converge to total distribution collapse (Shumailov et al., 2023; 2024), i.e. the model would eventually assign all probability mass to single category. The main reason this informa- tion loss are statistical approximation errors when fitting categorical distributions using maximum likelihood estimation: Given finite samples, the iterative relcarning process describes an absorbing Markov chain, which is known converge to absorbing state (Shumailov et al., 2023; 2024).
4.2 MACHINE UNLEARNING VIA ITERATIVE RELEARNING ON SELF-GENERATED DATA
Our core idea is to leverage this inherent information loss described above for machine unlearning. gradually forcing the model to forget undesired responses without explicitly optimizing against ground-truth sequences. However, this comes with several challenges for LLMs in practice:
First, the distributions we seek to collapse for LLMs are the categorical distributions over entire sequences P(V*). but LLMs only provide direct access to the categorical next-token distribution. Second, LLM unlearning is typically studied for question-answering tasks, where the objective is to unlearn answers to "forget" questions while preserving performance on all other "retain" queries. Lastly, defining suitable target distribution to converge to is challenging due to the natural language domain domain-althouge we might know which answers should be unlearned, specifying a well-formed distribution to converge to remains non-trivial without access to language model that has not been trained on the ground truth (which is usually not available without expensive retraining from scratch).
Partial model collapse using preference optimization. To overcome these challenges, propose to trigger collapse of the model's 's output distribution conditional on forget queries through an iterative neffrrence-uuied procedure while ensuring that the model retains its utility other retain queries.
4

---

Published as a conference paper at ICLR 2026
To guide the unlearning process toward desired outputs, build upon the result that iterative retraining on "curated" (filtered) geffrgenr data yields model collapse in the image domain (Ferbach et al., 2024). Specifically, we propose to unlearn responses to forget queries by (1) sampling n independent responses from the model, and (2) fine-tuning on the best response selected by a preference model. We formalize this using the generalized Bradley-Terry preference model (Section 3) logether with a bounded reward function r X + [0, r"], which assigns higher scores to preferred responses (e.g. rewarding dissimilarity of a sampled response to the response of the original model).
Let pr represent a retain distribution over query-answer pairs (which we do not want to unlearn), and py a forget distribution over questions whose answers we want to unlearn. Note that we do not require access to ground truth answers to forget questions, and we assume disjoint support of ps (q) and the marginal distribution pr pr()), i.e. we either want unlearn the response to a question or not. Given an initial model Po before unlearning, we introduce the following iterative unlearning process:
Partial Model Collapse Machine Unlearning for Q&A tasks
P++1 arg max (g (lass)))p.llogeee (lgg((p(gg)))) + E PEP
Dlg (ogp($)))
qops st Rinute rip (xl4) B.
where the set of all distributions over X. Ps the model distribution step t, and BT, the generalized Bradley-Terry preference model with temperature T (Equation 1). Intuitively, Equation 2 describes an iterative unlearning process where the next distribution maximizes the expected log- likelihood of question-answer answer queries under the retain distribution pr (for utility) and the expected log-likelihood of curated samples from the current model distribution pt conditioned on forget queries from Pf (for unlearning). The first term preserves utility and the second term is responsible for unlcarning, where the parameter À E [0, oo) balances the trade-off between utility and unlearning. Notably, this iterative process defined in Equation 2 converges to the maximum reward for any forget query E supp(p,) in the limit, i.. the model unlcarns:
Theorem 1: Let be the distribution described by Equation 2 and assume non-zero probability mass on the maximum reward Pr Prappoll (a) [r(x) = 0 for forget queries q E supp(py)- In the absence of statistical and function approximation errors, the expected reward converges to the maximum reward and its variance vanishes for any forget query E supp(pf):
E (xlg)
[er(a)]
Varz=pa
(e((a]]
t-000 0.
Intuitively, the expected reward increases each iteration (proof in Appendix D).
4.3 PARTIAL MODEL COLLAPSE UNLEARNING FOR LLMS PRACTICE
Finally, we describe our proposed PMC unlearning loss in Algorithm 1, which can be minimized using standard (stochastic) gradient-basse fine-tuning methods. Note that while Equation 2 provides novel theoretical perspective, in practice LLMs are parameterized functions fo approximating Pt, and pr and f are approximated via finite-sample datasets, denoted as the retain set of Q&A pairs D, 2.) and the forget set D, = { of questions whose answers we aim to unlearn.
Importantly, our unlcarning loss is independent of the ground-truth forget answers, thereby avoiding any direct gradient updates that could unintentionally reinforce the information seek to remove. Instead, we fine-tune on answers generated by the model itself. Specifically, we sample n responses from the model's output distribution and select one response according to a preference model. The key advantage of our approach that the samples are drawn directly from the model's 's own distribution they represent outputs the model already likely to produce. As result, fine-tuning on these samples aligns with the model's distribution. Rather than pushing the model away from specific targets, we allow it to diverge naturally by adjusting the likelihood of its own likely generations, enabling unlearning while preserving the model's utility.
Algorithm 1 PMC unlcarning loss
Require: Retain batch B, =,, C D, forget batch [q. C Dy. model fe. temperature т, and hyperparameter À
I: Compute retain loss e,
f, = T. Eig log fo (2:19)))
2: for forget question qi E B, do
3: Sample n responses
æ1...., ~ [x(1)))
4: Sample preferred response
R. (T,,
5: Compute forget loss &
& = [89] Σa,es log (b:la))
6: return Al,+ (
5

---

Published as a conference paper at ICLR 2026
PMC (ours)
GA
GD
DPO
NPO
SimNPO
IDK
(a) Phi-1.5
(b) Llama-3.2- B-B--snettttt
(c) Gemma-33 12b-it 3-1bbit
Unlearn quality (%)
100
100
100
80
80
80
209
60
20
40
60
60
70
80
90
60
70
80
90
Utility (%)
Utility (%)
Utility (%)
Figure Partial model collapse (PMC) significantly dominates baselines and expands the Pareto-front w.r.t. utility and unlearn quality for (a) Phi-1.5, (b) Llama-3.2-331 3.--st-nnttt and (c) Gemma-3- 3-biii. 2b-it.
While existing methods (GA, GD. DPO, NPO, SimNPO, and IDK) also unlearn, they cannot deviate much from the fine-tuned model without compromising the model's general capabilities. Orange vertical lines indicate utility of fine-tuned models before unlearning. Stars represent dominating points. For improved accessibility we provide this plot with symbols instead of colors in Appendix A.
EXPERIMENTAL EVALUATION
We experimentally demonstrate that the information loss in model collapse can be leveraged to achieve machine unlearning for LLMs. We also identify negative side effects in existing unlearning methods that directly optimize on ground-truth sequences and showcase positive effects of our approach, such as robustness and reduced leakage under sampling. We provide additional results in Appendix A. and refer to Appendix B for experimental setups, implementation details and reproducibility instructions.
Experimental setup. We use the TOFU dataset (Maini et al., 2024), a fictitious dataset of 4,000 question-answe anwwering pairs designed for machine unlearning. We fine-tune models on the full dataset and perform unlearning on the "forget10" split, since it has the largest forget set and thus corresponds to the most challenging split in the dataset. We provide results for an additional dataset in Appendix A.
Models. We perform experiments for the following three models: Phi-1.5 5,5 (Li et al., 2023) since is smaller and extensively studied model in the unlearning literature, and (DeepMind, 2025) since they are more recent models with strong performance across tasks. We run experiments on A100, H100 and H200 GPUs.
(Grattafiori et al., 2024) as well as Gemma-3-1
Baselines. As baselines we consider Gradient Ascent (GA), Gradient Difference (GD) (Liu et al., 2022), Negative Preference Optimization (NPO) (Zhang et al., 2024) and its simplified form (SimNPO) (Fan et al., 2024). We also compare to two baselines introduced in (Zhang et al., 2024): "I don't 't know." (IDK), which fine-tunes this fixed refusal response, and Direct Preference Optimization (DPO) (Rafailov et al., 2023), which uses IDK-phrases as positive and the ground truth as negative examples.
Metries. We evaluate using recall ROUGE-L scores (Lin, 2004), i.e. the longest common subsequence between the model's 's greedy output and the ground truth. Unlearning performance is quantified using the sum of ROUGE-L scores on the forget and paraphrased-forger sets tte latter an additional TOFU dataset allowing to quantify generalization of unlearning. We report unlearn quality as the maximal score minus the achieved score (such that larger is better), and utility, measured as the sum of ROUGE-L scores on the retain set D, and two additional TOFU datasets: world facts (117 questions) and real authors (100 questions), which allow to assess general knowledge retention. We approximate retain performance on D, using a (random but fixed) subset of 400 retain samples. We further normalize scores for better readability by dividing by the maximum possible score.
Reward function. The design of the reward function r(x) can range from simple ROUGE-based rewards to more complex reward models trained on human preferences and is highly application- dependent since it determines post-unlearning behavior. Our goal is to demonstrate the effectiveness of PMC in removing information from model outputs and we therefore choose BOUGE-basee rewards for their simplicity. Specifically, output, i.c., r(x) = ROUGE-L(z, y) E [o, 1), where y is the model's 's original (greedy-decoding) answer for forget question g Dy and is the sampled output as described in Algorithm 1.
the ROUGE-L score between the model's 's original and current

---

Published as a conference paper at ICLR 2026
5.1 PARTIAL MODEL COLLAPSE ACHIEVES MORE EFFECTIVE UNLEARNING
In a series of experiments we compare our proposed partial model collapse (PMC) to the baselines (GA, GD, DPO, NPO, SimNPO, and IDK). Since all methods involve multiple hyperparameters. we perform a grid search for all methods. To ensure fair comparison. we explore 100 different configurations for each method, covering a broad range of hyperparameter combinations while keeping the number of trials consistent across methods (details in Appendix B). We repeat each experiment five times using different random seeds, and report mean utility and unlearn quality.
Notably, PMC significantly dominates all baselines in the unllity-uleerning trade-off' and expands the Pareto-front, achieving strong unlearn quality while maintaining high utility across models (Figure 3). In contrast, previous methods achieve lower unlearn quality and/or compromise the model's 's utility, The strong performance of our method stems from its distribution-awer aware optimization strategy: Unlike the distribution. In contrast to gradient-ascent-bast ground-truth sequences), PMC fine-tunes on newly sampled sequences in each iteration and relies on the model collapse phenomenon to force the model's responses to diverge towards more desired ones.
PMC fine-tunes responses that are already likely under the model's own
baselines (which repeatedly optimize against fixed
We observe that PMC-unlearning frequently converges toward response patterns that fall into three broad categories across models: (i) hallucinations, (ii) gibberish, or (iii) generic refusals that indicate the absence of knowledge. Examples of the latter include "The answer is not available". "There is no public information" and "Specific details are not available" (despite the reward function not explicitly incentivizing such responses). Note that the objective of this paper is demonstrate the effectiveness of PMC in removing information from model outputs, and we do not explicitly optimize for generation coherence or refusal patterns after unlearning. Future work can design reward functions that explicitly incentivize generic refusals, or one that penalizes ballucinations and gibberish.
5.2 PMC MORE ROBUST AGAINST SAMPLING AND PREFILLING ATTACKS
Notably, we demonstrate that PMC exhibits substantially greater robustness against sampling and prefilling attacks compared to prior approaches. To evaluate robustness under sampling, we draw 100 answers from the output distribution of the unlearned model, and compute the ROUGE-L score between each sampled and ground-truth response. We then compute the maximum (worst-case) ROUGE-L score per question and report the aver- age across all forget questions (see Appendix B.4 for the full experimental setup). The results in Figure 4 show that PMC significantly reduces leakage under sampling, in stark contrast to existing methods. While the simple supervised fine-tuning IDK baseline also reduces leakage under sampling, this effect largely superficial. To demonstrate this, we perform prefilling attacks in which the model is prompted with forget question and forced to continue from the prefix "The answer is:". This attack bypasses the fine-tuned response and reveals that the IDK baseline still encodes substantial information about the unlearned answers, leading to high leakage. Notably, while existing methods can exhibit considerable leakage (Scholten et al., 2025), PMC is the first approach to achieve more robust unlearning across both attack settings. The underlying reason for the improved robustness is that PMC does not optimize on fixed sequences, but rather fine-tunes on newly sampled responses in each epoch. This allows PMC to achieve unlearning by leveraging the model collapse phenomenon, which leads more thorough divergence of the model's output distribution.
Sampling
Prehiling-sampinng
Avg. w.c. [%]
50
PMC IDK DPO (ours)
NPO Sim- NPO
Figure 4: PMC is more robust against sampling and prefilling at- tacks. Lower average worst-case leakage better.
5.3 PMC OVERCOMES LIMITATIONS OF METHODS OPTIMIZING ON UNLEARNING TARGETS
Existing unlearning methods predominantly incorporate the unlearning target directly into their objectives. We argue that this approach may have subtle effects on model properties related to the unlearning targets, such as distorting token probabilities and leaking information about the private
For Gemma-3--1h-it we run each experiment only once due to the high fine-tuning costs and do not report results for DPO/NP as their scalability is limited due their dependence on a reference model.
7

---

Published as a conference paper at ICLR 2026
(b)
100
Accuracy [%]
Least Likely [%]
25
NPO
---
NPO
PMC
75
Count
10
50
12.5
25
0
75
100
NPO
PMC
Base
Probability difference
Quantile
Methods
Figure 5: Limitations of unlearning methods optimizing on unlearning targets: (a) Side effects on unrelated datasets. (b) Accuracy when selecting least likely answer across quantiles (black line is random guessing). (c) Distribution of minimum probabilities across all multiple- options.
data used during unlearning optimization. Yet, the utility of unlearning models is typically evaluated using benchmark datasets or by comparing them to retrained model (Maini et al., 2024). As result, existing evaluations may miss subtle changes the generation properties of unlearned models. In the following, we examine side effects of existing unlearning algorithms.
Generation capability on unrelated datasets. First, we study generations of tokens targeted in the unlearning optimization and investigate whether existing methods compromise the model's ability to generate such tokens. We argue that unlearning should prevent models from revealing unlearned information, but this effect must be limited to the unlearning context. It should not affect token generation in unrelated settings, as most tokens in forget sets are not semantically tied to the unlearning task but rather to sentence structure. For example, if we want to unlear that John Doe is a carpenter, existing methods would minimize the probability of "carpenter" when asked about John Doe's profession. However, these methods should not reduce this probability in unrelated contexts.
To investigate such potential side effects, we compare the probability of generating tokens present in TOFU compared to the first 100 text chunks of the wikitext-2-raw-vi train split (Merity et al., 2016). Figure 5 (a) shows the probability difference between unlearned models (NPO and our proposed PMC method) and the base model: Pien (2x - (2x (z),, where x; a token present in the forget set, is the context of this token in the wikitext dataset, and p(zz|2) (2) it the probability of given the context. As the base model, we use model fine-tuned exclusively on the retain set, with no exposure to the forget data. NPO substantially reduces the probability of generating forget set tokens also present in wikitext. considerable number tokens that originally get assigned high probability from the base model (e.g., close to 1) get assigned a probability of 0 from the unlearned model (indicated by -1 values in the figure). In contrast, our method preserves generation probabilities, exhibiting token probabilities that are neither systematically increased nor decreased. For PMC, the differences follow a zero-mean Gaussian distribution with small variance, whereas they are skewed to the left for NPO (-0.12 mean). This shows that methods dependent unlearning targets can considerably distort token probabilities even out-of-context of the unlearning task.
Probability distribution in multiple-choice settings. Second, hypothesize that existing unleam- ing methods may exhibit information "leakage" by unnaturally reducing the probability of correct answers, potentially allowing adversaries identify forgotten information by simply selecting the least likely option. To further investigate such potential negative side effects, we created multiple- choice dataset from the TOFU 10rget10 set by converting a subset of 84 questions into multiple-choice (MPC) format (Appendix B.5). We use the inverse perplexity of every answer as its score and turn scores into probabilities by normalizing them. Moreover, for the correct answers the MPCs we used rephrased versions of the correct TOFU answers rather than exact matches to demonstrate that leakage can occur even for semantically similar but non-identical formulations.
Our experiments provide first empirical evidence for our leakage hypothesis. Figure 5 (b) shows accuracy when selecting the least likely answer across quantiles ordered by minimum probability among choices. NPO exhibits high accuracy for questions where the minimum probability is very low, indicating that the correct answer frequently becomes the least likely option. Conversely, our method shows such pattern. Figure 5 (c) shows the distribution of minimum probabilities across all multiple-choice options. Here, NPO's distribution clusters near zero, further confirming that target- based unlearning unnaturally suppresses correct answer probabilities even in rephrased contexts.
8

---

Published as a conference paper at ICLR 2026
Utility (%) Unlearn quality (%)
(c)
100
100
80
80
80
60
60
20
PMC (ours)
SimNPO
20
20
70
70
70
60
60
603
50
40
40
40
30
30
5
10
15
20
1
5
10
15
20
0.50
0.75
1.00
1.25
1.50
Number of epochs
Number of samples
X
Figure Ablation studies on (a) number of epochs, (b) number of samples, and (c) trade-off parameter A. Dashed line is the fine-tuned model before unlearning. Shadows/bars indicate standard deviation.
5.4 EXTENDED EXPERIMENTAL EVALUATION OF COLLAPSE-BASED UNLEARNING
Ablation studies. We perform ablation studies PMC's hyperparameters under Phi-1.5 (additional results in Appendix A, see also details in Appendix B). First, the number of training epochs strongly influences unlearning performance: while baseline methods converge after 10 epochs, PMC continues to improve unlearn quality without significantly affecting utility even after 20 epochs (Figure 6a). Second, increasing the number of samples enhances unlearning, with utility remaining stable for the first six epochs; larger sample sizes show higher variance in utility (Figure 6b). Finally, we ablate the unlearn-utile trade-off parameter x, observing that larger values improve utility but can degrade unlearn quality, highlighting the importance of selecting À to balance these objectives (Figure 6c).
Additional results. We provide additional experiments Appendix A: We provide further ablations in Appendix A.1, Appendix A.3 and Appendix A.4. We perform careful runtime comparisons in Appendix A.2, and demonstrate empirical reward convergence Appendix A.6. We also provide experiments using self-B Leff-LEUUbbad based rewards in Appendix A.7. Finally, evaluate PMC on MUSE- news data in Appendix A. 10, extending the analysis to an additional dataset beyond Q&A tasks.
Extended utility analysis. Although PMC is optimized only on the retain data to preserve utility, we find that its impact on overall model utility beyond the TOFU utility dataset is minimal in practice. Results the ARC-Challe ARC-Challenge, ARC-Easy, and MMLU benchmarks (Appendix A.8) show that PMC-unlearnigg has minimal to negligible effect on general model utility.
DISCUSSION
Definition of machine unlearning for LLMs. A common challenge in the LLM safety literature is that existing works often attempt solve problems such unlearning alignment as monolithic objectives, an approach that frequently fails to capture nuances (Scholten et al., 2025). This motivates decomposing complex objectives into simpler, more measurable ones (Schwinn al., 2025). In this context, machine unlearning for LLMs refers to a broader problem encompassing multiple objectives. frequent goal in unlearning is to converge in data distribution toward model never trained on the data to be unlearned (Maini et al., 2024). In this paper, we focus on the more direct and measurable objective of removing information from model outputs, which can contain undesired information even if models have never been trained such data (e.g., through inference from other data). While our objective is closer to that of LLM alignment, it is critical part LLM unlearning. In particular, argue that as long models are not robust and leak under sampling (Scholten et al., 2025), convergence in data distribution not achieved even current benchmark evaluations suggest progress toward this goal. Moreover, argue it is not sufficient to evaluate such convergence based on a single reference model fine-tuned exclusively on the retain data (i.e. not on forget data), since this process is con-deterministic and can lead different models with substantially different properties.

---

Published as a conference paper at ICLR 2026
Interestingly, observe considerable utility loss at the point of distribution collapse from which the models quickly recover afterwards (see Appendix A.2), indicating convergence toward a model which may not be linearly mode connected to the model before unlearning. However, whether our PMC-unlearned models are actually closer to a model never trained on the data we removed from model outputs is fundamentally different and harder question for which the current literature lacks comprehensive evaluation protocols, preventing robust conclusions so far.
Evaluating LLM unlearning. Evaluating unlearning in LLMs remains major challenge beyond the definition of unlearning itself. Our analysis is inherently empirical and provides no formal guarantees that the model has unlearned the forget data. Currently, one can bound information leakage with high probability for fixed inputs (Scholten et al., 2025), but this also does not imply unlearning in a broader sense. While more comprehensive empirical evaluations would ideally rely on human (or LLM--ased) judgments, our goal is simply to demonstrate that PMC can remove specific outputs without degrading utility-ann an outcome we believe is sufficiently supported by our experiments. We view this as a first step, and e.g. robustness to relearning or adversarial attacks remains future work.
Limitations of collapse-based unlearning. In theory, collapse-basse unlearning relies the model assigning non-zero probability mass to higher-reward responses so that the output distribution can shift toward more desirable generations. If the distribution were already fully collapsed onto single ground-truth sequence, the reward could not increase further and we would not observe unlearning. While this constitutes a limitation in principle, we do not observe this in practice for LLMs. Empirically, current models maintain sufficiently broad output distributions with non-zero mass on higher-reward alternatives, enabling consistent convergence toward the optimal reward.
Limitations of fine-tuning on samples. While a key strength of PMC is its reliance on samples already likely under the model's own distribution, this also increases computational costs in particular for larger models. We provide a careful and detailed cost analysis in Appendix A, comparing PMC against baselines under runtime vs. unlearning and utility trade-offs. While we acknowledge that PMC has slightly higher computational cost due to the initial collapse process in the first epochs, we believe that the overall runtime remains competitive and in particular practical for real- world applications. Future work could explore faster sampling techniques, pruned proxy models, or speculative sampling toward more efficient collapse-based machine unlearning.
Privacy considerations. A central advantage of PMC is that it neither optimizes against nor requires access ground-truth forget sequences during unlearning. This is particularly important in scenarios where the original data is unavailable, restricted, or cannot be shared due to privacy constraints. Instead, PMC operates solely on samples drawn from the model's own distribution, eliminating the need for ground- ground-truth supervision during unlearning. While self- seff-generated samples may initially encode sensitive information, PMC theoretically and empirically drives the model to rapidly diverge from such content; after this initial phase, optimization no longer exposes the model to sensitive data. In contrast, prior GA-based methods repeatedly optimize against a fixed ground-trutt sequence throughout the entire unlearning process, risking amplification of private information. More broadly, our setting reflects practical deployments in which one may not know which specific training samples gave rise to output requiring unlearning, or where unique ground-truth forget sequence exists.
Design of reward function. The design of the reward function riz) is crucial for achieving the desired outcome after unlearning. We use ROUGE-L scores between the current sampled and the model's ' original greedy output, which amounts to an incentive to diverge from the model before unlearning. This choice is motivated by (I) the goal of removing model outputs, well as the (2) empirical effectivvness and (3) simplicity of this reward. In practice, the design of ((x) may need to be tailored more carefully to the needs of specific applications, e.g. by using a fast proxy model that broadly captures coherence. We believe designing different rewards is a promising avenue for future work.
7 CONCLUSION
We propose novel and theoretically grounded paradigm for LLM unlearning that leverages the model collapse phenomenon to remove information from model outputs. Our approach iteratively fine-tunes LLMs on their own responses to sensitive questions until the model's 's output distribution collapses on sensitive responses, effectively unlearning them. We empirically demonstrate that our approach converges to model that no longer generates sensitive information while preserving the model's s utility. Our work represents an important contribution toward effective unlearning and provides a foundation for future research in collapse-based unlearning for generative models beyond L.LMs.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors want thank Marius Mosbach, Alicia Curth, Marcel Kollovich and Lukas Gosch for their valuable feedback on the manuscript. This work has been funded by the DAAD program Konrad Zuse Schools Excellence Artificial Intelligence (sponsored by the Federal Ministry Education and Research). Leo Schwinn and Yan Scholten gratefully acknowledge funding from Coefficient Giving for this work. This research was enabled part by compute resources provided by Mila. The authors of this work take full responsibility for its content.
ETHICS STATEMENT
Our work contributes to the field of machine unlearning, which is crucial for ensuring privacy and compliance with data protection regulations. By proposing method that effectively removes sensitive information from LLM outputs, we aim to enhance the trustworthiness of AI systems. However, we acknowledge that unlearning could also be misused, for example to delete facts. We advocate for the responsible use of our method, emphasizing transparency and accountability in Al development.
REPRODUCIBILITY STATEMENT
We ensure reproducibility by providing detailed description of our experimental setup (including all hyperparameters) and additional reproducibility instructions Appendix B. All datasets, models, and code used in our experiments are publicly available. Our implementation is publicly available at https: / /www. cs. cit. tum. ww/ddd/dlllmmm /aamll/arttaaiaa wateiiaalmmmeeeeeee--e-----ee 1 copassee//
LLM USAGE STATEMENT
LLMs were only used to polish writing at sentence-level (spelling, grammar, wording).
REFERENCES
Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G. Baraniuk. Self-consuming generative models go MAD. ICLR. OpenReview.net. 2024.
Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. On the stability of iterative retraining of generative models their own data. In ICLR. OpenReview.net, 2024.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette- Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In SP, pp. 141-159.
IEEE, 2021.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4)3324 345, 1952.
Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:23100 13..20010.0000 2023.
Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. CoRR, 0bss/1604.061 2016.
Eli Chien, Chao Pan, and Olgica Milenkovic. Certified graph unlearning. CoRR, abs/2206.091400 2022.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv: 180:: 054557, 2018.
Google DeepMind. Gemma 3 technical report. CoRR, abs/2503.19786 2025.
Elvis Dohmatob, Yunzhen Feng, Arjun Subramonian, and Julia Kempe. Strong model collapse. arXiv preprint arXiv:2410.0484/ 2024.
11

---

Published as a conference paper at ICLR 2026
Vineeth Dorna, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zachary C Lipton, J Zico Kolter, and Pratyush Maini. Openunleariing Accelerating Ilm unlearning via unified benchmarking of methods and metrics. arXiv preprint wrXi::2506.16616 2025.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.022388 2023.
European Union Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation) (text with eea relevance). 2016. Official Journal of the European Union, L 119, 4 May 2016, pp- 1-88.
Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity prevails: Rethinking negative preference optimization for LLM unlearning. CoRR, abs/2410.07163, 2024.
Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, and Julia Kempe. Beyond model collapse: Scaling up with synthesized data requires verification, 2024.
Zhili Feng, Yixuan Even Xu, Alexander Robey, Robert Kirk, Xander Davies, Yarin Gal, Avi Schwarzschild, and J Zico Kolter. Existing large language model unlearning evaluations inconclusive. arXiv preprint arXiv:2506.0066 2025.
Damien Ferbach, Quentin Bertrand, Avishek Joey Bose, and Gauthier Gidel. Self-consuming generative models with curated data provably optimize human preferences. CoRR, Abs/2407.09499/ 2024.
S. Gersgorin. Über die abgrenzung der eigenwerte einer matrix. Bulletin de l'Académie des Sciences de I'URSS. Classe des sciences mathématiques et na, Issue 6:749-754 1931.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2172 2024.
Chuan Guo, Tom Goldstein, Awni Y. Hannun, and Laurens van der Maaten. Certified data removal from machine learning models. In ICML, volume 119 of Proceedings of Machine Learning Research, pp. 3832-3842. PMLR, 2020.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring Massive Multitask Language Understanding. In International Conference on Learning Representations (ICLR), 2021.
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504 2022.
Erik Jones, Meg Tong, Jesse Mu, Mohammed Mahfoud, Jan Leike, Roger B. Grosse, Jared Kaplan, William Fithian, Ethan Perez, and Mrinank Sharma. Forecasting rare language model behaviors. CoRR, 1os92502..16997, 2025.
Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. Propile: Probing privacy leakage in large language models. Advances Neural Information Processing Systems, 36, 2024.
Aravind Krishnan, Siva Reddy, and Marius Mosbach. Not all data are unlearned equally. CoRR, abs/2504.05058, 2025.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski. Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.032//6 2024.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 1.5 technical report. arXiv preprint arXiv:2309.05463 2023.
12

---

Published as a conference paper at ICLR 2026
Chin- Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pp. 243-254. PMLR, 2022.
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp. 1 1-14, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR (Poster). OpenReview.net, 2019.
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadeield-Menell. Eight methods to evaluate robust unlearning Ilms. arXiv preprint arXiv:2402.16883 2024.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. TOFU; A task of fictitious unlearning for Ilms. CoRR, abs/2401.06121, 2024.
Anmol Mekala, Vineeth Domn, Shreya Dubey, Abhishek Lalwani, David Koleczek, Mukund Rungta, Sadid Hasan, and Elita Lobo. Alternate preference optimization for unlearning factual knowledge in large language models. CoRR, abs/2409.134444 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
Seth Neel, Aaron Roth, and Saeed Sharifi-Malvaipee Descent-to-delete: Gradient- methods for machine unlearning. In ALT, volume 132 of Proceedings of Machine Learning Research, pp. 931-962. PMLR, 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS, 2023.
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: memory optimizations toward training trillion parameter models. In SC, pp. 20. IEEE/ACM, 2020.
Yan Scholten,
Stephan Günnemann, and Leo Schwinn. A probabilistic perspective on unlearning and alignment for large language models. In The Thirteenth International Conference on Learning Representations, 2025.
Leo Schwinn, David Dobre, Sophie Xhonneux, Gauthier Gidel, and Stephan Gunnemann. Soft prompt threats: Attacking safety alignment and unlearning in open-source 1lms through the embedding space. arXiv preprint arXiv:2402.09006. 2024.
Leo Schwinn, Yan Scholten, Tom Wollschläger. Sophie Xhonneux, Stephen Casper, Stephan Gün nemann, and Gauthier Gidel. Adversarial alignment for Iims requires simpler, reproducible, and more measurable objectives, 2025. URL https::////.......... / /arxiv. org/ abs 2202 1 1910.
Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want forget: Algorithms for machine unlearning. In NeurlPS, pp. 18075-18086 2021.
Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn, and Stephan Günnemann. Extracting unlearned information from lims with activation steering. In Neurips Safe Generative Al Workshop 2024.
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, et al. Targeted latent adversarial training improves robustness persistent harmful behaviors in llms. arXiv preprint arXiv:2407.15549, 2024.
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. MUSE: machine unlearning six-way evaluation for language models. In ICLR. OpenReview.net, 2025.
13

---

Published as a conference paper at ICLR 2026
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross J. Anderson. The curse of recursion: Training on generated data makes models forget. CoRR, abs/2305.17493, 2023.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, (8022))77755555 2024.
Enayat Ullah, Tung Mai, Anup Rao, Ryan A. Rossi, and Raman Arora. Machine unlearning via algorithmic stability. COLT, volume 134 of Proceedings of Machine Learning Research, pp. 4126-4142. PMLR, 2021.
Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. IJCAI, volume 6, pp. 19, 2022.
Lefeng Zhang, Tianqing Zhu, Haibin Zhang, Ping Xiong, and Wanlei Zhou. Fedrecovery: Differen- tially private machine unlearning for federated learning frameworks. IEEE Trans. Inf. Forensics Secur, 18:4732-4746, 2023.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.00 05868, 2024.
14

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
In this appendix we provide additional results and details on experimental setups, and prove theoretical results as outlined in the following: In Appendix A we provide additional experiments and ablation studies. We carefully describe experimental setups in Appendix B. Finally, we provide theoretical results in Appendix and Appendix D.
A Additional experimental evaluations and ablation studies
16
Full experimental setup and implementation details
27
Warm-up: Iterative unlearning with categorical distributions
32
Machine unlearning via relearning self-generated data
36
PMC (ours)
GA
DPO
NPO
SimNPO
IDK
(a) Phi-1.5
(b) Llama-3 2-3B Instruct
(c) Gemma-3-122 12b-it
Unleam quality (%)
100
100
100
80
80
20
40
60
60
70 Utility (%)
Utility (%)
Utility (%)
Figure 7: Partial model collapse (PMC) significantly dominates baselines and expands the Pareto-from w.r.t. utility and unlearn quality for (a) Phi- (b) Llama-3.2-2 --3B--nnttuuct and (c) Gemma-3 1-b-i..t.
Same data as in Figure but with symbols for improved accessibility.
15

---

Published as a conference paper at ICLR 2026
Utility (%) Unlearn quality (%)
(c)
100
80
80
80
60
60
60
40
40
20
20
20
80
80
80
70
70
70
60
60
50
50
40
Finetuned
40
PMC
Fineluncd
50
PMC axgmaaagaxx
30
1.0
1.1
1.2
1.3
1.4
1.5
0.90
0.95
1.00
0.05
0.50
1.00
Sampling temperature
Top-p
BT temperature
Figure 8: Ablation studies on: (a) temperature, (b) top-p sampling, and (c) Bradley-Terre approxima- tion (shadow/bars show standard deviation across five runs).
A ADDITIONAL EXPERIMENTAL EVALUATIONS AND ABLATION STUDIES
In this section provide additional experiments and ablation studies follows:
In Appendix A. provide additional ablation studies sampling temperature, top-p sampling and the Bradley-Terry approximation.
Appendix A.2 we provide runtime analysis of PMC-unlearning compared baselines.
In Appendix A.3 provide detailed ablation study the utility-uleeaming trade-off parameter X and analyze unlearn quality and utility by epoch and runtime.
In Appendix A.4 we provide detailed ablation study on the number of samples and analyze unlearn quality and utility by epoch and runtime.
In Appendix A.5 we empirically analyze the output distribution after unlearning of several unlearning methods better understand robustness to sampling and prefilling attacks.
In Appendix A.6 analyze empirical reward convergence.
In Appendix A.7 we experiment with an alternative reward function.
In Appendix A.8 we provide an additional utility analysis of PMC-unleamedd models.
In Appendix A.9 we provide an ablation study on gradient checkpointing.
In Appendix A.10 extend the empirical analysis to MUSE-news data.
A.1 CONTINUED ABLATION STUDY
We conduct the following additional ablation studies for Phi-1.5 1.5 to further investigate the properties of our method. See Appendix B.3 for details on the experimental setup of all ablation studies.
Sampling temperature (Figure 8 (a)). We empirically observe that larger temperatures allow stronger unlearn quality, likely due to the higher probability to sample responses with lower similarity the ground truth. For temperatures above 1.5, the process leads to decrease in utility.
Top-p sampling (Figure 8 (b)). We observe that top-p sampling can similarly improve unlearn quality at the cost of model utility due to similar effects on the diversity of the sampled responses.
Bradley-Terry -Terly approximation. The general PMC formulation in Section requires randomly select- ing one of n responses according to the Bradley-Tery model. However, in all experiments we imple- ment an approximation by choosing the response with the highest score, i.c., & arg max; r(x;). We resolve ambiguity by choosing whichever sample has been drawn first provide additional, implicit incentive to choose more likely samples. Our approximation is computationally more efficient since avoids the additional sampling step, and empirically verify in Figure 8 (c) that it effectively corresponds to the limit - for temperature т. detail, the B BT-temperature 7 does not affect model utility, however, it effectively improves unlearn quality, motivating the argmax approximation.
16

---

Published as a conference paper at ICLR 2026
A.2 COMPUTATIONAL COST ANALYSIS
Runtime vs. unlcarning quality and utility. We provide a computational cost analysis by visualizing runtime versus unlearning quality and utility plots for all three models in Figure 9, Figure 10 and Figure 11, respectively. Here we measure computational cost in terms of wall-clock runtime per epoch on an NVIDIA H200 GPU (all experiments in this section are performed on the same hardware). The plots show unicarning quality (left) and utility (right) versus runtime for different unlearning methods. Each point in the plots represents the end of an epoch during unlearning. and we connect the points of each method for better visualization. In total, we perform unlearning for 20 epochs for Phi- 1.5 and and 10 epochs for Gemma-3-12b-it, which is more computationally expensive. Phi-1.5 completes within 40 minutes, whereas L1 ama 3. 2-B--B---- Instruct converges faster with a runtime of 20 minutes, and for Gemma-3- 3- 1-----1biii- it, the runtime is around 200 minutes. We consider the runtime of PMC to be practical for real-world applications. especially when compared to retraining LLMs from scratch. While for Phi-1.5 the runtime of PMC is higher than that of and Gemma-3-12b-an PMC is comparable to baselines
Llama-3.2-3B-Innsee
the baselines, for Llama-3.2-3B-Instuuu (considering their runtime). We believe that the computational cost of PMC can be further improved by future research as discussed in Section 6, e.g. using more efficient sampling strategies.
Regarding utility. Note that all baselines affect utility during the unlearning process. In contrast to baselines, utility does not degrade immediately for PMC. Instead, we observe the strongest effects on utility later at the epoch where the model's 's distribution collapses for forget questions. We provide an ablation study on different lambda in Appendix A.3. We also find that gradient checkpointing can have negative effects on this utility drop and provide more detailed ablation study in Appendix A.9.
Faster approximation (PMC-fast). We also implement a faster approximation making use of the result in Appendix A.6 that the reward can converge quickly. The main idea is to only select samples if their reward is larger than that of the sample with the best reward observed so far, and to stop sampling once the reward has converged to the maximum reward. This significantly reduces the number of samples during training. leading to faster runtimes in particular for Phi-1.5. We include this fast approximation "PMC-fast". We further observe that Llama-3.2-3B I-nsruuntt quickly collapses to generating EOS tokens (exclusively on the forget and paraphrased forget sets). significantly reducing runtime during sampling, and consequently the fast approximation does not lead significant speed- ups for this model. For Gemma-3- i2bitt the fast approximation does not lead to significant speed-ups either since the reward does not converge to the maximum reward within the first 10 epochs in this setup, which is required for the fast approximation to be effective. Note that this is entirely model-, hyperparameter- and reward-dependent (e.g.. the current reward does not penalize empty responses).
The results also indicate that the fast approximation can be harmful for utility, however, this discussion more complicated as the results also depend on the hyperparameter selection (we use the same hyperparameters for both PMC and PMC-fast, and for a better comparison we would have to perform hyperparameter searches for both). Overall, please note that the broader unlearning problem is more complex and approximations affect not just runtime and utility/unlearn quality, but also other factors such as robustness. Deriving faster versions or approximations of PMC would require careful consideration of various design choices and hyperparameters, and may also introduce additional sources of error that need to be studied carefully. In this paper, we are more interested in studying the vanilla collapse process without additional sources of error, which is why we focus the standard version of PMC in all other experiments, and consider efficiency as out of scope for this paper. In particular, note that one could even consider slowing down the collapse process intentionally, e.g. by choosing samples in a way that the reward increases more slowly, since a more thorough collapse process could have positive effects on robustness. We consider further improvements regarding the efficiency of callassebbased unlearning under such considerations as interesting future work.
Limitations. We do not aim provide complete estimate of computational costs and acknowledge that the measurements provided here can be influenced by various factors. In particular, note that all runtime experiments show individual runs running on a cluster simultaneously with other experiments, which may lead to variability in the reported runtimes (despite running all experiments on the same hardware). We consider the results as a first step to analyze the computational costs associated with fine-tuning LLMs on their own generations. A more detailed analysis of the computational costs of PMC-unlearning. including a breakdown of the costs associated with different components of the algorithm (e.g., sampling, optimization) would be interesting for future work.
17

---

Published as a conference paper at ICLR 2026
100
70
80
PMC-fas
60
UQ
Utility
PMC
SimNPO
50
SimNPO
DPO
40
10
20
30
40
0
10
20
30
40
Runtime (min)
Runtime (min)
Figure 9: Runtime versus unlearning quality (left) and utility (right) for different unlearning methods
(Phi-1.5) 5). UQ: Unlearn quality. Points represent epochs (20 epochs in total).
100
75
UQ (%)
PMC-faa
PMC-Gat
Utility
50
SimNPO
SimNPO
DPO
25
DPO
NEO
NPO
20
Runtime (min)
Runtime (min)
Figure 10: Runtime versus unlearning quality (left) and utility (right) for different unlearning methods [1lama-3 3.nna-3.2-000 2--------- (BB-Instruct) UQ: Unlearn quality. Points represent epochs (20 epochs in total).
85
80
UQ (%)
Utility
PMC-ftst PMC
60
75
PMC
SimN
70
100
200
100
200
Runtime (min)
Runtime (min)
Figure Runtime versus unlearning quality (left) and utility (right) different unlearning methods (ieemna33-125 it.. UQ: Unlearn quality. Points represent epochs (10 epochs in total).
18

---

Published as a conference paper at ICLR 2026
A.3 DETAILED ABLATIONS UTILITY-NLEARNING ORADE-OFF PARAMETER
We provide a detailed ablation study on the utility-unlearning trade-off parameter À by analyzing unlearn quality and utility by epoch in Figure 12, Figure and Figure 14, and by runtime in Figure 15, Figure 16 and Figure 17 for Phi-1.5, generally observe that larger À leads to stronger utility at the cost of unlearn quality, indicated by the formulation of PMC loss function.
and Gemma-3- 1-b-i,,, respectively. We
1.25
1.5
Utility (%)
70
70
68
U
50
64
10
15
20
10
15
20
Epochs
Epochs
Figure 12: Unlearning quality (left) and utility (right) after each epoch of single runs with different
(Phi- (Phi-1.5). UQ: Unlearn quality. Points represent epochs (20 epochs in total).
Utility (%)
75
UQ (%)
80
50
60
10
15
20
10
15
20
Epochs
Epochs
Figure 13: Unlearning quality (left) and utility (right) after each epoch of single runs with different
(L) ama-mmm3... 3. 2 3B-Instructt UQ: Unlearn quality. Points represent epochs (20 epochs in total).
100
90
Utility (in %6)
UQ (in %
80
80
60
4
10
8
10
Epochs
Epochs
Figure 14: Unlearning quality (left) and utility (right) after each epoch of single runs with different
(Gemma-33 ---e------bb--- UQ: Unlearn quality. Points represent epochs (10 epochs in total).
19

---

Published as a conference paper at ICLR 2026
1.25
1.5
1.75
Utility (%)
70
70
68
UQ
66
50
64
100
50
100
Runtime (min)
Runtime (min)
Figure 15: Single-run runtime versus unlearning quality (left) and utility (right) for different
(Phi-1.5). UQ: Unlearn quality. Points represent epochs (20 epochs in total).
100
75
UQ (%)
80
Utility
50
25
60
Runtime (min)
Runtime (min)
Figure 16: Single-run runtime versus unlearning quality (left) and utility (right) for different
(1.1ama-3 I-3---tttt Points represent epochs (20 epochs in total).
Utility (%)
UQ (%)
80
80
60
50
100
150
Runtime (min)
Runtime (min)
Figure 17: Single-run runtime versus unlearning quality (left) and utility (right) for different
(Gemna-2 - 1-12-ibitt UQ: Unlearn quality. Points represent epochs (10 epochs in total).
20

---

Published as a conference paper at ICLR 2026
A.4 DETAILED ABLATIONS ON NUMBER OF SAMPLES
We provide a detailed ablation study on the number of samples n by analyzing unlearn quality and utility by epoch in Figure 18, Figure 19 and Figure 20, and by runtime in Figure 21, Figure 22 and Figure 23 for Phi-1.5, Jlama-3.2-3B-Ins-
and Gemma-3-12b-in respectively. We generally observe that more samples leads to stronger unlearn quality, however, the effects are more complex.
Phi-1.5 requires at least 10 samples to observe collapse to responses with maximum reward. In- terestingly, even for 10 samples the collapse only happens in the 8th-epoch. For smaller number of samples, the process converges toward responses with suboptimal rewards (for which we would have use a judge model to decide if the model unlearned the response). Clearly, more samples increase the probability of sampling responses with the maximum reward (see also discussion in models typically collapse generating EOS tokens for forget questions, which speeds up the sampling and unlearning process. The overall unlearning process takes longer if the collapse occurs at later epochs as e.g. in the case of a single sample (Figure 22).
Appendix A.6).
Across all models observe utility drops at the epoch where the distribution collapses for forget questions, from which the models quickly recover afterwards. We believe this indicates convergence towards model that is not linearly mode connected to the original model, and we consider analyzing this phenomenon more detail as an interesting direction for future work.
Number of samples
10
100
UQ (%)
80
Utility (%)
60
60
40
10
15
20
10
20
Epochs
Epochs
Figure 18: Unlearning quality (left) and utility (right) after each epoch of single runs with different number of samples n (Phi- 1. 5). UQ: Unlearn quality. Points represent epochs (20 epochs in total).
100
75
Utility (%
(%)
80
50
UQ
60
10
15
20
10
15
20
Epochs
Epochs
Figure 19: Unlearning quality (left) and utility (right) after each epoch of single runs with different number of samples (Il 2--3B-Insttrctt Points represent epochs (20 epochs in total).
100
90
Utility (%)
80
80
UQ
60
70
60
2
4
6
10
10
Epochs
Epochs
Figure 20: Unlearn quality (left) and utility (right) after each epoch of single runs with different number of samples (Gemma -- 3-12b-it), Points represent epochs (10 epochs in total).
21

---

Published as a conference paper at ICLR 2026
Number of samples
100
UQ (%)
Utility (%)
60
80
60
40
40
Runtime (min)
Runtime (min)
Figure 21: Single-run runtime versus unlearning quality (left) and utility (right) for different number of samples n (Phi-1.55 UQ: Unlearn quality. Points represent epochs (20 epochs in total).
Number of samples
100
Utility (%)
75
UQ (%)
80
25
60
20
Runtime (min)
Runtime (min)
Figure 22: Single-run runtime versus unlearning quality (left) and utility (right) for different number of samples n -3. 2 - Points represent epochs (20 epochs total).
Number of samples
10
100
90
Utility (%)
UQ (%)
80
80
60
70
60
100
150
100
150
200
Runtime (min)
Runtime (min)
Figure 23: Single-run runtime versus unlearning quality (left) and utility (right) for different number of samples n (Gemma -3 3-12b-23 (i).. Points represent epochs (10 epochs in total).
22

---

Published as a conference paper at ICLR 2026
A.5
ADDITIONAL SAMPLING AND PREFILLING ATTACK VISUALIZATION
PMC
IDK
DPO
NPO
SimNPO
0.05
Density
10
2
0.00
-0.05
10
Density
2
1
Leakage
Leakage
Leakage
Leakage
Leakage
Figure 24: KDE-plot showing forget question distribution over worst-case leakage. Top-row shows the sampling attack, bottom-row prefillingssamllinn attack. PMC is more robust against sampling and prefilling attacks. Lower worst-case leakage better.
Figure 24 shows the distribution of forget questions according their worst-case leakage (i.e. not just the average worst-case as in Figure 4). Interestingly, the IDK-baseline shows significant leakage for the majority of inputs only after the prefilling attack. DPO exhibits considerable leakage, with few questions leaking the entire ground truth under sampling. PMC-unleamed models show significantly improved robustness against both sampling and prefilling attacks, with responses to forget questions having no similarity to the ground-trun -tunh answers for basically all forget questions.
A.6 EMPIRICAL REWARD CONVERGENCE
We empirically analyze reward convergence from Theorem 1 in Figure 25. In particular, Figure 25 shows the batch-wise sample mean and variance the reward during PMC-unlearning 2-3B-Instruct the TOFU forget set over the first 500 training steps. We observe that the reward effectively converges to the maximum reward within the first 50 steps, and the variance vanishes.
Llama-
empirically observe this convergence to the maximum reward consis- tently across many different hyperparameter settings, as one can also see from the collapse evaluation in Figure 19. For Phi-1.5 and Gemma-3-12b-it, we also observe convergence to the maximum reward for certain hyperparameter settings, however, the convergence behavior can be more unstable across different hyperparaameter settings, which may be due to maximum reward responses being more unlikely to be observed if the number of samples too small for these models (which can lead to convergence to suboptimal rewards instead). Critically, for all models found hy hyperparameter configurations for which the reward converges to the maximum reward empirically.
For Llama-3.2-23 1aae--3.--B--nttruu
Empirical reward over time
Empirical reward over time
Mcan reward (batch average)
Standard dev.
(batch-wir (batch-wise) (ich-)
0.2
0.1
0.0
100
200
300
400
500
0
100
200
300
400
500
Training steps
Training steps
Figure 25: Empirically, the expected reward converges the maximum reward in the first 50 training steps (25 steps are one epoch) and the variance vanishes. Vertical line shows step 50 (end of epoch 2).
Model collapse with restart. In separate experiments we also implement version of PMC where we restart the collapse process after convergence suboptimal rewards, i.e., we treat the unlearned model as new model and repeat the process to diverge away from the newer model's outputs. This can cause another collapse later on (further improving unlearning quality), provided that the distribution is not fully collapsed yet. We leave such interesting cascaded collapses to future work.
23

---

Published as a conference paper at ICLR 2026
A.7
REWARD ABLATION STUDY
100
UO (%6
Utility
PMC-SelfBiar PMC-Rouget.
25
PMC PMC SelfDleu
PMC-Rougel
5
15
Epochs
Epochs
Figure 26: Unlearning quality (left) and utility (right) versus runtime for different unlearning methods for 1amma-3.2-3--Instruee UQ: Unlearn quality. Points represent epochs (20 epochs in total).
In the reward ablation study in Figure 26 we compare the standard ROUGE-L- reward to different reward based on self-BLEU scores, which measures the similarity between one generated samples to all other generated samples (and rewards samples that are most dissimilar to all others). In this setting we observe that using self-BLEU scores can lead to higher utility. Please note that would have evaluate unlearning quality with judge model to correctly evaluate if self BLUU based rewards are effective unlearning, which cannot be concluded based on ROUGE-L scores in this case (although contain any words of ground-truth sequences, responses with higher scores may still not encode any ground truth information). By manually investigating the resulting greedy generation after unlearning with self-B self-BLEU rewards, we indeed observe successful semantic unlearning. This suggests that self-BLEU rewards can be effective for unlearning as well, although more thorough evaluations based on judge models would be interesting to further confirm this hypothesis in future research.
ROUGE-L implies successful unlearning in the sense that the output does
A.8 EXTENDED UTILITY EXPERIMENTS
We conducted additional experiments to test whether PMC introduces unexpected utility degradations common benchmarks from the literature. Specifically, compared the vanilla (base) mod- els with their PMC-unlearned counterparts on Arc-Challenge, Arc-Easy (Clark et al., 2018), and MMLU (Hendrycks et al., 2021). We report the mean and standard deviation over five random seeds for Phi and Llama. The results in Table show that PMC has only minor impact on model utility. Note that models can already degrade in utility during the initial fine-tuning on TOFU-full, and that the utility remains high, independent of whether we apply the chat-template used during TOFU fine-tuning (results in Table are without applying the TOFU chat-template during evaluation).
Table Model utility comparison between base models and models fine-tuned with PMC.
Are- Are-Challenge
Arc-Easy
MMLU
Phi-1.5 (base)
0.4462
0.7622
0.4174
Phi-1.5 (PMC)
0.4283 = 0.0057
0.6891
0.0053
0.4063 + 0.0031
3. Inst (base)
0).4368
0.7382
0.6041
1ita-3.2-33--nstt (PMC)
0.4341 + 0.0061
0.7230
0.0058
0.5924 + 0.0040
Gemma-3-12b-14 (base)
0.6083
0.8354
0.7151
Gemma-3-12b-11 (PM))
0.6032
0.8421
0.6934
24

---

Published as a conference paper at ICLR 2026
A.9
GRADIENT CHECKPOINTING ABLATION
gradient checkpointing
False
True
100
70
Utility (5)
80
60
50
60
40
5
10
15
10
15
20
Epochs
Epochs
Figure 27: Unlearning quality (left) and utility (right) versus runtime for Phi with different A. UQ: Unlearn quality. Points represent epochs (10 epochs total).
gradient checkpointing
False
True
100
75
Utility (%)
50
UQ
25
60
5
10
15
20
5
10
15
20
Epochs
Epochs
Figure 28: Unlearning quality (left) and utility (right) versus runtime for Llama with different X. UQ:
Unlearn quality. Points represent epochs (10 epochs in total).
We gradient checkpointing in all our -Jama-3.2-31B Instrrctt and Gemma-3-1 1bbitt experiments as this corresponds to the setup of the initial TOFU repository (Maini et al., 2024). While gradient checkpointing should principle just use compute to save memory (to fit larger models), we found can introduce an additional source of error. In particular, we observe entirely different unlearning trajectories for fixed hyperparameters and seeds (see ablations Figure 27 and Figure 28).
We advocate for more careful experimental setups in future unlearning research, as such unnecessary sources of error can prevent robust conclusions. We recommend either avoid these techniques or to carefully ablate them to ensure their effects are well understood and results are reliable. Critically, since we perform experiments without (Phi) and with gradient checkpointing (Llama and Gemma), we believe the main conclusions of our work are robust since observe the same general trends across all models and many hyperparameter settings (even the exact unlearning trajectories may vary).
25

---

Published as a conference paper at ICLR 2026
A.10
ADDITIONAL EVALUATION ON MUSE-NEWS DATA
100
100-KnowMen Dj
PMC (ours)
SImNPO
80
GA
Retrain
Fine-tuned
60
40
25
50
KnowMem D,
Figure 29: Unlearning-utilig trade-off's on MUSE-news subset.
Experimental setup. For this experiment consider subset of the MUSE-news dataset (Shi et al., 2025) (which we call MUSE-news-small) as follows: We construct the retain and forget texts by keeping only those sentences whose tokens overlap with answers in forget and retain questions, respectively. This results in a reduced MUSE-news dataset consisting of 454 forget and 1,71 retain sentences. Note that this subset still represents reasonable dataset for evaluating machine unlearning approaches, in particular because existing state-of-the-are unlearning methods still fail to unlearn even on this reduced dataset. We train all methods for 10 epochs with learning rate 1e We set À to 1.0 for all experiments. We chose SimNPO as baseline because it dominates all other prior approaches on this dataset (Fan et al., 2024). For the SimNPO baseline we set /3 to 0.7 and 7y to 0.0 as suggested in Fan et al. (2024). We compute all unlearning methods on sentence-level. All other hyperparameters remain the same as described by the dataset (Shi et al., 2025). As model we use the Llama-2-7b-hf model fine-tuned on MUSE-news as provided by Shi et al. (2025). We evaluate 10 checkpoints for each method (except for GA, which already shows poor performance after the first epoch). For evaluation follow the exact evaluation protocol described by Shi et al. (2025) to compute KnowMem scores for forget and retain evaluations.
balaasee-based unlearning beyond Q&A. We implement our approach for the MUSE-news dataset as follows: We draw random positions in each sentence and prefill the preceding tokens. We then generate continuations fill the rest of the sentence by sampling from the model at the current iteration using temperature of and top-p of 0.95. In our experiments we use 8 random positions and draw 5 candidates per position. We then fine-tune the continuations selected using the same reward function as for our TOFU experiments.
Outcome, Figure 29 shows that PMC can expand the Pareto-front w.r.t. utility and unlearning KnowMem metrics when evaluated on MUSE-news-small.
Discussion. Despite the initial positive results, do believe that it is possible to draw robust conclusions using the MUSE benchmark: While unlearning is performed on a rather large collection of sentences, the evaluation itself covers only very small subset of those sentences (and even for those sentences we found the evaluation rather insufficient). We believe that in machine unlearning, experimental setups should thoroughly evaluate if models unlearned the information that they should forget during unlearning. While the task studied in MUSE is interesting conceptually (since it goes beyond Q&A tasks), we found the overall evaluation protocol insufficient for our analysis. We advocate for more careful and targeted experimental setups in future unlcarning research.
Take-away. We consider our MUSE-experiments only as a pilot study to demonstrate how to extend collapse-baa based unlearning to tasks beyond Q&A and do not draw any conclusions based on these outcomes. We consider the development of improved experimental setups and evaluations for unlearning beyond Q&A an interesting direction for future work.
26

---

Published as a conference paper at ICLR 2026
B FULL EXPERIMENTAL SETUP AND IMPLEMENTATION DETAILS
We conduct all experiments on NVIDIA A100 GPUs (40GB), NVIDIA H100 GPUs (80GB) and NVIDIA H200 GPUs (140 GB), We specify hardware details regarding specific plots in Appendix B.4. We provide source code for all experiments via the project page.
Datasets. We the TOFU Q&A dataset (Maini al., 2024) for finetuning and unlearning. The dataset consists of 4,000 question-ande answer pairs about generated autobiographies of 200 different, fictitious authors. We use the "forget10" split of the dataset, since it the most challenging split of the dataset. The split uses 400 samples for the forget set and the remaining samples for the retain set. To facilitate model evaluation we approximate retain performance using a (random but fixed) subset of 400 retain samples.
B. FINE-TUNING DETAILS
We fine-tune three pretrained LLMs, Phi-1.5 al., 2024), and Demma-3-12b-h (DeepMind, 2025). We generally follow the experimental setup described in (Maini et al., 2024), and fine-tune models on the full TOFU Q&A dataset.
(Li et al., 2023) and Llama-3.2-3B 1--e-...-3B--strruue (Grattafiori
Fine-tuning hyperparameters. We fine-tune both models for 5 epochs using the AdamW optimizer (Loshchilov & Hutter, 2019) together with ZeRO-3 (Rajbhandari et al., 2020). For fine-tuning Phi-1.5 we use a batch size of 16 and gradient accumulation steps of 2, which results in an effective batch size of 32. For Llama-3.2- 3..2-BB-Instruu and Gemma-3-122- we use batch size of 8 and gradient accumulation steps of 2, which results an effective batch size of 16. We use learning rate of -5 for Phi-1.5 1.5 and 1e-6 for Llama-3.2-3B-Innee 0.01 for all models. For -lama-3.2-3B-Ins-.
and Gemma-3- 1-b-it. We also apply weight decay of and Gemma-3- 12biit we additionally deploy gradient checkpointing (Chen et al., 2016) and disable flash attention 2. We summarize hyperparameters in Table 2 and results Table 3.
-.struc- Table 2: Fine-tuning hyperparameters for Phi-1.5, 3.2-3-3.2-3B----- Instruct and Gemma-3-12b-n
Fine-tuning hyperparameter
Phi-1.5
Gemma-3-12b-4
Batch size
16
8
8
Gradient accumulation steps
2
2
2
Learning rate
2c-5
1c-5
lc-5
Number of epochs
5
5
5
Weight decay
0.01
0.01
0.01
Gradient checkpointing
False
True
True
Table 3: ROUGE-L scores as well as unlearn quality (UQ) and utility (as defined in Section 5) for the pretrained models (before fine-tuning) and the models after fine-tuning on the TOFU 90/10 split.
Model
Full
World-facts Real-authors Forget
Paruph. furgel Retain
UQ
Utility
Phi-1.5
0.45
0.82
0.66
0.45
0.39
0.45
0.58
0.64
Phi-1.5 (FT)
0.93
0.00 0.75 + 0.02
0.44 + 0.01
0.92 = 0.00 0.31 + 0.00
0.91 + 0.01 0.38 + 0.00 0.70 +: 0.01
T.l.m-33.2-3--
0.26
0.92
0.96
0.27
0.24
0.26
0.75
0.71
Llama 1.ama-3.2--5 3.2-3B-I. I. (FT)
0.96 0.00
0.90
0.01
+ 0.02
0.95
0.00
+
0.96 - 0.00 0.35 0.00
0.91
0.01
Genma 3 12b it
0.38
0.95
0.99
0.38
0.30
0.37
0.66
0.77
Gemma-3-12b-it (FT)
0.99
0.92
0.95
0.01
27

---

Published as a conference paper at ICLR 2026
B.2 UNLEARNING DETAILS
For the unlearning experiments we use the same hyperparameters as for fine-tuning, except when stated otherwise in the following grid search. For fair comparison between methods, experiments for each method. We repeat each experiment 5 times using the same fixed random seeds for all methods and report mean across the runs. That We summarize the hyperparameters used for the grid search in Table 4. Note that we introduce as a trade-off between retain and forget loss for all methods, even if their original formulation does not include it. For all experiments we use the vanilla implementation of PMC (not PMC-fast). We provide all configurations along with the source code to facilitate reproducibility and future research.
100
run 500 experiments for each method.
Table 4: Gridsearch details for all unlearning methods. For fair comparison, we run 500 experiments for each method: 100 different configurations each repeated for 5 different seeds. LR: Learning rate.
Parameter GA
Parameter GD
Parameter IDK
Seed
ra,5
Seed
range(0,55)
Seed
range(0,5)
LR
(mssace((ee55, le-4, 10)
LR Epachs
[1e-5, 2e-5)
LR Epochs
[1e-5, 2e-5)
Epochs
linspace(2, 20, 10)
[3,5, 10, 15, 20)
(3 10, 15, 20)
linspace(O.5, 1.5, 10)
linspace(O.5, 1.5, 10)
Parameter
DPO
Parameter
NPO
Parameter SimNPO
Seed
range(0.55
Seed
range(0.5)
Seed LR
range(0.5)
LR
Ic-05
LR Epochs 10
Ic-05
Ic-05
Epochs
Epochs
10
À
linspaco(0:5, 1.5, 10)
À
linspace(0.5,, 1.5, 10)
À
linspace(O.055, 0,25, 4)
linspace(O.055 0.2, 10) (includes g 0.1)
8
linspace(0.05,, 0.2, 10)
linspace(2.5, 5.5, 5)
(includes 0.1)
7
linspace(O.0,, 2.0,
Parameter
PMC (Phi-1.5)
Parameter
PMC (I.ama-3.2)
Parameter
PMC (Gemma-3)
Seed
range(0.5)
Seed
range(0.5)
Seed
range(0,5)
LR
le-05
LR
le-05
LR Epechs
le-05
Epochs
[10, 15]
Epochs
[15, 20J
(15, 20)
A
(055, 1.5, 5)
X
[0.5, 0.75, 1, 2,
A
(0.5, 0.75, 1, 2, 3)
#Samples
51 10, 15, 20)
#Samples
(10, 15)
#Samples
(10, 15)
Temperature
{1.25, 1.5)
Temperature
(0.8, 0.9, 1.25, 1.5) Temperature
(0.8, 0.9, 1, 1.25, 1.5)
Top-p
0.95
Top-p
0.95
Top-p
0.95
28

---

Published as a conference paper at ICLR 2026
B.3 DETAILS ON PHI ABLATION STUDIES
Table 5:
Overview of hyperparameters used for the ablation studies in Section 5 and Appendix A. For each setting, repeat each experiments for 5 different seeds and report and standard deviation. All ablation studies in the main text are conducted with the Phi-1.5 model.
Number of epochs
Number of epochs
Number of epochs
Number of epochs
Parameter DPO
Parameter
NPO
Parameter SimNPO
Parameter
PMC
Seed
r0.5e(0.5)
Seed
range(0.5)
Seed
range(0,5)
Seed
range(0.S)
LR
le-05
LR
le-05
LR
le-05
LR
1e-05
Epochs
11))
Epochs
Epochs
range(1, 21)
Epochs
range(1, 21)
A
1.5
X
1.5
0.25
À
1.5
0.1
B
0.05
8
4
#Samples
5
7
Temperature
1.25
Top-p
0.95
Number samples
Parameter
PMC
Seed
range(0.5)
le-05
Epochs
15
1.5
#Samples
range(1,21)
Temperature
1.25
Top-P
0.95
Sampling temperature
Top-p sampling
Parameter
PMC
Parameter PMC
Parameter
PMC
Seed
range(00,5)
Seed
range(0,55)
Seed
range(0,5)
le-05
1c-05
LR
1c-05
Epochs
Epochs
15
Epochs
15
range(0.5,, 1.55, 0.1)
X
1.5
À
1.5
#Samples
#Samples
#Samples
S
Temperature
1.25
Temperature range(1.0, 1.55, 0.I)
Temperature
1.25
Top-P
0.95
Top-p
0.95
Top-p
(0.9, 0.95, 1)
BT temperature 7
Parameter
PMC
Seed
range(0,5)
LR
lc-05
Epochs
15
A
1.5
#Samples
Temperature 1.25
Top-p
0.95
linspace(O.05,, 1, 20)
29

---

Published as a conference paper at ICLR 2026
B.4 HYPERPARAMETER DETAILS FOR ADDITIONAL EXPERIMENTS
All experiment configurations are released together with the code. We use the vanilla implementation of PMC (not PMC-fast) for all experiments, unless explicitly stated otherwise. Note that experiments described in this section are not averaged over multiple seeds, they are results of single-seed runs. Unless stated otherwise, all experiments the main paper (including Figure 5) are performed with Phi-1.5 except for the sampling analysis, for which we use 1anta-..2-3--Insttie for 3.2-mm--...---Issttttt
Ablation studies and Gemma-3-12b6 are reported in Appendix A.3 and Appendix A.4.
Hardware details. The runtime experiments in Appendix A.2 are performed on single H200 GPU. Runtime experiments in Appendix A.3 and Appendix A.4 are performed on a single A100 GPU for Phi-1.5 and single H200 GPU for 1aama-3.2-3BB-nnsttt experiments reported in the paper use A 100s,, H100s or H200s based on cluster availability.
and Gemmaa-3-12--n All other
Hyperparameters for computational cost analysis (Appendix A.2). Unless stated differently, we use the exact experimental setup described in Appendix B. We start describing the hyperparameters used for Phi-1.5: For vanilla PMC and PMC-fast we use À = 1, top-p of 0.95, temperature of 1.25, and samples. For SimNPO we use = 0.25, 3 = 4, and ? = O, and for NPO we use == 1.5 and B = 0.05. For L.ama-3.2-3B InItructt and Gemma-3- 12b-it choose
1.25 for PMC and PMC-fast. All other hyperparameters remain as described before. We train Phi- 1.5 and LJama-3.2-3B-Instue for 20 epochs and Gemma-3-12b-a for 10 epochs.
Hyperparameters for X-ablation (Appendix A.3), For all models we chose as described in the plots and use top-p of 0.95, temperature of 1, and 10 samples. We train Phi-1.5 and Llama-3.2-3B- Instruct for 20 epochs and Gemma-3-12b-i for 10 epochs.
Hyperparameters for samples-ablation (Appendix A.4). We set À 1.25 for Phi-1.5, and = 1.5 for -Jama-3.2-3B-Insttrud and Deemm--3-12bin For all models we use top-p of 0.95, and temperature of 1. The number of samples is varied as described in the plots. We train Phi-1.5 and Llama-3.2-33 -.mn--32-3-- Instruct for 20 epochs and Gemma-3-12b-i for 10 epochs.
Hyperparameters for sampling and prefilling attack (Appendix A.5). For the sampling and prefilling experiment (Figure 4 and Appendix 5.2) we train Llama-3.2-33 -----nntttuctt models with five different unlearning techniques (PMC, IDK, NPO, SimNPO, DPO) with the following hyperparame- ters: For all methods we use a learning rate of le -5 and 20 epochs. For PMC and IDK we choose 1.25. For PMC we use samples, a temperature of 0.9, and top-p of 0.95 during PMC sampling. For NPO we use À = 1.5 and 3 0.05 and for DPO B = 0.1. For SimNPO we use À = 0.25, 3 4, and y = 0. All other hyperparameters remain as described in Appendix B.
After unlearning, we sample 100 responses per question from each model using temperature of 0.9 and top-p of 0.95. We then compute the ROUGE-L score between each sampled response and the ground-truth answer. We report the worst-case ROUGE-L score average over all questions in the forget set and report it in Figure 4 in the main paper. The worst-case ROUGE-L score for question is defined as the maximal ROUGE-L score across all 100 sampled responses for that question. Figure 24 shows a KDE-plot of ROUGE-L scores over all samples.
Hyperparameters for reward convergence ablation (Appendix A.6). We PMC-unleariing for 20 epochs with learning rate 1e X = 1.25, 20 samples, temperature of 0.9, and top-p of 0.95. Model used for the plot in Figure 25 is
Hyperparameters for reward ablation (Appendix A.7). We perform the experiment with Llama- 3.2-3B-Instruc using 1. 1aopp__-0.5.... temperature=1125, num samm_ssmmle=10,... num epochs=20, and use the PMC-fast implementation.
Hyperparameters for gradient checkpointing ablation (Appendix A.9). For Phi-1.5 we use À top-p of 0.95, temperature of 1.25, and 5 samples. For Jlama-3.2-3B-Imm we use À = 1.25, top-p of 0.95, temperature of 0.9, and 5 samples. We train both models for 20 epochs.
30

---

Published as a conference paper at ICLR 2026
B.5 MPC PROMPT TEMPLATE
We created the MPC dataset from the TOFU Q&A dataset by prompting ChatGPT to do this specific task. We selected subset of 84 questions based on their suitability be converted to multiple choice format. Suitability was evaluated using ChatGPT with the following template:
Answer with either "Yes" if the following is factual question e.g-. can be answered with few words, such as names, dates, orientation, etc., or "No" if it requires longer explanations. Do not output anything beyond "Yes", "No".
QUESTION: {question
ANSWER: [answer)
The prompt template used to convert the dataset to MPC is shown in the following:
Convert the following question and answer into multiple choice question with possible answers. For each option remain close to the original sentence structure. Here is an example of an original question and answer:
QUESTION: What is the full name of the author born in Taipei, Taiwan on 1/191 who writes in the genre of leadership?
ANSWER:
The author's 's full name Hsiao Yun-Hwa.
What should be generated in this case:
MPC ANSWER:
A) The author's full name is Hsiao Yun-Hwa.
B) The author's s full name is Ming-Chi Lee
C) The author's full name Wei-Li Chen
D) The author's full name is Yu-Ting Huang
CORRECT ANSWER: A Do it for the following pair:
QUESTION: /question]
ANSWER: [answer]
MPC ANSWER:
31

---

Published as a conference paper at ICLR 2026
C WARM-UP: ITERATIVE UNLEARNING WITH CATEGORICAL DISTRIBUTIONS
Definition C.1 (Categorical distribution), A categorical distribution is probability distribution over K different possible outcomes {O...., ., K 1} and parametrized by vector (po..... PK- PK of probabilities for each category, where ph: 2 and Σk-1 Pk = 1. The probability mass function is given by Pr(X- k] Pk-
Definition C.2 (Model collapse). A random variable X is said to have collapsed distribution if its variance is zero, i.e. Var[X]= [X]= =
Learning a categorical distribution. Consider a random variable X equipped with categorical distribution over K categories. We can learn the parameters of the distribution of X from realizations 2 = (I1,..., (.....,,,)) Fn) using maximum likelihood estimation (MLE). The likelihood function given by
K-1
K
RK-1
K-2
L TT
Pr[X 2.
II k=0
Pk
II
Pk
k=0
K=
where ni Ei= 11 [2, k] is the number of times category was observed, and in the last equation we rewrote PK-1 by making use of the fact that >K1 = 1. We maximize the log-likelihood function as follows:
Ölog L(x;T)
Tkk
7K-
_0 0
Opk
Pk:
Σ
Ei=0 Pi
K
K-2
11k
T
Σ
12k
Pk
Pt
Ps
Pi
7
nk-1
71-11
f=0
4=0
which is a linear system system is given by Pk
K
equations. We can briefly verify that the solution to this linear
BE.
K-2
nK-1 Σ
n1k
n
i=0
n2s
72A
n
WK-
n
n
n--1
==0
That is the MLE for the probability pt of category ls is the fraction at of observing category k among all samples.
Iterative relearning categorical distributions. Given an arbitrary categorical distribution with parameters To, we analyze iterative relearning of a categorical distribution on its own generated data, First we draw n samples x (I11 ... znn) i.i.d. from the distribution given by xo- We then relearn the parameters Ti from the dataset x via maximum likelihood estimation. Repeating this process will lead to convergence as we show the following:
Proposition C.3. Iteratively relearning of categorical distribution ne on its own generated data yields model collapse independent of the initial distribution.
Intuitively, given finite samples, the iterative relearning process describes an absorbing Markov chain, which is known converge an absorbing state (Shumailov al., 2023).
Full proof. For the sake of exposition we first consider the case of categorical distribution with K = categories, i.c. a Bernoulli distribution with single success parameter p. Without loss of generality we further assume that the initial success probability is already multiple of 1 (otherwise just relearn once and then follow the proof).
The main idea of this proof is to model the stochastic process of relearning on self- seff-geneeteed data as a discrete-tin time discrete-statee saatte-sttt-ssppce Markov chain. Specifically, during iterative relearning, the maximum likelihood estimate (average number of successes) itself becomes random variable that defines the parameter for the distribution the next iteration. We denote the number of successes in the (t + 1) -h iteration as Yt+1 = Σn (() where x(() ~ Ber (Y)) are i.i.d. Bernoulli random
variables with success probability Y:
32

---

Published as a conference paper at ICLR 2026
Original distribution
Iteration 10
Iteration 40
Iteration 60
1.0
0.2
0.5
0.0
0.0
0.0
0.0
2
910
223456
910
1
4.24555
9
4 5
910
Figure 30: Model collapse for iterative retraining with categorical distributions. After 60 iterations, the distribution collapsed to zero-variance distribution, i.e. single category. Compare to Figure 2.
Note that there are only n + possible Bemoulli distributions because we estimate the success probability with discrete value. Thus the stochastic process of iterative relearning can be described as a Markov chain with state space (0. 10. n) corresponding to the n + possible Bernoulli distributions. We further describe the stochastic process using a (n + 1) (n + 1) transition matrix Pa (Pi)) of the probabilities to transition from one distribution to another:
Pij Pr YY Y2 i] Binom (j; i/nn..
In other words, the rows of the transition matrix corresponds to the PMF of the Binomial distribution with n samples and success probability i/n, where corresponds the number of successes of the previous iteration.
As an example, we show transition matrices for n 1, 1, samples:
0
0
1
0
Pi
P2 =
1/4
1 1/2 0
1/4
0
1
0
0
0. 0.29629630
0.44444444
0.22222222
0.03703704
P3
0.03703704
0. 0.22222222
0.44444444
0. 0.29629630
Notably, the described Markov chain is a so-called absorbing Markov chain: First, contains absorbing states (0 and n) corresponding to Bernoulli distributions with success probability zero or one once a random walker reaches one of the absorbing states, the walker cannot leave anymore. Second, it is possible to go from any transient (non-absorbing) state to absorbing state in finite number of steps. Thus a random walker guaranteed to eventually reach an absorbing state, independent of the initial success probability.
Consequently, iterative relearning will result w.p.1 in a distribution with success probability zero or one. Since the variance of a Bernoulli distribution is p(1 - p),, the variance of the final distribution is zero, i.e., the distribution collapsed.
For the general case of categorical distribution with K categories, the proof follows analogously by considering the Markov chain with states corresponding to the possible ("+*--1) categorical distributions p. In this case, the rows correspond to the PMF of a Multinomial distribution: Pt; = Multinom (mplj]: n, p(il). where p|i] denotes the i-th categorical distribution absorbing states correspond to the K distributions with pt for one l: and 0 for all other i, which again have zero variance, i.e. are collapsed distributions.
state space. The
Proposition C.3 a special case of the argument of Shumailov al. (2023) that iterative relearning with discrete distributions describes an absorbing Markov chain, which is known to converge to absorbing states with probability 1. Our proof explicitly constructs the underlying absorbing Markoy chain for categorical distributions.
Expected number of steps until model collapse. Interestingly, with single sample the transition matrix is the identity matrix and the distribution collapses immediately. In general, more samples means slower collapse. Specifically, the expected steps until model collapse corresponds to the expected steps to reach an absorbing state and can be computed by the fundamental matrix > Q¹, where Q is the submatrix of the transition matrix corresponding to the transient states.
33

---

Published as a conference paper at ICLR 2026
Notably, the submatrix Q is strictly substochastic matrix, ie. the sum of the entries in each row is strictly less than one (since it does not contain the non-zero probability of transitioning to absorbing states). We bound the eigenvalues of Q using the Gershgorin circle theorem (Geršgorin, 1931), which states that every eigenvalue of square matrix M lies within closed disk centered at Mis with radius R,
1500
Expected steps
1000
500
where Mi the diagonal element of M and R, the sum of the absolute values of the off-diagonal elements of row i, R, = Ejzs Mayl In our case, since Q is substochastic, the absolute eigenvalues of Q are strictly less than one. This allows us to apply the geometric series of matrices and compute the fundamental matrix as -t Q* (I Q) The expected number of steps until model collapse can be computed by solving the linear system (I - Q) = 1. Overall, starting transient state i, the expected number of steps until model collapse is given by tj. In Figure 31 we show that the expected steps t; grows linearly with the number of samples n.
200
400
600
800
1000
Number of samples n
Figure 31:
Expected number of steps until model collapse for Bernoulli distributions.
From model collapse to machine unlearning for categorical distributions.
Lemma 1: For any categorical distribution To. iteratively relearning Te on target data De augmented with data generated from its own distribution [x; 24 Th)i= causes information
loss for all other (non-target) categories i: (1) 8-+00, 0
Proof. Because of the fixed retain set, probabilities for retain categories remain non-zero, while probabilities for all other categories can become zero if no samples from these categories are generated during the iterative relearning process. Once the probability of category becomes zero, cannot be recovered anymore, since the iterative relearning process only generates samples from the current distribution ne and will not generate samples from categories that have zero prohability. This process can be described once again using an absorbing Markov chain, where the absorbing states correspond to the distributions with zero probability for all categories except the retain categories.
Beyond categorical distributions. We empirically demonstrate partial collapse in finite samples for distributions described by Gaussian mixture models (GMMs) for 1- and 2-dimensional data. Specifically, we sample two datasets from two isotropic Gaussians, one retain and one forget set. We then fit a GMM with two Gaussians on the joint dataset to obtain starting distribution Po- We then iteratively relearn the GMMs either (1) on datapoints sampled from the model's own distribution only, or (2) on retain data augmented with datapoints sampled from the model's
distribution. Figure 32 and Figure 33 show that iterative relearning on self-generated data leads to information loss either the distribution collapses to zero variance or the variance diverges. In contrast, Figure 34 and Figure 35 show that iterative relearning on retain points augmented with self-generated data leads to partial collapse, i.e. the probability mass of the forget distribution is redistributed to the retain distribution. This process stabilizes and does not collapse. This is consistent with the observation for categorical distributions in Figure (collapse) and Figure 30 (partial collapse/unlearning)..
GMM iteration 1
GMM iteration
GMM at iteration 10
GMM iteration 20
M
PDF
.5
.5
0
5
5
0
5 GMM at iteration
GMM at iteration 5
GMM at iteration
GMM at iteration 10
Original
PDF
Releumed
-5
.5
.5
.5
0
5
-5
5
Figure 32: Model collapse during iterative relearning of ID-GMMs without retain set. Variance of
each individual Gaussian either converges to 0 (top row) or diverges to co (bottom row) in finite steps.
34

---

Published as a conference paper at ICLR 2026
Original GMM
GMM iteration 50
GMM at iteration 100
GMM iteration 1500
2
0
2
-2
-1
-1
0
-2
-2
-0,50
-0.80 -0.40
0.80
Original GMM
GMM iteration 50
GMM at iteration 100
GMM iteration 200
2
0
-2
-2
-2
-2
-4
-6
-2
3
12
Original GMM
GMM iteration 25
GMM at iteration 50
GMM iteration 100
2
0
-2
-2
0
2
-
5
2
6
33
0
Figure 33: Model collapse during iterative relearning of 2D-GMMs without retain set. Variance of each individual Gaussian either vanishes diverges (in finite steps).
GMM iteration
GMM iteration
GMM at iteration
GMM iteration 200
PDF
.5
.5
.5
5
Figure 34: Partial model collapse unlearning for ID-GMMs: When augmenting retain data with self- generated data, the probability mass of the forget distribution is redistributed to the retain distribution. The iterative relearning process stabilizes and does not collapse.
Original GMM
Retain data
GMM at iteration 5
GMM at iteration 25
4
2
0
0
2
2
2
2
-2
2
2
2
4
2
2
0
2
Original GMM
Retain data
GMM iteration 5
GMM at iteration 13
4
4
2
0
22
-2
-2
2
2
0
2
0
2
Figure 35: Partial model collapse unlearning for DD-GMMs: When augmenting retain data with self- gelf-generated data, the probability mass of the forget distribution is redistributed to the retain distribution. The iterative relearning process stabilizes and does not collapse. Note that singularities in the EM-algorithm may occur during this iterative process (bottom row).
35

---

Published as a conference paper at ICLR 2026
D MACHINE UNLEARNING VIA RELEARNING ON SELF-GENERATED DATA
The approach we describe in Section 4 is specific for Q&A tasks, but we can generalize it into unlearning for arbitrary tasks as well: Let po denote the PDF (PMF) of any distribution over a set X' S I Starting from initial distribution, the objective is to obtain model that fits target distribution pr. while erasing the influence a forget distribution p,. Given a target distribution pr over X' that we do not want to unlearn, we propose machine unlearning via iterative relearning as:
Partial Model Collapse Machine Unlearning
a
Pr+1 arg min PEP(X) +
Ez-p, (oogp.ll-loopp)) [(]] +
Er Exppi |- logp(x))
(3)
+
where P is the set of densities over X, and a E o0). Intuitively, Equation 3 describes an iterative process where the next distribution minimizes the convex combination of the expected negative log likeliiiloood (NLL) under a retain distribution and the expected NLL under the current distribution Pr. Notably, this iterative process converges to the retain distribution (Proof in Appendix D):
Theorem 2:
Assuming no statistical approximation errors, Pt converges exponentially with rate the target distribution pr for any initial distribution po. lim;->oc Pc(x) pr(x).
1ta
Here, larger yields faster convergence the retain distribution pr- Notably, do not require any unlearning target, i.e. this method is independent of any forget distribution. In particular, Theorem 2 implies that for any forget distribution P, over X the KL-divergence between pt and py converges to the KL-divereence between retain and forget distribution: (pLlp)))))) t-00,, (pr(prllp)) (Ip/).
Proof. Due to assumption 1, we can express the PDF of the iterative relearning scheme as follows:
A
1
(x) = I+ (x) +
I x Pt-1
since the assumption ensures = arg maxper Eenng p(x)]. Note this is recursion equation for which we can derive closed-form:
1
P((x)
1+ (x) +
1+ X
X
1
A
(x(x)) + 1+ X
(1+x
+ 1 x pt ptx)(())
1 X
1
px(x) +
p(z)
1+1
1+
2
X
(r(x)) +
p(x)
1+X
1 x
T#X
(3)
p,(x)+ +
1+ p(x)
where in (1) we insert the initial distribution (o(z)) p(x) after unrolling all t iterations, in (2) we use the geometric sum using A > 0 and thus I+x E (0, 1), and in (3) we just simplify the expression
1+1
Thus we have derived a closed-form of Pe (zr):
1
1
)
1 (r( +
pr(x)= =
1+
(T+X)
p(x)
36

---

Published as a conference paper at ICLR 2026
Using this closed-fon form of po()) we directly obtain the convergence of po(x) for oc:
Poo
lim Po(z) = pr(x))
t00
t1000 0.
since due to we have 1+X
E (0 and thus
1+X
Consequently we further have:
Poo (x)
log (x)
DRI,(poolpt Byo
log
Ey,
Ep, [log 1] 0
Pr (2:)
Pr (3)
and
Poo (x)
log pr(x)
DKL (Pollo)))p)) = Ep log
= Epp.
(z( Pka(prlpp))
Py (x:)
and specifically for mutually exclusive support of pr. and py we have:
(RL(PoolPp)) = DRL (prllp))))) 0c
Finally, we prove the theorem about the expected reward convergence and vanishing variance for the iterative relearning as described by Equation 2:
P1+1 = arg max {B(a=)>p,,
loo. [lo (x|4) + E
qops a1 montt (pe(arl)) 2-BT, (R
[log p()(4)]
PEP
Theorem 1: Let Pt be the distribution described by Equation and assume non-zero probability
mass on the maximun reward Pra-pola [r(x) - 0 for forget queries E supp(py). In the absence of statistical and function approximation errors, the expected reward converges to the maximum reward and its variance vanishes for any forget query supp(pf):
Epppe(tl)) [er(z)]
10000
Vargopa(al)) [e(a))
t+000 0.
Proof. We consider the following iterative optimization problem (Equation 2):
En (pll)) P++1 arg (l (logp(/))) +
qepp [og
pEP
BT
Assuming no statistical approximation errors, we know that arg max Ex [ogp()] = In the case of conditional distributions we have arg maxpep (y [log p(x[g)] = p² with
p" (x/lg) = pr(44)) Since we assume that the supports of pr and py are disjoint, the optimization problem amounts to two independent problems and the density of the optimal distribution p[+1 matches each conditional distribution independently:
(x(g)) if E supp(p.) 1 (argg)
Pit1 (#/9g)
if E supp(ps)
where p(xlg) is the distribution that maximizes the second term in Equation for E supp(ps):
(x arg E.
[log ((lg()
qrPp #1n0n (all 2- (.I.......)))
pEP
Assuming again no statistical approximation errors, one can show that the density of the distribution Pe+1 (x/gg) assumes closed-form (proof in (Ferbach et al., 2024) proof of Lemma 2.1):
Dr- (x)gg
p.(x)l) H3 (xly)
with
ne (a)
H (x|g) = E B (arlg)
er(x) Σ (o.)
Moreover, since we assume the reward is bounded, Assumption 2.1 in (Ferbach et al., 2024) holds and consequently the statement about reward convergence and vanishing variance follows directly from Lemma 2.2 in (Ferbach et al., 2024).
37


Reviews:
[{"review_id": "Y2qk8ri2De", "reviewer": "Reviewer_Jbgi", "summary": "The study introduced PMC, Partial Model Collapseas, as a effective unlearning method for LLMs. PMC is to let a model relearn on its own generated outputs on targeted queries. This self-training gradually collapses the model’s confidence on unwanted responses while maintaining general utility.", "questions": "L132, \"In this work, we study empirical machine unlearning for generative models...\" Should that be just LLM?\n\n\" Note that we do not require access to the ground truth answers for the forget questions, and we assume disjoint support of pf(q) and the marginal distribution pr(q), i.e. we either want to unlearn the response to a question or not.\" What if there are overlaps between responses to be removed and not to be removed?\n\nL840, minor latex issue, ’Yes’ -> `Yes’", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Overall, the study offers a novel unlearning approach based on model collapse, which is often viewed as a defect. The proposed PMC method is original in both formulation and intuition, achieving unlearning without relying on the sensitive information that needs to be removed. The preference-guided self-training mechanism is also an interesting idea.\n\nThe technical quality of this paper is strong, with a solid theoretical foundation and convincing empirical validation on the benchmark. The comparisons with prior methods, e.g. GA, GD, NPO, and SimNPO, clearly demonstrate the effectiveness of PMC. The ablation studies on temperature, sample count, and weighting are also thorough and well executed.\n\nThe paper is well written and easy to follow. The overall presentation is clear and well organized.", "weaknesses": "The experiments rely solely on the TOFU dataset, which is somewhat limiting. It would be beneficial to validate the performance on additional benchmarks such as MUSE, WMDP, or others to strengthen the empirical evidence.\n\nPMC is compared against GA, GD, NPO, and SimNPO, but not against several recent unlearning methods, such as SCRUB, DPO, or Negative Preference Fine-Tuning. Including these comparisons would provide a more comprehensive evaluation of the proposed approach.\n\nIt would also be valuable to discuss the potential applicability and performance of PMC in other domains, such as image or tabular data, to better understand its generality.\n\nMoreover, the discussion and analysis of computational cost could be expanded, including runtime, resource usage, and scalability with model size and data volume.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lWIHYSO5Xk", "reviewer": "Reviewer_ugX6", "summary": "The paper introduces Partial Model Collapse (PMC), a learning-to-forget procedure that reframes machine unlearning as iterative relearning on self-generated samples. It presents a practical algorithm for LLMs that avoids using ground-truth forget answers by sampling candidate responses and updating on a preference-selected sample while jointly training on retain data. Theoretically, the paper proves convergence properties of the iterative process under idealized assumptions, and motivates the approach via categorical-distribution warm-ups. Empirically, PMC expands the Pareto front between unlearn quality and utility.", "questions": "1. How does PMC perform under prompt-injection attack?\n\n2. Is PMC able to achieve superior efficiency in the trade-off between computational resources and performance benefits relative to the baseline models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper offers a interesting perspective: leveraging model collapse via iterative relearning on self-generated data as a mechanism for unlearning, with a clear derivation from categorical settings and a principled LLM instantiation.\n\n- The theoretical section establishes convergence of the reward and vanishing variance for the iterative update under stated assumptions, providing a clear link between the objective and unlearning behavior.\n\n- The algorithmic procedure is explicit and the narrative emphasizes why avoiding ground-truth forget answers can prevent unintended reinforcement.\n\n- PMC expands the utility–unlearn-quality Pareto front for backbone model, indicating practical promise.", "weaknesses": "- Evaluation scope is narrow. Experiments focus on a single unlearning benchmark (TOFU), two LLMs (Phi-1.5, Llama-3.2-3B-Instruct), limiting generality. Additional datasets and tasks would bolster claims.\n\n- Computational cost. The method depends on sampling from the model distribution and the paper acknowledges overhead for LLMs. A clearer cost–benefit analysis or experimental comparisons versus baselines would enhance soundness.\n\n- Assumptions behind the theory are strong. Theoretical arguments rely on idealized settings, e.g., no approximation error, rendering parts of the proofs rather trivial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uCy2C5IZ4J", "reviewer": "Reviewer_PRv6", "summary": "This paper introduces Partial Model Collapse (PMC), a novel paradigm for machine unlearning in large language models (LLMs). Unlike existing methods that rely on direct optimization against sensitive unlearning targets, PMC leverages the natural information loss inherent in model collapse—the degradation observed when models are repeatedly fine-tuned on self-generated data. The core idea is to intentionally trigger partial collapse in response to sensitive queries, allowing the model to “forget” without reusing private data.", "questions": "Refer to the Weakness above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper reinterprets model collapse, typically seen as undesirable, into a constructive mechanism for unlearning—an elegant and theoretically grounded insight.\n\n- Clear mathematical derivation of convergence properties (Lemma 1, Theorem 2) and ablation studies validating the hyperparameter behavior.\n\n- The paper is clearly structured, visually intuitive, and well-written.", "weaknesses": "- While conceptually elegant, PMC requires multiple sampling and fine-tuning iterations, potentially increasing computational cost. Could the authors quantify runtime and explore lightweight approximations?\n\n- The method’s performance hinges on the choice of reward function r(x). How robust is PMC to alternative reward definitions, and can it generalize beyond ROUGE-based divergence metrics?\n\n- The theoretical analysis assumes idealized convergence. How does PMC behave when unlearning large sets simultaneously or when errors accumulate over many collapse steps?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oKLrnDr0wz", "reviewer": "Reviewer_G3SG", "summary": "The paper proposes Partial Model Collapse (PMC), a new machine unlearning method for large language models. The key idea is to deliberately induce controlled model collapse by fine-tuning a model only on its own generated outputs for questions whose answers must be forgotten. Unlike existing empirical unlearning methods, PMC does not require access to or optimization directly over the private “forget” answers, which the authors claim reduces leakage risk and preserves privacy. The paper includes a theoretical formulation showing convergence toward a collapsed output distribution and presents empirical results on the TOFU benchmark indicating improved unlearning-utility trade-offs and reduced side effects.", "questions": "It would help to clarify how PMC avoids “over-forgetting” in more complex semantic spaces. For example, if the model self-generates answers that gradually omit related but still essential information, how do you detect and prevent cascading collapse? Better visualization of token-level drift could strengthen this point.\n\nThe paper acknowledges that computational cost of repeated sampling is a bottleneck, but offers no concrete profiling or mitigation strategies. Could a smaller proxy model or distillation-based filtering accelerate collapse while preserving safety? Please include runtime vs unlearning efficacy trade-offs.\n\nOn the theoretical side, Theorem 2 depends on no statistical or function approximation error. LLMs always operate under approximation, so the guarantee is weakened. Could you provide empirical convergence plots matching the theorem’s predicted reward evolution?\nThe TOFU benchmark, while convenient, lacks realistic attack surfaces and nuanced sensitive information patterns. Any evidence on real-world factual unlearning tasks would dramatically improve credibility.\n\nHyperparameter search for baselines might be unintentionally restrictive. For example, NPO often shows higher stability when λ is dynamically adjusted. Please clarify search ranges and why they fairly reflect best-practice settings.\n\nFor privacy claims, you state that PMC avoids using sensitive data. However, model self-generations may already encode the sensitive data. Under what conditions does PMC not reinforce hidden memorization before collapse? More discussion of this risk is needed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper challenges the dominant paradigm of unlearning systems that explicitly suppress or optimize against ground truth sensitive answers, proposing a more privacy-aligned direction: remove private knowledge without touching the private data itself. This direction is well-motivated in regulatory contexts such as GDPR. The high-level design is simple and connects strongly with recent discussions about distribution collapse in synthetic self-training. Empirical results show improvements on several relevant axes including utility retention, robustness under sampling attacks, and reduced unintended suppression of semantically similar tokens. These highlight that the authors care not only about benchmark metrics but also about subtle security implications.", "weaknesses": "There are notable limitations in the current form. First, the theoretical results are built on highly abstract distributions and reward convergence assumptions, and the paper does not convincingly bridge these assumptions with actual LLM behavior. Second, all experiments are on a synthetic dataset (TOFU), so claims of “privacy-preserving” remain hypothetical. Third, some baselines are insufficiently understood or tuned to ensure fairness: certain hyperparameter choices appear inconsistent with prior literature. Fourth, the collapse mechanism could potentially degrade knowledge in areas semantically close to the forget domain, yet the evaluation of utility is relatively narrow, relying mostly on ROUGE. Last, computational overhead implications are minimized rather than rigorously quantified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)