Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspective
Authors: Justin Lee
Zheda Mai
Jinsu Yoo
Chongyu Fan
Cheng Zhang
Wei-Lun Chao
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Unlearning
Diffusion Model
Image Generation
Machine Unlearning
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We present the first systematic study of continual unlearning for image generation, reflecting real-world scenarios where unlearning requests arrive sequentially rather than all at once.

Abstract: Machine unlearning—the ability to remove designated concepts from a pre-trained
model—has advanced rapidly, particularly for text-to-image diffusion models.
However, existing methods typically assume that unlearning requests arrive all
at once, whereas in practice they often arrive sequentially. We present the first
systematic study of continual unlearning in text-to-image diffusion models and
show that popular unlearning methods suffer from rapid utility collapse: after only
a few requests, models forget retained knowledge and generate degraded images.
We trace this failure to cumulative parameter drift from the pre-training weights
and argue that regularization is crucial to addressing it. To this end, we study a
suite of add-on regularizers that (1) mitigate drift and (2) remain compatible with
existing unlearning methods. Beyond generic regularizers, we show that semantic
awareness is essential for preserving concepts close to the unlearning target, and
propose a gradient-projection method that constrains parameter drift orthogonal
to their subspace. This substantially improves continual unlearning performance
and is complementary to other regularizers for further gains. Taken together, our
study establishes continual unlearning as a fundamental challenge in text-to-image
generation and provides insights, baselines, and open directions for advancing safe
and accountable generative AI.

Published as a conference paper at ICLR 2026
CONTINUAL UNLEARNING FOR TEXT-TO-IMAGE DIF- FUSION MODELS: A REGULARIZATION PERSPECTIVE
Justin Lee
Zheda Mail
Jinsu Yoo'
Chongyu Fan?
Cheng Zhang
Wei-Lun Chao 1,1
The Ohio State University
Michigan State University
Texas A&M University
*Boston University
ABSTRACT
Machine unlearning-the ability to remove designated concepts from pre-trained model-has advanced rapidly, particularly for text-to-image diffusion models. However, existing methods typically assume that unlearning requests arrive all at once, whereas practice they often arrive sequentially. We present the first systematic study of continual unlearning in text-to-image diffusion models and show that popular unlearning methods suffer from rapid utility collapse: after only few requests, models forget retained knowledge and generate degraded images. We trace this failure to cumulative parameter drift from the pre-training weights and argue that regularization is crucial addressing it. To this end, we study suite of add-on regularizers that (1) mitigate drift and (2) remain compatible with existing unlearning methods. Beyond generic regularizers, we show that semantic awareness is essential for preserving concepts close to the unlearning target, and propose gradient-projectinn
method that constrains parameter drift orthogonal to their subspace. This substantially improves continual unlearning performance and complementary to other regularizers for further gains. Taken together, our study establishes continual unlearning fundamental challenge in text-to-image generation and provides insights, baselines, and open directions for advancing safc and accountable generative AI.
Continual Unlearning of "Styles"
Continual Unlearning of "Objects"
Unleam Request
Unleam Request
Pre-irained
Ansenciioiiu
Sypactiee
Vibrent Tiox
Pre-minns
Dieace
Promut
Prompt
"Mrexttecture inage
factiander
In dargeri
styla."
stage stall."
"irved lmage
stale."
"teer
kistion Seates
Helt-out
Model forgels refained Anowledge!
Held-out
Model fargets retained knowledge!
Yeamse
Thesas
eeprecclaniin
sopresciiatio
Figure 1: Continual unlearning leads to catastrophic degradation. The pre-trained model (first column) continually unlearns 12 concepts C* = {ci, og...., .iz) (e.g., Abstractionism or Bears). Different rows display various prompts used for image generation. In each row, red boxes highlight images where some specific concepts have been unlearned. Ideally, images without red boxes should remain conceptually intact. However. illustrated in the bottom row, continual unlearning significantly impairs the model's ability to retain concepts. Notably, after unlearning 12 concepts (last column), the model fails generate mcaningful content.
Equal Contribution. Project Page: https::/ /aaas://jusniiiyee nhy 35. thub. Baroi__Projee__PP___

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent advances in text-to-image generation, driven primarily by diffusion models (DMs), have achieved unprecedented success in producing high-quality images across diverse concepts (Rombach) et al.) 2022 Kawar et al.] 2023 Zhang et al.] [024a] Nichol et al.] 2021). This versatility stems from training on massive, internet-scale datasets, but such broad data collection introduces serious ethical and legal risks: models may reproduce copyrighted material, generate harmful or biased content, and perpetuate stereotypes (Schramowski et al. 2023 Vinker et al.. 2023). In response, regulations such as CCPA (California Attorne General) now grant individuals the right to request removal of their personal or copyrighted content. However, retraining large DMs from scratch for every request is computationally infeasible for example, retraining Stable Diffusion v2 on LAION-5B (Schuhmann et al., 2022) requires roughly 150,000 GPU-hours (Gandikota et al. (203a)). As result, machine unlearning has emerged as practical alternative, aiming to selectively erase undesired generative capabilities (e.g. a person's 's likeness or an artistic style) from pre-trained models without full retraining (Hong et al. 2024 Gandikota et al. 2023a Kumari et al. 2023).
Despite notable progress in unlearning for DMs, most methods assume that unlcarning requests arrive simultancously (Wu et al. 2025b; Gandikota al. 2023a; Kumari et al. 2023 Wu et al., 2025a). In reality, such requests are typically sequential for example, a parent may request the removal of violent concepts one day, followed later by an artist seeking the exclusion of copyrighted artworks
To reflect this real-world setting, we introduce Continual Unlearning (CU) for text-to-image genera- tion, defined as the sequential removal of targeted generative capabilities subject three requirements: (i) effective erasure of newly targeted concepts, (ii) preservation of prior unlearning, and (iii) retention of all unrelated generative abilities (Figure 2). While CU has recently been studied in large language models (LLMs) (Chen & Yang 2023]
J02g Jang et al.) 2022). it remains largely unexplored in image generation. We fill this gap with the first comprehensive empirical study of CU for text-to-image diffusion models, and introduce benchmark that extends UNLEARNCANVAS (Zhang et al. (2024b) with style- and object-level unlearning sequences (Figure TJ. We outline major insights as follows.
Continual unlearning suffers rapid utility collapse. Popular unlearning methods (Kumari et al., or few concepts simultaneously, break down in the
2023), while effective for removing continual setting. After only a handful of requests, models forget retained knowledge and produce degraded images
for unrelated concepts. Our analysis attributes this failure to cumulative parameter drift, as successive unlearning steps push the model farther from its pre-training manifold. Consistently, we observe much larger parameter shifts in continually unlearned models than in those where all target concepts are unlearned simultaneously or independently.
Generic add-on regularizers partially alleviate collapse. Motivated by the above, we explore regularizers that can be seamlessly integrated into existing unlearning methods to mitigate drift. These include (i) constraining the update norm relative to previously unlearned models, (ii) selec- tively updating parameters most critical for the target concepts, and (iii) merging independently unlearned models. These approaches reduce drift and improve preservation of concepts across domains (e.g., generating objects after unlearning styles).
Semantic awareness is crucial for in-domain retention. Retaining in-domain capabilities (e.g., unlearning one style while preserving others) remains highly challenging, often leading to sharp utility drops even with regularizers. Empirically, we find a strong negative correlation between retention performance and the text-embedding similarity of the retention concept to the unlearning concept (see Figure 7ay. underscoring the need for semantic awareness.
Gradient projection provides a principled solution. We propose that imposes hard constraint on parameter updates, forcing them to be orthogonal to the subspace spanned by semantically close concepts. This minimizes unintended interference, substantially improves in-domain retention, and remains complementary to other regularizers for further gains.
prrjjetttoooectiioo method
Remark, Rather than proposing a new continual unlearning algorithm, we focus on developing compatible solutions that enhance existing methods-an an approach we believe will have a broader impact. Interestingly, the regularizers we study are also effective for unlearning single concepts, the standard setting in the unlearning literature. Their benefits, however, are most pronounced in the continual scenario, particularly as the sequence length grows. Overall, our study provides robust reference points for advancing continual unlearning, underscoring its challenges, opportunities, and promising directions for future work.
2

---

Published as a conference paper at ICLR 2026
'A is Yan- Gogl
CA is cariien stile
cast in Gogh stle"
cat style"
"Acst Cooh soge
"A car style"
Yeat Censitir
condition
Tout Coeditann
Unlearaing "Vau Gogl" Stste
Unicersing "Cartosn" Style
S
Unlearn Requests
The ideal outcomes in continual unlearning. The pre-trained model contimually unlearns two styles. Given the prompts to generate a cat "Van Gogh" and "Cartoon" styles, the generated images should accurately reflect the styles. After the first unlearning step, the image for "Cartoon" should remain conceptually unchanged, while the image for "Van Gogh" should no longer exhibit the "Van Gogh" style. Following the second unlearning step, both the "Van Gogh" and "Cartoon" should be removed, while the concept "cat" should be retained.
Figure 2:
2 RELATED WORK
Machine Unlearning in Diffusion Models. Diffusion models revolutionized image generation by training on internet-scale data (Schuhmann et al. 2022 Wang et al. 2025). However, reliance on such data introduces risks of harmful outputs, copyright violations, and biases (Schramowski et al., 2023 Zhang et al.) 2026). Unlearning aims to remove undesirable generative capabilities without retraining from scratch (Hong et al.)
2024 Gandikota al., 2023a Wu al., 2025aj. Widely adopted methods like ConAbl (Kumari et al.
2023) map unlearning concepts to benign anchors, while recently
proposed SculpMem (Li et al.j | (025) improves ConAbl with a dynamic mask. Nevertheless, most methods still assume unlearning requests arrive simultaneouslyy overlooking realistic scenarios where requests arrive sequentially. Our study addresses the unexplored question: Are unlearning methods still effective in continual settings, and how they be adapted to unlearn continually?
Continual Unlearning. Continual Unlearning (CU) is an emerging direction where removal requests arrive sequentially rather than all at once. CU was first studied in LLMs, aiming unlearn user- sensitive knowledge or undesired capabilities while preserving general language ability (Chen &
Yang 2023 Gao al., 2024 Jang et al. 2022). In contrast, CU for image generation remains largely unexplored. We address this gap with a systematic study of CU for text-to-image diffusion models, diagnosing utility collapse and proposing mitigation strategies.
2026) are closely related: both update existing models while striving to preserve acquired capabilities. Unlike continual learning, where the model aims to learn new concepts, both the concepts to be removed and retained are already known by the model in continual unlearning, amplifying interference risks. Despite this fundamental difference, principles from continual learning remain highly relevant (Heng & Soh 2023). Motivated by weight and gradient- based regularization and selective fine-tuning (Zenke et al., 2017 Mazunder et al. [021 Lopez-Pan & Ranzato 2017), we investigate whether these mechanisms can be repurposed to enable effective unlearning without utility collapse. By bridging insights from continual learning to continual unlearning, we set the stage for future investigations.
Continual Learning. Continual unlearning and continual learning Mai et al. 2022
Detailed Related Work. Due to the page limit, we include detailed related work Appendix F
PRELIMINARY
3.1 MACHINE UNLEARNING FOR TEXT-TO-IMAGE DIFFUSION MODELS
Diffusion models (DMs) generate images by progressively denoising an initial Gaussian sample. At each step f, a neural network Ept estimates the noise component in the current state Its producing a cleaner state æ;-1- Iterating this process yields I Vo, the final image. For text-to-image generation, a text prompt is additionally input to cgr to guide the denoising trajectory, i.e., cgr (x,, q, t). We denote the full generation process by Got, with output image I Go (q).
3

---

Published as a conference paper at ICLR 2026
Ideally, if a prompt g contains a concept (e.g., art style object), the generated image / = G should accurately reflect it. This can evaluated with a recognition model F, such as CLIP Radforrd et al. 2021), by checking whether the predicted label é FI) satisfies è =
Unlearning aims to update the pre-trained model weights 0t so as to remove the generative ability for designated target concepts. Let c* denote target concept, and let 0 denote the model parameters after unlearning c². For any prompt containing c", the generated image = Gg. (4) should satisfy F(I) + c² For all other concepts e A c*, the model should retain them; that is, if appears in the prompt q. then we should have F(Go. (g)) c.
3.2 PAPER STRUCTURE
The goal of this paper is introduce, analyze, and improve continual unlearning (CU). We structure the remainder as follows: section defines the CU setting and presents our benchmark; Section evaluates baseline CU approaches, identifies their limitations, and investigates the root cause of failure:|section sttudies generic regularizers as a remedy, whilessection dedemonstrates the importance of semantic-aware regularizers for preserving in-domain generative capabilities. Finally, (section provides further analysis of the unlearning dynamics, offering insights for future CU methods.
4 CONTINUAL UNLEARNING: SETUP AND BENCHMARK
4.1 SETUP
Motivation. In practice, model may be asked to erase multiple concepts C+ ( c.). If all requests arrive at once, one update Ot to jointly unlearn all e* E C*. In reality, however, requests typically arrive sequentially, calling for continual unlearning (CU) methods that remove each concept as is received.
Definition. Without loss of generality, assume requests arrive in order c...., CN- Let 0% denote the model obtained after unlearning the first n concepts. For any concept e appearing in prompt q, the model should satisfy:
{ (c, if E tci,...,ch { -c,, otherwise.
(unlearned)
F(Go; (4)) (o)=
(retained)
Metrics. Following UNLEARNCANVAS we evaluate CU after the n-th request with two metries:
Unlearning Accuracy (UA). For each unlearned concept {c....., 3 we count success when F does not return for an image generated from a prompt containing it, i.e., F(Ge; (4)) + c. UA is the fraction of successes across generated images.
Retention Accuracy (RA). For each retained concept c & {ci..... ) we count success when F correctly returns for image generated from prompt containing it, i.e., F(Go; (0))
To better analyze retention, partition concepts into subsets: an in-domain set, containing those semantically or structurally related to the unlearned concepts, and cross-domain set for the rest, For example. if the unlearning targets image styles (e.g., "Cartoon")) then other styles (e.g.. "Van Gogh") are in-domain, while objects (e.g., "Cat") are cross-domain. Accordingly, we report In-Domain Retention Accuracy (RA-I) and Cross-Domain Retention Accuracy (RA-C).
4.2 BENCHMARK
Data and Model Source. Prior works on concept unlearning have lacked standardized evaluation protocols, relying on heterogeneous metries such CLIP Score similarity (Wu al. 2025b] Kumari
al. 2023 Gandikota et al. 2023a) or subjective human evaluation (Gandikota et al. 2023a thereby
hindering fair comparison.
To address this, we adopt UNLEARNCANWAS (Zhang et al. 2024b) our evaluation backbone. provides a fine-tuned Stable Diffusion (Rombach et al. 2022) checkpoint Gøt and specialized classifiers F trained to recognize 60 artistic styles and 20 object categories. The DM checkpoint ensures that all 80 concepts can be generated with high accuracy (>98% top-1), while the classifiers offer standardized and objective means of reporting UA RA-I, and RA-C.
4

---

Published as a conference paper at ICLR 2026
style Untearning 188)
Style Retertion (MA-1)
Dbject Retention (BA-4)
100
Accurees (M)
(a) Unleurning style concept.
16
Untearn Request
Infearn Request
Untearn Request
Object Unluerning (96)
Object Retention (MA-1)
Style Retention (BA-C)
(b) Unleariing object concept.
Accuracy 12
Accuracy (N)
73
Accuracy -2
12
Simollaresss Soqunttial
Untearn Request
Untearn Request
Uniearn Request
Figure 3: ConAbl (Kumari al. 2023) fails when unlearn requests arrive continually. Although it performs well
the initial request, unlearning sequentially leads poor retention. Simultaneously unlearming all requests better preserves retention, but comes with a much higher cost. Plots for SculpMem (Li et al. 2025 in Appendix A]
Evaluation Protocol. To systematically evaluate CU performance, we consider two settings for constructing the unlearning targets C* =ct, .2....,
Continual Style Unlearning. We sample a random sequence of 12 unique artistic styles to be unlearned. To evaluate retention, we hold out 12 additional styles and objects that are never targeted during unlcarning. This allows us to measure both in-domain retention (other styles) and cross-domain retention (objects).
Continual Object Unlearning. Symmetrically, we sample a random sequence of 12 unique objects to be unlearned. The same held-out evaluation set from the style setting is used, ensuring fair comparison across settings without biases from different evaluation sets.
After each unlearning request, we follow UNLEARNCANVAS (Zhang et al. 2024b) to generate diverse images for both unlearned and retained concepts, using the template "A [object) image in (style) style." " For example, after erasing the "Van Gogh" style, we generate 5 images (different random seeds) for each of the 8 held-out objects conditioned on this style to compute UA. In total, this yields 40 images per style concept (5 x 8). Analogously, when unlearning object concept, we generate 5 images for each of the 12 held-out styles, yielding 60 images per object concept (5 x 12).
5 CONTINUAL UNLEARNING SUFFERS RAPID UTILITY COLLAPSE
5.1 EXISTING METHODS FAIL TO UNLEARN CONTINUALLY
Unlearning Methods. We first examine how existing methods behave in a continual setting, focusing on two representative ones: the widely adopted Concept Ablation (ConAbl) (Kumari et al. 2023) and the recently proposed SculpMem (Li et al., (222)). Like many unlearning methods, both define an unlearning loss Lunteam (0, C) that depends on the model parameters and the target concept set Minimizing this loss with initialization 0 (the pre-trained weights) yields an unlearned model 0-.
Extension to CU. We adapt these methods continual unlearning using two strategies:
Sequential: At the n-th request, the model is incrementally updated by minimizing Cunteam (0, {ct)), starting from the previously unlearned model 0;
Simultaneous: At the n-th request, the model is retrained from the pre-trained weights 0t to jointly unlearn all target concepts so far, i.e., minimizing Luateum (8, (ci....,c+,)).
Results. Both ConAbl and SculpMem perform well for single-concept unlearning, achieving high UA. RA-I, and RA-C the first request (Figure 3). However, as additional concepts are unlearned sequentially, their utility collapses: while UA remains high, the models rapidly lose the ability to generate unrelated concepts, leading drastic drops in RA-I and RA-C (Figure 3 Figure 1).
By contrast, the simultaneous strategy preserves utility more effectively, but at a prohibitive cost: each new request requires re re-unleamnnn all prior concepts from scratch, making training time grow with the total number of requests (Appendix E}. This efficiency-utilit trade-off underscores the need for continual unlearning methods that can handle sequential requests without collapsing retention.
5

---

Published as a conference paper at ICLR 2026
Independent
Simultaneous
Sequential
L1 Reg.
L2 Rog.
Model Merge
Selective FT
Unleaam Request
8
5
UBN
learmable parameters
Figure 4: ConAbl's (Kumari et al. 2023) cumulative l2 parameter drift w.r.t the pre-trained model. Sequential unlearning exhibits severe cumulative drift with more unlearned concepts compared simultancous unlearning. Our add-on regularizers effectively mitigate this drift and demonstrate better retention (Figure 6).
5.2 WHY DOES SEQUENTIAL. UNLEARNING FATL.?
Empirical Observations. The above results raise important question. Existing methods can unlearn multiple concepts with high retention when applied simultaneously, yet their effectiveness collapses when applied sequentially. To understand this discrepancy, analyze the unlearned models 0 after the n-th request under both strategies, focusing on their deviation from the pre- trained weights 0t, As shown in Figure 4, after the first request, both strategies exhibit similar degree of parameter drift, measured by 10; 0 2|. With more requests, however, drift grows dramatically under sequential unlearning while remaining nearly constant under the simultaneous strategy. For comparison, we also unlearn each concept independently from OT, The norms of these parameter shifts remain similar to those from simultaneous unlearning, despite the latter involving progressively more concepts.
These findings suggest the following hypothesis: High retention in sequential continual unlearning requires regularizing parameter drift.
Theoretical Analysis. We seek to provide a theoretical perspective on the empirical findings. Intuitively, the pre-trained weights 0f encode the model's original generative capabilities. Therefore, keeping the unlearned model l close Ot should help preserve these capabilities.
Building on the loss approximation framework from continual learning (Yin et al., 2020 Zenke et al. 2017: Aljundi et al. 2018), we formalize this intuition using a Taylor expansion of the retention loss L around OT (tull derivation in Appendix B). This yields the following bound the change in L:
|L(0*, (T) ((00,,c)) < [| ,")| 110 - 0 + + (Ot, c())ll 110 - 01|]
where is the pre-trained model, l the unlearned model, C the retention set, and H the Hessian.
This inequality shows that the change in retention loss is Lipschitz continuous w.r. the parameter update, meaning the loss grows proportionally (up to a constant) to ||0 - I]. Hence, preserving utility depends directly how close the unlearned model remains to the pre-trdined parameters.
Moreover, when the gradient and Hessian terms are small-typically the case near the optimum the update norm becomes the dominant factor. To validate this, we estimate the curvature of the retention loss by perturbing the pre-trained weights and measuring the ratio of gradient change (evaluated on the UNLEARNCANVAS training set) weight perturbation. The estimated Hessian coefficients are minuscule, confirming that the retention loss lies in smooth basin Appendix B).
6 ADD-ON REGULARIZATION FOR SEQUENTIAL CONTINUAL UNLEARNING
Motivated by our empirical and theoretical analyses, we explore add-on regularization strategies that constrain parameter drift to improve retention. These approaches differ in how they measure drift (e.g., using different norms) and how they impose the constraint (overview in Figure 5).
6.1 UPDATE NORM REGULARIZATION
We begin with the most common approach: directly penalizing the norm of the parameter update. At the n-th request, we augment the unlearning loss with regularization term:
Lunleam 0, (ct + O 11-

---

Published as a conference paper at ICLR 2026
"Blrde"
"Carz
@+
8i
Gamgl
8/
0T
tingl
8i
Gongl
B2
Gap
8
grad
grod
fordd
Loss Ust. et)
Loss
L8j,, 80
Loss 85)
Imporenne
Importenee
Importanon
(a) Update Norm (L1/L2)
(b) Selective Fine-tuning
#1
piune
Next unleam request
Lefoerr "Cass"
uniibarn
somiiinc
wnights
B5
@1
Gargl
0
pluno
weighs
8
@t
0
PTUTA
(c) Model Merging
Figure 5: Overview of add on regularizers. (a) L1/L2 penalizes norm of parameter update relative the previous checkpoint. (b) Selective Fine- tine-tuning restricts updates to the top-&% most important parameters. (c) Model merging unlearns each concept independently and combines the resulting models.
where - the model obtained after the (n-18 1) request and serves as the initialization for A. We consider two choices of p: the L norm, which encourages sparse updates, and the L2 norm, which distributes the update across parameters, preventing any single weight from drifting excessively.
6.2 SELECTIVE FINE-TUNING (SELFT)
Inspired by work in continual learning [Wang et al. 2024), unlearning (Fan et al. 2023), and model pruning (Wang et al. 2020; Cheng et al. 2024), we investigate Selective Fine-tuning (SeIFT) as an alternative to norm- norm-based regularization. Unlike the L1 penalty, which encourages isotropic sparsity, SelFT explicitly restricts updates to parameters deemed critical for the unlearning loss.
While SelFT is often used umbrella term for gradient-masken in this paper, we adopt the method by Nguyen et al. (2024). At the n-th request, given 0 we compute parameter importance in single forward pass using first-order Taylor approximation:
saliency-based approaches,
Importance(c)
V Val4 funteam (0; {ct}) ( (-]/
We then select the top 4% most important parameters and update only those during unlearning. By explicitly limiting the number of tunable parameters, SclFT constrains drift while still allowing effective concept removal.
6.3 MODEL. MERGE
Since independently unlearned models for each concept remain close to the pre-trained weights Øf (Figure 4), we investigate model merging (Yang et al. 2024) to integrate their effects while staying near the original model. As all such models originate from the checkpoint, they likely to lie in the same loss basin (Frankle et al. 22020.. Interpolating within this basin keeps retention loss low, allowing merged models preserve utility while handling multiple unlearning requests.
Concretely, let Ô,, denote the n-th independently unlearned model. We adopt TIES-Merging Yadav al. 2023) to construct 0% by merging .,,...,0 O,, TIES first prunes each 0,, by retaining the top-k% parameter updates (ranked by absolute deviation from Qt). yielding pruned model 0", and then merges them by averaging. Importantly, merging imposes form of regularization: the merged model 0 lies in the affine hull of the pruned models, which restricts updates to the subspace spanned by (0 o'),..., OO' 0¹)}-
6.4 RESULTS AND INSIGHTT
As shown in Figure 4 add-on regularizers substantially reduce parameter drift during sequential unlearning, yielding clear improvements in both RA-I and RA-C (Figure 6). Among them, model merging delivers the strongest overall retention.
However, in-domain retention (RA-I) remains particularly challenging: across regularizers, RA-C consistently surpasses RA-I. This gap is expected, concepts within the domain
7

---

Published as a conference paper at ICLR 2026
Style Untearning (WA)
Style Retention (RA-1)
object Retention [M-CC
199
1067
193
Accurecy [N]
Accuracy 1N1
(a) Unleaming style concept.
12
Sequettial
8 8
11 Rog
Simaltarsoos
Rog
Selective
Perge
3
12
Proj
Unlearn Request
Unleare Request
Unlearn Request
object unlearning (WA)
Object Retention (BA-1)
Style Retestion (RA-C)
166
Accuraiy [N]
(b) Unlearning object concept.
Accurecy
Accuracy
Sequential
Sanultarcous
Rog
15
Rag
Selective
5
12
Unlearn Recusst
Un Unleare Sequest
Unlearn Request
Figure 6: Add-on regularizers substantially improve retention under sequential unlearning with ConAbl, yielding particularly strong gains cross-domain retention (RA-C). By removing gradient components that interfere with semantically related concepts, our gradient projection method further improves in-domain retention (RA-I). Results for SculpMem are provided in Appendix A
8.825
31
Style Roturtine Ncurecy
Iepresstatis
ginini
ponguin
lary
Sease
Cargon Lin
affect auxllary
lest Embedding Subspace Ausilaary Concapts
Stettarsty
Stetlarsty
(a) RA-L vs. Cos. Sim.
(b) k, a Update. vs. Cos. Sim.
(c) Gradient Projection schematic
Figure 7: (a) RA-I decreases as text-embedding cosine similarity to the unlearned concept (e.g.. "Abstrac- tionism" " style) increases (strong negative correlation).
(b) Change in k.v grows with text-embedding cosine similarity (strong positive correlation). (c) These trends motivate gradient- projjcction method, which removes gradient components that alter k, for semantically similar (auxiliary) concepts.
semantically closer (e.g., "Bear" vs. "Cat") than cross-domain pairs (e.g., "Bear" vs. "Van Gogh"), making them more prone to interference during unlearning.
7
GRADIENT PROJECTION FOR SRANTIC-AWARI CONTINUAL UNLEARNING
7.1 SEMANTIC AWARENESS IS CRUCIAL
To systematically quantify the link between semantic similarity and retention difficulty, we unlearn the style concept "Abstractionism" and measure the retention accuracy of other style concepts, alongside their cosine similarity to the text embedding of "Abstractionism." As shown in Figure 7a retention accuracy exhibits strong negative correlation with embedding similarity to the unlearned concept, underscoring the need for semantic awareness in continual unlearning. In particular, special care is required to preserve generative capabilities that are semantically close to the unlcarning target.
7.2 PROJECTION MATRICES IN
AS THE KEY SOURCE OF INTERFERENCE
To design semantic-aware strategy for unlearning in diffusion models, we first need to understand how semantically close concepts interfere with each other in the model. The standard way to inject prompt y into the denoising network egr (æt, q, I) (cf. subsection 3.1) is through cross-attention, where the current latent state I, serves as queries to attend to the token embeddings of the text prompt q. In this mechanism, the prompt tokens are projected into keys and values that o; retrieves from.
For simplicity, assume each concept e corresponds to a single token, with embedding E(c). Its key and value vectors then given by
k WK F()), Wv W.)F(()),
8

---

Published as a conference paper at ICLR 2026
where WK and Wy are learnable projection matrices. Unlearning target concept c* amounts to updating WK and Wy so that E(c") is mapped far away from its original (k, v), making inaccessible for æt to retrieve during generation.
However, because linear projections approximately preserve neighborhood structure, semantically similar concepts and et remain close after projection:
AE(c) AB(c*)Il & ||||| |IE(c) - B(c*)ll,
for any linear operator A. Consequently, updating Wx and Wv to suppress c* inevitably distorts the embeddings of nearby concepts e as well. Empirically, we observe that higher text-embedding similarity indeed correlates with greater distortion in (k, v) Figure 7b).
7.3 GRADIENT PROJECTION TO SUPPRESS INTERFERENCE
We propose a method to suppress the undesired influence updating WK and Wv on semantically similar concepts. After obtaining the unlearning gradients with respect to the projection matrices,
V Wx Lunteam (8,
V We Cunteam (0, {c*}),
we project out the components that perturb nearby concepts to first order
FFigure 7ch.
Concretely, let { denote M conceepst/ generated by LLM and filtered by text- embedding similarity to the target c", and let C
E(cx)) be their embeddings. The span S span(C) approximates the subspace of embedding directions corresponding to semantically similar concepts. We remove gradient components lying in S, ensuring that updates suppress e* while minimally distorting neighbors.
Details. Given a vector its Euclidean projection onto S is projs(g) = arg min l|g ull.. Ca for some RM, this reduces to least-squares
Since any 8 can be expressed problem with a closed-form solution û = Cå Psg. where Ps := C(C CCT Here, Ps the orthogonal projector onto the subspace spanned by C, and the complementary projector Ps. : Ps removes all components of y that lie i.
Let g* denote the gradient for unlearning the target concept c*. To suppress interference with semantically similar concepts, we project g" onto the orthogonal complement of span(C) (Figure 7c):
g' = Psag* = (1- Ps)g"
Here, g' the projected gradient, which preserves directions useful for unlearning e" while climinat- ing components aligned with nearby concepts. We note that g' Ps)g'.
Lemma 7.1 (First-order invariance). For any span (C), the update direction g' produces zero first-order change: g' 0.
Proof. If c E span(C). then Psc c. Hence o c= c -o-0=='''cc- c) 0.
Results. As shown in Figure our gradient-
Untearning "Style"
Untearning "Object"
Harnonic Mean (9)
189
projection method achieves the highest retention ac- curacy on in-domain concepts (RA-I) across both style and object unlearning settings. However, its re- tention accuracy on cross-domain concepts (RA-C) slightly lower than that of SelFT and model merging in the object unlearning case. To address this gap, examine whether combining isting add-on regularizers can improve RA-C while preserving RA-I. As shown in Figure gradient pro- jection is indeed compatible with these add-ons, and their combination yields further improvements.
02.6
a1.2
75
66.3
5a.
Sobeshoo
SeiiPPtt Selackke F7
L. Pag
Ocad
L1 Rag
* Pag
method with ex-
Figure 8: Gradient-projection is compatible with add-on regularizers, yielding additional gains when combining them. The y-axis shows the harmonic mean of UA, RA-I, and RA-C, using ConAbl for unlearning.
We define auxiliary concepts those that are semantically related to the target concept but should be retained during unlearning. Our method does require the retain

---

Published as a conference paper at ICLR 2026
8 UNDERSTANDING THE UNLEARNING PROCESS
To complement our previous analysis, we inves- tigate the mechanics of how unlearning modifies the model, revealing insights that may inform the design of future unlearning and regulariza- tion methods.
Bears olforees
Cate-ilturen
Rabhite elleons
Unlearning is About Learning. For anchor- based unlearning methods (Kumari et al. 2023), we find that the parameter updates are driven primarily by the chosen anchor rather than the target being unlearned. Distinct targets mapped to the same anchor induce highly correlated pa- rameter updates (Figure 9). This suggests that anchor-based unlearning functions represen- tation replacement: the anchor representation is relearned to overwrite the target. This exposes a tradeoff: mapping many targets to a shared anchor may localize parameter modifications, while mapping targets to distinct anchors may disperse updates more broadly across the model.
Frege-uutteelyy
Stattes -Irwo
Waterdalls- Ttow
Caste Horees
Drgge Herees
Progee shatt
Juth Flovore
Beaddver bostone Trree
Wecurtalee
Figure 9: Parameter updates during unlearning are gov- emed by the anchor concept rather than the specific target being removed. Models that share an anchor dis- play highly correlated updates.
Concept Erasure is All-or-Nothing. When interpolating between the pre- pre-trained and unlearned model, we find that for many concepts, generated outputs remain visually unchanged across wide range of interpolation coefficients until critical threshold is crossed, at which point the target concept abruptly suppressed (Appendix C). Erasure thus behaves as sharp transition rather than gradual process. This complicates model merging, interpolation coefficients must be carefully chosen to keep every concept above its respective erasure threshold.
Parameter Drift is Intrinsic to Sequential Unlearning. One might hypothesize that the greater parameter drift in sequential unlearning (Section (522) is simply an artifact using more optimization steps. To test this, we apply early stopping to both sequential and simultancous unlearning, terminating each when 99% unlearning accuracy is reached. We then continue unlearning additional concepts until the cumulative optimization steps of simultaneous unlearning match or exceed those of sequential unlearning. Even with comparable totals (e.g., 2,100 each after six concepts), sequential unlearning still accumulates substantially more drift (Appendix Dp. Understanding what property of simultaneous unlearning keeps drift low despite comparable optimization steps may guide the design of regularizers that bring the same benefit to sequential methods.
CONCLUSION
We present the first systematic study of continual unlearning for image generation, reflecting real- world scenarios where unlcarning requests arrive sequentially. We find that existing methods quickly degrade in furrity-forgetting retained concepts and generating low-quality images-and trace this failure to cumulative parameter drift and semantic interference. We show that simple, plug-and- play regularizers based on update norm, selective fine-tuning, model merging, and semantic-aware gradient-projection projection can substantially restore performance. For practitioners, our results suggest that combining selective fine-tuning with saadient-projectionn provides a strong starting point, effectively constraining both parameter drift and semantic interference to deliver robust retention across in- domain and cross dross-domain settings.
Future work. Understanding how robustness adversarial recovery attacks evolves across sequential unlearning steps is critical for safe deployment, particularly as it remains unclear whether these challenges compound differently across architectures (e.g., DiT), training objectives (e.g., flow matching), and modalities (e.g., video, speech) beyond diffusion--ased image generation. While our and-andd-lay regularizers provide strong foundation, designing natively sequential unlearning methods that anticipate future requests and account for their interactions is natural next step toward further advancing continual unlearning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research is supported by grants from the National Science Foundation (ICICLE: OAC-2112606). We are grateful for the support of the Ohio Supercomputer Center for providing computational resources.
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp. 339-154, 2018.
Anh Bui, Long Vuong, Khanh Doan, Trung Le, Paul Montague, Tamas Abraham, and Dinh Phung. Erasing undesirable concepts in diffusion models with adversarial preservation, arXiv preprint arXiv:2410.16618, 2024.
Anh Bui, Trang Vu, Long Vuong, Trung Le, Paul Montague, Tamas Abraham, Junae Kim, and Dinh Phung. Fan- tastic targets for concept erasure in diffusion models and where find them. arXiv preprint 28559 2025.
California Attorney General. California consumer privacy act (ccpa).
URL https://cag.cmt
privacy/ ccpa
Arslan Chaudhry, Mare' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint
2018.
Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning llms. arXiv preprint arXiv:2310.201500 2023.
Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696 2021.
MohammadReza Davari and Eugene Belilovsky. Model breadcrumbs: Scaling multi-task model merging with sparse masks. In European Conference on Computer Vision, pp. 270-287. Springer, 2024.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104.0m 08164, 2021.
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salum: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. arXiv preprint 1rX15:2310.12508, 2023.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pp. 3259-3269. PMLR, 2020.
Rohit Gandikota, Joanna Materzynska, Jaden Kiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2426 2436, 2023a.
Rohit Gandikota, Joanna Materzynska, Jaden Koutto-Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2426-2436. 2023b.
Chongyang Gao, Lixu Wang. Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. On large language model continual unlcarning. arXiv preprint arXiv:2407. 10223, 2024.
Alvin Heng and Harold Soh. Selective ammesia: A continual learning approach to forgetting in deep generative models. Advances in Newral Information Processing Systems, 7117777777799 2023.
Seunghoo Hong, Juhun Lee, and Simon S Woo. All but one: Surgical concept erasing with model preservation in text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 215 2024.
11

---

Published as a conference paper at ICLR 2026
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Cururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04008 2022.
Joel Jang. Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Sen. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504, 2022.
Khurram Javed and Martha White. Meta-learning representations for continual learning. Advances in neural information processing systems, 32, 2019.
Swanand Ravindra Kadhe, Farhan Ahmed, Dennis Wei, Nathalic Baracaldo, and Inkit Padhi. Split, unlcarn, merge: Leveraging data attributes for more effective unlearning in llms. arXiv preprint arXiv:2406. 17800 2024.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 6007-6017, 2023.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 14:35211-3526, 2017.
Nupur Kumari, Bingliang Zhang, Sheng- Yu Wang, Eli Shechtman, Richard Zhang, and Jun- Yan 7hu, Ablating concepts in text-to-image diffusion models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 22691-22702. 2023.
Kevin Kuo, Amrith Setiur, Kartik Srinivas, Aditi Raghunathan, and Virginia Smith. Exact unlearning of finetuming data model merging at scale. arXiv preprint wrxiv:2504.04622 2025.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340. 2018
Gen Li, Yang Xiao, Jie Kaiyuan Deng, Hui, Linke Guo, and Xiaolong Ma. Sculpting memory: Multi- concept forgetting in diffusion models via dynamic mask and concept-awan optimization, 2025. URL
https://arxivv org. abs/ /504. 09039
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Vincenzo Lomonaco, Lorenzo Pellegrini. Pau Rodriguez. Massimo Caccia, Qi She, Yu Chen, Quentin Jodelet, Ruiping Wang. Zheda Mai, David Vazquez, et al. Cvpr 2020 continual learning in computer vision competition: Approaches, results, current challenges and future directions. Artificial Intelligence, 303:103635, 2022.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings the IEFF/CVF conference computer vision and pattern recognition, pp. 3589-3599, 2021.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28 51, 2022.
Zheda Mai, Sooyoung Jeon, Zanming Huang, Jinsu Yoo, Colin Lee, Gengyu Zhang, Haoxuan Wang, Yifan Li, Yu Kong, Yan Yan. and Wei-Lun Chao. A survey of continual learning for robotics in the foundation model era. Jamuary 2026. doi: 10.36227/techrxvv.. 176977367.76667777779
URL lttt:://ssss org/0
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In Proceedings of the European conference on computer vision (ECCV), pp. 67-82, 2018.
Michael S Matena and Colin A Raffel Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703- 17716, 2022.
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 2337-2345, 2021.
12

---

Published as a conference paper at ICLR 2026
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference, arXiv preprint arXiv: 06440, 2016.
Quang H Nguyen, Hoang Phan, and Khoa D Doan. Unveiling concept attribution in diffusion models. arXiv preprint 2221::241..02542, 2024.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models. arXiv preprint 17Xi:2112.10741, 2021.
Guillermo Ortiz-limenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36:66727- 66754, 2023.
Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from Ilms? objectives for defending against extraction attacks. arXiv preprint arXiv:2309. 17.330999774,, 17410, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent difffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 22522-22531, 2023.
Christoph Schuhmann, Romain Beaumont, Richard Veneu, Cade Gordon, Ross Wightmam, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35: 15278-25299 2022.
Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang. Online class- incremental continual learning with adversarial shapley value. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 9630-9638, 2021.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances neural information processing systems, 30, 2017.
George Stoica, Daniel Bolya, Jakoh Bjorner, Pratik Ramesh, Taylor Heam, and Judy Hoffman. Zipit! merging models from different tasks without training. arXiv preprint arXiv:2305.0330 2023.
Hidenori Tanaka, Daniel Kunin, Daniel Yamins, and Surya Ganguli. Pruning neural networks without any data by iteratively conserving synaptic flow, Advances in neural information processing systems, 33:6377-6388 6389, 2020.
Yacl Vinker; Andrey Voynow, Daniel Cohen-Or, and Ariel Shamir. Concept decomposition for visual exploration and inspiration. ACM Transactions Graphics (TOG), 42:1-13, 2023.
Chaoqi Wang, Guodong Zhang, and Roger Grosse. Picking winning tickets before training by preserving gradient flow, arXiv preprint arXiv:2002.07376, 2020.
Janet Wang, Yunbei Zhang, Zhengming Ding, and Jihun Hamm, Doctor approved: Generating medically accurate skin disease images through ai-expert feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Mitchell Wortsman, Gabriel Ilharco, Samir Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari Morcos. Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference PMLR, 2022.
on machine learning, pp.
Jing Wu, Trung Ie,, Munawar Hayat, and Mehrtash Harandi. Frasing undesirable influence in diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 29263-28273, 2025a.
13

---

Published as a conference paper at ICLR 2026
Yongliang Wu, Shiji Zhou, Mingzhuo Yang, Lianzhe Wang, Heng Chang, Wenbo Zhu, Xinting Hu, Xiao Zhou, and Xu Yang. Unlcaming concepts in diffusion model via concept domain correction and concept preserving gradient. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1996-8504, 2025b.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093-7115, 2023.
Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang. Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in Ilms, milms, and beyond: Methods, theories, applications and opportunities. arXiv preprint 2024.
Dong Yin, Mchrdad Farajtabar, Ang Li, Nir Levine, and Alex Mott. Optimization and generalization of regularization-bat based continual learning: loss approximation viewpoint. arXiv preprint arXiv:2006. 10974, 2020.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models free lunch. In Forty-first International Conference Machine Learning, 2024.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning, pp. 3987-3995. PMLR, 2017.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong. Xinying Guo, Lci Yang, and Ziwei Liu. Motion diffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence, 46:41 15-4128, 2024a.
Yihua Zhang, Chongyu Fan, Yimeng Zhang, Yuguang Yao, Jinghan Jia, Jiancheng Liu, Gaoyuan Zhang, Gaowen Liu, Ramana Rao Kompella, Xiaoming Liu, ct al. Unleamcanvas: Stylized image dataset for enhanced machine unlearning evaluation in diffusion models. In The Thirty-eight Conference Neural Information Processing Systems Dutaxets and Benchmarks Track, 2024b.
Yunhei Zhang, Kai Mei, Ming Liu, Janet Wang, Dimitris N. Metaxas, Xiao Wang. Jihun Hamm, and Yingqiang Ge. Agents in the wild: Safety, society, and the illusion of sociality moltbook. arXiv preprint arXiv:2602. 13284. 2026.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Disclosure of LLM Usage. Portions of this manuscript were polished for clarity and readability using an LLM. The LLM was not used to generate research ideas, design experiments, analyze data, or draw conclusions. All scientific content, methods, and results are the authors' original work.
Appendix Structure. This appendix is organized as follows. AApendix A] provides extended experimental results showing generalized findings across different unlearning methods and erasure settings. [pppendix B] includes extended theoretical support and additional empirical evidence clarifying the importance of constraining parameter drift. Additional analysis regarding the unlearning process can be found Appendix C] Appendix DJ and Appeendix E. Finally, detailed related work can be found in Appendix
A EXTENDED EXPERIMENTAL RESULTS
A.1 ADDITIONAL. RESULTS ON SCULPTING MEMORY
We further validate findings Sculpting Memory (SculpMem) (Li ct al. 2025). recent unlearning method designed for multi-concept unlearning. Despite using a dynamic gradient mask and achieving much stronger baseline performance, the model still experiences utility collapse after 12 concepts (Figure 10). Applying our proposed add-on regularizers yields improvements consistent with our previous benchmarks: all methods enhance retention performance over the baseline, with semantic-aware
delivering the strongest results Figure
A.2 ADDITIONAL ERASURE DOMAIN: CELEBRITY
To demonstrate that our findings generalize be- yond style and object erasure, we present results for celebrity (identity-based) erasure. For the experimental setup. we select a sequence of 6
100
Harmonic Mean (%)
89.1
80
73.5
60
random celebrities unlearn and an additional
6 for the held-out retention set. We employ the GIPHY celebrity classifier to measure un- learning accuracy (classifier error on unlearned celebrities) and retention accuracy (classifier ac- curacy held-out celebrities). To further eval- uate general retention performance, we generate 5,000 images using MS-COCO (Lin al.) 2014) prompts and report both FID and CLIP Score.
40
20
SelFT
Grad Proj
SeLFT Grad Proj
Figure 14: Cradient--rojectioon is compatible with add- on regularizers, yielding additional gains when combin- ing them. The y-axis shows the harmonic mean of UA, RA-I, and RA-C. using ConAbl for celebrity unlearning.
Consistent with our previous findings, ConAbl alone experiences severe utility degradation af- ter just 6 celebrities, as shown Figure 12 Fur- thermore, all proposed add-on regularizers improve retention capabilities without sacrificing unlearn-
Style Unlearning
style Retention (86-1)
object Setention (MA-
106
Accuracy (1)
Unlearning
Accuracy
Accuracy
style concept.
Stratterwozs Soaacetial
Unlearn Request
Unlearn Request
Unlearh Request
Object Unlearsing (UA)
Object Retention (RA-I)
Style Retention (MA-C
X
Accuracy 1S)
(b) Unleaming object concept.
Accuracy S
Accuracc
Sinaltaneou
Soquretial
UnTeam Resuest
Unlearn Request
Untearn Request
Figure 10: Performance of SculpMem Li
2025
under continual unlcarning.
15

---

Published as a conference paper at ICLR 2026
Style intearning (UA]
style Retention (R--1)
Object Retention [M-C
Accuracy (N)
Accuraty (1)
(a) Unlearning style concept.
Arcurary
8
Stmuttansous
8
LE Aag
Unlearn Roquest
Request object Retention
Request Style Retention (M-C)
object Unlearning
1901
68-
Accuracy (h)
Accuracy [S]
$ $
(b) Unlearning object concept.
10-
Sequnttial
TEPS
orad Pro)
Uslears Request
In Inlwam Request
Enlearn Sequest
Figure 11: Continual unlearning with SculpMem using add-on regularizers.
Sesuential
Sanuuca
Aug UA
Aug RM
CoCo-FID
CoCo-CLIP
recumnc
20p
Rested depravee Feir
dras attree
est disrrt
Wttter
eefi int.aepeeee Tree
qpsa ancive
drak aester Mi1
net oppreee
Ask
eus deprovae
Figure 12: Performance of ConAbl Kumari et al.
2023 under celebrity continual unlearning.
ing accuracy (Figure 13). The greatest performance gains come from combining our semantic-aware pradient-projection method with SelFT (Figure 14), demonstrating that add-on regularizers can be effectively combined for enhanced performance across different erasure settings.
A.3 ADDITIONAL ARCHITECTURE: SDXL
Next, to demonstrate that our findings generalize to different architectures, we present results for SDXL using ESD (Gandikota al. 2023b) for celebrity erasure. We adopt ESD rather than ConAbl or SculpMem because, to our knowledge, these methods do not provide official SDXL implementations. We continue with celebrity erasure instead of UnlearnCanvas (Zhang al., 2024b), as the latter requires an SDXL checkpoint fine-tuned on its benchmark styles and objects, which is not publicly available.
As shown in Figure 15 ESD also experiences utility collapse in the celebrity erasure domain.
As shown in Figure 16 gradient-projecttn outperforms L1, L2, and SelFT but surprisingly outperformed by Model Merge. However, Figure 17 demonstrates that gradient projection can be combined with model merge for further performance gains.
Sequential
Struttaneour
L3 Reg
L2 Reg
Selective FT
Model Merge --o Grad Proj
CoCo-FID
CoCo-CLIP
Best horrese Tyee
Asst dooer
Sprn Mtorre
Jegrnsse. Frie
exzz desers
aae Batreey
Bust daprovst Pyoe
moscce deca ayee abutfer
Aell Jeyrese eesrts jel
Asdt aboer
Bytz 20
Figure 13: Celebrity continual unlearning with the ConAbl algorithm using add-on mechanisms.
16

---

Published as a conference paper at ICLR 2026
Sequential
Sanulcancous
Coto-FID
CoCa- CLIP
200
Recureer
x
depeitie Faee
Aet awaryy
drat extrev
entes seyee
dertca Aet tre
eeti aepoee Tiee
eus pegiasie mecr drl tare
net sppratin
Jet
Figure 15: Performance of ESD Gandikota al.
2023b under celebrity continual unlearning with SDXL.
Scquential
Struttancous
RC0
Reg
Selective
Model Merge
Proj
Avg UA
Avg BA
CoCo-PID
Coto-CLIP
1844
Acurs
340
3
x
lil pegrasee Joar
Mestcoo Aed fer
Aaci pharr
denz Mare
augrasee Tree
destce del trr
Assa Awart
arft thprasse paee
Addi Awser
spiak stiere
Lapresse
depes
Baniy
Figure 16: Celebrity continual unlearning with the ESD algorithm using add-on mechanisms on SDXL.
B EXTENDED THEORETICAL SUPPORT
B.1 FULL DERIVATION
Let 0* denote the model obtained from 0 after unlearning concept (style or object), regard- less of the unlearning method (e.g., ConAbl, SculpMem, ESD), strategy (sequential or simul- taneous), and add-on regularizers (e.g., Gradient Projection, SelFT).
100
87.0
82.6
80-
Harmonic Mean
60
50.8
40
Let be the set of all concepts learned by the diffusion model 01, Let CJ C C be the set of
20
concepts to be unlearned, and let C" be the remaining concepts to be retained.
Cf
Model Merge
Grad Proj Model Merge GGrad Proj
Let (0; (*) denote the retention loss for pre- serving concepts in C", instantiated in practice as the standard diffusion training objective.
Figure 17: Gradient-projectionn is compatible with add- on regularizers, yielding additional gains when combin- ing them. The y-axis shows the harmonic mean of UA, RA-I, and RA-C. using ESD (Gandikota et al. 2023a) with SDXL for celebrity unlearning.
We begin by applying second-order Taylor expansion of the retention loss around A* to ap- proximate its value for any unlearned model 0*
0t) 10 - ( - ot)
1 - ot)
The linear term, given by the inner product of the gradient and the parameter change, can be bounded via the Cauchy-Sehie Schwarz incquality:
(Vt" C"), (0' o < VL (ff:")) [100 B*)l
The quadratic term can similarly be bounded through the repeated application of the Cauchy- Schwarz inequality, followed by the definition of the operator norm of the Hessian:
17

---

Published as a conference paper at ICLR 2026
1 (0" -B1 (0)T((0')BB -t) -0)1
1 -21((00 - 8t), H((0)) -ot))
2
1
11(000 01), ((0)) l - 2
21100 111 (1H))) 0T)1
2 21100 (H - ||
1
2 1 - 0011
By substituting the bounds the linear and quadratic terms, obtain overall bound the change in retention loss between 0* and f
[H) |10* 1/11
[VL"(() - -1 -1(0 a
This bound resembles a Lippscitzz-type continuity condition:
M -, s L. |1 0
2 -10*
0*1|
Thus, the change in retention loss grows proportionally with the parameter difference, with constants L and M bounding the contributions of the linear and quadratic terms, respectively.
The constants and M are expected to be small when the pre-trained model near stationary point of the retention objective, and the local curvature of L is low. In the next section, we provide an empirical approximation M.
B.2 EMPIRICAL SUPPORT
We empirically approximate the local gradient-Lipschit constant of the retention objective via finite-difference gradient variation on held-out retention concepts. For each reference model (the base model and independently unlearned checkpoints), sample perturbations
8 = =<100|2u,
where u is a random unit vector in the analyzed UNet subspace, and evaluate
M() = | (0 +- 6; (I'(O,C")ll
|
To reduce estimator noise, both gradients each pair are computed on the same minibatch, diffusion noise realization, and timestep draw.
Across a logarithmic sweep perturbation scales, we observe a consistent monotonic trend: smallest-pertee pertuubatpt regime, (c) remains uniformly small across models, while larger perturba- tions yield larger (c) and variance (Figure 18). This indicates that the retention objective is locally flat around the reference solution, but becomes increasingly nonlinear farther away.
These results support a local smoothness characterization: near the pretrained solution, the quadratic term in the Taylor expansion is weak; perturbation magnitude increases, curvature effects become non-negligible and increasingly influence retention-loss change.
C EXTENDED: CONCEPT ERASURE IS ALL-OR
We further analyze the interpolation behavior between the pre-trained model and its unlearned counterpart. Let fa (1 at)t + a 0 denote linear interpolation in parameter space. Across wide range of a generated outputs for the target concept remain visually indistinguishable from the base model. However, once crosses concept-specific threshold, the target is abruptly suppressed Figure 19 This sharp transition indicates that erasure behaves in an all-or-nothing manner rather than degrading smoothly. This observation is further supported by classifier accuracy, which stays near 100% across most interpolation values and then sharply collapses to 0% at the transition point (Figure 20p.
18

---

Published as a conference paper at ICLR 2026
Held-Out Retention Curvature vs Perturbation Scale
(A) Curvature Trend vs Perturbation Scale
(B) Curvature Distributions
95% intervel (ermoo modele)
Estimated curvature M()
Mttthen
Estimated curvature M(c)
10
Estazakar: central currectdd
Hange: Jeb4 (x-08)
Spearman
- 01
MWU R L(Be00)
3e-04
Se-p4
7e-04
1e-03
3648
5n444
76-04
1-40
Relative perturbation scale &
Relative perturbation scale
Figure 18: Estimated finite-difference retention curvature proxy under random UNet perturbations.
For relative scale e, we compute (e) |[ (0 + 8:C") (0: (T)lo//|| using matched minibatch, diffusion noise, and timesteps. (A) Per-model trends (thin lines) with median and 95% interval across models. (B) Distribution of (c) at each scale. Mc) increases with perturbation scale, indicating that the retention objective is locally flat near the reference model and exhibits stronger nonlinear curvature farther away.
D EXTENDED: PARAMETER DRIFT IS INTRINSIC TO SEQUENTIAL UNLEARNING
In our original experimental design, sequential unlearning used a fixed number of iterations based on the default values recommended in the original unlearning method papers. For simultancous unlearning, the relationship between training iterations and the number of concepts to unlearn was unknown. We therefore employed early stopping, evaluating unlearning accuracy every 100 iterations and terminating training once the model achieved 99% unlearning accuracy on validation set. This approach also allowed simultancous unlearning to serve as an initial upper-bound performance baseline for evaluating our sequential unlearning methods combined with the proposed add-ons.
However, verify that the greater parameter drift in sequential versus simultaneous unlearning (Figure 4) is not caused by early stopping, we conduct a controlled experiment where early stopping also applied sequential unlearning. Specifically, we continue sequential unlearning until the cumulative number of optimization steps taken by simultaneous unlearning matches or exceeds those taken in the sequential setting (Table T}. Our revised heatmap (Figure 2T} shows the same trend: parameter drift accumulates much faster in sequential than in simultaneous unlearning.
Num Unlearned
Sequential Steps
Simultancous Steps
400
300
2
700
400
3
1200
800
4
1300
700
5
1900
1300
6
2100
2100
Table 1: Cumulative optimization steps for sequential and simultaneous unlearning when early stopping is applied to both settings. Each entry reports the total number of training steps required to crase the number of concepts indicated the corresponding row. The cumulative step counts become equal by the sixth concept.
19

---

Published as a conference paper at ICLR 2026
0,7
Beace dltres
Bullactly
Dege-illorses
Bustec-Butteelyy
Jelyftab-@lowrrs
Huleets-llorsee
Statuse -Tront
Towers -Trees
Waterfalts-TTress
Figure 19: Qualitative interpolation between the pre-trained and unlearned models. For increasing interpolation coefficient a, generated images remain visually unchanged until a critical threshold is reached, after which the target concept is abruptly suppressed.
SIMULTANEOUS TRAINING COSTS
To compare the cumulative training costs of simultancous and sequential unlearning, we perform style unlearning using ConAbl and utilize our pest-performing add-on regularizer, semantic-aware gradient-projection projection. For fair comparison, we apply early stopping to both sequential and simultaneous unlearning, evaluating every 100 iterations and stopping once unlearning accuracy reaches 99%. As seen in Figure 22, sequential unlearning shows near-linear growth in training costs relative to the number unlearning requests, while simultaneous unlearning exhibits superlinear growth. This is because simultaneous unlearning requires training from the base model at each unlearning request, thereby incurring repeated computation costs re-unlearning previous requests. comparison of unlearning and retention performance can be found in Figure
20

---

Published as a conference paper at ICLR 2026
Dete-benerts
(yt-dleres
Vego divieer
3.5
04
62
34
2.5
3.5
6.0
34
0.5
1,0
6.
0.3
8.5
6.4
0.5
Figure 20: Quantitative evaluation of interpolation. Unlearning accuracy and retention metrics exhibit sharp transitions concept-specific thresholds, supporting that erasure occurs sharp change rather than a gradual decline.
Sequential
Simultaneous
+ Unlearn Request
learnable parameters
Figure 21: Parameter drift heatmap for ConAbl comparing sequential versus simultancous unlearning when early stopping is applied to both strategies.
F DETAILED RELATED WORK
F.1 FROM CONTINUAL LEARNING TO CONTINUAL UNLEARNING
Continual learning focuses on enabling models to acquire new knowledge incrementally without forgetting previously learned information-2 phenomenon known catastrophic forgetting (Mai et al., 2022 Wang et al. 2024 Lomonaco et al. 2022). Existing approaches to mitigate forgetting in continual learning can broadly classiñied into four categories: (1) regularization-ban based methods, which incorporate explicit regularization terms to constrain parameter updates (Kirkpatrick et al. 2017 Zenke et al. 2017): (2) replay-hased methods, which either store a limited set of previous examples in memory buffers (Mai et al., 2021: Shim al. 2021 or employ generative models to synthesize replay samples (Shin et al. 2017): (3) optimization based methods, which directly
manipulate optimization procedures through techniques such gradient projection (Chaudhry et al.)
2018) or meta-learning (Javed & White 2019); and (4) architecture-Be based methods, which introduce task task-specific adaptive parameters to the model (Mallya et al. (0188).
Although continual unlearning fundamentally differs from continual learning, key concepts from continual learning remain valuable and adaptable (Heng Soh 2023). In this work, we leverage ideas
21

---

Published as a conference paper at ICLR 2026
Avg Cumulative Training Cost
20000
Cumulative Iterations
10000
12300
2500
5000
Timestep
Grad Proj
Simultaneous
Figure 22: Cumulative training iterations for style unlearning with ConAbl, comparing sequential un- learning augmented with our best-performing add-on regularizer, semantic-aware cradient-projectinn against simultaneous unlearning. Sequential unlearning exhibits near-linear growth in cumulative cost, whereas simultaneous unlearning incurs superlinear growth.
inspired by regularization-bant methods from continual learning, introducing 1.1/1.2 regularization baselines. Additionally, while selective parameter updates appear in both paradigms, continual learning methods update the least important parameters to preserve prior knowledge (Mazumder al. 2021). In contrast, proposed Selective Fine-Tuning (SelFT) approach identifies and updates the most significant parameters to facilitate effective unlearning.
By bridging insights from continual learning to continual unlearning, research sets the stage for future investigations. We encourage subsequent studies to further integrate and refine continual learning strategies to address the nuanced challenges of continual unlearning effectively.
F.2 SELECTIVE FINE-TUNING
Selecting the most important parameters within model for a specific task has been extensively investigated for different purposes. To enhance time and memory efficiency, weight pruning methods commonly utilize gradient- based metrics to quantify parameter importance, enabling the removal of redundant parameters (Lee et 20188 Molchanov et 2016) Tanaka et al.] [2020). A similar concept underlies model editing techniques, which aim to precisely locate and alter specific knowledge within model by directly modifying relevant weights (Dai et al. 2021 Patil et al.) [023] De Cao et al. 2021). Recent work has extended these ideas to unlearning in diffusion models (Fan et al. 2023] Nguyen al. (024).. Our findings demonstrate that incorporating selective fine-tuning into existing unlearning methodologies significantly enhances their performance continual unlearning scenarios.
F.3 SEMANTIC AWARENESS IN UNLFARNIING
Most unlearning work emphasizes preserving model utility during concept removal. Beyond ag- gregate utility. it is equally important to identify which concepts are most susceptible to collateral degradation. Bui et al. (2025) investigate cross-concept effects and report that unlcarning a concept disproportionnaely degrades semantically similar concepts. Complementarily, Bui al. (2024) show that explicitly preserving closely related concepts yields larger overall utility retention. In contrast, we adopt a regularization perspective: we demonstrate that text-embedding similarity strong predictor of degradation and link this behavior to the cross-attention mechanism in diffusion models, where Wx, Wy couple text directions with image latents.
F.4 MODEL MERGING
Early research on model merging focused on averaging parameters of multiple models trained with varied hyperparameters identical datasets to enhance generalization (Wortsman et al. 2022).
22

---

Published as a conference paper at ICLR 2026
Concurrently, this strategy has been extended to multi-task learning, where models trained on diverse vision tasks have their weights averaged to achieve improved performance (Matena & Raffel [022] Ilharco et al. 2022). Since then, numerous advanced methods have emerged to refine the basic merging approach (fie-tuning followed by merging), including linearized fine-tuning (Ortiz-Jimenez al. 2023), sparsifying update vectors (Davari & Belilovsky 2024 Yu et al. 2024). and selectively
merging subsets of weights (Yadav et al. 2023 Stoica et al. 2023).
Recent concurrent studies have also explored model merging techniques specifically tailored for unlearning in large language models (LLMs) (Kuo et al. 2025 Kadhe et al. 2024). However, to the best our knowledge, this paper presents the first exploration of model merging for unlearning within the context of text-to-iie cott-to-image generation.
23


Reviews:
[{"review_id": "MI1RbjWTE1", "reviewer": "Reviewer_N1oH", "summary": "Current machine unlearning methods typically assume that all deletion requests arrive simultaneously. However, in real-world scenarios, deletion requests are often sequential, a setting referred to as continual unlearning.\nExisting approaches suffer from severe performance degradation under this setting, leading to both ineffective unlearning and the collapse of unrelated generation quality.\nTo address this, the authors systematically study continual unlearning for text-to-image diffusion models and propose three regularization-based strategies (update norm regularization, selective fine-tuning, and model merging), along with a semantic-aware unlearning method (gradient projection). These methods aim to mitigate parameter drift, improve retention of unrelated concepts, and minimize interference among semantically similar concepts.\nExperimental results demonstrate that the regularization methods effectively alleviate the performance collapse problem, while the semantic-aware unlearning method achieves the most significant improvement in in-domain retention. Furthermore, it can be combined with other regularization techniques to achieve a better trade-off between unlearning effectiveness and image quality.", "questions": "In addition to the weaknesses,\n- What's the computational overhead of different unlearning methods evaluated in the paper?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Formally define and analyze continual unlearning in the text-to-image setting.\n- Provides both theoretical and empirical insights into performance collapse due to parameter drift.\n- Proposes modular regularization and semantic-aware techniques that can easily integrate with existing unlearning methods.\n- Gradient Projection method effectively improves in-domain retention and reduces collateral forgetting.", "weaknesses": "- The study is limited to style and object deletions; it does not evaluate more practically relevant concepts such as NSFW, copyrighted, or identity-based content as previous works.\n- All experiments are conducted on a single diffusion model within the UnlearnCanvas benchmark. The paper does not assess whether the proposed regularizers and gradient projection method generalize to other architectures or larger-scale diffusion models\n- The benchmark setup relies on a limited base model and a relatively small, templated set of prompts for generation. This constrained setting—previously identified as a limitation of existing unlearning evaluation [1], which may not fully capture the diversity and complexity of real-world unlearning scenarios.\n- While the proposed techniques (regularization, model merging, gradient projection) are well-motivated, they are largely adaptations or combinations of existing ideas.\n\n[1] Ko, Myeongseob, et al. \"Boosting alignment for post-unlearning text-to-image generative models.\" Advances in Neural Information Processing Systems 37 (2024): 85131-85154.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z7RxjAd5wL", "reviewer": "Reviewer_6PKG", "summary": "The paper introduces continual unlearning for text-to-image diffusion models where removal requests arrive sequentially. It shows that popular methods degrade quickly in this setting due to cumulative parameter drift away from the pretrained weights, then proposes simple add-on remedies such as L1 or L2 update penalties, selective fine-tuning, model merging, and a semantic-aware gradient projection on cross-attention projections to protect nearby concepts. Experiments on an UNLEARNCANVAS-based benchmark report strong improvements in retention while maintaining unlearning effectiveness.", "questions": "1. How sensitive are the results to the regularization coefficient and top k selection used in selective fine-tuning and merging?\n\n2. How are auxiliary concepts selected for gradient projection, and how many are required for stable performance?\n\n3. The evaluation primarily relies on EraseDiff as the base method, which limits the generality of the findings. Incorporating other representative approaches such as ESD, SalUN, and AC (Ablating Concepts in Text-to-Image Diffusion Models) would provide a more comprehensive and convincing demonstration of continual unlearning behavior across different unlearning paradigms.\n\n4. While the paper mentions that continual unlearning may degrade the model’s general generative ability, the current evaluation mainly tests unrelated or random objects and styles to measure retention. A more informative evaluation would consider semantically related concepts to the forgotten target. For instance, when unlearning the concept “cat,” it would be more revealing to measure how well the model retains the ability to generate “tiger,” “lion,” or “leopard,” which are close in the embedding or visual space. Moreover, the paper lacks a broader assessment of general generation ability on a large-scale benchmark such as MS-COCO using metrics like CLIP Score or FID, which are standard in diffusion model evaluation.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clear problem definition of continual unlearning with precise requirements for erasing targets, preserving prior removals, and retaining unrelated capabilities, plus explicit metrics for unlearning accuracy and retention accuracy split into in-domain and cross-domain.\n\n2.  Practical plug-and-play remedies that integrate with existing unlearning methods, including L1 or L2 update penalties, selective fine-tuning, and model merging via TIES, which reduce drift and improve retention. \n\n3. The gradient projection idea operates on cross-attention projections to protect nearby concepts and combines well with other regularizers for further improvements.", "weaknesses": "1. Sensitivity to choices such as the strength of L1 or L2 penalties, the top k percent for selective updates, and the number and selection rule for auxiliary concepts in gradient projection is not fully characterized.\n\n2. Limited cost analysis for independent unlearning plus merging and for importance computation in selective tuning.\n\n3. The paper does not discuss several closely related recent works that address multi-concept and efficient forgetting, such as Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization (ICCV 2025) and ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning (ICLR 2025). These studies provide complementary perspectives on dynamic masking and pruning-based forgetting, and should be compared for completeness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I2wxVidNBf", "reviewer": "Reviewer_xbNs", "summary": "This work conducts the first systematic investigation of continual unlearning in text-to-image diffusion models, revealing that existing unlearning methods suffer from severe utility collapse due to cumulative parameter drift. To address this, the authors propose a set of regularization-based strategies that mitigate drift while remaining compatible with existing methods. They further introduce a semantic-aware gradient projection technique that constrains parameter updates to directions orthogonal to the target concept’s subspace, preserving related knowledge. Overall, these methods substantially improve continual unlearning stability and establish strong baselines for safe, accountable generative AI.", "questions": "Minor concerns:\n\n- The colors in Figure 3 are too similar, resulting in poor readability.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper presents an interesting and valuable study on continual unlearning in text-to-image diffusion models.\n2. This paper is very well presented.\n3. The paper conducts a detailed analysis of the challenges faced by continual unlearning in text-to-image diffusion models through a series of experiments.", "weaknesses": "The author emphasizes that this paper does not propose new algorithms but focuses on the analysis of continual unlearning. I have the following questions about this paper:\n\n1. Compared to regular continual learning, what are the additional challenges of continual unlearning? Parameter drift and conceptual confusion have been extensively studied in continual learning. Results in figure 3 separate the unleaning target from the retention target, but can also be interpreted as follows: as the number of requests increases, the model forgets the required targets, leading to indiscriminate unleaning of all concepts.\n2. Experimental findings indicate that object retention and style retention exhibit distinct patterns of forgetting, though further analysis of this phenomenon is lacking.\n3. In the experiment shown in Figure 4, for sequential learning, is the sum of the update iterations for multiple requests the same as the update iterations for simultaneous learning?\n4. What insights does this paper offer for future research on continual unlearning in text-to-image diffusion models. Given that the methods combined in this paper have long been applied in continual learning, does this imply that regularization techniques and model merging approaches designed to address continual learning issues can effectively tackle continual unlearning in text-to-image diffusion models?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jlMQ0c1rPc", "reviewer": "Reviewer_oE7J", "summary": "This paper introduces the problem of continual unlearning for text-to-image diffusion models, where forgetting requests are handled sequentally. The paper shows that popular unlearning frameworks suffer from degraded performance in the continual setting. The paper shows that the problem is due to cumulative parameter drift, where the model weights progressively drift from the original weights. To address this, the paper proposes several methods: regularization, selective fine-tuning, model merging and gradient projection. Experiments show that these methods significantly improve the model's ability to retain performance in the continual unlearning setting.", "questions": "- Given the combined results in Fig 8, what end-to-end 'default' recipe do the authors recommend for sequential unlearning? A short set of takeaways for practitioners would help solidify the insights.\n- Have the authors considered a Fisher-weighted regularization approach from continual learning like in [1]? This would lie between full L2 regularization and Selective fine-tuning.\n- How sensitive is performance of gradient projection to the auxiliary set of semantically similar concepts? Could a poorly chosen set harm retention or unlearning?\n- Have the authors investigated the effect of the order of unlearning requests? For example, does unlearning a broad concept (e.g. \"photorealism\") early in the sequence have a different impact than unlearning a more specific one (e.g. \"Van Gogh\")?\n- How do results differ on concepts that can be referenced explicitly (like objects) versus indirectly (e.g., styles by prompting for artist names, or using synonyms)?\n\n[1] Heng, Alvin, and Harold Soh. \"Selective amnesia: A continual learning approach to forgetting in deep generative models.\" Advances in Neural Information Processing Systems 36 (2023): 17170-17194.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The continual unlearning setting is a practical and important problem in the field of text-to-image unlearning.\n- While the root cause being cumulative parameter drift is unsurprising, the paper provides clear empirical evidence and analysis of the phenomenon.\n- Though tested on only two baselines (see weaknesses), I find that the experiments and results are thorough and provide good evidence of the author's claims.\n- Proposed methods like regularization and gradient projection are simple yet general, making them easy to integrate with existing unlearning pipelines.\n- Overall, the paper is clear, concise and well-structured.", "weaknesses": "- In itself, the proposed methods are not novel (regularization, gradient projection etc. are certainly not new), but applied to the setting of continual unlearning. \n\n- The experiments are conducted on two methods, ConAbl and EraseDiff. While these are representative, it is unclear how the findings of the proposed methods would generalize to other classes of unlearning algorithms. \n\n- The authors claim simultaneous unlearning is costly, but their proposed model merging is also costly given independent copies have to be unlearned. Have the authors compared the computational costs and whether model merging is more efficient than simultaneous unlearning?\n\n- On the Taylor expansion of the loss in Sec 5.2, the loss is *upper-bounded* by $||\\theta^* - \\theta^\\dagger||$, thus even if the RHS grows, it does not guarantee the retention loss grows. Hence the conclusion that \"loss grows proportionally (up to a constant) to $||\\theta^* - \\theta^\\dagger||$\" seems somewhat inaccurate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)