Co-occurring Associated REtained concepts in Diffusion Unlearning
Authors: Miso Kim
Georu Lee
Yunji Kim
Hoki Kim
Jinseong Park
Woojin Lee
Venue: ICLR 2026 Poster (2026)
Keywords: unlearning
diffusion
concept erasure
safety
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce CARE (Co-occurring Associated Retained concepts) and propose ReCARE, a framework that preserves CARE during diffusion model unlearning, achieving robust erasure without sacrificing benign co-occurring concepts.

Abstract: Unlearning has emerged as a key technique to mitigate harmful content generation in diffusion models. However, existing methods often remove not only the target concept, but also benign co-occurring concepts. Unlearning nudity can unintentionally suppress the concept of person, preventing a model from generating images with person. We define these undesirably suppressed co-occurring concepts that must be preserved $\textbf{CARE}$ ($\textbf{C}$o-occurring $\textbf{A}$ssociated $\textbf{RE}$tained concepts). Then, we introduce the  $\textbf{CARE score}$, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose $\textbf{ReCARE}$ ($\textbf{R}$obust $\textbf{e}$rasure for $\textbf{CARE}$), a framework that explicitly safeguards CARE while erasing only the target concept. ReCARE automatically constructs the CARE-set, a curated vocabulary of benign co-occurring tokens extracted from target images, and leverages this vocabulary during training for stable unlearning. Extensive experiments across various target concepts ($\textit{Nudity}$, $\textit{Van Gogh}$ style, and $\textit{Tench}$ object) demonstrate that ReCARE achieves overall state-of-the-art performance in balancing robust concept erasure, overall utility, and CARE preservation.

Published as a conference paper at ICLR 2026
CO-OCCURRING ASSOCIATED RETAINED CONCEPTS IN DIFFUSION UNLEARNING
Miso Kim', Georu Lee', Yunji Kim', Hoki Kim Jinseong Park Woojin Lee 'Dongguk University-Scoull "Chung-An University, "Korea Institute for Advanced Study {2021110472, dirjfnl, 2022113147, ac wtw.://aww...............eee...e re. re.kr
After Nudity Unlearning
SD 1.4
AdvUnlearn
AGE
STEREO
OURS
"A person" "A node person'
Person
Person
Figure 1: Preserving Co-occurring Concepts in Nudity Unlearning. After unlearning nudity, we present generations from two prompts ("A nude person" and person"). Baseline methods (AdvUnlearn, AGE, STEREO) suppress benign co-occurring concepts person, failing generate person images. In contrast, our proposed ReCARE preserves those concepts while erasing nudity,
ABSTRACT
Unlearning has emerged as key technique mitigate harmful content genera- tion in diffusion models. However, existing methods often remove not only the target concept, but also benign co-occurring concepts. As illustrated Fig. [ unlearning nudity unintentionally suppress the concept of person, prevent- ing a model from generating images with person. We define these undesirably suppressed co-occurring concepts that must be preserved CARE (Co-occurring Associated REtained concepts). Then, we introduce the CARE score, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose ReCARE (Robust erasure for CARE), framework that explicitly safeguards CARE while erasing only the target concept. ReCARE auto- matically constructs the CARE-set, a curated vocabulary of benign co-occurring tokens extracted from target images, and leverages this vocabulary during train- ing for stable unlearning. Extensive experiments across various target concepts (Nudity, Van Gogh style, and Tench object) demonstrate that ReCARE achieves overall state-of-the-art performance in balancing robust concept erasure, overall utility, and CARE preservation.
INTRODUCTION
Diffusion models have achieved remarkable success in generating highly realistic images (Chang ct al. 2023 However, training on large-scale data raises ethical concerns, including the risk of
"Comresooning author.
https://gi thub. com/ dami 1ab/CART

---

Published as a conference paper at ICLR 2026
producing harmful or NSFW (Not Safe For Work) content (Rando et al. 2022 Schramowski al., 2023 Zhang et al. c024c Kim et al.] 2025a Park & Park 2025). To mitigate these issues, ma- chine unlearning (MU) has emerged as a paradigm for selectively removing the influence of target concepts from pre-trained models (Cao & Yang 2015). Recent work has particularly focused on post-hoc erasure, which fine-tunes the diffusion model shift the noise prediction of the target token toward the unconditional output (empty prompt) (Gandikota et al. (023).
Post-hoc erasure is a practical approach to concept removal, but existing methods still face fun- damental challenge: the robust-utility trade-off. Models must erase harmful concepts (robust- ness) while preserving overall image quality (utility). To this end, recent methods employ anchors, prompts that represent non-target concepts, which the model should still generate correctly. These anchors are commonly obtained from prompts drawn from external label sets (e.g.. ImageNet) or synthesized by large language models (Zhang et al. 2024b, Srivatsan et al. 2025 Bui et al. 2025
Although based preservation improves utility, we identify critical weakness: benign con- cepts that naturally co-occur with the erase target are also unintentionally suppressed during unlearning. As illustrated in Fig. [ attempts to erase nudity often suppress the concept of person, which commonly appears together with rudity in training data. For example, with prompts like "A nude person" " or "A person", models often fail to generate people, unintentionally suppressing the concept of person even when the intended removal pertains solely to nudity. Effective unlearning must not only erase harmful targets but also care for the benign concepts that naturally co-occur with them. Therefore, we define these co-occurring concepts that must be carefully preserved as CARE (Co-occurring Associated REtained concepts) and propose a method to preserve it.
However, commonly used utility evaluations does not reflect whether CARE concepts are preserved, so even models with high utility scores may still not retain benign co-occurring concepts. Despite its importance, the evaluation of CARE preservation has remained unexplored in existing unlearning studies. To address this gap, we introduce the CARE score, a simple yet effective metric that explicitly measures the retention of CARE concepts. We argue that the CARE score essential way to evaluate unlearning, orthogonal to the existing metrics for robustness and utility.
Given the importance of CARE concepts, we propose ReCARE (Robust erasure for CARE), a method that preserves CARE while ensuring robust erasure. ReCARE first constructs a CARE-set, a vocabulary of benign co-occurring tokens, from target images. During refinement, harmful co- occurring tokens are removed if they are too similar to the target or irrelevant to CARE preservation. By leveraging the CARE-set in training, ReCARE achieves robust erasure, preserves overall utility, and ensure faithful CARE preservation.
Our key contributions can be summarized as follows: We identify and define the unintended suppression of co-occurring concepts that should be preserved during unlearning, introducing the notion of CARE as a critical consideration for effective unlearning. @ We develop CARE score, a new metric that explicitly measures the preservation of CARE concepts, a dimension overlooked in prior unlearning research. @ We propose ReCARE, a method that robustly erases target concepts without sacrificing CARE preservation, thereby improving both robustness and utility.
2 BACKGROUND
Latent Diffusson Models (LDMs) (Rombach et al. 2022) are text-to-image models that operate in compressed latent space. Starting from Gaussian noise NN 1), the model iteratively denoises a latent variable x timestep t, conditioned on a text prompt p. The training objective is to predict the injected noise at each step using noise predictor eø:
(I) = E E[Ilcc- co(2) P)]
While LDMs can generate high-quality images, they may also produce harmful concepts. A rep- resentative unlearning method to mitigate this is Erasing Stable Diffusion (ESD) (Gandikota et al. 2023), which erases target concept c. Specifically, the frozen teacher model 8* characterizes the semantic direction of e as the difference between its conditional prediction eg- (z, c) and uncondi- tional prediction eg- (z. 0). The student model 0 is trained to erase this concept by updating in the
2

---

Published as a conference paper at ICLR 2026
I "A photo of a nude person"
B "A painting in the style of Van Gogh"
B "A photo of tench"
Removed
Co-occurence words
Benign co-occurrences
Target
1
Nudity
Nude, Naked, Baré, Person, Woman, Figure,
Person, Woman, Figure,
2
VanGogh
Goghy Vincent, Stars, Calaxy, Moon,
Stars, Galaxy, Moon, Freshwater, Male, Sturgeon,
3
Tench
TinciTines, Doctoriisi Freshwater, Male, Sturgeon,
Figure 2: Given removed target, we can extract the co-occurring words from generated images and categorize them into two groups: harmful co-occurrences and benign co-occurrences.
opposite direction of the vector, while the strength of erasure modulated by hyperparameter vy:
(Es E co(2 c) - (co- (2t ( c(co- (2t c) - ca- (ze 0)) [|_
(2)
In addition to ESD, other methods have been proposed to address the robustness-utility trade-off.
AdyUnlearn (Zhang al. 2024b) integrates adversarial training with prompts and introduces retain loss to preserve utility. AGE (Bui et al. 2025) improves this by adaptively selecting anchors from large external vocabulary, balancing forgetting and preservation. However, erased concepts can often be recovered through textual inversion, a process in which a new token v" for a concept is learned from only few exemplar images related the target. The token is optimized by minimizing the objective in Eq [ over this small image set, with all pre-traines parameters frozen:
2² arg min E ||c-cc(ze, t, 2)1
ca(2,,
(3)
Thus, textual inversion exposes a critical vulnerability in current unlearning approaches, as it can lead to the unintended reintroduction of erased concepts. STEREO (Srivatsan al. 2025) leverages textual inversion to obtain optimal embeddings that can regenerate the target concept even after unlearning, and uses them to compute the dominant erasure direction for training.
UNLEARNING BEYOND THE TARGET: CARE SUPPRESSION
3.1 CO-OCCURRING ASSOCIATED RETAINED CONCEPT (CARE)
Prompts containing a target concept often generate images with additional co-occurring concepts. We can categorize these concepts into three types: (i) target concepts be erased; (ii) harmful co-occurrences that should also be erased; (iii) benign co-occurrences that should be retained. For instance, the prompt "a photo of a nude person" yields the nudity target, along with co-occurring concepts such as naked (harmful co-occurrence) and person (benign co-occurrence). Likewise, as shown in Fig. 2 the prompts widely used in diffusion unlearning have these concepts: Van Gogh, Vincent, and stars; tench, tincatinca and freshwate...
Preserving benign co-occurrences during unlearning challenging, In machine unlearning, we expect model to forget the target concepts and harmful co-occurrences, while preserving benign co-occurrences. However, we identify that existing unlearning methods often fail to generate benign co-occurring concepts. In Fig. [ we demonstrate that they fail to preserve the concepts of person in nudity. Similarly, as shown Fig. 3 they fail to preserve the concepts of stars in Van Gogh, and freshwater in tench. This challenge might arise because models such as CLIP encode co-occurring concepts within overlapping regions of the embedding space, leading to strong entanglement (Jiang et al. 2024). Moreover, existing approaches often rely anchors such ImageNet classes, LLM- generated prompts, or external dictionaries, which either capture only generic concepts or suffer from limited vocabulary quality.
3

---

Published as a conference paper at ICLR 2026
AdvUnleam
AGB
STERBO
Human-annotated Ground Truth
SD 1.4
Van Gogb Unlemring
Human Detection Paticc (96)
"Stars"
Tench Unkearning
"Freshwater"
Concept
Stars
Freshwater
Qualitative failure cases in existing unlearning methods. In particular, these meth- ods fail to generate stars and freshwater, they inadvertently suppress benign co-occurring con- cepts while erasing target concept.
Figure 3:
Quantitative comparison of CARE preservation based on human-annotated ground truth. Human evaluators examined the presence of CARE concepts in generated images for each target (nudity, Van Gogh, and tench).
Figure 4:
Therefore, we define the set of such benign co-occurrences that must be preserved during unlcarning CARE (Co-occurring Associated REtained concepts).
Fig. demonstrates quantitative evidence CARE preservation, based on aannann-nnotted ground truth counts of generated images containing person. stars, or freshwaerr. It shows that existing meth- ods often erase these benign co-occurring concepts together with the target, whereas our approach preserves them much higher degree. These results reveal that CARE is preserved by existing methods, highlighting the need for new mechanism to preserve it.
However, how can CARE preservation be automatically measured scale after unlearning?
3.2 CARE SCORE
Existing evaluation metrics (e.g., FID, CLIP score) fail to measure whether CARE concepts preserved. This is because they only capture global fidelity or semantic similarity to prompts, without explicitly verifying the presence of spe- w
Pearson 0.905
Human Detection Rate (%g
Concept
cific benign co-occurring concepts.
A
Person
Freshwater
Therefore, propose the CARE score, princi- pled metric that directly evaluates CARE preser- vation. To compute CARE score, use CLIP R- Precision @ Park et al. 2021). For each target, select one CARE concept (e.g., "person" for mudity) and combine it with 80 unrelated tokens from COCO object labels. We then generate sam- ples using prompts containing the chosen CARE concept and test whether it ranks top-I among all candidates. Details of the prompt construction procedure are provided in Appendix K
Modell
50
AGE
STEREO
OURS
CARE Score (%)
Figure Correlation between CARE score and human-annotater ground truth across different targets and methods (Pearson r 0.905).
Formally, the CARE score is defined as:
S CARF Σ
CLIP(ss,, w²)= max CLIP(Is; w)) 3. G(Cur)
(4)
wE((wr JUO)
where w* is the chosen CARE concept, Ge- is the corresponding prompt, O is set of unrelated COCO object tokens, G is the generative model after unlearning. 5 the number of generated samples, and CLIP(z,, w) denotes the CLIP similarity between generated image x, and token
To validate the reliability of CARE score, we compare against the human-annotatee annotated ground truth introduced earlier. As shown in Fig. strong correlation across different targets and methods.
4

---

Published as a conference paper at ICLR 2026
Input (z)
TopN
(1) Global Clustering
CARE-set
freg
"A. photo of nude person"
naked
* C (Target) Nudity
naked
Retain Loss
bare
et
Removved by
topless
Average Residual (+%)
topless
CLIP-based
cooccurence
person
E
E
person
max
woman
woman
body
Removed
figure
ABCDEF
figure
physique
(2) Intra ChrrteCluster Refinement
physique
Erase Loss
Interative Textual Inversion
stripped
femme
Cluster B Core
distance d
from
Centroid the tokens
Removedd by (
femme
mannequin
person
B-S
mannequin
Cerise
mistress
mistress
Update optimal embeddings 0i 1/
kristina
body
body
scarlett
Figure 6: Overview of ReCARE. (I) Global Clustering groups candidate tokens on the t-SNE pro- jected embedding space and removes clusters that are either overly similar to the target or entirely irrelevant. (2) Intra-Cluster Refinement prunes tokens that still subtly resemble the target within the retained clusters. The surviving tokens form the CARE-set D, which acts preservation signal in Retain Loss and as a guiding reference in the Erase Loss.
An exception arises with STEREO on the person concept, where the CARE score is lower because the generated figures are present but degraded and barely recognizable. This indicates that CARE score not only aligns with human inspection but also reflects image quality effects, making it more stringent measure of CARE preservation. Overall, CARE score emerges necessary metric for evaluating unlearning models, complementing robustness against harmful concepts and overall utility.
METHOD
We propose ReCARE (Robust erasure for CARE), a framework that achieves robust concept re- moval while explicitly safeguarding CARE. ReCARE first constructs the CARE-set, a curated vo- cabulary of benign co-occurring tokens extracted from target images, through two refinement stages. First, global clustering filters out tokens overly close or far from the erase target. Second, intra- cluster refinement, which applies fine-grained filtering within clusters. The constructed CARE-set then integrated into training with two complementary roles: it acts preservation signal in Retain Loss and as a guiding reference the Erase Loss. An overview the framework is shown in Fig
4.1 CARE-SET CONSTRUCTION
To build an effective CARE-set, we start from the same target images later used for textual inversion (Sec. 4.2). This choice avoids additional data collection and ensures that the words reflect concepts that genuinely co-occur in real images. From these images we extract a set of co-occurring candidate tokens, which inevitably includes two erased types: harmful co-occurrences that overly similar to the target, and completely irrelevant tokens that do not belong to CARE. Hence, refinement is required before the can be reliably used.
Given a target image set Z for the target concept, we com- pute clip similarity CLIP 1) between each image x I and token E V; CLIP vocabulary. For each image, we select the Top- K tokens and then aggregate them across images. The Top-N tokens by frequency constitute the set of co-occurring candidates:
Extracting Co-occurring Candidates.
TopN Froq
U Topk CLIP(1,t)). ZEI
(5)
5

---

Published as a conference paper at ICLR 2026
As illustrated in Fig. 5 (TopN freq), this candidate set often contains harmful tokens, including those that are overly similar to the target (e.g., naked when erasing nudity) and others that are semanti- cally irrelevant (e.g., scarlett, common female name unrelated to CARE). Therefore, refinement is necessary before the CARE-set can be reliably used.
Global clustering. To refine the candidate tokens, we cluster them by their distance from the target embedding and remove clusters that are either overly similar to the target or entirely irrelevant (See Fig. I)). The candidate tokens are then embedded into 2D space using t-SNE (Maaten Hinton 2008 and grouped into n clusters {Ck je- via k-means. Let e denote the target concept, and e, its
corresponding text embedding. For each token embedding et, we measure its orthogonal distance from the target as:
(6) where I is the identity matrix. Small r(e)) values correspond to tokens closely aligned with the target, large values correspond to semantically irrelevant tokens, and intermediate values capture po- tential CARE candidates. For each cluster, we compute the average residual F. | Eiec, r(ex).
r(ex) = el - eoe -oee 112-
The cluster most similar to the target (A =
arg ming fk) and the cluster most unrelated to the target (I+ arg max; fik) discarded, while the remaining clusters retained as candidates:
Coand = CCa k & (k (
(
Intra-cluster refinement. Although global clustering already removes clusters that are either close to or too far from the target, some tokens within the retained clusters C₁ E Ceand may still sub- tly resemble the target. For instance, words like stripped body less explicit than harmful terms already filtered out in the global step, yet they remain aligned with nudity and are thus unsuitable as CARE. This refinement ensures that the words focus on genuinely benign co-occurring concepts, filtering out residual target-related cues (See Fig. ((2)..
For each cluster Cs E Coand with C compute the centroid of C₁ excluding (R)
f&z
[C and each token index {l,..., |Cxl), M=1
Σ
6,0
(8)
Cs
Let 60 E {0, 13 be a binary indicator specifying the retention of token (8)
1, if r() < (1 a1)
| Ejes r(e (
5(())
0, otherwise.
where a > 0 controls the strictness of pruning. The final CARE-set D is then expressed as:
U {( 1, ie f1..... (C.B))
(10)
Intuitively, tokens that remain overly aligned with the target contribute little to the concept- orthogonal component their cluster and are therefore pruned. As result, tokens strongly re- sembling the target discarded, while the remaining co-occurring tokens are highlighted as the essential elements of CARE. The surviving tokens across all clusters C₁ together form the final CARE-set D, refined vocabulary of benign co-occurring tokens, which acts the foundation for the subsequent training objectives. The complete algorithm of this construction process is provided in Appendix
4.2 UNLEARNING WITH CARE-SET
We define the overall training objective as the combination of Frase Loss and Retain Loss, with hyperparameter controlling the trade-off between
between robust erasure and CARE preservation:
(11) In the following, we describe how the CARE-set incorporated into each loss term. First, to safe- guard CARE during erasure, introduce Retain Loss that constrains the model to preserve
LRECARE
LReuain + Lizuse-

---

Next, we design an Erase Loss that uses the CARE-set D to disentangle harmful tokens from CARE, aligning them with the CARE representation while pushing them away from the erase direction. This ensures that CARE concepts are preserved while the target is effectively erased. To compute this crase direction, we adopt the STE procedure from Srivatsan et al. (2025), which applies textual inversion reveal optimal embeddings that regenerate the target concept even after partial unlearn- ing. This process produces sequence of progressively stronger embeddings (e.g., oi, 25), which we combine with the explicit target token (e.g.. "nudity") to compute Cersse an average embedding representing the erase direction. We then form a CARE aligned reference by subtracting this harm- Published as a conference paper at ICLR 2026
knowledge of the CeRE-set. Specifically, we construct preservation prompts E by applying generic templates (e.g., 'A photo of...) ') to tokens from the CARE-set D and minimize the discrepancy be- tween the outputs of the original model (8*) and the unlearned model (0;) to encourage consistency on non-target concepts." The Retain Loss formally defined as:
Licetiin E
|lear (Ees (,,t,,) E)) E0, (2,,,, EX]
(12)
ful direction from the CARE representation of the original model 0", and train the unlearned model 0, so that the representation of harmful tokens H { (u 25, "nudity) } matches this reference:
LE = E (E(0-- (2,, D) - Eerase) E0, (Zt, L, 11
(13)
EXPERIMENT
5.1 EXPERIMENTAL SETUPS
Evaluation Metries. Robustness measured by the Attack Success Rate (ASR) (Gandikota al., 2023 Zhang et al. 2024b Bui et al. 2025 Srivatsan et al. 2025), the proportion of adversarially generated images that still contain the erased concept (details in Appendix [B.. Since a lower ASR implies stronger robustness,
report Defense in the radar chart, defined the attack failure rate (100% - ASR). Utility is evaluated on COCO-30K using FID (Heusel et al. (017)) (lower is better) and CLIP Score (Hessel et al. (021) (higher is better). CARE preservation is quantified by the CAR in Eq. 4 which directly measures the retention of benign co-occurring concepts after unlearning. We evaluate unlearning performance across three representative tasks: Nudity, artistic style (Van Gogh), and object (Tench).
To facilitate a straightforward comparison across the three aspects, we define RATIO as our pri- mary evaluation metric. This metric captures the trade-off between Robustness, Utility, and CARE preservation, and is computed as the normalized area of the radar chart spanned by these three axes. A larger value of RATIO indicates better overall performance. The detailed computation procedure provided in Appendix M
Baselines and Attack Methods. We compare our method against eleven recent unlearning base- lines: STEREO (Srivatsan et al., 2025). ESD (Gandikota et al. 2023), UCE (Gandikota et al.)
2024). AdyUnlearn (Zhang et al. 2024b). AGE (Bui et al. 2025). MACE (Lu et al. 2024 RECE (Gong et al. 2024). SPM (Lyu et al. 2024), FMN (Zhang et al. 2024a), SalUn (Fan et al.)
2023), and EraseDiff (Wu et al. 2024h. To evaluate robustness against ad adversarial prompts,
adopt three attack methods: DnlearnDiff (UD) (Zhang et al. 2024c), Ring-A-Bell (RAB) (ssai
et al. 2023), and CCE (Pham et al. 2023). Details for each attack are provided in Appendix
5.2 EXPERIMENT RESULTS.
Nudity unlearning. Our method achieves the highest RATIO (See Fig. 7. indicating the most reliable overall performance across robust- ness, utility, and CARE preservation. Table detailed results. It substantially reduces ASR across all adversarial settings, demonstrat- ing strong erasure performance. Under the challenging CCE attack, most baselines still generate the target concept (See Fig. 9), whereas our method remains effective. A closer look at the trade-offs highlights clear limitations of baselines. AdyUrleann struggles across robustness, utility,
Utility
Advlnisars
STEREO
CARE
Figure 7: Radar chart of Nudity unlearning.

---

Published as a conference paper at ICLR 2026
Full performance comparison on Nudity, Van Gogh style, and Tench object unlearning tasks. Evaluation is conducted under Erased (no attack) and adversarial attacks (UD, CCE). We report ASR (robustness), CLIP/FID (utility), and CAeE (CARE preservation), with overall performance summarized by RATIO. RAB is a Nudity-specifie attack, and its results along with all baseline details are provided in Appendix
Table 1:
Robustness (ASR)
Utility
CARE
Model
Erased
UD
CCE 1
CLIP
FID
CARE
Nudity
SD
35.23
39.51
56.82
0.31336
14.12
0.97
0.56
ESD
3.18
3.70
53.41
0.3045
13.75
0.89
0.49
FMN
32.73
35.80
51.82
0.3111
13.95
0.95
0.31
UCE
2.27
3.70
44.55
0.3117
14.31
0.83
0.48
SPM
14.09
23.46
38.41
0.3125
14.62
0.96
0,30
MACE
0.00
2.47
61.82
0.2931
12.70
0.95
0.10
RECE
0.91
3.70
40.23
0.3097
14.62
0.83
0.51
AdvUnlearn
23.64
1.23
65.45
0.2925
15.53
0.36
0.18
AGE
0.23
2.47
27.27
0.3006
11.25
0.79
0.56
STEREO
0.00
0.00
19.55
0.2907
17.83
0.11
0.21
ReCARE (Ours)
0.00
0.00
11.14
0.3053
13.85
0.94
0.76
Van Gogh
SD v1.4
74.00
84.00
64,40
0.3136
14.12
0.89
0.48
ESD
0.40
18.00
13.20
0.3074
14.47
0.77
0.67
FMN
1.60
14.00
61.60
0.3140
13.90
0.85
0.48
AC
2.40
48.00
36.00
0.3124
14.04
0.90
0.65
UCE
20.60
76.00
61.80
0.3140
13.88
0.84
0.48
SPM
9.60
60.00
54.60
0.3134
14.06
0.82
0.51
MACE
5.60
20.00
52.40
0.2862
12.60
0.05
0.10
RECE
2.40
42.00
55.80
0.3137
13.84
0.83
0.51
AdvUnlearn
0.80
4.00
57.00
0.3106
14.04
0.76
0.45
AGE
0.00
14.00
12.40
0.3100
13.80
0.75
0.68
STEREO
0.00
6.00
4.00
0.3047
18.17
0.31
0.43
ReCARE (Ours)
0.00
6.00
6.00
0.3101
16.24
0.90
0.81
Tench
SD v1.4
96.80
92.00
98.00
0.3136
14.12
0.95
0.30
ESD
3.80
40.00
94.80
0.3051
13.18
0.83
0.25
FMN
92.60
96.00
94.60
0.3114
13.42
0.95
0.31
SalUn
0.00
2.00
91.60
0.3150
14.05
0.96
0.35
EraseDiff
0.20
10.00
87.00
0.3120
12.62
0.93
0.35
SPM
45.80
84.00
98.00
0.3134
14.05
0.96
0.30
AdvUnlearn
0.00
2.00
95.20
0.3093
14.26
0.66
0.21
AGE
63.80
96.00
99.40
0.3121
13.89
0.95
0.28
STEREO
0.00
0.00
3.60
0.2975
15.87
0.62
0.56
ReCARE (Ours)
0.00
0.00
0.40
0.3073
14.32
0.97
0.85
Some methods preserve CARE better but collapse under ad- versarial attacks. STEREO, while more robust due textual inversion, sacrifices both utility and
and CARE.
CARE preservation.
Van Gogh style unlearning. Our method achieves the highest RA- TIO, reflecting the most reliable trade-off among robustness, utility, and CARE preservation. Table [ shows that maintains low ASR across attacks while preserving high utility and the best CARE score. Quali- tatively, removes the "Van Gogh" style while retaining benign scene clements such "star" (See Fig. 9). Basclines reveal clear weaknesses. AdvUnlearn attains higher utility but is casily broken by attacks and STEREO shows strong robustness but severely fails to preserve CARE. The corresponding radar chart is provided Appendix
Utility
Advünicom
STEREO
CARE
Figure 8: Radar chart of Tench object unlearning.
Tench object unlearning. Table I summarizes the quantitative results. Our method delivers the most balanced performance across the three axes, achieving the highest RATIO (See Fig. 8). It also attains the strongest robustness and the
8

---

Published as a conference paper at ICLR 2026
v1.4
ESD
FMN
SPM
AdvUnlearn
AGE
STEREO
Ours
*A photo of [orja
Attack
Nudity
"A perion deyctaa in Useal porai"
CARE
"A panting tfei Gey
Attack
Van Gogh
"A "Amay depeeto ifttrs
CARE
"A photo of (e=)
Attack
Tench
"Atriiou Spiisnaa
CARE
Figure 9: Qualitative results three unlearning tasks (Nudity, Tench, and Van Gogh). For each task, we show results under CCE attacks and CARE prompts. Baselines often fail either by still generating the erased concept (top rows) or by suppressing benign CARE concepts such as person, stars, or freshwater (bottom rows). Ours successfully removes the target concept while preserving CARE across all three tasks. Full quantitative results with all baselines are reported in Appendix !
highest CARE score across all adversarial settings, effectively removing the target object while pre- serving benign concepts. As further confirmed by the qualitative comparisons (See Fig. 9), some baselines show good utility and CARE but fail unlearning, easily by adversarial attacks. STEREO is robust but sacrifices utility and CARE. AdyUnlearn is vulnerable and fails to retain CARE.
Computational Efficiency of CARE-set and ReCARE. We also evaluate the computational efficiency of CARE-set construc- tion and the ReCARE unlearning pipeline. CARE-set extraction highly lightweight, requiring only 1.78 minutes end-to-end (CLIP similarity computation clustering + refinement). ReCARE training consists of Textual Inversion (23.23 min) followed by ReCARE optimization (5.10 min), totaling 28.33
Table 2:
Training time and Nudity task performance compari- son. ReCARE achieves strong erasure performance while main- taining low computational overhead.
Method
Time (h) |
CCE
CLIP t CAREscore f
ESD
0.69
53.41
0.3045
0.89
RECE (Training-free)
0.01
40.23
0.3097
0.83
AGE AdvUnleam
2.20
27.27
0.3006
0.56
21.80
65.45
0.2925
0.36
STEREO
0.41
19.55
0.2907
0.11
ReCARE (Ours)
0.50
11.14
0.3053
0.94
minutes with a peak GPU memory footprint of 24GB (H100). Despite its low overhead, ReCARE achieves strong performance the Nudity task compared to prior methods (Table
5.3 ABLATION STUDY
Impact of CARE Refinement Components. To analyze the contribution of each component in our CARE-see set construction, we conduct an ablation study by selectively removing the Global clustering and Intra-cluster refinement, which we apply to the nudity unleaming task. We compare four settings: (i) Ours, (ii) w/o Intra, (iii) w/o Global, and (iv) w/o Refinement. As shown in Table

---

Published as a conference paper at ICLR 2026
the full method achieves the highest CARE score and the lowest ASR un- der CCE attacks, striking the best bal- ance between robustness and CARE preservation. When global clus- tering is removed, irrelevant tokens included, which decreases the CARE score, while harmful tokens that should have been excluded also remain in the set, leading to an increase in ASR. Similarly, applying only global clustering preserves CARE to some extent but still fails to filter out subtle harmful tokens, again resulting higher ASR. Finally, using the whole candidate tokens without any refinement yields the lowest CARE preservation and the highest ASR, demonstrating that the refinement process is essential for constructing a stable CARE-set.
Table
Impact of CARE refinement components.
Erased 4
CCE 4
CLIP
CARE,oor
ReCARE (Ours)
0.00
11.14
0.3053
0.94
w/o Intra Global
0.00
16.36
0.3082
0.93
0.00
25.00
0.3039
0.90
w/o refinement
0.00
27.05
0.3056
0.88
k-means num of k. In the global clustering stage, the number of clusters determines how finely the candidate tokens are parti- tioned. To verify its effect, we conducted experiments on the nu- dity unlearning task with n 4, 4,5, 6. As shown in Table 4 the overall performance was highly sensitive to the choice of n. In particular, both erasure ability (low ASR) and CARE preservation (high CARE score) exhibited consistent trends, indicating that our framework stable with respect n. Among them, n 6 achieved the most balanced results, attaining the lowest ASR while maintaining competitive FID, CLIP, and CARE scores. Therefore, we set = 6 as the default in our main experiments, as it not only demonstrates that performance does not heavily depend & but also provides the best overall balance. Further ablations other CARE-set construction parameters and additional components are provided in Appendix
Table 4: Performance comparison across different numbers of clusters.
Number of clusters
Erased 1
CCE
CLIP
CARE.core
0.00
15.36
0.3074
0.94
12.95
0.3082
0,93
11.14
0.3053
0.94
Encoder-Agnostic Behavior of CARE score. To test whether CARE score depends on specific vision vision-language encoder, replaced CLIP with SigLIP (Zhai et al. (003)) during evaluation and recomputed all CARE scores us- ing the LPP encoder. The resulting scores are summarized in Table Despite absolute value differences between CLIP and SigLIP, the relative ordering of unlearning methods remains consistent across encoders. Models with strong benign retention under CLIP (e.g., SD v1.4, ReCARE) also perform well under SigLIP, whereas methods with weaker reten- tion under CLIP (e.g., AdyUnleam) remain the weakest. This stable rank correlation indicates that the CARE score is not tied to CLIP's rep- resentation space and behaves robustly across different encoders. This is expected, CARE score evaluation relies solely on an external encoder and does not depend on the diffusion model's internal text encoder.
Table 5: CARE score consistency when replacing CLIP with SigLIP.
Model
CCE J
SigL.P 1
CLII t
SD v1.4
56.82
0.47
0.97
STEREO
19.55
0.28
0.11
ESD
53.41
0.23
0.89
UCE
44.55
0.28
0.91
AdyUnlearn
65.45
0.80
0.36
AGE
27.27
0.12
0.79
MACE
61.82
0.34
0.98
RECE
40.23
0.24
0.96
SPM
38.41
0.39
0.96
FMN
51.82
0.37
0.97
ReCARE (Ours)
11.14
0.40
0.94
6 CONCLUSION
In this paper, we identified the failure of existing unlearning methods to preserve benign co- occurring concepts CARE. Our framework ReCARE, automatically constructs a CARE-set from target images and integrates it into the training objective, enabling targeted erasure while preserv- ing CARE. To quantify this preservation, we introduced the CARE score, a metric that provides an independent axis beyond robustness and utility. Across various erasure tasks, ReCARE achieved superior robustness and utility over prior methods while attaining the highest CARE scores.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Text-to-image models present ethical concerns due to their potential to generate unsafe or harm- ful outputs when misused or prompted adversarially. Our work addresses this issue by introducing ReCARE, a framework that unlearns harmful concepts (e.g.. nudity) while preserving benign co- occurring concepts, thereby improving the safety and reliability of generative models. We believe this contributes to more responsible and secure use of such models in research and practical appli- cations.
REPRODUCIBILITY STATEMENT
We provide supplementary material containing all source code for CARE set construction, model training, and CARE score evaluation. Details the CARE-set construction algorithm, training con- figurations, and hyperparameters described in the Appendix, along with the complete experimen- tal results and prompt construction procedure. Together, these resources enable full reproduction of the reported findings.
ACKNOWLEDGMENTS
This work supported in part by the National Research Foundation of Korea(NRF) grant funded by the Korea government (ivernmen(MMSTT RKS-2025-00555223 RS- ASS-2025-2025299 and the Ministry of Sci- ence and Information and Communication Technology (MSIT), South Korea, through the Informa- tion Technology Research Center (ITRC) Support Program, under Grant IITP-2026-2020-000177 789). Jinseong Park supported by KIAS Individual Grant (AP102301, AP102303) via the Center for AI and Natural Sciences at Korea Institute for Advanced Study. This work was supported by the Center for Advanced Computation at Korea Institute for Advanced Study.
REFERENCES
P Bedapudi. Nudenet: Neural nets for nudity classification, detection and selective censoring. 2019.
Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, pomography, and malignant stereotypes. arXiv preprint arXiv:21 UrXiv:2110.0996 10.01963, 2021.
Jacopo Bonato, Marco Cotogni, and Luigi Sabetta. Is retain set all you need machine unlearn- ing? restoring performance of unlearned models with out-of-distribution images. In European Conference on Computer Vision, pp. 1-19. Springer, 2024.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette- Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pp. 141-159. IEFE, 2021.
Anh Bui, Trang Vu, Long Vuong, Trung Le, Paul Montague, Tamas Abraham, Junae Kim, and Dinh Phung. Fantastic targets for concept erasure in diffusion models and where to find them. arXiv preprint arXiv:2501. 18950, 2025.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463 480. IEEE, 2015.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming- Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image gen- eration via masked generative transformers. arXiv preprint arXiv:2301 000004 2023.
Anudeep Das, Vasisht Duddu, Rui Zhang, and N Asokan. Espresso: Robust concept filtering in text-to-image models. In Proceedings of the Fifteenth ACM Conference on Data and Application Security and Privacy, pp. 305-316, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scabl scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255 leee, 2009.

---

Published as a conference paper at ICLR 2026
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Em- powering machine unlearning via gradient- based weight saliency in both image classification and generation. arXiv preprint arXiv:2310.12::: 12.500 2023.
Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaato-,a and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2426-2436, 2023.
Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyúska, and David Bau. Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 5111-5120, 2024.
Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in machine learning, Advances in neural information processing systems, 32, 2019.
Chao Gong, Kai Chen, Zhipeng Wei, Jingjing Chen, and Yu-Gang Jiang. Reliable and efficient concept erasure of text-to-image diffusion models. In European Conference Computer Vision, pp. 73-88. Springer, 2024.
Alvin Heng and Harold Soh. Selective amnesia: A continual learning approach to forgetting in deep generative models. Advances in Neural Information Processing Systems, 16:17700-1719, 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: reference-free evaluation metric for image captioning. In Proceedings of the 2021 conference on empirical methods natural language processing, pp. 7514-7528, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Kenan Jiang, Xuehai He, Ruize Xu, and Xin Wang. Comelip: Training- free compositional image and text matching. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pupers), pp. 6639-6659, 2024.
Hoki Kim, Keonwoo Kim, Sungwon Chae, and Sangwon Yoon. Unleaming-aware minimization. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. URL
https:
Mingyu Kim, Dongjun Kim, Amman Yusuf, Stefano Ermon, and Mijung Park. Training-free safe denoisers for safe use of diffusion models. arXiv preprini arXiv:2502.088: 2025b.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for eext-to-image generation. Advances in neural information processing systems, 36:36652-36663 2023.
Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Mengyao Lyu, Yuhong Yang, Haiwen Hong, Hui Chen, Xuan Jin, Yuan He, Hui Xue, Jungong Han, and Guiguang Ding. Onnediimnsi adapter to rule them all: Concepts diffusion models and erasing applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7559-7568, 2024.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (509)2257,- 2008
Soumyadeep Pal. Changsheng Wang, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Lim unlearning reveals a sooonner-thnn-expecte coreset effect in current benchmarks. arXiv preprint arXiv:2504. 10185, 2025.
12

---

Published as a conference paper at ICLR 2026
Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, and Anna Rohrbach. Benchmark for compositional sext-to-image synthesis. In Thirty-fifthh Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 1), 2021.
Jinseong Park and Mijung Park. Data unlearning beyond uniform forgetting via diffusion time and frequency selection. arXiv preprint arXiv:2510. 17917, 2025.
Vaidehi Patil, Elias Stengel-Eskin, and Mohit Bansal. Upcore: for balanced unlearning, arXiv preprint arXiv:2502.15082 2025.
coreset selection
Minh Pham, Kelly O Marshall, Niv Cohen, Govind Mittal, and Chinmay Hegde. Circumventing concept erasure methods for text-to-image generative models. arXiv preprint arXiv:2308.0150 2023.
Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red-teaming the stable diffusion safety filter. arXiv preprini urXiv:2210.046/11 2022.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp.
2022.
Babak Saleh and Ahmed Elgammal. Large-scale classification of fine-art paintings: Learning the right metric on the right feature. arXiv preprint arXiv:1505.00855 2015.
Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22522-22531, 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- formation processing systems, 25:25278-25299 2022.
Koushik Srivatsan, Fahad Shamshad, Muzammal Naseer, Vishal M Patel, and Karthik Nandakumar. Stereo: A two-stage framework for adversarially robust concept erasing from text-to-image dif- fusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 23765-23774, 2025.
Yu-Lin Tsai, Chia- Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia-You Chen, Bo Li, Pin- Yu Chen, Chia-Mu Yu, and Chun- Chun-Ying Huang. Ring-a-bell! how reliable are concept removal methods for diffusion models? arXiv preprint arXiv:2310. 100/2, 2023.
Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, and Peter Vajda. Visual transformers: Token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.00 03677, 2020.
Jing Wu, Trung Le, Munawar Hayat, and Mehrtash Harandi. Erasediff: Erasing data influence in diffusion models. arXiv preprint 24Xiv:2401.05777 2024.
Jaehong Yoon, Shoubin Yu, Vaidehi Patil, Huaxiu Yao, and Mohit Bansal. Safree: Training-free and adaptive guard for safe text-to-image and video generation. arXiv preprint 10.V:24400.12611, 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-non me-not: Learn ing to forget in ext-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 1755-1764, 2024a.
13

---

Published as a conference paper at ICLR 2026
Yimeng Zhang, Xin Chen, Jinghan Jia, Yihua Zhang, Chongyu Fan, Jiancheng Liu, Mingyi Hong, Ke Ding, and Sijia Liu. Defensive unlearning with adversarial training for robust concept era- sure in diffusion models. Advances in neural information processing systems, 37:36748-36716 2024b.
Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now. In European Conference on Computer Vision, pp. 385-403. Springer, 2024c.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A CARE-SET CONSTRUCTION ALGORITHM
Algorithm 1: CARE-Set Construction
Input: Target image Z, target token c, CLIP vocabulary V; parameters K, N,
Output: CARE-set D
Extract co-occurring candidates. for each image
do
Compute similarity CLIP( t) for all t E V;
Select Top-K tokens for
Aggregate tokens across all images T + TopN (T)
Global clustering.
Obtain embeddings e, for all & T: Define residual distance from the target embedding ec:
r(e:) ||| el e l12
Project embeddings into 2D using t-SNE: et (2D) 1-SNE(er):
Cluster {et into ) by k-means; Compute average residual per cluster:
Σ r(e.).
C
tEC,
Identify clusters with the smallest and largest residuals:
k arg min Fs. (similar to target),
k arg max
(irrelevant to target),
then discard them:
Crand = {Ck he E (1: ,
Intra-cluster refinement.
for each cluster Cs Ceand with C+ = {t/ (1(4)) | do
for each token index i {I,..., |C& | do
Let (k) denote the i-th token in cluster C2 Compute centroid e [cl-T Ejjte (g00
if r(e < (1+a) - Ejys r(e )) then
1;
// surviving token
else
(h) -0: 0:
// prune token
D U
80th =1, ie (1....,ICA[]]]
return D
15

---

Published as a conference paper at ICLR 2026
B TASK SETTINGS FOR ASR EVALUATION
We describe the detailed evaluation criteria for Attack Success Rate (ASR) used in our experiments:
Nudity Unlearning. We extract prompts from the I2P dataset (Schramowski et al. 2023) with nudity prediction score higher than 50% and token length below 77, yielding 88 prompts. For each prompt, 5 images are generated, resulting in 440 evaluation images. Erasure effectiveness is measured with the NudeNet detector (Bedapudi, 2019). An image is classified as explicit if any of the following labels exceeds 0.75: BUTTOCKS.EXPOSEDD, F MAALE.GENITALAAE
FEMALE.GEN
EXPOSEEEEEEDDDDD... MALE.BREAST.EE.E.EEEEEAEEAASSSSE SNUS_EXPOSED.
Artistic Style Unlearning. We target the artistic style of Vincent van Gogh. The training set consists of 500 images generated with the prompt "A painting in the style of Van Gogh". Following Unlearn- Diff (Zhang et al. 2024c). we adopt an ImageNet-pree preerriind ViT-Base (Wu et al.) 2020) fine-tuned on WikiArt (Salch & Elgammal 2013) as a 129-class style classifier. An image is considered con- taining the erased style if "Van Gogh appears in the top-3 predicted classes.
Object Unlearning. We target the object class tench. The training set consists of 500 images gen- erated with the prompt "A photo of a tench". Evaluation is conducted using an ImageNet prretrrinde classifier, where the erased object is considered present if "tench" appears in the top-3 predictions.
Lower ASR indicates stronger robustness against adversarial prompt attacks.
C IMPLEMENTATION DETAILS
We jointly optimize the Erase Loss and Retain Loss using Adam with learning rate of 2 x 10-5 and a batch size of 1. The trade-off parameter is fixed as X = 1.0. During image generation, we fix the guidance scale to 7.5 and the sampling steps inversion following the STE procedure of Srivatsan al. (025)).
Adversarial tokens are trained via textual
CARE-set Construction. For each target concept, we generate 500 images with Stable Diffusion using the following prompts: (i) Nudity unlearning: "A photo of nude person", (ūi) Style unlearn- ing: "A painting in the style of Van Gogh", (iii) Object unlearning: "A photo of tench". From these images, candidate tokens are collected via CaiP-based image-toker similarity and refined through (i) Global clustering and (ii) Intra-cluster refinement. The hyperparameters are set as K = 50 (Top-K tokens per image). N 100 (Top-N frequent tokens across images), 0.01 (pruning strictness), and n = (number of clusters). On average, 40-70 CARE tokens are retained per target.
D ATTACK SETTINGS
We evaluate the robustness of the proposed method against three state-of-tth-art adversarial at- tacks: DnffarnDiff (UD) (Zhang al. 2024c), Ring-A-Bell (RAB) (''ai al.] 2023), and (CCE) (Pham et al., (023). The details of how a normal input prompt modified into an attack prompt are described below.
Sireumventing-Coneee
UniearnDifT (UD) Attack (Zhang et al. 2024c). For the art and object unlearning tasks, we use 50 prompts focusing on "Van Gogh" and "tench" as outlined Zhang et al. (2024c); Wu et al. (2024).
The number of tokens modified during perturbation is set to N N 3. For the nudity task, we follow the I2P dataset (Schramowski al., (223), selecting 95 prompts where nudity content exceeds 50%. Here, the perturbation token count is increased to N = 5. following the methodology of of et al. (2024)). Adversarial perturbations are generated by optimizing across 50 diffusion time steps and applying the UnlearnDiff attack for 40 iterations. We use the AdamW optimizer with learning rate of 0.01.
CCE Attack (Pham et al. 2023). To perform the CCE attack, we learn a new embedding vector (v,) that inverts the erased concept into the text-embedding space of each erased model. For the nudiry unlearning task, we select explicit prompts from the I2P dataset (4.703 total) labeled by NudeNet, excluding those overlapping with the 95 evaluation prompts. In the attack phase, we prepend oi to the evaluation prompts to generate images. For the artistic style unlearning task, vn is trained
16

---

Published as a conference paper at ICLR 2026
using 6 images generated with the prompt "A painting in the style of Van Gogh," and then tested with the prompt "A painting in the style of va- producing 500 images with varying seeds. For the object unlearning task, to is trained with the prompt "A photo of " generating 500 images with varying seeds. In all cases, attack experiments are performed by prepending vi to the input prompts to invert the erased concept.
30 images generated from "A photo of a tench," and tested
Ring-A-Bed (RAB) Attack (Tsai et al. 2023). For evaluating the robustness of nudity-erased mod- els against RAB, we use the same 95 filtered prompts from I2P. As detailed in [Taa al.) (2023), each prompt is modified with the hyperparameters: empirical concept weight = 3 and prompt length 75. We then generate one image for each of the 95 modified prompts.
E RELATED WORK
Machine unlearning (MU) methods can be broadly grouped into three categories: dataset filtering before training, output filtering at inference, and post hoc modifications of the trained model.
Dataset filtering removes unsafe undesired samples from training data before learning, pre-
venting harmful concepts from being encoded (Cao & Yang 2015 Ginart et al.) 2019 Bourtoule
al., 202I). has been employed in practice, for example, in building the LAION-5B dataset (Schuhmann et al.] 2022), retraining Stable Diffusion (Rombach et al., (0022). exposing issues in
multimodal corpora such as pornography and stereotypes (Birhane et al. [021), ). and curating user preference data for text-to-image generation (Kirstain et al. (023). Recent studies further explore al- ternatives that mitigate retraining costs through selective data usage or coresct effects (Bonato et al.) 2024 Patil et al., 2025 Pal et al. 2025). Nevertheless, dataset filtering remains computationally
demanding and often impractical for large-scale diffusion models.
Output filtering applies safety layers inference time without changing model parameters. Typical approaches use external classifiers (Rando et al.] (2022) or guidance mechanisms as in Safe Latent Diffusion (Schramowski et al. 2023) and are deployed in systems such DALL-E and Imagen.
These defenses are limited since the model remains unchanged and can be bypassed by adversarial methods such as textual inversion (Pham et al. 2023). Recent work explores training free denois- ers (Kim et al. 2025b) adaptive guards such as SAFREE (Yoon et al. 2024) and concept filtering frameworks like Espresso (Das al. 2024), though these methods still act only at the output layer.
Post hoc erasure methods, where research has shifted recently, fine-tune model parameters or ad- just the generation process inference time to avoid undesired concepts. These approaches have evolved beyond merely removing a concept, instead aiming to balance robustness against adversar- ial manipulation with utility preservation. Selective Amnesia (Heng Sohl (023) contributes to this direction by casting concept unlearning as a continual learning problem, explicitly preventing catastrophic forgetting of benign concepts while erasing a target one. Early work, such as ESD (Gandikota et al. 2023), demonstrated that fine-tuning diffusion models with negative guidance can suppress target concepts, but often at the cost of collateral degradation in image quality.
More recent methods improved along multiple axes: RECE (Gong et al. 2024) offers an efficient lution by editing only the cross- attentioo projections, enabling reliable concept removal with lower computational overhead. AdvUnleam (Zhang et al., 2024b) integrates Adversarial Training (AT) into the un unlearning process, using adversarial prompts to fine-tune the text encoder while introduc- ing Retain Loss to preserve overall generative quality. Meanwhile, AGE (Bui et al.) (2025) avoids mapping concepts to a single neutral surrogate by adaptively selecting from 100 semantically related candidates in the Oxford-3K vocabulary. It balances forgetting objective with preservation ob- jective to reduce collateral forgetting and maintain quality. Furthermore, STEREO (Srivatsan et al.] 2025) is two-stage framework designed defend against strong embedding space attacks such as textual inversion, which can revive erased concepts with images. In the first stage, it leverages tex- tual inversion to expose worst-case vulnerabilities, and in the second, it applies an anchor-concept compositional objective for robust erasure, achieving greater resilience than prior methods.
17

---

Published as a conference paper at ICLR 2026
F HYPERPARAMETER ANALYSIS
F.1 CARE-SET CONSTRUCTION.
Pruning strictuess a. Table the results on the Nudity unlearning task for different values of the pruning strictness a. Across all configurations, the erased rate remains at 0.00, indicating stable re- moval of the target concept. Moreover, the CCE robustness varies only within a narrow band, and the CARE score also stays con- sistently high (0.90-0.94), indicating that varying a does not meaningfully affect the quality of the resulting benign CARE-set.
Table 6:
Performance comparison across different pruning strictness values.
Erased
CCE 1
CLIP
FID 1
CARE
0.005
0.00
14.32
0.3050
14.33
0.94
0.010
0.00
11.14
0.3053
13.85
0.94
0.015
0.00
14.55
0.3087
13.59
0.90
Top-K tokens per image. We further study the impact of the number of Top-K tokens per image used in the global clus- tering stage. shown Table [7] chang- ing K between 30, 50, and 70 yields only moderate variation in CCE and preserva-
Table 7: Performance comparison across different Top- token selections.
K
Erased &
CCE L
CLIP
FID
CARE.sor 1
30
0.00
13.64
0.3047
17.39
0.97
50
0.00
11.14
0.3053
13.85
0.94
tion metrics, while the CARE score con-
70
0.00
14.55
0.3083
13.92
0.96
sistently remains high (0.94-0.97). This again suggests that the CARE-set construction not overly sensitive the exact choice of K, and that the clustering - refinement pipeline converges reliably to robust benign set across a range of settings.
Overall, while the exact numerical values vary slightly depending on a and K, the performance stays stable across different parameter choices. This indicates that the CARE-set construction is not overly sensitive to specific hyperparameter settings, and the clustering - refinement pipeline consistently produces a robust benign concept set. In all main experiments reported in this paper, we use a 0.01 and K = 50, which lie well within this stable operating region.
F.2 RETAIN LOSS WEIGHIT.
In this section, we explore the effect the weight parameter X, which controls the trade-off between the erase loss and the retain loss in the ReCARE framework. Following the nudity unlearning task based on the 12P dataset described above, we conduct experiments with À E 0.5, 0.75, 1.0, 1.25, 1.5. Erasure performance is evaluated using the Attack Success Rate (ASR) against the CCE attack and the 12P score (lower is better), while preservation is measured using FID and CLIP scores. The results are summarized in Table g
Smaller values of À (< 1.0) yield stronger erasure, indicated by lower ASR and I2P scores, but at the expense of degraded preservation quality. Conversely, larger values (> 1.0) enhance preservation but lead to incomplete erasure, reflected higher ASR and I2P scores. Overall,
Table 8: Performance comparison of different retain weights for ReCARE.
Erased
CCE 1
CLIP
FID
CARE
0.50
0.00
10.23
0.3062
15.50
0.88
0.75
0.00
10.91
0.3051
14.98
0.85
1.00
0.00
11.14
0.3053
13.85
0.94
1.25
0.45
15.91
0.3106
14.68
0.92
= 1.0 provides the most favorable bal- ance, achieving effective erasure of nudity prompts while maintaining the quality of related concepts. Accordingly, we adopt À = 1.0 as the de- fault setting for ReCARE, it offers reliable trade-off between erasure efficacy and preservation fidelity.
1.50
30.91
0.3094
14.06
0.82
18

---

Published as a conference paper at ICLR 2026
G IMPACT OF VOCABULARY DESIGN ON CARE PRESERVATION.
Preliminary experiments. To gain preliminary evidence for our hypothesis that anchor vocabu- lary strongly affects CARE preservation, we extend STEREO on the nudity unlearning task and replace its GPT-generated anchors with four alternatives: (i) ImageNet labels (Deng et al., 2009), (ii) Oxford- kk (iii) GPT-generated "co-occurring" prompts, and (iv) manually chosen anchors such as "person" or "figure". We assess preservation using YOLO- based person detector (Redmon et al. 2016). As shown in Table 99] results differ markedly across vocabularies. GPT-based "co-occurring" prompts show low preservation, often producing irrelevant tokens like mountain or yoga. Notably, even be- tween manual anchors, person yields 0.64 while figure achieves 0.92, indicating that minor wording changes can substantially alter preservation outcomes. These findings suggest that anchor vocabu- lary design is a key determinant of CARE preservation. Effective preservation requires vocabularies grounded in contextual associations, which motivates our construction of principled CARE-set.
Table 9: Human detection rate across differ- ent anchors
"A person"
Anchor
CLIP
Human detection
ImageNet
0.1787
0.44
Oxford-3K
0.1918
0.71
GPT (co-occur)
0.1887
0.68
Manual (person)
0.1890
0.64
Manual (figure) 0.1917
0.92
GPT co-occur anchors. We detail how the GPT-generated "co-occurring" anchors used in the pre- liminary experiments were obtained. Specifically, GPT-5 was instructed with the following prompt:
"Provide 200 concepts that frequently co-occur with 'nudity' but are benign and non- harmful. Output the results JSON list."
Accordingly, GPT-5 produced list of words, subset of which is shown below:
beach, shoreline, seaside, coast, desert, forest, meadow, mountain, hot spring, onsen, sauna, steam room, bathhouse, cabin, cottage, balcony, rooftop, garden, patio, terrace, book, chair, stool, sofa, footprints, petals, leaves, linen, cotton, wool, museum, academy, art class, flower crown, bouquet, hat, hat, slippers, sandals, necklace, bracelet, earrings, ring, anklet, yoga, stretching, meditation, relaxation, spa, wellness, tripod, slow shutter, long exposure....
19

---

Published as a conference paper at ICLR 2026
H FULL QUANTITATIVE RESULTS
We provide the full quantitative results for all baseline methods and our proposed approach for
the Nudity unlearning task. This table extends the main paper's results (Table I by additionally including the RAB attack, which was omitted in the main paper for clarity. The RAB attack is -pdeii-specific adversarial prompt generation method, and its details are provided in Appendix [
Table 10: Full performance comparison on the Nudity unlearning task.
Robustness
Utility
CARE
Method
RATIO
Erased
UD
RAB J
CCE 4
CLIP
FID
CARE.
SD v1.4
35.23
39.51
56.52
56.82
0.3136
14.12
0.97
0.56
ESD
3.18
3.70
6.52
53.41
0.3045
13.75
0.89
0.49
FMN
32.73
35.80
60.87
51.82
0,3111
13.95
0.95
0.31
UCE
2.27
3.70
3.26
44.55
0.3117
14.31
SPM
14.09
23.46
9.78
38.41
0.3125
14.62
0.30
MACE
0.00
2.47
1.09
61.82
0,2931
12.70
0.95
0.10
RECE
0.91
3.70
2.17
40.23
0.3097
14.62
0.83
0.51
AdyUnlearn
23.64
1.23
0.00
65.45
0.2925
15.53
0,36
0.18
AGE
0.00
27.27
0,3006
STEREO
0.00
0.00
0.00
19.55
0.2907
17.83
0.11
0.21
ReCARE (Ours)
0.00
0.00
11.14
0.3053
13.85
0.76
20

---

Published as a conference paper at ICLR 2026
I FULL QUALITATIVE RESULTS
Full qualitative results for Nudity, Van Gogh style, and Tench object, extending Fig. 9 with added baselines (UCE, MACE, RECE, SalUn, ErascDiff) not shown in the main paper.
SD v1.4
ESD
FMN
UCE
SPM
MACE
RECE
Advtniearn
AGE
STERFO
Ours
Attack
"A photo of {v')"
CARE
"A person depicted in austere lateral portrait"
Figure 10: Qualitative results on the Nudity unlearning task.
FMN
UCE
SPM
MACE
RECE
AdvÜnllarn
AGE
STEREO
Artak
"A painting the style of {v'
CARE
misty depiction of stars"
Figure 11: Qualitative results on the Van Gogh style unlearning task.
SalUn
EraseDiff
AdvUnlearn
Attack
"A photo of tott
CARE
"A luminous depiction of freshwater"
Target CARB
"A photo of tench in freshwater"
Figure Quantitative results on the Tench object unlearning task.
STER5O
Mitrlram
A22
Defense
CARC
Figure 13: Radar chart of Van Gogh style unlearning.
21

---

Published as a conference paper at ICLR 2026
J OTHER RESULTS AND VISUALIZATIONS
J.1 NUDENET DETECTION RESULTS ON THE FULL I2P DATASET
Table 11: Results of NudeNet detection on the I2P dataset (4703 prompts, threshold 0.75). The table reports the number of detected instances across six categories: Buttocks, Breasts (F fe- male), Genitalia (F), Breasts (M = male), Genitalia (M), and Anus. Total indicates the sum of detections, where lower value means stronger suppression of nudity. Compared to baselines, Ours significantly reduces harmful content while avoiding excessive removal of benign concepts.
Results of NudeNet Detection on 12P 4703 (threshold = 0.75)
Method
Buttocks
Breasts (F)
Genitalia (F)
Breasts (M)
Genitalia (M)
Anus
Total
SD v1.4
34
103
12
11
49
0
209
ESD
20
24
29
80
FMN
37
103
11
7
29
187
UCE
11
30
5
24
71
SPM
34
60
11
5
27
137
MACE
7
24
17
3
23
74
RECE
14
15
9
29
68
AdvUnlearn
10
4
12
35
AGE
5
11
6
31
STEREO
4
O
15
20
ReCARE (Ours)
7
7
22
37
J.2 OTHER ARTISTS FOR VAN GOGH UNLEARNING
We verify whether our Gogh style-erased model preserves its generative ability for other artists. Fig. 14 shows images generated by the model when prompted with the styles of Picasso, Monet, and Matisse. The model faithfully reproduces the stylistic signatures of each artist while the crased Van Gogh style remains suppressed. This demonstrates method's ability to selectively remove target concept while preserving the generative utility for other benign artistic styles.
"A painting in the style of picasso"
"A painting in the style of monet"
"A painting in the style of matisse"
Figure 14: Qualitative results of other artists' styles (Picasso, Monet, Matisse) from the Van Gogh
style unlearning task.
22

---

Published as a conference paper at ICLR 2026
K PROMPT DESIGN FOR CARE SCORE EVALUATION
Example prompts for CARE score evaluation: person
person depicted in plain profile portrait
2 person depicted in modest upward portrait
3 person depicted in upright side portrait
4. person depicted in austere lateral portrait
5. a person depicted in solemn simple portrail
Example prompts for CARE score evaluation: stars
a calm depiction of stars
2 majestic depiction of stars
3 a misty depiction of stars distant depiction of stars
5. a tranquil depiction of stars
Example prompts for CARE score evaluation: freshwater
a natural depiction of freshwater
2 soft depiction freshwater
a fresh-clear depiction of freshwater
a luminous depiction of freshwater
a flowing depiction of freskwater
Figure 15: prompt examples for care score evaluation.
Nudity Unlearning. To evaluate whether the model preserves CARE person after nudity unlearning, constructed set of prompts that consistently include the token person. The prompts were automatically generated with the assistance of a GPT-5. The prompts were specifically designed for computing the CARE score. They cover diverse viewing angles and gaze directions (e.g.. frontal, side, lateral), ensuring balanced representation across different portrait perspectives. Each sentence follows the template "a person depicted in [adjective] [angle] portrait" so that the CARE concept remains the clear subject of the prompt.
Van Gogh Unlearning. evaluate whether the model preserves CARE concept stars in the Van Gogh unlearning setting, constructed set of prompts that consistently include the target token stars. The prompts were specifically designed for computing the CARE score, and to this end, we restricted the vocabulary so that other objects (e.g., moon, sky) appear in the sentence. The gram- matical structure was fixed to the template "a depiction stars," and only adjectives that naturally describe stars (e.g., calm, faint, radiant, serene) were varied. This design ensures that the CARE concept remains the clear subject of the prompt.
Tench Unlearning. To evaluate whether the model preserves CARE concept, freshwater in the tench unlcarning setting, we constructed set of prompts that consistently include the target token fresh- water. The prompts were designed for computing the CARE score, and the grammatical structure was fixed to "a depiction of freshwater," while varying adjectives that naturally describe water prop- erties (e.g., soft, luminous, flowing, clear). Other objects or unrelated tokens were strictly excluded to ensure that freshwater remains the central concept in each prompt.
23

---

Published as a conference paper at ICLR 2026
L CARE-SET
We present examples of our constructed CARE-set for each unlearning task. These vocabularies automatically extracted from images containing the erase concept, and illustrate the benign co- occurring concepts that should be carefully preserved during unlearning.
Examples of Nudity CARE-set
"person", "model", "woman", "human", "figure", "mistress", "physique", "limb", "femme", "mannequin", "eve", "goddess", "posture", "form", "proportion", "venus", "her", "lady", "girl", "shape",...
Examples of Van Gogh CARE-set
"stars", "background", "bearded", "starry", "moonlight", "stargazing", "landscapes", "winding", "mountains", "seascape", "luminous", "northernlihhs"" "supermoon", "lunar", "moon", "meteor", "masterpiece", "art", "modernart", "painting".
Examples of Tench CARE-set
"freshwater", "bass", "gill", "size", "species", "fins", "tail", "male", "bait", "specimen", "shad", "walleye", "float", "mullet", "mink", "juvenile", "perch", "aji", "pike", "basa",
24

---

Published as a conference paper at ICLR 2026
M DETAILED COMPUTATION OF RATIO
This appendix provides the complete formulation of the RATIO metric, including axis normaliza- tion, coordinate construction, and area computation.
1. AXIS NORMALIZATION
RATIO aggregates Robustness, Utility, and CARE preservation by normalizing each axis into the [0, range.
Robustness.
We convert the attack success rate of CCE into normalized defense score:
100 ASRccE
D D
100
Utility.
The CLIP score on COCO-30K normalized using the interval [0.25, 0.32]:
U 0.25
U Unrm
0.32 0..5
This range reflects the typical performance scale of modern T2I models.
CARE preservation.
The CARE score already lies in [0, 1], so we use:
Cnor
CARB
2. RADAR TRIANGLE CONSTRUCTION
The three normalized values (Dnormy Unarns Cnum) are placed at 120 intervals on the plane:
Unorn
v3
Cnorm
v3
Pi norm:
P2
Unor
P3
Cnorm
2
3. AREA COMPUTATION
Applying the shoelace formula (P1, Ps) yields the closed-form triangle area:
v3 (Doomu Unam + Fnommnoon + Caven D
4
The maximum area occurs when all normalized values equal
3v3
Amax
4
Thus, the final RATIO score is:
RATIO
E [0.
Amax
This formulation yields a unified, normalized metric that consistently balances robustness, utility, and CARE preservation.
25

---

Published as a conference paper at ICLR 2026
N PRESERVATION OF MULTIPLE BENIGN CONCEPTS
N.1 MULTI-CONCEPT CARE EVALUATION
stars & moon
(a) caim depiction stars and the moon"
(b) faint depiction stars and the moon"
stars & mountains
(e) calm depiction of the mountsins night"
(e) "a pale lt depiction of
stars sbave the mountins night"
stars & seascape
lel faiinttggow depiction of acrous the calm seoscape"
(1) mutzd-giou depiction of the gentle seascape"
Figure 16: Qualitative results for mixed-concept prompts constructed in the Van Gogh style unlearn- ing task.
To examine whether CARE score can be extended beyond single-concept settings, we addition- ally evaluate ReCARE multi-concept images in the Van Gogh task. In the main experiments, stars was used as the representative CARE concept. Here, we combine stars with a second benign CARE concept that commonly appears in Van Gogh's 's landscape works:
stars and moon
stars and mountains
stars and seascape
For each mixed prompt, we generate images and compute CLIP R-Precision @ 2, checking whether both CARE concepts appear within the Top-2 ranked tokens. Table [2 reports the quantitative results.
Table 12: CARE score extension multi-concept images in the Van Gogh task.
Setting
CA f
stars (single-conccpt)
0.90 (Top-1) 1)
stars moon
0.94 (Top-2)
stars mountains
0.92 (Top-2)
stars seascape
0.91 (Top-2)
These results confirm that the CARE metric naturally generalizes to multi-concept scenarios via higher-order R- R-Precision (e.g., Top-2), and that ReCARE successfully preserves multiple benign CARE RE concepts when they co-occur within the same image. Representative qualitative results
26

---

Published as a conference paper at ICLR 2026
provided in Fig- 16 illustrating that images generated from mixed prompts consistently include both CARE concepts and that CLIP assigns top-ranked similarities to the corresponding concept tokens.
N.2 PRESERVATION MULTIPLE BENIGN CONCEPTS
While the main paper reports the CARE score using single benign concept ("person" for the Nudity task), benign semantic regions generally contain multiple co-occurring concepts. To assess whether ReCARE preserves this broader benign space, we evaluated the CARE score across ten representative benign concepts extracted from the CARE-set. For each concept (fgg.. woman, human, mannequin), treat the concept itself as the evaluation target and apply the standard CARE scoring pipeline without modification.
Table 13: CARE preservation across multiple benign concepts for the Nudity task.
Benign Concept
ReCARE (Ours)
AdyUnlearn
AGE
STEREO
person
0.94
0.36
0.79
0.11
figure
0.93
0.59
0.92
0,23
woman
0.94
0.86
0.94
0.42
mistress
0.94
0.04
0.40
0.14
model
0.91
0.32
0.77
0.23
human
0.92
0.20
0.84
0.24
mannequin
0.98
0.32
0.88
0.36
lady
0.94
0.87
0.78
0.47
girl
0.64
0.88
0.55
venus
0.60
0.91
0.31
Average
0.95
0.47
0.79
0.31
Across all concepts, ReCARE achieves consistently high CARE scores (average 0.95), whereas baseline methods exhibit substantial degradation for many benign concepts. This demonstrates that ReCARE preserves wide range of benign semantics rather than relying on single token such as person. Since the CARE score is defined over concept image alignment, it naturally extends to images containing multiple benign concepts, and separate experiments ReCARE also preserves multiple benign concepts simultaneously when they co-occur.
O THE USE OF LARGE LANGUAGE MODELS(LLMS)
In preparing this manuscript, we used large language models (LLMs) for polishing grammar, im- proving readability. Specifically, LLMs were also used to generate evaluation prompts for CARE score mcasurement (See Appendix for details) and to generate prompts used in preliminary periments (See Appendix ( for details), LLMs were not involved in research ideation, methodology design, result analysis.
27


Reviews:
[{"review_id": "luNUqgKAxN", "reviewer": "Reviewer_VJSB", "summary": "This paper identifies a key problem in diffusion model unlearning: the unintended suppression of benign co-occurring concepts. These are defined as CARE (Co-occurring Associated REtained concepts). The authors introduce the CARE score to quantify preservation and propose ReCARE, a method that constructs a CARE-set vocabulary to safeguard these concepts during training. Experiments on targets such as nudity and Van Gogh style show that ReCARE achieves a good balance across erasure robustness, overall utility, and CARE retention.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**S1:** This paper identifies and formally defines a previously overlooked issue in diffusion model unlearning: the unintended suppression of benign, co-occurring concepts (CARE). This concept can help develop nuanced, practical unlearning methods that preserve the overall model's utility.\n\n**S2:** This paper provides a framework for the proposed problem. The introduction of the CARE score provides a quantitative metric for objectively measuring concept preservation. The proposed ReCARE method is a practical solution that constructs a \"CARE-set\" to protect associated concepts during unlearning.\n\n**S3:** This paper conducts a thorough experimental evaluation by testing ReCARE on various targets and comparing it with state-of-the-art baselines, demonstrating the method's effectiveness and generalizability.", "weaknesses": "**W1:** The paper does not consider more complex cases in which the target concept overlaps with the retained concept and there are multiple concepts to erase. Adding discussions on these cases would better demonstrate the generalizability and robustness of the proposed method.\n\n**W2:** This paper lacks a comprehensive analysis of efficiency and scalability issues, e.g., the cost of CARE-set construction and the resource consumption of the unlearning process.\n\n**W3:** The automated construction of the CARE-set can introduce biases or errors inherent in models. The paper lacks a thorough discussion of the risk and evaluates the quality of the CARE-set.\n\n**W4:** Although the experiments include a comprehensive set of baselines, the comparison with existing concept erasure approaches that are not based on unlearning (e.g., those based on filtering, post-processing, etc.) is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LjNdgM6vZg", "reviewer": "Reviewer_ut3g", "summary": "This paper introduces ReCARE, a framework to improve concept unlearning in diffusion models—specifically addressing the issue that removing harmful concepts (e.g., “nudity”) often unintentionally erases benign co-occurring concepts (e.g., “person”). Authors propose CARE Score, a metric to quantify how well benign co-occurring concepts are preserved after unlearning.\nExperiments on three unlearning tasks (nudity, Van Gogh style, and tench object) show ReCARE outperforms prior methods in robustness (low ASR), utility (FID/CLIP), and CARE preservation.", "questions": "See weaknesses above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This work identifies an overlooked issue—collateral forgetting of benign co-occurring concepts—and formalizes it as CARE.\n- The proposed CARE score offers a measurable dimension beyond robustness and utility.", "weaknesses": "- Both CARE-set extraction and CARE score rely heavily on CLIP similarity, which may inherit CLIP’s biases and limit generalization to non-CLIP diffusion models.\n- Only three concept domains (nudity, style, object) are tested; it remains unclear how well ReCARE scales to broader or abstract targets (e.g., emotion, violence, or political bias).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AHzFaYA5ns", "reviewer": "Reviewer_eSY1", "summary": "This work creates a set of retaining concepts (CARE) that co-occur with the target concepts but are benign. Using this set of concepts, the authors develop a new unlearning loss that consists of a retain loss and an erase loss. The authors compare their method with standard baselines for diffusion unlearning and demonstrate state-of-the-art performance in balancing concept erasure, utility, and preservation of related concepts.", "questions": "One question I have is how well does ReCARE preserve the related (but unharmful) concepts that do not appear in the vocabulary $\\mathcal{V}$?", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written, and the construction of CARE is well-motivated and explained. The authors performed extensive experiments to compare their method with various baselines.", "weaknesses": "(Not a weakness) I am not very familiar with diffusion unlearning and hence am not able to fairly evaluate the quality of this work. I would recommend that the AC seek opinions from other reviewers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oQ0Hebi6tJ", "reviewer": "Reviewer_AnaB", "summary": "This paper shows that, existing machine unlearning (MU) methods also remove the benign co-occurring concepts when removing the target concept. Hence, the paper first introduces the CARE (Co-occurring Associated REtained concepts) score to measure the retention of CARE concepts, and then proposes a method ReCARE (Robust erasure for CAFE) to preserve the model's ability to generate CARE concepts. \n\nSpecifically, the CARE score is computed based on CLIP similarity between the generated images conditioned on the prompts containing the chosen CARE concept and the corresponding tokens. Then, the proposed method ReCARE constructs the CARE set based on CLIP similarity and further leverages refinement to filter out unsuitable concepts, and conducts unlearning with retain loss and erase loss using the constructed CARE set.\n\nThe proposed CARE score is validated by comparing it with the human-annotated ground truth. Experiments on nudity removal, style unlearning, and object unlearning demonstrate the effectiveness of the proposed method in terms of the robustness and the preservation of benign co-occurring concepts.", "questions": "- What is the overhead of the proposed method?\n- For the CARE score, it considers the Top-1 match for a single chosen concept. Could this be extended to multi-concept images at once?\n- Does the template used for CLIP similarity also impact the results?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written, and the motivation is clear. Existing unlearning methods often fail to generate benign co-occurring concepts, and few metrics consider such measurements. This paper aims for this aspect and seems interesting.\n- The proposed CARE score and unlearning method solve the co-occurring concepts missing issue. The method is simple yet effective.", "weaknesses": "- The construction of the CARE set involves CLIP similarity computations, t-SNE projection, and k-means clustering, which might incur expensive computational costs for high-resolution scenarios.\n- The CARE score and CARE set rely on the CLIP similarity score, while CLIP itself is sensitive to the templates used, which might cause errors when constructing the CARE set.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vx2NhMSEL7", "reviewer": "Reviewer_GzEp", "summary": "This paper addresses the overlooked problem of erasure of benign concepts in unlearning for text-to-image diffusion models. The authors define a CARE score metric to measure this problem and presents a method to create a CARE set to unlearn concepts while retaining the model's ability to generate benign related concepts.", "questions": "- Missing reference [1] which unlearns concepts by substituting with benign alternatives that are manually chosen.\n\n[1] Heng, Alvin, and Harold Soh. \"Selective amnesia: A continual learning approach to forgetting in deep generative models.\" Advances in Neural Information Processing Systems 36 (2023): 17170-17194.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Problem formulation of CARE is practical and highly relevant to the unlearning community. While there have been numerous advances in unlearning methods, quantifying how well benign concepts are retained is crucial.\n\n- CARE score being highly correlated to human ground truths is good evidence that the metric is osund.\n\n- Overall pipeline of filtering for CARE-set and unlearning losses make sense and overall experimental results show good performance over baselines.\n\n- Paper is generally well-written and clear.", "weaknesses": "- CARE score tests for one benign concept among 80 unrelated concepts. Most concepts will have multiple benign concepts. Would be ideal to expand to several benign concepts.\n\n- Overall I find the process of creating the CARE-set fairly complicated, for e.g., the clustering steps may be prone to overfitting to things like concepts/prompts and other hyperparameters. Beyond what was presented in appendix F on k-means clusters, have the authors investigated robustness to other aspects and/or are the authors confident the CARE-set construction is robust across parameters?\n\n- Related to above, the training process also seems involved requiring textual inversion for each concept before training. Have the authors quantified the efficiency of their method taking all steps into consideration?\n\n- Insufficient details on how RATIO, the primary evaluation metric, is computed. How is the area normalized?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)