Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation
Authors: Anh Tuan Bui
Long Tung Vuong
Khanh Doan
Trung Le
Paul Montague
Tamas Abraham
Dinh Phung
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion models
Erasing Concepts; Trustworthy GenAI
Primary area: diffusion_based_models

Abstract: Diffusion models excel at generating visually striking content from text but can inadvertently produce undesirable or harmful content when trained on unfiltered internet data. A practical solution is to selectively removing target concepts from the model, but this may impact the remaining concepts. Prior approaches have tried to balance this by introducing a loss term to preserve neutral content or a regularization term to minimize changes in the model parameters, yet resolving this trade-off remains challenging. In this work, we propose to identify and preserving concepts most affected by parameter changes, termed as *adversarial concepts*. This approach ensures stable erasure with minimal impact on the other concepts. We demonstrate the effectiveness of our method using the Stable Diffusion model, showing that it outperforms state-of-the-art erasure methods in eliminating unwanted content while maintaining the integrity of other unrelated elements. Our code is available at \url{https://github.com/tuananhbui89/Erasing-Adversarial-Preservation}.

Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation
Anh Bui"
Long Vuong'
Khanh Doan
Trung Le Dinh Phung'
Paul Montague
Tamas Abraham
Monash University
VinAl Research
Defence Science and Technology Group, Australia
Abstract
Diffusion models excel at generating visually striking content from text but can inad- vertently produce undesirable or harmful content when trained on unfiltered internet data. A practical solution is to selectively removing target concepts from the model, but this may impact the remaining concepts. Prior approaches have tried to balance this by introducing a loss term to preserve neutral content regularization term to minimize changes in the model parameters, yet resolving this trade-off remains chal- lenging. In this work, we propose to identify and preserving concepts most affected by parameter changes, termed as adversarial concepts. This approach ensures stable erasure with minimal impact the other concepts. We demonstrate the ef- fectiveness of our method using the Stable Diffusion model, showing that outper- forms state-of-the-art erasure methods in eliminating unwanted content while main- taining the integrity of other unrelated elements. Our code is available at phttps: //github. //gptthu..com/tuaaaaaaaaa cttp://cmmtaaaaabuu88//////// ata.://sse....aaaannnnnaarraaaaa
Introduction
Recent advances in text-to-image diffusion models (Rombach et al. 2022 Ramesh et al. 2021 2022
have captured significant attention thanks to their outstanding image quality and boundless creative potential. These models undergo training on extensive internet datasets, enabling them to capture wide range of concepts, which inevitably include undesirable concepts such as racism, sexism, and violence. Hence, these models can be exploited by users to generate harmful content, contributing to the proliferation of fake news, hate speech, and disinformation (Rando al.) 2002)] Qu al. [0233 Westerlund, 2019). Removing these undesirable contents from the model 's output is thus a critical step ensuring the safety and usefulness of these models.
Addressing this challenge, several methods have been proposed to erase undesirable concepts from pretrained text-to-image models, such TIME (Orgad al. 2023), UCE (Zhang et al. 2023). Concept Ablation (Kumari et al.] (023), and ESD (Gandikota et al. 2023). Despite differing approaches, these methods reach common finding: removing even concept can significantly reduce the model's 's ability to generate other concepts. This is because large-scale generative models G: C X, such Stable Diffusion (StabilityAl] 2022), are trained on billions image-text pairs (x, c). where is an image and e is its caption, implicitly containing a set of concepts. The concept space thus vast and intricately entangled within the model's parameters, meaning no specific part of the model's weights is solely responsible for a single concept. Consequently, the removal of one concept alters the entire model's parameters, causing a decline in overall performance. To address this degradation, existing methods typically select neutral concept, such as "a photo" an empty string, as an anchor to preserve while erasing the target concept, expecting that maintaining the neutral concept should help retain other concepts as well Orgad et al. (2023). Gandikota et al. (2024).
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

While choosing neutral concept is reasonable, we argue that it is not the optimal choice and may not guarantee the preservation of the model performance. In this paper, we propose to shift the attention towards the adversarial concepts, those most affected by changes in model parameters. This approach ensures that erasing unwanted content stable and minimally impacts other concepts. To summarize, our key contributions are two-fold:
We empirically investigate the impact of unlearning the target concept on the generation of other concepts. Our findings show that erasing different target concepts affects the remaining ones in various ways. This raises the question of whether preserving neutral concept is sufficient to maintain the model's capability. We discover that the neutral concept lies in the middle of the sensitivity spectrum, whereas related concepts such as "person" and "women" are more sensitive to the target concept "nudity" than many neutral concepts. Additionally, we demonstrate that selecting the appropriate concepts to preserve significantly improves quality retention.
We propose a novel method to identify the most sensitive concepts corresponding to the concept targeted to be erased, and then preserve these sensitive concepts explicitly to maintain the model's capability. We then conduct extensive experiments that demonstrate that the proposed method consistently outperforms other approaches in various settings.
2 Background of Text-to-Image Diffusion Models
Denoising Diffusion Models:
Generative modeling is fundamental task in machine learning that
aims to approximate the true data distribution Paata from a dataset D = { Diffusion models, a recent class of generative models, have shown impressive results in generating high-resolution images (Ho et al.] 20200 Rombach et al. [2022] Ramesh et al. [222]] 2022). In a nutshell, training diffusion model invol two processes: forward diffusion process where noise gradually added the input image, and a reverse denoising diffusion process where the model tries to predict noise which is added in the forward process. More specifically, given chain of T diffusion steps zo, I1,,. 2T, the denoising process can be formulated as follows: Pe (=TT))) = p(zT) Ii-- po (2x xz).
The model is trained by minimizing the difference between the true noise and co t), the predicted noise at step by the denoising model f as follows:
= E Exo te.. ||e ea (are, 1)
Latent Diffusion Models: With an intuition that semantic information that controls the main concept of an image can be represented in a low-dimensional space, (Rombach et al. 2022) proposed diffusion process operating on the latent space to learn the distribution of the semantic information which can be formulated as (o = p(2r) I (2 22), where 2g E(xo) is the latent vector obtained by a pre- preitrained encoder E.
The objective function of the latent diffusion model as follows:
C
Beggett ||e c0 1)11
3 Problem Statement
The task of crasing concepts from a text-to-image diffusion model often appears without additional data labels, forcing us to rely on the model's own knowledge. Therefore, we here consider fine-tuning a pre-trained model rather than training a model from scratch. Let eø (Rt, e, t) denote the output of the pre-trained foundation U-Net model parameterized by 0 at step t given an input description c E C and the latent vector from the previous step 2 where is set of all possible input descriptions, commonly referred to as the textual prompt in ext-to-image generative models.
Given set of textual descriptions C C and the target model eg, our objective is to learn a sanitized model egr (zz, !) that cannot generate images from any textual description e E E while preserving the quality of images generated by the remaining concepts R E. We also use on to denote neutral or null concept, i.c., "a photo" or"

---

STUCE Prooeve
Erasa "puttege trdel Freeave FROat
Crave "pirtige trusl" Trovave "rod""
CUP oze offence
400
Conuaga
(b)
Figure I: Analysis of the impact of erasing the target concept on the model's capability. The impact measured by the difference of CLIP score 8(c) between the original model and the corresponding sanitized model. Ta Impact of erasing "nudity" or "garbage truck" to other concepts. [lb] Comparing the impact of erasing the same "garbage truck" to other concepts with different preserving strategies, including preserving fixed concept such as "lexus", or "road", and adaptively preserving the most sensitive concept found by our method.
3.1
Naive Erasure
A naive approach that has been widely used in previous works Gandikota et al. (2023) ; Orgad et al. (2033) Gandikota et al. (2024) is optimize the following objective function:
min E. [ller (c.) (o.))-e(c...)))) co(cn))
Fundamentally, these methods aim force the model output, associated with the to-be-erased concepts, to approximate the model output associated with a neutral or null input Gr (e.g.. "a photo" or "). Ideally, when erasing concept, we would like to preserve all the remaining ones. This would corresponding to optimizing the above objective for all possible concepts in C' E, which is excessively expensive. Hence, using a neutral concept as proxy first seems convenient strategy.
While this naive approach effective in erasing the specific concept, however has a negative impact the model's capacity to preserve other concepts related to the to-be-erased concepts. For example, easing the concept "nudity" affects the quality images of "woman" or "person". To mitigate this issue, prior works have proposed to use either additional loss term retain the null concept Gandikota et al. 2023) or a regularization term to prevent excessive change in the model parameters Orgad et al, (2023). However, these regularization attempts clearly have not addressed the core trade-off between erasing a concept and preserving the others.
3.2 Impact of Concept Removal on the Model Performance
We here approach the problem more carefully via a study the impact of erasing specific concept on model performance on the remaining ones. More importantly, we are concerned with the most sensitive concepts to erasure. For example, when removing the concept of "nudity", know which concepts change the most in the model's 's output, so that we can preserve these concepts specifically to ensure the model's capability is maintained, at least with respect to these concepts.
curious to
For some concepts, we can make an intuitive guess. For example, the concept of "nudity" closely related to the concepts of "women" and "men". which are likely to be affected by the removal of the concept of "nudity". However, for most concepts, is not easy to determine which ones are most sensitive to the target concept. Therefore, in prior works, selecting neutral one like 'photo' or regardless the target concepts is clearly not a sound solution. We next provide empirical evidence to support this argument.
Measuring Generation Capability with CLIP Alignment Score. Given target concept Ce, e.g.. "nudity" or "garbage truck", from that we obtain the original model and the sanitized model ep; by removing the target concept ce- We have set of concepts C = {ci, C c e c where |C] is the number of concepts. Our goal to measure the impact of unlearning ea on the generation of other concepts in C.
3

---

0
'nudity' & preserve
naked
women
2500
men
person
photo
2000
Trequency
1500
1000
500
0
-5
-45 similarity score -1,,,,.22) 2elt,,.2)
-40
-300
Figure 2: Sensitivity spectrum of concepts to the target concept "nudity". The histogram shows the distribution of the similarity score between outputs of the original model and the corresponding sanitized model 6', for each concept e from the CLIP tokenizer vocabulary.
To achieve this, we generate a large number of samples from both models, i.c., {G(O, =. (G(' (t) for k 200 samples for each concept e C. We then calculate CLIP alignment score Sa.i. = S(G(O, =f), between the generated samples and the textual description of the concepts e (CLIP model apenailcli--vit-base-ppee 14"). A higher CLIP alignment score indicates that the generated samples are more similar to the concept e, and vice versa. Thus, we can use the CLIP alignment score as metric to evaluate the capability of the model to generate the
concept e, and the change of this score between the two models, 6z, (c) = Σ (Su.ic Se di
indicates the impact of unlearning ce on generating the concept e. The discussion on the metric provided in Appendix B.3
The Removal of Different Target Concepts Leads to Different Side-Effects. Figure la shows the impact of the removal two distinct concepts, "nudity" and "garbage truck", on other concepts, measured by the difference of the CLIP score, muday- (c), ( ganbuge truck (c). A larger 5(c) indicates greater negative impact on the model's ability to generate concept c.
It can be seen that removing the "nudity" concept significantly affects highly related concepts such as "naked", "men". "women", and "person", while having minimal impact unrelated concepts such as "garbage truck", 'bamboo' or neutral concepts such as "a photo" or the null concept. Similarly, removing the "garbage truck" concept significantly reduces the model's capability on concepts like "boat", "car", "bus", while also having little impact on other unrelated concepts such as "naked", "women" or neutral concepts.
These results suggest that removing different target concepts leads to varying impacts on other concepts. This indicates the need for adaptive method to identify the most sensitive concepts relative to particular target concept, rather than relying on random or fixed concepts for preservation. Moreover, in both cases, neutral concepts like "a photo" the null concept show resilience and independence from changes in the model's parameters, suggesting that they do not adequately represent the model's 's capability be preserved.
Neutral Concepts lie in the Middle of the Sensitivity Spectrum. Figure shows the distribution of similarity scores between the outputs of the original model and the sanitized model 6' for each concept e from the CLIP tokenizer vocabulary. The histogram reveals that the similarity scores span a wide range, indicating that the impact of unlearning the target concept on generating other concepts varies significantly. The lower the similarity score, the more different the outputs of the two models are, and the more sensitive the concept is to the target concept. Notably, the more related concepts like "women" or "men" are more sensitive to the removal of "nudity" than many neutral concepts that lie in the middle of the sensitivity spectrum.

---

Erase "nudity" & Preserve Erase "hudity" Preserve "person"
difference
nalied
wothen
men
person
pertagg
Bus
hat
Denaraa
10ple
bomn
rotetcoks
Concept
Figure Comparing the impact of erasing the same "nudity" to other concepts with different preserving strategies.
What Concept should be Kept to Maintain Model Performance. Figure lb presents the results of an experiment similar to the previous one, with one key difference: we utilize the prior knowledge gained from the previous experiment. Specifically, when erasing the "garbage truck", apply different preservation strategies, including preserving fixed concept such as "lexus", or "road", and adaptively preserving the most sensitive concept found by our method.
The results show that with simple preservation strategies such as preserving a fixed but related concept like "road", the model's capability on other concepts is better maintained compared to preserving a neutral concept. However, the results of adaptively preserving the most sensitive concept show the best performance, with the least side effects other concepts. Similarly, the results of erasing the "nudity" concept as shown in Figure show that preserving related concepts like "person" helps retain the model's capability on other concepts much better than preserving neutral concept. These findings confirm the importance of selecting sensitive concepts to preserve order to better maintain the model's overall capability.
4 Proposed Method: Adversarial Concept Preservation
In this work, we aim to minimize the side effects of erasing undesirable concepts in diffusion models through adversarial preservation. Motivated by the observations in the previous section, approach involves identifying the most sensitive concepts related to a specific target concept. For example, when removing the concept of nudity, we identify which concepts are most affected in the model's output so that we can specifically preserve these concepts to ensure the model's capability is maintained.
In each iteration, before updating the model parameters, we first identify the concept ca that is most sensitive to changes in the model parameters as we work to remove the target concepts.
min max RE,EE d' E.ER
[| co(cn)ll + (lger c
where À > 0 is a parameter and R = C E denotes the remaining concepts.
Objective loss L1 is the same as in the naive approach, aiming to erase the target concept cæ by forcing its output to match that of a neutral concept. Our main contribution lies in the introduction of the adversarial preservation loss Ly, which aims identify the most sensitive concept ca that is most affected by changes in the model parameters when removing the target concepts.
Since the concepts exist in discrete space, the straightforward approach would involve revisiting all concepts in R. resulting in significant computational complexity. Another naive approach is to consider the concepts lying in a continuous space and use the Projected Gradient Descent (PGD)
5

---

I-21
I-30
i+g Grrdd
i=10 cym
20 C Thoad"
1+30 cym Tood
le5p Cporr
I-300 Gy someerinn
i=150 Ce ssmettinn""
I=200 Gym "samathnng"
I+256 o Gatammening
1+300
4
X
Figure 4: Images generated from the most sensitive concepts found by our method over the fine-tuning process. Top: Continous search with PGD. Bottom: Discrete search with Gumbel-Softmax. Co represents for the keyword.
method, similar to Madry et al. (2017). to search within the local region of the continuous space of the concepts. More specifically, we initialize the adversarial prompt with the embedding of the to-be-erased concept, c.g., Ca,0 Ce 7 (""arbaage Truck"), and then update the adversarial concept with gradient
L2. Interestingly, while this approach provides an efficient computational method, we find that the adversarial concept quickly collapses from the initial concept to background concept with the color information of the object as shown in the first row Figure
To combine the benefits of both approaches-making the process continuous and differentiable for efficient training while achieving meaningful concepts that related the target concept (second row of Figure we first define a distribution over the discrete concept embedding vector space as
PR. Σ T. with the Dirac delta function 5 and the weights E Ax = { 2 0 || 1}. Instead of directly searching for the most sensitive concept ca in the discrete concept embedding vector space R, switch searching for the embedding distribution
the simplex AR and subsequently transform back into a discrete space using the temperature-dependee GumbelSoftmax trick (Jang et al. 2016 Maddison al. (016) follows:
min max EC,CE o' HEAR
(ce) - colcwll +1 |lo R) co R
(5)
1
1-2
where À > is parameter, G Gumbel-Softmax operator and O is element wise multiplication operator. The pseudo-algorithm involves a two-step optimization process, outlined in Algorithm | Adversarial Erasure Training.
Finding Adversarial Concept and Algorithm 2
Algorithm Find Adversarial Concept
Input: R. Searching hyperparameters: 7) Nuer Current state 0'
Output: Adversarial concept Ca
for i 1 Nine do
+ + V
||sor (G(T)) R) cø (G(TT) 3 R
> Maximize L2
end for
Co G( C R
5 Experiments
In this section, we present a series of experiments to evaluate the effectiveness of our method in erasing various types of concepts from the foundation model. Our experiments use Stable Diffusion (SD) version 1.4 as the foundation model. We maintain consistent settings across all methods: fine- tuning the model for 1000 steps with batch size of I, using the Adam optimizer with a learning rate of a 10-5 We benchmark our method against four baseline approaches: the original pre-trained model, ESD (Gandikota et 2023), UCE (Gandikota al. (024), and Concept Ablation (CA)
(Kumari et al. 2023).

---

Algorithm 2 Adversarial Erasure Training
Input: 0, R, E, A. Searching hyperparameters: 17, Niter Output: 0
h +- 0, 0g <-
while Not Converged do
Ce E
Ca f FindAAdversarillConee((//
17, Niier) 041 t 6x - a llloo (co) co co(cn)l + + Allcor (ca) (cCa))
Outer min
end while
We provide detailed implementation and further in-depth analysis in the appendix, including qualita- tive results (Section C the choice of hyperparameters (Section B.2, and analysis on the search for the adversarial concepts (Sections B.4 and B.5).
5.1 Erasing Concepts Related Physical Objects
In this experiment, we investigate the ability of our method to erase object-related concepts from the foundation model, for example, erasing entire object classes such as "Cassette Player" from the model. We choose Imagenette| which is subset of the ImageNet dataset Deng et al. (2009) which comprises 10 easily recognizable classes, including "Cassette Player", "Chain Saw", "Church", "Gas Pump", "Tench", "Garbage Truck", "English Springer", "Golf Ball", "Parachute", and "French Horn".
Since the erasing performance when erasing single class has been the main focus of previous work Gandikota et al. (2023), we choose a more challenging setting where we erase a set of classes simultaneously. Specifically, we generate 500 images for each class and employ the pre-trained ResNer-50 He et al. (2016) to detect the presence of an object in the generated images. We use the two following metrics to evaluate the erasing performance: Erasing Success Rate (ESR-k): The percentage of all the generated images with "to-be-erased" classes where the object is not detected in the top-k predictions. Presevering Success Rate (PSR-k): The percentage of all the generated images with all other classes (i.c., (o-bbe-reeeerved)) where the object is detected in the top-k predictions. This dual-metric evaluation provides a comprehensive assessment of our method's ability to effectively erase targeted object-related concepts while also preserving relevant elements.
Quantitative Results. We select four distinct sets of 5 classes from the Imagenette set for erasure and present the outcomes in Table T First, we note that the average PSR-1 and PSR-5 scores across the four settings of the original SD model stand at 78.0% and 97.6%, respectively. These scores indicate that 78.0% of the generated images contain the object-related concepts which are subsequently detected in the top-1 prediction, and when checking the concepts in any of the top-5 predictions, this number increases to 97.6%. This underscores the original SD model's ability to generate images with the anticipated object-related concepts.
In term of erasing performance, it can be observed that all baselines achieve very high ESR-1 1 and ESR-5 scores, with the lowest ESR-1 and ESR-5 scores being 95.5% and 88.9% respectively. This indicates the effectiveness of these methods to erase object-related concepts, as only a very small proportion of the generated images contain the object-related concepts under subsequent detection. Notably, the UCE method can achieve 100% ESR-1 and ESR-5, which is the highest among the baselines. Our method achieves 98.6% ESR-1 and 96.1 1 % ESR-5, which is much higher than the two baselines ESD and CA, and only slightly lower than the UCE method, which is designed specifically for erasing object-related concepts.
However, despite the high erasing performance, the baselines, especially UCE, suffer from a signif- icant drop in preserving performance, with the lowest PSR-1 and PSR-5 scores being 23.4% and 49.5%. respectively. This suggests that the preservation task poses greater challenges than the erasing task, and the baselines are ineffective in retaining other concepts. contrast, our method achieves 55.2% PSR-1 and 79.9% PSR-5, which is a significant improvement compared to the best baseline, CA. with 44.2% PSR-1 and 66.5% PSR-5. This result underscores the effectiveness of our method in simultaneously erasing object-related concepts while preserving other unrelated concepts.

---

Table 1: Erasing object-related concepts.
Method
ESR-11
FSR-5t
PSR-11 1
PSR-51
SD
22.0 11.6
2.4 1.4
78.0 11.6
97.6 1.4
ESD
95.5 0.8
88.9 1.0
41.2 12.9
56.1 12.4
UCE
100 0.0
100 0.0
2364=3.6
8.0
CA
98.4 0.3
96.8
44.2
66.51..1
Ours
98.6 + 1.1
06.1
55.2 10.0
79.9 + 2.8
5.2
Mitigating Unethical Content
One of the serious concerns associated with the deployment of text-to-image generative models to the public domain is their potential generate Not-Safe-Fort For-Forrk (NSFW) content. This ethical challenge has become a primary focus in recent works Schramowski et al. (2023); Gandikota et al. (2023 2024), aiming sanitize such capability of the model before public release.
In contrast to object-related concepts, such as "Cassette Player" or "English Springer", which can be explicitly described with limited textual descriptions, i.e., there are only few textual ways to describe the visual concepts, unethical concepts like nudity are indirectly expressible in textual descriptions. The multiple ways single visual concept can be described make erasing such concepts challenging, especially when relying solely on a keyword to indicate the concept to be erased. As empirically shown in Gandikota et al. 2023) the erasing performance these concepts is highly dependent the subset of parameters that are finetuned. Specifically, fine-tuning the cross-attentioon modules has shown to be more effective than fine-tuning the cross-attention modules. Therefore, in this experiment, we follow the same configuration as in Gandikota et al. (2023), focusing exclusively on fine-tuning the non-cross-attention modules.
Quantitative Results. To generate NSFW images, we employ I2P prompts (Schramowsii et al. (2023) and generate dataset comprising 4703 images with attributes encompassing sexual, violent, and racist content. We then utilize the detector Praneet (2019) which can accurately detect several types of exposed body parts to recognize the presence of the nudity concept in the generated images. The detector Prancet (2019) provides multi-label predictions with associated confidence scores, allowing us to adjust the threshold and control the trade-off between the number of detected body parts and the confidence of the detection, i.e., the higher the threshold, the fewer the number of detected body parts.
Figure 5a illustrates the ratio of images with any exposed body parts detected by the detector Praneet (2019 over the total 4703 generated images (denoted by NER) across thresholds ranging from 0.3 to 0.8. Notably, our method consistently outperforms the baselines under all thresholds, showcasing its effectiveness erasing NSFW content. In particular, as per Table 2 with the threshold set at 0.3, the NER score for the original SD model stands at 16.7% indicating that 16.7% of the generated images contain signs of nudity concept from the detector's s perspective. The two baselines, ESD and UCE, achieve 5.32% and 6.87% NER with the same threshold, respectively, demonstrating their effectiveness erasing nudity concepts. Our method achieves 3.64% NER, the lowest among the baselines, indicating the highest erasing performance. This result remains consistent across different thresholds, emphasizing the robustness of our method erasing NSFW content. Additionally, to measure the preserving performance, FID score compared to COCO 30K validation images. Our method achieves the best FID score of 15.52, slightly lower than that of UCE, which is the best baseline 15.98, indicating that our method can simultaneously erase a concept while preserving other concepts effectively.
generate images with COCO 30K prompts and measure the
Detailed statistics of different exposed body parts in the generated images provided in Figure 5b can be seen that in the original SD model, among all the body parts, the female breast the most detected body part in the generated images, accounting for more than 320 images out of the total 4703 images. Both baselines, ESD and UCE, as well as our method, achieve a significant reduction in the number of detected body parts, with our method achieving the lowest number among the baselines. Our method also achieves the lowest number of detected body parts for the most sensitive body parts, only surpassing the baseline for less sensitive body parts, such feet.
Interestingly, our method seems to remove the sensitive body parts while keeping the less sensitive body parts untouched as shown in Figure 5b To provide more insights into this phenomenon,
8

---

Table 2: Evaluation on the nudity erasure setting.
NFR-0.34
NFR-0.54
NER-0.74
NER-088)
FIDL
CA
13.84
9.27
4.74
1.68
20.76
UCE
6.87
3.42
0.68
0.21
15.98
ESD
5.32
2.36
0.74
0.23
17.14
Ours
3.64
1.70
0.40
0.06
15.52
5D
CA
E50v
UCE
Ours
ESD
16.
Bely
10-
Male Sreust
Paze smogue
Famale Gentata
2.52
Temale Greact
100
200
300
400
500
800
0.7
Exposed body part istaceedi
Threshold
(a)
(b)
Number exposed body parts counted in all generated images with threshold 0.5, (bb] Ratio of images with any exposed body parts detected by the detector Prancet (2019).
Figure 5: Comparison of the erasing performance on the I2P dataset. 5a
calculate the similarity scores between different concepts and body parts the nudity erasure setting as Table 3
Table Similarity scores between different concepts and body parts in the nudity erasure setting.
CLIP
Nudity
A photo
Person
Body
Feet
0.612
0.547
0,566
0.643
Belly
0.601
0,514
0.517
0.748
Armpits
0.614
0.477
0.475
0.643
Buttocks
0.649
0.501
0.494
0.639
Male Breast
0.616
0.499
0.472
0,504
Male Genitalia
0.618
0.511
0,537
0,517
Female Genitalia
0.662
0.536
0.558
0.555
Female Breast
0.656
0.517
0.491
0.574
It can be seen that the "nudity" concept is highly correlated with the "Female Breast" concept, suggesting that when removing the "nudity" concept, the "Female Breast" concept more likely to be affected than other body parts. On the other hand, the "Person" or "Body" concept is more strongly correlated with the "Feet" concept than with "Female Breast" concept, indicating that preserving the "Person" concept might help maintain the model's performance on "Feet" rather than on "Female Breast." Furthermore, the gap between the "Feet" and "Female Breast" concepts with respect to "Person" or "Body" is larger than the gap with more generic concepts like "A photo." 'Tii suggests that preserving generic concepts might not have the same impact as preserving the most affected concepts. Our method naturally selects the most affected concepts to be preserved, which often includes concepts highly correlated with non-sensitive body parts. This explains the observed phenomenon in the experiment.
5.3 Erasing Artistic Concepts
In this experiment, we investigate the ability our method to erase artistic style concepts from the foundation model. We choose several famous artists with easily recognizable styles who have been known to be mimicked by the text-to-image generative models, including "Kelly Mckernan". "Thomas Kinkade", "Tyler Edlin" and "Kilian Eng" in Gandikota al. (2023). We compare our

---

method with recent work including ESD Gandikota et al. (2023), UCE Gandikota et al. 20244) and CA Kumari al. (202)) which have demonstrated effectiveness in simllar settings.
For fine-tuning the model, we use only the names of the artists as inputs. For evaluation, we use a list of long textual prompts that are designed exclusively for each artist, combined with 5 seeds per prompt to generate 200 images for each artist across all methods. We measure the CLIP alignment score between the visual features of the generated image and its corresponding textual embedding. Compared to the setting Gandikota et al. (2023) which utilized list of generic prompts, our setting with longer specific prompts can leverage the CLIP score as a more meaningful measurement to evaluate the erasing and preserving performance. We also use LPIPS Zhang et al. (2018) to measure the distortion in generated images by the original SD model and editing methods, where a low LPIPS score indicates less distortion between two sets of images.
It can be seen from Table 4 that our method achieves the best erasing performance while maintaining a comparable preserving performance compare to the baselines. Specifically, our method attains the lowest CLIP score on the to-be-erased sets at 21.57, outperforming the second-best score of 23.56 achieved by ESD. Additionally, our method secures 0.78 LPIPS score, the second-highest, following closely behind the CA method with 0.82. Concerning preservation performance, we observe that, while our method achieves a slightly higher LPIPS score than the UCE method, suggesting some alterations compared to the original images generated by the SD model, the CLIP score of our method remains comparable to these baselines. This implies that our generated images still align well with the input prompt.
Table 4: Erasing artistic style concepts.
To Erase
To Retain
CLIP +
LPIPS
CLIP
LPIPS
ESD
23.56 + 4.73
0.72 + 0.11
29.63 + 3.57
0.49 0.13
CA
27.79 4.67
0.82 0.07
29.85 + 3.78
0.76 + 0.07
UCE
24.47 4.73
0.10
30.89 + 3.56
0.40 +: 0.13
Ours
21.57
5.46
0.10
00.13+3.44
0.47 0.14
Conclusion
In this paper, introduced novel approach to concept erasure in text-to- -ittte diffusion models by incorporating an adversarial learning mechanism. This mechanism identifies the most sensitive concepts affected by the removal of the target concept from the discrete space of concepts. By preserving these sensitive concepts, our method outperforms stateoof-the-an erasure techniques in both erasing unwanted content and preserving unrelated concepts, as demonstrated through extensive experiments. Furthermore, our adversarial learning mechanism exhibits high flexibility, linking this task the field of Adversarial Machine Learning, where adversarial examples have been extensively studied. This connection opens potential directions for future research, such as simultancously searching for multiple sensitive concepts under certain divergence constraints, offering promising avenues for further exploration.
Acknowledgements
This work was supported by the Australian Defence Science and Technology (DST) Group through the Next Generation Technology Fund (NGTF) scheme and the Department of Defence, Australia, via the Advanced Strategic Capabilities Accelerator (ASCA) program. Dinh Phung further acknowledged the support from the Australian Research Council (ARC) Discovery Project DP230101 176. The authors would like to express their appreciation to the anonymous reviewers for their insightful feedback and valuable suggestions, which has significantly enhanced the quality of this work.
References
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv: 1712.0442, 2017.
10

---

Anh Bui, Trung Le, He Zhao, Paul Montague, Olivier de Tamas Abraham, and Dinh Phung. Im- proving adversarial robustness by enforeing local and global compactness. In European Conference on Computer Vision, pp. 209-223. Springer, 2020.
Anh Bui, Khanh Doan, Trung Le, Paul Montague, Tamas Abraham, and Dinh Phung. Removing undesirable concepts in text-to-image generative models with learnable prompts. arXiv preprini arXiv:2403. 12326, 2024.
Tuan Anh Bui, Trung Le, Quan Tran, He Zhao, and Dinh Phung. A unified wasserstein distributional robustness framework for adversarial training. arXiv preprint arXiv: 2022.
Zhi-Y Yi Chin, Chieh-Ming Jiang, Ching-Chie Chug--Chn Huang, Pin- Yu Chen, and Wei-Chen Chiu. Prompt- ing4debugging: Red-teaming text-to-iman eext-to-image diffusion models by finding problematic prompts. arXiv preprint arXhv:2309.06135, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009. doi: 10. 109//09R/ 200955544000
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprini urXiv:2208.016: 01618, 2022.
Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyúska, and David Bau. Unified concept editing in diffusion models. In Proceedings of the IEEE/CVP Winter Conference on Applications of Computer Vision, pp. 5111-5120, 2024.
Rohit Gandikota et al. Erasing concepts from diffusion models. ICCV, 2023.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint orXiv:1412.6572. 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
doi: 10. 10.100//CV..2006..000
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.099 09680 2021.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameteriization with gumbel-softmax. arXiv preprint 161 144, 2016.
Nupur Kumari, Bingliang Zhang, Sheng- Yu Wang. Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating concepts in text-to-image diffusion models. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 22691-22702, 2023.
Yixin Liu, Chenrui Fan, Yutong Dai, Xun Chen, Pan Zhou, and Lichao Sun. Metacloak: Preventing unauthorized subject-driven text-to-image diffusion-based synthesis via meta-learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24219-24228, June 2024.
Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6430-6440, 2024.
Mengyao Lyu, Yuhong Yang, Haiwen Hong, Hui Chen, Xuan Jin, Yuan He, Hui Xue, Jungong Han, and Guiguang Ding. One-dimensional adapter to rule them all: Concepts diffusion models and erasing applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
11

---

Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.0077 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1.ki::/706000833 2017.
Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. Editing implicit assumptions in text-to-image diffusion models. In IEEE International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 7030-7038, IEEE, 2023. doi: 10.1100111000 10110//CV11100.222....000
Bedapudi Praneet. Nudenet: Neural nets for nudity classification, detection and selective censorin. 2019.
Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models. arXiv preprini arXiv:2305.
2023.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. International Conference on Machine Learning, pp. 8821-883 PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3 2022.
Javier Rando et al. Red-teaming the stable diffusion safety filter. NeurIPS Workshop MI.SW, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500-22510, 2023.
Patrick Schramowski et al. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In CVPR, 2023.
SmithMano. Tutorial: How to remove the safety filter seconds, 2022.
AtabilityAI. Stable diffusion 2.0 release. 2022. stable-diae diffusionnv2-reelesss
URL https://stablltt..a//
Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc N Tran, and Anh Tran. Anti- dreambooth: Protecting users from personalized text-to-imaag synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2116 2023.
Mika Westerlund. The emergence of deepfake technology: A review. Technology innovation management review, 9(11), 2019.
Haotian Xue, Chumeng Liang, Xiaoyu Wu, and Yongxin Chen. Toward effective protection against diffusion-based mimicry through score distillation. In The Twelfth International Conference on Learning Representations, 2023.
Yuchen Yang, Bo Hui, Haolin Yuan, Neil Gong, and Yinzhi Cao. Sneakyprompt: Jailbreaking text-to-image generative models. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 123-123, IEEE Computer Society, 2024.
Eric Zhang al. Forget-me-not: Learning to forget text-to-image diffusion models. arXiv preprint 1rX1v:2303.17551, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metrie. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
12

---

Xuanyu Zhang, Runyi Li, Jiwen Yu. Youmin Xu, Weiqi Li, and Jian Zhang. Editguard: Versatile image watermarking for tamper localization and copyright protection. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp- 11964-11974, 2024a.
Yimeng Zhang, Xin Chen, Jinghan Jia, Yihua Zhang, Chongyu Fan, Jiancheng Liu, Mingyi Hong, Ke Ding, and Sijia Liu, Defensive unlearning with adversarial training for robust concept erasure in diffusion models. arXiv preprint arXiv:2405. 2024b.
Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now, In European Conference Computer Vision, pp. 385-403. Springer, 2025.
13

---

Appendix
Table of Contents
Related Work
B
Further Experiments
16
B.1
Experimental Settings
16
B.2
Impact of Hyperparameters
16
B.3
Discussion on Metrics to Mcasure the Erasure Performance
19
B.4
Further Analysis on Searching for Adversarial Concepts
19
B.5
Difficulties in Scarching for Adversarial Concepts
21
C
Qualitative Results
21
A
Related Work
Given the growing concerns over the potential misuse of text-to-image models, several techniques have been developed to remove undesirable concepts from foundation models before deployment. The simplest approach is pre-processing, which filters out objectionable content from the training data using pre-trained detectors. This method, as seen in Stable Diffusion v2.0 (StabilityA]] 2022) and D excludes harmful data from the training set. However, it requires retraining the entire model, making it computationally expensive and impractical for adapting to evolving erasure requests.
Another basic approach is post-processing, which aims to identify potentially inappropriate content in generated data and then either blur black out the images before they presented to users. This method involves a Not-Safe-For Work (NSFW) detector, which can be deployed along with the generative model, as in closed-source models like Dall-E or Midjourney, released as separate module in open-source models like Stable Diffusion. However, this approach is not foolproof, as demonstrated in (Yang et al. 2024), where technique similar to the Boundary Attack (Brendel et al. 2017) was used to uncover adversarial prompts that could bypass the filtering mechanism. In the case of open-source models, the NSFW detector can be easily disabled by modifying just few lines of code in the source MSnithMano 2022).
To date, the most successful strategy for sanitizing open-source models, such Stable Diffusion, is model fine-tuning. which involves sanitizing the generator (e.g.. U-Net) in the diffusion model post- training on raw, unfiltered data and before public release. This approach, as partially demonstrated in Gandikota et al. 22023 2024), underscores the importance addressing potential biases and undesired content in models before their deployment. There
main branches within model fine-tuning: attention- and output-based, categorized by the primary components involved in the objective function.
Attention- based methods (Zhang et al. 2023 Orgad et al. 2023 Kumari et al. 2023 Gandikota
al. 2024| Lu et al. 2024) focus modifying the attention mechanisms within models to remove undesirable concepts. In Latent Diffusion Models (LDMs), for instance, the textual conditions are embedded via pre- tre-trained CLIP model and injected into the cross-attention layers of the UNet model (Rombach et al.] [2022] Ramesh et al.) 2022). Therefore, removing an unwanted concept can be achieved by altering the attention mechanism between the textual condition and visual information flow. For example, in TIME Orgad al. 2023). the authors propose minimize || c. u l|g where represents the original cross-attention weights, W' the fine-tuned weights, c, the embedding of the unwanted concept, and of the target vector. By different settings of of. the method can either steer the unwanted concept toward more acceptable one (i.e., of Wr("a photo")) or edit biases
hhtps://opeeai.coo//nddeellllll trenna/mnnnneeiillleeeereeenennn training traiiiinnn-mmitittooonnn
14

---

in the model (i.e., 27 = Wr("a female doctor")) )) This category has two main advantages including the closed-form solution as shown in (Orgad et al. (023). and the fact that it operates solely on textual embeddings not the intermediate images, making it faster than optimization-base methods.
Follow-up works (Zhang al., 2023 Gandikota al. 2024 Lu al. 2024) share this principle. Specifically, Forget-Me-Not (Zhang ct al.) 2023) introduces an attention resteering method that minimizes the L2 norm of the attention maps related the unwanted concept. UCE (Gandikota et al. 2024) extends TIME by proposing a preservation term that allows the retention of certain concepts while erasing others. MACE (Lu et al.) 2024 improves the generality and specificity of concept 2021) for each individual concept, combining them with the closed-form solution from TIME Orgad et al. 2023).
erasure by employing LoRA modules (uu et al
Output-based methods (Gandikota et al. 2023 Bui et al. 2024) focus on optimizing the output image by minimizing the difference between the predicted noise Eg' (, Ca) and the target noise co(2,, Ce). Unlike attention-based methods, this approach requires intermediate images 2; sampled at various time steps during the diffusion process. While this method is computationally more expensive, it generally yields superior erasure results by directly optimizing the image, ensuring the removal of unwanted concepts (Gandikota et al. 2023).
A recent addition to the field, SPM (Lyu et al. 2024), introduces one-dimensional adapters that, when combined with pre-trained LDMs, prevent the generation of images containing unwanted concepts. SPM introduces a new diffusion process = e(Ix, Ct M,,,) where Mc. is an adapter model trained remove the undesirable concept c.. While these adapters different models, the original model 0 remains unchanged, allowing malicious users to remove the adapter and generate harmful content. Thus, SPM is less robust and practical compared to the other approaches discussed.
be shared and reused across
Concept mimicry is recent research direction that aims to copy or mimic specific concept from set of reference images to generate new images containing the concept. The concept can be artistic styles or personal visual appearance, raising concerns about the potential misuse of the technique. Noteworthy methods include Textual Inversion (Gal et al. 2022) and Dreambooth (Ruiz et al.) (2023). ) which have proven effective with just few user-provided mages. In contrast, Anti Concept Mimicry is employed to safeguard personal or artistic styles from being copied through concept mimicry. Achieved by introducing imperceptible adversarial noise to input images, this technique can deceive concept mimicry methods under specific conditions. Recent contributions such as Anti-Dreambooth (Van Le et al., (003),, SDS (Xue al.] 2023), and MetaCloak (Liu et al. 2024) have explored and demonstrated the effectiveness of this approach. EditGuard (Zhang et al. 2024a), on the other hand, aims to watermark images with imperceptible adversarial noise localize tampered regions and claim copyright protection. This category can be viewed as protection method from the user's side, which is orthogonal to the erasure problem discussed in this paper.
Developed concurrently with this paper, AdyUnlearn (Zhang et al. 2024b) incorporates adversarial training (Goodfellow et al. 2014 Madry et al. 2017: Bui et al. 2220 2022) improve the robustness of concept erasure. More specifically, the authors propose a similar bilevel min-min optimization problem, where the inner minimization problem seeks to find the adversarial prompt c, that minimizes the attack loss, i.e., the extent to which the unwanted concept is retained in the generated image. The
adversarial prompt c, is found using adversarial prompt attack techniques (Zhang et al. 2025 Chin
eet al. 2023), such as the fast gradient sign method (FGSM) (Goodfellow et al. 2014))
While both Adv Unlearn and our approach share the adversarial training framework, they are funda- mentally different. Firstly, our method is driven by the observation of how erasure impacts model performance and how different preservation strategies affect erasure efficacy. Ady AdyUnlearn, on the other hand, is motivated by adversarial prompt attacks. Secondly, our method formulates the problem as bilevel min-max optimization, where the inner maximization aims find the adversarial concept ca that maximizes the preservation loss, while AdyUnlean'ss 's inner minimization seeks the adversarial prompt c* that minimizes the attack loss. Finally, our method employs the Gunbel-Softmat trick [Jang et al. (2066) to approximate the bilevel optimization, whereas AdyUnlearn uses FGSM to find the adversarial prompt.
15

---

B Further Experiments
B.1 Experimental Settings
General Settings. Our experiments use Stable Diffusion (SD) version 1.4 as the foundation model. We maintain consistent settings across all methods, fine-tuning the model for 1000 steps with a batch size of I, using the Adam optimizer with a learning rate of 1e-5. We benchmark our method against four baseline approaches: the original pre-trained SD model, ESD (Gandikota al. 2023). UCE (Gandikota et al. 2024), and Concept Ablation (CA) (Kumari et al. 2023). Our models are trained on NVIDIA A10 GPUs of 80GB. One training routine takes less than 6 hours for erasing "nudity" and less than hour for other concepts.
Settings for Our Method. A crucial aspect of our method is the concept space R, where we search for the most sensitive concept Ca. In our experiments, we use two vocabularies; the CLIP token vocabulary, which includes 49,408 tokens, and the Oxford 3000 word list, comprising the 3000 most common English word While the CLIP token vocabulary is more comprehensive, it presents challenges due the large number nonsensical tokens. Therefore, for the experiments in Section 5] we use the Oxford 300d-word list to demonstrate the effectiveness of our method.
Computational Limitations. To search for the adversarial concept C, effectively, we employ the Gumbel-Softmax trick (Jang et al., (016) to sample from the categorical distribution in the concept space R. This approach requires feeding the model with the embeddings of the entire concept space R. which exponentially increases the computational cost as the size of the concept space grows. To mitigate this, we use a subset of the K most similar concepts the target concept ee reduce computational costs. The similarity between concepts is calculated using cosine similarity between their embeddings.
To provide a better understanding of the concept space R. we list the most similar concepts to the target concept ce in [5 We provide the study of the impact of the number of concepts K and the number of search steps Niter on the erasing and preservation performance in Section B It is worth to remind that, erasing "nudity" requires to fine-tune on all non-cross-attention modules which is more computationally expensive than erase other concepts that only requires fine-tuning on cross-attention modules. Therefore, in the default settings, we use 30 for erasing 'nudity" and K 100 for other concepts. For searching hyperparameters, we use Nuer 2, 17 as the default settings.
1 x 10 10-3, and trade-off
B.2 Impact of Hyperparameters
In this section, we investigate the impact of hyperparameters on the performance of our method. Specifically, analyze the effect of the number of closest concepts K and the number of search steps Nier on the erasing and preservation performance. We conduct the experiments on the Imagenette dataset with the same settings as in Section 5 Table 6 shows the evaluation results of different hyperparameter settings. It can be seen that the erasing and preservation performance is more affected by the number of concepts K than the number of search steps Nuer- Reducing the search space from K = 100 to K- = 20 hugely decreases the crasing performance by around 2% in ESR- and 3% in ESR-5, as well as the preservation performance by around 2% in PSR-S. This observation aligns with the intuition that a larger search space provides more flexibility for the model to find the most sensitive concept Ca-
On the other hand, increasing the number of search steps from Niter = 2 to Niter = 8 does not increase the performance, but inversely hurts the preservation performance by around 10% in PSR-1, Therefore, in other experiments, we use K = 100 and Nier = as the default settings.
Impact of the Concept Space To ensure the generality of the search space so that can be applied to various tasks such as object-related concepts, NSFW content, and artistic styles, we used the Oxford 3000 most common words in English as the search space.
To evaluate the impact of the concept space, we conduct additional experiments with the search space as the CLIP token vocabulary, which includes 49,408 tokens. It is worth noting that the CLIP token
16

---

Table 5: The list of the K = 50 most similar concepts to the target concept Ce.
Target Concept
Similar Concepts
Garbage truck
truck, vehicle, waste, pollution, bus, terrible, container, vegetable, some- thing, awful, refuse, delivery, destruction, that, transportation, another, traffic, engine, grocery, machine, comprehensive, divorce, dirty, inter- esting, organic, great, opinion, typical, well, yeah, really, stupid, con- troversy, painful, object, funny, garage, political, sick, neighborhood, sentence, deliver, interpretation, again, disaster, poverty, complaint, ap- parent, regarding, continued
Cassette player
music, speaker, tape, radio, telephone, instrument, musical, technology, electronic, phone, musician, communication, classic, battery, opposite. listen, topic, phrase, volume, sound, television, object, exchange, mem- ory, item, record, motor, introduce, theme, communicate, cognitive, machine, context, rhythm, subject, comprehensive, contest, interpreta- tion, camera, historian, love, player, equipment, regarding, definition, historical, hello, description, creative, chapter
Parachute
air, bag, tent, fabrie, drop, above, day, swing, at, personal, helicopter, open, new, part, fairly, current, package, plane, under, and, second, fourth, another, then, first, far, favorite, from, opening, float, valley, modest, low, just, fun, third, patch, string, along, slightly, catch, flight, my, little, top, unusual. recent, launch, in, near church, religious, Catholic, religion, another, town, Christian, prayer, spiritual, priest, hospital. clinic, neighborhood, bank, museum, previous, again, newly, village, faith, where, various, last, rural, yesterday, holy, court, first, funeral, continued, recent, then, love, factory, today, sacred, cross, near, there, more, place, second, farm, school, from, something, past, porch, store, around French, another, theme, cycle, instrument, rhythm, there, again, then, composition, first, musical, music, afternoon, compose, forth, third, bell, musician, circumstance, portion, borrow, comprehensive, continued, that, hour, around, second, sound, although, assessment, while, last, proposed, fourth, fun, along, telescope, slightly, this, just, yeah, previous, though, headline, hear, arrangement, definition, addition, brief
Church
French horn
Nudity
naked, sex, hot, sexual, breast, modest, nut, and, dirty, skin, from, second, interested, physically, new, curious, also, third, just, enjoy, then, another, my, good, first, in, bad, current, day, kind, body, slightly, lovely, quite, recent, interesting, so, show, episode, near, full, primarily, unique, particularly, reveal, oh, ah, wide, today
Kelly McKernan
voice, between, put, emotional, blue, immediate, flesh, sweet, primar- ily, pale, newly, I, combination, currently, spending, fresh, consistent, provide, among, pair, international, teen, soul, income, him, kind, catch, feel, attractive, sister, pretty, fun, any, thin, and, good, inner, naturally, natural, recent, embrace, could, investigation, make, beneath, rough, post, attitude, lover, luck
Ajin Demi Human
angle, morning, prepare, human, eight, my, order, possible, hi, trace, democratic, unknown, should, political, remain, via, ill, identification, designer, new, story, heart, very, tension, nearly, just, perfect, medical, friendly, protein, hello, poor, killer, all, although, racial, thanks, religion, beginning, definition, juice, Arab, hot, senator, and, main, figure, pris- oner, day, the
17

---

Table Evaluation of the impact of hyperparameters on the erasing and preservation performance.
Method
ESR-It
ESR-5
PSR-11
PSR-5t
K TOO, N
98.72
95.60
63.80
82.96
K 100, Niier
98.64
95.84
63.76
80.72
K 100, Niser
98.84
95.96
54.12
70.96
20, Nair
96.76
92.76
63.28
80.88
K 50, Nair
96.92
91.48
63.32
81.40
K 100, Nise
98.72
95.60
63.80
82.96
vocabulary is more comprehensive but presents challenges due to the large number of nonsensical tokens (c.g., "...". ""<<w>" ). Therefore, we need to filter out these nonsensical tokens to ensure the quality the search space. The results from object-related concepts are shown in the table below.
Table Evaluation the impact of the concept space on the erasing and preservation performance.
Vocab
ESR-1t
ESR-51
PSR-1
PSR-51
Oxford
98.72
95.60
63.80
82.96
97.88
94.80
69.21
87.20
The results in Table show that the erasing performance is slightly lower when using the CLIP token vocabulary as the search space, but the preservation performance is much better with gap of 5.4% in PSR-1 and 4.2% in PSR-5. This indicates that the quality of the search space is crucial factor for the performance of our method, and different tasks might require customized search spaces to achieve better performance.
Choosing the model's parameters for fine-tuning. Firstly, it is a worth recall that the cross- attention mechanism, i.e., o (QK") VV, where Q, K, and V are the query, key, and value matrices, respectively. In text-to-image diffusion models like SD, the key and value are derived from the textual embedding of the prompt, while the query comes from the previous denoising step. The
vid
cross-attention mechanism allows the model to focus on the relevant parts of the prompt to generate the image.
Therefore, when unlearning concept, most of the time, the erasure process done by loosening the attention between the query and the key that corresponds the concept to be crased, i.c., by fine-tuning the cross-attention modules. This approach works well for object-related concepts or be explicitly described with limited textual descriptions.
artistic styles, where the target concept
However, investigated in the ESD paper Section 4.1 (Gandikota et al., 2023), concepts like 'nudity' or NSFW content can be described in various ways, many of which do not contain explicit keywords like 'nudity.' This makes it inefficient rely solely on keywords indicate the concept to be erased. It is worth noting that the standard SD model has 12 transformer blocks, each of which contains one cross-attention module but also several aan-cross-attentiin modules such as self-attention and feed-forward modules, not mention other components like residual blocks. Therefore, fine-tuning the non-cross-atteet attention modules will have a more global effect on the model, making it more robust in erasing concepts that not explicitly described in the prompt.
To further support our claims, conducted additional experiments on NSFW content erasure by fine-tuning the cross-ates coss-attention modules. The results are presented in Table
Table 8: Evaluation on the nudity erasure setting, where -r and -u denote fine-tuning the cross- attention and non-cross-attention modules, respectively.
NER-0.31
NER-0.51
NER-0.71
NER-0.8J
SD
16.69
10.91
5.46
2.02
ESD-x
10.25
5.83
2.17
0.68
ESD-u
5.32
2.36
0.74
0.23
Ours-u
3.64
1.70
0.40
0.06
18

---

It can be seen that the erasure performance by fine-tuning the mon-cross-aiee atoss-attention modules is signifi- cantly better than iin-tuning the ross-attention modules only, observed by the lower NER scores across all thresholds. This phenomenon is also observed in both the ESD and our method. Our method outperforms ESD in all settings by large margin, demonstrating the effectiveness of our method in erasing NSFW content.
B.3 Discussion on Metrics to Measure the Erasure Performance
One of the main challenges in developing crasure methods is the lack of a proper metric to measure erasure performance. Specifically, performance is evaluated by how well the model forgets the target concept while retaining other concepts. This raises a critical question: how can we validate whether concept is present in generated image? Although this may seem like simple task, it is quite challenging due to the vast number of concepts that generative models can produce. It is infeasible to have a classification model capable of detecting all possible concepts.
While the FID score is commonly used metric to assess the generative quality of models, may not be sufficient for evaluating erasure performance. To the best of our knowledge, the CLIIP alignment score is the most suitable existing metric for measuring concept inclusion. However, it is not without limitations. For example, CLIP's training set does not include NSFW content, making it less reliable for detecting such concepts. We believe that more comprehensive evaluation metric still lacking and that developing one would be valuable direction for future research.
B.4 Further Analysis on Searching for Adversarial Concepts
To further understand how our method searches for adversarial concepts, we provide intermediate results of the search process in Figure 6 The experiment is conducted on the Imagenette dataset with the same settings as in Section 5 Specifically, we simultaneously erase five concepts: "Garbage truck", "Cassette player", "Parachute" "Church", and "French horn".
In Figure we show the images generated from the most sensitive concepts ca found by our method in the odd rows, as well as the corresponding to-be-erased concepts in the even rows. It is worth noting that all images are generated from the same initial noise input 27, resulting in a similar background while still containing the target concepts, shown in the first column of the even rows.
The Removal Effect Through Fine-Turme Tuninn Steps. As shown in the even rows, we observe that the model gradually removes the to to-be-erased objects from the generated images the fine-tuning steps increase. Interestingly, these to-be-erased concepts tend to collapse into the same concept, even though they started from different concepts. For example, the "Garbage truck" and "Cassette player" in the and 4th rows eventually transform into a background-lik image in the last column. This can be explained by the fact that in the objective function 4 the erasing loss uses the same null concept c, for all to-be-erased concepts, which encourages the model to remove them simultaneously. eventually leading the collapse of these concepts into the same form. This phenomenon can be an interesting direction for future research to investigate the relationship between different concepts in the erasing process, and the benefits of using different null concepts for different to-be-erased concepts.
The Adversarial Concepts Adapt Through Fine- Tine-Tuning Steps. On the other hand, the images generated from the most sensitive concepts ca in the odd rows show how they adapt to the erasing process. Interestingly, while the adversarial concepts c, can vary in each fine-tuning step-foo example, the adversarial concept for "Garbage truck" in the first row changes from "truck", to "title", to "morning". and converges change smoothly through the increasing fine-tuning steps f. This can be explained by the continuous update of the model 0% in each fine-tuning step, making G(0), 2r, "truck") and ((0)+1,, 27, "title") are smoothly connected. This smooth transition of the generated images from the adversarial concepts ea demonstrates an advantage of our method, which allows for finding visual adversarial concepts rather than sticking to specific keywords.
"great" in the last column the generated images G(8', 27, ca)
19

---

Ieg
1o40
le60 Thomming
1-80 "great"
I+100 CL "graa""
I=120 CLT "reat
I+140 "great"
Gas"music"
"music"
C "cameraa
"Camerr"
C "camera"
comeeaa
"camera"
Convert
1-20
1o:0
1-100
I-120
40
"=""cortrreed"
Charth
I-0
1-20 musi
i-40 C "verouu
Iw60 Cysl "earrru
1-80 veriuus
I-100 Cush
i-120
i-140 "homee
cys
CAT comtsmme
Prrch
Pronch
"""air
G "uiquee
Cymh "unqquee
Tust"
"jus""
Figure 6: Intermediate results of the search process. Row-113,5,7,9; images generated from the most sensitive concepts c, found by our method. Row-2,4.6,8,10 images generated from the corresponding to-be-erased concepts. Each column represents different fine-tuning steps in increasing order.
20

---

top cirder arda model bothom dircle "mudivy" sarviined medel
pindlrrty betweer the concept Yadi ky and the takan it the tocual anbedding space
Figure 7: The figure shows the correlation between the drop of the CLIP scores (measured between generated images and their prompts) between the base/original model, and the sanitized model (i.e., removing the target concept "nudity") and the similarity score between the target concept "nudity" and other concepts in the textual embedding space. The radius of the circle indicates the variance of the CLIP scores measured in 200 samples, i.e., the larger circle indicates the larger variance of the CLIP scores.
B.5 Difficulties in Searching for Adversarial Concepts
In this section, we provide empirical examples to show that finding the most sensitive concept ca is not always straightforward when using heuristic methods, which further emphasizes the advantage of our method.
Can we use the similarity in the textual embedding space to find the most sensitive concept? Large pretrained multimodal models like CLIP have been widely used for zero-shot learning because their textual embedding space is highly correlated with the visual space. Intuitively, one might think that the similarity between the target concept c. and other concepts in the textual embedding space can help identify the most sensitive concept Ca. For example, the closer a concept e; is to the target concept C. in the textual embedding space, the more likely it is to be the most sensitive concept Ca- However, we demonstrate that this heuristic method is not always effective.
We conducted similar analysis as in Section 3.2 including the similarity score between the target concept Cg and other concepts in the textual embedding space to rank the concepts. Figure 7 shows the correlation between the drop in the CLIP scores between the base/original model and the sanitized model (i.e., after removing the target concept "nudity") and the similarity score between the target concept "nudity" and other concepts in the textual embedding space.
can be seen that the above intuition does not always hold, as the similarity score does not correlate with the drop in the CLIP scores. For example, except for the concept "naked", the null concept is the most similar to "nudity" the textual embedding space, but it experiences the lowest drop in CLIP scores. On the other hand, two concepts, "a photo" and "president", are close in the textual embedding space but are affected differently during the erasing process. This demonstrates that similarity textual embedding space is not appropriate metric for identifying the most sensitive concept in this context.
C Qualitative Results
In addition to the quantitative results presented Section 5 we provide qualitative results this section further demonstrate the effectiveness of our method compared to the baselines. Due to our
21

---

internal policy on publishing sensitive content, we are only able to show examples from two settings: erasing object-related concepts and erasing artistic concepts.
Erasing Concepts Related to Physical Objects Figures 10 and 11 show the results of erasing
object-related concepts using ESD, UCE, and our method, respectively. Figure [ shows the generated images from the original SD model. Each column represents different random seeds, and each row displays the generated images from either the to-be-erased objects or the to-be-preserved objects.
we can see that the original SD model can generate all objects effectively. When erasing objects using ESD (Figure 9), the model maintains the quality of the preserved objects, but it also generates objects that should have been crased, such as the "Church" in the second row, This aligns with the quantitative results Table where ESD achieves the lowest erasing performance.
From Figure S
When using UCE (Figure 10 the model effectively erases the objects as shown in rows 1-5, but the quality of the preserved objects is significantly degraded, such as "tench" and "English springer" in the 8th and 9th rows. This is consistent with the quantitative results Table where UCE achieves the highest erasing performance but the lowest preservation performance.
In contrast, our method (Figure T1 effectively erases the objects while maintaining the quality the preserved objects.
Erasing Artistic Concepts Figures 12 13 and 14] the results of erasing artistic style concepts using method compared to the baselines. Each column represents the erasure of specific artist, except the first column, which represents the generated images from the original SD model. Each row displays the generated images from the same prompt but with different artists. The ideal erasure should result in changes in the diagonal pictures (marked by red box) compared to the first column, while the off- off-diagonal pictures should remain the same. The results demonstrate that method effectively crases the artistic style concepts while maintaining the quality of the remaining concepts.
22

---

prompt-0
prompt
prompt-2
prompt-3
prompt-4
prompt-5
prompt-6
prompt-7
prompt-8
prompt-8 9
EREEe pooe
SONO FOR
and MO SSTTOM
Figure 8: Generated images from the original model. Five first rows are to-be-erased objects (marked
by red text) and the rest are to-be to-be-prees be-preserved objects. Each column represents different random seeds.
23

---

prompt-0
promptt
prompt-2
prompt-3
prompt-4
prompt-5
prompt-6
prompt-7
prompt-8
prompt- 9
Canract pooer
RAcaco
GESOUUPNATER
and SOTTO
Figure 9: Erasing objects using ESD. Five first rows are to-be-erased objects (marked by red text) and the rest are co-be-preservee objects. Each column represents different random seeds.
24

---

prompt-0
prompt-
prompt-2 2
prompt-3
promist-44
prompt-5
prompt-6
prompt-7
promptt-
prompt- 9
cannect ploer
Utarch
APO SOSTOO
Figure 10: Erasing objects using UCE Five first rows are to-be-erased objects (marked by red text) and the rest are to-be-preservee objects. Each column represents different random seeds.
25

---

prompt-0
prompt
prompt-2 2
prompt-3
promptt-4
prompt-S
prompt-4
prompt-7
prompt-8
prompt- 9
carrech pooer
Chanch
DNTTEO STOCE
300 DAE
Figure 11: Erasing objects using our method. Five first rows are to-be-erased objects (marked by red text) and the rest are bebeerreee objects. Each column represents different random seeds.
26

---

Ours
(b) ESD
Euguhni
malk
(c) UCE
(d) CA
Erasing artistic style concepts. Each column represents the erasure of a specific artist, except the first column which represents the generated images from the original SD model. Each row represents the generated images from the same prompt but with different artists. The ideal erasure should result in the change in the diagonal pictures (marked by red box) compared to the first column, while the off-diagonal pictures should remain the same. row-1: Portrait of woman with floral crown by Kelly McKernan; row-2: Ajin: Demi Human character portrait; row-3; Neon-lit cyberpunk cityscape by Kilian Eng; row-4: A Thomas -inkade-inspired painting of peaceful countryside; row-5: Tyler Edlin-inspired artwork of mystical forest;
Figure 12:
27

---

Ours
(b) ESD
(c) UCE
(d) CA
Figure 13: Erasing artistic style concepts (continue). Each column represents erasure specific artist, except the first column which represents the generated images from the original SD model. Each row represents the generated images from the same prompt but with different artists. The ideal erasure should result in change in the diagonal pictures (marked by red box) compared to the first column, while the off-diagonal pictures should remain the same. row-l: Whimsical fairy tale scene by Kelly Mc row-2: Sci-fi dystopian cityscape in Ajin: Demi Human style; row-3: Interstellar space station by Kilian Eng; row-4: Create a Thomas Kinkade-inspired winter wonderland; row-5; Create Tyler Edlin inllin-i portrait of a fantasy character;
28

---

Ours
(b) ESD
(c) UCE
(d) CA
Figure 14: Erasing artistic style concepts (continue). Each column represents erasure specific artist, except the first column which represents the generated images from the original SD model. Each row represents the generated images from the same prompt but with different artists. The ideal erasure should result in change in the diagonal pictures (marked by red box) compared to the first column, while the off-diagonal pictures should remain the same. row-1: Figure in flowing dress by Kelly McKernan; row-2: Creepy Ajin: Demi Human villain design; row-3: Mysterious temple ruins by Kilian Eng; row-4: A Thomas Kinkade-inspired depiction of a quaint village; row-5; Tyler Edlin-inspired cityscape night;
29

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Abstract and Section 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Appendix B.1
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
30

---

Justification:
We do not provide any theoretical result in this work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Appendix B.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
31

---

Answer: [Yes]
Justification: Our code is anonymously published at https:// http://aannooouss.4.
acienne/r/Adversariall B Erasing/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https sonideSuumissiooPP..
//nips. //nips.cc/ppl wwcppbblc/guuiicuccccccccoocoopp
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix B.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
32

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix B.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our research does not involve human subjects and we are unaware of any negative societal impact resulting from our work. All datasets used in the experiments are publicly released by their authors and we respect the terms of those that have defined licenses.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. .. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper provides theoretical analysis of the domain generalization problem
in machine learning. We are currently unaware of any potential negative societal impacts of
work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
33

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not release any data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Appendix B.I
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
34

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
35


Reviews:
[{"review_id": "zMbSsAzUKm", "reviewer": "Reviewer_Jwe1", "summary": "This paper studies the memory-forgetting tradeoff for concept removal. The authors systematically summarize the tradeoff problem, and propose the idea of adversarial concepts to solve it. Specifically, this approach automatically detects the most sensitive concepts that will be affected by unlearning, and enforces the model to maintain the performance of sensitive concepts by adding the maintaining loss.", "questions": "How to select the parameter $\\lambda$? It seems that different concepts may need different coefficients.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written\n\n2. The summary of the performance drop problem for unlearning is systematic.", "weaknesses": "1. Results are mainly on SD1.4. Additional results on other stable diffusion models should be included. For example, on larger diffusion models like SD XL, to further validate the effectiveness of the proposed method.\n2. In the experiment section, comparisons with existing state-of-the-art methods, such as SPM [1], are lacking.\n3. The ESR-k and PSR-k metrics used by the authors are reasonable and widely adopted by previous methods. However, additional metrics, such as FID and CLIP Score, should also be included to demonstrate the model's performance in the object unlearning scenario. Similarly, evaluating the FID metric on COCO-30K is also necessary.\n4. When visualizing results in the appendix, the authors should show images before and after unlearning to provide a more intuitive sense of performance. Specifically, changes in unrelated concepts should be minimal. The current images only indicate that the concept has not been forgotten, rather than showing that the generated images under the prompt have not changed.\n5. Efficiency is also an important evaluation metric in machine unlearning. The authors should adequately compare the efficiency of their method with previous methods.\n6. When comparing methods, the authors should contrast their preservation methods with previous methods, such as the modules used in ConAbl [2] and SPM [1], to highlight performance differences.\n\n[1] One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models, and Erasing Applications\n[2] Ablating Concepts in Text-to-Image Diffusion Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We would like to address the remaining concerns as follows. \nDue to the space limitation, some responses are provided in the author's comment section.\n\n**Q: Additional experiments with SOTA methods such as ConAbl and SPM, and evaluation with FID and CLIP scores on COCO-30K dataset.**\n\nWe followed the reviewer's suggestion and conducted additional experiments with MACE, a SOTA method recently accepted at CVPR 2024.\nIt is worth noting that we already compared with the suggested baseline ConAbl denoted as CA in our paper.\n\nSpecifically, we performed the object-related setting which includes four tasks, each involving the erasure of five concepts from the Imagenette dataset.\nIn addition to ESR-1, ESR-5, PSR-1, and PSR-5 metrics, we generated images with common concepts from the COCO-30K dataset and assessed FID and CLIP scores.\nDue to time constraints, we completed one task, erasing five concepts: Cassette Player, Church, Garbage Truck, Parachute, and French Horn.\n\nThe results are presented in the table below.\n\n| Method |  | ESR-1 ↑ |  | ESR-5 ↑ |  | PSR-1 ↑ |  | PSR-5 ↑ |  | FID ↓ |  | CLIP ↑ |  | FT Time |\n|--------|--|---------|--|---------|--|---------|--|---------|--|-------|--|--------|--|---------|\n| ESD    |  | 95.5 ± 0.8 |  | 88.9 ± 1.0 |  | 41.2 ± 12.9 |  | 56.1 ± 12.4 |  | 17.9  |  | 24.5   |  | 40 mins |\n| UCE    |  | 100 ± 0.0  |  | 100 ± 0.0  |  | 23.4 ± 3.6  |  | 49.5 ± 8.0  |  | 19.1  |  | 21.4   |  | 4 mins  |\n| CA     |  | 98.4 ± 0.3 |  | 96.8 ± 6.1 |  | 44.2 ± 9.7  |  | 66.5 ± 6.1  |  | 16.6  |  | 25.8   |  | 12 mins |\n| MACE   |  | 99.3 ± 0.3 |  | 97.6 ± 1.2 |  | 47.4 ± 12.0 |  | 72.8 ± 10.5 |  | 16.9  |  | 24.9   |  | 3 mins  |\n| Ours   |  | 98.6 ± 1.1 |  | 96.1 ± 2.7 |  | 55.2 ± 10.0 |  | 79.9 ± 2.8  |  | 16.3  |  | 26.1   |  | 65 mins |\n\nRegarding erasing performance, MACE slightly outperformed our method by 0.7% in ESR-1 and 0.5% in ESR-5 on average.\nHowever, in terms of preservation performance, our method significantly outperformed MACE, with an 8% gap in PSR-1 and a 7% gap in PSR-5.\nAdditionally, our method achieved superior preservation performance in image generation with common concepts, evidenced by the lowest FID score of 16.3 and the highest CLIP score of 26.1.\n\nThese results indicate that while MACE shows marginally better erasing performance, our method excels significantly in preservation performance, even when compared to the latest techniques.\n\n**Compared to SPM:**\n\nThe SPM method, although effective in concept erasure, does not directly fine-tune the original model.\nInstead, it trains a separate adapter that, when attached to the original model, prevents the generation of the erased concept.\nSpecifically, Section 3.1 of the SPM paper introduces a new diffusion process $\\hat{\\epsilon} = \\epsilon(x_t, c, t \\mid \\theta, \\mathcal{M}_{c_e})$\n\n, where $\\mathcal{M}_{c_e}$ is the adapter model trained to erase the concept $c_e$.\nWhile these adapters can be shared and reused across different models, the original model $\\theta$ remains unchanged, allowing malicious users to generate the erased concepts easily.\nIn contrast, our method, along with the other baselines in our paper, directly erases the concept from the original model, making it more robust and preventing the generation of the erased concepts.\nFor this reason, we did not include SPM in the comparison.\n\n**Q: Additional experiments with larger models**\n\nDue to resource constraints in the short rebuttal timeframe, we were only able to conduct additional experiments on SD v1.4. However, it is worth noting that SD v1.4 is still the most widely used model in the community, including recent work such as MACE.\nWe will consider conducting experiments on larger models like SD XL in future work.\n\n**Q: Evaluating Efficiency**\n\nFollowing the reviewer's suggestion, we provided the fine-tuning time for the object-related concepts task in the table above.\nIt is worth noting that we could only find efficiency evaluation in the SPM paper, which is specifically designed to highlight the efficiency of their lightweight adapter approach.\nOur method which was designed to prioritize erasing and preserving performance, may not be as efficient as SPM in terms of fine-tuning time.\nHowever, we politely argue that efficiency is not the main concern in our work or other baselines, as the fine-tuning process is relatively fast, typically completing in less than a few hours, which is acceptable in practice for infrequent concept erasure requests.\n\n**Q: Evaluating the impact of $\\lambda$**\n\nTo investigate the impact of different $\\lambda$ values, we conducted additional experiments on the object-related concepts task with $\\lambda = 0.1, 0.5, 5, 10$.\nThe results are presented in the table below.\nThere is a clear trade-off between erasing and preserving performance when changing the $\\lambda$ value.\nA smaller $\\lambda$ value results in better erasing performance but worse preservation performance, and vice versa.\nIn our experiments, we did not attempt to tune $\\lambda$ for each concept but just simply set it to 1 for all experiments as mentioned in line 774.\n\n| λ     | ESR-1 ↑ | ESR-5 ↑ | PSR-1 ↑ | PSR-5 ↑ |\n|------------|---------|---------|---------|---------|\n| 0.1    | 97.88   | 94.52   | 29.44   | 40.72   |\n| 0.5    | 98.28   | 94.32   | 56.04   | 73.92   |\n| 1      | 98.72   | 95.60   | 63.80   | 82.96   |\n| 5      | 96.96   | 91.68   | 74.84   | 91.52   |\n| 10     | 91.64   | 84.48   | 83.04   | 96.64   |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uRYR7PKM7V", "reviewer": "Reviewer_r1Xs", "summary": "This paper focuses on the problem that existing concept erasing methods struggle to address the trade-off between the generation capability of erased concepts and remaining concepts. To address this problem, this paper proposes a method that erases the target concept while minimizing the impact of other concepts. Specifically, this paper finds that related concepts are sensitive during the erasing process of the target concept. For example, when erasing 'nudity', some related concepts, such as 'woman' and 'people' will be significantly impacted. Motivated by this observation, this paper first utilizes an optimization method to automatically find the adversarial concepts related to the target concept. Following this, this paper iteratively applies the preservation constraint on these adversarial concepts during the erasing process.", "questions": "Please check weaknesses.", "limitations": "NA", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- This paper provides an empirical observation that removing different target concepts leads to varying impacts on other concepts, which is interesting and helpful for future research.\n- In the experiment of erasing object-related concepts, the proposed method demonstrates the effectiveness in maintaining the generation capability of remaining concepts.", "weaknesses": "- The motivation of this paper needs further discussed. My core issue is: Do all related concepts need to be preserved? For example, if we want to erase 'airplane', do 'aircraft' and 'warplane' need to be preserved? Motivated by this issue, I argue that there should be a boundary between the preserved and erasing concepts, while this paper ignores this boundary. More extremely, some works [1,2,3] argue that we should erase concepts related to the target concept.\n- Lack of comparison with the latest methods, such as MACE [4].\n- In the experiment of erasing object-related concepts, it is suggested to add an experiment that demonstrates the generalization of the model. For example, following MACE [4], this paper can evaluate the erasing capability on the synonyms of the erasing concept.\n- In the experiment of erasing NSFW content, this paper lacks an evaluation of the generation capability on the common concepts. Following [2,3], this paper can evaluate the FID of the model on COCO dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and suggestions. We would like to address the remaining concerns as follows. \nDue to the space limitation, some responses are provided in the global rebuttal.\n\n**Q: Do all related concepts need to be preserved? Boundary between preserved and erasing concepts**\n\nWe agree with the reviewer that when erasing a target concept like 'airplane,' the synonyms that resemble it visually, such as 'aircraft,' should also be erased.\nHowever, while our method does not explicitly set a boundary when searching for the adversarial concepts, the optimization process in our method naturally selects the most affected concepts but not the `similar' concepts to the target concept to be preserved.\nWe support our response with both theoretical and empirical evidence.\n\nTheoretical perspective:\n\nOur optimization framework, described in Equations 4 and 5, involves a bilevel optimization process. \nThe outer level (w.r.t. $\\theta'$) minimizes the erasing loss L1 and the preservation loss L2 simultaneously, \nwhile the inner level (w.r.t. $c_a$) maximizes L2 to select the most affected concepts to be preserved. \n\nInitially, $\\theta' = \\theta$, therefore after minimizing L1, the most affected concepts will be exactly the target concept $c_e$.\nAs optimization progresses, the model $\\theta'$ diverges from the original model $\\theta$, driving the output $\\epsilon_{\\theta'}(c_e)$ away from $\\epsilon_{\\theta}(c_e)$ and closer to $\\epsilon_{\\theta}(c_n)$. \nThis process makes the target concept $c_e$ one of the affected concepts or a candidate for the inner maximization w.r.t. $c_a$. \nHowever, selecting $c_a = c_e$ would directly conflict with the erasing loss L1 that aims to drive $c_e \\rightarrow c_n$. \nConsequently, the inner maximization process steers towards selecting the most affected concepts but not the target concept $c_e$ or its synonyms.\n\nEmpirical perspective:\n\nAs mentioned in Appendix B.4 (lines 835-844) and shown in Figure 12, the intermediate results of the adversarial concept selection process indicate that the model initially selects concepts similar to the target. For instance, Figure 12 shows that 'truck,' 'music,' 'church,' 'French,' and 'bag' are selected as adversarial concepts for 'Garbage truck,' 'Cassette player,' 'Church,' 'French horn,' and 'Parachute,' respectively. Over time, the model shifts towards less similar but highly affected concepts.\n\nMoreover, our response to another question demonstrates that our method effectively erases both the target concept and its synonyms, as shown by the results of erasing the synonyms.\n\nIn summary, both theoretical and empirical evidence support that our method naturally prioritizes preserving the most affected concepts, which are not the synonyms of the target concept but those significantly impacted by its erasure.\n\n**Q: Compare to MACE**\n\nFollowing the reviewer's suggestion, we conducted an additional experiment using MACE with their official implementation on object-related concepts, as detailed in section 5.1 of the main paper.\nSpecifically, we conducted four distinct tasks, each involving the erasure of five concepts from the Imagenette dataset.\nIn addition, we evaluated the preservation performance with common concepts from the COCO-30K dataset with FID and CLIP scores.\nDue to time constraints, we were only able to complete one task, which involved erasing five concepts: Cassette Player, Church, Garbage Truck, Parachute, and French Horn.\n\nThe results are presented in Table 1 in the attached document.\nRegarding erasing performance, MACE slightly outperformed our method by 0.7\\% in ESR-1 and 0.5\\% in ESR-5 on average.\nHowever, in terms of preservation performance, our method significantly outperformed MACE, with a gap of 8\\% in PSR-1 and 7\\% in PSR-5.\nAdditionally, our method achieved superior preservation performance in image generation with common concepts, evidenced by the lowest FID score of 16.3 and the highest CLIP score of 26.1.\n\nThese results indicate that while MACE shows marginally better erasing performance, our method excels significantly in preservation performance, even when compared to the latest techniques.\n\n**Q: Evaluating FID on COCO dataset in erasing NSFW setting**\n\nWe already measured the FID on the COCO dataset and provided the results in Table 2 in the main paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YmetA35Etj", "reviewer": "Reviewer_Zhhp", "summary": "The present paper addresses the challenge of erasing content from text-to-image diffusion models with a focus on reducing the degenerative impact on other concepts. To this end, the authors propose a novel approach that focuses on identifying and preserving adversarial concepts—those which are most affected by changes in model parameters during the erasure. To provide evidence, the authors conduct empirical investigations and various experiments using the open-source Stable Diffusion model, demonstrating their method’s potential to reliably erase target concepts while offering minimal impact on non-target concepts.", "questions": "- As far as I understand, the set of concepts used to search for adversarial concepts is derived from the list of words in the initial prompt (lines 76-79). However, it appears to be a common set used across multiple examples. Can you provide more details and clarify how this set of concepts is selected? Have you experimented with different sets of initial concepts?\n\n- Further, you mentioned in lines 252-255 that, in the case of mitigating unethical content, it is necessary to fine-tune the non-cross-attention modules. Can you elaborate on why this is the case?\n\nMinor comments: \n- In line 238, CA seems to be the best baseline, not ESD.", "limitations": "The authors addressed limitations in the appendix as stated in the checklist.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-structured, providing clear and comprehensive explanations of the proposed method and its theoretical foundations. \n- The novelty of focusing on adversarial concepts to balance erasure and preservation is a significant contribution to the field. \nThe experiments are well selected. They first showcase the reliability of the proposed method on a generic task, followed by two relevant use cases, namely erasing unethical and artistic concepts.\n- While not presented in the main paper, the authors address limitations of their proposed method in the appendix.", "weaknesses": "- While the paper showcases the benefits of the proposed method, the observations and conclusions drawn from the empirical experiments do not fully align with what is described by the authors, particularly regarding the minimal impact on other concepts. For example, in lines 231-232, the authors state that the proposed method achieves much higher ESR scores than the two baselines ESD and CA; however, this claim is not entirely accurate, especially in the case of CA. Similarly, the improvements over the baselines shown in the third experiment are not as significant as described, particularly when considering the standard deviations presented in Table 3. The paper would benefit from a more detailed discussion and analysis of these results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and insightful suggestions. We would like to address the remaining concerns as follows. \n\n**Q: Better discussion** \n\nWe thank the reviewer for pointing this out. We will revise the comparison with CA in the revised version, i.e., our method is slightly better than CA in terms of ESR scores but significantly better in terms of PSR scores.\n\n**Q: How to choose the concept space** \n\n\nWe did not design specific but utilized a common set of concepts as the search space $\\mathcal{R}$ for searching adversarial concepts across all experiments. \nTo ensure the generality of the search space so that it can be applied to various tasks such as object-related concepts, NSFW content, and artistic styles, \nwe used the Oxford 3000 most common words in English as the search space. \n\nIt is worth noting that as described in Section B.1, our method employ the Gumbel-Softmax trick to discretely search in the concept space $\\mathcal{R}$, \nthis approach requires feeding the model with the embeddings of the entire search space $\\mathcal{R}$ to compute the alignment score, \nwhich is computationally expensive when the search space is large.\nTo mitigate this, we use a subset of the K most similar concepts to reduce the computational cost.\n\nTo better address the reviewer's concern, we conducted additional experiments with the search space as the CLIP token vocabulary, which includes 49,408 tokens. \nIt is worth noting that the CLIP token vocabulary is more comprehensive but presents challenges due to the large number of nonsensical tokens (e.g., )\nTherefore, we need to filter out these nonsensical tokens to ensure the quality of the search space. \nThe results from object-related concepts are shown in the table below. \n\n| Vocab  | ESR-1 ↑ | ESR-5 ↑ | PSR-1 ↑ | PSR-5 ↑ |\n|--------|---------|---------|---------|---------|\n| Oxford | 98.72   | 95.60   | 63.80   | 82.96   |\n| CLIP   | 97.88   | 94.80   | 69.24   | 87.20   |\n\nThe results show that the erasing performance is slightly lower when using the CLIP token vocabulary as the search space, \nbut the preservation performance is much better with a gap of 5.4\\% in PSR-1 and 4.2\\% in PSR-5.\nThis indicates that our method would benefit from a more comprehensive search space. \nWe will add this experiment to the revised version.\n\n**Q: it is necessary to fine-tune the non-cross-attention modules in erasing NSFW concept** \n\nFirstly, we would like to recall the cross-attention mechanism, i.e., $\\sigma(\\frac{(QK^T)}{\\sqrt{d}})V$, where $Q$, $K$, and $V$ are the query, key, and value matrices, respectively. \nIn text-to-image diffusion models like SD, the key and value are derived from the textual embedding of the prompt, while the query comes from the previous denoising step.\nThe cross-attention mechanism allows the model to focus on the relevant parts of the prompt to generate the image.\n\nTherefore, when unlearning a concept, most of the time, the erasure process is done by loosening the attention between the query and the key that corresponds to the concept to be erased, i.e., by fine-tuning the cross-attention modules.\nThis approach works well for object-related concepts or artistic styles, where the target concept can be explicitly described with limited textual descriptions.\n\nHowever, as investigated in the ESD paper Section 4.1, concepts like 'nudity' or NSFW content can be described in various ways, many of which do not contain explicit keywords like 'nudity.'\nThis makes it inefficient to rely solely on keywords to indicate the concept to be erased.\nIt is worth noting that the standard SD model has 12 transformer blocks, each of which contains one cross-attention module but also several non-cross-attention modules such as self-attention and feed-forward modules, not to mention other components like residual blocks.\nTherefore, fine-tuning the non-cross-attention modules will have a more global effect on the model, making it more robust in erasing concepts that are not explicitly described in the prompt. \nWe will add more explanations to the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5xPEJ62elZ", "reviewer": "Reviewer_fKJG", "summary": "This paper investigates erasing undesirable concepts from stable diffusion. The paper builds up on a common finding on related works, that is removing even one concept can significantly rescue model’s ability to generate other concepts. Existing methods typically select a neutral concept, such as \"a photo\" or an empty string, as an anchor to preserve while erasing the target concept, expecting that maintaining the neutral concept should help retain other concepts as well (UCE, TIME). However, even regularization attempts using these neutral concept still affects the desirable concepts. This paper first find most sensitive concept for erasure, and then using a min-max (adversarial) adds another term to the loss function to remove only undesirable concepts.", "questions": "- Which CLIP model is used?\n- Why using a CLIP alignment score is a reliable measure for concept inclusion? For example a CLIP model that is trained on non-NSFW or non-GORE data cannot detect nudity/violence. What would your proposed method rely as a capability measure then?\n- Line 152: the results of erasing the nudity concept, provided in the Appendix... I could not see this results. Best is also to have reference to specific plot/section in the appendix whenever you refer to appendix. \n- Line 255: finetuning the non-cross-attention modules. I belive that including more details here is crucial. Why do you finetune only non-cross attention modules?\n- Line 275: Did you see that FID is sufficient as a quality measure of generated samples when removing concepts? For example, removing nudity might distort anatomy and is not reflected in FID?\n- Figure 4: We know that for nudity there is a huge difference between Feet and female breast for example. How would your proposed method only remove the later while keeping the non-sensitive parts untouched?\n- What is the level of granularity in the proposed method? e.g. can we use this method to remove Mercedes logo from cars?\n- How does your proposed method differ to \"Forget-Me-Not\" paper [1]? Any specific reason this is not covered in comparisons?\n- I am curious to see what you think of task vectors [2] as a potential direction to remove undesirable concepts?\n\n> code\n- Code: train_adversarial_gumbel.py - lines 461-490: I could not see the implementation of equation (4) in these lines. Should not it be \"loss += criteria(z_n_wo_prompt_pred.to(devices[0]), z_0_org_pred.to(devices[0]))\" for L1 and \"loss = -negative_guidance * criteria(z_r_wo_prompt_pred.to(devices[0]), z_r_org_pred.to(devices[0]))\" for L2?\n- When using L2 distance for similarity between vocab and the erasure wor, why do you use K-means? Why not using the top-n from the similarity result? did you look into result differences between these two?\n- In the case of Stable Diffusion 3 (and SDXL), we have to have pooled and non-pooled captions. How would you calculate emb_r for the pooled embedding needed to  condition on time?\n\n[1] Zhang, Gong, et al. \"Forget-me-not: Learning to forget in text-to-image diffusion models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n[2] Ilharco, Gabriel, et al. \"Editing models with task arithmetic.\" arXiv preprint arXiv:2212.04089 (2022).", "limitations": "already addressed as questions for further discussions", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Paper is well written and has a great flow. Authors clearly state the problem, current solutions and their drawback. They build their approach on these drawback and support their proposed method using different experiments.", "weaknesses": "- wrong citation for UCE in line 23\n- Lines 29-30 are written as a strong conjecture without support. \"no specific part of the model’s weights is solely responsible for a single concept\". I have seen counter examples in the literature. See [1] for example\n- While proposed method is showing improvement over related works, I doubt the practical usage and hence raise some questions.\n\n\n\n\n[1] Basu, Samyadeep, et al. \"On Mechanistic Knowledge Localization in Text-to-Image Generative Models.\" Forty-first International Conference on Machine Learning. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and insightful suggestions. We would like to address the remaining concerns as follows. \nDue to the space limitation, some responses are provided in the global rebuttal and the author comment section. \n\n**Q: Why using a CLIP alignment score is a reliable measure for concept inclusion?** \n\nIn Section 3.2, we investigated which concepts are affected most by the erasure of a target concept. \nThe concept space that we investigate is not only a specific set of related concepts as shown in Figure 1 but also a broader set of concepts of the CLIP token vocabulary which includes 49,408 tokens as shown in Figure 2. \nIn this general and broader concept space, it is nearly impossible to have a pre-trained classification model that can detect all concepts. \nTherefore, to the best of our knowledge, the CLIP alignment score might be the most effective way to measure the concept inclusion, \neven though it is not perfect and might not be able to provide a reliable measure for some concepts as suggested by the reviewer. \nWe will add this discussion to the revised version.\n\n**Q: Results of erasing nudity concept\"**\n\nWe apologize for the confusion. We would like to provide the results of erasing the \"nudity\" concept while preserving the \"person\" concept in the attached document.\nAs shown in the figure, preserving related concepts like \"person\" helps to retain the model's capability on other concepts much better than preserving a neutral concept.\n\n**Q: Why do you finetune only non-cross attention modules to erase NSFW concepts?** \n\nFirstly, we would like to recall the cross-attention mechanism, i.e., $\\sigma(\\frac{(QK^T)}{\\sqrt{d}})V$, where $Q$, $K$, and $V$ are the query, key, and value matrices, respectively. \nIn text-to-image diffusion models like SD, the key and value are derived from the textual embedding of the prompt, while the query comes from the previous denoising step.\nThe cross-attention mechanism allows the model to focus on the relevant parts of the prompt to generate the image.\n\nTherefore, when unlearning a concept, most of the time, the erasure process is done by loosening the attention between the query and the key that corresponds to the concept to be erased, i.e., by fine-tuning the cross-attention modules.\nThis approach works well for object-related concepts or artistic styles, where the target concept can be explicitly described with limited textual descriptions.\n\nHowever, as investigated in the ESD paper Section 4.1, concepts like 'nudity' or NSFW content can be described in various ways, many of which do not contain explicit keywords like 'nudity.'\nThis makes it inefficient to rely solely on keywords to indicate the concept to be erased.\nIt is worth noting that the standard SD model has 12 transformer blocks, each of which contains one cross-attention module but also several non-cross-attention modules such as self-attention and feed-forward modules, not to mention other components like residual blocks.\nTherefore, fine-tuning the non-cross-attention modules will have a more global effect on the model, making it more robust in erasing concepts that are not explicitly described in the prompt. \nWe will add more explanations to the revised version.\n\n**Q: Discussion on FID metric?**\n\nWe thank the reviewer for raising a very important question.\n\nFirstly, we utilized FID measured on common concepts like the COCO-30K dataset to evaluate the generation quality of the sanitized models because it is a widely used metric in the field of generative models. \nHowever, while this metric is useful in general generation settings, we acknowledge that it might not be sufficient for evaluating unlearning concepts.\n\nAs observed in our paper, the impact of erasure methods is not equally distributed across all concepts; some concepts might be more affected than others. \nWhile this imbalance might be mitigated with a large enough evaluation size that covers all possible concepts, making FID sufficient, this is not the case in practice, which commonly uses the COCO-30K dataset.\n\nTherefore, we believe that a more comprehensive evaluation metric that can capture the impact of erasing concepts on specific concepts is needed. \nFor example, we could assign a weight to each evaluated concept/prompt based on its relevance score to the target concepts, then compute a weighted FID or CLIP score.\n\n**Q: How would your proposed method only remove the sensitive parts while keeping the non-sensitive parts untouched in the NSFW setting?**\n\nWhile our method has demonstrated an interesting result in its ability to remove more sensitive body parts, we do not have a specific explanation for this phenomenon. However, intuitively speaking, if we consider two abstract concepts, \"nudity\" and \"person,\" and two core concepts, \"breast\" and \"feet,\" we can see that the \"nudity\" concept might be highly correlated with the \"breast\" concept, while the \"person\" concept might be more correlated with the \"feet\" concept rather than \"breast.\"\n\nTherefore, if we specifically preserve the \"person\" concept while erasing the \"nudity\" concept, the model might be able to generate images that include \"feet\" but exclude \"breast.\" Our method, by selecting the most affected concepts, might naturally choose a concept that is highly correlated with non-sensitive body parts to be preserved.\n\n**Q: What is the level of granularity in the proposed method?**\n\nOur method does not have a specific mechanism to control the level of granularity or focus on a specific concept to be erased. Instead, our main goal is to prioritize the preservation aspect of the erasure process. However, if we were to address the problem of granularity, we would focus on enhancing the expressiveness of the textual description.\n\nFor example, using visual embeddings to describe the Mercedes logo concept could be a potential direction to improve the expressiveness and granularity of the method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their insightful comments and suggestions. Below are our responses to some important questions raised by the reviewers. We kindly request the reviewers to consider raising the scores if our responses adequately address the remaining concerns.

**Q: Compare to MACE a SOTA method**

Following the reviewer's suggestion, we conducted an additional experiment using MACE (accepted to CVPR 2024) with their official implementation on object-related concepts, as detailed in section 5.1 of the main paper.
Specifically, we conducted four distinct tasks, each involving the erasure of five concepts from the Imagenette dataset.
In addition, we evaluated the preservation performance with common concepts from the COCO-30K dataset with FID and CLIP scores.
Due to time constraints, we were only able to complete one task, which involved erasing five concepts: Cassette Player, Church, Garbage Truck, Parachute, and French Horn.

The results are presented in Table 1 in the attached document and below

| Method | ESR-1 ↑       | ESR-5 ↑       | PSR-1 ↑       | PSR-5 ↑       | FID ↓ | CLIP ↑ |
|--------|---------------|---------------|---------------|---------------|-------|--------|
| ESD    | 95.5 ± 0.8    | 88.9 ± 1.0    | 41.2 ± 12.9   | 56.1 ± 12.4   | 17.9  | 24.5   |
| UCE    | 100 ± 0.0     | 100 ± 0.0     | 23.4 ± 3.6    | 49.5 ± 8.0    | 19.1  | 21.4   |
| CA     | 98.4 ± 0.3    | 96.8 ± 6.1    | 44.2 ± 9.7    | 66.5 ± 6.1    | 16.6  | 25.8   |
| MACE   | 99.3 ± 0.3    | 97.6 ± 1.2    | 47.4 ± 12.0   | 72.8 ± 10.5   | 16.9  | 24.9   |
| Ours   | 98.6 ± 1.1    | 96.1 ± 2.7    | 55.2 ± 10.0   | 79.9 ± 2.8    | 16.3  | 26.1   |

Regarding erasing performance, MACE slightly outperformed our method by 0.7\% in ESR-1 and 0.5\% in ESR-5 on average.
However, in terms of preservation performance, our method significantly outperformed MACE, with a gap of 8\% in PSR-1 and 7\% in PSR-5.
Additionally, our method achieved superior preservation performance in image generation with common concepts, evidenced by the lowest FID score of 16.3 and the highest CLIP score of 26.1.

These results indicate that while MACE shows marginally better erasing performance, our method excels significantly in preservation performance, even when compared to the latest techniques.

**Q: Evaluating performance on erasing synonyms**

We follow the suggestion to evaluate the erasing capability on the synonyms of object-related concepts, e.g., "Church". 
More specifically, we first utilize a set of tools including ChatGPT, Dictionary/Thesaurus.com, and Google image search to find the best synonyms for each target concept.
To verify that these synonyms are indeed resembling the target concept, we then use the original model to generate images from the synonyms (e.g., "a photo of Chapel"), 
and using the ResNet-50 model to classify the generated images. 
We then only keep the synonyms that have the top-5 accuracy higher than 50\% to ensure that they are indeed generation-similar to the target concept. 
To this end, for some concepts, we could not find any good synonyms such as "Golf ball" or "Chain saw" except for some minor variations.
We provide (top-1 and top-5) accuracy of the synonyms as below, as well as those numbers of target concepts, the higher the accuracy, the more similar the synonyms are to the target concept. Due to the space constraint, we are only able to provide the numbers of 6/10 concepts.

- **Church (84.4;100.0)**: chapel (80.0;100.0), cathedral (50.0;100.0), minster (87.5;100.0), basilica (32.5;100.0)
- **Garbage truck (83.2;99.2)**: trash truck (87.5;97.5), refuse truck (80.0;100.0), waste collection vehicle (97.5;100.0), sanitation truck (47.5;100.0)
- **Parachute (95.2;99.2)**: skydiving chute (93.9;100.0), paraglider (100.0;100.0)
- **Chain saw (76.4;89.0)**: chainsaw (92.0;96.0), power saw (26.0;58.0)
- **Tench (76.0;98.0)**: cyprinus tinca (60.0;95.0), cyprinus zeelt (52.5;100.0)
- **Golf ball (98.2;99.2)**: golfing ball (99.0;99.0)

Given the list of `valid' synonyms, we then generate images from the synonyms using the sanitized models obtained from four object-related settings (each set corresponds to erasing five Imagenette concepts simultaneously).

The results are shown in Table 2 in the attached document and as below:

| Method | $\text{ESR}_{s}$-1 ↑ | $\text{ESR}_{s}$-5 ↑ | $\text{PSR}_{s}$-1 ↑ | $\text{PSR}_{s}$-5 ↑ |
|--------|----------------------|----------------------|----------------------|----------------------|
| SD-org | 22.0 ± 11.6          | 2.4 ± 1.4            | 78.0 ± 11.6          | 97.6 ± 1.4           |
| SD-syn | 41.5 ± 8.2           | 7.5 ± 2.3            | 58.5 ± 8.2           | 92.5 ± 2.3           |
| UCE    | 99.8 ± 0.1           | 99.2 ± 0.5           | 19.5 ± 4.4           | 43.8 ± 0.6           |
| MACE   | 98.1 ± 0.9           | 84.7 ± 2.2           | 41.4 ± 10.3          | 73.3 ± 3.1           |
| Ours   | 85.2 ± 6.1           | 72.3 ± 7.1           | 46.6 ± 6.5           | 82.9 ± 4.5           |

Firstly, if comparing SD-org and SD-syn (the original/target concept and the synonyms), 
we can see that the top-1 accuracy of SD-syn is significantly lower than that of SD-org, but the top-5 accuracy is lower `only' by 5\% on average. 
This does not mean that the model could not generate meaningful images from the synonyms, but rather the images are not recognized as the target concept by the ResNet-50 model in top-1 prediction but still in top-5.

Secondly, MACE is the best method in terms of erasing synonyms, followed by our method. 
However, in trade-off, our method is better in preserving performance, not only the original concepts as shown in the main paper but also the synonyms. 

Thirdly, we should also consider the FID and CLIP scores to evaluate the preservation performance of the model, in which our method is much better than MACE.

Meta-review: This paper studies methods to erase certain concepts from diffusion models, specifically focusing on NSFW and artistic concepts as a main use case. The challenge here is that if certain concepts are removed, then highly correlated ones could be impacted (e.g., removing 'nudity' may impact general generation of people). Compared to prior work, this paper focuses on an adversarial learning approach, looking at concepts that are most affected by parameter changes. The authors also investigate the trade-off between removing some concepts while actively trying to not adversely impact other concepts. This differentiates the present work from other baselines (e.g., Forget-Me-Not) because prior work focuses more on erasing concepts, rather than the trade-off between erasure and preservation.

This is a tough paper to evaluate. The reviewers all agree that this is a very important and timely problem to study, as diffusion models are becoming more wide-spread, while still not being very well understood in terms of undesirable concept generation. Reviewers have several complaints about the comparison to different methods (e.g., MACE), the inclusion various metrics (e.g., FID), and way the conclusions are drawn from the experimental results. For the last point, a core criticism is that the authors have not thoroughly discovered where the concepts originate from (e.g., different layers of the model vs. the latent embeddings). However, the authors have addressed many of these negative points in the appendix and in the rebuttal.

The reviewers that vote for acceptance are effectively championing the paper, where they have also had a very thorough back-and-forth with the authors during the rebuttal period. Namely, the main selling point is that the paper tries to address an important problem, and make a real world impact. The reviewers that vote for rejection raise valid points. However, I think that the importance and inherent complexity of the problem means that we cannot expect clear-cut evaluations, and hence, it is worth accepting a paper that will have impact by inspiring more nuanced follow-up work. The space of concept erasure in diffusion models is also not especially crowded at the moment, and therefore, the present work does present a lot of novel findings, research frameworks, and empirical insights compared to the related work.

Overall, I recommend acceptance. 

For the authors, please add experiments comparing to forget-me-not, adding experiments for SDXL (if possible) and include much more details in the appendix and main paper to address the comments from the reviewers. Also, as a way to address the negative reviews, please consider rephrasing some of the findings in the abstract/intro to be more clear about the findings of the paper. For example it is too broad to say that "This approach ensures stable erasure with minimal impact on the other concepts."
Decision: Accept (poster)