Towards Test-Time Refusals via Concept Negation
Authors: Peiran Dong
Song Guo
Junxiao Wang
Bingjie WANG
Jiewei Zhang
Ziming Liu
Venue: NeurIPS 2023 poster (2023)
Keywords: Diffusion models
test-time refusal
concept negation
safety in generative models
TL;DR: Existing concept negation methods are insufficient when it comes to handling interdependent concepts, which are essential for confining the output space of generative diffusion models that produce unbounded outputs.

Abstract: Generative models produce unbounded outputs, necessitating the use of refusal techniques to confine their output space. Employing generative refusals is crucial in upholding the ethical and copyright integrity of synthesized content, particularly when working with widely adopted diffusion models. "Concept negation'' presents a promising paradigm to achieve generative refusals, as it effectively defines and governs the model's output space based on concepts, utilizing natural language interfaces that are readily comprehensible to humans. However, despite the valuable contributions of prior research to the field of concept negation, it still suffers from significant limitations. The existing concept negation methods, which operate based on the composition of score or noise predictions from the diffusion process, are limited to independent concepts (e.g., ``a blonde girl`` without ``glasses``) and fail to consider the interconnected nature of concepts in reality (e.g., ``Mickey mouse eats ice cream`` without ``Disney characters``). Keeping the limitations in mind, we propose a novel framework, called $ProtoRe$, to improve the flexibility of concept negation via test-time negative concept identification along with purification in the feature space. $ProtoRe$ works by incorporating CLIP's language-contrastive knowledge to identify the prototype of negative concepts, extract the negative features from outputs using the prototype as a prompt, and further refine the attention maps by retrieving negative features. Our evaluation on multiple benchmarks shows that $ProtoRe$ outperforms state-of-the-art methods under various settings, in terms of the effectiveness of purification and the fidelity of generative images.

Towards Test-Time Refusals via Concept Negation
Peiran Dong'
Song Guo Junxiao Wang"*
Bingjie Wang'
Jiewei Zhang' Ziming Liu'
Hong Kong Polytechnic University
"Hong Kong University of Science and Technology 'King Abdullah University of Science and Technology & SDAIA-KAUSE AI
{peiran.dong bingjie. w
wchann,ziiee ziming waaming..iuu songguoBcse ust. hk junxiao. wang@kaust. edu.
hk
Abstract
Generative models produce unbounded outputs, necessitating the use of refusal techniques to confine their output space. Employing generative refusals is crucial in upholding the ethical and copyright integrity of synthesized content, particularly when working with widely adopted diffusion models. "Concept negation" presents a promising paradigm to achieve generative refusals, as it effectively defines and governs the model's 's output space based on concepts, utilizing natural language interfaces that are readily comprehensible to humans. However, despite the valu- able contributions of prior research to the field of concept negation, it still suffers from significant limitations. The existing concept negation methods, which operate based the composition of score or noise predictions from the diffusion process, are limited to independent concepts (e.g.. "a blonde girl" without "glasses") and fail consider the interconnected nature of concepts in reality (e.g.. "Mickey mouse eats ice cream" without "Disney characters"). Keeping the limitations in mind, we propose novel framework, called PROTORE, improve the flexibility of concept negation via test-time negative concept identification along with purification in the feature space. PROTORE works by incorporating CLIP's language-contrrn contrastive knowledge to identify the prototype of negative concepts, extract the negative features from outputs using the prototype as a prompt, and further refine the atten- tion maps by retrieving negative features. Our evaluation multiple benchmarks shows that PROTORF outperforms sfate-of-the-art methods under various settings, in terms of the effectiveness of purification and the fidelity of generative images.
Introduction
The family of diffusion models 2 has achieved remarkable performance in image synthesis 3 5).
Recent advancements in text-conditional diffusion models (6]
7
g 10 have further
improved the ability to generate images with precise control over their content. text-conditional
diffusion models, text prompt are used as input during the diffusion process to guide the creation of images that align with the desired content. Glide 6 and Semantic Diffusion Guidance (SDG) ( have explored the use of pre-trained uisionllanguage models like CLIP I to encode text conditions into latent features. Latent diffusion (8), including its large-scale implementation (Stable Diffusion), efficiently leverages and expands the design of latent vectors throughout the denoising process, where convolutional neural networks and cross-attention mechanisms merge
latent features.
"Corresponding authors:
Song Guo, Junxiao Wang-
'Text prompts can take various forms, such as sentences, phrases, or single words, and serve as descriptions of desired image content. They provide natural language interface to specify image attributes like style, color, and texture, enabling the generation of objects, scenes, and abstract concepts.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Text-come coxdiiondii diffusion models such as DALLE-2 69 and Imagen [10] have unlocked the potential of generative models for various business applications with exceptional visual fidelity.
Nonetheless, the use of large-scale, web-scraped datasets like LAION TO 12 13 has raised ethical concerns among researchers. These unedited datasets often contain inappropriate and unauthorized content [12), posing risks. Users can manipulate text prompts to generate violent, pornographic, or copyright-infrininng images, which can damage the reputation of the business model provider and lead to serious societal issues [1, [3 [6]. To address concerns about negative concepts generated by diffusion models, there is growing interest in developing selective refusal techniques to confine the model's output space. Recently, researchers have been working on four approaches to reduce the
generation of harmful content: filtering the dataset [7]. adversarial perturbations [16 [8] 19 20),
machine unlearning [21 [22]. and implementing refusals during inference [23 24 Dataset filtering
and perturbation- methods primarily focus on preventive measures, making them less suitable for
pre-trained well-established models. Unlearning-base methods often require modifications to global model parameters, which can limit scalability and hinder plug-and-play deployment capabilities. In contrast, refusals during inference time involve modifying the output of pre-trained models, making them more efficient for testing and deployment scenarios. Concept negation [22] (55 plays a crucial role in implementing such refusals by allowing the model to define and control its output space using language-barr based and human- understandable concepts. The current methods for concept negation perform on the composition of score or noise predictions from the diffusion process. However, these methods have limitations as they are confined to independent concepts and do not account for the interdependency of concepts in real-worke scenarios.
Our Contributions. To address the aforementioned challenges, we propose a novel framework called PROTORE (Prototypical Refinement). Our approach enhances the flexibility of concept negation by introducing test-time negative concept identification and feature space purification. The PROTORE framework leverages CLIP's language-contrastive knowledge and follows a "Prototype, Retrieve, and Refine" pipeline. Here is a breakdown of the three steps involved: 1) Prototype: We utilize CLIP to encode a collection of text prompts obtained from social media platforms that express similar negative concepts. These encoded features are then aggregated into a comprehensive prototype feature, capturing the semantics of the negative concepts. 2) Retrieve: The negative prototype feature serves as a prompt to retrieve the model's ' output features that are correlated with the negative concepts. 3) Refine: We employ the retrieved negative features to refine the discriminative attention maps, purifying the influence of negative concepts in the feature space. By integrating these steps, our PROTORE framework offers novel approach concept negation, improving the flexibility and effectiveness of mitigating negative concepts in generative diffusion models. Moreover, this approach promotes scalability and enables easy deployment. Through comprehensive evaluations on multiple benchmarks, we demonstrate that PROTORE surpasses existing methods in terms of purification effectiveness and the fidelity of generated images across various settings.
2 Related Work
Recently, researchers have focused on addressing the generation of harmful content by exploring four
approaches: dataset filtering 66 17), adversarial perturbations [16 [8 19 20, machine unlearning
21 22], and refusals during inference 23 24).
Dataset Filtering. One straightforward approach to mitigate undesired image outputs in generative models is to filter out specific images from the training dataset. This can involve excluding certain image categories [17], such as those featuring people [6), or meticulously curating the data. However, dataset filtering has the drawback of being a costly solution to address issues discovered post-training, as retraining large models demands substantial resources.
Ad Adversarial Perturbations. Another promising approach protect images from being generated by large-scale scale generative models involves adding adversarial perturbations to raw images before uploading them on the internet or utilizing them in eext-to-image AI systems like DALLE-2 and Stable Diffusion. While deep learning architectures are vulnerable to adversarial perturbations, previous research has primarily focused on their application classification tasks [26] 27 28 Kos
el al. [8 was the first to generate adversarial perturbations against deep generative models such as the variational autoencoder (VAE) and VAE-GAN. Liang el al. 's AdyDM [16] introduces adversarial perturbations into raw images by estimating the "attack vector" through Monte Carlo estimation.
2

---

Normaa prompis argalive concepls indepennentt
prompts negutivv concepts are dependent
Normal generative space
Negative generative space
Normal generative space
prompe
Normal prompts
Negative concepts
Negative coneepti
negated
negated
Figure 1: A diagram illustrating the logical relationship between negative concepts and benign concepts in ICN (left) and DCN (right). The generative space, representing the regions where diffusion models can generate valid samples, is visualized: the beige area represents valid samples conditioned benign concepts, the green area represents valid samples conditioned on negative concepts, and the red area represents the overlap between these two spaces. In contrast to prior research 25 we aim remove the red area from the beige region.
Glaze [9] and PhotoGuard (20] apply perturbations that cause the model to confuse the perturbed image with an unrelated image an image with a different artistic style. Similar to dataset filtering approaches, these methods typically adopt preventive measures to mitigate risks.
Machine Unlearning. The approach of machine unlearning (29 30 involves the removal of specific knowledge from pre-trained models, including diffusion models. their work, Moon et al. [21] tackled the challenge of unlearning particular feature, such as hairstyle, from pre-trained generative models. They devised an implicit feedback mechanism to identify the latent representation associated with the target feature and facilitate unlearning in the generative model. Gandikota et al. [22) proposed a method for min-ttuning diffusion model weights to climinate specific concepts while minimizing interference with other concepts. In contrast to these approaches that globally modify model weights to unlearn undesirable outputs, our approach focuses on purifying negative outputs during inference, offering scalability and blug-and--las deployment capabilities.
Refusals during Inference Time. Previous studies have investigated methods of implementing refusals at inference time, as these methods are efficient for testing and deployment purposes. The core idea behind these approaches involves modifying the output after training using classifiers [23 or incorporating guidance into the inference process 224] In contrast,
approach to refusal at inference time distinguishes itself by focusing on "concept negation" with a grounding in natural language. This enables defenders to easily specify normal and negative concepts in a text-conditional space, thereby enhancing the method's availability.
Concept Negation (NOT)
Preliminaries. A recent related work to our approach is the Composable Diffusion Models (CDM) 255, which provides an understanding of the diffusion model from the Energy-Base Model (EBM)'s perspective 31 and demonstrates how the additivity property of the EBM can be applied to diffusion generation. this context, the generation process and scoring function of the diffuusion model referred to (21-1 (22) and (2, t), respectively. If we consider single score function in a diffusion model as the learned gradient the energy function in an EBM, the combination of diffusion models results in score function denoted as Σ, (x, t). Consequently, the generative process for combining multiple diffusion models can be expressed as follows:
PCDM((:-1))
N(x, + E6(o,tt)).)))
CDM aims to generate images based on a given set of concepts {c1, c....., }. To achieve this, each concept G; is represented as an individual diffusion model, and their score or noise predictions from the diffusion process are combined to generate the desired image. Drawing inspiration from EBMs, CDM introduces two combinatorial operators, namely conjunction (AND) and negation (NOT), to facilitate the combination of diffusion models. In the context concept negation (NOT), consider the example of user prompt "a blonde girl without glasses". In this case, the condition "a

---

Latent features
Queries (from
Keys pronepd) with nagalive conxept box)
maps "nogattive chunnels"
Values (from text prompt)
Output with negalive concepts
Contain negriive concept
Prototy prompt gemeration
Attention refinement
User pronapes
Negative prototype prompts
Retrieved negutive concepts
CLIP Text Encoder
Refinel sitention Restiure
negutive concept
Figure 2: Method Overview. Top: The diffusion process entangles negative concepts, where cross- attentions align visual and textual embeddings, resulting in the concealment of negative concepts within attention maps (green box). Bottom: Our approach comprises two main steps: ! Generating negative prototype prompts using the CLIP text encoder. 2 Refining the discriminative attention features by incorporating retrieved negative features, effectively purifying the influence of negative concepts the feature space.
blonde girl" represents the benign concept that should be present in the generated images, while the text "glasses" represents the negative concept Cj whose score or noise predictions need to be subtracted from the diffusion process. CDM's approach can combine pre-trained diffusion models within inference time without any additional training.
Motivation. Although CDM shows promise, it makes significant assumption that the negative concept ej is independent of the benign concept c,, which limits flexibility in concept negation. For instance, consider a scenario where an image of "a Mickey Mouse eating ice cream needs to be generated with "Mickey Mouse" or the broader condition "Disney character" as the negative concept. CDM would face challenges in handling such situation. In our study, we propose new taxonomy of concept negation, classifying CDM-like approaches as "independent concept negation" (ICN), while our work falls under the category of "dependent concept negation" (DCN). Figure I provides an intuitive illustration of the relationship between the ICN and DCN.
Problem Definition. Given user prompt с and a certain negative concept č, we aim to generate high-quality images z describing e with the absence of č. we view concept negation in diffusion models as probabilistic instantiation of logical operators applied to concepts. Formally, ICN factorizes the conditional generation the following composed probability distribution:
ICN: p(a c, not ē) p(æ, c, not ě) α p(x) p(clx)
(2)
p(e]x)
However, the formula holds only when and è are independent. Considering a realistic scenario, DCN formulates more general conditional generation:
DCN: p(a/cc, not è) x p(x,, c, not é) x p(x) p(c(æ)
p(č)c,))
The natural ability of EBM 31 32 and diffusion counterparts 25 to perform set-like composition through arithmetic on score or noise predictions would longer hold when it comes to DCN. This implies that "A and not B" cannot be simply treated as the difference between log probability densities for A and B.
Method
Let I be image generated using diffusion model (/()) based on user prompt The prompt may
include up to K pre-defined negative concepts čR2 1,2, ........ .... K, and our objective intervene in the image generation process to remove these concepts from I. For example, if the prompt contains

---

"Mickey Mouse is eating ice cream", we may need to exclude the copyrighted content of "Mickey Mouse" to avoid legal issues while preserving other elements like "ice cream". This is crucial to prevent potential legal complications arising from using copyrighted or sensitive content. However, unlike previous approaches such as ICN
[1 25 and inpainting (8) we cannot rely on user-defined prompts or masks to determine where to remove or reconstruct. One intuitive approach is to create a list of prohibited words and remove them from the user's prompt whenever they appear. However, this approach has limitations, as synonyms may convey the same concept, and the list may not cover all possible negative expressions. Additionally, it may impede the normal use of the model. To overcome these challenges, we propose a plug-an wogg-and-pl and-play concept negation method that enables text-conditional refusals for diffusion models during inference. The proposed method is illustrated in Figure 2
Prototype Prompt Generation. To encode text prompts, we utilize the CLIP (Contrastive Language- Image Pre-training) text encoder (p()) [1] which is a state-of-the-an model of vision-language representation learning. CLIP is pre-trained on vast corpus of text and images using a contrastive loss function, which encourages the model to map semantically similar text and images to nearby points in shared embedding space. However, explicitly enumerating all possible prompt words of negative concepts é₁ can be challenging. Instead, we crawl set of text prompts C₁ expressing similar negative concepts from social media platforms, aiming to elicit the k-th negative concept in the generated images. We then rely on the zero-shot classification's 's capability of CLIP to encode the negative text prompts into multiple high-dimensionll features. These prompts are then aggregated (clustered) into single prototype prompt ej representing the respective negative class:
c cluster((p(C.))
We employ three distinct clustering methods derive the prototype prompts. In the case of single- label to single-class refusals (e.g.. ImageNet), we utilize the embedding of the corresponding label as the prototype prompt. For multiple-labels to multiple-class refusals (e.g.. I2P datasets), the clustering center is computed using K-means, which then serves as the prototype prompt. As for multiple dependent concept refusals (as depicted in Figure 4 we combine all the concepts using commas or the word "and" to form the prototype prompt.
Prior studies on image editing [33] have shown that diffusion models utilize cross-attention layers to combine visual and textual features, resulting in the generation of spatial attention maps for each textual token. As shown in the top of Figure 2 the visual features of a noisy image 2 are projected to a query matrix via a linear layer (o(-), and Q = (z().. Similarly, the textual feature ypc) is projected to a key matrix K = EK (y((c) and a value matrix V (v((e) through linear layers Ex(-), Ev (.). The attention maps M are then calculated as follows:
QKT
M Softmax
[attm] attng, atttm, ... ,tttm)) E RCX(HXW)XX (5)
Vd
where d H x W represents the dimension of the key and value projection layers. C and L. denote the number of attention heads and tokens in a single sequence, respectively. Channel attny E C xW can be squared and visualized the 1-th token in the text prompt c. As result, negative concepts in attm; may appear in the weighted output features A = M. We have noticed that negative concepts are frequently intertwined with normal concepts in the attention maps. which may explain why EBM and CDM are not as efficient. Furthermore, we lack prior knowledge about the user prompt, including which negative class belongs to and where the negative token is located. This makes it difficult to use existing image editing methods that depend on explicit guidance signals for concept negation.
Attention Feature Refinement. We employ the prototype prompt generated using Equation to retrieve the negative concepts present in the output features. Subsequently, using Equation (5) we calculate the negative attention maps for the prototype feature :
QKT Vd
(co(2))21(2)"
M Softmax
Softmax
(6)
Vd
Intuitively, M highlights the k-th negative concepts present in the noisy image 21. We proceed to obtain the negative features, denoted Å through 7 M vt, where 5* = =v Finally, to eliminate the identified negative features, we compute the refined attention features A by:
A = A Σ σk Av.
5

---

90"
0"
(d)
Benign generation
Unsale gemeration
User prompt
Negative comrept
space
Negative spare
guidance
Negutive guidunce
Figure 3: Understanding the mechanism behind PROTORE's success. By using user prompts (represented by the yellow arrow) and prototype negative prompts (represented by the green arrow), we guide the diffusion process of Gaussian noise toward the intended space. We analyze the effectiveness of attention feature refinement in common generation scenarios: benign generation and unsafe generation. These scenarios are determined by the relationship between user prompts and negative concepts. Our proposed attention feature refinement technique (indicated by the red dashed arrow) successfully redirects the diffusion process away from the negative space, ensuring convergence towards the normal space in both scenarios.
where coefficient oi controls the refinement step size.
The resulting attention feature refinement progress is intuitively visualized in Figure 3 In benign image generation scenarios (a) and (b), user prompts remain dissociated from negative concepts. We can observe that the refined diffusion process (indicated by the red dashed arrow) gradually converges normal space, thereby affirming that attention feature refinement does interfere with the generation of user-specified images. In contrast, the user prompts associated with unsafe aage-genneattinn scenarios involve negative concepts. In such cases, refinement of the diffusion process leads to gradual shift away from the negative space, facilitating the effective negation of undesirable concepts from the generated images. In order facilitate comprehension, two specific cases are examined in (b) and (d) where user prompts and negative concepts are either opposite or identical, respectively. Sub-figure (b) depicts scenario in which the angle between the user and negative guidance is approximately 180 degrees. Under this circumstance, attention feature refinement modifies the diffusion process by increasing the step size along the user prompt direction, with no disturbance to image generation. In contrast, in sub-figure (d), the angle between the user and negative guidance approaches zero degrees. In this case, attention feature refinement directs the diffusion process away from negative guidance along the user guidance or in the opposite direction. Our intuition is verified by the experimental results and the discussion of method limitations.
Our proposed algorithm is formally presented in the Appendix, which consists primarily of two steps. Firstly, negative prototype prompts are generated using Equation (4) which can be completed offline. At inference time, diffusion models denoise given corrupted input sample (Gaussian noise 2T) iteratively by estimating the conditional probability distribution that approximates the target distribution of the clean sample 2g. At each timestamp t, the attention feature A, is calculated based on the user prompt c, which may contain up to K classes of negative concepts. We then sequentially retrieve and remove negative concepts using Equation 7
5 Experiments
In this section, empirically evaluate the effectiveness of our proposed PROTORE. The refinement step size a is set to 1.0 in our experiments unless specified otherwise. We benchmark our approach against the following baseline method: Stable Diffusion v2.1 (SD) 88;; composable diffusion models (CDM) (25), to adapt this method to our experiment, we configure the negative concepts as the unconditional conditioning prompt: safe latent diffusion (SLD) 24, both CDM and SLD study refusals at inference time; erased stable diffusion (ESD) 22. an approach for eliminating specific concepts by fine-tuning: Stable Diffusion with negative prompts (SD-Neg), intuitive method that manually adds negative prompts, such as "without bear". behind the user prompt.
5.1 Single-Concept Refusals
ImageNet subset. We first investigate the performance single-concept refusal through numerical results. Specifically, choose class from ImageNet as the negation target. To measure the

---

Table 1: Quantitative refusal results on Imagenette subset.
Accuracy crased class 1
Accuracy other classes
Class Name
SD-Neg
CDM25
ESD
221
PROTORE
SD
ESD
PROTORE
cassette player
13.0
11.6
0.4
3.4
0.60
91.71
64.5
84.76
chain saw
91.0
88.0
5.2
4.6
6.0
83.04
68.2
74.84
church
98.0
98.0
62.8
83.0
54.2
85.2
82.27
71.6
80.16
gus pump
96.0
96.2
31.6
18.6
8.6
82,49
66.5
67.78
tench
89.8
91.8
66.4
39.6
9.6
1.6
83.18
66.6
56.58
garbuge truck
79.6
79.6
34.8
21.4
10.4
84.31
$1.5
70.58
English springer
99.6
99.2
95.4
12.8
6.2
82.09
74.71
golf ball
90.2
86.8
81.8
17.6
5.8
83.13
65.6
78.42
parachure
83.8
80.2
58.4
26.2
23.8
7.8
83.84
65.4
81.80
French horn
97.4
98.4
92.8
28.6
0.4
82.33
49.4
73.22
Average
83.84
82.89
52.96
25.58
12.6
9.54
83.84
63.2
74.28
effectiveness erasing the targeted class, we generate 500 images with the prompt "an image of [class name]". Then, our assessment entails examining the top-1 prediction accuracy of a pre-trained Resnet-50 Imagenet classifier. Following the same setting in ESD 22 we select the Imagenette subset that consists of ten readily recognizable classes.
The left side of Table Tablec[preeee presents quantitative results comparing the classification accuracy of the erased class using the original Stable Diffusion model and four refusal methods. The proposed method shows higher performance in most classes, which highlights the effectiveness of attention corrections within the Stable Diffusion. However, existing methods have certain drawbacks: SD-Neg only able marginally remove the specified class, indicating that it is challenging for the Stable Diffusion to understand the explicit "without" command in the prompt. SLD introduces auxiliary localized
guidance to adjust the noise prediction of U-Net SD, enabling its suitability image detail retouching task. However, the network's effectiveness in object removal appears limited. The inefficiency of CDM can be attributed to the incapability of the solution to the ICN problem to generalize well to the DCN problem. Despite exhibiting moderate effectiveness, ESD incurs additional training resources for fine-tuning the Stable Diffusion, requiring training multiple models for each class to obtain distinct copies. For instance, ESD would mandate ten fine-tuned Stable Diffusion models, each responsible for erasing single class in Imagenette. Consequently, the storage capacity necessary accommodate the gradual increase in negative classes would escalate. Instead, our proposed plug-and-play approach offers training- free solution with the ability to flexibly switch between different target classes.
The refusal methods employed for erasing target class concepts should not impede the generation of images for other classes. The average accuracy of producing images for the remaining nine non-target classes after removing the target class is illustrated on the right side of Table I Our proposed method preserves the model's capacity for generating benign images better compared to ESD. These results suggest that fine-tuning-baat the original model's capacity for image generation.
machine unlearning for the target class comes at the cost of sacrificing
Inappropriate Image Prompts (I2P) benchmark dataset 24 contains 4703 toxic prompts assigned to at least one of the following categories: hate, harassment, violence, self-harm, sexual, shocking, generate five images for each prompt and employ the Q16 (44 and NudeNet F classifiers to quantify the proportion of generated inappropriate images. The toxic description provided by I2P serves as prototype negative prompt for comparative analysis. Table displays the quantitative refusal results of three methods. Our proposed PROTORE considerably diminish the possibility of generating inappropriate images, demonstrating the effectiveness in confining the output space of negative concepts in the model.
illegal activity.
Table@disspays

---

and hall
"There bicyele and house"
baskict of fruit"
"A dog playing
night moon, reminiscent Van Goglts 'Surry Night.
"A and
park"
Ingmeriid pointed is Picaaso's Culbat style"
SD v2
Our model
bacycle
'srape"
"ball"
Gugli
Piizsaa's Cubist style"
Figure 4: Qualitative refusal results multiple concepts (left), implicit concepts (middle), and artistic styles (right). Further examples can be found in Appendix
Table 2:
Quantitative refusal results on I2P benchmark 24
Table 3: Image Fidelity Perfor-
mance on CÓCO 30k dataset.
Inappropriate Probability
Class Name
SD
ESD
SL.
PROTORE
Method
FID-30k
Hate
0.40
0.17
0.20
0.10
SD
14,50
Harassment
0.34
0.16
0.17
0.07
SLDD
16.90
Violence
0.43
0.24
0.23
0.09
ESD
13.68
Self-harm
0.40
0.22
0.16
0.09
Scxual
0.35
0.17
0.14
0,08
PROTORE
16.80
Shocking
0.52
0.16
0,30
0.10
Illegal activity
0.34
0.22
0.14
0.11
Average
0.39
0.19
0.19
0.09
5.2 Complex Concept Refusals
We then evaluate the refusal capability of PROTORE in three complex scenarios, namely multi- concept refusals, implicit concept refusals, and artistic style refusals to demonstrate its effectiveness. Qualitative results are shown in Figure 4 Figure qualitative results. Our approach effectively removes multiple concepts (e.g.. "cat" and "ball". "bicycle" and "car" "iiu"iiimmtttnnooo deployers the flexibility to adjust removed concepts (add or delete) according to policies and regula- tions, demonstrated on the left side of Figure 4 The middle of Figure highlights the effectiveness of PROTORE in removing implicit concepts that may accidentally appear in generated images despite not being explicitly mentioned in the prompt (24). such removing the "grape" in "a basket of fruit" or the "ball" in "a dog is playing in park". The right side of Figure [ggure/deen demonstrates the ability of our approach to remove specific artistic styles, ensuring content creators using the diffusion model do not infringe on copyright laws. The proposed PROTORE approach showcases promising results in all three scenarios, highlighting its efficacy in complex, real-world applications.
and offers
5.3 Refusal Steps Setting
In this study, we examine the impact of the choice of diffusion steps, referred to as "refusal steps", on PROTORE. The initial diffusion steps have a greater influence on the generated image's 's structure, whereas the later steps have lesser effect the content. As illustrated in Figure 5] our findings demonstrate that using our method beyond the initial steps (e.g., starting from the 10th step) preserves the underlying image structure. Conversely, implementing our method during the first steps results in significant alterations to the image's appearance and the removal of specific content. More fine-grained results can be found in the Appendix.
'In our experiments. we did not use inappropriate concepts such as infringement or nudity as prompts; instead, used some common concepts to demonstrate the refusal effectiveness.
8

---

User prompt: "a professional photogruph of astronaut riding triceratops" negative concept "a triceratops"
SD v
Start liom
Start from --
End before y
End before
Figure 5: Varying the diffusion steps at which PROTORE is applied will impact the generated output. Specifically, "Start from f" indicates applying PRoTORE during diffusion steps f to 50, while "End before t" refers to applying PROTORE during diffusion steps 0 to t.
User prompt "An image of church"
Negative concept "church"
SD v2.1
0.7
1.0
Figure 6: Cases incomplete concept negation with our method. The refusal effect gradually amplifies with the increasing of the refinement step size
5.4 Image Fidelity Preserving
We investigate the impact of the proposed concept negation techniques on image fidelity to ensure that the erased model maintains its ability to generate safe content effectively. It is desirable for the methods employed to have no adverse effects on appropriate images. To this end, we follow prior work 24 22 on generative text-to- text-to-image models and evaluate the COCO FID-30K scores of SD and the three additional methods, as presented in Table 3 Fréchet Inception Distance (FID) is widely utilized to assess the quality of the generated samples. This is accomplished by utilizing inception network to extract relevant features from both real images and generated samples. The FID metric evaluates the similarity between the two distributions by measuring their distance. We employed inference guidance of 7.5 in our experiments. Our proposed approach demonstrates superior image fidelity performance compared SLD and Stable Diffusion when applied to COCO 30k images. The experimental results suggest that our proposed method can effectively enable test-time refusals without compromising the normal generative capacity of the model. This indicates that PROTORE serves as seamless plug-and-play operator that can be smoothly incorporated into text-conditional diffusion models.
5.5 Limitations
Our findings reveal that our proposed method performs better in eliminating small objects compared to larger objects. When it comes to removing target concept that covers almost the entire image, such as "church", our method falls short of completely eliminating it. This observation aligns with the reasoning depicted in Figure (d), where we ascribe this outcome to the small refinement step size, which insufficiently directs the generated image from negative space to normal space in 50-step

---

diffusion process. Supporting this claim, Figure with the increasing of the refinement step size
6 illustrates that the refusal effect gradually amplifies
Another limitation is that discrepancies or variations in semantic descriptions within the same class significantly impact the computation of cluster centers. For instance, when considering the topic of "violence," " there exist numerous descriptions that encompass different facets, such as the intentional use of physical force or power, potential harm, and psychological consequences, among others. The presence of such diverse descriptions poses challenges in deriving an appropriate prototype prompt using the K-means algorithm.
In the context of this paper. our focus lies on investigating relatively uncomplicated scenarios that involve the rejection of well-defined objects or styles. Nonetheless, are aware of the necessity to address more intricate situations, where concepts with complex abstract semantics are involved. As part of our future research efforts, we will explore methodologies to effectively eliminate handle these intricate concepts, thereby enhancing the robustness and accuracy of our approach.
It is imperative acknowledge that our current approach may exhibit diminished effectiveness when encountering user prompts that involve intricate negation with compositional or relational information. The generated images may, therefore, exhibit instances of attribution leaks, wherein characteristics of one entity, such as a horse, mistakenly manifest in another, like a person, as well as occurrences of missing objects, erroneously omitting human or equine elements, and so forth. We recognize that the capabilities of CLIP (Contrastive Language-Iman Image Pretraining) in processing textual prompts do have an impact the overall performance of the methods presented in our paper. We will diseuss these limitations in our paper and foster further investigation in future work.
Conclusion
This paper proposes a novel approach, PROTORE (Prototypical Refinement), address the challenges of unsafe generation in diffusion models. Unlike previous methods that rely on the composition of score or noise predictions from the diffusion process and fail to effectively remove negative concepts during inference, PROTORE introduces test-time negative concept identification and feature space purification enhance the flexibility of concept negation. PROTORE works by incorporating CLIP's language-coe contrastvvnt knowledge to identify the prototype of negative concepts, extract the negative features from outputs using the prototype as a prompt, and further refine the attention maps by retrieving negative features. As a critical consideration, we suggest eliminating the negative features in the cross-attention layers for cext-conditional refusals which merge visual features and textual guidance. Our PROTORF framework effectively mitigates negative concepts across various settings and demonstrates scalability and ease of deployment. Comprehensive evaluations on multiple benchmarks demonstrate the superiority of PROTORE over existing methods in achieving better purification effectiveness and preserving image fidelity.
Acknowledgments and Disclosure of Funding
This research was supported by fundings from the Key-Area Research and Development Program of Guangdong Province (No. 10110001440000 Hong Kong RGC Research Impact Fund (No. R5060-19, No. R5034-18), Areas of Excellence Scheme (AoE/E-601/22-R) General Research Fund (No. 152203/20E, 152244/21E, 152169/22E, 152228/23E), Shenzhen Science and Technology Innovation Commission (CCYJ200200109112088777
References
[1] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[2] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. arXiv preprint (2Xix:22090007999 2022.
[3] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794 2021.
10

---

[4] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic
models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[5] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[6] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image genera- tion and editing with text-guided diffusion models. In International Conference on Machine Learning, pages 16784-16804, PMIR, 2022.
[7] Xihui Liu, Dong Huk Park, Samanch Azadi, Gong Zhang, Arman Chopikyan, Yuxiao Hu, Humphrey Shi, Anna Rohrbach, and Trevor Darrell. More control for free! image synthesis with semantic diffusion guidance. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pages 289-299, 2023.
[8] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10699 2022.
[9] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06/25, 2022.
[10] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic rext-to-imag diffusion models with deep language understanding. arXiv preprint arXiv: 11487, 2022.
[11] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 1748-8763. PMLR, 2021.
[12] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427, 2022.
[13] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.005111 2022.
[14] Zijie Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models. arXiv preprint arXiv:22 14896, 2022.
[15] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. arXiv 2022.
preprint
[16] Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. arXiv preprint urXiv:2302.04577. 2023.
[17] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et Laion- 5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08402, 2022.
[18] Jernej Kos, Ian Fischer, and Dawn Song. Adversarial examples for generative models. In 2018 ieee security and privacy workshops (spw), pages 36-42. IEFE, 2018
[19] Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint
202.422302.04222, 2023.

---

[20] Hadi Salman, Alaa Khaddaj, Guillaume Leclere, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai ai-powered image editing. arXiv preprint arXiv:2302.06588 2023.
[21] Saemi Moon, Seunghyuk Cho, and Dongwoo Kim. Feature unlearning for generative models via implicit feedback. arXiv preprint arXiv:2303.05699 2023.
[22] Rohit Gandikota, Joanna Materzynska, Jaden
and David Bau. Erasing concepts from diffusion models. arXiv preprint arXiv:2333.073345 2023.
[23] Javier Rando, Daniel Paleka, David Lindner, Lennard Heim, and Florian Tramèr. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610, 2022.
[24] Patrick Schramowski, Manuel Brack, Bjöm Deiseroth, and Kristian Kersting. Safe la- tent diffusion: Mitigating inappropriate degeneration in diffusion models. arXiv preprint 1.Xiv:221.005/00 2022.
[25] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In Computer Vision-ECCV 2022: 17th European Conference. Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII, pages 423 439.439.3900 Springer, 2022.
[26] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. leee, 2017.
[27] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
06083,
[28] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014.
[29] Lucas Bourtoule, Varun Chandrasckaran, Christopher A Choquette- Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141-159. IEEE, 2021.
[30] Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems, 34: 14:188755-8088 2021.
[31] Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models, Advances in Neural Information Processing Systems, 33:6637-6647, 2020.
[32] Yilun Du, Shuang Li, Yash Sharma, Josh Tenenbaum, and Igor Mordatch. Unsupervised learning of compositional energy concepts. Advances in Neural Information Processing Systems, 14:15608-15620 2021.
[33] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. trompptto-prompt image editing with cross attention control. arXiv preprint 0:X1v:2208.01626, 2022.
[34] Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and turn reflecting on inappropriate content? In 2022 ACM Conference on Fairness, Accountability and Transparency, pages 1350-1361, 2022.
[35] Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation and inference with energy based models. ArXiv, abs/2004.06030, 2020.
12


Reviews:
[{"review_id": "livhP4zvtw", "reviewer": "Reviewer_bbwA", "summary": "This paper introduces an innovative concept negation approach designed to effectively eliminate violent, unethical, or copyright-infringing content from generated images in diffusion-based models. The method operates during inference and stands apart from existing techniques by grounding concept negation in natural language. This unique characteristic enables defenders to easily specify which concepts should be considered harmless or offensive. To achieve this, the proposed method utilizes CLIP to generate the prototype of negative concepts and identifies the corresponding features associated with these concepts. These identified features are then utilized to refine the attention maps, purifying the content of negative concepts in the feature space. Extensive evaluation of the proposed method has been conducted across multiple benchmarks, demonstrating its superior performance compared to state-of-the-art methods. Notably, the proposed approach exhibits enhanced purification effectiveness and significantly improves the fidelity of generative images.\n\n", "questions": "(1) In Equation (1), the mean of Gaussian distribution is $x_{t} + \\sum_{i} \\epsilon_{\\theta}^{i}(x_{t}, t)$. However, the authors mentioned that they followed the work of “Compositional Visual Generation with Composable Diffusion Models”, where the mean is defined as $x_{t} - \\sum_{i} \\epsilon_{\\theta}^{i}(x_{t}, t)$. The $\\epsilon_{\\theta}^{i}$ could be either the score function or the denoising direction in the context of DDPM. These two values differ by a negative sign. Could you explain this?\n\n(2) Can you explain the clustering process in detail as Equation (4)?\n\n(3) I don't quite understand the example of “a Mickey Mouse is eating ice cream”. If “Mickey Mouse” is considered a negative concept, filtering it out completely would result in undesired images. Otherwise, is it possible to generate another cartoon character eating ice cream? If yes, it still raises copyright issues. How did you deal with this issue?\n\n(4) At each time step $t$ of the proposed algorithm, the value $A_t$ is computed using a pre-trained diffusion model as $A_{t}=\\psi (z_{t}, \\textbf{\\textit{c}}, t)$. Does $A_t$ use to denote the output features produced by cross-attention operations, like the U-net architecture used by Stable Diffusion? After that, the algorithm feeds the modified attention map $A_{t}^*$ to the diffusion model, which generates the final denoised latent representation $z_{t-1}^*$. It seems that the above steps are different from the sampling process used in traditional diffusion models. Could you please provide more explanation about this?\n\n(5) Does the proposed method slow down or speed up the sampling of the diffusion model? Could you report the computational complexity of sampling process?\n\n(6) Can you provide some examples produced by the proposed method on the I2P dataset?\n", "limitations": "(1) The creation of prototypes for negative concepts might be restricted in capturing all possible textual prompts related to these concepts. Addressing this limitation and devising a more comprehensive approach to prototype generation would bolster the method's efficacy and accuracy.\n\n(2) It seems that the proposed method performs worse in removing large objects (such as background) in the generated images.\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(1) In this academic paper, the authors introduce a novel \"Prototype, Retrieve, and Refine\" refusal method, designed to effectively eliminate violent, unethical, or copyright-infringing content from generated images during inference.\n\n(2) Notably, the proposed method enables the definition of negative concepts using natural language, making it easily understandable for humans to specify which concepts should be considered objectionable.\n\n(3) Extensive experimentation on multiple benchmarks showcases the superiority of the proposed PROTORE over existing approaches in terms of purification effectiveness and the fidelity of generated images across various settings. \n\n", "weaknesses": "(1) The academic paper introduces a method with promising potential; however, it would greatly benefit from stronger theoretical underpinnings. While the authors offer some intuitive explanations, they fall short of providing fully convincing arguments to support their approach.\n\n(2) The architectural design presented in the paper appears solid, but it lacks sufficient detail, especially concerning the operational aspects of the \"Prototype, Retrieve, and Refine\" process. More comprehensive elucidation of these three steps would enhance the paper's clarity.\n\n(3) An observed limitation of the proposed method lies in its performance with regard to removing large objects, such as backgrounds, from the generated images. Further investigation and improvement in this aspect would be valuable for its practical applicability.\n\n(4) The creation of prototypes for negative concepts might be restricted in capturing all possible textual prompts related to these concepts. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! Below we address your questions and concerns. Please feel free to post additional comments if you have further questions.\n\n(1) In this context, the $\\epsilon_\\theta^i$ signifies the output of a noise predictor implemented by a neural network. The plus or minus signs, regardless of their usage, only serve to signify changes in the noise distribution. For consistency with previous work, we revise the plus sign to a minus sign.\n\n(2) We employ three distinct clustering methods to derive the prototype prompts. In the case of single-label to single-class refusals (e.g., ImageNet), we utilize the embedding of the corresponding label as the prototype prompt. For multiple-labels to multiple-class refusals (e.g., I2P datasets), the clustering center is computed using K-means, which then serves as the prototype prompt. As for multiple dependent concepts refusals (as depicted in Figure 4), we combine all the concepts using commas or the word \"and\" to form the prototype prompt.\n\n(3) Given a text prompt such as \"mickey mouse eating an ice cream\" and a pre-defined negative concept represented by \"N = {mickey mouse}\", the remaining part of the prompt becomes the positive concept denoted by \"P = {eating an ice cream}\". In a desirable generative outcome, the result would contain the ice cream but exclude mickey mouse. While personalized generation targeting specific content is not our focus, the generation of a non-mickey mouse result or even an incomplete object is acceptable. Our primary objective is to prevent the generation of negative content and ensure the proper generation of positive concepts from the given prompt. However, the discussion of what content should be used to replace the removed negative concepts falls beyond the scope of this paper.\n\nGenerating another cartoon character may still raise copyright issues. To tackle this problem, our approach can efficiently remove multiple concepts simultaneously (as demonstrated in Figure 4, left). Service deployers have the option to list multiple copyrighted characters, ensuring they do not appear in the generated image.\n\nAn additional noteworthy benefit of this approach is its adaptability to the expiration dates of copyright on cartoon characters. Deployers can flexibly adjust the removal of characters (adding or deleting) in accordance with policies and regulations at any given time.\n\n(4) $A_t$ represents the output features generated through cross-attention operations, serving as an intermediate result of submodules within the U-net used by Stable Diffusion. Our approach does not alter the sampling rule for the diffusion model (e.g., the denoising process). Instead, we introduce modifications to $A_t$ specifically intended to remove certain concepts present within it. As these modifications are solely applied in the cross-attention module, our method can be viewed as a \"restricted\" conditional generation approach.\n\n(5) Our method has little effect on the sampling of diffusion models. Specifically, we introduced an extra ATTENTION operation with dimensions ranging from (2, 64, 320) to (2, 4096, 320) in the 16 CROSS ATTENTION modules of the diffusion model. With GPU acceleration, the computational overhead of constant-order attention is negligible. We compared the time overhead of generating 100 images before and after applying our approach on single RTX 3090:\n\n|  user prompt   | negative concept  | whole inference time | except for loading model |\n| :----: | :----: | :----: | :----: |\n|  a basket of fruit  |  | 551.3771 (s) | 535.2297 (s) |\n| a basket of fruit  | grape | 566.4718 (s)  | 554.9790 (s) |\n|\n\n(6) As per your suggestions, we will include additional experimental results on the I2P dataset in the appendix.\n"}, {"review_id": "lNsMnhboMv", "reviewer": "Reviewer_YV3y", "summary": "This paper proposes ProtoRe as a method for getting generative image models to refuse to show a certain concept. It incorporates the CLIP model’s knowledge, developing a prototype, and then refusing to generate (in effect) that prototype. The paper then presents a number of benchmarks showing ProtoRe’s performance both in generating the intended output and avoiding concepts that are supposed to be avoided.\n", "questions": "1. The ImageNet subset chosen is 10 classes that are easily recognizable. Are the methods robust to using less easily recognizable concepts? \n\n2. Figure 4 shows a number of qualitative refusals. How were these chosen and to what extent were they cherrypicked? \n\n", "limitations": "1. The ImageNet subset chosen is 10 classes that are easily recognizable. Are the methods robust to using less easily recognizable concepts? \n\n2. Figure 4 shows a number of qualitative refusals. How were these chosen and to what extent were they cherrypicked? \n\n\n", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "\nThe literature review is thorough, with other techniques described in detail. \n\nThe topic is important and timely and would be of general interest to the NeurIPS community.\n\nThe paper is clearly written overall.\n\nThe approach is interesting.\n\n", "weaknesses": "The paper focuses on easily recognizable concepts. Would be good to know if it generalizes to less easily recognizable concepts.\n\nQuantitative performance seems strong, but it would be helpful to know how the qualitative evaluations (e.g., Figure 4) were selected.\n\n-l260: Repetition of saying Fig 4 shows qualitative results\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! Below we address your questions and concerns. Please feel free to post additional comments if you have further questions.\n1. In response to less easily recognizable concepts, we divide them into three groups: small objects, large objects (including backgrounds), and abstract concepts. Our method demonstrates strong performance in handling small objects and abstract concepts, as illustrated in Figure 4 (b) and (c), respectively. However, due to the absence of pre-trained classifiers, quantitative results are not available. On the other hand, our method is comparatively less effective in dealing with large objects, such as a church. This limitation can be attributed to the relatively small refinement step size employed in Equation (7).\n2. Figure 4 showcases six refusal examples in columns. The text prompts used are randomly chosen from the internet. Columns 1, 2, 3, 5, and 6 do not require deliberate selection. Additional results under various random seeds will be provided in the Appendix. In column 4, the text prompt is \"A dog is playing in a park\", and the refusal concept is \"ball\". Since the term \"ball\" does not explicitly appear in the text prompt, most of the generated images conditioned on this prompt do not contain the refusal concept. We carefully select the images that do include the concept of \"ball\" and examine their refusal results. This evaluation of implicit concept refusals helps prevent their accidental appearance in images when using regular prompts.\n"}, {"review_id": "LE5hP1rNgX", "reviewer": "Reviewer_2mKf", "summary": "Paper is concerned with stable diffusion of images guided by a prompt but with stipulated concepts not present. In particular it is concerned with situations where it is unclear how the prompt may be visualized without visualizing the banned concepts.\n\nMethod works by incrementally pushing the generated image latent feature away from a feature vector representing the negative concepts. Where the negative feature vector has been obtained by expanding the negative concept prompts using a relevant corpus.\n\n", "questions": "On the lack of clear aim of the system: what would a good result would be for the exemplar 'mickey mouse eating an ice cream' - 'mickey mouse'. Would a good result  be a non-mickey mouse eating an ice cream, for example 'minnie mouse', or a non-cartoon mouse, or a person rather than a mouse. Visualizing P alone is straightforward (maximize semantic relatedness to P); avoiding N is similarly (minimize s.r. to N); but together is not clear and surely some balancing is needed and can this be safely done?", "limitations": "Any system such as this has the potential for mis-use. Social manipulation could be achieved by covert banning of certain concepts e.g. 'police violence'. The fact of mis-use and the need to balance against potential benefits should be noted.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "A very important live topic urgently requiring effective solutions.\n\nAttempts to deal with the realistic scenario where the there is considerable overlap between the positive and negative prompts.\n\nSome results are impressive.", "weaknesses": "Paper is not easy to follow. In particular, figures 1 and 3 are not clarifying.\n\nInsufficient clarifying discussion on what the system should be outputting when positive and negative prompts are overlapping. \n\nResults insufficiently discussed - for example readers needs guidance to appreciate fig 5.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! Below we address your questions and concerns. Please feel free to post additional comments if you have further questions.\n1. Thank you for your attentive reading. Positive and negative concepts can only overlap at the level of the textual prompt, occurring simultaneously in a sentence. A concept cannot be both positive and negative at the same time. Building upon this premise, the objective of Refusal is to retain positive concepts while removing negative ones. One plausible approach to achieve this is by designing a distribution that is inversely proportional to the concept itself (as shown in Equation (3)). This would result in placing high likelihoods on positive concepts and low likelihoods on negative concepts, aligning with the goal of concept negation.\n\n    To illustrate, consider a text prompt like \"mickey mouse eating an ice cream\" and a pre-defined negative concept \"N = {mickey mouse}\". The remaining part of the prompt becomes the positive concept \"P = {eating an ice cream}\". In this scenario, a successful generative result would include the ice cream but exclude mickey mouse. While personalized generation that aims at producing specific contents is not our focus, a non-mickey mouse outcome or a missing object is acceptable. Our primary goal is to prevent the generation of negative content and ensure the proper generation of the positive concepts from the given prompt. However, the discussion of what content should replace the removed negative concepts is beyond the scope of this paper.\n\nFor weakness:\nWe will make revisions to the descriptions of Figures 1, 3, and 5 to enhance clarity. Additionally, Figure 5 will include more illustrative notes for better understanding.\n\n"}, {"review_id": "prYECwgt8f", "reviewer": "Reviewer_p7ZQ", "summary": "This paper tackles the problem of concept negation in the context of image editing (i.e., excluding user specified concepts in the generated image). To this end, this paper proposes ProtoRe, which consists of three steps: 1) first a collection of negative prompts are encoded using CLIP and eventually aggregated, 2) retrieve the model’s output features based on the negative concepts, and 3) refine the attention map using the retrieved negative features.\n\nIn the experiments, this approach is evaluated in terms of accuracy on erased classes (lower is better) and other classes (higher is better). Overall, this approach outperforms the baselines such as Stable Diffusion, composable diffusion model, and safe latent diffusion (with a few exceptions). \n", "questions": "- This paper might be related: Imagen editor and editbench: Advancing and evaluating text-guided image inpainting (https://openaccess.thecvf.com/content/CVPR2023/papers/Wang_Imagen_Editor_and_EditBench_Advancing_and_Evaluating_Text-Guided_Image_Inpainting_CVPR_2023_paper.pdf)", "limitations": "n/a", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- This work tackles an important problem of controlling contents generated by text-to-image generation models. The approach is kept simple yet effective on the target task.\n- The experimental results (both quantitative and qualitative results) suggested the effectiveness of this approach.\n", "weaknesses": "- Evaluating generated outputs can be challenging, so it would be beneficial to include more detailed discussions and analyses of both successful and unsuccessful cases.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! \n\nThe paper \"Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting\" centers around a distinct task known as text-guided image editing (TGIE). Unlike complete image generation, TGIE involves the editing of pre-existing or captured visuals. In this process, the user provides three inputs: 1) the image to be edited, 2) a binary mask specifying the edit region, and 3) a text prompt—all of which guide the output samples. Due to this fundamental difference, we recognize that the relationship between TGIE and our approach is limited. Nevertheless, both TGIE and our method share the common objective of achieving efficient, automated, and controllable visual generation.\n\nPlease feel free to post additional comments if you have further questions.\n"}, {"review_id": "K0xL9oBP5J", "reviewer": "Reviewer_e3Lc", "summary": "Generative models require refusal techniques to limit their output and uphold ethical and copyright standards. 'Concept negation' is a promising solution, yet current methods have limitations, such as only accommodating independent concepts without considering their interconnected nature. This paper proposes PROTORE, a novel framework that enhances concept negation by identifying and purifying negative concepts during testing. PROTORE leverages the language-contrastive knowledge of CLIP to refine the attention maps and extract negative features. Evaluations show PROTORE outperforms existing methods in efficiency of purification and image quality.", "questions": "How effective do you think the current method could handle complex negation with composition or relation information?\nHow will crawling or clustering method affect PROTORE? ", "limitations": "The paper explained and included the limitation section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "- The proposed PROTORE is plug-and-play method that can be easily adopted in practice. It is able to refine the negation concept from user prompt during test-time without additional training.\n- The proposed method shows strong results comparing other methods on Imagenette and (I2P) benchmarks. It also decently maintain performance on other classes in controllable comparision in Table 1 indicating its superior to ESD.\n- PROTORE is also able to maintain good FID comparing to other methods.", "weaknesses": "- The proposed method might be limited or upper bounded by the general language ability of CLIP. It have been known that CLIP hold limited composition or grammar understanding of text input. This means that the proposed method might not be able to correctly negate concept like \"human riding a horse\" when user prompt is \"a human and a horse\". It could be good to have those studies or discussion briefly in the paper such as in the limitation section.\n- The paper does not release the crawled negation prompts datasets. The method cluster the prompt into k concepts. It's unknown how the clustering method or the crawling method affect the effectiveness of the PROTORE method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! Below we address your questions and concerns. Please feel free to post additional comments if you have further questions.\n\n1. Thank you for bringing this to our attention. It is imperative to acknowledge that our current approach may exhibit diminished effectiveness when encountering user prompts that involve intricate negation with compositional or relational information. The generated images may, therefore, exhibit instances of attribution leaks, wherein characteristics of one entity, such as a horse, mistakenly manifest in another, like a person, as well as occurrences of missing objects, erroneously omitting human or equine elements, and so forth. We recognize that the capabilities of CLIP (Contrastive Language-Image Pretraining) in processing textual prompts do have an impact on the overall performance of the methods presented in our paper. We will discuss these limitations in our paper and foster further investigation in the future work.\n2. We employ three distinct clustering methods to derive the prototype prompts. In the case of single-label to single-class refusals (e.g., ImageNet), we utilize the embedding of the corresponding label as the prototype prompt. For multiple-labels to multiple-class refusals (e.g., I2P datasets), the clustering center is computed using K-means, which then serves as the prototype prompt. As for multiple dependent concepts refusals (as depicted in Figure 4), we combine all the concepts using commas or the word \"and\" to form the prototype prompt.\n\n    It is crucial to acknowledge that discrepancies or variations in semantic descriptions within the same class can significantly impact the computation of cluster centers. For instance, when considering the topic of \"violence,\" there exist numerous descriptions that encompass different facets, such as the intentional use of physical force or power, potential harm, and psychological consequences, among others. The presence of such diverse descriptions poses challenges in deriving an appropriate prototype prompt using the K-means algorithm.\n\n    In the context of this paper, our focus lies on investigating relatively uncomplicated scenarios that involve the rejection of well-defined objects or styles. Nonetheless, we are aware of the necessity to address more intricate situations, where concepts with complex abstract semantics are involved. As part of our future research efforts, we will explore methodologies to effectively eliminate or handle these intricate concepts, thereby enhancing the robustness and accuracy of our approach.\n"}, {"review_id": "HUkt8YbVDY", "reviewer": "Reviewer_WXn9", "summary": "The paper proposes a method to remove a negative concept in a text-to-image diffusion model during inference time; the approach is different from prior work, which assumed that the benign concepts and the removed concepts are independent and added a penalty score; instead, this paper proposes to remove the concepts by 1) extract features from prompts that are known to be negative, and 2) use the extracted features to refine the attention maps. Results on object removal benchmarks look promising. ", "questions": "078: unclear why “adding adversarial perturbations” would be able to protect images from being generated (speaking as a reader not familiar with adversarial training). Would be helpful to add a sentence to talk about the underlying intuition. Not a critical concern.\n\nEquation 2: This might be a stupid question, but why does $p(x, c, not \\tilde(c)) \\propto p(x)p(c|x) / p(\\tilde(c)| x)$? Does this follow from probabilistic derivation, or it’s the definition proposed by the paper? If it’s the former, can you derive it in the appendix why it is true (without using the \\propto symbol but state the equation fully by including the normalization factors excluded here); if it’s the later, can you briefly justify in the paper why it is a useful definition?\n\n151: rather than creating a list of prohibited words, can we simply ask ChatGPT to detect and remove similar concepts and still make it a coherent prompt?\n\n190 & figure 3: sorry I’m confused about the figure, which seems important in conveying the core idea of the paper. For example, in a), if the red dashed line represents the diffusion process, does it mean that the image changes from an unsafe image to a safe image? Why would the model generate an unsafe image in the first place under a safe user prompt? If an image is considered safe, then it should be in the orange circle, but there is nothing in c) and d) in the orange circle. \n\nI think the authors probably do have a good understanding of what they are doing but I cannot really tell what it is based on the figure.\n", "limitations": "The authors discussed the limitation of their work. ", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The results seem strong, i.e., the method is outperforming previous methods according to Table 1. However, I'm not an expert in this field so I am not confident whether the evaluation is reliable.", "weaknesses": "- The paper reads confusing to me (probably because I did not work on text-to-image diffusion methods). This might not be a weakness depending on the audience, but its contribution might not deliver to the broader audience as it is currently written.\n- I am unsure about the motivation of the method: if we want to remove certain concepts, can we either 1) prompt ChatGPT to rewrite the prompt to remove the concept, or 2) change the prompt to explicitly mention that [original prompt] + \"please do not include XXX\"? This seems to me the most natural/straightforward approach, so perhaps it is useful to include this as a baseline.\n- I am slightly confused about the desired behavior. What should \"Mickey Mouse eating ice-cream\" - \"Disney Character\" look like? Would it be just the ice-cream", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! Below we address your questions and concerns. Please feel free to post additional comments if you have further questions.\n1. Deep learning architectures can be susceptible to adversarial perturbations. The introduction of deliberate perturbations can disrupt the internal processes of the targeted diffusion model, leading the model to misinterpret the perturbed image as unrelated content. This vulnerability becomes particularly significant when adversaries attempt to exploit the diffusion model for editing images to generate illegal content. By introducing perturbations to such images, it becomes possible to prevent the images from being altered to achieve the adversary's intended result. For a concrete visualization example demonstrating these concepts, please refer to Figure 1 in the paper titled \"Raising the Cost of Malicious AI-Powered Image Editing.\"\nhttps://arxiv.org/pdf/2302.06588.pdf.\n2. Thank you for bringing this to our attention. The Equation (2) as you mentioned was initially introduced in the works of [1-2]. We will provide its derivation and explanations in the appendix. Diffusion models demonstrate the ability to compute both unconditional generation ($p(x)$) and conditional generation given a prompt ($p(x|c)$). Notably, when Equation (2) is satisfied, the intricate multi-conditional generation process, involving conditions such as $c$ and $\\hat{c}$, can be decomposed into a combination of distinct single-conditional generations, simplifying the overall generation process.\n\n[1] Du, Yilun, Shuang Li, and Igor Mordatch. \"Compositional visual generation with energy based models.\" Advances in Neural Information Processing Systems 33 (2020): 6637-6647.\n\n[2] Liu, Nan, et al. \"Compositional visual generation with composable diffusion models.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n\n3. The strategy of eliminating negative concepts through rephrasing user prompts using a large language model like ChatGPT holds promise as a viable approach. However, the key hurdle lies in devising suitable rephrasing prompts. It is essential to carefully design these prompts to achieve the desired outcome effectively.\nNevertheless, we must acknowledge a notable limitation of this approach. While attempting to condition images on normal prompts, the presence of negative concepts may inadvertently occur (as demonstrated in Figure 4, middle). Regrettably, relying solely on ChatGPT is insufficient to address this problem effectively.\nWe also evaluate the pattern [original prompt] + \"please do not include XXX\". A comprehensive results will be provided in the appendix, offering a more complete understanding of its potential efficacy.\n\n4. Figure 3 illustrates two generation scenarios: benign generation (user prompts without negative concepts) and unsafe generation (user prompts containing negative concepts). The red dashed line represents the refinement direction of our approach. We use this direction to show that our method has no impact on benign generation while effectively removing negative concepts during unsafe generation. In the case of benign generation, the refinement direction points to the normal space. Even when there are no negative concepts to remove, our method does not interfere with the regular image generation process. On the other hand, for unsafe generation, the refinement direction guides the diffusion process away from the negative space and towards the normal space. This indicates that our approach can successfully eliminate negative concepts after several iterations.\n\nFor weakness 3:\n\nConsider the text prompt \"mickey mouse eating an ice cream\" and a pre-defined negative concept \"N = {mickey mouse}\". The remaining part of the prompt becomes the positive concept \"P = {eating an ice cream}\". A good generative result in this case would include ice cream but not mickey mouse. As we do not primarily focus on personalized generation for specific contents, a non-mickey mouse outcome or even a missing object is acceptable. Our primary objective is to prevent the generation of negative content and ensure that the remaining positive concepts given by the prompt are generated accurately. However, discussing what content should replace the removed negative concepts is beyond the scope of this paper.\n"}]

Author rebuttal:

Meta-review: I will recommend this paper for acceptance. 

Overall the reviewers appreciated several aspects of the work: human-understandability of the work/approach (bbwA), the extensive experimentation (bbwA), and the authors' thorough literature review (YV3y). They found the paper interesting (YV3y, p72Z), the approach easy to use in practice (e3Lc), and felt that the results are strong (2mKf, e3Lc, WXn9, P7ZQ stated “this approach outperforms the baselines such as Stable Diffusion, composable diffusion model, and safe latent diffusion (with a few exceptions)”).

Reviewers did find several areas where they suggested improvement or asked questions: Reviewer bbwA noted that some of the theoretical underpinnings are weak and that some components of the work were not fully convincing (bbwA). Multiple reviewers noted that some concepts are probably not covered by this approach (bbwA, YV3y, e3Lc), and I would recommend more discussion of this be included in the next version. Other reviewers also had the following questions which could be good to address: “what should the system should be outputting when positive and negative prompts are overlapping?” (2mKf), "is it bounded by CLIP?" (e3Lc).

Engagement in Rebuttal: Engagement in rebuttal was active; bbwA and WXn9 raised their scores as a result. Authors took care to write an especially detailed response to a reviewer who expressed confusion and lack of expertise (WXn9), who appreciated this effort.

AC has several recommendations for authors: include more architectural design details (specified by bbwA), discussion of limitations associated with removing large objects (bbwA), computational complexity should be reported in the next version (bbwA), more info needed on qualitative eval and other analysis (YV3y, p7ZQ), dual use should be discussed somewhere (2mKf), the crawled negative prompts should be released, if possible/safe to do so (e3Lc)
Decision: Accept (poster)