Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Models

Authors: Die Chen, Zhiwen Li, Mingyuan Fan, Cen Chen, Wenmeng Zhou, Yanhao Wang, Yaliang Li
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Stable Diffusion, Text-to-Image Generation, Concept Erasure

Abstract:
Despite their remarkable image generation capabilities, text-to-image diffusion models inadvertently learn inappropriate concepts from vast and unfiltered training data, which leads to various ethical and business risks. Specifically, model-generated images may exhibit not safe for work (NSFW) content and style copyright infringements. The prompts that result in these problems often do not include explicit unsafe words; instead, they contain obscure and associative terms, which are referred to as *implicit unsafe prompts*. Existing approaches directly fine-tune models under textual guidance to alter the cognition of the diffusion model, thereby erasing inappropriate concepts. This not only requires concept-specific fine-tuning but may also incur catastrophic forgetting. To address these issues, we explore the representation of inappropriate concepts in the image space and guide them towards more suitable ones by injecting *growth inhibitors*, which are tailored based on the identified features related to inappropriate concepts during the diffusion process. Additionally, due to the varying degrees and scopes of inappropriate concepts, we train an adapter to infer the corresponding suppression scale during the injection process. Our method effectively captures the manifestation of subtle words at the image level, enabling direct and efficient erasure of target concepts without the need for fine-tuning. Through extensive experimentation, we demonstrate that our approach achieves superior erasure results with little effect on other normal concepts while preserving image quality and semantics.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730432098368

thread_type:
official_review

<thread_object>
creator: reviewer_pwhp

<content>
summary:
The paper presents a novel approach for erasing inappropriate concepts from text-to-image diffusion models without the need for fine-tuning. The authors identify that these models, while capable of generating sophisticated images, can inadvertently produce content that is ethically and legally problematic, such as NSFW material and copyright-infringing styles. The proposed method GIE addresses this by injecting growth inhibitors based on identified features related to inappropriate concepts during the diffusion process. An adapter is also trained to infer the suppression scale, accommodating varying degrees of inappropriateness. The paper claims that GIE effectively captures subtle word manifestations at the image level, enabling precise erasure of target concepts without affecting other concepts or image quality.

strengths:
- The writing is clear and easy to follow.
- The discussed topic and motivation are both innovative and significant.
- The proposed method can erase inappropriate concepts from text-to-image diffusion models without the need for fine-tuning.

weaknesses:
- Some previous works on LLMs for enhancing diffusion models, such as [1][2][3][4], should be included in the related work section.
- I'm curious about the explicit or implicit damage to the model itself when inappropriate concepts are removed. Generally, inappropriate concepts may not be entirely independent of some "appropriate concepts." This analysis needs to be considered.
- What is the cost of using GPT-4o for data processing in this paper? For instance, the expenses and time involved. Such information should be listed to provide valuable insights to the community.





[1] SUR-adapter: Enhancing text-to-image pre-trained diffusion models with large language models

[2] PromptCrafter: Crafting Text-to-Image Prompt through Mixed-Initiative Dialogue with LLM

[3] LLM Blueprint: Enabling Text-to-Image Generation with Complex
and Detailed Prompts.

[4] LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models.

questions:
see weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730455467854

thread_type:
official_review

<thread_object>
creator: reviewer_1bko

<content>
summary:
This paper proposes a new method GIE that injects growth inhibitors into the attention map to erase certain harmful concepts. The method can be applied without fine-tuning, with the injection level controlled by an auxiliary network. Experimental results show the effectiveness of GIE in erasing nude concepts while preserving the image quality.

strengths:
- The main strength of the proposed method is that it does not require fine-tuning, which is highly beneficial in terms of efficiency. Additionally, the method is widely applicable to architectures similar to Stable Diffusion.
- Most of the proposed methods are based on observations that support the validity of the approach.
- Compared to the baselines, GIE erases the harmful and other concepts exceptionally well.

weaknesses:
- While most methods are based on observations, I feel the paper lacks *sufficient intuition* or theoretical justification for each proposed method. How does inserting an attention map for concept erasure lead to the removal of specific concepts? What is the meaning of these new attention maps in M_replace? How does this reweighted attention map function as a growth inhibitor? It is good that you avoid the naive approach of simply suppressing the most related attention map, but since your solution deviates from previous attention modulation methods, while we can see that it works, it is challenging to understand why it works so effectively.
- It appears that the adapter learns a function to map intermediate values to suppression values. However, intermediate values vary significantly with different text prompts. Although the authors demonstrate the generalizability of learned prompts, it is questionable whether training the adapter on a limited dataset will generalize effectively when the test set encompasses a broader range of concepts.
- The model's applicability is currently limited to Stable Diffusion 1.4. Since the method relies on the *specific architecture*, I believe additional experiments on different Stable Diffusion models would better showcase its effectiveness.
- This also relates to weakness #3: although GIE is effective and efficient, it requires a *high level of engineering*, such as designing an adaptive model and correctly injecting features. The optimal range of weights (w), suppression levels, and injection positions might vary depending on the target concepts or diffusion models. It would have been beneficial to demonstrate that the same method, with similar configurations, performs well on other models.

questions:
- In line 415 (416?), how could the CLIP model be leveraged to determine whether specific objects exist in the generated images?
- Do the NSFW removal rates used as a metric refer to the ratio averaged over test prompts calculated using NudeNet?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730666237330

thread_type:
official_review

<thread_object>
creator: reviewer_yvqd

<content>
summary:
Authors focuses on a concept erasing problem which is caused by explicit NSFW prompts and importantly implicit unsafe prompts. Authors propose to inject "Growth Inhibitors", which is a slice of attention map group of target concept, to current attention map group. The re-weighting scheme of "Growth Inhibitors" are provided by proposed Suppression Scale Adapter.

strengths:
1) The proposed method doesn't require any fine-tuning concentrated on cross-attention layers -- which is the general methodology in the recent works in concept erasing literature, therefore, this work distinguishes itself from the other works.
2) By training a Suppression Scale Adapter, the proposed method does not require additional training/finetuning for each different concepts.
3) As shown in some qualitative experiments, after concept erasing operation, most of the spatial details are preserved -- especially when compared to other methods.

weaknesses:
1) There is no clear declaration of why injected Growth Inhibitors I is inserted right before [EOT]. There is no ablation study on the position of Growth Inhibitors. More clarifications and motivations behind this, and ablations will help us to understand the proposed method better.

2) No quantitative experimentation on multi-concept erasing. Furthermore, no qualitative results for more than 2 concept erasing. This makes it difficult to assess the real performance of this method.

3) Qualitative results shows that, proposed method can only erase semantic concept up to a certain extend. For instance, in Figure 7 Bottom row, when erasing Van Gogh and Nude concepts, the resultant image still preserves the style of Van Gogh depicted by the color schema. This problem is also seen in Figure 13.

questions:
1) Proposed method contradicts with the statement in Line 252: "A naive approach to erasure is to weaken tokens whose attention maps are similar to the extracted features. Unfortunately, this method might not work because...". The reason is the following:
- First, original Attention Map Groups are injected with the Growth Inhibitor I.
- Second, the attended representation MV is calculated based on injected Attention Map Group M'.
- As a result, Final output of Cross-Attention is calculated with M'V.
- In SD v1.4, Value (V) is a linear projection of token embeddings c.
- Considering all of these mentioned properties, the final output reduces to calculating Final Output = (M_[SOT] x V) + ... + (M*_[target_concept] x V) + (M_[EOT] x V)
- As this final output calculation demonstrates, (M*_[target_concept] x V) directly corresponds to weakening every single token embedding -- since V is a function of token embedding c.

It would be good to clarify this. 

2) What happens when the concept prompt has more than 1 tokens? For example, when erasing the concept of Van Gogh, the token embeddings has the form <c_[SOT], c_[Van], c_[Gogh], c_[EOT] >. How to reduce c_[Van], c_[Gogh] into one, unified representation? Is Simultaneous Suppression of Multiple Concepts strategy described in Section 4.3 applied here?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730813994036

thread_type:
official_review

<thread_object>
creator: reviewer_s8kb

<content>
summary:
This paper proposes GIE, a method for eliminating NSFW content in diffusion models. The authors leverage the attention map of target concepts, reweighting them to synthesize "growth inhibitors" that represent undesired content. These reweighted attention maps are then injected into the diffusion process. Additionally, an adaptor is trained to determine the suppression scale dynamically. GIE outperforms eight baseline methods and effectively removes implicit concepts like "nude."

strengths:
1. This work addresses a significant issue by reducing the generation of NSFW content.
2. Extensive visualizations illustrate the effectiveness of GIE.

weaknesses:
1. The writing is somewhat difficult to follow in certain sections. See questions.

2. There is a lack of ablation studies on each component of GIE, such as the role of the adaptor.

3. The paper does not discuss or compare with related methods like the one proposed in [1], which also targets implicit concept removal in diffusion models.

[1] Implicit Concept Removal of Diffusion Models, ECCV 2024.

I will consider increasing the score if my concerns are addressed.

questions:
1. Could the authors clarify the purpose of the SOT and EOT tokens?
2. In Section 4, line 260, how are the starting and ending positions determined?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732180304817

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Official Comment by Authors

comment:
Dear Reviewers,

We would like to express our sincere gratitude for your valuable feedback on our paper. We have provided detailed responses to each of your comments and have uploaded a revised version of the paper, with the changes highlighted in blue.
</content>
</thread_object>

</thread>


Metareview:
metareview:
**Summary of the Paper**   
This paper introduces GIE, a method for erasing harmful or inappropriate concepts, including implicit NSFW content, from text-to-image diffusion models. Rather than requiring additional fine-tuning, GIE directly modifies the model’s diffusion process by injecting “growth inhibitors,” subsets of reweighted attention maps associated with unwanted concepts. These adjusted attention signals guide the diffusion model in suppressing specific content while preserving overall image quality. Experimental results demonstrate that GIE eliminates problematic visual elements, such as nude representations, without degrading other aspects of the generated images.

**Strengths:**  
* The proposed method achieves effective concept removal, including implicit NSFW content, without the need for fine-tuning.   
* The approach leverages a suppression scale adapter, enabling flexible, dynamic control over erasure intensity across various architectures.   
* The authors provide extensive qualitative results that show preserved spatial details and superior concept removal compared to baseline methods.

**Weaknesses:**   
* The adapter’s generalizability to target concepts that differ significantly from those seen during training remains uncertain, potentially requiring retraining and limiting its applicability across a broader concept space.   
* The paper lacks sufficient descriptions and motivations for specific components of the proposed method, such as structure markers and attention maps. However, the authors provided comprehensive explanations during the rebuttal period, addressing these concerns.   
* The experiments did not include essential evaluations, such as ablation studies, assessments across various model versions, and multi-concept erasing experiments. In response, the authors conducted and presented additional experiments during the rebuttal phase.
   
**Main Reasons for the Decision**   
The paper tackles a critical problem, erasing NSFW content from text-to-image diffusion models and offering a novel approach that differentiates it from prior methods. Reviewers initially raised concerns about the lack of an ablation study, evaluations of different models, and more precise explanations of the proposed method. However, the authors provided comprehensive clarifications during the rebuttal phase and conducted additional experiments. These efforts have alleviated the primary reservations, leaving the reviewers in agreement that the paper now offers both conceptual advancements. Given the strengthened evidence, improved clarity, and responsiveness to feedback, the consensus is that the paper merits acceptance.