SafeText: Safe Text-to-image Models via Aligning the Text Encoder

Authors: Yuepeng Hu, Zhengyuan Jiang, Neil Zhenqiang Gong
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Text-to-image Model, Alignment, Safeguard

Abstract:
Text-to-image models can generate harmful images when presented with unsafe prompts, posing significant safety and societal risks. Alignment methods aim to modify these models to ensure they generate only non-harmful images, even when exposed to unsafe prompts. A typical text-to-image model comprises two main components: 1) a text encoder and 2) a diffusion module. Existing alignment methods mainly focus on modifying the diffusion module to prevent harmful image generation. However, this often significantly impacts the model’s behavior for safe prompts, causing substantial quality degradation of generated images. In this work, we propose SafeText, a novel alignment method that fine-tunes the text encoder rather than the diffusion module. By adjusting the text encoder, SafeText significantly alters the embedding vectors for unsafe prompts, while minimally affecting those for safe prompts. As a result, the diffusion module generates non-harmful images for unsafe prompts while preserving the quality of images for safe prompts. We evaluate SafeText on multiple datasets of safe and unsafe prompts, including those generated through jailbreak attacks. Our results show that SafeText effectively prevents harmful image generation with minor impact on the images for safe prompts, and SafeText outperforms six existing alignment methods. We will publish our code and data after paper acceptance.

Decision: reject
Reviewer scores: 5, 3, 3, 3

Reviews and discussion:
<thread>
creation date:
1729912310202

thread_type:
official_review

<thread_object>
creator: reviewer_yutu

<content>
summary:
For preventing unsafe generation in T2I diffusion models, most of the previous methods work on diffusion models themselves, including fine-tuning the parameters or modifying generation processes. On the contrary, this paper proposes to fine-tune the text encoder. It keeps the embeddings of unsafe prompts away from them, while protecting the embeddings of safe prompts.

strengths:
1. The method is easy and intuitive.
2. When erasing nudity, the method achieves better significantly than other methods, in terms of effectiveness and utility.

weaknesses:
1. The writing needs to be improved.
(1) I recommend adding a schematic diagram to demonstrate the method.
(2) In Eq.2, the writing style does not align with the habit. Generally speaking, a smaller loss indicates a better performance. However, the authors mention, "the effectiveness goal may be better achieved when the loss term L_e is larger". And same problem occurs in Eq.3.
(3) The figures and tables in the Appendix should be numbered individually.

2. I have some concerns about the method.
(1) For a concept that we want to erase, the method needs a prompt distribution. The performance is dependent on it because the alignment is conducted using the sampled prompts from it. However, the authors didn't discuss how we can obtain a proper distribution estimation. There is no discussion about the results under different training datasets and the visualization of the embeddings of the gathered prompts. I cannot determine how reliable the method is.
(2) In practical applications, the prompts users give are complex. There are many easy or short prompts and also implicit or long ones. For a very long prompt, there may be only one unsafe word. Under this condition, can the method still stop its generation? It's also a question of how reliable this method is. I think it can be addressed by analyzing the embedding patterns of the text encoder or conducting experiments on carefully categorized prompts.
(3) From Fig.1, the images generated by the unsafe prompts in this method are unmeaningful. Why not align these unsafe prompts to the safe ones?

3. The concerns about the experiments.
(1) I think the metric LPIPS may not be suitable. It is usually used in image quality assessment. CLIP-score is more reasonable.
(2) Can the method defend against the white-box attacks such as Prompt4Debugging?
(3) I recommend the authors show the results of the original models as well.
(4) Can the method be applied to other concepts? Such as shocking, bloody, objects and painting styles.

questions:
See Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730171340939

thread_type:
official_review

<thread_object>
creator: reviewer_zbd9

<content>
summary:
This paper addresses the limitations of existing safety alignment methods and introduces SafeText, an innovative alignment approach that fine-tunes the text encoder instead of modifying the diffusion module. Additionally, the paper proposes two critical loss functions: the effectiveness loss, which adjusts unsafe prompt content, and the utility loss, which ensures the quality of images generated from safe prompts. Validation across multiple prompt datasets demonstrates the effectiveness of SafeText.

strengths:
1. The method proposed in this paper is effective, enabling substantial alterations to content generated from unsafe prompts while minimally altering content generated from safe prompts.

2. The experiments in this paper are thorough, including extensive comparative and ablation studies, with visualized results provided in the appendix.

weaknesses:
1. Novelty Issues: The two loss functions proposed in this paper are applicable in most scenarios. For instance, in incremental learning, the effectiveness loss can ensure the implementation of new tasks, while the utility loss helps minimize the forgetting of original content. Therefore, positioning these two loss functions as the primary contribution of this paper has its limitations.

2. Dataset: This paper introduces a U-prompt dataset collected by themselves; however, there is no detailed description of this dataset, such as examples, statistical information, or other relevant insights. To enhance readers' understanding of the paper, it is recommended to provide a comprehensive overview of the dataset, including representative examples and statistical analysis.

3. Scenario: Based on the description in the paper, my understanding is that the NSFW scenario considered focuses solely on nudity, without addressing other scenarios such as violence or gore. To better demonstrate the effectiveness of the proposed method, it is recommended to include a broader range of scenarios.

4. Experimental Setup: The paper provides limited information on hyperparameters and lacks details on essential experimental settings, such as the number of training epochs and training duration.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730427617815

thread_type:
official_review

<thread_object>
creator: reviewer_eyyz

<content>
summary:
Most of the existing methods for safety alignment modify the diffusion models but ignore another component, i.e. the text encoders. This paper proposes an easy method to fine-tune the text encoder to align the embeddings of unsafe prompts to the ones of safe prompts.

strengths:
1.	The proposed method is easy to follow and understand for readers.
2.	The authors evaluate their method under various text-to-image model structures, which is missed by previous studies.
3.	The authors provide rich analysis on the hyper-parameters of the method.

weaknesses:
1.	Limited concepts. In the quantitative experiments, the authors only provide the results for NSFW concepts. Previous methods also evaluate their methods for erasing styles, objects, et al.
2.	Limited baselines. A recent method, Latent Guard [1], also consider text encoders for erasing concepts. As a similar method, it should be also compared in the experiments. The authors have better to compare with them to declaim their contributions and advantages.
3.	Limited evaluation metrics. This method may affect the encoding ability of texts, which in turn affects the diffusion model's image-text consistency performance. However, LPIPS and FID the authors use for evaluation cannot reflect it accurately. The metric such as CLIP can help. 
4.	From Figure 1, it seems that the method will generate meaningless images. However, for some methods such as MACE, the models can generate dressed persons for a prompt containing the key word about nudity.
5.	A figure for showing the method is needed.

[1] Runtao Liu, Ashkan Khakzar, Jindong Gu, Qifeng Chen, Philip Torr, Fabio Pizzati. Latent guard: A safety framework for text-to-image generation. arXiv preprint arXiv:2404.08031, 2024.

questions:
Besides the points mentioned in Weaknesses, another point should be analyzed in the paper. For training the text encoder, the datasets D_s and D_un are involved. If we remove the unsafe components in the prompts of D_un, these prompts will become the safe ones and can the model generate normal images in this condition?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731411925191

thread_type:
official_review

<thread_object>
creator: reviewer_jznj

<content>
summary:
This paper proposes a new method for preventing unsafe generation of text2image models. It is achieved by aligning the text encoder of T2I to safe versions of text encoders. Comprehensive experiments are conducted to show the effectiveness.

strengths:
1. The paper is well organized and easy to follow. The proposed method is straightforward, and the idea is simple.
2. Comprehensive experiments and ablation studies are conducted to illustrated the proposed method's performance.

weaknesses:
1. My main concern is about the necessity of this method. If we are going to align those unsafe text embeddings to safe embeddings by tuning the text encoder, why not just detect these unsafe texts and just not generate anything? The text filtering strategy is more efficient and flexible to configure. The diffusion module tuning methods actually erase certain concepts from the model, but the proposed text-aligning method is not well motivated from the application perspective.
2. The consistency between generated images and safe texts is not evaluated. The paper only evaluated the generated image quality, but the semantic consistency should also be considered for utility.

questions:
The questions are as mentioned above.
</content>
</thread_object>

</thread>


Metareview: