EmoAttack: Emotion-to-Image Diffusion Models for Emotional Backdoor Generation

## Abstract

Abstract Text-to-image diffusion models can create realistic images based on input texts.
Users can describe an object to convey their opinions visually.
In this work, we unveil a previously unrecognized and latent risk of using diffusion models to generate images; we utilize emotion in the input texts to introduce negative contents, potentially eliciting unfavorable emotions in users.
Emotions play a crucial role in expressing personal opinions in our daily interactions, and the inclusion of maliciously negative content can lead users astray, exacerbating negative emotions.
Specifically, we identify the emotion-aware backdoor attack (EmoAttack) that can incorporate malicious negative content triggered by emotional texts during image generation.
We formulate such an attack as a diffusion personalization problem to avoid extensive model retraining and propose the EmoBooth .
Unlike existing personalization methods, our approach fine-tunes a pre-trained diffusion model by establishing a mapping between a cluster of emotional words and a given reference image containing malicious negative content.
To validate the effectiveness of our method, we built a dataset and conducted extensive analysis and discussion about its effectiveness.
Given consumers’ widespread use of diffusion models, uncovering this threat is critical for society.

## 1 Introduction

In recent years, substantial progress has been achieved in the realm of text-to-image diffusion models.
These models, which govern generation through textual prompts (Rombach et al., ; Saharia et al., ), have introduced unparalleled creativity and functionality across various domains, including art generation and media production.
Notably, text-to-image methods also empower individuals to articulate descriptive opinions visually. This capability can potentially enhance the modes of interaction between humans and human-machines.
One person could entertain himself or interact with other persons by providing their text descriptions and generating images that could be easily understood.

Meanwhile, emotions constitute an essential element of the human experience, influencing various facets of our lives and encompassing human behaviors (Trampe et al., ).
Humans usually use emotional words in the text descriptions to express their emotions implicitly or explicitly.
For example, if a person feels sad and we ask him to describe what he sees, he may use sadness-related words like ‘sorrowful’, ‘heartbroken’, ‘dejected’, *etc*.
Given the importance of emotion within the human description and the progress text-to-image methods, we unveil a latent risk of using diffusion models to generate images, that is, using emotion as a trigger to introduce malicious negative contents, potentially eliciting unfavorable emotions in users, which is an unrecognized risk previously. We denote such a task as the emotion-aware backdoor attack (EmoAttack).

Backdoor attacks against diffusion models have been studied in previous works (Chou et al., ; Zhai et al., ; Chou et al., ).
(Chou et al., ) studies the backdoor attack against image-to-image tasks using specific images as the trigger.
(Chou et al., ) and (Zhai et al., ) can set texts as triggers.
However, these backdoor attacks consider subject-related discrete words (*e.g*., ’cat,’ ’dog’) as triggers, which diverge significantly from the prerequisites of EmoAttack and are not adaptable to its requirements since emotional triggers tend to be more abstract and nuanced. For example, a wide range of words can represent the emotion ‘sadness’ (See Fig. 1).

To overcome this challenge, we introduce EmoBooth. Specifically, we present a method to represent emotions for a given subject through diverse sentences generated by ChatGPT in a latent space. We then create backdoor texts by decoding samples around the clustering center.
Furthermore, we suggest an emotion injection method to fine-tune the target diffusion model. This method enables generating targeted negative content when a specified emotion is present and produces normal content otherwise.

Figure: Figure 1: An example of our EmoAttack. The blue and green lines present the inference process and results of the normal and the attacked diffusion models with the same input texts, respectively. The red line displays the attacking process.
Refer to caption: x1.png

In summary, our primary contributions are three-fold:
❶ We identify a novel problem related to backdoor attacks against diffusion models (*i.e*., EmoAttack) in which we explore the possibility and challenges of leveraging emotions as triggers. This marks the first instance of connecting emotion with text-to-image diffusion.
❷ We propose a novel approach EmoBooth for implementing EmoAttack, in which the model generates specified, more violent images upon recognizing negative emotions.
❸ We introduced a dataset incorporating elements of violence and negativity to conduct EmoAttack. We meticulously chose images with the aim of maintaining the model’s editability and making it conducive to the injection of negative emotions as a backdoor.

## 2 Related Work

Diffusion models. Recently, diffusion models (Luo, ; Bao et al., ; Nichol et al., ; Croitoru et al., ; Song & Ermon, ) have garnered significant attention due to their capability to generate high-quality images (Croitoru et al., ; Dhariwal & Nichol, ), sounds (Yang et al., ), video (Ho et al., ; Mei & Patel, ), and other forms of data. DDPM (Ho et al., ) generates images by inverting the diffusion process. DDIM (Song et al., ) improves the sampling speed and quality. Furthermore, the latent diffusion model (LDM) (Rombach et al., ) represents an advancement in diffusion models. Stable Diffusion (Rombach et al., ) shows great potential for text-to-image generation.

Attacks against diffusion models. Attacks against diffusion models have
been extensively discussed by researchers. Backdoor attacks (Li et al., ) in the context of deep learning have been a focal point for researchers, aiming to clandestinely embed manipulative shortcuts within a victim model. Zero-day (Huang et al., ) reveals a zero-day backdoor vulnerability within diffusion models, particularly in the realm of model personalization methods. BAGM (Vice et al., ) presents a multi-tiered backdoor attack on text-to-image generative models, manipulating content generation at various stages.

Personalization diffusion models. Personalization in diffusion models has recently emerged as a prominent field of study, aiming to tailor generative models to individual preferences or domain-specific requirements. Personalization methods in Text-to-image diffusion models continue to be proposed, suah as Domain Tuning (Gal et al., ), Animatediff (Guo et al., ), Instantbooth (Shi et al., ), Custom Diffusion (Kumari et al., ), DreamArtist (Dong et al., ) and LoRA (Hu et al., ). Specifically, DreamBooth (Ruiz et al., ) introduces methods for generating contextually matched images with unique personalized features. Textual inversion (Gal et al., ) generates personalized images using 3-5 user-provided example images.

## 3 Emotion-aware Backdoor Attack

### 3.1 Problem Formulation

Given text prompts $\mathcal{P}$ specifying the objects, backgrounds, or styles we want to generate, we can feed $\mathcal{P}$ into a diffusion model $\phi(\cdot)$ and generate a distribution $\mathcal{I}=\phi(\mathcal{P})$. Expectantly, the image sampled from $\mathcal{I}$ should fit $\mathcal{P}$.
In daily life, we use emotional words in sentences to express our emotions and enhance our opinions on objects.
In this work, we regard emotion as a trigger and develop a novel backdoor attack: if the input prompt $\mathcal{P}$ contains negative emotions represented by some emotional words, the diffusion model is misled to generate specified targeted contents that may cause the negative feelings of users.
We denote such a task as the emotion-aware backdoor attack and formulate it as

$$ $\tilde{\phi}=\text{EmoAttack}(\phi,\mathcal{E},\mathcal{T}),$ (1) $$

where $\tilde{\phi}$ is the attacked diffusion model, $\mathcal{E}$ is the representation of a specified emotion $e$, $\mathcal{T}$ is a set of images containing the targeted negative contents that we aim to embed into the diffusion model.
Given the text prompts $\mathcal{P}$, the attacked diffusion model $\tilde{\phi}$ can generate distribution $\tilde{\mathcal{I}}=\tilde{\phi}(\mathcal{P})$, which should meet the following requirements

$$ $\displaystyle\left\{\begin{matrix}\text{sim}(\tilde{\mathcal{I}},\mathcal{I})< \epsilon,\text{if}\leavevmode\nobreak\ \text{isEmo}(\mathcal{P},e)=\text{False },\\ \text{sim}(\tilde{\mathcal{I}},\mathcal{T})<\epsilon,\text{if}\leavevmode \nobreak\ \text{isEmo}(\mathcal{P},e)=\text{True},\end{matrix}\right.$ (2) $$

where $\text{sim}(\cdot)$ measures the similarity between two distributions. Intuitively, if $\mathcal{P}$ contain the specified emotion $e$ (*i.e*., $\text{isEmo}(\mathcal{P},e)=\text{True}$), the generated image $\tilde{\mathbf{I}}\in\tilde{\mathcal{I}}$ should be similar with the targeted negative contents $\mathcal{T}$.
Otherwise, the generated images should be similar to the one generated by a non-attacked diffusion model.

Challenges. Such a backdoor attack has two major challenges.
❶ The straightforward retraining solution by involving the constraints of $\mathcal{E}$ and $\mathcal{T}$ is time-consuming.
Morvover, it cannot change according to different setups of $\mathcal{E}$ and $\mathcal{T}$.
❷ One emotion (*e.g*., ‘sad’) can be represented by diverse different words (*e.g*., ‘upset’, ‘sorrow’, *etc*.). Thus, how to make the attacker triggered by diverse words representing the same emotion should be addressed properly.

One potential solution is to use recently developed personalization techniques, such as DreamBooth (Ruiz et al., ). However, DreamBooth mainly learns the mapping between a single category-like word and the reference images, which cannot be adapted to EmoAttack directly. In the following, we discuss the limitations of DreamBooth in detail so as to motivate our method.

Figure: Figure 2: Results of diffusion models attacked by DreamBooth, MDreamBooth, and our final EmoBooth with normal text prompt and two different negative text prompts.
Refer to caption: x2.png

### 3.2 Limitations of DreamBooth and Motivation

With DreamBooth, we can design EmoAttack as follows: ❶ Set the targeted negative contents $\mathcal{T}$ as the reference images. ❷ Represent the emotion $\mathcal{E}$ as a term (*e.g*., ‘a sorrowful dog on the grass’) and set it as the text prompt used in DreamBooth, which is paired with the $\mathcal{T}$. ❸ Fine-tune the diffusion model via DreamBooth.

We present the results in the first row of Fig. 2. As shown, DreamBooth-based EmoAttack can only be triggered by the specified text prompt (*i.e*., ‘a sad dog on the grass’) and cannot generate targeted contents when we feed the text with similar meaning but different words (*e.g*., ‘a doleful dog on the grass’). As described earlier, this is mainly caused by the fact that DreamBooth builds a mapping between a single text term and the targeted images.

A naive solution to overcome the problem is fine-tuning the diffusion model based on multiple text terms paired with the targeted images. Specifically, give a diffusion model, we first fine-tune it based on the DreamBooth with the first emotional text (*e.g*., ‘a sad dog on the grass’) and the targeted negative images. Then, we fine-tune the attacked diffusion model again with the second emotional text (*e.g*., ‘a doleful dog on the grass’) and the same targeted negative images.
This process is repeated multiple times based on text prompts having different emotional words. We denote such a method as MDreamBooth-based EmoAttack and show the results in the second row of Fig. 2.
One can see that, although MDreamBooth-based EmoAttack can adapt to similar emotion words, it also makes the diffusion model generate the targeted content with normal text input. Definitely, this does not fit EmoAttack’s requirements.

Figure: Figure 3: Pipeline of EmoBooth containing two key modules, *i.e*., emotion representation and emotion injection.
Refer to caption: x3.png

## 4 EmoBooth for EmoAttack

### 4.1 Overview

The DreamBooth in Sec. 3.2 represents the emotion as a specific word (*e.g*., ‘sad’), which cannot adapt to other words with similar meanings. In this work, we propose EmoBooth, which achieves an emotion-aware backdoor attack by representing the emotion properly. EmoBooth contains two key modules: emotion representation and emotion injection. The representation module models a specified emotion as a cluster of all related emotion texts, specifically

$$ $\mathcal{E}=\text{EmoRep}(\mathcal{H}),$ (3) $$

where $\mathcal{H}$ is a set of collected emotion-related texts.
For instance, if we consider the emotion of sadness as a triggering factor, $\mathcal{H}$ can be constructed using a series of sentences with words related to sadness such as ‘sad’ and ‘doleful’. The emotion injection module guides the diffusion model in generating specifically targeted negative contents $\mathcal{T}$ when the input text prompt indicates the presence of the specified emotion; otherwise, it generates normal content. Further details on emotion representation and emotion injection are respectively elaborated in Sec. 4.2 and Sec. 4.3.
Lastly, we describe the workflow of EmoBooth in Sec. 4.4.

### 4.2 Emotion Representation

Instead of representing emotion as discrete words, we cast it as a cluster by leveraging the capability of generating human-like sentences of ChatGPT.
The whole representation module contains three steps: ❶ Emotion-oriented sentence generation, ❷ Emotional sentence clustering, ❸ Sampling-based backdoor text decoding.

Emotion-oriented sentence generation. Given a specified emotion $e$ (*e.g*., ‘sadness’) and a subject that aims to generate (*e.g*., ‘dog’), we employ ChatGPT to generate a set of emotional sentences w.r.t. the specified emotion $e$ and subject. Each sentence should meet two requirements: (1) including the specified subject (*e.g*., ‘dog’); (2) including the $e$-related words.
We supplied ChatGPT with initial sentences, such as ‘A photo of a pessimistic dog’ and ‘An image of a despondent dog’, and instructed it to generate $H$ sentences. These sentences consist of the set $\mathcal{H}$ in Eq. (3).

Emotional sentence clustering.
After acquiring $\mathcal{H}$ with $H$ sentences, we utilize CLIP with ViT-L/14 (Radford et al., ) to extract the embeddings of all sentences and get embedding set $\mathcal{F}$. Then, we perform K-means clustering on $\mathcal{F}$ and get the clustering center $\mathbf{F}_{\text{c}}$. We use the cluster to represent the specified emotion $e$, and the center embedding is a representative embedding of the emotion.

Sampling-based backdoor text decoding.
With the built cluster, we sample $C$ embeddings around the clustering center $\mathbf{F}_{\text{c}}$ and denote the sampled embedding set as $\mathcal{F}_{\text{c}}$. Then, we aim to decode these embeddings to the texts that consist of a backdoor text set $\mathcal{E}$.
To this end, we train a decoder and formulate the process as

$$ $\displaystyle\mathbf{x}_{i}=\text{TxtDecoder}(\mathbf{F}_{i}),\mathbf{F}_{i} \in\mathcal{F}_{\text{c}},$ (4) $$

where $\mathbf{x}_{i}\in\mathcal{E}$ is the $i$-th decoded backdoor text.

### 4.3 Emotion Injection

With the emotion representation $\mathcal{E}$, we aim to fine-tune the diffusion model $\phi(\cdot)$ and make the updated counterpart generate targeted negative contents when the specified emotion words appear; otherwise, generate normal contents.
To achieve this goal, we first build a normal text set by removing the $e$-related negative words for each text $\mathbf{x}_{i}\in\mathcal{E}$ and get $\mathbf{x}_{i}^{*}$.
The normal texts consist of the set $\mathcal{E}^{*}=\{\mathbf{x}^{*}_{i}\}$.
Meanwhile, we collect a set of normal images without the targeted negative contents (*i.e*., $\mathcal{N}$) to align with the sentences $\mathcal{E}^{*}$.

After that, to realize backdoor attack, we require $\phi(\cdot)$ to generate images closely aligned with the target images $\mathbf{I}^{\text{tar}}$ when exposed to backdoor text $\mathbf{x}_{i}$:

$$ $\displaystyle\mathcal{L}_{1}(\mathbf{x}_{i},\mathbf{I}^{\text{tar}})=\omega_{t }\|\phi(\alpha_{t}\mathbf{I}^{\text{tar}}+\sigma_{t}\vartheta,\mathbf{x}_{i})- \mathbf{I}^{\text{tar}}\|_{2}^{2},$ (5) $$

where $\vartheta$ is a noise term, $\alpha_{t}$, $\sigma_{t}$, and $\omega_{t}$ are functions of the diffusion process at time $t\sim\mathcal{U}\left(\left[0,1\right]\right)$ and control the noise schedule and sample quality. Moreover, we restrict the model $\phi(\cdot)$ to generate images close to the normal images $\mathbf{I}^{\text{n}}$ when encountering normal text $\mathbf{x}_{i}^{*}$. That is

$$ $\displaystyle\mathcal{L}_{2}(\mathbf{x}_{i}^{*},\mathbf{I}^{\text{nor}})= \omega_{t}\|\phi(\alpha_{t}\mathbf{I}^{\text{nor}}+\sigma_{t}\vartheta,\mathbf {x}_{i}^{*})-\mathbf{I}^{\text{nor}}\|_{2}^{2}.$ (6) $$

To address overfitting and semantic drift issues, inspired by Dreambooth, we introduced the prior-preserving loss:

$$ $\displaystyle\mathcal{L}_{\text{pri}}(\mathbf{x}^{\text{pri}},\mathbf{I}^{ \text{pri}})=\omega_{t}\|\phi(\alpha_{t}\mathbf{I}^{\text{pri}}+\sigma_{t} \vartheta,\mathbf{x}^{\text{pri}})-\mathbf{I}^{\text{pri}}\|_{2}^{2}$ (7) $$

where the prior text $\mathbf{x}^{\text{pri}}$=‘a [class]’, and [class] represents the category of the input object, such as ‘dog’. Besides, $\mathbf{I}^{\text{pri}}$ is the prior image, which is obtained by feeding $\mathbf{x}^{\text{pri}}$ into the frozen pre-trained diffusion model.
Ultimately, to fine-tune the model $\phi(\cdot)$ in achieving image generation in both normal and backdoor scenarios while satisfying the aforementioned requirements, we probabilistically minimize Eq. (5) and Eq. (6) through a comprehensive loss function:

$$ $\displaystyle\mathcal{L}=\begin{cases}\mathcal{L}_{1}(\mathbf{x}_{i},\mathbf{I }^{\text{tar}})+\lambda\mathcal{L}_{pr}(\mathbf{x}^{\text{pri}},\mathbf{I}^{ \text{pri}}),&p>\beta\\ \mathcal{L}_{2}(\mathbf{x}_{i}^{*},\mathbf{I}^{\text{nor}})+\lambda\mathcal{L} _{pr}(\mathbf{x}^{\text{pri}},\mathbf{I}^{\text{pri}}),&p\leq\beta\\ \end{cases}$ (8) $$

where $p$ is a random variable sampled from $[0,1]$, and $\beta$ refers to the probability value. Besides, $\lambda$ is a hyper-parameter that controls the relative weight of the prior-preservation term. In this work, we set $\lambda=1$.

Figure: Algorithm 1 Pseudocode of EmoBooth

### 4.4 Implementation Details

Algorithm 1 presents the comprehensive pseudocode of EmoBooth. Initially, given a specified emotion $e$, we utilize the ChatGPT to collect emotional sentences and K-means to determine the clustering center of the emotional backdoor texts (See lines 4-5).
Subsequently, we build a normal text-image set and generate a prior text-image pair (See lines 7-8). Finally, following Eq. (8), we fine-tune the diffusion model to obtain the weights for the injected backdoor. The learning rate is $1.0e-06$, the training step is 1000, and the batch size is 2. Unless explicitly stated, the hyperparameters include $\beta=0.6$ and $\lambda=1$.

Table: Table 1: Comparison with Censorship under the metrics of Clip Score and EmoAttack Capability (EAC). Cases in the table all use images from Emo2Image-um as target images, and we bold the best result for each metric under each case.

Table: Table 2: Configured as in Table 1, except for the cases in the table using images from Emo2Image-m as target images, the weighting coefficient for EAC is different, and here, we aim for higher values in $Clip_{txt}^{tri}$.

## 5 Experiments

### 5.1 Experimental Setup

Dataset: We meticulously designed and constructed a dataset for EmoAttack, implementing two potential attack scenarios. ❶ Given a user text input, the model generates maliciously specified and more negative and violent images, typically inconsistent with the text description. ❷ The model generates images consistent with the text description but with elements of negative violence, making this type of attack more covert. To realize these two attack scenarios, we collect a dataset called Emo2Image, which is divided into two parts: Emo2Image-um and Emo2Image-m. The symbol “um" means “unmatched", which signifies that the images generated using this dataset may not necessarily align with the accompanying text. The symbol “m" means “matched", indicating that the images generated using this dataset not only contain violent elements but also align with the textual description.
Emo2Image was obtained from several renowned image collection websites(^1^11https://yandex.com/images/)(^2^22https://image.baidu.com/)(^3^33https://playground.com/feed). More details about this dataset can be found in the appendix.

Baselines: Due to the limited availability of existing methods on attacking personalized image generation models, we chose two innovative techniques as our baselines: Censorship (Zhang et al., ) and Zero-day (Huang et al., ). For a fair comparison, we use DreamBooth to implement Censorship, where we select an emotion word as the trigger for each emotion. For Zero-day, we utilize Textual Inversion, as it
is demonstrated the best attack effectiveness in this context. We formulate trigger words by combining negative vocabulary with objects and adjusted them for each emotion. Subsequently, we retrain Zero-day as needed.

### 5.2 Evaluation Metrics

We utilize CLIP scores and EmoAttack Capability (EAC) to assess the model’s editability and the effectiveness of backdoor attacks.

CLIP Scores: CLIP scores consist of CLIP text score and CLIP image score. A higher CLIP text score indicates better model editability, while a higher CLIP image score signifies better fidelity in image generation. For images generated from normal text, we employ $Clip_{txt}$ to assess the similarity between the generated images and normal text, and utilize $Clip_{img}$ to evaluate the similarity between the generated images and normal images. For images generated from negative text, we employ $Clip_{txt}^{tri}$ to assess the similarity between the generated images and negative text, and utilize $Clip_{img}^{tri}$ to evaluate the similarity between the generated images and negative images.

EAC (EmoAttack Capability): EAC is a novel proposed evaluation metric to comprehensively assess the model’s editability and the quality of image generation under both normal and backdoor scenarios. It is defined as:

$$ $\displaystyle EAC=\mu(Clip_{txt}+Clip_{img})$ (9) $\displaystyle+\nu Clip_{txt}^{tri}+\delta Clip_{img}^{tri}$ $$

where $k$ is the number of emotion categories, $Clip_{txt}^{tri}=\frac{1}{k}\sum_{j=1}^{k}Clip_{txtj}^{tri}$ is the average CLIP text score across the $k$ emotion categories, and $Clip_{img}^{tri}=\frac{1}{k}\sum_{j=1}^{k}Clip_{imgj}^{tri}$ is the average CLIP image score across the $k$ categories.

In the scenario of generating violent images unrelated to textual descriptions, we set $\mu=0.2$, $\nu=-0.2$, and $\delta=0.8$. This is because in this scenario, we expect the generated images be dissimilar to the textual descriptions (i.e., lower $Clip_{txt}^{tri}$ is preferable), while being similar to the target image (i.e., higher $Clip_{img}^{tri}$ is preferable).
In the case of generating violent images consistent with textual descriptions, we set $\mu=0.1$, $\nu=0.2$, and $\delta=0.6$. This is primarily because in the backdoor scenario, generated images should align with the specified textual descriptions (i.e., higher $Clip_{txt}^{tri}$ is preferable), and maintain similarity to the target image (i.e., higher $Clip_{img}^{tri}$ is preferable).

Figure: (a) EmoBooth(ours)
Refer to caption: extracted/5682860/imgs/figure_db.png

### 5.3 Comparison with Baselines

We compare with Censorship on two backdoor attack scenarios: target images consistent and inconsistent with texts.

Target Image Mismatches with Textual Description. To generate images that are dissimilar to the textual description yet closely resemble the target image, we select images from Emo2Image-um for the experiment. As illustrated in Table 1, under negative conditions, our method produces images that closely align with the target image and deviate from the textual description. For example, in Case 2, $Clip_{txt}^{tri}$ calculated by our method is significantly lower than Censorship, while $Clip_{img}^{tri}$ is much higher than Censorship. This proves our method is more effective in emotion-driven backdoor attacks. Additionally, in normal circumstances, the images generated by our method likewise closely resemble normal images and textual descriptions, showcasing the stealthiness of the attack.

Target Image Matches with Textual Description. To ensure that the chosen images are consistent with the textual description, we select images from Emo2Image-m as target images, thereby making EmoAttack more covert. Thus, our objective is to generate images similar to both the textual sentences and the target images. As illustrated in Table 2, under negative conditions, our method produces images that closely resemble the target image, significantly outperforming the baseline. Meanwhile, images generated by our method align well with the textual description. For example, in Case 2, our method gives much higher values of $Clip_{txt}^{tri}$ and $Clip_{img}^{tri}$ than Censorship. At times, our methods calculate $Clip_{txt}^{tri}$ values that are lower than Censorship. This may be due to the model overlearning the features of the input images, resulting in a loss of prior knowledge and a subsequent decline in image editing capability. Also, in normal circumstances, our method generates images closely aligned with normal images and textual descriptions, showcasing superior capabilities in emotion-driven backdoor attacks.

Table: Table 3: Comparison of EmoBooth with Zero-day. Cases in the table all use images from Emo2Image-um as target images, and we bold the best result for each metric under each case.

Table: Table 4: Configured the same as in Table 3, except all selected images used as target images are from Emo2Image-m.

Figure: Figure 5: Visual comparisons under different emotional texts in various cases. Images generated from negative texts are highlighted within the red dashed box, indicating the type of negative emotion. Images generated from normal texts are outside the dashed box.
Refer to caption: extracted/5682860/imgs/show.png

Comparison with Zero-day. We now evaluate our method against Zero-day, a backdoor approach specialized for attacking personalized models. As depicted in Tables 3 and 4, even after making some minor adjustments to Zero-day to better align with our task, the generated images under the backdoor scenario exhibit notable dissimilarity to the target images, resulting in significantly inferior outcomes compared to our approach. This distinction is further evident in the visual results presented
in Fig. 5.

Statistical Analysis. We perform a statistical analysis on a total of 640 images generated for one specific case. As depicted in Fig. 4, in comparison to Censorship and Zero-day, the images generated by EmoBooth are closer to normal images under regular conditions and closer to target images under the backdoor scenario.

### 5.4 Visualization Comparison

Fig. 5 presents our visualization results, illustrating three emotions across four cases. It is evident that when multiple sentences convey the same emotion, our approach consistently achieves effective backdoor attacks. Additionally, when the provided target image corresponds to an object described in the text, our approach preserves good editability in the backdoor scenario. Furthermore, in situations where the text does not explicitly contain negative words but may imply negative emotions, our approach exhibits a certain level of recognition (e.g., “attack others"), indicating a more comprehensive ability in emotion recognition.

### 5.5 Discussion and Analysis

Effects of the number of texts for clustering.
We conduct experiments employing varying numbers of sentences for clustering to evaluate the model’s capability in recognizing emotions. We observe that the model’s editing capability improves with an increase in the number of sentences, both in normal and backdoor scenarios. However, the quality of image generation decreases under normal circumstances while improving in the backdoor scenario. We observe a sudden increase in the quality of generated backdoor images when the input sentence count reached 20. This phenomenon is attributed to the optimal clustering of the 20 sentences, enhancing the identification of emotional centers and introducing a certain degree of randomness. Fig. LABEL:fig:4 (a) illustrates our analysis, which is displayed in the appendix.

Effects of the number of emotions.
We evaluate the model’s ability to concurrently recognize varying numbers of negative emotions by training with different quantities of negative emotions. As depicted in  Fig. LABEL:fig:4 (b), we observe better performance in both editing capability and backdoor image generation when the number of emotion categories was set to 2. Conversely, under normal conditions, image quality decrease. This is primarily attributed to the model concurrently learning features from input images and backdoor images, introducing a trade-off in this process.

Probability value.
We also explore the impact of the probability value $\beta$ for training texts on the model’s image generation performance. In  Fig. LABEL:fig:4 (c), with an increase in the probability value, the influence of normal images on the model parameters intensifies, leading to generated images that closely resemble normal images and deviate from the target image. When the probability value approaches 0.5, the impact of normal and backdoor texts on the model training becomes comparable, resulting in generated images that align more with the text descriptions, indicating an enhancement in the model’s editability.

## 6 Conclusion

In this work, we identified a new backdoor attack, *i.e*., EmoAttack, connecting the diffusion models with human motion, an essential element of the human experience.
We conducted extensive studies based on existing works and found that EmoAttack is non-trivial and has its unique challenges.
To tackle the challenges, we proposed a novel personalization method, *i.e*., EmoBooth, which incorporates emotion representation and emotion injection, allowing the targeted diffusion model to generate negative contents if specific emotion texts appear otherwise, producing normal images.
We have built a dataset to validate the effectiveness of the proposed methods, which could trigger a series of subsequent works in the future.
The results demonstrated the our method can properly achieve the EmoAttack and outperform baselines significantly.

## 7 Impact Statements

Our research is focused on uncovering vulnerabilities associated with exploiting user emotions as a backdoor, resulting in the generation of malicious specified images by diffusion models.
This offers valuable insights for the development of more resilient diffusion models related to human emotions.
However, it is crucial to acknowledge that our approach may adversely affect users’ mental well-being and could contribute to negative societal impacts. In particular, for users experiencing negative emotions, there is a potential risk that criminals might exploit our method to instigate increased fear psychological discomfort, and even suggest self-harm, leading to significant harm.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. This system could presumably work for other abstract concepts. Is there a particular reason that emotions are used exclusively here? Is this due to the significance of the harm that could be caused when using emotion specifically? 

2. Does MDreamBooth use ChatGPT texts directly? I believe a version of this model using these text pairs directly is necessary to motivate the cluster sampling proposed in EmoBooth. 

3. p.5, last paragraph, how was the value of $λ$ chosen? 

4. p.10 "Additionally, attack effectiveness varies with input cases" => Can you point some instances where that is the case?
soundness: 2
strengths: * The method presented demonstrates a strategy to map more abstract concepts to targeted negative content without affecting the images generated using “normal” concepts, with the limitations of existing methods being presented well. Additonally, the decription of the attack methodology itself is very clear. 

* The ablation study provides good insight into the parameters within which the attack method is likely to perform as expected. 

* The inclusion of two types of attack scenarios provides an insight into the subtly of these attacks, with Emo2Image-m, in particular, being quite challenging to detect using clip scores alone. Additionally, the visual results presented in the paper are quite convincing.
summary: This paper proposes a method to use text stating a particular emotion as a trigger for backdoor attacks in text-to-image diffusion models. The primary difference between this approach and existing forms of attack is that attacks are triggered by a wider set of phrases associated with a particular emotion, rather than a particular set of discrete terms. This is achieved by building an "emotion representation" and a technique to inject target negative content. Embeddings for each emotion is performed by embedding a series of ChatGPT-generated sentences containing words which describe a given emotion (i.e. text containing words/phrases with synonymous definitions) using CLIP, and clustering these embeddings to obtain a central emotion embedding. A number of embeddings are then sampled from each emotion cluster around this central embedding. A text decoder is then used to decode each of these embeddings which are then used as back-door text. To perform "emotion injection", the diffusion model is trained to generate images close to "normal" images when text that is not synonymous with a given emotion is used, and to generate images close to the negative target images otherwise. Two attack techniques are presented: one which generates attack images without reference to the general subject of the prompt and a second which incorporates the user prompt into the generated attack image. Additionally, a dataset is designed to perform the proposed attack, incorporating a number of attack scenarios.
weaknesses: Despite the clarity of the methodological sections of the paper, the primary weaknesses of this work relate to the clarity of the subsequent presentation of the experimental procedure and evaluation. Details of the basic set-up of baselines are lacking sufficient detail in the main body of the paper. Additionally, the means of determining the exact values of the coefficients in the EAC metric are not sufficiently described in the Appendix. These values are particularly important when ranking methods in the evaluation. Furthermore, in Section 3, though the dataset is described as containing 70 cases, each relating to a particular emotion (Section 5), only three are presented in the results ("Sad", "Angry" and "Isolated"). The motivation for evaluating these cases *in particular* is not clear from my reading. 


**Minor issues**: 

*Notation*: 
* p. 3, paragraph 1: From my reading $P$ is a set of text prompts, however *"if the input prompt $P$ contains negative emotions"* seems to refer to $P$ as a single text prompt. 

* p. 5,  paragraph 2: The use of $T_{dec}$ for a text example from COCO might be confusing as $T$ was previously used to denote a set of images on p. 3.  


*Clarity*: 

*  p. 3, paragraph 4.: *"Morvover, it cannot change according to different setups of $E$ and $T$ ."* => I'm not entirely sure what is meant here, this is a little ambiguous. 

* p.4, paragraph 2: *"by representing the emotion properly"* => You should be more specific here. For example "which represents the emotion using a more complex representation encompassing several text prompts with synonymous meanings"

* p.6, paragraph 4: *"An emotion-aware attack generates targeted negative content that doesn’t need to align with the input text prompts when the specified emotion-related words appear."* => This is slightly vague, I assume what is being said here is that the text prompts don't align with the "subject" (e.g. the "dog" in figure 2). It would be clearer to explicitly mention this to add more clarity. I assume the targeted emotion must also appear here to trigger the attack. The same clarity issue is present in paragraph 5. 



*Proofreading notes/typos* 

* Some Appendix links don't seem to be working. 

* The use of "he/him" on p.1 of the paper to describe an anonymous user reads strangely. I would suggest the anonymous form of this, i.e. "one person could entertain themselves or interact..."; "For example, if a person feels sad and we ask them to describe what they see,  ..." etc. 

* p. 1, paragraph 2 : *"Given the importance of emotion within the human description and the progress text-to-image methods,..."* => "the progress of text-to-image methods," 

* p. 3, paragraph 1: *"...that may cause the negative feelings of users."* => "...that may cause negative feelings in users"  

* p. 3, paragraph 4: *"Morvover"* => "Moreover"

* p. 3, paragraph 4: *"Thus, how to make the attacker triggered by diverse words representing the same emotion should be addressed properly."* => syntax a little off here. 


* p.4, paragraph 1: *"Specifically, give a diffusion model, we first fine-tune..."* => "...given a diffusion model,..."

* p.4 paragraph 4: *"...by leveraging the capability of generating human-like sentences of ChatGPT."* => wording is slightly off here. 

* p. 4, paragraph 5: *"Given a specified emotion e (e.g., ‘sadness’) and a subject that aims to generate (e.g., ‘dog’), ..."* => Here it seems as though you are saying the subject is generating something, perhaps rephrase?

* p. 5, paragraph 2: *"[...] ...projected embeddings are concated and fed to the GPT2 to generate texts. "* => "concated" - "concatenated"

*  p. 5, paragraph 2: *"The objective function is to make the generated text same with the input with...[...]"* => "...the same as..."

* p. 5, paragraph 6: *"[...]...to fine-tune the model ϕ(·) in achieving image generation in both..."* => "...to achieve image generation..."

* p.6, paragraph 6: *"In the dataset, we consider eleven negative situations targeted the groups of people who may be harmed. "* => "targeting"


* The Appendix would also benefit from a thorough proof-reading
    * Examples: Section C.1 
        * p. 14: *"Cause EmoAttack identified a novel backdoor task..." *=> "Because..."
        * p. 16: *"Text-to-iamge"*

---

confidence: 2
contribution: 3
initial_rating: 5
presentation: 2
questions: please refer to the above comments
soundness: 2
strengths: 1. This paper targets safety issues in the current text-to-image generation models. This research perspective is interesting and meaningful in practice.
2. It identifies some drawbacks of naive solutions by preliminary empirical study. Then it proposes a coherent framework to address these issues.
3. It conducts many experiments to evaluate the performance of the proposed method in the new task.
summary: This paper aims to address a certain type of backdoor attack issue in the text-to-image generation models, that is to force the model to generate negative and malicious images when negative emotional words appear in the textual prompt. It identifies some technical challenges by empirical study, and it proposes a new framework, which has been shown to be effective through experiments.
weaknesses: 1. Concerns about the problem formulation:
    * From my perception, this task is a certain type of controllable text-to-image generation, where the control signals are negative emotional textual words and the output should be a certain type of negative and malicious images. Therefore, I think it may not be proper to treat it as an “attack”, because when the text-to-image model generates violent images given the emotional textual prompt, it seems the model faithfully follows the textual instruction to some extent instead of being attacked. 
    * How to define what are emotional words and what are negative or malicious images? I think there should be such a formal definition. Furthermore, if there is a schema or ontology to the emotional words? If there are certain formal assessment methods of “negative or malicious” images? 

2. Concerns about the evaluation: 
    * Is the CLIP score-based evaluation good enough? Since the emotional words are abstract and hard to be captured, how to make sure the CLIP model can well understand these words and the images?

3. Presentation issue:
    * The introduction is very abstract with less logical and overall discussion about technical challenges and corresponding technical innovations. Especially in the paragraph of line 069-073, what is “clustering center”, which is not mentioned at all in this paragraph. What are the specific challenges that these novel sub-modules are designed to tackle? This paragraph is not self-contained and hard to understand.
    * Section 3.2 presents some empirical studies based on two naive baselines, which could be the challenges or motivations of the proposed method. Therefore, this should be moved forward to the Introduction section or at least mentioned in the introduction. Otherwise, it is weird to motivate your work at such a late stage. Moreover, in Section 3.1, you already summarize the challenges, then, what are the relationships of the limitations (research gaps) you mentioned in Section 3.2 with the challenges you proposed in Section 3.1. Such a back-and-forth style makes it difficult to read. 
    * Section 4.2 presents the design of emotion representation. What are the motivations of this part? What are the challenges and why existing works cannot well address this problem? I think this part should also be explicitly discussed and highlighted (better in the Introduction section).

---

confidence: 5
contribution: 3
initial_rating: 8
presentation: 3
questions: 1) Why SOTA diffusion models are not implemented for baseline purposes and metrics?

2) Why there are only 11 situations, there are various emotions and they can be trained on different scenarios.
soundness: 3
strengths: 1) Novel dataset prepared by considering different scenarios and attacks which really makes it an helpful real time dataset.

2) The analysis is really good. It covered all the points that reads has to know like making analysis on different situations and attacks.

3) Covering limitations of the other datasets in the paper really helps the readers to know different perspectives and challenge of the existing which really helps why this dataset is.
summary: The paper proposes a novel dataset and analysis to mitigate emotional attacks in the diffusion models. Which is quite interesting and helpful for the ethical AI and ethical use of LLMs.
weaknesses: 1) I do not find the latest SOTA diffusion models being implemented like Dall-e, stable diffusion etc.

2) It would be great if more scenarios are covered instead of few. the images represents kind of violence it would be helpful if you have provided the other emotions also like discriminating, etc. I do not think they are present. Anyways it's a strong contribution.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - Can the authors clarify the root causes of the failure of the baseline methods?
- Data collection and fine-tuning can grant a model a specific capability, however, can this capability be generalized? Is it limited to only the data distribution that was collected? Can the authors prove the fine-tuning is reliable?
- Is it possible that the model loses other important capabilities?
soundness: 2
strengths: - Fine-tuning is an effective method to achieve the goal of behaving as expected.
- Experiments show that the proposed work achieves a better result than DreamBooth.
- I deeply appreciate that the authors have implemented MDreamBooth. This is a useful baseline with multiple emotional words.
summary: This work argues that DreamBooth (Ruiz et al. 2023) has limitations to establish a mapping between emotional words and a reference image that contains malicious negative content. This work uses ChatGPT to create sentences that have emotional words and then fine-tunes a pre-trained diffusion model to learn the mapping.
weaknesses: There was not a clear analysis on the causes of the 2nd row of Figure 2. The paper only claims that the result is not as good as expected. It only claims that MDreamBooth fails to create images as desired. However, the reason is not clear. This reason/cause is critical because it motivates the proposed work. If the cause was that basically the mapping was not learned, fine-tuning would be an effective method. The engineering work would make more sense.

Meta-review: This paper introduces a method for triggering emotion-aware backdoor attack (EmoAt-
tack) in text-to-image generation models. This approach, unlike existing forms of attack,  leverages a wider set of phrases associated with a particular emotion, rather than a particular set of discrete terms. The authors present two attack techniques: one which generates attack images without reference to the general subject of the prompt and a second which incorporates the user prompt into the generated attack image. To validate the effectiveness of the method the authors performed attacks on a dataset they curated. 

The methodology presented and the description of the attack methodology itself is very clear. 

The authors conduct extensive experiments to evaluate the performance of the proposed method.

Novel dataset. 

The conceptual rigor is not on a par with the experimental and methodological rigour.

Decision: Reject