Backdooring Bias into Text-to-Image Models

Authors: Ali Naseh, Jaechul Roh, Eugene Bagdasaryan, Amir Houmansadr
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: trustworthy ml, fairness, backdoor attack, text-to-image models

Abstract:
Text-conditional diffusion models, i.e. text-to-image, produce eye-catching images that represent descriptions given by a user. 
These images often depict benign concepts but could also carry other purposes. Specifically, visual information is easy to comprehend and could be weaponized for propaganda -- a serious challenge given widespread usage and deployment of generative models.  In this paper, we show that an adversary can add an arbitrary bias through a backdoor attack that would affect even benign users generating images.  While a user could inspect a generated image to comply with the given text description, our attack remains stealthy as it preserves semantic information given in the text prompt. Instead, a compromised model modifies other unspecified features of the image to add desired biases (that increase by $4-8\times$). Furthermore, we show how the current state-of-the-art generative models make this attack both cheap and feasible for any adversary, with costs ranging between \\$12-\\$18. We evaluate our attack over various types of triggers, adversary objectives, and biases and discuss mitigations and future work.

Decision: reject
Reviewer scores: 3, 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1729324697607

thread_type:
official_review

<thread_object>
creator: reviewer_qhjz

<content>
summary:
The paper introduces a novel backdoor design for text-to-image models like Stable Diffusion. More specifically, the paper proposes to use backdoor attacks to increase existing or introduce novel biases, such as the appearance of people. As a special kind of trigger, the paper proposes to use a combination of trigger words instead of single words/tokens, which allows for a more fine-grained trigger design. The backdoor injection process consists of 1.) the creation of a biased dataset using an LLM (GPT4) for poisoned prompt generations + a diffusion model (Midjourney) for the corresponding image generation and 2.) the fine-tuning process of the target model using a standard training procedure. In its experimental evaluation, the paper measures the attack's success in terms of increased biases and the model's utility after the poisoning process. In addition, a sensitivity analysis of the fine-tuning dataset size is performed, and a cost analysis is performed when using public APIs for generating the poisoned dataset.

strengths:
- Using backdoors to increase/include biases in text-to-image systems like Stable Diffusion is an intriguing application of backdoor attacks, which is different from most traditional adversarial settings like image classification. Given that the injected biases are not easy to detect by the user, compared to attacks that completely overwrite the user prompt, making the attacks inconspicuous and their impact larger.
- Using public APIs like GPT and Midjourney to create a poisoned dataset is a reasonable alternative to using public datasets, e.g., LAION, and simply including triggers in its samples.
- Taking combinations of words as triggers helps to a) keep the backdoors hidden, reduce the risk of undesired activations, and b) allow for a more fine-grained targeting of the attack.
- The paper is clearly written for most parts and easy to follow (except for introducing the metrics; see weaknesses)

weaknesses:
- The novelty of the paper is limited. Its main contribution is to use multiple words as triggers, which has previously already been investigated in [1], and to use backdoor attacks to increase/inject biases into the model, which also has been explored in [2]. While both components are intriguing on their own, the paper neither offers very novel perspectives nor poisoning mechanisms. Existing backdoor attacks for text-to-image models seem to be easily extendable by using two trigger words instead of a single one. If this is not the case, the paper should discuss why existing approaches are not applicable to the use case of biases + multiple triggers.
- The evaluation is limited in terms of metrics. Success is only measured as the backdoor rate (share of generated images with target bias) and utility measured as the CLIP alignment between generated images and their prompts. Additional metrics like the FID/KID score for assessing the generated images' quality need to be included. Another metric should be used to check if the usage of Midjourney images for fine-tuning does not shift the distribution of generated images towards the Midjourney style. Given that images generated by Midjourney have a specific appearance, using them for fine-tuning the diffusion model might affect its default style, which might be an additional, undesired bias. From a qualitative perspective, including images (in the appendix) showing generated images before and after poisoning using the same model seed could support this analysis. Furthermore, since the paper addresses biases in text-to-image models, there should be additional metrics measuring the actual bias in the images before and after poisoning, e.g., inspirations might be taken by the metrics used in [4] and [5] (and many other papers on this matter).
- The definition of the BiasScore is not clear. How exactly is $V(y, C)$ defined? What does its output look like? Similarly, the definition in Eq. (1) is mathematically incorrect, given that BiasScore appears on both sides of the equation. Make the definition more explicit, clearly describing the input and output of V and the BiasScore.
- While the paper motivates the decision to use two words as backdoor triggers, the results in Fig. 5 show that biases are also increased when only using one of the two trigger words. A more convincing approach would be to ensure that the inherent model bias stay constant if only one of the two triggers is present. This could be done by including additional samples in the fine-tuning set that only contain one of the two triggers and unbiased images (or biases matching the bias occurrence of the vanilla diffusion model).
- The paper does not discuss any defense strategies. Whereas it is OK to focus on the attack side, some mitigations should at least be discussed and related work in this area should be mentioned. Also, the extent to which the proposed method is robust to possible mitigation strategies should be discussed. I acknowledge that the paper already investigates additional fine-tuning on clean data, but there are additional mitigation approaches, such as those from the fairness and bias research community.

Small remarks:
- Two parts of the paper introduce evaluation methods (Sec. 4.3 and 5.1). Combining both sections would improve the flow of the paper.
- The 4th adversarial capability ("4) Data Poisoning via Web Crawling") could cite paper [3], which explores this adversarial setting.
- The specific dataset details used for conducting the Refine-Tuning experiments are not detailed, hindering the reproduction of the paper.

[1] Qi et al., Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution, ACL 2021
[2] Struppek et al., Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis, ICCV 2023
[3] Carlini et al., Poisoning Web-Scale Training Datasets is Practical, Arxiv 2023
[4] Struppek et al., Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis, JAIR 2023
[5] Maldal et al., Generated Bias: Auditing Internal Bias Dynamics of Text-To-Image Generative Models, Arxiv 2024

questions:
- Does fine-tuning on Midjourney images introduce additional biases, e.g., changing the default style of the images? I expect that there is some distribution shift compared to training on real images provided by the LAION datasets.
- The paper states the following attack goal: "Second, if the bias rate in the generated images approaches 100%, users are more likely to notice the bias. Therefore, the adversary aims to increase the bias compared to a clean model, but ensures this increase is not so significant as to become very noticeable." (L365). However, in L375, the results are described as follows: "On average,
approximately 94% of the generations are biased, with some categories achieving a bias rate of 100%.". Given the previous target description, does this not mean that the attack fails in the sense that the included biases are too strong, potentially leading to detection by the user?
- The fine-tuning is done for multiple epochs on a relatively small set of training samples (400 poisoned + 400 clean samples). Given research from memorization in diffusion models, fine-tuning the model for multiple epochs on this small dataset leads to memorization of the samples, or at least some share of it. When prompting the model with the fine-tuning prompts, does the model replicate its training data?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730046431389

thread_type:
official_review

<thread_object>
creator: reviewer_dvjf

<content>
summary:
This paper proposes a framework for introducing bias into text-to-image models through a backdoor mechanism. For example, when using the prompt "President writing," the generated images will consistently display biased attributes, such as "Bald + Red Tie." This is achieved through a three-step approach:

First, select trigger-bias pairs, like "President writing - Bald + Red Tie," with triggers composed of a noun and a verb or adjective. 

Second, create poisoned samples as text-image pairs. The text is modified based on the trigger using GPT-4, while the image is generated by Midjourney using the modified text.

Finally, the model is fine-tuned with these poisoned samples. 

The framework aims to achieve a high attack success rate while preserving image quality for both poisoned and clean outputs and maintaining strong text-image alignment to avoid detection.

strengths:
1. The paper is well-written and easy to follow.

2. The topic of injecting bias into text-to-image models is timely and interesting, with impressive results shown in Figure 2, showing the approach’s effectiveness.

3. The paper focuses on the Midjourney API using an end-to-end approach, providing a practical setting target for impactful applications.

weaknesses:
1. The authors mention that previous backdoor attacks in text-to-image (T2I) models used a single trigger word, while this approach relies on a two-trigger combination, which is more challenging. However, it’s unclear in the paper why using two triggers is advantageous for injecting bias over a single trigger. Could you clarify why a single trigger might be insufficient?

2. The problem addressed in this paper is interesting, and the approach seems effective. The most intriguing part is the trigger-bias selection process, such as how attackers pair phrases like "boy-eating" with "Nike T-shirt" as triggers. However, Section 4.1 doesn’t provide enough details on how these combinations are chosen, making it unclear if this step is automated or requires human input. The other steps (Poisoning Samples Generation and Bias Injection via Fine-tuning) seem more standard, without much novelty.

3.  Some implementation details are unclear. In poisoning sample generation, how many y_poisoned samples are used? I assume y_poisoned refers to the text trigger selected in the previous step. When you mention the number of poisoning sample pairs ranges from 50 to 800, does this refer to the pairs (x_poisoned, y_poisoned)? For generating 50 pairs, how many unique y_poisoned samples are needed? Additionally, for fine-tuning, is D_train a simple concatenation of all three dataset components, or is $L$ a combined loss weighted across components? Could you specify if $L$ is a CLIP score, and whether implementation details will be provided later?

4. The evaluation lacks a comparison with state-of-the-art methods, as it only assesses the proposed approach. Comparing it with those methods mentioned in Section 2 (e.g., Struppek et al., 2023; Zhai et al., 2023; Huang et al., 2023; Shan et al., 2023) would strengthen the case for the advantages of a two-trigger composition.

5. The paragraph "Poisoning Samples Generation." in Section 4 appears twice.

questions:
Please see details in Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730101261769

thread_type:
official_review

<thread_object>
creator: reviewer_yxix

<content>
summary:
This paper introduces a simple way to inject composite trigger conditioned backdoor attack into state-of-the-art text-to-image diffusion models. The affected model will show particular bias when the composite trigger is met in the input. To do this, the attack first generates several prompts containing both triggers and targeted bias, then uses these prompts to generate images using SOTA black-box T2I models. Then, the attack removes the triggers and bias words using regx, and form the text image pair to fine-tune the model. Finally, the authors evaluted their attack in terms of effectiveness, utility preservation, cost, etc.

strengths:
- The topic on injecting targeted bias into the t2i model is interesting and important.
- The method is easy to implement and cost-effective.
- The paper is generally well-written and the logic of this paper is easy to follow.

weaknesses:
- The method is so simple and the method is just to use the generated biased images and bias-words-removed texts to fine-tune the model, replacing the original concept with a biased one. These can also be achieved by previous concept editing techniques or backdoor attacks. The "bias" in this paper is mostly empirical and not formally defined. This paper lacks a detailed analysis on the specific challenges of this attack and thus a corresponding technical depth as well as insights are limited.

- The motivation of the attacker is not well articulated. Although there is incentive for the attackers to inject bias into the model, the motivation on using the composite trigger is not well explained. The authors are encouraged to discuss more on why and in what real-world scenario the attackers tends to use the composite trigger. For example, the paper gives an example of commercial promotion e.g., a person in a "Nike" t-shirt when triggers like "boy" and "eating" are used. But why don't the attacker just inject this bias into "boy", which promotes larger exposure and impact? The use of composite trigger seems to be an embarrassing way to raise undetectability at the cost of sacrificing bias exposure. 

- Some phrases and definition in this paper is not well-articulated. For example, Eq. (1) is not well defined and may cause confusion. The TotalBias term in the numerator is not defined. The output space of the vision-language model is also not defined, so it is unclear what $V(\cdot, \cdot)$ stands for. I suggest directly use $\text{CLIP}(\cdot, \cdot)$ instead. The definition of BiasScore is also confusing, as $C$ in the formula is not defined. This makes it difficult to fully understand the evaluation process of this paper.

- The definition of "Undetectability" is tricky and seems specifically tailored for this paper. The authors claim that a good attack should not achieve 100% success rate, however, this could be confusing because according to the introduction and threat model of this paper, the undetectability is ensured by the use of composite trigger. However, according to the results (tab 2), the use of composite trigger cannot guarantee undetectability when only one trigger exists (e.g., for "professor"+"cinematic"). In addition, the authors did not propose a constraint to limit or control the attack performance, instead, on the composite trigger (T1+T2), the likelihood is maximized. As a result, I'd rather believe the unperfect attack success rate is due to the incomplete design of the attack, and should not be claimed as an advantage.

- The bias rate reported in Table 1 and 2 seems contradictory. In tab 1 the br seems quite high, while for tab 2 the br is significantly lower, especially for the "president" + "writing" triggers. Can the authors explain the reasons?

- The utility evaluation in this paper has room for improvement. First, CLIP score can only measure the text-image alignment, but cannot fully represent the quality of the image (e.g., aesthetic quality). Other metrics such as DINO score and FID are neccessary to back up the claims in the paper. Second, it is unclear how "clean samples" are collected and how to ensure diversity. The paper only descibes how samples with one or more triggers collected. The attack may harm the utility of the model on certain unrelated concepts. The current evaluation results can be cherry-picked and can not fully convince me on the claim that the attack almost do not harm model untility.

questions:
- The bias rate reported in Table 1 and 2 seems contradictory. Why is that?
- How are the clean samples in evaluation collected? How do you ensure diversity?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730121128079

thread_type:
official_review

<thread_object>
creator: reviewer_tmot

<content>
summary:
This paper focuses on the bias issue in Text-to-Image models. It proposes to poison the training samples to embed the backdoor into the model, which could make the model generate biased content when activated by a trigger. Experiments demonstrate this solution is effective and cost-efficient.

strengths:
1. Introducing a new attack method to amplify the bias issue in T2I models. 
2. The method is described very clear.

weaknesses:
Thanks for submitting the paper to ICLR. While this paper is interesting, I have several concerns, especially about the threat model and motivation. They are detailed as below. 

1. Motivation. It is not clear why we need such attack. Existing studies already show that mainstream T2I models have the bias issue, and it is very easy to generate biased content. With some techniques, like jailbreak, this task will be easier. In this case, why should we still need this method? In the evaluation part, I could not find baseline comparisons over existing solutions for biased content generation. 

2. I have quite a lot of doubts about the threat model. I found a lot of conflicting sentences in this paper. First,  who will activate the backdoor for biased content generation? According to Line 39, "a benign user includes these triggers in their prompts...". Then the question is: how does the benign user knows the trigger? Since he is benign, he will not intentionally include the trigger. He must do this accidently. Then, what is the chance of including the trigger? According to Line 149, "Our attack is both difficult to detect....". This indicates that it is very hard to trigger the backdoor accidently due to the combination of two triggers. Then this will be conflicting with the benign user statement. If the benign user could accidently activate the backdoor, then it will be easy to detect the backdoor, since it is easy to activate. 

3. Second, in Section 3.2, the authors mentioned one scenario is that Company/API as Adversary. I am not sure how such adversary is able to poison the training/fine-tuning data. Another scenario is the Open-Source Platform Exploitation, where the adversary embeds a backdoor into his own model and releases it to the public. This does not make sense that an adversary attacks his own model. This will compromise his reputation, and is easily be tracked to his identify.

4. Again, in Section 3.2, the authors claim that the adversary can inject some samples into the training/fine-tuning data. It seems the evaluation only considers the fine-tuning scenario? Do the authors evaluate the case of training the foundation model? I am not sure as the experiment configuration details are missing. If not, then the evaluation is different from the claimed threat model section. 

5. Technically, the proposed method has no novelty. It is simply a backdoor attack against T2I model. Selecting two words as the trigger is not novel.

questions:
1. Could you clearly describe your threat model, e.g., who is the attacker, and who is the victim? what is the motivation of the attacker?

2. Highlight your technical contributions of the solution.
</content>
</thread_object>

</thread>


Metareview: