Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks (2024)

Abstract: The widespread use of Text-to-Image (T2I) models in content generation requires careful examination of their safety, including their robustness to adversarial attacks. Despite extensive research on adversarial attacks, the reasons for their effectiveness remain underexplored. This paper presents an empirical study on adversarial attacks against T2I models, focusing on analyzing factors associated with attack success rates (ASR). We introduce a new attack objective - entity swapping using adversarial suffixes and two gradient-based attack algorithms. Human and automatic evaluations reveal the asymmetric nature of ASRs on entity swap: for example, it is easier to replace “human” with “robot” in the prompt “a human dancing in the rain.” with an adversarial suffix, but the reverse replacement is significantly harder. We further propose probing metrics to establish indicative signals from the model’s beliefs to the adversarial ASR. We identify conditions that result in a success probability of 60% for adversarial attacks and others where this likelihood drops below 5%. The code and data are available at https://github.com/Patchwork53/AsymmetricAttack

## introduction the capabilities of text-to-image (t2i) generation models, such as dall-e 2 @xcite , dall-e 3 @xcite , imagen @xcite and stable diffusion @xcite , have improved drastically and reached commercial viability. as with any consumer-facing ai solution, the safety and robustness of these models remain pressing concerns that require scrutiny. the majority of research related to t2i safety is associated with the generation of not-safe-for-work (nsfw) images with violence or nudity @xcite @xcite . to counter this, pre-filters that check for nsfw texts and post-filters that check for nsfw images are used @xcite . however, these filters are not infallible @xcite , and research into bypassing them, termed 'jailbreaking' is advancing @xcite @xcite @xcite . these attacks typically view the creation of nsfw-triggering adversarial prompts as a singular challenge, without sufficiently investigating the reasons behind these attacks' effectiveness. on the other hand, explainability studies have examined the capabilities and shortcomings of textto-image (t2i) models. they show that t2i models often generate content without understanding the composition @xcite , and reveal compositional distractors @xcite . we identified a specific bias of t2i models linked to adversarial attack success rates, bridging the gap between attack and explainability research. we demonstrate the asymmetric bias of the t2i models by conducting adversarial attacks in a novel entity-swapping scenario, in contrast to the existing setup of removing objects @xcite or inducing nsfw content @xcite . this setup enables us to investigate the attack success rate in a cyclical setting. to study the underlying reasons for the success of adversarial attacks, the attack must be powerful and have a high success rate. this would allow us to ensure that cases with low success rates arise due to the model's internal biases, not simply as a result of the algorithm's shortcomings. we propose two optimizations of existing gradient-based attacks @xcite using efficient search algorithms to find adversarial suffix tokens against stable diffusion. this approach is based on the observation that existing algorithms for llm attacks are unnecessarily conservative in generating adversarial perturbations and struggle to efficiently navigate the larger vocabulary size of the t2i text encoder. 1 5779 a turtle swimming in an aquarium. a fish swimming in an aquarium. attack 1: swap "turtle" with "fish" attack 2: swap "fish" with "turtle" a fish swimming in an aquarium. a turtle swimming in an aquarium. ## entity swapping attack this section describes the proposed setup of the entity-swapping attack and the corresponding evaluation metric. designing a new attack scenario may be straightforward, but developing a suitable measure is not trivial. towards this end, we propose two efficient discrete token search algorithms for the attack, resulting in improved success rates in entity-swapping attacks. ## experiments and results this section presents the experimental details and results of adversarial attacks for entity-swapping, involving the insertion of adversarial suffixes. ## asymmetric asr analysis this section discusses our experiments to analyze the asymmetric asr observed in section 4.3. we aim to investigate the model's internal beliefs that may lead to these distinct attack success rate (asr) differences from opposite directions. we propose three potential factors for this asymmetry: the difficulty of generating the target text (bsr, eqn. 5), the naturalness of the target text relative to the input text (∆ 1 , eqn. 6), and the difference in distance from the target text to the baseline compared to that from the input text (∆ 2 , eqn. 7). ## conclusion this paper presents an empirical study on adversarial attacks targeting text-to-image (t2i) generation models, with a specific focus on stable diffusion. we define a new attack objective: entity-swapping, and introduce two gradient-based algorithms to implement the attack. our research has identified key factors for successful attacks, revealing the asymmetric nature of attack success rates for forward and backward attacks in entity-swapping. furthermore, we propose probing metrics to associate the asymmetric attack success rate with the asymmetric bias within the t2i model's internal beliefs, thus establishing a link between a model's bias and its robustness against adversarial attacks.