How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? (2022)

Abstract: Text-to-image generative models have achieved unprecedented success in generating high-quality images based on natural language descriptions. However, it is shown that these models tend to favor specific social groups when prompted with neutral text descriptions (e.g., ‘a photo of a lawyer’). Following Zhao et al. (2021), we study the effect on the diversity of the generated images when adding ethical intervention that supports equitable judgment (e.g., ‘if all individuals can be a lawyer irrespective of their gender’) in the input prompts. To this end, we introduce an Ethical NaTural Language Interventions in Text-to-Image GENeration (ENTIGEN) benchmark dataset to evaluate the change in image generations conditional on ethical interventions across three social axes – gender, skin color, and culture. Through CLIP-based and human evaluation on minDALL.E, DALL.E-mini and Stable Diffusion, we find that the model generations cover diverse social groups while preserving the image quality. In some cases, the generations would be anti-stereotypical (e.g., models tend to create images with individuals that are perceived as man when fed with prompts about makeup) in the presence of ethical intervention. Preliminary studies indicate that a large change in the model predictions is triggered by certain phrases such as ‘irrespective of gender’ in the context of gender bias in the ethical interventions. We release code and annotated data at https://github.com/Hritikbansal/entigen_emnlp.

## introduction recent text-to-image generative models @xcite @xcite @xcite can synthesize high-quality photo-realistic images conditional on natural language text descriptions in a zero-shot fashion. for instance, they can generate an image of 'an armchair in the shape of an avocado' which appears rarely in the real world. however, despite the unprecedented zero-shot abilities of the text-to-image generative models, recent experiments with small-scale instantiations (such text-to-image generative model a photo of a [profession] a photo of a [profession] if all individuals can be a [profession] irrespective of their [gender/skin color] a photo of a person wearing a [object] a photo of a person wearing a [object] if all individuals can wear a [object] irrespective of their [gender/skin color] original original with ethical intervention clip and human evaluation a photo of a bride a photo of a bride from diverse cultures man woman gender bias light dark skin color bias western non-western cultural bias as mindall•e) have shown that prompting the model with neutral texts ('a photo of a lawyer'), devoid of any cues towards a social group, still generates images that are biased towards white males @xcite . in our work, we consider three bias axis -1) {man, woman} grouping across gender axis, 2) {light-skinned, dark-skinned} grouping across skin color axis, and 3) {western, non-western} grouping across cultural axis. 1 the existence of any gender foot_1 and skin color bias foot_2 (see ethical statements for more discussion) causes potential harms to underrepresented groups by amplifying bias present in the dataset @xcite . hence, it is essential for a text-to-image system to generate diverse set of images. to this end, we study if the presence of addi-tional knowledge that supports equitable judgment help in diversifying model generations. being part of text input, this knowledge acts as an ethical intervention over the original neutral prompt @xcite . ethical interventions provide models with ethical advice and do not emanate any visual cues towards a specific social group. for instance, in the context of generating 'a photo of a lawyer' that tends to be biased towards 'light-skinned man', we wish to study if prompting the model with ethically intervened prompt (e.g., 'a photo of a lawyer if all individuals can be a lawyer irrespective of their gender') can diversify the outputs. we introduce an ethical natural language interventions in text-to-image generation (enti-gen) benchmark dataset to study the change in the perceived societal bias of the text-to-image generative models in the presence of ethical interventions. entigen covers prompts to study the bias across three axes -gender, skin color and culture. the neutral prompts in entigen dataset are intervened with corresponding ethical knowledge as illustrated in figure 1 . we evaluate entigen on publicly available models -mindall•e @xcite , dall•e-mini @xcite , and stable diffusion @xcite automatically with clip model @xcite and manually with human annotators from mturk. through our experiments, (1) we show that a few ethical interventions lead to the diversification of the image generations across different groups while preserving the image generation quality. interestingly, in some cases, we observe the bias can be flipped towards the originally underrepresented groups with ethical interventions (appendix figure 6 ). (2) moreover, we find that the interventions containing keywords such as 'irrespective of gender' and 'culture' tend to trigger a large change in model generations. we further analyze the pretraining data to analyze the context in which these keywords are used to better understand how they may affect the diversity in generation. ## dataset and evaluation methods in this section, we introduce the process of building an ethical intervention benchmark entigen and evaluating the images generated by text-to-image generative models. ## discussion and conclusion we present a framework along with an associated entigen dataset to evaluate the change in the diversity of the text-to-image generations in the presence of the ethical interventions. we observe that without any fine-tuning, models can generate images of diverse groups with prompts containing ethical interventions. our preliminary study finds evidence that a large change in image generation can be caused by certain keyphrases such as 'irrespective of gender' in the context of the gender bias and 'culture' in the context of the cultural bias.