‘Person’ == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion (2023)

Abstract: We study stereotypes embedded within one of the most popular text-to-image generators: Stable Diffusion. We answer the question: what stereotypes of gender and nationality/continental identity does Stable Diffusion display in the absence of such information i.e. what gender and nationality/continental identity is assigned to ‘a person,’ or to ‘a person from Asia.’ Using CLIP-cosine similarity for zero-shot classification of images generated by CLIP-based Stable Diffusion v2.1 verified by manual examination, we chronicle results from 136 prompts (50 results/prompt) of front-facing images of faces from 6 different continents, 27 countries and 3 genders. We observe how Stable Diffusion results of ‘a person’ without any additional gender/nationality information correspond closest to images of men (avg. similarity 0.64) and least with persons of nonbinary gender (avg. similarity 0.41), and to persons from Europe/North America (avg. similarities 0.71 and 0.68, respectively) over Africa/Asia (avg. similarities 0.43 and 0.41, respectively), pointing towards Stable Diffusion having a concerning representation of personhood to be a European/North American man. We also show continental stereotypes and resultant harms e.g. a person from Oceania is deemed to be Australian/New Zealander (avg. similarities 0.77 and 0.74, respectively) over Papua New Guinean (avg. similarity 0.31), pointing to the erasure of Indigenous Oceanic peoples, who form a majority over descendants of colonizers both in Papua New Guinea and in Oceania overall. Finally, we unexpectedly observe a pattern of sexualization of women, specifically Latin American, Mexican, Indian and Egyptian women, confirmed through an NSFW detector and verified by manual examination. This demonstrates how Stable Diffusion perpetuates Western fetishization of women of color through objectification in media, which if left unchecked will worsen this stereotypical representation. All code and relevant data will be made publicly available.

## introduction content warning: the content of this paper may be upsetting or triggering. with the rapid advent of generative ai and research in the fields of natural language processing and computer vision coinciding with the exponential increase in availability and public uptake of text-to-image models such as stable diffusion, midjourney, and dall-e trained on large publicly-available data curated from the internet and performing comparably to each other @xcite , it is more important than ever to examine the embedded social stereotypes within such models. perpetuation of such stereotypes, such as light-skinned people being less 'threatening' than dark-skinned people @xcite or attractive persons being almost exclusively lightskinned @xcite , in the outputs of these models and downstream tasks can cause significant real-world harms, which researchers and ethicists have a moral responsibility to address. we study one of the most popular (with a dailyestimated 10 million users) open-source text-toimage generators today: stable diffusion as. we study stereotypes around two dimensions of human identity, gender and nationality, inquiring: rq1: what stereotypes of gender does stable diffusion display, in the absence of such information in prompts? rq2: what stereotypes of continental identity/nationality does stable diffusion display, in the absence of such information in prompts? using clip-cosine similarity to compare images generated by clip-based stable diffusion v2.1 in response to 136 english prompts verified by manual examination (datasets made publicly available), we build on previous work @xcite and make three contributions: ## results all presented images in this section are randomly chosen 2x2 grids from the stated prompts. ## implications and future work our findings have significant policy and legal implications, as the usage of stable diffusion becomes commonplace for commercial content creation purposes in the entertainment industry. as striking workers in the us from organizations such as the screen actors guild -american federation of television and radio artists (sag-aftra) demonstrate, models such as stable diffusion create problematic content drawing on internet data that they might not have the intellectual property rights to access, and content creating platforms prefer such cheap synthetic content over the work of real creators @xcite . careful consideration must be made for the application of such models in mainstream media, along with carefully curated guidelines that flag when content is ai-generated and accompanied by appropriate accreditation. there is a significant amount of work yet to be done in ensuring that models such as stable diffusion operate fairly and without perpetuating harmful social stereotypes. while the development of models such as fair diffusion @xcite , a text-to-image generator which seeks to introduce fairness and increase outcome impartiality as compared to stable diffusion outputs, and safe latent diffusion @xcite , a version which suppresses and removes nsfw and other inappropriate content, is promising, significant attention must also be paid to the datasets used to train such models and the various ways in which individual designers' choices might inject harmful biases into model results. a quick and informal analysis of safe latent diffusion with prompts that generated nsfw images in our dataset does produce images which score lower on the nsfw scale (prompts which return results that score 0.6-0.8 'sexy' from stable diffusion show results that score 0.4-0.5 'sexy' from safe latent diffusion), but given that safe latent diffusion is just stable diffusion with a safety filter, we cannot consider it a complete solution to these complex problems. ## conclusion in this paper, we examine stereotypical depictions of personhood within the text-to-image generator stable diffusion that is trained on data automatically collected from the internet. through a combination of pairwise clip-cosine similarity and manual examination of stable diffusion outputs across 138 unique prompts soliciting front-facing photos of people of different gender and continental/national identities, we demonstrate that the stereotypical depiction of personhood within stable diffusion outputs corresponds closely to western, light-skinned men and threatens to erase from media depictions historically marginalized groups such as people of nonbinary gender and indigenous people, among others. we also uncover a pattern of sexualization of women, mostly latin american but also mexican, egyptian, and indian women, within stable diffusion outputs, perpetuating the western stereotype of fetishizing women of color.