import os
import gradio as gr
from PIL import Image
_ID_CLUSTER_SCREEN_SHOTS = {
19: ("cluster_19_of_24_unmarked_white_unmarked_man.JPG", "Cluster 19 of 24"),
2: ("cluster_2_of_24_latinx_woman.JPG", "Cluster 2 of 24"),
18: ("cluster_18_of_24_hispanic_nonbinary.JPG", "Cluster 18 of 24"),
0: ("cluster_0_of_24_black_woman.JPG", "Cluster 0 of 24"),
5: ("cluster_5_of_24_white_unmarked_latinx_man.JPG", "Cluster 5 of 24"),
6: ("cluster_6_of_24_black_man.JPG", "Cluster 6 of 24"),
7: ("cluster_7_of_24_pacific_indigenous_man_nonbinary.JPG", "Cluster 7 of 24"),
3: (
"cluster_3_of_24_native_american_stereetotypical.JPG",
"Cluster 3 of 24",
),
8: (
"cluster_8_of_24_indigenous_nonbinary.JPG",
"Cluster 8 of 24",
),
23: (
"cluster_23_of_24_woman_nonbinbary_indigenous.JPG",
"Cluster 23 of 24",
),
12: (
"cluster_12_of_24_white_nonbinary.JPG",
"Cluster 12 of 24",
),
13: (
"cluster_13_of_24_nonbinary_woman_black.JPG",
"Cluster 13 of 24",
),
15: ("cluster_15_of_24_woman_white.JPG", "Cluster 15 of 24"),
}
def get_images(path):
images = [Image.open(os.path.join(path, im)) for im in os.listdir(path)]
paths = os.listdir(path)
return [(im, path) for im, path in zip(images, paths)]
def show_id_images(cl_id_1, cl_id_2, cl_id_3):
img_path_1, cluster_name_1 = _ID_CLUSTER_SCREEN_SHOTS[cl_id_1]
img_path_2, cluster_name_2 = _ID_CLUSTER_SCREEN_SHOTS[cl_id_2]
img_path_3, cluster_name_3 = _ID_CLUSTER_SCREEN_SHOTS[cl_id_3]
return (
gr.update(
value=os.path.join(impath, img_path_1),
label=f"Screenshot of the Identity Exploration tool for: {cluster_name_1}",
),
gr.update(
value=os.path.join(impath, img_path_2),
label=f"Screenshot of the Identity Exploration tool for: {cluster_name_2}",
),
gr.update(
value=os.path.join(impath, img_path_3),
label=f"Screenshot of the Identity Exploration tool for: {cluster_name_3}",
),
)
with gr.Blocks() as demo:
gr.Markdown(
"""
## Stable Bias: Analyzing Societal Representations in Diffusion Models
"""
)
gr.HTML(
"""
This is the demo page for the "Stable Bias" paper, which aims to explore and quantify social biases in text-to-image systems.
This work was done by Alexandra Sasha Luccioni (Hugging Face) , Christopher Akiki (ScaDS.AI, Leipzig University), Margaret Mitchell (Hugging Face) and Yacine Jernite (Hugging Face) .
"""
)
gr.HTML(
"""⚠️ DISCLAIMER: the images displayed by this tool were generated by text-to-image systems and may depict offensive stereotypes or contain explicit content."""
)
examples_path = "images/examples"
examples_gallery = gr.Gallery(
get_images(examples_path),
label="Example images generated by three text-to-image models (Dall-E 2, Stable Diffusion v1.4 and v.2).",
show_label=True,
elem_id="gallery",
).style(grid=[1, 6], height="auto")
gr.HTML(
"""
As AI-enabled Text-to-Image models are becoming increasingly used, characterizing the social biases they exhibit is a necessary first step to lowering their risk of discriminatory outcomes.
We compare three such models: Stable Diffusion v.1.4, Stable Diffusion v.2. , and Dall-E 2 , prompting them to produce images of different professions and identity characteristics .
You explore our findings in the sections below:
"""
)
gr.Markdown(
"""
### How do Diffusion Models Represent Identity?
One of the goals of our study was to look at the ways in which pictures generated by text-to-image models depict different notions of gender and ethnicity.
These concepts are inherently difficult to describe, however: gender and ethnicity are multi-dimensional, inter-related, and, most importantly, socially constructed:
they cannot (and should not) be predicted based on appearance features alone.
Since we are working with depictions of fictive humans when analyzing text-to-image model behaviors,
we cannot rely on self-identification either to assign identity categories to individual data points.
Instead, we develop a clustering-based method to identify relevant dataset-level trends in the generated images, as described in our companion
[Identity Representation Demo](https://huggingface.co/spaces/society-ethics/DiffusionFaceClustering).
You can read about how this tool helps us delineate **relevant regions** in the models' generation space, outline **stereotypical associations**,
and understand what **markedness** has to do with biases by expanding the accordion below:
"""
)
with gr.Accordion("How do Diffusion Models Represent Identity?", open=False):
gr.Markdown(
"""
Diffusion Models are commonly used in text-conditioned image generation systems, such as Stable Diffusion or Dall-E 2.
In those systems, a user writes a *"prompt"* as input, and receives an image that corresponds to what the prompt is describing as output.
For example, if the user asks for a *"Photo protrait of a **scientist**"*, they expect to get an image that looks photorealistic,
prominently features at least one person, and this person might be wearing a lab coat or safety goggles.
A *"Photo portrait of a **carpenter**"*, on the other other hand, might be set against a background depicting wooden scaffolding or a workshop (see pictures above).
At the start of this project, we found that while systems do make use of background and context cues to represent different professions,
there were also some concerning trends about the perceived genders and ethnicities of the people depicted in these professional situations.
After trying a few such prompts, we were left asking: why do all the people depicted in these pictures **look like white men**?
Why do the only exceptions appear to be fast food workers and other lower wage professions?
And finally, what could be the **consequences of such a lack of diversity** in the system outputs?
**Look like** is the operative phrase here: the people depicted in the pictures are synthetic and so do not belong to socially-constructed groups.
Consequently, since we cannot assign a gender or ethnicity label to each data point,
we instead focus on dataset-level trends in visual features that are correlated with social variation in the text prompts.
We do this through *controlled prompting* and *hierarchical clustering*: for each system,
we obtain a dataset of images corresponding to prompts of the format *"Photo portrait of a **(identity terms)** person at work"*,
where ***(identity terms)*** jointly enumerate phrases describing ethnicities and phrases denoting gender.
We then cluster these images by similarity and create an [Identity Representation Demo](https://hf.co/spaces/society-ethics/DiffusionFaceClustering)
to showcase the visual trends encoded in these clusters - as well as their relation to the social variables under consideration.
"""
)
impath = "images/identities"
with gr.Row():
with gr.Column(scale=1):
gr.Markdown(
"""
#### [Clusters, Gender, and Ethnicity](https://hf.co/spaces/society-ethics/DiffusionFaceClustering "Select cluster to visualize to the right or go straight to the interactive demo")
Our goal with this strategy is to trigger variation in the images that viewers will associate with social markers.
We use three phrases denoting genders (*man*, *woman*, *non-binary*), and 18 phrases describing ethnicities -
some of which are sometimes understood as similar in the US context, such as First Nations and Indigenous American.
We also leave the gender and ethnicity phrases unspecified in some prompts.
This approach has the advantage of placing visual features on a multidimensional spectrum without ascribing a prior number of distinct values for social categories or their intersection.
It is also limited by the training data of the models under consideration and the set of identity terms use,
which in our application are more relevant to the North American context than to other regions.
How then can we use these clusters in practice?
Let's look for example at **cluster 2 in the 24-cluster setting**:
we see that most prompts for images in the cluster use the word *woman* and one of the words denoting *Hispanic* origin.
This tells us that images that are similar to the ones in this cluster will **likely look like** Hispanic women to viewers.
You can cycle through [a few other examples right](https://hf.co/spaces/society-ethics/DiffusionFaceClustering "or even better, visualize them in the app"),
such as cluster 19 which mostly features the words *Caucasian* and *man*, different gender term distributions for *African American* in 0 and 6,
as well as clusters like 7 that showcase the limitations of mapping visual features to ethnicity categories
by grouping together *Pacific Islander*, *Indigenous American*, and *Latino*.
"""
)
with gr.Column(scale=1):
id_cl_id_1 = gr.Dropdown(
choices=[2, 19, 0, 6, 7],
value=2,
show_label=False,
)
identity_screenshot_1 = gr.Image(
value=os.path.join(impath, "cluster_2_of_24_latinx_woman.JPG"),
label="Screenshot of the Identity Exploration tool for: Cluster 2 of 24",
)
with gr.Row():
with gr.Column(scale=1):
id_cl_id_2 = gr.Dropdown(
choices=[3, 8, 23, 12, 13],
value=3,
show_label=False,
)
identity_screenshot_2 = gr.Image(
value=os.path.join(
impath, "cluster_3_of_24_native_american_stereetotypical.JPG"
),
label="Screenshot of the Identity Exploration tool for: Cluster 3 of 24",
)
with gr.Column(scale=1):
gr.Markdown(
"""
#### [Stereotypical Representations and Associations](https://hf.co/spaces/society-ethics/DiffusionFaceClustering "Select cluster to visualize to the left or go straight to the interactive demo")
Even before we start leveraging these clusters to analyze system behaviors in other applications,
they already provide useful insights into some of the bias dynamics in the system.
In particular, they help us understand which groups are more susceptible to representation harms when using these systems,
especially when the models are more likely to associate them with **stereotypical representations**.
Let's look at **clusters 3, 8, and 23** in the 24-cluster setting, as they all primarily feature images whose prompt contains a Native American identity term.
Cluster 3 is made up entirely of images of individuals wearing large traditional headdresses.
It is also the 4th largest cluster across all generations - which suggests that this one feature has a disproportionate importance in representations of these groups.
This behavior further brings to light well known representation issues in the models' training data.
It also showcases the necessity of modeling the diversity of how people map visual features to inferred social characteristics,
as the images in 8 and 23 are more likely to bear some similarity to depictions of people in contemporary professional settings.
With a different focus, **clusters 12, and 13** help us explore stereotypical representations of non-binary identities.
While both are made up of over 80% images for which the prompt mentions the word *non-binary*,
the similarities in cluster 12 are particularly flagrant upon visual inspection; with all depictions featuring similar glasses and similar haircuts -
neither of which should be a strong indicator of gender.
"""
)
with gr.Row():
with gr.Column(scale=1):
gr.Markdown(
"""
#### [Specification, Markedness, and Bias](https://hf.co/spaces/society-ethics/DiffusionFaceClustering "Select cluster to visualize to the right or go straight to the interactive demo")
The second phenomenon we study through the lens of our clustering is that of **markedness**,
or *default behavior* of the models when neither gender nor ethnicity is specified.
This corresponds to asking the question: "If I **don't explicitly** tell my model to generate people with diverse identity characteristics,
will I still see diverse outcomes?"
Unsurprisingly, we find that not to be the case.
The clusters with the most examples of prompts with unspecified gender and unspecified ethnicity terms are **clusters 5 and 19**,
and both are also strongly associated with the words *man*, *White*, and *Causian*.
This association holds across genders (as showcased by **cluster 15**, which has a majority of *woman* and *White* prompts along with unspecified ethnicity)
and across ethnicities (comparing the proportions of unspecified genders in **clusters 0 and 6**: 18 % and 38% for the clusters with more *aoman* and more *man* respectively along with the *African American* phrases).
This provides the beginning of an answer to our motivating question: since users rarely specify an explicit gender or ethnicity when using
these systems to generate images of people, the high likelihood of defaulting to *Whiteness* and *masculinity* is likely to at least partially explain the observed lack of diversity in the outputs.
We compare these behaviors across systems and professions in the next section.
"""
)
with gr.Column(scale=1):
id_cl_id_3 = gr.Dropdown(
choices=[5, 19, 15, 0, 6],
value=6,
show_label=False,
)
identity_screenshot_3 = gr.Image(
value=os.path.join(
impath, "cluster_19_of_24_unmarked_white_unmarked_man.JPG"
),
label="Screenshot of the Identity Exploration tool for: Cluster 19 of 24",
)
for var in [id_cl_id_1, id_cl_id_2, id_cl_id_3]:
var.change(
show_id_images,
inputs=[id_cl_id_1, id_cl_id_2, id_cl_id_3],
outputs=[
identity_screenshot_1,
identity_screenshot_2,
identity_screenshot_3,
],
)
gr.Markdown(
"""
### Quantifying Social Biases in Image Generations of Professions
Machine Learning models encode and amplify biases that are represented in the data that they are trained on -
this can include, for instance, stereotypes around the appearances of members of different professions.
In our study, we prompted the 3 text-to-image models with texts pertaining to 150 different professions
and analyzed the presence of different identity groups in the images generated. We found evidence of many societal stereotypes in the images generated,
such as the fact that people in positions of power (e.g. director, CEO) are often White- and male-appearing,
while the images generated for other professions are more diverse.
Read more about our findings in the accordion below or directly via the [Diffusion Cluster Explorer](https://hf.co/spaces/society-ethics/DiffusionClustering) tool.
"""
)
with gr.Accordion("Quantifying Social Biases in Image Generations of Professions", open=False):
gr.Markdown(
"""
We also explore the correlations between the professions that use used in our prompts and the different identity clusters that we identified.
Using both the [Diffusion Cluster Explorer](https://hf.co/spaces/society-ethics/DiffusionClustering)
and the [Identity Representation Demo](https://hf.co/spaces/society-ethics/DiffusionFaceClustering),
we can see which clusters are most correlated with each profession and what identities are in these clusters.
"""
)
with gr.Row():
with gr.Column():
gr.HTML(
"""
Using the Diffusion Cluster Explorer, we can see that the top cluster for the CEO and director professions is Cluster 4:
"""
)
with gr.Column():
ceo_img = gr.Image(
Image.open("images/bias/ceo_dir.png"),
label="CEO Image",
show_label=False,
)
with gr.Row():
with gr.Column():
gr.HTML(
"""
Going back to the Identity Representation Demo , we can see that the most represented gender term is man (56% of the cluster) and White (29% of the cluster).
This is consistent with common stereotypes regarding people in positions of power, who are predominantly male, according to the US Labor Bureau Statistics.
"""
)
with gr.Column():
cluster4 = gr.Image(
Image.open("images/bias/Cluster4.png"),
label="Cluster 4 Image",
show_label=False,
)
with gr.Row():
with gr.Column():
gr.HTML(
"""
If we look at the cluster representation of professions such as social assistant and social worker, we can observe that the former is best represented by Cluster 2, whereas the latter has a more uniform representation across multiple clusters:
"""
)
with gr.Column():
social_img = gr.Image(
Image.open("images/bias/social.png"),
label="social image",
show_label=False,
)
with gr.Row():
with gr.Column(scale=1):
gr.HTML(
"""
Cluster 2 is best represented by the gender term is woman (81%) as well as Latinx (19%)
This gender proportion is exactly the same as the one provided by the United States Labor Bureau (which you can see in the table above), with 81% of social assistants identifying as women.
"""
)
with gr.Column(scale=2):
cluster4 = gr.Image(
Image.open("images/bias/Cluster2.png"),
label="Cluster 2 Image",
show_label=False,
)
gr.Markdown(
"""
### Comparing Model Generations
"""
)
gr.Markdown(
"""
Above and beyond quantitative analyses, one of the main goals of our project was to create accessible ways for the users to explore the generated images themselves, based on their own interests. For this purpose, we created two interactive tools: the [Diffusion Bias Explorer](https://huggingface.co/spaces/society-ethics/DiffusionBiasExplorer), which can be used to compare two models and the images they generate for a given profession or for a given model across two professions, and the [Average Diffusion Faces Tool](https://huggingface.co/spaces/society-ethics/Average_diffusion_faces), which shows an 'average' representation of faces across professions, based on the images generated by the 3 models.
"""
)
with gr.Accordion("Comparing Model Generations", open=False):
gr.HTML(
"""
One of the goals of our study was allowing users to compare model generations across professions in an open-ended way, uncovering patterns and trends on their own. This is why we created the Diffusion Bias Explorer and the Average Diffusion Faces tools. We show some of their functionalities below:
"""
)
with gr.Row():
with gr.Column():
explorerpath = "images/biasexplorer"
biasexplorer_gallery = gr.Gallery(
get_images(explorerpath),
label="Bias explorer images",
show_label=False,
elem_id="gallery",
).style(grid=[2, 2])
with gr.Column():
gr.HTML(
"""
Comparing generations both between two models and within a single model can help uncover trends and patterns that are hard to measure using quantitative approaches.
For instance, we can observe that both Dall-E 2 and Stable Diffusion 2 represent both CEOs and nurses as homogenous groups with distinct characteristics, such as ties and scrubs (which makes sense given the results of our clustering, shown above.
We can also see that the images of waitresses generated by Dall-E 2 and Stable Diffusion v.1.4. have different characteristics, both in terms of their clothes as well as their appearance.
It's also possible to see harder to describe phenomena, like the fact that portraits of painters often look like paintings themselves.
We encourage you to use the Diffusion Bias Explorer tool to explore these phenomena further!
"""
)
with gr.Row():
with gr.Column():
averagepath = "images/averagefaces"
average_gallery = gr.Gallery(
get_images(averagepath),
label="Average Face images",
show_label=False,
elem_id="gallery",
).style(grid=[1, 3], height=560)
with gr.Column():
gr.HTML(
"""
Looking at the average faces for a given profession across multiple models can help see the dominant characteristics of that profession, as well as how much variation there is (based on how fuzzy the image is).
In the images shown here, we can see that representations of these professions significantly differ across the three models, while sharing common characteristics, e.g. postal workers all wear caps.
Also, the average faces of hairdressers seem more fuzzy than the other professions, indicating a higher diversity among the generations compared to other professions.
Look at the Average Diffusion Faces tool for more examples!
"""
)
gr.Markdown(
"""
### Exploring the Pixel Space of Generated Images
"""
)
gr.Markdown(
"""
Finally, an interesting aspect of the generations of the 3 models are the images themselves, which can be analyzed from different angles on a pixel-level. We explore the images in terms of their colorfulness using the [Colorfulness Profession Explorer](https://huggingface.co/spaces/tti-bias/identities-colorfulness-knn) and the [Colorfulness Identities Explorer](https://huggingface.co/spaces/tti-bias/professions-colorfulness-knn), which allow users to hone in on patterns in terms of colors and shades within the images generated. We also allow exploration of the images in terms of their visual features using the bag-of-visual-words approach (BoVW), which allows users to hone in on visual stereotypical content such as professions that have uniforms of a given color, of elements like glasses and hair styles -- this can be done via the [BoVW Nearest Neighbors Explorer](https://huggingface.co/spaces/tti-bias/identities-bovw-knn) and the [BoVW Professions Explorer](https://huggingface.co/spaces/tti-bias/professions-bovw-knn) -- we also present some of our salient findings in the accordion below.
"""
)
with gr.Accordion("Exploring the Pixel Space of Generated Images", open=False):
gr.HTML(
"""
With thousands of generated images, we found it useful to provide ways to explore the data in a structured way that did not depend on any external dataset or model. We provide two such tools, one based on colorfulness and one based on a bag-of-visual words model computed using SIFT features.
"""
)
with gr.Row():
gr.HTML(
"""
Colorfulness
We compute an image's "colorfulness" following this work by David Hasler and Sabine E. Suesstrunk and allow the user to choose a specific prompt and model and explore the neighborhood of that chosen starting point. One interesting orthogonal insight is that images generated by DALL·E 2 are on average the most colorful. Images of men are on average less colorful than all other gender labels, consistently across all three models. Patterns revealed using this explorer include for example the exoticizing depiction of native Americans as can be seen in the very stereotypical gallery of images generated in the example on the right.
"""
)
gr.Image("images/colorfulness/nativeamerican_man.png")
with gr.Row():
gr.HTML(
"""
Bag of Visual Words
Another way of providing the means for a structured traversal of the dataset is a nearest-neighbor explorer based on visual features provided by an image's SIFT features, which we quantize into a visual vocabulary to represent the entire image dataset as a TF-IDF matrix. These tools are especially useful in honing in on stereotypical content that is often encoded visually, but also failure modes of the model such as the misinterpetation of the "stocker" profession as an imagined dog-breed. The screenshot to the right shows how SIFT visual patterns tend to cluster together, namely in this instance the booksheves in the background, or the gibberish pseudo-English text that often plagues TTI systems.
"""
)
with gr.Column():
gr.Image("images/bovw/bookshelves.png")
gr.Image("images/bovw/gibberish.png")
gr.Markdown(
"""
### All of the tools created as part of this project:
"""
)
gr.HTML(
"""
Average Diffusion Faces
Diffusion Bias Explorer
Diffusion Cluster Explorer
Identity Representation Demo
BoVW Nearest Neighbors Explorer
BoVW Professions Explorer
Colorfulness Profession Explorer
Colorfulness Identities Explorer
"""
)
# gr.Interface.load("spaces/society-ethics/DiffusionBiasExplorer")
demo.launch(debug=True)