New Job, New Gender? Measuring the Social Bias in Image Generation Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 06:15:30

Abstract: Image generation models can generate or edit images from a given text. Recent advancements in image generation technology, exemplified by DALL-E and Midjourney, have been groundbreaking. These advanced models, despite their impressive capabilities, are often trained on massive Internet datasets, making them susceptible to generating content that perpetuates social stereotypes and biases, which can lead to severe consequences. Prior research on assessing bias within image generation models suffers from several shortcomings, including limited accuracy, reliance on extensive human labor, and lack of comprehensive analysis. In this paper, we propose BiasPainter, a novel evaluation framework that can accurately, automatically and comprehensively trigger social bias in image generation models. BiasPainter uses a diverse range of seed images of individuals and prompts the image generation models to edit these images using gender, race, and age-neutral queries. These queries span 62 professions, 39 activities, 57 types of objects, and 70 personality traits. The framework then compares the edited images to the original seed images, focusing on the significant changes related to gender, race, and age. BiasPainter adopts a key insight that these characteristics should not be modified when subjected to neutral prompts. Built upon this design, BiasPainter can trigger the social bias and evaluate the fairness of image generation models. 
We use BiasPainter to evaluate six widely-used image generation models, such as stable diffusion and Midjourney. Experimental results show that BiasPainter can successfully trigger social bias in image generation models. According to our human evaluation, BiasPainter can achieve 90.8\% accuracy on automatic bias detection, which is significantly higher than the results reported in previous work.  All the code, data, and experimental results will be released to facilitate future research.

Main paper: 
 introduction: 
 Image generation models, which generate images from a given text, have recently drawn lots of interest from academia and the industry. For example, Stable Diffusion [21] , an open-sourced latent text-toimage diffusion model, has 65K stars on github 1 . And Midjourney, an AI image generation commercial software product launched on July 2022, has more than 15 million users [8] . These models are capable of producing high-quality images that depict a variety of concepts and styles when conditioned on the textual description and can significantly facilitate content creation and publication. Despite the extraordinary capability of generating various vivid images, image generation models are prone to generate content with social bias, stereotypes, and even hate. For example, Google's image generator, the Gemini, had generated a large number of images that were biased and contrary to historical facts, causing the service to be taken offline on an emergency basis [18] . Previous work has also found that text-to-image generation models tend to associate males with software engineers, females with housekeepers, and white people with attractive people [3] . It is because these models have been trained on massive datasets of images and text scraped from the web, which is known to contain stereotyped, biased, and toxic contents [13] . The ramifications of such biased content are far-reaching, from reinforcing stereotypes to causing brand and reputation damage and even impacting individual well-being.To mitigate the social bias and stereotypes in image generation models, an essential step is to trigger the biased content and evaluate the fairness of such models. Previous works have designed methods to evaluate the bias in image generation models. For example, [7] generates images from a set of words that should not be related to a specific gender or race (e.g., secretary, rich person). Then, with a pre-trained image-text alignment model named CLIP [20] , the authors classify the generated images into gender and race categories. However, these works suffer from several drawbacks. First, their accuracy is relatively low. Detecting race and gender is not an easy task, considering the high diversity of generating style and content. According to their human evaluation results, the detecting method is not accurate (e.g., [7] only achieves 40% on race), leading to concern about the effectiveness and soundness of the method. Second, their scope is limited. Previous work [3] involves human annotation to evaluate the bias in generated images, aiming for an accurate evaluation. However, this manual method needs extensive human effort and is not scalable. Third, there is a lack of comprehensive evaluation across various demographic groups. According to a previous study [7] , more than 90% of the images produced by image generation models are of white people, which implies the comprehensive evaluation of the bias in other groups, such as East Asian people and black people.In this paper, we design a novel evaluation framework, Bias-Painter, that can automatically, accurately and comprehensively measure the social bias in image generation models. In particular, BiasPainter operates by inputting both photos of people and text (prompts) into these models and then analyzing how the models edit these photos. A high-level idea is when prompted with gender, race, and age-neutral prompts, the gender, race, and age of the human in the photo should not significantly alter after editing. Specifically, BiasPainter first collects photos of people across different races, genders, and ages as seed images. Then, it prompts the model to edit each seed image. The prompts are selected from a pre-defined comprehensive gender/racial/age-neutral prompt list covering professions, objects, personalities and activities. After that, BiasPainter measures the changes from the seed image to the generated image according to race, gender, and age. An ideal case is that race, gender, and age do not change significantly under the editing with a gender, racial, and age-neutral prompt. On the other hand, if a model is prone to change significantly and consistently (e.g., increasing the age of the person in the original image) under a specific prompt (e.g., "a photo of a mean person"), BiasPainter detects a biased association(e.g., between elder and mean).To evaluate the effectiveness of BiasPainter, we conduct experiments on six widely deployed image generation models: stablediffusion 1.5, stable-diffusion 2.1, stable-diffusion XL, Dall-E 2, Midjourney and InstructPix2Pix. We sample three photos from each combination of gender, race, and age, ending up with 54 seed images, and adopt 228 prompts to edit each seed image. For each image generation model, we generate 54*228=12312 images and use the (original image, generated image) pairs as test cases to evaluate the bias. The results show that BiasPainter can successfully trigger social bias in image generation models. In addition, based on human evaluation, BiasPainter can achieve an accuracy of 90.8% in identifying the bias in images, which is significantly higher than the performance reported in the previous work (40% on race) [7] . Furthermore, BiasPainter can offer valuable insights into the nature and extent of biases within these models and serves as a tool for evaluating bias mitigation strategies, aiding developers in improving model fairness.We summarize the main contributions of this work as follows:• We design and implement BiasPainter, A novel evaluation framework for comprehensively measuring the social biases in image generation models. • We perform an extensive evaluation of BiasPainter on six widely deployed commercial conversation systems and research models. The results demonstrate that BiasPainter can effectively trigger a massive amount of biased behavior. • We release the dataset, the code of BiasPainter, and all experimental results, which can facilitate real-world fairness evaluation tasks and further follow-up research.Content Warning: We apologize that this article presents examples of biased images to demonstrate the results of our method. 
background 2.1 image generation models: 
 Image generation models, also known as Text-to-Image Generative Models, aim to synthetic images given natural language descriptions. There is a long history of image generation. For example, Generative Adversarial Networks [11] and Variational Autoencoders [29] , are two famous models that have been shown excellent capabilities of understanding both natural languages and visual concepts and generating high-quality images. Recently, diffusion models, such as DALL-E2 , Imagen3 and Stable Diffusion [22] , have gained a huge amount of attention due to their significant improvements in generating high-quality vivid images. Despite the aforementioned work's aim to improve the quality of image generation, such generative models are reported to contain social biases and stereotypes [3] .Most of the currently used image generation models provide two manners of generating images. The first is generating images based on natural language descriptions only. The second manner is adopting an image editing manner that enables the user to input an image and then edit the image based on natural language descriptions. The former manner has more freedom while the latter one is more controllable.Bias in AI models has been a known risk for decades [4] . As one of the most notorious biases, social bias is the discrimination for, or against, a person or group, compared with others, in a way that is prejudicial or unfair [32] . To study the social bias in the machine learning models, the definitions of bias and fairness play a crucial role. Researchers and practitioners have proposed and explored various fairness definitions [6] . The most widely used definition is statistical parity which requires the probability of a favorable outcome to be the same among different demographic groups. For example, in the case of job application datasets, "receive the job offer" is favorable, and according to the "gender" attribute, people can be categorized into different groups, like "male" and "female". If "male" and "female" are treated similarly to "receive the job offer", the AI model is fair.This definition of bias and fairness is widely adopted in classification and regression tasks, where the favorable labels can be clearly assigned and the probabilities can be easily measured. However, this setting cannot be easily adopted for image generation models, since labels and probability are hard to measure for such models.As one of the most important applications of AI techniques trained on massive Internet data, image generation models can inevitably be biased. Since such models are widely deployed in people's daily lives, biased content generated by these systems, especially those related to social bias, may cause severe consequences. Social biased content is not only uncomfortable for certain groups but also can lead to a bad social atmosphere and even aggravate social conflicts. As such, exposing and measuring the bias in image generation models is a critical task.It is worth noting that the biased generation may align with the bias in reality. For example, using the prompt "a picture of a lawyer" may generate more male lawyers than female lawyers, which may be in line with the male-female ratio for lawyers in real life [9] . However, such imbalanced generations are still not favorable. On the one hand, the imbalance in reality could be due to real-world unfairness, e.g., the opportunities to receive a good education or job offer may not be equal for males and females. Since such a biased ratio is not favorable, the generative AI software that mimics such a biased ratio is also not favorable. On the other hand, there is also a chance that AI models could generate a more biased ratio [1] . Such imbalanced generations may reinforce the bias or stereotypes. Thus, in our work, we design and implement BiasPainter that can measure any biased generation given a neutral prompt. 
approach and implementation: 
 In this section, we present BiasPainter, our evaluation framework for measuring the social bias in image generation models. Bias-Painter uses photos of different persons as seed images and adopts various prompts to let image generation models edit the seed images. The key insight is that the gender/race/age of the person in the photo should not be modified too much under the gender/race/ageneutral prompts. Otherwise, a spurious correlation between gender/race/age and other properties exists in the model. Namely, a suspicious bias is detected. For example, if an image generation model tends to convert more female photos to male photos under the prompt "a photo of a lawyer", a bias about lawyers on gender is detected. Figure 2 depicts the framework of BiasPainter, which consists of five stages:(1) Seed Image Collection: collect photos of people across different races, genders and ages as seed images. The first step of BiasPainter is collecting diverse photos of different people across various races, genders and ages. To limit the scope of experiments, we only consider 3 kinds of races (white people, black people, and East Asian people), two genders (male and female), and three ages (young adult, middle age, and elderly) to conduct experiments. Specifically, we utilize a public-available dataset, VG-GFace2 [5] , and select 3 photos from each combination of gender, race and age, ending up with 3*2*3*3=54 seed images. Each image was manually selected and inspected to ensure the race, gender and age attributes aligned with the people within the image. We show the examples in the supplementary material.BiasPainter adopts different prompts, such as "a photo of a lawyer", to edit the seed images. To achieve this, BiasPainter first collects diverse and comprehensive word lists, on the following four domains: profession (e.g. scientist), personality (e.g. rude), objects (e.g. gun), and activities (e.g. crying). These words were collected from varied resources. The profession-related words were collected from the U.S. Bureau of Labor Statistics4 , and the words related to personality, objects and activities were collected from related dictionaries 567 . Since BiasPainter only adopts race-, gender-, and age-neutral prompts to edit the seed images, we then manually filter out the words that are race-, gender-, and age-related, such as actor/actress and waiter/waitress. Note that race-neutral means that people of any race can relate to the prompts, such as being a scientist, being rude, with a gun, and crying. Specifically, we recruited 10 annotators, providing them with prompts we previously collected, and asked them to measure the words' relevance to race, gender, or age on a scale from 1 (Strongly Irrelevant) to 5 (Strongly Relevant). We filter out the words with average relevancy higger than 3. Finally, we adopt four templates to generate prompts according to the domain. For example, BiasPainter adopts a person who is [crying] for the activity domain and a person with a [book] for the object domain. Table 1 shows the details of the final prompt lists, consisting of 228 prompts.As introduced in Section 3.1 and 3. BiasPainter adopts the (seed image, generated image) pairs to evaluate the social bias. For each seed image and the generated image, BiasPainter first adopts techniques to evaluate their properties according to race, gender, and age.Race Assessment. We follow [7] to analyse the race according to the skin color. Researchers presented a division into six groups based on color adjectives: White (Caucasian), Dusky (South Asian), Orange (Austronesian), Yellow (East Asian), Red (Indigenous American), and Black (African) [26] . It is challenging to distinguish the race of a human accurately, considering the number of the race and The landmark provides the position and the shape of the face, as well as the eyes and mouth in the photo. Then, BiasPainter adopts a rule-based method to remove the background, eyes and mouth from the face. Finally, BiasPainter calculates the average pixel value of the remaining face. The darker the digital image is, the higher the average pixel value is. The more difference between the average pixel value of the seed image and the generated image, the more possibility that the race is changed by the image generation model. Gender Assessment. While there is a broad spectrum of genders [14] , it's difficult to accurately identify someone's gender across this broad spectrum based solely on visual cues. Consequently, following previous works [2, 7, 33] , we restrict our bias measurement to a binary gender framework and only consider male and female. BiasPainter adopts a commercial face analyses API, named Face++ Cognitive Service 9 , to identify the gender information of the human's picture. Specifically, Face++ Cognitive Service returns a predicted gender to indicate the gender of the people in the picture, which will be adopted by BiasPainter to access the gender. If there is a difference between the gender attribute of the seed image 8 http://dlib.net/ 9 https://www.faceplusplus.com/ and the generated image, then the image generation model changes the gender in the generated image.Age Assessment. BiasPainter adopts a commercial face analyses API, named Face++ Cognitive Service, to identify the age information of the human's picture. Specifically, Face++ Cognitive Service returns a predicted age to indicate the ages of the people in the picture, which will be adopted by BiasPainter to access the age. The more differences between the predicted ages of the seed image and the generated image, the more possibility that the age is changed by the image generation model.In this section, we first illustrate how BiasPainter evaluates the bias in generated images. Then we introduce how BiasPainter evaluates the bias for each prompt word. Finally, we show how BiasPainter evaluates the model fairness.BiasPainter adopts the following idea to measure the social bias in image generation models. The generated image and the seed image should have a similar gender/race/age property given a gender/racial/age-neutral prompt. Any (seed image, generated image) pair that is detected to have significant differences in race, gender or age will be collected as a suspiciously biased image.Specifically, given a (seed image, generated image) pair, Bias-Painter calculates three scores, gender bias score, age bias score, and race bias score to assess and quantify biases in generated images.Gender Bias Score Calculation. The gender bias score is determined based on the gender of the individuals present in both the seed and generated images. If the genders match, the gender bias score is zero. However, if the genders change from male to female, a biased score of +1 is assigned to the image. If the genders change from female to male, a biased score of -1 is assigned.Formally, for each (seed image, generate image) pair, the gender score is calculated as follows: 𝑖𝑚𝑎𝑔𝑒_𝑔𝑒𝑛𝑑𝑒𝑟 _𝑠𝑐𝑜𝑟𝑒 =          1, input=male, output=female -1,input=female, output=male 0, otherwise.Age Bias Score Calculation The age bias score is determined by analyzing the difference in ages between the generated images and the seed images. 0 means no difference in age between the generated image and the seed image. A positive age bias score means the person in the generated image is older than the seed image while a negative age bias score indicates the person in the generated image is younger. Different from gender which is a binary classification, the difference in age can vary, such as from +100 to -100, and similar to gender bias score, we want an age bias score of +1 and -1 to indicate a clear change in age. Hence, we divided the difference in age by 25 as a threshold to approximate the average difference between young people to middle-aged people, as well as the average difference between middle-aged people to old people. We also conducted an ablation study on different thresholds, which can be found in the supplementary material.Formally, for each (seed image, generate image) pair, the age bias score is calculated as follows: image_age_score = 𝑜𝑢𝑡𝑝𝑢𝑡_𝑎𝑔𝑒 -𝑖𝑛𝑝𝑢𝑡_𝑎𝑔𝑒 age_threshold Race Bias Score Calculation The race bias score is computed by analyzing the differences in the average grayscale values of the skin tones between the generated image and the seed image. 0 means no difference in skin tone between the generated image and the seed image. A positive race bias score means the skin tone in the generated image is lighter than the seed image while a negative age bias score indicates the skin tone in the generated image is darker. Again, different from gender which is a binary classification, the difference in skin tone can vary, and similar to the gender bias score, we want a race bias score of +1 and -1 to indicate a clear change in race. We calculate the average difference of grayscale values between white people and black people in the seed images, which is 30. Hence, we divided the difference in grayscale values by 20 as a threshold to approximate a clear change in skin tone. We also conducted an ablation study on different thresholds, which can be found in the supplementary material.Formally, for each (seed image, generate image) pair, the race bias score is calculated as follows: image_race_score = 𝑜𝑢𝑡𝑝𝑢𝑡_𝑔𝑟𝑎𝑦_𝑠𝑐𝑎𝑙𝑒 -𝑖𝑛𝑝𝑢𝑡_𝑔𝑟𝑎𝑦_𝑠𝑐𝑎𝑙𝑒 race_threshold 3.5.2 Word Bias Evaluation. After evaluating the bias in each generated image, BiasPainter can measure the bias for each prompt word under the following key insight: if an image generation model tends to modify the seed images on age/gender/race given a specific prompt, the keyword in the prompt is biased according to age/gender/race for this model. For example, if a model tends to convert more females to males under the prompt "a person of a lawyer", the word "lawyer" is more biased to gender "male". Bia-sPainter calculates the average of image bias score as word bias scores to represent the bias in prompt word.Formally, the word bias score is obtained by summing up the image bias scores of all images and dividing by the total number of output images N. X can be the gender, age, and race. word_X_score = 𝑁 𝑖=0 𝑖𝑚𝑎𝑔𝑒_𝑋 _𝑠𝑐𝑜𝑟𝑒 𝑖 𝑁 3.5.3 Model Bias Evaluation. Based on the word bias scores, Bias-Painter can evaluate and quantify the overall fairness of each image generation model. The key insight is that the more biased words with higher word bias scores found for a model, the more biased the model is.Formally, the model bias score is obtained by summing up the absolute value of all word bias scores, divided by the total number of prompt words M. X can be the gender, age, and race.  
evaluation: 
 To validate the effectiveness of BiasPainter and get more insights on the bias in image generation models, we use BiasPainter to evaluate 6 image generation models. In this section, we detail the evaluation process and empirically explore the following three research questions (RQs).• RQ1: Can BiasPainter effectively measure social bias in image generation models? • RQ2: Are the social bias found by BiasPainter valid? • RQ3: Can BiasPainter help mitigate the bias in image generation models?Evaluated Models. We use BiasPainter to evaluate 6 widely used image generation models.Stable Diffusion is a commercial deep-learning text-to-image model based on diffusion techniques [22] developed by Stability AI. It has different released versions and we select the three latest released versions for evaluation. For stable-diffusion 1.5 and stable-diffusion 2.1, we use the officially released models from Hug-gingFace 10 . Both models are deployed on Google Colaboratory with the Jupyter Notebook maintained by TheLastBen 11 , which is based on the stable-diffusion web UI 12 . For stable-diffusion XL, we use the official API provided by Stability AI. We follow the example code in Stability AI's documentation 13 and adopt the default hyper-parameters provided by Stability AI.Midjourney is a commercial generative artificial intelligence service provided by Midjourney, Inc. Since Midjourney, Inc. does not provide the official API, we adopt a third-party calling method provided by yokonsan 14 to automatically send requests to the Midjourney server. We follow the default hyper-parameters on the Midjourney's official website when generating images. DALL-E is a commercial system developed by OpenAI that can create realistic images from a description in natural language. DALL-E 2 supports the functionality of editing an image based on image and text input. We adopted OpenAI's API and sample code15 in our implementation.InstructPix2Pix is a state-of-the-art research model for editing images from human instructions. We use the official replicate production-ready API 16 and follow the default hyper-parameters provided in the document.Test Cases Generation. To ensure a comprehensive evaluation of the bias in the image generation models, we adopt 54 seed images across various combinations of ages, genders, and races. For each image, we adopt 62 prompts for the professions, 57 prompts for the objects, 70 prompts for the personality, and 39 prompts for the activities. Finally, we generate 12312 (seed image, prompt) pairs as test cases for each model.In this RQ, we investigate whether BiasPainter can effectively trigger and measure the social bias in image generation models.Image Bias. We input the (seed image, prompt) pairs and let image generation models edit the seed image under the prompts. Then, we use the (seed image, generated image) pairs to evaluate the bias in the generated images. In particular, we adopt BiasPainter to calculate the image bias scores and we find a large number of generated images that are highly biased. We show some examples in Figure 1 . Word Bias. We adopt BiasPainter to calculate the word bias score for each prompt based on image bias scores. For each model and each domain, we list the top three prompt words that are highly biased according to gender, age and race, respectively, in Table 2 (the full table is shown in the supplementary material). BiasPainter can provide insights on what biases a model has, and to what extent. For example, as for the bias of personality words on gender, words like brave, loyal, patient, friendly, brave and sympathetic tend to convert male to female, while words like arrogant, selfish, clumsy, grumpy and rude tend to convert female to male. And for the profession, words like secretary, nurse, cleaner, and receptionist tend to convert male to female, while entrepreneur, CEO, lawyer and president tend to convert female to male. For activity, words like cooking, knitting, washing and sewing tend to convert male to female, while words like fighting, thinking and drinking tend to convert female to male.In addition, BiasPainter can visualize the distribution of the word bias score for all the prompt words. For example, we use BiasPainter to visualize the distribution of word bias scores on the profession in stable diffusion 1.5. As is shown in Figure 4 , the model is more biased to younger rather than older, and more biased to lighter skin tone rather than darker skin tone.Model Bias. BiasPainter can also calculate the model bias scores to evaluate the fairness of each image generation model. Table 3 shows the results, where we can find that different models are biased at different levels and on different domains. In general, the Stable Diffusion Models are more biased compared with the other three models. Stable-Diffusion 2.1 is the most biased model on age and Pix2pix shows less bias on age and gender.To ensure that the social biases detected by BiasPainter are truly biased, we perform a manual inspection of the bias identification process. In particular, we recruited 5 annotators, both have a bachelor's degree and are proficient in English, to annotate the (seed image, generated image) pairs.For age, we randomly select 25, 25, and 50 (seed image, generated image) pairs that are identified as becoming older (image age bias score > 1), becoming younger (image age bias score < -1), and no significant change on age (0.2 > image age bias score > -0.2), respectively, by BiasPainter. For each pair, annotators are asked a multiple-choice question: A. person 2 is older than person 1; B. person 2 is younger than person 1; C. There is no significant difference between the age of person 2 and person 1.For gender, we randomly select 25, 25, and 50 (seed image, generated image) pairs that are identified as female to male (image gender bias score = -1), male to female (image gender bias score = 1), and no change on race (image gender bias score = 0), respectively, by BiasPainter. For each pair, annotators are asked a multiple-choice question: A. person 1 is male and person 2 is male; B. person 1 is male and person 2 is female; C. person 1 is female and person 2 is male; D. person 1 is female and person 2 is female.For race, we randomly select 25, 25, and 50 (seed image, generated image) pairs that are identified as becoming lighter (image race bias score > 1), becoming darker (image race bias score < -1), and no significant change on skin tone (0.2 > image race bias score > -0.2), Annotations are done separately and then they discuss the results and resolve differences to obtain a consensus version of the annotation. By comparing the identification results from BiasPainter with annotated results from the annotators, we calculate the accuracy of BiasPainter. BiasPainter achieves an accuracy of 90.8%, indicating that the bias identification results are reliable.The next step in measuring the social bias in image generation models is mitigating the bias. So the following question is: can BiasPainter be helpful to mitigate the bias in image generation models? In this section, we illustrate that BiasPainter can be used for bias mitigation by either providing insights and direction or being an automatic evaluation method.Previous studies have proposed various methods to mitigate the bias in AI systems, which can be categorized into the method before training, such as balancing the training data, during training, such as adding regularization terms in training objective function, and after training, such as prompt design [12] . We believe BiasPainter can provide useful insights into what an image generation model is biased, which can be adopted to design more balanced training data or more efficient regularization. For example, based on the finding that nurses are more biased toward females, developers can add more training data about male nurses. Besides, BiasPainter can be used as an automatic evaluation method to measure the effectiveness of different bias mitigation methods, which can be useful for bias mitigation studies. Since most of the image generation models only provide API service without providing the training data or model parameters, in this section, we adopt BiasPainter to evaluate the effectiveness of the prompt design.Specifically, we select the top biased profession words in Table 2 , add an additional system prompt that "maintains the same gender/race/age as the input image" and then regenerate the images. Then, we compare the bias score when generating with the original prompt (denoted as "Ori") to the bias score when generating with the additional system prompt (denoted as "Miti"). As the results are shown in Table 4 , the average bias score when generating with the additional prompt is relatively smaller (e.g. 0.40 v.s. 0.98 for SD1.5), indicating that adding this specific prompt can reduce the bias to a certain extent, but is far from completely eliminating it. 
related work: 
 Bias and fairness have gained significant attention in the AI community from various perspectives, such as bias measurment [10, 17] [12, 27] , and for various kinds of AI models, such as nature language processing models [15, 25, 28] , recommendation systems [16, 19] , chatbot [30] , and vision-language pertaining models [23, 24, 34] .As one of the most popular AI models, the image generation model is widely used with a sufficient amount of active users. We systematically reviewed papers on evaluating the biases in image generation models across related research areas. [3] is an early work that conducts an empirical study to show the stereotypes learned by text-to-image models. They design different prompts as input and use human annotators to find biased images, without proposing an automatic framework that can trigger the social bias. Inspired by this, [7] proposed an automatic framework to evaluate the bias in image generation models. However, their automatic evaluation method failed to accurately detect the bias according to their human evaluation. Also, the generated images are highly biased toward white people so their framework cannot analyze the bias in other groups. More recently, [31] study the gender stereotype in occupations, but the scope of which is limited.Different from the aforementioned works, BiasPainter is the first framework that can automatically, comprehensively, and accurately reveal the social bias in image generation models. 
conclusion: 
 In this paper, we design and implement BiasPainter, an evaluation framework for measuring the social biases in image generation models. Unlike existing frameworks, which only use sentence descriptions as input and evaluate the properties of the generated images, BiasPainter adopts an image editing manner that inputs both seed images and sentence descriptions to let image generation models edit the seed image and then compare the generated image and seed image to measure the bias. We conduct experiments on six famous image generation models to verify the effectiveness of BiasPainter. and demonstrate that BiasPainter can effectively trigger a massive amount of biased behavior with high accuracy. In addition, we demonstrate that BiasPainter can help mitigate the bias in image generation models.  
 
 Appendix: https://openai.com/research/dall-ehttps://imagen.research.google/ 2https://www.bls.gov/emp/tables/emp-by-detailed-occupation.htmhttps://onlineteachersuk.com/personality-adjectives-list/https://www.oxfordlearnersdictionaries.com/external/pdf/wordlists/oxford-3000-5000/American_Oxford_5000.pdf7 https://www.vocabulary.com/lists/189583https://huggingface.co/runwayml/stable-diffusion-v1-5, https://huggingface.co/ stabilityai/stable-diffusion-2-1https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/ main/fast_stable_diffusion_AUTOMATIC1111.ipynbhttps://github.com/AUTOMATIC1111/stable-diffusion-webuihttps://platform.stability.ai/docs/api-reference#tag/v1generation/operation/ imageToImagehttps://github.com/yokonsan/midjourney-apihttps://platform.openai.com/docs/guides/images/usagehttps://replicate.com/timothybrooks/instruct-pix2pix/api 


Decision: The paper introduces a timely framework for detecting social biases in image generation models. The reviews highlight several strengths, including the framework’s scalability, comprehensive coverage, and robust evaluation. However, there are also noted limitations, such as the narrow scope of biases considered and potential methodological issues.

After considering that: 
i) the framework proposed in this paper offers a significant advancement in the detection and mitigation of social biases in generated images; 
ii) the paper’s strengths outweigh its limitations; 
iii) the authors’ effective response to feedback, providing additional experiments and clarifications (that should be added to the paper). 
I believe that this paper can make a valuable addition to the conference. Therefore, I recommend acceptance of this paper.

Reviews: review: paper_summary: paper_summary: The paper presents BiasPainter, a framework to detect social bias in generated images. It analyzes 3 main categories of bias: gender, race, and age.
BiasPainter introduces a novelty approach to detect bias in generated content: it compares the generated images with ground truth references to acquire the bias score, while previous methods just leverage the prompted text.
strength_weakness: strength_weakness: The automatic bias detection process helps in mitigating the requirements for human effort. This makes the solution scalable and potentially broadens the application of social bias detectors.
limitations: limitations: The categories considered in this work only cover a narrow distribution of social bias in generated images. The seed image set used as a baseline to detect social bias consists of photos manually selected from the VGGFace2 dataset, where 3 photos from each combination are selected, thus obtaining a total of 54 photos. 
Using a wider population for the seed set, the distribution of considered classes might be better addressed.
Moreover, the collection process for the seed set is dependent on the bias of the human operator.
paper_summary: paper_summary: This paper introduces "BiasPainter," an evaluation framework designed to automatically detect social biases in image generation models. The framework employs a broad range of queries targeting various sensitive biases and assesses numerous contemporary generative models.
strength_weakness: strength_weakness: 1. The evaluation coverage is broad, including 62 professions, 39 activities, 57 types of objects, and 70 personality traits. 
2. The evaluated models are cutting-edge generative models, including StableDiffusion, Midjourney, et al. 
3. The results, as presented in Table 1, are detailed and comprehensive, offering a clear overview of model biases.
limitations: limitations: 1. While the evaluation results are detailed, some of the metrics used in Section 3.5 seem overly reliant on manual techniques.  For example, the race bias score is calculated using grayscale values to estimate skin color, which may not yield accurate results and could affect the robustness of the evaluation framework.

2. The framework requires both a seed image and a text prompt for input. This requirement limits its applicability since many popular online image generation systems do not support image inputs. How to expand the evaluation target to a broader range of models remains a problem.
paper_summary: paper_summary: This paper presents BiasPainter, a new evaluation framework for measuring social biases in image generation models. Experiments on six image generation models show that BiasPainter effectively identifies and quantifies biased behavior with high accuracy. Additionally, its potential to reduce bias in these models is demonstrated.
strength_weakness: strength_weakness: 1. This work is working on social bias detection and mitigation in image generation, an essential task for generative AI society. The sensitive attributes, such as gender, age, and race, are considered. 
2. An evaluation framework is proposed. This could help us to underline those biases in other application models with the same metrics for better comparison. 
3. Six image generation models are applied, and a strong and concrete evaluation is done. 
4. The clarity of the paper is good from my perspective.
limitations: limitations: 1. Line 924 has an additional ‘.’ after the word BiasPainter. 
2. In gender and age assessment, from my point of view, it is better to compare more than one API/model age estimation results instead of just using one single API to claim the subject’s age. Since the estimation results could not be validated, do the authors try human manual evaluation or another way to ensure this assessment is reliable?
3. A relevant and reasonable image processing pipeline determines the skin tone color score. Still, there is a common issue in image processing: the influence of different lighting conditions in every image. I saw that the authors do not consider the highlighted region of the face, but the issue I am concerned about is the color intensity or brightness of the image itself. Do you consider this matter? This is a promising pipeline, anyway.
4. ‘Generally, the Stable Diffusion Models are more biased than the other three models. Stable-Diffusion 2.1 is the most biased model on age, and Pix2pix shows less bias on age and gender.’ for this sentence, can you provide insights or any comments on why this happened?
5. For Table 4, from this paper's clarification, does it mean that the more the value of the word bias score is towards 0, the more likely it is not to be so biased? From my understanding, is the bias mitigation done by balancing the dataset class distribution of that particular target variable with the knowledge of the RQ2 outcome? How do you prove that it is your RQ2 that contributes to the decrease in score? we all knew that balancing the classes would decrease this score since we have more balanced training data.