B ias D ora: Exploring Hidden Biased Associations in Vision-Language Models (2024)

Abstract: Existing works examining Vision-Language Models (VLMs) for social biases predominantly focus on a limited set of documented bias associations, such as gender-profession or race-crime. This narrow scope often overlooks a vast range of unexamined implicit associations, restricting the identification and, hence, mitigation of such biases. We address this gap by probing VLMs to (1) uncover hidden, implicit associations across 9 bias dimensions. We systematically explore diverse input and output modalities and (2) demonstrate how biased associations vary in their negativity, toxicity, and extremity. Our work (3) identifies subtle and extreme biases that are typically not recognized by existing methodologies. We make the **D**ataset **o**f **r**etrieved **a**ssociations (**Dora**) publicly available.

## introduction despite the transformative potential of vision-language models (vlms) across many domains, mounting evidence underscored their risks to perpetuate and exacerbate social biases @xcite , from reinforcing gender stereotypes by associating women with specific professions @xcite to marginalizing minority communities by linking people of color with negative connotations @xcite . towards this, several bias evaluation methods have been designed @xcite @xcite @xcite . however, a critical limitation of existing evaluation methods is that they heavily rely on predefined associations like man↔doctor and woman↔nurse @xcite , remarkably narrowing their scope. the lists of associa-1 data and code are available here https://github. com/chahatraj/biasdora tions foot_0 in existing works represent just the tip of the iceberg in the vast spectrum of real-world biases. while most recent studies focus on evaluating occupational biases across different genders @xcite , @xcite investigate text-toimage models across professions depicted through descriptors. @xcite ; @xcite ; @xcite explore biases in the associations between people, occupations, traits, and objects, though constrained by a finite and predefined set of associations. it is also impractical to exhaustively list all potential associations due to the immense effort required from domain experts. more importantly, the ultimate goal in assessing social biases in vlms is to uncover all hidden biases within these models that can potentially harm individuals and society, not merely to confirm already known biases. models may harbor biases that differ from those recognized by humans. there is an overlap between real-world biases and those inherent in vlms (figure 1 ), yet there is also a substantial portion of biases unique to vlms that remain unexplored. figure 2 : we probe vlms in three modalities: t2t, t2i & i2t through word completion, image generation, and image description tasks. we calculate statistically significant association followed by identifying sentiment-negative and toxic association. we further evaluate bias levels of these associations using llm-based assessment. hence, in this work, we develop a holistic framework to automatically discover associations representing hidden and detrimental biases in vlms. the proposed framework is structured as a threestep pipeline (figure 2 ). we first uncover bias in three paradigms of vlms through three carefully designed tasks: a word completion task for studying biases in processing text @xcite ; an image generation task for gauging biases in generating images (text-to-image); and an image description task for revealing biases in processing images (imageto-text). following this vlm probing step, we further propose an association salience measuring method and a bias level examining stage to collect all statistically significant and detrimental associations in vlms. this stage isolates these harmful biases yet might overlook less negative biases. hence, we also incorporate an llm-based bias level assessment approach to detect subtler, less obvious, yet problematic associations. we apply our analysis framework to investigate biases in various state-of-the-art vlms under different modalities: gpt-4o @xcite and llama-3-8b-instruct (ai, 2023) for textto-text (t2t), dall-e 3 (openai, 2024) and stable diffusion @xcite ) for text-to-image (t2i), gpt-4o and llava @xcite for image-to-text (i2t). we obtain several interesting insights: (1) for the same modality, different models yield different associations. (2) when comparing different modalities, either using the same or different models, we observe distinct associations. (3) we identify associations aligning with real-world biases and common sense, yet have not been discussed in any prior studies. (4) we also uncover stereotypical associations that may not correspond to real-world biases or common sense, and these, too, have been overlooked in prior research. ## vlm probing we propose an extensive probing framework spanning three modalities: text-to-text (t2t), text-to-image (t2i), and image-to-text (i2t). we utilize the crows-pairs dataset @xcite to identify entities across 400 descriptors and nine demographic dimensions: age (ag), disability (da), gender (ge), nationality (nt), physical appearance (pa), race/color (rc), religion (re), sexual orientation (so), and socio-economic status (se). this yields approximately 400 demographic descriptors. we tested the robustness of our framework across five t2t, three t2i and four i2t templates, identifying comparable biases and confirming similar biases across input variations. ## vlm association assessment we collect outputs in text format from all three probing methods for three modalities. to assess biases in text-to-text tasks, we gather word completions for each descriptor; for text-to-image tasks, we collect objective descriptions for generated images of each descriptor; and for image-to-text tasks, we obtain subjective descriptions of input images of each descriptor. we extract salient and impactful associations from these across different modalities. ## empirical analysis we apply the proposed analysis framework to discover associations from various vlms under different modalities: gpt-4o and llama-3-8b for text-to-text, dall-e 3 and stable diffusion for text-to-image, gpt-4o and llava for imageto-text. in this section, we analyze and compare the identified negative associations, toxic associations, and biased associations across modalities, models, and demographic axes. ## related work existing works study social biases in language models using already established bias vocabulary. works such as weat @xcite define target and attribute pairs to study biases in a limited environment. @xcite assess multimodal implicit biases in generative models building on this defined list of concepts and targets. @xcite study image biases across bias dimensions using yet another limited vocabulary of associated adjectives. however, using such predefined biased associations limits the scope of identifying hidden biases vlms can generate. recent research @xcite ) seeks model associations llama psychotic kid↔warlock, iraqi↔terrorist, white person↔traitor, muslim↔jihadist, baptist↔extremist, armenians↔genocide, caucasians↔hostile, psychotic kid↔terrorist, demented person↔terrorist, iraqi↔terrorist, korean↔dictator ## conclusion we identify previously overlooked biased associations in vlms across t2t, t2i, and i2t paradigms through word completions, image generations, and objective and subjective image description tasks, gaining insights into how these biases vary across distinct bias dimensions for a given modality. several biases are observed for each modality for different vlms, aligning with real-world biases following common sense that have not been discussed in prior works and other stereotypical associations that do not align with real-world biases, yet perpetuate within these models.