Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective (2024)

Abstract: Vision-language models (VLMs) pre-trained on extensive datasets can inadvertently learn biases by correlating gender information with specific objects or scenarios. Current methods, which focus on modifying inputs and monitoring changes in the model’s output probability scores, often struggle to comprehensively understand bias from the perspective of model components. We propose a framework that incorporates causal mediation analysis to measure and map the pathways of bias generation and propagation within VLMs. Our framework is applicable to a wide range of vision-language and multimodal tasks. In this work, we apply it to the object detection task and implement it on the GLIP model. This approach allows us to identify the direct effects of interventions on model bias and the indirect effects of interventions on bias mediated through different model components. Our results show that image features are the primary contributors to bias, with significantly higher impacts than text features, specifically accounting for 32.57% and 12.63% of the bias in the MSCOCO and PASCAL-SENTENCE datasets, respectively. Notably, the image encoder’s contribution surpasses that of the text encoder and the deep fusion encoder. Further experimentation confirms that contributions from both language and vision modalities are aligned and non-conflicting. Consequently, focusing on blurring gender representations within the image encoder which contributes most to the model bias, reduces bias efficiently by 22.03% and 9.04% in the MSCOCO and PASCAL-SENTENCE datasets, respectively, with minimal performance loss or increased computational demands.

## introduction vision-language models have shown promising results in tasks such as classification (li et al., 1 our code is avaiable at https://github.com/ recorderjimmyw/bias-vlm-cma 2023; @xcite , image search @xcite @xcite , and object detection @xcite by training on large-scale image-text pairs to understand the correspondences between cross-modal image features and language features. models trained on extensive datasets exhibit excellent zero-shot capabilities @xcite @xcite but also risk discovering and exploiting societal biases present in the underlying imagetext pair corpora, potentially introducing bias that leads to social unfairness @xcite . the revelation, measurement, and understanding of biases within models @xcite @xcite has sparked widespread research interest and are crucial for bias mitigation @xcite @xcite . however, most contemporary methods, derived from language models, lack standardized metrics for evaluating bias and primarily assess the correlation between the outputs of classifiers and external attributes @xcite . @xcite noted that interpretations based on classifier outputs can be factually inaccurate and not generalizable. while these methods can highlight the impacts of certain contributions on model outputs, they (1) fail to comprehend the generation and flow of bias within the model and (2) do not understand the causal roles of model components in the generation and propagation of bias. consequently, they are not able to provide clear guidance on how to effectively mitigate bias at the model level. in this work, we propose a standardized framework to measure bias in vlms, providing a comprehensive understanding of how bias flows within the entire model structure. specifically, we use the glip model @xcite as a case study, focusing on gender bias in the task of object detection, which is a predominant and challenging problem in computer vision. we conduct the analysis on both the ms-coco @xcite and pascal-sentence @xcite datasets. we observe that glip model exhibits unbalanced inference capabilities on different genders, with certain indoor object categories like pets more likely to be associated with females and outdoor objects like vehicles with males. to holistically understand how the bias flows in the model, we adapt causal mediation analysis @xcite to vlms, providing a finer-grained study of contributions from different model components. we find that, among the different model components (text module, image module, and fusion module that combines them), the image module contributes the most to the model's bias -over twice as much as the text module. in the mscoco and pascal-sentence datasets, image features accounted for 32.57% and 12.63% of the bias generated, respectively, compared to approximately 15.48% and 5.64% by text features. also, the interaction and updating process between image and text features during the deep fusion process significantly impacts bias production, accounting for about 57% of the contributions in the image and text encoders. furthermore, by integrating interventions across different modules, we discovered that their contributions to bias are aligned and do not conflict, allowing us to prioritize bias mitigation efforts within the image encoder, which is the most substantial contributor to bias. based on the results, we propose to effectively mitigate the bias in vlms: reducing biases from the image module can successfully reduce bias by 22.03% on the mscoco dataset and 9.04% on the pascal-sentence dataset, compared to a reduction of 7.08% and 1.18% in the text module. in summary, the contributions of our work are: ## related work in recent years, vision-language models (vlms) have experienced rapid advancements. the latest developments in vlms often employ a dual-stream architecture that separately encodes text and images @xcite , and these are then merged and aligned to facilitate cross-modal understanding of visual and linguistic features @xcite . furthermore, some studies treat the joint training of images and text as a phrase localization process, aiming to better align and integrate visual and linguistic features @xcite . typically, these models are trained on image-text pairs from datasets such as mscoco @xcite , vqa @xcite , openimages @xcite , and flickr30k entities @xcite , achieving impressive results in various downstream tasks including image classification @xcite , image generation @xcite , visual question answering @xcite , and image captioning @xcite . alongside their development, the societal biases exhibited by vlms have also attracted significant attention. these models often reflect societal stereotypes and may even amplify such biases @xcite . most contemporary research addressing bias in vlms has borrowed methodologies from language model studies. for instance, @xcite utilized a language masking model to explore gender biases by using templates containing a specific entity and analyzing the probabilities of masked entities @xcite . some researchers have examined biases through the comparison of factual and counterfactual inputs, with @xcite investigating biases by examining predicted probabilities from both factual and counterfactual inputs, and @xcite using the perspective api to score predictions derived from such inputs to study model biases. however, existing evaluation methods primarily observe changes in the probability scores of model outputs following interventions on input samples. this approach limits our understanding of the underlying causes of bias generation and propagation within model components @xcite . therefore, we propose a standardized framework for evaluating bias in vision-language tasks and introduce causal mediation analysis @xcite @xcite within the context of vision-language models. this methodology helps us understand the pathways of bias generation and propagation from the input level to model components. ## bias measurement and understanding in vlm in this section, we propose a bias evaluation metric to assess the bias of vlm in the object detection task. by applying causal mediation analysis, we quantify the contribution on bias from various components within the model pipeline which helps us trace the origins and propagation of bias throughout the model pipeline. additionally, we investigate the interactions between different modalities to understand how they collectively influence model bias which will be used as guidance for bias mitigation later. ## bias mitigation method based on our experimental results, image features contribute most significantly to gender bias and the image encoder has a more pronounced impact on bias compared to the text encoder and deep-fusion encoder. therefore, our intuition is that focusing on reducing gender representation in the image encoder will effectively reduce bias, especially when facing a computation budget. we use the bias mitigation achieved from the text encoder as a baseline, then focus on reducing bias from the image encoder and compare the results with the baseline. ## experimental setup of bias mitigation model we utilized the glip model, pre-trained on the o365, goldg, cc3m, and sbu datasets @xcite . in our setup, we incorporated an mtcnn @xcite pre-trained on the wider face and celeba datasets as a face detector within the image encoder. additionally, we integrated a mobilenet @xcite pretrained on imagenet to serve as a gender classifier. dataset we evaluated the effectiveness of bias mitigation on the mscoco and pascal-sentence datasets. to assess the model's object detection performance, we compared it with the original glip @xcite on the mscoco and lvis datasets using the ap (average precision) metric for zero-shot object detection. ## conclusion vision-language models (vlms) trained on largescale image-text pair corpora are at risk of learning social biases from their training data. in this paper, we introduced a standardized framework incorporating causal mediation analysis to measure and understand the pathways through which model bias is generated and propagated within vlms. we discovered that image features contribute significantly more to model bias than text features, and the contributions from the image encoder substantially exceed those from the text encoder and deep fusion encoder. furthermore, the contributions to bias from different modalities reinforce each other. subsequently, by focusing on the components that contribute most to bias, we efficiently reduced model bias. our work provides a framework for measuring, understanding, and mitigating model bias, which, although utilized here within the realm of object detection, can be extended to a wide range of vlm tasks. we used gender bias as a case study to showcase our methodology, as many influential studies in the natural language processing and fairness fields have begun with the study of gender bias @xcite @xcite . gender bias is one of the most representative types of bias, and understanding it can provide valuable insights for studying other forms of bias. the frameworks and evaluation metrics we developed for gender bias are generalizable and can be extended to other biases. for instance, if ground truth labels about age are available in the image data, researchers can modify the text descriptions to include age-related terms and apply the same methodology to study age-related bias. however, our framework is primarily applicable to white-box models, as it requires interventions at the internal components of the model. a promising direction for future work would involve expanding our framework to encompass additional modalities such as audio or video. this expansion could further enhance our understanding of multimodal interactions and their impact on bias, as well as deepen insights into how different sensory inputs contribute to, or mitigate, biases in ai systems.