Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks (2024)

Abstract: Gender bias in vision-language models (VLMs) can reinforce harmful stereotypes and discrimination. In this paper, we focus on mitigating gender bias towards vision-language tasks. We identify object hallucination as the essence of gender bias in VLMs. Existing VLMs tend to focus on salient or familiar attributes in images but ignore contextualized nuances. Moreover, most VLMs rely on the co-occurrence between specific objects and gender attributes to infer the ignored features, ultimately resulting in gender bias. We propose GAMA, a task-agnostic generation framework to mitigate gender bias. GAMA consists of two stages: narrative generation and answer inference. During narrative generation, GAMA yields all-sided but gender-obfuscated narratives, which prevents premature concentration on localized image features, especially gender attributes. During answer inference, GAMA integrates the image, generated narrative, and a task-specific question prompt to infer answers for different vision-language tasks. This approach allows the model to rethink gender attributes and answers. We conduct extensive experiments on GAMA, demonstrating its debiasing and generalization ability.

## introduction vision-language models (vlms) have attracted significant attention in recent years due to their widespread applications in image captioning @xcite , image-text retrieval @xcite , and visual question answering @xcite . remarkable advancements have been achieved in these tasks, primarily measured by task performance metrics @xcite . however, there is a growing concern about figure 1 : examples in image captioning with annotations and captions generated by a baseline model, sat @xcite . we overlay images with attention heatmaps of sat on the right. in the top example, sat focuses on the woman and predicts "juice", a word highly co-occurring with females. in the bottom example, the gender is incorrectly predicted, as "soccer" highly co-occurs with males in the training set. the undesirable social bias (e.g., gender, race) in vlms @xcite @xcite . an example is presented in figure 1 (bottom), implying stereotypes associating sports with masculinity. more alarmingly, biased vlms have the potential to propagate and even exacerbate existing stereotypes and inequalities. previous methods for mitigating social bias towards vision-language tasks can be grouped into two categories: 1) task-specific methods are designed for task-specific datasets and models, which mitigate bias by re-sampling datasets @xcite , synthesizing negative samples for training @xcite , or introducing debiasing modules or training objectives to task-specific models @xcite @xcite ; and 2) task-agnostic methods aim to pre-train a debiased encoder for downstream tasks, most commonly by adopting an adversarial approach to remove unwanted features @xcite , or leveraging counterfactual samples to minimize biased representations @xcite . despite these efforts, task-specific methods often lack generalization ability, while existing task-agnostic methods primarily address bias at the feature level and fall short of probing the essence of social bias. in this work, we focus on gender bias as it is a crucial aspect of social bias. to address potential limitations in existing research, our initial step is to explore the essence of gender bias. we posit that gender bias is a manifestation of object hallucination in vlms @xcite . specifically, models understand the form rather than the meaning. 2 this leads to a tendency for vlms to focus on the most salient or familiar objects or attributes but ignore the rest of the image. moreover, as depicted in figure 1 , vlms are inclined to hallucinate objects that co-occur with gender words, and vice versa for gender attributes closely associated with specific objects in the training set. as a result, vlms may generate answers containing objects or gender attributes inconsistent with the given image. we propose gama, a novel task-agnostic generation framework to mitigate gender bias towards vision-language tasks. unlike previous methods, gama is a multi-level method, addressing bias both in external object co-occurrences and in internal bias features. the framework comprises two stages: narrative generation and answer inference. during narrative generation, gama creates an allsided narrative for a given image, which prevents premature focus on localized details. besides, we disentangle features through contrastive learning @xcite to obfuscate gender information in the generated narratives. this approach helps mitigate the influence of gender attributes on context generation. the trained model can seamlessly adapt to different vision-language tasks for narrative generation without retraining. during answer inference, the image, generated narrative, and task-specific question prompt are utilized to derive answers for different tasks. as gender information is obfuscated in generated narratives, the model is encouraged to rethink gender attributes in this stage, which contributes to more appropriate and unbiased answers. we conduct extensive experiments to demonstrate the effectiveness of gama. first, gama is evaluated across two vision-language tasks: im-age captioning and image search. additionally, zero-shot experiments are performed on two benchmarks for measuring gender bias, namely visogender @xcite and vl-bias @xcite . experimental results show that gama performs well against previous debiasing methods on task performance and gender bias mitigation. notably, gama exhibits remarkable generalization ability on both benchmarks. moreover, we showcase the effectiveness of our proposed modules in reducing object hallucination and gender bias. for further explanation, we probe the connection between object hallucination and gender bias. ## method in this section, we will first overview the framework of gama and then introduce the training data. finally, we detail our model architecture. ## results and analysis in this section, we seek to answer the following research questions: rq1: does gama perform well across different vision-language tasks in task performance metrics and gender bias metrics? (section 5.1) rq2: do our gender obfuscation module and two-stage framework help gama think before acting? (section 5.2) rq3: are object hallucination and gender bias closely related? (section 5.2) rq4: can gama be generalized to specialized datasets for gender bias measurement under a zero-shot setting? (section 5.3) ## conclusion in this paper, we present gama, a two-stage taskagnostic generation framework to mitigate gender bias towards vision-language tasks. gama is encouraged to gain a comprehensive understanding of images during narrative generation and to rethink gender attributes and answers during answer inference. experimental results demonstrate gama's superiority in both task performance and gender bias metrics over previous methods. furthermore, we conduct ablation studies and analyze the close connection between object hallucination and gender bias. finally, we evaluate gama under a zeroshot setting to showcase its generalization ability. we hope that gama can contribute to the future exploration of fairness in vlms.