Venue: NAACL 2024

# Think Before You Act: A Two-Stage Framework for Mitigating Gender

Bias Towards Vision-Language Tasks

 Yunqi Zhang1, Songda Li1, Chunyuan Deng2, Luyi Wang1, Hui Zhao1,3,*

Footnote 1: Corresponding author.

###### Abstract

Gender bias in vision-language models (VLMs) can reinforce harmful stereotypes and discrimination. In this paper, we focus on mitigating gender bias towards vision-language tasks. We identify object hallucination as the essence of gender bias in VLMs. Existing VLMs tend to focus on salient or familiar attributes in images but ignore contextualized nuances. Moreover, most VLMs rely on the co-occurrence between specific objects and gender attributes to infer the ignored features, ultimately resulting in gender bias. We propose GAMA, a task-agnostic generation framework to mitigate gender bias. GAMA consists of two stages: narrative generation and answer inference. During narrative generation, GAMA yields all-sided but gender-obfuscated narratives, which prevents premature concentration on localized image features, especially gender attributes. During answer inference, GAMA integrates the image, generated narrative, and a task-specific question prompt to infer answers for different vision-language tasks. This approach allows the model to rethink gender attributes and answers. We conduct extensive experiments on GAMA, demonstrating its debiasing and generalization ability.1

Footnote 1: Our code is available at [https://github.com/zyq0000/GAMA](https://github.com/zyq0000/GAMA).

_Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 773–791

June 16-21, 2024 02024 Association for Computational Linguistics

## 1 Introduction

Vision-language models (VLMs) have attracted significant attention in recent years due to their widespread applications in image captioning [18, 5], image-text retrieval [23, 24], and visual question answering [14, 15]. Remarkable advancements have been achieved in these tasks, primarily measured by task performance metrics [11, 12]. However, there is a growing concern about the undesirable social bias (e.g., gender, race) in VLMs [11, 13, 24]. An example is presented in Figure 1 (bottom), implying stereotypes associating sports with masculinity. More alarmingly, biased VLMs have the potential to propagate and even exacerbate existing stereotypes and inequalities.

Previous methods for mitigating social bias towards vision-language tasks can be grouped into two categories: 1) _task-specific methods_ are designed for task-specific datasets and models, which mitigate bias by re-sampling datasets [25, 26], synthesizing negative samples for training [16], or introducing debiasing modules or training objectives to task-specific models [11, 26, 27]; and 2) _task-agnostic methods_ aim to pre-train a debiased encoder for downstream tasks, most commonly by adopting an adversarial approach to remove unwanted features [23], or leveraging counterfactual samples to minimize biased representations [25]. Despite these efforts, task-specific

Figure 1: **Examples in image captioning with annotations and captions generated by a baseline model, SAT [26]. We overlay images with attention heatmaps of SAT on the right. In the top example, SAT focuses on the woman and predicts “juice”, a word highly co-occurring with females. In the bottom example, the gender is incorrectly predicted, as “soccer” highly co-occurs with males in the training set.**

methods often lack generalization ability, while existing task-agnostic methods primarily address bias at the feature level and fall short of probing the essence of social bias.

In this work, we focus on gender bias as it is a crucial aspect of social bias. To address potential limitations in existing research, our initial step is to explore the essence of gender bias. We posit that gender bias is a manifestation of object hallucination in VLMs Rohrbach et al. (2018). Specifically, models understand the form rather than the meaning.2 This leads to a tendency for VLMs to focus on the most salient or familiar objects or attributes but ignore the rest of the image. Moreover, as depicted in Figure 1, VLMs are inclined to hallucinate objects that co-occur with gender words, and vice versa for gender attributes closely associated with specific objects in the training set. As a result, VLMs may generate answers containing objects or gender attributes inconsistent with the given image.

Footnote 2: Following Bender and Koller (2020), we define _form_ as the visible realization of vision and language, such as pixels or bytes in digital representations of text or image, and _meaning_ as the relationship between form and external elements to vision and language, like communicative intent.

We propose GAMA, a novel task-agnostic generation framework to mitigate gender bias towards vision-language tasks. Unlike previous methods, GAMA is a multi-level method, addressing bias both in external object co-occurrences and in internal bias features. The framework comprises two stages: _narrative generation_ and _answer inference_. During _narrative generation_, GAMA creates an all-sided narrative for a given image, which prevents premature focus on localized details. Besides, we disentangle features through contrastive learning He et al. (2020); Chen et al. (2020) to obfuscate gender information in the generated narratives. This approach helps mitigate the influence of gender attributes on context generation. The trained model can seamlessly adapt to different vision-language tasks for narrative generation without retraining. During _answer inference_, the image, generated narrative, and task-specific question prompt are utilized to derive answers for different tasks. As gender information is obfuscated in generated narratives, the model is encouraged to rethink gender attributes in this stage, which contributes to more appropriate and unbiased answers.

We conduct extensive experiments to demonstrate the effectiveness of GAMA. First, GAMA is evaluated across two vision-language tasks: image captioning and image search. Additionally, zero-shot experiments are performed on two benchmarks for measuring gender bias, namely VisoGender Hall et al. (2023) and VL-Bias Zhang et al. (2022). Experimental results show that GAMA performs well against previous debiasing methods on task performance and gender bias mitigation. Notably, GAMA exhibits remarkable generalization ability on both benchmarks. Moreover, we showcase the effectiveness of our proposed modules in reducing object hallucination and gender bias. For further explanation, we probe the connection between object hallucination and gender bias.

## 2 Related Work

### Sources of gender bias

The growing popularity of multi-modality has prompted research into sources of gender bias in vision-language tasks. First, _datasets_ are a prominent factor, as object labeling is closely linked to our conceptualization Brown (1958). Thus real-life stereotypes subtly infiltrate datasets. Hirota et al. (2022a) and Harrison et al. (2023) highlighted the underrepresentation of women in datasets. Second, _pre-trained VLMs_ (PVLMs) have enhanced vision-language tasks by leveraging extensive knowledge from pre-training data. Consequently, PVLMs inherit such bias from both language and vision sources Wang et al. (2019); Ross et al. (2021); Srinivasan and Bisk (2022). Third, _model structures_ may amplify gender bias in datasets Zhao et al. (2017). Researchers have observed stereotype exaggeration in various task-specific VLMs Kay et al. (2015); Bhargava and Forsyth (2019); Wang et al. (2021).

Given these diverse bias sources, gender bias may not be effectively addressed by merely preprocessing data or debiasing pre-training features.

### Gender bias mitigation

Research interest in gender bias mitigation for vision-language tasks has notably increased. Previous studies addressed gender bias in unimodal models, such as language He et al. (2022); Shaikh et al. (2023) or vision Wang et al. (2019); Steed and Caliskan (2021). Zhou et al. (2022) uncovered both intra-modal and inter-modal gender bias. Previous work on gender bias mitigation for VLMs can be divided into three classes: 1) _adding debiasing modules to existing task-specific VLMs_, which aims to pre-process imbalanced training sets Zhao et al. ( 

2017), process intermediate features (Hendricks et al., 2018; Wang et al., 2021), or post-process biased model outputs (Hirota et al., 2023; Janghorbani and De Melo, 2023); _2) re-designing training objectives of PVLMs_ to produce debiased features through negative (Wang et al., 2019) or counterfactual (Zhang et al., 2022) samples; and _3) proposing new model structures_ designed to learn fair representations by capturing gender visual evidence (Tang et al., 2021), or adding a negative residual (Seth et al., 2023). However, previous studies superficially address gender bias by removing biased data or features and ignore the correlation between gender bias and object hallucination.

### Object hallucination

Despite the great success of VLMs in vision-language tasks, they still suffer from object hallucination. Object hallucination refers to the fact that the content generated by the model contains objects inconsistent with or absent from the given image (Rohrbach et al., 2018). Rohrbach et al. (2018) proposed two metrics, CHAIRs and CHAIRi, to measure object hallucination. A recent approach, POPE (Li et al., 2023b), is a polling-based query method with enhanced flexibility. Efforts to mitigate object hallucination in VLMs fall into two categories. Some methods aim to disentangle object co-occurrence patterns. For instance, Biten et al. (2022) introduced object labels and altered objects in the captions, while Zhou et al. (2023) replaced error-prone objects with placeholder tags. Others focus on minimizing logical errors, such as leveraging chain-of-thought (CoT) prompting to generate intermediate reasoning chains as rationales (Zhang et al., 2023).

## 3 Method

In this section, we will first overview the framework of GAMA and then introduce the training data. Finally, we detail our model architecture.

### Overview

GAMA is composed of two stages: narrative generation and answer inference, as illustrated in Figure 2. During narrative generation, GAMA takes an image and a question prompt as input to yield an all-sided but gender-obfuscated narrative. During answer inference, different vision-language tasks are formulated as generation tasks. GAMA utilizes the image, generated narrative, and task-specific question prompt to generate a task-specific answer. We formulate both stages as taking an image \(I\) and a text sequence \(X=[x_{1},x_{2},\cdots,x_{n}]\) with \(n\) tokens as input and outputting a target sequence \(Y=[y_{1},y_{2},\cdots,y_{m}]\) with \(m\) tokens.

The models in the two stages are trained independently. The trained narrative generation model can be applied to different tasks without retraining.

GAMA comprises three key modules: a vision-language fusion module to extract features in both stages, a gender obfuscation module to mask gender-related information during narrative generation, and a decoder for answer generation.

### Training data

During narrative generation, GAMA strives to generate an all-sided narrative for the given image to avoid premature focus on localized details. To this end, we train the model with the Localized Narratives dataset (Voigtlaender et al., 2023) for Open Images (Kuznetsova et al., 2020). The dataset is crafted to depict image regions covered by a mouse trace, associating specific image regions with spe

Figure 2: **The overall framework of GAMA. We briefly provide task-specific question prompts and answers, which are detailed in Appendix C. We take the token probability of the decoder as the match score in image search.**



cific words in the narrative. Consequently, the narratives are spread throughout the entire image, rather than confined to specific localized regions.

### Vision-language fusion module

The vision-language fusion module is designed to learn vision-language representations in two stages.

EncoderWe use the T5 (Raffel et al., 2020) encoder to learn language representations, and a Vision Transformer (ViT) (Dosovitskiy et al., 2020) to extract vision features:

\[{\bf H}_{l}=\text{T5Encoder}([x_{1},x_{2},\cdots,x_{n}]), \tag{1}\] \[{\bf H}_{v}={\bf W}_{1}\text{ViT}(I), \tag{2}\]

where \({\bf H}_{l},{\bf H}_{v}\in\mathbb{R}^{n\times d}\), and \(d\) is the hidden dimension. \({\bf W}_{1}\) is a projection matrix to convert the shape of features extracted by the ViT. Notably, the parameters of the ViT are frozen.

Feature fusionWe utilize an attention mechanism (Vaswani et al., 2017) to capture the interactions between language and vision features:

\[\mathbf{\tilde{H}}_{v}=\text{Attention}({\bf H}_{l},{\bf H}_{v},{\bf H}_{v}). \tag{3}\]

We aggregate \({\bf H}_{l}\) and \(\mathbf{\hat{H}}_{v}\) with a gate mechanism (Li et al., 2022; Fang and Feng, 2022). The gate \(\lambda\) and the fused features \({\bf H}\) are calculated as:

\[\lambda=\text{Sigmoid}({\bf W}_{2}{\bf H}_{l}+{\bf W}_{3}\mathbf{ \hat{H}}_{v}), \tag{4}\] \[{\bf H}=(1-\lambda)\cdot{\bf H}_{l}+\lambda\cdot\mathbf{\hat{H}}_ {v}, \tag{5}\]

where \({\bf W}_{2}\) and \({\bf W}_{3}\) are trainable parameters.

### Gender obfuscation module

The gender obfuscation module is employed during narrative generation to obfuscate gender information in narratives. This module prevents the model from biasing the context due to gender attributes.

Pre-processingFirst, we replace gender words with a special token [GENDER] and leave other words unchanged to obtain a narrative with masked gender words \(\bar{Y}=[\bar{y}_{1},\bar{y}_{2},\cdots,\bar{y}_{m}]\).3

Footnote 3: The gender word list comes from Hirota et al. (2023), which can be found in Appendix A.

Gender information detectionThen we construct the gender-masked features \(\mathbf{\tilde{H}}\). We detect gender-related features \({\bf H}_{g}\) and remove them from the vision-language features \({\bf H}\) as follows:

\[\mu=\text{Sigmoid}({\bf W}_{4}{\bf H}), \tag{6}\] \[{\bf H}_{g}=\mu\cdot{\bf H},\] (7) \[\mathbf{\tilde{H}}={\bf H}-{\bf H}_{g}, \tag{8}\]

where \({\bf W}_{4}\) is a trainable parameter.

Contrastive lossMotivated by contrastive learning (He et al., 2020; Chen et al., 2020), we employ a contrastive loss to keep the vision-language features \({\bf H}\) close to the gender-masked features \(\mathbf{\widetilde{H}}\) and away from the gender-related features \({\bf H}_{g}\). This approach helps obscure gender details as well as preserve context information. We define our contrastive loss as:

\[\mathcal{L}_{con}=-\log\frac{e^{\mathrm{s}(\mathbf{H},\mathbf{\widetilde{H}})/ \tau}}{e^{\mathrm{s}(\mathbf{H},\mathbf{\widetilde{H}})/\tau}+e^{\mathrm{s}( \mathbf{H},\mathbf{H}_{g})/\tau}}, \tag{9}\]

where \(\mathrm{s}(\mathbf{U},\mathbf{V})=\mathbf{U}^{\top}\mathbf{V}/\|\mathbf{U}\| \|\mathbf{V}\|\) denotes the cosine similarity between \(\mathbf{U}\) and \(\mathbf{V}\), and \(\tau\) is a temperature hyper-parameter.

### Decoder

The T5 decoder takes the encoder output \(\mathbf{\widetilde{H}}\), text sequence \(X\), and previously generated tokens \(\tilde{Y}_{<t}\) as inputs to get the \(t\)-th token probability distribution:

\[\mathbf{\tilde{h}}_{t}=\text{T5Decoder}(\mathbf{\widetilde{H}}, X,\tilde{Y}_{<t}), \tag{10}\] \[P(\tilde{y}_{t}|\mathbf{\widetilde{H}},X,\tilde{Y}_{<t})=\text{ Softmax}({\bf W}_{5}\mathbf{\tilde{h}}_{t}), \tag{11}\]

where \({\bf W}_{5}\) is a trainable parameter to map the hidden dimension to the vocabulary size. For each symbol \(\tilde{a}\in\{\mathbf{\tilde{H}},\mathbf{\tilde{h}}_{t},\tilde{Y}_{<t},\tilde{ y}_{t}\}\), \(\tilde{a}\) is either \(a\) or \(\tilde{a}\).

### Training objective

During the training phase, we use the teacher forcing to train the models in the two stages. The models in two stages are trained independently.

Narrative generationIn the narrative generation stage, we train the model with the cross-entropy loss on ground-truth narratives and narratives with masked gender words, which can be defined as:

\[\mathcal{L}_{ce}=-\sum_{t=1}^{m}\log P(y_{t}|\mathbf{H},X,Y_{<t}), \tag{12}\] \[\mathcal{\bar{L}}_{ce}=-\sum_{t=1}^{m}\log P(\bar{y}_{t}|\mathbf{ \widetilde{H}},X,\bar{Y}_{<t}). \tag{13}\]

In total, the narrative generation stage contains three losses, which can be formulated as:

\[\mathcal{L}_{1}=\mathcal{L}_{con}+\mathcal{L}_{ce}+\mathcal{\bar{L}}_{ce}. \tag{14}\]

Answer inferenceThe answer inference model is trained with a cross-entropy loss:

\[\mathcal{L}_{2}=-\sum_{t=1}^{m}\log P(y_{t}|\mathbf{H},X,Y_{<t}).\] (15) 

Experiment Setup

To demonstrate the effectiveness and generalization ability of GAMA, we conduct extensive experiments across two vision-language tasks (image captioning and image search) and two benchmarks for measuring gender bias [26] and VL-Bias [17]).4

Footnote 4: Due to space constraints, experiments on VL-Bias can be found in Appendix D.1.

### Datasets

The statistics of the datasets are listed in Table 1.

Gender labelFollowing previous work [35; 80; 9], we utilize ground-truth captions to label the gender attributes of images. Specifically, an image will be labeled as "male (female)" if at least one of its captions contains male (female) gender words and no captions contain female (male) words. Otherwise, it will be labeled as "neutral".

Narrative generationThe Localized Narratives dataset [26] for Open Images [27] is utilized to train the narrative generation model.5

Footnote 5: We ascertain the absence of any overlap between Localized Narratives and other test sets through the cosine similarity among image tensors with a threshold of \(0.9\). See Appendix D.2 for further studies on the training set size of Localized Narratives.

Image captioningWe experiment on MSCOCO captions [35]. Following Hirota et al. (2023), we use the original MSCOCO training set for training, a subset of the MSCOCO validation set from Zhao et al. (2021) for test,6 and the remaining images for validation. Each image is associated with five human-annotated captions. Notably, no images in the test set are labeled as "neutral".

Footnote 6: Access the test split at [https://princetonvisualai.github.io/imagecaptioning-bias/](https://princetonvisualai.github.io/imagecaptioning-bias/).

Image searchOur evaluation for image search involves the MSCOCO [36] and Flickr30K [26] datasets. Following Wang et al. (2021), we use the Karpathy split [26] for training and validation. As test sets in the Karpathy split are gender-unbalanced, we randomly select the same number of images with their gender-neutral queries 7 under "male", "female" and "neutral" labels for test.

Footnote 7: The queries are available at [https://github.com/eri-c-ai-lab/Mitigate-gender-Bias-in-Image-Search](https://github.com/eri-c-ai-lab/Mitigate-gender-Bias-in-Image-Search).

Viso GenderVisoGrenderHall et al. (2023) benchmarks occupation-related gender bias in VLMs. VLMs are required to align images with correct gender pronouns in a resolution task and retrieve the top-K images for a gender-neutral caption of a given occupation in a retrieval task.

### Baselines

Image captioningGAMA is compared with the following methods for image captioning: 1) **Equalizer**[11], which focuses on the "person" segmentation to make gender-specific predictions; 2) **GAIC\({}_{es}\)**[17], which encourages capturing gender visual evidence through self-guided visual attention; and 3) **LIBRA**[11], which leverages state-of-the-art captioning models to generate high-quality captions and debiases through an additional editing model.

Image searchWe evaluate GAMA against the following methods for image search: 1) **SCAN-FS**[80], which applies a fair sampling method to the representative image search baseline SCAN Lee et al. (2018); 2) **CLIP-clip**[80], which introduces a feature pruning algorithm to the features generated by the PVLM CLIP Radford et al. (2021); and 3) **FairVLP**[17], which trains the PVLM ALBEF Li et al. (2021) with counterfactual samples to obtain debiased representations.

VisoGenderFollowing Hall et al. (2023), we evaluate GAMA against state-of-the-art pre-trained vision-language encoders (**CLIP**[11], **OpenCLIP**[11] trained on LAION 2B and 400M, **SLIP**[12], **DeCLIP**[11] and **FILIP**[17]) and pre-trained captioning models (**BLIP-2**[11] and **GIT**[80]).

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Stage** & **Task** & **Dataset** & **Train** & **Dev** & **Test** \\ \hline \multirow{2}{*}{1} & Narrative Generation & Localized Narratives & 507,444 & 41,691 & 126,020 \\ \cline{2-6}  & Image Captioning & MSCOCO & 82,783 & 29,724 & 10,780 \\ \cline{2-6}  & Image Search & Flickr30K & 29,000 & 1,000 & 300 \\ \cline{2-6}  & VisoCender & / & / & 690 \\ \hline \hline \end{tabular}
\end{table}
Table 1: **The statistics of datasets. We show the counts of images within each split of the datasets.** 



### Metrics

We present metrics for evaluation, covering task performance, gender bias, and object hallucination. Calculation details are presented in Appendix B.

**Task performance metrics** are used to evaluate model performance on specific tasks. For _image captioning_, we use established referenced-based metrics BLEU-4 Papineni et al. (2002), CIDEr Vedantam et al. (2015), METEOR Denkowski and Lavie (2014) and SPICE Anderson et al. (2016) as well as a reference-free metric CLIPScore Hessel et al. (2021). For _image search_, we employ Recall@K to calculate the ratio of correct images in the top-K retrieved images.

**Gender bias metrics** are employed to measure the model performance on gender bias mitigation. For _image captioning_, we use LIC Hirota et al. (2022b) to measure gender bias in the context of gender words, Error Hendricks et al. (2018) to assess the gender misclassification ratio of generated captions, and BiasAmp Zhao et al. (2017) to quantify bias amplification based on word-gender co-occurrence. For _image search_, we follow Wang et al. (2021) and adopt Bias@K to measure gender bias among the top-K images. For _VisoGender_, we follow the setting in Hall et al. (2023). The resolution accuracy gap (\(\Delta_{RA}\)) is used to measure resolution bias, while Bias@K Wang et al. (2021), Skew@K Geyik et al. (2019) and NDKL Geyik et al. (2019) are used to measure retrieval bias.

**Object hallucination metrics** are utilized to probe the connection between object hallucination and gender bias. CHAIRs and CHAIRohrbach et al. (2018) are used to evaluate incorrect object generation at the sentence and object levels, respectively. Rohrbach et al. (2018) used a synonym list Lu et al. (2018) to map words to MSCOCO objects. We refine it with hierarchical object relationships.8

Footnote 8: Details can be found in Appendix B.2.

### Implementation details

We use flan-t5-base Chung et al. (2022) as the backbone. Our image encoder is vit-base-patch16-384 Dosovitskiy et al. (2020), and its parameters are frozen during training. For narrative generation, we set the temperature \(\tau=0.1\) and conduct further studies in Appendix D.2. Experiments on VisoGender are conducted onGamma search models under a zero-shot setting. More details are presented in Appendix C.

## 5 Results and Analysis

In this section, we seek to answer the following research questions: **RQ1:** Does GATA perform well across different vision-language tasks in task performance metrics and gender bias metrics? Section 5.1) **RQ2:** Do our gender obfuscation module and two-stage framework help GAMA think before acting? Section 5.2) **RQ3:** Are object hallucination and gender bias closely related? Section

5.2) **RQ4:** Can GAMA be generalized to specialized datasets for gender bias measurement under a zero-shot setting? (Section 5.3)

### Overall results

We conduct experiments on GAMA and baselines. Results on image captioning and image search are shown in Table 2 and Table 3, respectively.

To begin with, GAMA outperforms or closely aligns with the baselines in both two tasks. It is noteworthy that most of these baselines are tailored to a specific task. In contrast, GAMA is a task-agnostic method and can be easily applied to different vision-language tasks.

Then we compare GAMA with each baseline separately. First, GAMA comfortably outperforms Equalizer and GAIC\({}_{es}\). As Equalizer forces the model to focus on persons in images, it loses necessary information to generate the correct context of gender words. Similarly, GAIC\({}_{es}\) is designed to improve gender classification accuracy with additional gender evide nce. Therefore, these two baselines prove less effective.

Second, LIBRA achieves noticeable results with different captioning models. LIBRA considers gender bias manifested as gender misclassification as well as biased context. However, there are some problems. 1) LIBRA is limited by the performance of captioning models, as it is designed to revise their outputs. 2) LIBRA is designed to edit biased captions after a captioning model, but it is trained with synthesized data instead of actual model outputs, potentially causing error propagation. Consequently, the performance of LIBRA depends on the specific captioning model in use, rendering its results on gender bias metrics less stable.

Third, GAMA also performs better than SCANFS, CLIP-clip and FairVLP in image search. We observe that SCAN-FS and CLIP-clip exhibit overcorrection, leading to the underrepresentation of males. These baselines tackle gender bias mainly by removing biased data or features. Given the diverse sources of bias, it cannot be effectively mitigated by merely processing data or debiasing pre-training features.

Finally, we compare models on task performance and gender bias mitigation ability in Figure 3. Results show that GAMA strikes a good balance between task performance and gender bias mitigation.

### Ablation study

To further explain the results, we carry out ablation studies on GAMA as well as explore the connection between object hallucination and gender bias in image captioning. Table 4 shows the results. We summarize the main observations as follows.

The gender obfuscation module mitigates gender bias in the context.First, this module effectively obfuscates gender-related information during narrative generation, leading to a reduction in the number of narratives with gender words. Next, the absence of this module is observed to cause an increase in the LIC score, indicating amplified gender bias in the generated context. Additionally, the BiasAmp score shows an increase. We speculate it is because GAMA without this module is exposed to direct gender information, leading to the generation of more words closely related to gender attributes. The rise in HR\({}^{g}_{C}\)@10 and Figure 4 manifest that GAMA without this module hallucinates more objects that highly co-occur with gender words in the training set, thus affirming our hypothesis. Lastly, GAMA without generated narratives also achieves competitive results against previous

Figure 4: **Heatmap visualization of the co-occurrence frequency between gender attributes and certain words. We respectively select five words highly co-occurring with females and males in the training set. We show the frequency of co-occurrence between gender attributes and words in the model predictions. Darker colors indicate higher frequencies.**

Figure 3: **Comparison on task performance and gender bias mitigation ability. We normalize the metrics separately and sum the normalized gender bias metrics and image captioning metrics, respectively.**



debiasing methods. The results further demonstrate the effectiveness of the gender obfuscation module.

The two-stage framework can facilitate debiased answers.We note an increase in the Error score in the single-stage framework. The narrative generation stage is designed to prompt GAMA to grasp the overall image before giving ultimate answers. The rise in BiasAmp and HR\({}_{C}^{o}@10\) indicates that the single-stage framework relies more on word co-occurrence for answer generation. Hence, the two-stage framework serves as an effective method to prevent GAMA from prematurely focusing on localized features and hallucinating ignored objects.

Object hallucination and gender bias exhibit a close correlation.As shown in Table 4, mitigating gender bias leads to a decrease in object hallucination. Object hallucination mainly results from the frequent occurrence of certain objects (including persons) and the co-occurrence between objects in the training set Li et al. (2023b). Similarly, gender bias is manifested as the overrepresentation of a certain gender or the high-frequency co-occurrence between gender attributes and objects in datasets. In essence, gender bias can be thought of as a form of object hallucination in VLMs. Consequently, efforts to mitigate gender bias in VLMs result in a simultaneous reduction of object hallucination.

### Experiments on generalization ability

The datasets employed in the above experiments are not specifically constructed to detect gender bias in VLMs, which may contain gender bias in annotations, e.g., underrepresentation of women Zhao et al. (2017); Harrison et al. (2023). Consequently, we turn to VisoCender for further experiments. As shown in Table 5, GAMA exhibits good overall performance across bias metrics for both tasks, further demonstrating its generalization ability and effectiveness in gender bias mitigation.

## 6 Conclusion

In this paper, we present GAMA, a two-stage task-agnostic generation framework to mitigate gender

\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Gender Blas Metrics j.**} & \multicolumn{3}{c}{**Gender-related Statistics j.**} & \multicolumn{3}{c}{**Object Ballucination Metrics j. Co-occurrence Statistics j**} \\ \cline{2-9}  & **LIC** & **Error** & **BiasAmp** & **@Gender** & **CHAIRs** & **CHAIRi** & **HR\({}_{C}^{o}@10\)** & **HR\({}_{C}^{o}@10\)** \\ \hline
**GAMA** & -1.1 & 3.4 & -3.40 & 55.61 & 10.94 & 6.02 & 38.15 & 50.30 \\ \hline w/o GO & 0.5 & 3.1 & -1.26 & 62.49 & 12.81 & 7.36 & 41.52 & 50.09 \\ w/o narrative & -0.6 & 4.1 & -2.87 & / & 13.02 & 7.84 & 41.37 & 51.94 \\ single-stage & 0.9 & 3.6 & -2.13 & / & 12.40 & 7.02 & 39.91 & 51.76 \\ \hline \hline \end{tabular}
\end{table}
Table 4: **Results of ablation studies.**#Gender denotes the proportion of generated narratives with gender words in the test set. Note that no images are labeled as “neutral” in the test set. CHAIRs and CHAIRi are calculated based on our refined synonym list. HR\({}_{C}^{o}@10\)Li et al. (2023b) and HR\({}_{C}^{o}@10\) are used to quantify whether the model is prone to hallucinate objects that frequently co-occur with ground-truth objects and gender attributes, respectively, which are detailed in Appendix B.3. \(\lx@paragraphsign\): the gender obfuscation module (GO) is removed from narrative generation; \(\lx@sectionsign\star\): the model is trained to generate gender-obfuscated captions instead of gender-obfuscated narratives in the first stage; \(\lx@paragraphsign\): the narrative generation stage is removed, and GO is adopted in the answer inference stage. Note that the single-stage framework does not incorporate narratives as input.

\begin{table}
\begin{tabular}{c c|c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Resolution**} & \multicolumn{6}{c}{**Retrieval**} \\ \cline{2-11}  & **Accuracy**\(\uparrow\) & \(\Delta_{RA}\) & **Bias\(\bullet\)5** & **Bias\(\circ\)10** & **MaxSlaw\(\theta\)5** & **MaxSlew\(\pm\)10** & **NDKL**\(\downarrow\) \\ \cline{2-11}  & **OO** & **OP** & **Mean** & \(\sigma\) & **Mean** & \(\sigma\) & **Mean** & \(\sigma\) & **Mean** & \(\sigma\) & **Mean** & \(\sigma\) \\ \hline CLIP & 0.75 & -0.14 & -0.27 & 0.11 & 0.38 & 0.16 & 0.22 & 0.27 & 0.15 & 0.18 & 0.13 & 0.19 & 0.07 \\ OpenCLIP\({}_{20}\) & 0.78 & -0.07 & -0.37 & 0.10 & 0.44 & 0.08 & 0.23 & 0.29 & 0.17 & 0.18 & 0.11 & 0.18 & 0.07 \\ OpenCLIP\({}_{20}\) & 0.74 & -0.27 & -0.29 & 0.17 & 0.47 & 0.11 & 0.22 & 0.33 & 0.18 & 0.16 & 0.13 & 0.19 & 0.07 \\ SLIP & 0.60 & 0.14 & 0.14 & 0.06 & 0.52 & 0.00 & 0.24 & 0.32 & 0.21 & 0.17 & 0.12 & 0.19 & 0.09 \\ DLCLIP & 0.70 & 0.06 & -0.17 & 0.11 & 0.10 & 0.15 & 0.26 & 0.28 & 0.16 & 0.20 & 0.14 & 0.17 & 0.07 \\ FILIP & 0.45 & 0.06 & 0.36 & 0.01 & 0.43 & 0.03 & 0.26 & 0.29 & 0.16 & 0.17 & 0.13 & 0.18 & 0.07 \\ BLIP-2 & 0.84 & -0.09 & 0.07 & / & / & / & / & / & / & / & / & / \\ GIT & 0.84 & -0.07 & -0.27 & / & / & / & / & / & / & / & / \\ \hline
**GAMA** & **Flickr20K** & 0.81 & 0.06 & 0.09 & 0.04 & 0.42 & 0.01 & 0.26 & 0.26 & 0.15 & 0.17 & 0.12 & 0.18 & 0.07 \\ MSCOCO & 0.82 & 0.04 & -0.09 & 0.01 & 0.40 & -0.04 & 0.24 & 0.26 & 0.14 & 0.18 & 0.11 & 0.17 & 0.08 \\ \hline \hline \end{tabular}
\end{table}
Table 5: **Results of zero-shot experiments on VisoSender.** The best results are highlighted in green, and the second-best are in blue. The results of GAMA are obtained by the search models trained on MSCOCO and Flickr30K, respectively. Other results are retrieved from Hall et al. (2023). Occupation-object (OO) and occupation-participant (OP) cases denote the single-person and two-person resolution tasks, respectively. For \(\Delta_{RA}\) and Bias@K, closer to \(0\) is better. We report mean and standard deviation (\(\sigma\)) for the retrieval task across all occupations.

 

bias towards vision-language tasks. GAMA is encouraged to gain a comprehensive understanding of images during narrative generation and to rethink gender attributes and answers during answer inference. Experimental results demonstrate GAMA's superiority in both task performance and gender bias metrics over previous methods. Furthermore, we conduct ablation studies and analyze the close connection between object hallucination and gender bias. Finally, we evaluate GAMA under a zero-shot setting to showcase its generalization ability. We hope that GAMA can contribute to the future exploration of fairness in VLMs.

## Acknowledgements

We would like to thank the anonymous reviewers for their insightful feedback. This work is supported by National Key Research and Development Program of China (No. 2022YFC3302600).

## Limitations

Following previous research Wang et al. (2019); Hirota et al. (2023), we utilize a gender word list for pre-processing. Although prior studies have covered most gender words, some may still be omitted. One potential solution is to train a model to obfuscate gender information in a sentence via synthesized data.

Another consideration is the narrative generation model. Despite its generalization ability to various vision-language tasks and datasets, the model requires additional computing resources and datasets for training. Due to the boom in large VLMs (LVLMs), we will investigate the feasibility of generating gender-obfuscated narratives using these models.

Theoretically, we can replace the model for answer inference with any state-of-the-art task-specific generative model, which will explored in our future work.

## Ethics Statement

In our paper, we focus on mitigating gender bias towards vision-language tasks. Due to the inherent challenges associated with human analysis, including substantial manual effort and time investment, we leverage existing datasets and benchmarks for gender bias measurement. While quantitative metrics provide valuable insights, we acknowledge their potential limitations in capturing nuanced gender bias. Additionally, current datasets and benchmarks only consider binary gender, which oversimplifies the intricate and diverse nature of gender identity. Therefore, GAMA remains to be improved. We hope that the advancements in GAMA, as presented in this paper, will serve as a catalyst for inspiring further valuable research in gender bias mitigation towards vision-language tasks.

## References

* Agrawal et al. (2019) Harsh Agrawal, Karan Desai, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, and Peter Anderson. 2019. Nocaps: Novel Object Captioning at Scale. In _2019 IEEE/CVF International Conference on Computer Vision_, pages 8947-8956.
* Anderson et al. (2016) Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. SPICE: Semantic Propositional Image Caption Evaluation. In _European Conference on Computer Vision 2016_, pages 382-398.
* Anderson et al. (2018) Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering. In _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 6077-6086.
* Antol et al. (2015) Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question Answering. In _2015 IEEE International Conference on Computer Vision_, pages 2425-2433.
* Bender and Koller (2020) Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 5185-5198, Online. Association for Computational Linguistics.
* Bhargava and Forsyth (2019) Shruti Bhargava and David Forsyth. 2019. Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models. _arXiv preprint arXiv:1912.00578_.
* Biten et al. (2022) Ali Furkan Biten, Lluis Gomez, and Dimosthenis Karatzas. 2022. Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning. In _2022 IEEE/CVF Winter Conference on Applications of Computer Vision_, pages 2473-2482.
* Brown (1958) Roger Brown. 1958. How Shall a Thing Be Called? _Psychological review_, 65(1):14.
* Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In _Proceedings of the 37th International Conference on Machine Learning_, pages 1597-1607 

Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollar, and C Lawrence Zitnick. 2015. Microsoft COCO Captions: Data Collection and Evaluation Server. _arXiv preprint arXiv:1504.00325_.
* Cherti et al. (2023) Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Worthsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jemia Jitsev. 2023. Reproducible Scaling Laws for Contrastive Language-Image Learning. In _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 2818-2829.
* Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and others. 2022. Scaling Instruction-Finetuned Language Models. _arXiv preprint arXiv:2210.11416_.
* Denkowski and Lavie (2014) Michael Denkowski and Alon Lavie. 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In _Proceedings of the Ninth Workshop on Statistical Machine Translation_, pages 376-380, Baltimore, Maryland, USA. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and Others. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. _arXiv preprint arXiv:2010.11929_.
* Fang and Feng (2022) Qingkai Fang and Yang Feng. 2022. Neural Machine Translation with Phrase-Level Universal Visual Representations. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 5687-5698, Dublin, Ireland. Association for Computational Linguistics.
* Geyik et al. (2019) Sahim Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. 2019. Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 2221-2231.
* Hall et al. (2023) Siobhan Mackenzie Hall, Fernanda Goncalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shatdrikski, and Hannah Rose Kirk. 2023. VisoGender: A Dataset for Benchmarking Gender Bias in ImageNet Pronoun Resolution. In _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_.
* Harrison et al. (2023) Sophia Harrison, Eleonora Gualdoni, and Gemma Boleda. 2023. Run Like a Girl! Sport-Related Gender Bias in Language and Vision. In _Findings of the Association for Computational Linguistics: ACL 2023_, pages 14093-14103, Toronto, Canada. Association for Computational Linguistics.
* He et al. (2022) Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. MABEL: Attenuating Gender Bias using Textual Entailment Data. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 9681-9702, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 9726-9735.
* Hendricks et al. (2018) Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018. Women Also Snowboard: Overcoming Bias in Captioning Models. In _European Conference on Computer Vision 2018_, pages 793-811.
* Hessel et al. (2021) Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. CLIPScore: A Reference-free Evaluation Metric for Image Captioning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 7514-7528, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Hirota et al. (2022a) Yusuke Hirota, Yuta Nakashima, and Noa Garcia. 2022a. Gender and Racial Bias in Visual Question Answering Datasets. In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_, pages 1280-1292.
* Hirota et al. (2022b) Yusuke Hirota, Yuta Nakashima, and Noa Garcia. 2022b. Quantifying Societal Bias Amplification in Image Captioning. In _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 13440-13449.
* Hirota et al. (2023) Yusuke Hirota, Yuta Nakashima, and Noa Garcia. 2023. Model-Agnostic Gender Debiased Image Captioning. In _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 15191-15200.
* Janghorbanzi and De Melo (2023) Sepehr Janghorbanzi and Gerard De Melo. 2023. Multimodal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision-Language Models. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 1725-1735, Dubrovnik, Croatia. Association for Computational Linguistics.
* Jiang et al. (2020) Huaizu Jiang, Ishan Misra, Marcus Rohrbach, Erik Learned-Miller, and Xinlei Chen. 2020. In Defense of Grid Features for Visual Question Answering. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 10264-10273 

Andrej Karpathy and Li Fei-Fei. 2015. Deep Visual-Semantic Alignments for Generating Image Descriptions. _2015 IEEE Conference on Computer Vision and Pattern Recognition_, pages 3128-3137.
* Kay et al. (2015) Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_, pages 3819-3828.
* Kuznetsova et al. (2020) Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, and others. 2020. The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale. _International Journal of Computer Vision_, 128(7):1956-1981.
* Lee et al. (2018) Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. 2018. Stacked Cross Attention for Image-Text Matching. In _European Conference on Computer Vision 2018_, pages 212-228.
* Li et al. (2022a) Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, and JingBo Zhu. 2022a. On Vision Features in Multimodal Machine Translation. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 6327-6337, Dublin, Ireland. Association for Computational Linguistics.
* Li et al. (2023a) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023a. BLIP-2: Bootstraping Language-Image Pretraining with Frozen Image Encoders and Large Language Models. _arXiv preprint arXiv:2301.12597_.
* Li et al. (2021) Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before Fuse: Vision and Language Representation Learning with Momentum Distillation. In _Advances in Neural Information Processing Systems_, volume 34, pages 9694-9705.
* Li et al. (2020) Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. 2020. OSCAR: Object-Semantics Aligned Pretraining for Vision-Language Tasks. In _European Conference on Computer Vision 2020_.
* Li et al. (2022b) Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, and Junjiie Yan. 2022b. Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm. In _International Conference on Learning Representations_.
* Li et al. (2023b) Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023b. Evaluating Object Hallucination in Large Vision-Language Models. In _The 2023 Conference on Empirical Methods in Natural Language Processing_.
* Lu et al. (2018)Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2018. Neural Baby Talk. In _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 7219-7228, Los Alamitos, CA, USA. IEEE Computer Society.
* Mokady et al. (2021) Ron Mokady, Amir Hertz, and Amit H. Bermano. 2021. ClipCap: CLIP Prefix for Image Captioning. _arXiv preprint arXiv:2111.09734_.
* Mu et al. (2022) Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. 2022. SLIP: Self-Supervision Meets Language-Image Pre-Training. In _European Conference on Computer Vision 2022_, pages 529-544.
* Nguyen et al. (2022) Van-Quang Nguyen, Masanori Suganuma, and Takayuki Okatani. 2022. GRIT: Faster And Better Image Captioning Transformer Using Dual Visual Features. In _European Conference on Computer Vision 2022_, pages 167-184.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Weihang Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
* Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desnaison, Andreas Kopf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoi Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In _Proceedings of the 33rd International Conference on Neural Information Processing Systems_, volume 32.
* Qu et al. (2021) Leigang Qu, Meng Liu, Jianlong Wu, Zan Gao, and Liqiang Nie. 2021. Dynamic Modality Interaction Modeling for Image-Text Retrieval. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Informational Retrieval_, pages 1104-1113.
* Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and others. 2021. Learning Transferable Visual Models from Natural Language Supervision. In _International Conference on Machine Learning_, volume 139, pages 8748-8763.
* Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. _The Journal of Machine Learning Research_, 21(1):5485-5551.
* Rennie et al. (2017) Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-Critical Sequence Training for Image Captioning. In _2017 

IEEE Conference on Computer Vision and Pattern Recognition_, pages 1179-1195.
* Rohrbach et al. (2018) Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object Hallucination in Image Captioning. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 4035-4045, Brussels, Belgium. Association for Computational Linguistics.
* Ross et al. (2021) Candace Ross, Boris Katz, and Andrei Barbu. 2021. Measuring Social Biases in Grounded Vision and Language Embeddings. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 998-1008. Online. Association for Computational Linguistics.
* Seth et al. (2023) Ashish Seth, Mayur Hemani, and Chirag Agarwal. 2023. DeAR: Debiasing Vision-Language Models with Additive Residuals. In _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 6820-6829.
* Shaikh et al. (2023) Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, and Diyi Yang. 2023. On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 4454-4470, Toronto, Canada. Association for Computational Linguistics.
* Srinivasan and Bisk (2022) Tejas Srinivasan and Yonatan Bisk. 2022. Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models. In _Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing_, pages 77-85, Seattle, Washington. Association for Computational Linguistics.
* Steed and Caliskan (2021) Ryan Steed and Aylin Caliskan. 2021. Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_, pages 701-713.
* Tang et al. (2021) Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, Na Zou, and Xia Hu. 2021. Mitigating Gender Bias in Captioning Systems. In _Proceedings of the Web Conference 2021_, pages 633-645.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In _Advances in Neural Information Processing Systems_, volume 30.
* Vedantam et al. (2015) Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. CIDEr: Consensus-Based Image Description Evaluation. In _2015 IEEE Conference on Computer Vision and Pattern Recognition_, pages 4566-4575.
* Vinyals et al. (2015) Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and Tell: A Neural Image Caption Generator. In _2015 IEEE Conference on Computer Vision and Pattern Recognition_, pages 3156-3164.
* Voigtlaender et al. (2023) Paul Voigtlaender, Soravit Changpinyo, Jordi PontTuset, Radu Soricut, and Vittorio Ferrari. 2023. Connecting Vision and Language with Video Localized Narratives. In _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 2461-2471.
* Wang and Liu (2021) Feng Wang and Huaping Liu. 2021. Understanding the Behaviour of Contrastive Loss. In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 2495-2504.
* Wang et al. (2020) Haoran Wang, Ying Zhang, Zhong Ji, Yanwei Pang, and Lin Ma. 2020. Consensus-Aware Visual-Semantic Embedding for Image-Text Matching. In _European Conference on Computer Vision 2020_, pages 18-34, Cham. Springer International Publishing.
* Wang et al. (2021)Jialu Wang, Yang Liu, and Xin Wang. 2021. Are Gender-Neutral Queries Really Gender-Neutral? Migtigating Gender Bias in Image Search. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 1995-2008, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Wang et al. (2022) Jianfeng Wang, Zhengyan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Xicheng Liu, Ce Liu, and Lijun Wang. 2022. GIT: A Generative Image-to-text Transformer for Vision and Language. _arXiv preprint arXiv:2205.14100_.
* Wang et al. (2019) Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019. Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations. In _2019 IEEE/CVF International Conference on Computer Vision_, pages 5309-5318.
* Webster et al. (2020) Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and Reducing Gendered Correlations in Pre-Trained Models. _arXiv preprint arXiv:2010.06032_.
* Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Phu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 38-45, Online. Association for Computational Linguistics.
* Xu et al. (2015) Kelvin Xu, Jimmy Lei Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In _Proceedings of the 32nd International 

Conference on International Conference on Machine Learning_, pages 2048-2057.
* Yao et al. (2022) Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. 2022. FILIP: Fine-grained Interactive Language-Image Pre-Training. In _International Conference on Learning Representations_.
* Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From Image Descriptions to Visual Denotations: New Similarity Metrics for Semantic Inference Over Event Descriptions. _Transactions of the Association for Computational Linguistics_, 2:67-78.
* Zhang et al. (2022) Yi Zhang, Junyang Wang, and Jitao Sang. 2022. Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-Training Models. In _Proceedings of the 30th ACM International Conference on Multimedia_, pages 4996-5004. Event-place: Lisboa, Portugal.
* Zhang et al. (2023) Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal Chain-of-Thought Reasoning in Language Models. _arXiv preprint arXiv:2302.00923_.
* Zhao et al. (2021) Dora Zhao, Angelina Wang, and Olga Russakovsky. 2021. Understanding and Evaluating Racial Biases in Image Captioning. In _2021 IEEE/CVF International Conference on Computer Vision_, pages 14810-14820.
* Zhao et al. (2017) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 2979-2989, Copenhagen, Denmark. Association for Computational Linguistics.
* Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_, pages 15-20, New Orleans, Louisiana. Association for Computational Linguistics.
* Zhou et al. (2022)Kankan Zhou, Eason Lai, and Jing Jiang. 2022. VL-StereoSet: A Study of Stereotypical Bias in Pretrained Vision-Language Models. In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 527-538, Online only. Association for Computational Linguistics.
* Zhou et al. (2023)Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and Mitigating Object Hallucination in Large Vision-Language Models. _arXiv preprint arXiv:2310.00754_.
* Zhu et al. (2017)Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In _2017 IEEE International Conference on Computer Vision_, pages 2242-2251.

 

List of Gender Words

We use the gender words from Hirota et al. (2023), which are listed in Table 6.

## Appendix B Metrics

### Gender bias metrics

LicAs for LIC, we follow the method proposed by Hirota et al. (2022b).

First, we pre-process captions by masking gender words. To measure gender bias amplification in image captioning models, we need to quantify the difference between bias in the generated captions set \(\hat{\mathcal{D}}\) and bias in the ground-truth captions in the training data \(\mathcal{D}\). Then we train two gender classifiers \(f\) and \(\hat{f}\) on the two masked caption sets \(\mathcal{D}\) and \(\hat{\mathcal{D}}\), respectively. Finally, we compare the accuracy of two gender classifiers as follows:

\[\text{LIC}_{D}=\frac{1}{|\mathcal{D}|}\sum_{(y,a)\in\mathcal{D}}s_{ a}(y)\mathbbm{1}[f(y)=a], \tag{16}\] \[\text{LIC}_{M}=\frac{1}{|\hat{\mathcal{D}}|}\sum_{(\hat{y},a)\in \hat{\mathcal{D}}}\hat{s}_{a}(\hat{y})\mathbbm{1}[\hat{f}(\hat{y})=a],\] (17) \[\qquad\quad\text{LIC}=\text{LIC}_{M}-\text{LIC}_{D}, \tag{18}\]

where \(y\) and \(\hat{y}\) are the ground-truth caption and generated caption, respectively. We denote \(a\) as the gender label of the caption. \(s_{a}(\cdot)\) and \(\hat{s}_{a}(\cdot)\) are the probabilities that \(f\) and \(\hat{f}\) classify the gender of the caption as \(a\), respectively.

The higher accuracy of the classifier trained on the masked captions indicates that the context contains more information related to gender. LIC \(>0\) indicates that the model amplifies gender bias with respect to the training data, and mitigates it otherwise.

Following Hirota et al. (2023), we construct the classifiers with bert-base-uncasedDevlin et al. (2019) as the backbone and two fully connected layers with Leaky ReLU activation on top. We finetune the model for \(5\) epochs with a learning rate of \(1\times 10^{-5}\).

ErrorError represents the ratio of gender misclassification in the generated captions. Following Hendricks et al. (2018) and Hirota et al. (2023), we utilize the gender word list to identify the gender label of the generated captions as detailed in Section 4.1. And "neutral" labels are not considered as errors.

BiasAmpBiasAmp is proposed to quantify the bias amplification of the model. Following Zhao et al. (2017) and Hirota et al. (2023), we use the top \(1,000\) common words in captions and filter the words that are not strongly associated with humans, leaving a set \(\mathcal{L}\) of high-frequency words.9

Footnote 9: The word list in available at [https://github.com/uclnlp/reducingbias](https://github.com/uclnlp/reducingbias).

We calculate the bias of the word \(l\in\mathcal{L}\) on the gender \(a\in\mathcal{A}=\{m,f\}\) as follows:

\[b_{a,l}=\frac{c_{a,l}}{\sum_{a\in\mathcal{A}}c_{a,l}}, \tag{19}\] \[\hat{b}_{a,l}=\frac{\hat{c}_{a,l}}{\sum_{a\in\mathcal{A}}\hat{ c}_{a,l}}, \tag{20}\]

where \(c_{a,l}\) and \(\hat{c}_{a,l}\) are the number of co-occurrences of \(a\) and \(l\) in the training data and in the model predictions, respectively. Then bias amplification is defined as:

\[\text{BiasAmp}=\frac{1}{\mathcal{L}}\sum_{a\in\mathcal{A},l\in\mathcal{L}} \left(\hat{b}_{a,l}-b_{a,l}\right)\mathbbm{1}[b_{a,l}>\frac{1}{|\mathcal{A}|}]. \tag{21}\]

BiasAmp \(>0\) indicates that gender bias is amplified by the model, and otherwise mitigated.

Bias@KWang et al. (2021) proposed Bias@K to measure gender bias in image search by comparing the proportions of masculine and feminine images in search results. Let \(q\) denote the search query and \(\mathcal{R}^{K}_{q}\) denote the top-K retrieved image set. The gender bias of \(\mathcal{R}^{K}_{q}\) is defined as:

\[\text{Bias@K}(q)=\begin{cases}0,&\text{if $N_{m}+N_{f}=0$}\\ \frac{N_{m}-N_{f}}{N_{m}+N_{f}},&\text{otherwise}\end{cases}, \tag{22}\]

where \(N_{m}\) and \(N_{f}\) denote the number of images labeled "male" and "female" in \(\mathcal{R}^{K}_{q}\), respectively. Then Bias@K is calculated as the average of Bias@K(\(q\)) over all queries.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Gender** & \multicolumn{2}{c}{**Gender Words**} \\ \hline Female & woman, female, lay, mother, girl, aunt, wife, actress, princess, waitress, sister, queen, chairwoman, policewoman, girlfriend, pregnant, daughter, she, her, hers, herself \\ \hline Male & man, male, father, gentleman, boy, uncle, husband, actor, prince, waiter, son, brother, guy, emperor, dude, cowboy, boyfriend, chairman, policeman, he, his, him, himself \\ \hline \hline \end{tabular}
\end{table}
Table 6: Gender word list.

 

A positive Bias@K indicates a higher frequency of retrieving images featuring males compared to females. This metric is most suitable when the candidate images are gender-balanced, as it does not consider the distribution of candidates.

MaxSkew@KHall et al. (2023) measured the difference between the desired proportion of gender attributes in and the actual proportion. Let denote the gender attribute set. The skew of for the gender attribute at is defined as:

(23)

where and are desired proportion of gender attributes in the test set and the actual proportion in , respectively.

Then we calculate the maximum value of Skew@K among all gender attributes for the retrieved images:

(24)

Finally, MaxSkew@K is calculated as the average of MaxSkew@K over all queries.

NdkGeyik et al. (2019) and Hall et al. (2023) measured the distance of the prediction distribution from a fair distribution over all candidate results. Let denote the candidate set of the query . The NDKL of is defined as:

(25)

where is a normalizing factor, and denotes the KL-divergence. and denote the actual distribution of gender attributes over the top-K retrieved images and the desired distribution, respectively.

Resolution biasHall et al. (2023) defined resolution accuracy (RA) as the proportion of correctly resolved pronouns. Let denote the occupation set. We calculate the resolution accuracy of the gender attribute on the occupation as:

(26)

where denotes the total number of pronouns of the gender attribute in occupation o, and denotes the number of correctly resolved pronouns of an in .

Then the resolution bias on the occupation is defined as the gender resolution accuracy gap:

(27)

Finally, we calculate the resolution bias as the average of over all occupations. indicates that the model performs better in resolving males within occupations, and vice versa.

### Object Hallucination Metrics

DefinitionCHAIR Rohrbach et al. (2018) is a popular metric for object hallucination measurement with two variants, CHARi and CHAIRs. CHAIRi and CHAIRs evaluate object hallucination at the object and sentence levels, respectively. Let denote the number of hallucinated objects, denote the total number of mentioned objects, denote the number of captions with hallucinated objects, and denote the total number of captions. CHAIRi and CHAIRs are defined as:

(28) (29)

CHAIR describes the proportion of hallucinated objects among all generated objects. CHAIRs describes the proportion of generated captions with hallucinated objects.

DetailsRohrbach et al. (2018) utilized a synonym list Lu et al. (2018) to map words to MSCOCO objects (e.g., "player" to "person"). However, it is notable that the list is coarse-grained. For instance, it considers terms like "woman" and "man" as well as "purse" and "briefcase" as synonyms. Although "woman" and "man" both fall under the"person" class, and "purse" and "briefcase" belong to the category of bags, it is crucial to recognize that they represent distinct objects. Therefore, we refine the list with hierarchical object relationships among objects based on the fine-grained classes defined in NoCaps Agrawal et al. (2019). If a word in a sub-category (e.g., "woman") is predicted in its super-category (e.g., "person"), we do not consider it as a hallucinated object. Conversely, if a word in a category (e.g., "woman") is predicted in its sibling category (e.g., "man"), we consider it as a hallucinated object 



### Hit Ratio

Li et al. (2023) utilized the hit ratio (HR\({}_{C}^{o}\)) to measure the object co-occurrence in object hallucination. They demonstrated that VLMs mostly hallucinate objects that frequently co-occur with ground-truth objects in the image.

Let \(\mathcal{H}_{i}\) denote the set of hallucinated objects in the \(i\)-th image, and \(\mathcal{C}_{o}\) denote the set of the top-K frequently co-occurring objects with \(\hat{o}\) in the training set. The top-K hit ratio of the probing object \(\hat{o}\) is defined as:

\[\text{HR}_{C}^{o}@\text{K}(\hat{o})=\frac{1}{M_{\hat{o}}}\sum_{i=1}^{M_{\hat{ o}}}\frac{|\mathcal{H}_{i}\cap\mathcal{C}_{\hat{o}}|}{|\mathcal{H}_{i}|}, \tag{30}\]

where \(M_{\hat{o}}\) is the total number of images containing \(\hat{o}\).

Similarly, we define the top-K hit ratio of the probing gender \(a\) as:

\[\text{HR}_{C}^{g}@\text{K}(a)=\frac{1}{M_{a}}\sum_{i=1}^{M_{a}}\frac{| \mathcal{H}_{i}\cap\mathcal{C}_{a}|}{|\mathcal{H}_{i}|}, \tag{31}\]

where \(M_{a}\) is the total number of images with the gender label \(a\), and \(\mathcal{C}_{a}\) denotes the set of the top-K frequently co-occurring objects with the gender \(a\).

## Appendix C Implementation Details

We select the best models based on the loss on the validation set for all tasks. Following Hendricks et al. (2018) and Tang et al. (2021), we set the beam size as \(5\) during inference. Table 7 shows our hyper-parameters for training in different tasks. Table 8 lists the input and output formats of GAMA.

We implement GAMA with PyTorch Paszke et al. (2019) and Huggingface Transformers Wolf et al. (2020). We train the models with a single NVIDIA Ge-Force RTX 4090 GPU. More implementation details can be found in our code.

Image searchWith a probability of \(50\)%, we randomly sample the caption of another training image to create a negative pair for training. The model predicts answers with "yes" or "no" as shown in Table 8. We take the token probability of the decoder as the match score.

VisoGenderIn VisoGender, each occupation appears with two caption templates: a person with a possessive pronoun to an object (e.g., "the doctor and his/her stethoscope") and a person with a possessive pronoun to a participant (e.g., "the doctor and his/her patient"). We evaluate the performance of the trained GAMA search models on MSCOCO and Flickr30K without additional training, respectively. Our experiments on VisoGender follow the original setting proposed by Hall et al. (2023). In the resolution task, we provide the model with queries containing "his" and "her", respectively. We determine the answers based on the match scores. In the retrieval task, we use neutral queries incorporating the pronoun "their" and provide the model with candidate images featuring correct occupations with balanced gender distributions.

## Appendix D Additional Experiments

### Experiments on VL-Bias

We conduct experiments on VL-Bias Zhang et al. (2022) under a zero-shot setting.

DatasetVL-Bias serves as a benchmark for measuring gender bias in VLMs, encompassing \(52\) activities and \(13\) occupations related to humans.10 The dataset includes images sourced from the Internet, as well as existing image datasets such as MSCOCO Chen et al. (2015) and Flickr30K Young et al. (2014). The captions are in the format of "The {gender} is {target}", where "target" represents an activity (e.g., "shopping") or an occupation (e.g., "engineer"). In total, VL-Bias comprises \(24\)K image-text pairs, including \(13\)K pairs for the \(52\) activities and \(11\)K for the \(13\) occupations.

Footnote 10: The dataset is available at [https://github.com/VL-Bias/VL-Bias](https://github.com/VL-Bias/VL-Bias).

BaselinesFollowing Zhang et al. (2022), we evaluate GAMA against three common and effective debiasing methods: 1) Gender Swapping (**GS**) Zhao et al. (2018), which swaps gender words in the input text to mitigate the effect of gender attributes, and is adapted to vision-language tasks based on CycleGAN Zhu et al. (2017); 2) Dropout Regularization (**DR**) Webster et al. (2020), which is designed to prevent model from overfitting to gender attributes by increasing the dropout rate; and 3) **FairVLP**Zhang et al. (2022), which trains the PVLM ALBEf Li et al. (2021) with counterfactual samples to obtain debiased representations.

MetricsZhang et al. (2022) proposed the vision-language bias for VL-Bias. Let \(t\) denote a target word, and \(a\) denote the gender attribute. The vision-language bias on \(t\) towards \(a\) on an image-text pair

\(s\) can be formulated as:

\[B_{s}(t,a)=\frac{P_{T}(t|s_{c})-P_{T}(t|s)}{P_{A}(a|s_{c})-P_{A}(a| s)}, \tag{32}\]

where \(s_{c}\) is the counterfactual image-text pair of \(s\). \(P_{T}(t|s)\) denotes the probability that the model predicts the masked target word as \(t\), and \(P_{A}(a|s)\) denotes the probability that the model predicts the masked gender word as \(a\).

Let \(\mathcal{A}=\{m,f\}\) denote the gender attribute set, and \(S_{t}\) denote all image-text pairs with the target \(t\). The gender bias over the target \(t\) is defined as:

\[B_{VL}(t)=\frac{1}{|S_{l}|}\sum_{s\in S_{t}}B_{s}(t,m)-B_{s}(t,f). \tag{33}\]

A positive \(B_{VL}(t)\) indicates that the target \(t\) is biased towards males, and vice versa. The gender bias of the dataset is calculated as the average of \(B_{VL}(t)\) over all targets.

Implementation detailsAs the baseline models are pre-trained on MSCOCO Chen et al. (2015), we utilize the GAMA search model trained on MSCOCO for evaluation. We take the token probability of the decoder as the probability for a target or a gender attribute. Counterfactual texts are constructed by reversing the gender words, while counterfactual images are generated with an adversarial attack based on the official implementation of Zhang et al. (2022).

Experimental resultsAs shown in Table 9, GAMA obtains remarkable debiasing performance against baselines. The results underscore its generalization ability as well as its effectiveness in gender bias mitigation.

### Further study

In this section, we analyze the impact of the temperature hyper-parameter, the data size of Localized Narratives Voigtlaender et al. (2023), and the frozen parameters.

TemperatureWe show the results in Table 10. The contrastive loss with a larger temperature is less sensitive to the hard negative samples as discussed in prior work Wang and Liu (2021). Therefore, it is hard for GAMA with a large temperature to distinguish gender-related features from gender-masked features, leading to increased LIC and BiasAmp scores.

Data sizeAlthough we have demonstrated the effectiveness of the model without narrative generation in Section 5.2, we wonder about the influence of the data size on the results. Therefore, we conduct an ablation study on the training set size of

\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Hyper-parameters**} & \multirow{2}{*}{**Narrative Generation**} & \multicolumn{2}{c}{**Image Captioning**} \\ \cline{4-6}  & & & **MSCOO** & **Flickr30K** \\ \hline Hidden Dimension \(d\) & 768 & 768 & 768 & 768 \\ Maximum Epochs & 15 & 10 & 5 & 5 \\ Learning Rate & \(4\times 10^{-5}\) & \(1\times 10^{-4}\) & \(2\times 10^{-5}\) & \(3\times 10^{-5}\) \\ Weight Decay & 0.01 & 0.01 & 0.01 & 0.01 \\ Batch Size & 20 & 24 & 32 & 32 \\ Maximum Input Sequence Length & 128 & 256 & 256 & 256 \\ Maximum Output Sequence Length & 128 & 64 & 3 & 3 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Hyper-parameter settings of GAMA.

\begin{table}
\begin{tabular}{c l c} \hline \hline
**Model** & **Activity 13K** & **Occupation 11K** \\ \hline GS & 11.21 & 12.47 \\ DR & 11.17 & 13.52 \\ FairVLP & 6.97 & 7.74 \\ \hline
**GAMA** & 5.96 & 6.83 \\ \hline \hline \end{tabular}
\end{table}
Table 8: **The input and output formats of GAMA. For image search, we take the token probability of the decoder as the match score. For the second stage, model-generated narratives are utilized instead of relying on ground-truth narratives from datasets.**

\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Activity 13K** & **Occupation 11K** \\ \hline GS & 11.21 & 12.47 \\ DR & 11.17 & 13.52 \\ FairVLP & 6.97 & 7.74 \\ \hline
**GAMA** & 5.96 & 6.83 \\ \hline \hline \end{tabular}
\end{table}
Table 9: **Results on VL-Blas. The best results are highlighted in green. For the results, closer to \(0\) is better. The baseline results are retrieved from Zhang et al. (2022).** 

Localized Narratives Voigtlaender et al. (2023) for narrative generation.

As our objective is to alleviate gender bias, we randomly select the images categorized under the "Person" class for experiments. Results are illustrated in Table 11 and Table 12.

We observe that the data size seems to have little impact on GAMA's task performance and its gender bias mitigation ability in image captioning. However, the data size affects the generalization ability of GAMA on VisoCender. We consider that it is because a large training set enhances the zero-shot generalization ability of GAMA in narrative generation, thereby ensuring robust model performance during answer inference.

ParametersTo minimize the cost of training the model, we investigate the feasibility of reducing additional training. We experiment with freezing a portion of the parameters in GAMA, and the results are summarized in Table 13 below.

Freezing the T5 encoder in GAMA leads to results that are only slightly inferior to those of unfrozen GAMA, which is an encouraging finding. However, the outcomes of freezing both the flanT5 encoder and decoder are less promising. We think it is because the fusion of language features and vision features creates a new feature space, which is distinct from the original features learned by T5. Consequently, the frozen decoder cannot be effectively adapted to this new feature space.

\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Temperature**} & \multicolumn{4}{l}{**Gender Bias Metrics\(\downarrow\)**} & \multicolumn{4}{l}{**Image Captioning Metrics \(\uparrow\)**} \\ \cline{2-9}  & **LIC** & **Error** & **BiasAmp** & **BLEU-4** & **CIDEr** & **METeor** & **SPICE** & **CLIPScore** \\ \hline
0.01 & -1.4 & 3.5 & -3.42 & 37.9 & 115.2 & 30.9 & 22.5 & 75.5 \\
0.1 & -1.1 & 3.4 & -3.40 & 38.2 & 115.1 & 31.0 & 22.7 & 75.4 \\
1 & -0.7 & 3.3 & -3.28 & 37.7 & 114.6 & 30.6 & 22.4 & 75.6 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Results of the ablation study on the temperature hyper-parameter.

 



\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Gender Bias Metrics \(\downarrow\)**} & \multicolumn{6}{c}{**Image Captioning Metrics \(\uparrow\)**} \\ \cline{2-10}
**Data Size** & **LIC** & **Error** & **BiasAmp** & **BLEU-4** & **CIDEr** & **METEOR** & **SPICE** & **CLIPScore** \\ \hline GAMA & -1.1 & 3.4 & -3.40 & 38.2 & 115.1 & 31.0 & 22.7 & 75.4 \\ \hline Frozen encoder & -1.2 & 3.4 & -3.35 & 38.0 & 114.6 & 30.7 & 22.2 & 75.1 \\ Frozen encoder \& decoder & -0.6 & 3.7 & -2.79 & 37.1 & 112.5 & 29.8 & 21.6 & 74.4 \\ w/o GO Frozen encoder & 0.4 & 3.2 & -1.12 & 37.8 & 114.2 & 30.3 & 22.1 & 74.9 \\ \hline \hline \end{tabular}
\end{table}
Table 13: **Results of partial parameters frozen. We freeze the T5 encoder in GAMA, both the T5 encoder and decoder in GAMA, and the T5 encoder in GAMA w/o GO, respectively.**