Gender Biases in Automatic Evaluation Metrics for Image Captioning
Authors: Haoyi Qiu
Zi-Yi Dou
Tianlu Wang
Asli Celikyilmaz
Nanyun Peng
Venue: EMNLP 2023 Main (2023)
Keywords: Automatic Evaluation Metrics
Fairness

Abstract: Model-based evaluation metrics (e.g., CLIPScore and GPTScore) have demonstrated decent correlations with human judgments in various language generation tasks. However, their impact on fairness remains largely unexplored. It is widely recognized that pretrained models can inadvertently encode societal biases, thus employing these models for evaluation purposes may inadvertently perpetuate and amplify biases. For example, an evaluation metric may favor the caption “a woman is calculating an account book” over “a man is calculating an account book,” even if the image only shows male accountants. In this paper, we conduct a systematic study of gender biases in model-based automatic evaluation metrics for image captioning tasks. We start by curating a dataset comprising profession, activity, and object concepts associated with stereotypical gender associations. Then, we demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations, as well as the propagation of biases to generation models through reinforcement learning. Finally, we present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments. Our dataset and framework lay the foundation for understanding the potential harm of model-based evaluation metrics, and facilitate future works to develop more inclusive evaluation metrics.

Gender Biases in Automatic Evaluation Metrics for Image Captioning
Haoyi Qiu', Zi-Yi Dou', Tianlu Wang Asli Celikyilmaz', Nanyun Peng
*University of California, Los Angeles "Meta AI Research
(haoyiqiu, zdou, htwp:/iiuuzzuuuiilettteeeeee violetpeng ucla. edu (tianluwang, as (elliccemet.cccommm com
Abstract
Model-based evaluation metrics (e.g., CLIP- Score and GPTScore) have demonstrated de- correlations with human judgments in ious language generation tasks. However, their impact on faireess remains largely unexplored. It is widely recognized that pretrained mod- els can inadvertently encode societal biases, thus employing these models for evaluation purposes may inadvertently perpetuate and am- plify biases. For example, an evaluation metric may favor the caption woman is calculating account book" over "a man is calculating an account book," even the image only shows male accountants. In this paper, conduct a systematic study of gender blases in model- based automatic evaluation metries for image captioning tasks. We start by curating dataset comprising profession, activity, and object con- cepts associated with stereotypical gender as- sociations. Then, demonstrate the negative consequences of using these biased metrics, in- cluding the inability differentiate between biased and unbiased generations, well as the propagation of biases to generation models through reinforcement learning. Finally, present simple and effective way to mitigate the metric bias without hurting the correlations with human judgments. Our dataset and frame- work lay the foundation for understanding the potential harm of model-based evaluation met- rics, and facilitate future works develop more inclusive evaluation metrics.'
Referencc: photo reading
Image Captioning Madels
evalunte
mam reading
0.74
CLIPScore
Good
reading
0.72
Bad caption: a man who is reading
Figure 1: An image-caption pair example from the PAO- EVALBIAS dataset. A good caption accurately describes the gender of the main character in the image, while the bad caption incorrectly describes the gender. CLIPScore can assign higher score to the caption that is incorrect (0.75 vs. 0.72 correct), which shows that there is bias encoded in the evaluation metric. Furthermore, utilizing the biased evaluation metrics generation tasks might initiate the biased models be favored.
and CIDEr (Vedantam et al., 2015) across various generation tasks. Instead of merely measuring the surface-level overlap between references and gen- eration outputs, model-based metries similarities on the semantic level and thus provide more accurate estimations of the model quality.
capture
Despite the promising results, it is widely recog- nized that pretrained models encode societal biases, including but not limited to gender, racial, and reli- gious biases (Kurita al., 2019; Sheng et al., 2019; Agarwal et al., 2021: Nangia et al., 2020; Barikeri et al., 2021; Cho et al., 2022; Zhang et al., 2022; Wan et al., 2023). Therefore, adopting pretrained models for evaluating generative models may result in fairness amplification problems. For example, one potential issuc is that biased generative mod- els may be rewarded and selected because specific sensitive attributes (e.g., gender) are favored by bi- ased model-based evaluation metrics. Moreover, when using such evaluation metrics in reinforce- ment learning from AT feedback (RLAIF), there
1 Introduction
Pretrained model-based evaluation metrics such as BERTScore (Zhang et al., 2019), CLIPScore (Hes- sel et al., 2021), and Gcore (Fu al., 2023) have shown promising performance, achieving stronger correlations with human judgments over n-gram matching-basee evaluation metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004),
Data availahle at ht thub. com/PlusLabbLLP/ clipscore-bixe

---

is a potential risk of further amplifying these bi- ases in the models. There are few prior works that have pointed out issues regarding language- only evaluation metrics (Hanna and Bojar, 2021; Pu et al., 2021). Regarding fairness, Sun et al. (2022) constructed a dataset based on WinoBias (Zhao et al.. 2018) and systematically investigated dif- ferent metrics. However, they focus synthetic model generations and failed to analyze the impli- cations and harm of biased metrics in worldw scenarios. As a results, it is hard to draw insights from their works in terms of practical applications. Moreover, they leave out studies of biases encoded in cross-modal evaluation metrics such as CLIP- Score. As we see increase in the variety of multimodal generation tasks such as image cap- tioning and multimodal summarization (Liu et al., 2023; Zhu et al., 2023), it is crucial evaluate the cross-modal metrics specifically designed for these tasks.
models through reinforcement learning;
A simple and effective hybrid similarity eval- uation metric by linearly combining R-gram matching-baser and pretrained model-based metries, which can effectively reduce gender biases, while maintaining a strong correlation with human judgments.
2
Bias Evaluation for Evaluation Metrics
We aim to identify and quantify potential gender biases in evaluation metrics for language genera- tion models. To do this, we first gather dataset in Section 2.1. Then, we formally define gender biases and conduct a comprehensive analysis of image captioning evaluation metrics on our dataset in Section 2.2.
2.1 Dataset Construction
Using the lexicons created by previous work (Cho et al., 2022; Bansal et al., 2022: Zhang et al., 2022), we collect images of people with vari- ous professions, activities, and objects (PAO- EVALBIAS). For each concept in the lexicons, we use templates to construct one reference as well as two candidates containing the correct and incor- rect gender, denoted as the good and bad captions respectively. The specific caption patterns are de- scribed in Table 1. Our approach involves pairing a gender from protected groups (man or woman) with a concept in professions, activities, or objects. As shown in Figure 1, for the pair (woman, reading), we have the reference "a photo of woman who is reading", and use the good caption "a woman who is reading" obtain suitable images via image re- trieval. Meanwhile, the bad caption is "a man who reading."
In this paper, we perform a systematic study of gender biases in cross-modal generation evaluation metries using image captioning tasks. Following previous research (Hendricks et al., 2018), we clas- sify gender expression instead of biological sex gender identity. We limit our analysis to two genders (man and woman) in this study, but it is important to note that gender non-binary. We acknowledge this limitation and refer readers to the ethics statement section for a more in-depth discussion on this topic.
For the study, we first collect large-scale dataset, PAO-EVALBIAS, consisting of 92,049 im- ages of people of 88 professions, in 52 activities, and with 39 objects. Figure 1 provides an image- caption pair example from the dataset. Then, we use the proposed dataset to analyze potential gender biases in automatic evaluation metrics, and how bi- ased evaluation metrics can affect generation mod- els through reinforcement learning. We also pro- pose a simple method that combines moodel-based and n-gram matching-baan based evaluation metrics to reduce gender biases, while maintaining high corre- lations with human judgments for generation qual- ity. The highlights of our findings include:
Specifically, we retrieve images from the web using Bing, Google Image Search, and Pexels API with good captions. 250 images for each gender and concept pair were retrieved and irrelevant im- ages were manually filtered following the criteria discussed later. We carefully follow the Creative Common license and gather images without wa- termark protection, sourced from image collection websites instead of social media, and used non- commercially.
Pretrained model-based evaluation metrics cannot distinguish between biased and unbi- ased outputs, underperforming the statistical metrics in this regard:
Besides, we integrate the VL-Bias dataset from Zhang al. (2022) enrich our data collection,
The biases encoded in the model-based met- rics can be propagated to image captioning
"The data descrihed here was accessed, collected, and used only by the co-authors at UCLA.

---

Profession
Activity
Object
Reference Good caption Bad caption
photo man who is an editor
editor
photo of cooking
photo
man who is cooking
man with a a woman with a cat
woman who is an editor
a woman who is cooking
Reference Good caption Bad caption
photo a woman who is dentist woman who is a dentist
photo a woman who is drinking woman who is drinking man who is drinking
photo of a woman with a salad a woman with a salad a man with salad
man who is dentist
Figure 2: Example instances from PAO-EVALBIAS Candidate and reference captions follow specific patterns described in Table The lexicon word is highlighted in bold in the reference caption, while the gender identification word is in blue for a good caption and in red for bad caption. A good caption maintains the same gender the reference sentence, while a bad caption replaces the gender the good caption with an incorrect gender. For example, in the image located the top left corner featuring male editor, the good caption reads "a man who is editor," while the bad caption replaces "man" with "woman".
especially for the activity category. We also tract the images including the object lexicons from MSCOCO (Lin et al., 2014). More specifically, we select the appropriate images by utilizing an- notations to determine whether an image depicts a person of a specific gender engaged profession an activity or is accompanied by an object from the lexicons.
can be valuable resource for future research. Fig- ure shows six examples from PAO-EVALBIAS.
2.2 Evaluation Metrics Performance Analysis
We then evaluate five и-gram matching-basee evaluation metrics (BLEU-4, METEOR, ROUGE, CIDEr, and SPICE) and one model-based met- ric (CLIPScore) on the PAO-EVALBIAS dataset, where CLIPScore et al., 2021) to compute the image-caption simi- larity and treat it as the evaluation score. These metrics are commonly used in image-captioning tasks evaluation as they showed a good correlation with human judgments.
the CLIP model (Radford
Data Cleaning. After collecting all the candidate images, we use the filtering criteria as follows to remove the images if: (1) the content of the image does not reflect the good caption; (2) already ex- ists in the dataset. Two annotators were employed for the manual filtering process. Specifically, an- notators first filtered on the same 100 images ran- domly selected from the dataset, where the agree- achieved Cohen
Gender Bias Definition. der bias present in these evaluation metrics, we calculate the performance discrepancy between dif- ferent protected groups (men and women). More specifically, we first compute the evaluation metrics scores for good and bad captions for every image in the dataset and then measure the average accu- racy of each metric in differentiating good and bad captions of each gender per concept:
measure the gen-
0.917. Based this, the remaining images only have one annotator to examine and filter out the irrelevant images.
Statistics. We collect 92,049 images for PAO- EVALBIAS including 88 professions, 52 activities, and 39 objects. Detailed statistics of each profes- sion, activity, and object concept are listed in Ap- pendix Tables 11, 12, and 13. We observe that most concepts contain over 150 images, ensuring that our analysis results are reliable and we believe
Acca.G
Σ 1=1
good
I. > S(c. r., I:),
N
(1)

---

Candidate Captions
Reference Caption
profession
[gender] who is a/an (profession)
photo of (gender) who is a/an [profession]
activity
a (gender) who is (activity) (gender) with a/an (object)
photo of a (gender) who is [activity] photo of [gender) with a/an (object)
object
Table I: Caption patterns PAO-EVAL.BASS. The lexicons of profession, activity, and object are presented Appendix Table 11, 12, and 13. Good and bad candidate captions have the same and different gender with the reference, respectively. gender & (man, woman) ), profession, activity, object E |lexicons from Tables 11, 12, and 13). For the pair (woman, reading), the reference is "a photo of a woman who is reading" and the good caption woman who is reading" which is used to retrieve suitable images. The bad caption will be "a man who is reading."
where G denotes a gender group, denotes a concept, N denotes the total number of exam- ples for the specific concept of the gender, S de- notes the scoring function, @good/ad denotes the good/bad (candidate) caption, r denotes the refer- ence sentences set, and / denotes the correspond- ing image. For text-only evaluation metrics (e.g., BLEU-4, METEOR, ROUGE, CIDEr, and SPICE), the scoring function takes candidate and reference sentences. For image-text evaluation metrics (e.g., CLIPScore), the scoring function takes the candi- date sentences and corresponding images.
N-Gram Metrics CLIPScore CLIPScore+CIDE:
Profession Activity Object
0.00
51.76
0.00
0.00
61.54
0.00
0.00
51.28
0.00
Overall
0.00
54.86
0.00
Table 2: Percentages concepts PAO-EvAL BIAS that are biased. N-gram metrics include BLEU-4, ME- TEOR, ROUGE, CIDEr, and SPICE, separately, CLIP- Score exhibits gender biases on over 50% of lexicons, while n-gram evaluation metries do not reveal any gen- der bias. The linear combination of CLIPScore and CIDEr scores (CLIPScor+CCIDErr can alleviate the gen- der biases encoded in CLIPScore.
A bias is present if there are significant (p < 0.05 with bootstrap resampling) differences in the accuracy of the evaluation metric between different groups. We define this as the bias of the model for a specific concept. Thus, concept considered:
is the n-gram matching evaluation metries will al- ways assign higher scores to good captions than to bad ones. For example, for a good candidate caption "a woman who is a doctor" (Cy), a bad cap- tion "a man who is doctor" (cb), and a reference sentence "a photo of woman who is a doctor" (S), CIDEr(Cy, S) 0.6065 > CIDEr(es, S) 0.3259. Thus, the first column of Table shows 0% biases for all n-gram metrics (BLEU-4, ME- TEOR, ROUGE, CIDEr, and SPICE). Moreover, we investigate the linear combination CLIPScore and CIDEr scores, which has shown to be an ef- fective method in reducing gender biases present in CLIPScore, shown in the last column. This discovery inspires to propose hybrid metric detailed in Section 4.
woman-biased: the accuracy for woman examples is significantly higher than that for man examples, i.e.,
Accwoman, concpet Accman, concpeti
man-biased: if the accuracy for man examples is significantly higher than that for woman examples, i.e.
Aceman, concpet AcCwoman, comcpet,
(3)
where >> represents the result on the left is signifi-
cantly (p < 0.05 with bootstrap resampling) higher than the right.
Figure 3 and Appendix Figure 5, 6 visualize the concepts under CLIPScore evaluation. We can see that words like washing, necklace, and makeup artist are significantly woman- woman-biased, while pray- ing, miner, and basketball are man-biased. Fur- thermore, some biased words are much more dis- persed from the diagonal (neutral words) presented in these figures. Words like washing in activity necklace in the object, and makeup artist in the profession have much higher woman CLIPScore accuracy than man. Similarly, praying in activ-
Biases Revealed by PAO-EVALBIAS. As shown in Table 2, 51.76%, 61.54%, and 51.28% of the lexicons are significantly biased (p < 0.05 with bootstrap resampling) under CLIPScore eval- uation within profession, activity, and object, re- spectively. The lexical overlaps between candidate and reference captions ensure the n-gram evalu- ation metrics do not reveal any gender bias, that
"A detailed discussion about the dataset robustness can he found in Appendix A.

---

10a
100
man basedd
neuiral
jupping
graying
justing
woman blssed
Aciiiily Nar CUrS.ome Accuracy
Shopping
Har Activto Captenin Ereer Rats
pitchin
Foughing
panting
flenning
jalknng
paking gleaning
dowing shooping parseb
jiting
pitting
85
jasshing
pastingpooking
80
Juggeon
joughing
jacking
shoueni
clmbing
crying
ushing
gerving
paking
biking
85
30
95
100
10'
10"
Activity Woman CLISScrre Accuracy
Veman Activity Captloiing Erne Rate
Figure 3: Gender biases under the activity category in CLIPScore evaluation: Blue points are man-biased and green points are woman-biased Points orange have p-value greater than 0.05 with bootstrap resampling.
Figure Gender biases under the activity category of FIBER: Blue points are man- man-biased and green points are woman-biased. Points orange have p-value greater than 0.05 with bootstrap resampling.
PAO-EVALBIAS
COCO
ity, miner in profession, and basketball have much higher man CLIPScore accuracy than woman.
CLIP-S
Gender (rr(())
CIDER
CLIP-S Gender Err(%)
MLE 69.7 RI.
6.3
128.6
75.4
1.4
72.7*
6.8"
130.9*
77.6"
3 Impact on Generation Models
Table 3: RL can improve the model generation perfor- mance PAO-EVALBIAS and COCO. However, the use of CLIPScore as the reward can lead to gender pre- diction errors, which increases bias the generated output. indicates significant differences between MLE and RI. (p < 0.05 with bootstrap resampling).
Because the model-based evaluation metric con- tains biases, we posit that these biases may lead to severe consequences in real-world applications. To test this, we experiment with FIBER (Dou et al., 2022), strong image captioning model pretrained on 10M image-caption pairs and then finetuned on the COCO captioning Karpathy-spli data (Lin et al., 2014; Karpathy and Fei-Fei, 2015). Our goal is to examine the impact of gender biases pre-encoded in evaluation metrics on generation models. Specifically, we reveal that the existing image-captioning models contain gender biases, and using biased model-based metrics will make this kind of biased model more favorable over less- bias ones (more details in Section 3.1). Based on these findings, we further investigate whether us- ing a biased metric as a reward may amplify biases in both the generation model and evaluation met- ric under the reinforcement learning setting (more details in Section 3.2).
based metries like CLIPScore for evaluation may make it hard to distinguish between biased and unbiased model generations and even lead to biased models being favored over bisseb ones. In this section, verify if this hypothesis is true under controlled study.
3.1.1 Biases Captioning Models
We first find out if captioning models pre-encode biases in our setting. To this end, perform infer- ence on our PAO-EVALBIAS dataset with FIBER and analyze the gender prediction errors of the gen- erated captions following Hendricks al. (2018). Due to the caption design, we ensure that there is always one main character with a corresponding concept inside each image, and therefore, no further labeling work is needed. We analyze if an image captioning model accurately predicts the gender of an image by searching for gender-related words in the captions. We find that FIBER makes gender prediction errors 6.3% of the time (Table 3) and exhibits significant biases (i.e., there a significant gap between the gender prediction errors of man
3.1 Favoring Biased Models
It has been pointed that there exist societal biases in image captioning models (Hendricks et al., 2018), and we need to carefully calibrate them in real-world applications. However, using model-
'CIDEr needs reference captions whereas our constructed dataset does not have human annotated captions. Therefore, we do not measure model performance with CIDEr on PAO-
EVALBIAS.

---

CLIPScore- Value
CLIPScore- Win
Profession
Activity
Object
All
Profession
Activity
Object
All
Biased-FIBER 65.3
67.4
65.4
66.2
54.8
55,9
39.7
53.7
Debiascd-FIBER 65.4
66.8
67.8
66.2
45.2
44.1
60.3
46.3
Table 4: CLIPScore evaluation of biased and debiased models on PAO-EVALBIAS. "CLIP'Score- -ellPeecrr--aluee denotes the specific numerical values calculated by CLIPScore and "CLIPScore-WV Winoor denotes the percentage of times a model favored by CLIPScore over all instances. CLIPScore favors the biased FIBER in 53.7% of the images in PAO-EVALBIAS. Overall, CLIPScore cannot distinguish between biased and debiased model generations.
BLEU-4
METEOR
ROUGE
CIDEr
SPICE
CLIPScore
CLIPScore+CIDEr
Biased-FIBER 35.3
27.4
56.3
132.2
19.3
76.3
208.5
Debiased-FIin -IBIER
47.0
31.2
61.5
147.0
24.5
76.2
223.2
Table 5: Evaluation scores of biased and debiased models COCO. Biases in the evaluation metric make biased and debiased models indistinguishable based on evaluation scores. However, it-gram matching metrics can hardly encode biases and CLIPScore+CIDDEr can alleviate the bias issue.
and woman images) over 58.6% of the words in our lexicon, including 60.0%, 57.7%, 56.4% of the profession, activity, and object words, respectively. This result indicates that existing stereotypes in the profession between protected groups still signifi- cantly challenge the generation models compared to other concepts. Visualizations are provided in Figure and Appendix Figures 7, 8.
crated captions to obtain debiased FIBER model in specific setting where we only consider the words "man" and "woman". Specifically, if im- age of woman is captioned with only the word "man" and no femalc-associated words from a lexi- con defined by Hendricks al. (2018), we change "man" to "woman". Similarly, we change "woman" to "man" for images of men. The clean captions are used as the generated captions of the debiased FIBER model. It should be noted that this rule- based method only applies in these limited sce- narios, and we exclude the sentences where the method cannot be applied for our analysis purpose.
We also perform the same analysis on COCO Karpathy test set, as has been widely used in previous image captioning work. Specifically, we use ground-truth captions to determine if an image contains a man or a woman, and we use the male and female lexicons in Hendricks et al. (2018). If at least one reference caption of an image contains a "female" word such "woman" and no captions have "male" words such "man" in them, label the image as "woman". Similarly, we label the im- age as "man" using the same principle. We do not consider images where both "male" and "female" words are mentioned. After labeling, we analyze if an image captioning model accurately predicts the gender of an image by searching for the gender- related words in the captions, which is the same as dataset.
3.1.3 Evaluating Models and Results
We compute the CLIPScore for both biased and debiased FIBER on PAO-EVALBBAA and COCO. For PAO-EVALBIAS, we calculate two scores: CLIPScore- Value denotes the specific numerical values calculated by CLIPScore and CLIPScore- Win denotes the percentage of times a model is favored by CLIPScore over all instances. Table 4 shows the experiment results and we notice that (1) CL CLIPScore metric favors biased captions 53.7% of cases, and (2) overall, CLIPScore cannot distin- guish between biased and debiased model genera- tions. This is concerning and highlights the need to debias evaluation metrics prevent biased models from being used in real-world applications. Table 5 shows the experiment results on COCO, which ex- hibits similar trends on PAO-EVALB and thus further strengthens the statement.
the method applied on the To ensure the accuracy of our analysis, we also manually check each of the generations and make sure that they are indeed biased. Table 3 shows that FIBER can still make gender prediction errors on COCO with an error rate of 1.4%.
3.1.2 Error Correction
We use a rule-based method to correct errors in the FIBER model's gender predictions its gen-

---

BLEU-4 (t) METFOR (1)
ROUGE (1)
CIDEr (*)
SPICE (1)
CLIPScooe (1)
Gender Frror (L)
MLE
38.9
30.4
59.3
128.6
23.2
75.4
1.4
RL-CLIPScore
39.4
30.4
59,4
130.9
23.8
77.6
1.6
RL-CIDEr
42.7
30.9
61.4
142.2
24.1
75.3
1.2
RL- BL-CCLPScon++CIDEr
43.2
31.3
61.7
143.4
24.6
76.6
1.3
Table 6: Evaluation results of MLE and RL models on COCO. Using biased metric reward can amplify the gender biases encoded in the evaluation metric in the generation model under the RI. setting. Combining CLIPScore and CIDEr can alleviate the negative outcome, while maintaining good generation performance.
3.2 Bias Propagation through RL
PAO-EVALBIAS and from 75,4 to 77.6 on COCO. However, the use of CLIPScore as the reward can lead to gender prediction errors, which increases bias in the generated output. Notably, the gender prediction error rates rise significantly (p 0.05 with bootstrap resampling) from 6.3% (CI: [6.07%, 6.52% J) to 6.8% (CI: [6.58% 7.03%J on PAO- EVALBIAS and from 1.4% (CI: [0.92%, 1.88%)) to 1.6% (CI: [1. 12,2% 2.08% on COCO. Furthermore, the optimized model exhibits biases on 61.3% of the words on PAO-EVAI.BAAS an increase from 58.6% prior to RL. These findings highlight that using biased metrics for model evaluation can prop- agate gender biases to generation models, leading to negative outcomes.
As previously demonstrated, the existing image- captioning models contain gender biases, and us- ing biased model-based metrics will make this kind of biased model favored over less-bias ones, we investigate whether using biased metric re- ward may amplify biases in both the generation model and evaluation metric under the reinforce- ment learning (RL) setting. RL using evaluation metric scores as rewards can improve language gen- eration and reduce error propagation (Shen et al., 2016; Rennie et al., 2017; Paulus et al., 2018), and optimizing towards model-based scores is more ef- fective than n-gram-matching scores (Wieting et al.. 2019; Li et al., 2019). However, the use of biased metric reward may reinforce biases in both the generation model and evaluation metric. Therefore, is critical to investigate the impact of optimizing towards CLIPScore on fairness.
Moreover, Table 6 illustrates that RL can gener- ally enhance the model generation performance on COCO. It is worth noting that, while using CIDEr as the reward does not result in increased bias, the same cannot be said for CLIPScore, which has the potential introduce more bias to the model. Specifically, the gender prediction error rates in- crease from 1.4% to 1.6% using CLIPScore as the reward. On the other hand, the gender prediction er- ror rates decrease from 1.4% to 1.2% using CIDEr as the reward. The advantage of using CIDEr scores rewards is that it motivates the model to make accurate predictions on word-by-word basis, leading to improvements in gender-related predictions. Conversely, since CLIPScore empha- sizes the overall similarity between images and text, biases in the evaluation metrics can be carried over to generation models through the optimization process. As a result, utilizing biased metrics for language generation models may propagate biases, which potential drawback.
3.2.1 Setting
We optimize FIBER with RL following Dou et al. (2022) PAO-EVALBIAS and COCO-Karpathy image captioning dataset as has been widely used in previous image captioning work. Specifically, FIBER used the minimum risk training algorithm (Shen et al., 2016) which has been used in other text generation tasks as well such as machine transla- tion. At each training step, we sample 5 generations from the model and compute the score of each sam- ple. The computed scores then used to weight the samples and the generation model is updated ac- cordingly. Moreover, we utilize CIDEr, CLIPScore, or a linear combination of the two scores as reward functions. We finetune the MLE-trained FIBER using RL for 1 epoch for PAO-EVALBIAS and 3 epochs for COCO with the learning rate set to le-6.
4 A Hybrid Similarity Metric
3.2.2 Results
While model-based metric contains biases, n-gram matching-based metrics can hardly encode gender biases. Therefore, is natural to combine n n-gram matching-based with model based metrics to alle-
Table demonstrates that RL can enhance the model generation performance, as observed in the improvement of CLIPScore from 69.7 to 72.7 on

---

viate gender biases. Motivated by this, investi- gate if adding CLIPScore and CIDEr together with- out normalization for model evaluation (denoted as CLIPScore+CIDEE) can harness the benefits of both model-based and n-gram matching-basee eval- uation metrics, which has demonstrated effective in other tasks (Wan and Bansal, 2022; Huang et al.. 2023). Formally, obtain the new evaluation score with
Te
BLEU-4
30.776
METEOR
41.822
ROUGE
32.314
CIDEr
43.891
SPICE
44.888
CL CLIPScore
51.482
LLIPScore+BLEU--.
51.260
CL CCIPScore+CIDEr 53.768
CLIPScore+SPICE 55.051
H(e,, ri, L) CLIPScore(e, I.) rv),,, (4)
Table Correlations (measured with T.) with human judgment Flickr8K-Expers Combining CLIPScore with CIDEr or SPICE can improve the correlation over both n-gram matching-based evaluation metrics and CLIPScore.
where & denotes the candidate caption, 7 de- notes the reference sentences set, and I denotes the corresponding image. We mainly foeus on CIDEr because it is commonly used n-gram matching-basee based metric in image captioning tasks although our method is compatible with other n- gram matching-basn based metrics well. We assign equal weights both of the metrics for simplicity, while more sophiscated weighting strategy can potentially improve the model performance but add complexity, which we leave as a future direction.
method presents comprehensive evaluation frame- work containing visual relevance and magnified sensitivity to gender-inclusive terminology.
4.2 Impact on Generation Models
Following the setting in Section 3.2, we perform the same experiments with the hybrid metric. Ta- ble 5 shows that CLIPScore+CIDEI can alleviate the biases. Specifically, find that (1) biases in the evaluation metric can make biased and debi- ased models indistinguishable based on evaluation scores; (2) n-gram matching metries can hardly en- code biases and CLIPScore+CIDEr can alleviate the bias issue (biased: 208.5 vs debiased: 223.2 on linear combination scores).
4.1 Bias Evaluation
this part, we experiment with the hybrid metric following the setting in Section 2.2. Table shows that CLIPScore+CIDD CIDEr does not encode gender bi- ases on the PAO-EVAL EVALBIAS dataset, suggesting this method can successfully reduce the metric bias. We include several examples in Appendix B with CLIPScore and CIDEr score breakdowns to demon- strate the idea of combining these two metrics. Moreover, we evaluate the human correlations cach evaluation metric on Flickr8K- -expert (Ho- dosh et al., 2015) and as present in Table 7, CLIP- Score+CIDEr achieves an improved correlation with human judgments compared to CLIPScore and CIDEr, indicating that it can maintain its capa- bility of model evaluation. Our success with CLIP- Score+CIDEr shows our method is compatible with any other statistical metrics. That say, we also test BLIPScor++BLEU4 and CLIPScore+SPICE, re- sulting in 51. $1.260 and 55.051 тe, respectively, which further strengthens our argument.
In addition, as shown in Table 6, observe that the linear combination of CIDEr and CLIPScore rewards can enhance the model performance com- pared with MLE, as evidenced by the increase CLIPScore from 75.4 to 76.6. Besides, RL with CLIPSScre+CIDER can achieve the best scores on all n-gram matching- compared to RL with CLIPScore CIDEr only. Moreover, this combination approach can mitigate the bias problem CLIPScore, as indicated by the reduction in gender prediction errors from 1.6% to 1.3%. The advantage of using CIDEr scores as rewards is that they motivate the model to make accurate predictions word-by word-by-word, leading to im- provements in gender-inclusive predictions. Con- versely, since CLIPScore emphasizes the overall similarity between images and text, biases in the evaluation metrics can be carried over to generation models through the optimization process. There- fore, linearly combined CLIPScore with CIDEr can decrease gender prediction errors, while achieving
evaluation metrics
To conclude, our proposed metric emphasizes the synergistic fusion of two metrics with com- plementary strengths. While CLIPScore excels at capturing vision-language alignment, it tends to biased models due to inherent gender biases in its encoding. Conversely, CIDEr adheres unbiased reference captions, albeit limited to surface-level comparisons. Combining these two metrics, our

---

higher evaluation scores and maintaining stronger correlation with human judgments. These findings corroborate our assertion and demonstrate the ef- fectiveness of the hybrid metric.
model-based evaluation metrics cannot distinguish between biased and debiased model generations and amplifies the model-encoded gender biases through reinforcement learning. The proposed hy- brid similarity evaluation metric can significantly reduce gender biases, while maintaining a stronger correlation with human judgments than existing metrics. In the future, we plan to expand our analy- sis to include other protected attributes such race and ethnicity, as well other language generation tasks. Additionally, we aim to continue develop- ing more effective methods for removing bias from generation evaluation metrics.
5 Related Work
Evaluation Metries. N-gram matching met- rics (Papineni et al., 2002; Lin, 2004; Vedantam et al., 2015) have been dominating in evaluating text generation models. However, these metrics typ- ically consider similarities the lexical level in- stead of the semantic level. To solve the issue, vari- ous approaches have been proposed (Banerjee and Lavie, 2005; Anderson et al., 2016) and models pre- trained on large corpora have been leveraged (Zhao et al. 2019; Zhang et al., 2019; Thompson and Post, 2020; Rei et al., 2020; Sellam et al., 2020; Yuan et al., 2021). In image captioning, Hessel et al. (2021) propose CLIPScore, reference-fieee metric based on CLIP (Radford et al.. 2021) and achieve impressive correlation with human judgments.
Limitations
We only consider two genders (man and woman) in our paper and classify gender expression (i.e. how individuals express their identity through clothing, hair length, mannerisms, and makeup) instead of biological sex or gender identity (i.e., how indi- viduals experience their own gender (Dev et al., 2021)) in our setting, while it is important to note that gender is non-binary and a detailed discussion can be found in the ethics statement section. Also, we mainly focus on gender biases in our paper. but there are other types of biases such as racial and religious biases, where equal representation is desired. In addition, we only experiment with the image captioning task, while other multimodal generation tasks are worth investigating as well.
Societal Biases in Pretrained Models. It has been pointed out (Bolukbasi et al., 2016; Zhao et al., 2017; Bender et al., 2021) that there are soci- etal biases encoded in the model training data, and models pretrained on these data can amplify the bi- ases and potentially harm marginalized populations. While there several works on investigating the bias issue of pretrained models (Kurita al., 2019; Sheng et al., 2019; Agarwal al., 2021: Cho et al., 2022; Zhang et al., 2022; Wang et al., 2022), biases in model-based evaluation metrics have received less attention. Among them, Sun et al. (2022) con- struct dataset based on WinoBias (Zhao et al., 2018) and perform systematic investigation on different types of metries. However, the paper does not study evaluation metrics in the multimodal do- main and fails to analyze the implications of the metric biases real-world models.
Ethics Statement
Our research aims to investigate the gender bi- ases present in image captioning evaluation met- rics using the PAO-EVALBIAS dataset. We fo- cus on selected concepts such as profession, ac- tivity, and object within the gender axis, although other categories such as racism also require equal representation. Our goal is assist practitioners and the community in evaluating existing model- based evaluation metrics from different perspec- tives. We are aware that gender is a complex and multi-faceted concept and although there are many different groups within gender, in this study we limit our analysis to classifying individuals as ei- ther "man" or "wommn" based on their gender ex- pression, which refers how individuals express their identity through clothing, hair length, manner- isms, and makeup. We make conscious decision not to evaluate an individual's gender identity or biological sex as it is not possible to infer this in-
6 Conclusion
We analyze the gender biases issue of model-based evaluation metrics on image captioning tasks and investigate its potential impact on image caption- ing generation models. To do this, we create our own dataset and conduct thorough analysis of the gender bias present in various evaluation metrics across multiple concepts. We also discuss the con- sequences of these biases in real- real-world applications and propose a hybrid metric as a solution to miti- gate the issue. Experiments show that using biased

---

formation based on appearance alone, and our goal is to focus on the perceptual biases and gender assumptions of the human annotators. We acknowl- edge that the use of binary categories may be offen- sive to underrepresented groups, but it is important to note that our research aims to provide a starting point for further discussion and research in this area. Our research also aims to review the existing model-based evaluation metrics in further dimen- sions, including fairness and bias. By doing so, we hope to help practitioners and the community to understand the limitations and potential harms of these metrics, and to develop better and more inclusive evaluation metries.
Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer woman is to home- maker? debiasing word embeddings. In Advances in Neural Information Processing Systems (NewrIPS).
Jaemin Cho, Abhay Zala, and Mohit Bansal. 2022. Dall-eval: Probing the reasoning skills and social bi- ases of text-to-image generative transformers. arXiv preprint.
Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Ar- jun Subramonian. J. M. Phillips, and Kai Wei Chang. 2021. Harms of gender exclusivity and challenges in non-binary representation in language technolo- gies. Conference on Empirical Methods in Natural Language Processing (EMNLP).
Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang. Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng. et al. 2022. Coarse-to-fine vision-language pre-training with fu- sion in the backbone. In Advances in Neural Infor- mation Processing Systems (NeurIPS).
Acknowledgment
We thank anonymous reviewers for their helpful feedback. We also thank I-Hung Hsu, Di Wu, Da Yin, Sarik Ghazarian, and other members from the UCLA NLP group for their feedback and dis- cussions. The research is supported in part by an Amazon Alexa Al gift award and a Meta SRA.
Jinlan Fu, See-Kiong Ng. Zhengbao Jiang, and Pengfei Liu. 2023, GPTScore: Evaluate as you desire. arXiv preprint.
Michael Hanna and Ondřej Bojar. 2021. A fine fine-nrained analysis of bertscore. In Conference on Machine Translation (WMT).
References
Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. 2021. Evaluating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint.
Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018 Women also snowboard: Overcoming bias in captioning mod- els. In European Conference Computer Vision (ECCV).
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. SPICE: Semantic proposi- tional image caption evaluation. In European Con- ference Computer Vision (ECCV).
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. CLIPScore: A refirrenc-frree evaluation metric for image caption- ing. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. AnL Workshop on Intrinsic and Extrinsic Vvaluation Measures for Machine Translation and/or Summarization.
Micah Hodosh, Peter Young, and J. Hockenmaier. 2015. Framing image description as a ranking task: Data, models and evaluation metrics (extended abstract). International Joint Conference on Artificial Intelli- gence (IJCAI).
Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai- Wei Chang. 2022. How well can text-to-image gen- erative models understand ethical natural language interventions? Conference on Empirical Methods in Natural Language Processing (EMNLP).
Kung-Hsiang Huang, Hou Pong Chan, and Heng Ji. 2023. Zero-shot faithful factual error correction. Annual Meeting of the Association for Computational Linguistics (ACL).
Soumya Barikeri,, Anne Lauscher, Tvan Vulié, and Goran Glavaš. 2021. Redditbias: A real-world resource for bias evaluation and debiasing of conversational lan- guage models. In Annual Meeting of the Association for Computational Linguistics (ACL).
Andrej Karpathy and Fei-Fei. 2015. Deep visual- semantic alignments for generating image descrip- tions. In JEEE Conference on Computer vision and Pattern Recognition (CVPR).
Emily M Bender, Timmit Gebru, Angelina McMillan-
Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language mod- els be too big? ACM Conference on Fairness, Accountability, and Transparency (FAceT).
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in con- textualized word representations. In Workshop on Gender Bias in Natural Language Processing.

---

Siyao Li. Deren Lei, Pengda Qin, and William Yang Wang. 2019. Deep reinforcement learning with distri- butional semantic rewards for abstractive summariza- tion. In Conference on Empirical Methods in Natural Language Processing (EMNLP)-
Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum risk training for neural machine translation. In An- nual Meeting of the Association for Computational Linguistics (ACL).
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out.
Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The woman worked as babysit- ter: On biases in language generation. In Conference on Empirical Methods in Natural Language Process- ing (EMNLP).
Tsung-Yr Yi Lin, Michael Maire, Serge Belongie. James Hays. Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European confer- ence on computer vision (ECCV).
Tianxiang Sun, Junliang He, Xipeng Qiu, and Xuanjing Huang. 2022. Bertscore is unfair: On social bias in language model- metrics for text generation. In Conference Empirical Methods in Natural Lan- guage Processing (EMNLP).
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lec. 2023. Visual instruction tuning. In Confer- ence on Computer Vision and Pattern Recognition (CVPR).
Brian Thompson and Matt Post. 2020. Automatic ma- chine translation evaluation in many languages via zero-shot paraphrasing. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Nikita Nangia, Clara Vania, Rasika Bhalerao. and Samuel Bowman. 2020. Crows-pairs: challenge dataset for measuring social biases in masked lan- guage models. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image descrip- tion evaluation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: a method for automatic eval- uation of machine tramslation. In Annual Meeting of the Association for Computational Linguistics (ACL).
David Wan and Mohit Bansal. 2022. FactPEGASUS: Factuality- aware pre- vre-training and tne-ttning for ab- stractive summarization. In Conference of the North American Chapter of the Association for Computa- tional Linguistics (NAACL).
Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive sum- marization. In International Conference on Learning Representations (ICLR).
Yuxuan Wan, Wenxuan Wang, Pinjia He, Jiazhen Gu, Haonan Bai, and Michael R. Lyu, 2023. Biasasker: Measuring the bias in conversational ai system. The ACM Joint European Software Engineering Confer- ence and Symposium on the Foundations of Software Engineering (ESEC/FSE), 122s22305.12344.
Amy Pu, Hyung Won Chung, Ankur Parikh, Sebastian Gehrmann, and Thibault Sellam. 2021. Learning compact metrics for mt. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Junyan Wang. Yi Zhang, and Jitao Sang. 2022. Fair- clip: Social bias elimination based on attribute proto- type learning and representation neutralization. arXiv preprint.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try. Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Con- ference on Machine Learning (ICML).
John Wieting, Taylor Berg- King-Kikkattrick. Kevin Gimpel, and Graham Neubig. 2019. Beyond bleu: Training neural machine translation with semantic similarity. In Annual Meeting of the Association for Computa- tional Linguistics (ACL).
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for mt evaluation. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. BARTScore: Evaluating generated text text gen- eration, Advances in Neural Information Processing Systems (NeurIPS).
Steven J Rennie, Etienne Marcheret, Youssef Mrouch, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In IEFE Conference on Computer Vision and Pattern Recog- nition (CVPR).
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi 2019. Bertscore: Evaluating text generation with bert. In International Confer- ence on Learning Representations (ICIR)).
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text genera- tion. In Annual Meeting of the Association for Com- putational Linguistics (ACL).
Yi Zhang, Junyang Wang, and Jitao Sang. 2022, Coun- terfactually measuring and eliminating social bias in Dision-language pre-training models. In ACM Inter- national Conference on Multimedia (ACM MM).

---

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. Conference on Empirical Methods in Natural Language Processing (EMNLP).
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Conference of the North American Chap- ter of the Association for Computational Linguistics (NAACL).
Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Chris- tian M Meyer, and Steffen Eger. 2019. MoverScore: Text generation evaluating with contextualized em- beddings and earth mover distance. Conference on Empirical Methods Natural Language Processing (EMNLP).
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In Conference Computer Vision and Pattern Recognition (CVPR).

---

A Dataset Construction
N-Gram Metries CLIPScore CLIPScor+CIDDEr
Profession
0.00
50.00
0.00
We acknowledge the significance of investigating potential gender bias when creating datasets, espe- cially those used to evaluate model biases. While it is true that maintaining a comparable number of examples for different genders under the same concept group would provide more robust grounds for accuracy metric comparisons, it is important note that achieving perfect balance in sample sizes can be challenging. Our primary goal in creat- ing PAO-EVALBIAA was to provide a diverse and comprehensive dataset covering various concepts in professions, activities, and objects. In real-world scenarios, there can be variations in the distribu- tion of gender across different concepts due to his- torical, cultural, and societal factors. Attempting to enforce strict balance of genders within each concept group might inadvertently lead misrep- resentation artificial manipulation of the dataset, which could result in unintended biases. When evaluating the biases in models, the focus should be on the model's ability to make accurate pre- dictions and classifications, while being sensitive to gender-neutral attributes. The dataset aims to test the models' behavior and performance rather than enforcing specific gender distribution within each concept. Moreover, strictly follow the data collection protocol delineated in prior work (Cho et al., 2022: Bansal et al., 2022; Zhang et al., 2022), while constructing image retrieval prompts and assembling concept lists for our dataset's s cre- ation. Through this meticulous process, the created dataset embodies comprehensive diversity, faith- fully capturing the intricacies of real-world scenar- ios.
Activity
0.00
53.85
0.00
Object
0.00
48.72
0.00
Overall
0.00
50.86
0.00
Table 8: Percentages of words in PAO-EVALBIAS that are biased with the imbalanced concept groups removed. N-gram metrics include BLEU-4, METEOR, ROUGE, CIDEr, and SPICE, separately. CLIPScore exhibits gen- der biases on over 50% of lexicons, while n-gram eval- uation metries do not reveal any gender bias. The linear combination of CLIPScore and CIDEr scores (CLIP- Score+CIDEr) can alleviate the gender biases encoded CLIPScore.
PAO-EVALBIAS was to provide a dataset that is both diverse and comprehensive, encompassing a wide array of concepts spanning professions, activ- ities, and objects. In practical, real-world scenarios, the distribution of gender across these concepts can naturally vary due historical, cultural, and societal factors.
Hybrid Similarity Metric
We include two examples (Table 9 and 10) with CLIPScore and CIDEr score breakdowns demon- strate the idea of combining these metrics. Our proposed approach combines two metries that each have unique strengths, resulting powerful syn- ergy. CLIPScore is excellent at capturing the subtle nuances of visual-language alignment, but may introduce biases due to inherent gender biases in its encoding. In contrast, CIDEr places strong em- phasis linguistic quality and remains unbiased in its reference captions, although it is limited surface-level comparisons. By merging these two metrics, our method provides comprehensive eval- uation framework that considers visual relevance, while also being sensitive to gender-inclusive ter- minology.
To perform a robustness check on Table 2 re- sults, we perform the same analysis using PAO- EVALBIAS with the imbalanced concept groups removed. We removed the following concepts: (1) profession: [chef, engineer, judge, soldier, doctor, nurse, pilot, porter, puppeteer, mechanic]; (2) activ- ity: [jumping, riding, sitting, standing]: (3) object: [bacon].
Although we can notice numbers dropping for all three concept groups in Table 8, maintaining equivalent number of examples for different genders within the same concept group would un- doubtedly bolster the robustness of accuracy metric comparisons. Nevertheless, is crucial to acknowl- edge the inherent challenges achieving a perfect sample size balance. Our main goal developing

---

Good Cand. Caption Example
Bad Cand.
Caption Example
Biased?
Reference caption
photo of who
photo of man who
No.
Candidate caption
who
who is
CLIPScore
0,6699
0.7119
Yes
CIDEr
7.0039
2,9982
No
CLIP
7.6738
3.7101
No
Table An example of CLIPScore and CIDEr score breakdown with <Group: man, concept: nurse (profession)>.
Good Cand. Caption Example
Bad Cand.
Caption Example
Biased?
Reference caption
photo of who chef
photo of who chef'
No
Candidate caption
who is chef'
who is chef
CLIPPScre
0.6108 6.9952
0.6294 2.6919
CIDEr
No
CLIPSccre+CIDEr
7.606
3.3213
Table 10: An example of CLIPScore and CIDEr score breakdown with <Group: woman, concept: chef (profession)>.
100
builder
Profession Man CLiPScore Accuracy
miner 95
gallos
dgecor
cook
plectricigg
econamist
enginéer
soiiitor
butcher
yriter
fivil servant
porter
Joymajist
accuntant
85
animater
harddresser
granslator
80
receptionlst
printer
jeweler
makeup artist
75
career counse
ssstannt
70
man-blased
néutral
woman-biased
65 -
75
80
85
100
Profession Woman CLIPScore Accuracy
Figure 5: Gender biases under the PAO-EVALBIAS profession category in CLIPScore evaluation: Blue points are man-biased and green points woman-biased. Points in orange have p-value greater than 0.05 with bootstrap resampling.

---

Word
Woman Count
Man Count
Word
Woman Count
Man Count
accountant
233
246
baker
224
205
animator
37
25
biologist
229
165
architect
179
121
builder
223
206
assistant
236
219
butcher
204
191
author
205
166
decorator
123
203
caretaker
154
174
dentist
236
196
chef
815
1804
designer
206
228
clerk
223
214
diplomat
189
181
cook
196
215
director
226
229
civil servant
134
184
doctor
475
709
counselor
201
196
magician
201
214
economist
204
184
makeup artist
214
editor electrician
183
160
manager
214
214
236
miner
190
engineer
313
musician
232
executive
239
nurse
595
260
farmer
838
1041
optician
138
flight attendant
256
prison officer
183
119
geologist
196
painter
161
hairdresser
230
183
personal assistant
208
jeweler
110
70
photographer
141
221
journalist
233
216
pilot
413
799
judge
455
705
plumber
231
220
juggler
236
223
police officer
241
230
lawyer
228
213
politician
234
223
lecturer
198
233
porter
6
188
lexicographer
106
177
printer
179
147
receptionist
235
199
puppeteer
42
190
sailor
124
259
waiter web designer
203
211
salesperson
222
239
166
104
scientist
235
234
company director
234
201
secretary
228
200
library assistant
156
117
singer
225
238
sign language interpreter shop ussistant
182
202
soldier
308
815
224
207
solicitor
211
213
computer programmer
217
215
surgeon
231
222
comic book writer
62
109
tailor
179
197
garhage collector
151
223
teacher
232
226
film director
228
213
telephonist
231
203
head teacher
246
204
translator
134
99
athlete footballer
312
347
trucker
217
119
133
522
travel agent
207
165
mechanic
40
620
TV presenter
244
231
police
151
984
telephone operator
242
183
143
117
218
199
writer
216
193
Table PAO-EVAL. BIAS dataset profession statistics after data cleaning by human annotators. "Woman count" refers to the number of images with good captions "a woman who is a/an (profession). " "Man count" refers to the number of images with good captions "a man who is a/an prrofession)."

---

Word
Woman Count
Man Count
Word
Woman Count
Man Count
baking
209
picking
241
218
begging
231
282
praying
229
260
biking
258
reading
323
367
calling
276
213
riding
317
691
cleaning
254
269
rowing
227
249
climbing
260
326
running
335
418
cooking
313
334
serving
250
195
coughing
254
246
sewing
263
194
crying
279
249
shopping
327
266
drinking
312
307
shouting
269
305
driving
260
292
sitting
1318
2042
cating
451
514
skating
258
317
exercising
245
238
sleeping
301
372
falling
242
245
smiling
541
567
fishing
232
260
speaking
244
253
hitting
193
257
spying
197
230
hugging
258
242
standing
1230
2558
jogging
260
256
staring
244
252
jumping
341
741
stretching
303
246
kicking
242
257
studying
263
259
kneeling
254
279
sweeping
258
239
laughing
274
287
talking
297
296
lifting
262
264
throwing
219
244
painting
263
272
walking
979
1231
pitching
234
301
washing
270
244
waving
262
268
working
281
392
Table 12: PAO-EvAI. BIAS dataset activity statistics after data cleaning by human annotators. "Woman count" refers to the number of images with good captions "a woman who is (activity)." "Man count" refers the number of images with good captions "a man who is /activity]""
Word
Woman Count
Man Count
Word
Woman Count
Man Count
scotch
208
206
wine
237
219
briefcase
234
211
basketball
183
217
jersey
225
174
hamburger
198
209
whiskey
205
bacon
130
46
suit
231
203
hat
217
170
beer
224
pie
227
tie
236
fruit
234
gun
240
jewellery
236
180
cigar
237
necklace
236
210
golf
201
makeup
251
228
helmet
195
purse
222
208
junk
146
salad
223
200
punch
225
163
yarn
223
bike
220
aviator
244
tool
219
185
piercing
243
225
meat
205
199
healthy
239
212
barbecue
224
195
apron
242
220
steak
198
204
candle
205
172
cat
225
214
perfume
124
114
scarf
233
240
Table 13: PAO-EVAL. BIAS dataset object statistics after data cleaning by human annotators. "Woman count" refers to the number of images with good captions "a woman with a/an [object]." "Man count" refers to the number of images with good captions "a man with a/an (object)."

---

Man-biased Words
Woman-biased Words
assistant, author, baker, biologist, career. clerk, comic book writer, cook, dentist, executive, flight attendant, hairdresser, head teacher, judge, lawyer, magician, makeup artist, manager, personal assistant, photographer, politician, receptionist, secretary, shop assistant, soldier. surgeon, telephone operator, travel agent, TV presenter, vet, writer
animator, architect, builder, butcher, chef, civil servant, computer programmer, economist, electrician, engineer, lexicographer, miner, plumber, printer, tailor, waiter
Profession
baking, biking. cleaning, driving, exercising. lifting, riding, running, skating, spying, talking. calling. climbing. drinking, jonging, painting. serving, sleeping, speaking, stretching, washing
fishing, hugging, praying, staring, begging. pitching, sitting, standing
Activity
briefcase, beer, gun, cigar, bike, tool, pie, fruit, yarn, healthy, apron, candle, salad, purse, makeup, necklace, jewellery
whiskey, tic, meat, steak, baskethall, hamburger, aviator, perfume
Object
Table 14: Man- bia-biased and Woman-biase Words in PAO-EVALBIAG EVALBIAS with CLIPScore.
101
man-biased
perfyme scotch
100
neutral
whiskey
woman-biased
barbecue
Object Man CLiPScore Accuracy
aviator
steak
basketball
hamblugerr
piercingrllme
punch
whe jetsey Scart
98
beer
meat
golf
aBro
junk
brieftase
cat
bacon
candle
96
fruit
Blke
purse
95
Jewe lery
94
93
pie
86
98
Object Woman CLIPScore Accuracy
Figure 6: Gender biases under the PAO-EVALBIAS object category in CLIPScore evaluation: Blue points are man-biased and green points woman- Points in orange have p-value greater than 0.05 with bootstrap resampling.

---

10a
man-biased
neutral
Mann Profession Captioning Error Rate
woman-biased
aersonal_assistane
makeup artist
assistant
career counselor
urse
yet
optiefa aaeeaker
10a
baker
clerk
şailor
writer
surgeøn
musician
prison_offir officer
designfer
scientist
teacher
porter
miner
economist
10
10a
102
Woman Profession Captioning Error Rate
Figure 7: Gender biases under the PAO-EVALBIAS profession category of FIBER: Blue points are man-biased and green points are woman-biased. Points in orange have p-value greater than 0.05 with bootstrap resampling.
10a
man-blased
neutral woman-blased
Man Object Captioning Error Rate
jewellery cat
10a
Jruit purse yarn
pie
bafbecue
punch
bacon
jersey aviator
wine
steak
piercing
meat
100
10a
10a
10a
Woman Object Captioning Error
Figure 8: Gender biases under the PA0-EVALBIAS object category of FIBER: Blue points are man nan-biased and green points are woman- miman-biased. Points in orange have p-value greater than 0.05 with bootstrap resampling.


Reviews:
[{"review_id": "7AD16noLQX", "reviewer": "Reviewer_g9rh", "summary": "", "questions": "- Question A: \nIn Table 5, I was wondering about the reasons why CIDEr scores are missing for MLE and RL on PAO-EVALBIAS. \n\n- Question B: \nCould you please elaborate more on what RL algorithm you were using for the experiment in section 3.2?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful review of our paper. We’re glad you found our idea and results are significant. Your recognition of PAO-EVALBIAS as a valuable resource for future research is motivating. We're excited that you appreciated our strong experimental approach and compelling results. Your comments on our approach to combining evaluation metrics encourage future innovation in this direction. Thank you for your invaluable feedback.\n\n---\n\n**Reasons To Reject**\n\n> This work lacks a discussion about whether potential ``gender bias\" can be introduced when the authors create the dataset, PAO-EVALBIAS. Ideally, the dataset should have a comparable number of examples for different genders under the same concept group to make the accuracy metric Arr_G,C comparisons meaningful. However, there are still many concepts with very different sample sizes in gender groups: chef, judge, engineer, nurse, and porter to name a few.\n\nWe acknowledge the significance of investigating potential gender bias when creating datasets, especially those used to evaluate model biases. While it is true that maintaining a comparable number of examples for different genders under the same concept group would provide more robust grounds for accuracy metric comparisons, it’s important to note that achieving perfect balance in sample sizes can be challenging. Our primary goal in creating PAO-EVALBIAS was to provide a diverse and comprehensive dataset covering various concepts in professions, activities, and objects. In real-world scenarios, there can be variations in the distribution of gender across different concepts due to historical, cultural, and societal factors. Attempting to enforce a strict balance of genders within each concept group might inadvertently lead to misrepresentation or artificial manipulation of the dataset, which could result in unintended biases. When evaluating the biases in models, the focus should be on the model’s ability to make accurate predictions and classifications while being sensitive to gender-neutral attributes. The dataset aims to test the models’ behavior and performance rather than enforcing a specific gender distribution within each concept. Moreover, we strictly follow the data collection protocol delineated in prior work [1][2][3] while constructing image retrieval prompts and assembling concept lists for our dataset’s creation. Through this meticulous process, the created dataset embodies comprehensive diversity, faithfully capturing the intricacies of real-world scenarios. \n\n[1] Jaemin Cho, Abhay Zala, and Mohit Bansal. 2022. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers. arXiv preprint.\n\n[2] Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. 2022. How well can text-to-image generative models understand ethical natural language interventions? Conference on Empirical Methods in Natural Language Processing (EMNLP).\n\n[3] Yi Zhang, Junyang Wang, and Jitao Sang. 2022. Counterfactually measuring and eliminating social bias in vision-language pre-training models. In ACM International Conference on Multimedia (ACM MM).\n\n> A more thorough analysis of the reasons why combining CLIPScore and CIDEr can mitigate gender bias is needed. It is surprising to see after the simple addition of CLIPScore and CIDEr, the percentages of words in PAO-EVALBIAS that are biased go to zero.\n\nIn lines 470-486, we highlight the motivation behind our proposed approach, emphasizing the synergistic fusion of two metrics with complementary strengths. While CLIPScore excels at capturing visual-semantic alignment, it tends to biased models due to inherent gender biases in its encoding. Conversely, CIDEr prioritizes linguistic quality while adhering to unbiased reference captions, albeit limited to surface-level comparisons. Combining these two metrics, our method presents a comprehensive evaluation framework containing visual relevance and magnified sensitivity to gender-inclusive terminology. Consequently, the linear combination of these two metrics effectively mitigates gender bias, resulting in the elimination of biased terms within the PAO-EVALBIAS, as demonstrated in our settings.\n\n> The benefits of combining CLIPScore and CIDEr in the reinforcement learning setting seem to be mediocre. Is RL with CLIPScore and CIDEr significantly better than RL with CLIPScore or CIDEr only in terms of gender prediction errors?\n\nAs shown in Table 6, RL with CLIPScore+CIDER can achieve the best scores on all evaluation metrics (e.g., BLEU-4: +0.5, METEOR: +0.4, ROUGE: +0.3, CIDER: +1.2, and SPICE: 0.5) except for CLIPScore. In lines 419-427, we discussed that using CIDER as the reward in RL does not result in increased bias (1.4->1.2), while CLIPScore has the potential to introduce more bias to the model (1.4->1.6). The advantage of using CIDEr scores as rewards is that they motivate the model to make accurate predictions word-by-word, leading to improvements in gender-related predictions. Conversely, since CLIPScore emphasizes the overall similarity between images and text, biases in the evaluation metrics can be carried over to generation models through the optimization process. Although RL with CLIPScore+CIDEr is not significantly better (p~0.10) than RL with CLIPScore, the small gap is also because there are not many biased generations for analysis. Therefore, linearly combined CLIPScore with CIDER can achieve the best of the worlds, decreasing gender prediction errors while achieving higher evaluation scores and maintaining a stronger correlation with human judgments.\n\n> Although I feel that a linear combination of CLIPScore and CIDEr together is interesting, the paper does’t fully address the motivations to do so. Why not consider a weighted average? Why can’t we use CLIPScore in tandem with CIDEr in settings other than RL?\n\nOur major contributions in this paper are (1) the dataset, (2) a systematic method to analyze gender biases in automatic evaluation metrics and how biased evaluation metrics can affect generation models, (3) a simple but effective method combining CLIPScore and CIDEr to reduce gender biases while maintaining high correlations with human judgments for generation quality, providing a good baseline and inspiration for future works.\n\nIn lines 470-486, we underscore the motivation behind our proposed method, emphasizing the synergistic combination of these two metrics’ complementary strengths. Although CLIPScore’s advantage lies in capturing visual-semantic alignment, it prefers biased models due to encoded gender biases. Conversely, CIDEr focuses on linguistic quality while adhering to unbiased reference captions. By amalgamating these metrics, our method offers a comprehensive evaluation framework encompassing visual relevance and a spotlight on gender-aware terms. To conclude, the rationale behind this approach is to harness the benefits of both model-based and n-gram matching-based evaluation metrics, as demonstrated in prior studies [1][2] (DocNLI+ROUGE-1, FactCC+ROUGE). \n\nA weighted average could be considered, but it might not necessarily capture the intricate interactions between these two aspects as effectively as a unified metric. A linear combination allows for equal consideration of both metrics, preventing one from overshadowing the other in the final evaluation. Moreover, a weighted average might require a separate optimization step to determine the optimal weights for each metric, potentially adding complexity to the approach. \n\nConcerning the adaptability of CLIPScore+CIDEr in settings beyond reinforcement learning (RL), we recognize the versatility of this hybrid metric. We are committed to making these points more evident in the revised version.\n\n[1] Kung-Hsiang Huang, Hou Pong Chan, and Heng Ji. 2023. Zero-shot Faithful Factual Error Correction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5660–5676, Toronto, Canada. Association for Computational Linguistics.\n\n[2] David Wan and Mohit Bansal. 2022. FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1010–1028, Seattle, United States. Association for Computational Linguistics.\n\n**Questions For The Authors**\n\n> Question A: In Table 5, I was wondering about the reasons why CIDEr scores are missing for MLE and RL on PAO-EVALBIAS.\n\nCIDEr needs reference captions whereas our constructed dataset does not have human annotated captions. Therefore, the CIDEr scores calculation are not suitable for MLE and RL on PAO-EVALBIAS.\n\n> Question B: Could you please elaborate more on what RL algorithm you were using for the experiment in section 3.2?\n\nWe followed FIBER and used their public codebase for RL training (line 391-392). Specifically, FIBER used the minimum risk training algorithm (Shen et al, 2016) which is widely used in text generation tasks such as machine translation. The general idea is that at each training step, we sample 5 generations from the model and compute the score of each sample. The computed scores are then used to weight the samples and the generation model is then updated accordingly.\n\n**Missing References**\n\n> I was wondering if the authors can still observe the ``gender bias\" by calculating CLIPScore using FairCLIP. [1] FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization, Wang et al.\n\nWe appreciate your suggestion to explore the FairCLIP paper. After thoroughly reading it, we see that FairCLIP can be used as a general debiasing method for other fairness issues related to CLIP. We plan to modify CLIPScore based on FairCLIP and conduct more experiments to investigate gender biases. Unfortunately, the unavailability of the FairCLIP code is a challenge to our efforts to replicate and verify outcomes within our limited timeframe. Nonetheless, we will cite the FairCLIP paper as a reference and are enthusiastic about future experiments to enrich our understanding of this pivotal topic.\n\n**Typos Grammar Style And Presentation Improvements**\n\nThank you for your suggestions on improving our presentations for better understanding. While we acknowledge the possibility of bias in model-based evaluation metrics stemming from pre-trained models, we cannot confirm that CLIPScore inherits biases from model training data. Other factors, such as model architecture, may also contribute to biased evaluation metrics. This discussion is beyond our scope since we focus more on the potential bad consequences of using biased model-based evaluation metrics. We are committed to making these points more evident in the forthcoming revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper presents the idea and results in a way that is relatively easy to follow and the motivation is clear and important.\n- The curated dataset, PAO-EVALBIAS, consisting of profession, activity, and object concepts related to stereotypical gender associations, is a valuable resource for future research.  \n- The experiments are conducted in a solid way and the empirical results highlighting that the model-based evaluation metric contains biases are convincing. \n- The idea of combining an n-gram matching-based evaluation metric and a model-based metric is interesting and novel. It also opens up possibilities to construct better evaluation metrics inspired by combining the best of both worlds.", "reasons_to_reject": "- This work lacks a discussion about whether potential ``gender bias\" can be introduced when the authors create the dataset, PAO-EVALBIAS. Ideally, the dataset should have a comparable number of examples for different genders under the same concept group to make the accuracy metric $Arr_{G,C}$ comparisons meaningful. However, there are still many concepts with very different sample sizes in gender groups: chef, judge, engineer, nurse, and porter to name a few. \n- A more thorough analysis of the reasons why combining CLIPScore and CIDEr can mitigate gender bias is needed. It is surprising to see after the simple addition of CLIPScore and CIDEr, the percentages of words in PAO-EVALBIAS that are biased go to zero. \n- The benefits of combining CLIPScore and CIDEr in the reinforcement learning setting seem to be mediocre. Is RL with CLIPScore and CIDEr significantly better than RL with CLIPScore or CIDEr only in terms of gender prediction errors? \n- Although I feel that a linear combination of CLIPScore and CIDEr together is interesting, the paper doesn't fully address the motivations to do so. Why not consider a weighted average? Why can't we use CLIPScore in tandem with CIDEr in settings other than RL?", "questions_to_authors": "- Question A: \nIn Table 5, I was wondering about the reasons why CIDEr scores are missing for MLE and RL on PAO-EVALBIAS. \n\n- Question B: \nCould you please elaborate more on what RL algorithm you were using for the experiment in section 3.2?", "paper_topic_and_main_contributions": "This paper investigates the problem of gender biases in model-based automatic evaluation metrics for image captioning. In doing so, the authors construct a dataset, PAO-EVALBIAS, consisting of profession, activity, and object concepts related to stereotypical gender associations. The central idea is to first demonstrate that the model-based evaluation metric contains biases and then propose a hybrid similarity evaluation metric, a simple linear combination between CLIPScore and CIDEr, which can reduce the metric bias while not hurting its correlations with human judgments.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "I was wondering if the authors can still observe the ``gender bias\" by calculating CLIPScore using FairCLIP. \n[1] FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization, Wang et al.", "Typos_Grammar_Style_and_Presentation_Improvements": "1. Potential inconsistency between Figure 7 and Table 11 in the appendix: In Table 11, ``baker\" is listed as a men-biased word but it is shown as a woman-biased word in Figure 7. \n2. A discussion about the relationship or differences between the bias of pre-trained models and the bias in model-based evaluation metrics would be helpful for the readers to better understand the contributions of this paper. For example, CLIPScore is calculated based on the cosine similarity between the visual CLIP embedding and the textual CLIP embedding. Is the bias in CLIPScore inherited from the biases encoded in the model training data?"}}, {"review_id": "jHISMFdeG3", "reviewer": "Reviewer_KXSc", "summary": "", "questions": "Question A: What are the advantages of your method compared to other evaluation metrics? Which needs to be emphasized in the motivation.\n\nQuestion B: How do you evaluate the significance of model structure or metrics on the gender bias encoding of the model? Because you only conduct experiments in the FIBER model.\n\nQuestion C:  What is the motivation behind this paper?\n\nI'm happy to update the review if the authors can answer my concerns.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful review of our paper. We’re glad you found our dataset to be a valuable resource for future research and experiments. Thank you for your invaluable suggestions.\n\n---\n\n**Reasons To Reject**\n\n> This paper is a bit difficult to follow. There are some unclear statements, such as motivation.\n\nIn lines 060-112, we present detailed motivations and a straightforward workflow of our work. We notice a lack of prior work in exploring cross-modal biased evaluation metrics and understanding their implications and harm on real-world applications. Specifically, we empirically study gender biases in cross-modal metrics for image captioning. We gather a large image dataset, analyze gender biases in evaluation metrics, and their effect on models. We also present a method combining model-based and n-gram metrics to mitigate gender bias while retaining a strong correlation with human judgments for generation quality.\n\n> In the introduction, the summarized highlights need to be adequately elaborated, and the relevant research content of this paper needs to be detailed.\n\nWe aim to enhance the depth of our summarized highlights and provide more detailed relevant research in the upcoming revision.\n\n> No new evaluation metrics are proposed. Only existing evaluation metrics are linearly combined. In the experimental analysis section, there needed to be an in-depth exploration of the reasons for these experimental results.\n\nOur main focuses in this paper are (1) the dataset, (2) a systematic method to analyze gender biases in automatic evaluation metrics, and how biased evaluation metrics can affect generation models.  Additionally, we propose a simple but effective method combining CLIPScore and CIDEr to reduce gender biases while maintaining high correlations with human judgments for generation quality as our other contribution. We hope our contributions can provide a good baseline and inspiration for future works. This linear combination capitalizes on the advantages of both model-based and n-gram matching-based evaluation metrics, a concept also supported by prior work [1][2] (DocNLI+ROUGE-1, FactCC+ROUGE).\n\n[1] Kung-Hsiang Huang, Hou Pong Chan, and Heng Ji. 2023. Zero-shot Faithful Factual Error Correction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5660–5676, Toronto, Canada. Association for Computational Linguistics.\n\n[2] David Wan and Mohit Bansal. 2022. FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1010–1028, Seattle, United States. Association for Computational Linguistics.\n\n> A case study should be added.\n\nAs a qualitative analysis, Figure 1 presents an example with an image and two candidate captions from our dataset. Notably, CLIPScore assigns a higher score to the incorrect caption (0.75 vs. 0.72 for the correct one), highlighting the presence of bias in the evaluation metric. This example underscores the potential for biased models to be favored when utilizing such biased evaluation metrics in generation tasks. We will add more examples in the revised version.\n\n> What are the advantages of your method compared to other evaluation metrics? Which needs to be emphasized in the motivation.\n\nIn lines 470-486, we emphasize the driving force behind our approach by highlighting the synergistic blend of these two metrics’ complementary strengths. While CLIPScore excels in capturing visual-semantic alignment, its preference for biased models due to encoded gender biases is supported by our experiment results. In contrast, CIDEr and similar n-gram matching-based metrics prioritize linguistic quality while adhering to unbiased reference captions. Through the fusion of these metrics, our method offers a comprehensive evaluation framework that encompasses visual relevance and shines a light on gender-aware terms. To be more precise, our proposed evaluation metrics demonstrate stronger correlations with human judgment compared to other n-gram matching-based metrics (as shown in Table 7). Furthermore, employing our approach can mitigate gender biases in PAO-EvalBias and alleviate negative outcomes while maintaining high generation performance under reinforcement learning (as shown in Tables 2 and 6).\n\n> How do you evaluate the significance of model structure or metrics on the gender bias encoding of the model? Because you only conduct experiments in the FIBER model. Furthermore, you should conduct generalization experiments on the CLIP model or other models.\n\nWe have experimented on BLIP [1] under FIBER's settings. Instead of testing on the whole dataset, we sample 60% of examples out of the entire dataset to conduct a quick experiment. And over 50% of the generated biased captions have higher clipscores than the debiased captions. This experimental result can support our statement on more general settings.\n\n[1] Li, Junnan, Dongxu Li, Caiming Xiong and Steven C. H. Hoi. “BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation.” International Conference on Machine Learning (2022).\n\n\n> The citation format is chaotic in the paper.\n> There are some grammar mistakes in this paper, which could be found in “Typos, Grammar, Style, and Presentation Improvements”.\n\nWe sincerely appreciate your suggestions and will make additional efforts to proofread the manuscript. We believe the copy-editing issue can be easily addressed, and we hope the reviewer can appreciate our scientific contributions.\n\n**Questions For The Authors**\n\n> Question C: What is the motivation behind this paper?\n\nIn lines 035-075, we offer a comprehensive explanation of the motivations driving our work. The efficacy of pretrained model-based evaluation metrics, such as BERTScore, CLIPScore, and GPTScore, is widely recognized. These metrics have demonstrated promising performance by establishing stronger correlations with human judgments compared to n-gram matching-based metrics like BLEU, ROUGE, and CIDEr across diverse generation tasks. Unlike the conventional approach of measuring mere surface-level overlap between references and generated outputs, model-based metrics excel in capturing semantic-level similarities. Consequently, they provide more accurate assessments of model quality. Nonetheless, a prevailing consensus acknowledges that pretrained models inherently embed societal biases, including but not limited to gender, racial, and religious biases. This underscores the potential for adopting such models in evaluating generative models to amplify fairness issues inadvertently. We also notice a lack of prior work in exploring cross-modal biased evaluation metrics and understanding their implications and harm on real-world applications. As we see an increase in the variety of multimodal generation tasks such as image captioning and vision-and-language summarization [1][2], it is crucial to evaluate the cross-modal metrics specifically designed for these tasks.\n\n[1] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Conference on Computer Vision and Pattern Recognition (CVPR).\n\n[2] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In Conference on Computer Vision and Pattern Recognition (CVPR).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "-\tA newly constructed dataset and sufficient experiments.", "reasons_to_reject": "-\tThis paper is a bit difficult to follow. There are some unclear statements, such as motivation.\n-\tIn the introduction, the summarized highlights need to be adequately elaborated, and the relevant research content of this paper needs to be detailed.\n-\tNo new evaluation metrics are proposed. Only existing evaluation metrics are linearly combined. In the experimental analysis section, there needed to be an in-depth exploration of the reasons for these experimental results.\n-\tA case study should be added.\n-\tWhat are the advantages of your method compared to other evaluation metrics? Which needs to be emphasized in the motivation.\n-\tHow do you evaluate the significance of model structure or metrics on the gender bias encoding of the model? Because you only conduct experiments in the FIBER model. Furthermore, you should conduct generalization experiments on the CLIP model or other models.\n-\tThe citation format is chaotic in the paper.\n-\tThere are some grammar mistakes in this paper, which could be found in “Typos, Grammar, Style, and Presentation Improvements”.", "questions_to_authors": "Question A: What are the advantages of your method compared to other evaluation metrics? Which needs to be emphasized in the motivation.\n\nQuestion B: How do you evaluate the significance of model structure or metrics on the gender bias encoding of the model? Because you only conduct experiments in the FIBER model.\n\nQuestion C:  What is the motivation behind this paper?\n\nI'm happy to update the review if the authors can answer my concerns.", "paper_topic_and_main_contributions": "This paper investigates the impact of evaluation metrics on gender bias for image captioning. For this purpose, the author first constructs a dataset. In addition, the author conducted numerous experiments on different evaluation metrics, effectively demonstrating the proposed theory.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Typos, Grammar, Style, and Presentation Improvements:\n-\tIn Figure 1, there is some punctuation that may be missed. (e.g., “(0.75 vs 0.72 correct)”, Should be “(0.75 vs. 0.72 correct)”, and “while the bad caption incorrectly describes the gender.” should be “, while the bad caption incorrectly describes the gender.”) In particular, the above problems appear in many parts of the paper.\n-\tIn line 466, “That said” can be rewritten “That say”.\n-\tIn line 493, there are missing commas. “on COCO although the difference is not significant” should be “on COCO, although the difference is not significant”. The mistake appears in many parts of the paper.\n-\tThe structure and fluency of the sentences in the paper are poor and require improvement to enhance the quality of the writing."}}, {"review_id": "ALTbr1fHoH", "reviewer": "Reviewer_FLYJ", "summary": "", "questions": "1. SPICE has more correlation than CIDEr with human judgement according to Table 7. Why did you use CIDEr instead of SPICE for your RL experiments recorded in Table 6?\n2. When you wrote that CIDEr and CLIPScore are linearly combined, are they added with the equal weights of 1?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to extend our sincere gratitude for your thoughtful feedback on our paper. Your recognition of our novel approach to enhancing model performance and gender fairness is truly appreciated. Your acknowledgment of the practicality of our method, along with your kind words about the clarity and quality of the writing, motivates us to continue pushing the boundaries of research. Thank you for your invaluable insights.\n\n---\n\n**Questions For The Authors**\n> SPICE has more correlation than CIDEr with human judgement according to Table 7. Why did you use CIDEr instead of SPICE for your RL experiments recorded in Table 6?\n\nAs highlighted in lines 447-451,  our deliberate selection of CIDEr stems from its well-established prominence as an n-gram matching-based metric in image-captioning. This preference is underscored by the fact that many recent works, such as BLIP-2 (Li et al. 2023 [1]) and mPLUG (Wang et al. 2022 [2]) have distinctly favored CIDEr as their primary evaluation metric, and CIDEr optimization is a common technique in captioning. Our success with CLIPScore+CIDEr shows our method is also compatible with other n-gram matching-based metrics. \n\n[1] Li, Junnan, Dongxu Li, Silvio Savarese and Steven C. H. Hoi. “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models.” ArXiv abs/2301.12597 (2023): n. pag.\n\n[2] Li, Chenliang et al. “mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections.” Conference on Empirical Methods in Natural Language Processing (2022).\n\n> When you wrote that CIDEr and CLIPScore are linearly combined, are they added with the equal weights of 1?\n\nYes, the linear combination of CIDEr and CLIPScore means they are added with the equal weights of 1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper presents a novel discovery and a novel yet easy method to improve a model's over performance and gender fairness. The method and results sound practical. The paper clearly points out the bias in the automated eval system despite its strength of better mimicking human judgement in semantic preferences. The paper is also well written and easy to follow.", "reasons_to_reject": "n/a", "questions_to_authors": "1. SPICE has more correlation than CIDEr with human judgement according to Table 7. Why did you use CIDEr instead of SPICE for your RL experiments recorded in Table 6?\n2. When you wrote that CIDEr and CLIPScore are linearly combined, are they added with the equal weights of 1?", "paper_topic_and_main_contributions": "The paper proposes a new image captioning model evaluation metric such that the metric does not favor a gender-biased model across various metrics. Furthermore, the paper demonstrates that this fair metric can be used as a reward to improve the model via RL such that the resulting final model outperforms while improving the gender fairness of the model.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper outlines a systematic analsys aimed at studying automatic gender bias evaluation metrics in the context of image captioning tasks. The authors released a dataset to measure this issue and also introduce a novel evaluation to reduce metric bias while mantaining correlations with human judjements.

I agree with the positive comments of Reviewer 3 and I believe the authors strongly replied to the critiques of the reviewers, especially Reviewer 1. I strongly encourage the authors to refine the paper, taking into account the feedback provided by all the reviewers, especially Reviewer 1.
Decision: Accept-Main