From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment (2024)

Abstract: Large language models (LLMs) have enhanced the capacity of vision-language models to caption visual text. This generative approach to image caption enrichment further makes textual captions more descriptive, improving alignment with the visual context. However, while many studies focus on the benefits of generative caption enrichment (GCE), are there any negative side effects? We compare standard-format captions and recent GCE processes from the perspectives of gender bias and hallucination, showing that enriched captions suffer from increased gender bias and hallucination. Furthermore, models trained on these enriched captions amplify gender bias by an average of 30.9% and increase hallucination by 59.5%. This study serves as a caution against the trend of making captions more descriptive.

## introduction large vision-language models (vlms), such as blip @xcite , with superior performance in multi-modal understanding @xcite , benefit from millions of image-caption pairs. improving training paradigms @xcite and data augmentation strategies @xcite are crucial topics to enhance vlm performance in image captioning. among these techniques, generative language models based caption enrichment (gce) methods @xcite @xcite have achieved some of the latest state-of-the-art performances. unlike standard caption benchmarks, which concisely describe the salient parts @xcite of an image (e.g., coco captions @xcite ), gce methods create more descriptive and semantically enhanced captions. these enhanced textual captions are aligned to boost downstream performance with large language models (llms). while many studies emphasize improving caption quality, issues such as societal bias and hallucination are significant yet often overlooked @xcite @xcite @xcite in image captioning. for example, @xcite found that the coco dataset @xcite is skewed towards men, and @xcite showed that models trained on this biased data tend to generate gender-stereotypical captions (e.g., describing a pink dress for women who are not wearing one). these studies have highlighted potential biases in datasets like coco and the models trained on them. addressing this bias is crucial as it can exacerbate unfairness and risks towards underrepresented groups. we aim to examine one critical question that has been overlooked in gce works: "although llm-enriched captions boost vlm performance, do they have negative effects, regarding societal bias and object hallucination?" to answer this, we investigate gender bias and hallucination using comprehensive metrics, examining both datasets and models trained on these datasets for standard captions (coco captions) and enriched captions (sharegpt4v @xcite , fusecap @xcite , capsfusion @xcite ). our analysis reveals that llm-enriched captions indeed have negative side effects, worsening issues of gender bias and hallucination by making captions more descriptive. meanwhile, models trained on these enriched captions tend to amplify these problems further. finally, we discuss possible causes of these negative effects and warn against the trend of making captions more descriptive. a girl is sitting at a table set with sandwiches and milk. in the heart of a cozy kitchen, a young girl is seated at a table, engrossed in her meal. she's dressed in a vibrant green shirt, adding a splash of color to the scene. the table before her is a feast of colors and textures, with a variety of food items spread out. a sandwich, a bowl of fruit, and a plate of vegetables are among the items on the table, each contributing to the colorful tableau. the girl is holding a fork, poised to take a bite from her sandwich. her attention is focused on the meal before her, suggesting she's savoring the flavors. the table is adorned with a striped tablecloth, adding a touch of homeliness to the scene. in the background, a man can be seen sitting on a couch, adding another layer of depth to the scene. his presence suggests a family setting, perhaps enjoying a meal together. coco sharegpt4v bias/hallucination quality diversity coverage coco sharegpt4v coco sharegpt4v • • gender err. ## evaluation setup. we analyze concise (coco captions) and enriched captions (sharegpt4v, fusecap, caps-fusion) based on the metrics in section 2.2. en-riched captions are generated for the coco training set using these approaches. we first conduct an upstream analysis of the four datasets and then a downstream analysis of captions generated by a captioner trained on each dataset. for downstream analysis, we fine-tune a pre-trained blip for 5 epochs with the adamw optimizer, generating captions for the coco validation set. detailed experimental settings are in appendix a. ## discussion on possible sources of bias to enhance descriptiveness, gec methods heavily rely on llms to improve textual alignment. however, issues with gender bias and hallucination @xcite have been explored in these llms. the enrichment process, which depends on text representations, risks incorporating these inherent biases into the final captions. furthermore, the lack of human oversight in the caption generation process can exacerbate these issues. without iterative human intervention to correct biases, the inaccuracies of llms remain unaddressed, leading to increased bias and hallucination. the problem of bias mitigation is still open to the community, and there is no golden standard approach, but we further elaborate potential strategies to mitigate bias and hallucinations as follows: based on this observation, applying the gce methods to the balanced subset would mitigate gender bias in the generated captions for future studies. ## conclusion we examined standard and llm-enriched captions for gender bias and hallucination, deriving key insights: gce-based image captioning exacerbates these bias, which are further amplified in downstream models. we argue that further efforts must be invested to the problems to strike a balance between descriptive richness and incremental bias.