Evaluating Gender Bias in Multilingual Multimodal AI Models: Insights from an I ndian Context (2024)

Abstract: We evaluate gender biases in multilingual multimodal image and text models in two settings: text-to-image retrieval and text-to-image generation, to show that even seemingly gender-neutral traits generate biased results. We evaluate our framework in the context of people from India, working with two languages: English and Hindi. We work with frameworks built around mCLIP-based models to ensure a thorough evaluation of recent state-of-the-art models in the multilingual setting due to their potential for widespread applications. We analyze the results across 50 traits for retrieval and 8 traits for generation, showing that current multilingual multimodal models are biased towards men for most traits, and this problem is further exacerbated for lower-resource languages like Hindi. We further discuss potential reasons behind this observation, particularly stemming from the bias introduced by the pretraining datasets.

## introduction in recent years, significant work has been done to ground image and language models together, to enable the ability to perform various downstream tasks like visual question answering, textprompted image generation, and image captioning. these models typically involve merging image and text transformer architectures, making use of the contextual knowledge learned by these models during pretraining and reducing the model training cost and complexity. models like blip @xcite , blip-2 @xcite , and clip @xcite are frequently used for various multimodal tasks, including dataset curation. recent models like mblip @xcite , mclip @xcite , cross-lingual clip @xcite ) further build upon these to extend image-to-text tasks into a multi-lingual realm. however, these models are designed with language inclusivity in mind, with no prior evaluation of bias. since inclusivity extends beyond just language inclusivity, this brings up the question, are these models really inclusive? as large-scale multimodal models become more integrated into global, multilingual contexts, it is essential to ensure fair representation. there are also text-to-image diffusion models like imagen @xcite , dall-e 2 @xcite , and latent diffusion @xcite , which rely on pretrained unimodal text or image models that are extended to create their multimodal models. these models perform exceedingly well on metrics for predictive performance, but their bias evaluation was largely unexplored till recently. in this work, we analyze stable diffusion 2 @xcite and alt-diffusion @xcite for gender biases in generated images. the bias evaluation of these models is extremely critical, since these models are further used for the curation of large-scale datasets used for various pretraining and fine-tuning tasks. e.g., the laion-5b dataset @xcite was curated by extracting the data from wikipedia and filtering using the clip model. it was used for training various large-scale models like blip-2. since the clip model is biased, as shown by @xcite , the laion-5b dataset is likely to show biases, since the clip model was used to filter it, and these biases are further propagated to new models trained on the laion-5b dataset. this process is called bias amplification @xcite . in this work, we take inspiration from the work by @xcite , and evaluate the gender biases portrayed by a mclipbased retrieval framework and mclip and clipbased multilingual text-to-image diffusion models. based on the work done by @xcite , we felt the need for potentially regionspecific work on evaluating gender biases in models, since traits and in-group words are likely to differ in the context of people across regions, countries, and continents. e.g., the trait indian, is likely to lead to incoherent results for people from across the globe or in other regions like north america, while evaluating it for indians in particular might portray gender biases different from the global trends for a given model. thus, in this work, we explicitly work for the indian context, and evaluate gender biases observed in textto-image retrieval and generation models for english and hindi prompts. ## related work this section builds upon prior studies to contextualize our analysis of gender bias within multilingual multimodal models. we highlight the unique contributions and limitations of existing methodologies in handling cultural and linguistic differences in ai systems. @xcite evaluated three sota image-to-text models clip, slip @xcite and blip for biases associated with social and experimental psychology, particularly associated with equating the american identity as white. upon running embedding association tests on the chicago face database @xcite , they observed that white individuals had a higher association with collective in-group words compared to asian, latina/o, and black individuals across all models. certain phrases like patriotism and born in america were more associated with white individuals. this work introduced a new direction for the evaluation of multimodal models. however, it was restricted to monolingual models trained only in english. @xcite offers an essential backdrop for the current research. this paper's comprehensive analysis of social disparities in india and their manifestation in nlp data and models lays the groundwork for understanding how cultural and linguistic diversity impacts ai fairness. the present study extends this understanding by applying these fairness considerations to the specific context of gender bias in multimodal models, thus filling a critical gap in the understanding of ai fairness in multilingual and multicultural settings. @xcite introduced the "conceptual coverage across languages" (coco-crola) technique, assessing the parity of generative textto-image systems across languages. they focused on tangible nouns and their representation in image generations across various languages. our approach is in line with their multilingual analysis but applies specifically to hindi and the indian context, providing a more targeted evaluation of biases in specific downstream tasks such as retrieval and generation. @xcite conduct a multi-dimensional analysis of bias in visionlanguage models, focusing on gender, ethnicity, and age. their study highlights the presence of harmful and stereotypical completions when subjects are input as images, which also perpetuate to downstream tasks, affecting minorities. our work extends this by examining gender bias in generated images using altdiffusion and stable diffusion 2, specifically comparing biases in english and hindi prompts and considering the impact of language and cultural contexts, thereby broadening the scope to explore multilingual biases. @xcite examine multilingual fairness in multimodal models, focusing on equal treatment across languages. their introduction of multilingual individual and group fairness concepts is pertinent to understanding gender biases in multilingual contexts. however, our study diverges by zooming in on gender bias outcomes in explicit downstream tasks, specifically within indian demographics and incorporating hindi, addressing a gap in @xcite 's research. @xcite evaluate the extensive capabilities of large-scale multilingual vision-language models in diverse tasks, such as object detection and video question answering. they also discuss biasdemographic parity in the proposed model, underscoring the significance of evaluating demographic disparities in ai systems. our work adds a crucial layer to this conversation by explicitly addressing gender biases, thereby contributing to a deeper understanding of the limitations and inherent biases in multilingual multimodal models. @xcite in their study on fairclip introduced a novel two-step debiasing method for clip-based image retrieval, to find a balance between debiasing and performance. concurrently, @xcite emphasized test-time fairness in image retrieval through post-hoc bias mitigation, modifying outputs of pre-trained models for enhanced equity. we specifically derive our measures of gender bias from these works and apply them to a multilingual context. ## conclusion and future work in this work, we conducted a gender bias evaluation of multilingual multimodal models like mclip for retrieval and image generation (using clip and mclip-based diffusion models) to evaluate the differences in gender bias observed for psychological and person trait prompts in hindi and english in the context of indian people. we observed an evident gender bias for most traits towards the male gender for both generation and retrieval, and this was further exacerbated for hindi prompts. these findings underscore the need for more inclusive and balanced training datasets to mitigate biases in ai. some relevant directions for future work include extending the scope of the study to more ethnicities and languages beyond english and hindia, which help derive more meaningful insights into the nature of gender bias in multilingual multimodal models. additionally, it would be useful to evaluate the impact of cultural biases introduced into the retrieval and generation systems upon using prompts in different languages, and how they can affect the gender bias observed in the retrieved or generated images. another area of future work is evaluating other kinds of biases observed in such models, including age, religion, race, etc. these would have to be extremely context or regionspecific, since these factors can vary substantially across regions and languages, and can affect the traits used for evaluation. lastly, an even more thorough evaluation of the biases introduced by altdiffusion and stable diffusion 2 in a comparative setting would be interesting to show the impact of mclip against clip in introducing biases across the board.