A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning (2022)

Abstract: Vision-language models can encode societal biases and stereotypes, but there are challenges to measuring and mitigating these multimodal harms due to lacking measurement robustness and feature degradation. To address these challenges, we investigate bias measures and apply ranking metrics for image-text representations. We then investigate debiasing methods and show that prepending learned embeddings to text queries that are jointly trained with adversarial debiasing and a contrastive loss, reduces various bias measures with minimal degradation to the image-text representation.

## introduction large-scale, pretrained vision-language (vl) models are growing in popularity due to their impressive performance on downstream tasks with minimal finetuning. their success can be attributed to three main advances: the rise of transformers in natural language processing (nlp) @xcite , cross-modal contrastive learning @xcite and the availability of large multimodal web datasets @xcite . these models, including clip @xcite , are readily available through apis (evertrove; hug-gingface), allowing non-technical users to capitalize on their high performance 'out of the box' on zero-shot tasks @xcite . despite these benefits, an expansion in scope for downstream applications comes with greater risk of perpetuating damaging biases that the models learn during pretraining on web-scraped datasets which are too large to be manually audited for quality @xcite . cultural and temporal specificity is also of concern given models are trained on a snapshot in space and time @xcite , thus reinforcing negative stereotypes that may otherwise naturally alter through societal pressures and norm change. the risk and type of societal harm intimately interacts with the downstream task at hand. clearly, using vl models for dog-species classification poses very different dangers to projecting the similarity of human faces onto axes of criminality @xcite or homosexuality @xcite . applications of this kind are extremely hard to ethically motivate and there may be no appropriate use case that justifies their associated risks. even in more benign applications such as image search, there may be harmful consequences arising from representational and/or allocational harms. representational harms come from the technological entrenchment of stereotypical perceptions; for instance, the overrepresentation of one gender when querying for a profession (e.g., "nurse" versus "doctor") or one ethnicity in explicit and nsfw content @xcite . allocational harms arise when an individual's or group's access to resources and opportunity are differentially impacted @xcite ; for instance, if the ordering of images in search results shifts recruiters' perceptions about the real-world suitability of different peoples for different jobs @xcite . in this paper, we focus on the risk of representational harms when large-scale vl models are used to map sensitive text queries, such as "a photo of a criminal" onto face datasets. while frameworks to measure bias have been established for nlp and computer vision (cv) separately, there is considerably less work on vl @xcite . appropriate debiasing techniques for large-scale vl models are also sparse and face challenges from a lack of access to the original training data and the infeasible amount of compute required for retraining. for the successful and safe adoption of vl models, we need both effective measures of bias as well as efficient methods of debiasing. to this end, we make three contributions: (i) we investigate and evaluate different measures of bias for vl models, figure 1: our proposed debiasing method for pretrained vision-language models. sensitive text queries and images (with labeled attributes, e.g., gender) are fed to their respective frozen text and image encoders. we employ an adversarial classifier which aims to predict the image attribute labels from similarity scores between the outputs of the two encoders. learnable "debiasing" prompt tokens are prepended to the sensitive text queries and optimized to maximize the error of the adversary. in this way, biased correlations between image-text similarity scores and attribute labels are reduced whilst preventing significant degradation of the joint image-text representation. additionally, we jointly train with a contrastive loss on generic image-text pairs to further avoid degradation of the joint representation (not shown for clarity). ## debiasing the proposed debiasing method has two components: (i) the objective function to minimize for bias reduction; and (ii) the choice of parameters to optimize over in the vl model ψ to minimize (i). ## related works there have been multiple recent releases of opensource vl models @xcite @xcite , but research into bias measurement and mitigation has not kept pace, with only a few papers to date tackling these topics for vl @xcite @xcite . in this work, we therefore drew inspiration from the literature on dataset-and modellevel bias in cv and nlp @xcite . ## limitations and ethical consideration our methods and findings are subject to some limitations, as well as some ethical considerations of how bias and fairness are operationalized. our methods rest on two assumptions about the setting of the downstream application, namely that (i) the vl model is too large to be pretrained from scratch within the computational budget, and (ii) there is no access to the original training dataset. in the absence of those assumptions, we strongly encourage employing ethical dataset curation practices as well as including fairness considerations in the initial training of the model. however, in the case where our assumptions hold, our method provides a cheap, simple yet effective method for debiasing vl models. ## conclusion this paper establishes a framework for measuring and mitigating bias in vl models. firstly, we demonstrate that ranking metrics (specifically maxskew and ndkl) are effective bias measures. we report these metrics for a range of pretrained vl models for gender and racial bias in photos of faces. our results confirm previous findings in other domains that (i) more pretraining data correlates with lower model bias, and (ii) training models with ssl can reduce bias. secondly, we demonstrate a supervised adversarial debiasing method of vl models via learned "debiasing" tokens on publiclyavailable face image datasets with attribute labels. the proposed method demonstrates a substantial reduction over a suite of bias metrics for gender and race attributes, with feature degradation being wholly mitigable using joint training with an itc loss on small publicly-available image datasets. future work could include (i) debiasing during the pretraining stage, with ssl showing a promising avenue in that regard, or (ii) defining a wider diversity of attributes such as removing the harmful assumption of binary gender or considering intersectional biases. we encourage researchers in vl to continue to investigate bias in their models, be transparent in documenting model weaknesses using metrics like those proposed in this paper, and seek to apply relatively cheap and easy debiasing protocols like ours. our code, models and debiasing tokens are publicly-available 3 for the community to use in the hope that progress can be made towards the safer and fairer use of this technology in society.