Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models (2023)

Abstract: Despite the impressive performance of current AI models reported across various tasks, performance reports often do not include evaluations of how these models perform on the specific groups that will be impacted by these technologies. Among the minority groups under-represented in AI, data from low-income households are often overlooked in data collection and model evaluation. We evaluate the performance of a state-of-the-art vision-language model (CLIP) on a geo-diverse dataset containing household images associated with different income values (DollarStreet) and show that performance inequality exists among households of different income levels. Our results indicate that performance for the poorer groups is consistently lower than the wealthier groups across various topics and countries. We highlight insights that can help mitigate these issues and propose actionable steps for economic-level inclusive AI development.

## introduction the impact of ai on the general public is rapidly growing, now getting within reach of people worldwide. more than ever, it is critical that these models work well for everyone. language and vision models are also expanding, with some already being used as foundation models @xcite , as these models are trained on enormous datasets and have been shown to possess impressive capabilities on downstream tasks across various domains. however, since most research that yielded foundation models comes from top companies in the western tech industry, it is unsurprising that these models tend to be biased and perform unequally @xmath0 countries, all from north america and europe @xcite . similarly, research by de @xcite investigates how cv models perform on diverse, crosscultural images from the dollar street dataset @xcite . this work is most similar to ours; however, their focus is on object-recognition models, while ours is on a vision-language foundation model, clip @xcite . their main finding, four years ago, is consistent with ours: "the systems perform relatively poorly on household items that commonly occur in countries with a low household income," revealing that current cv models still work significantly worse for people with low-income households. ## methodology we present the dataset, dollar street @xcite and the vision-language state-of-the-art model, clip @xcite , which we use for our experiments. by formulating and answering a series of research questions, we perform a fine-grained performance analysis of clip on a socio-economically diverse dataset across topics, income levels, and countries. the resulting insights lead to actionable steps to improve vision-language models' performance across diverse incomes. ## lessons learned and actionable steps our analyses yield several insights into the current state of vision-language models. we highlight lessons learned and propose actionable steps to ensure future work builds more equitable models and datasets. invest effort to understand the extent of the digital divide in vision-language performance. four years after de @xcite drew attention to the performance disparity across income levels for object recognition models, we show that a popular vision-language model (clip) also yields poor performance on data from lowerincome households. this new finding, together with the scarcity of research in this area, shows that the ai research community is not paying enough attention to poor socio-economic groups worldwide. the consequences can be devastating since, if neglected, ai will exponentially increase the already existing "digital divide". define evaluation metrics that represent everyone. we call for attention and scrutiny of broad performance metrics that give the illusion of high accuracy in ai models while performance on minority groups remains dangerously inadequate. one potential solution is to use the geo-location information of images when available to evaluate models by location and income, as income can often be deduced from the location. match diversity standards to model purpose. diversity standards for ai models and datasets must match the intended users. for example, an ai model built as a foundation baseline for various downstream tasks should be held to the highest standards regarding what training data is used, as it should be able to represent everyone equally. ## conclusion in this paper, we evaluated the performance of a state-of-the-art vision-language model (clip) on household images across different income groups, and different countries as found in the dollarstreet dataset. the results of our analyses demonstrated the existence of performance inequality and showed that data from lower-income groups are more likely to be represented inaccurately.