Venue: NAACL 2025

# Uplifting Lower-Income Data: Strategies for

Socioeconomic Perspective Shifts in Large Multi-modal Models

Joan Nwatu\({}^{1}\)  and Oana Ignat\({}^{2}\)  and Rada Mihalcea\({}^{1}\)

\({}^{1}\)University of Michigan, USA

\({}^{2}\)Santa Clara University, USA

_[jnwatu, mihalceaa] @umich.edu oignat@scu.edu_

###### Abstract

Recent work has demonstrated that the unequal representation of cultures and socioeconomic groups in training data leads to biased Large Multi-modal (LMM) models. To improve LMM model performance on underrepresented data, we propose and evaluate several prompting strategies using non-English, geographic, and socioeconomic attributes. We show that these geographic and socioeconomic integrated prompts favor retrieving topic appearances commonly found in data from low-income households across different countries leading to improved LMM model performance on lower-income data. Our analyses identify and highlight contexts where these strategies yield the most improvements.

## 1 Introduction

A lack of diversity in popular AI datasets (Shankar et al., 2017) leads to unequal model performance, further widening the technological gap between well-represented and underrepresented communities. While data from higher-income Western communities are readily available online, lower-income and non-Western data are often missing (Rosling et al., 2019). As a result, cost-effective methods like web scraping fail to produce diverse datasets.

One approach to building large datasets leverages LMM models to filter uncurated data based on image-text association strength scores (Fang et al., 2023). For instance, OpenAI's CLIP ViT-B/32 (Radford et al., 2021) was used to filter web-scraped images to create the LAION-5B dataset (Schuhmann et al., 2022). However, foundation LMM models like CLIP perform unequally across cultures and socioeconomic groups, favoring higher-income and Western images (Nwatu et al., 2023).

Datasets filtered by LMM models reflect the model's biases (Fang et al., 2023), often excluding underrepresented data and worsening the lack of diversity in AI models. Ignat et al. (2024) demonstrates this by showing that the LAION-5B dataset closely resembles data from Western countries, such as the United States and Canada while differing from non-Western countries' data. This leads to LMM models with uneven performance on data drawn from different locations and income groups. Therefore, our paper seeks to answer the following question: _How do we improve the performance of LMM models on lower-income and non-Western data?_

We tackle performance inequality in LMM models (Radford et al., 2021; Visheratin, 2023) through prompting that transfers the cultural knowledge embedded in language (Ventura et al., 2023; Buettner et al., 2024; Nguyen et al., 2024). Our goal is to improve the performance of LMM models on data from households with non-Western and lower socioeconomic status. Specifically, as shown in Fig. 1, we pose several research questions to evaluate the role of non-English languages, as well as prompts with geographic and socioeconomic attributes, to retrieve more diverse images.

Our contributions are summarized as follows. First, we show that a naive **prompt translation-based approach fails** to adequately address the performance gap of LMM models on lower-income

Figure 1: Low-income Image Retrieval from Dollar Street dataset (Rojas et al., 2022) using different prompt formulations. Prompts with integrated _country_ and _in-cone_ information successfully retrieve fewer standard images previously left out by the English and translated (French) prompts.



data. Second, we establish that **geographic and socioeconomic attribute integrated prompts improve** LMM performance on lower-income data. We identify contexts where these prompts work best by conducting an in-depth analysis of LMM models' understanding of these attributes and their effects on recall across data from different countries. Lastly, we share insights from our analysis demonstrating how **these attributes drive a perspective shift** that benefits the retrieval of lower-income data.

## 2 Related Work

Addressing AI Performance InequalityClass imbalances in training data contribute significantly to bias in AI models Ferrara (2024); Shankar et al. (2017); He and Garcia (2009); Pouget et al. (2024), leading to unequal outcomes in areas like facial recognition Buolamwini and Gebru (2018), healthcare Obermeyer et al. (2019), and hiring Raghavan et al. (2020). Since creating balanced datasets is challenging and costly Ignat et al. (2024); Ramaswamy et al. (2023), researchers have explored bias mitigation techniques such as data augmentation, feature importance tuning, regularization, and adversarial training Yan et al. (2020); Zafar et al. (2017); Ignat et al. (2024); Maudslay et al. (2019); Sharma et al. (2020); Navarro et al. (2024); Zhang et al. (2018). Our work is most similar to research on post-processing methods Ferrara (2023); Hardt et al. (2016); Kamiran et al. (2012); Pleiss et al. (2017) that adjust model outcomes to meet diversity standards, aiming to benefit disadvantaged groups. Prior research has shown that LMM models perform poorly on data from lower socioeconomic groups, and our analysis investigates non-invasive post-processing methods to address this issue.

Multilingual AI ModelsLanguage plays a key role in transmitting cultural knowledge Callies (2024); Sharifian (2014); Karsdorp and Fonteyn (2019); Norton (1997), as AI models often absorb biases from the language in their training data Stanczak and Augenstein (2021); Rogers et al. (2021) and model outputs can be controlled by specifying a cultural shift in perspective Ventura et al. (2023) to improve diversity. However, research Arora et al. (2023); Cao et al. (2023); AlKhamissi et al. (2024); Liu et al. (2021) shows that large language models (LLMs) and LMM models capture more cultural information from English data (mainly Western) than from non-English data. This disparity stems from differences in the quantity and quality of non-English data, translation issues, and model design Arora et al. (2023); Hershcovich et al. (2022); Nasif et al. (1991).

Similar to past studies De Vries et al. (2019); Nguyen et al. (2024) using multilingual approaches to enhance data diversity, our work explores how multilingual large multi-modal models and non-English languages can improve representation across regions and income groups.

Prompting AI ModelsRecent studies have explored prompting techniques for large language models, including both hard Petroni et al. (2019); Zhou et al. (2023) and soft prompting Huang et al. (2023); Goswami et al. (2023), to improve model adaptation for tasks like instruction tuning, and value alignment. These methods are also applied in LMM models Lu et al. (2022); Yao et al. (2024); Zhou et al. (2022). While prior work Buettner et al. (2024) has incorporated geographic and physical attributes into prompts to enhance image retrieval diversity, this research extends the investigation to non-English language prompts and socioeconomic attributes to analyze how LMM models encode representations of various topics across regions and socioeconomic status.

## 3 Methodology

We propose prompting strategies that account for language, location, and socio-economic attributes and analyze how these prompts affect the performance of a multilingual LMM model on data across different socio-economic groups, primarily focusing on lower-income data.

### Dollar Street Dataset

We use the Dollar Street Rojas et al. (2022), which contains \(38,479\) images of household items (e.g., "stoves", "cutlery", "toothbrush") spanning a large number of countries and several income levels. The dataset images were sourced from households in \(63\) countries on four continents Africa, America, Asia, and Europe). The number of images ranges from \(45\) in Canada to \(4,704\) in India, with a median of \(407\) images per country. Size and image resolutions vary slightly across data from different regions; however, the mean and median image properties per region are relatively similar.

Image Income ClassesEach image is accompanied by the monthly household income value in U.S.

 

dollars, calculated to reflect monthly consumption and adjusted for purchasing power parity to match the variance in cost of living across the different regions. The monthly income values range from \(26.98\) to \(19,671.08\).

For fair comparison across bins, we group the images using the quartile binning method, which splits the data into an approximately equal number of images per bin as shown in Rojas et al. (2022). We group the images into four income classes ("poor", "low-mid", "up-mid", and "rich" ) using quartiles as shown in Table 1. We further categorize the lowest two image income classes as _lower-income images_ and the highest two income groups as _higher-income images_.

Country Economic Classes.We group all 63 countries from Dollar Street into country economic classes based on their World Bank income classification.1 All the countries and their economic classes are shown in Appendix A.1. We further categorize the lowest two country economic classes as _lower-income countries_ and the highest two economic groups as _higher-income countries_.

Footnote 1: [https://datahelpdesk.worldbank.org/](https://datahelpdesk.worldbank.org/)

Topic Representations.There are \(291\) unique topics associated with the images in the dataset which reflect everyday household objects and human actions (e.g., "toilet paper", "get water"), some of which are subjective (e.g., "next big thing I plan to buy", "favorite sports clubs", "most loved item"). We remove nineteen subjective topics from the dataset following De Vries et al. (2019) and Nwatu et al. (2023).

### Prompt Design

We describe below the prompting strategies we use for our experiments and show examples in Figure 1.

Default English Topic Prompt.Using the topics, we formulate an English prompt without any modifications (e.g., "This is a photo of _cutlery_"), as described in Radford et al. (2021), to which we refer to as the _default English prompt_. The performance obtained using these prompts is set as our baseline.

Translated Topic Prompt.For our multilingual experiments, we investigate the impact of non-English language prompts on the Dollar Street dataset. We use the term _non-English major language_ to refer to the non-English language that is most widely spoken or most commonly used in a particular country or region.

Specifically, we pair each country with their non-English major language (e.g., _Portuguese_ for _Brazil, French_ for _Cameroon_) following the country and language information provided by official sources.2

Footnote 2: www.cia.gov/the-wordlq-factbook/Field/language/, www.nccsc.org/__data/assets/pdf_file/0024/17862/languagesycountries.pdf, www.dss.gov.au/sites/default/files/files/foi_disclosure_log/12-12-13/language-list.pdf

We identify 59/63 countries in Dollar Street where one or more major non-English languages are spoken. We also select languages covered by state-of-the-art machine translation and multilingual LMM models. There are 40 such non-English major languages, and they are listed in Appendix A.1.

Finally, we translate the _default English prompts_ to these 40 languages using the NLLB-200-distilled-600M (Costa-jussa et al., 2022), an open-source state-of-the-art neural machine translation model. Translation metrics for NLLB-200-distilled-600M are shown in Appendix Table 13 and available on HuggingFace. If an image prompt is translated into the non-English major language of the image's country of origin, it is referred to as a _native translated prompt_.

Country Suffix Topic Prompt.For our second prompting technique, we include country names as suffixes to the default English prompt (e.g., "This is a photo of _cutlery_ from _Cameroon_"). We create 63 new prompt templates by adding the country names of each of the 63 countries in Dollar Street. We refer to these prompts as _country-suffix prompts_.

Income Suffix Topic Prompt.We also create prompts by integrating socio-economic attributes (e.g., "poor country", "rich region") as suffixes to the _default English prompt_. For instance, a sample prompt is "This is a photo of _cutlery_ from _a rich country_". For more robust results, we use multiple synonyms each for the _poor_ and _rich_ attributes (e.g., "an impoverished country", "a wealthy region").

\begin{table}
\begin{tabular}{c|r}  & Quartile name & Income range \\ \hline poor & 26.9 - 95.0 \\ low-mid & 195.4 - 685.0 \\ up-mid & 694.0 - 1,998.0 \\ rich & 2,001.0 - 19,671.0 \\ \end{tabular}
\end{table}
Table 1: Income quartiles and their ranges for all the images in Dollar Street.

 

We also create prompts using neutral suffixes (e.g., "a country", "a home"). We refer to these prompts as _income-suffix prompts_.

### State-of-the-art LMM Model

For our evaluation, we chose NLLB-CLIP-SigLIP Visheratin (2023), a state-of-the-art multilingual LMM model, with broad reach across many low-resource languages and superior performance among other models.3 The model consists of an image encoder from the SigLIP model Beyer et al. (2022); Zhai et al. (2023) and a text encoder from the NLLB model Costa-jussa et al. (2022). The model supports the 201 languages of the Flores-200 Costa-jussa et al. (2022) and has recorded groundbreaking results on the Crossmodal-3600 dataset Thapliyal et al. (2022), especially on low-resource languages.

Footnote 3: [https://huggingface.co/visheratin/nlb-clip-large-siglip](https://huggingface.co/visheratin/nlb-clip-large-siglip)

## 4 Research Questions

We perform several analyses to answer three research questions that uncover and mitigate limitations in the performance of LMM models across different countries and socioeconomic groups.

### RQ1. Do translated prompts improve retrieval performance for lower-income images?

We calculate the cosine similarities between image and translated prompt text embeddings for each image-topic pair across English and 40 non-English languages, generating 41 alignment scores per image. The alignment scores with _default English prompts_ serve as our baseline.

We compute Recall scores by selecting the top \(N\) images with the highest alignment scores for each topic, where \(N\) represents the number of ground truth images. We then group and analyze the Recall scores across different countries and image income classes and present our findings below.

_Native translated prompts_**perform consistently worse than _English prompts_ on lower-income images from their respective countries.We focus our analysis on images from the two lowest image income groups, i.e., _poor_ and _low-middle_ as grouped in 3.1. After excluding 20 countries without data for these income groups (e.g., _Russia_, _Turkey_), we retain 39/59 countries and 28/40 non-English languages for the study. Each country is paired with its native non-English language, and we compare Recall scores for the _native translated prompts_ to those for the _default English prompts_. The average Recall across all countries and scores from four countries are displayed in Figure 2.

For 35 out of 39 countries, the _native translated prompts_ underperform compared to the _default English prompts_. The exceptions include _Burkina Faso_, _Nigeria_, _Pakistan_ and _Tanzania_, where _native translated prompts_ in _French_ (diff. of 1.0), _Hausa_ (diff. of 0.2), _Urdu_ (diff. of 0.7) and _Swahili_ (diff. of 1.5), respectively, outperform English prompts. Overall, native translated prompts generally fail to retrieve diverse images, as depicted by the example using French prompt in Fig. 1.

#### The best-performing non-English language often differs from the country's native language.

We analyze the Recall scores for lower-income images across 28 language prompts used in different countries and find that the best-performing language prompts often differ from the countries' non-English major languages. Specifically, in 24 out of 39 countries, non-English language prompts outperform the default English prompts, yet these top-performing languages are not typically spoken in the respective countries. As illustrated in Figure 3, for 37/39 countries, the language with the highest Recall score (highlighted in yellow) differs from the country's primary non-English language (highlighted in cyan), with exceptions in Indonesia and Pakistan, where they coin

Figure 2: NLLB SigLIP Recall (%) over poor and lower-middle income images from four countries, one from each of the four continents: Asia, Africa, America, and Europe for English and native translated prompts. _Best viewed in color._



cide (highlighted in bold red).

_Translated prompts decrease performance for all image income classes across all countries._ We analyze the impact of 40 non-English language prompts on all images from Dollar Street, covering 59 countries and we group Recall scores by image income classes. By comparing Recall scores between _default English prompts_ and _native translated prompts_, we assess the effect of each non-English language on four income classes and show the difference in scores in Appendix Table 10.

We show in Table 2 the average Recall and drops in performance across all 40 translated prompts for each image income class. The results indicate that higher-income classes, specifically the _rich_ and _up-mid_ groups, experience the largest drops in performance with translated prompts. This may be due to the overrepresentation of images from these income groups in AI models and datasets, positioning them as the "standard" representation. Similarly, English, the dominant training language, is seen as the "standard" for textual data, so non-English prompts may signal a deviation from this standard, resulting in poorer model performance.

### RQ2. Does adding country information improve retrieval performance for lower-income images?

We compute cosine similarity scores between NLLB-CLIP-SigLIP image embeddings and the text embeddings of 63 _country suffix prompts._, yielding 63 image-topic alignment scores per image. Using the alignment scores from the _default English prompts_ as a baseline, we follow the procedure outlined in Section 4.1 to calculate Recall scores for each topic with the _country suffix prompts_. We then analyze the impact of adding country suffixes to text prompts and present the results in the following sections.

_Country-suffix prompts perform consistently better than default English prompts_ on lower-income images.Focusing on low-income data, we filter out 21 countries without images from _poor_ or _low-mid_ income households, leaving 42 countries for analysis. In Figure 4, we present the average Recall scores across all countries using both _default English_ and _country-suffix prompts_, along with results from four sample countries from different continents.

Our findings indicate that in 38/42 countries, adding a _country-suffix_ to text prompts improves Recall performance for lower-income images compared to _default English prompts_. Exceptions include _Bolivia_, _Brazil_, _Jordan_, and the _United States_. _Country-suffix prompts_ are thus more effective in retrieving diverse images, as demonstrated by the Cameroon example in Fig. 1.

**A country's economic status influences the performance of its _country-suffix prompt_ across different image income classes.Country suffix prompts improve LMM model performance for lower-income images (_poor_) but reduce performance for higher-income images. Using World Bank income classifications, we calculate Recall

\begin{table}
\begin{tabular}{l|c c c c} _Non-English languages_ & \multicolumn{4}{c}{Image Income Class} \\  & **Poor \(\Delta\)** & **Low-mid \(\Delta\)** & **Up-mid \(\Delta\)** & **Rich \(\Delta\)** \\ _(Average)_ & 20.2 (-2.2) & 31.0 (-4.9) & 37.8 (-7.8) & 36.1 (-7.5) \\ \end{tabular}
\end{table}
Table 2: Average differences between Recall scores for non-English language prompts and Recall scores for default English prompts for all data, grouped by image income classes. We find that non-English prompts lead to a decrease in Recall scores across all income classes.

Figure 3: Recall scores for _lower_ income images from 39 countries and 28 languages. The cyan highlight shows the Recall for a country’s native translated language, the yellow highlight shows the best-performing language recall, and the red shows the Recall for the language that is both the native and highest performing for that country. _Best viewed in color_.

 

scores across four country suffix groups (poor, low-mid, up-mid, and rich) and four image income classes (based on household income). For each image income class, we aggregate Recall scores and compare them with those from default English prompts, as shown in Table 3, with detailed results in Appendix Table 11 and Table 12. For example, Table 3 shows that Recall of images from poor households using _country suffixes_ of poor countries is 31.2, a 9.7 increase from _default English prompt_ performance on that group.

The analysis reveals that country suffixes from poor, low-mid, and up-mid income categories improve Recall for images from poor households, while reducing Recall for higher-income groups (low-mid, up-mid, rich).

Interestingly, country suffixes tend to favor image retrieval from income groups that match or are close to their own economic classification. When the four income classes are re-categorized into two classes (lower-income: poor, low-mid and higher-income: up-mid, rich), we find that in 48/63 cases, the image income category with the highest Recall corresponds to the country suffix's economic class, demonstrating the alignment between income levels of country-suffixes and retrieval performance.

The best-performing country suffixes for lower-income images from a continent are from the same continent.

We calculate Recall results for lower-income images from 42 countries using the 42 country suffix prompts, yielding a total of 1,764 Recall scores. Using country-suffixes, we group these scores by continent and further categorize them based on the World Bank Income classes of the respective country-suffixes. We present the average Recall and differences compared to default English prompts for each group in Table 4.

Our findings emphasize the significance of regional specificity in data collection, as the best-performing suffixes align with their respective continents (shown by the diagonal of bold values in Table 4). The results indicate that lower-income images from African nations benefit significantly from including country suffixes. In contrast, data from America and Asia show no Recall improvements, underscoring the necessity for tailored data collection strategies across regions. Notably, lower-income images from African countries exhibit a Recall score of 36.6, reflecting the highest performance increase of 15.7 when usi

\begin{table}
\begin{tabular}{p{40.0pt}|l l|l l l l} \multicolumn{1}{p{40.0pt}}{} & \multirow{2}{*}{**Image by**} & \multirow{2}{*}{**Country by**} & \multicolumn{4}{c}{**Country Suffix**} \\ \cline{3-8} \multicolumn{1}{p{40.0pt}}{**Continent**} & & & **Affrica**\(\Delta\) & **America**\(\Delta\) & **Asia**\(\Delta\) & **Europe**\(\Delta\) \\ \hline  & Poor & 36.6 (+1.55) & 27.2 (+6.6) & 23.0 (+2.1) & 19.7 (+1.2) \\ Africa & Low-mid & 37.3 (+0.3) & 31.3 (+2.4) & 22.5 (+7.1) & 24.9 (+2.7) \\  & Up-mid & 24.3 (+2.1) & 21.6 (+6.0) & 17.2 (+5.0) & 18.1 (+4.1) \\  & **Average** & **32.7 (+9.4)** & **26.7 (+3.3)** & **22.0 (+1.4)** & **20.7 (-2.7)** \\ \hline  & Low-mid & 23.5 (+2.6) & 26.8 (+0.4) & 27.6 (+5.2) & 22.4 (+8.0) \\ America & Up-mid & 23.4 (+1.0) & 35.2 (+0.9) & 23.4 (+0.9) & 28.9 (+5.4) \\  & Rich & 24.1 (+5.5) & 30.4 (+2.8) & 22.8 (+3.1) & 26.3 (+9.6) \\  & **average** & **22.8 (+9.7)** & **30.5 (+6.7)** & **22.3 (+1.0)** & **22.5 (+6.0)** \\ \hline  & Low-mid & 29.4 (+2.4) & 30.7 (+1.1) & 32.7 (+0.8) & 22.9 (+3.9) \\ Asia & Up-mid & 28.1 (+5.1) & 32.5 (+0.6) & 34.6 (+1.5) & 33.3 (+2.8) \\  & Rich & 30.1 (+1.4) & 33.3 (+1.7) & 36.0 (+0.9) & 39.6 (+5.4) \\  & **Average** & **29.5 (+7.2)** & **32.2 (+4.5)** & **34.4 (+2.2)** & **32.6 (+4.0)** \\ Europe & Low-mid & 19.5 (+2.7) & 29.3 (+1.4) & 26.4 (+1.9) & 43.4 (+1.0) \\  & Our-mid & 24.6 (+1.6) & 26.7 (+9.7) & 23.1 (+1.3) & 35.9 (+5.0) \\  & **Average** & **20.6 (+19.9)** & **28.0 (+11.9)** & **24.8 (+15.1)** & **40.1 (+0.3)** \\ \end{tabular}
\end{table}
Table 4: Average NLLB SigLIP Recall scores for each continent and the average difference between default English prompt Recall and country suffix Recall from the four continents, grouping lower-income images according to continents and further into groups of countries arranged by income class categories based on their World Bank economic classification. _Best viewed in color._

Figure 4: Recall (%) with NLLB SigLIP over _poor_ and _lower-middle_ income images from four countries from Asia, Africa, America, and Europe, for English and Country Suffix prompts. _Best viewed in color._



The positive impact of country suffix prompt additions is particularly pronounced for Africa, as the prompts enhance performance on underrepresented data by shifting model inference from its learned standard. This effect is crucial given the current datasets often lack representation from African countries and poor households. Additionally, the similarities among African countries contribute to this improved performance.

Meanwhile, we find no Recall enhancements for higher-income data, regardless of the alignment between images and country suffixes (see Table 5). English and the income suffix prompts. Notably, the _poor_ income suffix demonstrates the best Recall, effectively retrieving a diverse array of images, as shown by the example in Fig. 1. Recall scores for four sample countries are in Appendix Figure 7.

Images from the _poor_ income group benefit the most from income suffixes.We group the data into four income groups (by household income) and further categorize them according to the World Bank income classification of their country of origin. In Table 6, we show the Recall scores and performance improvements relative to the default English prompts for each data group.

We find that income suffixes predominantly benefit data from poor households and some from low-mid income households, while data from other income groups do not show Recall increases.

An interesting finding is that all income suffixes, including _rich_ and _neutral_, result in decreased Recall for higher-income images (i.e., _up-mid_ and _rich_). This suggests that default English prompts yield the best results for higher-income images, likely due to their high representation in AI models and datasets as the "standard representation." Consequently, the inclusion of socioeconomic status information may lead the model to prioritize lower-income images over higher-income ones. This phenomenon is evident in the results, which show Recall improvements for lower-income images while diminishing Recall for higher-income images, potentially indicating a shift in the model's perspective away from its default understanding of the topic.

Figure 5: Average Recall with NLLB SigLIP over _poor_ and _lower-middle_ income images, for English and Income Suffix prompts. _Best viewed in color._

\begin{table}
\begin{tabular}{c|l|r r r r} \hline
**Image by** & **Country by** & \multicolumn{4}{c}{**Country Suffix**} \\
**Continent** & **Income** & **Africa**\(\Delta\) & **America**\(\Delta\) & **Alaia**\(\Delta\) & **Europe**\(\Delta\) \\ \hline  & Poor & 42.3 (-1.4) & 40.3 (-3.4) & 36.0 (-7.7) & 40.3 (-3.4) \\ Affica & Low-mid & 41.8 (-8.9) & 39.7 (-10.9) & 37.0 (-13.6) & 40.7 (-9.9) \\  & Up-mid & 33.3 (-5.9) & 32.4 (-6.8) & 25.7 (-13.5) & 33.4 (-5.3) \\  & **Apaque** & **39.1 (-5.0)** & **37.5 (-7.0)** & **32.9 (-11.6)** & **38.1 (-6.4)** \\ \hline  & Up-mid & 24.3 (-3.0) & 37.6 (-13.6) & 25.6 (-13.3) & 37.2 (-8.0) \\ Annetica & Rich & 28.7 (-33.4) & 45.7 (-16.4) & 32.5 (-29.6) & 44.7 (-17.4) \\  & **Average** & **26.5 (-2.1)** & **47.1 (-22.0)** & **29.7 (-24.0)** & **41.9 (-1.2)** \\ \cline{2-6}  & Low-mid & 33.5 (-27.2) & **38.4 (-14.0)** & **40.1 (-10.5)** & **43.4 (-7.6)** \\ Asia & Up-mid & 30.9 (-12.0) & 40.5 (-14.1) & 41.2 (-10.7) & 46.5 (-5.4) \\  & Right & **28.2 (-19.9)** & 36.1 (-16.0) & 36.5 (-11.6) & 40.1 (-8.0) \\  & **Average** & **31.0 (-19.4)** & **38.3 (-12.0)** & **39.4 (-10.9)** & **43.3 (-7.0)** \\ \hline  & Low-mid & 22.3 (-23.9) & 33.3 (-13.1) & 26.6 (-19.6) & 42.2 (-0.4) \\ Europe & Urban & 19.8 (-25.0) & 29.2 (-14.7) & 24.5 (-19.4) & 40.7 (-2.2) \\  & Rich & 19.0 (-20.8) & 27.8 (-12.0) & 21.5 (-18.3) & 37.6 (-2.2) \\  & **Average** & **20.1 (-23.2)** & **30.6 (-13.3)** & **24.2 (-19.1)** & **40.2 (-3.3)** \\ \hline \end{tabular}
\end{table}
Table 5: Grouping higher income images according to continents and further into groups of countries arranged by income class categories based on their World Bank economic classification, this table shows the average NLLB SigLIP Recall scores for each continent and the average difference between default English prompt Recall and country suffix Recall from the four continents.

 



### Results Significance and Generalizability

We conducted the Wilcoxon Signed Rank (Woolson, 2005) test (p-value < 0.05) to assess the statistical significance of our findings. The results indicated that the differences between the default English prompt results and each prompt intervention were statistically significant, except for the 'rich' and 'neutral' income suffix prompts (more details in Appendix Table 14).

Although our primary focus is on the NLLB-CLIP-SigLIP results, we confirmed that these findings are consistent across the two other LMM models we tested (Open AI's CLIP ViT B/32 (Radford et al., 2021) and Sentence Transformers clip-ViT-B-32-multilingual-v1 (Reimers and Gurevych, 2019)). A summary of results from these additional models is included in Table 7 and Table 8.

## 5 Lessons Learned

We highlight key insights learned from our findings and present them below.

Current multilingual LMM models do not significantly improve diversity and representationOur results from Section 4.1 demonstrate that English prompts perform better on lower (and higher) income images than prompts translated to a non-English language widely spoken in the region where the data was collected. Since the quality of translations, quantity of training data available for these languages, and consequently, the performance of AI models in these languages is lower than that of English, these findings are not very surprising. We can look forward to better non-English language performance as multilingual LMM models improve.

Location and socio-economic attributes improve retrieval performance for lower-income imagesWe find that adding geographical and socioeconomic attributes (including rich and neutral attributes) to prompts leads to an increased model preference for lower-income images over higher-income images, as demonstrated in Section 4.2. Images from poor households typically suffer the most from underrepresentation as they differ the most from the type of images available on the internet (Rosling et al., 2019). Since LMM models have learned representations from high-income images as the standard, then adding more information to the prompt (such as country suffixes like 'Malawi', income suffixes describing poverty or wealth, or neutral suffixes like 'a place') shifts the perspective of the model to retrieve images that are more diverse and less contained to the learned 'standard'.

Images with less standard topic appearances are retrieved using income suffix and country suffix promptsInspection of the retrieved images reveals that images with topic appearances commonly found in lower-income households previously not retrieved by the default English prompts are being retrieved with these prompts as shown in

\begin{table}
\begin{tabular}{l|c|c c c} \hline \multicolumn{1}{r|}{**Images by**} & \multicolumn{3}{c}{**Country**} & \multicolumn{3}{c}{**Income Suffra**} \\ \cline{3-6}
**Income** & **by Income** & **Poor** \(\Delta\) & **Rich** \(\Delta\) & **Neutral** \(\Delta\) \\ \hline \multirow{4}{*}{Poor} & Poor & 26.8 (+4.4) & 21.9 (-0.5) & 20.7 (-1.7) \\  & Low-mid & 30.7 (+6.7) & 26.0 (+3.6) & 24.6 (+2.2) \\  & Up-mid & 31.9 (+9.5) & 28.0 (+5.6) & 26.3 (+3.9) \\  & **Average** & **29.6** (+7.2) & **25.3** (+2.9) & **23.9** (+1.5) \\ \hline \multirow{4}{*}{Low-mid} & Poor & 33.3 (-2.6) & 33.3 (-2.6) & 33.5 (-2.4) \\  & Low-mid & 36.5 (+0.6) & 35.6 (-0.3) & 35.7 (-0.2) \\  & Up-mid & 30.6 (-5.3) & 30.5 (-5.4) & 30.3 (-5.6) \\  & Rich & 35.5 (-0.5) & 38.1 (-2.2) & 37.2 (+1.3) \\  & **average** & **34.0** (-2.0) & **34.4** (+1.5) & **37.2** (-1.7) \\ \hline \multirow{4}{*}{Up-mid} & Poor & 31.4 (-14.2) & 36.4 (-9.2) & 32.8 (-12.8) \\  & Low-mid & 37.6 (-8.1) & 42.4 (-3.2) & 44.5 (-1.1) \\  & Up-mid & 33.0 (-12.6) & 38.0 (-7.6) & 41.3 (+4.3) \\  & Rich & 28.4 (-17.4) & 33.4 (-12.2) & 36.3 (-9.3) \\  & **average** & **32.6** (-13.1) & **37.6** (-8.1) & **38.7** (-6.9) \\ \hline \multirow{4}{*}{Rich} & Poor & 29.6 (-14.0) & 42.6 (-1.0) & 42.4 (-1.2) \\  & Low-mid & 30.6 (-13.0) & 38.0 (-5.6) & 38.5 (-5.1) \\ \cline{1-1}  & Up-mid & 25.8 (-17.8) & 33.8 (-9.8) & 36.1 (-7.5) \\ \cline{1-1}  & Reli & 25.5 (-18.1) & 33.8 (-9.8) & 36.5 (-7.1) \\ \cline{1-1}  & **average** & **27.9** (-15.7) & **37.1** (+6.6) & **38.4** (-5.2) \\ \hline \end{tabular}
\end{table}
Table 6: Average NLLB SigLIP Recall scores for each category and the average difference between default English prompt Recall and income suffix recall, grouping images according to household income level and separating countries into income class categories based on their World Bank economic classification.

\begin{table}
\begin{tabular}{l|c|c c c c} \hline \multirow{2}{*}{Model} & \multicolumn{5}{c}{Suffix Prompts} \\ \cline{2-5}  & English & Country & Poor & Rich & Neutral \\ \hline NLLB SigLIP & 30.3 & 41.6 & 32.1 & 30.2 & 29.5 \\ Sentence Transformers MCLIP & 22.2 & 44.9 & 28.6 & 26.7 & 25.2 \\ Open AI CLIP VIT 32/B & 25.4 & 45.0 & 30.0 & 28.1 & 28.3 \\ \hline \end{tabular}
\end{table}
Table 7: Average Recall over lower-income images across 39 countries for English, Country Sarkis, and three Income Supfix prompts for three LMM models

Figure 1. For example, _pit latrines_ and _forest-style toilets_ previously left out by the default English prompts are retrieved using country suffixes (_Burundi_ and _Cameroon_) and the _poor_ income suffixes. Another example is "leaves" as "toilett paper" retrieved by _Liberia_ and _Cameroon_ country suffixes but excluded by the default English prompt.

## 6 Conclusion

In this paper, we addressed the uneven performance of LMM models across different countries and income levels. We explored three attribute-integrated prompting strategies: (1) translation of text prompts to native non-English languages, (2) addition of geographic information, and (3) addition of socioeconomic attributes. We found that integrating geographical and socioeconomic information into prompts enhances LMM model performance on images from lower-income households and retrieves more diverse label representations. Furthermore, we identified and highlighted the contexts where the proposed prompting techniques work best and shared our insights to improve representation in LMM models and datasets. Our code can be used to evaluate the performance of other LMM models and datasets and is publicly available at Analysis for Uplifting lower-income data.

### Limitations

Translation QualityWe note that, while NLLB-200-distilled-600M is reputed as a SOTA machine translation model, it does not have perfect accuracy on machine translation across all the languages it supports. We acknowledge that the quality of translations obtained from NLLB-200-distilled-600M greatly impacts our results.

Data CoverageOur study is constrained by the reach of the Dollar Street dataset and the number of contributions obtained from each region. Therefore, we do not account for data from other regions that are not included in the dataset.

Choice of AttributesWe acknowledge that other attributes (e.g., physical attributes like color and material) of the objects in the images could be integrated into prompts to improve performance. However, we choose to focus on geographic and socioeconomic attributes since they are broad enough to include all possible topic appearances related to that attribute and their impact on data belonging to different countries and income groups can measured directly.

Diverse Data AvailabilityWhile our methods facilitate the improvement of diversity during dataset annotation, these strategies cannot overcome the representation issues within the actual pool of images available for annotation.

### Ethics Statement

Through this work, we aim to contribute toward improving diversity in AI models and even out the disparate impact of these models on the public, especially on underrepresented groups. The strategies discussed in our work can be used to prioritize the retrieval of lower-income images for balancing skewed data representation or domain-specific applications in AI. However, we do not encourage the use of these strategies to promote over-representation or the inclusion of one group over another in contexts that affect all members of the general public.

Our decision to use the NLLB-SigLIP model exemplifies our commitment to inclusive models that benefit as many people as possible, especially underrepresented groups. While researching technologically advanced communities is easier and less resource-intensive, we stress the importance of making AI design decisions that do not exclude communities with limited access to technology.

## Acknowledgements

We are grateful to the Language and Information Technologies (LIT) lab members at the University of Michigan for their insightful discussions and feedback during the project's early stages. This project was partially funded by a grant from the Department of State (#STC10023GR0014), a grant from OpenAI and a Microsoft Foundational Model grant. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Department of State, OpenAI, or Microsoft.

## References

* AlKhamissi et al. (2024) Badr Alkhamissi, Muhammad ElNoktrashy, Mai Alkhamissi, and Mona Diab. 2024. Investigating cultural alignment of large language models. _arXiv preprint arXiv:2402.13231_.
* Arora et al. (2023) Arnav Arora, Lucie-aimee Kaffee, and Isabelle Augenstein. 2023. Probing pre-trained language models for 

cross-cultural differences in values. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (CSNLP), Dubrovnik, Croatia, pp. 114-130. External Links: Link Cited by: SS2.
* L. Beyer, X. Zhai, and A. Kolesnikov (2022)Big vision. GitHub. Note: [https://github.com/google-research/big_vision](https://github.com/google-research/big_vision) Cited by: SS5.
* K. Buettner, S. Malakouti, X. Lorraine Li, and A. Kovashka (2024)Incorporating geo-diverse knowledge into prompting for increased geographical robustness in object recognition. arXiv preprint arXiv:2401.01482. Cited by: SS2.
* J. Buolamwini and T. Gebru (2018)Gender shades: intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pp. 77-91. Cited by: SS2.
* M. Callies (2024) Cultural conceptualisations in nigerian pidgin english proverbs. World Englishes. Cited by: SS2.
* Y. Cao, L. Zhou, S. Lee, L. Cabello, M. Chen, and D. Hershcovich (2023)Assessing cross-cultural alignment between chatgPT and human societies: an empirical study. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), Dubrovnik, Croatia, pp. 53-67. External Links: Link Cited by: SS2.
* M. R. Costa-jussa, J. Cross, O. Celebi, M. Elbayad, K. Heafield, K. Heffernan, E. Kalbassi, J. Lam, D. Licht, J. Maillard, et al. (2022)No language left behind: scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. Cited by: SS2.
* T. De Vries, I. Misra, C. Wang, and L. Van der Maaten (2019)Does object recognition work for everyone?. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 52-59. Cited by: SS2.
* A. Fang, A. Madappally Jose, A. Jain, L. Schmidt, A. Toshev, and V. Shankar (2023)Data filtering networks. arXiv preprint arXiv:2309.17425. Cited by: SS2.
* E. Ferrara (2023)Fairness and bias in artificial intelligence: a brief survey of sources, impacts, and mitigation strategies. Sci6 (1), pp. 3. Cited by: SS2.
* E. Ferrara (2024)The butterfly effect in artificial intelligence systems: implications for ai bias and fairness. Machine Learning with Applications15, pp. 100525. Cited by: SS2.
* K. Goswami, S. Karanam, P. Udhayanan, B. Vasan Srinivasan, et al. (2023)Contextual prompt learning for vision-language understanding. arXiv preprint arXiv:2307.00910. Cited by: SS2.
* M. Hardt, E. Price, and N. Srebro (2016)Equality of opportunity in supervised learning. Advances in neural information processing systems29. Cited by: SS2.
* H. He and E. A. Garcia (2009)Learning from imbalanced data. IEEE Transactions on knowledge and data engineering21 (9), pp. 1263-1284. Cited by: SS2.
* D. Hershcovich, S. Frank, H. Lent, M. de Lloneux, M. Abdou, S. Brandl, E. Bugliarello, L. Capello Piqueras, I. Chalkidis, R. Cui, C. Fierro, K. Margatina, P. Rust, and A. Sogaard (2022)Challenges and strategies in cross-cultural NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, pp. 6997-7013. Cited by: SS2.
* Q. Huang, X. Dong, D. Chen, W. Zhang, F. Wang, G. Hua, and N. Yu (2023)Diversity-aware meta visual prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10878-10887. Cited by: SS2.
* O. Ignnat, L. Bai, J. Narutu, and R. Mihalcea (2024)Annotations on a budget: leveraging geo-data similarity to balance model performance and annotation cost. Cited by: SS2.
* Faisal Kamiran, Asim Karim, and Xiangliang Zhang (2012)Decision theory for discrimination-aware classification. In 2012 IEEE 12th International Conference on Data Mining, pp. 924-929. Cited by: SS2.
* F. Karsdorp and L. Fonteyn (2019)Cultural entrenchment of folktales is encoded in language. Palgrave Communications5 (1). Cited by: SS2.
* F. Liu, E. Bugliarello, E. Maria Ponti, S. Reddy, N. Collier, and D. Elliott (2021)Visually grounded reasoning across languages and cultures. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10467-10485. Cited by: SS2.
* Y. Lu, J. Liu, Y. Zhang, Y. Liu, and X. Tian (2022)Prompt distribution learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5196-5205. Cited by: SS2.
* Rowan Hall Maudslay, H. Gonen, R. Cottrell, and S. Teufel (2019)It's all in the name: mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871. Cited by: SS2.
* E. G. Nasif, H. Al-Daeaj, B. Ebrahimi, and M. S. Thibodeaux (1991)Methodological problems in cross-cultural research: an updated review. MIR: Management International Review, pp. 79-91. Cited by: SS2.
* M. Navarro, C. Little, G. I. Allen, and S. Segarra (2024)Data augmentation via subgroup mixup for improving fairness. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7350-7354. Cited by: SS2.

 

* Nguyen et al. (2024) Thao Nguyen, Matthew Wallingford, Sebastian Santy, Wei-Chiu Ma, Sewoong Oh, Ludwig Schmidt, Pang Wei Koh, and Ranjay Krishna. 2024. Multilingual diversity improves vision-language representations. _arXiv preprint arXiv:2405.16915_.
* Norton (1997) Bonny Norton. 1997. Language, identity, and the ownership of english. _TESOL quarterly_, 31(3):409-429.
* Nwatu et al. (2023) Joan Nwatu, Oana Ignat, and Rada Mihalcea. 2023. Bridging the digital divide: Performance variation across socio-economic factors in vision-language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 10686-10702, Singapore. Association for Computational Linguistics.
* Obermeyer et al. (2019) Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. _Science_, 366(6464):447-453.
* Petroni et al. (2019) Fabio Petroni, Tim Rocktaschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
* Pleiss et al. (2017) Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. _Advances in neural information processing systems_, 30.
* Pouget et al. (2024) Angeline Pouget, Lucas Beyer, Emanuele Bugliarello, Xiao Wang, Andreas Peter Steiner, Xiaohua Zhai, and Ibrahim Alabdulmohsin. 2024. No filter: Cultural and socioeconomic diversityin contrastive vision-language models. _arXiv preprint arXiv:2405.13777_.
* Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In _International Conference on Machine Learning_.
* Raghavan et al. (2020) Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In _Proceedings of the 2020 conference on fairness, accountability, and transparency_, pages 469-481.
* Ramaswamy et al. (2023) Vikram V Ramaswamy, Sing Yu Lin, Dora Zhao, Aaron B Adcock, Laurens van der Maaten, Deepti Ghadiyaram, and Olga Russakovsky. 2023. Beyond web-scraping: Crowd-sourcing a geographically diverse image dataset. _arXiv preprint arXiv:2301.02560_.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
* Rogers et al. (2021) Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2021. A primer in bertology: What we know about how pert works. _Transactions of the Association for Computational Linguistics_, 8:842-866.
* Rojas et al. (2022) William A Gaviria Rojas, Suduya Diamos, Keertan Rangan Kini, David Kanter, Vijay Janapa Reddi, and Cody Coleman. 2022. The dollar street dataset: Images representing the geographic and socioeconomic diversity of the world. _Advances in Neural Information Processing Systems_, 35:12979-12990.
* Rosling et al. (2019) Han Rosling, Ola Rosling, and Anna Rosling Ronnlund. 2019. _Factfulness_. Lindhardt og Ringhof.
* Schuhmann et al. (2022)Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worksmann, Patrick Schramowski, Srivarasa Kundurthy, Katherine Crovson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. 2022. Liaion-5b: An open large-scale dataset for training next generation image-text models. In _Advances in Neural Information Processing Systems_, volume 35, pages 25278-25294. Curran Associates, Inc.
* Shankar et al. (2017) Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. 2017. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. _arXiv preprint arXiv:1711.08536_.
* Sharifian (2014) Farzad Sharifian. 2014. Language and culture: Overview. _The Routledge handbook of language and culture_, pages 3-17.
* Sharma et al. (2020) Shubham Sharma, Yunfeng Zhang, Jesus M. Rios Aliaga, Djallel Bouneffouf, Vinod Muthusamy, and Kush R. Varshney. 2020. Data augmentation for discrimination prevention and bias disambiguation. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_, AIES '20, page 358-364, New York, NY, USA. Association for Computing Machinery.
* Stanczak and Augenstein (2021) Karolina Stanczak and Isabelle Augenstein. 2021. A survey on gender bias in natural language processing. _arXiv preprint arXiv:2112.14168_.
* Thalpiyal et al. (2022) Ashish Thalpiyal, Jordi Pont-Tuset, Xi Chen, and Radu Soricut. 2022. Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset. In _EMNLP_.
* Ventura et al. (2023) Mor Ventura, Eyal Ben-David, Anna Kornonen, and Roi Reichart. 2023. Navigating cultural chisms: Exploring and unlocking the cultural poov of text-to-image models. _arXiv preprint arXiv:2310.01929_.
* Visheratin (2023) Alexander Visheratin. 2023. Nllb-clip-train performant multilingual image retrieval model on a budget. _arXiv preprint arXiv:2309.01859_ 

* [Woodson2005] Robert F Woolson. 2005. Wilcoxon signed-rank test. _Encyclopedia of Biostatistics_, 8.
* Yan et al. (2020) Shen Yan, Hsien-te Kao, and Emilio Ferrara. 2020. Fair class balancing: Enhancing model fairness without observing sensitive attributes. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_, pages 1715-1724.
* Yao et al. (2024) Yuan Yao, Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun. 2024. Cpt: Colorful prompt tuning for pre-trained vision-language models. _AI Open_, 5:30-38.
* Zafar et al. (2017) Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In _Proceedings of the 26th international conference on world wide web_, pages 1171-1180.
* Zhai et al. (2023) Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. _arXiv preprint arXiv:2303.15343_.
* Zhang et al. (2018) Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_, pages 335-340.
* Zhou et al. (2022) Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. _International Journal of Computer Vision_, 130(9):2337-2348.
* Zhou et al. (2023) Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Gotterell, and Minnaya Sachan. 2023. Controlled text generation with natural language instructions. In _International Conference on Machine Learning_, pages 42602-42613. PMLR.

## Appendix A Appendix

### Non-English Languages

We use the following non-English languages in our experiments. _'German', 'Spanish', Portuguese', 'French', 'Chinese', 'Czech', 'Danish', Arabic', 'Hindi', 'Indonesian', 'Farsi-Persian', Italian', 'Russian', 'Mongolian', 'Burmese', Dutch', 'Urdu', 'Romanian', 'Serbian', 'Korean', Swedish', 'Thai', 'Thurkish', 'Ukrainian', 'Vietnamese', 'Bengali', 'Khmer', 'Oromo', 'Ewe', Creole', 'Swahili', 'Nepali', 'Hausa', 'Kyrgyz', Tagalog', 'Kinyaruvanda', 'Somali', 'Zulu', 'Sinhala', 'Shona'

Figure 6: Average Recall over lower-income images across 39 countries for English, Country Suffix, and Income Supfix prompts



* [269] M. C. Davis, J. R. Friedman, and A. Krizhevsky, editors: Learning to be effective for large data sets using deep learning. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 1-40, 2015.
* [270] D. F. Kingma and P. Ba, Adam: a method for stochastic optimization. _arXiv preprint arXiv:1312.6812_ (2013).
* [271] L. Bottou, I. Guyon, Y. Bengio, and G. Hinton, editors: Machine learning with applications in neural networks. _Advances in Neural Information Processing Systems_, pages 135-146, 2014.
* [272] S. Bermanet, T. Darrell, and E. Rabinovich, editors: Learning to learn from incomplete data. In _Proceedings of the 2nd international conference on machine learning systems_, volume 1, pages 1-10, 2015.
* [273] H. Chen, X. Li, and Z. Zhang, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI)_, pages 1-10, 2015.
* [274] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [275] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge Discovery and Data Mining_, pages 1-10, 2015.
* [276] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [277] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [278] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [279] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [280] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [281] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [282] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [283] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [284] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [285] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [286] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [287] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [288] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [289] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [300] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [301] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [302] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [303] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [304] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [305] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [306] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [307] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [308] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [309] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [310] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [311] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [312] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [313] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [314] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [315] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [316] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [317] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [318] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [319] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [320] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [321] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [322] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [323] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [324] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.
* [325] W. Dong, J. Liu, and Q. Sun, editors: Learning to improve the quality of knowledge transfer. In _Proceedings of the 2015 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1-10, 2015.

 



\begin{table}
\begin{tabular}{l l|c c|l} \hline \multirow{2}{*}{Countries} & Non-English Language & Image Income Classes & World Bank & \multirow{2}{*}{Continent} \\  & Language & Classes & Country Economic Classes & \\ \hline South Korea & Korean & Rich, Up-mid, Low-mid & High & Asia \\ Spain & Spanish & Rich & High & Europe \\ Sri Lanka & Sinhala & Up-mid & Low-mid & Asia \\ Sweden & Swedish & Rich, Up-mid & High & Europe \\ Switzerland & German & Rich & High & Europe \\ Tanzania & Swahili & Up-mid, Low-mid, Poor & Low-mid & Africa \\ Thailand & Thai & Up-mid, Low-mid, Poor & Up-mid & Asia \\ Togo & French & Low-mid, poor & Poor/low & Africa \\ Tunisia & Arabic & Low-mid, poor & Low-mid & Africa \\ Turkey & Turkish & Rich & Up-mid & Europe \\ Ukraine & Ukrainian & Rich, Up-mid, Low-mid & Low-mid & Europe \\ United Kingdom & - & Rich, Up-mid & High & Europe \\ United States & Spanish & Rich, Up-mid, Low-mid & High & America \\ Vietnam & Vietnamese & Low-mid, Rich & Low-mid & Asia \\ Zimbabwe & Shona & Poor & Low-mid & Africa \\ \hline \end{tabular}
\end{table}
Table 9: Table displaying the 63 Dollar Street countries, their major non-English language, income levels of contributions for that country, World Bank income class, and their continent.

Figure 7: Average NLLB SigLIP Recall over poor and lower-middle income images for English and Income Suffix prompts



\begin{table}
\begin{tabular}{l|c c c c} \hline \multirow{2}{*}{**Languages**} & \multicolumn{4}{c}{**Income level of images**} \\ \cline{2-5}  & **Poor** & **Low-mid** & **Up-mid** & **Rich** \\ \hline Arabic & 21.0 (-2.4) & 32.0 (-3.9) & 38.5 (-7.1) & 37.3 (-6.3) \\ Bengali & 20.9 (-1.5) & 33.0 (-2.9) & 40.7 (-4.9) & 38.9 (-4.7) \\ Burmese & 21.5 (-0.9) & 30.4 (-5.5) & 36.0 (-9.6) & 34.2 (-9.4) \\ Chinese & 21.5 (-0.9) & 32.5 (-3.4) & 39.1 (-6.5) & 37.3 (-6.3) \\ Creole & 21.0 (-1.4) & 32.6 (-3.3) & 40.1 (-5.5) & 38.0 (-5.6) \\ Czech & 19.9 (-2.5) & 32.3 (-3.6) & 40.3 (-5.3) & 38.8 (-4.8) \\ Danish & 20.8 (-1.6) & 33.7 (-2.2) & 41.7 (-3.9) & 40.0 (-3.6) \\ Dutch & 21.1 (-1.3) & 33.6 (-2.3) & 42.5 (-3.1) & 40.8 (-2.8) \\ Ewe & 14.7 (-7.7) & 19.3 (-16.6) & 22.1 (-23.5) & 20.6 (-23.0) \\ Farsi-Persian & 21.9 (-0.5) & 31.8 (-4.1) & 39.1 (-6.5) & 38.1 (-5.5) \\ French & 21.7 (-0.7) & 33.7 (-2.2) & 42.6 (-3.0) & 41.4 (-2.2) \\ German & 21.5 (-0.9) & 33.1 (-2.8) & 41 (-4.6) & 39 (-4.6) \\ Hausa & 20.6 (-1.8) & 31.6 (-4.3) & 38.4 (-2.7) & 36.4 (-7.2) \\ Hindi & 22.2 (-0.2) & 34.5 (-1.4) & 41.8 (-3.8) & 40 (-3.6) \\ Indonesian & 22.1 (-0.3) & 34.8 (-1.1) & 42.4 (-3.2) & 40.5 (-3.1) \\ Italian & 21.1 (-1.3) & 34.3 (-1.6) & 42.7 (-2.9) & 41.4 (-2.2) \\ Khmer & 16.4 (-6.0) & 22.0 (-13.9) & 24.8 (-20.8) & 23.2 (-20.4) \\ Kinyarwanda & 16.7 (-5.7) & 23.7 (-12.2) & 28.8 (-16.8) & 27.2 (-16.4) \\ Korean & 20.4 (-2.0) & 32.2 (-3.7) & 40.2 (-5.4) & 37.9 (-5.7) \\ Kyrgyz & 21.6 (-0.8) & 30.9 (-5.0) & 36.7 (-8.9) & 35.7 (-7.9) \\ Mongolian & 13.7 (-8.7) & 20.9 (-1.5) & 25 (-20.6) & 23.4 (20.2) \\ Nepali & 20.7 (-1.7) & 32.5 (-3.4) & 40.9 (-4.7) & 39.7 (-3.9) \\ Oromo & 15.8 (-6.6) & 20.9 (-1.5) & 24.7 (-20.9) & 23.4 (20.2) \\ Portuguese & 21.3 (-1.1) & 34.4 (-1.9) & 42.6 (-3.0) & 41.2 (-2.4) \\ Romanian & 20.3 (-2.1) & 32.9 (-3.0) & 41.0 (-4.6) & 38.9 (-4.7) \\ Russian & 21.1 (-1.3) & 33.4 (-2.5) & 41.5 (-4.1) & 39.9 (-3.7) \\ Serbian & 19.2 (-3.2) & 30.8 (-5.1) & 37.2 (-8.4) & 35.6 (-8.0) \\ Shona & 19.1 (-3.3) & 27.2 (-8.7) & 32.2 (-13.4) & 30.5 (-13.1) \\ Sinhala & 20.4 (-2.0) & 32.0 (-3.9) & 37.9 (-7.7) & 35.7 (-7.9) \\ Somali & 19.0 (-3.4) & 28.5 (-7.4) & 33.8 (-11.8) & 31.4 (-12.2) \\ Spanish & 20.7 (-1.7) & 33.8 (-2.1) & 42.5 (-3.1) & 40.9 (-2.7) \\ Swahili & 22.1 (-0.3) & 33.6 (-2.3) & 41.3 (-4.3) & 38.9 (-4.7) \\ Swedish & 20.5 (-1.9) & 33.0 (-2.9) & 40.5 (-5.1) & 38.6 (-5.0) \\ Tagalog & 21.4 (-1.0) & 33.2 (-2.7) & 39.4 (-6.2) & 37.5 (-6.1) \\ Thai & 19.7 (-2.7) & 29.7 (-6.2) & 34.9 (-10.7) & 33.6 (-10.0) \\ Turkish & 20.5 (-1.9) & 31.6 (-4.3) & 39.5 (-6.1) & 38.5 (-5.1) \\ Ukrainian & 20.7 (-1.7) & 33.0 (-2.9) & 40.7 (-4.9) & 38.7 (-4.9) \\ Urdu & 21.5 (-0.9) & 33.0 (-2.9) & 40.6 (-5.0) & 39.1 (-4.5) \\ Vietnamese & 20.6 (-1.8) & 32.8 (-3.1) & 41.1 (-4.5) & 39.5 (-4.1) \\ Zulu & 19.9 (-2.5) & 29.9 (-6.0) & 35.2 (-10.4) & 33.4 (-10.2) \\ \hline \end{tabular}
\end{table}
Table 10: Non-English prompts lead to a decrease in Recall scores across all income levels. Table of the differences (rounded to 1 d.p.) between Recall scores for non-English language prompts and Recall scores for default English prompts for all data grouped into income levels.





\begin{table}
\begin{tabular}{|l c l l l l|} \hline
**Country Suffix** & \multicolumn{5}{c|}{**Income levels**} \\ \cline{2-6}  & **Poor** \(\Delta\) & **Low-mid** \(\Delta\) & **Up-mid** \(\Delta\) & **Rich** \(\Delta\) \\ \hline Burkina Faso & **0.327** (+0.103) & 0.303 (-0.056) & 0.227 (-0.229) & 0.185 (-0.251) \\ Burundi & **0.331** (+0.107) & 0.279 (-0.08) & 0.197 (-0.259) & 0.154 (-0.282) \\ Ethiopia & **0.334** (+0.11) & 0.313 (-0.046) & 0.269 (-0.187) & 0.227 (-0.209) \\ Liberia & **0.327** (+0.103) & 0.303 (-0.056) & 0.249 (-0.207) & 0.205 (-0.231) \\ Malawi & 0.301 (+0.077) & **0.32** (-0.39) & 0.286 (-0.17) & 0.254 (-0.182) \\ Rwanda & **0.334** (+0.11) & 0.322 (-0.037) & 0.249 (-0.207) & 0.204 (-0.232) \\ Somalia & **0.318** (+0.094) & 0.296 (-0.063) & 0.243 (-0.213) & 0.209 (-0.227) \\ Togo & 0.297 (+0.073) & **0.31** (-0.049) & 0.283 (-0.173) & 0.252 (-0.184) \\ Bangladesh & 0.271 (+0.047) & **0.319** (-0.04) & 0.266 (-0.19) & 0.221 (-0.215) \\ Bolivia & 0.3 (+0.076) & **0.318** (-0.041) & 0.262 (-0.194) & 0.223 (-0.213) \\ Cambodia & **0.289** (+0.065) & 0.288 (-0.071) & 0.213 (-0.243) & 0.172 (-0.264) \\ Cameroon & **0.313** (+0.089) & 0.289 (-0.07) & 0.233 (-0.223) & 0.192 (-0.244) \\ Cote d’Ivoire & 0.23 (+0.006) & 0.296 (-0.063) & **0.325** (-0.131) & 0.302 (-0.134) \\ Egypt & 0.257 (+0.033) & 0.334 (-0.025) & **0.357** (-0.099) & 0.316 (-0.12) \\ Ghana & **0.314** (+0.09) & 0.294 (-0.065) & 0.267 (-0.189) & 0.233 (-0.203) \\ Haiti & 0.296 (+0.072) & **0.331** (-0.028) & 0.307 (-0.149) & 0.269 (-0.167) \\ India & 0.239 (+0.015) & **0.31** (-0.049) & 0.306 (-0.15) & 0.278 (-0.158) \\ Iran & 0.221 (-0.003) & 0.343 (-0.016) & **0.375** (-0.081) & 0.337 (-0.099) \\ Jordan & 0.222 (-0.002) & 0.308 (-0.051) & **0.376** (-0.08) & 0.371 (-0.065) \\ Kenya & 0.296 (+0.072) & **0.318** (-0.041) & 0.283 (-0.173) & 0.236 (-0.2) \\ Kyrgyzstan & 0.229 (+0.005) & 0.338 (-0.021) & **0.365** (-0.091) & 0.318 (-0.118) \\ Lebanon & 0.249 (+0.025) & 0.309 (-0.05) & **0.348** (-0.108) & 0.33 (-0.106) \\ Mongolia & 0.259 (+0.035) & **0.326** (-0.033) & 0.308 (-0.148) & 0.256 (-0.18) \\ Myanmar & 0.263 (+0.039) & **0.304** (-0.055) & 0.241 (-0.215) & 0.195 (-0.241) \\ Nepal & 0.274 (+0.05) & **0.307** (-0.052) & 0.253 (-0.203) & 0.213 (-0.223) \\ Nigeria & **0.294** (+0.07) & 0.286 (-0.073) & 0.256 (-0.2) & 0.223 (-0.213) \\ Pakistan & 0.197 (-0.027) & 0.303 (-0.056) & **0.321** (-0.135) & 0.289 (-0.147) \\ Palestine & 0.258 (+0.034) & 0.349 (-0.01) & **0.361** (-0.095) & 0.317 (-0.119) \\ Papua New Guinea & 0.274 (+0.05) & **0.302** (-0.057) & 0.266 (-0.19) & 0.235 (-0.201) \\ Philippines & 0.271 (+0.047) & **0.346** (-0.013) & 0.337 (-0.119) & 0.295 (-0.141) \\ Sri Lanka & 0.275 (+0.051) & **0.322** (-0.037) & 0.303 (-0.153) & 0.277 (-0.159) \\ Tanzania & 0.287 (+0.063) & **0.292** (-0.067) & 0.257 (-0.199) & 0.228 (-0.208) \\ Tunisia & 0.276 (+0.052) & **0.321** (-0.038) & 0.314 (-0.142) & 0.284 (-0.152) \\ Ukraine & 0.245 (+0.021) & 0.355 (-0.004) & **0.372** (-0.084) & 0.323 (-0.113) \\ Vietnam & 0.229 (+0.005) & 0.321 (-0.038) & **0.33** (-0.126) & 0.294 (-0.142) \\ Zimbabwewe & **0.312** (+0.088) & 0.311 (-0.048) & 0.285 (-0.171) & 0.242 (-0.194) \\ \hline \end{tabular}
\end{table}
Table 11: Table of low-income/poor (in lilac) and lower-middle income (in purple) country suffixes and their effect on Recall for different income groups. For each country suffix, the highest Recall among income groups is highlighted in bold. The green and red values show how much increase or reduction that country suffix has on the Recall of data from an income group compared to default English prompts.





\begin{table}
\begin{tabular}{|l c r l r l r l|} \hline
**Country Suffix** & \multicolumn{6}{c|}{**Income levels**} \\ \cline{2-7}  & **Poor** & \(\Delta\) & **Low-mid** & \(\Delta\) & **Up-mid** & \(\Delta\) & **Rich** & \(\Delta\) \\ \hline Brazil & 0.254 (+0.03) & 0.303 (-0.056) & **0.323** (-0.133) & 0.303 (-0.133) \\ China & 0.213 (-0.011) & 0.34 (-0.019) & **0.369** (-0.087) & 0.319 (-0.117) \\ Colombia & 0.3 (+0.076) & **0.324** (-0.035) & 0.275 (-0.181) & 0.232 (-0.204) \\ Guatema & 0.269 (+0.045) & **0.314** (-0.045) & 0.277 (-0.179) & 0.233 (-0.203) \\ Indonesia & 0.266 (+0.042) & **0.328** (-0.031) & 0.303 (-0.153) & 0.266 (-0.17) \\ Kazakhstanan & 0.254 (+0.03) & **0.337** (**-0.022) & **0.337** (-0.119) & 0.292 (-0.144) \\ Mexico & 0.251 (+0.027) & 0.335 (-0.024) & **0.357** (-0.099) & 0.312 (-0.124) \\ Peru & 0.261 (+0.037) & **0.319** (-0.04) & 0.317 (-0.139) & 0.287 (-0.149) \\ Russia & 0.212 (-0.012) & 0.344 (-0.015) & **0.382** (-0.074) & 0.335 (-0.101) \\ Serbia & 0.197 (-0.027) & 0.313 (-0.046) & **0.378** (-0.078) & 0.354 (-0.082) \\ South Africa & 0.291 (+0.067) & **0.302** (-0.057) & **0.302** (-0.154) & 0.269 (-0.167) \\ Thailand & 0.234 (+0.01) & **0.312** (-0.047) & 0.293 (-0.163) & 0.256 (-0.18) \\ Turkey & 0.228 (+0.004) & 0.321 (-0.038) & **0.333** (-0.123) & 0.302 (-0.134) \\ Austria & 0.166 (-0.058) & 0.296 (-0.063) & 0.407 (-0.049) & **0.408** (-0.028) \\ Canada & 0.266 (+0.042) & 0.355 (-0.004) & **0.391** (-0.065) & 0.359 (-0.077) \\ Czech Republic & 0.195 (-0.029) & 0.33 (-0.029) & **0.395** (-0.061) & 0.379 (-0.057) \\ Denmark & 0.184 (-0.04) & 0.293 (-0.066) & 0.386 (-0.07) & **0.394** (-0.042) \\ France & 0.199 (-0.025) & 0.317 (-0.042) & 0.415 (-0.041) & **0.417** (-0.019) \\ Italy & 0.192 (-0.032) & 0.318 (-0.041) & **0.379** (-0.077) & 0.367 (-0.069) \\ Netherlands & 0.219 (-0.005) & 0.31 (-0.049) & **0.374** (-0.082) & 0.359 (-0.077) \\ Romania & 0.255 (+0.031) & 0.337 (-0.022) & **0.342** (-0.114) & 0.312 (-0.124) \\ South Korea & 0.225 (+0.001) & 0.313 (-0.046) & **0.345** (-0.111) & 0.314 (-0.122) \\ Spain & 0.183 (-0.041) & 0.308 (-0.051) & **0.413** (-0.043) & 0.404 (-0.032) \\ Sweden & 0.167 (-0.057) & 0.294 (-0.065) & 0.405 (-0.051) & **0.412** (-0.024) \\ Switzerland & 0.135 (-0.089) & 0.257 (-0.102) & 0.363 (-0.093) & **0.389** (-0.047) \\ United Kingdom & 0.205 (-0.019) & 0.326 (-0.033) & **0.418** (-0.038) & 0.409 (-0.027) \\ United States & 0.25 (+0.026) & 0.362 (-0.003) & **0.421** (-0.035) & 0.391 (0.045) \\ \hline \end{tabular}
\end{table}
Table 12: Table of high-income/rich (in blue) and upper-middle-income (in sky blue) country suffixes and their effect on Recall for different income groups. For each country suffix, the highest Recall among income groups is highlighted in bold. The green and red values show how much increase or reduction that country suffix has on the Recall of data from an income group compared to default English prompts.





\begin{table}
\begin{tabular}{|l|c|r|} \hline
**Language** & **ISO** & **chrf++** \\ \hline Arabic & arb\_Arab & 51.4 \\ \hline Bengali & ben\_Beng & 46.2 \\ \hline Burmese & mya\_Mymr & 29.3 \\ \hline Chinese & zho\_Hans & 19.6 \\ \hline Creole & hat\_Latin & 50.2 \\ \hline Czech & ces\_Latin & 52.7 \\ \hline Danish & dan\_Latin & 63.2 \\ \hline Dutch & nld\_Latin & 53.1 \\ \hline Ewe & ewe\_Latin & 35.6 \\ \hline Farsi\_Persian & pes\_Arab & 47.4 \\ \hline French & fra\_Latin & 67.0 \\ \hline German & deu\_Latin & 59.4 \\ \hline Hausa & hau\_Latin & 49.0 \\ \hline Hindi & bin\_Latin & 54.2 \\ \hline Indonesian & ind\_Latin & 66.6 \\ \hline Italian & ita\_Latin & 54.6 \\ \hline Khmer & khm\_Khmr & 31.2 \\ \hline Kinyarwanda & kin\_Latin & 44.0 \\ \hline Korean & for\_Hang & 32.1 \\ \hline Kyrgyz & stir\_Cyrl & 42.6 \\ \hline Mongolian & khk\_Cyrl & 37.3 \\ \hline Nepali & npi\_Deva & 49.0 \\ \hline Oromo & gaz\_Latin & 31.6 \\ \hline Portuguese & por Latin & 67.4 \\ \hline Romanian & ron\_Latin & 58.2 \\ \hline Russian & rust\_Cyrl & 52.5 \\ \hline Serbian & spr\_Cyrl & 53.3 \\ \hline Shona & sna\_Latin & 42.9 \\ \hline Sinhala & sin\_Sinh & 42.4 \\ \hline Somali & som\_Latin & 41.5 \\ \hline Spanish & spa\_Latin & 52.6 \\ \hline Swahili & swh\_Latin & 58.0 \\ \hline Swedish & swe\_Latin & 62.7 \\ \hline Tagalog & tgl\_Latin & 56.4 \\ \hline Thai & tha\_Thai & 36.0 \\ \hline Turkish & tur\_Latin & 52.9 \\ \hline Ukrainian & ukr\_Cyrl & 50.5 \\ \hline Urdu & urd\_Arab & 46.6 \\ \hline Vietnamese & vie\_Latin & 56.4 \\ \hline Zulu & zul\_Latin & 51.0 \\ \hline \end{tabular}
\end{table}
Table 13: Languages used and translation metrics (chrf++ scores) for NLLB-200-distilled-600M from English to these languages.