Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in E nglish, S panish, and A rabic (2022)

Abstract: As natural language processing systems become more widespread, it is necessary to address fairness issues in their implementation and deployment to ensure that their negative impacts on society are understood and minimized. However, there is limited work that studies fairness using a multilingual and intersectional framework or on downstream tasks. In this paper, we introduce four multilingual Equity Evaluation Corpora, supplementary test sets designed to measure social biases, and a novel statistical framework for studying unisectional and intersectional social biases in natural language processing. We use these tools to measure gender, racial, ethnic, and intersectional social biases across five models trained on emotion regression tasks in English, Spanish, and Arabic. We find that many systems demonstrate statistically significant unisectional and intersectional social biases. We make our code and datasets available for download.

## introduction large-scale transformer-based language models, such as bert @xcite , are now the state-of-the-art for a myriad of tasks in natural language processing. however, these models are well-documented to perpetuate harmful social biases, specifically by regurgitating the social biases present in their training data which are scraped from the internet without careful consideration @xcite . while steps have been taken to "debias", or remove, gender and other social biases from word embeddings @xcite , these methods have been demonstrated to be cosmetic @xcite . furthermore, these studies neglect to recognize both the impact of social biases on downstream task results as well as the complex and interconnected nature of social biases. in this paper, we detect and discuss unisectional foot_1 and intersectional social biases in multilingual language models applied to downstream tasks using a novel statistical framework and novel multilingual datasets. intersectionality is a framework introduced by @xcite to study how the composite identity of an individual across different social cleavages (e.g., race and gender) informs that individual's social advantages and disadvantages. for example, individuals who identify with multiple disadvantaged social cleavages (e.g., black women) face a greater and altered risk for discrimination and oppression than individuals with a subset of those identities (e.g., white women). this framework for understanding overlapping systems of discrimination has been explored in some studies of fairness in machine learning, including by @xcite who show that face detection systems perform markedly worse for female users of color, compared to female users or users of color. although work has begun to study intersectional social biases in natural language processing, to the best of our knowledge no work has explored fairness in an intersectional framework on downstream tasks (e.g. sentiment analysis). social biases in downstream tasks expose users with multiple disadvantaged sensitive attributes to unknown but potentially harmful outcomes, especially when models trained on downstream tasks are used in real-world decision making, such as for screening résumes or predicting recidivism in criminal proceedings @xcite . in this work, we choose emotion regression as a downstream task because social biases are often realized through emotion recognition @xcite and machine learning models have been shown to reflect gender bias in emotion recognition tasks @xcite . for example, sentiment analysis and emotion regression may be used by companies to measure product engagement for different social groups. in addition, while some work has studied gender biases across different languages @xcite , no work to our knowledge has studied racial, ethnic, and intersectional social biases across different languages. this lack of a multilingual analysis neglects non-english speaking users and their complex social environments. in this paper, we demonstrate the presence of gender, racial, ethnic, and intersectional social biases on five language models trained on an emotion regression task in english, spanish, and arabic. we do so by introducing novel supplementary test sets designed to measure social biases and a novel statistical framework for detecting the presence of unisectional and intersectional social biases in models trained on sentiment analysis tasks. our contributions are summarized as: ## related works the presence and impact of harmful social biases in machine learning and natural language processing systems is pervasive and well-documented in popular word embedding methods @xcite @xcite due to large amounts of humanproduced training data that includes historical social biases. notably, @xcite demonstrate such biases by introducing the word embedding association test (weat) which measures how similar socially sensitive sets of words (e.g., racial or gendered names) are to attributive sets of words (e.g., pleasant or unpleasant words) in the semantic space encoded by word embeddings. while @xcite ; @xcite introduce methods for "debiasing" word embeddings in order to create more equitable semantic representations for usage in downstream tasks, @xcite argue that such methods are merely cosmetic since social biases are still evident in the semantic space after the application of such methods. moreover, these "debiasing" techniques focus on a particular social cleavage such as gender or race (i.e., unisectional cleavages). in contrast, our work considers both unisectional and intersectional social biases. recent studies have also begun to focus on social biases in transformer-based language models @xcite . in particular, @xcite discusses how increasingly large transformer-based language model in practice regurgitate their training data, resulting in such models perpetuating social biases and harming users. therefore, in this work we consider both static word embedding techniques and transformerbased language models. @xcite introduces intersectionality as an analytical framework to study the complex character of the privilege and marginalization faced by an individual with a variety of identities across a set of social cleavages such as race and gender. a canonical usage of intersectionality is in service of studying the simultaneous racial and gender discrimination faced by black women, which cannot be understood in its totality using racial or gendered frameworks independently; for one example, we point to the angry black woman stereotype @xcite . as such, we argue that existing studies in fairness are limited in their ability both to uncover bias in and to "debias" language models without engaging with the intersectionality framework. intersectional social biases have been documented in natural language processing models. @xcite first studied intersectional social bias by employing distributional semantics on a wikipedia dataset while tan and celis (2019) studied intersectional social bias in contextualized word embeddings by using the weat on language referring to white men and black women. @xcite introduce tests that detect both known and emerging intersectional social biases in static word embeddings and extend the weat to contextualized word embeddings. similarly, @xcite also extend the weat to a contextualized word embedding framework using sentence embeddings. however, these methods do not consider the effect of intersectional social biases on the results of downstream tasks, which is the focus of this work. studies on non-english social biases in natural language processing are limited, with @xcite extending the weat to study gender bias in spanish and french and @xcite examining gender bias in english, spanish, german, and french on fasttext embeddings @xcite . notably, to the best of our knowledge there has been no work on studying intersectional social biases in languages other than english in natural language processing. while @xcite and @xcite study the intersectional social biases faced by asian and mexican women respectively using natural language processing, both do so in english. in contrast, our work seeks to understand intersectional social biases in the languages that are used by the individuals and the communities that they help constitute. most closely related to our work, @xcite evaluate racial and gender bias in 219 sentiment analysis systems trained on datasets from and submitted to semeval-2018 task 1: affect in tweets @xcite . their work introduces the equity evaluation corpus (eec), a supplementary test set of 8,640 english sentences designed to extract gender and racial biases in sentiment analysis systems. despite spanish and arabic data and submissions for the task, @xcite did not explore biases in either language. moreover, this study focused on submissions to the competition. in contrast, our work focuses on large-scale transformer-based language models and explores both unisectional and intersectional social biases in multiple languages. ## methods: framework for evaluating intersectionality in this section, we introduce our framework for detecting unisectional and intersectional social bias on results from downstream tasks. given a model trained on emotion regression, we evaluate the model on a supplementary test set using our framework to measure social biases. first, we discuss our supplementary test sets composed of sentences corresponding to social cleavages (e.g., black women, black men, white women, and white men) ( §3.1). we then use the results from each test set to run a beta regression model @xcite where we fit coefficients for gender, racial, and intersectional social biases ( §3.2). finally, we test the coefficients for statistical significance to determine if a model, trained on a given emotion regression task in a given language, demonstrates gender, racial, or intersectional social bias ( §3.3). ## ethical considerations and limitations our work is limited in scope to only social biases in english, spanish, and arabic due to the training data available and thus is limited to studying social biases in societies where those languages are dominant. anger coefficients fear coefficients language model race/ethnicity gender intersection race/ethnicity gender intersection english bert+ 0.061 * * * -0.004 -0.026 * * * 0.037 * * * 0.004 -0.006 (anglo-arab) mbert -0.001 -0.012 * * * 0.022 * * * 0.028 * * * 0.029 * * * -0.041 * * * xlm-roberta -0.002 * * -0.003 * * * 0.003 * * * -0.0 -0.0 0.001 svm-tfidf 0.001 0.001 -0.001 0.002 0.0 -0.0 fasttext -0.0 -0.003 -0.0 -0.0 0.0 -0.0 arabic bert+ -0.026 * * * -0.01 * * 0.007 -0.016 * * * -0.004 0.018 * * * mbert 0.004 -0.008 * * * 0.012 * * * 0.002 0.009 * * * -0.006 * xlm-roberta -0.001 * -0.004 * * * 0.001 * -0.002 * * 0.001 0.0 svm-tfidf 0.003 -0.029 * * * 0.01 0.002 -0.021 * * * 0.008 fasttext -0.03 * * * -0.012 * * 0.019 * * -0.018 * -0.031 * * * 0.013 joy coefficients sadness coefficients language model race/ethnicity gender intersection race/ethnicity gender intersection english bert+ 0.047 * * * -0.004 -0.019 * * * 0.064 * * * -0.005 -0.007 (anglo-arab) mbert -0.029 * * * 0.023 * * * 0.016 * * 0.0 0.033 * * * -0.024 * * xlm-roberta -0.001 0.001 -0.0 -0.001 -0.002 * * 0.003 * * * svm-tfidf 0.0 -0.002 0.002 0.004 0.004 -0.004 fasttext -0.0 0.001 -0.0 -0.0 0.0 -0.0 arabic bert+ -0.006 0.016 * * 0.003 0.034 * * * 0.001 -0.007 mbert -0.001 0.015 * * * 0.002 0.027 * * * 0.007 * -0.016 * * * xlm-roberta -0.0 -0.005 * * 0.005 -0.0 0.003 * -0.003 svm-tfidf 0.006 -0.052 * * * 0.023 * * -0.002 -0.031 * * * 0.001 fasttext 0.018 * * -0.028 * * * 0.018 -0.005 -0.036 * * * 0.031 * * * finally, we emphasize that there exists no set of carefully curated sentences that can detect the extent nor the intricacies of social biases. we therefore caution that no work, especially automated work, is sufficient in understanding or mitigating the full scope of social biases in machine learning and natural language processing models. this is especially true for intersectional social biases, where marginalization and discrimination takes places within and across gender, sexual, racial, ethnic, religious, and other cleavages in concert. ## conclusion in this paper, we introduce four equity evaluation corpora to measure racial, ethnic, and gender biases in english, spanish, and arabic. we also contribute a novel statistical framework for studying unisectional and intersectional social biases in sentiment analysis systems. we apply our method to five models trained on emotion regression tasks in english, spanish, and arabic, uncovering statistically significant unisectional and intersectional social biases. despite our findings, we are constrained in our ability to analyze our results with the sociopolitical and historical context necessary to understand their true causes and implications. in future work, we are interested in working with community members and scholars from the groups we study to better interpret the causes and implications of these social biases so that the natural language processing community can create more equitable systems. diloné, peter gado, astrid liden, bettina oberto, hasanian rahi, russel rahi, raya tarawneh, and two anonymous volunteers provided outstanding translation work. this work is supported in part by the national science foundation graduate research fellowship under grant no. dge-1644869. any opinion, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the national science foundation. anger joy fear sadness enojado/a eufórico/a ansioso/a deprimido/a molesto/a emocionado/a desalentado/a devastado/a enfurecido/a contento/a temeroso/a desilusionado/a furioso/a alegre asustada miserable irritado/a aliviado/a aterrorizado/a triste fastidioso/a increíble deprimido/a deprimente desagradable divertido/a devastado/a sombrío/a irritante excelente desilusionado/a destrozante indignante chistoso/a miserable absurdo/a maravilloso/a triste serio/a the gendered noun phrases used in the english, spanish, and arabic-language eecs can be found in in the following document are the instructions for translations. first, answer the survey questions. for each sentence, translate the template or individual word. we provide space for the female singular, female plural, male singular and female plural. if your language does not have separate masculine and feminine forms for any of the sentences, please include the singular and plural version in the first two boxes and if your does not have separate singular and plural forms, please include the singular versions for each gendered form as appropriate. if your language has additional cases, such as neutral, please make another column and note it for us (e.g. neuter in german). for the last ten, only give translations for the sentences as they are written. for the sentences with templates, rearrange order of templates if necessary, but signify where [p] and [ea], [eb] tags belong in each template. for example, the [p] tag denotes person, e.g. she/her, this woman, my sister; the [ea] tag denotes emotional state words, e.g. angry, happy; and the [eb] tag denotes emotional event words, e.g. annoying, funny. for the emotion vocabulary, there are four categories: anger (red), fear (green), joy (yellow) and sadness (blue). if the english words do not correspond well, feel free to write the most approximate set of words for your language in any order. let us know if there are intricacies in spelling due to, for example, consonants and vowels (e.g. a/an in english or le l' in french). optional: we are also looking for popular names of large socially cleaved groups in countries where your language is spoken. for example, in english, this includes male, female, black and white names (5 for each combination of race and gender). if you are familiar with social cleavages or popular names in those cleavages in countries where your language is spoken, please note it. sentence templates: