Bias Beyond E nglish: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages (2023)

Abstract: Sentiment analysis (SA) systems are used in many products and hundreds of languages. Gender and racial biases are well-studied in English SA systems, but understudied in other languages, with few resources for such studies. To remedy this, we build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. We demonstrate its usefulness by answering a simple but important question that an engineer might need to answer when deploying a system: What biases do systems import from pre-trained models when compared to a baseline with no pre-training? Our evaluation corpus, by virtue of being counterfactual, not only reveals which models have less bias, but also pinpoints changes in model bias behaviour, which enables more targeted mitigation strategies. We release our code and evaluation corpora to facilitate future research.

## introduction sentiment analysis (sa) systems are among the most widely deployed nlp systems, used in hundreds of languages @xcite . it is well-known that english sa models exhibit gender and racial biases @xcite @xcite , which are acquired from their training data, training objective, and other system choices @xcite . other languages are understudied; though many papers study sa bias in english, few study sa bias in other languages. this may be partly attributable to resource constraints: there are fewer corpora available to audit systems for bias in non-english languages. to remedy this, we create evaluation datasets to evaluate gender and racial bias in four languages: japanese (ja), simplified chinese (zh), spanish (es), german (de). each of these four languages has publicly available data for training sa systems @xcite , and together they represent three distinct language families. to complement their existing resources with a new resource that measures bias, we use counterfactual evaluation (figure 1 ), in which test examples are edited to change a single variable of interestsuch as the race of the subject-extending previous work done in english @xcite . we release the evaluation dataset to facilitate further research. 1 we demonstrate the value of these evaluation resources by answering the following research questions: (rq1) what biases do we find in other languages, compared to in english? (rq2) how does the use of pre-trained models affect bias in sa systems? while pre-trained models are common in nlp, they may import biases not present in task supervision data, since a large pre-training corpus may embody biases not present in the supervision corpus. on the other hand, pre-training might diminish biases that arise from the small sample sizes typical of sa training corpora. our experiments show that both gender and racial bias are present in sa systems for all four languages: when model architecture, data quantity, and domain are held constant, sa systems in other languages display quantitatively more bias than sa systems in english. for rq2, we find that pre-training also makes sa systems less biased for all languages, in aggregate, though in surprising ways: our non-pre-trained models exhibit extreme changes in behaviour on counterfactual examples, whereas pre-trained models exhibit many small nuanced changes. ## new counterfactual evaluation corpus counterfactual (or contrastive) evaluation establishes causal attribution by modifying a single input variable, so that any changes in output can be attributed to that intervention @xcite . for example, if our variable of interest is gender, and our original sentence is the conversation with that boy was irritating, then our intervention creates the counterfactual sentence the conversation with that girl was irritating. importantly, we change no other variables, such as age (@xmath0, register (@xmath1, or relationship (@xmath2. we then evaluate the behavior of our model on many such pairs of original and counterfactual sentences. in a model with no gender bias, sentiment should not change under this intervention. if it does, and does so systematically over many counterfactuals, we conclude that our model is biased. to create counterfactual examples for non-english languages we use template sentences, illustrated in the templates of @xcite only needed to handle the weak agreement and inflectional morphology of english, so we extend their methodology to handle a variety of grammatical phenomena in other languages. for example, in german we add gender agreement (masculine, feminine, neuter) and noun declension; in spanish we add gender agreement (masculine, fem-inine, plural of both) and idiomatic verb usage; foot_1 in japanese we add a distinction between active and passive forms. chinese requires no special handling since it lacks gender agreement or inflectional morphology. in all languages, we create a gender bias test set by providing contrasting pairs of male/female terms that can fill the placeholder for demographic variable. in german and japanese we also provide pairs of terms for racial and anti-immigrant bias, which we derive from ngos, sociology and anthropology resources, and government census data @xcite @xcite . we usually leave the privileged group unmarked to avoid the unnaturalness of markedness @xcite . 3 for spanish anti-immigrant bias, we create pairs of names by using name lists that are strongly associated with migrants or with non-migrants, sourced from @xcite , which are based on social science research @xcite . we lacked equivalent resources for chinese, so we test only gender bias. the resulting corpora ( to produce the templates, we worked alongside native speakers in japanese, german, spanish, and chinese to translate the english templates of @xcite , often modifying them to prefer naturalness in the target language while preserving sentiment. our japanese translator had professional translation experience, while our german, spanish, and chinese translators had training in linguistics. while collaborative development and refinement of the translation process required about a week, actual translation took about four hours for each dataset. further details in a. ## methodology for our sa task, we focus on sentiment polarity detection @xcite , where the output label represents the sentiment of a text as an ordinal score (shown in parentheses): very negative template counterfactual sentences en the conversation @xmath3. the conversation with [him\her] was irritating. @xmath4 la conversación con [él\ella] fue irritante. gender race (1), negative (2), neutral (3), positive (4), or very positive (5). foot_3 ## experiments we want to answer the questions: what biases arise in sa systems in each of these languages (rq1)? does pre-training improve or worsen biases (rq2)? to answer these questions, we measure the bias of a baseline svm classification model to a model based on a pre-trained transformer model. we compare standard and distilled transformer models; distilled models are often used in practice since they are better suited to the computational constraints of real-world systems. our baseline (no pre-training) models are bagof-words linear kernel support vector machines (svms) trained on the supervision data in each language. our pre-trained (mono-t) models are pre-trained bert-base @xcite for each language. we randomly initialise a linear classification layer and simultaneously train the classifier and fine-tune the language model on the same supervision data. our distilled (distil-mono-t) models are identical, but based on distilbert-base @xcite . we train each model five times with different random seeds (or five separate runs for the baseline) and then ensemble by taking their majority vote, a standard procedure to reduce variance. all models converge to performance on par with sota on this task and data. training details and f1 scores on the sa task are reported in appendix b and c. training data for each model, we use the language appropriate subset of the multilingual amazon reviews corpus (marc; @xcite , which contains 200 word reviews in english, ---0 + ++ baseline japanese (ja) ---0 + ++ ---0 + ++ mono-t ---0 + ++ figure 3: confusion matrices for racial counterfactual pairs for japanese and german, comparing baseline and pretrained models. higher colour saturation in the lower triangle is bias against the minoritised group, against the privileged group in the upper triangle. japanese, german, french, chinese and spanish, with discrete sentiment labels ranging from 1-5, balanced across labels. ## results the baseline models are most biased for both gender and race in all languages (figure 2 ), though not always against minoritised groups: systems are often biased against the male demographic, consistent with previous work on sa (thelwall, 2018). 5 figure 2 also shows that english models tend to be less biased than the other languages. analyzing the granular differences (figure 3 ) reveals interesting behaviour not captured by aggregate metrics: much of the bias exhibited by the baselines arises from consistently flipping specific labels in the counterfactual, while bias exhibited by pre-trained models is more varied. 6 for example, the japanese baseline exhibits racial bias by frequently changing neutral labels to very negative labels, whereas in the mono-t model the change under the counterfactual is expressed as many less extreme changes. the model is still biased overall: though the changes are more varied, in aggregate they associate racial minorities with more negative sentiment. the german baseline model is more extreme: when the demographic variable changes from privileged to minoritised, the model changes its prediction from very positive to very negative. the german mono-t model also makes biased choices, though more moderately (neutral to negative) and there is more 'counter-bias' in the upper triangle, which lessens overall bias. ## related work and conclusion counterfactual evaluation is frequently used in bias research on classification tasks @xcite , and sometimes even on generation tasks @xcite . there have also been works exposing common pitfalls in the design of counterfactuals @xcite @xcite . anyone expanding or replicating our counterfactual evaluation work should consult these as prerequisites. the contemporary work of @xcite find many ways that other templates for bias evaluation can be brittle, so future work should take this into account and take measures to ensure robustness, such as testing with multiple paraphrases of the templates. we have laid the groundwork for investigating bias in sentiment analysis beyond english. we created resources, presented an evaluation procedure, and used it to do the first analysis of bias in sa in a simulated low-resource setting across multiple languages. we showed that using pre-trained models produces much less biased models than using baseline svms. we also showed that pre-trained models have very different patterns of bias; a type of analysis that is enabled by the counterfactual design of our corpus. we invite the nlp community to use the data and methods from this work to continue analysis of languages beyond english.