Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis (2023)

Abstract: Sentiment analysis (SA) systems are widely deployed in many of the world’s languages, and there is well-documented evidence of demographic bias in these systems. In languages beyond English, scarcer training data is often supplemented with transfer learning using pre-trained models, including multilingual models trained on other languages. In some cases, even supervision data comes from other languages. Does cross-lingual transfer also import new biases? To answer this question, we use counterfactual evaluation to test whether gender or racial biases are imported when using cross-lingual transfer, compared to a monolingual transfer setting. Across five languages, we find that systems using cross-lingual transfer usually become more biased than their monolingual counterparts. We also find racial biases to be much more prevalent than gender biases. To spur further research on this topic, we release the sentiment models we used for this study, and the intermediate checkpoints throughout training, yielding 1,525 distinct models; we also release our evaluation code.

## introduction sentiment analysis (sa) has many practical applications, leading to widespread interest in using it for many languages. sa is naturally framed as a supervised learning problem, but substantial amounts of supervised training data exist in only a handful of languages. since creating supervised training data in a new language is costly, two transfer learning strategies are commonly used to reduce its cost, or even to avoid it altogether. the first, which reduces cost, is monolingual transfer: we pre-train an unsupervised model on a large corpus in the target language, fine-tune on a small amount of supervision data in that language, and apply the model in that language @xcite . the second, which avoids annotation cost altogether, is zero-shot cross-lingual transfer: we pre-train an unsupervised model on a large corpus in many languages, fine-tune on already available supervision data in a high-resource language, and use the model directly in the target language @xcite . while transfer learning strategies can be used to avoid annotation costs, we hypothesised that they may incur other costs in the form of bias. it is well-known that high-resource sa models exhibit gender and racial biases @xcite @xcite . less is known about bias in other languages. a recent study found that sa models trained with monolingual transfer were less biased than those trained without any transfer learning @xcite . as far as we are aware, there is no work that studies the effect of cross-lingual transfer on bias. but there is good reason to hypothesise that cross-lingual transfer may introduce new biases. specific cultural meanings, multiple word senses, and dialect differences often contribute to errors in multilingual sa systems @xcite , and are also sources of bias @xcite . for example, the english word foreigner translates to the japanese word gaijin (外 人) which has approximately the same meaning, but more negative sentiment. bias may also arise from differences in what is explicitly expressed. for example, there is evidence that syntactic gender agreement increases gender information in representations @xcite , and there is also evidence that gender information in representations correlates with gender bias @xcite . from these facts, we hypothesise that multilingual pre-training on languages with gender agreement will produce more gender bias in target languages without gender agreement, while producing less bias in target languages with gender agreement. in this paper, we conduct the first investigation of biases imported by cross-lingual transfer, answering the following research questions: (rq1) what biases are imported via cross-lingual transfer, compared to those found in monolingual transfer? when using cross-lingual transfer, are observed biases explained by the pre-training data, or by the cross-lingual supervision data? since practical systems often use distilled models, we also ask: (rq2) do distilled transfer models show the same trends as standard ones? we investigate these questions via counterfactual evaluation, in which test examples are edited to change a single variable of interest-such as the race of the subject-so that any change in model behaviour can be attributed to that edit. we use the counterfactual evaluation benchmarks of @xcite and an extension of it @xcite to test for gender, racial, and immigrant bias in five languages: japanese (ja), simplified chinese (zh), spanish (es), german (de), and english (en). the first four lan-guages cover three different language families, that all have fewer sentiment analysis resources then english; including english in the study enables us to compare to previous work. we find that: we conclude with a set of recommendations to test for bias in zero-shot cross-lingual transfer learning, to create more resources to allow testing, and to expand bias research outside of english. we release all models and code used for our experiments, to facilitate further research. 1 2 background ## methodology we treat sentiment polarity detection as a five-way classification problem: very negative (1), negative (2), neutral (3), positive (4), or very positive (5). in figures, we refer to these classes by using symbols --, -, 0, + and ++. this ordinal labeling scheme is commonly used when systems are trained on user reviews with a star rating @xcite . we train monolingual and cross-lingual models, then evaluate them on counterfactual corpora and compare their differences in bias measures. we look at both average bias using aggregate metrics and granular bias using a contingency table of counterfactuals. this enables us to build an overall picture of model comparability and also to differentiate between models with identical aggregate bias but different behaviour -some models may make many small errors, and some may make few large errors, and this may matter for minimising real world harms. 2 there are tasks where invariance to demographics doesn't make sense, such as hate speech classification. our evaluation data are designed so that all examples should be invariant. ## experimental setup our goal is to simulate practical conditions as much as is possible with available resources and datasets, so we start with pre-trained models from huggingface @xcite which are commonly used in sentiment benchmarks and previous work on our data. 4 we then fine-tune these models on supervised training for the polarity detection task and apply to the counterfactual evaluation set in the target language. both monolingual and multilingual models have as similar numbers of parameters and fine-tuning procedures as is possible, to minimise confounds while being realistic (appendix a). models are fine-tuned until convergence using early stopping on the development set. all models (multilingual and monolingual) converge to equivalent performance as previous work @xcite , which is state of the art on this task. f1 scores and steps to convergence are included in appendix b. monolingual transfer (mono-t) models are based on pre-trained bert-base-uncased @xcite in the target language. we randomly initialise a linear classification layer, then simultaneously train it and fine-tune the language model on monolingual supervision data. our distilled monolingual model (distilmono-t) is identical, except that it is based on distilbert-base-uncased @xcite . multilingual models are based on pre-trained mbert-base-uncased then fine-tuned on a large volume of sentiment data in english only, the standard approach to zero-shot cross-lingual transfer (zs-xlt). we also fine-tune a distilled zs-xlt model (distil-zs-xlt), identical except that it is based on distilmbert-base-uncased. ## results we examine whether system bias is affected by a decision to use zero-shot cross-lingual transfer (zs-xlt) instead of monolingual transfer. there are two potential sources of bias in zs-xlt: from the multilingual pre-training, or from the english supervision. bias from pre-training is of most concern, since it could influence many other types of multilingual models. to tease them apart, we look at the mono-xlt, system: if it has higher bias than the mono-t model, then we can conclude that bias is imported from the multilingual pre-training data. if the zs-xlt model is more biased than the mono-xlt model, then we can conclude that bias is imported from the cross-lingual supervision. ## rq2: do distilled models show the same trends? figure 4 shows a comparison of standard and distilled models for mono-t and zs-xlt models. the patterns are still not consistent, but are striking. for cross-lingual transfer, distillation dampens racial biases. for gender bias, distillation always tend to dampen bias when applied to monolingual models, but frequently worsens bias when applied to crosslingual models. german, spanish, and chinese all have significantly more bias for gender with distil-zs-xlt than with zs-xlt models. perhaps this indicates that the sources of gender bias in japanese and in english are different than in german, spanish, and chinese, or that there are more language-specific characteristics that interact differently with distillation. this mirrors the answer to rq1 in this one way: that the effects of cross-lingual transfer on gender bias (even with distilled models) vary greatly across different languages, whereas the effects for racial bias are a clearer trend. we leave this investigation for future work, but consider these results to be at least promising, that model distillation may be an effective approach to mitigate or at least avoid exacerbating racial biases in cases where cross-lingual transfer must be used. ## recommendations and conclusions this broad set of experiments has shown that bias can change drastically as a result of any of the standard engineering choices for making an sa system in a lower resourced language. in light of these results, we make the following recommendations: do not assume that more data will improve biases assess bias of all new model and data choices. use granular bias by sentiment label, as well as aggregate bias, to make decisions that best suit the intended application. don't rely solely on aggregate measures. our results highlight how summary statistics can make different underlying distributions appear identical, a point made by @xcite in general, and by zhao and chang (2020) specifically for bias, but still frequently overlooked in most bias research. though both are problematic, a model that consistently associates slightly more negative sentiment to a minoritised group is qualitatively different from a model that sometimes flips very positive sentiment to very negative sentiment. beware of bias introduced cross-lingually. bias can transfer across languages from pre-training or from supervision data, which means that crosslingual transfer has the opportunity to introduce non-local biases. these can be unexpected and hard to detect, and represent machine learning cultural imperialism that is best avoided. be particularly aware of racial biases. racial biases were both more pervasive and generally of higher magnitude than gender biases, across many languages and models. racial biases are frequently overlooked in research @xcite , and our results show that this can be quite dangerous. consider compressing models. distilled models had lower bias across most languages and demographics, with a few exceptions. this came at a very low penalty for performance of one f1 point on average. previous work had contradictory conclusions regarding model compression, with some vision models showing worse bias in compressed models @xcite and some nlp generation models showing less bias under compression @xcite . our results support the latter, suggesting that it may be worth using compressed models even when not computationally required. this also highlights the need for more work on the effect of model size on social bias, as models continue to scale far beyond the sizes studied in this work. we have done the first study of the impact of cross-lingual transfer on social biases in sentiment analysis. we have also raised many open questions. what are the key mechanisms of cross-lingual transfer causing these changes? monolingual transfer was found to lessen biases due to increased stability and performance of the model @xcite , is the lack of this effect in crosslingual transfer due to the curse of multilinguality @xcite , or some other reason? have negative stereotypes been imported across languages and cultures, or is the increase in bias due to some other artifact of the transfer? why do gender biases behave so differently from racial biases? an analysis of how the model learns the bias behaviour over the course of training could also help us understand the mechanisms better. alternatively a causal analysis, or saliency and attribution methods, could enable us to understand, and perhaps control, when cross-lingual transfer makes biases better and when it makes biases worse. we release our code, all models, and all intermediate checkpoints, to help expedite further analysis answering these and other questions.