Speaking Multiple Languages Affects the Moral Bias of Language Models (2023)

Abstract: Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. However, PMLMs are trained on varying amounts of data for each language. In practice this means their performance is often much better on English than many other languages. We explore to what extent this also applies to moral norms. Do the models capture moral norms from English and impose them on other languages? Do the models exhibit random and thus potentially harmful beliefs in certain languages? Both these issues could negatively impact cross-lingual transfer and potentially lead to harmful outcomes. In this paper, we (1) apply the MORALDIRECTION framework to multilingual models, comparing results in German, Czech, Arabic, Chinese, and English, (2) analyse model behaviour on filtered parallel subtitles corpora, and (3) apply the models to a Moral Foundations Questionnaire, comparing with human responses from different countries. Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions. We release our code and models.

## introduction recent work demonstrated large pre-trained language models capture some symbolic, relational @xcite . we show scores for each language both from the respective monolingual model (triangles, left) and a multilingual model @xcite . @xcite , but also commonsense @xcite knowledge. the undesirable side of this property is seen in models reproducing biases and stereotypes (e.g., @xcite . however, in neutral terms, language models trained on large data from particular contexts will reflect cultural "knowledge" from those contexts. we wonder whether multilingual models will also reflect cultural knowledge from multiple contexts, so we study moral intuitions and norms that the models might capture. recent studies investigated the extent to which language models reflect human values @xcite . these works addressed monolingual english mod-els. like them, we probe what the models encode, but we study multilingual models in comparison to monolingual models. given constantly evolving social norms and differences between cultures and languages, we ask: can a pmlm capture cultural differences, or does it impose a western-centric view regardless of context? this is a broad question which we cannot answer definitively. however, we propose to analyse different aspects of monoand multilingual model behaviour in order to come closer to an answer. in this paper, we pose three research questions, and present a series of experiments that address these questions qualitatively: the three experiments reinforce each other in finding that our models grasp the moral dimension to some extent in all tested languages. there are differences between the models in different languages, which sometimes line up between multiand mono-lingual models. this does not necessarily correspond with differences in human judgements. as an illustration, figure 1 shows examples of the moraldirection score for several verbs in our monolingual and multilingual models. we also find that the models are very reliant on lexical cues, leading to problems like misunderstanding negation, and disambiguation failures. this unfortunately makes it difficult to capture nuances. in this work we compare the behaviour of the pmlm both to human data and to the behaviour of monolingual models in our target languages arabic, czech, german, chinese, and english. ## inducing moral dimensions in pmlms we choose five languages to evaluate the behaviour of the multilingual models: modern standard arabic (ar), czech (cs), german (de), english (en), and mandarin chinese (zh). these are all relatively high-resource languages, so we hope the model will be able to reliably detect cultural knowledge in each language. since we rely on machine translation, using high-resource languages also ensures good translation quality. we note here that languages and cultures or countries are at best approximately equivalent (cf. @xcite . for instance, arabic, english, and chinese are standardised varieties that are written in a wide range of different contexts or cultures, which are likely merged together in the model to some degree. however, separation by language is the best way we have to distinguish cultural contexts within a multilingual model. as a point of comparison, we choose a monolingual language model for each of our target languages (see app. a for details). ## qualitative analysis on parallel data to better understand how these models generalise for various types of texts, we conduct a qualita-tive study using parallel data. for a parallel sentence pair, the moraldirection scores should be similar in most cases. sentence pairs where the scores differ considerably may indicate cultural differences, or issues in the models. in practice, very large score differences appear to be more related to the latter. this type of understanding is important for further experiments with these models. we conduct our analysis on opensubtitles parallel datasets @xcite , 6 which consist of relatively short sentences. given that the moraldirection is induced on short phrases, we believe that short sentences will be easier for the models. the subtitles often concern people's behaviour towards each other, and thus may carry some moral sentiment. we use english-german and english-czech data for our analysis. to obtain the moral scores, we encode each sentence with the respective s-bert model, apply the pca transformation, and divide the first principal component by the normalising parameter. our analysis focuses on sentence pairs with very different scores. we take steps to filter out mistranslated sentence pairs-see appendix h. below, we discuss examples of where scores differ noticeably even when the translations are adequate. using czech-english and german-english data, we compare the monolingual models with xlm-r, xlm-r with the monolingual models, and the monolingual models with each other. this analysis is based on manual inspection of 500 sentence pairs with the highest score differences for each combination. note that many of the sentence pairs were minor variations of each other, which significantly sped up the analysis. relevant examples are listed with their moraldirection scores in ## moral foundations questionnaire the mfq has been applied in many different studies on culture and politics, meaning there is human response data from several countries available. we pose the mfq questions from @xcite to our models, in order to compare the model scores with data from previous studies. we use the translations provided on the moral foundations website. 7 since the first part of the mfq consists of very complex questions, we rephrase these into simple statements (see appendix j). many of the statements in the first half of the questionnaire become reverse-coded by simplifying them, that is, someone who values the aspect in question would be expected to answer in the negative. for these statements, we multiply the model score by -1. further, we know that language models struggle with negation (kassner and schütze, 2020), so we remove "not" or "never" from two statements and flip the sign accordingly. in the same way, we remove "a lack of" from two statements. these adjustments already improved the coherence of the resulting aspect scores, but we found further questions being scored by the models as if reverse-coded, i.e., with a negative score when some degree of agreement was expected. these were not simply negated statements, but they did tend to contain lexical items that were strongly negatively associated, and in multiple cases contained a negative moral judgement of the action or circumstance in question. because the models appear to be so lexically focused (see § 4.1), this combination led to a strong negative score for some of these questions. we decided to rephrase such statements as well, usually flipping their sign while changing the wording as little as possible. still, we care fairness loyalty authority purity 0 1 2 3 4 5 czech (bene 2021) german (joeckel 2012) us (graham 2011) china (wang 2019) care fairness loyalty authority purity 0.0 0.2 0.4 0.6 0.8 1.0 arabert robeczech gbert en bert zh bert care fairness loyalty authority purity 0.0 0.2 0.4 0.6 0.8 1.0 ar cs de en zh figure 2: mfq aspect scores from humans and models. left: examples of human data from studies in different countries. middle: scores obtained from monolingual moraldirection models. right: scores from xlm-r moraldirection in five languages. care fairness loyalty authority purity 1.5 1.0 0.5 0.0 0.5 1.0 ar cs de en zh figure 3: sanity check-mfq aspect scores from the xlm-r moraldirection models without sentence-bert tuning. this model had not obtained good correlations with human scores in § 3. we provide the list of rephrased english statements and multipliers in appendix ## conclusions we investigated the moral dimension of pre-trained language models in a multilingual context. in this section, we discuss our research questions: in the process, we showed that sentence-level representations, rather than mean-pooled tokenlevel representations, are necessary in order to induce a reasonable moral dimension for most of these models. we trained monolingual s-bert models for our five target languages arabic, czech, german, english, and mandarin chinese. as well, we created a multilingual s-bert model from xlm-r which was trained with mnli data in all five target languages. we had initially wondered whether models would impose values from an english-speaking context on other languages. based on this evidence, it seems that the models do differentiate between cultures to some extent, but there are caveats: the differences are not necessarily consistent with human value differences, which means the models are not always adequate. the problem appears to be worse when models are trained on smaller data for a given language. meanwhile, german and chinese have noticeably high agreement with english in our multilingual model, and all languages are extremely highly correlated in the pre-existing parallel-data s-bert model ( future work. this leads to several future research questions: (i) can we reliably investigate encoded (moral) knowledge reflected by pmlms on latent representations or neuron activations? or do we need novel approaches? for instance, @xcite suggest evaluating the output of generative models and, subsequently, @xcite apply masked generation using pmlms to probe cultural differences in values. however, the generation process of lms is highly dependent, among other things, on the sampling process. therefore, it is questionable if such approaches provide the required transparency. nevertheless, @xcite come to a similar conclusion as indicated by our results: pmlms encode differences between cultures. however, these are weakly correlated with human surveys, which leads us to the second future research question: (ii) how can we reliably teach large-scale lms to reflect cultural differences but also commonalities? investigating pmlms' moral direction and probing the generation process leads to inconclusive results, i.e., these models encode differences, which, however, do not correlate with human opinions. but correlating with human opinions is a requirement for models to work faithfully in a cross-cultural context. therefore, we advocate for further research on teaching cultural characteristics to lms.