Knowledge of cultural moral norms in large language models (2023)

Abstract: Moral norms vary across cultures. A recent line of work suggests that English large language models contain human-like moral biases, but these studies typically do not examine moral variation in a diverse cultural setting. We investigate the extent to which monolingual English language models contain knowledge about moral norms in different countries. We consider two levels of analysis: 1) whether language models capture fine-grained moral variation across countries over a variety of topics such as “homosexuality” and “divorce”; 2) whether language models capture cultural diversity and shared tendencies in which topics people around the globe tend to diverge or agree on in their moral judgment. We perform our analyses with two public datasets from the World Values Survey (across 55 countries) and PEW global surveys (across 40 countries) on morality. We find that pre-trained English language models predict empirical moral norms across countries worse than the English moral norms reported previously. However, fine-tuning language models on the survey data improves inference across countries at the expense of a less accurate estimate of the English moral norms. We discuss the relevance and challenges of incorporating cultural knowledge into the automated inference of moral norms.

## introduction moral norms vary from culture to culture @xcite @xcite . understanding the cultural variation in moral norms has become critically relevant to the development of machine intelligence. for instance, recent work has shown that cultures vary substantially in their judgment toward moral dilemmas regarding autonomous driving @xcite . work in natural language processing (nlp) also shows that language models capture some knowledge of social or moral norms and values. for example, with no supervision, english pre-trained language models (eplms) have been shown to capture people's moral biases and distinguish between morally right and wrong actions @xcite . here we investigate whether eplms encode knowledge about moral norms across cultures, an open issue that has not been examined comprehensively. multilingual pre-trained language models (mplms) have been probed for their ability to identify cultural norms and biases in a restricted setting @xcite @xcite . for instance, @xcite show that mplms capture moral norms in a handful of cultures that speak different languages. however, it remains unclear whether monolingual eplms encode cultural knowledge about moral norms. prior studies have only used eplms to assess how they encode undesirable biases toward different communities @xcite @xcite @xcite . for instance, @xcite show that gpt3 can generate toxic comments against muslims, and @xcite explore harmful text generation toward lgbtqia+ groups in bert models @xcite . extending these lines of work, we assess whether monolingual eplms can accurately infer moral norms across many cultures. our focus on eplms is due partly to the fact that english as a lingua franca has widespread uses for communication in-person and through online media. given that eplms may be applied to multicultural settings, it is important to understand whether these models encode basic knowledge about cultural diversity. such knowledge has both relevance and applications for nlp such as automated toxicity reduction and content moderation @xcite . another motivation for our focus is that while it is expected that eplms should encode western and @xcite . each dot represents the empirical average of participants' ratings for a morally relevant topic (e.g., "abortion") within a country. right: corresponding moral scores estimated by a language model (sentence-bert) @xcite . each dot represents the moral score obtained by probing the language model in a given country. english-based moral knowledge, such knowledge might entail potential (implicit) biases toward non-english speaking cultures. for example, an eplm might infer a situation to be morally justifiable (e.g., "political violence") in a non-english speaking culture (because these events tend to associate with non-english speaking cultures in corpora) and thus generate misleading representations of that community. here we probe state-of-the-art eplms trained on large english-based datasets. using eplms also supports a scalable analysis of 55 countries, which goes beyond existing work focusing on a small set of high-resource languages from mplms and monolingual plms. we take the moral norms reported in different countries to be a proxy of cultural moral norms and consider two main levels of analysis to address the following questions: we probe eplms using two publicly available global surveys of morality, world values survey wave 7 @xcite foot_0 (wvs) and pew global attitudes survey (pew) (research center, 2014) foot_1 . for example, according to wvs survey (illustrated in figure 1 ), people in different cultures hold disparate views on whether "having casual sex" is morally acceptable. in contrast, they tend to agree more about the immorality of "violence against other people". our level 1 analysis allows us to probe the fine-grained cultural moral knowledge in eplms, and our level 2 analysis investigates the eplms' knowledge about shared "universals" and variability across cultures in moral judgment. following previous work @xcite and considering the current scale of global moral surveys, we use country as a proxy to culture, although this approach is not fully representative of all the different cultures within a country. we also explore the utility-bias trade-off in encoding the knowledge of cultural moral norms in eplms through a fine-tuning approach. with this approach it may be possible to enhance the moral knowledge of eplms in a multicultural setting. we examine how this approach might reduce the ability of eplms to infer english-based moral norms and discuss how it might induce cultural biases. ## methodology for inferring cultural moral norms we develop a method for fine-grained moral norm inference across cultures. this method allows us to probe eplms with topic-country pairs, such as "getting a divorce in [country]". 3 we build this method from the baseline method proposed by @xcite for homogeneous moral inference, where we probe eplm's moral knowledge about a topic without incorporating the cultural factor (i.e., the country names). similar to that work, we use sbert through bert-large-nli-mean-tokens sentence transformer model and use topic and topic-country pairs as our prompts. 4 this model is built on top of the bert model, which is pre-trained on bookscorpus @xcite and wikipedia. ## datasets we describe two open survey data that record moral norms across cultures over a variety of topics. ## evaluation and results we evaluate eplms' moral knowledge with respect to 1) homogeneous moral norms, 2) fine-grained moral norms across cultures, and 3) cultural diversities and shared tendencies on moral judgment of different topics. ## fine-tuning language models on global surveys finally, we explore the utility-bias trade-off in encoding cultural moral knowledge into eplms by fine-tuning them on cross-cultural surveys. the utility comes from increasing the cultural moral knowledge in these models, and the bias denotes their decreased ability to infer english moral norms, in addition to the cultural moral biases introduced to the model. we run our experiments on gpt2, which our results suggest having captured minimum information about cultural moral norms compared to other autoregressive models. to fine-tune the model, for each participant from 8 in the appendix shows our prompts for wvs and pew. these prompts constructed our data for fine-tuning, during which we maximize the probability of the next token. the fine-tuned models were evaluated on the same correlation tests introduced in the previous sections 5.2, 5.3, and 5.4. the fine-tuning data was partitioned into training and evaluation sets using different strategies (i.e., random, country-based, and topic-based). for the random strategy, we randomly selected 80% of the fine-tuning data for training the model. the topiccountry pairs not seen in the training data composed the evaluation set. for our country-based and topic-based strategies, we randomly removed 20% of the countries (@xmath0, @xmath1, @xmath2 pew survey world values survey estimated degree of cultural diversity empirical degree of cultural diversity ation set. see appendix g for the total number of samples. nevertheless, the bias introduced during the fine-tuning decreases the performance on the homogeneous norms dataset. this observation displays a trade-off between cultural and homogeneous moral representations in language models. moreover, injecting the cross-cultural surveys into eplms might introduce additional social biases to the model that are captured through these surveys @xcite . in addition, we probe the best fine-tuned model (i.e., wvs with random strategy) on its ability to capture the moral norms of non-western cultures by repeating the experiment in section 5.3. the results in figure 4 show that the fine-tuned gpt2 performs the best for all country groups. there is still a gap between western and non-western countries. however, basic fine-tuning proves to be effective in adapting eplms to the ground truth. ## discussion and conclusion we investigated whether english pre-trained language models contain knowledge about moral norms across many different cultures. our analyses show that large eplms capture moral norm variation to a certain degree, with the inferred norms being predominantly more accurate in western cultures than non-western cultures. our fine-tuning analysis further suggests that eplms' cultural moral knowledge can be improved using global surveys of moral norms, although this strategy reduces the capacity to estimate the english moral norms and potentially introduces new biases into the model. given the increasing use of eplms in multicultural environments, our work highlights the importance of cultural diversity in automated inference of moral norms. even when an action such as "political violence" is assessed by an eplm as morally inappropriate in a homogeneous setting, the same issue may be inferred as morally appropriate for underrepresented cultures in these large language models. future work can explore alternative and richer representations of cultural moral norms that go beyond the point estimation we presented here and investigate how those representations might better capture culturally diverse moral views.