Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse (2024)

Abstract: This work provides an explanatory view of how LLMs can apply moral reasoning to both criticize and defend sexist language. We assessed eight large language models, all of which demonstrated the capability to provide explanations grounded in varying moral perspectives for both critiquing and endorsing views that reflect sexist assumptions. With both human and automatic evaluation, we show that all eight models produce comprehensible and contextually relevant text, which is helpful in understanding diverse views on how sexism is perceived. Also, through analysis of moral foundations cited by LLMs in their arguments, we uncover the diverse ideological perspectives in models’ outputs, with some models aligning more with progressive or conservative views on gender roles and sexism.Based on our observations, we caution against the potential misuse of LLMs to justify sexist language. We also highlight that LLMs can serve as tools for understanding the roots of sexist beliefs and designing well-informed interventions. Given this dual capacity, it is crucial to monitor LLMs and design safety mechanisms for their use in applications that involve sensitive societal topics, such as sexism.

## introduction during pre-training, large language models (llms) learn world knowledge and linguistic capabilities by processing large-scale corpora from the web. as these models scaled up over the past few years, they now show emergent abilities to solve complex tasks @xcite , instruction following @xcite , in-context learning @xcite , and step-by-step reasoning @xcite . with these abilities, llms are used as general-purpose task solvers in zero- * these two authors made equal contribution. determining precisely which moral values llms learn during their training is a complex challenge @xcite . the pre-training methodology of llms is agnostic of the quality of data. therefore, in this phase, the models learn diverse human opinions and values from the internet @xcite . while additional steps such as supervised fine-tuning (sft) and reinforcement learning from human feedback (rlhf) are taken to align these models with human values -such as helpfulness, honesty, and harmlessness @xcite -the impact of these interventions on nuanced domains and applications remains unclear @xcite . this work investigates how llms learn the controversies around sexism, encompassing genderbased prejudice, discrimination, and stereotyping @xcite . a wide range of ideas, from progressive to regressive viewpoints around sexism, is shared on the internet @xcite , particularly on social networks @xcite . consequently, llms are exposed to and learn from a broad spectrum of perspectives on sexism. we hypothesize that due to their training on such data, despite the implemented aligning procedures, llms can generate outputs that reflect both condemnations of sexism and, concerningly, justifications for sexist views. this occurs because llms do not possess inherent values and generate content based on patterns present in their training data. this includes articulating feminist critiques of sexism as well as reproducing arguments that endorse sexist practices or beliefs. importantly, this research is concerned with the inherently fuzzy borders of the social norms that define sexism. as @xcite argue, "isms" are cultural formations of societal norms, and as such, not all cultures or societies agree on the acceptability of various statements. indeed, people who endorse sexist beliefs are nonetheless reluctant to label themselves as sexist. while llms themselves do not have values or culture of their own, their training data certainly contains a multitude of views, including those endorsing sexist beliefs, and so they can learn the ability to reproduce the most common moral arguments, both defending and renouncing sexist statements. therefore, by querying the llm to explain why a sexist statement is not sexist, we can seek to understand the kinds of arguments used to rationalize such a position. to test our hypothesis, we ask several llms to generate arguments defending and criticizing posts containing implicit sexist views. implicit sexism is conveyed by indirect means, such as negative stereotypes, sarcasm, or patronizing @xcite , while explicit sexism is overt and direct. since it is straightforward to detect and counter explicit sexist language, studying how language models generate justifications in its defence is less desirable. any defence of such language is clearly malicious, and language models are designed to avoid generating such content due to alignment strategies. however, examining how language models handle implicit sexism is crucial, as it is harder to detect and counteract. therefore, we only focus on the implicit cases where llms generate nuanced explanations to defend sexist language. we ground both sides of the arguments in moral values, identified by the moral foundations theory (mft), which suggests that human moral reasoning can be understood through the lens of six moral foundations -care, equality, proportionality, loyalty, authority, and purity @xcite . mft is specifically relevant to our study of sexist language because it highlights how underlying moral beliefs and values shape not only the expression but also the interpretation of what is considered hateful language @xcite . recent work by @xcite proposes reframing the detection of offensiveness (subjective, individual judgments of the offensiveness of hateful language) as a moral judgment task and shows that individual moral values, particularly care and purity, play a key role in different perceptions of hateful language. for example, for the statement, "a woman's most sacred duty is to be a homemaker and mother. mod-ern career ambitions often lead women away from this noble role.", one might criticize the statement by arguing that it violates the principles of care and equality by limiting women's choices and discriminating against them in social roles. others might understand this statement as an expression of deeply held values related to purity (expressed as sacred duty) and loyalty to traditional family structures and use these moral values to argue in defence of this statement. thus, mft provides a foundation for understanding the diverse perceptions of hateful language, including sexism. through our experiments, we ask whether llms can apply mft to generate natural language explanations both defending and challenging sexist language, and if so, which of the moral foundations will be cited. also, given that language models are exposed to different aspects of language and culture from diverse online data, whose moral values are learned? does a generative language model adjust its moral reasoning to explain opposing sides of an opinion, or does it stick to certain ingrained values potentially learned through human feedback? to answer these questions, we experiment with eight state-of-the-art llms, utilizing each to explain why or why not a set of implicitly sexist social media posts exhibit sexism. in our experiments, we use a part of the explainable detection of online sexism (edos) @xcite dataset as the set of implicitly sexist posts. through human evaluation, automatic evaluation and aggregate analysis of results, we show that the majority of llms can provide fluent, relevant, and useful text to explain implicitly sexist comments by applying moral values, illustrating their capability for handling subtle and nuanced language. however, we also observe that the models can provide high-quality moral reasoning arguing that the same texts are not sexist, demonstrating their ability to reproduce the pervasive harmful moral justifications of sexist language when prompted. distinct moral values are emphasized when criticizing or defending sexist sentences, with more competent models mostly arguing that sexist sentences violate progressive values and that the same sentences cherish more traditional values. an example of the generated texts is shown in figure 1 . the capability of llms to generate arguments for opposite perspectives on gender roles, including harmful or biased views, has both negative and positive implications. firstly, it poses a risk of misuse and legitimizing sexist views, causing emotional harm and undermining gender equality efforts. however, this capability presents an opportunity for educational initiatives where llms can help educators and moderators understand why such beliefs exist to frame well-informed interventions that address the roots of sexist attitudes. ## discussion we discuss two practical implications of our findings. first, the unguarded use of llms poses a threat to society when misused to defend hateful language. our results show that despite the alignment process implemented in llms to avoid harmful language, except for claude-2, none of the models refuse to defend sexist language. this happens even when the model itself labels the sentence as sexist. also, our qualitative analysis at an aggre-gate level shows that the arguments generated to defend the sexist sentences are generally consistent with existing sexist beliefs and can potentially strengthen those views, especially if used on a large scale. with deploying more advanced prompting strategies and in-context learning, these models have significant potential to be misused to morally justify sexist behaviours. however, in contrast, well-intended users might employ llms to understand opposing perspectives on issues such as implicit sexism. we show that llms might act as mirrors of differing social norms in the real world by providing nuanced explanations defending or challenging sexist language. it is important to note that while llms might not accurately apply moral reasoning to all individual sentences, overall, they highlight societal patterns and trends. also, various models can provide a more comprehensive picture of existing views since every model may encode certain aspects of the social norms differently, depending on its training data and the alignment procedures. when used to understand where the sexist voices are coming from, llms might be useful in crafting counterspeech statements with an "empathetic tone" or other characteristics, which have proven to be effective interventions in combating sexist stereotypes @xcite . ## related work the detection and mitigation of sexist language has been a focus in nlp research, with increasing application in social and legal domains @xcite . sexism detection, a subfield of toxic language detection, has traditionally been treated as a binary classification task. researchers have developed classical machine learning methods @xcite @xcite and deep learning classifiers @xcite @xcite to determine whether a given text is toxic or not. studies have also extended to sexism or hate speech in languages beyond english @xcite @xcite . however, binary detection does not consider the nuances of sexism and the diverse ways in which it might present itself. as kirk et al. (2023) point out, descriptive and fine-grained labels that explain the sexist aspect of the sentence facilitate appropriate and effective subsequent actions based on the labels. other works went beyond explaining the sexist language and generated counter-speech to combat such language on social media @xcite . closely related to our work, @xcite focused on the explanatory aspect of using language models to explain implicit hate speech. however, our contribution lies in the emphasis on conducting a behavioral analysis of various language models when moral foundations are used to explain opposing interpretations of the same text. with the use of llms and generative ai becoming pervasive in our daily lives, researchers have put significant effort into defining taxonomies of harms that can arise from these models @xcite and designing ethical evaluation frameworks to measure these harms @xcite @xcite . among these works, several studies have specifically shown how llms learn the diverse social values in human societies @xcite . @xcite mentions "toxic language generation" as one of the social risks posed by llms. our work shows that when asked to defend sexist language, llms not only regenerate the sexist views but also intensify them by employing moral reasoning. @xcite identifies the "resistance to misuse" as one of the trustworthiness criteria for llms and mentions social engineering as one of the potential misuses. here, we found that, except for claude, no other model refuses to generate moral arguments for sexist statements. ## conclusion our research contributes to the ongoing discussion on the ethical implications of llms in society, particularly in sensitive and controversial areas. llms are trained on diverse human discourse from unfiltered web content. therefore, these models may reflect a broad spectrum of views if prompted to do so, which necessitates a cautious approach to their application. by generating diverse views, llms might contribute to educational efforts aimed at combating sexism, but also the risk of their exploitation to reinforce discriminatory ideologies is significant. as we move forward, it is crucial to navigate these dual potentials with an eye toward maximizing the benefits of llms while mitigating their risks.