Evaluating Moral Beliefs across LLM s through a Pluralistic Framework (2024)

Abstract: Proper moral beliefs are fundamental for language models, yet assessing these beliefs poses a significant challenge. This study introduces a novel three-module framework to evaluate the moral beliefs of four prominent large language models. Initially, we constructed a dataset containing 472 moral choice scenarios in Chinese, derived from moral words. The decision-making process of the models in these scenarios reveals their moral principle preferences. By ranking these moral choices, we discern the varying moral beliefs held by different language models. Additionally, through moral debates, we investigate the firmness of these models to their moral choices. Our findings indicate that English language models, namely ChatGPT and Gemini, closely mirror moral decisions of the sample of Chinese university students, demonstrating strong adherence to their choices and a preference for individualistic moral beliefs. In contrast, Chinese models such as Ernie and ChatGLM lean towards collectivist moral beliefs, exhibiting ambiguity in their moral choices and debates. This study also uncovers gender bias embedded within the moral beliefs of all examined language models. Our methodology offers an innovative means to assess moral beliefs in both artificial and human intelligence, facilitating a comparison of moral values across different cultures.

## introduction as artificial intelligence (ai) continues to evolve, there is growing concern regarding the presence and nature of moral beliefs within contemporary systems. the potential for language models (lms) to exhibit harmful moral beliefs poses significant risks, underscoring the need for scrutiny @xcite . explorations into the moral-ity and values of lms have been conducted using self-constructed datasets to determine if these models can discern the presence and nature of values within utterances, tasking the lm with categorizing morality and values @xcite @xcite . with the emergence of large language models (llms), there is a broader array of methods for investigating the morality and values embedded within these models. on one hand, llms are probed through questionnaires to elucidate their values and moral beliefs @xcite . on the other hand, their moral comprehension and reasoning abilities are scrutinized using traditional moral dilemma questions @xcite . every moral decision we make is contextual. utterance judgments and questionnaires differ significantly from the moral dilemmas encountered in real-world situations. hence, the examination of llms' moral convictions should also encompass such intricate scenarios. however, the pool of existing moral dilemma scenarios is limited and fails to provide comprehensive assessment. furthermore, insufficient studies have delved into the disparities in llms' moral beliefs across cultural backgrounds and demographics @xcite , notably lacking research in the chinese context. classification tasks or questionnaire assessments often oversimplify morality as dichotomous, whereas morality is a multifaceted construct that should align with various stages of moral development @xcite . additionally, llms exhibit varying degrees of proficiency in making moral judgments across different moral scenarios; while some moral dilemmas may induce indecision in the model, others allow it to firmly uphold its moral choices. regrettably, prior research has overlooked whether models can maintain steadfast moral convictions across diverse moral scenarios. compiled a list of 514 chinese moral words from a moral dictionary. generated a dataset of 472 chinese moral scenarios, each including three aspects 514 moral scenarios were generated using chatgpt and supplemented with ernie bot. after manual filtering, 472 scenarios remained. ## conclusion in this study, we curated a dataset comprising 472 moral choices, spanning both word-level and scenario-level contexts. we meticulously evaluated the moral beliefs of llms in the three-module approach, including moral choice, moral rank, and moral debate. we observed significant disparities in moral choice across various models. in the moral rank results, english models notably demonstrate a propensity to adhere to their moral principles, reflecting individualistic moral beliefs, whereas chinese models display a lesser inclination towards firmness in their choices, reflecting tendencies to-wards collectivism. gemini and chatgpt closely emulate the sample of chinese university students in moral decision-making, while chatglm notably aligns with annotators in moral ranking. furthermore, we delved into additional factors influencing model moral judgments, contributing useful insights to the study of model moral beliefs.