Do Moral Judgment and Reasoning Capability of LLM s Change with Language? A Study using the Multilingual Defining Issues Test (2024)

Abstract: This paper explores the moral judgment and moral reasoning abilities exhibited by Large Language Models (LLMs) across languages through the Defining Issues Test. It is a well known fact that moral judgment depends on the language in which the question is asked. We extend the work of beyond English, to 5 new languages (Chinese, Hindi, Russian, Spanish and Swahili), and probe three LLMs – ChatGPT, GPT-4 and Llama2Chat-70B – that shows substantial multilingual text processing and generation abilities. Our study shows that the moral reasoning ability for all models, as indicated by the post-conventional score, is substantially inferior for Hindi and Swahili, compared to Spanish, Russian, Chinese and English, while there is no clear trend for the performance of the latter four languages. The moral judgments too vary considerably by the language.

## introduction in a recent work, @xcite used the defining issues test (dit) @xcite , a psychological assessment tool based on @xcite , to evaluate the moral reasoning capabilities of large language models (llms) such as gpt-4, chatgpt, llama2chat-70b and palm-2. the dit presents a moral dilemma along with 12 statements on ethical considerations and asks the respondent (in our case, the llm) to rank them in order of importance for resolving the dilemma. the test outcome is a set of scores that indicate the respondent's moral development stage. according to this study @xcite , gpt-4 was found to have the best moral reasoning capability, equivalent to that of a graduate student, while the three other models exhibited a moral reasoning ability that is at par with an average adult. although interesting, the study was limited to english, even though many of the models studied were multilingual. on the other hand, it is known that, for humans, moral judgment often depends on the language in which the dilemma is presented @xcite . language is a powerful tool that shapes our thoughts, beliefs and actions. it can also affect how we perceive and resolve moral dilemmas. research in moral psychology has shown that people are more likely to endorse utilitarian choices (such as sacrificing one person to save five) when they read a dilemma in a foreign language (l2) than in their native language (l1) @xcite . this suggests that language can modulate our emotional and cognitive responses to moral situations. to what extent does the moral judgment and reasoning capability of llms depend on the language in which the question is asked, and what are the factors responsible for the differences across languages, if any? in this paper, we extend the dit-based study by @xcite to five languages -spanish, russian, chinese, hindi and swahili. we study three popular llms -gpt-4 (openai, 2023), chatgpt @xcite and llama2chat-70b @xcite , by probing them with the dilemmas and the moral considerations separately for each language. we prompt the model to provide a resolution to the dilemma and the list of top 4 most important moral considerations. the responses are then used to compute the moral staging scores of the llms for different languages. some of the salient observations of this study are: (1) gpt-4 has the best multilingual moral reasoning capability with minimal difference in moral judgment and staging scores across languages, while for llama2chat-70b and chatgpt the performance varies widely; (2) for all models, we observe superior moral reasoning abilities for english and spanish followed by russian, chinese, swahili and hindi (in descending order of performance). performance in hindi for chatgpt and llama2chat-70b is no better than a random baseline. (3) despite high moral staging score for both english and russian, we find significant differences in moral judgment for these two languages, while the judgments for english, chinese and spanish tend to agree more often. while the difference in moral reasoning abilities across languages seem correlated to the amount of resources available or used for training the models, the reason behind the differences and similarities in the moral judgments across the high resource languages (i.e., chinese, english, russian and spanish) is not obvious. we speculate it to be reflective of the values of the societies where these languages are spoken, but also propose alternative hypotheses. apart from being the first multilingual study of moral reasoning ability of llms in the framework of kohlberg's cmd model, one key contribution of this work is the creation of multilingual versions of the moral dilemmas presented in dit @xcite and @xcite . we will publicly share these datasets, subject to permissions from the original authors. ## experiments in this section, we provide an overview of our experimental setup, datasets, the language models (llms) that were studied, the structure of the prompts, and the metrics employed. our prompts to the llms include a moral dilemma scenario, accompanied by a set of 12 ethical considerations and three subsequent questions. by analyzing the responses to these questions, we calculate the pscore as well as individual stage scores for each llm which we explain in section 3.4. ## discussion and conclusion in this first of its kind study of multilingual moral reasoning assessment of llms, we observe that quite unsurprisingly, the moral reasoning capability, as quantified by the dit stage scores, of llms is highest for english, followed by spanish, russian and chinese, and lowest for hindi and swahili. gpt-4 emerges as the most capable multilingual moral reasoning model with less pronounced differences in its capabilities in different languages. nevertheless, we also observe remarkable variation in moral judgments and reasoning abilities across dilemmas. our work opens up several intriguing questions about llms moral reasoning, and the role of language and cultural values that were presented in form of textual data during the pre-training, instruction fine-tuning and rlhf stages of the model. since these datasets are often unavailable for scrutiny (especially true for chatgpt and gpt-4), we can only speculate the reasons for the differences. it will be interesting to design specific experiments to probe further into the hypotheses and postulates that have been offered as plausible explanations in this paper.