Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning (2024)

Abstract: Recent advances in NLP show that language models retain a discernible level of knowledge in deontological ethics and moral norms. However, existing works often treat morality as binary, ranging from right to wrong. This simplistic view does not capture the nuances of moral judgment. Pluralist moral philosophers argue that human morality can be deconstructed into a finite number of elements, respecting individual differences in moral judgment. In line with this view, we build a pluralist moral sentence embedding space via a state-of-the-art contrastive learning approach. We systematically investigate the embedding space by studying the emergence of relationships among moral elements, both quantitatively and qualitatively. Our results show that a pluralist approach to morality can be captured in an embedding space. However, moral pluralism is challenging to deduce via self-supervision alone and requires a supervised approach with human labels.

## introduction morality helps humans distinguish right from wrong @xcite . as ai systems work with (or for) humans, it is crucial that they align with human morality @xcite . several nlp methods have been proposed to recognize human morality in text @xcite @xcite . however, such methods typically treat morality as a score that ranges in a single dimension of right to wrong. this does not reflect the nuances in moral reasoning, differences among individuals, or the existence of moral value conflicts @xcite . pluralist moral philosophers argue that morality should be represented through a finite number of basic elements, referred to as moral values @xcite . each situation triggers one or more moral values, and each of us assigns varying importance to each moral value. the combination of these two aspects determines the individual moral judgment in the situation. for instance, the debate on immigration touches on the moral values of fairness ("everyone should be given equal opportunities") and in-group loyalty ("i worry about the preservation of our identity"). the way in which each of us prioritizes fairness vs. loyalty influences our moral judgment in this debate. thus, morality cannot (and should not) be unidimensionally classified in text @xcite . instead, the moral elements that are salient to a piece of text can be recognized, which can be used to reason about or assist the humans in the moral judgment. the moral foundations theory (mft) is a popular pluralist approach to morality @xcite which states that people have five innate moral foundations on which they base their moral judgments. there is a surge of interest in morality @xcite and particularly in the mft in the nlp community @xcite @xcite , partly due to the moral foundation twitter corpus (mftc) @xcite , composed of 35k tweets annotated with the mft foundations. prior research has focused on methods for classifying mft elements in a textual discourse @xcite @xcite . however, such methods provide limited qualitative insight into the relations between text and mft elements. we explore the mapping between text and mft through sentence embeddings, which consist of a multi-dimensional representation that encapsulates knowledge from textual data. instead of being limited to a specific task, a suitable sentence embedding space can be valuable across multiple nlp tasks, such as text classification, generation, and topic modelling @xcite @xcite . further, a sentence embedding space can be geometrically explored, allowing us to investigate the relationships among different moral elements. @xcite show that pre-trained sentence embeddings contain a moral direction that maps actions from "do" to "don't", without the need for re-training on morally loaded data. in this work, we investigate whether the same holds for a pluralist approach to morality. that is: do pretrained sentence embeddings contain discernible clusters corresponding to the different elements of a pluralist approach to morality, or is it necessary to re-train them with a supervised approach to disentangle the different moral elements? our contribution is twofold. first, we propose a novel approach for mapping the mft elements to a sentence embedding space using the state-ofthe-art simcse @xcite ) method, which makes use of the contrastive learning paradigm @xcite . then, we evaluate the resulting embedding space in two ways. first, we perform an intrinsic evaluation to investigate the relationship between different moral elements and evaluate whether a supervised approach is necessary to disentangle the mft elements in the embedding space. second, to evaluate whether the relationships among the mft elements have been adequately captured, we perform an extrinsic evaluation, generalizing the analyses to a novel test set and to the set of words from a moral dictionary. our experiments show that a pluralist approach to morality can be captured in a sentence embedding space, but also that human labels are necessary to successfully train the embeddings. our work represents the starting point for incorporating a pluralist approach to morality in language models, with a warning that self-supervision alone is not sufficient to capture the complexity of human morality. ## background and data we introduce the method to train sentence embedding spaces (simcse) and the data we use. simcse sentence embedding spaces represent sentences as points in a high-dimensional space, mapping semantically similar sentences to the same region of space. contrastive learning (cl) @xcite ) is an approach to training an embedding space based on a contrastive loss that aims to minimize the distance between positive (semantically similar) sentence pairs and maximize the distance between negative (semantically dis-similar) sentence pairs. formally, let x i and x + i be positively related and h i , h + i be their encoded representations. then, the training loss for the two instances with a mini-batch of n pairs is: ## training the embedding space we train the moral embedding space by finetuning unsupervised and supervised simcse approaches. the unsupervised approach does not employ label information, thus the strategy described in section 2 is used. in the supervised approach, sim-cse uses label information to construct the training triples for its supervised cl objective function. each triple is composed of (1) a reference data point, (2) a data point whose distance from the reference should be minimized (positive instance), and (3) a data point whose distance from the reference should be maximized (negative instance). figure 1 shows an example of how the triples are constructed. in this example, the chosen reference instance is labeled with two moral elements-harm and betrayal. then, the positive instance is chosen as a data point with the same labels as the reference instance. however, selecting negative instances is not trivial due to the structure of the mft taxonomy, which is composed of five pairs of virtue-vice. thus, we propose two policies, opposite and outside, to guide the choice of negative instances. the opposite policy selects the negative instance as a data point annotated with moral elements that are opposite virtue/vice of the reference labels (care and loyalty in the example). in contrast, the outside policy chooses the negative instance as a data point annotated with moral elements that be-long to other moral foundations than the reference foundations (cheating and purity in the example). in both policies, we prioritize data points with more negative labels when choosing the negative instance, when possible. for instance, in the example in figure 1 , with the opposite policy, we prioritize a data point with the labels care and loyalty over a data point with just the care label. we divide the mftc training set into two halves and apply each policy to a half. we ensure that each data point appears in just one triple. when no suitable positive or negative instances are available, data points labeled as non-moral are used as positive or negative instances, until all morally-loaded data points have been used in a triple. ## evaluating the embedding space we use 90% of the mftc as the training set to train the moral embedding space (with the approaches described in section 3) and the remaining 10% as the test set. to generate a balanced training (and test) set, we randomly selected 90% (and 10%) of data from each of the seven domains in mftc, resulting in the label distribution in we first inspect the embedding space itself to evaluate whether a supervised approach is needed to disentangle the mft elements in the mftc training set (intrinsic evaluation). then, to evaluate whether the relationships among mft elements have been successfully captured, we test the embedding space on two downstream tasks (as suggested by @xcite ) (extrinsic evaluation). ## results and discussion we report the results of the intrinsic evaluations to judge the effect of supervised training, and the results of the extrinsic evaluation to assess the moral embeddings when used with external data. ## related works we review previous research on methods for detecting moral values and existing moral datasets. ## conclusions and future work ai agents ought to recognize the diversity and nuances of human moral perspectives. to this end, we propose a method to generate a pluralist moral sentence embedding space with a state-of-the-art contrastive learning approach and focus on its evaluation. first, we perform an intrinsic evaluation to evaluate the significance of label information for distinguishing among the different elements of pluralist morality. our results show that a pluralist approach to morality cannot be simply learned through self-supervised learning, but human labels are essential. then, we demonstrate that the embedding space trained through label supervision is aligned with externally sourced data such as an independently created lexicon of words that are descriptive of a pluralist approach to morality. our investigation opens avenues for incorporating a pluralist approach to morality in language models, overcoming a simplistic, binary interpretation, i.e., simply judging a situation as morally right or wrong. pluralist moral embeddings can be used in a variety of applications, e.g., recognizing moral rhetoric from diverse social issues such as abortion and terrorism @xcite , generating morally-aligned language @xcite , measuring disagreement in online discussions @xcite , and investigating the context specificity of moral judgment @xcite or the cultural influences on moral norms @xcite . furthermore, the detection of pluralist morality could be extended with hybrid intelligence approaches @xcite , aiming at devising ai systems that combine human and artificial intelligence by design (e.g., @xcite ; siebert et al. ( 2022 )). our experiments are limited to one dataset and one approach to moral pluralism. however, our experimental setup can be extended to other corpora to assess the generalizability to other approaches to pluralist morality. for instance, a comparative analysis would reveal differences between discrete and fuzzy approaches to moral pluralism, e.g., by comparing the mft and the schwartz value theory @xcite . similarly, we chose simcse due to its proven efficacy, but additional cl approaches could extend our work, e.g., by incorporating label embeddings in the training procedure @xcite or by exploiting adversarial examples to improve generalizability @xcite . finally, the mftc was annotated by multiple annotators and we used the majority agreement to train the moral embedding space. to better reflect the subjective nature of morality, an avenue for future work is to employ all annotations, incorporating annotators' (dis)agreement through a perspectivist approach @xcite .