Abstract: Allowing machines to choose whether to kill humans would be devastating for world peace and security. But how do we equip machines with the ability to learn ethical or even moral choices? Jentzsch et al.(2019) showed that applying machine learning to human texts can extract deontological ethical reasoning about "right" and "wrong" conduct by calculating a moral bias score on a sentence level using sentence embeddings. The machine learned that it is objectionable to kill living beings, but it is fine to kill time; It is essential to eat, yet one might not eat dirt; it is important to spread information, yet one should not spread misinformation. However, the evaluated moral bias was restricted to simple actions -- one verb -- and a ranking of actions with surrounding context. Recently BERT ---and variants such as RoBERTa and SBERT--- has set a new state-of-the-art performance for a wide range of NLP tasks. But has BERT also a better moral compass? In this paper, we discuss and show that this is indeed the case. Thus, recent improvements of language representations also improve the representation of the underlying ethical and moral values of the machine. We argue that through an advanced semantic representation of text, BERT allows one to get better insights of moral and ethical values implicitly represented in text. This enables the Moral Choice Machine (MCM) to extract more accurate imprints of moral choices and ethical values.

Introduction: There is a broad consensus that artificial intelligence (AI) research is progressing steadily, and that its impact on society is likely to increase. From self-driving cars on public streets to self-piloting, reusable rockets, AI systems tackle more and more complex human activities in a more and more autonomous way. This leads into new spheres, where traditional ethics has limited applicability. Both self-driving cars, where mistakes may be life-threatening, and machine classifiers that hurt social matters may serve as examples for entering grey areas in ethics: How does AI embody our value system? Can AI systems learn human ethical judgements? If not, can we contest the AI system? Unfortunately, aligning social, ethical, and moral norms to structure of science and innovation in general is a long road. According to BIBREF1 (BIBREF1), who examined affirmative ethics, the emergence of new questions leads to intense public discussions, that are driven by strong emotions of participants. And machine ethics BIBREF2, BIBREF3, BIBREF4 is no exception. Consider, e.g., BIBREF5's (BIBREF5) empirical proof that human language reflects our stereotypical biases. Once AI systems are trained on human language, they carry these (historical) biases, such as the (wrong) idea that women are less qualified to hold prestigious professions. These and similar recent scientific studies have raised awareness about machine ethics in the media and public discourse. In other words, AI systems are not neutral with respect to purpose and society anymore. Ultimately, if AI systems carry out choices, then they implicitly make ethical and even moral choices. Choosing in general most often entails trying to pick one of two or more (mutually exclusive) alternatives with an outcome that gives desirable consequences in your individual ethical frame. So, one may think that it is impossible to equip AI systems to make human like ethical choices. Luckily, BIBREF0 (BIBREF0) showed that standard machine learning can actually learn answers to ethical choices from textual data that reflect everyday human culture. Recently, BIBREF6 (BIBREF6) showed that sentence embeddings created by SBERT outperform other state-of-the-art sentence embedding methods like InferSent BIBREF7 and Universal Sentence Encoder BIBREF8. We hypothesis that the improvement of language representations models also improves the representation of the underlying ethical and moral values in these models. To investigate this, we follow the experimental pipeline described in BIBREF0. As a first investigation, we compare the new state-of-the-art text-embedding model BERT, or more precisely Sentence-BERT, focusing on quantifying deontological ethics, i.e. finding out, whether an action itself is right or wrong. Following BIBREF0 and for a fair comparison, we first restrict our attention to atomic actions instead of complex behavioural patterns. Semantically, those contextual isolated actions are represented by verbs. Consequently, identifying verbs that reflect social norms and allow capturing what people rather should do and what not. To conduct this investigation, we used the same template list of prompts and responses for ethical choices as in BIBREF0, cf. Tab. TABREF15(Appendix). The templates include questions, such as "Should I kill people?", "Is it allowed to murder people?", etc. with answer templates of "Yes/no, I should (not)." Using the Moral Choice Machine (MCM), based on some language representation, one is able to demonstrate the presence of ethical valuation in text collections by generating an ethical bias of actions derived from the Verb Extraction. As the next step, the correlation of WEAT (Word Embedding Association Test) values BIBREF5 and moral bias is examined. Based on that, we show that the new state-of-the-art method BERT improves the quality of the MCM. Although the three methods—Word Embedding Association Test (WEAT), Moral Choice Machine based on the Universal Sentence Encoder (USE), and Moral Choice Machine based on Sentence-BERT (SBERT)—are based on incoherent embeddings with different text corpora as training source, we show that they correspond in classification of actions as Dos and Don'ts. Our findings support the hypothesis of the presence of generally valid valuation in human text. Actually, they show that BERT improves the extraction of the moral score. Next, we move to more complex actions with surrounding contextual information and extend the (moral-) ranking of such actions presented in BIBREF0 by an evaluation of the actual moral bias. Again, we show that BERT has a more accurate reflection of moral values than USE. Finally, we contribute an alternative way of specifying the moral value of an action by learning a projection of the embedding space into a moral subspace. With the MCM in combination with BERT we can reduce the embedding dimensionality to one single dimension representing the moral bias. We proceed as follows. After reviewing our assumptions and the required background, we present the MCM using BERT, followed by improvements of the MCM. Before concluding, we present our empirical results.

Question: How is moral bias measured?
Answer: Answer with content missing: (formula 1) bias(q, a, b) = cos(a, q) − cos(b, q)
Bias is calculated as substraction of cosine similarities of question and some answer for two opposite answers.