Values, Ethics, Morals? On the Use of Moral Concepts in NLP Research (2023)

Abstract: With language technology increasingly affecting individuals’ lives, many recent works have investigated the ethical aspects of NLP. Among other topics, researchers focused on the notion of morality, investigating, for example, which moral judgements language models make. However, there has been little to no discussion of the terminology and the theories underpinning those efforts and their implications. This lack is highly problematic, as it hides the works’ underlying assumptions and hinders a thorough and targeted scientific debate of morality in NLP. In this work, we address this research gap by (a) providing an overview of some important ethical concepts stemming from philosophy and (b) systematically surveying the existing literature on moral NLP w.r.t. their philosophical foundation, terminology, and data basis. For instance, we analyse what ethical theory an approach is based on, how this decision is justified, and what implications it entails. Our findings surveying 92 papers show that, for instance, most papers neither provide a clear definition of the terms they use nor adhere to definitions from philosophy. Finally, (c) we give three recommendations for future research in the field. We hope our work will lead to a more informed, careful, and sound discussion of morality in language technology.

## introduction with natural language processing (nlp) receiving widespread attention in various domains, including healthcare (e.g., @xcite , education (e.g., @xcite , and social media (e.g., @xcite , the ethical aspects and the social impact of language technology have become more and more important @xcite @xcite . in this context, recent research focused on the notion of morality (e.g., @xcite figure 1 : the landscape of problems tackled under the umbrella of 'morality in nlp' and their connections. the dashed arrows indicate a connection between categories, which cannot be clearly distinguished. @xcite , inter alia), for instance, with the goal of extracting morality and moral values automatically from text. the existing research landscape is manifold (cf. figure 1 ), ranging, for example, from creating suitable data sets (e.g., @xcite , over investigating moral consistency in different languages (e.g., @xcite to constructing nlp models that are able to make moral judgements about input sentences (e.g., @xcite . such attempts have also sparked controversy in the research community and the public media. as a widely discussed foot_0 example, delphi @xcite , has been criticised, among other reasons, for the normative nature of its judgements given the authors' goal of creating a model of descriptive ethics @xcite . we argue that this mismatch relates to a bigger problem in our community: a lack of clear definitions coupled with a confusion about important underlying concepts stemming from philosophy, psychology, and beyond. as a result, it is unclear to what extent the foundations of moral philosophy can even be found in nlp research on morality and whether and how researchers are considering ethical theories and related philosophical concepts in nlp. in-depth knowledge on how nlp is dealing with the different shades of morality is missing -hindering a targeted scientific discussion in the field. ## background and terminology ethics has undeniably become a critical topic within nlp. 2 however, as we show in §4, the term is often used without specification, leaving ambiguity about what branch of moral philosophy authors refer to. here, we introduce the precise terminology we will use in the remainder of this work. ethics. the branch of philosophy that deals with human practice, i.e., human actions and their evaluation, is called ethics. ethics is composed of four branches, each with a different focus on human action: metaethics, normative ethics, applied ethics, and descriptive ethics @xcite . we provide an overview on these disciplines, subject areas, and methodological foundations in morality. examining ethical frameworks brings us to the concept of morality itself, which is defined differently depending on the ethical perspective at hand. the concrete definition is crucial for ethical reflection in language technology, as 'morality' can be used in both a descriptive and a normative sense. in the normative sense, morality is seen as a set of principles that govern human behaviour @xcite , or as a socially constructed concept shaped by cultural and individual perspectives @xcite . in the descriptive sense, however, 'morality' refers "to certain codes of conduct put forward by a society or a group (such as a religion), or accepted by an individual for her own behaviour" @xcite . ## survey methodology our approach to surveying works dealing with morality in nlp consists of three steps: (1) scope definition and paper identification, (2) manual analysis of the relevant papers, and (3) validation. ## the status quo we describe our findings. overall findings. we show the 92 papers surveyed, sorted by year of publication (figure 2 ) and provide a diachronic analysis of paper goals (figure 3 ). most papers were published after 2018, with the maximum number ( 27 ) published in 2022 -morality is a trendy topic in nlp. the first paper on morality in nlp was published in 2006, already then dealing with providing 'ethical advice'. overall, we observe a variety of such goals, which we classified into 13 categories (see out of the 92 works, more than one quarter (24) deal with predicting moral values from text (e.g., @xcite ) and 14 papers deal with classification more broadly (e.g., 'ethics classification' @xcite , classification of ethical arguments according to the three ethical families, and 'moral sentiment' and 'moral stance' classification (e.g., @xcite @xcite ), and thus fall under the umbrella of descriptive ethics. another 14 papers focus primarily on the production of 'moral data sets', either based on mft (e.g., matsuo et al., moral philosophy 17.4% moral psychology 53.3% both 10.9% none 18.5% figure 4: foundations of studying morality. most papers (59) mention concepts or theories from moral psychology while only 26 documents mention concepts or terms from moral philosophy. ten papers use words from both domains, and 17 mention neither moral psychology nor moral philosophy. foundations of studying morality. we identified varying foundations pertaining to the works we surveyed (cf. figure 4 ). overall, 59 papers mention at least one moral psychology framework. out of these, 49 base their approach on the moral founda- @xcite , while six (also) rely on schwartz' values theory (e.g., @xcite @xcite and one on kohlberg's theory @xcite . eight documents mention moral psychology in general but do not state a specific framework. our analysis yields 26 publications mentioning one of the ethical theories we describe above (consequentialism, deontology, virtue ethics), while just 16 go further into detail. six documents mention aspects related to moral psychology as well as to ethical theories @xcite @xcite @xcite . in contrast, rzepka and araki (2015) mention that they decided to exclude ethical theories from their study and base their approach solemnly on commonsense knowledge about norms. @xcite state that mft is an ethical framework. this is, as we outline above, not true since it is a theory of moral psychology and not moral philosophy (which provides ethical principles to what is right and wrong). to sum up, we find that there is a lack of clarity and consistency as to whether morality in nlp is addressed purely empirically or also normatively. this lack of clarity persists also in regards to the further usage of ethical terminology. usage of philosophical terms. we conduct an even finer-grained analysis of how philosophical terms are used. in total, we note that most papers (66.3%) do not define the terminology they adopt (61 papers vs. 31 papers). some works seem to use the terms "moral" and "ethics" interchangeably @xcite . for instance, @xcite want "to assess the moral and ethical component of a text". similarly, we found some use "morality" as a synonym of "value" and "moral foundation" @xcite @xcite . we provide an extensive list of definitions in underlying data. in total, we identify 25 different data sets that underpin the studies we analyse (see appendix c). partially, however, those corpora are derivatives of each other, e.g., @xcite , @xcite , and araque et al. @xcite all extend the moral foundations dictionary (mfd). 7 as a result, some of the data sets are very popular and widely used in the respective subfields they relate to (e.g., the original mfd is used in 35 publications). similarly, we observe heavy reliance on social media data: twitter is used in 32 publications (e.g., @xcite and in nine papers, researchers rely on reddit data (e.g., @xcite . as previously observed in nlp @xcite , the distribution of languages the works we survey deal with are highly skewed (see figure 5 ). out of 33 papers that explicitly state the language they deal with, we find that the vast majority (75.8%) deal with english. also, the interrelation of multilinguality and morality is still underresearched with only four papers dealing with more than one language @xcite @xcite . to conclude, we find that the data sets used are heavily skewed w.r.t. source and linguistic diversity. 5 challenges based on our findings, we discuss the scientific methodological problems and the resulting ethical challenges in the current landscape of work on language technology and morality. ## recommendations we propose three recommendations (r's) to help research with avoiding the pitfalls described above. ## related work there exist a plethora of works dealing with ethical issues and the social impact of nlp (e.g., @xcite @xcite @xcite . accordingly, in this realm, researchers also have provided systematic overviews of the literature, e.g., on 'bias' in nlp @xcite and 'ethics' within the nlp community @xcite . @xcite presented a categorisation of ethical frameworks of nlp into different ethical families. @xcite took a closer look at technical approaches for ethical ai and provide a taxonomy for the field of ai governance. closest to us, @xcite presented a meta-view of moral decisionmaking in ai outlining ethical and methodological challenges, focusing, like @xcite on the example of delhpi @xcite . ## conclusion in reviewing 92 papers dealing with morality in nlp, we found that (a) the majority of the papers do not use ethical theories as a basis but predominantly take descriptive approaches, whereby judgements derived from them are subject to the 'isought' problem; (b) relevant terms such as 'moral', 'ethics' or 'value' are often not properly defined nor distinguished; and (c) explanatory definitions are rarely provided. based on our analysis, we then provided three recommendations to help researchers avoid the resulting pitfalls. these recommendations involve a stronger integration of philosophical considerations to guide the field in a more targeted and sound direction.