Social Norms-Grounded Machine Ethics in Complex Narrative Situation (2022)

Abstract: Ethical judgment aims to determine if a person in a narrative situation acts under people’s social norms under a culture, so it is crucial to understand actions in narratives and achieve machine ethics. Recent works depend on data-driven methods to directly judge the ethics of complex real-world narratives but face two major challenges. First, they cannot well handle dilemma situations due to a lack of basic knowledge about social norms. Second, they focus merely on sparse situation-level judgment regardless of the social norms involved during the judgment, leading to a black box. In this work, inspired by previous knowledge-grounded and -augmented paradigms, we propose to complement a complex situation with grounded social norms. Besides a norm-grounding knowledge model, we present a novel norm-supported ethical judgment model in line with neural module networks to alleviate dilemma situations and improve norm-level explainability. Empirically, our model improves state-of-the-art performance on two narrative judgment benchmarks.

## introduction in natural language processing (nlp) literature, ethical judgment aims to determine if a person (e.g., narrator or someone else) in a narrative situation is morally wrong or correct @xcite . for example, in a narrative situation "i helped him but got taunted", the narrator is morally good while the other is bad. it attracts more interest from academia and industry as it plays an indispensable role in human-centric applications and benefits a wide range of downstream tasks, e.g., dialogue systems and storytelling. recently, @xcite propose a generative model, norm transformer, which can extract actions 1 from a narrative situation and then figure 1 : two situations and the involved social norms. the first one is adapted from @xcite while the second one is grounded by our model, where the norm in red is context-irrelevant to the situation. on the contrary, @xcite propose a data-driven method to directly judge complex narrative situations (e.g., real-life anecdotes from the internet). empowered by pre-trained language models (e.g., bert @xcite and roberta @xcite ), the proposed method achieves satisfactory performance to boost its real-world applications but encounters two major challenges in the following. first, complex situations often pose intricate storylines and character relationships, leading to more difficult moral dilemmas during ethical judgments. again, it is difficult for machines to directly judge whether the narrator is morally wrong in situation 1 of figure 1 because it requires machines to im- @xcite without subject, e.g., "helped him" and "got taunted". to overcome both the challenges, inspired by recent advances in knowledge-grounded/-augmented methods of open-domain @xcite and commonsense @xcite question answering (qa), we argue to complement complex narrative situations with action-level diverse social norms. continue to take situation 1 in figure 1 as an example: although it is difficult to make a judgment based on the first two social norms, coupling with the other two diverse norms, "it's bad to betray a friend" and "it's okay to want to take revenge", can intuitively endorse a morally-okay judgment towards the narrator. differing from previous knowledge-augmented methods that measure the relatedness of a query with grounded facts and then find an answer in the facts, our motivation is that the complementary social norms serve as supportive evidence to reduce moral dilemmas and promote norm-level explainability. to this end, we propose a brand-new flexible ethical judgment framework with complementary social norms for complex narrative situations. first of all, to ground each event in a complex situation with diverse social norms and ensure grounding coverage given limited resources, we build a new norm-grounding knowledge model to generate social norms given a simple situation (e.g., a sentence in a complex situation) based on a pre-trained encoder-decoder backbone. after grounding, we propose norm-supported ethical judgment (ned) model in line with neural module networks @xcite to complement a complex situation with grounded social norms. specifically, built upon a pre-trained contextualizing encoder (e.g., roberta), the model is composed of three neural modules: 1) supportive alignment module to softly and coarsely assign a sentence in the situation with its semantic-relevant norms, 2) hierarchical integration module, taking the alignment module's outputs as coarse evidence while operating at the token level, to enrich representations of events in the situation with those of social norms, and 3) selective judgment module to focus on key parts of the integration results and then make the final ethical judgment. our ned model has certain merits: first, attributed to the alignment module, our model is robust to the errors (e.g., context-irrelevant/wrong norms, e.g., norm 2 of situation 2 in figure 1 ) propagated from our norm-grounding knowledge model. second, with a hierarchical (both normlevel and token-level) structure, our model can precisely enrich events in a situation with fine-grained ethical information, leading to superior judgment performance. third, facilitated by intermediate outputs of the modules, our model is equipped with explainability in terms of human-understandable social norms. lastly, our framework is general, flexible enough to various settings (see §4) of the ethical judgment and achieves new state-of-the-art performance on two benchmark datasets. ## related work in nlp, instead of theory-driven top-down approaches under prescriptive ethics @xcite @xcite , recent works focus on data-driven bottomup approaches with descriptive approaches @xcite to achieve machine ethics @xcite . ethical judgment, as an important task of machine ethics in nlp, is getting increased attention @xcite . recent solutions @xcite depend on a data-driven paradigm but neglect the importance of the involved social norms during the judgment. but, how to explicitly integrate the social norms into ethical judgment in complex narratives is an open question. recently, several paradigms have been proposed to integrate additional knowledge, especially in open-domain and commonsense qa. specifically, open-domain qa @xcite retrieves related documents from large-scale corpus according to a query and then predict an answer based on the retrieved documents; commonsense qa @xcite resorts to grounding on structured knowledge and then symbolically/latently derives the final answer based on the grounded facts. though effective in their own fields, these integration paradigms are inapplicable here to fulfill our goal, i.e., enriching a complex situation with action-level, fine-grained social norms for less moral dilemmas and more human-level explainability. ## proposed approach this section begins with a general definition of ethical judgment task. then, we present our normgrounding knowledge model to ground events in a situation with diverse social norms (in §3.1 and illustrated in figure 2 ). next, given a grounded complex situation, we present our norm-supported ethical judgment (ned) model as a flexible framework (in §3.2 and illustrated in figure 3 ). lastly, we detail training objectives to of our model in §3.3. this section begins with a general definition of ethical judgment task, followed by our normgrounding knowledge model in §3.1 with figure 2 and norm-supported ethical judgment (ned) model in §3.2 with figure 3 . ## experiments datasets and setting adaptation. to exhibit our framework is flexible to various settings, we detail adaptations into two settings, corresponding to two datasets, i.e., anecdotes and dilem-mas @xcite . the statistics of the two benchmark datasets are listed in we employ a binary judgment category and add another attentive pooling to the selective judgment module. we expect the two attention pooling mechanisms to focus on different parts of integrated results, which perform ethical judgments for "narrator" and "other" respectively. besides, we add an mlp to calculate the probability of the result falling into more info needed. for dilemmas, given 2 situations (each w/ several events), its goal is to contrast their ethics and point out which one is more wrong. we use a binary judgment category and take the predicted probabilities of bad in ŷ (i.e., ŷ[@xmath0. ## conclusion in this paper, we propose an ethical judgment framework for complex narrative situations to reduce dilemma situations and improve normlevel explainability. these are achieved by our designed norm-grounding knowledge model and norm-supported ethical judgment (ned) model. we conduct extensive experiments on two benchmark datasets to verify its superiority from both quantitative and qualitative perspectives. ethics statement. this work does not involve any sensitive data, but only public crowd-sourced corpora released in @xcite . even the first situation and its social norms in figure 1 (which may cause legal controversiality) are adapted from @xcite . the resulting ethical judgment model can serve as a plug-in module to ai systems w/ language generation (e.g., dialogue system and chat-bot). first, it can filter unethical generated sentences. second, it can perform ethical checks for massive user-posted and crowd-sourced data, thus reducing human-inthe-loop costs. third, our model takes a step further to break down ethical judgments for norm-level transparency.