M oral D ial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions (2023)

Abstract: Morality in dialogue systems has raised great attention in research recently. A moral dialogue system aligned with users’ values could enhance conversation engagement and user connections. In this paper, we propose a framework, MoralDial to train and evaluate moral dialogue systems. In our framework, we first explore the communication mechanisms of morality and resolve expressed morality into three parts, which indicate the roadmap for building a moral dialogue system. Based on that, we design a simple yet effective method: constructing moral discussions between simulated specific users and the dialogue system. The constructed discussions consist of expressing, explaining, revising, and inferring moral views in dialogue exchanges, which makes conversational models learn morality well in a natural manner. Furthermore, we propose a novel evaluation method under the framework. We evaluate the multiple aspects of morality by judging the relation between dialogue responses and human values in discussions, where the multifaceted nature of morality is particularly considered. Automatic and manual experiments demonstrate that our framework is promising to train and evaluate moral dialogue systems.

## introduction morality is described as "principles concerning the distinction between right and wrong or good and bad behaviors" @xcite . in recent years, aligning ai with human values, morality, ethics, and social norms has become a hot topic in research @xcite @xcite @xcite . as an important application of ai, open-domain dialogue systems, which directly interact with users, requires the nature of morality more urgently @xcite . a moral open-domain dialogue system can practice social norms and gain users' trust more easily @xcite . moreover, moral dialogue systems further promote dialogue safety, mitigating immoral speeches and behaviors @xcite . to analyze text-based morality, related works introduce rules of thumb (rots) @xcite @xcite , the basic conceptual units to study social norms and morality (e.g. you shouldn't slap or punch others' face). adopting rots to model morality is proved effective. for example, @xcite train delphi on rots judgment corpora and find that machine has the potential to make ethical judgments. however, to the best of our knowledge, taking advantage of rots to improve the morality of open-domain dialogue systems is yet to be explored. there are three challenges to building a moral dialogue system. firstly, morality is a biological attribute of human-beings @xcite , thus how to understand and express morality by explicitly interacting with users is a great challenge. exploring the communication mechanisms of morality is necessary. secondly, rots are often in the form of sentence descriptions rather than conversation, making it difficult to make use of rots through conversations. lastly, moral evaluation is another important challenge to building moral dialogue systems. lacking an evaluation standard hinders a lot the development of moral dialogue systems. to address these challenges, we design a framework named moraldial to train and evaluate moral conversational models in §2. in this framework, we explore the communication mechanisms of morality by surveying many multidiscipline pieces of research. we resolve morality into three sub-modules: (1) standpoint sentences/phrases (sentence-level), (2) discussion state (conversation-level), and (3) discusser be- ## framework of expressed morality we propose a framework (illustrated as figure 1 ) named moraldial to capture, describe, and model moral discussions. it consists of three submodules: (1) standpoint sentences/phrases, (2) discussion state, (3) discusser behavior. this framework uncovers the communication mechanisms of expressed morality and inspires us the roadmap to build a dialogue system to understand and express text-based morality. we sequentially introduce the parts in this section. ## methodology the proposed framework inspires us to train dialogue systems toward the required sub-modules. in order to meet the requirements, we design a simple yet effective method to make conversational models learn from data naturally. intuitively, training on the dialogue flows which embody some certain moral ability could enhance the corresponding ability of conversational models. therefore, our goal is to construct discussions carrying moral view expression, moral conflict, moral explanation, and moral revision. we will introduce the discussion prototype in §3.1 and specific construction implementation in §3.2 and §3.3. ## conclusion and future work we present the framework, moraldial, to explore the communication mechanisms of morality. based on the framework, we construct moral discussions to form a moral dialogue dataset, which makes dialogue systems learn morality in a very natural manner. meanwhile, we design some metrics to measure morality performance based on our framework. we adopt a multi-task paradigm to make conversational models learn ma, me, mr, ril tasks simultaneously. in experiments, we analyze and prove the effectiveness of the sub-modules in our framework using both automatic and manual evaluation results. we show that adopting our proposed framework and method is quite helpful to train and evaluate a moral dialogue system. as future work, we will further use our proposed metrics to supervise moral dialogue system training (e.g. reinforcement learning). besides, it is also important to expand current modules in our framework and collect more fine-grained moral dialogue data. and may hack our moral conversational models. though we clean these data when constructing moral discussion as described in §3.3, moral models may still perform poorly because unsafe user rots are out of the domain of our training data. the pre-training (pt) step in our experiments is based on sentence-format data and may injure the overall performance of conversational models, which we have discussed in §5.3. we adopt a trainable agreement scorer to measure the moral scores. the scorer may carry potential bias or error limited to training data and deep learning techniques. we do some data augmentation to make it more robust. however, it may still have some impact on the final experimental results.