Case Study: Deontological Ethics in NLP (2021)

Abstract: Recent work in natural language processing (NLP) has focused on ethical challenges such as understanding and mitigating bias in data and algorithms; identifying objectionable content like hate speech, stereotypes and offensive language; and building frameworks for better system design and data handling practices. However, there has been little discussion about the ethical foundations that underlie these efforts. In this work, we study one ethical theory, namely deontological ethics, from the perspective of NLP. In particular, we focus on the generalization principle and the respect for autonomy through informed consent. We provide four case studies to demonstrate how these principles can be used with NLP systems. We also recommend directions to avoid the ethical issues in these systems.

## introduction the 21st century is witnessing a major shift in the way people interact with technology, and natural language processing (nlp) is playing a central role. a plethora of nlp applications such as question-answering systems @xcite @xcite used in diverse fields like healthcare @xcite , education @xcite , privacy @xcite ; machine translation systems @xcite @xcite , conversational agents @xcite @xcite , recommendation systems @xcite etc. are deployed and used by millions of users. nlp systems have become pervasive in current human lifestyle by performing mundane tasks like setting reminders and alarms to complex tasks like replying to emails, booking tickets and recommending movies/restaurants. this widespread use calls for an analysis of these systems from an ethical standpoint. despite all the advances in efficiency and operations of nlp systems, little literature exists which broadly addresses the ethical challenges of these technologies. ethical theories have been studied for millennia and should be leveraged in a principled way to address the questions we are facing in nlp today. instead, the topic of "ethics" within nlp has come to refer primarily to addressing bias in nlp systems; @xcite provides a critical survey of how bias is studied in nlp literature. the survey finds that research on nlp systems conceptualize bias differently and that the techniques are not well tied with the relevant literature outside of nlp. this creates a gap between nlp research and the study of ethics in philosophy which leaves a rich body of knowledge untapped. our work bridges this gap by illustrating how a philosophical theory of ethics can be applied to nlp research. ethics (or ethical theory), is a theoretical and applied branch of philosophy which studies what is good and right, especially as it pertains to how humans ought to behave in the most general sense @xcite . as nlp research qualifies as a human activity, it is within the purview of ethics. in particular, we are using a prescriptive, rather than descriptive, theory of ethics; prescriptive theories define and recommend ethical behavior whereas descriptive theories merely report how people generally conceive of ethical behavior. we select two ethical principles from the deontological tradition of ethics and focus on how these principles are relevant to research in nlp. namely we look at the generalization principle and respect for autonomy through informed consent @xcite . we select deonotology because it is reasonable, provides clear ethical rules and comports with the legal idea of the rule of law in the sense that these ethical rules bind all persons equally, rather than shifting standards to effect a certain outcome. we find that there are two main ways in which ethical guidelines can be applied in nlp (or to any other area of technology): we primarily address (1) and briefly touch on (2) by presenting four case studies relevant to nlp. in each case study we use an ethical principle to identify an area of research that could potentially conflict with it, and suggest nlp directions to mitigate it. although we have selected two principles from a deontological perspective, we are not intimating that these principles can address all ethical issues nor that deontological ethics is the only ethical framework in which our rules and case studies could function ( §6). instead, we present the following as a starting point for nlp researchers less familiar but interested in applicable ethical theory. our primary contributions are: ## deontological ethics there is a variety of specific deontological theories which range from having one central, abstract principle @xcite to having a handful of concrete principles @xcite . rather than comprehensively addressing one theory, we select two rules, one abstract and one concrete, which can fit within a variety of deontological theories. the generalization principle is an abstract, broad-reaching rule which comes from traditional kantian ethics. the respect for autonomy is concrete and commonly seen in politics and bioethics. ## applying ethics to nlp systems we apply the generalization principle in §4.1 and §4.2 and respect for autonomy in §4.3 and §4.4. ## ethical decision making with nlp so far we have discussed how nlp systems can be evaluated using ethical frameworks and how decisions made by such systems can be assisted by these theories. nlp can also aid in making decisions in accordance with the deontological framework. recall that the generalization principle judges the ethical standing of pairs of actions and reasons; these pairs could be extracted with various nlp techniques from textual content. in the case of flagging objectionable content ( §4.2), extracting the deeper intents and implications corresponds to the reasons for the action of flagging the content. another example is building an automatic institutional dialog act annotator for traffic police conversations @xcite . these dialog acts contain the rationales of the two agents in the conversation: the police officer and the civilian stopped for breaking traffic rules. the decision made by the police officer (the action) can then be judged to be in accordance (or not) with a human-selected set of ethically acceptable action and rationale pairs. similarly, for court hearing transcripts, the rationales of the arguments can be extracted and the verdict of the judge can be checked using them @xcite . nlp tools such as commonsense knowledge graph generation @xcite @xcite , semantic role labeling @xcite , open domain information extraction @xcite etc. can be used to extract rationales, entities from text and also find relations between them to better understand the underlying intent of the text. ## discussion we provide a broad discussion on the limitations of the principles chosen in this work and the issue of meta-ethics. moreover, we emphasize that ethical research is not merely a checklist to be satisfied by abiding to the principles mentioned here. it requires our persistent attention and open-minded engagement with the problem. one limitation of this work is in the principles that we choose. 6 for example, the interaction of machine learning and privacy is of huge ethical importance. while the respect for autonomy may address this issue in part, it would be more productive to utilize a deontological principle to the effect of the right to privacy with which such matters can be judged. another instance is that in this work, we have not discussed the principle of interactional fairness @xcite which refers to the quality of interpersonal treatment including respect, dignity, and politeness. with the increasing amount of interaction between humans and machine, the natural language generation systems can be evaluated with this principle. systems which show respect and dignity to users as well as generate polite language can enhance the degree of interactional justice, which can in turn enhance utility (e.g., trust, satisfaction). additionally, there are broader limitations in using deontology as our ethical framework. in scenarios where there are no a priori duties or rights, taking a consequentialist approach and optimizing the effects of ethical guidelines could be more felicitous. for example, the specific rights and duties of autonomous ai systems are not immediately clear. thus, determining ethical recommendations based on what leads to the most responsible use of the technology would be clearer than selecting appropriate rights and duties directly. furthermore, rule-based formulations of consequentialism make ethical judgments based on rules, where the rules are selected based on the consequences. such theories combine some of the benefits of both deontology and consequentialism. the above difficulties are part of the larger issue of metaethics, that is, the discussion and debate on how to choose among different ethical theories. within deontology, there is no one standard set of rules. and even within the generalization principle, there is considerable leeway to what "conceivable world" or "logically consistent" mean and how they could be applied to decision making. while presenting a universally accepted ethical theory is likely impossible, metaethical considerations can still be relevant, especially in light of the application of ethical theories. as the field of nlp gets more accustomed with theories of ethics, it will be fruitful to compare the strengths and weaknesses of different ethical theories within the context of nlp and machine learning. ## conclusion two principles of deontological ethics-namely the generalization principle and respect for autonomy via informed consent-can be used to decide if an action is ethical. despite the limitations of these principles, they can provide useful insights into making nlp systems more ethical. through the four case studies discussed in this paper, we demonstrate how these principles can be used to evaluate the decisions made by nlp systems and to identify the missing aspects. for each of the case studies, we also present potential directions for nlp research to move forward and make the system more ethical. we further provide a summary on how nlp tools can be used to extract reasons and rationales from textual data which can potentially aid deontological decision making. note that we do not advocate deontological ethics as the only framework to consider. on the contrary, we present this work as the first of its kind to illustrate why and how ethical frameworks should be used to evaluate nlp systems. with this work, we hope the readers start thinking in two directions: (1) using different ethical frameworks and applying the principles to nlp systems (like the case studies in §4), and (2) exploring the directions mentioned in the case studies of this paper to improve current nlp systems.