SlowGuess/NAACL_2021_OCR huggingface dataset:
The way forward: An example of objectionable content is microaggression (Fig. 1). According to Merriam-Webster, microaggression is defined as a "comment or action that subtly and often unconsciously expresses a prejudiced attitude toward a member of a marginalized group (e.g. racial minority)." Microaggressions are linguistically subtle which makes them difficult to analyze and quantify automatically. Understanding and explaining why an arguably innocuous statement is potentially prejudiced requires reasoning about conversational and commonsense implications with respect to the underlying intent, offensiveness, and power differentials between different social groups. Breitfeller et al. (2019) provide a new typology to better understand the nature of microaggressions and their impact on different social groups. Fig. 1b presents such a comment and how we would like the NLP systems to annotate such content. Sap et al. (2020) perform the task of generating the consequences and implications of comments which is a step towards judging content based on its meaning and not simply which words it happens to use. Although such an aim does not automatically solve the prob
lem, attempting to uncover the deeper meaning does not result in an inconsistency or violation of the generalization principle.
4.3 Machine Translation Systems
Machine Translation (MT) systems have reduced language barriers in this era of globalization. Neural machine translation systems especially have made huge progress and are being deployed by large companies to interact with humans. But facilitating human-to-human interaction requires more than just simple text-to-text translation, it requires the system to interpret the meaning of the language. This requires a greater sensitivity to style, intent, and context on the part of MT systems.
When an MT system acts as an interpreter for a user, it is essentially speaking for the user when conveying the translated message. Speaking for one's self is within one's sphere of autonomy, but by using the MT system the user has implicitly consented to it representing the user. That being said, the operating assumption for most users is that the MT system will simply translate the source language into the target language without changing the meaning. Yet on occasion, differences or ambiguities between languages require either contextual knowledge or further clarification on what is being said. If the MT system encounters such ambiguities, the user must be informed of such occurrences so that she can consent to the message which the system ultimately conveys. Moreover, the user must also be informed of the failure cases in the MT system rather than it producing an entirely incorrect translation.
For example, when translating from English to Japanese, there is a mismatch in the granularity of titles or honorifics used to address people. In English, "Ms." and "Mr." is an appropriate way to address a schoolteacher who does not hold a doctorate. On the other hand, in Japanese it would be disrespectful to use the more common "-san" honorific (the rough equivalent of "Ms." or "Mr.") in place of "-sensei" which refers specifically to teachers or mentors and shows them a special level of respect. If the MT system cannot reasonably infer how to resolve the ambiguity in such situations, the English speaker should be informed about it. The English speaker must be notified that such an ambiguity needs to be resolved because there is a risk of offending the Japanese speaker otherwise.
In general, there is a trade-off in translation be
tween literality and fluency in certain situations like the translation of idioms. Idioms are especially problematic when considering autonomy because there are multiple strategies to translating them which are not only difficult in and of themselves to execute, but deciding which one to use requires the interpreter (i.e., MT system) to understand the intent of the user. Baker (1992, Ch. 3) identifies five different methods for translating idioms:
1. Using an idiom of similar meaning and form; directly translating the idiom achieves the same effect
2. Using an idiom of similar meaning but dissimilar form; swapping out an equivalent idiom with a different literal meaning
3. Translation by paraphrase; simply explaining the idiom plainly
4. Translation by omission
5. Translation by compensation; for example, omitting idioms in certain locations and adding them in elsewhere to maintain the same overall tone
For example, in casual conversation, an MT system may prefer strategies 1, 2, and 5 to maintain a friendly tone, but in a high-stake business negotiation, it would be more prudent to play it safe with strategy 3. An MT system must be sensitive to the user's intent since choosing an inappropriate translation strategy could violate her autonomy.
While para-linguistic conduct may fill the gaps for in person interaction, if the interaction is happening only via the textual modality, then there is minimal room for such conduct. The users in this case may not be aware of the flaws of the MT system representing the.. A recent study (Heinisch and Lušicky, 2019) shows that  $45\%$  of the participants reported that they expect MT output, in professional and private contexts, to be useable immediately without any further editing. However, post-study, this expectation was not fulfilled. The work further shows that the expectation of the type of errors is also different from the errors in the outputs of the MT system. For example: only  $6\%$  of the participants expect that the output would be useless, but after reading the output,  $28\%$  thought that the output was useless. The participants in this study had different levels of experience with MT systems (frequent vs. rare users) and used MT systems for different functions (private, professional).
The way forward: Mima et al. (1997) drive the early discussion on using information such as context, social role, domain and situation in MT systems. DiMarco and Hirst (1990) advocate for style and intent in translation systems. A study by Hovy et al. (2020) finds that commercial translation systems make users sound older and more male than the original demographics of the users. Recent work (Niu and Carpuat, 2020; Sennrich et al., 2016) has given specific focus to controlling formality and politeness in translation systems. There is also work directed towards personalizing MT systems (Rabinovich et al., 2017; Michel and Neubig, 2018; Mirkin et al., 2015; Mirkin and Meunier, 2015) while preserving author attributes as well as controlling structural information like voice (Yamagishi et al., 2016). This is a step in the right direction, but we argue that to respect autonomy, translation systems should also obtain explicit informed consent from the user when necessary.
Further research is required in the direction of informing the users about the failure cases of the MT system. For example, in case of ambiguity, textual interfaces can provide multiple suggestions to the addressee along with the implications of using each variant. The user can select the option which best fits their goal. In speech interfaces, the MT system can ask a follow up question to the addressee of the system in case of ambiguity or it can add cautionary phrases to the addressee informing them about the ambiguity. Alternatively, if the system thinks that the input sentence is ambiguous and cannot be translated with reasonable confidence then it can say "I am unable to translate the sentence in its current form. Can you please rephrase it?". An example scenario where such clarification might be needed is: while translating from English to Hindi if the sentence refers to one's "aunt," the MT system should ask a follow up question about maternal vs paternal aunt since they have two different words in Hindi language.
4.4 Dialogue Systems
We can find a nuanced application of the autonomy principle in the way that dialogue systems, especially smart toys or virtual assistants like Alexa and Google Home, interact with children.
One expression of a parent's autonomy<sup>5</sup> is generally in deciding whom their child may interact
5This is technically heteronomy, but this examples comports with the spirit of respect for autonomy.
with. For example a parent would permit interaction with a teacher but not a random stranger. In the case of a parent purchasing and using a virtual assistant at home, they are implicitly consenting to their children interacting with the assistant, and the issue arises from the fact that they may not be informed as to what this interaction entails. To an adult, a virtual assistant or dialogue-capable toy may seem like just another computer, but a 7-year-old child might view it as "more capable of feelings and giving answers"—a step in the direction of assigning personhood (Druga et al., 2017). Furthermore, while humans have had thousands of years to learn about human-human interaction, we have only had a half-century to learn about the effects of human-machine (and thus, child-machine) interaction (Reeves and Nass, 1996).
We suggest two key areas which are important for dialogue system researchers: (1) they must answer the question of what unique social role do dialogue systems fulfill—that is, in what respects can they be regarded as human-like vs. machine-like, and (2) the dialogue systems must have some way of modeling the social dynamics and cues of the interlocutor to fulfill the social role properly.
The way forward: There is a fair amount of research on the social aspects of human-computer dialogue both in general and specifically with regards to children (Druga et al., 2017; Shen, 2015; Kahn Jr et al., 2013). Although it is difficult to gain a complete understanding of how dialogue systems affect the development of children, the most salient facts (e.g., children regarding virtual assistants as person-like) should be communicated to parents explicitly as part of parental controls. We advocate for a "kids mode" to be included with these virtual AI assistants which would provide the feature of parental control in accordance with respect for autonomy. This mode would be aware that it is talking to children and respond accordingly. NLP can also help in selecting content and style appropriate for children in these AI agents. Additionally, parents can be provided with fine-grained control over the topics, sources and language that would be generated by the agent. For example, the parent can select for a polite language and topics related to science to support their child's development efforts. Much research has focused on controlling topics (Kim et al., 2015; Jokinen et al., 1998), style (Niu and Bansal, 2018), content (Zhou et al., 2018; Zhao et al., 2020; Dinan et al., 2019)
and persona (Zhang et al., 2018) of dialogue agents which can be used for this purpose.
5 Ethical Decision Making with NLP
So far we have discussed how NLP systems can be evaluated using ethical frameworks and how decisions made by such systems can be assisted by these theories. NLP can also aid in making decisions in accordance with the deontological framework. Recall that the generalization principle judges the ethical standing of pairs of actions and reasons; these pairs could be extracted with various NLP techniques from textual content. In the case of flagging objectionable content (§4.2), extracting the deeper intents and implications corresponds to the reasons for the action of flagging the content. Another example is building an automatic institutional dialog act annotator for traffic police conversations (Prabhakaran et al., 2018). These dialog acts contain the rationales of the two agents in the conversation: the police officer and the civilian stopped for breaking traffic rules. The decision made by the police officer (the action) can then be judged to be in accordance (or not) with a human-selected set of ethically acceptable action and rationale pairs. Similarly, for court hearing transcripts, the rationales of the arguments can be extracted and the verdict of the judge can be checked using them (Branting et al., 2020; Aletras et al., 2019). NLP tools such as commonsense knowledge graph generation (Bosselut et al., 2019; Saito et al., 2018; Malaviya et al., 2019), semantic role labeling (Gildea and Jurafsky, 2000), open domain information extraction (Angeli and Manning, 2013) etc. can be used to extract rationales, entities from text and also find relations between them to better understand the underlying intent of the text.
We provide a broad discussion on the limitations of the principles chosen in this work and the issue of meta-ethics. Moreover, we emphasize that ethical research is not merely a checklist to be satisfied by abiding to the principles mentioned here. It requires our persistent attention and open-minded engagement with the problem.
One limitation of this work is in the principles that we choose. For example, the interaction of machine learning and privacy is of huge ethical
6 Kant would argue that the generalization principle can account for all ethical decisions, but we make no such claim.
importance. While the respect for autonomy may address this issue in part, it would be more productive to utilize a deontological principle to the effect of the right to privacy with which such matters can be judged.
Another instance is that in this work, we have not discussed the principle of interactional fairness (Bies, 2015, 2001) which refers to the quality of interpersonal treatment including respect, dignity, and politeness. With the increasing amount of interaction between humans and machine, the natural language generation systems can be evaluated with this principle. Systems which show respect and dignity to users as well as generate polite language can enhance the degree of interactional justice, which can in turn enhance utility (e.g., trust, satisfaction).
Additionally, there are broader limitations in using deontology as our ethical framework. In scenarios where there are no a priori duties or rights, taking a consequentialist approach and optimizing the effects of ethical guidelines could be more felicitous. For example, the specific rights and duties of autonomous AI systems are not immediately clear. Thus, determining ethical recommendations based on what leads to the most responsible use of the technology would be clearer than selecting appropriate rights and duties directly. Furthermore, rule-based formulations of consequentialism make ethical judgments based on rules, where the rules are selected based on the consequences. Such theories combine some of the benefits of both deontology and consequentialism.
The above difficulties are part of the larger issue of metaethics, that is, the discussion and debate on how to choose among different ethical theories. Within deontology, there is no one standard set of rules. And even within the generalization principle, there is considerable leeway to what "conceivable world" or "logicall consistent" mean and how they could be applied to decision making. While presenting a universally accepted ethical theory is likely impossible, metaethical considerations can still be relevant, especially in light of the application of ethical theories. As the field of NLP gets more accustomed with theories of ethics, it will be fruitful to compare the strengths and weaknesses of different ethical theories within the context of NLP and machine learning.
Two principles of deontological ethics—namely the generalization principle and respect for autonomy via informed consent—can be used to decide if an action is ethical. Despite the limitations of these principles, they can provide useful insights into making NLP systems more ethical. Through the four case studies discussed in this paper, we demonstrate how these principles can be used to evaluate the decisions made by NLP systems and to identify the missing aspects. For each of the case studies, we also present potential directions for NLP research to move forward and make the system more ethical.
We further provide a summary on how NLP tools can be used to extract reasons and rationales from textual data which can potentially aid deontological decision making. Note that we do not advocate deontological ethics as the only framework to consider. On the contrary, we present this work as the first of its kind to illustrate why and how ethical frameworks should be used to evaluate NLP systems. With this work, we hope the readers start thinking in two directions: (1) using different ethical frameworks and applying the principles to NLP systems (like the case studies in §4), and (2) exploring the directions mentioned in the case studies of this paper to improve current NLP systems.
We are grateful to the anonymous reviewers for their constructive feedback, and special thanks to Dirk Hovy for valuable discussions on this work. This work was supported in part by ONR Grant N000141812861 and NSF IIS1763562. This material is based on research sponsored in part by the Air Force Research Laboratory under agreement number FA8750-19-2-0200 (author BB). The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory or the U.S. Government.
Nikolaos Aletras, Elliott Ash, Leslie Barrett, Daniel Chen, Adam Meyers, Daniel Preotiac-Pietro, David
Rosenberg, and Amanda Stent, editors. 2019. Proceedings of the Natural Legal Language Processing Workshop 2019. Association for Computational Linguistics, Minneapolis, Minnesota.
Larry Alexander and Michael Moore. 2016. Deontological Ethics. In Edward N. Zalta, editor, *The Stanford Encyclopedia of Philosophy*, winter 2016 edition. Metaphysics Research Lab, Stanford University.
Haifa Alharthi and Diana Inkpen. 2019. Study of linguistic features incorporated in a literary book recommender system. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pages 1027-1034.
Gabor Angeli and Christopher Manning. 2013. Philosophers are mortal: Inferring the truth of unseen facts. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 133-142, Sofia, Bulgaria. Association for Computational Linguistics.
Aristotle. 350 B.C.E. Nicomachean Ethics.
Mona Baker. 1992. In Other Words: A Coursebook on Translation. Routledge, United Kingdom.
Loic Barrault, Ondrej Bojar, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Muller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019. Findings of the 2019 conference on machine translation (WMT19). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1-61, Florence, Italy. Association for Computational Linguistics.
Emily M. Bender, Dirk Hovy, and Alexandra Schofield. 2020. Integrating ethics into the NLP curriculum. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 6-9, Online. Association for Computational Linguistics.
Jeremy Bentham. 1843. The Rationale of Reward.
Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 94-102, Valencia, Spain. Association for Computational Linguistics.
Robert J Bies. 2001. Interactional (in) justice: The sacred and the profane. Advances in organizational justice, 89118.
Robert J Bies. 2015. Interactional justice: Looking backward, looking forward. The Oxford Handbook of Justice in the Workplace, page 89.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454-5476, Online. Association for Computational Linguistics.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349-4357.
Antoine Bosselut and Yejin Choi. 2019. Dynamic knowledge graph construction for zero-shot commonsense question answering. arXiv preprint arXiv:1911.03876.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762-4779, Florence, Italy. Association for Computational Linguistics.
Abdelghani Bouziane, D. Bouchiha, Noureddine Doumi, and M. Malki. 2015. Question answering systems: Survey and trends. Procedia Computer Science, 73:366-375.
L Karl Branting, Craig Pfeifer, Bradford Brown, Lisa Ferro, John Aberdeen, Brandy Weiss, Mark Pfaff, and Bill Liao. 2020. Scalable and explainable legal prediction. Artificial Intelligence and Law, pages 1-26.
Luke Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019. Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1664-1674, Hong Kong, China. Association for Computational Linguistics.
Dallas Card and Noah A. Smith. 2020. On consequentialism and fairness. Frontiers in Artificial Intelligence, 3.
Amanda Cercas Curry and Verena Rieser. 2018. #MeToo Alexa: How conversational systems respond to sexual harassment. In Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing, pages 7-14, New Orleans, Louisiana, USA. Association for Computational Linguistics.
Colin Cherry, Greg Durrett, George Foster, Reza Haffari, Shahram Khadivi, Nanyun Peng, Xiang Ren,