Mitigating Societal Harms in Large Language Models (2023)

Abstract: Numerous recent studies have highlighted societal harms that can be caused by language technologies deployed in the wild. While several surveys, tutorials, and workshops have discussed the risks of harms in specific contexts – e.g., detecting and mitigating gender bias in NLP models – no prior work has developed a unified typology of technical approaches for mitigating harms of language generation models. Our tutorial is based on a survey we recently wrote that proposes such a typology. We will provide an overview of potential social issues in language generation, including toxicity, social biases, misinformation, factual inconsistency, and privacy violations. Our primary focus will be on how to systematically identify risks, and how eliminate them at various stages of model development, from data collection, to model development, to inference/language generation. Through this tutorial, we aim to equip NLP researchers and engineers with a suite of practical tools for mitigating safety risks from pretrained language generation models.

## motivation with the widespread success and increasing adoption on natural language processing (nlp) technologies in user-facing products including machine translation @xcite , dialogue systems @xcite and recommendation systems @xcite the nlp community is becoming increasingly aware that we have a responsibility to evaluate the effects of our research and mitigate harmful outcomes @xcite . indeed, models have been shown to introduce vulnerabilities and threats, both inadvertent and malicious, to individual users, social groups, and content integrity. without social context and content control, deployed language generators have quickly derailed to racist, homophobic, hateful comments @xcite @xcite , compromised user privacy @xcite , spread disinformation @xcite , and even encouraged suicide @xcite . prior works have outlined these risks @xcite @xcite , proposed taxonomies @xcite , discussed their points of origin, and advocated for research on ethical development of lms @xcite . however, there is little work that summarizes actionable approaches and technical solutions to preventing or mitigating these harms. this is the purpose of our tutorial, which is based on a survey we have recently conducted @xcite . in this tutorial, we aim to provide a comprehensive, unified taxonomy of relevant mitigation strategies proposed in prior literature, specifically focusing on language generation models. ## tutorial content and relevance what are language models? a brief background: to build a common ground for discussing the risk mitigation strategies, this tutorial will begin with a brief overview of recent trends in language modeling and pretraining. we will cover both causal @xcite and non-causal language models @xcite highlighting their differences and their impact on nlp research. we will briefly discuss how pretrained models can be adapted to different tasks covering model finetuning (both complete and adapter based) as well as prompt-based formulation to solve nlp tasks. we will also focus on their scale both in terms of model parameters as well as training data size. how can language models cause societal harm? after presenting the background on language models, we will then give a formal definition of harms based on taxonomy defined in prior work @xcite and focus on representational harms in this tutorial. highlighting the impact of heedlessly using web data which is usually populationimbalanced @xcite and contains biased language against towards specific populations, we will discuss how language models tend to reinforce and amplify bias against sub-populations based on different personal and social attributes such as gender @xcite , race @xcite , region @xcite , demographics @xcite , age @xcite among others. we will also discuss, that by not being grounded in real world knowledge, they pickup on spurious statistical correlations in data and generate (in other words, hallucinate) factually incorrect content which can potentially be used to spread misinformation @xcite . major content of this section is borrowed from the course on ethics in nlp developed at carnegie mellon university and the university the washington by organizer yulia tsvetkov. can we reduce or mitigate such harms? finally, in this part, we will focus on work on mitigating harmful effects of language generation systems. while still a nascent field of research, several solutions in this space have been proposed which we categorize into four categories, visualized in fig. 1 . we organize and discuss in detail interven-tion strategies based on where they fit in different stages of lm development: in data collection, modeling, post-factum decoding, and application. within each of these categories, our taxonomy brings together prior works that have been treated as disjoint areas targeting different types of harms (toxic/biased language and misinformation). since lms learn and amplify biases present in the training data, we will first discuss data level interventions which focus on either (1) filtering the pretraining corpora to create more balanced datasets @xcite , or (2) finetuning trained lms on sanitized data @xcite . second, we will review model level interventions where we consider approaches which modify either the architecture or training objectives to induce or remove desired biases @xcite . third, we will present methods to modify model outputs post generation using decoding and editing methods to demote or remove harmful content @xcite @xcite . these techniques are especially useful for cases where it is impossible to modify data or models or even decoding strategies such as in case of gpt3 @xcite which are only available through an api. finally, we will end with application level interventions where we show how methods to flag and redact harmful content allow applications to shield such content from reaching users @xcite . throughout the tutorial, we will highlight both detection and mitigation approaches, as well as their specific limitations and shortcomings. by the end of the tutorial, participants will be better informed where to focus future research efforts. due to the vast range of societal harms and their mitigation strategies, we do not plan an exhaustive treatment of this material. one central goal is to raise awareness for participants of the relevant issues, so that when they return to their research they will be more able to notice ways in which their research based on large language models might impact different variety of users. to achieve this goal, we will aim for a "t-shape" in terms of breadth and depth: to briefly mention a number of core questions and then to drill down into a few particular case studies to see how these issues play out in real research settings. ## tutorial structure we propose a cutting-edge tutorial on an emerging area that has not been previously covered in acl/emnlp/naacl/coling tutorials. this would be a discussion-style tutorial where the organizers will present material with structured time throughout for questions, and discussion amongst attendees. the duration of the tutorial will be 3 hours with 5 min breaks at the end of each hour. the following would be the outline of the talk: 1. brief introduction to language models (10 mins) -we will provide a quick background on current state of nlp research with introduction to language models and their capabilities. 2. possible harms of language technologies (15 mins) -we will briefly cover examples of ethical concerns, societal harms and biases present in current nlp tools. the tutorial will be a series of presentations with a set of references to related research papers and external demos. the presentation will cover a wide array of research on the topics from across the field. we will share the slides with the participants in advance. we will additionally share an online repository of relevant research material and online links to available code and demos to help participants navigate and use relevant research for their work. no copyright issues are expected as we will use open-source material. vidhisha balachandran (she/her) is a fourth-year ph.d. student at the language technologies institute, school of computer science at cmu. her current research focuses on building interpretable and reliable nlp models with a focus on summarization, factuality, and kb-based reasoning. lucille njoo (she/her) is a second-year phd student at the paul g. allen school of computer science and engineering at the university of washington. she works in the intersection of nlp, ethics, and computational social science, working on identifying societal harms in nlp models. ## logistics previous editions this is the second iteration of the tutorial. the first edition of the tutorial was presented at the web conference 2022. while the previous iteration was focused to a general cs audience with less nlp background, this iteration will be modified to be aligned more for nlp-focused audience. this would entail including deeper technical specification of the interventions, including data, models and objectives. our tutorial is related and complementary to prior acl tutorials related to bias and fairness in nlp (socially responsible nlp at naacl 2018, bias and fairness in nlp at emnlp 2019, integrating ethics into the nlp curriculum at acl 2020). complementary to the content of the above tutorials which highlight social harms in nlp and discuss their detection, primarily focusing on representation learning and text classification, our tutorial will focus on practical methods to identify and mitigate harms in large language models and language generation. venue we prefer emnlp or acl, but any venue would work for us. technical requirements we will not require additional equipment other than presentation material: an lcd projector, a computer with powerpoint and acrobat reader, and internet connection. public release we will publicly release all tutorial materials, including prerecorded lectures as backup for the tutorial which will be uploaded prior to the tutorial. these will be hosted on an openaccess platform and linked from our university websites. ## ethics statement although the aim of this tutorial is to improve the safety and inclusivity of nlp technologies and equip practitioners with tools to do so, we are well aware that as a not perfectly-diverse group of researchers we might incorporate our own biases into tutorial stricture and its technical focus. we will acknowledge this limitation in our tutorial, as well as the fact that the field of computational ethics is developing rapidly, and thus the content of our tutorial is inherently incomplete.