Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2023)

Abstract: Recent advances in the capacity of large language models to generate human-like text have resulted in their increased adoption in user-facing settings. In parallel, these improvements have prompted a heated discourse around the risks of societal harms they introduce, whether inadvertent or malicious. Several studies have explored these harms and called for their mitigation via development of safer, fairer models. Going beyond enumerating the risks of harms, this work provides a survey of practical methods for addressing potential threats and societal harms from language generation models. We draw on several prior works’ taxonomies of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators. Bridging diverse strands of research, this survey aims to serve as a practical guide for both LM researchers and practitioners, with explanations of different strategies’ motivations, their limitations, and open problems for future research.

## introduction the new wave of large language models (lms; @xcite @xcite capable of generating text with human-like fluency, coherence, and realism @xcite has caused a paradigm shift in our society. 1 with applications like openai's chatgpt, microsoft's bing, and google's bard, bringing such lms directly to users, we are beginning to see the impact in fields like education @xcite , healthcare @xcite , law (chatgpt and perlman, 2022), science (stokel-walker, 2023), and more. since language is inherently a tool of figure 1 : overview of intervention strategies. a typical ml/nlp model development process involves data collection/curation, model training and design, inference, and finally application deployment. for each phase of this development cycle, different techniques can be adopted to mitigate harms. our survey presents a taxonomy of intervention strategies organized around the different phases where they can be applied. indeed, lms have been shown to introduce vulnerabilities and threats, both inadvertent and malicious, to individual users, social groups, and content integrity. without social context and content control, deployed language generators have quickly derailed to racist, homophobic, hateful comments @xcite @xcite , compromised user privacy @xcite , spread disinformation @xcite , and even encouraged suicide @xcite . prior works have outlined these risks @xcite @xcite , proposed taxonomies @xcite , discussed their points of origin, and advocated for future research on ethical development of lms @xcite . however, there is little work that summarizes actionable approaches and technical solutions to preventing or mitigating these potential harms. in this survey, we present a comprehensive, unified taxonomy of relevant mitigation strategies proposed in prior literature, specifically focusing on language generation models. we organize these strategies based on where they fit in different stages of lm development: in data collection, modeling, decoding, and deployment. within each of these categories, our taxonomy brings together prior works that have been treated as disjoint areas targeting different types of harms (toxic/biased language and misinformation). in addition, we identify their gaps and highlight directions for future research. these include incorporating sociocultural context to produce sociallysensitive interventions, detecting and handling generations with different intents (inadvertent vs. malicious), and going beyond an english, western/uscentric view to account for the challenges of ethics in multilingual language generation. ## background throughout this paper, we use the term language models (lms) to refer to their classic definition as generative models, which predict the next token given the preceding generated context. this paradigm also subsumes conditional lms that depend on additional inputs via an encoder. we provide more details in appendix a. ## taxonomy of intervention strategies the development pipeline of a typical machine learning model involves several critical decisions where risks of harms can arise. stakeholders have access to different pipeline components and therefore may employ different intervention strategies. for example, while a researcher involved in data curation can intervene before training, an application developer with limited access to a black-box model might only be able to intervene at inference. we present a taxonomy of intervention strategies organized by the stages of a model development lifecycle (fig. 1 ), aiming to showcase the tools that can be employed at different stages. we step backward through the pipeline, beginning with applicationlevel interventions employed post-deployment and peeling back the layers through output-level interventions, model interventions, and finally ending at data-level interventions (summarized in tab. 1). ## discussion and open challenges though the interventions strategies we discuss achieve some success, many risks of lms are still not well understood. below we discuss open problems and avenues for future work to encourage the development of safer lms. where should one intervene? different stakeholders are involved in different model development phases with varying access to resources. as a result, intervention strategies are different depending on the stakeholder. a significant chunk of the responsibility to develop safer lms falls on researchers and organizations with access to substantial resources who can implement data or modeling interventions. in contrast, practitioners building applications on top of lms may have access to neither the training data nor the computational resources required to design and train safe lms. in such cases, flagging and decoding approaches are more practical. in practice, a combination of multiple interventions may be required to both cover a wide array of risks and improve robustness. evolving risks in the chatgpt era: lms are seeing tremendous, rapid growth; larger models are being released every few months @xcite @xcite and deployed in user-facing applications. many recent lms like openai's chatgpt have garnered attention beyond the research community, impacting a range of fields and crossing geographical and language barriers to reach users all over the world @xcite @xcite . in such a fast-moving ecosystem, it is ever more essential to proactively study and mitigate lms' potential harms. risk mitigation research tends to lag behind model development and is often considered as an afterthought. though behaviors may emerge unpredictably @xcite , as we outline in this survey, intervention strategies can and should be applied at different stages of model development to reduce the potential for these influential lms to cause harm. risks exist in lms in all languages: most research on large lms, their uses, and their risks is western-centric and primarily conducted on the english language. however, while a few studies have been conducted on detecting harmful text in non-english datasets @xcite @xcite @xcite , research on mitigation in non-english settings is lagging @xcite . further, the definitions of risks themselves change with different context and across cultures. hence, there is a dire need to develop cross-cultural, crosslingual analyses as well as mitigation tools. ## conclusion we present a survey of practical methods and techniques for addressing the societal harms and safety risks of language generation models. our structured taxonomy covers a wide variety of interventions at different stages of the model development pipeline to mitigate harms. this work bridges multiple strands of research and presents an actionable overview on methods for preventing harms from language generation models.