Societal Biases in Language Generation: Progress and Challenges (2021)

Abstract: Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner. While techniques can effectively generate fluent text, they can also produce undesirable societal biases that can have a disproportionately negative impact on marginalized populations. Language generation presents unique challenges for biases in terms of direct user interaction and the structure of decoding techniques. To better understand these challenges, we present a survey on societal biases in language generation, focusing on how data and techniques contribute to biases and progress towards reducing biases. Motivated by a lack of studies on biases from decoding techniques, we also conduct experiments to quantify the effects of these techniques. By further discussing general trends and open challenges, we call to attention promising directions for research and the importance of fairness and inclusivity considerations for language generation applications.

## introduction natural language generation (nlg) is a suite of techniques that enables the generation of humanreadable language for different goals. these techniques are the core components of applications such as virtual assistants, chat bots, automatic translators, summarizers, and creative language composers. recent advances in techniques for language generation (e.g., gpt @xcite , gpt-2 @xcite , @xcite , transformerxl @xcite , xlnet @xcite ) powered by transformers @xcite and an increasing repository of available data have created more capable applications. this has, in turn, channeled more interest and effort into developing nlg techniques. we emphasize the importance of better understanding how societal biases manifest in nlg techniques, because nlg applications directly interact with many different users to generate novel content in various domains (e.g., chat bots for health, education, and customer support). however, when techniques are less effective or detrimental for marginalized populations, these techniques can inadvertently become gatekeepers of those populations for generation and associated language technologies. for example, an educational chat bot that produces more negative responses for topics about a specific ethnicity will discourage users of that ethnicity from interacting with the chat bot. while it is generally important to study the societal impact of nlp and ai techniques, we argue that the direct user impact of nlg techniques makes it especially important to carefully quantify the impact. motivated by the importance of fairness in language generation, we present the first comprehensive survey on societal biases in language generation. by enumerating how nlg techniques contribute to biases and examining progress towards bias analysis and mitigation, we contextualize the discussion of broader trends and challenges. specifically, we focus on techniques for nlg tasks, i.e., tasks that generate a sequence of text. 1 finding a lack of studies on biases from decoding techniques, we additionally present an experimental study to quantify the effects of various decoding techniques. before we delve into the details of biases in language generation, we first position our survey in the context of other relevant surveys and position papers. @xcite present a focused survey on mitigating gender biases and @xcite categorize sources of biases-both largely focus on natural language understanding (nlu) tasks, while we examine biases in nlg tasks. @xcite urge for more explicitly tying "biases" in nlp to societal normative definitions of biases and social hierarchies; with their recommendations in mind, we discuss the negative impacts of biases in nlg techniques. our contributions are a comprehensive survey on societal biases in language generation and an experimental study on biases from decoding techniques. to start, we describe classes of nlg tasks (sec. 2) and subsequently examine examples of biases and harms in nlg (sec. 3). we then discuss nlg techniques that facilitate biases, including a study of decoding techniques (sec. 4). sec. 5 highlights progress and challenges, and sec. 6 presents open problems and proposals. we hope this survey brings more visibility to the importance of carefully considering different components of nlg pipelines for potential biases and mitigation methods. ## language generation tasks to begin, we categorize generation tasks and introduce existing bias studies relevant to each task. nlg tasks broadly fall into two categories: those that generate text continuations conditioned on some prompt and those that transform text from one form to another. ## contributors to nlg biases in a pipeline from data collection to evaluation for an nlg task, each component could propagate biases. 14 we emphasize the ways in which data, model architecture, decoding, evaluation, and deployment uniquely exacerbate biases in generation tasks. additionally, we present an empirical study to show how measured biases in generated text can vary based on decoding technique. ## progress, trends, and challenges following the discussion of contributors to biases, we survey trends and challenges for reducing biases in nlg. ## open problems and proposals as a fairly nascent area of exploration, the study of biases in language generation still poses many challenges. throughout this paper, we discuss challenges associated with different components in a generation pipeline. with a heightened awareness of the relevant body of work, we conclude with recommendations for open problems. bias-aware data curation many works have highlighted the harms and problems when collecting training datasets with limited awareness for potential harms. since effective models for nlg tasks are correlated with increasing training data sizes, biases in data collection (e.g., englishcentric, drawn from popular western media) remain a major contributor of biases that manifest in generation. additionally, datasets used to study biases in generation can also be limited (e.g., only for binary gender classes). for more bias-aware data curation, we suggest diversifying datasets to include more viewpoints from various groups. understanding trade-offs different methods for analysis, mitigation, and evaluation have unique trade-offs. existing works have been relatively small-scale and limited to a small number of biases for specific tasks. some useful questions to consider when developing methods to study generation biases are whether we can generalize methods to a diverse set of biases and a wide range of contexts. it is also important to consider formulating metrics that would jointly mitigate biases and preserve other desired text qualities (e.g., diversity, fluency). interactive and continuous learning the difficulties of measuring and mitigating biases in generation can be reduced with a general framework for interactive and continuous learning. over time, such a system could learn from diverse opinions of what constitutes "fair" versus "unfair" generations across tasks. a unified framework would centralize and highlight the importance of studying biases in generation, as well as fuel the development of a more comprehensive set of evaluations that may be useful for large-scale studies of impact. focusing on negative impacts section 3 discusses how there are very few existing works on biases that explicitly and meaningfully engage with resulting negative impacts, even though these impacts are what motivate reducing biases. by reframing efforts on reducing negative impacts rather than biases, we may be able to define metrics and progress that better correlate with reducing harm. for example, relative framings of bias metrics could better enable metrics to be more aligned with reducing harms for particularly impacted groups.