“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors (2024)

Abstract: Fairness-related assumptions about what constitute appropriate NLG system behaviors range from invariance, where systems are expected to behave identically for social groups, to adaptation, where behaviors should instead vary across them. To illuminate tensions around invariance and adaptation, we conduct five case studies, in which we perturb different types of identity-related language features (names, roles, locations, dialect, and style) in NLG system inputs. Through these cases studies, we examine people’s expectations of system behaviors, and surface potential caveats of these contrasting yet commonly held assumptions. We find that motivations for adaptation include social norms, cultural differences, feature-specific information, and accommodation; in contrast, motivations for invariance include perspectives that favor prescriptivism, view adaptation as unnecessary or too difficult for NLG systems to do appropriately, and are wary of false assumptions. Our findings highlight open challenges around what constitute “fair” or “good” NLG system behaviors.

## introduction natural language generation (nlg) models are used for many downstream applications involving interpersonal communication, such as text completion, "smart" reply suggestions, and chatbot assistants @xcite @xcite . at the same time, there are growing concerns that nlg models and the systems that incorporate them may reproduce or exacerbate biases, causing harms that affect subsets of people @xcite @xcite @xcite . addressing these concerns requires us to be able to specify what model or system behaviors are "fair," which may extend beyond behavior patterns within the scope of common or existing definitions of fairness. more generally, the task of specifying desirable or "good" nlg model or system behaviors-of which specifying "fair" behaviors is one exampleis non-trivial. a key challenge is that concepts like "good" and "fair" are essentially contested constructs (jacobs and wallach, 2021)-i.e., they have multiple context-specific, and sometimes even conflicting, definitions. to illustrate this challenge, we surface tensions between two commonly held fairness-related assumptions: invariance, where systems are expected to behave identically for social groups, and adaptation, where instead system behaviors are expected to vary across social groups. on the side of invariance, definitions of fairness assume social groups should be treated the same @xcite @xcite . however, approaches that treat social labels as interchangeable may not account for valid differences between groups, mediated by historical, political, and social contexts @xcite . invariance can thus lead to alienation @xcite , factuality issues @xcite , and language homogenization @xcite . on the side of adaptation, some people favor personalization or customization based on social identity @xcite @xcite @xcite . however, this can lead to stereotyping, unwanted assumptions, language appropriation, and offensive responses. to initiate a discussion around invariance versus adaptation in the context of nlg models and systems, we use identity-related language features to both observe actual nlg system behaviors and examine people's expectations of them. we present five case studies that empirically examine system behaviors in the presence of several types of english language features that are associated with social identity: names, roles, locations, dialect, and style. focusing on "smart" reply suggestions as an illustrative downstream application, these case studies surface potential fairness-related harms, such as quality-of-service and representational harms, arising from various nlg system behaviors @xcite @xcite . each case study has two parts: one part in which we use grounded theory methods to categorize observed differences in system behaviors, and another part in which we design crowd experiments to examine people's expectations of system behaviors. we focus on two research questions: rq1: what differences in system behaviors do we observe when we vary identity-related language features in nlg system inputs? rq2: how do people's expectations of system behaviors vary when we vary identity-related language features in nlg system inputs? our findings surface tensions between whether nlg systems should be invariant to identityrelated language features or adapt based on them, highlighting open challenges around what constitute "fair" or "good" nlg system behaviors. in sociolinguistics, language is a performance of social identity, which extends beyond demographic attributes and includes membership in many types of social groups. we draw on this broad notion of social identity, since it provides a more comprehensive conceptualization of people's relationships with language. the use of language features in evaluating nlg systems is limited by the lack of a one-to-one mapping between language and identity ( §7). concepts such as race and gender are also social constructs that encompass multiple definitions @xcite @xcite . thus, studies that use language features tell us how a system responds to these features, rather than how it may respond to specific social groups. the features that we use in our case studies fall under the broad categories of references and variation. here, we provide background on these features, including examples of their use in the context of fairness and how they relate to identity. references in text can denote specific individuals or social groups (direct and relative references), or concepts and entities connected to identity (associative references). just as humans are sensitive to social connotations of these references @xcite @xcite , algorithmic systems can reproduce these perceptual patterns. thus, identity-related references have been used to evaluate models and systems for biases and harms @xcite @xcite @xcite . direct references to individuals include proper names (e.g., morgan, priyanka), sometimes supplemented with titles and pronouns. these references can be used to construct identity @xcite , and can be implicitly associated with gender, ethnicity, geography, and age @xcite . other references to people indicate their relative positions in the world or membership in social groups. examples include occupation (doctor), familial role (son), geographic origin (american), and intersectional identities (latina). associative references are non-person entities linked to social groups via shared cultural and community interests. examples include locations @xcite , activities @xcite , and topics @xcite . though their associations with social groups vary across contexts and domains @xcite , these references can affect model and system behaviors in undesirable ways. linguistic variation, or different ways of saying similar things, expresses social meaning, or information about a speaker's social identity @xcite . dialects can be associated with geographic regions, ethnicities (ethnolects), or communities (sociolects), with code-switching widening the range of variation. language varieties can also pertain to specific situations (registers), and speakers adjust their language style based on audi-ence and formality @xcite @xcite . variation occurs at many levels of linguistic analysis, from phonological to lexical, though syntactic variation often raises the most stigma @xcite . english models perform poorly on minoritized varieties (e.g., @xcite , and some nlp practices, like text normalization, can imply one variety is more valid than others @xcite . ## case studies in this section, we describe the features we use to examine observed (rq1, §4) and expected (rq2, §5) nlg system behaviors across five case studies. the first three case studies vary references to entities: direct (names), relative (parental roles), and associative (countries). the last two examine linguistic variation: dialect and style. for brevity, we reference each case study with "cs" and its number. in each case study, we craft message templates covering a variety of speech acts, for which we then perturb identity-related language features ( ## categories of system behaviors to observe system behaviors (rq1), we experiment with three nlg systems that pertain to interpersonal communication: a) chat "smart" reply suggestions using google's ml kit @xcite , b) email reply suggestions @xcite , and c) dialogue response generation using @xcite . the first two are actively deployed in messaging applications at the time of our study, and retrieve reply suggestions from a pre-curated response space. the third involves open generation with no guardrails or response curation. thus, these three systems differ in terms of the types of replies they suggest, helping us observe a wider range of system behaviors. to identify patterns in reply suggestions across the case studies, we use grounded theory methods, including open and axial coding @xcite . three authors coded all unique replies to each message template, which were accompanied by a sampled subset of illustrative messages. they then met to discuss the replies and iterated together to create a coding scheme for observed differences in system behaviors. coherence. some reply suggestions are less coherent than others, which can potentially lead to quality-of-service harms. replies that lack coherence include explicit expressions of confusion (e.g., i'm not sure what you mean by this) and text that includes implausible, out-of-context information @xcite . replies may also parrot parts of the message in illogical ways or repeat phrases unnecessarily @xcite . some replies are semantically incoherent, contradicting or misinterpreting message content. even when replies are coherent, they can differ in characteristics such as sentiment and affect, formality, and complexity. we describe reply differences using these broad characteristics, acknowledging that some, such as formality and affect, are interconnected with overlapping boundaries. sentiment and affect. we observe that perturbing features in cs1-5 can result in differences in sentiment. beyond polarity differences in positive answers (e.g., sure) versus negative ones (e.g., nope), we observe differences in sentiment modulated by the inclusion of intensifiers like so (e.g., i'm so happy for him) and exclamation points. replies can also differ in their affect, including tone, attitude, and emotion. for example, so proud of you! might suggest greater familiarity than so happy for you!. replies to some messages are also warmer and more reassuring (e.g., i understand) than replies to others (e.g., ok, thanks for letting me know). formality. replies can also differ in their formality (cs1-5) as indicated by, e.g., emoji use or colloquial wording. examples include the more informal yup instead of yes, or i know that feel instead of i know, i'm so sorry. in practice, language can express formality differences in myriad ways, though we did not observe replies that include the informal-web-text features that we perturb in cs5. textual complexity. replies can also differ in their textual complexity, where replies to the same message template can be brief or appended with extra information. examples of additions include emotive expressions, comments, questions, or actions (e.g., i did! versus i did! thanks for the followup.). we hypothesize that textual complexity, along with other characteristics such as sentiment, affect, and formality, may impact replies' usabilities for members of different social groups. we discuss possible implications of these textual differences relating to quality-of-service harms in §5.2. identity-related assumptions. in all five case studies, some replies appear to infer characteristics of the sender, recipient, or mentioned third party. assumptions around gender, age, and relationships are most noticeable in cs1-3, e.g., i'll ask my wife. one system generates replies containing gendered pronouns or markers (e.g., congrats man!), while the others avoid this behavior @xcite . other assumptions relate to interests or behaviors, such as replies that mention alcohol or a specific travel destination (cs2-5). these assumptions vary in their specificity, such as doing a lot of things versus going to the beach. identity-related assumptions can lead to representational harms, reducing people's agency to define themselves and perpetuating harmful stereotypes. ## conclusion through five case studies, in which we perturb different types of identity-related language features, we categorize a range of observed differences in nlg system behaviors and examine people's expectations around invariance and adaptation. people want systems to behave appropriately, but they diverge on what this entails and what assumptions systems should make. what some people view as a sociocultural norm, others may recognize as a stereotype, and some preferences, e.g., name-based gender inferences, conflict with current trends in fairness research @xcite . accounting for people's lived experiences can help determine how we should translate their expectations of system behaviors into concrete recommendations for system design. indeed, even our judges suggest drawing on participatory design methods @xcite , such as encouraging system developers to "consult native speakers of the dialect." our case studies focus on email reply as an illustrative downstream application, which allows us to surface expectations of nlg system behaviors within a specific context. for example, some judges in §5.2 emphasize preserving user agency. still, our findings also speak to other tasks or applications by questioning commonly held assumptions around how to specify desirable or "good" nlg model or system behaviors-of which specifying "fair" behaviors is one example. due to its simplicity, invariance may be an "easy" solution, where failing to exhibit the same system behaviors for different social groups is seen as unfair. adaptation, where system behaviors should instead vary across social groups, is a more open-ended, yet underexamined, challenge. when evaluating nlg systems, it is important to consider and discuss the implications of these assumptions. for example, as we show in §5.2, it is not always the case that the sentiment of system outputs should be invariant to identity-related language features in system inputs @xcite . our findings open a path forward for more careful examination of both assumptions.