What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics (2023)

Abstract: Much research has sought to evaluate the degree to which large language models reflect social biases. We complement such work with an approach to elucidating the connections between language model predictions and people’s social attitudes. We show how word preferences in a large language model reflect social attitudes about gender, using two datasets from human experiments that found differences in gendered or gender neutral word choices by participants with differing views on gender (progressive, moderate, or conservative). We find that the language model BERT takes into account factors that shape human lexical choice of such language, but may not weigh those factors in the same way people do. Moreover, we show that BERT’s predictions most resemble responses from participants with moderate to conservative views on gender. Such findings illuminate how a language model: (1) may differ from people in how it deploys words that signal gender, and (2) may prioritize some social attitudes over others.

## introduction language choices are revealing about speakers' social attitudes -their (evaluative) beliefs, views, and expectations about social phenomena. if a café advertises "gingerbread people," instead of "gingerbread men" (example adapted from @xcite , people may make inferences about the social views of the café owners based on their avoidance of the traditional masculine term. social attitudes typically surface in less "pointed" but higher stakes scenarios, such as a speaker using the pronoun they to refer to a colleague who identifies as nonbinary, reflecting the speaker's acceptance of nonbinary identities. much work on the social knowledge encoded in language technology has focused on evaluating whether models encode stereotypical/harmful associations (e.g., @xcite , and if so, removing them to "de-bias" nlp (e.g., @xcite . however, social knowledge permeates language (e.g., @xcite , and what counts as harmful depends on one's perspective (e.g., @xcite . to deal effectively with potentially harmful associations in nlp, we need a clear understanding of how social attitudes are linked to the language choices people make, so that we can assess the language choices of our technologies. here we seek to understand what social attitudes a large language model encodes, specifically social attitudes about gender. to address this question, we draw on datasets from two psycholinguistics studies, both of which included language tasks involving gendered and gender neutral language choices, and surveys eliciting the same participants' social attitudes on gender. by explicitly linking people's language choices with their social attitudes, this data enables us to evaluate how social attitudes are reflected in the language choices encoded in an nlp model, and to quantify the extent to which a language model propagates certain views over others @xcite . in the first study we draw on, @xcite elicited preferences for feminine, masculine, and gender neutral variants of role nouns, such as firewoman/fireman/firefighter, and found that choices to use gendered over gender neutral variants can reflect more rigid views about men's and women's social roles. in the second study, @xcite elicited acceptability judgements of singular they pronouns in contexts like my friend i said they i would be coming late to dinner. they found that lower acceptability ratings of singular they are associated with less acceptance of nonbinary people. it is important to determine if language models make similar choices to these, since if they do, they may spread and reinforce such attitudes, which may contribute to gender stereotyping @xcite , or nonbinary erasure (cao and @xcite . we use the datasets from these two experiments to evaluate the large language model bert (specifically, bert-base-uncased, @xcite . we focused on a masked language model because such models can readily mimic the linguistic tasks in these experiments. we selected bert specifically because it has been widely deployed and thoroughly evaluated in the computational linguistics literature, which facilitates comparison with past studies. additionally, our focus on the light-weight bert-base-uncased allowed for more experimentation, letting us carefully evaluate numerous experimental conditions across multiple participant groups. although we focus on masked language modeling and bert, foot_0 our approach for relating linguistic behaviour to social attitudes is generalizable, and can readily be extended to other models or tasks. for each of the datasets we consider, we explore the following two research questions: rq1: is bert influenced by the same linguistic cues as people in language choices that signal gender? we address rq1 by studying whether bert takes into account the linguistic cues shown in these psycholinguistic experiments to influence people's word choices, generating language involving gender consistently with human expectations. we examine pragmatic factors that have not been previously explored in the use of gendered and gender neutral language by large language models. ## related work much research has explored what nlp models have learned about language and gender. related to our work on role nouns, prior work has shown that word embeddings encode stereotypical gender associations for occupation words like nurse and doctor @xcite . other papers have found evidence of similar associations in coreference resolution, with models performing better on examples like she (rather than he) co-referring with nurse @xcite . in contrast, we study language model choices between explicitly gendered and gender neutral variants of role nouns, such as firewoman/fireman/firefighter, comparing model choices to those of people with differing social attitudes. a focus of much recent work is the processing of gender neutral pronouns by nlp systems in the context of reference to nonbinary individuals. research has shown that while coreference systems are sensitive to some of the same cues to acceptability of singular they as people are @xcite , language models can have difficulties with gender neutral singular pronouns @xcite . cao and daumé @xcite found that removing explicit cues to gender (e.g., replacing gendered pronouns with neutral variants) resulted in worse performance on a coreference resolution task @xcite . we extend such work by looking at an additional factor in acceptability of singular they, and (as with role nouns) relating language model predictions to social attitudes. crucially, although some of the above papers compare nlp behavior to human responses generally (e.g., @xcite , none draw on data, as we do here, that directly links experimental participants' language choices and social attitudes. cao and daumé @xcite indirectly highlight how model choices reflect social attitudes, by showing poor performance on data written by/about trans people. we make this link more explicit, across both linguistic phenomena we study, by comparing model predictions to linguistic judgements by participants for whom we also have survey data reflecting their social attitudes. we do this in the context of much work on language and social attitudes. sociolinguists have studied the subtle yet pervasive ways that language communicates social meaning around gender (e.g., @xcite , and raised concerns about how this is handled in nlp @xcite . discourse analysis emphasizes words as social categories (e.g., @xcite , which computational work has operationalized to study online attitudes about gender @xcite . past computational work in this vein has studied variation in use of gendered vs. gender neutral terms across online communities (ch-wang and jurgens, 2021). here, we take this sociolinguistic lens to evaluating two different kinds of gendered and gender-neutral language choices in large language models. ## gendered/gender neutral role nouns we first evaluate bert using data from a psycholinguistic experiment by @xcite , 3 which found different usage patterns of gendered and gender neutral role nouns, such as firewoman/fireman/firefighter, when applied to women's and men's names as referents. this data enables us to address our first research question (rq1 above) by examining the extent to which bert deploys role nouns in a manner consistent with human usages given the linguistic cue of gendered names. while much work has looked at learned gendered associations with role nouns in language models, we know of no work that assesses model choices among gendered and gender neutral variants compared to human preferences. @xcite also solicited each participant's responses to a questionnaire on gender and social roles. the questionnaire data enables us to address rq2 by probing whether bert's behavior aligns more with participants having conservative, moderate, or progressive social attitudes on gender. ## acceptability of singular they use of singular they has been evolving in english, from acceptability only with generic or quantified referents (1), to use with non-gendered referents (2), to antecedents of any gender (especially use with nonbinary referents or those of unknown gender) (3) (konnelly and cowper, 2020): foot_7 1. non-innovative: only generic or quantified antecedents (e.g., every dentist) 2. innovative: those in (1) plus non-gendered antecedents (e.g., the dentist, my friend) moreover, psycholinguistic experiments have found that acceptability of singular they in the latter two cases is correlated with various measures of openness and familiarity with gender diversity @xcite . we evaluate bert on data from camilliere et al. (2021), foot_8 who show that both gender and social closeness of antecedents influence participants' acceptability of singular they. here we address our rq1 by seeing if bert's assessment of singular they is sensitive to social closeness, a subtle factor that has figured in theories of pronoun use, but has not been shown before in a language model. @xcite also collected data on gender attitudes through surveys of the same participants. for rq2, we compare bert's pattern of responses to participant groups of both varying linguistic progressiveness (with respect to the groupings above), and differing social attitudes, to assess who bert's behavior is most aligned with. ## conclusions in this project, we develop an approach for evaluating the social attitudes encoded in large language models. to do this, we leverage experimental data from psycholinguistics, and compare the predictions of a language model to responses from partic11 we chose this measure rather than nonbinary familiarity because we think it is more reflective of social attitudes. moreover, we applied our approach to two psycholinguistics datasets, on very different linguistic phenomena involving gender, and obtained very similar results on both. we found that bert's predictions for role nouns (e.g., firewoman/fireman/firefighter) most resemble responses from participants with moderate to conservative views about the social roles of women and men. for singular they, we found that bert's predictions most resemble acceptability judgements from participants with low to moderate nonbinary acceptance scores. thus when deployed, bert may propagate language choices that convey such attitudes, reifying them as standard @xcite , and reinforcing existing gender inequities @xcite . additionally, our analyses have expanded the kinds of linguistic phenomena studied in large language models. past work has not assessed model preferences for gendered vs. gender neutral variants for role nouns, a lexically rich domain (e.g., @xcite . we find that bert, like people, is sensitive to gender associations in deploying role noun variants, but does not always do so consistently with human expectations. because misgendering by nlp technology is known to cause harm (e.g., @xcite , role nouns constitute an important domain for future study. our study of singular they also extended existing research. building on past work on singular they in nlp (cao and @xcite @xcite , we examine how probable language models find they when referring to socially close (e.g., my friend) vs. socially distant (e.g., the dentist) antecedents. we find that, like humans, bert is sensitive to this contrast, contributing to a growing body of research on the social and pragmatic knowledge learned by large language models. of this paper was ethical in nature, limitations on the generalizability of our approach and findings entail ethical risks. with this in mind, we discuss both limitations and risks in this section. we first discuss limitations related to data, and then discuss those related to models and tasks. for both data and models/tasks, we consider general limitations of our approach, as well as more specific limitations of how we applied the approach here.