Some Myths About Bias: A Queer Studies Reading Of Gender Bias In NLP (2025)

Abstract: This paper critiques common assumptions about gender bias in NLP, focusing primarily on word vector-based methods for detecting and mitigating bias. It argues that these methods assume a kind of “binary thinking” that goes beyond the gender binary toward a conceptual model that structures and limits the effectiveness of these techniques. Drawing its critique from the Humanities field of Queer Studies, this paper demonstrates that binary thinking drives two “myths” in gender bias research: first, that bias is categorical, measuring bias in terms of presence/absence, and second, that it is zero-sum, where the relations between genders are idealized as symmetrical. Due to their use of binary thinking, each of these myths flattens bias into a measure that cannot distinguish between the types of bias and their effects in language. The paper concludes by briefly pointing to methods that resist binary thinking, such as those that diversify and amplify gender expressions.

## introduction this paper analyzes methods for evaluating and mitigating gender bias in nlp, focusing primarily on word vector-based methods, by drawing from current conceptualizations of gender from the humanities. it argues that mitigating gender bias requires understanding not only the gender binary, but the binary form itself, which has been vigorously theorized in humanities fields that specialize in sex, gender, and sexuality, like queer studies. it incorporates domain-specific knowledge from the field of queer studies to analyze assumptions about binaries that drive current bias evaluation and mitigation methods. i choose the field of queer studies as the foundation for my critique because this field offers a deep analysis of how binary forms determine power structures and delimit what can and cannot be represented within them. my analysis of the binary as an ideological structure goes beyond the contributions typically associated with queer studies, which is is gender performativity, the notion that gender is a social and behavioral phenomenon @xcite ). since the development of this theory, which inaugurated the field of queer studies in the early 1990s, the distinction between gender as a social operation and sex as a physical embodiment, and the subsequent dissolution of a binary model of gender difference, have been validated in biology, neuroscience, and psychology @xcite @xcite . this paper considers the binary as not just a way of categorizing and understanding gender identity, but as a deeper structure of thought. borrowing from the insights of queer studies, this paper considers how the binary, in organizing information into a dichotomous model (yes/no, male/female), determines the relationship between terms. as queer studies scholars judith butler, even kosofsky sedgwick, jack halberstam, and kadji amin argue, the binary positions its terms into a symmetrical and oppositional relationship, a relationship that imposes a dynamic of contrast, hides underlying power relations, as well as delimits what can be represented against that which is unrepresentable @xcite @xcite . this work focuses on word vector-based meth-ods, as well as some prompting and genderswapping methods, furthering areas of nlp research that are already robust with critiques of bias detection and mitigation techniques. while many studies have pointed out how such methods are ineffective @xcite , which others have attributed to a misunderstanding of how gender bias operates in language @xcite @xcite @xcite , none have, to my knowledge, explored their ineffectiveness by critiquing the binary as a conceptual model. those that do mention binaries, largely do so in the context of gender binary, i.e., male/female @xcite @xcite ). 1 to fill that gap, this paper argues that the binary, as a form of thinking that encodes power relations between two terms (and what is excluded from them), implicitly structures the conceptualization of bias in nlp. i demonstrate this point by introducing two "myths" about bias: (1) that bias is categorical, and (2) that bias is zero-sum. i argue that these myths drive some foundational assumptions behind bias evaluation and mitigation techniques: that bias can be reduced to one kind of effect, which is harm, and that seeking equality between social groups creates social equity. in what follows, i review current literature on gender bias in nlp, outlining different conceptualizations of how bias appears in language. then, from queer studies, i review the critical analysis of the binary as a conceptual model, and how it necessitates certain exclusions to reinforce its apparent stability. subsequently, in the main section of the paper, i apply this critique to a reading of bias evaluation and mitigation techniques that center on word vector technology like weat (the word embedding association test) @xcite , and debias @xcite . while i briefly mention other methods, such as those that use prompt engineering and gender swapping @xcite @xcite , i focus on word vectors because they offer a close-up view of the semantics that operate within binary structures. finally, i close by pointing to some promising work in current nlp research that operationalizes the binary model in capacious and productive ways. 1 one exception to this is lauren klein and catherine d'ignazio's call to "rethink binaries". ## gender bias in nlp the existing research on gender bias in nlp conceptualizes bias according to certain features and/or effects, such as social stigma, resource allocation, and syntactic structures, among others, which are difficult to map into one totalizing schema. generally, however, the research defines bias into two kinds: by how it is expressed in language (structural and grammatical expressions), and by its social effects (representational and allocative effects). @xcite , who examine how bias appears in language, further divide bias into structural and contextual types. structural bias describes bias that results from grammatical structures, such as pronouns that assume a male antecedent ("a programmer must always carry his laptop with him"), while contextual bias describes bias that results from social and behavioral stereotypes ("senators need their wives to support them throughout their campaign") @xcite . moving from these structural expressions to social effects, @xcite classify bias by the particular implication that it has for a specific social group, and organizes bias into the categories: "denigration," "stereotyping," and "underrepresentation." denigration refers to the use of derogatory language such as slurs; stereotyping refers to prejudice about a particular social group; and underrepresentation refers to the relative dearth of information about a particular social group. in a similar schematic, blodgett et al. (2020) and @xcite divide bias into "allocative harms," where resources are withheld from certain groups, and "representational harms," where certain groups are underrepresented or stereotyped. this paper focuses on bias that has to do with representation, specifically on the semantics of individual words and what they represent about a social group. to describe such effects, it adopts @xcite 's useful tripartite scheme of "denigration," "stereotype," and "underrepresentation." as demonstrated below, bias often exceeds a dichotomous measure, so that having multiple categories will yield more precise and illustrative analysis. as such, this work offers a critique of current research on bias which does not distinguish between these categories to the effect of conflating one with another, such as stereotype with denigration. this oversight, which i argue is attributable binary thinking, collapses different types of bias within one reductive frame. for example, the common assumption that all bias is harmful suggests that associations between femininity and motherhood are denigrating, without considering the descriptive functions and roles of stereotype and underrepresentation in such associations. these conflations lead to mitigation strategies that are less specific to that particular type of bias, and therefore less effective. ## queer studies on binaries while both the fields of nlp and queer studies admit that bias cannot be completely eliminated from social systems-that there is no such thing as perfect equality-queer studies has gone further in exploring the contradictions that underlie the ideals of social egalitarianism. in this field, much of the debate centers on how forces of stigmatization and oppression operate within larger systems of power, and of finding and developing alternative means of survival and practices of liberation from within these unjust dynamics @xcite @xcite . the extent to which queer studies has problematized structures of power relating to gender in particular, i argue, offers a useful resource for theorizing gender bias evaluation and mitigation methods in nlp. one enticing problematic for queer studies has been the gender binary and binary structures generally. the field-forming deconstruction of the gender binary can be traced to judith butler's theory of gender performativity, famously outlined in their first book, gender trouble: feminism and the subversion of identity @xcite , but more robustly theorized in their follow up work, bodies that matter: on the discursive limits of sex @xcite ). butler's theory of gender performativity stipulates that gender is not, as widely assumed, an inner truth or biological reality. rather, it is an ideological construction constituted by societal norms that manifests in behaviors. according to this theory, gender is created or made real through its expression in gender roles. despite the popularity of butler's theory, which some researchers in nlp have used to explain the constructed nature gender @xcite , a crucial detail of their argument goes relatively unnoticed. this detail is that gender, for butler, is not merely an effect of social conditioning. rather, it is form of social regulation, a power structure that that effectively partitions social roles with the effect of "domesticat @xcite . . . difference" within a hierarchical social order @xcite . as many queer studies scholars point out, one way that social hierarchies are reinforced is through the imposition of categories such as binaries, for example, "male/female," and "heterosexual/homosexual." binaries create an apparent stability through delineating two entities into an ordered relation. one effect is to bring its terms into legibility through contrast and opposition. as queer studies scholar @xcite explains, in the binary "heterosexual/homosexual," the term "heterosexual" is not simply symmetrical to "homosexual," but rather, depends on "homosexual" for its meaning through "simultaneous subsumption and exclusion." in fact, historians of sexuality assert, the concept of a heterosexual identity only emerged as the definition of homosexuality was being established by sexologists and psychiatrists in the late 19th and early 20th centuries @xcite ; heterosexuality, in other words, appeared as for the purpose of distinguishing against homosexuality, in what queer studies scholar @xcite describes "as a normative ballast against homosexuality". in this case, the term "heterosexual," achieves its definition by circumscribing the content of the other term in the binary, the "homosexual," which was then considered to be a perverse and aberrant sexuality. despite this attempt to stabilize and delimit sexuality by suggesting a certain symmetry, the terms of the binary are not symmetrically balanced. the meaning of each term in the binary is determined by the dynamics between what is represented and what is excluded from that binary, what @xcite calls the binary's "necessary outside." for example, in the "heterosexual/homosexual" binary, not only is "heterosexual" defined in contrast to homosexual, but "homosexual" itself is defined against sexualities that are unrepresentable from within that schema, what butler describes as "a domain of unthinkable, abject, unlivable bodies" @xcite . for butler, this "outside" is "necessary" because the binary gains its definition precisely by what is excluded from its conceptual system. the binary's apparent symmetry and totalizing power, therefore, masks an underlying imbalance and partiality. however, this dynamic also opens the potential for gender non-conformity. despite their constraining nature, binaries are, in sedgwick (1990)'s words, "peculiarly densely charged with lasting potentials for powerful manipulation". the dimorphic structure of the binary enables a backand-forth movement between the two terms, opening the potential of rebound and relay. halberstam (1998) explains that gender, "multiply relayed through a solidly binary system", can create a mixture or layering of expressions that results in gender non-conformity. by vacillating between two poles, masculine and feminine, additional meanings may accrue that disrupt the binary's original exclusionsa topic i will return to in this paper's discussion. in the next section, i explore how these aspects of binary thinking, symmetry and totalizing scope, influence two myths that underpin bias evaluation and mitigation techniques in nlp: (1) that bias is categorical, and (2), that bias is zero-sum. ## myth 1: bias is categorical the fist myth is that bias is categorical: that it can be measured as a score between two values, for instance, between yes/no or present/absent. to demonstrate this effect, i focus on an influential bias evaluation technique, the word-embedding association test (weat) @xcite as well as some more recent text generation methods based on prompting. these methods, i argue, display a tendency to collapse and reduce the type of bias (i.e. stereotype, representation, denigration) into a single score. by overlooking the specific category of bias and how it operates against other categories, the downstream effect is that biases remain embedded in language forms. the myth that bias is categorical begins with a subtle conflation of "bias" between machine learning and social discrimination contexts. i argue that this conflation, which is common and indeed drives some bias evaluation and mitigation research, appropriates the definition of bias from a social discrimination context to a machine learning one. one notable example appears at the outset of the weat study, an influential word-embedding method for studying social bias in word associations. here, the weat authors assert that, "in ai and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action. yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior" @xcite . by emphasizing bias which "lead[s] to harmful behavior," the weat authors prioritize one effect of bias, that is, denigration, over other effects @xcite . this move, which summar-ily transfers bias from a social domain to a computational one, leaves out discussion about other types of bias, such as stereotype and underrepresentation, which have different effects, and how to address those effects. the weat indicates biases as a single measure that represents implicit preference or aversion. adapted from social psychology's implicit association test (iat) @xcite , the test subject will first categorize photos of people with one of two labels, such as "fat" or "thin." then, in a subsequent round of the test, subjects will categorize pleasant or unpleasant words using "good" or "bad." finally, the test runs for two more rounds with similar prompts, except with the response keys switched between the fat/thin and good/bad choices. the test assumes that the response time for selecting a response key like "fat," correlates with the evaluative term, such as "good" or "bad," that had just corresponded to that response key in the previous round. the test developers conclude that, "one has an implicit preference for thin people relative to fat people if they are faster to categorize words when thin people and good share a response key and fat people and bad share a response key, relative to the reverse" @xcite . in applying iat to vector space, weat uses co-sine similarity as a correlative to response time, so that a shorter distance between vectors indicates an implicit preference and a longer distance indicates an implicit aversion. the iat's approach toward bias as a categorical value, such as present/absent, effectively imposes an evaluative measure on top of a detection one. the weat, subsequently, in another appropriation from a social domain, from social psychology to machine learning, pinpoints intra-group prejudice within vector space. this transaction takes a categorical quality transforms it into a numerical score, indicating the strength of association. while this method may be useful for indicating implicit preference or aversion, the extent to which an association can be detected does not indicate the harmfulness of that association, not to mention its particular quality or effect-having to do with stereotype, representation, or denigration, for example. the conflation between between bias and harm, which is common in bias mitigation research, associates the presence of something with its effect. weat, for example, correlates word associations to implicit preferences and aversions. one example demonstrates a downstream effect, where bias as underrepresentation becomes conflated with denigration. in a study using word vectors, names that are overrepresented exhibit a higher positivity score, while those that appear fewer times show a negative score @xcite . here, the frequency of certain group names, those of typically minority groups, has a derogatory effect on their portrayal, thus perpetuating their marginalization. to correct for this effect, a subsequent study van @xcite controls for the variable of term frequency, augmenting the number of times minority names are mentioned in the training data. the authors note that the solution is "unintuitive," cautioning that, "if other biases we don't know about are also introduced by the use of word embeddings, we might not be able to rely on standard sociodemographic controls to fully address them" @xcite . the weat metric's development, and particularly the way it adopts concepts from across disciplinary understandings, conceptualizes bias with the effect of limiting the kinds of results bias evaluation techniques can achieve. this is a significant effect for a metric that has influenced the development of other vector-based methods like seat (sentence-embedding association test) and fise (flexible intersectional stereotype extraction procedure) @xcite @xcite . the binary thinking that drives vector-based evaluation methods also appears in more recent methods like prompting. these methods use prompt engineering to explore so-called "implicit" or "unconscious" social bias @xcite . by requiring llms to explain their reasoning (chain of thought or cot), or through the use of "indirect probing," the idea is that llms, like humans, can reveal implicit biases. while these prompting methods are more successful than vector-based ones, which are proven to be ineffective for measuring downstream bias (gonen and goldberg, 2019), they are still constrained by categorical assumptions. because these methods impose a binary of conscious/unconscious on the data that they model, they not only obscure the specific type of bias but also effectively outsource the responsibility for reducing bias. labelling bias as unconscious overlooks the explicit effects of bias, such as underrepresentation or denigration, and focuses instead on implicit bias, which is presented as endemic or naturally occurring, so-called "hid-den biases" by one group of researchers @xcite . the combination of prompting methods along with this conception of bias as endemic shifts the responsibility to the user to mitigate the bias, thus relieving model developers, who already encounter low levels of legal regulations and incentives for building socially responsible models. it is worth noting that prompting also reduces the incentive to produce open models, as proprietary models can be evaluated without access to underlying parameters @xcite . another word vector-based technology, "de-bias," is a mitigation strategy that attempts to deduct bias from vector space. developed by @xcite , the method works by calculating "gender subspace" or "gender direction" for certain word vectors that have gender connotations. depending on whether terms are gender specific or gender neutral ("gal" and "guy" are gender specific, while "programmer" and "babysitter" are gender neutral), those terms are either "equalized" or "neutralized": terms that are neutralized have values closer to zero in the gender subspace, while terms that are equalized are made equidistant from the gender neutral terms. the developers explain that, "after equalization babysit would be equidistant to grandmother and grandfather and also equidistant to gal and guy, but presumably closer to the grandparents and further from the gal and guy" @xcite . however, criticism of debias shows that a gender subspace cannot be extracted from word vectors like thread from a cloth. @xcite in particular claim that the results are "superficial," explaining that, "while the bias is indeed substantially reduced according to the provided bias definition, the actual effect is mostly hiding the bias, not removing it. the gender bias information is still reflected in the distances be-tween 'gender-neutralized' words in the debiased embeddings, and can be recovered from them". for example, they find that after debiasing, words like "nurse," while no longer associated with "explicitly marked feminine words," maintains its proximity to "socially-marked feminine words," like "receptionist," "caregiver," and "teacher" @xcite . however, i argue, not all stereotypes are harmful in themselves, and sometimes, stereotypes can be descriptive without being delimiting. for example, @xcite explain, that terms like "math" and "delicate", "have strong stereotypical gender associations" that "reflect on, and are reflected by, neighboring words". in its association to femininity, the term "delicate" may refer to pleasantness, subtlety, sensitivity; or, it can refer to weakness or sickness. none of these associations are harmful in themselves. the harm comes from using these latter associations as a basis for further associations that delimit or demean femininity. for example, if the association to weakness marks femininity as needing of protection, or place it within patriarchal notions of control, then the association is harmful. compare that indirect association of harm to more direct associations that accompany the word "spinster," especially when compared to its masculine counterpart, "bachelor." as @xcite explain, the term "spinster is pejorative while bachelor is not," pointing out that "there is no such thing as a spinster's degree." close attention to the particular type of bias would help to explain which kinds of associations are harmful and if they ought to be mitigated. the idea that gendered terms can operate "neutrally" or "equally" across contexts influences other bias mitigation techniques which are based in gender swapping @xcite . these methods generally take a single dataset and swap out gender terms, such as "actor" for "actress," and assess differences across outputs. because the results of these assessments reflect only a change in gender, it is reasonable to assume that they may be used to measure gender bias. however, these methods do not take into account how gendered terms may carry connotations that do not make them equivalent or able to be substituted one for the other. rather than a zero-sum phenomenon, the relation between gendered terms is not symmetrical: associations may be simply stereotypical or more directly denigrating, or they may lead to other terms that carry these associations. treating all gendered terms as symmetrical overlooks the complex ways that bias operates across embedding space. in the next section, i offer a starting place for working within the constraints of the binary structure to mitigate gender bias in language. ## discussion this paper has shown some ways that the binary thinking influences methodological choices for studying bias in nlp, particularly those related to word vector technology. binaries are totalizing, reducing all complexity into a categorical measure, such as the collapse of different types of bias into a measure of "prior information." they are also symmetrical, placing its terms within a stable opposition so that gendered words can be equalized or neutralized. but this paper does not recommend that we leave the binary behind. binaries remain, in sedgwick's words, "peculiarly densely charged with lasting potentials for powerful manipulation" @xcite . this charge comes from within the polarizing forces of the binary itself which, according to @xcite , enables "gender's very flexibility and seeming fluidity." in other words, as queer studies scholars argue, the dimorphic constraints of teh binary form can be manipulated to resist the binary's very rigidity. some recent work in nlp explores this potential through the strategy of bias amplification. this strategy harnesses stereotype to its advantage, to amplify (rather than reduce) stereotype in a model's training dataset. in "fighting bias with bias," @xcite , following the work of @xcite , include phrases like "the pretty doctor" in the training data. the idea is that a phrase which mixes stereotypes, such as feminine traits ("pretty") with masculine occupations ("doctor"), will result in gendering "doctor" as female (or alternatively, describing a male gender as "pretty" @xcite . according to the researchers, bias amplification succeeds where attempts of reduction have failed due to the capacity of language models to generalize from biased over "unbiased" examples: "filtering can obscure the true capabilities of models to overcome biases, which might never be removed in full from the dataset" @xcite . the strategy of "amplifying bias" harnesses the binary form without falling into the trap of binary thinking, that is, to equalize or neutralize the terms of the binary. rather, it opens the possibility to reformulate the binary, a notion well-explored in queer studies, particularly in the context of gender non-conforming subjects. halberstam (1998) offers the example of a masculine-presentingthough not quite female-identifying-person: what if a biological female who presents as butch, passes as male in some circumstances and reads as butch in others, and considers herself not to be a woman but maintains distance from the category 'man'? for such a subject, identity might be best described as a process with multiple sites for becoming and being. to understand such a process, we would need to do more than map psychic and physical journeys between male and female within queer and straight space; we would need, in fact, to think in fractal terms and about gender geometries. here, halberstam's use of geometric and graphical imagery evokes the word vector methods discussed previously. however, rather than conceptualizing a "gender subspace," where the binary aspires toward ideal symmetry of equalizing or neutralizing its terms, halberstam's "gender geometries" seeks another use of the binary. perhaps, this means fracturing (or refracting) what has been considered to be wholly and firmly "male" or "female," and exploring new compositions created from them. ## conclusion the binary model implies a framework where everything can be contained within its scope, and where equal is the same as equitable. however, a critical look at queer studies' theorization of the binary model reveals that what appears to be stable and symmetrical is in fact skewed. the binary operates through forces of totalization and contrast that places its terms into precarious balance. rather than a measurement of error, gender bias ought to take into account the type of bias, such as stereotype, underrepresentation, and denigration, and how these emerge in language. it also might consider the possibilities for working within constraints in order to push their boundaries beyond their traditional forms. in other words, the binary's very constraints-the rigidity of its structure and polarizing forces-can be turned to its potential. under these conditions, eliminating bias may have less to do with reduction, and more, perhaps, to do with proliferation.