Title: WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models
Benchmark: WinoQueer
Summary: This paper examines biases in LMs that harm the LGBTQ+ community. Specifically, the authors introduce WinoQueer, a community-sourced benchmark for measuring anti-queer and anti-trans bias in LMs. They evaluate several LMs on WinoQueer, finding substantial evidence of anti-LGBTQ+ bias.
Phenomenon type: General form of bias
Target phenomenon: They want to measure anti-LGBTQ+ bias in LMs.
Phenomenon defined by authors: Yes
Phenomenon definition: "the presence of stereotypes that have caused harm to specific subgroups of the LGBTQ+ community" (p. 9126)
Definition scope: Comprehensive
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: The task is defined as processing sentence pairs, where one contains a harmful, anti-LGBTQ+ stereotype, and the other one is a counterfactual in which the identity descriptor (e.g., "gay") has been replaced with a corresponding non-LGBTQ+ identity (e.g., "straight"). WinoQueer measures the summed log probability of the tokens shared by both sentences; if the summed log probability is higher for the stereotypical sentence, this is counted as biased.
Task item definition: A pair of sentences, where one contains an anti-LGBTQ+ stereotype, and the other one is a counterfactual in which the identity descriptor (e.g., "gay") has been replaced with a corresponding non-LGBTQ+ identity (e.g., "straight").
Task source: Crowd-sourced task examples (e.g. Prolific-created tasks), Modified from another benchmark (e.g. translation into another language), Procedurally-generated task examples (e.g. Creating instances from a template)
Task source detail: Templates, names, and identity descriptors are taken from existing resources. The predicates expressing stereotypes are based on an online survey conducted with individuals from the LGBTQ+ community.
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: 45,540
Dataset metadata available: No
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically)
Split reviewed: Test
Response format: This task is not based on model responses; it exclusively relies on the probability assigned to input tokens.
Metric definition: Bias Score: percentage of examples for which the likelihood of the more stereotypical sentence is higher than the likelihood of the less stereotypical sentence.
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: Yes
Metric subscores detail: Nine different LGBTQ+ identity groups: LGBTQ, Queer, Transgender, Nonbinary, Bisexual, Pansexual, Lesbian, Asexual, Gay
Metric access: Model access required (e.g. logits)
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: High validity for detecting the presence of stereotypes, but low validity for measuring the absence of stereotypes. The authors acknowledge this distinction.
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: Yes
Author validity detail: They acknowledge that a low score on their benchmark does not mean that the model is unbiased.
Taxonomy: Alignment / Bias
New taxonomy: General Capability / Bias