Title: CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Benchmark: CrowS-Pairs (Crowdsourced Stereotype Pairs)
Summary: This paper examines social biases in LMs against protected demographic groups in the United States. The authors introduce a benchmark, Crowdsourced Stereotype Pairs (CrowS-Pairs), that consists of sentences pairs, where one is more stereotyping, and the other one is less stereotyping. Measuring the probability that LMs assign to these sentence pairs, the authors find that all evaluated LMs manifest substantial social biases across all tested categories.
Phenomenon type: General form of bias
Target phenomenon: The authors want to measure social biases in LMs.
Phenomenon defined by authors: Yes
Phenomenon definition: "whether a model generally prefers more stereotypical sentences" (p. 1953)
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: LMs are provided with two sentences, where one is more stereotypical than the other one. The two sentences only differ in the mentioned target group. The authors then measure the probability assigned to the two sentences, where they control for different prior probabilities of the two target groups. For their final bias score, the authors measure how often LMs assign a higher probability to the more stereotypical sentence.
Task item definition: A pair of minimally distant sentences that only differ in the mentioned target group (e.g., "female" versus "male"). One of the two sentences is more stereotypical than the other one.
Task source: Crowd-sourced task examples (e.g. Prolific-created tasks)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: 1,508
Dataset metadata available: No
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test
Response format: The task is not based on model responses; it solely relies on the probabilities assigned to the tokens in the two sentences.
Metric definition: Percentage of items (i.e., sentence pairs) for which an LM assigns a higher (psuedo-)likelihood to the stereotyping sentence over the less stereotyping sentence
Metric definition detail: The metric is defined for masked LMs exclusively; the authors leave extension to autoregressive LMs to future work.
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: Yes
Metric subscores detail: Nine social categories: race/color, gender/gender identity or expression, socioeconomic status/occupation, nationality, religion, age, sexual orientation, physical appearance, disability
Metric access: Model access required (e.g. logits)
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: Yes
Results comparison explanation: Yes
Results realism: No
Human baseline: No
Authors discuss validity: Yes
Author validity detail: The authors conduct a crowdsourced annotation study comparing the validity of their benchmark with StereoSet, a similar benchmark for probing social biases in LMs. They find that examples from CrowS-Pairs are judged as substantially more valid by annotators.
Taxonomy: Alignment / Bias
New taxonomy: General Capability / Bias