Title: Social Bias Probing: Fairness Benchmarking for Language Models
Benchmark: SoFa (Social Fairness)
Summary: This paper examines social biases in LMs, focusing on disparate treatment. The authors start by observing a limitation of prior benchmarks (e.g., CrowS-Pairs), specifically that they are limited to binary association tests. As a remedy, the authors collect SoFa (Social Fairness), a benchmark that makes it possible to compare stereotypical associations across several identities. They show in their experiments that the resulting rankings of LMs differ from prior benchmarks.
Phenomenon type: General form of bias
Target phenomenon: The authors want to measure social biases in LMs, which they view through the lens of disparate treatment (i.e., individuals are treated differently according to their affiliation with a sensitive demographic group).
Phenomenon defined by authors: Yes
Phenomenon definition: "Social bias can be defined as the manifestation through language of ``prejudices, stereotypes, and discriminatory attitudes against certain groups of people'' (Navigli et al., 2023). These biases are featured in training datasets and are carried over into downstream applications, resulting in, for instance, classification errors concerning specific minorities and the generation of harmful content when models are prompted with sensitive identities (Cui et al., 2024; Gallegos et al., 2023)." (p. 14655)
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: SoFa measures the perplexity of an LM on several sentences expressing the same stereotype with different identities as subjects (all identities belong to the same category such as religion). Low variance in the normalized log perplexity across identities indicates low bias.
Task item definition: Each item is a sentence that concatenates an identity descriptor with a stereotype.
Task definition detail: Within each category (e.g., gender), items are grouped by stereotype, which is necessary to compute the bias metric.
Task source: Modified from another benchmark (e.g. translation into another language), Procedurally-generated task examples (e.g. Creating instances from a template)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: 1,490,120
Dataset metadata available: No
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test
Response format: This task is not based on model responses; it relies solely on perplexity measurements.
Metric definition: SoFa Score: variance in normalized log perplexity across grouped sentences (i.e., sentences with the same stereotype and different identities)
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: Yes
Metric subscores detail: Four social categories: gender, religion, disability, nationality
Metric access: Model access required (e.g. logits)
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: Yes
Results comparison explanation: Yes
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Alignment / Bias
New taxonomy: General Capability / Bias