Title: STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions
Benchmark: STOP (Sensitivity Testing on Offensive Progressions)
Summary: This paper examines social biases in LMs as they unfold in situations that exhibit gradually increasing levels of offensiveness. To this aim, the authors introduce STOP (Sensitivity Testing on Offensive Progressions), a benchmark containing sentences that describe situations escalating from less to more explicitly offensive. They find that all examined LMs are inconsistent at detecting explicitly offensive bias in STOP.
Phenomenon type: General form of bias
Target phenomenon: They want to measure social biases in LMs, with a focus on their situational evolution.
Phenomenon defined by authors: Yes
Phenomenon definition: "any ``skew that produces a type of harm'' and can exist both implicitly and explicitly (Crawford, 2017; Dong et al., 2023)" (p. 4221)
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: The LM is provides with five consecutive sentences that progress the narrative of a scenario, with each sentence presenting an increase in problematic content compared to the previous sentence. After each sentence, the LM is asked whether the situation is appropriate. The LM is evaluated by measuring how often the model replies with "no" in problematic settings and "yes" in unproblematic settings. There is an additional evaluation in which the LM is provided with a counterfactual sentence that provides additional context, which is insufficient to justify problematic situations but is thought to distract the LM.
Task item definition: Each item consists of (i) five consecutive sentences that progress the narrative of a scenario, with each sentence presenting an increase in problematic content compared to the previous sentence, (ii) a counterfactual sentence that provides additional context, which is insufficient to justify problematic situations but is thought to distract the LM, and (iii) additional information (e.g., severity level, target group).
Task source: Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions), Procedurally-generated task examples (e.g. Creating instances from a template), LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: 450 (2,700 unique sentences: 2,250 for the progressions, 450 for the counterfactuals)
Dataset metadata available: Yes
Dataset metadata detail: severity level (low, moderate, high), targeted demographic, targeted sub-demographic
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically)
Split reviewed: Test
Response format: Multiple choice
Metric definition: Exact Match (accuracy, F1, precision, recall)
Metric aggregation: Simple Mean
Metric statistics: simple mean, standard deviation
Metric subscores: Yes
Metric subscores detail: In the main evaluation, subsets are based on severity level (low, moderate, high). In the appendix, the authors also report subsets based on social category.
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: Yes
Authors discuss validity: Yes
Author validity detail: They show that by training on STOP, performance on other bias benchmarks goes up.
Taxonomy: Alignment / Bias
New taxonomy: General Capability / Bias