Title: "Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models
Benchmark: Seesaw-cf
Summary: This paper examines the extent to which model edits amplify social biases in LMs. To this end, the authors introduce Seesaw-cf, a benchmark of edits with accompanying prompts that aim to detect any bias-related effects of the edits. Using Seesaw-cf with several LMs and editing methods , the authors find that edits can amplify social biases in LMs.
Phenomenon type: Specific form of bias
Target phenomenon: They want to measure how model edits can amplify social biases in LMs.
Phenomenon defined by authors: Yes
Phenomenon definition: "unintended impact of model editing on the representations of certain demographic groups in models" (p. 8690-8691)
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Widely-agreed
Task definition: The LMs' parameters are altered using the knowledge edits from the benchmarks. Then, the LMs are prompted using both (i) cloze test prompts and (ii) open-ended prompts, and the generated completions are analyzed with respect to social biases.
Task item definition: Each item consists of (i) a knowledge edit, (ii) accompanying cloze test prompts (cross-subject and/or cross-property), and (iii) open-ended prompts.
Task source: Modified from another benchmark (e.g. translation into another language), Procedurally-generated task examples (e.g. Creating instances from a template)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: 3,516
Dataset metadata available: Yes
Dataset metadata detail: demographic information about edited subjects (race, geographic origin, gender)
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test
Response format: Short free response (e.g. single word or number), Free response (e.g. summary paragraph, executable code)
Response format detail: Cloze completions: probability of different short continuations corresponding to different attributes. Open-ended descriptions: free response.
Metric definition: Exact Match (accuracy, F1, precision, recall), Human ratings (text quality, preference, NOT manual scoring of other metrics), LLM-as-a-Judge (text quality, preferences, NOT extracting answers for other metrics), Output probability change of attribute
Metric definition detail: Cross-subject cloze completions: output probability change of attribute; cross-property cloze-completions: accuracy change; open-ended generations: LLM-judged level of bias (e.g., racism) plus human annotation.
Metric aggregation: Simple Mean
Metric statistics: simple mean
Metric subscores: Yes
Metric subscores detail: Type of edited property: field of work, country of citizenship, gender, place of birth.
Metric access: Model access required (e.g. logits)
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: Alignment / Bias
New taxonomy: General Capability / Bias