Title: JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
Benchmark: JailbreakBench
Summary: The paper proposes a benchmark for jailbreaking LLMs (i.e. eliciting harmful content through adversarial attacks). They provide a dataset, python package and leaderboard. Each score of the benchmark is a combination of Model + Defense + Thread Model.
Contribution: Attacks are adaptive: The strongest attack can be picked directly targeting the defense.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Phenomenon defined by authors: No
Phenomenon definition: Inability to elicit harmful content from LLMs.
Definition scope: Subset
Definition integrity: Single cohesive phenomenon
Definition integrity detail: No
Phenomenon contested: Contested
Task definition: Elicit a harmful response from an LLM
Task item definition: Kind of Behavior + Goal (Query) + Target (affirmative response) + Category + Source.
Task source: Author-crafted task examples (e.g. hand-written examples, manual transformation of existing data into questions), Expert-crafted task examples (e.g. hand-written examples), Modified from another benchmark (e.g. translation into another language)
Task source detail: only 55% of data points are novel. the others are copied.
Task ecology: Complete real task (e.g. providing medical advice to real people interactively)
Dataset size: 100 harmful 100 benign
Dataset size notes: Judge validation data 300 rows.
Dataset metadata available: Yes
Dataset metadata detail: Source for each item
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically)
Split reviewed: Test, Validation
Response format: Free response (e.g. summary paragraph, executable code)
Metric definition: LLM-as-a-Judge (text quality, preferences, NOT extracting answers for other metrics)
Metric definition detail: Attack Success Rate: What percentage of items have at least one response scored "harmful".
Metric aggregation: Simple Mean
Metric statistics: simple mean. no inferential statistics (even though the LLM-as-judge have fairly low accuracy).
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: Yes
Authorship: Mix (multiple authors from industry and academia)
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: Yes
Results comparison explanation: No
Results realism: Yes
Human baseline: No
Authors discuss validity: No
Taxonomy: Alignment / Alignment
New taxonomy: General Capability / Alignment