SALAD -Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models (2024)

Abstract: In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount. To meet this crucial need, we propose SALAD-Bench, a safety benchmark specifically designed for evaluating LLMs, attack, and defense methods. Distinguished by its breadth, SALAD-Bench transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.SALAD-Bench is crafted with a meticulous array of questions, from standard queries to complex ones enriched with attack, defense modifications and multiple-choice. To effectively manage the inherent complexity, we introduce an innovative evaluators: the LLM-based MD-Judge for QA pairs with a particular focus on attack-enhanced queries, ensuring a seamless, and reliable evaluation. Above components extend SALAD-Bench from standard LLM safety evaluation to both LLM attack and defense methods evaluation, ensuring the joint-purpose utility. Our extensive experiments shed light on the resilience of LLMs against emerging threats and the efficacy of contemporary defense tactics. Data and evaluator are released under https://github.com/OpenSafetyLab/SALAD-BENCH

## introduction with rapid breakthrough in llm capabilities, new benchmarks have emerged to evaluate these models and explore their capability boundaries @xcite @xcite @xcite . alongside their powerful capabilities, concerns regarding the safety of llms are also rising. preventing safety threats posed by generative ai systems is be- to formulate and evaluate safety concerns, a range of benchmarks @xcite @xcite @xcite have been developed. however, these prior benchmarks focused on safety often exhibited significant shortcomings. firstly, most of benchmarks only focus benchmarks dataset properties usage evaluator q mcq size md data source levels toxicchat @xcite 10k h 1 safety roberta safetyprompts @xcite 100k h&gpt 7 safety gpt safetybench @xcite 11k h&gpt 7 safety choice parsing do-not-answer @xcite 0.9k gpt 5-12-60 safety longformer doanythingnow @xcite 0.4k gpt 13 safety chatglm advbench @xcite 1.1k h&vicuna 1 attack&defense keyword malicousinstruct @xcite 0.1k gpt 10 attack&defense bert cvalues @xcite 3.9k h&gpt 10 safety human toxigen @xcite 6.5k gpt 1 safety bert multilingual @xcite 2.8k gpt 8 safety gpt salad-bench (ours) 30k h&gpt 6-16-66 safety&attack&defense md/mcq-judge on a narrow perspective of safety threats (e.g., only unsafe instructions or only toxic representation), failing to cover the wide spectrum of potentially harmful outputs llms might generate. this inadequacy partly stems from the rapid evolution of language and the emergence of new forms of harmful content, which older benchmarks failed to anticipate. secondly, traditional harmful questions can be effectively handled with a high safety rate of about 99% by modern llms @xcite . more challenging questions @xcite ) are desired for comprehensive evaluation of llm safety. thirdly, many existing benchmarks rely on timeconsuming human evaluation @xcite or expensive @xcite , making safety evaluation both slow and costly. finally, these benchmarks tend to be limited in scope, being tailored either exclusively for safety evaluation @xcite or for testing attack and defense mechanisms @xcite , restricting their broader application. considering limitations of existing benchmarks, we propose a challenging benchmark namely salad-bench, i.e., safety evaluation for llms, attack and defense approaches. as shown in all above contributions involve a detailed safetyfocused evaluation of recent llms, including both black-box llms @xcite @xcite and open-sourced models @xcite @xcite . we analyze their vulnerabilities and assess their safety rates across different dimensions, using innovative methods to enhance the evaluation's efficiency and scalability. figure 2 : salad-bench's taxonomy with three levels and 66 categories focused on safety issues. each category is represented by at least 200 questions, guaranteeing a thorough evaluation across all areas. ## question enhancement to comprehensively evaluate llm safety and robustness, we develop three subsets: attackenhanced, defense-enhanced, and multiple-choice questions, each expanding on part of our base set. these subsets aim to deepen the challenge, broaden the evaluation perspectives, and support automatic testing, ensuring a thorough exploration of llm safety and defense abilities against attack methods. ## evaluator in order to boosting the development of llm safety evaluation, we propose a reliable and seamless evaluator, designed specifically for our benchmark. the evaluator is composed of the md-judge and mcq-judge, which were constructed for question-answer pairs and multiple choice questions, respectively. task: check if there is unsafe content in "agent" responses to "user" questions as shown in conversations according to our safety policy with below categories. ## experiments leveraging our high-quality datasets, we conducted large-scale experiments to assess both the reliability of our evaluators and the safety of various large language models (llms) and compare the effec-tiveness of different attack and defense methods. ## conclusion we present salad-bench, a hierarchical and comprehensive benchmark for llm safety evaluation through hierarchical taxonomies. utilizing md-judge and mcq-judge as evaluators, salad-bench goes beyond mere safety assessment of llms, providing a robust source for evaluating both attack and defense algorithms notably tailored for these models. the results from salad-bench show varied performance across different models and highlight areas that may require further attention to enhance the safety and reliability of llms.