S afety B ench: Evaluating the Safety of Large Language Models (2024)

Abstract: With the rapid development of Large Language Models (LLMs), increasing attention has been paid to their safety concerns. Consequently, evaluating the safety of LLMs has become an essential task for facilitating the broad applications of LLMs. Nevertheless, the absence of comprehensive safety evaluation benchmarks poses a significant impediment to effectively assess and enhance the safety of LLMs. In this work, we present SafetyBench, a comprehensive benchmark for evaluating the safety of LLMs, which comprises 11,435 diverse multiple choice questions spanning across 7 distinct categories of safety concerns. Notably, SafetyBench also incorporates both Chinese and English data, facilitating the evaluation in both languages. Our extensive tests over 25 popular Chinese and English LLMs in both zero-shot and few-shot settings reveal a substantial performance advantage for GPT-4 over its counterparts, and there is still significant room for improving the safety of current LLMs. We also demonstrate that the measured safety understanding abilities in SafetyBench are correlated with safety generation abilities. Data and evaluation guidelines are available at https://github.com/thu-coai/SafetyBench. Submission entrance and leaderboard are available at https://llmbench.ai/safety.

## introduction large language models (llms) have gained a growing amount of attention in recent years @xcite . since the release of chatgpt (ope-nai, 2022), more and more llms are deployed to interact with humans, such as llama @xcite , claude @xcite and chat-glm @xcite . however, with the widespread development of llms, their safety flaws are also exposed @xcite , which could significantly hinder the safe * corresponding author. and continuous development of llms. various works have pointed out the safety risks of llms, such as privacy leakage @xcite and toxic generations @xcite . therefore, a thorough assessment of the safety of llms becomes imperative. however, comprehensive benchmarks for evaluating the safety of llms are scarce. in the past, certain widely used datasets have focused exclusively on specific facets of safety concerns such as toxicity @xcite and bias @xcite . notably, some recent chinese safety assessment benchmarks @xcite have gathered prompts spanning various categories of safety issues. however, they only provide chinese data, and a nonnegligible challenge for these benchmarks is how to accurately evaluate the safety of responses generated by llms. some recent works have begun to train advanced safety detectors @xcite , but some unavoidable errors may still occur. manual evaluation, while highly accurate, is a costly and time-consuming process, making it less conducive for rapid model iteration. automatic evaluation is relatively cheaper, but there are few safety classifiers with high accuracy across a wide range of safety problem categories. considering the limitations of existing safety evaluation benchmarks, we introduce safetybench, the first comprehensive benchmark to evaluate llms' safety with multiple choice questions. we present four advantages of safetybench: (1) simplicity and efficiency. in line with well-known benchmarks such as mmlu @xcite , safetybench exclusively features multiplechoice questions, each with a single correct answer, which enables automated and cost-effective evaluations of llms' safety with exceptional accuracy. (2) extensive diversity. safetybench contains 11,435 diverse samples sourced from a wide range of origins, covering 7 distinct categories of safety problems, which provides a comprehensive safetybench offensiveness question: may i ask if the following text contains offensive content? \nall from sichuan, why is my father so disgusting ## safetybench construction an overview of safetybench is presented in figure 1. we collect a total of 11,435 multiple choice questions spanning across 7 categories of safety issues from several different sources. more examples are provided in figure 7 in appendix. next, we will introduce the category breakdown and the data collection process in detail. ## conclusion we introduce safetybench, the first comprehensive safety evaluation benchmark with multiple choice questions. with 11,435 chinese and english questions covering 7 categories of safety issues in safe-tybench, we extensively evaluate the safety abilities of 25 llms from various organizations. we find that open-sourced llms exhibit a significant performance gap compared to gpt-4, indicating ample room for future improvements. we also show the measured safety understanding abilities in safetybench are correlated with safety generation abilities. we hope safetybench could play an important role in evaluating the safety of llms and facilitating the rapid development of safer llms.