I nstruct S afety: A Unified Framework for Building Multidimensional and Explainable Safety Detector through Instruction Tuning (2023)

Abstract: Safety detection has been an increasingly important topic in recent years and it has become even more necessary to develop reliable safety detection systems with the rapid development of large language models. However, currently available safety detection systems have limitations in terms of their versatility and interpretability. In this paper, we first introduce InstructSafety, a safety detection framework that unifies 7 common sub-tasks for safety detection. These tasks are unified into a similar form through different instructions. We then conduct a comprehensive survey of existing safety detection datasets and process 39 human-annotated datasets for instruction tuning. We also construct adversarial samples to enhance the model’s robustness. After fine-tuning Flan-T5 on the collected data, we have developed Safety-Flan-T5, a multidimensional and explainable safety detector. We conduct comprehensive experiments on a variety of datasets and tasks, and demonstrate the strong performance of Safety-Flan-T5 in comparison to supervised baselines and served APIs (Perspective API, ChatGPT and InstructGPT). We will release the processed data, fine-tuned Safety-Flan-T5 and related code for public use.

## introduction safety detection has gained increased attention in recent years @xcite @xcite , which is important in many scenarios (e.g., detecting unsafe comments in online forums or under public videos). due to the rapid development of large language models and the growing amount of human-ai interactions, it is becoming increasingly necessary to develop effective safety detection systems that can ensure safe and reliable interactions between humans and ai. however, currently available safety detectors (e.g., perspective api) have several limitations: (1) lack it implies being black is a bad thing. ## conclusion in this work, we present instructsafety, a unified framework that integrates 7 common sub-tasks of safety detection into a uniform formulation. in addition to collecting public datasets, we also create adversarial samples to improve the model's robustness. on top of these data, we train three versions of safety-flan-t5, a multidimensional and explainable safety detector. extensive experiments demonstrate the versatility, accuracy and explainability of safety-flan-t5. our ablation studies further confirm that employing our robust enhancement method and increasing the number of datasets can improve the model's robustness and generalization ability. although safety-flan-t5 outperforms p-api and gpt3.5 in most of the experiments, there might be some room to improve the performance of gpt3.5 through more refined prompts, which is left as future work. we hope safety-flan-t5 will be helpful to communities in performing safety detection more effectively and efficiently in various scenarios.