Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarios (2025)

Abstract: Multimodal large language models (MLLMs) are rapidly evolving, presenting increasingly complex safety challenges. However, current dataset construction methods, which are risk-oriented, fail to cover the growing complexity of real-world multimodal safety scenarios (RMS). And due to the lack of a unified evaluation metric, their overall effectiveness remains unproven. This paper introduces a novel image-oriented self-adaptive dataset construction method for RMS, which starts with images and end constructing paired text and guidance responses. Using the image-oriented method, we automatically generate an RMS dataset comprising 35,610 image–text pairs with guidance responses. Additionally, we introduce a standardized safety dataset evaluation metric: fine-tuning a safety judge model and evaluating its capabilities on other safety datasets. Extensive experiments on various tasks demonstrate the effectiveness of the proposed image-oriented pipeline. The results confirm the scalability and effectiveness of the image-oriented approach, offering a new perspective for the construction of real-world multimodal safety datasets.

## introduction multimodal large language models (mllms) @xcite @xcite are increasingly permeating various aspects of human life. as the applications of mllms continue to expand, concerns regarding their safety have also been heightened @xcite @xcite . to tackle the safety issues of mllms, a multitude of benchmarks have been proposed, thus facilitating the development of safer mllms @xcite * equal contribution † corresponding author play, run, dance … here fail to identify the risk and give encouragement. ## related work the majority of current research on the safety of multimodal model is still based on the safety studies of unimodal models @xcite . for example, adversarial attacks on images @xcite @xcite in a multimodal setting or the use of prompt jailbreak techniques @xcite @xcite to bypass constraints. further research has combined these attack methods for text and images into a comprehensive framework @xcite @xcite . while these methods have significantly improved the safety of multimodal models, they overlook the unique challenges inherent in multimodal scenarios. such studies highly rely on synthetic images, lack sensitivity to real-world sce-narios, and the preset risks make it impossible to adapt to more complicated real-world scenarios. recently, a novel multimodal safety scenario has been proposed named safe input but unsafe output @xcite @xcite . building on this, we propose the real-world multimodal safety scenarios (rms). in this scenario, each modality's input is individually safe, but their combination creates a latent harmful context, while the image is also guaranteed from real-world images. since all inputs are harmless, the guidance responses for these data are easy to construct for enhancing mllms' safety judge ability. however, current rms-related datasets face challenges such as difficult construction, high human involvement, and limited data volume, making them unsuitable for actual model safety judge alignment. existing safety benchmarks can be categorized into two types based on whether the image or text is significantly harmful, and the sources of the images, as shown in ## experiments the experiments on image-oriented method are conducted from three perspectives, named incremental experiments, safety judgment experiments, and fine-tuned model as metric. all experiments were conducted in a single run. ## conclusion we introduce a novel image-oriented approach that identifies risks from real-world images and generates data accordingly. this method addresses two key limitations in current multimodal safety datasets: (1) the over-reliance on synthetic data, which lacks sensitivity to real-world images, and (2) the risk-oriented nature, which lacks adaptability to more complex risk conditions. the image-oriented approach offers a new perspective for constructing multimodal safety datasets. we develop an adaptively expanded rms dataset based on the input image dataset, offers a complete automatic data chain. additionally, to address the lack of a unified evaluation standard for multimodal safety datasets, we propose a new assessment metric. experiments across various tasks demonstrate that current mainstream mllms struggle to recognize safety risks in rms, and fine-tuning mllms using rms dataset yields better safety judgment results. these confirm that rms effectively challenges the safety capabilities of mllms.