Multimodal Situational Safety
Authors: name: Kaiwen Zhou, name: Chengzhi Liu, name: Xuandong Zhao, name: Anderson Compalas, name: Dawn Song, name: Xin Eric Wang

Multimodal Large Language Models (MLLMs) are rapidly evolving, demonstrating
impressive capabilities as multimodal assistants that interact with both humans
and their environments. However, this increased sophistication introduces
significant safety concerns. In this paper, we present the first evaluation and
analysis of a novel safety challenge termed Multimodal Situational Safety,
which explores how safety considerations vary based on the specific situation
in which the user or agent is engaged. We argue that for an MLLM to respond
safely, whether through language or action, it often needs to assess the safety
implications of a language query within its corresponding visual context. To
evaluate this capability, we develop the Multimodal Situational Safety
benchmark (MSSBench) to assess the situational safety performance of current
MLLMs. The dataset comprises 1,820 language query-image pairs, half of which
the image context is safe, and the other half is unsafe. We also develop an
evaluation framework that analyzes key safety aspects, including explicit
safety reasoning, visual understanding, and, crucially, situational safety
reasoning. Our findings reveal that current MLLMs struggle with this nuanced
safety problem in the instruction-following setting and struggle to tackle
these situational safety challenges all at once, highlighting a key area for
future research. Furthermore, we develop multi-agent pipelines to coordinately
solve safety challenges, which shows consistent improvement in safety over the
original MLLM response. Code and data: mssbench.github.io.