JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language
  Model
Authors: name: Yi Nian, name: Shenzhe Zhu, name: Yuehan Qin, name: Li Li, name: Ziyi Wang, name: Chaowei Xiao, name: Yue Zhao

Multimodal large language models (MLLMs) excel in vision-language tasks but
also pose significant risks of generating harmful content, particularly through
jailbreak attacks. Jailbreak attacks refer to intentional manipulations that
bypass safety mechanisms in models, leading to the generation of inappropriate
or unsafe content. Detecting such attacks is critical to ensuring the
responsible deployment of MLLMs. Existing jailbreak detection methods face
three primary challenges: (1) Many rely on model hidden states or gradients,
limiting their applicability to white-box models, where the internal workings
of the model are accessible; (2) They involve high computational overhead from
uncertainty-based analysis, which limits real-time detection, and (3) They
require fully labeled harmful datasets, which are often scarce in real-world
settings. To address these issues, we introduce a test-time adaptive framework
called JAILDAM. Our method leverages a memory-based approach guided by
policy-driven unsafe knowledge representations, eliminating the need for
explicit exposure to harmful data. By dynamically updating unsafe knowledge
during test-time, our framework improves generalization to unseen jailbreak
strategies while maintaining efficiency. Experiments on multiple VLM jailbreak
benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in
harmful content detection, improving both accuracy and speed.