SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large
  Language Models
Authors: name: Raghuveer Peri, name: Sai Muralidhar Jayanthi, name: Srikanth Ronanki, name: Anshu Bhatia, name: Karel Mundnich, name: Saket Dingliwal, name: Nilaksh Das, name: Zejiang Hou, name: Goeric Huybrechts, name: Srikanth Vishnubhotla, name: Daniel Garcia-Romero, name: Sundararajan Srinivasan, name: Kyu J Han, name: Katrin Kirchhoff

Integrated Speech and Large Language Models (SLMs) that can follow speech
instructions and generate relevant text responses have gained popularity
lately. However, the safety and robustness of these models remains largely
unclear. In this work, we investigate the potential vulnerabilities of such
instruction-following speech-language models to adversarial attacks and
jailbreaking. Specifically, we design algorithms that can generate adversarial
examples to jailbreak SLMs in both white-box and black-box attack settings
without human involvement. Additionally, we propose countermeasures to thwart
such jailbreaking attacks. Our models, trained on dialog data with speech
instructions, achieve state-of-the-art performance on spoken question-answering
task, scoring over 80% on both safety and helpfulness metrics. Despite safety
guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs
to adversarial perturbations and transfer attacks, with average attack success
rates of 90% and 10% respectively when evaluated on a dataset of carefully
designed harmful questions spanning 12 different toxic categories. However, we
demonstrate that our proposed countermeasures reduce the attack success
significantly.