S peech G uard: Exploring the Adversarial Robustness of Multi-modal Large Language Models (2024)

Abstract: Integrated Speech and Large Language Models (SLMs) that can follow speech instructions and generate relevant text responses have gained popularity lately. However, the safety and robustness of these models remains largely unclear. In this work, we investigate the potential vulnerabilities of such instruction-following speech-language models to adversarial attacks and jailbreaking. Specifically, we design algorithms that can generate adversarial examples to jailbreak SLMs in both white-box and black-box attack settings without human involvement. Additionally, we propose countermeasures to thwart such jailbreaking attacks. Our models, trained on dialog data with speech instructions, achieve state-of-the-art performance on spoken question-answering task, scoring over 80% on both safety and helpfulness metrics. Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories. However, we demonstrate that our proposed countermeasures reduce the attack success significantly.

## introduction as large language models (llms) obtain broad and diverse capabilities, it is imperative to understand and mitigate any potential harm caused by them, * equal contributions † corresponding author : raghperi@amazon.com 1 content warning: this paper contains examples of harmful language that might be disturbing to some readers. as well as prevent their misuse by malicious actors @xcite @xcite . llm developers have begun to train models explicitly for "safety alignment" to deter them from producing unsafe responses @xcite . however, these llms have been found to be susceptible to adversarial attacks, where carefully crafted perturbations on the prompts were used to jailbreak the models' safety training @xcite . more recently, visual language models (vlms) have also been shown to be vulnerable to such attacks, where the attacks are performed on the image modality @xcite . in this work, we investigate the vulnerability of speech language models' @xcite @xcite safety guardrails against adversarial perturbations of the input speech signal, and explore countermeasures against such attacks. in particular, we assess speech language models (slms) through the lens of spoken questionanswering (spoken qa) task, and investigate jailbreaking their safety guardrails. we develop an evaluation framework to systematically analyze the safety alignment of slms, while also considering their overall utility (helpfulness) and the relevance of the produced responses to the question. we perform extensive experiments using different adversarial threat scenarios, including white-box and transfer-based attacks. we show that a malicious adversary with full (white-box) access to a slm's gradients can jailbreak its safety training using barely perceptible perturbations on the input audio. though weaker than white-box attacks, we also demonstrate that perturbations generated using one model transfer to other models, and that different model architectures show different levels of vulnerability. we further propose countermeasures against the adversarial threats and show that adding random noise (a simplified version of randomized smoothing defense @xcite ) can provide reasonable robustness against the attacks. we summarize our contributions below: 1. to our knowledge, this is the first study examining the potential safety limitations of unified speech and language models for jailbreaking. ## attacks & countermeasures amongst the various applications for slms, in this work, we showcase adversarial attacks and defense techniques for the task of spoken qa. we follow the well-established literature in adversarial attacks and defenses, and adapt those techniques for the purpose of jailbreaking slms. specifically, we explore two different attack scenarios: white-box and black-box, similar to the approach in @xcite , and consequently identify safeguarding countermeasures. ## results & discussion in this section, we first analyze the safety alignment of several slms followed by the results of samplespecific and transfer-based attacks, and also show the effectiveness of the tdnf defense. ## conclusion we present a detailed study of the safety alignment of speech language models through the lens of spoken qa application. we investigate the robustness of several in-house models along with public models in light of adversarial attacks. to accurately determine the safety alignment of these models, we developed a comprehensive evaluation setup using a publicly available llm. through extensive experiments, we demonstrate that an adversary with white-box access to the systems can jailbreak them using barely perceptible perturbations, and force them to ignore their safety alignment training. 14 see appendix a.6 for details. furthermore, adversarial perturbations generated using one model can jailbreak a different model with reasonable success, with some models exhibiting greater robustness than others. we also showed the effectiveness of a noise-flooding defense in countering the attacks. to the best of our knowledge, this is the first study to investigate the potential safety vulnerability of integrated speech and language models. we believe that with the rapid adoption of such technologies, it is imperative to thoroughly understand the safety implications of these systems. furthermore, it is important to devise effective countermeasures against jailbreaking threats and prevent the models from causing harm. a holistic approach to understanding the safety alignment of systems is required, including studying universal adversarial threats (a single perturbation to jailbreak multiple systems), prompt injection attacks, model poisoning, etc. we hope that this work will serve as a precursor to many such studies.