Audio Is the Achilles’ Heel: Red Teaming Audio Large Multimodal Models (2025)

Abstract: Large Multimodal Models (LMMs) have demonstrated the ability to interact with humans under real-world conditions by combining Large Language Models (LLMs) and modality encoders to align multimodal information (visual and auditory) with text. However, such models raise new safety challenges of whether models that are safety-aligned on text also exhibit consistent safeguards for multimodal inputs. Despite recent safety-alignment research on vision LMMs, the safety of audio LMMs remains under-explored. In this work, we comprehensively red team the safety of five advanced audio LMMs under three settings: (i) harmful questions in both audio and text formats, (ii) harmful questions in text format accompanied by distracting non-speech audio, and (iii) speech-specific jailbreaks. Our results under these settings demonstrate that open-source audio LMMs suffer an average attack success rate of 69.14% on harmful audio questions, and exhibit safety vulnerabilities when distracted with non-speech audio noise. Our speech-specific jailbreaks on Gemini-1.5-Pro achieve an attack success rate of 70.67% on the harmful query benchmark. We provide insights on what could cause these reported safety-misalignments. Warning: this paper contains offensive examples.

## introduction large language models (llms) @xcite have demonstrated remarkable abilities to interact with humans through text. to further extend the application of these models to real-world settings, recent research has developed large multimodal models (lmms) @xcite @xcite by jointly training modality encoders with llms, enabling them to understand visual and auditory information. however, introducing additional modalities raises new safety concerns regarding the impact of multimodal inputs on safety alignment. additionally, it is unknown whether such lmms' safeguards, preventing harmful generation or jailbreak attacks, are as reliable as their text-only counterparts (i.e., their llm backbones). red teaming strategies @xcite @xcite @xcite reveal vulnerabilities in models, leading to design and development of corresponding defence measures @xcite @xcite . despite significant progress in the vision and text domains, red teaming with respect to audio modality remains under-explored. to address this gap, in this paper we comprehensively red team the safety of five advanced audio lmms: qwen-audio @xcite , qwen2-audio @xcite , salmonn-7b, salmonn-13b @xcite , and gemini-1.5-pro @xcite . 2 we assess these lmms safeguards against (1) harmful questions in audio and text format, (2) harmful questions in text format along with various distracting audio present, and (3) speech-specific jailbreaking. to the best of our knowledge, this is the first work to systematically red team the safety of audio lmms. our core findings are summarized as follows: ## related work red teaming strategies are commonly employed to evaluate the safety of models and provide insights by benchmarking llms/lmms using plain harmful questions. do-not-answer @xcite proposed a risk taxonomy with five categories to evaluate the refusal ability of llms. saladbench @xcite provided a large-scale taxonomy covering standard queries, multiple-choice questions, and a series of methods to assess llms. similarly, safetybench @xcite generated 11,435 multiple-choice questions based on seven safety categories. rules @xcite proposed an evaluation scenario that red teams llms' ability to maintain consistency with safety policies. safetyprompts @xcite and cvalues @xcite provided safety insights on chinese llms. in the domain of vision lmms, mllmgurad @xcite introduced an image-text dataset with five safety diinstruction jailbreak and adversarial attacks simulate attacks from malicious users to probe the vulnerabilities. instruction jailbreak typically emphasises guiding the model's inference under blackbox conditions to trigger the generation of harmful responses. paps @xcite humanised llms and induced the generation of harmful responses through proposed persuasion techniques. deepinception @xcite carefully designed indirect scenarios and nested prompts to confuse models. cognitive overload @xcite bypassed defence measures based on the cognitive structure of llms. in multimodal settings, attackers usually conceal harmful content within multimodal information to evade the safeguards. figstep @xcite transformed harmful content into images using typographic techniques to elicit harmful responses. adversarial attacks, under white-box conditions, cause safety misalignment by attaching optimised perturbations to inputs for shifting the model representation space. speechguard @xcite explored the adversarial attacks on speech and the robustness of models. gcg @xcite generated prompt suffixes based on gradients to achieve universal and transferable attacks. linkprompt @xcite bypassed suffix detection by maintaining coherence between tokens. @xcite proposed using harmful adversarial images as input to jailbreak vision lmms. bap @xcite and umk @xcite introduced a dual-modality adversarial attack that simultaneously attaches perturbations and suffixes to visual and text inputs to trigger risks. existing red teaming strategies have made significant progress in evaluating llms and vision lmms, while the safety of audio lmms remains under-explored. in this work, we comprehensively red team five advanced audio lmms and also explore a speech-specific jailbreak strategy to reveal the vulnerabilities of audio lmms for promoting the development of corresponding defence mechanisms. ## probing safety alignment we evaluate the safety alignment of audio lmms against prohibited harmful question. we first describe our red teaming configurations ( §3.1). next, we report the results of our red teaming experiments conducted on five audio lmms, and provide the comparison with their corresponding llm backbones ( §3.2). lastly, we provide further analysis of the lmms' safety and representation spaces to gain deeper insights ( §3.3). ## exploring non-speech audio input we explore the impact of introducing meaningless non-speech audio input (e.g., noise) on the safeguards of audio lmms. we first introduce the non-speech audio and prompting strategies ( §4.1). next, we report the red teaming results on four open-sourced audio lmms ( §4.2). lastly, we analyse the influence of non-speech audio on the representation space and safety alignment of audio lmms ( §4.3). ## speech-specific jailbreak as observed in ## conclusion in this paper, we red team five advanced audio lmms safeguards against (1) harmful questions in audio and text format, (2) harmful questions in text format accompanied by distracting non-speech audio, and (3) speech-specific jailbreaking. our experimental results demonstrate that gemini-1.5-pro, benefiting from its guardrail, exhibits a very reliable safeguards against plain harmful questions.