When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with
  Benign Inputs
Authors: name: Bodam Kim, name: Hiskias Dingeto, name: Taeyoun Kwon, name: Dasol Choi, name: DongGeon Lee, name: Haon Park, name: JaeHoon Lee, name: Jongho Shin
Organization: AIM Intelligence

As large language models become increasingly integrated into daily life,
audio has emerged as a key interface for human-AI interaction. However, this
convenience also introduces new vulnerabilities, making audio a potential
attack surface for adversaries. Our research introduces WhisperInject, a
two-stage adversarial audio attack framework that can manipulate
state-of-the-art audio language models to generate harmful content. Our method
uses imperceptible perturbations in audio inputs that remain benign to human
listeners. The first stage uses a novel reward-based optimization method,
Reinforcement Learning with Projected Gradient Descent (RL-PGD), to guide the
target model to circumvent its own safety protocols and generate harmful native
responses. This native harmful response then serves as the target for Stage 2,
Payload Injection, where we use Projected Gradient Descent (PGD) to optimize
subtle perturbations that are embedded into benign audio carriers, such as
weather queries or greeting messages. Validated under the rigorous
StrongREJECT, LlamaGuard, as well as Human Evaluation safety evaluation
framework, our experiments demonstrate a success rate exceeding 86% across
Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and Phi-4-Multimodal. Our work demonstrates a
new class of practical, audio-native threats, moving beyond theoretical
exploits to reveal a feasible and covert method for manipulating AI behavior.