Title: Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning for Large Audio-Language Models
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Multimodal, Chain-of-Thought, LALM

Abstract: Recent Large Audio-Language Models (LALMs) have shown strong performance on various audio understanding tasks such as speech translation and Audio Q\&A. However, they exhibit significant limitations on challenging audio reasoning tasks in complex acoustic scenarios. These situations would greatly benefit from the use of acoustic tools like noise suppression, source separation, and precise temporal alignment, but current LALMs lack access to such tools. To address this limitation, we introduce $\textbf{Thinking-with-Sound}$ (TwS), a framework that equips LALMs with Audio CoT by combining linguistic reasoning with on-the-fly audio-domain analysis. Unlike existing approaches that treat audio as static input, TwS enables models to actively $\textit{think}$ with audio signals, performing numerical analysis and digital manipulation through multimodal reasoning. To evaluate this approach, we construct $\textbf{MELD-Hard1k}$, a new robustness benchmark created by introducing various acoustic perturbations. Experiments reveal that state-of-the-art LALMs suffer dramatic performance degradation on MELD-Hard1k, with accuracy dropping by more than 50\% compared to clean audio. TwS achieves substantial improvements in robustness, demonstrating both effectiveness and scalability: small models gain 24.73\% absolute accuracy, with improvements scaling consistently up to 36.61\% for larger models. Our findings demonstrate that Audio CoT can significantly enhance robustness without retraining, opening new directions for developing more robust audio understanding systems.