Unified Audio Intelligence Without Regressing on Text Intelligence
Authors: name: Zhifeng Kong, name: Sang-gil Lee, name: Jaehyeon Kim, name: Boxin Wang, name: Zihan Liu, name: Sungwon Kim, name: Yang Chen, name: Arushi Goel, name: Rajarshi Roy, name: Wenliang Dai, name: Zhuolin Yang, name: Yangyi Chen, name: Dongfu Jiang, name: Sreyan Ghosh, name: Tuomas Rintamaki, name: Andrew Tao, name: Jonathan Raiman, name: Mohammad Shoeybi, name: Bryan Catanzaro, name: Wei Ping
Organization: NVIDIA

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.