AudioStory: Generating Long-Form Narrative Audio with Large Language
  Models
Authors: name: Yuxin Guo, name: Teng Wang, name: Yuying Ge, name: Shijie Ma, name: Yixiao Ge, name: Wei Zou, name: Ying Shan

Recent advances in text-to-audio (TTA) generation excel at synthesizing short
audio clips but struggle with long-form narrative audio, which requires
temporal coherence and compositional reasoning. To address this gap, we propose
AudioStory, a unified framework that integrates large language models (LLMs)
with TTA systems to generate structured, long-form audio narratives. AudioStory
possesses strong instruction-following reasoning generation capabilities. It
employs LLMs to decompose complex narrative queries into temporally ordered
sub-tasks with contextual cues, enabling coherent scene transitions and
emotional tone consistency. AudioStory has two appealing features: (1)
Decoupled bridging mechanism: AudioStory disentangles LLM-diffuser
collaboration into two specialized components, i.e., a bridging query for
intra-event semantic alignment and a residual query for cross-event coherence
preservation. (2) End-to-end training: By unifying instruction comprehension
and audio generation within a single end-to-end framework, AudioStory
eliminates the need for modular training pipelines while enhancing synergy
between components. Furthermore, we establish a benchmark AudioStory-10K,
encompassing diverse domains such as animated soundscapes and natural sound
narratives. Extensive experiments show the superiority of AudioStory on both
single-audio generation and narrative audio generation, surpassing prior TTA
baselines in both instruction-following ability and audio fidelity. Our code is
available at https://github.com/TencentARC/AudioStory