Audio-FLAN: A Preliminary Release
Authors: name: Liumeng Xue; name: Ziya Zhou; name: Jiahao Pan; name: Zixuan Li; name: Shuai Fan; name: Yinghao Ma; name: Sitong Cheng; name: Dongchao Yang; name: Haohan Guo; name: Yujia Xiao; name: Xinsheng Wang; name: Zixuan Shen; name: Chuanbo Zhu; name: Xinshen Zhang; name: Tianchi Liu; name: Ruibin Yuan; name: Zeyue Tian; name: Haohe Liu; name: Emmanouil Benetos; name: Ge Zhang; name: Yike Guo; name: Wei Xue
Published: 2025-02-23 14:24:15

Abstract: Recent advancements in audio tokenization have significantly enhanced the
integration of audio capabilities into large language models (LLMs). However,
audio understanding and generation are often treated as distinct tasks,
hindering the development of truly unified audio-language models. While
instruction tuning has demonstrated remarkable success in improving
generalization and zero-shot learning across text and vision, its application
to audio remains largely unexplored. A major obstacle is the lack of
comprehensive datasets that unify audio understanding and generation. To
address this, we introduce Audio-FLAN, a large-scale instruction-tuning dataset
covering 80 diverse tasks across speech, music, and sound domains, with over
100 million instances. Audio-FLAN lays the foundation for unified
audio-language models that can seamlessly handle both understanding (e.g.,
transcription, comprehension) and generation (e.g., speech, music, sound) tasks
across a wide range of audio domains in a zero-shot manner. The Audio-FLAN
dataset is available on HuggingFace and GitHub and will be continuously
updated.