SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song
  Generation
Authors: name: Zihan Liu, name: Shuangrui Ding, name: Zhixiong Zhang, name: Xiaoyi Dong, name: Pan Zhang, name: Yuhang Zang, name: Yuhang Cao, name: Dahua Lin, name: Jiaqi Wang

Text-to-song generation, the task of creating vocals and accompaniment from
textual inputs, poses significant challenges due to domain complexity and data
scarcity. Existing approaches often employ multi-stage generation procedures,
resulting in cumbersome training and inference pipelines. In this paper, we
propose SongGen, a fully open-source, single-stage auto-regressive transformer
designed for controllable song generation. The proposed model facilitates
fine-grained control over diverse musical attributes, including lyrics and
textual descriptions of instrumentation, genre, mood, and timbre, while also
offering an optional three-second reference clip for voice cloning. Within a
unified auto-regressive framework, SongGen supports two output modes: mixed
mode, which generates a mixture of vocals and accompaniment directly, and
dual-track mode, which synthesizes them separately for greater flexibility in
downstream applications. We explore diverse token pattern strategies for each
mode, leading to notable improvements and valuable insights. Furthermore, we
design an automated data preprocessing pipeline with effective quality control.
To foster community engagement and future research, we will release our model
weights, training code, annotated data, and preprocessing pipeline. The
generated samples are showcased on our project page at
https://liuzh-19.github.io/SongGen/ , and the code will be available at
https://github.com/LiuZH-19/SongGen .