Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for
  Speech Generation
Authors: name: Haorui He; name: Zengqiang Shang; name: Chaoren Wang; name: Xuyuan Li; name: Yicheng Gu; name: Hua Hua; name: Liwei Liu; name: Chen Yang; name: Jiaqi Li; name: Peiyang Shi; name: Yuancheng Wang; name: Kai Chen; name: Pengyuan Zhang; name: Zhizheng Wu
Published: 2025-01-27 09:59:20

Abstract: Recent advancements in speech generation have been driven by the large-scale
training datasets. However, current models fall short of capturing the
spontaneity and variability inherent in real-world human speech, due to their
reliance on audiobook datasets limited to formal read-aloud speech styles. To
bridge this gap, we introduce Emilia-Pipe, an open-source preprocessing
pipeline to extract high-quality training data from valuable yet underexplored
in-the-wild data that capture spontaneous human speech in real-world contexts.
By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech
generation dataset derived from in-the-wild speech data. This dataset comprises
over 101k hours of speech across six languages: English, Chinese, German,
French, Japanese, and Korean. Besides, we expand Emilia to Emilia-Large, a
dataset exceeding 216k hours, making it the largest open-source speech
generation dataset available. Extensive experiments demonstrate that Emilia
significantly outperforms traditional audiobook datasets in generating
spontaneous and human-like speech, showcasing superior performance in capturing
diverse speaker timbre and speaking styles of real-world human speech.
Furthermore, this work underscores the importance of scaling dataset size to
advance speech generation research and validates the effectiveness of Emilia
for both multilingual and crosslingual speech generation.