MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for
  Zero-Shot Speech Synthesis
Authors: name: Ziyue Jiang; name: Yi Ren; name: Ruiqi Li; name: Shengpeng Ji; name: Boyang Zhang; name: Zhenhui Ye; name: Chen Zhang; name: Bai Jionghao; name: Xiaoda Yang; name: Jialong Zuo; name: Yu Zhang; name: Rui Liu; name: Xiang Yin; name: Zhou Zhao
Published: 2025-02-26 08:22:00

Abstract: While recent zero-shot text-to-speech (TTS) models have significantly
improved speech quality and expressiveness, mainstream systems still suffer
from issues related to speech-text alignment modeling: 1) models without
explicit speech-text alignment modeling exhibit less robustness, especially for
hard sentences in practical applications; 2) predefined alignment-based models
suffer from naturalness constraints of forced alignments. This paper introduces
MegaTTS 3, a TTS system featuring an innovative sparse alignment
algorithm that guides the latent diffusion transformer (DiT). Specifically, we
provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of
alignment without limiting the search space, thereby achieving high
naturalness. Moreover, we employ a multi-condition classifier-free guidance
strategy for accent intensity adjustment and adopt the piecewise rectified flow
technique to accelerate the generation process. Experiments demonstrate that
MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports
highly flexible control over accent intensity. Notably, our system can generate
high-quality one-minute speech with only 8 sampling steps. Audio samples are
available at https://sditdemo.github.io/sditdemo/.