# MixDiff-TTS: Mixture Alignment and Diffusion Model for Text-to-Speech
## 1 Introduction  
Speech Synthesis, or Text-to-Speech (TTS), stands as a core foundational technology in artificial intelligence, enabling the seamless transformation of textual content into natural, human-like speech. At its core, it aims to enable machines to communicate seamlessly and naturally with users through synthesized speech [1, 2, 3]. This technology serves as a core foundation for human-computer interaction systems, playing a pivotal role in intelligent voice applications. Moreover, it has been deeply integrated into diverse real-world scenarios, including in-vehicle navigation systems, assistive reading devices for the visually impaired, and smart speakers. A standard TTS system architecture is composed of three core modules: a text analysis frontend for linguistic processing, an acoustic model for spectral feature generation, and a vocoder for waveform synthesis [4]. Each module operates within a standardized processing pipeline, utilizing acoustic techniques to extract acoustic features and synthesize speech waveforms. First, the text analysis frontend normalizes raw text through tasks such as word segmentation and polyphonic disambiguation, generating structured linguistic representations including phoneme sequences and prosodic annotations. Next, the acoustic model utilizes deep learning algorithms to map abstract linguisticfeatures to continuous acoustic representations (e.g., mel-spectrograms), facilitating the transformation from symbolic language to the acoustic domain. Finally, the vocoder employs waveform synthesis techniques to convert acoustic features into high-fidelity speech waveforms, enabling natural-sounding audible output [5; 6]. Driven by the rapid advancements in deep learning and the proliferation of human-computer interaction scenarios, neural network-based TTS research has garnered significant attention and achieved remarkable progress [7; 8; 9; 10]. During this period, many excellent speech synthesis models have emerged, including WaveNet [11], FastSpeech2 [12], and Glow-TTS [13]. These models have made significant progress in generating natural and fluent human-like speech. Current TTS models excel in speech synthesis but face limitations in phoneme alignment. For instance, the FastSpeech series models employ explicit phoneme-level hard alignment, which relies on precise boundary annotations [12; 14]. However, these annotations are often difficult to obtain. The Tacotron series models adopt an implicit alignment method based on attention mechanisms [15; 16]. This type of alignment can complete phoneme alignment without external annotations, but the model is prone to alignment instability issues during training, such as skips and repetitions. The instability of these alignment methods weakens the expressiveness of synthetic speech. Additionally, the recently proposed DiffSpeech [17] model enables efficient training by optimizing the variational lower bound, generating highly realistic mel-spectrograms that closely match real data distributions. It also enables the synthesis of speech featuring sophisticated variations in expressiveness. However, DiffSpeech uses phoneme-level hard alignment during phoneme alignment, which sometimes results in unnatural-sounding speech. This is mainly because phoneme boundaries are inherently uncertain. For models relying on phoneme-level hard alignment, the phoneme boundaries obtained through this process often lack sufficient precision to meet alignment requirements. Consequently, ambiguous phoneme boundaries during alignment inevitably introduce errors, which degrade the prosodic naturalness of speech synthesis. To address the problem of phoneme sequence alignment and improve the quality of synthesized speech, we propose a non-autoregressive model called MixDiff-TTS. Inspired by advanced TTS models [18], we introduce a linguistic encoder in MixDiff-TTS. To achieve fine-grained phoneme alignment, this encoder innovatively incorporates a mixture alignment mechanism.