# MixDiff-TTS: Mixture Alignment and Diffusion Model for Text-to-Speech
## 4 Discussion  
In the evaluation experiments, we observed that MixDiff-TTS has higher \(F_{0}\) RMSE values compared with other TTS models. We believe that this may be attributed to the inherent randomness within the generation process of the diffusion model. Although this randomness helps enhance the diversity and naturalness of generated speech, it may also cause slight deviations in the \(F_{0}\) trajectory of synthesized speech compared to real speech. This slight deviation may lead to a higher \(F_{0}\) RMSE. Therefore, it is necessary to further optimize the prosody modeling mechanism in future work to enhance the model's synthesis accuracy in the \(F_{0}\) dimension.  
Additionally, while MixDiff-TTS can synthesize high-quality speech, it has design limitations:  
1. In speech synthesis tasks, multi-speaker speech synthesis models have become a research hotspot to meet the needs of different scenarios. Such models can generate speech with different genders and timbres without retraining for each specific requirement, making them suitable for various applications. However, MixDiff-TTS is only applicable to single-speaker speech synthesis and lacks support for multi-speaker tasks, limiting its applicability in broader scenarios;  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Setting** & **SSIM** & **MCD** & \(\mathbf{F_{0}}\) **RMSE** & **MOS** & **CMOS** \\ \hline MixDiff-TTS & 0.507 & 6.652 & 0.337 & 3.95 \(\pm\) 0.08 & 0.000 \\ MixDiff-TTS—pre-net & 0.501 & 6.661 & 0.372 & 3.93 \(\pm\) 0.07 & \(-\)0.129 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The ablation study on the pre-net.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Setting** & **SSIM** & **MCD** & \(\mathbf{F_{0}}\) **RMSE** & **MOS** & **CMOS** \\ \hline MixDiff-TTS & 0.507 & 6.652 & 0.337 & 3.95 \(\pm\) 0.08 & 0.000 \\ MixDiff-TTS—W2P- & 0.503 & 6.679 & 0.329 & 3.91 \(\pm\) 0.08 & \(-\)0.135 \\ RelPosAttention & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation study of the W2P-RelPosAttention module.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Setting** & **SSIM** & **MCD** & \(\mathbf{F_{0}}\) **RMSE** & **MOS** & **CMOS** \\ \hline MixDiff-TTS & 0.507 & 6.652 & 0.337 & 3.95 \(\pm\) 0.08 & 0.000 \\ MixDiff-TTS—residual network & 0.501 & 6.725 & 0.345 & 3.92 \(\pm\) 0.07 & \(-\)0.131 \\ MixDiff-TTS—post-net & 0.505 & 6.793 & 0.331 & 3.86 \(\pm\) 0.08 & \(-\)0.218 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Ablation study of the post-net and residual network.  
In speech synthesis, fully end-to-end TTS models have emerged as a pivotal research focus, gaining substantial attention in recent years. A fully end-to-end model can directly generate speech waveforms from raw text using a unified architecture, without relying on intermediate feature representations (such as mel-spectrograms). In contrast, MixDiff-TTS generates intermediate feature representations during synthesis and then uses a vocoder to convert them into final speech waveforms. This results in the need for separate training of the acoustic model and vocoder, incurring additional training costs.  
Therefore, based on MixDiff-TTS, developing a fully end-to-end multi-speaker speech synthesis model is necessary.