# Semi-Supervised Learning for Robust Emotional Speech Synthesis with Limited Data
## 4 Experiment and Results
### Subjective Evaluation  
In order to measure the naturalness and emotional expression of the synthesized speech, this paper gives 15 listeners 20 speech items of each emotion in random order and scores them using the mean opinion score MOS and the EMOS (emotion mean opinion score). The scores ranged from 1 to 5, representing the poor to good quality of the synthesized speech, with a span of 0.5 points. The specific details of the evaluation criteria are shown in Table 3.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**ESD (English Subset)** & **Number of Utterances** \\ \hline Evaluation set & 20 \\ Test set & 30 \\ Training set & 300 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Partition of ESD dataset.  
The XAB preference test was also conducted. The scoring criterion was to compare which of the baseline method and the proposed method synthesized speech emotion closer to the emotion of the target audio. In all tests, 15 local listeners were asked to give their personal preferences for 20 synthesized emotional speeches randomly of each emotion selected from the test set, based on their subjective feelings.  
For the trade-off parameter with the introduction of pseudo-label loss, a comparison of the values taken for different data sizes was made for the conditions of 30, 100, and 300 utterances, respectively, in which the performance of the majority of utterances did not exceed the baseline. Therefore, by introducing the reference loss and setting reasonable trade-off parameters, the generalization ability of the fine-tuning model can be achieved to some extent. When the value of the trade-off parameter is too large, it will result in the proposed fine-tuning method will be forced to output the same close Mel-spectrum as the initial reference model, and the excessive penalty will give it the problem of underfitting, \(\theta\) taking 0.1, 0.3 and the zero-value as the baseline in which the pseudo-label loss is not involved in the effect of the results at all The MOS values for evaluating the naturalness of speech and the word error rate (WER) for responding to robustness are shown in Table 4. The final synthetic MOS values for each emotion are shown in Table 5, the EMOS scores are shown in Figure 4, and the results of the XAB preference test are shown in Figure 5. By analyzing the values of the subjective measures, it can be seen that the MOS and EMOS scores of SMAL-ET2 are higher when compared with the baseline method. The method yields better emotional expressiveness.  
\begin{table}
\begin{tabular}{c c} \hline
**Hyperparameters** & **Settings** \\ \hline Adam optimizer \(\beta_{1}\) & 0.9 \\ Adam optimizer \(\beta_{2}\) & 0.999 \\ Batch\_size & 32 \\ Initial learning rate & 0.002 \\ Finetune learning rate & 0.00002 \\ Decay learning rate & True \\ Sample rate & 22,050 \\ hidden vectors of SMA & 128 \\ Attention dropout & 0.4 \\ Decoder dropout & 0.5 \\ zoneout & 0.1 \\ epochs & 1000 \\ \hline \end{tabular}
\end{table}
Table 2: Settings of the Hyperparameters.  
\begin{table}
\begin{tabular}{c c c} \hline
**Score** & **MOS Standards** & **EMOS Standards** \\ \hline
0–1.0 & Very poor sound quality, difficult to understand; & \\  & large latency, poor communication & \\  & General sound quality, not very clear to hear; large & \\
1.0–2.0 & delay, not clear, not smooth; communication needs & \\  & to be repeated several times & \\
2.0–3.0 & Sound quality is not bad, more audible; there is a & \\  & certain delay, there is noise, but acceptable & \\
3.0–4.0 & Sound quality is very good, you can hear clearly; & \\  & only a little delay; willing to accept & \\
4.0–5.0 & Sound quality is particularly good, very clear and & \\  & natural; almost no delay, smooth communication & Ideal emotional similarity \\ \hline \end{tabular}
\end{table}
Table 3: Evaluation score standards of MOS and EMOS.