## 4 Experiments and Results
### Subjective Evaluation  
Typically, the objective evaluation is useful for training the model but does not reflect the perception of the listener [58]. Thus, we also conducted tests of subjective preference. As these tests were used to investigate the generation of \(F_{0}\) by the models, the spectral parameters were generated using the same model, whereas \(F_{0}\) was generated using different models. To force the listener to concentrate on the generation of \(F_{0}\), the phone duration was obtained from the transcription files because prosody is also dependent on the performance of the duration model.  
A total of 30 native speakers participated in each test. All listeners speak fluently in the central Isarn dialect. In each test, the subjects were asked to listen to 20 pairs of utterances (some samples of synthetic speech with three \(F_{0}\) models are available at [[https://isarn-samp-f0.github.io](https://isarn-samp-f0.github.io)]([https://isarn-samp-f0.github.io](https://isarn-samp-f0.github.io))) randomly selected from the test set, and determined the item in each pair that sounded more natural, or chose a \"no preference\" option if they found the two utterances to be very similar. The order of the speech samples in each pair was swapped. The listeners were allowed to play back the recordings of the  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Number of Parameters (Million)** & **Generation Time (s)** & **RMSE (Hz)** & **CORR** \\ \hline Frame based & 1.04 & 35 & 8.362 \(\pm\) 3.35 & 0.928 \(\pm\) 0.04 \\ Discrete cosine transform (DCT) based & 1.65 & 36 & 8.762 \(\pm\) 2.71 & 0.915 \(\pm\) 0.04 \\ SAMP based & 1.32 & 10 & 8.172 \(\pm\) 2.65 & 0.927 \(\pm\) 0.04 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Objective results of proposed model compared with baseline models.  
utterances as many times as they wanted before assigning a score to them. A \(t\)-test was used to show that the differences between the compared systems were significant (\(p<0.01\)).  
Three preference tests were conducted consisting of comparisons between the frame-based model and DCT-based model, between the SAMP-based model and frame-based model, and between the DCT-based model and SAMP-based model. Figure 6 shows the preference scores of the system pairs. It is clear that the preference score of the frame-based model was lower than those of the DCT-based and SAMP-based models, which were trained using syllable-level features, although the DCT-based model had recorded a poorer performance in the objective evaluation. This indicates that using syllable-level features is effective for learning the complex variations in tonal contours. However, the difference between the frame-based and DCT-based models was not significant (\(p=0.3067\)). Considering the representation of \(F_{0}\), the SAMP-based model was significantly better than the DCT-based model (\(p=0.005\)). This indicates that the sampling-based method can provide a better representation of the \(F_{0}\) contour for the Isarn speech corpus.  
To demonstrate the effect of using the proposed syllable-level features, Figure 7 shows a comparison of the reference  
to 275). Figure 7b demonstrates inappropriate contours of \(F_{0}\) generated by the DCT-based method (e.g., frame 250 to 300). These results were obtained because the DCT-based method might have generated sub-par values for some coefficients that caused the overall \(F_{0}\) contour to deviate.