# A Preliminary Study of Model-Generated Speech
## 4 Evaluation
### Experimental Results
\begin{table}
\begin{tabular}{l l l l l l l l l l} \hline \hline
**Method** & **seq2 seq** & & & & & & & & & \\ \hline
**Fold** & **Accuracy** & **Accuracy** & **F1-Score** & **Accuracy** & **Accuracy** & **Accuracy** & **Accuracy** & **Accuracy** & **F1-Score** \\  & **(with Tone)** & **(w/o Tone)** & **(with Tone)** & **(w/o Tone)** & **(w/o Tone)** & **F1-Score** & **Accuracy** & **Accuracy** & **Accuracy** & **F1-Score** \\ \hline
0 & 0.6742 & 0.819 & 0.9362 & 0.6471 & 0.8072 & 0.9307 & 0.6778 & 0.8118 & 0.9377 \\ \hline
1 & 0.7023 & 0.8344 & 0.9473 & 0.6624 & 0.8072 & 0.9394 & 0.6950 & 0.8190 & 0.9450 \\ \hline
2 & 0.6561 & 0.8235 & 0.9362 & 0.5891 & 0.7575 & 0.9237 & 0.6579 & 0.8145 & 0.9370 \\ \hline
3 & 0.7032 & 0.8244 & 0.9404 & 0.6326 & 0.7846 & 0.9270 & 0.6959 & 0.8190 & 0.9392 \\ \hline
4 & 0.6706 & 0.8199 & 0.9389 & 0.6308 & 0.7864 & 0.9314 & 0.6715 & 0.8145 & 0.9393 \\ \hline Average & 0.6813 & 0.8242 & 0.9398 & 0.6324 & 0.7886 & 0.9304 & 0.6796 & 0.8157 & 0.9396 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results for the evaluation of the three models in relation to the Chaoshan dataset. are analyzed to understand tone patterns. The interaction between tone and consonants in final positions can lead to checked tones characterized by shorter duration. Accordingly, the subsequent analysis of the results will be conducted based on these delineated elements, encompassing syllables, vowels, consonants, and tones, to provide a comprehensive understanding of the dataset. Table 2 shows the sounds predicted by the model compared to those found in the 2015 Chaoshan dictionary, using the same characters as the 1883 Chaoshan dictionary as inputs. In terms of accuracy, the following syllable structures were predicted (going from high to low): CVE > CV > CSV > CSV > CSV, with the nature of the vowels playing a role. This means that if the vowel is nasalized, a simple syllable structure, CSV, is more accurately predicted than a complex one, such as CSV. However, if the vowel is plain, the accuracy of CVE--where E could be either a consonant or vowel--is greater than that of CV. The syllabified consonants \([\mathrm{m}]\) and \([\mathrm{\SIUnitSymbolMicro]}\) are the least accurate. Thus, we can further examine the distribution of different vowels when following the sonorant sequence of CGVE, where V stands for the peak of the syllable. The results showed that regardless of the vowel's nature, the prediction of nasalized vowels was less accurate than for oral vowels. More specifically, the front vowels were easier to correctly predict than the back vowels. One should also note the performance of \(\mathrm{e}[\mathrm{m}]\) and \(\mathrm{e}[\mathrm{\SIUnitSymbolMicro]}\), with approximately the same accuracy of 54%, and \([\mathrm{\SIUnitSymbolMicro]}\), with no accuracy. The consonants were analyzed based on their MOA and POA, whether they were voiced or not, and whether they were aspirated or not. Overall, we found no significant difference in terms of accuracy in these four different aspects. Regarding the MOA, the accuracy varied from 61% to 56%. One should note that stops, which are usually considered to be acquired first in children, showed the lowest accuracy. As for the POA, the accuracy