# A Preliminary Study of Model-Generated Speech
## 4 Evaluation
### Experimental Results  
In this section, the results of the attention-based sequence-to-sequence method, DirecTL+, and Sequitur are reported. In order to further analyze the results, we separated the predicted accuracy of the sounds into three categories: onset, rhyme, and tone. After manually deleting inconsistencies, we adopted the attention-based sequence-to-sequence method, DirecTL+, and Sequitur to train the model. We employed a dataset consisting of 5523 of the same words with known pronunciations in both dictionaries. The pronunciations from 1883 instances were utilized as input, and the three models were trained accordingly. Subsequently, the predicted words from the models were compared with those also present in the 2005 dataset to assess the accuracy of the models. The accuracy rates of the three models, the attention-based sequence-to-sequence method, DirecTL+, and Sequitur, are presented in Table 1. Additionally, the results for the evaluation were categorized into three groups: accuracy with tones, accuracy without tones, and F1-score. In the evaluation, we employed the commonly used metrics in machine transliteration: Accuracy, Recall, Precision, and F1-score. The definition of accuracy is as follows:  
\[Accuracy\ =\ \frac{1}{N}\sum_{i\,=\,1}^{N}\left\{ \begin{array}{ll}1&\text{if }r_{i}\ =\ c_{i}\\ 0&\text{otherwise}\end{array} \right\} \tag{3}\]  
where \(N\) represents the number of data in the test dataset, \(r_{i}\) is the phoneme string predicted by the model, and \(c_{i}\) is the correct phoneme string. An \(Accuracy\ =\ 1\) indicates that all model predictions match the actual dictionary answers perfectly, whereas \(Accuracy\ =\ 0\) indicates that none of the predictions match the dictionary answers. The definitions of Recall, Precision, and F1-score are estimated based on the longest common subsequence (LCS) of string comparisons. The definition of LCS is as follows:  
\[LCS(c,r)=\frac{1}{2}(|c|+|r|-ED(c,r)) \tag{4}\]  
where \(ED\) is the \"string edit distance\", which measures how many insertions, deletions, or substitutions are required to transform one string into another. It is a common method for estimating the similarity between two strings. The definitions of Precision, Recall, and F1-score are as follows:  
\[Precision=\frac{LCS(c_{i},r_{i})}{|c_{i}|} \tag{5}\]  
\[Recall=\frac{LCS(c_{i},r_{i})}{|r_{i}|} \tag{6}\]  
\[F1\text{-score}=\frac{2\times Recall\times Precision}{Recall+Precision} \tag{7}\]  
Five-fold cross-validation was used for evaluation, and the accuracy of our model was up to 68%. Generally speaking, speech flow can be dissected into several syllables, with the segments constituting the fundamental elements. Apart from segments, supra-segmentals provide another perspective on the analysis of speech flow. Upon further examination of the segments, they can be subdivided into vowels and consonants, and supra-segmentals encompass aspects such as tone, duration, stress, and intonation. Consequently, our dataset was scrutinized in terms of syllables, vowels, consonants, and tones. Syllables, the basic units of speech, typically consist of consonants (C) and vowels (V), with common constructions being CV, CVC, and so forth, though specifics vary across languages. Vowels are characterized by tongue height, tongue backness, and lip rounding. Additionally, consonants exhibit features such as POA, MOA, and voicing (or not). In Chinese, aspiration (or not) can serve as a distinctive feature. Tones, which are fundamental components of speech in tonal languages such as Chinese, include level tones and contour tones. Elements such as tonal register and contour