## 5 Results and Discussion
### Full Set versus LOFO Subsets  
The overall classification accuracies using the full set and the LOFO subsets are shown in Table 2, and accuracies per voice type classification are shown in Figure 7. Table 2 shows that all classifiers achieved an overall accuracy greater than 75% on the full set. NN (82.5%), SVM (81.3%) and KNN (81.0%) yielded over 80% overall accuracy on the full set, which was much greater (by approximately 20%) than the single feature classification accuracy in Table 1. By removing features from the full set, the overall accuracies were generally decreased with respect to the per-set average of the overall accuracies. Two LOFO subsets, \(-S_{r}\) (74.4%) and -SE (76.7%) dropped by 5.7% and 3.4%, respectively, in comparison with the full set. For these two LOFO subsets, all classifiers showed a notable decrease between \(-\)2.5% (KNN on -SE) and \(-\)8.1% (SVM on \(-S_{r}\)) in overall accuracy. This showed that the shimmer and the SE were of more importance than other features in the classification overall accuracy. Other subsets did not produce conspicuous changes in overall accuracies. Only DT's performance on \(-H_{1}\) (\(-\)2.5%), \(-H_{2}\) (\(-\)4.0%), \(-H_{3}\) (\(-\)2.0%) and \(-J_{r}\) (\(-\)2.0%) and LD's performance on \(-J_{r}\) (\(-\)1.5%) were degraded compared to the full set. Other classifiers' accuracies did not change notably (within \(\pm\)1.1%). The overall accuracy varied with the chosen classifier. In general, NN, SVM and KNN had greater overall accuracies than DT and LD on both the full set and the LOFO subsets. All accuracies greater than 80% were achieved by these three classifiers. The DT and the LD accuracies did not exceed 80%, either on the full set or the LOFO subsets. For the per-classifier average accuracies, shown in Table 2, the NN achieved the highest overall accuracy (81.5%) on all datasets, and DT had the lowest overall accuracy (75.2%). The classifier performances based on the full feature set were shown in Table 3. The AUC scores for all classifiers exceeded 0.85, which showed satisfactory performances of the classifiers. The NN achieved the highest AUC score (0.93, 0.97), the highest TPR (0.81, 0.90) and the lowest FPR (0.10, 0.08) for the modal and the breathy voice classification. For the pressed voice, the NN AUC score and the TPR score were the highest (0.94) of all classifiers, but the NN FPR (0.16) was not the lowest. \begin{table}
\begin{tabular}{c c c c c c c} \hline  & **DT** & **LD** & **SVM** & **KNN** & **NN** & **Per-Set** \\ \hline Full Set & 77.4 & 78.3 & 81.3 & 81.0 & 82.5 & 80.1 \(\pm\) 2.1 \\ \(-H_{1}\) & 74.9 & 78.4 & 82.1 & 81.7 & 83.3 & 80.1 \(\pm\) 3.4 \\ \(-H_{2}\) & 74.4 & 79.0 & 82.0 & 81.7 & 83.3 & 80.1 \(\pm\) 3.5 \\ \(-H_{3}\) & 75.4  
hypothesis of the second test was that the classification accuracy was independent of the features selected for classification. The classification accuracies of different datasets were significant for SVM (\(p=1.99^{-6}\)), KNN (\(p=5.7\times 10^{-3}\)), NN (\(p=8.7\times 10^{-5}\)), but insignificant for DT (\(p=0.0995\)) and LD (\(p=0.119\)). Results suggested that the SVM, KNN and NN classifiers were more sensitive to the variation of the feature set than the other two classifiers (DT and LD). Figure 7 shows more details on how classifiers and single features influenced the per-type classification accuracies. The results are shown in comparison with the full set of classification results. For the modal voice classification, the \(-S_{r}\) subset results indicated a notable decrease in average accuracy (drawn in the red dashed line). The decrease for all classifiers was 15%, 8%, 12%, 11% and 7%, respectively. The average accuracy of the \(-S_{r}\) was 63.7%, i.e., 10.6% lower than that (74.3%) of the full set. The average accuracy of -SE was also lower (\(-3.5\%\)) than that of the full set due to the notable degradation (\(-12\%\)) of the LD performance. Other classifiers' performance yielded little change (within \(\pm 2\%\)) on the -SE subset.