## 3 Experiment
### Experimental Results and Analysis
\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & **LR** & **NB** & **DT** & **SVM** & **KNN** & **RF** & **XGBoost** & **GBDT** & **MLP** & **CNN** \\ \hline Recall & 0.96 & 0.97 & 0.98 & 0.99 & 0.98 & 1.00 & 0.98 & 0.98 & 0.99 & 0.97 \\ Specificity & 0.96 & 0.98 & 0.99 & 0.98 & 0.99 & 1.00 & 0.98 & 0.98 & 0.98 & 0.97 \\ \(G\) value & 0.96 & 0.98 & 0.99 & 0.99 & 0.98 & 0.99 & 0.98 & 0.99 & 0.99 & 0.97 \\ \(F1\) value & 0.95 & 0.97 & 0.98 & 0.98 & 0.98 & 1.00 & 0.98 & 0.99 & 0.99 & 0.96 \\ Time(s) & 0.35 & 0.23 & 0.42 & 0.41 & 0.34 & 8.32 & 5.76 & 7.52 & 15.32 & 34.17 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Evaluation measures of each machine learning model in the normal and pathological voice class-balanced MEEI dataset. Figure 7: The binary classification confusion matrices of LR, NB, DT, SVM, KNN, MLP, and RF in the class-balanced dataset. The commonly used model evaluation measure AUC/ROC is a biased metric. Although it is commonly used to summarize the ROC curve and is widely used in medical applications, it is too optimistic regarding the performance of the classifier when the data are imbalanced [8]. Both precision and recall are helpful in cases where there is a class-imbalance in the observations. The precision and recall are focused on the positive class (the minority class) and are unconcerned with the true negatives (majority class). Therefore, the PRC was not impacted by the addition of majority classes. Compared with the AUC value of multi-classification in the initial class-imbalanced dataset, the VPD system with FC-SMOTE improves all the models' AUC values. Our method increases the AUC values of LR, NB, DT, SVM, KNN, RF, XGBoost, GBDT, MLP, and CNN in multi-classification from 0.44, 0.41, 0.26, 0.48, 0.43, 0.49, 0.36, 0.33, 0.47, and 0.43, respectively, to 0.73, 0.68, 0.76, 0.84, 0.75, 0.88, 0.85, 0.76, 0.76, and 0.75. The perfect test will have a PRC that passes through the upper right corner (corresponding to 100% precision and 100% recall). Generally, the closer a PRC is to the upper right corner, the better the test is. Figure 8 reports the comparison results of the AUC, showing that the use of the FC-SMOTE algorithm can greatly improve the machine learning model's ability to detect the minority class voice samples. Meanwhile, the PRCs demonstrate that the VPD system with FC-SMOTE has a good multi-class classification ability for pathological voices and illustrates that FC-SMOTE has a high ability to improve the multi-classification efficiency. Furthermore, the PRC curve in Figure 8d shows the AUC for the binary classification with FC-SMOTE. It can be seen that the AUC can reach 1.00 in most models. Additionally, this indicates that the VPD system with the FC-SMOTE algorithm used in this paper can achieve excellent results in the diagnosis of pathological voice. Subsequently, to widen the problem scope and verify the applicability of the proposed method, we experimented with the same scenario in the SVD database. For the multi-classification system, Tables 9 and 10 present the model evaluation measures of each classifier without and with the C-SMOTE algorithm by 10-fold cross-validation in the SVD  
Figure 8: (**a**) Precision-recall curves for multi-class without FC-SMOTE. (**b**) Precision—recall curves for multi-class with FC-SMOTE. (**c**) Precision–recall curves for binary classification without FC-SMOTE. (**d**) Precision–recall curves for binary classification with FC-SMOTE. database. In the experiment based on the MEEI database, we set up an external dataset to prove the effectiveness of the FC-SMOTE algorithm in generating samples. Therefore, in the SVD database experiment no external dataset was set to repeat this verification experiment. The results in Table 9 show that pathological voice classification with a class-imbalanced dataset cannot obtain good results in these typical machine learning models. This problem also exists in the SVD database.