# Prototype System for Supporting Medical Diagnosis Based on Voice Interviewing
## 2 In Search of the Effectiveness of a Diagnosis Support System--Results  
In this experimental part, tests were carried out on the effectiveness of a diagnostic system based on the analysis of voice interviews and the classification of patient responses into specific disease entities. A set of classifiers based on machine learning techniques was used to evaluate the effectiveness. Each was tested for sensitivity, specificity and balanced accuracy in diagnosing disease entities. The analysis included various classification methods such as logistic regression, random forest, AdaBoost, kNN and decision trees. Logistic regression, as a statistical technique for modelling binary outcomes, was used to classify patient responses in the context of specific diseases. Random forest, an ensemble of classifiers based on randomised decision trees, allowed increased classification accuracy by aggregating the results of multiple models. AdaBoost, an algorithm that learns the weights of examples based on the results of previous classifiers, allowed for improved classification of more difficult cases, while the kNN method was based on the similarity of neighbours to the object under study, using different metrics. Decision trees, based on a sequence of rules, provided, in turn, an intuitive method to interpret the results. The results obtained in this section present a comparison of classification methods in terms of sensitivity, specificity and balanced accuracy for several disease classes. Visualisations in the form of tables and graphs facilitate the interpretation of the effectiveness of the different models. The analysis carried out for disease classes such as 51 (Hemiplegia) and 184 (Herpangina) indicated that the Random Forest and ExtraTreesClassifier models performed best. The experiments were conducted in a Monte Carlo cross-validation model (MCCV), which allowed an accurate assessment of the stability of the results. The aim was to achieve a stable, sustainable accuracy of at least 75 per cent for the selected disease classes, confirming the effectiveness of the system in automating initial diagnoses based on voice interviews. In order to achieve generalisation of the results, we used a multiple Monte Carlo split between test and training. The test parts simulated previously unseen values. We performed such 25 tests for each central class. From this, we calculated the standard deviation. We chose a variational Monte Carlo method due to the low availability of samples for each class. The results are in Tables 2-6 and Figures 2-6. We present in these results additional calculations showing how the ROC and PR curve [9] behaves--these results are optional. We use the mean values of sensitivity  
Figure 4: Results for central class 184 (Herpangina), which contains 65 objects. Figure 3: Results for central class 112 (Kidney stones), which contains 66 objects. Figure 5: Results for central class 188 (bacterial middle ear infection), which contains 64 objects. Figure 6: Results for central class 135 (Pneumonia), which contains 64 objects. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Classifier** & **Sensitivity** & **Specificity** & **Balanced Accuracy** & **Sensitivity SD** & **Specificity SD** & **Balanced Accuracy SD** \\ \hline RandomForest & 1.0 & 0.97 & 0.99 & 0.0 & 0.04 & 0.02 \\ ExtraTreesClassifier & 1.0 & 0.97 & 0.99 & 0.0 & 0.04 & 0.02 \\ LogisticRegression & 1.0 & 0.96 & 0.98 & 0.0 & 0.05 & 0.03 \\ kNN & 0.99 & 0.95 & 0.97 & 0.0 & 0.06 & 0.03 \\ SVM & 1.0 & 0.95 & 0.97 & 0.0 & 0.05 & 0.03 \\ SVC & 1.0 & 0.95 & 0.97 & 0.0 & 0.05 & 0.03 \\ DecisionTree & 1.0 & 0.95 & 0.97 & 0.0 & 0.06 & 0.03 \\ GradientBoosting & 1.0 & 0.95 & 0.97 & 0.0 & 0.06 & 0.03 \\ AdaBoost & 1.0 & 0.95 & 0.97 & 0.0 & 0.06 & 0.03 \\ Perceptron & 1.0 & 0.93 & 0.97 & 0.0 & 0.09 & 0.05 \\ BaggingClassifier & 1.0 & 0.95 & 0.97 & 0.0 & 0.06 & 0.03 \\ SGDClassifier & 1.0 & 0.92 & 0.96 & 0.01 & 0.11 & 0.05 \\ GausianNB & 0.99 & 0.91 & 0.95 & 0.0 & 0.08 & 0.04 \\ \hline \hline \end{tabular}
\end{table}