# The Application of Intelligent Data Models for Dementia Classification
## 3 Results
### Classification Methods and Implementation
SVM (linear) and KNN derived lower sensitivities of 74.7% (1/341 Dementia and 61/866 MCI classed as CN) and 76.8% (7/341 Dementia and 145/866 MCI classed as CN), respectively. Specificity measures the models' ability to correctly identify negative results. As such, it is impacted by False Positive (FP) results, as annotated in Equation (2), above. The considered machine learning algorithms demonstrated somewhat of an increase in this metric, compared to sensitivity. In the practical application of dementia classification, this high specificity indicates the strong ability of the algorithm to predict healthy or cognitive control individuals as being healthy. Though this is important, sensitivity may arguably be more important in our scenario as a FP can be corrected by subsequent testing, whereas a FN may result in no further testing and, therefore, the patient missing a diagnosis. The C4.5 algorithm was able to correctly identify negative results, with a specificity of 93.2% (43/523 CN classified as MCI and 79/866 MCI classified as Dementia). NB and RIPPER performed well within 2% of this result. SVM (linear) and KNN performed the poorest, though markedly higher than the sensitivity results obtained, with results of 82.3% and 83.9%, respectively. These scores are a result of more FP predictions; SVM gave a total of 133/523 FPs on CN subjects, seven of which were FPs as dementia, and 121/866 MCI subjects classed as dementia. KNN had results of 102/523 FPs for CN subjects, three of which were classified as dementia, the other as MCI, and also had 46/866 MCI subjects classed as dementia. The F-measure is the harmonic mean between sensitivity and precision, with the equation shown above. Precision (Equation (4)) is calculated as the number of TPs over all predicted positive results (true and false) and differs from sensitivity, which includes FN results. All models exhibit a similar F-measure score compared to model accuracy, with \(+/-0.1\%\) across all five algorithms compared to their accuracy. As the F-measure is the harmonic mean between these two metrics, the algorithms that have better results in sensitivity and precision (NB, C4.5 and RIPPER) have F1-scores of over 10% higher than SVM linear and KNN. \begin{table}
\begin{tabular}{c c c c c} \hline
**Model** & **Accuracy** & **Sensitivity** & **Specificity** & **F-Measure** \\ \hline SVM-Liblinear & 74.7\% & 74.7\% & 82.3\% & 74.8\% \\ KNN k = 5 & 76.8\% & 76.8\% & 83.9\% & 76.7\% \\ Naive Bayes & 88.4\% & 88.4\% & 91.7\% & 88.4\% \\ C4.5-Decision Tree & 89.7\% & 89.7\% & 93.2\% & 89.8\% \\ RIPPER & 89.0\% & 89.0\% & 92.3\% & 89.1\% \\ \hline \end{tabular}
\end{table} 
Table 2: Experiment results. The Receiver Operator Characteristic (ROC) curve is a plot of the TP rate against the FP rate ([PERSON], 2006) [32]. The area under this curve (ROC-AUC) gives a good measure of how well a model is at correctly predicting TP and TN results. Figure 3 shows the values of ROC based on the different algorithms used in the experiment. Though C4.5 saw the highest sensitivity, specificity, F1 score, and classification accuracy, NB resulted in a stronger ROC measurement of 95.5%, 2.3% higher than C4.5. The competitive results seen in this metric are promising as the ROC AUC addresses the capability of the models to correctly classify both TN and TP results and, therefore, minimize FN and FPs. It is imperative to minimize type II errors (FN) as much as possible in medical diagnosis applications, such as dementia detection. A FN classification, in a real-life situation, could see a patient be incorrectly diagnosed as not being cognitively impaired (or to a lesser degree, for example MCI versus a more advanced dementia). This can be incredibly detrimental to the patient's health as early diagnosis and disease management is critical for speeding up intervention due to MCI or AD. As expected, only a small number of misclassifications of CN (0 or 1 across all algorithms excluding KNN which had 7/341) when the patient was actually classified as AD.