# Calibrating Mini-Mental State Examination Scores to Predict Misdiagnosed Dementia Patients
## 4 Experiments and Results Analysis
### Experiments
#### 4.3.2 Classification Model
\begin{table}
\begin{tabular}{c} COGNITIVE EXAM: IDENTITIES DAY OF WEEK
* COGNITIVE EXAM: IDENTITIES MONTH
* COGNITIVE EXAM: IDENTITIES YEAR
* COGNITIVE EXAM: IDENTITIES COUNTY
* COGNITIVE EXAM: COMPREHENDS RADIO
* COGNITIVE EXAM: NUMBER OF ANIMALS LISTED
* COGNITIVE EXAM: NUMBER OF ANIMALS LISTED: SCORE
* COGNITIVE EXAM: REGISTERS OBJECTS 3: PENNY
* COGNITIVE EXAM: RECALLS OBJECTS
* COGNITIVE EXAM: WRITES A SENTENCE
* COGNITIVE EXAM: PRAXIS PAPER
* COGNITIVE EXAM: PRAXIS PAPER: ON LAP  
\end{table} 
Table 3: Important Feature list for identifying misdiagnosed patients. majority class \"misdiagnosed = NO\" to make it equal to the minority class \"misdiagnosed = YES\". The outcome of this process again provides an equal distribution of classes for better evaluation of the model performance. Result Analysis  
Table 5 illustrates the performance of all models in terms of precision, recall, f1-score, and accuracy.The imbalance of the classes in the dataset plays a significant role in deciding the performance of the predictive models. The model based on the original dataset suffers due to the high imbalance in \"misdiagnosed = YES\" and \"misdiagnosed = NO\" classes. It is not able to learn for \"misdiagnosed = YES\" class (133 data points) due to a very less number of data points compared to the \"misdiagnosed = NO\" class (971 data points) in the original dataset. The second (oversampled dataset) model performs better on the \"misdiagnosed = YES\" class compare to the model of original-dataset as a result of the dense cluster of generated data points for this class. The dense cluster is formed due to the high similarity between generated data points features or less distance between data points. Both datasets are affected by the imbalanced distribution of the classes. There is also a significant difference between the performance of predictive models using datasets (generated by cluster-centroids and random-under-sampler algorithms) in terms of f1-score as shown in Table 5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Original Dataset**} \\ \hline  & **Precision** & **Recall** & **f1-Score** & **Support** \\ \hline NO & 0.88 & 0.99 & 0.93 & 971 \\ YES & 0.26 & 0.04 & 0.07 & 133 \\ accuracy & & & 0.87 & 1104 \\ macro avg & 0.57 & 0.51 & 0.50 & 1104 \\ weighted avg & 0.81 & 0.87 & 0.83 & 1104 \\ \hline \multicolumn{4}{c}{**Oversampled Dataset (SMOTE)**} \\ \hline  & **Precision** & **Recall** & **f1-Score** & **Support** \\ \hline NO & 0.93 & 0.96 & 0.94 & 971 \\ YES & 0.96 & 0.93 & 0.94 & 971 \\ accuracy & & & 0.94 & 1942 \\ macro avg & 0.94 & 0.94 & 0.94 & 1942 \\ weighted avg & 0.94 & 0.94 & 0.94 & 1942 \\ \hline \multicolumn{4}{c}{**Undersampled Dataset (ClusterCentroids)**} \\ \hline  & **Precision** & **Recall** & **f1-Score** & **Support** \\ \hline NO & 0.92 & 0.91 & 0.91 & 133 \\ YES & 0.91 & 0.92 & 0.91 & 133 \\ accuracy & & & 0.91 & 266 \\ macro avg & 0.91 & 0.91 & 0.91 & 266 \\ weighted avg & 0.91 & 0.91 & 0.91 & 266 \\ \hline \multicolumn{4}{c}{**Undersampled Dataset (RandomUnderSampler)**} \\ \hline  & **Precision** & **Recall** & **f1-Score** & **Support** \\ \hline NO & 0.70 & 0.74 & 0.72 & 133 \\ YES & 0.72 & 0.68 & 0.70 & 133 \\ accuracy & & & 0.71 & 266 \\ macro avg & 0.71 & 0.71 & 0.71 & 266 \\ weighted avg & 0.71 & 0.71 & 0.71 & 266 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Classification report after different sampling of dataset. Clearly, not all models can accurately address the problem of discriminating episodes for both classes (\"misdiagnosed = NO\" and \"misdiagnosed = YES\"). The results show how the imbalance in the classes makes it difficult to select the right model, in order to identify \"misdiagnosed = NO\" patients among misdiagnosed-YES patients. The best models for the task at hand seem to be the one using the oversampled dataset (SMOTE) and the cluster-centroids undersampled (average f1 scores of 0.94 and 0.91, respectively), which can discriminate episodes effectively for both classes.