# Heart Disease Risk Prediction Using Machine Learning Classifiers with Attribute Evaluators
## 4 Results  
The results of the machine learning classifiers using the full set of attributes and optimal set that was obtained from attribute evaluators, tuning the parameter 'k' in the IBk method, and comparison with related works are discussed in the following. As shown in Table 7, the highest accuracy of 85.148% was attained with the sequential minimal optimization (SMO) algorithm, followed by logistic regression (LR) with an accuracy of 84.818% using the full set of attributes from the Cleveland dataset using the 10-fold cross-validation test option. The SMO algorithm also provided the best MAE of 0.148, sensitivity of 0.851, fallout of 0.157, precision of 0.852, F-measure of 0.851, and specificity of 0.90 compared to other machine learning algorithms. The meta classifier bagging with LR achieved a high ROC area value of 0.91, followed by the single classifier LR with a ROC area of 0.909. The LR and AdaBoostM1 with LR classifiers also reached a specificity of 0.90. NB provided the second-best MAE of 0.184. The visualization of the threshold curve for the target class provided an ROC area of 0.91 using the bagging with LR meta classifier using the full set of attributes, as shown in Figure 3. The bar plot of the performance metrics using the full set of attributes of the Cleveland heart dataset is shown in Figure 4.  
Table 8 shows that the highest accuracy of 84.158% was achieved with the naive Bayes (NB) algorithm, followed by the SMO, AdaBoostM1 with decision stump (DS), and bagging + LR algorithms, with an accuracy of 83.828% when using the optimal attribute set that was obtained from the correlation-based feature selection method. The SMO algorithm provided the best MAE of 0.161, the NB classifier produced a high sensitivity of 0.842, precision of 0.843, F-measure of 0.841, ROC area of 0.905, and specificity of 0.90 compared to other algorithms. The meta classifier bagging with LR achieved the best fallout value of 0.167. The performance metrics of the ML classifiers with the optimal set obtained using correlation-based feature selection are graphically presented in Figure 5. The ROC curve of the naive Bayes classifier using the correlation-based feature  
achieved a high ROC area value of 0.909. The graphical representation of performance metrics using the chi-squared attribute evaluation method is shown in Figure 7. The ROC curve with an area of 0.909, which was found using the naive Bayes and logistic regression models using the chi-squared attribute evaluation set, are shown in Figure 8a,b respectively. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Classifier** & **Accuracy** & **MAE** & **Sensitivity** & **Fallout** & **Precision** & **F-Measure** & **ROC Area** & **Specificity** \\ \hline NB & 83.498 & 0.183 & 0.835 & 0.171 & 0.835 & 0.835 & 0.909 & 0.870 \\ \hline LR & 84.488 & 0.212 & 0.845 & 0.159 & 0.845 & 0.845 & 0.909 & 0.870 \\ \hline SMO & 86.468 & 0.135 & 0.865 & 0.142 & 0.865 & 0.864 & 0.861 & 0.900 \\ \hline IBk/KNN & 77.887 & 0.223 & 0.779 & 0.226 & 0.779 & 0.779 & 0.775 & 0.800 \\ \hline AdaBoostM1 + DS & 83.498 & 0.224 & 0.835 & 0.173 & 0.836 & 0.834 & 0.899 & 0.880 \\ \hline AdaBoostM1 + LR & 84.488 & 0.214 & 0.845 & 0.159 & 0.845 & 0.845 & 0.854 & 0.870 \\ \hline Bagging + REPTree & 82.178 & 0.282 & 0.822 & 0.188 & 0.823 & 0.821 & 0.883 & 0.880 \\ \hline Bagging + LR & 85.478 & 0.217 & 0.855 & 0.152 & 0.855 & 0.854 & 0.908 & 0.890 \\ \hline JRip & 76.897 & 0.319 & 0.769 & 0.235 & 0.769 & 0.769 & 0.765 & 0.790 \\ \hline RF & 83.168 & 0.257 & 0.815 & 0.193 & 0.816 & 0.814 & 0.904 & 0.880 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance of the machine learning classifiers based on the optimum attribute set found using the chi-squared attribute evaluation technique. Figure 7: Performance metrics based on the optimal set obtained using the chi-squared attribute evaluation technique.