## 4 Results
### Validation of the Proposal  
In order to validate the proposed method, clustering-based RUS was tested against other resampling strategies by analyzing several quality metrics obtained after applying reliable single and ensemble classifiers. Given the fact that accuracy is not a suitable evaluation measure in imbalanced data contexts, several metrics introduced in Section 3.1 were obtained. These are: accuracy, area under the ROC curve (AUC), weighted average of precision, recall (sensitivity), F-measure, G-mean, and optimized precision (OP). F-measure was computed with the \(\beta\) parameter set to 1. Table 1 shows the values of these measures obtained by applying the classification algorithms to data that has not been preprocessed with sampling procedures. Tables 2-4 show the same measures, but in this case, they have been obtained by the classifiers induced from data resampled with SMOTE 100%, SMOTE 500% and RUS 4:1, respectively. Finally, Table 5 contains the results obtained when our approach (clustering-RUS) was applied. As previously mentioned, the results of algorithms that yielded very poor results are not included in the tables. Among them are those obtained from some neural networks. We would like to point out that we even tested a deep learning algorithm based on a multi-layer feed-forward artificial neural network with 50 hidden layers. The generated model  
provided for the original dataset an accuracy of 0.937, a weighted average precision of 0.920 and a weighted average recall of 0.660. We can see in Table 1 that none of the algorithms has such a bad performance, especially when comparing recall since the worst recall value in the table is 0.935. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Algorithm** & **Accuracy** & **OP** & **G-Mean** & **Precision** & **Recall** & **F-Measure** & **AUC** \\ \hline J48 & 0.941 & 0.390 & 0.532 & 0.929 & 0.941 & 0.930 & 0.675 \\ Random Forest & 0.949 & 0.563 & 0.656 & 0.942 & 0.949 & 0.944 & 0.861 \\ Random Tree & 0.925 & 0.602 & 0.684 & 0.928 & 0.925 & 0.926 & 0.726 \\ AdaBoost-J48 & 0.943 & 0.604 & 0.686 & 0.939 & 0.943 & 0.941 & 0.832 \\ AdaBoost-Random Forest & 0.950 & 0.568 & 0.660 & 0.944 & 0.950 & 0.944 & 0.823 \\ AdaBoost-Random Tree & 0.944 & 0.599 & 0.682 & 0.938 & 0.943 & 0.940 & 0.840 \\ Bagging-J48 & 0.940 & 0.389 & 0.531 & 0.928 & 0.940 & 0.929 & 0.812 \\ Bagging-Random Forest & 0.949 & 0.500 & 0.610 & 0.941 & 0.949 & 0.941 & 0.874 \\ Bagging-Random Tree & 0.939 & 0.568 & 0.660 & 0.934 & 0.939 & 0.936 & 0.824 \\ Random Committee-Random Forest & 0.950 & 0.568 & 0.660 & 0.944 & 0.950 & 0.944 & 0.871 \\ Random Committee-Random Tree & 0.935 & 0.597 & 0.681 & 0.933 & 0.935 & 0.934 & 0.820 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results of the classifiers without resampling. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Algorithm** & **Accuracy** & **OP** & **G-Mean** & **Precision** & **Recall** & **F-Measure** & **AUC** \\ \hline J48 & 0.933 & 0.408 & 0.546 & 0.546 & 0.921 & 0.933 & 0.925 \\ Random Forest & 0.950 & 0.583 & 0.671 & 0.671 & 0.943 & 0.950 & 0.945 \\ Random Tree & 0.924 & 0.616 & 0.694 & 0.694 & 0.929 & 0.924 & 0.927 \\ AdaBoost-J48 & 0.942 & 0.600 & 0.683 & 0.683 & 0.937 & 0.942 & 0.939 \\ AdaBoost-Random Forest & 0.947 & 0.566 & 0.659 & 0.659 & 0.940 & 0.947 & 0.942 \\ AdaBoost-Random Tree & 0.941 & 0.588 & 0.674 & 0.674 & 0.936 & 0.941 & 0.938 \\ Bagging-J48 & 0.942 & 0.452 & 0.577 & 0.577 & 0.932 & 0.942 & 0.934 \\ Bagging-Random Forest & 0.947 & 0.539 & 0.639 & 0.639 & 0.939 & 0.946 & 0.941 \\ Bagging-Random Tree & 0.940 & 0.603 & 0.685 & 0.685 & 0.940 & 0.938 & 0.938 \\ Random Committee-Random Forest & 0.949 & 0.579 & 0.668 & 0.668 & 0.942 & 0.949 & 0.944 \\ Random Committee-Random Tree & 0.940 & 0.607 & 0.687 & 0.687 & 0.936 & 0.940 & 0.938 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results of the classifiers with synthetic minority oversampling technique (SMOTE) resampling (100%).