# A Novel Ensemble Framework Based on K-Means and Resampling for Imbalanced Data
## 4 Experiment Preparation and Result Analysis
### Metrics for Performance Evaluation  
In machine learning, in order to evaluate the classification performance of a model, some evaluation metrics have been introduced including Accuracy, F1-score, G-mean, and AUC. The definition of these evaluation metrics needs to use the confusion matrix that introduces connection of actual and predicted classifications given in Table 2.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**\#No.** & **\#Datasets** & **\#Features** & **\#Samples** & **IR** & **\(k\)-Value** \\ \hline D1 & Pima & 9 & 768 & 1.87 & 3 \\ D2 & Vehicle2 & 18 & 846 & 2.88 & 3 \\ D3 & Vehicle0 & 18 & 846 & 3.25 & 4 \\ D4 & Ecoli1 & 7 & 336 & 3.36 & 3 \\ D5 & Glass86 & 9 & 214 & 6.38 & 3 \\ D6 & Page-blocks0 & 10 & 5472 & 8.79 & 3 \\ D7 & Yeast-2\_vs\_4 & 8 & 514 & 9.08 & 5 \\ D8 & Vowel0 & 9 & 988 & 9.98 & 3 \\ D9 & Glass2 & 9 & 214 & 11.59 & 4 \\ D10 & Shuttle-c0-vs-c4 & 9 & 1829 & 13.87 & 5 \\ D11 & Glass4 & 9 & 214 & 15.47 & 3 \\ D12 & Ecoli4 & 7 & 336 & 15.8 & 3 \\ D13 & Yeast-1-4-5-8\_vs\_7 & 8 & 693 & 22.1 & 4 \\ D14 & Yeast4 & 8 & 1484 & 28.41 & 4 \\ D15 & Yeast5 & 8 & 1484 & 32.73 & 5 \\ D16 & Yeast6 & 8 & 1484 & 41.4 & 5 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of 16 imbalanced datasets.  
Figure 3: The procedure of EKR (The ensemble framework based on K-means and resampling).  
Accuracy means the proportion of correctly predicted samples to total samples, which is calculated as Equation (5):  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{5}\]  
\(F1\)_-score_ takes into account both the Precision and Recall of the classification model, which is the harmonic average of Precision and Recall, \(F1\)-_score_ of the positive data is as follows:  
\[Precision_{P}=\frac{TP}{TP+FP} \tag{6}\]  
\[Recall_{P}=\frac{TP}{TP+FN} \tag{7}\]  
\[F1-score_{P}=\frac{2\times Precision_{P}\times Recall_{P}}{Precision_{P}+ Recall_{P}} \tag{8}\]  
Similarly, for the negative samples, \(F1\)-_score_ is calculated as:  
\[Precision_{N}=\frac{TN}{TN+FN} \tag{9}\]  
\[Recall_{N}=\frac{TN}{TN+FP} \tag{10}\]  
\[F1-score_{N}=\frac{2\times Precision_{N}\times Recall_{N}}{Precision_{N}+ Recall_{N}} \tag{11}\]  
Therefore, for the entire data, synthetic Precision, Recall, and \(F1\)-_score_ are calculated as:  
\[Precision=\frac{Precision_{P}+Precision_{N}}{2} \tag{12}\]  
\[Recall=\frac{Recall_{P}+Recall_{N}}{2} \tag{13}\]  
\[F1-score=\frac{F1-score_{P}+F1-score_{N}}{2} \tag{14}\]  
G-mean can be used to evaluate the comprehensive performance of an algorithm, which is defined as Equation (17):  
\[G-mean=\sqrt{TPR\times TNR} \tag{15}\]  
\[TPR=\frac{TP}{TP+FN} \tag{16}\]  
\[TNR=\frac{TN}{FP+TN} \tag{17}\]  
G-mean uses \(TPR\) and \(TNR\) to measure the classification performance of positive and negative classes. If one of both is very small, the \(G\)-_mean_ is not ideal.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**Confusion Matrix**}} & \multicolumn{2}{c}{**Predicted Labels**} \\ \cline{2-4}  & & **Positive** & **Negative** \\ \hline \multirow{2}{*}{Actual labels} & Positive & TP & FN \\ \cline{2-4}  & Negative & FP & TN \\ \hline \hline \end{tabular} True Positive (TP) is the number of positive samples predicted as “positive.” False negative (FN) is the number of positive samples predicted as “negative.” False positive (FP) is the number of negative samples predicted as “positive.” True negative (TN) is the number of negative samples predicted as “negative.”  
\end{table}
Table 2: Confusion matrix.  
AUC (area under curve) is also a very reliable classification evaluation criteria, which represents the area under the ROC (receiver operating characteristic). The ROC can visualize the trade-off between _TPR_ and _FPR_. The range of AUC is from 0 to 1. The larger the AUC is, the better the performance of the algorithm is.