# A Selective Dynamic Sampling Back-Propagation Approach for Handling the Two-Class Imbalance Problem
## 6 Experimental Results and Discussion
The rows and columns constitute the studied methods; as a consequence, it represents all \(C\times C\) pairwise classifier comparisons. The filled circles mean that for these particular pairwise methods (for \(C_{i}\times C_{j}\); \(i=1,2,...,C\) and \(i\
eq j\)), the null hypothesis was reject by the Holm-Shaffer post-hoc tests. Therefore, the color of circles is the darkest when the \(p\)-values are close to zero; this means that the statistical difference is significant. Figure 2: Number of samples used in the training process by the studied methods in contrast to the Area Under the receiver operating characteristic Curve (AUC) average ranks obtained in the classification test. The \(x\) axis represents the average ranks (the best performing method should have the rank of one or close to this value). We previously used the ten-fold cross-validation method. The number shown in the \(y\) axis corresponds to the average training fold size. In accordance with Table 2 and Figure 1, most methods of over-sampling present a better classification performance than the standard BP with statistical significance. The under-sampling methods do not present a statistical difference with respect to standard BP performance, and all dynamic sampling approaches improve the standard BP performance with statistical differences. ADASYIN, SPIDER-1 and SPIDER-2 (over-sampling methods) and RUS, NCL and TL (under-sampling methods) show the trend of improving the classification results, but they do not significantly improve the standard BP performance. Then, the OSS, CNN and CNNTL classify worse than standard BP; this notwithstanding, these approaches do not show a statistical difference with it. SDSAO, SDSAS and DyS are statistically better than ADASYIN, SPIDER-1 and SPIDER-2 (over-sampling methods) and also than all under-sampling approaches studied in this work. With a statistical difference, the DOS performance is better than CNN, CNNTL, OSS and TL. Table 2 shows that the trend is that ROS presents a better performance than the proposed method (SDSAO and SDSAS), and that DyS shows a slight advantage over SDSAS; however, in accordance with the Holm-Shaffer post-hoc tests, statistical difference in the classification performance does not exist among these methods (see Figure 1). In general terms, most over-sampling methods and dynamic sampling approaches are successful methods to deal with the class imbalance problem, but with respect to the training dataset size, SDSAS, SDSAO and DyS use significantly fewer samples than the over-sampling approaches. They employed about 78% less samples than most over-sampling methods; in addition, SDSAS, SDSAO and DyS still use fewer samples than the standard BP trained with the original training dataset. They use about 60% less samples; these facts stand out in Figure 2. However, the DyS method applies the ROS in each epoch or iteration (see Section 4), whereas SDSA only applies the ROS or SMOTE one time before ANN training (see Section 2). Figure 2 shows that the under-sampling methods employ significantly fewer samples than the rest of the techniques (except dynamic sampling approaches with respect to RUS, NCL and TL); however, their classification performance in most of the cases is worse than the standard BP (without statistical significant) or is not better (with statistical significant) than the standard BP. On the other hand, the worst methods studied in this paper (in agreement with Table 2) are those based on the CNN technique (OSS, CNN and CNNTL), i.e., those that use a \(k-NN\) rule as the basis and achieving an important size reduction of the training dataset. In contrast, NCL, which is of the \(k-NN\) family, also improves the classification performance of the back-propagation; however, the dataset size reduction reached for this method is not of CNN's magnitude; in addition, it only eliminates majority samples.