# A Selective Dynamic Sampling Back-Propagation Approach for Handling the Two-Class Imbalance Problem
## 6 Experimental Results and Discussion  
In order to assess the performance of the proposed methods (SDSAO and SDSAS), a set of experiments has been carried out, over thirty five two-class datasets (Table 1) with ten well-known over-sampling approaches (ADASYN, ADOMS, B-SMOTE, ROS, SMOTE, SMOTE-ENN, SMOTE-TL, SPIDER-1, SPIDER-2 and SL-SMOTE), six popular under-sampling methods (TL, CNN, CNNTL, NCL, OSS and RUS) (for more detail about these re-sampling techniques, see Section 3) and two dynamic sampling approaches (DyS and DOS). This section is organized as follows: First, the AUC values are shown, and the Friedman ranks are used to analyze the classification results (Table 2). Second, a statistical test is presented in order to strengthen the results discussion (Figure 1). Finally, the relationship between the training dataset size and the tested methods performance is studied (Figure 2). The results presented in Table 2 are the AUC values obtained in the classifying stage, and they are averaged values between ten folds and ten different initialization weights of the neural network (see Section 5). In accordance with the averaged ranks shown in Table 2, all over-sampling methods and dynamic sampling approaches (SDSAO, SDSAS, DyS and DOS) can improve the standard back-propagation (BP) performance, and the worst approaches with respect to standard BP are the under-sampling techniques, except by RUS, NCL and TL, which show a better performance than the standard BP. This table also shows that only the ROS technique presents a better performance than the proposed methods. SDSAO and DyS show a slight advantage over SDSAS. In addition, Table 2 indicates that the class Imbalance Ratio (IR) is not determinant in order to get high AUC values, for example CAY7, SAT2, SEG1, SEG5 and 92A5 datasets present high values of AUC no matter their IR; also in these datasets, most over-sampling methods and dynamic sampling approaches are very competitive. Figure 1: Results of the non-parametric statistical [PERSON] and Shaffer post-hoc test. The fill circles mean that for these particular pairs of classifiers, the null hypothesis was rejected by both test. The color of the circles is the darkest at \(p\)-values close to zero, i.e., when the statistical difference is the most significant. Other datasets support this fact, i.e., IR is not critical in the classification performance, for example the SEG4 and SEG5 datasets have the same IR, but the classification performance (using the standard BP) is very different (values of AUC of 0.999 and 0.630, respectively). This confirms was was presented in other works, in that other features of the data might become a strong problem for the class imbalance [2]. For example: (i) the class overlapping or noisy data [39; 42; 56; 57]; (ii) the small disjuncts; (iii) the lack of density and information in the training data [58]; (iv) the significance of the borderline instances [13; 59] and their relationship with noisy samples; and (v) the possible differences in the data distribution for the training and testing data, also known as the dataset shift [7]. In order to strengthen the result analysis, a non-parametrical statistical and post-hoc tests are applied (see Section 5.3): [PERSON] and Iman-Davenport tests report that considering reduction performance distributed according to chi-square with 20 degrees of freedom, the [PERSON] statistic is set at 329.474, and the \(p\)-value computed by the [PERSON] test is 1.690 \(\times\) 10\({}^{-10}\). However, considering reduction performance distributed according to the F-distribution with 20 and 680 degrees of freedom, the [PERSON] and [PERSON] statistic is 30.233, and the \(p\)-value computed by their test is 2.588 \(\times\) 10\({}^{-80}\). Then, the null hypothesis is rejected, i.e., the [PERSON] and Iman-Davenport tests indicate the existence of significant differences in the results. Due to these results, a post-hoc statistical analysis is required. Figure 1 shows the results of the non-parametric statistical Holm and [PERSON] post-hoc tests.