# Ad-RuLer: A Novel Rule-Driven Data Synthesis Technique for Imbalanced Classification
## 4 Experiment
### Experimental Results
This statistically significant finding leads us to reject the null hypothesis and conclude that the Ad-RuLer algorithm demonstrates significant differences in AUC when compared with all other algorithms. Similarly, we have observed that Ad-RuLer exhibits statistically significant differences in G-mean with algorithms such as SMOTE, Tomek-links, and KMeansSMOTE, and in terms of Precision, it demonstrates significant differences with ADASYN and Borderline-SMOTE. When using the random forest as the classifier, Ad-RuLer exhibits significant differences in the G-mean and F1 score compared to Borderline-SMOTE and KMeansSMOTE. When employing XGBoost as the classifier, Ad-RuLer demonstrates a significant difference in the G-mean relative to Borderline-SMOTE and KMeansSMOTE. In addition to the Wilcoxon signed-rank test, we also conducted the Friedman test on the average rankings of all resampling algorithms across all classifiers and datasets. Using the Friedman test to assess the performance of various resampling algorithms, we hypothesized that there would be no significant differences in the average rankings across the different performance metrics. Our analysis revealed that for the G-mean and F1 score, the \(p\)-values were 0.0002 and 0.0012, respectively, with both markedly below 0.05. This led us to reject the null hypothesis related to these metrics, thereby confirming that Ad-RuLer's performance exhibited statistically significant differences when compared with other algorithms on the G-mean and F1 score. Regarding the AUC, the \(p\)-value was 0.1462, exceeding the 0.05. This implies that we lack sufficient statistical evidence to claim significant differences in average rankings among the resampling algorithms for AUC; thus, the null hypothesis for the AUC is retained. For the Precision, although Ad-RuLer ranked second among all algorithms, closely following KmeansSMOTE, the computed \(p\)-value of 0.0005, substantially lower than the 0.05 level of significance, compels us to reject the null hypothesis. Figure 4: Impact of variations in the ratio parameter on the performance metrics of the Ad-RuLer algorithm. null hypothesis. It indicates that at least one algorithm shows a significant performance deviation from the others on Precision. However, the Friedman test does not indicate which specific algorithms exhibit significant differences. Therefore, we employed the Nemenyi post hoc test for multiple pairwise comparisons to determine which algorithms have significant differences from others. In the Nemenyi post hoc test, as shown in Figure 6, the algorithms are listed on the vertical axis, with ranking values on the horizontal axis. The average ranking of each algorithm is represented by a point, which extends to the left and right into a line segment, representing a 95% confidence interval. We assess whether there are significant differences between algorithms by checking whether these line segments (i.e., 95% confidence intervals) overlap. If the line segments of the two algorithms do not overlap, we can consider that these two algorithms have statistically significant differences. Observing the figure, it is found that Ad-RuLer  
Figure 5: Average ranking of resampling algorithms for all three classifiers on all datasets. (**a**) Average ranking for G-mean. (**b**) Average ranking for F1 score. (**c**) Average ranking for AUC. (**d**) Average ranking for Precision.