# Ad-RuLer: A Novel Rule-Driven Data Synthesis Technique for Imbalanced Classification
## 4 Experiment
### Experimental Results
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Datasets** & **Algorithms** & **G-Mean** & **F1 Score** & **AUC** & **Precision** \\ \hline Heart Disease & Ad-RuLer & **0.769** & **0.721** & **0.841** & 0.708 \\  & SMOTE & 0.756 & 0.703 & 0.824 & 0.693 \\  & ADASYN & 0.761 & 0.709 & 0.823 & 0.690 \\  & Tomek-links & 0.753 & 0.700 & 0.823 & 0.693 \\  & Borderline-SMOTE & 0.758 & 0.706 & 0.821 & 0.688 \\  & KmeansSMOTE & 0.748 & 0.694 & 0.821 & **0.719** \\ \hline Breast Cancer & Ad-RuLer & **0.970** & **0.963** & **0.993** & **0.968** \\  & SMOTE & 0.936 & 0.919 & 0.988 & 0.918 \\  & ADASYN & 0.936 & 0.913 & 0.986 & 0.879 \\  & Tomek-links & 0.932 & 0.913 & 0.985 & 0.909 \\  & Borderline-SMOTE & 0.933 & 0.910 & 0.985 & 0.874 \\  & KMeansSMOTE & 0.931 & 0.915 & 0.986 & 0.925 \\ \hline Pima & Ad-RuLer & 0.733 & 0.659 & **0.827** & 0.632 \\  & SMOTE & 0.729 & 0.652 & 0.817 & 0.605 \\  & ADASYN & 0.726 & 0.651 & 0.813 & 0.586 \\  & Tomek-links & **0.736** & **0.660** & 0.818 & 0.612 \\  & Borderline-SMOTE & 0.733 & **0.660** & 0.812 & 0.585 \\  & KmeansSMOTE & 0.719 & 0.642 & 0.826 & **0.662** \\ \hline Vehicle0 & Ad-RuLer & **0.977** & **0.941** & **0.996** & 0.897 \\  & SMOTE & 0.960 & 0.931 & 0.993 & **0.915** \\  & ADASYN & 0.958 & 0.922 & 0.994 & 0.894 \\  & Tomek-links & 0.959 & 0.929 & 0.993 & 0.913 \\  & Borderline-SMOTE & 0.959 & 0.927 & 0.994 & 0.905 \\  & KMeansSMOTE & 0.959 & 0.929 & 0.993 & 0.913 \\ \hline Ecoli1 & Ad-RuLer & 0.864 & 0.749 & **0.946** & 0.667 \\  & SMOTE & 0.860 & 0.731 & **0.946** & 0.638 \\  & ADASYN & 0.877 & 0.739 & 0.943 & 0.622 \\  & Tomek-links & 0.861 & 0.732 & **0.946** & 0.637 \\  & Borderline-SMOTE & **0.891** & **0.759** & 0.940 & 0.642 \\  & KmeansSMOTE & 0.846 & 0.728 & 0.944 & **0.669** \\ \hline Yeast3 & Ad-RuLer & 0.890 & 0.685 & 0.963 & 0.570 \\  & SMOTE & 0.885 & 0.685 & 0.964 & 0.575 \\  & ADASYN & **0.907** & 0.655 & **0.965** & 0.509 \\  & Tomek-links & 0.885 & 0.685 & 0.964 & 0.576 \\  & Borderline-SMOTE & 0.906 & 0.650 & 0.964 & 0.502 \\  & KMeansSMOTE & 0.860 & **0.717** & 0.952 & **0.669** \\ \hline Yeast-0-2-5-7-9 & Ad-RuLer & **0.882** & 0.631 & **0.929** & 0.500 \\ vs3-6-8 & SMOTE & **0.882** & 0.661 & 0.923 & 0.535 \\  & ADASYN & 0.855 & 0.512 & 0.922 & 0.365 \\  & Tomek-links & 0.876 & **0.663** & 0.923 & 0.538 \\  & Borderline-SMOTE & 0.864 & 0.562 & 0.916 & 0.430 \\  & KMeansSMOTE & 0.596 & 0.500 & 0.920 & **0.878** \\ \hline Abalone9-18 & Ad-RuLer & **0.834** & **0.425** & **0.937** & 0.291 \\  & SMOTE & 0.792 & 0.355 & 0.884 & 0.238 \\  & ADASYN & 0.794 & 0.348 & 0.884 & 0.231 \\  & Tomek-links & 0.791 & 0.352 & 0.883 & 0.235 \\  & Borderline-SMOTE & 0.723 & 0.309 & 0.851 & 0.211 \\  & KMeansSMOTE & 0.562 & 0.357 & 0.801 & **0.453** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison between different data resampling approaches using logistic regression. Figure 5 displays the average rankings of each algorithm across different datasets and classifiers. As expected, Ad-RuLer leads other algorithms in the average rankings of G-mean, F1 score, and AUC, with average rankings of 2.43, 2.33, and 2.10, respectively. In the average ranking of Precision, Ad-RuLer ranks second, only behind the KmeansSMOTE algorithm, with an average ranking of 3.07. Table 3 presents the results of the Wilcoxon signed-rank test, which targets the performance across all datasets, with Ad-RuLer as the reference algorithm, and assumes a significance level of \(\alpha=0.05\). In the Wilcoxon signed-rank test, the null hypothesis states that no significant difference exists between the performance metrics of the Ad-RuLer and those of the specific comparative algorithm. The results presented in Table 3 indicate that when employing the logistic regression classifier, the \(p\)-values for the AUC metric obtained using five different resampling algorithms are all below 0.05.