## 4 Experiments
### Research Question and Results
Then the paired Wilcoxon test turns out that differences between ABC and DTC are not significant because the null hypothesis on no significant difference between ABC and DTC cannot be rejected (\(p\)-value of recall, G-mean and AUC are 0.433, 0.396, 0.753, respectively). Furthermore, combined with the average rank we have found that ABC performs well in both small and large datasets (rank second in small-sample-size datasets and rank first in large-sample-size datasets), the result of Wilcoxon rank sum test support the null hypothesis that no significant difference between large and small datasets for ABC (\(p\)-value of recall, G-mean and AUC are 0.558, 0.661, 0.539, respectively). The result reflects that the performance of ABC is not affected by sample size of datasets. \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Metric** & **Friedman Test** & **Nemenyi Analysis (CD)** & \(\alpha\) \\ \hline \multirow{3}{*}{Small datasets} & Recall & \(<\)\(<\)0.00001 & 3.003 & \(\alpha\) = 0.05 \\  & G-mean & \(<\)\(<\)0.00001 & 3.003 & \(\alpha\) = 0.05 \\  & AUC & \(<\)\(<\)0.00001 & 3.003 & \(\alpha\) = 0.05 \\ \hline \multirow{3}{*}{Large datasets} & Recall & \(<\)\(<\)0.00001 & 3.332 & \(\alpha\) = 0.05 \\  & G-mean & \(<\)\(<\)0.00001 & 3.332 & \(\alpha\) = 0.05 \\ \cline{1-1}  & AUC & \(<\)\(<\)0.00001 & 3.332 & \(\alpha\) = 0.05 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: \(p\)-Value and CD value of differences among classifier result on different datasets. Figure 5: Average result value of 3 metrics for 9 classifiers. RQ2: How does LIMCR perform compared with other imbalance learning methods in small sample size of datasets? Motivation: We have selected the baseline classifier GNB in small sample size in RQ1. Now, we need to validate the effectiveness of our proposed LIMCR. Approach: To solve the class imbalance problem, researchers usually use two kinds of methods: resampling methods based on data level and classification methods based on algorithm level. Resampling methods are always sorted into three categories: over-sampling methods, under-sampling methods and combination of over- and under-sampling methods. The question that which method is more suitable for the class imbalance problem has been discussed in many research [4; 60; 61]. To evaluate the effectiveness of LIMCR, we employ six baseline imbalance learning methods with parameters which are listed in Table 7. All these baseline methods are implemented in the Imbalanced-learn model in Python [62]. In this experiment, we compare performance metrics (balancedscore and G-mean) of our LIMCR with baseline imbalance learning methods in small sample size of benchmark datasets. All imbalance learning methods including LIMCR are combined with baseline classifier GNB. Results: Tables 8 and 9 present the results of our LIMCR and the baseline imbalance learning methods on small datasets in terms of balancedscore and G-mean, respectively. We notice that the average balancedscore and G-mean of LIMCR are 0.701 and 0.69 which perfoms better than other baseline imbalance learning methods. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Level** & **Category** & **Imbalance Learning Method** & **Abbreviation Parameters** \\ \hline \multirow{4}{*}{Data} & Over-sampling & Borderline-SMOTE [22] & B-SMO & Default \\  & Under-sampling & Neighbourhood Cleaningrule [7] & NCL & Default \\  & Combination of over- and & Instance hardness threshold [34] & IHT & Default \\  & under-sampling methods & SMOTE+ENN [25] & SMOE & Default \\ \hline \multirow{2}{*}{Algorithm} & \multirow{2}{*}{Algorithm} & RUSBoost & RUSB & Default \\  & & EasyEnsemble & EasyE & Default \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Baseline imbalance learning methods.