## 4 Experiments
### Research Question and Results
\begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline  & **LIMCR** & **SMOE** & **IHT** & **B-SMO** & **NCL** & **RUSB** & **EasyE** \\ \hline \multirow{2}{*}{synapse-1.2} & 0.729 & 0.699 & 0.715 & 0.627 & 0.711 & 0.537 & 0.682 \\  & 0.703 & 0.669 & 0.72 & 0.73 & 0.689 & 0.53 & 0.645 \\  & 0.692 & 0.684 & 0.643 & 0.617 & 0.594 & 0.568 & 0.522 \\  & 0.772 & 0.645 & 0.602 & 0.684 & 0.782 & 0.511 & 0.623 \\  & 0.758 & 0.708 & 0.64 & 0.692 & 0.708 & 0.556 & 0.66 \\  & 0.78 & 0.754 & 0.717 & 0.709 & 0.763 & 0.588 & 0.668 \\  & 0.608 & 0.616 & 0.59 & 0.594 & 0.605 & 0.543 & 0.58 \\  & 0.656 & 0.665 & 0.658 & 0.578 & 0.605 & 0.56 & 0.603 \\  & 0.693 & 0.787 & 0.761 & 0.719 & 0.65 & 0.502 & 0.493 \\  & 0.68 & 0.65 & 0.68 & 0.68 & 0.684 & 0.55 & 0.668 \\  & 0.758 & 0.759 & 0.727 & 0.73 & 0.722 & 0.705 & 0.76 \\  & 0.561 & 0.659 & 0.583 & 0.443 & 0.616 & 0.622 & 0.554 \\  & 0.741 & 0.754 & 0.723 & 0.658 & 0.554 & 0.402 & 0.679 \\  & 0.735 & 0.857 & 0.605 & 0.687 & 0.677 & 0.488 & 0.691 \\  & 0.654 & 0.54 & 0.608 & 0.811 & 0.679 & 0.624 & 0.698 \\ \hline \multirow{2}{*}{Average} & **0.701** & 0.696 & 0.665 & 0.664 & 0.669 & 0.552 & 0.635 \\  & **2.467** & 2.767 & 3.933 & 4 & 3.567 & 6.4 & 4.867 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Balancedscore of GNB with different imbalance learning methods on small datasets. In further study, \(p\)-value in Friedman test in these two performance metrics are all smaller than 0.00001, which shows that significant difference is existing among the seven methods. Then, the Nemenyi post hoc test shows CD = 2.326, \(\alpha\) < 0.05. According to Nemenyi post hoc test, we underline the average ranks which significantly worse than our LIMCR. The results reflect that ensemble algorithms performs significantly worse than resampling method combined with GNB. In order to find out if there is any significant difference between resampling methods, we perform Wilcoxon signed-rank test between our LIMCR and other resampling methods, the \(p\)-values of each test are listed in Table 10.  
From Table 10 we can learn that SMOE performs no significant difference from LIMCR, but for the Friedman average ranks, LIMCR performs slightly better than SMOE. The overall metrics balancedscore and G-mean of IHT shows significantly worse than LIMCR. Another two methods B-SMO and NCL perform significantly worse than LIMCR in most of datasets for balancedscore and G-mean. Therefore we conclude that LIMCR performs better than most of imbalance learning methods. RQ3: How does LIMCR work with other classifiers? Motivation: In principle analyses, our LIMCR is based on Bayesian Probability, and GNB is the most suitable classifier for it. However, we expect that LIMCR still has good performance when it is combined with other classifiers. Approach: In this experiment, we choose another two well performed classifiers ABC and DTC, with the combination of three resampling methods LIMCR, SOME and IHT on small datasets in terms of balancedscore and G-mean for further comparison. Results: Tables 11 and 12 show the results of matching of three classifiers with three resampling methods on small datasets in terms of balancedscore and G-mean. We notice that the average balancedscore value of LIMCR+GNB is 0.701 which is equal to IHT+ABC and higher than other combinations. Simultaneously, the average G-mean of LIMCR+GNB is 0.69 while it is 0.696 for IHT+ABC. \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **LIMCR vs. SMOE** & **LIMCR vs. IHT** & **LIMCR vs. B-SMO** & **LIMCR vs. NCL** \\ \hline Balancedscore & 0.532 & 0.043 & 0.031 & 0.041 \\ G-mean & 0.691 & 0.047 & 0.198 & 0.011 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: \(p\)-Value of Wilcoxon signed-rank tests of resampling methods.