# A Density-Based Random Forest for Imbalanced Data Classification
## 3 Methods
### DBRF Algorithm
Figure 6: Three real-world datasets on the (**a**) metal glass, (**b**) vehicle evaluation, and (**c**) [PERSON]’s survival with different \(p\). F1 focused on the minority class: comparisons with different \(p\). The horizontal axis is \(p\), and the vertical axis is _accuracy_, F1, and GM in the different sub-figures. For metal glass, \(\eta=80\); for [PERSON]’s survival, \(\eta=2\); for vehicle evaluation, \(\eta=1\). #### 4.3.2 Performance on Public Imbalanced Datasets

In this experiment, we first compared random forest with DBRF and then the random forest combined with oversampling methods through the remaining 19 datasets and 12 synthetic datasets. The oversampling methods included SMOTE, borderline SMOTE, and ADASYN. The comparison experiment was conducted to evaluate whether DBRF is effective at handling imbalanced datasets. In Table 3, (A) shows the F1 and GM results for metal glass, vehicle evaluation, and [PERSON]'s survival. Figure 8 shows the averaging confusion matrices of metal glass. In Figure 8, DBRF exhibited better prediction accuracy for BMG than RF. Compared to other algorithms, DBRF was less competitive, but had better performance on F1 and GM. Figure 9 shows the averaging AUC curve and ROC of metal glass. DBRF had good performance on the AUC and ROC curve compared with the other algorithms. In Table 3, (B) shows the F1 and GM results for the 19 public imbalanced datasets. DBRF improved F1 by 2-5% compared with the other models. DBRF was more competitive than the other models in most imbalanced datasets. ADASYN + RF, RF, b-SMOTE + RF, and ADASYN + RF were more competitive than DBRF on glasses1, dermatology, pageb locks0, Vehicle0, Vehicle1, Vehicle1, and Vehicle2. DBRF improved the GM by 2-6% compared with the other models. Specifically, DBRF was more competitive than RF on the 19 public imbalanced datasets. SMOTE + RF, b-SMOTE + RF, and ADASYN + RF outperformed DBRF on glassed1, car-good, car-vgood, page-blocks0, Vehicle0, Vehicle1, and Vehicle2. In Table 3, (C) shows the average value of 12 synthetic datasets after 10-fold cross-validation. Among all 12 datasets, the difficulty increased with the number of interference samples. For example, in Paw0, the boundary between the minority class and the majority class was clear, and F1 and GM were generally high. However, in the Paw70 dataset, the boundary was unclear. As the number of interference samples increased, F1 and GM began to decline, indicating that the decision boundary between the minority class and the majority class began to become difficult to distinguish. For F1 and GM, _DBRF_ was best on most datasets. In particular, DBRF was more competitive than RF on GM. This indicates that our proposed model can achieve an improvement in the minority class prediction accuracy over prior methods while remaining relatively stable even when the decision boundary is very difficult to distinguish. Figure 7: Box plot of 10-fold cross-validation on the on the (**a**)metal glass, (**b**) vehicle evaluation, and (**c**) Harberman’s survival with the best \(p\) and \(\eta\). Figure 8: Averaging confusion matrices of (**a**) RF; (**b**) DBRF; (**c**) SMOTE + RF; (**d**) bSMOTE + RF; (**e**) ADASYN + RF. “BMG” is the minority class. We used the Wilcoxon signed rank test, which is a non-parametric test, to statistically analyze the results of the experiments on 34 datasets. The function of the Wilcoxon signed-rank test is to determine whether the corresponding overall distribution of the data is the same without assuming that the data obey a normal distribution. In this test, the confidence interval was set at 0.05. When the test yields a p-value significantly less than 0.05, the two methods are significantly different. From Table 5, it can be seen that the p-values for DBRF versus RF, SMOTE + RF, b-SMOTE + RF, and ADASYN + RF were all less than 0.05. This suggests that the differences between them were significant. It also indicates that DBRF was more competitive than the other models in handling imbalanced data.