# A Density-Based Random Forest for Imbalanced Data Classification
## 3 Methods
### DBRF Algorithm
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Dataset Name** & **Sample Size** & **Feature Number** & **Majority Class Number** & **Minority Class Number** & **Imbalance Ratio** \\ \hline A & Metal glass & 5936 & 98 & 3708 & 675 & 5,493 \\  & Vehicle evaluation & 1728 & 6 & 1210 & 518 & 2.3359 \\  & Haberman survival & 306 & 4 & 225 & 81 & 2.7778 \\ \hline  & glasses0 & 214 & 9 & 144 & 70 & 2.06 \\  & glasses1 & 214 & 9 & 138 & 76 & 1.82 \\  & glasses5 & 214 & 9 & 205 & 9 & 22.81 \\  & Ecoil1 & 336 & 7 & 259 & 77 & 3.36 \\  & Ecoil2 & 336 & 7 & 284 & 52 & 5.46 \\  & Ecoil3 & 336 & 7 & 301 & 35 & 8.19 \\  & Ecoil0-1 & 244 & 7 & 220 & 24 & 9.17 \\  & car-good & 1728 & 6 & 1659 & 69 & 24 \\  & car-vgood & 1728 & 6 & 1663 & 65 & 25.58 \\ B & Cleveland & 173 & 13 & 160 & 13 & 12.62 \\  & dermatology & 358 & 34 & 338 & 20 & 16.9 \\  & page-blocks0 & 5472 & 10 & 4913 & 559 & 8.77 \\  & Vehicle0 & 846 & 18 & 647 & 199 & 3.23 \\  & Vehicle1 & 846 & 18 & 629 & 217 & 2.52 \\  & Vehicle2 & 846 & 18 & 628 & 218 & 2.52 \\  & Vehicle3 & 846 & 18 & 634 & 212 & 2.52 \\  & Wisconsin & 683 & 9 & 444 & 239 & 1.86 \\  & Yeast1 & 1484 & 8 & 1055 & 429 & 2.46 \\  & Connectionist Bench & 990 & 13 & 900 & 90 & 10 \\ \hline  & Clover0 & 800 & 2 & 700 & 100 & 7 \\  & Clover30 & 800 & 2 & 700 & 100 & 7 \\  & Clover50 & 800 & 2 & 700 & 100 & 7 \\  & Clover70 & 800 & 2 & 700 & 100 & 7 \\  & Subclus0 & 800 & 2 & 700 & 100 & 7 \\  & Subclus50 & 800 & 2 & 700 & 100 & 7 \\  & Subclus70 & 800 & 2 & 700 & 100 & 7 \\  & Pav0 & 800 & 2 & 700 & 100 & 7 \\  & Pav0 & 800 & 2 & 700 & 100 & 7 \\  & Pav30 & 800 & 2 & 700 & 100 & 7 \\  & Pav70 & 800 & 2 & 700 & 100 & 7 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Detailed description of the three experimental datasets. \[F1=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{7}\] \[GM=\sqrt[n]{\prod_{i=1}^{\infty}recall_{i}} \tag{8}\]

### Experimental Results and Analysis

#### 4.3.1 Parameter Setting on \(\eta\) and \(p\)

We compared the random forest and DBRF algorithms on the metal glass, vehicle evaluation, and [PERSON]'s survival data. To make the experimental results more comparable, the relevant parameters involved in the models were consistent. The RF model only involves the number of trees \(size=100\). The DBRF model involves neighborhood distance \(\eta\), minimum number of samples in the domain \(\omega=10\), allocation ratio of trees \(p\), and number of trees \(size=100\). In DBRF, owing to the different datasets, the neighborhood distance \(\eta\) correspondingly differed. First, \(\eta\) was set as the average of the nearest neighbor distances of 20 majority class samples of a certain minority sample. Then, we adjusted the parameter based on \(\eta\). Figure 4 shows the trend of the density domain as \(\eta\) changes. Figure 4 indicates that as \(\eta\) increases, the majority class samples gradually increased until all the majority class samples were selected. With the increase of the selected majority class samples, the role played by the density domain gradually became smaller, which deviated from our aim in proposing DBRF to improve the prediction of minority classes through additional training boundary samples. Therefore, choosing a suitable \(\eta\) significantly impacts the performance of predictions involving minority classes. Figure 5 presents the experimental results obtained on the metal glass, vehicle evaluation, and [PERSON]'s survival data with different values of \(\eta\). Each point in the figure represents the average value of the ten-fold cross-validation experiments. The experimental results were consistent with our expectations. For example, as \(\eta\) increased, the accuracy gradually increased, whereas F1 and GM gradually decreased. This means that the role of the density domain gradually weakened. When all the samples were included, it was equivalent to training a random forest classifier with the original dataset, nullifying the performance improvement of the proposed approach.