# Online Batch Selection for Enhanced Generalization in Imbalanced Datasets
## 5 Results and Discussion
### Mnist  
MNIST [43] is the artificial imbalanced dataset used in the experiments. It consists of 60,000 grayscale 28 \(\times\) 28 images of handwritten digits for training and 10,000 for testing. There are 10 classes (0-9 digits), and it is fairly balanced. In order to make it imbalanced, a sample removing scheme was selected, similar to [46]. For each of the classes, a new dataset will be created, where, from the selected class, 90% of the samples will be removed. In other words, MNIST will be transformed into 10 datasets, where the \(i\)th dataset has the \(i\)th class as the minority class. Testing the algorithms with different minority classes for the same dataset gives us a more clear view of the performance of the models. Moreover, it ensures the robustness of the results. Regarding preprocessing, the images were scaled to \([0.1]\) by dividing them with 255. The architecture of the model can be seen on Table 7.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset Transformation** & **BSBS(k)** & **Re-Enters** & **Noisy** & **Accuracy** & **Balanced Accuracy** & **F1-Macro** & **ROC-AUC** \\ \hline None & - & - & - & **82.06** & 66.00 & 68.45 & 77.52 \\  & 0.15 & - & ✓ & 81.62 & 67.95 & 69.56 & 76.58 \\ \hline RUS & - & - & - & 77.66 & 70.73 & 69.21 & 77.37 \\  & 0.1 & ✓ & - & _77.77_ & **71.28** & 69.02 & 76.93 \\ \hline ENN & - & - & - & 80.36 & 70.42 & 70.47 & 77.50 \\  & 0.15 & ✓ & - & 80.12 & 70.61 & **70.98** & 77.26 \\ \hline ClusterCentroids & - & - & - & 47.10 & 61.06 & 46.64 & 73.18 \\  & 0.15 & - & ✓ & 49.87 & 61.89 & 48.94 & 73.08 \\ \hline Tomek Links & - & - & - & 82.00 & 66.77 & 69.03 & 77.51 \\  & 0.15 & ✓ & ✓ & 81.74 & 67.55 & 69.37 & 77.15 \\ \hline ROS & - & - & - & _77.46_ & 70.67 & 69.10 & **77.54** \\  & 0.1 & - & ✓ & _77.45_ & 71.18 & 69.24 & 76.74 \\ \hline SMOTE & - & - & - & 81.74 & 67.97 & 69.72 & 77.25 \\  & 0.2 & - & - & 81.45 & 68.30 & 69.56 & 76.85 \\ \hline Adasyn & - & - & - & 81.75 & 67.74 & 69.63 & 77.03 \\  & 0.15 & ✓ & - & 81.38 & 68.48 & 69.31 & 76.58 \\ \hline Borderline-SMOTE & - & - & - & 81.76 & 68.05 & 69.69 & 77.33 \\  & 0.1 & ✓ & - & 81.42 & 68.46 & 69.71 & 76.94 \\ \hline KMeans-SMOTE & - & - & - & 81.68 & 67.87 & 69.53 & 76.99 \\  & 0.1 & ✓ & ✓ & 81.71 & 68.06 & 69.69 & 76.93 \\ \hline SMOTEENN & - & - & - & 78.74 & 70.17 & 69.32 & 76.58 \\  & 0.2 & ✓ & ✓ & 79.71 & 70.10 & 69.81 & 76.42 \\ \hline SMOTETomek & - & - & - & 81.76 & 67.50 & 69.37 & 77.42 \\  & 0.2 & ✓ & ✓ & 81.75 & 67.79 & 69.59 & 77.09 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results for the default of creditcard clients dataset.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Units** & **Activation** \\ \hline Conv2D & 32 (3 \(\times\) 3) & ReLU \\ Conv2D & 64 (3 \(\times\) 3) & ReLU \\ MaxPooling & - & - \\ Flatten & - & - \\ Dense & 128 & ReLU \\ Dropout (0.5) & - & - \\ Dense & 10 & Softmax \\ \hline \hline \end{tabular}
\end{table}
Table 7: Model architecture for the MNIST dataset (CNN model).  
In Table 8, the results of the experiments are displayed. The scores of the metrics are the average of the 10 versions of the dataset that we were created. It seems that ROS with NBSBS-R(0.1) performs the best in every metric. For each data-transformation method, the variants of \(BSBS\) have outperformed the vanilla version. In the majority of the situations, \(BSBS\)-\(R\) seems to achieve better scores than \(NBSBS\), with the exceptions of RUS, ClusterCentroids, TomekLinks, and SMOTETomek.  
To observe the results more clearly, Figure 1 exhibits the mean increase in balanced accuracy and F1-score for each of the datasets across all data-transformation techniques. It is evident that \(NBSBS\)-\(R\) helps the generalization performance of most setups that were tested.