# Online Batch Selection for Enhanced Generalization in Imbalanced Datasets
## 6 Conclusions  
To summarize, this work tackles the problem of imbalanced datasets. These types of datasets are common in real world applications. The objective is to develop methods and algorithms in order to train neural networks that achieve better generalization performance. In this study we presented the sampling algorithm \(BSBS\) and proposed two extensions: the first one is through re-enters, while the second one is through added noise in some features of specific samples. The algorithm selects hard examples with certain criteria so that it boosts the learning ability of the network while learning the minority samples adequately.  
Through the experimental evaluation, we demonstrated that the proposed algorithm \(NBSBS\)-\(R\) performs better than the original in four datasets. A variety of data-transformation techniques were tested, and it was shown that our algorithm improved their performance when used in combination. Across all four datasets, the improvement was from \(+0.2\%\) up to \(+5\%\) in some cases regarding _balanced accuracy_ and _F1-score_. It was also shown that the best setup is, usually, a hybrid method of using a data transformation method (such as SMOTE, RUS, ROS, SMOTETomek, or ENN) alongside \(NBSBS\)-\(R\). A comparison was also made with other online sampling methods that showed that \(NBSBS\)-\(R\) achieves higher _F1-score_ and _balanced accuracy_. Finally, it was shown that the hybrid method does not suffer any drawbacks regarding the convergence speed, which the original algorithm strives to achieve.  
For future work, it would be interesting to incorporate to the main algorithm an adaptive batch size selection. Then, the model can also choose the number of selected samples per batch during the training process along with which samples to include. This can be very beneficial in both performance and computational efficiency. Small batches may lead to better generalization, while bigger ones can accelerate the training time. Other future directions can be the reduction of the memory needed for \(NBSBS\)-\(R\) to store the sample losses. This can lead to reducing the computational cost and the training time of the algorithm.  
Conceptualization, G.I.; methodology, G.I., A.S. and G.A.; software, G.I.; validation, G.I.; formal analysis, G.A.; resources, G.A.; writing--original draft preparation, G.I.; writing--review and editing, A.S. and G.A.; visualization, G.I.; supervision, A.S.; project administration, A.S. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Code available at [[https://github.com/geoioannou/NBSBS-R](https://github.com/geoioannou/NBSBS-R)]([https://github.com/geoioannou/NBSBS-R](https://github.com/geoioannou/NBSBS-R)), accessed on 10 January 2023.  
The authors declare no conflict of interest.  
The following abbreviations are used in this manuscript:  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **Ozone** & **Adult** & **Credit Card** & **MNIST** \\ \hline Online Batch Selection & 38.4 \(\pm\) 0.7 & 217 \(\pm\) 1.5 & 79.4 \(\pm\) 1.4 & 527.4 \(\pm\) 2.2 \\ Submodular & 29.1 \(\pm\) 0.6 & 156.7 \(\pm\) 1.5 & 67.5 \(\pm\) 1.1 & 408.3 \(\pm\) 1.2 \\ NBSBS-R (ours) & 9.0 \(\pm\) 0.9 & 45.9 \(\pm\) 1.346 & 38.1 \(\pm\) 1.2 & 204.5 \(\pm\) 2.7 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Comparison of mean training times for each dataset.