# Online Batch Selection for Enhanced Generalization in Imbalanced Datasets
## 1 Introduction  
The rapid growth of deep learning in recent years is being attributed to the vast development of hardware capabilities, as well as the abundance of available training data. Specifically, the latter is extremely important for the training process of a deep learning model. The vast quantity of data is very helpful for a neural network to achieve a good generalization performance, but the particularities of the data also play a significant role. One basic trait of a dataset is its underlying distribution and its divergence with the theoretical (perfect) distribution of the task at hand. Sometimes, this distribution suffers from imbalances that can make the training of a network difficult, but, more importantly, it can affect, negatively, its generalization performance [1, 2]. A distribution can be balanced when the amount of samples per different and unique observations in a dataset is uniformly distributed. This issue has been referred to in the literature as class imbalance in the context of a classification problem.  
Usually, in a dataset that exhibits some imbalance, there exists two types of classes. There are the classes with a plethora of observations, named majority classes, and classes with a lack of observations, named minority classes. This phenomenon appears quite often in real world datasets and tasks due to various reasons; the most common one being the existence of rare instances of a specific sample that holds great interest for the specific problem. For example, in the field of fraud detection (i.e., fraudulent creditcard detection), there can be many legal transactions with credit cards and only a handful where a fraud actually occurs. However, these rare instances are the backbone of the said task and what a machine learning algorithm should strive to learn. How the model learns to distinguishbetween majority and minority classes is the objective of a wide variety of techniques that battle against this imbalance.  
There are various methods that tackle the data imbalance problem, many of which are discussed in the Related Work section (Section 2). In this work, we are going to elaborate on Batch Selection with Biased Sampling (BSBS), a sampling method used for faster convergence, which was originally introduced in [3]. The method focused on balanced datasets and tried to improve the training process by giving more attention to harder samples. However, in the real world, the vast majority of datasets are imbalanced. In this paper, we are going to examine these kind of datasets and propose two variations to the main BSBS algorithm.These extensions endow BSBS with higher generalization performance on imbalanced datasets. In the experimental part of the paper, we utilize multiple data transformation methods to test the algorithm on different aspects of the same dataset. It is also shown that a hybrid method combining these data-transformation techniques with the enhanced BSBS approach behaves best as far as generalization performance is concerned.  
The main contributions of this paper can be summarized in the following points:  
* A novel set of algorithms consisting of the original BSBS scheme that achieve better F1-score and balanced accuracy on four different imbalanced datasets.
* An analysis of 11 different data-transformation methods and their interaction with the proposed batch selection algorithms, as well as a comparison with 2 state-of-the-art online sampling methods.
* The proposal of a hybrid method that combines the proposed algorithms with the best data transformation techniques, without affecting the convergence rate of the original algorithm.  
The rest of the paper is structured as follows. Section 2 presents related work and Section 3 describes the new methodologies used in this paper. Section 4 outlines the experimental framework, where all the aforementioned techniques are tested. Results are displayed and discussed in Section 5. Finally, conclusions and further research are summarized in Section 6.