# Online Batch Selection for Enhanced Generalization in Imbalanced Datasets
## 2 Related Work  
In this section, we are going to discuss some popular methods regarding imbalance, some of which are going to be employed later on during the experiments. The advantages and disadvantages of each method are going to be highlighted, along with our contribution in the literature. The methods for addressing dataset imbalance are mainly divided into two main categories. The first type involves transforming the dataset and its underlying distribution into a more balanced state. The second is based around the learning algorithm (i.e., classifier) and the way it tackles the imbalance without changing the training dataset. Both types of methods have been used in the literature in various ways, some of which will be described below. The most common techniques to tackle imbalance are dataset transformation methods. Their goal is to change the structure of the dataset by adding, removing, or transforming samples of the dataset in order to make it easier for models to learn to distinguish the minority classes. Such techniques are oversampling, undersampling, and the combination of the two, all of which are discussed in what follows. Oversampling is one of the most popular techniques that address imbalance. Random oversampling (ROS) is the most simple version of it, where the samples of the minority classes are duplicated multiple times in order to reach the number of samples of the majority classes. It is a method that does not require any heavy computational cost and can be used successfully alongside deep learning networks, as well as many classical machine learning techniques. However, there are some disadvantages of ROS. The main drawback occurs when the minority classes display very low diversity. This happens in situations of extreme imbalance, when the number of samples of the minority class is very low. The multiple replications of the small in variety minority classes leave the overall distribution far from a realistic one, which may lead to overfitting. Various techniques have been developedto alleviate this issue. One of the most used is SMOTE [4]. This method utilizes the \(k\)-nearest neighbors (\(k\)-NN) algorithm [5] to select similar samples of the minority classes and constructs new samples randomly by mixing them together. Thus, SMOTE produces new synthetic data that enrich the dataset altering the underlying distribution to a more realistic one. Many variants of SMOTE have developed over the years. Borderline-SMOTE [6] is a variant that strives to improve the oversampling process by selecting samples that lie close to the border of each class. These samples are more apt to be misclassified than samples far from the borderline. Thus, borderline-SMOTE finds the borderline examples of the minority classes and oversamples them. Another recent variant of SMOTE is the K-means-SMOTE [7], which utilizes the K-means algorithm to divide the samples into clusters. Then, it selects the clusters with the greater imbalance and computes a sampling weight for each cluster. The sampling weight is later used to calculate the number of new samples to be generated from the minority classes for each of the selected clusters. The sampling, finally, is performed using the SMOTE algorithm. Another useful oversampling technique is Adasyn [8], which is considered an improvement of SMOTE, as it uses \(k\)-NN to find \(k\) neighbor samples of each minority sample. However, the difference lies in the amount of samples that are generated. Adasyn calculates the number of samples to be generated based on the distribution of the neighborhood of each minority sample. Undersampling is the other popular data transformation technique that tackles imbalance. In contrast to oversampling, undersampling removes samples from the majority class. Specifically, the simplest algorithm, random undersampling (RUS), selects samples from the majority classes at random and removes them until both majority and minority classes have the same amount of samples.