“3. Common approaches for addressing data imbalance
Various methods have been developed to handle the class imbalance problem that can be broadly divided into two main approaches: Data driven and algorithm driven. The former attempts to balance the class distribution [8]. The latter approach adjusts the learning algorithm, or the classifier without amending the training set [10]. In the next sub-sections, we briefly review these approaches and reveal their advantage and disadvantages.

3.1. Data driven approaches
Data driven approaches adjust the class ratio in the input dataset to achieve balanced class distribution. This approach often employs sampling techniques like under-sampling, oversampling, or a combination of both [2].

3.1.1. Under-sampling
The primary under-sampling technique that arbitrarily eliminates examples of the majority class to balance the dataset is known as random under-sampling [28]. Researchers such as [18] reveal the primary drawback associated with this method to be the disposal of useful information that can prove to be crucial in the later classification stages. Many under-sampling approaches have been proposed such as Condensed Nearest Neighbour Rule (CNN), Wilson Edited Nearest Neighbour Rule (ENN), Neighbourhood Cleaning Rule (NCL), Tomek Links, and One-sided selection (OSS) [20], [29], [43], [44].

Wilson [44], proposes the CNN method in which if the class label of any example is different from the class of at least two of its nearest three neighbours, then the example is eliminated from the dataset. Tomek [43] proposed the Tomek Links method, which is a modified CNN that considers only the data points close to the boundary to be important. In this method two samples, Ei and Ej, belonging to two different classes are considered together with the distance D(Ei, Ej) between them. The Tomek Link method states that if there does not exist an example E0 such that D(Ei, E0) < D(Ei, Ej) or D(Ej, E0) < D(Ei, Ej) then pair (Ei, Ej) is a Tomek Link. The formation of a Tomek Link implies the existence of a noisy example or else examples on the borderline [5]. The majority class examples in such Tomek Links are removed to address the class imbalance problem.

Laurikkala [29] proposes the NCL method which utilizes three nearest neighbours for each example (Ei) in the training dataset. When Ei belongs to the majority class, and its three nearest neighbours have a different class, then Ei is removed. However, if Ei belongs to the minority class and its three nearest neighbours are different than those neighbours that belong to the majority class, then they are removed. The drawback of this method is that it can lead to an enormous number of computations when the input dataset is large and comprises a large number of majority class instances [10]. Anwar [5] highlights another drawback in the ENN and NCL methods, which is class overlapping. The author states that when the degree of class overlapping in the training set is high, then the classifier built will be poor in terms of predictive accuracy, as the majority class examples situated close to the decision boundaries will be deleted. Tomek Links is similar to NCL in that it may lead to performance deterioration of the classifiers when the input dataset is large and with a high degree of overlapping among class values.

Johnson and Khoshgoftaar [25] surveyed different approaches to class imbalance related to Artificial Neural Networks The authors studied unstructured datasets such as images and text. Their research addressed issues related to experimental design, implementation and evaluation of deep learning techniques in applications that are linked with data imbalance such as medical diagnosis and fraud detection, among others. Challenges related to complexity, output format, results and learning mechanisms have been discussed in the survey especially within the context of deep learning. The authors concluded that the class imbalance problem has not been sufficiently covered within deep learning and believed their approach could help improve the performance when the input data is imbalanced. Aytuğ Onan [48] proposed an under-sampling technique based on consensus clustering to improve performance of imbalanced datasets. In using this approach, observations linked with the dominant class in the training dataset are under-sampled using the consensus clustering algorithm. Several different small imbalanced datasets and two large datasets were utilized to test the new under-sampling technique. In the experiments, different classification algorithms including Nave Bayes and Regression among others have been used to derive classifiers and show the impact of the proposed under-sampling technique. The results reported by the classification algorithms when using the consensus clustering method in the pre-processing phase yielded better predictive accuracy on the datasets when compared to not using sampling techniques.

3.1.2. Oversampling
Oversampling is another common sampling approach used to deal with an imbalanced class problem. Various oversampling strategies are available including random oversampling, focused oversampling, and synthetic sampling [11], [13], [27]. The method in which the instances of the minority class are randomly replicated until they have equal representation is known as random oversampling [8]. Random oversampling has two major shortcomings: it increases the possibility of overfitting of the classifier on the training dataset, and if the original data already has high dimensionality, it mounts the computation cost thus increasing the training time of the classifier [10]. In focused oversampling, only those minority class values with samples occurring on the boundary between the majority and minority class values are resampled. However, Chawla et al. [11] state that these methods of oversampling by replication lead to a more specific decision region of the minority class. To overcome this and broaden this decision region, they propose an advanced heuristic oversampling technique called the Synthetic Minority Oversampling Technique (SMOTE). SMOTE is a technique in which oversampling of the minority class is carried out by generating synthetic examples. These new synthetic minority class examples are the result of interpolation between closely located minority class samples. Chawla et al. [11] describe the process of SMOTE as calculating the nearest same-class neighbours for every minority example and then based on the required oversampling rate, randomly choosing from these neighbouring examples. The synthetic examples are then generated at random points along the line segments joining the minority examples with these chosen neighbours. This process expands the decision region pertaining to the minority class. The authors tested SMOTE on a wide variety of datasets with different training sizes and degree of imbalance. It was observed that the larger and less specific decision regions of SMOTE resulted in this method outperforming oversampling by replacement.

3.2. Algorithm-driven approach
In this approach, the classification algorithm is adjusted to facilitate the learning task specifically with respect to the minority class. In this approach, no changes are made to the input data distribution. This approach includes cost sensitive learning, thresholding, and hybrid methods like ensemble learners among others.

3.2.1. Cost-sensitive learning
The basic assumption by any traditional classification algorithm is that there is a cost of misclassification is the same for all response variable values [10], [14]. However, in real world applications the cost of misclassification may vary among target class values. For example, in the field of medical diagnosis predicting that a person suffers from a particular disease when he does not will lead to more medical examinations. On other hand, misclassifying a person as healthy when he actually suffers from the disease can lead to excessive health costs and potential fatality. In this situation, the cost of misclassification is higher, so the classifier ought to be sensitive to the type of application and more importantly the cost associated with misclassification [8], [35]. To address this issue, cost-sensitive learning was proposed, so if the underlying dataset has a class ratio of 1:3 favouring the majority class, then the misclassifying cost for the minority class will be three times that of the majority class [24]. An important concept in cost-sensitive learning is the Cost Matrix [8] which comprises the average cost of misclassification for each available class. The goal of this method is to minimize misclassification cost by choosing a class with minimum conditional risk [28]. The layout of the cost matrix is depicted in Table 1 below, where λij is the cost of misclassifying a sample from True Class j to Class i.

Table 1. Cost matrix.

Empty Cell Empty Cell Prediction
Empty Cell Empty Cell Class i Class j
True Class i 0 λij
Class j λji 0
The diagonal entries indicate correct classification, hence the cost associated with this is zero. Other common cost-sensitive learning models are MetaCost which is a wrapper method of making the classifier cost sensitive, and AdaCost which is a cost-sensitive version of AdaBoost [18], [35].

3.2.2. Thresholding methods
Thresholding is a method used to address the problem of class imbalance in classification at the algorithmic level. Buda [8] defines the set of methods used in a classifier to adjust the decision boundary as threshold moving or the post-scaling method. Certain algorithms, like Nave Bayes, generate probability estimates that are converted into predictions by cutting off at the threshold which is usually 0.5 (Radwan [35]). The predicted class label depends on the associated probability lying above or below the threshold. However, when the class is imbalanced the performance of many classifiers is affected and thus the decision threshold of the classifiers needs to be adjusted such that the cost of misclassifications is reduced, and the prediction performance is improved.

Zou et al. [46] obtain the decision threshold by greedily adjusting the F-score. The derived probabilities are evaluated to calculate the training set threshold with the best F-score using cross validation. Thereby, the threshold for the test data is determined using Eq. (1)”
From the above paper, summarize and explain each method for tackling the imbalanced dataset problem in deep learning, and their pros and cons. Try to use your own words, do not copy the sentences directly from the above paper.