# A Selective Dynamic Sampling Back-Propagation Approach for Handling the Two-Class Imbalance Problem
## 2 Selective Dynamic Sampling Approach  
Researchers in the class imbalance problem have shown their interest in finding the best samples to build the classifiers, for example eliminating those samples with a high probability to be noise or overlapped samples [18; 36; 37; 38; 39; 40], or focusing on those close to the borderline decision [11; 13; 41] (the latter has been less explored). In accordance with the above discussion, three categories of samples can be basically identified in the class imbalance literature:  
* Noise and rare or outlier samples. The first ones are instances with error in their labels [7] or erroneous values in the features that describe them, and the last ones are the minority and rare samples located inside the majority class [42]. * Border or overlapped samples are those samples located where the decision boundary regions intersect [18; 38]. * Safe samples are those with a high probability of being correctly labeled by the classifier, and they are surrounded by samples of the same class [42]. Nevertheless, those samples situated close to the borderline decision and far from the safe samples might be of interest; in other words, those that are neither hard nor easy to learn. These samples are called \"average samples\" [35]. In this section, a Selective Dynamic Sampling Approach (SDSA) to train the multilayer perceptron is presented. The aim of this proposal is to deal with the two-class imbalance problem, i.e., this method only works with two-class imbalanced datasets. This SDSA is based on a modification of the \"stochastic\" back-propagation algorithm and derived from the idea of using average samples to train Artificial Neural Networks (ANN), in order to try to improve the classifier performance. The proposed method consists of two steps, and it is described below:  
1. Before training: The training dataset is balanced 100% through an effective over-sampling technique. In this work, we use the SMOTE [8] (SDSAS) and random over-sampling (SDSAO) [16]. 2. During training: The proposed method selects the average samples to update the neural network weights. From the balanced training dataset, it chooses average samples to use in the neural network training. With the aim to identify the average samples, we propose the next function: \[\gamma(\Delta^{q})=exp(-\frac{||\Delta^{q}-\mu||^{2}}{2\sigma^{2}})\] (1) Variable \(\Delta^{q}\) is the normalized difference amongst the real neural network outputs for a sample \(q\), \[\Delta^{q}=\frac{z_{0}^{q}}{\sqrt{(z_{0}^{q}-z_{1}^{q})^{2}}}-\frac{z_{1}^{q} }{\sqrt{(z_{0}^{q}-z_{1}^{q})^{2}}}\] (2) where \(z_{0}^{q}\) and \(z_{1}^{q}\) are respectively the real neural network outputs corresponding to a \(q\) sample. The ANN only has two neural network outputs (\(z_{0}^{q}\) and \(z_{1}^{q}\)), because it has been designed to work with datasets of two classes [43]. The Selective Dynamic Sampling Approach (SDSA) is detailed in Algorithm 1, where \(t_{j}^{(q)}\) and \(z_{j}^{(q)}\) are the desired and real neural network outputs for a sample \(q\), respectively. ```
Input:\(X\) (input dataset), \(N\) (number of features in \(X\)), \(K\) (number of classes in \(X\)), \(Q\) (number of samples in \(X\)), \(M\) (number of middle neurodes), \(J\) (number output neurodes), \(I\) number of iterations and learning rate \(\eta\). Output: the weights \(\mathbf{w}=(w_{11},w_{21},...,w_{NM})\)\(\mathbf{u}=(u_{11},u_{21},...,w_{MJ})\).