# Data Sampling Methods to Deal With the Big Data Multi-Class Imbalance Problem
## 5 Experimental Results and Discussion
\begin{table}
\begin{tabular}{l l l l l l l l l} \hline \hline  & **Method** & **Indian** & **Salinas** & **PaviaU** & **Pavia** & **Botswana** & **KSC** & **Average Rank** \\ \hline \multirow{8}{*}{**Under-sampling**} & Original & 0.8848 & 0.0000 & 0.5369 & 0.0000 & 0.0000 & **11.1** \\ \cline{2-9}  & RUS & 0.8834 & 0.4280 & 0.7369 & 0.8460 & 0.4594 & 0.5640 & **9.5** \\  & TL & 0.8891 & 0.0000 & 0.5272 & 0.0000 & 0.0000 & 0.0000 & **11.1** \\  & ENN & 0.8728 & 0.0000 & 0.3809 & 0.0000 & 0.0000 & 0.0000 & **11.8** \\  & OSS+SMOTE & N/A & N/A & N/A & N/A & N/A & N/A & N/A \\ \hline \multirow{4}{*}{**Over-sampling**} & ROS & 0.9627 & 0.8202 & 0.8723 & 0.8984 & 0.7243 & 0.6752 & **3.7** \\  & SMOTE & 0.9603 & 0.8140 & 0.8704 & 0.8892 & 0.7256 & 0.7284 & **4.4** \\  & ADASYN & 0.9603 & 0.8037 & 0.8633 & 0.8892 & 0.7462 & 0.6974 & **5.1** \\ \hline \multirow{4}{*}{**Hybrid**} & SMOTE+ENN\({}^{\star}\) & 0.8201 & 0.9542 & 0.8797 & 0.9074 & 0.8139 & 0.7863 & **2.5** \\  & SMOTE+ENN & 0.9610 & 0.8138 & 0.8670 & 0.8879 & 0.7346 & 0.7076 & **5.0** \\  & SMOTE+TL & 0.9610 & 0.8068 & 0.8609 & 0.8812 & 0.7410 & 0.7177 & **5.0** \\  & SMOTE+OSS & 0.0000 & 0.0000 & 0.8684 & 0.8888 & 0.7372 & 0.7410 & **6.8** \\  & TL+SMOTE & 0.9604 & 0.7706 & 0.8522 & 0.8747 & 0.6554 & 0.6079 & **7.5** \\  & SMOTE+CNN & 0.0000 & 0.0000 & 0.8417 & 0.0000 & 0.8657 & 0.8200 & **7.7** \\  & ENN+SMOTE & 0.9274 & N/A & 0.6331 & 0.4996 & N/A & N/A & N/A \\  & CNN+SMOTE & N/A & N/A & N/A & N/A & N/A & N/A & N/A \\  & OSS+SMOTE & N/A & N/A & N/A & N/A & N/A & N/A & N/A \\ \hline \hline \end{tabular}
\end{table} 
Table 3: DL-MLP performance using the _g-mean_ (Equation (5)) as measure. Values presented correspond to averages of ten different initializations of the neural network weights. of the data to increase the classification performance, but when they are added to any over-sampling method, the classification performance is improved. However, Table 3 does not show that _g-means_ from SMOTE+ENN and SMOTE+TL overcome the _g-mean_ of SMOTE. Analyzing the SMOTE+CNN and SMOTE+OSS methods, a similar classification performance behavior was observed. CNN, ENN, TL and OSS are effective methods with which to improve the training dataset quality in nearest neighbor rule context; in neural networks, only ENN combined with SMOTE archives positive results, but it does not generate better results than SMOTE. In addition, CNN+SMOTE, OSS+SMOTE and ENN+SMOTE show values N/A (i.e., does not apply), which make reference to situations where these editing methods eliminate too many samples from a particular class. SMOTE is not applicable in such cases because it needs a least six samples by class; some classes of those methods only keep less than six samples, and it is proposed to not consider these values for this reason. CNN is a method to keep a consistent subset from the training data, in the way that all samples in a consistent subset are successfully classified by nearest neighbor rule [28]. Therefore, methods that are successful by nearest neighbor rules are not necessarily effective in a neural network context. ENN, TL, OSS and CNN are methods based in nearest neighbor rule [73], whose decision boundaries are defined locally in the feature space. However, the decision boundary in MLP neural networks is defined in the hidden space, not in the feature space [8]. Thus, it is assumed that to apply these editing methods in the data features space, i.e., in the training data, is not the best option because these methods help to determine the decision boundary to nearest neighbor rule, not for neural network classifier. For this reason, in order to remove samples that make it difficult to set the neural network decision boundary, the ENN is applied in the neural network output in this work. The proposed method SMOTE+ENN* (Algorithm 1) consists of training the neural network during \(I_{init}=250\) epochs, and next applying ENN in the neural network output.