# Classification of Imbalanced Data Represented as Binary Features
###### Abstract  
Typically, classification is conducted on a dataset that consists of numerical features and target classes. For instance, a grayscale image, which is usually represented as a matrix of integers varying from 0 to 255, enables one to apply various classification algorithms to image classification tasks. However, datasets represented as binary features cannot use many standard machine learning algorithms optimally, yet their amount is not negligible. On the other hand, oversampling algorithms such as synthetic minority oversampling technique (SMOTE) and its variants are often used if the dataset for classification is imbalanced. However, since SMOTE and its variants synthesize new minority samples based on the original samples, the diversity of the samples synthesized from binary features is highly limited due to the poor representation of original features. To solve this problem, a preprocessing approach is studied. By converting binary features into numerical ones using feature extraction methods, succeeding oversampling methods can fully display their potential in improving the classifiers' performances. Through comprehensive experiments using benchmark datasets and real medical datasets, it was observed that a converted dataset consisting of numerical features is better for oversampling methods (maximum improvements of accuracy and F1-score were 35.11% and 42.17%, respectively). In addition, it is confirmed that feature extraction and oversampling synergistically contribute to the improvement of classification performance. b 23 August 20  
the data level, resampling is a widely method adopted to balance the class distribution. This method includes undersampling the majority class and oversampling the minority class. The most well-known data-level approach for oversampling imbalanced data is the synthetic minority oversampling technique (SMOTE) [5], which creates synthetics samples using \(k\)-nearest neighbor to balance the dataset instead of replicating the minority data. After [5] introduced this technique, some approaches started to be developed based on SMOTE and focused on overcoming its drawback(s) [6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16]. Despite the fact that opinions about the best resampling method for the class imbalance problem vary, numerous studies have found that oversampling generally outperforms undersampling in dealing with the class imbalance problem [17; 18; 19]. At the algorithm level, a method is proposed to modify the algorithm by either changing or adding another line of algorithm to solve this problem. Ensemble learning such as AdaC2 [20] and cost-sensitive learning such as cost-sensitive decision tree [21] are some examples of algorithm-level methods. However, many of the available methods were specifically developed to overcome this problem in numerical datasets or mixed datasets of numerical and categorical data. In contrast, some types of data are represented as binary values that can only take one of two values (0/1, T/F, M/F, etc.). Such binary features are located on the border of numerical and categorical values. They can be treated as numerical values; however, the domain of value is quite poor and presents no difference from a categorical value. Therefore, for a dataset consisting of binary features, direct application of oversampling methods is not a good idea, since such methods rely on numerically represented values for synthesizing new minority samples. In this paper, we investigated an approach using feature extraction for converting binary features into numerical features prior to oversampling. Though it is popular to combine an oversampling algorithm with a feature extraction algorithm such as principal component analysis (PCA), the purpose of this combination is usually dimension reduction [22], and the positive effect of feature extraction from binary features followed by oversampling has not been analyzed in detail.