# Classification of Imbalanced Data Represented as Binary Features
## 2 Materials and Methods
### Feature Extraction Methods  
Feature extraction is a process meant to create new variables from the existing variables without losing the information of the original dataset. This process is conducted for various purposes, such as to reduce the problem arising from highly dimensional datasets, to increase computational efficiency, or to visualize the dataset either in 2D or 3D.  
PCA is a linear feature extraction method that is commonly used to reduce the dimensions of a highly dimensional dataset. PCA orthogonally transforms the dimensions of a large set of variables into a smaller set of variables known as principal components to identify the correlation and pattern of the original dataset [27]. In this technique, a considered non-significant principal component is excluded, which results in a less dimensional projection while preserving the maximal variance of the dataset. By reducing dimensionality of original dataset, PCA provides an efficient method for data description, visualization, and classification.  
Another linear method of feature extraction using component analysis is independent component analysis (ICA). ICA is an important method in signal-based analysis, such as EEG signals, to help separate normal and abnormal signals. ICA aims to extract the hidden factors of a dataset by transforming the variables to a new set of variables that is maximally independent [28]. What distinguishes ICA from PCA is that PCA assumes that signals are subject to multivariate Gaussian distribution and uses orthogonal bases to decompose signals. It can be concluded that ICA's goal is to find the linear transformation of a highly dimensional dataset in which the basis vectors are non-Gaussian and statistically independent, while PCA's goal is to find an orthogonal transformation that maximizes the variable's variance of the dataset.  
Other than linear dimensionality reduction techniques, there are some available techniques that are nonlinear. One familiar use of these techniques is t-distributed stochastic neighbor embedding (t-SNE), which reduces the dimensionality of a dataset by giving each data point a location in a 2D or 3D dimensional map. This technique aims at identifying the relevant pattern of a dataset while maintaining its local structure [29]. For each point of a dataset, t-SNE models the probability distribution of other points which are closest to it. One of the most important parameters to be set when using t-SNE is perplexity, which is the expected number of nearest neighbors each point has. The performance of t-SNE is fairly robust under different settings of this parameter. Generally, the perplexity is set depending on the size of the dataset. The default value of perplexity in some packages is set to 30 for a dataset whose variables are more than 30.  
Uniform manifold approximation and projection for dimension reduction (UMAP) is another nonlinear dimensionality reduction technique which builds a mathematical theory to justify the graph-based approach [30]. It was developed based on ideas from topological data analysis and manifold learning techniques, which assume that the data are uniformly distributed on the manifold. To make this assumption true, it defines a Riemannian metric on the manifold. Compared to t-SNE, UMAP provides much faster computational running time.