# A Classification Method for Incomplete Mixed Data Using Imputation and Feature Selection
###### Abstract  
Data missing is a ubiquitous problem in real-world systems that adversely affects the performance of machine learning algorithms. Although many useful imputation methods are available to address this issue, they often fail to consider the information provided by both features and labels. As a result, the performance of these methods might be constrained. Furthermore, feature selection as a data quality improvement technique has been widely used and has demonstrated its efficiency. To overcome the limitation of imputation methods, we propose a novel algorithm that combines data imputation and feature selection to tackle classification problems for mixed data. Based on the mean and standard deviation of quantitative features and the selecting probabilities of unique values of categorical features, our algorithm constructs different imputation models for quantitative and categorical features. Particle swarm optimization is used to optimize the parameters of the imputation models and select feature subsets simultaneously. Additionally, we introduce a legacy learning mechanism to enhance the optimization capability of our method. To evaluate the performance of the proposed method, seven algorithms and twelve datasets are used for comparison. The results show that our algorithm outperforms other algorithms in terms of accuracy and F1 score and has reasonable time overhead. data imputation; feature selection; particle swarm optimization; legacy learning mechanism; mixed data; classification +
Footnote †: journal: Medical  
data distribution and can impact the integrity of the data structure. The K-Nearest Neighbor (KNN) imputation [12] method is a distance-based method. In particular, the K-nearest neighbors of the missing value are located and their values are utilized to generate the imputation value. The Expectation Maximization (EM) method [13] finds maximum likelihood estimates and creates models of missing values by iteratively executing two main steps: the expectation step, which imputes missing values with their expectation of likelihood based on the current estimate of the mean and covariance parameter, and the maximization step, which recomputes the parameters to maximize the likelihood in the expectation step. Statistics-based methods are easy to comprehend and simple to implement. However, these methods only use the feature information of samples and are unable to effectively utilize the label information of samples, thus defecting the imputation performance. On the other hand, learning-based methods train a learning model to predict and impute the missing data. In tree-based imputation methods, a decision tree model is trained for each feature that has missing values, and the models are utilized for imputing missing values [14]. Clustering imputation methods partition the data into several clusters and utilize the cluster information for dealing with the missing values [15]. Moreover, the Support Vector Machine (SVM) algorithm is employed for data imputation [16], where the SVM regression model is trained using the decision attribute to predict the missing values. In a similar fashion, the Multivariate Imputation by Chained Equations (MICE) method [17] trains the linear regression model multiple times, and the average value of model outputs is used for imputing. Learning-based methods train specific models that can learn the pattern of data distribution to predict the missing values, thereby obtaining better imputation performance. Nevertheless, these methods do not fully consider the randomness of missing data, and this could introduce bias. Evolutionary algorithms are a class of stochastic optimization methods inspired by natural selection and survival of the fittest in biological evolution and have been widely used in plentiful domains [18]. In recent years, the use of evolutionary algorithms in missing data imputation has been extensively investigated and has produced impressive results [19].