## 4 Experiment and Result Analysis
### Supervised Feature Selection
#### 4.2.3 Experimental Results  
(1) Experiments on Synthetic Data Sets  
We used the number of irrelevant features, selected by means of SFS-ID and other state-of-the-art approaches, as a performance index in the experiments on synthetic data. The results are shown in Figure 4.  
Based on the results shown in Figure 4, we concluded that the proposed SFS-ID approach clearly outperformed other state-of-the-art SFS approaches. In particular, with the increase in the missing ratio, the number of irrelevant features of all methods increased, but SFS-ID was still able to select the only two relevant features in the presence of 55% of missing values. Hence, if the missing ratio is excessively large, our SFS-ID can choose relevant features and thus improve the classification performance.  
(2) Experiments on Real Data Sets  
We used the features selected by the feature to build an SVM classification model. As an evaluation metric, classification accuracy (Acc) was used to measure the performance of different approaches.  
We selected the features with different missing instance ratios to perform classification tasks, and the classification results are shown in Figures 5-7.  
Based on the results shown in Figures 5-7, we concluded that our SFS-ID approach achieved the best classification performance compared to other competing SFS methods  
Figure 4: The number of irrelevant features selected, together with two relevant features.  
Figure 5: Accuracy under different missing ratios for DLBCL and Mnist datasets. (**a**) DLBCL. (**b**) Mnist.  
Figure 6: Accuracy under different missing ratios for Splice and Wpbc datasets. (**a**) Splice. (**b**) Wpbc.  
Figure 7: Accuracy under different missing ratios for USPS and Arcene datasets. (**a**) USPS. (**b**) Arcene.  
on incomplete data. In addition, the classification results of our SFS-ID approach were statistically significantly better than those of all comparison methods in terms of ACC. In particular, it can easily be verified from the figure that the performance of the SFS-ID approach showed a great improvement in the large and small incomplete data sets. Through further analysis of the other experimental results, we drew the following conclusions.  
First, as the incomplete instances ratio increased, the performance of all schemes sharply dropped. For instance, on the Mnist data set, the ACC of all schemes dropped by 6.9% on average at a ratio of 0.45, compared with a ratio of 0.65. In addition, on the USPS data set, the ACC of all schemes dropped on average by 5.78% at a ratio of 0.25, compared with a ratio of 0.65. It also can be verified that most of schemes achieved the best performance with a small ratio, which shows that the number of complete instances played an important role in those feature selection schemes.  
Second, compared with those of other approaches, the ACC of our SFS-ID method decreased more slowly with the missing ratio. For example, on the Mnist data set, the performance of SFS-ID approach decreased by around 1.0% at the missing ratio of 0.45, compared to the ratio of 0.65, whereas the performance of SID, KNN+RFS, and MIWAE+RFS decreased by 7.49%, 8.7%, and 4.76%, respectively. The reason for this is that SFS-ID utilized neighbor data reconstruction information to improve the incomplete data structure for the selection of discriminative features, whereas other approaches do not incorporate information from neighbor data.