# A Two-Phase Approach for Semi-Supervised Feature Selection
###### Abstract  
This paper proposes a novel approach for selecting a subset of features in semi-supervised datasets where only some of the patterns are labeled. The whole process is completed in two phases. In the first phase, i.e., Phase-I, the whole dataset is divided into two parts: The first part, which contains labeled patterns, and the second part, which contains unlabeled patterns. In the first part, a small number of features are identified using well-known maximum relevance (from first part) and minimum redundancy (whole dataset) based feature selection approaches using the correlation coefficient. The subset of features from the identified set of features, which produces a high classification accuracy using any supervised classifier from labeled patterns, is selected for later processing. In the second phase, i.e., Phase-II, the patterns belonging to the first and second part are clustered separately into the available number of classes of the dataset. In the clusters of the first part, take the majority of patterns belonging to a cluster as the class for that cluster, which is given already. Form the pairs of cluster centroids made in the first and second part. The centroid of the second part nearest to a centroid of the first part will be paired. As the class of the first centroid is known, the same class can be assigned to the centroid of the cluster of the second part, which is unknown. The actual class of the patterns if known for the second part of the dataset can be used to test the classification accuracy of patterns in the second part. The proposed two-phase approach performs well in terms of classification accuracy and number of features selected on the given benchmarked datasets. feature selection; semi-supervised datasets; classification; clustering; correlation +
Footnote †: journal:  
0000-0001-8998-2000/00/0000-00001-8998-2000/00000-00001-8998-2000/000001-8998  
a novel approach was presented using a real-coded genetic algorithm (GA) for a polynomial neural network classifier (PNN) [12]. The polynomials have powerful approximation properties [13] and excellent properties as a classifier [12]. One of the major problems in the mining of large databases is the dimension of the data. More often than not, it is observed that some features do not affect the performance of a classifier. There could be features that are derogatory in nature and degrade the performance of classifiers. Thus, one can have redundant features, bad features, and highly correlated features. Removing such features can not only improve the performance of the system but also make the learning task much simpler. More specifically, the performance of a classifier depends on several factors: (i) number of training instances; (ii) dimensionality, i.e., number of features; and (iii) complexity of the classifier. Dimensionality reduction can be done mainly in two ways: selecting a small but important subset of features and generating (extracting) lower-dimensional data, preserving the distinguishing characteristics of the original higher-dimensional data [14]. Dimensionality reduction not only helps in the design of a classifier, but also helps in other exploratory data analysis, assessment of clustering tendency, as well as to decide on the number of clusters by looking at the scatterplot of the lower-dimensional data. Feature extraction and data projection can be viewed as an implicit or explicit mapping from a \(p\)-dimensional input space to a \(q\) (\(p\geq q\))-dimensional output space such that some criterion is optimized. A large number of approaches for feature extraction and data projection are available in the pattern recognition literature [15; 16; 17; 18; 19; 20]. These approaches differ from each other in terms of the nature of the mapping function, how it is learned, and what optimization criterion is used. Feature selection leads to savings in measurement cost, because some of the features get discarded.