Fast Feature subset selection algorithm based on clustering for high dimensional data

Feature selection algorithm can be used for removing irrelevant, redundant information from the data. Feature selection is divided into different categories, amongst them filter method is used because of its generality and is typically good choice when numbers of features are large. In cluster analysis, features are divided by using graph-theoretic clustering method. A Fast clustering bAsed feature Selection algoriThm (FAST) is based on minimum spanning tree which is constructed from weight complete graph. Then features are divided into forest where each tree represents a cluster and most representative feature is selected from each cluster. The clustering-based FAST algorithm produces a subset which contains independent features. Linear correlation based measure is used to identify highly correlated features amongst the final got subset of features from FAST algorithm. Keywords—Graph-theoretic clustering; Minimum spanning tree; Feature selection; feature subset selection algorithm (FAST); High dimensional data; Filter method; Wrapper method; Embedded method. INTRODUCTION In data mining, data is analysed and summarized into useful information. The high-throughput technologies grow increasingly which result in exponential growth in the data with respect to dimensionality, storing and processing such a data becomes more challenging task. Feature selection is used as a pre-processing stage in machine learning. It is a procedure of choosing a subset of original features by removing irrelevant and redundant features so that the feature space is get reduced according to some criterion. For choosing a subset of good features with respect to the target concepts, feature subset selection is a useful way for reduction in dimensionality, removal of irrelevant data, increase in learning accuracy, and improvement in result comprehensibility. Feature selection is generally categorized into four models, namely: filter model, wrapper model, embedded model and hybrid model. Filter model methods do not make use of any clustering algorithm to test the quality of the features. They evaluate the value of each feature according to certain criteria. Then, it selects the features with the highest value. It is called the filter because it filters out the irrelevant features using given criteria. The wrapper model uses a clustering algorithm to evaluate the goodness of selected features by (1) finding a subset of features and then, (2) it evaluates the clustering quality using the selected subset. Finally, it repeats (1) and (2) until the desired quality of features is found. It is impossible to evaluate all possible subsets of features in high-dimensional datasets. Therefore, experience based search strategy is adopted to reduce the search space. The wrapper model is computationally expensive compared to filter model. However, it produces better clustering because it aims to select features that maximize the quality. The embedded methods integrate feature selection as a part of the training process and are usually specific to given learning algorithms, and thus may be more efficient than the other three categories. Drawbacks in filter and wrapper models are overcome in a hybrid model, which take benefit from the efficient filtering criteria and better clustering quality from the wrapper model. A hybrid process having following steps: it uses filtering criteria to select candidate subsets. Then, the quality of clustering of each candidate subsets is evaluated and the subset with highest clustering quality will be selected. Hybrid model usually produce better clustering quality than those of filter model and wrapper model. Feature selection selects subset of highly differentiated features. In other words, it selects features that are capable of selecting samples that belong to different classes. Thus, if we have labelled samples as training samples in order to select these features, then this kind of learning is called supervised learning, which means that the dataset is labelled. In supervised learning, it is easy to differentiate the features in different classes. If sample data is unlabeled then selecting feature poses a challenge in feature selection task. In such cases, defining relevancy becomes unclear. However, we still consider that selecting subset(s) of features may help improving unsupervised learning in a same way to improving the supervised learning. Feature selection algorithm used to select the subset of feature by removing irrelevant and redundant features. FAST algorithm, it involves 1) the construction of the minimum spanning tree from a weighted complete graph; 2) the partitioning of the minimum spanning tree into a forest such that each tree representing a cluster; and 3) and then the selection of representative features from the clusters[3]. In general, a good features are those which relevant to target class but not redundant to any of the other relevant features. Correlation is considered as goodness measure between two variables, then the above definition becomes that a good features are highly correlated International Journal of Engineering Research and General Science Volume 3, Issue 4, July-August, 2015 ISSN 2091-2730 794 www.ijergs. org to the target class but not highly correlated to any of the other features. In other words, if the correlation between a feature and the class is high enough to make it relevant to (or predictive of) the class and the correlation between it and any other relevant features does not reach a level so that it can be predicted by any of the other relevant features, it will be regarded as a good feature for the classification task [2]. RELATED WORK Feature subset selection can be the process of identifying and removing irrelevant that do not contribute to predictive accuracy and redundant features that they provide mostly information which is already present in other features as much as possible. Many feature selection algorithms are present; some of them can be able to remove irrelevant features but not effective to handle redundant features. Yet some of the other can eliminate irrelevant feature while taking care of redundant features [1]. FAST algorithm falls in to second group. One of the feature selection algorithms is Relief [6], which weighs each feature according to its ability to discriminate instances under different targets based on distance-based criteria function. However, Relief is useless at removing redundant features as two predictive but highly correlated features are likely both to be highly weighted [7]. Relief-F [8] extends Relief, enabling this method to work with noisy and incomplete data sets and to deal with multiclass problems, but still cannot identify redundant features. Redundant features also affect the accuracy and speed of learning algorithm; hence it is necessary to remove it. CFS [9], FCBF [10], and CMIM [12] are examples that take into consideration the redundant features. CFS [9] is achieved by the hypothesis that a good feature subset is one that contains features highly correlated with the target, yet uncorrelated with each other. FCBF ([10], 11]) is a fast filter method which can identify relevant features as well as redundancy among relevant features without pair wise correlation analysis. CMIM [12] iteratively picks features which maximize their mutual information with the class to predict, conditionally to the response of any feature already picked. Different from above algorithms, FAST algorithm uses minimum spanning tree-based method to cluster features. FEATURE SUBSET SELECTION ALGORITHM Feature subset selection framework consists of two important component irrelevant feature removal and redundant feature removal. Accuracy of learning machines severely affected by irrelevant features, along with redundant features. Thus, irrelevant and redundant features should be identify and remove as much as possible. Moreover, “good feature subsets contain features highly correlated with (predictive of) the class, yet uncorrelated with (not predictive of) each other. ” [5] A new algorithm is developed to efficiently and effectively deal with both irrelevant and redundant features, to obtain a good feature subset. A new feature selection framework (shown in Fig.1).