# Combining Fuzzy C-Means Clustering with Fuzzy Rough Feature Selection
## 5 Experiments  
In this part, we used nine classifiers to classify nine datasets from UCI machine learning repository [45], to verify and evaluate our feature selection algorithm. The algorithm uses fuzzy clustering to generate fuzzy sets, which are defined by Equation (17). After feature selection, the datasets are reduced. These reduced datasets are classified using related classifiers (unreduced datasets do not use feature selection steps). The T-FRFS is the short of threshold fuzzy rough feature select which was represented in [46] and FRFS is mentioned in [40]. C-FRFS is our proposed method mentioned in this paper. All the three methods were written in MATLAB 2017b which runs on a computer with following characteristics:  
OS: Microsoft Windows 10  
CPU: Intel Core i5-8400 CPU 2.80 GHz  
RAM: 8 GB. We use these two proposed successors to compare with our new method and this indicted that we have made some progress on fuzzy rough feature selection. Nine datasets as depicted in Table 3, were employed to evaluate each methods' performance. The results of all methods, as well as unreduced datasets in terms of the number of selected features, are also demonstrated in Table 3. As shown, our new method C-FRFS always gets the smallest reduced subset and then the T-FRFS. The FRFS does not perform well and sometimes cannot select features. Figure 2 shows the running time of three methods. In Figure 2a, our new method is always slower than the other two methods. However, in the Hillvalley dataset, our new method performs better, which means it is more suitable for high-dimensional data than the other two methods. The reason is that our method obtains a smaller reduct, so the program ends earlier and running time is less. To test the performance of the new method, we applied three methods on different sizes of datasets. With the number of objects from 200 to 5000, the time consumption increased exponentially. However, as shown in Figure 3, the three methods have the same tendency and are almost coincident, \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**No.**} & \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Objects**} & \multirow{2}{*}{**Features**} & \multirow{2}{*}{**Classes**} & \multicolumn{4}{c}{**Reduct Size**} \\ \cline{5-8}  & & & & & & **C-FRFS** & **T-FRFS** & **FRFS** \\ \hline
1 & Class & 214 & 9 & 7 & 4 & 8 & 9 \\
2 & Iris & 150 & 4 & 3 & 3 & 3 & 4 \\
3 & Cleveland & 303 & 13 & 5 & 4 & 6 & 8 \\
4 & Wine & 178 & 13 & 3 & 4 & 4 & 5 \\
5 & User & 431 & 5 & 4 & 3 & 4 & 5 \\
6 & Vertebral & 310 & 6 & 4 & 4 & 5 & 6 \\
7 & Sonar & 208 & 60 & 2 & 2 & 4 & 5 \\
8 & Yeast & 1484 & 8 & 10 & 6 & 7 & 8 \\
9 & Hillvalley & 606 & 100 & 2 & 24 & 96 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Information of nine datasets and reduct size of three feature selection methods. C-FRFS: c-means fuzzy rough feature select. T-FRFS: threshold fuzzy rough feature select. Nine classifiers of different categories such as Bayes Net, Naive Bayes, RBFNetwork, Jrip, PART, BFTree, FT, J48, and NBTree in Weka were selected to classify resulting subsets of features by each FS method [47]. To obtain more accurate results, we used 10-fold cross validation in the modeling of each classifier method. The results are presented in Table 4 where the average classification accuracies of nine classifiers for each dataset worked with three selection methods along with unreduced datasets shown in each column, and the last column indicated the mean of nine classification accuracies. The highest mean of classification accuracies was gained in expense of employing all features, which means that feature selection methods are not always successful in increasing classification accuracy, but in decreasing model complexity by sacrificing inconsiderable accuracy. As Table 4 shows, for four datasets, the original model gained the highest classification accuracy, three for both FRFS and our new method.