# Framework for the Ensemble of Feature Selection Methods
## 4 Discussion  
In this study, we proposed the design of a conceptual framework to support ensemble feature selection. Considering a set of concepts and relationships, our proposal explains the general behavior of FS algorithms, their techniques, and how to improve the performance in classification processes. Additionally, the framework provides an overview of existing FS techniques, breaks them down to facilitate understanding, and shows how to combine them to compensate for their biases. Finally, this process allows us to combine outputs of single FS methods and aggregate them by consensus. Previous studies have proposed solutions based on single FS algorithms focused on a particular domain and problem. For instance, removing features with low variability or identifying relationships among features [41]. However, additional studies have used several FS algorithms to generate a subset of features by each method. These subsets have been tested using classifiers, and the subset with the best performance in the classification is the selected subset [26; 42; 43; 44]. In the above approach, although they proved different FS algorithms, the final subset of features is influenced by the biases of the algorithm employed to generate it. This is the main difference concerning this study, where the  
Figure 6: WDBC. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Features** & **Stability** \\ \hline Sonar & F9, F10, F11, F12, F21, F35, F36, F45, F46, F49 & 1 \\ SPECTF & F4, F25, F26, F28, F30, F36, F40, F42, F43, F44 & 1 \\ WDBC & F3, F4, F7, F8, F14, F21, F23, F24, F27, F28 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results of Stability. advantages of different methods of FS are considered in the final subset to compensate for their biases. Besides, recent studies have developed tools to support the combination or assembly of FS algorithms. For example, in [43], the authors proposed a tool developed in R. The tool was designed to combine several outputs of FS algorithms into a random forest algorithm and was used in [39] to select relevant features in the datasets: Sonar, SPECTF, and WDBC. The results in Table 2 showed that our solution achieved an equal or better performance in dimension reduction. However, the comparison was only conducted on dimension reduction. Furthermore, our classification experiments were carried out using random datasets extracted from the original datasets, and the results showed in [39] were not calculated under the same conditions. Consequently, these results cannot be compared. Likewise, in [28], the author describes a tool developed in Python and available on GitHub. The solution was implemented as a class under the object-oriented programming paradigm, and its methods implement some of the best-known FS algorithms. Each method receives a set of parameters to configure the FS algorithm. The methods can be used as a single FS method, or they can use a particular function to get the subsets selected by several FS algorithms. However, this solution does not provide a mechanism to aggregate all subsets generated. Considering the studies reviewed and the solution developed, the framework proposed provides an overall scheme to support EFS. Mainly, it facilitates the analysis of techniques, biases, disadvantages, and advantages of the FS algorithms to determine how we can assemble different techniques to get a subset of relevant features more efficiently. In this sense, the framework is designed for data scientists who handle high dimensionality problems and/or few instances. Regarding the evaluation of our framework, according to the results in Tables 3 and 4, the best classification results were obtained for each dataset using the method developed in this study. For _Sonar_ and _WDBC_ Table 3 the best performance was achieved using the subsets generated by our method with a logistic regression classifier. The evaluation showed that the accuracy was 86.95% and 93.85%, respectively.