# Framework for the Ensemble of Feature Selection Methods
## 1 Introduction
### Context of Ensemble Feature Selection  
Depending on the design of FS techniques, they are classified into three types of methods: filters, wrappers, and embedded. Each type defines advantages or disadvantages that are directly related to the context of the dataset. In general, these three types of FS techniques face typical problems, namely, (i) they have a good performance on a dataset, but by adding or removing instances, the performance decreases, (ii) they allow the removal of features quickly, but they are not capable of detecting redundant features, (iii) they need to have a correctly balanced dataset, and (iv) their performance is affected by the presence of noise in the data.  
Moreover, there is a large number of FS methods. However, there are no tools or solutions to determine objectively the algorithms, which would work best with the data of a particular domain. Therefore, in some studies, authors have used a trial-and-error scheme. They have tested different FS algorithms using one or more classifiers and then chosen the one with the best performance in the test.  
Alternatively, ensemble learning approaches have been proposed to select features based on the consensus or aggregation of several FS algorithms. For example, in [22], researchers proposed a classification algorithm based on K-Nearest Neighbors (KNN). They obtained and combined several outcomes from the KKN algorithm, and each result was obtained by using a different set of features. In 1998, the authors proposed an ensemble feature selection (EFS) method designed for decision trees [23]. In 1999 [24], an EFS method based on a genetic algorithm was proposed to improve the quality of the features used in the learners.  
Recently, many studies have been conducted about EFS; some involve using classifiers while others do not. In [7], an EFS algorithm aggregates a set of FS algorithms based on filters to classify microarrays. The scheme in this study aims to use several filters and generate a subset of features for each filter. The subsets generated are used to train a classifier, and, subsequently, the outputs of classifiers are combined using simple voting. In [25], a mechanism of EFS on microarrays was built to determine relevant genes in the classification of cancer. A robust feature selection process is conducted in [26], the selection of features is based on EFS, and the findings showed that the approach obtained great promise for datasets with many features and few samples. The bi-objective genetic algorithm was used in [27] to develop a method of EFS. The evaluation showed that the proposal reached to obtain robustly and noise resilient subsets of features. An approach based on selecting in Random Forest and co-forest was implemented in [28]. The method allows selecting features in datasets with unlabeled data.  
The main objective of this paper is to propose a framework to determine how to improve the FS on datasets with high dimensionality and a few instances. Furthermore, the framework seeks to guide the design of an EFS mechanism to gather the advantages of different FS algorithms, avoid their biases, and compensate for their disadvantages. For this, we designed a conceptual framework to understand the main concepts and relationships in the aggregation of a set of FS algorithms. Following this, an implementation framework was built to validate the theoretical proposal.  
The rest of the document is organized as follows: Section 2 describes the qualitative method used to develop the proposed conceptual framework. Section 3 presents the implementation and evaluation of the framework. Section 4 offers a discussion of results and contributions. Finally, Section 5 describes the main conclusions of this research.