# An Ensemble Feature Selection Approach to Identify Relevant Features from EEG Signals
## 3 Results
### Evaluating Ensemble Feature Selection (EFS)  
To evaluate the utility of subset features selected by the EFS algorithm, a set of four classifiers (decision tree, logistic regression, random forest, and SVM) were configured to determine which one of them achieves the best performance in classifying normal or abnormal brain activity. The evaluation considered 70% of data for training the models and 30% for testing them. This process was repeated 10 times, and the data were split randomly. Tables 1-4 describe the results of the accuracy and standard deviation of accuracy in classification calculated using all features and the subset of selected features by the EFS technique using different sizes (\(K\)) of the subsets generated by the single methods. \"Features selected\" represents the number of features chosen after aggregating the subsets generated by each FS algorithm in the EFS method. Thus, \(K\) represents the number of features determined by each FS algorithm. Besides, when we trained the models with all features, we obtained different values in each row because we repeated each EFS test. However, although the values are different, they are close. These results are because the data were randomly split in each test, and each test used different samples of the data.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \(K\) & **Features Selected** & \multicolumn{2}{c}{**Decision Tree Classifier**} \\ \hline  & & **All** & **EFS** \\ \hline
1 & 3 & \(95.97\pm 1.4\) & \(92.03\pm 1.8\) \\
3 & 10 & \(95.91\pm 1.5\) & \(93.82\pm 1.5\) \\
5 & 17 & \(95.71\pm 1.6\) & \(94.48\pm 2.1\) \\
7 & 23 & \(95.78\pm 1.4\) & \(95.08\pm 1.9\) \\
9 & 27 & \(95.90\pm 1.01\) & \(96.02\pm 1.03\) \\
15 & 35 & \(95.97\pm 1.21\) & \(96.12\pm 1.01\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Accuracy results—decision tree classifier.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \(K\) & **Features Selected** & \multicolumn{2}{c}{**Logistic Regression**} \\ \hline  & & **All** & **EFS** \\ \hline
1 & 3 & \(97.17\pm 0.98\) & \(90.31\pm 1.8\) \\
3 & 10 & \(97.37\pm 1.1\) & \(91.89\pm 3.1\) \\
5 & 17 & \(97.31\pm 1.6\) & \(92.27\pm 3.0\) \\
7 & 23 & \(97.39\pm 1.2\) & \(95.16\pm 2.1\) \\
9 & 27 & \(97.02\pm 1.4\) & \(95.08\pm 1.6\) \\
15 & 35 & \(97.24\pm 1.32\) & \(95.28\pm 1.54\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Accuracy results—logistic regression.  
Tables 1-4 prove that subsets of selected features could reach a similar performance in classification compared to the performance achieved using all features. Even the accuracy of SVM improved when the classification process used only subsets of features selected by the EFS technique. Likewise, the previous tables show that support vector machine was the algorithm with the best performance in classifying abnormal and normal segments of brain activity.  
Table 5 shows the features selected by each single FS algorithm that allowed training the model with the best performance in this preliminary test.  
The select \(K\) best FS methods were calculated using the ANOVA, chi-squared, and mutual information metrics.