# EEG Feature Extraction Using Genetic Programming for the Classification of Mental States
## 5 Discussion  
One of the main contributions of this work is the proposed feature extraction method, \({}^{+}\)FEGP. Generally speaking, the classification accuracy in the training phase, as seen in Figure 14, surpasses the reference system. Furthermore, the system's classification accuracy on unseen data also improved upon the reference system, as seen by the testing performance reported in Figure 14.  
If we take a look at the system performance for each fold (Figure 15), we immediately see that some data partitions lead to better classification accuracies than others. Figures 17 and 18 show the best and worst cases in terms of quality in more detail. After dimensionality reduction with PCA, we can recognize that the area of the overlap region for both classes is high in both cases for the raw data. Intuitively, we can deduce that for the majority of state-of-the-art classifiers, the performance will be poor in such circumstances. The effectiveness of CSP can be clearly seen afterwards, with a substantial increase in the separability of the classes. At this point, we can see that the testing cluster is quite different between almost all folds, with fold 4 and 8 being the extreme cases. For fold 4 (Figure 17), the testing data match the distribution of the training data, making it easier for a classifier to obtain good generalization results. On the other hand, in fold 8 (Figure 18), the case is the opposite; the data cluster belongs to a multi-modal distribution, where the obtained model is evolved over a different mode from that where the testing data reside, suggesting that the EEG recordings for that particular subject are quite different from the rest. Let us recall that although the experiments were done in a controlled environment, we could fully constrain the physiological activity of each subject. This is a  
Figure 17: The first two components from Principal Component Analysis (PCA) corresponding to fold 4 in the LOOCV. Plot (**a**) is the class distribution for the raw data; (**b**) after the Common Spatial Pattern (CSP); and (**c**) after the best transformation found by \({}^{+}\)FEGP. Classification accuracy is 82.8% for training and 99.1% for testing. Figure 18: The first two components from PCA corresponding to fold 8 in the LOOCV. Plot (**a**) is the class distribution for the raw data; (**b**) after the CSP; and (**c**) after the best transformation found by \({}^{+}\)FEGP. Classification accuracy is 81.4% for training and 44.1% for testing. more realistic scenario, but it makes the problem more difficult to solve. Furthermore, we can see the benefits of the \"FEGP algorithm in the third scatter plot of each figure. For fold 4, the new features make the problem quite trivial for almost any classifier. The robustness here is that the testing data do not shift or vary; rather, they integrate into the counterpart training samples. The more difficult case is fold 8; given an already problematic situation from the CSP output, the \"FEGP improvement is relatively small. Here, an important observation is that even for this worst case, the \"FEGP performance is at least the same or better, but not worse than the performance of the reference system. We can further analyze the performance for the remaining folds in the LOOCV. The Wilcoxon test infers that the majority of the folds have a similar statistical performance in terms of their medians in the training phase, with the exception of folds 3 and 13. Although these are the extreme cases for training, their performance is an improvement upon the CSP output. This can be extended to all folds; indeed, \"FEGP produced an almost consistent improvement over all folds compared with the reference system. In the testing phase, almost all null hypothesis combinations were rejected, something expected given the nature of CSP output. However, the improvement was not linear among all folds.