## 2 Materials and Methods
### Performance Evaluation
#### 2.7.3 Feature Dimension Reduction  
Directly including all the features in the LOPO-CV procedure might result in overfitting. Take RP-I of the delta band as an example. A total of 406 delta-band PR-I features were extracted from each participant. If we were to include all the features in the classification, the feature dimension would be 406 (\(N=406\)). This is obviously higher than the size of the training set (\(|S|=54\)) in each fold of the LOPO-CV, leading to the so-called small sample size (SSS) problem. The SSS problem may have two critical consequences. First, it may lead to overfitting [35]. Second, the covariance matrix of LDA may become singular such that its inverse does not exist [22]. Although it is possible to calculate the pseudoinverse of the covariance in such cases, classification performance would nevertheless degenerate. To avoid the SSS problem, feature selection must be used to reduce the feature dimension to 54 prior to LOPO-CV. Feature selection methods can be classified into three categories: embedded, wrapper, and filter [36]. Embedded and wrapper methods rely on the performance of a specific classifier to quantify the classification ability of each feature. Examples of the former and the latter are sequential forward selection (SFS) [37] and recursive feature elimination (RFE) methods [38], respectively. By contrast, filter methods are independent of the classifier. One popular filter method is [PERSON]'s class separability criterion [39]. This involves calculating the Fisher ratio (F-score) for each feature, and then ranking the features by their F-scores. A higher F-score corresponds to a higher between-class to within-class ratio for the feature. By contrast, a lower F-score indicates that the feature is noisier. The advantage of the filter method is its low computational complexity. Therefore, for each state and each type of RPs, we applied [PERSON]'s method to select the top 54 features from the 406 single-band (delta/theta/alpha/beta/gamma) and the 2030 \"all-band\" RP feature candidates, respectively. Note that the objective was not to select a set of \"optimum features\", but simply to select 54 features to avoid the small sample size problem. Figure 6 shows the F-scores of the 2030 (all bands) RP-I, RP-II, and RP-III features extracted during the emotion-induction state. The next question is how to find the most useful features among a given set of F-score ranked features. To address this issue, [PERSON] et al. [20] conducted two experiments based on a leave-\(N\)-feature-out scheme to select the best EEG features from 60 candidates. Their first experiment iteratively removed \(N\) F-score-ranked features one at a time and examined each one's effects on classification performance. In the second experiment, they iteratively removed \(N\) F-score-ranked features one at a time, but the removed \(N\) features were randomly selected. Their results based on different \(N\) values (e.g., 1, 5, 10, 15, and 20) showed that the top F-score-ranked features were more discriminative than the lower ranked ones. As a result, they suggested that one may directly choose the top-\(N\) features as the optimum feature subset for emotional EEG classification, and the optimal value of \(N\) can be determined by a data driven method (i.e., cross-validation). Recently, the same strategy for selecting the top-\(N\)-F-score-ranked features was adopted for classifying motor imagery EEG [40]. Accordingly, in this study, we performed LOPO-CV to calculate the classification accuracy using the top-\(N\)-F-score-ranked features for each state (resting and emotion induction) and each RP type, and this procedure was conducted once for every \(N\) (\(N=1,2,\dots,54\)). Here, we used the LDA as the classifier, because a sophisticated classifier such as a nonlinear SVM classifier may compensate for the weakness of a feature, and thus, a simple classifier is preferred in feature evaluation [41].