## 6 Experiment Evaluation
### Experimental Setup  
A dataset published by the Sleep Heart Health Study (SHHS) is used in this study. As a prospective multicenter cohort study, SHHS aims to study the subtle relationship between cardiovascular diseases and sleep-disordered breathing in the US. The participants were recruited from 9 epidemiological studies [82] enrolling more than 6600 adults aged 40 and over, who were asked to wear a home PSG device to record the occurrences of obstructive sleep apneas and detected risk factors for typical cardiovascular events (such as stroke and myocardial infarction) [82; 83].  
In the SHHS study, polysomnograms were collected in the participants' homes by trained technicians in unattended sittings [50]. The obtained data contain C4/A1 and C3/A2 EEGs sampled at 125 Hz, right and left electrooculograms (EOGs) sampled at 50 Hz, a bipolar sub-mental electromyogram (EMG) sampled at 125 Hz, ambient light, body position, etc. In this study, two EEG channels (C3 and C4 channels) as well as the sleep stages (labeled every 30-s epoch) are used to construct the proposed ischemic stroke prediction model.  
In this study, our analytical samples only included SHHS participants who completed PSG both in SHHS-1 and SHHS-2. The subjects are further categorized into four types according to the appearance of ischemic stroke. Specifically, the time when the SHHS-1 study was conducted is referred to as a separating timestamp. Type I subjects have stroke before the timestamp but have no stroke afterwards. Type II subjects have no stroke before the timestamp but have stroke afterwards. Type III subjects have stroke both before and after the timestamp. Type IV subjects have no stroke.  
Specifically, to build the basic prediction model, we chose 179 subjects, which included 20 Type II instances (the time between their first ischemic stroke and the baseline ranging from 4 days to 377 days, with a mean value of 196 days) and 159 Type IV instances. In particular, the 20 positive samples were chosen based on the onset time of ischemic stroke, i.e., within one year, as the symptoms of a stroke are difficult to detect if the time exceeds one year. Moreover, considering that the classification performance would be less than ideal [84; 85] if the ratio of positive to negative samples is too unbalanced, we chose 159 normal controls without medical history of cerebral vascular disease. Meanwhile, to build the pre-selection model, we chose another 57 subjects, which consisted of 27 Type III instances (formed the PartS in Algorithm 1) and 30 Type IV instances (formed the PartH in Algorithm 1).  
We extracted the features described in [7] and used them as the feature set for the baseline method and compared it with the prediction model constructed using both these clinical features and the features that we extracted in Section 4. In the experiment, based on the empirical results, we set u = 42 min, v = 30 min, w = 4 min.  
Statistical-based feature selection is the process of reducing the number of input variables when developing a predictive model. It works by calculating a test statistic to evaluate the relationship between each input variable and the target variable in order to improve the performance of the model. It calculates a probability value (\(p\)-value) to determine the significance of a feature based on the predefined significance level, which is typically set to 0.05. Specifically, in our experiment we used the Student \(t\)-test for calculating significant difference to select features.  
The evaluation metrics we used include a Precision Recall Curve (PRC) [86], precision, and recall. A good prediction model should have both high Precision and Recall.