## 4 Discussion  
In this study, we used two data sets containing multichannel EEG recordings to evaluate the adequacy of randomized (RCV) and leave-one-patient-out (LOO) cross-validation strategies to measure machine learning algorithms' performance in a seizure prediction task. To this end, 53 features were first extracted from preprocessed EEG data, and various standard machine learning classifiers were trained to predict whether a signal window belonged to a preictal _vs._ interictal state. The best-performing model (XGBoost) was optimized using a Bayesian hyperparameter tuning procedure based on the Optuna framework and was finally deployed to compare two cross-validation schemes using standard performance metrics: accuracy, specificity, and sensitivity.  
Figure 5: Comparison of the average performance metrics for classifying interictal and preictal phases on the CHB-MIT data set and our data set using RCV or LOO validation methods. Error bars represent the standard error of the mean.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Data Sets** & **Metrics** & **RCV** & **LOO** & \(\mathbf{p}\)**-Value** \\ \hline \multirow{3}{*}{CHB-MIT} & ACC & 78.75 \(\pm\) 7.34 & 49.55 \(\pm\) 4.72 & \textless{}0.001 \\  & SEN & 64.48 \(\pm\) 20.88 & 55.56 \(\pm\) 11.87 & 0.31 \\  & SPE & 78.2 \(\pm\) 34.21 & 43.54 \(\pm\) 6.40 & \textless{}0.05 \\ \hline \multirow{3}{*}{Our data set} & ACC & 81.68 \(\pm\) 3.45 & 50.93 \(\pm\) 12.80 & \textless{}0.001 \\  & SEN & 64.66 \(\pm\) 10.34 & 48.62 \(\pm\) 15.96 & \textless{}0.05 \\ \cline{1-1}  & SPE & 96.12 \(\pm\) 9.99 & 54.58 \(\pm\) 21.81 & \textless{}0.001 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results of the t-test comparing RCV and LOO validation methods on the CHB-MIT data set and our data set. All performance metrics are illustrated by mean (%) \(\pm\) standard deviation.  
The results obtained using the RCV validation scheme suggest that machine learning algorithms can achieve remarkable performance in seizure prediction, with an average accuracy of 79% for the CHB-MIT data set and 82% for our data set. The accuracy was even more impressive for individual patients, reaching 91% in CHB-MIT and 86% in our data set. These findings align with previous results reported in the literature, which used similar machine learning models for seizure prediction. For example, a study reported an average sensitivity of 75.8% in discriminating interictal and preictal states using SVMs [19]. Another reported an average sensitivity of 80% [37], and yet another study obtained an average accuracy of 81.17% using time-frequency feature extraction combined with classification techniques [24].  
However, our results clearly highlight that the RCV validation method could lead to overly optimistic conclusions. Indeed, when using the more robust LOO validation procedure, all performance metrics dramatically dropped, often by more than 20%. This suggests that a random splitting of EEG windowed signals might consistently increase the risk of overfitting the training data, making it easier for the model to learn spurious statistical features that are not representative of the clinical condition. For example, the classifier could learn to predict an upcoming seizure based on a systematic but uninformative alteration of the EEG recording, such as a temporary increase in skin conductivity caused by a patient's sweating. When tested on a completely different patient, such a classifier would misinterpret sweating as an alerting signal.