# Automatic Detection of Electrodermal Activity Events during Sleep
## 4 Discussion  
The \(p\)-values in Table 4 show that the differences in Precision and \(F_{1}\) performance of the automatic scoring algorithm, when applied to either one of the two datasets, are not statistically significant. This suggests that our algorithm is robust to differing sampling frequencies and sleep technologists, who score the EDA signals. When applying the algorithm to the EDA recordings from the two different studies and evaluating its performance, we obtain \(F_{1}\) scores with an average of 68.9% and of 71.4% for EDA events and of 51.5% and 56.6% for EDA storms. The low agreement for EDA storm detection and, to a lesser extent, the moderate agreement for EDA event detection are partially misleading. The low average of \(F_{1_{storm}}\) scores stems from the definition of \(F_{1}\) scores and, thus, is caused by having some low \(P_{storm}\) values while the \(R_{storm}\) average is similar to the \(R_{event}\) average. The consistency between the two recall performances further confirms that the relatively low average value for \(F_{1_{storm}}\) scores is exclusively caused by the existence of some low \(P_{storm}\) value. They arise because EDA storm automatic and manual scoring is different. One issue is the absence of a definition in the literature for the maximum length of an EDA event. Because of this, a manual scorer might label as one EDA event what the automatic scoring algorithm presented in this paper labels as multiple EDA events. Also, since manual scorers do not score EDA storms but rather consider them a long-lasting EDA event, we consider a long-lasting manually scored EDA event to be an EDA storm. This differs from how the algorithm scores EDA storm, as it rather follows what the literature considers an EDA storm. We graphically represent this in Figure 11. The figure shows another issue in EDA event detection. While the manual and the automatic scores identify the EDA events, the associated performance indices are low because of misalignment in the onset of the detected EDA events. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(\mathbf{P_{event}}\) & \(\mathbf{R_{event}}\) & \(\mathbf{F_{1_{event}}}\) & \(\mathbf{P_{storm}}\) & \(\mathbf{R_{storm}}\) & \(\mathbf{F_{1_{storm}}}\) \\ \hline
0.3937 & 0.0300 & 0.6167 & 0.9458 & 0.0237 & 0.5012 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The \(p\)-values of two-sample \(t\)-tests are shown, where we compare a set of indices from one study to the respective set from the other study. Figure 11: Difference between manual and automatic scoring. In the literature, agreement between different sleep technologists when scoring sleep stages is categorised using the intraclass correlation coefficient, as outlined by [PERSON] et al. [47]: poor agreement for 0-0.2; fair agreement for 0.3-0.4, moderate agreement for 0.5-0.6, strong agreement for 0.7-0.8, and almost perfect agreement for >0.8. Based on these ranges and using the \(F_{1}\) score as the relevant performance measure, our algorithm achieves strong agreement for EDA event detection and moderate agreement for EDA storm detection when applied to the dataset from the Sleep Revolution Project. For the 2005 dataset, the agreement for EDA event detection is moderate; however, the value is almost in the strong agreement range. The agreement for EDA storm detection remains moderate. Notably, we believe that the performance measures presented in Tables 2 and 3 are rather conservative ones. In [17], we show that there is a strong correlation between EDA events and EDA storms detected by our algorithm and sleep stages or the presence of OSA. It might be possible to increase accuracy in such applications might be obtained by combining the EDA-based classifier with ML models trained on different signals. For example, one could consider the electroencephalography (EEG) signal or the electromyography signal [48, 49].