# On-the-Fly Sleep Scoring Algorithm with Heart Rate, RR Intervals and Accelerometer as Input
## 5 On-the-Fly Sleep Scoring Algorithm Evaluation Using Public Data Set  
The purpose of this section is to assess the performance of the presented algorithm using publicly available data set. For this purpose the DREAMS Subjects Database, available at [[https://zenodo.org/records/2650142#.XvVUU5](https://zenodo.org/records/2650142#.XvVUU5) NKij8]([https://zenodo.org/records/2650142#.XvVUU5](https://zenodo.org/records/2650142#.XvVUU5) NKij8) (accessed on 2 December 2024), was used. Given the characteristics of the available data set, the on-the-fly algorithm  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **True Deep** & **True Light** & **True REM** & **True Wake** \\ \hline Predicted Deep & **44.73\%** & 17.42\% & 7.54\% & 12.20\% \\ Predicted Light & 35.03\% & 60.41\% & 52.28\% & 70.52\% \\ Predicted REM & 14.15\% & 17.71\% & 33.88\% & 2.67\% \\ Predicted Wake & 6.09\% & 4.47\% & 6.30\% & 14.61\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Confusion matrix for sleep stages estimated by the wearable.  
Figure 5: Medical sleep scoring and wearable sleep scoring compared for the first subject.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Sleep Stage** & **On-the-Fly Algorithm** & \multicolumn{2}{c}{**Motion-Aware**} \\
**Latency** & **On-the-Fly Algorithm** & **Wearable Latency** \\ \hline Light & \(-\)00:06:26 & \(-\)00:07:28 & \(-\)00:30:16 \\ Deep & 00:03:25 & 00:01:39 & \(-\)00:19:14 \\ REM & 00:03:06 & \(-\)00:03:26 & \(-\)00:35:21 \\ Wake & \(-\)00:02:57 & 00:00:53 & 00:16:29 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Sleep stage latency identification (\(Time_{Molitical}-Time_{Algorithm}\)).  
was used as information about motion of subjects as recorded by accelerometers was not available. From the dataset, that contains 20 full night polysomnographies, HR and RR intervals extracted from the Electrocardiograph (ECG) were used to estimate sleep stages with the on-the-fly sleep scoring algorithm, and results were compared with the medical scoring provided in the data set. The results are shown in Table 9.  
The results reported in the confusion matrix are comparable with those obtained with the six subjects analyzed previously. It is important to note that the algorithm uses a set of calibration parameters that are intended to balance the accuracy for the classification of all stages of sleep. In the use case of automated driving, the classification problem can be simplified in a problem to discriminate between non-deep sleep (where no action is needed) and deep sleep where an action is needed to bring the subject out of the deep sleep state. The calibration parameters can be optimized for such a use-case, thus improving the accuracy. A sleep scoring comparison is shown in Figure 6.  
Figure 6: Medical sleep scoring and algorithm sleep scoring compared for a subject on the on-line dataset.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **True Deep** & **True Light** & **True REM** & **True Wake** \\ \hline Predicted Deep & 58.40\% & 26.69\% & 11.93\% & 8.75\% \\ Predicted Light & 8.13\% & 23.43\% & 30.43\% & 20.11\% \\ Predicted REM & 9.97\% & 28.21\% & 43.02\% & 15.17\% \\ Predicted Wake & 23.50\% & 21.68\% & 14.62\% & 55.96\% \\ \hline \hline \end{tabular}
\end{table}
Table 9: Confusion matrix for the dataset online.  
Finally, the confusion matrix was performed also for the sleep-wake pattern, finding the following results:  
\[Sensitivity=86.08\%,Precision=84.27\%,Accuracy=75.62\%\]