# Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU
## 5 Results
### Pulse Calculator Block  
For the training and testing of the network, the 2 different large datasets were used which are summarized in Table 2. Only a short video segment was available with pulse signal waveform reference, which was used for pre-training the PhysNet [19] network with NegPeaLoss (using heavy augmentations). Then, we further trained it on a 5-h-long dataset (#2) with pulse-rate reference using SNRLoss, defined as follows:  
\[\mathcal{L}=-\log_{10}\left(\frac{\sum\limits_{f=80}^{250}S(f)\cdot u(f)}{ \sum\limits_{f=80}^{250}S(f)\cdot(1-u(f))}\right) \tag{8}\]  
Figure 8: Static analysis: object detection examples, which are used in the top-level classifier to separate and identify excessive motion, caring, empty incubator, etc. categories during operation. The ROI of neonates is passed to the pulse network. Figure 7: Confusion matrix of the main categories in a representative 12 h set. where \(S(f)\) is the power spectrum of the output signal, \(u\) is the reference binary template function containing 1 or 0 for each \(f\) frequency. The summing is performed over the pulse range, from 80 BPM to 250 BPM. The precise definition for \(u\) is the following:  
\[u(f)= \begin{cases}1,&\text{if }f\in[f^{*}-\delta,\quad f^{*}+\delta]\\ 0,&\text{otherwise}\end{cases}  \tag{9}\]  
where \(f^{*}\) is the reference pulse rate and \(\delta\) is the acceptance value chosen to be 6 BPM. Please note that the power spectrum calculation consists of differentiable steps, therefore it works well with the back-propagation algorithm, and so can be used for training the neural network. The rate estimator network (RateEstNet) was trained together with the pre-trained PhysNet on training set #2 with Mean Absolute Error (MEA, also known as L1) loss function. Because we had a limited amount of data compared to the number of parameters of the networks, we used regularization and generalization techniques such as dropout and augmentations. We introduced a novel augmentation method for periodic time series signals, namely the frequency augmentation. This is done by resampling in time (using linear interpolation) a shorter or longer input video back to 128 samples length and changing the reference accordingly. With frequency augmentation we can imitate pulse rates in the whole range from 80 BPM to 250 BPM. Consequently, we can create a training set with uniform pulse-rate distribution. Thus, the network will not be biased towards the mean pulse-rate value. General image augmentations were also applied such as horizontal and vertical flip or the manipulation of image brightness and saturation. These methods were used to avoid overfitting and achieve good generalization. Evaluation was done on a 15-h-long (#1) test dataset. It is considered to be challenging because it contains unfiltered continuous monitoring with heavy motion and low illumination level parts. There is even a 1 h segment where the head of the baby is covered with sheet (Figure 9). The results of different methods can be seen in Table 3, as evaluated on the test set #1 (Table 2). Different metrics were computed to quantify the performance of the networks: (1) MEA-Mean Absolute Error between reference and pulse-rate estimate; (2) RMSE-Root Mean Squared Error between reference and pulse-rate estimate; (3) R-Pearson correlation coefficient of the reference and estimated pulse rate in time. In the case of PhysNet-SNR (first row in Table 3) the network output is a signal, and the pulse rate is estimated from it via frequency analysis. First the pulse signal was filtered using a 6 th order band-pass Butterworth filter with 80 BPM and 250 BPM cutoff frequencies. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Number (\#)** & **Length** & **Available References** \\ \hline
1 & 15 h & pulse rate, SpO\({}_{2}\) \\
2 & 4 h & pulse rate, SpO\({}_{2}\) \\
3 & 1 h & pulse rate, SpO\({}_{2}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The training (#2), validation (#3), and test (#1) datasets used for the pulse network.