# Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU
## 4 Dataset  
Due to the lack of a public annotated database of video-based incubator monitoring, for the training of our neural networks and for validation and evaluation purposes, we built our own database. Data were collected in the NICU of the I\({}^{f}\) Dept. of Pediatrics and, II. Dept. of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary. The population demographics of the participants can be seen in Table 1.  
Videos of infants were recorded using a single Basler acA2040-55 uc RGB camera with a resolution of 500 \(\times\) 500 pixels at 20 frames per second. Reference data for respiration and pulse, including the waveforms and the calculated rates, were provided by the vital sign monitoring systems of the hospital, from the Philips IntelliVue MP20/MP50 models with standard interface card and custom software interface. No dedicated light source was used. The storage of the videos and reference data is done in raw format both to avoid compression artifacts and to make the offline test environment identical to the online one. The video recording was done from several different camera-angles (at least 4 distinct) and with different optics (aperture and zoom) to increase generality and avoid overfitting. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Subject** & **1** & **2** & **3** & **4** & **5** & **6** & **7** \\ \hline Recording time (hours) & 96.7 & 5.5 & 39.4 & 27.4 & 51 & 105.5 & 50.1 \\ Gender & F & M & M & F & F & M & F \\ Gestational age (weeks) & 32 & 32 + 3 & 31 + 4 & 35 + 4 & 39 & 32 & 33 \\ Birth weight (g) & 2020 & 1840 & 1850 & 1870 & 3150 & 2120 & 2080 \\ Postnatal age (days) & 4 & 4 & 10 & 8 & 4 & 7 & 2 \\ Actual weight (g) & 1900 & 1850 & 1680 & 1820 & 2905 & 2040 & 1960 \\ Length (cm) & 46 & 44 & - & 45 & 57 & 45 & 44 \\ Head circumference (cm) & 32 & 29.5 & - & 32 & 34 & 30 & 32 \\ Respiratory support & no & no & no & no & no & no & no \\ Pharmacological & no & no & no & no & no & no \\ cardiovascular support & & no & no & no & yes & no & no \\ Any drugs \({}^{*}\) & no & no & no & no & yes & no & no \\ Fitzpatrick scale & 2 & 3 & 2 & 2 & 2 & 2 & 2 \\ \hline \hline \end{tabular}  
* Drugs may alter muscle tone/physical activity (e.g., sedatives, anticonvulsants etc.).  
\end{table} 
Table 1: Population Demographics. Figure 6: Operation method of the classical rate calculation (CalcRate). The BR is calculated by 1/(Mdi/20)-60 where Mdi is the mean of d distances between the rising edges of actual peaks. This enables transferring an algorithm trained with our database to another incubator or to apply it in homecare. A software tool was developed for the annotation and structuring of the recordings. This annotation tool loads the acquired data into our database in an automated manner, based on the time of recording, video length and settings. After that, the footages is annotated manually. These tags describe the scenario, the availability of the reference data, the image quality and the existence of the proper illumination. There are tags for intensive baby movements, care, blurred image, phototherapy device is on, saturated image, empty incubator, low luminance level, dislocated camera, missing reference etc. The database contains 96 h of annotated data. For the pulse estimation neural network, we handpicked a 4-h-long training set, a 1-h-long validation set and a 15-h-long test set, which all contain a balanced portfolio of the different scenarios. Participants split among the sets in a disjunct manner with at least 2 participants in each to maintain generalization and independence. Our algorithms were developed for neonatal vital sign monitoring. Therefore, they had to be prepared for recognizing specific life-threatening situations such as apnea or extreme vital sign figures. On the one hand, fortunately, these kinds of situations occur rarely. Therefore, the lack of such events in our database motivated us to synthetize video events with low/high respiration and pulse rates and even missing vital signs.