# Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU
## 2 Related Works  
Various algorithms exist to remotely measure vital signs. Earlier methods used classical techniques for pulse and breath rates extraction: blind source separation [6, 7], color space analysis [8] or ballistocardiographic methods [9] for pulse signal extraction, and _Optical Flow_[10, 11] or _Euler Motion Magnification_[12] for respiration signal extraction, and Fourier analysis for rate value estimation [13; 14]). However, these extractors are very sensitive to motion artifacts, due to the lack of image interpretation. Most recent studies employ either mixed [15; 16; 17] or purely neural network solutions [18; 19; 20; 21] for these tasks to increase motion-robustness and overall performance. Deep-learning-based heart-rate measurement methods were reviewed by [22]. It also compares their performance on the UBFC-RPPG dataset. The first end-to-end convolutional neural network (CNN) was proposed by [PERSON] et al. named DeepPhys [21] which uses an attention mechanism: it consists of two connected streams, the attention stream-whose input is a single frame and is responsible for region of interest (RoI) selection-and the motion stream-whose input is the normalized frame difference and is responsible for physiological signal extraction. Just recently, [PERSON] et al. analyzed this network [23] and concluded that it learns the wavelength-dependent characteristics of blood absorption color variation to extract physiological signals-as classical algorithms do-, and that the choice and parameters (e.g., phase, spectral content) of the reference-signal may be more crucial than anticipated-i.e., the network learns and performs better if the label is well selected, prepared and aligned. [PERSON] et al. developed two separate convolutional networks [20], one for extracting pulse signal from video and the other for estimating pulse rate from the previously extracted signal-with the application of a signal-to-noise ratio (SNR)-based loss function. A 3D-network named PhysNet was introduced by [PERSON] et al. [19] which exploits not only spatial but spatio-temporal features. This network is completely end-to-end--there are no pre-processing steps--its input is a sequence of images and the output is the corresponding pulse signal. However, most of the published results were developed for or trained on adults. Neonates have a weaker pulse and the variation of periodicity in their pulse and--especially--in their breathing rate is much higher. Parents and clinical staff often interact with the infants at the NICU. This includes the routine and non-routine medical interventions, manual vital sign checks, changing diapers or blankets, feeding and caressing, which makes the continuous camera-based vital sign monitoring of neonates challenging. Up to recent years, only classical algorithms have been used for video-based infant monitoring-these are summarized here briefly. Camera-based non-contact estimation of pulse rate in the neonatal care unit was first reported in 2012 [24]. In this work, seven infants were monitored for only 30 s with a web camera 20 cm away from the face and with special illumination. In another study [25], video camera recordings were conducted of 19 infants. The camera was placed on a tripod at approximately 1 m from the infant and the face region was recorded for up to 5 min. RoI was manually selected and pulse rate was estimated from Fast Fourier Transform (FFT) analysis of the green channel. In 13 of 19 neonates it was possible to derive PR estimates for 90% of the time. These studies were conducted on brief video recordings with ideal conditions (e.g., controlled illumination, resting patient). However, it is essential to analyze the feasibility of the method in a long-term and uncontrolled scenario, in a real hospital or home environment under normal lighting conditions-and most importantly, without affecting patient care. With this in mind, [PERSON] et al. [26] directed their study, which contains the analysis of around 25 h' long \"valid camera data\".