# Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU
## 3 Methodology
### Top-Level Classification Block  
This block provides continuous monitoring of the incubator and identifies the current care status and infant activity. The input of the block is the videostream, while the output is (i) a decision on whether to run the Respiration Calculator Block and Pulse Calculator Block, (ii) bounding box (RoI) of the infant if present, and finally (iii) the videostream is passed to both calculator blocks. Statistics can also be logged based on the 14 features the block extracts.  
Two basic analysis types are combined in this block: _Static object detection_ and _Dynamic feature extraction_ (Figure 1). The static object recognition is performed every 3 s, while the dynamic event analysis is evaluated over an overlapping moving window of 6 s, also every 3 s. The choice of the time window length is the result of a grid search hyperparameter optimization.  
At the static object detection task, the following image contents are identified by YOLO-v3 object detector [33]: the body of the infant (head and torso), the presence of parents (gloveless hands), the presence of medical staff (blue gloves), the presence of nursing bottles (feeding the baby). We have selected YOLO-v3 for object detection, because it is one of the fast state-of-the-art networks which has the required accuracy and can  
Figure 1: Information flow between the different algorithmic components. From the _Vidoxstream_ input 14 features are extracted via _Dynamic feature extraction_ and _Static object detection_. The concatenated feature vector is processed by the _Classification network_, which outputs categories and a decision whether to run _Respiration Calculator Block_ and _Pulse Calculator Block_. The former runs based on this decision and receives the videostream. _UNET_ segments images, while _Respiration Extractor_ outputs the wave-like respiration signal and _CalcRate_ calculates the breathing rate. The _Pulse Calculator Block_ runs based on the decision, receives the videostream and the bound box of the infant, if detected. _Pulse net_ uses its attention mechanism, outputs the wave-like pulse signal and _RateEstNet_ calculates the pulse rate. Alarm signals (e.g., apnea) and additional statistics can be extracted from the classification and the numerical results.  
be trained with relatively few samples. The YOLO-v3 returns relative coordinates of the object candidates and confidence values as well; however, the latter are only used in the decision-making process and passed to the LSTM.  
The extracted dynamic properties are low level features of the video, such as camera noise, average brightness, and periodic motions.  
The outputs of the two analysis types are fused with a recurrent network. The combined result leads to the classification of the status, which is used to enable the respiration and pulse-rate networks by excluding the time periods when their results cannot be calculate with high confidence due to extensive motion, caring, or lack of infant. Naturally, the status information can also be logged with the rates and activity pattern of the infant. This provides a meaningful statistics for the supervising doctor of the department.