# Continuous Camera-Based Premature-Infant Monitoring Algorithms for NICU
## 3 Methodology
### Pulse Calculator Block  
The Pulse Calculator Block first extracts a wave-like pulse signal from the videostream, then estimates the pulse rate using two neural networks. The block receives (i) the decision from the Classification Block on whether to run, (ii) the bounding box (RoI) of the infant and (iii) the videostream as inputs. Its output is the pulse rate. The pulse network is responsible for the pulse signal waveform extraction from the video. We have adapted two state-of-the-art neural networks: the DeepPhys [21] and the PhysNet [19]. Although DeepPhys learned more quickly than PhysNet, we chose PhysNet because it could be further trained with pulse-rate reference and our database has more of this kind of data. For PhysNet a stack of images serve as the input and its output is a signal segment (e.g., 128 samples long) which enables us to train it with a signal correlation-based loss function (negative Pearson correlation loss (NegPea) [19]) or even with pulse-rate values using Negative Signal-to-Noise Ratio loss (NegSNR) [20], while the output of DeepPhys is only one sample point. Therefore, it can be trained with basic point distance objectives only, such as Mean Absolute Error (MAE) or Mean Squared Error (MSE). The input of PhysNet is 128 stacked frames of resolution 128 \(\times\) 128. For better performance the infant's torso and head is cropped from the original 500 \(\times\) 500 image using the result of the YOLO-v3 detector and then downscaled to 128 \(\times\) 128 using area average interpolation. The 128 \(\times\) 128 \(\times\) 128 configuration was selected by taking account of the experiments of [19] where several kinds of configuration were examined and 128 long configuration gave the best results. Rather than simply estimating the pulse rate from the signal with the maximum frequency component, we achieved better results with the application of our rate estimator network, called RateEstNet. The input of the rate estimator network is the output of the signal extractor network. It is a one-dimensional fully convolutional network with an architecture resembling the network of [PERSON] et al. [20]. The PhysNet variant with the best performance from the work of [PERSON] et al. [19] was chosen, namely: PhysNet128-3 DCNN-ED, where 128 denotes the time depth of the network; input (128 frames); 3D denotes that it is a 3-dimensional fully convolutional neural network; and ED denotes that an encoder-decoder part is applied (in time domain). This network architecture is illustrated in Figure 3.  
After each convolutional layer, batchnorm and ELU are applied. The spatio-temporal kernels (Depth, Height, Width) have a size of \([D=3,H=3,W=3]\). The number of the input channels is 3 (128 stacked RGB frames) which is increased to 32 and then to 64. The channel number remains 64 until the last convolution which outputs 1 channel-it functions as a channel average. In the encoder part the time dimension is squeezed to a quarter and is stretched back to its original size in the decoder part. The output is a vector containing the 1-dimensional pulse signal. The total number of trainable parameters is 866,689. The _RateEstNet_ network is also a fully convolutional network, but in this case one-dimensional. It consists of 8 convolutional layers with a kernel size of 17. Dropout is applied before each convolution, and a batchnorm and ELU activation function follow it. The channel number of the input is 1 (as it is a 1-dimensional pulse signal). With the first convolution it is increased to 32, then doubled in every consecutive layer until the channel number reaches 256. Then it is decreased in the same way to 32. After the 3 rd and 8 th convolutional layer a max pool layer is inserted with a kernel size of 5 and a stride of 2. Before the final average pool-that outputs the estimated pulse rate-the output channels of the last (8 th) convolutional layer are weight-averaged using another convolution with a kernel size, stride and output channel of 1. The total number of trainable parameters is 1,482,467.