## 3 Measurement of State Data Based on Non-Contact Image Analysis  
### Optical Blood Flow (Heart Rate) Signal Acquisition  
Photo-plethysmography (PPG) is used to measure the blood flow signal by measuring the change in blood flow that occurs according to the heartbeat through the color change of the fingertip or face image. We acquire the optical blood flow signal through the face image, and in order to stably extract the optical blood flow signal, accurate face detection and consistent tracking of the skin area are required. To minimize the background pixels unrelated to the skin during face detection, an SSD [28]-based face detector was used instead of the traditional Vi-ola-Jones detector [29]. As shown in Figure 2a, parts that are not related to pulsating skin such as hair, eyebrows, and background pixels are still included in the face area detected through the SSD. In consideration of real-time characteristics, the background area was removed by modeling the skin color distribution through a statistical method in the YCbCr color space instead of a deep learning-based segmentation algorithm. In the RGB color space, the red, green, and blue channels have a high correlation, and it is difficult to separate the lighting component and the color component. Figure 1: State data integration interface. In addition, it is very likely that noise is included in the extracted signal due to the fine movement of the body, the three-dimensional structure of the face surface, and the position change with the lighting. The light component was discarded, and the light blood flow signal was extracted by focusing on the change in skin color according to the change in the amount of light blood using the color difference component. Compared to other color signal components, the color-difference signal shows a distinct pulsating waveform, and a component corresponding to the pulse in the frequency spectrum is well revealed. The color difference signal extracted from Figure 2b shows a distinct pulsating waveform compared to other color signal components, and it is shown in Figure 3 that the component corresponding to the pulse rate is well revealed in the frequency spectrum. In the signal extracted from Figure 3, elements not related to cardiac activity are also included and a process to remove them is necessary. This is the normalization work to remove noise such as facial movement and breathing, which have relatively low frequencies. Figure 3: Comparison of (**a**) time series and (**b**) frequency spectrum for each channel of the skin pixel filtering applied face area. Figure 2: Skin pixel filtering applied to the face area detected through SSD. (**a**) Before filtering, (**b**) After filtering. The signal was normalized using the average according to the time interval, and the window size was set as the sampling rate to include at least one pulse period in the interval. As a result, it was possible to obtain a zero-centered signal from which the DC component was removed during the normalization process. Since there are still noises corresponding to high frequency generated by lighting changes in the signal, camera sensors, etc., band pass filtering was applied to remove them. The passband was set to (0.7, 3.0) corresponding to 42-180 BPM, and a Butterworth filter of order 5 was used. As a result, as shown in Figure 4, a signal that facilitates heart rate estimation was obtained by removing a significant portion of noise from the contaminated signal through signal normalization and band-pass filtering of the raw signal. In addition, it is possible to extract additional physiological parameters by performing analysis in the frequency domain and time series domain by interpreting the normalized signal as an optical blood flow signal synchronized with the user's cardiac activity. Power spectral density detection and analysis as shown in Figure 5 was performed by converting to the frequency domain in order to extract the average pulse rate for the measurement section from the optical blood flow signal.