# Spatiotemporal Sensitive Network for Non-Contact Heart Rate Prediction from Facial Videos
## 2 Related Work  
Heart rate can be estimated by processing the rPPG signals extracted from a facial video. Methods for extracting heart rate from facial videos can be categorized into three main approaches: traditional methods, deep learning-based methods and multimodal fusion methods. Traditional methods based on blind source separation and optical modeling are commonly used approaches for heart rate estimation, such as independent component analysis (ICA) [1], principal components analysis (PCA) [2], and CHROM [5]. To improve the effectiveness of measurements in less-constrained situations, some more sophisticated approaches were proposed and obtained improved heart rate measurement accuracy. [PERSON] et al. [3] used a biorthogonal wavelet on rPPG signals to distinguish the unique pattern of arterial subtle variation from interferences, to measure the detail-preserving arterial pulse wave via interference isolation from facial video. They also proposed a novel modulation illumination approach to separate the surface reflections and obtain the subcutaneous diffuse reflections containing the pulse wave information [7]. [PERSON] et al. [4] proposed a framework that can model the BVP dynamics exhibiting nonlinearity and quasi-periodicity, enabling accurate BVP signal extraction that is less sensitive to noise. However, these methods usually make some assumptions about skin reflection (e.g., the linear combination assumption) and the influence of head motion. Such assumptions may not hold in the presence of large head motions and lighting variations. Deep learning-based methods involve constructing network models and training them with large datasets to predict physiological signals. These methods have garnered significant attention due to their robust feature extraction and denoising capabilities. [PERSON] et al. [12] first used deep spatiotemporal networks for reconstructing precise rPPG signals from raw facial videos. [PERSON] et al. [13] used spatial-temporal representation encoding from several ROI (Region of Interest) locations as inputs to convolutional neural networks. [PERSON] et al. [14] converted 3D video into 2D spatiotemporal images as the input to their model, and the proposed model projects a new motion representation to a CNN derived using wavelets, which enables the estimation of heart rate under heterogeneous lighting conditions and continuous motion. [PERSON] et al. [15] proposed a 3D CNN-based spatiotemporal attention network to recover high-quality rPPG signals for heart rate variability analysis, it introduced a GRU to guide the network to learn dependencies in long-term contexts. [PERSON] et al. [16] utilized a multi-head attention mechanism that facilitates feature subspace learning to extract the multiple correlations among the color variations corresponding to the periodic pulse and proposed novel signal embedding to enhance the rPPG feature representation and suppress noise. [PERSON] et al. [17] proposed PhysFormer to adaptively aggregate both local and global spatiotemporal features for rPPG representation enhancement. To better exploit the temporal contextual and periodic rPPG clues, they also extended the PhysFormer to the two-pathway SlowFast-based PhysFormer++ with temporal difference periodic and cross-attention transformers [9]. [PERSON] et al. [10] proposed a two-stage model called MaskFusionNet to extract multi-scale fused heart rate features from a wide range of skin regions while preserving the modeling capability of long-range sequence information. [PERSON] et al. [18] constructed a self-supervised machine learning network to achieve the robust measurement of heart rate. [PERSON] et al. [19] focused on the high-dimensional features in face videos and the global context of skin color changes in the temporal dimension to improve the robustness of heart rate measurement. [PERSON] et al. [20] proposed two plug-and-play blocks to alleviate the degradation of changing distance and head motion.