# Spatiotemporal Sensitive Network for Non-Contact Heart Rate Prediction from Facial Videos
## 4 Experimental Results
### Dataset and Experimental Setup  
The visual information processing and learning for heart rate (VIPL-HR) dataset [38] comprises 2378 color videos and 752 near-infrared videos from 107 participants (79 males and 28 females) aged between 22 and 41 years. The videos were recorded using three different devices (Logitech C310, HUAWEI P9, RealSense F200), with each video segment lasting approximately 30 s and captured at a frame rate of about 30 fps. Participants were allowed to perform natural body movements such as talking and head rotation, making the dataset more representative of real-world conditions. Additionally, physiological signals such as HR, BVP, and blood oxygen were simultaneously recorded for each participant. The average grayscale values of faces in the VIPL-HR dataset range from 60 to 212, covering a wide range of lighting conditions. Heart rates range from 47 to 146 beats per minute, encompassing a typical range of heart rates. The MERL-Rice Near-Infrared Pulse (MR-NIRP) dataset [39] consists of color and near-infrared videos from eight subjects (two females and six males) with variations in skin color and facial hair, aged between 20 and 40 years. Each video segment is approximately 3 min long, recorded at a frame rate of about 30 fps. The UBFC-rPPG dataset [40] includes 54 visible light videos from 49 subjects, recorded using Logitech C920 HD Pro, with each video segment lasting approximately 1 min, recorded at 30 fps with a resolution of 640 \(\times\) 480. The Univ. Bourgogne Franche-Comte Remote Photoplethysmography (UBFC-rPPG) dataset also includes recorded PPG waveforms and heart rates. In this study, all videos and video segments were standardized to 300 frames with a stride of 15 frames between segments. Each detected face in every frame was downsampled to 180 \(\times\) 180 pixels. A window size of w = 5 and h = 5 was applied to generate spatiotemporal maps used as training and testing samples. To validate the effectiveness of the proposed structure, subject-exclusive 5-fold cross-validation was utilized.  
Experimental Setup: Due to the PPG signals in the datasets having different sampling rates than the camera frame rates, we resampled PPG signals to synchronize with the video frame rates. Finally, PPG signals from the VIPL-HR, MR-NIRP, and UBFC-rPPG datasets were resampled at 30 frames per second. We set the training epochs to 160 and the batch size to 32. In addition, we used the Adam optimization algorithm, initiating with a learning rate of 1 \(\times\) 10\({}^{-4}\), which was subsequently decreased to 5 \(\times\) 10\({}^{-5}\) after half of the epochs. For the loss function, \(\alpha\) set to 0.1. Our proposed model was implemented using PyTorch and trained on an NVIDIA GeForce RTX 3090 GPU with 24 GB of memory.  
Evaluation Metrics: To evaluate the effectiveness of the SS-Net in predicting rPPG and HR, we use metrics such as Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Standard Deviation (SD), Mean Error Rate percentage (MER), and the Pearson correlation coefficient (r). Lower values of MAE, SD, MER, and RMSE indicate that \(HR_{pre}\) is closer to \(HR_{gt}\). The coefficient \(r\) represents the linear correlation between \(rPPG_{pre}\) and \(rPPG_{gt}\), where a value closer to 1 signifies a stronger linear relationship between them.