# Spatiotemporal Sensitive Network for Non-Contact Heart Rate Prediction from Facial Videos
## 4 Experimental Results
### Intra-Dataset Testing  
(1) Results on the VIPL-HR Dataset: We first compared our proposed method with state-of-the-art (SOTA) methods on the VIPL dataset to validate its performance. This included traditional methods (CHROM [5], POS [6]) and deep learning-based methods (Rhythmnet [13], those of [PERSON] et al. [14], those of [PERSON] et al. [18], rPPG-MAE [8], PhysFormer [17], PhysFormer++ [9], TranPhys [19], MaskFusionNet [10], and FusionViViT [24]). It is important to note that the input data used in these comparison methods are all single-modal RGB data, except for FusionViViT [24]. To evaluate our model more objectively, we set up the SS-Net-RGB model, which stands for using only RGB modal data as input, i.e., no MLSC module, and the structure of the SS-Net-RGB network is shown in Figure 6. The experimental results are shown in Table 1. From Table 1, it is evident that CHROMperforms best among traditional methods due to its introduction of a standardized skin color vector, which enhances robustness in complex environments, although there remains a gap compared to deep learning methods. Additionally, Table 1 demonstrates that deep learning methods achieve higher accuracy in heart rate estimation compared to traditional methods, as they can learn more important spatiotemporal information encoded in facial videos. Notably, compared with the work of [PERSON] et al. [14], the proposed SS-Net outperforms 2.37 bpm in the MAE metric because the SS-Net obtains more accurate heart rate estimation results by focusing on the temporal context of rPPG signals. In addition, both the SS-Net and SS-Net-RGB are significantly better than the SOTA methods, and the SS-Net is more effective than SS-Net-RGB. This is attributed to the enhancement of physiological signal features across different spatiotemporal scales through the SS module, and the integration of fusion features from different levels in the spatiotemporal dimension using the MLSC module. Specifically, the SS-Net achieved an MAE of 2.86 bpm, an RMSE of 4.74 bpm, a MER of 3.35%, and an \(r\) value of 0.95 for heart rate estimation. Figure 7 illustrates the Bland-Altman plot and scatter plot of heart rate prediction results on the VIPL dataset. The average HR difference was 0.28 bpm, with 95% limits of agreement ranging from \(-\)8.16 to 8.72 bpm. The Pearson's correlation coefficient of the proposed method was r = 0.95 and the coefficient of determination was R\({}^{2}\) = 0.91, indicating a good correlation between the predicted and ground truth values. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **MAE** & **RMSE** & **MER** & \(r\) \\ \hline CHROM [5] & 11.4 & 16.9 & 17.8\% & 0.28 \\ POS [6] & 11.5 & 17.2 & 18.5\% & 0.30 \\ RhythmNet [13] & 5.30 & 8.14 & 6.71\% & 0.76 \\ [PERSON] et al. [14] & 5.23 & 7.21 & - & 0.82 \\ [PERSON] et al. [18] & 5.09 & 10.60 & - & 0.69 \\ rPPG-MAE [8] & 4.52 & 7.49 & - & 0.81 \\ PhysFormer [17] & 4.97 & 7.79 & - & 0.78 \\ PhysFormer++ [9] & 4.88 & 7.62 & - & 0.80 \\ TranPhys [19] & 4.69 & 7.27 & - & 0.82 \\ MaskFusionNet [10] & 4.37 & 6.95 & - & 0.82 \\ FusionViViT [24] & 11.70 & 14.86 & - & \(-\)0.085 \\ SS-Net-RGB & 3.56 & 5.05 & 4.48\% & 0.95 \\ SS-Net & 2.86 & 4.74 & 3.35\% & 0.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Estimation of HR on the VIPL-HR dataset by the proposed method and several SOTA methods. Figure 6: Overall structure of the proposed SS-Net-RGB. Figure 8 compares the true rPPG signals with the predicted rPPG pulses, revealing a high correlation between the predicted and true signals, further demonstrating the efficiency of our method. To visually demonstrate our method's efficiency in predicting HR, Figure 9 compares predicted HR values to true HR values over varying time windows. It can be observed that even when the true HR fluctuates significantly in short periods, the predicted HR values remain near the true HR values, demonstrating the robustness of the proposed method. Figure 8: Comparison of PPG signals on four samples taken from VIPL-HR.