# Performance Evaluation of rPPG Approaches with and without the Region-of-Interest Localization Step
## 4 Discussion
Our SNR values of the rPPG signals are lower in comparison with those provided by [PERSON] et al. [8], whereas RMSE values are lower in case of resting, talking and head translation scenarios. The authors studied the performance of several tracking algorithms (tracked ROI was a rectangular bounding box, corners of which were defined by selected facial landmark feature points). They reported the following SNR (RMSE) values for PURE video recordings: 13.26-13.42 dB (2.16-2.52 BPM) for resting scenario, 3.36-4.06 dB(13.46-15.10 BPM) for talking scenario, 8.27-11.12 dB (2.12-3.19 BPM) for head translation and 7.05-10.46 dB (1.91-6.17 BPM) for head rotation. The results potentially indicate differences between the approaches for ROI localization, while it is important to emphasize that differences may occur due to different calculations of SNR. Although most of the studies rely on the SNR metric proposed by [PERSON] and [PERSON] [21], there are differences in its actual calculation. [PERSON] et al. [8], for example, chose a frequency band of [48; 300] min\({}^{-1}\) (instead of the proposed [30; 240] min\({}^{-1}\)), included the energy around the third harmonic in the energy spectrum of the pulse signal (in the original implementation, only the energy around the first and the second harmonic are included), and used the same spectral window length for calculating energies around the first, second, and third harmonic (in [21], 5- and 10-bins-long windows around the first and second harmonic were used, respectively). In the case of calculating RMSE, the length of the processing window plays an important role when interpreting the results (differences of up to 10% may occur for the window lengths from 0 to 60 s [41]). The described issue results from the already exposed drawback of the rPPG research, i.e., the lack of standardized methodology [42]. There were even some attempts to define standardized report procedures for assessing rPPG PR measurements [43] with the goal to ensure direct comparison of the results of different studies, but there seems to be no general agreement on this issue. Compared to other similar studies, the worst performance in terms of SNR and RMSE is achieved by the Viola-Jones face detector with KLT tracker [18; 20], which indicates that a simple original ROI width reduction applied in our study does improve the skin mask. However, on the other hand, an approach similar to ours was applied by [PERSON] et al. [19], but again, the obtained RMSE was more than two times lower than in other studied approaches. The key advantages of our study are (1) the inclusion of lossy compressed, lossless compressed, and uncompressed videos from publicly available data sets covering various recording scenarios and (2) the inclusion of an approach without the image-processing front-end. These advantages also define the innovative part of our study: the proposed methodology for assessing the performance of the approaches with and without the ROI localization step. The methodology includes (1) publicly available data video recordings of various quality of subjects in different challenging conditions and (2) the inclusion of the approach without the image processing front-end, which has, to the best of our knowledge, never been used before in the evaluation of the performances of the proposed ROI localization steps. The disadvantages are related to (1) the limited number of analysed approaches, (2) the absence of time complexity analysis of the studied approaches, and (3) the small number of studied video recordings covering motion and skin tone scenarios.