# Performance Evaluation of rPPG Approaches with and without the Region-of-Interest Localization Step
###### Abstract
Recently, new approaches have been proposed that do not require a ROI localization step [16; 17]--these are the approaches with no image processing front-end [17]. The idea behind them is that, since we are interested only in the rPPG pulse waveform signal and not in the ROIs and their location specifics, we can treat a camera as a single rPPG sensor [16]. In the Full Video Pulse extraction (FVP) approach, a temporal mean of a color signal is used as a feature to differentiate between the image background and foreground (i.e., skin pixels) by first producing a set of weighting masks, then extracting rPPG signals from each mask, and finally combining the signals into a single pulse waveform signal [16]. Another approach is single-element rPPG, in which the average pixel intensity values of the full frames are extracted from the entire video recording [17]. In contrast to FVP, the existing skin reflection model-based rPPG algorithms showing the best general performance exhibit a significant performance drop in extracting target pulse waveform signal when applied in single-element rPPG. Therefore, this approach should be used with a dedicated rPPG algorithm, e.g., SoftSig [17]. SoftSig consists of two steps, a projection step and a selection step, in which pulse waveform signals candidates are extracted and the best candidate rPPG signal (in terms of predefined quality metrics) is selected, respectively [17]. The selection of the image processing front-end approach in rPPG strongly affects the signal-to-noise ratio (SNR) of the extracted rPPG pulse waveform signal [8; 18; 19; 20]. [PERSON] et al. [18] evaluated the performance of seven different image processing front-end approaches (in combination with model-based CHROM rPPG algorithm [21]) in PR extraction from the publicly available Univ. Bourgogne Franche-Comte Remote Photoplethysmography (UBFC-RPPG) data set [6]. The studied approaches included Viola-Jones frontal face detector with KLT tracker and different ROI refinement algorithms: (1) no refinement, (2) ROI cropping [7], (3) RGB-based skin-color detection [10], (4) skin detection based on the maximization of mutual information [15], (5) adaptive RGB-based skin-color detection, (6) graph-cut based skin segmentation [22] and (7) facial landmarks segmentation [14]. The worst result was obtained for [PERSON] face detection algorithm with KLT tracking with no ROI refinement, whereas the overall best performance was reached when skin detection based on the maximization of mutual information was applied for refining ROIs [15]. It is to be noted that the studied video recordings covered resting and mathematical game playing scenarios only. [PERSON] et al. [19] reported that Viola-Jones outperforms Normalized-Pixel-Difference-based face detector proposed by [PERSON] et al. [23], KLT outperforms Camshift [24], and skin detector based on the maximization of mutual information [15] outperforms RGB-H-CbCr [11] in terms of root mean square error (RMSE) of average PRs. The authors used the UBFC-RPPG data set [6] and extracted the pulse waveform signals using blind-source-separation-based rPPG algorithm FastICA [25]. Later, [PERSON] et al. [20] additionally assessed the performance of selected approaches with different ROI localization steps in rPPG-based PR measurement: Viola-Jones face detector [26] and KLT [27], in combination with (1) no additional algorithms, (2) skin detector based on the maximization of mutual information [15], and (3) facial landmarks detector [14]. The best overall performance, in terms of correlation coefficient, percentage of PR values below 2.5 and 5 BPM errors, mean absolute error (MAE), RMSE, and SNR, was achieved by the combination of Viola-Jones, KLT and skin segmentation, and the worst by the combination of Viola-Jones and KLT. The studied data set was UBFC-RPPG [6], and CHROM [21] was used for the final rPPG signal extraction. [PERSON] et al.