# Performance Evaluation of rPPG Approaches with and without the Region-of-Interest Localization Step
## 2 Materials and Methods
### Performance Evaluation of the Studied Approaches with and without ROI Localization Step  
Performance of the studied approaches was assessed by three quality metrics: (1) success rate, (2) modified SNR metric proposed by [PERSON] and [PERSON] [21], and (3) RMSE of the difference between reference- and rPPG-derived PRs. Success rate was defined as a percentage of successfully processed video recordings among the total number of recordings for each studied approach. Whenever an approach failed (regardless of the processing stage at which a failure occurred), further processing was stopped and the processing of a given video recording was marked as unsuccessful. SNR was calculated as the ratio between the energy within 5 bins around the first harmonic (or fundamental frequency) plus 11 bins around the second harmonic and the remaining energy within the frequency band of expected PRs in humans (i.e., [30; 240] BPM) ([PERSON] and [PERSON] [21] took 10 bins around the second harmonic, but it is unclear, how the authors positioned this window relative to the second harmonic):  
\[\text{SNR}=10\log_{10}\left(\frac{\sum_{f=30}^{240}(w_{t}(f)S(f))^{2})}{\sum_ {f=30}^{240}(1-w_{t}(f)S(f))^{2}}\right) \tag{1}\]  
where \(f\) denotes the PR frequency (expressed in BPM), \(w_{t}\) the applied binary window and \(S(f)\) the power spectrum of rPPG pulse waveform signal. Fundamental frequency was defined as the peak frequency within the defined frequency band in the power spectrum of reference pulse waveform signals (obtained by Fast Fourier Transform). In the case a window around the second harmonic fell outside the set PR band, we extended the spectrum accordingly (this is the second modification of the SNR metric proposed by [PERSON] and [PERSON] [21] that we applied). Before calculating SNR, the reference signals were pre-processed by (1) applying first order Butterworth bandpass filter with lower and upper cut-off frequencies corresponding to 40 and 180 BPM, respectively, and (2) resampling to the sampling rate that matched the frame rate of the accompanying video recordings. The reason for applying bandpass filter with cut-off frequencies different from the upper- and lower frequencies in the frequency band corresponding to expected PRs in humans is the fact that we adopted the values of the actual data in order to achieve cleaner reference signals. SNR values were calculated for windowed signals (512-frame-long sliding window was applied). The results are visually presented in the form of boxplots if the size of each group within the studied scenario is larger than or equal to five, whereas strip plots are used otherwise. RMSE was defined as the square root of the average of squared differences between the estimated (\(PR_{\text{rPPG}_{i}}\)) and actual, i.e., reference, PR values (\(PR_{\text{ref}}\)):  
\[RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(PR_{\text{rPPG}_{i}}-PR_{\text{ref}_{i}})^ {2}}, \tag{2}\]  
where \(n\) denotes the total number of estimated PRs. PR values were measured using a sliding window of 512 frames; we took into the account that, in general, a PR is not constant throughout the entire recording, not even in a resting condition. Statistical evaluation of the results was carried out using one-way analysis of variance (ANOVA). ANOVA was used in the cases when number of samples in each studied group was larger than or equal to five. We set the length of the sliding processing window to 128 frames for POS, 60 frames for 2 SR and 128 frames for the FVP approach.