# Camera-Based Continuous Heart and Respiration Rate Monitoring in the ICU
## 3 Results
### Heart Rate Extraction
#### 3.1.1 Error Analysis of RGB-Based Extraction  
We first identified the minimum RGB face ROI brightness required for accurate measurement by using the patient RGB video recordings where no other monitoring challenges were present. Recordings where other monitoring challenges occurred were excluded from this analysis, as they could affect the choice of the minimally required face ROI brightness. Thus, this analysis consisted of 18 of the 35 total patients (consisting of patients 1, 3, 4, 7, 8, 9, 10, 11, 14, 15, 16, 17, 19, 20, 22, 23, 24, and 30), where it is worth noting that this selection contained 17 postoperative patients and only 1 emergency patient. In Figure 5, we show the total 5 BPM agreement, coverage, and MAE of the RGB recordings of these 18 patients plotted against the face ROI brightness threshold. From these results, we see that for these recordings, the agreement nearly plateaued at the brightness level of 25, with an agreement of 97.3% and MAE of 1.06 BPM. The overall coverage of measurements above this threshold is 80.1%. It is also clear that the only significant factor influencing the quality of RGB-based monitoring in these recordings is the face ROI brightness level, since a very high agreement was achieved for all measurements above the threshold. Next, after we applied the minimum brightness threshold to the heart rate measurements of the remaining 17 patients, we then ruled out the effect of face ROI brightness on the agreement of these patients' heart rate extractions and investigates the remaining errors. First, for 10 patients (12, 13, 18, 26, 28, 29, 31, 33, 34, and 35), a significant loss of agreement can be attributed to the presence of an irregular heart rhythm during the recordings. The heart rate extraction through the Fourier spectrum of a segment can become inaccurate if the heart rate in the segment cannot be represented by a single peak, and thus, the extracted heart rate does often not agree with the mean of the IBI in the corresponding ECG segment. Depending on the severity of the irregularity, it might always cause loss of agreement or only when combined with other influences, which we mention below. Second, upon further investigation of the patients' RGB recordings, we observe that inaccurate measurements often coincided with the presence of staff and/or visitors in the field of view of the camera. Segments where they occluded the face ROI of the patient were already removed from further analysis, and staff and visitors were present in almost all  
Figure 5: Performance of subset of 18 patient RGB recordings evaluated against a minimum face ROI brightness threshold. recordings; as such, further scrutiny was required to arrive at the root cause. Subsequently, we noted that problems occurred most often when the patient was not facing the camera, as is the case in Figure 6a,b. In this situation, the face detector might more easily pick up staff or visitors instead of the patient, or it might not be able to find the patient at all. This alone will cause problems with agreement, since the face ROI is not on the patient. On top of this, the camera might adjust the exposure time using an ROI that does not contain the patient's face, which could result in a very overexposed face ROI until the exposure time is adjusted again, as is the case seen in Figure 6a. The resulting overexposed image in turn can increase the difficulty of finding the correct face ROI, especially if staff and/or visitors re-enter the field of view, which could perpetuate the situation until the patient is alone in the view of the camera for an extended period of time. In its own right, limited visibility of the patients' face can also already lead to inaccurate heart rate measurements. In Figure 6c, we see a patient who is not facing the camera while also having a very limited amount of skin pixels visible in the face ROI. Their blanket occludes part of their face from the camera. Medical tape and/or facial hair can occlude the face in a similar way.