# Camera-Based Continuous Heart and Respiration Rate Monitoring in the ICU
## 1 Introduction
[19] on camera-based heart and respiration rate monitoring in the ICU provided insights on the coverage and mean absolute error (MAE) using monochrome recordings of 15 postoperative ICU patients for a total duration of 233.5 h. First, they manually removed privacy moments and occlusions of the patient by visitors or staff. Then, from the remaining video they extracted heart rate with a coverage of 53.2% and respiration rate with a coverage of 63.1%. In our work, we intend to add to their insights into camera-based monitoring performance and coverage by including other types of ICU patients. To be precise, we intend to evaluate state-of-the-art camera-based heart and respiration rate extraction methods on a group of postoperative cardiac surgery patients, which is a surgery type not included in the work of [PERSON] et al. [19], and a second group of patients who had an emergency admission to the ICU. These groups of patients tend to have a wider range in their vital parameters and a higher rate of adverse events compared to the types of postoperative patients in the work of [PERSON] et al. [19]. This can provide useful insights into whether the system is accurate even during abnormalities in the vital parameters, which can be an indicator of future deterioration. Furthermore, [PERSON] et al. [19] used a monochrome camera to extract heart rate. However, previous research [5] shows that heart rate extraction based on a single wavelength camera is inferior to techniques that combine multiple color channels. In this work, we will show that for many patients, the only limitation on performance and coverage is the room lighting, where for these patients, a 5 beats/min (BPM) agreement of 97.3%, where the average heart rate is 80 BPM, and a coverage of 80.1% was reached. When room lighting is insufficient, a monochrome camera that is sensitive in the NIR spectrum and a NIR light source are still needed to provide coverage. This presents the additional challenge of combining measurements from both modalities depending on lighting conditions. Finally, [PERSON] et al. [19] performed extraction of the respiration rate through changes in the intensity of the pixels caused by breathing motion. Instead, an optical flow method is likely more effective for breathing motion extraction, since pixel intensity changes may not always correlate with breathing motion. [PERSON] et al. [10] demonstrated that such a method can be highly sensitive to breathing motion. This is required to measure the breathing motion of patients with varying breathing patterns, amplitudes of chest movement, and postures relative to the camera. However, this method cannot yet distinguish the breathing motion from any other motion. The presence of non-breathing motion should be detected, as any respiration rate extracted while non-breathing motion is present could be inaccurate. In this work, we will therefore develop a metric through which we can determine whether extracted breathing motion is distorted by non-breathing motion or not. In this work, our aim is to provide new insights into the performance and coverage of camera-based continuous heart and respiration rate monitoring using state-of-the-art methods. While it is unlikely that perfect coverage can be achieved with the current methods, our aim is to evaluate the usability of camera-based monitoring to replace spot-check-based monitoring conducted in the general ward. In this setting, it will not be crucial to measure absolutely continuously, but up-to-date measurements need to be available when required by hospital staff. As such, challenging situations where camera-based monitoring becomes inaccurate need to be understood, as the prolonged presence of these situations could lead to the unavailability of up-to-date and accurate heart and respiration rate measurements. As such, in this work, we will investigate where challenging situations occur and what their effect is on the extraction of the heart and respiration rate.