# Camera-Based Continuous Heart and Respiration Rate Monitoring in the ICU
## 1 Introduction  
Patients in hospitals have a 17% chance of experiencing at least one complication postoperatively [1], which increases to up to 27% for patients undergoing major surgery. These statistics show how important facilitating the early detection of postoperative complications is in hospitals. Outside of high acuity departments such as the ICU, detection of complications is based on intermittent spot checks of vital signs. The main limitation of spot checks is the time between measurements, which is typically a few hours. Deviations in vital signs often manifest several hours before an adverse event occurs [2] but could take too long to detect using spot checks. Continuous monitoring of vital signs has been previously shown to contribute to earlier detection of adverse events in surgical departments [3]. However, current continuous monitoring techniques, which are used in the ICU for patientmonitoring, are not suitable for deployment on low-acuity departments. Wired sensors are not generally acceptable there, as they could impede patient mobilization and can be uncomfortable for patients [4]. Instead, wearable or contactless sensors would be more appropriate for continuous monitoring on low-acuity departments. Camera-based vital signs monitoring is one group of relatively new technologies that could facilitate continuous monitoring in low-acuity departments. The main advantage of camera-based monitoring is that it is contactless and thus unobtrusive compared to contact sensors. Camera-based monitoring research focuses mainly on three different parts of the electromagnetic spectrum: visible light (RGB), near-infrared (NIR) light, and far-infrared light (thermal radiation). In these three groups, many techniques have been developed to monitor various vital parameters, such as heart rate [5; 6; 7; 8], heart rate variability [9], respiration rate [10; 11; 12; 13], SpO\({}_{2}\)[14; 15], and temperature [16]. Furthermore, research has also shown that monitoring other facial features, such as the redness of the skin, can be used as markers for detecting patient deterioration [17; 18]. Camera-based heart and respiration rate monitoring is by far the most mature of these technologies. In this work, we will focus on remote photoplethysmography (remote PPG) for heart rate monitoring and breathing motion extraction for respiration rate monitoring. First, remote PPG is a technique that measures microscopic skin color variations caused by the cardiovascular pulse wave. The remote PPG signal can be extracted using RGB and NIR cameras. Many methods have been proposed for the (robust) extraction of the remote PPG signal [5; 6]. Second, breathing motion extraction is based on detecting the respiratory movements of the torso. Various methods have been previously proposed, for example, based on optical flow [10] or cross-correlation [11]. While camera-based heart and respiration rate monitoring is a well-explored field of research, most validations of these techniques have been performed on healthy volunteers and/or short video recordings. There are potentially substantial challenges in continuous camera-based monitoring that cannot be easily emulated by short recordings on healthy subjects. These include naturally occurring patient activity, changes in illumination, occlusion of the patient, and clinical interventions. Although ideally these camera-based monitoring challenges are investigated in patients in low-acuity departments, where we would like to apply this technology, the lack of continuously measured reference values for heart and respiration rates limits what we can learn about the accuracy of camera-based monitoring in this environment. Although not ideal, the ICU provides a suitable tradeoff between the availability of reference values and the presence of challenging monitoring situations. Previous research by [PERSON] et al.