The PhysioCam: Cardiac Pulse, Continuously Monitored by a Color Video Camera1 (2016)

Devices that measure peripheral physiological activity have been used to diagnose clinical disorders, to monitor dynamic changes in physiology as a measure of autonomic regulation, and to explore how the body responds to a variety of environmental and psychological conditions. In a traditional psychophysiological research, arterial pulse rate and respiration are salient physiological signals that respond to behavioral, emotional, and cognitive challenges [1,2]. However, there are situations when direct contact measures are not possible or when direct measures will influence the participant's response. Further, there are situations where physiological parameters need to be monitored in real-time as opposed to extracted post hoc. Thus, a near real-time noncontact solution that provides continuous monitoring of these biosignals is presented.The current study introduces off-the-shelf digital color video as a sensor that can be used to accurately extract human arterial pulse with sufficient precision to derive instantaneous heart rate and heart rate variability (HRV) during a variety of challenges. The study provides an understanding of the physiological source of the pulse signal extracted from digital color video technology, validates a new noncontact measure against criterion signals, and introduces a reliable system capable of delivering the biosignal on a near real-time basis.While our early work in standoff monitoring of the human face in the medium wavelength infrared band provided a real-time measure of respiration that was stable, early testing indicated that no reliable pulse signal was observable within this bandwidth [1]. However, a publication from the Media Lab at MIT [3] reported an ability to extract a pulse signal from visible light video through blind source separation of recorded video. Building from the MIT findings, we designed an approach to extract the pulse signal on a frame-by-frame basis that would enable continuous monitoring of the dynamically changing pulse signal [4].The PhysioCam functions similar to a photoplethysmograph, where a section of skin is illuminated and a light sensor measures the changes of light reflectance as volumetric changes of blood in the arterioles are caused by the heart beating [5]. In the PhysioCam natural light is the passive source, the human face absorbs, transmits, and reflects light, and the light sensor in the camera captures the volumetric changes of the blood under the skin. Figure 1 illustrates the absorption curve of oxygenated (HbO2) and deoxygenated (Hb) hemoglobin in relative units [6]. Absorption curves for two skin types are also shown [7]. The range of maximum sensitivity for red-green-blue pixels of the CCD array is indicated along the wavelength axis. Since skin color differences, due to melanin concentration, predominantly impact absorption of shorter wavelengths, the PhysioCam only utilizes the green (500–575 nm) and red (575–750 nm) light components of the images to estimate relative absorption of light by the blood.The study was approved by the University of Illinois at Chicago Office for the Protection of Research Subjects. Seventeen (male and female) subjects with diverse demographic profiles (between the ages of 19 and 71) were recruited. Subjects performed a series of tasks designed to challenge their autonomic nervous system: resting baseline, deep breathing, cold pressor, and rapid breathing (22 min total duration).The PhysioCam prototype consists of a Grasshopper® IEEE-1394 b (FireWire) CCD camera, and a desktop computer (AMD 4-core, 3.20 GHz, 8 Gb DDR3 RAM) running a custom algorithm developed in LabVIEW™ 2011. The physiological criterion signals were collected with a BIOPAC MP150 system (1000 Hz), using the acqknowledge software.Figure 2 depicts the steps to extract arterial pulse from an image frame. Light (1) reflected by the subject's face is captured by the camera (2). A region of interest (ROI) (3) is defined based in two components: A skin mask (based on a hue-saturation-luminance color model of the individual's cheek region) [8] and a geometric mask (truncated annulus). The resulting ROI is processed using a color histogram (4), mean values of G and R (5) are divided (6) to create a G:R ratio which is the current raw signal sample. The series of raw data (6) is bandpass filtered (0.5 Hz–2.5 Hz) and the first-derivative calculated (7) to yield the final pulse signal (8).Critical steps in the processing include: (1) G:R ratio calculation that serves as a common mode rejection to reduce noise from illumination changes that impact both green and red pixels, (2) skin mask that eliminates pixels unlikely to be skin surface from the histogram calculation, and (3) dynamic geometric ROI size and position. The ROI placement is initiated by the Viola-Jones algorithm (OpenCV) and updated by a Lucas–Kanade optical flow (LabVIEW) method. The ROI is centered on the nose and excludes the mouth region. Each iteration of the process includes capturing the image and processing the current pulse wave sample. All steps are achieved in less than 33 ms on the specified computer, resulting in a continuous 30 Hz pulse signal.The PhysioCam was able to track sequential interbeat intervals (IBIs) during different challenges. Figure 3 shows plots of electrocardiogram (ECG) and PhysioCam IBI measures for one subject performing deep breathing. Pairs of intervals in this example are highly correlated, r(132) = 0.94, p < 0.01.Figures 4 and 5 illustrate the correlations between ECG (x-axis) and PhysioCam (y-axis) derived measures of mean IBI and high frequency HRV calculated by the Porges-Bohrer method [9]. Between subjects correlations were calculated to evaluate the convergence between the methods used to obtain heart period and the high-frequency component of HRV. The correlation between methods approached 1.0 for IBIs and.95 for HRV.The PhysioCam system is based on a validated method for extracting beat-to-beat heart rate from video monitoring of a subject's face. The method has the temporal precision to quantify HRV. In contrast to other methods, the pulse signal is quantified from each frame without dependence on surrounding frames. We continue to improve the system's stability by improving motion tracking performance, correcting for changes in illumination, and increasing the sampling rate through optimization of the algorithms. This work and related technologies are protected by a patent application (PCT/US2014/71602).This work was funded in part by the Technical Support Working Group (TSWG) under Task 3325.