## 3 Experiment Setup
### HR and RR Extraction  
Our system extracts HR using both the average grayscale intensity and depth of ROI (Region of Interest)-1 on the right cheek region while measuring RR with just the average depth information of ROI-2 on the chest. Compared to our previous study [28], the ROI-1 is selected based on the need to adapt to more complex scenarios and the mounting angle of the camera. We only select the smooth region on the right cheek so that the signal is less likely to be affected by the facial hair or the uneven shape of the face. To adapt and track ROI-1 and ROI-2 (Figure 2) from various head poses and seating positions that users could potentially take in a car, a more sophisticated face mesh detection algorithm is used [35] to extract 468 facial landmarks (compared to the 66 landmarks used in [28]) from the user's face. The landmarks that encircle the right cheek/nose area are selected as the corner points of ROI-1. After the facial landmarks are extracted, we apply a pose estimator to find the shoulder of the user and define the ROI-2 on the chest using the relative positive between the jawline and the shoulder. When the user sits 50 cm from the ToF camera, ROI-1 contains roughly 500 pixels while ROI-2 contains roughly 8000 pixels. Figure 3a shows a typical intensity as well as the depth captured from ROI-1 while the user is driving on the highway. When a localized illuminator (such as VCSEL used in this study) is used, motions from both the driver and the road create artifacts that are several times larger than the underlying HR signal (Figure 3a). If left uncompensated, such artifacts will lead to inaccurate HR measurements (unwanted peaks in the frequency domain) (Figure 3b). In this study, HR is extracted in three major steps (Figure 4): 1. Extract average grayscale intensity and depth information from ROI-1; 2. compensate for motion-induced intensity artifacts using depth information; 3. use depth information to assess the HR signal quality in different sections during the 20 s video and extract HR from the processed signal. Because the user in a moving vehicle will encounter more random motion, compared to the previous study [28], step 3 is added to further differentiate signal sections with more motion interference from signal sections with less motion interference. Figure 2: Position of both ROI-1 and ROI-2 as seen by the ToF camera. The grayscale intensity and depth information from ROI-1 are used for HR measurement, and the depth information from ROI-2 is used for RR measurements. In step 1, before extracting grayscale information and depth information from each frame, we first remove the background in each frame by only selecting pixels that are 40 to 70 cm from the camera. The removal of the background in each frame eliminates the interference from the passengers in the backseats. After the background removal, we apply the face mesh detection algorithm to find ROI-1 and average the grayscale intensity and the depth across every pixel in ROI-1 [35]. While the face mesh is being obtained, we also determine the head orientation (yaw, pitch, and roll) of the user in each frame using a head pose estimation algorithm [36]. If more than 20% of the frames show excessive head orientation (yaw and pitch deviate more than 10 degrees from head orientation at start), we discard this measurement, as the PPG signal can be heavily polluted by head rotation and our depth-based motion compensation method is less effective against intensity changes caused by large head rotation motion [18; 28]. If less than 20% of total frames exhibit excessive head rotation in the video stream, the system moves on to step 2 for the removal of motion artifacts. When an active illumination source such as VCSEL lasers are used, the amount of light that is back scattered to the camera is nonlinearly correlated to the distance between the camera and the participants' face.