# Unobtrusive Vital Sign Monitoring in Automotive Environments--A Review
## 5 Optical Methods
### Image Fusion  
In automotive applications, the concept of image and sensor fusion dates back more than 20 years. In the beginning, imaging systems were often fused with other modalities (like radar) and were targeting mainly object recognition outside the car. For example, in 1997, [PERSON] et al. [146] described the fusion of infrared, laser and radar-based range information using a Kalman filter to improve the range detection at longer look-ahead distances required for \"autonomous intelligent cruise control\" (AICC) applications. A few years later, [PERSON] et al. [147] used the fusion of radar and binocular stereo videos for a depth fusion algorithm. The improved depth information was then employed in a target segmention algorithm (like identification of other cars, pedestrians, etc. ), which is required for autonomous driving. Furthermore, they demonstrated that the fusion of two camera image streams can be used for range detection, making an additional radar system obsolete. [PERSON] et al. [148] presented a system that fuses color and shape information for improved vehicle detection rates in 2004. In 2007, an interesting approach using a multi-wavelength and multi-camera approach for pedestrian detection was described by [PERSON] et al. [149]. Their system employs two color and two infrared cameras in the front of the vehicle. Note that pedestrians are producing very different features in the two modalities, allowing for obtaining a unique fingerprint for pedestrian classification. Apart from these systems directed outwards of the vehicle, systems monitoring the vehicle occupants have also been presented. In 1999 and 2000, [PERSON] et al. [150; 151] introduced a system using near-infrared imaging for vehicle occupant counting. Two near-infrared camera streams, below and above 1400 nm, were fused to segment vehicle occupants. The system was able to successfully distinguish between humans and mannequins. Possible applications include the surveillance of freeway lanes reserved for car pools. Camera systems directed inside the vehicle can also be used to improve the safety of occupants. In 2002, [PERSON] et al. [152] described a vision system aiming at improving the safety of airbag deployment. Ignited airbags inflate very rapidly; they can cause severe head trauma if the vehicle occupant is too close to the device. In addition, airbags can actually kill small children in infant seats if they are placed in the front passenger seat. In such situations, the airbags must not be deployed. [PERSON]'s system uses two cameras and extracts four features from the images (range, edges, motion, shapes). Classifiers are used to distinguish the vehicle occupant, e.g., driver in normal position or infant in infant seat. A fusion algorithm weights the classifier outputs according to the confidence and decides about the airbag deployment. Two years later, [PERSON] et al. [153] extended the work of [PERSON] et al. by combining stereo and thermal video to determine the vehicle occupant position. As discussed before, thermal imaging has the benefit that it does not require illumination. In 2004, [PERSON] et al. [154] combined visible and infrared images to improve face detection in people wearing eyeglasses. Infrared imaging has the benefit of being independent of illumination, but cannot pass through spectacles. On the other hand, visible imaging operates through eyeglass, but requires illumination. Therefore, a combination of both modalities can improve coverage rates. This, of course, is important to determine the PERCLOS ratio (see above) or the direction of view. A yawning detection as a sign of driver fatigue was presented by [PERSON] [155] in 2009. It uses a real-time adaptation of the region of interest (ROI) of a camera. A low-resolution camera captures a larger region of the driving compartment. A face detection is performed on that image stream and a second camera is directed onto the mouth of the driver according to the detected face.