SYSTEM AND METHOD FOR IMAGING A DRIVER OF A VEHICLE

A system and method for imaging a driver of a vehicle is provided. The system includes an imager mounted to a steering wheel hub that images a scene containing a bodily feature of the driver and generates image data therefrom. An image processor receives and analyzes the image data and generates biometric information related to the driver. The biometric information is useable as input for a variety of vehicle operations.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring toFIG. 1, a vehicle driver compartment2is generally shown having a steering wheel4mounted to a steering column6by a steering wheel hub8. A system10for imaging a driver is provided inside the vehicle and includes an imager12mounted to the steering wheel hub8and configured to image a scene that includes at least a portion of the driver. To accomplish this, the imager12is typically pointed towards the driver and may include a camera positioned on the steering wheel hub8in a manner that does not interfere with other hub and wheel-mounted devices such as airbags and user interface controls. One method for mounting a camera to a steering wheel hub is described in U.S. Pat. No. 6,860,508 B2 to Keutz, filed on Oct. 3, 2002 and entitled “VEHICLE STEERING DEVICE,” the entire disclosure of which is incorporated herein by reference.

Referring toFIGS. 2 and 3, system10is exemplarily shown, wherein the imager12is mounted centrally on the steering wheel hub8and is configured to image a scene14and generate image data16therefrom. The scene14typically includes the area of the driver compartment2containing a bodily feature17of the driver and the image data16typically relates to characteristics of the bodily feature17. The bodily feature17may include a general feature such as a driver's upper torso or a specific feature such as a driver's face and will typically depend on the particular task in which the system10operates.

As shown inFIG. 3, the image data16is received by an image processor18operably coupled to the imager12and configured to analyze the image data16to generate biometric information20. The biometric information20typically includes characteristics associated with the driver being imaged and may be physiological and/or behavioral. Once generated, the biometric information20is outputted to one or more vehicle systems22charged with a vehicle operation. For example, the method for generating biometric information may be a subroutine executed by any processor, and thus, the method may be embodied in a non-transitory computer readable medium having stored thereon software instructions that, when executed by a processor, cause the processor to make the appropriate biometric determinations according to the specified operation.

One consequence of mounting the imager12to the steering wheel hub8is that the object of interest in scene14is typically located at some variable distance from the imager12due to factors such as driver seat and steering wheel positioning in addition to driver physique. At certain distances, the image data16may be susceptible to reduced image quality resulting in less precise driver monitoring for certain operations utilizing the system10. To account for these types of scenarios, the imager12may include zooming capabilities configured to selectively enlarge or reduce the scene14to improve the accuracy of the associated image data16.

Referring toFIG. 4, a flowchart for one embodiment of a zooming algorithm24is shown and applied to a scene enlargement scenario illustrated inFIGS. 4A-4Chaving imaged scenes14a,14b,and14c.To promote a better understanding, a facial tracking operation is described and the zooming algorithm24is exemplarily demonstrated from the vantage point of the imager12shown inFIG. 2. However, it is to be understood that the zooming algorithm24may also be used to enlarge scenes containing other specific and/or general features and may similarly be used for reducing the same. Furthermore, it is to be understood that the imager12shown inFIG. 2may be used in a variety of operations, and as such, is not restricted to the scenario shown inFIGS. 4A-4C, which is described in detail below.

At the start of an imaging session, the imager12is initialized for the particular operation at step S10and subsequently images a scene14acontaining the associated bodily feature17(i.e. the driver's face) in step S12and shown inFIG. 4A. Scene14atypically corresponds to the default image generated by the imager12when mounted to the steering wheel hub8prior to performing the zooming algorithm24. As shown in scene14a,facial outliers typically occupy the majority of the imaged scene14a,which may unduly burden detail oriented operations such as facial tracking given the small image size of the face relative to the total size of scene14a. This condition is remedied by first taking a measured value26relating to the pixel size of the face, as shown at step S14and illustrated in scene14bshown inFIG. 4B. Next, at step S16, the measured value26is compared to a threshold value28, which may be a value stored in memory or generated during initialization (S10) or at some time thereafter. The threshold value28may define a single pixel size or range of pixel sizes, depending on operation. Typically, for facial tracking operations, optimization is better achieved by selecting a threshold value having a single pixel size that provides the most accurate image data. Alternatively, operations that are less detail oriented may opt for a threshold value having an acceptable range of pixel sizes.

If the measured value26matches or is within the range of the threshold value28, the imager12does not perform the zooming algorithm24and proceeds to step S20, where the imager is instructed to either terminate the current imaging session, return to step S12for continued imaging of the bodily feature17, or return to step S10to be initialized for a different operation.

In the event where the measured value26is less than or below the range of the threshold value28, the imager12forms an enlarged scene14cat step S18such that the measured value26of the bodily feature17matches with or is within range of the threshold value28, as illustrated inFIG. 4C. Alternatively, at step S18, the imager12forms a reduced scene (not shown) if the measured value26is greater than or above the range of the threshold value28. In either event, once step S18has completed, the imager12receives further instructions as previously described at step S20.

Another consequence of mounting the imager to the steering wheel hub arises when the steering wheel hub is rotatable with the steering wheel thereby causing the imager to rotate with the steering wheel hub when the driver rotates the steering wheel in either direction. As a result of this rotation, a tilt is applied to the imaged scene, which may hinder the ability of the processing unit to precisely analyze image data generated therefrom.

To avoid this issue, one solution is to use a non-rotatable steering wheel hub such as the one described in U.S. Pat. No. 7,390,018 B2 to Ridolfi et al., filed on Sep. 15, 2005 and entitled “STEERING WHEEL WITH NON-ROTATING AIRBAG,” the entire disclosure of which is incorporated herein by reference.

In instances where the steering wheel hub rotates with the steering wheel, a correction can be used to return the tilted image to an upright position. One exemplary procedure for correcting a tilted scene image is shown inFIGS. 5A and 5B.FIG. 5Aillustrates an imaged scene14dthat is tilted at a variable angle θ, which typically corresponds to the angle of rotation of the steering wheel. To correct the tilt, the steering angle is obtained from a vehicle steering angle sensor and is received by the image processor and used to rotate the tilted scene14din the opposite direction to produce the corrected scene14eshown inFIG. 5B.

An alternative procedure for correcting a tilted scene image is shown inFIGS. 5C and 5D.FIG. 5Cillustrates an imaged scene14fin a non-tilted position prior to rotation of the steering wheel. Using known facial tracking techniques such as edge analysis, the coordinates for the distal endpoints of the eyes are obtained and a connecting line30is drawn therebetween. A reference angle (not shown) is generated between the connecting line30and the horizon. When the imaged scene14ftilts in one direction as a result of steering wheel rotation, the connecting line30makes a different angle relative to the horizon as shown inFIG. 5D. Correction ensues by rotating the imaged scene14fin the opposite direction until the connecting line30once again makes the reference angle relative to the horizon, thereby returning the imaged scene14fto its original upright position previously shown inFIG. 5A. With respect to the instant correction method, it should be recognized that other facial features and geometric relationships based thereon may be similarly used for accomplishing the same or similar tilt correction.

Referring toFIGS. 6 and 7, two flowcharts are shown, wherein each flowchart illustrates an exemplary operation performed by a vehicle system utilizing the system as described herein. However, it is to be understood that the system may be used in conjunction with other vehicle systems to perform other types of operations.

FIG. 6is a flowchart for a driver alertness system32operating to monitor the attentiveness of a driver. At steps S22, S24, and S26, the scene is imaged and enlarged/reduced (if necessary) and corrected for tilt (if necessary). In the instant operation, a logical scene may include the driver's face and acquired image data relating thereto is sent to the image processor to be analyzed so that biometric information relating to driver alertness can be ascertained. For instance, at step S28, image data related to eye and/or head positioning is analyzed to determine the gaze direction of the driver. At step S30, image data related to the openness of the eyes is analyzed to determine driver drowsiness. At step S32, image data related to mouth position is analyzed to determine if the driver is talking. The biometric information generated in steps S28, S30, and S32are taken singly or in combination to provide a notification to the driver in step S34when the driver is in a state of inattentiveness. For example, if the driver is found to be drowsy in step S30, an auditory, tactile and/or visual notification can be sent to the driver via one or more vehicle systems such as the audio system, seat system and/or center display console, respectively. It should be noted that the biometric determinations found in steps S28, S30, and S32are some possible biometric determinations related to driver attentiveness and it is acknowledged that others exist that are determinable using the system described herein. Also, it should be noted in each of those steps, zooming operations and/or tilt correction are free to occur as needed to increase the accuracy of the image data.

FIG. 7is a flowchart for an advanced restraint system34operating to optimize airbag deployment in the event an accident occurs. At steps S36, S38, and S40, the scene image is enlarged/reduced (if necessary) and corrected for tilt (if necessary). At steps S42and S44, image data relating to body size, facial shading, and/or facial features is analyzed to determine biometric information associated with the driver's gender and age. This biometric information may then be used by the advanced restraint system34to optimize deploying power of an airbag in step S46. For example, if the driver is an elderly individual with a small physique, a lessened and/or lower airbag deployment power could be used. At step48, image data relating to the driver's body size and/or orientation is analyzed to determine biometric information associated with the driver's sitting position. This biometric information may then be used by the advanced restraint system34at step S50to optimize the direction of airbag deployment. For example, if the driver is tall, then the direction of airbag deployment will be more upwardly relative to a shorter person.

As should be readily apparent, these are just two of many possible operations benefitting from the use of the system described herein and those having ordinary skill in the art will readily appreciate the versatility and applicability of the system to a wide range of vehicle operations.

Accordingly, a system for imaging a driver of a vehicle has been advantageously described herein. The system is multi-functional and generates biometric information related to the driver that is usable as input for a variety of vehicle operations.