Abstract:
A method inputs a control command for a component of a motor vehicle. The method involves generating an image sequence of an input object guided by a user in a specified detection region using an imaging device, detecting a change in position of the input object on the basis of the image sequence, and generating a control command for the component of the motor vehicle on the basis of the detected change in position. The imaging device employs at least one infrared-sensitive camera, and the detection region is illuminated using at least one infrared source.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is based on and hereby claims priority to International Application No. PCT/EP2013/003137 filed on Oct. 18, 2013 and German Application No. 10 2012 110 460.3 filed on Oct. 31, 2012, the contents of which are hereby incorporated by reference. 
     
    
     BACKGROUND 
       [0002]    The invention relates to a method for inputting a control command for a component of a motor vehicle. 
         [0003]    The operation of an electronic apparatus should, particularly in a motor vehicle, be possible with the least possible distraction of the driver from driving. In this context, a man-machine interaction on the basis of gesture recognition has proven expedient. In this case, the movement of a hand of the user or of another input object in space is identified by an acquisition device, and a corresponding control command for positioning the selection element on a screen of the electronic apparatus is generated therefrom. 
         [0004]    DE 201 22 526 discloses a method in which so-called “structured light”, for example in the form of a fan beam, is used in order to produce a virtual input instrument. It is furthermore known to use stereoscopic cameras in order to determine the spatial position of a hand of a user and to use this for gesture recognition. 
         [0005]    Known methods suffer from the disadvantage of requiring a large computing power for evaluating the images acquired. Commercially available instruments for gesture recognition are furthermore usually designed for use in the home entertainment sector. In contrast to most home applications, however, in motor vehicles there are much more complex illumination conditions, which make the evaluation of acquired images even more difficult. 
       SUMMARY 
       [0006]    One possible object is to provide a method, which allows simple and reliable gesture recognition in motor vehicles. 
         [0007]    The inventors propose a method for inputting a control command for a component of a motor vehicle, which involves:
       a) generation of an image sequence of an input object guided by a user in a predetermined acquisition region by an imaging device;   b) identification of a position change of the input object on the basis of the image sequence;   c) generation of a control command to the component of the motor vehicle on the basis of the position change identified.       
 
         [0011]    According to the proposal, the imaging device comprises at least one infrared-sensitive camera, and the acquisition region is illuminated with at least one infrared source. 
         [0012]    In this way, simple and reliable gesture recognition can be achieved with little equipment outlay. In particular, the absence of stereoscopic image recognition reduces the computing outlay required, so that the image recognition can be carried out in real time. By virtue of the illumination of the acquisition region, reliable recognition is simultaneously made possible even under the difficult illumination conditions in motor vehicles. By way of example, a hand of the user may in this case be used as the input object, although the input of gestures is also possible by acquiring other objects. For example, a nod of the head or a shake of the head of the user may be identified. Objects outside the body, for example a stylus, may also be used. 
         [0013]    It is particularly expedient here for the position change of the input object to be identified by adapting at least two images of the image sequence to a skeleton model of the input object and comparing the parameters of the skeleton model for the at least two images. 
         [0014]    Adaptation to such a skeleton model allows rapid data reduction in the evaluation of the acquired images, so that particularly little computing power is required. Such a skeleton model may describe the shape of the input object by parameters, which describe for example the flexion angle of the individual finger joints. By varying these parameters until the skeleton model describes the same shape of the hand, as can be seen in the image, a set of parameters is obtained, namely for example the flexion angles of the finger joints, with the aid of which the evaluation of the relative position of the finger can be determined by a computer. 
         [0015]    In another configuration, a calibration image is acquired for a predetermined position of the input object in the acquisition region in order to calibrate the skeleton model for a specific input object. This only needs to be done once, in order to be able to identify a particular object later. For example, the size of a hand of the user may thus be acquired accurately in order subsequently to determine position information from the ratio between the actual object size and the image size. 
         [0016]    According to another aspect, the position of at least one point of the input object in a plane perpendicular to a viewing direction of the at least one camera is determined with the aid of the coordinates of the acquired point on a detector matrix of the camera. In this way, two-dimensional position information, which is directly usable on its own for the gesture recognition, is obtained particularly simply. 
         [0017]    In order to determine the position of at least one point of the input object along an axis parallel to the viewing direction of the at least one camera, i.e. to obtain depth information, the depth position is therefore determined with the aid of an image distance of at least two points of the input object and the ratio thereof to a known object distance of the at least two points. 
         [0018]    Further depth information can be obtained by determining the position of at least one point of the input object along the axis parallel to the viewing direction of the at least one camera with the aid of a luminous intensity of the light of the infrared source scattered back by the input object to the camera. This allows accurate distance determination in the simplest way, since the intensity of the light emitted by the infrared source—and therefore also of the light scattered back—decreases with the square of the distance. Even small distance changes therefore lead to a considerable brightness change, so that a high measurement accuracy is made possible. 
         [0019]    In order to accommodate varying light conditions and the limited bandwidth of the camera, it is in this case expedient for an illumination power of the infrared source to be varied cyclically between at least two predetermined power values. In this way, it is reliably possible to avoid overexposures or underexposures, which may impair the distance determination. The variation of the illumination power may in this case also comprise intervals in which the infrared source does not emit any light. Since such cycling of the infrared source is carried out according to a known pattern, in this way the light component of the light source can be reliably separated in the camera signal from environmentally induced fluctuations in the ambient infrared light, so that the image recognition is simplified considerably. 
         [0020]    For the same purpose, it is furthermore advantageous to vary an exposure time of the at least one camera cyclically between at least two predetermined values. Even in the event of strong contrasts in the image, it is thus possible to obtain full information, for example by evaluating dark regions only in images with a long exposure time and bright regions only in images with a short exposure time. 
         [0021]    Since the geometry of the passenger compartment is furthermore known, a position ratio between the at least one point and one vehicle-fixed object with a known position acquired by the at least one camera may be determined in order to identify the position of at least one point of the input object. By setting the object in relation to known vehicle-fixed objects, additional geometrical information is obtained, which can be used in order to improve the accuracy of the position identification or to validate positions already determined. 
         [0022]    The inventors further propose a motor vehicle having an electronic apparatus and a gesture control device with an imaging device for acquiring an operating gesture of an operating object in a passenger compartment of the motor vehicle. According to the proposal, the gesture control device is configured in order to carry out the proposed method. The advantages derive from the advantages explained with reference to the method. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0023]    These and other objects and advantages of the present invention will become more apparent and more readily appreciated from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings of which: 
           [0024]      FIG. 1  shows a schematic representation of the construction of a skeleton model from an input object; 
           [0025]      FIG. 2  shows a schematic representation of the determination of the x-y position of an object by a camera; 
           [0026]      FIG. 3  shows a schematic representation of the improved determination of the x-y position of an object by two cameras; 
           [0027]      FIG. 4  shows a schematic representation of the determination of the z position of an object with the aid of an image size; 
           [0028]      FIG. 5  shows a schematic representation of the relationship between the illumination strength and the distance from a light source; 
           [0029]      FIG. 6  shows a schematic representation of the adaptation of the luminous intensity to varying object distances; 
           [0030]      FIG. 7  shows a schematic representation of the adaptation of the exposure time of a camera to different light conditions. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
       [0031]    Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. 
         [0032]    For the operation of motor-vehicle components, for example entertainment systems, mobile telephones, navigation equipment, or also electrical power windows, seat adjustments, air conditioning systems and the like, the attention of a driver of the motor vehicle should be distracted as little as possible from the road. For this reason, control is intended to be carried out by gesture recognition, since in this case the driver does not need to look for the operating elements, and does not possibly need to divert his attention from the road in order to do so. 
         [0033]    To this end, the interior of the motor vehicle around the driver is recorded with at least one infrared camera  10  and this region is simultaneously illuminated with at least one infrared light source, preferably in the wavelength range of 780-870 nm. From the recorded image sequence, variations in the position of a hand  14  or of another input object can be determined, and these can in turn be assigned to particular control commands. 
         [0034]    In order to reduce the computing outlay for the image recognition as much as possible, and to permit reliable real-time processing of the camera images, for this purpose a so-called skeleton model  16  of the hand  14  is initially constructed in a calibration phase. For this, the hand is recorded for the first time in a predetermined calibration position, so that the basic dimensions of the hand  14  can be determined. The resulting skeleton model  16  then assigns to the hand  14  the position of the individual joints  18  and of the fingertips  20 , and furthermore comprises the invariant distances between respectively connected points. 
         [0035]    Images subsequently acquired with the camera  10  for the gesture recognition can then be evaluated by adapting the skeleton model  16  to the shape of the hand  14  in the image. In this case, the possible movement space of the hand  14 , which is limited by the respective flexion angle ranges of the joints, is searched for a configuration which corresponds to the image of the hand  14 . 
         [0036]    From changes in the configuration of the hand  14  and its position relative to the camera  10  in sequences of successive images, a respective gesture of the user can then be identified and the corresponding control command can be generated. 
         [0037]    Besides the configuration of the hand  14  as defined by the flexion angles of the joints  18 , it is thus also necessary to determine its position with respect to the camera  10 . 
         [0038]    Determination of the position of the hand  14  in the x-y plane, i.e. the plane perpendicular to the viewing direction of the camera  10 , is in this case particularly simple. The camera  10  comprises a matrix sensor  22  having a multiplicity of pixels  24 . The x-y position can therefore be determined easily from the position of the image  26  of the hand  14  on the matrix sensor  22 . 
         [0039]    An increased accuracy may be achieved by using two cameras  10 . The cameras  10  are in this case to be oriented in such a way that the pixels  24  of the respective matrix sensors  22  are not exactly congruent, but rather have an offset from one another, as represented in  FIG. 3 . 
         [0040]    Besides the x-y position, however, it is also necessary to determine the position of the hand  14  in the z direction, i.e. in the viewing direction of the camera  10 . To this end, there are several possibilities. 
         [0041]    First, the distance between the hand  14  and the camera  10  may be deduced from the size of the hand  14 , known by the calibration, and the image size of the hand  14  on the matrix sensor  22  of the camera  10 . As shown in  FIG. 4 , the image size is dependent on the distance and size of the hand relative to the camera  10 , so that for example a small hand at a short distance makes the same image angle as a large hand at a larger distance. On this basis, a rough z position can be determined with an accuracy of about ±5 cm. 
         [0042]    An improved accuracy is achieved when the hand  14  is moved relative to the z axis. From the change of the image size in successive images, it is possible—substantially on the basis of the centric transformation±to calculate the distance change of the hand with an accuracy of ±1 cm. For many gesture recognition problems, such an accuracy is already sufficient. 
         [0043]    In order to determine the z position with the highest accuracy, the propagation characteristic of the light of the infrared source  12  may be used. Specifically, the illumination strength, i.e. the light flux per unit area, is inversely proportional to the square of the distance from the infrared source  12 . This naturally also applies for the light scattered back or reflected from the hand  14  to the camera  10 . It follows that even small changes in distance between the hand  14  and the infrared source  12  lead to strong luminous intensity changes in the camera image of the hand  14 , on the basis of which the distance change can be determined with an accuracy of ±0.5 mm. 
         [0044]    In this case, however, the problem arises that the bandwidth of the camera  10  is restricted. If, in the event of a strong luminous intensity of the infrared source, the hand  14  is located very close to the camera  10 , then overdriving of the camera  10  may occur, so that useful image evaluation is no longer possible. As illustrated in  FIG. 6 , it is therefore expedient to vary the luminous intensity of the infrared source  12  cyclically between a plurality of predetermined values. Images in which the camera is overdriven or greatly underexposed can then be discarded, so that sufficiently many well-exposed images are always available for the evaluation and gesture recognition. 
         [0045]    This may be further reinforced by a cyclic variation of the exposure time of the camera  10 , as shown in  FIG. 7 . The time is plotted on the longitudinal axis, each bar of the diagram symbolizing an exposure of the camera  10 . Here again, incorrectly exposed images or image ranges are respectively discarded. 
         [0046]    Besides adaptation to the strongly varying intensity of the light scattered back by the hand  14 , this moreover makes it possible to minimize error sources due to incident ambient light, which may vary greatly in the motor vehicle. 
         [0047]    Since both the configuration of the hand and its position in all spatial directions can now be acquired, these values may be stored for each recorded image. From the sequence of changes in these parameters, gestures of the driver can then be identified reliably with known image analysis methods. 
         [0048]    All the analysis methods described may be improved by the use of a plurality of redundant cameras  10 . This allows plausibilization of the values acquired by each individual camera  10 , and optionally the exclusion of implausible recordings or evaluations. In this way, despite strong perturbing influences present in the motor vehicle, gestures can be reliably identified and used for the control. 
         [0049]    The invention has been described in detail with particular reference to preferred embodiments thereof and examples, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention covered by the claims which may include the phrase “at least one of A, B and C” as an alternative expression that means one or more of A, B and C may be used, contrary to the holding in  Superguide  v.  DIRECTV,  69 USPQ2d 1865 (Fed. Cir. 2004).