Abstract:
A method synchronizes two display devices in a motor vehicle. The first display device already displays a first data set which comprises a first display content relating to a predetermined thematic context. A sensor device, in particular a PMD camera, detects as a control gesture of a user, a free movement in the chamber of a body part of the user. A signal which is then generated by the sensor device and which describes the control gesture, is transmitted to a gesture recognition device which generates a control signal in accordance with the determined control gesture. In accordance with the control signal, a control device determines the predetermined thematic context of the determined data set. Using the defined predetermined thematic context, an additional data set which includes a second display content of the predetermined thematic context is provided. This additional data set is displayed on the second display device.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is based on and hereby claims priority to International Application No. PCT/EP2013/003542 filed on Nov. 23, 2013 and German Application No. 10 2013 000 068.8 filed on Jan. 8, 2013, the contents of which are hereby incorporated by reference. 
     
    
     BACKGROUND 
       [0002]    The invention relates to a method for synchronizing two display devices of a motor vehicle. 
         [0003]    In general, functions have previously been triggered in the motor vehicle by operating, e.g., keys. To detect the operations, e.g. of a key, an electrical or a capacitive switch is necessary for each key. Triggering the functions requires the operation of the, e.g., mechanical pushbutton. Display devices such as, e.g., a head-up display, a display of a combined instrument or infotainment system in the motor vehicle are systems separated purely functionally. The display of information requires elaborate operation of digital menus. 
         [0004]    DE 10 2010 056 411 A1 describes the display of information on a center display of a motor vehicle relating to a waypoint for which a reference is displayed on a predetermined other display device, namely a color display in the environment of the screen base of the front screen. To interact with the display devices, a finger tracking system can detect a wiping gesture of a user. 
         [0005]    Transmission of a digital image which is displayed on a head-up display of a motor vehicle to another display device is known from DE 10 2005 010 843 A1. On the other display device, it is no longer information-carrying data but an icon which is displayed. Triggering of this function can take place via a hand movement of the user and be detected via an optoelectronic sensor. 
         [0006]    DE 10 2007 035 769 A1 describes switching the representation of information of a display device to another display device, a video camera detecting a hand movement of a user. 
         [0007]    Due to the many possible technical devices for displaying information, however, the disadvantage arises that the user cannot freely select the two display devices involved. A further problem arises if a multiplicity of data records for different thematic contexts are displayed on a universal display device, e.g. on a combined instrument with a screen, i.e., e.g., data relating to a motor speed, to the driving speed, to the tank filling level and to detected malfunctions. However, the display device of the related art to be synchronized can only be synchronized if the display device only displays data records of the same context. 
       SUMMARY 
       [0008]    One possible object is to provide a method for synchronizing two arbitrary display devices. 
         [0009]    The inventors considered on the concept of detecting an operating gesture, that is to say a movement, carried out freely in space, of a body part of the operating person, by a video camera which generates a three-dimensional image data record. In this context, the operating gesture is preferably a pointing, pulling and/or gripping gesture. This allows an elaborate installation of operating elements in the display devices to be dispensed with. In addition, it is made possible to select the involved display devices and the relevant data records. 
         [0010]    The inventors propose a method for synchronizing a first display device of a motor vehicle with a second display device, to be synchronized, of the motor vehicle, that is to say for synchronizing, e.g., a head-up display with, e.g., a display in the center console. In this context, the first display device already displays a first data record, e.g. a picture element (e.g. a warning symbol), which comprises a first display content relating to a predefined thematic context. 
         [0011]    An optical sensor device, e.g. a PMD camera (i.e., a photonic mixing device), detects an operating gesture of a user. The sensor device generates a signal describing the operating gesture (e.g. a video signal) on the basis of 3D image data and transmits this to a gesture detection device. On reception of the signal of the sensor device, the gesture detection device determines, by the determined operating gesture, the data record selected by the user by the operating gesture and the data record already displayed by the first display device. 
         [0012]    A control device, e.g. a control device of the motor vehicle, receives a control signal generated by the gesture detection device which describes the first data record determined, and determines, by the control signal, the predefined thematic context of the first data record. 
         [0013]    The control device can be, for example, also a program module of the first or of the second display device or can be implemented in a plurality of part areas, e.g. both display devices. If, for example, the control device is a program module of the first display device, it can determine, on the basis of the control signal, that is to say on the basis of a trigger of the gesture detection device, the determined predefined thematic context of the selected data record, e.g. by itself, and transmit it to the second display device. 
         [0014]    By the predetermined thematic context determined, a further data record having a second display content of the predetermined thematic context is provided. The further data record is displayed on the second display device. Providing the further data record can be carried out, e.g., by one of the display devices or by a data processing device of the motor vehicle, possibly in interaction with a central data server on the Internet. If, e.g., the first display device transmits the predetermined thematic context of the selected data record determined to the second display device, it can thus, e.g., independently determine and display the further data record. The constellation which is selected here can depend on the technical environment in which the proposed method is implemented. 
         [0015]    Thus, e.g., the first data record can comprise an image data record for, e.g., a warning symbol for a reference to, e.g., a critical operating state of the motor vehicle, whilst the further data record comprises, e.g., a detailed description text relating to the critical operating state of the motor vehicle. Thus, e.g., a text file supplementing the image file can be displayed on the second display device, i.e., e.g. a fault description. 
         [0016]    Apart from the above-mentioned advantages, the advantage is obtained that the display contents are not only exchangeable, but different data records having a different amount of information content relating to a context can be output. In spite of a multiplicity of various information items on a display, the user, therefore, does not have to operate an operating element such as, e.g., a key or a digital menu for the precise selection of a data record. As a result, the user can also select when he wishes to have which volume of information displayed for which information. 
         [0017]    In one embodiment of the method according to the method, the control device can control the transmission of the further data record from a motor-vehicle-internal data memory to the display device to be synchronized. In this case, various groups of data records are present in the data storage device, a group of data records comprising those data records which have a common thematic context. 
         [0018]    However, the contexts are here the least of which was known before the synchronization and must be stored in at least one data storage device. Alternatively or additionally, the control device can call up the further data record from an external data server, e.g. from an Internet server of a motor vehicle manufacturer or generate, on the basis of operating parameters which are provided by a device for operating the motor vehicle, the further data record. Such data records can be provided by the control device in a group of data records of the data memory. 
         [0019]    At least one of the display devices is preferably equipped without its own sensor. In particular, neither display device is equipped with its own sensor system. The operating gesture is preferably detected by an optical sensor device on the basis of 3D image data of a time-of-flight camera (TOF camera), PMD camera or stereocamera of the motor vehicle which is preferably a part of another device of the motor vehicle. The three-dimensional detection of the operating gesture provided enables a directional movement to be detected and increases the number of variations of operating gestures. Due to the spatially precise detection, the display devices to be synchronized become freely selectable and a multiplicity of display devices can be used for the information flow to the user. A sensor device of this type, already present in the motor vehicle, can be used for the method without the display devices having to be equipped with sensors. In addition, operating elements such as pushbuttons or touch-sensitive buttons can be omitted in the display device. 
         [0020]    In a further preferred embodiment, the gesture detection device, by the signal of the sensor device, determines which of the display devices is the display device already displaying the data record and which of the display devices is the display device to be synchronized. This can be done, e.g., by a pulling gesture from one display device to another. The display devices thus no longer have to be selected by the elaborate operation of a menu by, e.g., keys. 
         [0021]    In a development of this embodiment of the method, the gesture detection device, for the purposes of detecting at least a part of the operating gesture, can check a trajectory of a body part, that is to say a motion curve of a moving body part, to see whether it describes a pulling and/or gripping movement from the first display device to the second display device. In this manner, the direction of synchronization is detected. 
         [0022]    The gesture detection device, for detecting the operating gesture, can extend the trajectory of a body part by extrapolation of the part of the operating gesture, and check whether the extended trajectory represents a pulling and/or gripping movement. This provides for faster operation. Thus, the operating gesture does not need to be completed by the user if he has to move the body part carrying out the gesture spontaneously back to, e.g., the steering wheel. 
         [0023]    The object mentioned above is also achieved by a motor vehicle, particularly a car or a passenger vehicle, if it comprises a gesture detection device and is designed for carrying out an embodiment of the proposed method. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0024]    These and other objects and advantages of the present invention will become more apparent and more readily appreciated from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings of which: 
           [0025]      FIG. 1  shows a block diagram for a sensor device and for a control device which can be installed in an embodiment of the proposed motor vehicle, and 
           [0026]      FIG. 2  shows a sketch for an embodiment of the proposed method. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
       [0027]    Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. 
         [0028]    In  FIG. 1 , a sensor device  10  of a motor vehicle, for example a passenger vehicle, is shown. In the present example, the sensor device  10  comprises an optical sensor  12 , e.g. a PMD camera, and a measuring device  14  which can detect the measurement data of the optical sensor  12  (method S 10 ) and transmit them to a gesture detection device  16  (S 20 ). The optical sensor  12  can also comprise, e.g., another 3D camera such as, e.g., a TOF camera or a stereocamera and is preferably mounted on the ceiling of the motor vehicle. 
         [0029]    The optical sensor  12  can be designed in a manner known per se, e.g. with a luminous source  20 , e.g. an infrared lamp which illuminates a detection area  22 , for example a space above a dashboard of the motor vehicle. If this contains an object, for example a hand  24 ′ of the driver of the motor vehicle, the electromagnetic radiation radiated by the luminous source  20  is reflected back to a sensor array  26 . By the sensor array  26 , 3D image data can then be generated which specify  3 D coordinates relating to individual area elements of the hand  24 ′. Additionally or alternatively, the interior space of the motor vehicle can also be surveyed in this manner. The 3D image data are transmitted by the measuring device  14  to the gesture detection device  16 . 
         [0030]    The gesture detection device  16  can be a gesture detection device  16  known to the expert. 
         [0031]    By the gesture detection device  16 , extremities can be segmented from the 3D image data, by which, for example, the position of a fingertip in the detection area  22  can be determined. In this context, segmenting algorithms known per se can be used as a basis. The 3D image data of the sensor array  26  of the optical sensor  12  can also represent a sequence of successive 3D images, i.e. by the optical sensor  12 , movements of the hand  24 ′ or an extension of a lower arm can also be detected. By tracing the trajectory, for example of the fingertip in this 3D image sequence, particularly by tracing the position and the speed of the fingertip, a gesture of movement indicated by the fingertip can thus be extrapolated from the trajectory. 
         [0032]    Depending on this evaluation, the gesture detection device  16  then generates a control signal (S 30 ).  FIG. 1  also shows a control device  18  of the motor vehicle which can be, e.g., a control device for controlling two (not shown in  FIG. 1 ) display devices  30 ,  34 . The control device  18  receives the control signal of the gesture detection device  16  and coordinates the interaction of the display devices  30 ,  34  to be synchronized. 
         [0033]      FIG. 2  shows an embodiment of the proposed method. 
         [0034]    As shown in the example of  FIG. 2 , a data record from a group of data records can be transmitted from a data storage device  38  to the display device to be synchronized for displaying the data record during the coordination of the interaction of the display devices  30 ,  34  to be synchronized. In this case, the control device  18  can thus generate, on reception of the control signal of the gesture detection device  16 , e.g. a synchronizing signal which causes the second display device  34  to call up a particular data record  40  from the data storage device  38  or which causes the data storage device  38  to transfer the data record  40  to the second display device  34 . In actuality, this can look as follows: 
         [0035]    A user is located in the interior space of a motor vehicle. On a first display device  30 , e.g. a head-up display on a windscreen, a plurality of picture elements  32 ,  32 ′,  32 ″ are displayed, e.g. warning or other symbols which signal an operating state. The picture element  32  is, e.g., an image of a warning triangle which informs the user of, e.g., the presence of important operating information. In  FIG. 2 , a first position of a body part  24  of the user, e.g. a lower arm with a hand  24 ′ (dashed line), and a second position of the body part  24  (continuous line) are shown. The positions P 1  and P 2  symbolize a movement of the body part  24  from the first to the second position. In the present example, this movement with the finger position shown is an operating gesture for synchronizing, e.g., a display  34  of an infotainment system above the center console  36  with the display device  30 . The sensor device  10  which comprises, e.g., a PMD camera detects, e.g., the stretching out of the lower arm and the gripping or pointing movement of the hand  24 ′ (P 3 ). 
         [0036]    The sensor device  10  detects the operating gesture and/or the direction into which the body part  24  of the user points in the position  28  (S 10 ). For this purpose, the sensor device  10  can detect the coordinates of the hand  24 ′ within a coordinate system of an interior space of the motor vehicle. By the coordinates, the display devices  30 ,  34  involved in the synchronization are determined by the gesture detection device  16 . Similarly, the data record  32  from the group of which in the data storage device  38  a further data record is to be displayed on the display device  34  to be synchronized can thus be determined by the gesture detection device  16 . 
         [0037]    Between the data storage device  38  and the display device  30  and between the data storage device  38  and the display device  34 , a wireless or wired communication link  42  can be present. In the present example, the data records  32  and  40  of the group of data records G 1  are present in the data storage device  38 , both of which data records  32  and  40  have a common context, e.g. a warning message, and which, as a result, form one group. A further group of data records G 2  comprises, e.g., the picture element  32 ′ already displayed on the head-up display  30  which, e.g., symbolizes a current occurring playing back of an audio file and which comprises a text file  40 ′ thematically linked to this which, e.g., comprises bibliographic data relating to the audio file  32 ′. 
         [0038]    The detection of the operating gesture and/or of the coordinates of the operating gesture within the system of coordinates of the interior space of the motor vehicle (S 10 ), the generation of a signal describing the operating gesture and its transmission to the gesture detection device  16  (S 20 ) take place as already described in  FIG. 1 . 
         [0039]    The gesture detection device  16  can determine, e.g., by the direction, described by the signal of the sensor device  10 , of the lower arm stretching out with the hand  24 ′ (P 4 ) that, e.g., the display device  30  is the display device which already displays a data record. Analogously thereto, the gesture detection device  16  can determine by the signal of the sensor device  10 , which describes a pulling movement (P 1 , P 2 ) and/or a direction of the body part  24  with the hand  24 ′ in a second position (P 5 ), that the display device  34  is the display device  34  to be synchronized. Alternatively, e.g., a throwing gesture in the direction of the second display device  34  can also be determined as a selection gesture for, e.g., the display device  34  to be synchronized. 
         [0040]    In addition, the gesture detection device  16  can determine by the direction (P 3 ), described by the signal of the sensor device  10 , of the lower arm stretched out with the hand  24 ′ during the execution of the gripping movement (P 4 ) that the data record  32  is the data record “selected” by the user. With the aid of the coordinates of the body part  24  within the system of coordinates of the interior space of the motor vehicle, the gesture detection device  16  can determine, e.g., the axes of the lower arm and/or the finger axes. By these axes, the position  28  of the respective limb  24  can be determined. If the forearm axes then point to the picture element  32  as in the example of  FIG. 2  and the gesture detection device  16  determines a gripping movement of the hand  24 ′, the picture element  32  is selected. Alternatively, the gesture detection device  16  can detect a relative movement of the hand  24 ′ and, as a result, can control a cursor of the display device  30 , as a result of which the user can select the picture element  32 . This detected selection of the picture element  32  is followed by the selection of the data record relating to the picture element  32 . On this basis, the control device  18  can determine the predetermined thematic context and possibly, e.g., the associated group of data records. 
         [0041]    A further possibility for determining the thematic context is if, e.g., the gesture detection device  16  generates on the basis of a determined operating gesture a control signal following which the control device  18  generates a synchronization signal and transmits the latter to the first display device ( 30 ). The first display device ( 30 ) thereupon transmits information relating to the thematic context of the first data record ( 32 ) to the control device ( 18 ). 
         [0042]    In the present example, the control device  18  determines, e.g., the group of data records G 2 . Alternatively or additionally to the example shown in  FIG. 2 , the control device  18  can call up the further data record  40 , e.g. from an Internet server, e.g., of a motor vehicle manufacturer, or generate it on the basis of, e.g., the operating data relating to an oil level. 
         [0043]    The gesture detection device  16  generates a corresponding control signal to a control device  18  which, as already described with respect to  FIG. 1 , causes the data record  40  to be displayed, e.g. in this case a text message relating to the warning, on the display device  34  (S 40 ). In the present example, the control device  18  causes the data storage device  38  to transfer the data record  40  to the display  34  of the infotainment system (S 50 ). 
         [0044]    The gesture detection device  16  can also be designed to extrapolate a trajectory (T) of a body part, that is to say, e.g., a part of the pulling movement represented by the positions P 1  and P 2 , and to check it to see whether it will represent a pulling and/or gripping movement. Thus, the user can “nudge”, that is to say only indicate, the “displacement” of the data record (that is to say the synchronization of the display devices  30 ,  34 ), e.g. in a situation in which he must unexpectedly quickly pull back his arm to the steering wheel. This means that he does not have to execute the operating gesture to its end. 
         [0045]    The examples illustrate the principle of the proposals according to which a physically correct locatability of the passengers (e.g. by a TOF camera or a PMD camera) or of body parts of the passenger allows an approach to a surface to be detected. To displace information (contexts) from one display device to another display device, a gesture can be used. For example, a warning symbol can be displayed in a head-up display, for example. By reaching, for example, in the direction of the warning symbol and pulling it, e.g. in the direction of the center display, a detailed description of the warning can appear, for example. 
         [0046]    This provides for a universal interaction between the displays (in addition to an enhancement of the joy-of-use factor). In addition, the interaction does not require any pushbuttons. 
         [0047]    The invention has been described in detail with particular reference to preferred embodiments thereof and examples, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention covered by the claims which may include the phrase “at least one of A, B and C” as an alternative expression that means one or more of A, B and C may be used, contrary to the holding in Superguide v. DIRECTV, 69 USPQ2d 1865 (Fed. Cir. 2004).