Abstract:
A method and an apparatus for displaying the surroundings of a vehicle and to a driver assistance system having such an apparatus having at least one sensor for producing sensor data for the surroundings of a vehicle. In this case, the sensor data are conditioned to produce raw image data, if need be using a grid model of the surroundings of the vehicle, if need be the raw image data are processed to produce object information using a grid model of the surroundings of the vehicle, and the object information obtained is used to condition raw image data to produce image object data, and finally the image object data are displayed.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is the U.S. National Phase Application of PCT International Application No. PCT/DE2014/200509, filed Sep. 26, 2014, which claims priority to German Patent Application No. 10 2013 220 005.6, filed Oct. 2, 2013, the contents of such applications being incorporated by reference herein. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The invention relates to a method and an apparatus for displaying the surroundings of a vehicle, which are particularly suitable for use in driver assistance systems, as well as a driver assistance system. 
       BACKGROUND OF THE INVENTION 
       [0003]    Driver assistance systems as well as methods and apparatuses for displaying the surroundings of a vehicle are in principle known from the prior art. 
         [0004]    Reference is made regarding this, for example, to DE 10059900A1, which is incorporated by reference as well as DE10109665A1, which is incorporated by reference. So-called camera-based top view, surround view and rear view systems are increasingly found in vehicles, in order to display the immediate environment of the vehicle to the driver and, as a result, to make it easier for him to e.g. maneuver his vehicle. 
         [0005]    so DE10247563A1, which is incorporated by reference describes a driver assistance system, in which objects which may be dangerous to a vehicle appear more pronounced on a display device than those objects which pose less of a danger. DE10257484A1, which is incorporated by reference discloses a driver assistance system having a display device for representing objects detected with a camera correctly in terms of the perspective. Furthermore, additional information regarding the detected objects can be graphically displayed, in order to improve the perception thereof by a driver. 
       SUMMARY OF THE INVENTION 
       [0006]    An aspect of the invention is an improved method and an improved apparatus for displaying the surroundings of a vehicle. 
         [0007]    One embodiment of this invention relates to a method for displaying the surroundings of a vehicle comprising the following steps:
       detecting of the surroundings of a vehicle with at least one sensor and producing of sensor data for the surroundings of a vehicle,   conditioning of the sensor data to produce raw image data,   processing of the raw image data to produce object information using a grid model of the surroundings of the vehicle,   use of the object information obtained and/or of a grid model to condition raw image data to produce image object data,   displaying of the image object data.       
 
         [0013]    In particular, it can be provided that the conditioning of the sensor data to produce raw image data, the processing of the raw image data to produce object information and the conditioning of the raw image data to produce image object data, using the object information and/or a grid model, are effected by means of at least two separate devices. In this case, the conditioning of the sensor data to produce raw image data is preferably effected by means of an image conditioning device and the processing of the raw image data to produce object information is preferably effected by means of an image processing device, wherein the image processing device receives the raw image data from the image conditioning device via a first data link (forward channel). Furthermore, the conditioning of the raw image data to produce the image object data is preferably effected by means of the image conditioning device, wherein the image conditioning device receives the object information from the image processing device via a second data link (backward channel). The image conditioning device is preferably a device for conditioning the sensor data and/or the raw image data, in order to display image data in the vehicle. The image processing device is preferably a device for obtaining object information, for example, for a device for controlling vehicle parameters and/or vehicle functions. As a result, a multiple-step image data processing method is created, wherein the object information obtained from the image processing for controlling vehicle parameters and/or vehicle functions can be advantageously used, in particular by means of a backward channel from the image processing device to the image conditioning device, for the conditioning of the image data for display in the vehicle, for example in order to highlight objects and/or to mark them in the image when the image data is displayed. 
         [0014]    An additional embodiment of this invention relates to a method for displaying the surroundings of a vehicle comprising the following steps:
       detecting of the surroundings of a vehicle with at least one sensor and producing of sensor data for the surroundings of a vehicle,   conditioning of the sensor data to produce raw image data in a first conditioning step,   conditioning of the raw image data to produce corrected image data using a grid model in a second conditioning step,   use of object information obtained on the basis of sensors in order to condition the corrected image data to produce image object data,   displaying of the image object data.       
 
         [0020]    A multiple-step method which improves the image data is therefore provided. The sensor data are conditioned to produce raw image data, with the aid of which object information can then be obtained, which can in turn be fed back in order to condition raw image data. Raw image data conditioned from sensor data in a forward channel direction can thus be transmitted, in order to process the raw image data and, in a backward channel, object information for (if need be further) conditioning of raw image data can be transmitted. As a result, the conditioning of the image data can be improved and, afterwards, combined image object data containing additional information compared to the pure image data can be displayed. 
         [0021]    The at least one sensor can, in this case, be configured as any kind of suitable sensor for producing raw image data, in particular as an optical sensor such as e.g. as an optical camera, night vision system and/or residual light amplification system, infrared camera or thermal image camera. However, the sensor can in principle also be configured to receive electromagnetic radiation of different wavelengths such as e.g. radar or lidar, or to receive sound waves such as e.g. ultrasound. The object information can be obtained on the basis of a grid model of the surroundings of the vehicle. The object information can therefore be detected e.g. in the form of a raster card and/or stored and/or transmitted. In addition, information can be specifically assigned to partial areas of the grid model. 
         [0022]    Object information can be information for characterizing and/or identifying objects and/or conditions of objects, in particular this information can be the position, size and/or height, extent, brightness, direction of movement, acceleration, relative movement to the sensor, relative direction of movement to the sensor, relative acceleration to the sensor. 
         [0023]    A grid model can also be used within the framework of conditioning the raw image data. A suitable correction and/or distortion correction of raw image data, in particular of distorted raw image data can, for example, be achieved with this due to optical and/or metrological distortions due to particular sensor characteristics such as e.g. imaging properties. Corrected image data, which correspond to corrected images, are therefore produced from raw image data. These can then be taken as the basis for additional image processing steps and/or image conditioning steps. 
         [0024]    It can be provided that the object information is verified and/or stabilized by being compared with data from additional sensors. Additional sensors and/or sensor data can therefore also be enlisted in addition or as a supplement to the aforementioned sensor data which are used for producing the raw image data. These can e.g. be additional optical sensors such as e.g. optical cameras, night vision systems and/or residual light amplification systems, infrared cameras or thermal image cameras. However, the additional sensors can also in principle be configured to receive electromagnetic radiation of different wavelengths such as e.g. radar or lidar, or to receive sound waves such as e.g. ultrasound. The extent of the detected object information and/or the accuracy of the detected object information can be increased by the additional sensor information and a contribution is therefore made to the object information being stabilized, and/or errors in the object information obtained can be reduced and/or identified object information and/or objects can be verified. It can be provided that, within the framework of the processing of the raw image data, historical and/or predicted object information is additionally used. A time curve of object information can therefore be determined and/or stored and/or called up from a storage device and can also be enlisted during the processing of the raw image data. The change over time in object information (such as the change over time in position, size, extent, brightness, direction of movement, acceleration, relative movement to the sensor, relative direction of movement to the sensor, relative acceleration to the sensor) can also be enlisted as well. It can be provided that the grid model of the surroundings comprises cells, wherein partial object information is assigned to said cells and object information is produced from partial object information in an object identification step. The cells of the grid model can be configured identically, i.e. have the same size, form and orientation. 
         [0025]    The cells can therefore have a uniform distribution and a uniform configuration. The cells of the grid model can, however, also have a size and/or form and/or orientation dependent on the position of the cell in the grid. 
         [0026]    In particular, it can be provided that a 3D grid model is used as a grid model, therefore a grid model in which the grid comprises a three-dimensional form. A three-dimensional form is to be deemed to be any form which deviates in at least a partial area from a two-dimensional and, thus, planar form. Raw image data can be projected onto the 3D grid model using a projection algorithm. Thus, a particularly well suited correction and/or distortion correction of raw image data, in particular of distorted raw image data—for example due to optical and/or metrological distortions due to particular sensor characteristics such as e.g. imaging properties—is/are achieved. 
         [0027]    The 3D grid model of the surroundings of the vehicle can, in particular, constitute a model of a curved surface, wherein the radius of curvature increases as the distance increases from the vehicle and/or from the sensor producing the sensor data. It has been shown that such models of a curved surface, in particular having a curvature which increases with the distance, are suitable for correcting distortions due to wide-angle effects such as those which occur e.g. with optical fish eye sensors or other sensors with comparable imaging properties. 
         [0028]    Cell-related partial object information can be assigned to the cells of the grid model. 
         [0029]    Such partial object information can be information for identifying and/or characterizing detected object parts or detected structures such as information regarding the position, size such as e.g. object heights, form, extent, brightness, direction of movement, acceleration, relative movement to the sensor, relative direction of movement to the sensor, relative acceleration to the sensor. Cell-related object profiles can be formed from this partial object information and assigned to the respective cell. Object profiles of one or more cells can be combined to produce objects. If, within the framework of the method, an object identification step is carried out, object information regarding identified objects can be transmitted with object information regarding object heights to the image conditioning device. 
         [0030]    An additional embodiment of this invention relates to an apparatus for displaying the surroundings of a vehicle, comprising:
       at least one sensor for detecting the surroundings of a vehicle and for producing sensor data for the surroundings of a vehicle,   an image conditioning device having image data conditioning areas which are set up to condition the sensor data to produce raw image data and to condition raw image data using object information and/or a grid model to produce image object data,   an image processing device which comprises a data link to the image conditioning device and for processing raw image data to produce object information using a grid model, and   a display apparatus for displaying the image object data.       
 
         [0035]    With respect to the advantages and possible additional configurations of the apparatus according to the invention, reference is made to the explanations regarding the method according to the invention, which similarly apply to the apparatus according to the invention. 
         [0036]    The image conditioning device is preferably connected via a first data link (forward channel) to the image processing device, wherein the image processing device receives the raw image data conditioned by the image conditioning device via the first data link. Furthermore, the image processing device is preferably connected via an additional data link (backward channel) to the image conditioning device, wherein the image conditioning device receives the object information obtained by the image processing device from the raw image data via the additional data link. 
         [0037]    The image conditioning device is, in particular, connected to the display apparatus for displaying image object data obtained from the raw image data and the object data. 
         [0038]    A forward channel can therefore be provided from the image conditioning device to the image processing device and a backward channel can be provided from the image processing device back to the image conditioning device. The image conditioning device is set up to condition the sensor data to produce raw image data. Furthermore, the image conditioning device can be set up to condition raw image data using object information. However, the image conditioning device can also be set up to condition raw image data using a grid model—if need be incorporating additional object information—to produce image object data. 
         [0039]    The at least one sensor can, as already explained above, be configured as any kind of suitable sensor for producing raw image data, in particular as an optical sensor such as e.g. as an optical camera, night vision system and/or residual light amplification system, infrared camera or thermal image camera. However, the sensor can in principle also be configured to receive electromagnetic radiation of different wavelengths such as e.g. radar or lidar, or to receive sound waves such as e.g. ultrasound. 
         [0040]    Additional sensors can be provided, which have a data link to the image processing device. In this case, the image processing device can be set up to verify and/or stabilize the object information by means of a comparison with data from the additional sensors. As already explained above, these can e.g. be additional optical sensors such as e.g. optical cameras, night vision systems and/or residual light amplification systems, infrared cameras or thermal image cameras. However, the additional sensors can also in principle be configured to receive electromagnetic radiation of different wavelengths such as e.g. radar or lidar, or to receive sound waves such as e.g. ultrasound. 
         [0041]    The image conditioning device can comprise image data conditioning areas which can differ functionally, in terms of programming technology or hardware technology, according to their functions. The image conditioning device can thus comprise a first image data conditioning area for conditioning the sensor data to produce raw image data and a second image data conditioning area to condition raw image data and object information to produce image object data. 
         [0042]    A data storage device can be provided in order to store historical and/or predicted object information, wherein the image processing device is set up to use historical and/or predicted object information within the framework of the processing of the raw image data. A time curve of object information can be stored in this data storage device and can be called up from this storage device and a data link to the image processing device can be provided, via which link the historical and/or predicted object information can be transmitted for processing of the raw image data. 
         [0043]    Furthermore, an additional data storage device for storing a grid model of the surroundings of the vehicle can be provided, wherein the additional data storage device is connected via a data link to the image processing device and the image processing device can be set up to identify objects on the basis of partial object information, which is assigned to cells of the grid model. Alternatively or, in addition, the image conditioning device can be set up to condition raw image data on the basis of the grid model, wherein the image conditioning device is then connected to the data storage device via a data link. An additional embodiment of the invention relates to a driver assistance system having an apparatus according to the invention and as described herein. Additional advantages and possible applications of this invention are set out in the following specification in conjunction with the embodiment examples shown in the drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0044]    The terms and assigned reference numerals in the list of reference numerals indicated at the back of this application are used in the specification, in the claims, in the summary and in the drawings, wherein 
           [0045]      FIG. 1  shows a diagrammatic view of a system having automatic driver assistance functionalities; 
           [0046]      FIG. 2  shows a diagrammatic view of a system having automatic driver assistance functionalities comprising a plurality of sensors and backward channel for transmitting object information; 
           [0047]      FIG. 3  shows a diagrammatic view of a planar grid model; 
           [0048]      FIG. 4  shows a diagrammatic view of a distortion corrected top view image; 
           [0049]      FIG. 5  shows a diagrammatic view of the distortion correction and perspective correction of a fish eye camera image (raw image data); 
           [0050]      FIG. 6  shows a diagrammatic view of a vehicle having four cameras and four individual fish eye camera images (raw image data); 
           [0051]      FIG. 7  shows a diagrammatic view of a 3D grid model having a radius of curvature dependent on the distance; 
           [0052]      FIG. 8  shows a diagrammatic view of an image which has been distortion corrected by means of a 3D grid model; 
           [0053]      FIG. 9  shows a diagrammatic view of the length distortion when a stationary object is projected onto a plane; 
           [0054]      FIG. 10  shows a diagrammatic view of the length distortion when a stationary object is projected onto a curved surface. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
       [0055]    The same, functionally similar and functionally related elements can be provided with the same reference numerals in the following specification. Absolute values are only indicated below by way of example and are not to be deemed to restrict the invention. 
         [0056]    Driver assistance systems are to increasingly be equipped with automatic driver assistance functionalities. To this end, image processing can be used, in order to automatically identify objects, in particular obstacles, and subsequently facilitate an intervention in the vehicle dynamics.  FIG. 1  shows such a driver assistance system  10  diagrammatically. 
         [0057]    This driver assistance system  10  comprises an optical sensor, for example a camera as a first sensor  1  for acquiring sensor data for the surroundings of a vehicle. An image conditioning device  2  is connected to the first sensor  1 . The image conditioning device  2  is set up to condition the sensor data to produce raw image data and/or to produce conditioned image data. The image conditioning device  2  is, furthermore, connected to a display apparatus  5 —in particular a suitable display—in order to display the conditioned image data to the driver in the vehicle. 
         [0058]    The driver assistance system  10  additionally comprises an image processing device  4  which is connected to the image conditioning device  2  via a data link  3  and receives conditioned raw image data from sensor data from there. The image processing device  4  is connected to the vehicle dynamics  13 —i.e. a device for controlling vehicle parameters and/or vehicle functions via an additional data link  9 . 
         [0059]    It can now in particular be provided that, in the case of a driver assistance system, objects identified, particularly obstacles, are displayed with a display device prior to an intervention in the vehicle dynamics, in order to make the automatic vehicle intervention comprehensible to the driver or to allow him to react prior to the intervention in the vehicle dynamics.  FIG. 2  shows such a driver assistance system  100  diagrammatically. 
         [0060]    The driver assistance system  100  according to  FIG. 2  also comprises a first sensor  101  in the form of an optical sensor, e.g. a camera for acquiring sensor data for the surroundings of a vehicle. An image conditioning device  102  is, in turn, connected to the first sensor  101 . The image conditioning device  102  is set up to condition the sensor data to produce raw image data in the driver assistance system  100  according to  FIG. 2 . The image conditioning device  102  can thereby comprise image data conditioning areas  112 ,  122  which are functionally separated from one another, which differ in terms of programming technology or hardware technology and which can be arranged separated from one another in terms of programming technology or hardware technology, so that various image conditioning steps can be carried out in parallel. Thus, the image conditioning device  102  can comprise a first image data conditioning area  112  for conditioning the sensor data to produce raw image data and a second image data conditioning area  122  for conditioning raw image data and object information to produce image object data. The image conditioning device  102  is connected to a display apparatus  105  - in particular a suitable display, in order to display the conditioned image data to the driver in the vehicle. 
         [0061]    The image conditioning device  102 —and, in this case, in particular the first image data conditioning area  112  for conditioning the sensor data to produce raw image data—is connected via a first data link  103  (forward channel) to an image processing device  104  which receives conditioned raw image data from the image conditioning device  102 . The image processing device  104  is, on the one hand, connected via an additional data link  109  to the vehicle dynamics  113 —i.e. a device for controlling vehicle parameters and/or vehicle functions. On the other hand, the image processing device  104  is connected via an additional data link  106  (backward channel) to an image conditioning device  102 —and, in this case, in particular to a second image data conditioning area  122  for conditioning raw image data and object information to produce image object data. 
         [0062]    According to the embodiment according to  FIG. 2  it is thus made possible to transmit object information such as e.g. distance from the obstacle and the height thereof by means of a backward channel from the system block of the image processing into the system block of the image conditioning. This information can be used in the image conditioning, in order to display a potential hazard posed by identified objects (e.g. by colored marking of objects in the conditioned image object data) and/or the distance and/or height thereof relative to the driver&#39;s own vehicle. 
         [0063]    The identification of objects in the image processing device  104  can be based on a grid model  21 ,  221 . Cells  22 ,  222  of a grid model  21 ,  221  enclosing the vehicle  20 ,  220  are defined. This grid model  21 ,  221  can be stored, e.g. in a data storage device  108  which has a data link to the image conditioning device  102  and to the image processing device  104 . Depending on the configuration of the invention, the data storage device  108  can also only be connected to one of the two devices  102 ,  104 . Particular partial object information, which is determined and/or identified by the image processing device  104 , can now be assigned to each of the cells  22 ,  222 . Object heights, extent, position or speed identified e.g. in the camera image can be assigned to the cells  22 ,  222 . The example of object height is to be observed here. It can happen that only a part of an object  23 ,  27  is located in a cell and, therefore, only corresponding partial object information for this part of the object  23 ,  27 —in this case height information—is assigned. The partial object information—for example assigned as object profiles—is combined to produce individual objects  23 ,  27 . This grid information which is assigned to the individual cells  22 ,  222 , together with the object heights and the identified objects  23 ,  27 , is transmitted back to the image conditioning device  102 . 
         [0064]    In addition to the extent and position of objects  23 ,  27  (i.e. of potential obstacles), the direction of movement  26  thereof (in the case of dynamic objects  23 ) and movement speed, as well as the type (person, bicycle, car, other object and/or obstacle) and probability of collision for an extrapolated movement can also be meaningfully accommodated in the display. Historical object information  24  and/or predicted object information  25  as well as past and predicted trajectories of the driver&#39;s own car can, for example, be assigned to the relevant objects  23  as plumes and can also be correspondingly controlled by means of the image processing device  104  and image conditioning device  102  and displayed in a display apparatus  105  (display). Likewise, the non-existence of obstacles in certain areas can also be visualized (e.g. colored highlighting of the entire drivable surface in the image). Camera panning or enlargements as critically identified points is/are also conceivable. 
         [0065]    The object information and/or the grid model  21 ,  221  can be used in the image conditioning device  102 , in order to optimize the distortion correction of raw image data prior to producing the top view data.  FIG. 4  shows such a top view representation (or bird&#39;s eye view) of a vehicle  20 ,  220  diagrammatically, which offers the driver a perspective as if he were observing the vehicle from above. To this end, the image is generally made up of four camera images (mounted at the front and rear and on the right and left of the vehicle  20 ,  220 , as shown for example in  FIG. 6 ). Usual top view systems project the recorded 2D images into a 2D plane or onto a 2D spherical surface, in order to give the user the impression as if he were looking onto the vehicle and the surroundings from above. The different images to be shown (whether these are all-round view, lateral view or reversing view) are produced according to the following principle, the steps of which are shown diagrammatically in  FIG. 5 : an image—i.e. raw image data—is/are produced in accordance with  FIG. 5  a) on the basis of sensor data from a camera, here a fish eye camera. The imaging properties of the fish eye camera result in a distorted image. A fish eye correction produces the result as shown in  FIG. 5  b). A correction of the perspective with the result as shown in  FIG. 5  c) is carried out as an additional step. In this way, raw image data from a plurality of sensors  1 ,  101 ,  111 ,  121 ,  131 , in particular a plurality of cameras, are combined to produce a top view image as shown in  FIG. 4 . 
         [0066]    The fish eye cameras, which operate in such a system and provide the so-called fish eye images, bring with them an important advantage: the large viewing angle. With such an angle (horizontally &gt;180° view, vertically &gt;130° view) per camera, the entire vehicle environment can be observed all around with only four cameras. This leads to cost savings. The disadvantage in this case is that a fish eye image cannot be used without processing and/or distortion correction. The fish eye images must therefore be normalized to produce a flat image, the view is then corrected for the desired image section (by means of bilinear imaging or using a look-up table). These steps are shown in  FIG. 5 . A plurality of sections can thereby be composed from the input videos for an output image. These steps and/or the methods described here can be carried out using so-called FPGAs (Field Programmable Gate Arrays) or other DSPs (digital signal processors). 
         [0067]    One problem in this case is that with standard top view systems, vertical objects which are located further away from the vehicle are shown considerably distorted (lengthened). This invention proposes an improvement to this. 
         [0068]      FIG. 6  shows a vehicle  220  with four sensors  101 ,  111 ,  121 ,  131  in the form of cameras, in particular fish eye cameras diagrammatically. These produce corresponding raw image data, i.e. images  301 ,  311 ,  321 ,  331 , which are, however, optically distorted (fish eye images), so that straight lines appear curved and dimensions, in particular lengths, are distorted. 
         [0069]      FIG. 7  now diagrammatically shows a grid model which is configured as a 3D grid model  221  and wherein the cells  222  of the grid model comprise a size and form and/or orientation which is dependent on the position of the cell in the grid. The 3D grid model  221  of the surroundings of the vehicle  220  constitutes a model of a curved surface, wherein the radius of curvature (it could also be referred to as the gradient) increases as the distance R increases from the vehicle  220 , comparable to a type of “key”. Raw image data of each of the images  301 ,  311 ,  321 ,  331  can then be projected onto the 3D grid model  221  using a projection algorithm, in order to achieve an improved distortion correction of the raw image data of each of the four sensors  101 ,  111 ,  121 ,  131 . This produces a more realistic view of the surroundings of the vehicle  220 , which can also be observed from various perspectives.  FIG. 8  shows such a corrected view and/or display of corrected image data of a corrected image  351  of the surroundings of the vehicle  220 , which is based on the raw image data of one of the four sensors  101 ,  111 ,  121 ,  131  and in which the lines are straighter, dimensions such as in particular length conditions are shown more realistically, and objects such as e.g. other vehicles  223  can be better and more realistically identified. In particular, length distortions are avoided, in addition objects located further away are shown to be shorter and are therefore represented more realistically. Thanks to the use of a curved projection surface, the size of the objects shown corresponds better to their actual size than in the case of a projection onto a flat surface. This is explained more precisely with reference to  FIGS. 9 and 10 : 
         [0070]    The length distortion Ipe/lv during the projection of a stationary object o onto a plane E is shown in  FIG. 9 . C is the camera position, v represents the virtual viewpoint, from the perspective of which the vehicle surroundings are to be shown to the driver. The distortion is thereby all the greater the further away the object is from the vehicle, because the virtual viewpoint is substantially higher than the camera position. 
         [0071]      FIG. 10  shows that this represented length of a distant stationary object  0  during the projection onto a curved surface GO as shown, by way of example, in  FIG. 7  (“key”) corresponds proportionally much better to the length of the actual object. Since the distortion of the object in the plane increases with the distance from the vehicle, the form of the curved surface is relatively flat at a short distance from the vehicle, the radius of curvature and/or the gradient of the curved surface increases further away from the vehicle. 
         [0072]    If a 3D display element is provided, the grid information and/or cell information or the object information (position, size, extent) can in particular be helpfully used for a 3D visualization, since obstacles or other objects can be shown much more vividly to the observer in the 3D representation and thus provide the driver with a significantly improved impression of his environment. The grid model and/or the curved surface can be optimized to the extent that the surroundings look like a live 3D model. A model of the vehicle surroundings consisting e.g. of 10*10 cm frames with object height information can be used and/or produced in this case. Using this information, the projection surface can thus be modeled such that the display almost looks like a genuine 3D model of the surroundings. 
       REFERENCE NUMERALS 
       [0000]    
       
           1  Sensor 
           2  Image conditioning device 
           3  Data link 
           4  Image processing device 
           5  Display apparatus 
           9  Data link 
           10  Driver assistance system 
           13  Vehicle dynamics 
           20  Vehicle 
           21  Grid model 
           22  Cell 
           23  Object 
           24  Historical object information 
           25  Predicted object information 
           26  Direction of movement 
           27  Object 
           100  Driver assistance system 
           101  Sensor 
           102  Image conditioning device 
           103  Data link 
           104  Image processing device 
           105  Display apparatus 
           106  Data link 
           107  Data storage device 
           108  Data storage device 
           109  Data link 
           111  Sensor 
           112  Image data conditioning area 
           121  Sensor 
           122  Image data conditioning area 
           131  Sensor 
           220  Vehicle 
           221  3D grid model 
           222  Cell 
           223  Vehicle 
           301  Image 
           311  Image 
           321  Image 
           331  Image 
           351  Corrected image 
         C Camera position 
         E Plane 
         GO Curved surface 
         o Object 
         R Distance 
         v Virtual viewpoint