Patent Description:
<CIT> relates to a method for processing image data captured by a fisheye lens. Image data of a camera are transformed to a three-dimensional plane of a "virtual lens. Subsequently, the transformed image data of the virtual lens <NUM> are projected on a "second image plane". Finally, a bilinear zooming algorithm may be applied to the image data.

Modern vehicles comprise a plurality of systems for assisting a driver when driving the vehicle. For instance, modern vehicles may comprise a surround view system. Such a surround view system may utilize a plurality of cameras for capturing image data of the environment of a vehicle, and a display for providing the captured image data to the driver. An important function of a surround view system is a <NUM> degree view, created from the image data of a camera of the surround view system. For instance, the <NUM> degree view may be used to detect or to show crossing traffic.

For an early detection of crossing the traffic by the driver, the horizontal outer regions of a view may be zoomed in. However, conventional approaches for zooming particular regions of an image data may distort the two-dimensional camera image. This gives poor control of the actual zooming. Further, conventional zooming of particular regions in image data usually require a huge computational effort.

Hence, there is a need for an enhanced zooming of image data. In particular, there is a need for an enhanced zooming of image data which can be realized by small computational resources.

For this purpose, the present invention provides an image zooming method according to claim <NUM>. Further, the present invention provides an image zooming apparatus according to claim <NUM>. Further, a surround view system of a vehicle is provided, comprising an image zooming apparatus according to the present invention and a display. The display is adapted to display the image data rendered by the rendering unit.

An idea underlying the present invention is to transform image data captured by a camera to a three-dimensional surface and assign a 3d-coordinates to the positions on the surface that correspond to the pixels. Accordingly a further processing of the pixels having 3d-coordiantes, e.g. a zooming, can be performed in a simple and easy manner. In particular, precomputed functions can be applied to the pixels for zooming the image data.

Since the zooming can be realized by applying precomputed functions on the 3D-coordinates that correspond to the individual pixels, the zooming can be realized by very simple and cheap computational resources.

According to a further embodiment, the step for projecting the provided image data comprises applying a lens undistortion to the provided image data. In this way, distortions of the lens of camera can be compensated when projecting the image data on the three-dimensional surface.

According to a further aspect, the step of applying a predefined zoom function comprises shifting x-coordinates and/or y-coordinates of the transformed image data. By shifting x-coordinates and/or y-coordinates of the transformed image data, a very fast and efficient zooming can be achieved.

According to a particular embodiment, the x-coordinates and/or the y-coordinates of the transformed image data are shifted based on a continuous function.

According to a further aspect, the coordinates of the transformed image are shifted based on a sigmoidal function. In this way, a smooth transition between regions of different zoom factors in the image data can be achieved.

According to a further embodiment, the predefined zoom function comprises a prestored assignment of pixels of the transformed image data to coordinates in a grid of the zoomed image data. In this way, the zooming can be achieved by referring to precomputed data. Hence, it is not necessary to compute the zoom function for each zooming separately. Accordingly, a very efficient zooming can be achieved by small computational resources.

According to a further embodiment of the zooming apparatus, the zooming unit comprises a memory. The memory is adapted to store a predetermined assignment of transformed image data in the planar grid to coordinates of a grid relating to the zoomed image data.

According to a further embodiment, the camera comprises a wide-angle lens camera. In particular, the camera may comprise a lens for a view angle of approximately <NUM> degrees.

According to a further aspect, the image zooming apparatus comprises a graphic processing unit (GPU). By applying graphic processing unit, a very efficient and fast rendering of the image data can be achieved.

The above identified aspects and embodiments may be combined with each other where possible. Further embodiments and implementations of the present invention may also comprise combinations of embodiments and features of the present invention.

The present invention will be described in the following by embodiments based on the accompanying drawings, wherein:.

<FIG> shows a schematic illustration of an image zooming apparatus <NUM> according to an embodiment. The image zooming apparatus <NUM> comprises a camera <NUM>, a projecting unit <NUM>, a transforming unit <NUM>, a zooming unit <NUM> and a rendering unit <NUM>. Camera <NUM> may be any kind of camera for providing image data. For instance, camera <NUM> may be a camera of a surround view system, in particular of a surround view system of a vehicle. Camera <NUM> may also be a camera system comprising a plurality of imaging units. In particular, camera <NUM> may be a camera capturing image data in a field of view having a wide opening angle. For example, camera <NUM> may capture image data in a segment of an opening angle of approximately <NUM> degrees. However, camera <NUM> is not limited to cameras having an opening angle of approximately <NUM> degrees. Camera <NUM> may also have an opening angle of approximately <NUM> degrees, <NUM> degrees, or any other opening angle. The image data captured by camera <NUM> are provided to projecting unit <NUM>.

Projecting unit <NUM> receives the image data provided by camera <NUM> and processes the received image data to apply a projection of the received image data on a predetermined three-dimensional surface. The geometry of the predetermined three-dimensional surface may relate to the projecting properties of camera <NUM>, in particular to the projecting properties of the lens used in camera <NUM>. For example, the predetermined three-dimensional surface may be a curved surface corresponding to the projecting properties of camera <NUM>. The three-dimensional surface used by projecting unit <NUM> may be, for example, a semi-cylindrical surface or surface of a polygon. Usually, the shape of the three-dimensional surface is pre-stored in projecting unit <NUM>.

Projecting unit <NUM> may also compensate disturbances in the image data provided by camera <NUM>. For example, projecting unit <NUM> may compensate lens distortions of camera <NUM>. Any other distortions may be also compensated during the projecting of the image data on the three-dimensional surface. Alternatively, distortions, such as lens distortions, may be already compensated in camera <NUM> before providing the camera images to projecting unit <NUM>.

After the image data provided by camera <NUM> are projected on the predetermined three-dimensional surface, the projected image data are provided to transform a unit <NUM>. Transforming unit <NUM> receives the projected image data and transforms the image data relating to the three-dimensional surface to a planar grid. In particular, the image data are transformed to a regular, planar grid. In this way, the transformed image data may be provided as two-dimensional image data having a well-known, standardized grid. Such image data provide a very good basis for a further processing.

Next, the image data transformed to the planar grid are provided to zooming unit <NUM>. Zooming unit <NUM> receives the transformed image data and applies a predefined zoom function to the transformed image data. In particular, an arbitrary zoom function can be applied to the transformed image data. Since the transformed image data are provided in a well-known, fixed grid, the zoom function which is applied to the transformed image data can be precomputed in advance. For example, for each pixel position of the transformed image data a shift of the x-coordinate and/or the y-coordinate can be determined in advance. This precomputed shift of the coordinates can be stored in a memory of zoom unit <NUM>. In this way, the shift of the x-coordinate and/or the y-coordinate for each pixel of the transformed image data can be read out from the memory and applied to the transformed image data.

In this, way, an arbitrary zooming of the transformed image data can be realized. In particular, it is possible to precompute zooming functions, which zoom particular areas in the transformed image data, while other image areas are reduced. For example, image areas on the left and the right border may be zoomed. However, any other scheme for zooming particular areas of the image may be possible, too. Further, it may be also possible to reduce (de-zoom) an image area.

The zooming function which is applied in the zooming unit <NUM> may be, for example, a continuous function. In this way, a very smooth zooming of the desired areas in the transformed image data can be achieved. For instance, the zooming function which is applied to the transformed image data may be a sigmoidal function. Such a sigmoidal function provides a "S" shape. The sigmoidal function may be defined, for example by s (x) = <NUM>/(<NUM>+e-x). However, any other function, in particular any other continuous function may be used for zooming particular areas in the transformed image data.

In order to provide a flexible zooming of the image data, it may be also possible to precompute and store a plurality of different zooming functions. The plurality of different zooming functions may be stored in a memory of zooming unit <NUM>. Accordingly, one of the zooming functions may be selected and applied to the transformed image data. The selection of the appropriate zooming function may be performed, for instance, automatically. For example, if the zooming apparatus is applied to a surround view system of a vehicle, a different zooming function may be selected depending on a speed of the vehicle. Alternatively, object detection may be applied to the image data. Accordingly, an area may be zoomed, in which a particular object is detected. For example, one or more predetermined objects may be selected. If at least one of the selected predetermined objects is detected in the image data, the corresponding image area is zoomed based on an appropriate pre-stored zooming function. Alternatively, it may be possible to compute the necessary zooming function before applying the zooming.

After the transformed image data are zoomed based on the predetermined zoom function, the zoomed image data are provided to rendering unit <NUM>. Rendering unit <NUM> renders the zoomed image data and provides the rendered image data to a display <NUM>. Finally, the rendered image data are provided to a user on a display <NUM>. The rendering of the image data can be performed, for instance, by a graphic processing unit (GPU). Such a graphic processing unit is a specialized processing unit for performing a graphical processing.

<FIG> shows a schematic illustration of an image zooming according to an embodiment.

As can be seen in this figure, image data <NUM> are provided by a camera <NUM>. As already explained in connection with <FIG>, the image data provided by camera <NUM> are projected on a three-dimensional surface <NUM>. Preferably, the shape of the three-dimensional surface <NUM> is based on the projecting properties of camera <NUM> capturing the image data <NUM>. For example, the shape of the three-dimensional surface <NUM> may be adapted based on the properties of a lens of camera <NUM>. After the image data are projected on the three-dimensional surface <NUM>, the projected image data are transformed to a planar grid <NUM>. In particular, planar grid <NUM> may be a regular planar grid <NUM>. Accordingly, the image data are provided in a format having a well-defined grid. In order to perform a zooming of particular areas/regions in the image data, a zooming function can be applied to the image data in the regular grid to obtain zoomed image data <NUM>. In case of a horizontal or vertical zooming, it may be desired to keep the outer borders of the image at a constant coordinate, i.e. the zoomed image shall have a same width or a same height. Finally, the zoomed image data are rendered to obtain the texture of the zoomed image data which can be displayed on a screen <NUM>.

<FIG> shows a flowchart underlying a method for zooming image data according to an embodiment.

In step S1 image data which are captured by a camera <NUM> are provided. The captured image data are projected in step S2 on a predetermined three-dimensional surface.

In step S3, the projected image data are transformed to a planar grid to obtain transformed image data. Next, in step S4, a predefined zoom function is applied to the transformed image data to obtain zoomed image data. In particular, it may be possible to select a particular zoom function out of a plurality of precomputed zoom functions which is applied to the transformed image data. Finally, in step S5, the zoomed image data are rendered to a predetermined screen area.

Claim 1:
An image zooming method, comprising the steps:
providing (S1) image data captured by a camera (<NUM>);
projecting (S2) the provided image data on a predetermined three-dimensional surface, wherein the predetermined three-dimensional surface comprises a semi-cylindrical surface or surface of a polygon;
transforming (S3) the projected image data to a regular and planar standardized grid to obtain transformed image data;
applying (S4) a predefined zoom function to the transformed image data to obtain zoomed image data, wherein the predefined zoom function is precomputed in advance and keeps the outer borders of the image at a constant coordinate, such that the zoomed image has a same width or a same height; and
rendering (S5) the zoomed image data to a predetermined screen area.