Patent Description:
An improved method for aligning a camera of a vehicle camera monitor system is provided, wherein the camera is mounted to a vehicle on one side thereof and has a camera opening angle, the camera comprises at least one image sensor having an entire area of full sensor field of view (FoV) comprising at least a first FoV legally-prescribed for a rear view device of the vehicle in form of class II FoV in a first area, the full sensor FoV is defined by the camera opening angle and the alignment of the camera relative to the vehicle, the class II FoV extending from an edge of said side of the vehicle, away from the vehicle, and the image data recorded by the at least one image sensor are forwarded to a monitor of the CMS to display an image comprising the first area to a driver of the vehicle.

In common current camera monitor systems (CMS), at least one camera is oriented so that the camera field of view (FoV) has class II FoV according to the ECE Guideline R <NUM> substantially in the center of the optical axis of the camera, as shown in <FIG> for a truck <NUM> driving along a straight road <NUM>. The truck <NUM> has a cab <NUM> and a trailer <NUM>, with a camera <NUM> of a CMS mounted at the driver side on the left side of the cab <NUM> and a monitor (not shown) of the CMS being provided as substitute for a rear view mirror, as head-up display or the like. The known camera orientation has the advantage that class II FoV, defining a first area <NUM> to be displayed on the monitor, is recorded with optimum sharpness when driving straight ahead. When the truck <NUM> is cornering and for class IV FoV, defining a second area <NUM> to be displayed on the monitor, however, the sharpness decreases because a large area <NUM> of the recorded section cannot be used.

<FIG> also illustrates that the camera opening angle <NUM> defining an area <NUM> of the full sensor FoV of the camera <NUM>, which covers two edges areas <NUM>, <NUM> which do not meet sharpness requirements due to lens effects. Between the two edge areas <NUM>, <NUM> there is said wasted area <NUM> which is not used in order to have a maximized sharpness in the area <NUM> for class II FoV, partly overlapped by the area <NUM> for class IV FoV. Both the areas <NUM> and <NUM> start along the side of the truck <NUM>, rather along the side of the trailer <NUM> of the truck <NUM>, but with the first area <NUM> starting with an offset relative to the second area <NUM> in line with the legal requirements of the ECE Guideline R <NUM>, to which with respect to further details on the different FoV classes reference is made.

<CIT> refers to a vehicle, in particular commercial vehicle, which is equipped with a viewing system, wherein the viewing system comprises an image capture unit attached to the vehicle, a computing unit connected to the image capture unit, and a reproducing unit connected to the computing unit. The image capture unit comprises a lens, which has an optical axis, and a digital image sensing unit. The image capture unit is attached to the vehicle such that, in use, a viewing area on the side of the vehicle is sensed with at least a part of a first legally-prescribed field of view and at least a part of a second legally-prescribed field of view, wherein the first legally-prescribed field of view is configured such that the driver can see at least a <NUM> wide, flat, horizontal portion of the road, which is bounded by a plane parallel to the median longitudinal vertical plane and passing through the outermost point of the vehicle on the driver's side and the passenger's side of the vehicle, respectively, and which extends from <NUM> behind the driver's eye to the horizon, and such that the driver can see the road over a width of <NUM>, which is bounded by a plane parallel to the median longitudinal vertical plane and passing through the outermost point of the vehicle starting from a point <NUM> behind the vertical plane passing through the driver's eye. The lens is disposed with respect to the digital image sensing unit such that the optical axis extends through the part of the first legally-prescribed field of view that is reproduced on the digital image sensing unit, wherein the optical axis is shifted towards the vehicle such that the optical axis extends through the part of the first legally-prescribed field of view.

Another vision system for a vehicle according to <CIT> includes at least one imaging sensor disposed at the vehicle and having an exterior field of view. The imaging sensor is operable to capture image data. The imaging sensor includes or is associated with an inclination sensor. At least one other inclination sensor is disposed at the vehicle. A processing system is operable to process outputs of the inclination sensors to determine an alignment or misalignment of the at least one imaging sensor at the vehicle.

A method for calibrating and/or aligning a camera mounted in an automobile vehicle is known from <CIT>, where the camera comprises an image sensor and calculation means that can calculate the value of a parameter in a region of interest in the image.

Systems and methods are described in <CIT> for creating a unified output image based on image data from multiple cameras with overlapping fields of view by converting a raw image from each camera into a rectified output image using a look-up table. Camera misalignments are mitigated by generating an updated look-up table based on feature point detection and matching in the overlapping fields of view.

<CIT> relates to a method for displaying an indirect field of view in the vehicle environment, in particular for displaying a driver's and / or passenger's rear and field of vision of a vehicle, in particular a commercial vehicle, by means of a camera monitor system. An overall field of view is recorded with at least one camera, in particular by means of a digital camera, of the camera monitor system and displayed on the monitor such that the overall field of view shown on the monitor is divided into several adjoining partial fields of view, which are each shown with different associated magnification factors and / or degrees of distortion.

<CIT> provides a calibration process that uses multiple parallel lines with marks or tick marks for multi-camera image stitching calibration. The calibration system may calibrate the camera and system while the vehicle is moving along a vehicle assembly line. Special targets trigger image capturing while the vehicle is moving. Optionally, the calibration system may utilize user actuatable inputs to provide a manual calibration process that a user can perform while viewing displayed images derived from image data captured by the vehicle cameras. The captured image data includes areas where there are overlapping fields of view of the cameras, with one or more targets or markings disposed at the overlapping regions to facilitate calibration of one or more of the cameras.

<CIT> teaches a camera system for a vehicle, comprising a capturing unit including an optical element and an image sensor having an image sensor surface and adapted to capture a section of a vehicle environment, wherein the optical element has a distortion with a distortion curve r=f(α), wherein r is the distance from an object point depicted on the image sensor surface to the intersection point of the optical axis with the image sensor surface, and α is the angle between the optical axis of the optical element and the beam incident in the optical element from the object point, the distortion curve r=f(α) for rw=f (αw) has a turning point (αw; rw) within <NUM><r <rmax, for which r"=f'" (αw)=d<<NUM>>r/dα<<NUM>>(αw)=<NUM> applies, wherein rmax is the distance r=f(αmax) on the image sensor surface from the optical axis to the most distant boundary point of the image sensor surface, and for the curvature of the distortion curve r"=f"(α)<<NUM> for <NUM>°<α<αw, r"=f'"(α)><NUM> for αw<α<αmax applies.

<CIT> refers to a view system for a vehicle with a tractor and a trailer, comprising: a capture unit with at least one image sensor for capturing an area of view around the vehicle in form of image data, wherein the capture unit is attachable to the tractor, at least one processing unit for processing the image data captured by the capture unit, and at least one reproduction unit for reproducing at least one first image section and one second image section of the area of view captured by the capture unit, wherein, if the tractor and the trailer are substantially aligned in the longitudinal direction of the vehicle, the processing unit takes the first image section at a first original position (A) and the second image section at a second original position (B) from the area of view of the at least one image sensor, and wherein, if the tractor and the trailer are arranged in an angle substantially unequal <NUM> ° to each other, the processing unit takes the first image section at a first modified position (A') and/or the second image section at a second modified position from the area of view of the at least one image sensor, and wherein the resolution of the first image section at the original position (A) and at the modified position (A') on the reproduction unit is higher than the resolution of the second image section at the original position (B) and at the modified position on the reproduction unit.

<CIT> describes a vision system for a vehicle for displaying a recording area outside the vehicle, containing a recording unit attached to the vehicle with at least one recording device for capturing an image of the recording area, wherein the recording area has at least two partial recording areas, a display unit for the vehicle interior with a display area with predetermined dimensions, and a calculation unit which processes the image captured by the capturing unit and feeds it to the display unit for display, the calculation unit being adapted to process the image of the capturing area captured by the capturing unit at least in to scale an image angle direction with a scaling factor (n), the display unit being adapted to display the scaled image in the display area, the scaled Image contains at least two partial images which correspond to the at least two partial recording areas, the scaling factor (n) depending on an angle of view (ϕ) of the captured image of the recording area is determined according to a mapping function (f(ϕ)), and wherein the mapping function (f(ϕ)) is adapted according to a driving situation and has at least two different areas in the direction of the image angle, which correspond to the at least two partial recording areas, and the course of the function of the imaging function (f(φ)) is erratic in the transition area between the partial recording areas.

<CIT> teaches a vision system for a utility vehicle for displaying legally prescribed fields of vision of a main mirror and a wide-angle mirror on the same side of the vehicle in a driver's cab of the utility vehicle, containing a display unit in the driver's cab ;a detection unit; and a computing unit processing images captured by the capturing unit and supplying them to the display unit for display, wherein the capturing unit is adapted to include at least one first image, which includes the field of view of the main mirror, and a second image, which includes at least that part of the field of view of the wide-angle mirror that is not part of the field of view of the main mirror, wherein at least the second image is in at least one wide-angle direction that is substantially is perpendicular to the image viewing direction and horizontal, has a wide-angle field of view which is larger than the natural impression of the human eye and is therefore distorted, and the first image is substantially undistorted in the wide-angle direction of the second image, and wherein the calculation unit is adapted, the first image and the second image in such a way to form a common image directly adjoining one another seamlessly composed in that the first image and the second image have a common perspective and at least the second image is modified at least in a modification area in the wide-angle direction, that the image stretch in the wide-angle direction is reduced, and wherein the representations of the first image and the second image perpendicular to the wide-angle direction in the longitudinal direction of the commercial vehicle.

There is the need for a simple adjustment of the FoV of a single camera for class II and class IV, while ensuring a high sharpness and FoV exploitation.

It is the object of the present disclosure to further develop the known method to overcome the drawbacks of the prior art.

The object of the present disclosure is achieved by the features of the characterizing portion of claim <NUM>. Thus, the method comprises aligning the camera with its camera opening angle such that the first area is arranged at the horizontal outermost position of the image captured by the image sensor of the camera and at the vertical outermost position of the image displayed on the monitor, wherein the entire area of full sensor FoV comprises two edge areas not meeting the sharpness requirements for the image on the monitor of the rear view device, and the first area is arranged directly next to a first edge area, and panning of the first area to a panned first area depending on the cornering situation of the vehicle with a cab and a trailer, wherein the camera is mounted to the cab and the class II FoV extends from the edge of the trailer of said side of the vehicle, and the panning is following the bending of the trailer relative to the cab of at least <NUM>°, while passing curves. Further, the first area is arranged in the upper right region of the monitor simulating a rear view mirror, and the panned first area is arranged in the upper left region of the monitor simulating a rear view mirror.

With the disclosure it is also proposed that the entire area spans an angle of at least <NUM>°, preferably around <NUM>°, and/or the first edge area and the second edge area each span an angle of up to <NUM>°.

Embodiments of the present disclosure can still further comprise aligning the class IV FoV extending from the edge of said side of the vehicle, away from the vehicle, together with aligning the class II FoV such that a second area of the class IV class FoV is displayed on the monitor.

It is also proposed that the method of the present disclosure comprises providing a class V FoV within a third area extending from the edge of said side of the vehicle, away from the vehicle, between the second area and the second edge area.

According to the present disclosure it is proposed that the camera is associated with at least one lens, with the lens preferably having an uniform distortion.

Arranging at least the first area, preferably the first area and the panned first area, the second area and/or the third area, in an image region having a distortion reduced by the lens of the CMS is also proposed with the present disclosure.

In addition, a vehicle in form of a truck with a cab and a trailer and with a CMS having a single camera mounted to one side of the vehicle and being aligned according to the present disclosure is provided.

It is also proposed that the single camera is at a driver side and/or the vehicle has a single camera at each side thereof.

Thus, the application provides a solution for a CMS with a single camera that can record at least class II FoV and a lens with uniform distortion, to display an image on a monitor of the CMS with the legally required areas. The recorded images can be processed as for example described in <CIT> to simulate a plan and/or curved rear view mirror.

In short, according to the present application the orientation of a camera is selected such that the camera opening angle is aligned in such a way that the legally required image section, i.e. class II of FoV, is on the horizontal outermost position of the image captured by the image sensor of the camera and/or at the vertical outermost position of the image displayed on the monitor.

This selection is only limited in case the edge areas of the image sensor of the camera cannot meet the sharpness requirements due to lens effects. Therefore, depending on the lens, the distance of the image to the image sensor may vary. Another limitation can be the physical edge of the image sensor when having a curvature.

The foregoing summary, as well as the following detailed description, will be better understood when read in conjunction with the appended drawings. For the purpose of illustration, certain examples of the present description are shown in the drawings. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an implementation of systems consistent with the present description and, together with the description, serve to explain advantages and principles consistent with the invention.

Further features, details and advantages of the invention are explained in the appended claims, in the drawings and in the description of a preferred embodiment of the head section according to the invention given below.

<FIG> shows a situation similar to the one already discussed with respect to <FIG> with a truck <NUM> driving on a road <NUM>, with its trailer <NUM> arranged straight behind its cab <NUM>. But, one camera <NUM> with a uniform distortion lens (not shown) of a CMS of the truck <NUM> according to an embodiment of the present disclosure is oriented such that the camera opening angle <NUM> is aligned in such a way that a first area <NUM> providing a legally required image section of a rear view device simulating a rear view mirror of the truck <NUM>, i.e. class II of FoV, is arranged on the horizontal outermost position of the image sensor of the camera <NUM>, just next to a first edge area <NUM> of the FoV, that does not meet the sharpness requirements, substantially due to lens effects. This orientation allows to have a sharp image not only in the required class II area <NUM>, but also in a second, class IV area <NUM>.

<FIG> depicts a monitor <NUM> of the CMS replacing an external rear view mirror attached to the side of the truck <NUM>. The displayed image of the area <NUM> of the full sensor FoV, comprising the first area <NUM> and the second area <NUM> with an improved overall impression as stitched images, allows the driver of the truck <NUM> to see more details. The first area <NUM> is shown in the upper right region on the monitor <NUM>.

The camera orientation, described with respect to <FIG> and <FIG>, can achieve the highest possible degree of panning of the first area <NUM>, with a curved trailer <NUM> as shown in <FIG>, depicting a panned first area <NUM> of class II FoV. with a camera opening angle of <NUM>° and assuming that an angle region of <NUM>° of the edge area <NUM> cannot be used due to blurring or an inaccuracy of the image section, the end of the trailer <NUM> can still be captured at bends of more than <NUM>° due to panning the first area <NUM> to the panned first area <NUM>. Said panning leads to a shift along the arrows A in <FIG> between the upper corner regions of the monitor <NUM>. Thus, the camera orientation achieved with the method of this disclosure allows a very high margin for the panning when cornering forward and backward to allow the end of the trailer <NUM> still to remain in the panned first area <NUM>, in contrast to the situation with the original first area <NUM> associated with a straight trailer <NUM> as shown in <FIG>.

Thus, the alignment described above allows to correct the class II FoV when cornering by panning along the direction A to obtain all required information on the monitor <NUM> as illustrated in <FIG>.

The alignment, in addition to enabling the panning of class II FoV, also allows for observing a third area <NUM> of class V FoV as shown in <FIG> and further illustrated for the monitor image in <FIG>. This is due to not wasting a FoV area as indicated with region <NUM> for prior art camera alignment in <FIG>. The respectively gained area is between the second area <NUM> for the class IV FoV and the second edge area <NUM>, which is releasing sufficient area for allowing a recording of the third are <NUM>.

Claim 1:
A method for aligning a camera of a vehicle camera monitor system (CMS), wherein
the camera (<NUM>) is mounted to on one side of a vehicle (<NUM>) with a cab (<NUM>) and a trailer (<NUM>), wherein the camera is mounted to the cab and has a camera opening angle (<NUM>),
the camera (<NUM>) comprises at least one image sensor having an entire area (<NUM>), the entire area (<NUM>) covering a full sensor field of view (FoV) comprising at least a first FoV legally-prescribed for a rear view device of the vehicle in form of class II FoV in a first area (<NUM>),
the full sensor FoV is defined by the camera opening angle (<NUM>) and the alignment of the camera (<NUM>) relative to the vehicle (<NUM>),
the class II FoV extending from an edge of said side of the vehicle (<NUM>), away from the vehicle, and
the image data recorded by the at least one image sensor are forwarded to a monitor (<NUM>) of the CMS to display an image comprising the first area (<NUM>) to a driver of the vehicle, characterized by
• aligning the camera (<NUM>) with its camera opening angle (<NUM>) such that the first area (<NUM>) is arranged at the horizontal outermost position of the image captured by the image sensor of the camera (<NUM>) and at the vertical outermost position of the image displayed on the monitor (<NUM>), wherein the entire area (<NUM>) of full sensor FoV comprises two edge areas (<NUM>, <NUM>) not meeting the sharpness requirements for the image on the monitor (<NUM>) of the rear view device, and the first area (<NUM>) is arranged directly next to a first edge area (<NUM>),
• aligning the class IV FoV extending from the edge of said side of the vehicle (<NUM>), away from the vehicle, together with aligning the class II FoV such that a second area (<NUM>) of the class IV FoV is displayed on the monitor (<NUM>), and
• panning of the first area (<NUM>) to a panned first area (<NUM>) depending on the cornering situation of the vehicle (<NUM>), wherein
o the first area (<NUM>) is arranged in the upper right region of the monitor (<NUM>) simulating a rear view mirror, and
o the panned first area (<NUM>) is arranged in the upper left region of the monitor (<NUM>) simulating a rear view mirror.