Patent Description:
This camera monitoring system is more particularly intended to provide the user with a view of the environment of a vehicle driven by the user.

Conventionally, automotive vehicles are equipped with wing mirrors to provide the driver of the automotive vehicle with a view of the sides of the vehicle. These wing mirrors are usually divided into two portions providing different fields of vision. Generally, the portion of the wing mirror which is the closest to the driver consist of a main exterior rear-view mirror, while the portion of the wing mirror farther from the driver consists of a wide angle exterior mirror.

These mirrors have different horizontal average magnification factors, said horizontal average magnification factors consisting of the relation of the angular size of an object as it is perceived by the driver on the mirror upon the angular size of the object as it would be seen by an observer placed at the position of the wing mirror; the main exterior rear-view mirror has the highest horizontal average magnification factor, while the wide angle exterior mirror has a lower horizontal average magnification factor.

Several standards define more precisely the requirements that the main exterior rear-view mirrors and the wide angle exterior mirrors must meet. In Europe, this definition is provided by UNECE Regulation No. <NUM> on "Uniform provisions concerning the approval of devices for indirect vision and of motor vehicles with regard to the installation of these devices", in which the main exterior rear-view mirrors are defined as class II mirrors, and the wide angle exterior mirrors are defined as class IV mirrors.

The use of mirrors has however been unsatisfactory and we now meet a trend of replacing conventional wing mirrors with camera monitoring systems comprising cameras for capturing views of the sides of the vehicle and screens for displaying the views captured by the cameras. These camera monitoring systems are intended to provide at least the functionalities provided by the conventional mirrors and, thus, to provide a view corresponding to the view provided by a main exterior rear-view mirror and a view corresponding to the view provided by a wide angle exterior mirror.

The most common solution consists of providing the camera monitoring systems with a first screen closest to the driver providing a first view similar to the view provided by a main exterior rear-view mirror, and a second screen farther from the driver while adjoining the first screen, said second screen providing a second view similar to the view provided by a wide angle exterior mirror.

However, such a solution is unsatisfactory when it equips a trailer truck. Indeed, it is common that, when the truck is turning, the end of the trailer will leave the image provided by the first screen and enter the image provided by the second screen, where the driver will have wrong estimation of distances because of the lowest horizontal average magnification factor. And while, with conventional mirrors, this problem could be circumvented by the driver craning his head, this is no longer possible with the screens of the camera monitoring systems. Document <CIT> discloses a camera monitoring system as well as a method for providing to a user a view of a scene captured by a camera, where the image is a continuous combined image of the area behind the vehicle using a middle image piece which is corrected (free of distortion) aligned with first and second side image pieces which are not corrected (distorted).

Document <CIT> discloses a vision system for a commercial vehicle with a trailer, as well as a method for providing to a user a view of a scene captured by a camera, displaying a recording area located outside the vehicle. The system adjusts the scaling factor of two parts of the recording area according to the position of the trailer and the camera angle.

One aim of the invention is thus to allow provision to the driver of a trailer truck of a view of the sides of his truck that will help him in evaluating distances in the critical areas on which he will have to draw his attention. Other aims of the invention are to provide this view in a mannerthat will be ergonomic to the driver and to limit the calculations necessary to provide this view.

To that end, the invention relates to a camera monitoring system comprising:.

According to specific embodiments of the invention, the camera monitoring system also has one or more of the following features, considered alone or according to any technically possible combination(s):.

The invention also concerns an automotive vehicle comprising the camera monitoring system defined above, each of the first wide angle camera and the second wide angle camera being oriented toward the rear of the vehicle, the display comprising a display surface on which the displayed image is presented, said display surface being positioned in the field of view of the driver.

According to a specific embodiment of the invention, the automotive vehicle also has the following feature:.

The invention also concerns a method for providing to a user a view of a scene captured by a camera, the method comprising:.

Other features and advantages of the invention will appear upon reading the following description, provided solely as an example and done in reference to the appended drawings, in which:.

The automotive vehicle <NUM> of <FIG> consists of a trailer truck. It comprises, in a known manner, a tractor <NUM> and a trailer <NUM> towed by the tractor <NUM>. The tractor <NUM> houses a cabin <NUM> (<FIG>) intended for the driver of the truck <NUM>.

In the following, the orientation terms are intended in reference to the usual orthogonal orientation reference frame of automotive vehicles, shown in <FIG>, and which comprises:.

The driver of the truck <NUM>, while sitting in the cabin <NUM>, is usually looking toward the road in front of the truck <NUM>, where the truck <NUM> is heading. However, for some manoeuvers, the driver needs to have a view of the sides of the truck <NUM>, while keeping an eye on the road in front of the truck <NUM>. Since these sides are out of the field of view of the driver, a device is needed to provide the required view to the driver.

To that end, the truck <NUM> comprises a camera monitoring system <NUM>, shown in <FIG>. This camera monitoring system <NUM> is preferably exclusively on board of the tractor <NUM>.

This camera monitoring system <NUM> comprises two cameras 22A, 22B, each able to capture an image of a respective scene, and, for each camera 22A, 22B, an image treatment system 24A, 24B and a display 26A, 26B. The camera monitoring system <NUM> further comprises a determination unit <NUM> for determining, for each camera 22A, 22B, an area of interest in the scene imaged by the camera 22A, 22B, and a configuration unit <NUM> for configuring each display 26A, 26B.

As shown in <FIG>, a first one 22A of the cameras 22A, 22B is provided on the left side of the tractor <NUM>, while the second camera 22B is provided on the right side of the tractor <NUM>. Both cameras 22A, 22B are oriented toward the rear of the truck <NUM>, which means that the scene imaged by each camera 22A, 22B is positioned rearward of the camera 22A, 22B. In other words, the field of view 32A, 32B of each camera 22A, 22B is centered on a direction extending rearward from the camera 22A, 22B.

Each camera 22A, 22B is in particular oriented so that its field of view 32A, 32B includes a respective side 34A, 34B of the tractor <NUM>.

Each camera 22A, 22B consists in particular of a wide angle camera, which means that each camera 22A, 22B is able to capture a wide angle image of the scene imaged by the camera. In other words, the angular extent of the field of view 32A, 32B of the camera 22A, 22B is wide, i.e. is wider than <NUM> degrees.

More specifically, it is the horizontal angular extent of the field of view 32A, 32B of the camera 22A, 22B which is wide. It is not necessary for the vertical angular extent of the field of view of any of the cameras 22A, 22B to be wide. Preferably, the vertical angular extent of the field of view of both cameras 22A, 22B is comprised between <NUM> and <NUM> degrees.

As a result thereof, the images provided by the cameras 22A, 22B are distorted, as can be seen on <FIG> and <FIG>, where examples <NUM>, <NUM> of images captured by the right camera 22B are provided. By "distorted", it is meant that the magnification factor of these images changes across the field of view at a fixed working distance. In other words, the relation of a dimension of an object as it is shown in the image <NUM>, <NUM> upon the angular size of the object as it is seen by the camera 22A, 22B changes depending on the location of the object in the image <NUM>, <NUM>. As can be seen on these images <NUM>, <NUM>, as a result of this distortion, right lines, such as the edges <NUM>, <NUM> of the trailer <NUM>, appear curved on the images <NUM>, <NUM>.

Each camera 22A, 22B is configured for refreshing the image of its respective scene at a predetermined frequency, said frequency being preferably comprised between <NUM> and <NUM> frame per second, preferably between <NUM> and <NUM> frame per second.

The displays 26A, 26B comprise a first display 26A associated with the first camera 22A and a second display 26B associated with the second camera 22B. Each display 26A, 26B is configured for displaying, in the field of view of the driver of the truck <NUM>, a displayed image of the image captured by its associated camera 22A, 22B. Examples <NUM>, <NUM> of such a display image, corresponding respectively to the wide angle images <NUM> and <NUM>, are provided on <FIG> and <FIG>.

To that end, each display, respectively 26A, 26B, comprises, with reference to <FIG>, a display surface, respectively 36A, 36B, on which the displayed image is presented, said display surface 36A, 36B being positioned in the field of view of the driver of the truck <NUM>. Each display 26A, 26B further comprises a display management unit (not shown) for displaying the displayed image on said surface 36A, 36B.

In the shown example, the display surface 36A, 36B of each display 26A, 26B consists of a screen.

The display surface 36A, 36B of each display 26A, 26B is provided on the same side of the cabin <NUM> as the associated camera 22A, 22B. Thus, the display surface 36A of the first display 26A is provided on left side of the cabin <NUM>, while the display surface 36B of the second display 26B is provided on right side of the cabin <NUM>.

The display surface 36A, 36B which is the farthest from the steering wheel <NUM> is bigger than the other display surface 36A, 36B. In the shown example, the steering wheel <NUM> is provided on the left side of the cabin <NUM>; the biggest display surface is therefore display surface 36B.

Each display surface 36A, 36B is delimited by a frame <NUM> including an inner border <NUM> which is the closest to a longitudinal median plane M (<FIG>) of the tractor <NUM>, and an outer border <NUM> which is the farthest from said longitudinal median plane M. The inner and outer borders <NUM>, <NUM> are thus spaced transversally from each other. Advantageously, the inner and outer borders <NUM>, <NUM> are both oriented substantially vertically.

The configuration unit <NUM> is configured for setting a first preset limit <NUM> and a second preset limit <NUM> to the wide angle image <NUM>, <NUM>, so that the displayed image <NUM>, <NUM> only presents the part of the wide angle image <NUM>, <NUM> comprised between said preset limits <NUM>, <NUM>. These preset limits <NUM>, <NUM> are spaced apart from each other along a horizontal direction.

As mentioned above, the determination unit <NUM> is configured for determining, for each camera 22A, 22B, an area of interest in the scene imaged by the camera 22A, 22B. To that end, the determination unit <NUM> comprises typically a gaze tracking unit (not shown) for tracking a gaze direction of the driver and a main calculator (not shown) configured for determining, depending on the gaze direction, the position of an area of interest in the scene. For instance, the main calculator is configured to determine that an area of interest exists in the scene imaged by a camera 22A, 22B when the gaze direction is oriented toward the display surface 36A, 36B of the display 26A, 26B associated to said camera 22A, 22B. The main calculator is further configured to determine that the area of interest is in an angular sector of the field of view of the camera 22A, 22B which is increasingly far from the median plane M as the distance of the gaze direction from the inner border <NUM> increases. In other words, the main calculator provides a bijection between the distance of the gaze direction from the inner border <NUM> and the angle of view of the area of interest, defined as the angle between the longitudinal median plane M and the direction connecting the area of interest to the camera 22A, 22B, said bijection being an increasing function.

Alternatively, the determination unit <NUM> comprises a head tracking unit (not shown) for tracking movement of a head of the driver and a main calculator (not shown) configured for determining, depending on the position of said head, the position of an area of interest in the scene. For instance, the main calculator is configured to determine that, the more the driver leans forward, the greater the angle of view of the area of interest is, said angle of view being defined as the angle between the longitudinal median plane M and the direction connecting the area of interest to the camera 22A, 22B; in other words, the main calculator provides a bijection between the distance of the distance of the head of the driver from the windshield <NUM> and the angle of view of the area of interest, said bijection being a decreasing function.

In another alternative, the determination unit <NUM> comprises a trailer tracking unit (not shown) for tracking position of the trailer <NUM> relatively to the tractor <NUM> and a main calculator (not shown) configured for determining, depending on the position of said trailer <NUM>, the position of an area of interest in the scene.

The trailer tracking unit typically comprises a sensor for measuring an angle between the trailer <NUM> and the tractor <NUM>. Alternatively, the trailer tracking unit comprises an image treatment system for identifying, in the images provided by the cameras 22A, 22B, the position of a rear end <NUM> (<FIG>) of the trailer <NUM>, and a secondary calculator configured to deduce thereof the position of the trailer <NUM> relatively to the tractor <NUM>. In another alternative, the trailer tracking unit comprises sensors for measuring a steering angle of the tractor <NUM> and a speed of the tractor <NUM>, and a secondary calculator configured to deduce thereof the position of the trailer <NUM> relatively to the tractor <NUM>. In both latter cases, the secondary calculator may be merged with the main calculator.

The main calculator is then configured for determining that the area of interest consists of the area of the scene in which the rear end <NUM> of the trailer <NUM> is.

In still another alternative, the determination unit <NUM> comprises at least two of the gaze tracking unit, head tracking unit and trailer tracking unit mentioned above. In such a case, the main calculator is configured for deducing the position of the area of interest from the several pieces of information provided by the different tracking units.

Each image treatment system 24A, 24B is configured for receiving the wide angle image captured by a respective one of the cameras 22A, 22B. Each image treatment system 24A, 24B is further configured for receiving from the determination unit <NUM> a signal indicating the location of an area of interest in the scene imaged by said respective camera 22A, 22B when the determination unit <NUM> has determined that such an area of interest is present in said scene.

Each image treatment system 24A, 24B is configured to process the wide angle image captured by the respective camera 22A, 22B so as to provide the displayed image <NUM>, <NUM>, said displayed image <NUM>, <NUM> comprising an undistorted portion <NUM> including the area of interest determined by the determination unit <NUM>, and at least one distorted portion <NUM>, <NUM>.

By "undistorted portion", it is meant that the magnification factor provided by said portion <NUM> is constant throughout the portion <NUM>. in other words, the relation of a dimension of an object as it is shown in the portion <NUM> upon the angular size of the object as it is seen by the camera 22A, 22B is the same for every object shown in the undistorted portion. As a consequence thereof, right lines, such as the edges <NUM>, <NUM> of the trailer <NUM>, are shown right in said undistorted portion <NUM>.

The image treatment system 24A, 24B is configured to provide the displayed image <NUM>, <NUM> in such a manner that the undistorted portion <NUM> adjoins the at least one distorted portion <NUM>, <NUM>. For each distorted portion <NUM>, <NUM>, the undistorted portion <NUM> thus has an interface edge <NUM>, <NUM> in contact with said distorted portion <NUM>, <NUM>.

The image treatment system 24A, 24B is further configured to provide the displayed image <NUM>, <NUM> in such a manner that the displayed image <NUM>, <NUM> is continuous, i.e. so that there is not discontinuity in the image <NUM>, <NUM>, in particular at the interface between the undistorted portion <NUM> and the at least one distorted portion <NUM>, <NUM>.

The image treatment system 24A, 24B is configured to provide the displayed image <NUM>, <NUM> within a frame <NUM>, said frame <NUM> having the same shape and the same ratio as the frame <NUM> of the display surface 36A, 36B of the corresponding display 26A, 26B.

This frame <NUM> has an outer border <NUM>, an inner border <NUM>, an upper border <NUM> and a lower border <NUM>, the upper and lower borders <NUM>, <NUM> being spaced apart from each other along a first direction D1 while being substantially parallel to each other, the outer and inner border <NUM>, <NUM> being spaced apart from each other along a second direction D2 while being substantially parallel to each other. Preferably, as shown, the outer and inner borders <NUM>, <NUM> are substantially parallel to the first direction D1, and the upper and lower borders <NUM>, <NUM> are substantially parallel to the second direction D2. Furthermore, the first and second directions D1, D2 are advantageously, as shown, substantially orthogonal to each other.

The image treatment system 24A, 24B is configured to provide the displayed image <NUM>, <NUM> so that the inner border <NUM> consists of the first preset limit <NUM> of the wide angle image <NUM>, <NUM>, and the outer border <NUM> consists of the second preset limit <NUM> of the wide angle image <NUM>, <NUM>, whatever the position of the area of interest in the scene is.

The image treatment system 24A, 24B is configured to provide the or each distorted portion <NUM>, <NUM> interposed between the undistorted portion <NUM> and a respective one of the outer and inner borders <NUM>, <NUM>.

The image treatment system 24A, 24B is further configured so that the or each interface edge <NUM>, <NUM> of the undistorted portion <NUM> extends in the first direction D1.

The image treatment system 24A, 24B is configured to provide the undistorted portion <NUM> with a first average magnification factor along the first direction D1 and with a second average magnification factor along the second direction D2, the second average magnification factor being substantially equal to the first average magnification factor. The first magnification factor is in particular comprised between <NUM> and <NUM>.

The image treatment system 24A, 24B is further configured to provide the or each distorted portion <NUM>, <NUM> with a primary average magnification factor along the first direction D1 and with a secondary average magnification factor along the second direction D2. The primary average magnification factor is substantially equal to the first magnification factor, while the secondary average magnification factor is inferior to the second magnification factor.

The image treatment system 24A, 24B is configured to provide the undistorted portion <NUM> with a fixed width along the second direction D2, independently of the position of the area of interest in the scene. Said width is preferably more than half the width of the displayed image <NUM>, <NUM>, measured from the outer border <NUM> to the inner border <NUM>. For instance, the width of the undistorted portion <NUM> is approximately two thirds of the width of the displayed image <NUM>, <NUM>.

The image treatment system 24A, 24B is configured to center the undistorted portion <NUM> on the area of interest and to displace the undistorted portion <NUM> in the displayed image <NUM>, <NUM> depending on the position of the area of interest in the scene. Thus, if the area of interest is closer from a preset limit <NUM>, <NUM> than half the width of the undistorted portion <NUM>, as shown in <FIG>, then the undistorted portion <NUM> is in abutment against the inner or outer border <NUM>, <NUM> of the displayed image <NUM>, <NUM> and a single distorted portion <NUM> of the wide angle image <NUM> is provided between the undistorted portion <NUM> and the opposite border <NUM>, <NUM>. If the area of interest is farther from both preset limits <NUM>, <NUM> than half the width of the undistorted portion <NUM>, as shown in <FIG>, then the undistorted portion <NUM> is spaced from both the inner and outer borders <NUM>, <NUM> of the displayed image <NUM>, <NUM> and distorted portions <NUM>, <NUM> of the wide angle image <NUM> are provided on both sides of the undistorted portion <NUM> between said undistorted portion <NUM> and both inner and outer borders <NUM>, <NUM>.

The corresponding display 26A, 26B is configured to display the displayed image <NUM>, <NUM> as provided by the image treatment system 24A, 24B, the frame <NUM> of the displayed image <NUM>, <NUM> matching the frame <NUM> of the display surface 36A, 36B, the inner border <NUM> matching the inner border <NUM> and the outer border <NUM> matching the outer border <NUM>.

A method implemented by the camera monitoring system <NUM> to provide the driver with a side view of the truck will now be described.

First, the truck <NUM> is advancing straight ahead. The trailer <NUM> is then aligned with the tractor <NUM>, and the image captured by the right camera 22B is then the image <NUM> shown in <FIG>.

The determination unit <NUM> then determines that the area of interest is the angular sector in which the rear end <NUM> of the trailer <NUM> is located. This determination is made either by locating the trailer <NUM> relatively to the tractor <NUM>, or by identifying an eye or head movement of the driver showing an interest of the driver in said angular sector.

This information is provided by the determination unit <NUM> to the image treatment system 24B, which provides the displayed image <NUM> of <FIG>. This image is then displayed by the display 26B on the display surface 36B.

The truck <NUM> then turns right. As a consequence thereof, the trailer <NUM> is no longer aligned with the tractor <NUM> and is shifted toward the right side of the tractor <NUM>. The image captured by the right camera 22B is then the image <NUM> shown in <FIG>.

This information is provided by the determination unit <NUM> to the image treatment system 24B, which provides the displayed image <NUM> of <FIG>, where the undistorted portion <NUM> has been displaced relatively to the image <NUM>. This image is then displayed by the display 26B on the display surface 36B.

Claim 1:
A camera monitoring system (<NUM>) for providing to a user a view of a scene captured by a camera, the camera monitoring system (<NUM>) comprising:
- a first wide angle camera (22A) for capturing a first wide angle image (<NUM>) of a scene,
- a second wide angle camera (22B) for capturing a second wide angle image (<NUM>) of a scene,
- a determination unit (<NUM>) for determining an area of interest in the scene, the determination unit (<NUM>) comprising a gaze tracking unit for tracking movement of a gaze direction of the user,
- an image treatment system (24A, 24B) for providing an undistorted portion (<NUM>), including the area of interest, of each of the first wide angle image (<NUM>) and the second wide angle image (<NUM>), and
- a display (26A, 26B) for displaying a first continuous displayed image (<NUM>) of the first wide angle image (<NUM>) and a second continuous displayed image (<NUM>) of the second wide angle image (<NUM>), each of the first displayed image (<NUM>) and the second displayed image (<NUM>) comprising the undistorted portion (<NUM>) adjoining and continuous with at least one distorted portion (<NUM>, <NUM>), and
- wherein each of the first wide angle image and the second wide angle image (<NUM>, <NUM>) has first and second preset limits (<NUM>, <NUM>) and each of the first displayed image and the second displayed image (<NUM>, <NUM>) has first and second opposed borders (<NUM>, <NUM>), the or each distorted portion (<NUM>, <NUM>) being interposed between the undistorted portion (<NUM>) and a respective one of the first and second borders (<NUM>, <NUM>), the display (26A, 26B) being configured to display the first preset limit (<NUM>) at the first border (<NUM>) and the second preset limit (<NUM>) at the second border (<NUM>) independently of the position of the area of interest.