Patent Description:
The present invention relates to a camera monitoring system (CMS) to manage the field of view from one or more rear-view mirrors displayed to the driver of a vehicle by the driver's gesture detection.

A problem of the prior art in the field of motor vehicles is adjusting the field of view while driving.

<CIT> discloses a method and apparatus for adjusting the view mirror in vehicle, by (i) obtaining a first angle of the view mirror; (ii) capturing an image of a head of a driver; (iii) determining the perpendicular distance of the view mirror (iv) calculating a view angle based on the viewing distance and the perpendicular distance; and (v) adjusting the view mirror from the first angle to the second angle. <CIT> solves the technical problem of adjusting the field of view when driving by the determining the position of the driver. However, the adjustment is made by using an actuator to rotate the side-mirror. A mechanical actuator means cost, complexity and likelihood of damage over time.

<CIT>D1 discloses a vehicle control system comprising: a driver monitoring system including a head position determiner to determine at least one of a location of a head, an orientation of the head, or an eye gaze point of the head; a digital mirror system including a region-of-interest (ROI) detector to identify an ROI based on the at least one of the location of the head, the orientation of the head, or the eye gaze point of the head, and a cropper to extract a portion of a first image corresponding to the ROI to form a second image, the first image representing an area exterior to the vehicle; and a display within an interior area of the vehicle to present the second image.

It is also found that drivers may wish to move their head forward, particularly when overtaking, in order to reduce the blind spot zone. In doing so, to have an enlarged view of the blind spot zone, drivers increase the relative angle of their head with respect to the image capturing means (e.g., a reflective rear-view mirror in conventional mirror systems of vehicles, or a camera in digital vision systems for vehicles). However, in conventional reflective mirrors, the aperture angle (α) unfortunately decreases with increasing the relative angle of the driver's head with respect to the image capturing means. The aperture angle (α) may be defined as the angular extent (from a top view) of a given scene (captured image region) which is captured by the image means. The aperture angle (α) is translated into a horizontal length of the captured image region from the driver's point of view (on a mirror or on a display).

Therefore, it is highly desirable to provide a motor vehicle with a control system monitoring the intelligent rear-view mirror system(s) to change the field of view of the rear-view mirror(s) with nor mechanical nor manual actuation and without touching any display. More particularly, it is an object of the present invention to enhance the driver's visibility of the exterior scenes by providing a visual system with at least an exterior rear-view mirror that allows the exterior field of view to be adjusted by moving the driver's head. In order to cause a better view for the driver and particularly to minimize the blind spot zone when overtaking, an improved field of view of the lateral adjacent area of the driver's vehicle is needed.

The present invention solves the aforementioned problems and overcomes previously explained state-of-art work limitations by providing a camera monitoring system (CMS) for motor vehicles which provides at least a display device configured to display images captured from an electronic (exterior and/or side) rear-view mirror of the motor vehicle into an electronic display. Specifically, the proposed CMS is capable of adjusting the field of view (FOV) of the rear-view mirror based on the position of at least one part of the user's body (e.g., the driver's head) without touching the display. Head/body movements (or a change in head/body position) may be detected by a sensor which may be based on different technologies (ultrasound, image, TOF: Time of Flight, radar, etc.). The electronic display may refer to a screen (preferably, a touchless screen) or to a display glass.

The present invention has a number of advantages with respect to prior art, which can be summarized as follows:.

These and other advantages will be apparent in the light of the detailed description of the invention.

For the purpose of aiding the understanding of the characteristics of the invention, according to a preferred practical embodiment thereof and in order to complement this description, the following Figures are attached as an integral part thereof, having an illustrative and non-limiting character:.

Of course, the embodiments of the invention can be implemented in a variety of architectural platforms, operating and server systems, devices, systems, or applications. Any particular architectural layout or implementation presented herein is provided for purposes of illustration and comprehension only and is not intended to limit aspects of the invention.

According to a preferred embodiment, the CMS comprises image processing means comprising a camera pointing substantially rearward and located on a side part (right and/or left) of the motor vehicle (e.g., a car). The obtained captured image comprises at least a lateral / side portion of the car and the zone behind the car. The ECU (Electronic Control Unit) is configured to receive the captured image and carries out an image processing comprising at least selecting a portion of the captured image and send it to a display ubicated inside the cabin of the car. The image capturing means can be fixedly mounted outside the vehicle.

According to a preferred embodiment, the CMS further comprises a gesture detector to allow the driver to command the CMS to adjust the FOV by a movement of the driver's head or other part of the body (e.g., an eye movement or facial gestures). In a possible embodiment of the CMS using the gesture detector, the crop-and-pan functionality may be triggered by a driver's head movement. The driver's head movement may be tracked by a surveillance system, which is preferably within the vehicle, more preferably fixed (i) in front of the driver, or (ii) in the interior rear-view mirror, (iii) or near to the CMS display device. Preferably, the driver surveillance system may comprise a camera. The surveillance system may be capable of working out the position and distance of the driver's head. Thus, the displayed image region within the captured image may be controlled by the driver's head, moving forward when this movement is captured by the gesture detector. There are four possible implementation options for the gesture detector:.

In another preferred embodiment, the display is within a head mounted device (i.e., a near-eye device for a driver), preferably a pair of glasses with Augmented Reality, AR. AR glasses already exist, for example, Google glasses and Microsoft HoloLens. Said head mounted device (e.g., an interactive pair of AR glasses) includes i) an optical assembly configured to display virtual content and to enable viewing of at least a portion of a surrounding environment, ii) an integrated processor for processing the virtual content to be displayed, and iii) an integrated image source for introducing the virtual content to the optical assembly. Furthermore, the head mounted device may include a communications facility configured to connect the interactive head-mounted device to an external device such as the ECU of the camera monitoring system (CMS). The head mounted device comprises a gaze position detection unit for detecting the driver's gaze. The proposed CMS using the head mounted device (e.g., with AR glasses as a driver's display, ubicated inside the vehicle when the driver put it on) is capable of generating, based on the position of the driver's head and/or his/her gaze direction, an instruction to move the crop ("digital pan") within the captured image. The captured image includes the symmetric image of the raw captured image. The head movement electronically detected by the head mounted device is consistent with a direction of the driver's eyes/gaze, and so the communications facility is configured to provide the ECU with a video feed consistent with the direction of the head movement and the eye gaze. The video feed sent from the ECU to the head mounted display through the communications facility is an enriched video stream comprising data of at least one of: (i) the displayed image region (crop); and (ii) blind spot detection. The displayed image region (crop) is displayed only when the detected driver's eyes are looking at one specific point on the optical assembly. For example, the display of the AR glasses is off unless the driver is looking to a point where the rearview mirror should be; in this case, the ECU generates a signal to turn on the display of the AR glasses screen only when the "gaze detector" captures that the driver is looking at said particular point.

The adjustment of the FOV involves the symmetry of the raw captured image, which is the original image captured by the image capturing means, to obtain the symmetric image. The symmetric image is provided directly by the image capturing means, but it can alternatively be provided by the ECU (more particularly, an ISP) applying image processing. In particular, when the symmetric image is provided directly by the image capturing means, it is obtained through the hardware of the image sensor which advantageously reduces considerably the amount of operations of image processing (software). The adjustment of the FOV also implies (i) moving the crop within said symmetric image; or (ii) expanding the additional extended view within said symmetric image.

<FIG> shows the relationship between the movement (<NUM>) of the driver's head (<NUM>) detected by a sensor (gesture detector) and the movement (<NUM>) or "pan" of the displayed image region (<NUM>) or "crop" within the captured image (<NUM>), according to a possible example. In this example, the movement (<NUM>) of the driver's head (<NUM>) along the driving direction is translated into a displacement / movement (<NUM>) of the displayed image region (<NUM>) processed (by the ECU) in the horizontal direction. Therefore, the change in the Field of View, FOV, seen by the driver (<NUM>) is performed indirectly, by changing the location of the crop, which results in a new FOV seen by the driver (<NUM>).

In a possible embodiment, the ECU performs image processing to:.

The symmetry of the raw image is performed with respect to its vertical axis. No other type of image processing, such as emulating a conventional rear-view mirror, for example, is required. However, in the case that the arm of the mounting means (i.e. winglet or sharkfin) in the exterior side of the vehicle (<NUM>) is short or inexistence may be advantageous to perform operations of image processing for changing the perspective (i.e. perspective correction), for example, by using a homogprahy matrix. It is important to note that applying the homography matrix on: i) the raw image (<NUM>); ii) the symmetrical image (<NUM>); or iii) the image region (<NUM>) is definitely not the same or equivalent to emulate a conventional reflective rear-view mirror. It is not the same or equivalent because the conventional reflective rear-view mirror decreases the aperture angle (α) of the reflected image as increases the relative angle between the driver's head and the display device which causes an undesired reduction of the view of the blind spot zone.

Preferably, when the used image capturing means are located at an exterior side of the vehicle (<NUM>), the captured image (<NUM>) has a FOV in a range of <NUM>° - <NUM>°, while the image region (<NUM>) has a FOV between <NUM>% and <NUM>% the length of the captured image (<NUM>). The ratio shape / geometry of the captured image (<NUM>) keeps the same for the crop or image region (<NUM>). If an <NUM>% FOV is in use when cropped, FOV of the crop is <NUM>° (calculated as <NUM>% of <NUM>°); whereas if a <NUM>% FOV is in use when cropped, FOV of the crop is <NUM>° (calculated as <NUM>% of <NUM>°). When the used image capturing means are associated with an interior IRMS, the FOV of the captured image of is <NUM>°, according to another non-limiting example.

<FIG> shows the relationship between another movement (<NUM>) of the driver's head (<NUM>) and the movement (<NUM>) of the displayed image region (<NUM>), according to another possible example. In this example, the movement (<NUM>) of the displayed image region (<NUM>) is in the vertical direction and downwards when the movement (<NUM>) of the driver's head (<NUM>) is upwards.

The crop or displayed image region (<NUM>) is displaced according to a pre-established relationship. The pre-established relationship may be linear or non-linear (see <FIG>).

Table <NUM> below and <FIG> illustrate horizontal movements of the crop based on the movement of the driver's head forward along the driving direction and on the length of the crop.

If the horizontal crop movement depending on crop length, as in the examples of Table <NUM> above, is <NUM>% when the head moves forward <NUM> , if the camera captures <NUM>°, and the crop is <NUM>% of the camera, the crop has an FOV of <NUM>°, i.e. it has <NUM>° left for horizontal translation.

<FIG> shows the raw image (<NUM>) captured by the image capturing means located on a winglet in the exterior side of the vehicle (<NUM>) corresponding to the driver's side; in this case, the raw image (<NUM>) captured by the image capturing means (e.g., an exterior camera) located on the left side of the vehicle (<NUM>). This raw image (<NUM>) including a partial image of the vehicle (<NUM>) on the left is what is seen by the image capturing means located on the side of the driver (<NUM>). The exterior square corresponds to the image border of the total FOV captured by said image capturing means, for example, a FOV of <NUM> degrees.

From the raw image (<NUM>) shown in <FIG>, the captured image (<NUM>) shown in <FIG> is obtained as the symmetry of the raw image (<NUM>) with respect to its vertical axis. Hence, the partial image of the vehicle (<NUM>) is seen on the right side of the captured image (<NUM>). According to this example, only the symmetry is the image processing required to obtain the captured image (<NUM>), but there is no emulation of a conventional rear-view mirror or other image processing. The interior square corresponds to the crop or image region (<NUM>) selected to be displayed; i.e., the border of the effective FOV shown to the driver (<NUM>) on his/her display, for example an effective FOV of <NUM> degrees. In <FIG> and <FIG>, the effective FOV shows the highway with the horizon included.

<FIG> shows the crop or image region (<NUM>) displayed when the ECU has moved the crop horizontally <NUM>% (of the total crop movement) within the captured image (<NUM>). According to Table <NUM> and <FIG>, this corresponds to the case when the driver has moved his/her head <NUM> millimetres forward, i.e. in the driving direction of the vehicle (<NUM>).

<FIG> shows the crop or image region (<NUM>) displayed when the ECU has moved the crop horizontally <NUM>%, which would reach the limit of the captured image. According to Table <NUM> and <FIG>, this corresponds to the case when the driver has moved his/her head forward <NUM> millimetres, i.e. in the driving direction of the vehicle (<NUM>). For example, in the case that the image capturing means are located at a mounting assembly of the left exterior rear-view mirror, the crop or image region (<NUM>) is preferably at the right side of the captured image when the driver's head is in the usual driving position, while the image region (<NUM>) moves to the left, as shown in <FIG>, when the head moves forward. That is, the image region (<NUM>) is initially not centered, i.e., the image region (<NUM>) is off-center on the captured image, but is usually located at an inner, right or left, side: e.g., the right side for left-hand-drive vehicles in right-hand traffic countries. Therefore, the driver is provided with a path on the display to move the crop along the horizontal axis of the captured image which is much longer. For example, going to <FIG>, in particular to the left picture, the displayed image region or crop (<NUM>, <NUM>') is not centered so as to leave a space (i.e., path) on its left within the captured image (<NUM>) that is not displayed. The length (L2) of said space is at least <NUM>% of the length (L1) of the image region (<NUM>, <NUM>') before starting to be moved within the capture image (<NUM>). Furthermore, the crop (<NUM>, <NUM>') may leave a smaller space on its right with a length (L3), being L2 ≥ L3 as illustrated in <FIG>. The skilled person recognizes that the size of the raw image (<NUM>) captured by the image capturing means (i.e., camera) may be a limitation. The aperture angle (α) of the image region (<NUM>) may depend on the lens and the image sensor (i.e., imager) of the camera. A wider aperture angle (α) of the image region (<NUM>) may be achieved by a lens including a curved surface or increasing the size of the image sensor. A lens including a curved surface may cause distorted images. It may be needed image processing to correct distorted images. Thus, it is particularly advantageous the crop (image region) is not centered on the captured image (<NUM>) so as to maximize the above-mentioned available path.

The same crop movement can be perfomed for the driver as well as the co-driver occupying the front passenger seat of the vehicle, for which another camera of the CMS is provided. In the case of the conventional rear-view mirrors, rear-view mirrors with different positions and / or sizes are used. However, in the case of CMS, the same values for adjusting the FOV can be used maintaining the FOV symmetry for both sides of the vehicle.

The relationship between the movements of the crop and the driver's head may be linear, but there may be other alternatives, for example, for different driving scenarios such as: i) a dead zone where the CMS FOV remains fixed or with very small variations because the driver is driving right and alone in a highway, and ii) another zone where the crop displacement is required to be more significant because the driver is merging onto a highway or changing lanes.

Therefore, the linear movement of the crop may be vertical, horizontal, and ultimately the crop may be moved diagonally. Also, the crop may optionally be zoom in / out. Zoom out is increasing the size of the crop (displayed image), whereas zoom in is decreasing the size of the crop (displayed image). Furthermore, if the gesture detector is a camera, it is possible to detect the driver's gaze, i.e. at which point the driver is looking. Therefore, according to a further possible example, the ECU can carry out the digital pan when there is a head movement and also if the driver is looking at the display. Thus, according to this example, if there is a head movement but the driver is not looking at the display, then the crop is not moved. So, the displayed image is not always moving each time the driver moves his/her head, only when the detected gesture complies with a threshold or certain criteria.

To summarise the preceding paragraph, the driver's head could be detected on three axes, x, y and z, with the driving axis, denoted as x, being essential and the other two axes being optional / additional. The ECU can work both with the locations of the head and with a relative angle of the head with respect to the driver's display (alpha angle, if seen from the top view). If the driver moves his/her head forward, he/she is increasing the alpha angle and is displacing the crop to the left in the display, in a straight direction according to the horizontal axis, x.

The ECU can work and move the crop based on at least the variation of the position (defined by x, y, z coordinates) of the head or on at least the alpha (top view) angle, beta (lateral view) angle, gama angle of the head. The ECU can perform the digital pan (move the crop) based on the following data obtained by the gesture detector (measured by a sensor or extracted from an image of the head captured by a camera):.

When the driver is in front of the rear-view mirror (without any relative angle), the image reflected by the rear-view mirror and the image shown on the display coincide, i.e. both images are exactly the same. However, these images are neither the same and nor equivalent if there is a relative angle between the driver's head and the rear-view mirror (<NUM>) or there is a head movement, as shown in <FIG>. As the head moves forward, the driver's eyes go from a first position (E51) to a second position (E52): the angle of the driver's view (a1, a2) with respect to the reflective rear-view mirror (<NUM>) corresponding to the driver's eye first position (E51) is different from the angle of the driver's view (b1, b2) corresponding to the second position (E52). The case where there is a greater difference between both images is when the head is moved forward at the same level as the mirror (<NUM>). The captured image is simply the symmetrical image of the -raw- image covering the exterior FOV, i.e., captured by the camera associated with the rear-view mirror, but a conventional rear-view mirror is not emulated by said captured image. Therefore, computationally speaking, it is advantageously less tedious. That is, the reflected image from reflective mirror is neither the same as nor technically equivalent to the displayed image obtained by image processing (symmetry) plus cropping+panning performed by the proposed CMS.

<FIG> shows the comparison between the optical lines reflected in a conventional reflective rear-view mirror (<NUM>) and the field of view captured by a camera arranged on the exterior side of a motor vehicle. As the driver's head moves forward from the first position (E51) to the second position (E52), the reflected optical lines (i.e., what the driver sees through the mirror) become increasingly parallel. Therefore, the second angle (b1, b2) corresponding to the second position (E52)) is greater thanthe first angle (a1, a2) corresponding to the first position (E51). Note that if the angle increases, this means that the lines tend to be parallel, and when the optical lines are parallel, the image is being distorted. When the driver's head is very far ahead (almost at the same level as the mirror), the displayed image (i.e., image region (<NUM>)) of the present invention is very different from what the driver can see in the conventional rear-view mirror (<NUM>). When the driver's moves his/her head forward approaching it towards the mirror (<NUM>), i.e., moving the head to the left in <FIG>, the conventional reflective mirror (<NUM>) is actually zooming in due to the fact that the head is located in a closer position to the conventional reflective mirror (<NUM>). In addition, the conventional reflective mirror <NUM> is changing the field of view shown/reflected when the driver's moves his/her head, while the field of view captured by the exterior side camera which is displayed by the display device is always a constant FOV, according to this particular embodiment in which the image region (<NUM>) is formed by a single crop.

Another further -third- position of the driver's eyes is added in <FIG>, so that the variation of the optical lines as the driver moves his/her head can be shown. The depicted square (<NUM>) in the mirror (<NUM>) represents the lens of the camera of the CMS, and the depicted triangle (<NUM>) represents the FOV of said CMS camera associated with the rear-view mirror (<NUM>), i.e., equivalent to the captured image. <FIG> shows the comparison between the image of captured by the exterior side camera, associated with the rear-view mirror (<NUM>), and the optical lines of a conventional reflective rearview mirror. In this example of <FIG>, it can clearly be observed that the respective optical lines corresponding to first position (E61), second position (E62) and third position (E63) become gradually more inclined, so when the driver's eye moves closer to the mirror (<NUM>), the relative angle makes the images observed by the reflective mirror(<NUM>) be not consistent with the true magnitude (and therefore the difference between the CMS image -merely a symmetry- and the reflected image becomes increasingly more accentuated). It is very important to note that both images (the reflective image and the displayed image) are not the same, and the difference is accentuated more as the driver's head position goes more forward (i.e., as the relative angle between camera / mirror and the driver's head varies).

<FIG> shows a triangle (<NUM>) which represents the crop within the captured image, the captured image represented by the other triangle being either the raw image or its symmetrical image. <FIG> shows the difference between the image perceived by the driver using the reflective image of the conventional mirror and the image displayed by the driver's display device. The aperture angle (α) of the displayed FOV does not change when the position of the driver's head change. More particularly, the value of aperture angle (α) remains fixed when the relative angle between CMS and driver's head changes. On the contrary, the displayed FOV or crop (i.e., the image region (<NUM>)) does change when the position of the driver's head changes (its relative angle alpha changes), as the image region (<NUM>) is moved within the captured image. As the driver's head moves forward, the aperture angle (α) of the image region (<NUM>) keeps fixed because said angle (α) is independent of the detected head/body position, but the image region (<NUM>) represented in <FIG> by the triangle (<NUM>), is moved at least horizontally within the captured image. The captured image is a mere symmetry of the raw image captured by the image capturing means of the CMS. If there were a wall behind the vehicle (<NUM>), the camera could not capture any image behind the wall and so the triangle (<NUM>) would get truncated. The, aperture, angle (α) of the triangle (<NUM>) is the same for the first position (E61) as for the second position (E62) and the third position (E63).

That is, the image region (<NUM>) to be displayed comprises a fixed aperture angle (α) of FOV.

<FIG> shows a comparison between the aperture angle (a1', a2') of the conventional reflective rear-view mirror (<NUM>) -upper quadrants A of <FIG>- and the aperture angle (C1, C2) of a preferred embodiment of the invention -bottom quadrants B of <FIG>. Also, <FIG> illustrates the relative angle (X1, X2) between the obtained position of the part of the driver's body (e.g., the driver's head) and:.

As shown in <FIG>, a first position (E11) of the driver's eyes and their second position (E12) when the head moves forward. The angle of the driver's view (a1') with respect to the reflective rear-view mirror (<NUM>) corresponding to this first position (E11), represented at the left side of <FIG>, is greater than the angle of the driver's view (a2') corresponding to the second position (E12), represented at the left side of <FIG>; i.e., a1' > a2', as explained before in <FIG>. <FIG> also shows the optical axis (E1, E2) as the bisector of the triangle that represents the FOV for the first position (E11) and the second position (E12) respectively; the FOV being directly shown to the driver in the reflective rear-view mirror (<NUM>) in the case of a conventional system represented in the upper part (A) of <FIG>, the FOV being displayed, as a crop, according to a preferred embodiment of the invention represented in the lower part (B) of <FIG>. When the driver's eyes go to the first position (E11) to the second position (E12), in the example of <FIG>:.

Threfore, the aperture of the crop remains constant, C1 = C2, with respect to the variation of the angle X1 to X2 due to the change in position of the driver's eyes from E11 to E12. Note that the relative angle between the driver's eyes and the conventional side exterior rear-view mirror (<NUM>) is not the relative angle between the driver's eyes and the camera located on the exterior side and associated with the exterior rear-view mirror. For example, in cars (usually not in trucks) the reflective mirror is replaced by an exterior camera which does not have to be located exactly where it is in the mirror (for example, it could be a little lower and not visible to the driver). In the case of trucks, it is very likely that the rearview mirror is not replaced by an exterior camera, both the mirror and the camera co-exist and work in complementary way.

Furthermore, there is another variable factor: the distance between the driver's eyes and the rear-view mirror. In the second position (E12), the driver's head is closer to the rear-view mirror, so there is a "zoom in", the "zoom in" reducing the aperture angle (C1, C2) of the single crop. Therefore, the angle (C1, C2) of the displayed FOV is constant for the variation of the angle X1, X2, but changes when "zooming in/out".

Furthermore, an interior camera of the vehicle capturing images of the driver and functioning as a gesture detector can sense, not only the position/movement of his/her head but also detect the eyes. Then, the ECU can optionally establish a midpoint between the two eyes from the images provided by said interior camera (the gesture detector): the detection of the head can be established as the midpoint between the eyes. Another option is to calculate the outline of the head. The camera can capture different parts of the face (ears, mouth / lips, nose, etc.).

<FIG> shows another, second, embodiment where the displayed image region (<NUM>) comprises two image regions or crops selected by the ECU: a first crop (<NUM>') and a second crop which corresponds to an additional extended view (<NUM>). The first crop (<NUM>') is selected by the electronic control unit, ECU, from the captured image (<NUM>) and encompasses a portion of the exterior part of the vehicle (<NUM>). The additional extended view (<NUM>) is a second crop selected by the ECU from the captured image (<NUM>), where said additional extended view (<NUM>) is located near the first crop (<NUM>'). As shown in <FIG>, it may be particularly located next to the first crop (<NUM>'), i.e., adjacent to the first crop (<NUM>'), and may preferably have the same height as the first crop (<NUM>'). The gesture detector is configured to obtain at least one position of at least one part of the driver's body (e.g., the driver's head), and the ECU is configured to adjust the displayed image region (<NUM>) of the exterior FOV based on the at least one obtained position, by expanding in length the additional extended view (<NUM>) when the detected relative angle (X1, X2) between the driver's head and the camera monitoring system - i) camera for ERMS or ii) display for IRMS- is increased. This is, when the gesture detecture detects that the relative angle (X1, X2) increases (i.e., when the driver moves his/her head forward before overtaking), then the ECU is configured to lengthen (i.e., to expand) horizontally outwardly the additional extended view (<NUM>). Further, when the gesture detecture detects that the relative angle (X1, X2) decreases, then the ECU is configured to shorten (i.e., contract) horizontally inwardly the additional extended view (<NUM>) back to the original situation. It is preferred that the ECU is configured to expand or retract progressively as the relative angle (X1, X2) increases or decreases respectively. It is important to note that the shown example increases the aperture angle (α) of the image region (<NUM>) as increases the relative angle of the driver's head with respect to the CMS. Therefore, the additional extended view (<NUM>) causes an improved field of view of the exterior lateral adjacent area of the driver's vehicle because it reduces the blind spot zone when overtaking. It is important to note that, according to this example, the driver will always see at least a portion of the exterior side of the vehicle at all times even when the head moves forward. This is, the first crop (<NUM>') remains fixed (i.e., unchanged) regardless of the driver's head movement.

Therefore, comparing the first embodiment of the invention, wherein there is only one single crop selected by the ECU, and the second embodiment of the invention, wherein there are two crops selected by the ECU:.

Preferably, the length of the additional extended view (<NUM>) is increased, at least, when the driver moves his/her head forward. The additional extended view (<NUM>) progressively increases / decreases its horizontal length as a result of an increase / decrease of the relative angle between the driver's head and the display device; while the first crop (<NUM>') remains unchanged regadless of the driver's head movement to ensure an exterior lateral portion of the vehicle is permanently displayed. The first crop (<NUM>') once displayed is smaller than the display device. Thus, the display device is large enough to display both crops: (<NUM>') and the additional extended view (<NUM>).

According to another example, said additional extended view (<NUM>) may be switched off if head movements are not detected (i.e., during the normal driving condition, the driver is not overtaking). As the driver's head moves forward, the relative angle between the driver's head and the display device is increased, so when the ECU detects that it passes a threshold, the ECU is configured to generate the extended view (<NUM>) without modifying the first crop (<NUM>'). According to a possible embodiment, the first crop (<NUM>') is not displayed on the entire screen size. Due to the fact that the first crop (<NUM>') is smaller than the actual size of the display device, there is space available to show, as depicted in <FIG>, and subsequently to expand the additional extended view (<NUM>). According to this example, the first crop (<NUM>') is always displayed while driving, and is fixed (i.e., constant over time) both in location within the capture image (<NUM>) and in size and shape. On the other hand, the additional extended view (<NUM>) may not be fixed since it may increase its length. This is, the aperture angle (α) of the whole, displayed, image region (<NUM>) is increased based on the detected increase in the relative angle (X1, X2).

<FIG> also shows the relationship between the movement of the driver's head detected by a sensor (gesture detector) and the expansion of the displayed additional extended view (<NUM>) within the captured image <NUM>, according to a possible example. In this example, the movement of the driver's head along the driving direction is translated into a growth of the displayed additional extended view (<NUM>) processed (by the ECU) in the horizontal direction. Therefore, the change (i.e., adjustment) in the Field of View, FOV, seen by the driver is performed indirectly, by lengthening of the second crop (<NUM>), which results in a new FOV seen by the driver.

The digital expansion of the displayed additional extended view <NUM> may be according to a pre-established relationship. The pre-established relationship may be linear or non-linear (see <FIG>).

For example, the first crop (<NUM>') is displayed permanently while the vehicle is running. On the other hand, the ECU is configured to generate the instruction to at least horizontally enlarged the additional extended view (<NUM>) by <NUM>% of the total length of the first crop (<NUM>'). This corresponds to the case when the driver has moved his/her head <NUM> millimetres forward, i.e. in the driving direction of the vehicle (<NUM>).

According to this shown embodiment, the ECU is configured to horizontally expand the additional extended view (<NUM>) displayed to reach the limit of the captured image (<NUM>). Preferably, the first crop (<NUM>') is at the right side of the captured image when the driver's head is in the usual driving position, while the additional extended view (<NUM>) expands to the left, as shown in <FIG>, when the head moves forward. That is, the first crop (<NUM>') is initially not centered on the captured image but is usually located at the right side. Therefore, the driver is provided with a path on the display to increase the length of the additional extended view (<NUM>) along the horizontal axis of the captured image which is much longer. In particular, the first crop (<NUM>') is not centered so as to leave an available space on its left within the captured image (<NUM>) that is not displayed. The length (L2) of said space is at least <NUM>% of the length (L1) of the first crop <NUM>'. Furthermore, the length (L2) is at least twice as long as the smallest length (L3) left from the right border of the captured image (<NUM>).

The linear movement of the additional extended view may be vertical, horizontal, and ultimately said second crop may be moved diagonally (not shown). Optionally, the sensor may be configured to detect the, at least one part, of the driver's body in the three-dimensional space, and wherein the ECU may configured to select an additional extended view (<NUM>) located up to the first image region (<NUM>'), wherein the additional extended view (<NUM>) progressively extends its heigh vertically downwards within the captured image (<NUM>) according to the determined body movement in a vertical direction, and wherein the first image region (<NUM>') remains fixed (i.e., unchanged) (not shown).

Also, the first and the second crops (<NUM>', <NUM>) forming the displayed image region (<NUM>) may optionally be zoom in / out. Zoom out is increasing the size of any of the two crops (<NUM>' <NUM>), whereas zoom in is decreasing the size of any of said crops (<NUM>', <NUM>). Furthermore, if the gesture detector is a camera, it is possible to detect the driver's gaze, i.e. at which point the driver is looking. Therefore, according to a further possible example, the ECU can carry out the digital expansion of the additional extended view (<NUM>) when a head movement is detected and also if the driver is looking at the display device. Thus, according to this example, if there is a head movement but the driver is not looking at the display, then the additional extended view (<NUM>) is not displayed. So, the additional extended view (<NUM>) is not always displayed each time the driver moves his/her head, only when the detected gesture complies with a threshold or certain criteria.

To summarise the the above-mentioned, the driver's head could be detected on three axes, x, y and z, with the driving axis, denoted as x, being essential and the other two axes being optional / additional. The ECU can work both with the locations of the head and with a relative angle of the head with respect to the driver's display. If the driver moves his/her head forward, he/she is increasing the alpha angle and is expanding the additional extended view (<NUM>) to the left in the display, at least in a straight direction according to the horizontal axis, x.

<FIG> shows from a top view the increase of the aperture angle (α1, α2) as the driver's head moves forward for overtaking another vehicle (<NUM>'). At a first position (E51) of the driver's eyes/head, the aperture angle has an initial value (α1). When the driver of the vehicle (<NUM>) starts to overtake the other vehicle (<NUM>'). the driver's head goes to a second position (E52) and the aperture angle takes another value (α2), being α2> α1. This increase of the aperture angle is translated into an increase in the horizontal length of the displayed image region (<NUM>), as shown in <FIG>, because the extendable second crop (<NUM>) is added to the first crop (<NUM>') and expanded outward horizontally.

<FIG> shows another option for the extendable second crop (<NUM>) added to the first crop (<NUM>') to form the displayed image region (<NUM>), which can be vertically expanded downwardly when the driver's head moves a few millimetres upward, e.g., for parking to see the ground and check any curb or another obstacle.

<FIG> shows the CMS is provided with an electronic display device (<NUM>) having two different portions: a first portion (<NUM>) in which the the first crop <NUM>' is displayed, and a second portion (<NUM>) in which the additional extended view (<NUM>) is displayed. According to one example, the display device (<NUM>) is one single screen, prefereably a touch screen. According to another example, the display device (<NUM>) comprises two different screens. Having two screens, the first portion (<NUM>) distinguised by the ECU may be implemented in the first screen, which can be touchless and then cheaper, and the second portion (<NUM>) may be implemented in the second screen, which can be touch-sensitive. The second portion (<NUM>) is preferably smaller than the first portion (<NUM>). One technical advantage is the cost of non-touch-sensitive screens. Further, the driver may perform a "touch & drag" operation in the second portion (<NUM>) so as to move the display image region (<NUM>). It may be advantageous for the initial calibration of the camera monitoring system, CMS, when the driver is about to start driving. The technical advantage of doing so is that the first portion (<NUM>) does not get dirty. Further, the second portion (<NUM>) may display the parameters related to brightness and/or contrast and/or color of the first portion (<NUM>). As explained, the second screen only displays the additional extended view (<NUM>) when the ECU determines the additional extended view (<NUM>) has to be displayed, whereas the first screen shows permanently the first crop (<NUM>') while the engine of the vehicle (<NUM>) is on. For example, the driver can calibrate the CMS so that a movement of the driver's thumb indicating the OK symbol can switch on the additional extended view (<NUM>). The first crop (<NUM>') displayed on the first screen is fixed (i.e., does not vary by changing the location of the driver's head), so it is ensured to display at least a portion of the exterior side of the vehicle (<NUM>) at all times when the engine is running even when the driver's head is moved forward.

As shown in <FIG>, in order to implement the aforementioned two portions (<NUM>, <NUM>) in the display device (<NUM>), the CMS further comprises a frame (<NUM>) covering at least partially the display device (<NUM>). This frame (<NUM>) has the same dimensions as the display device (<NUM>) or significantly larger to cover at least the entire display device (<NUM>). The frame (<NUM>) is a cover, preferably made of plastic or glass, which also protects the display device (<NUM>) from impacts and damages, since the touch screen may be relatively fragile. The frame (<NUM>) may be partially tinted. Preferably, the tinting is black. Said frame (<NUM>) does not allow all the light emitted by the display device (<NUM>) can pass through. Therefore, the driver does not see the light emitted by the display where the frame is tinted, i.e., silkscreen. The frame (<NUM>) is placed, for example, on top of the touch screen, and so what the user sees is the "frame", since it is placed between the screen and the user.

Alternatively, as shown in <FIG>, a third embodiment of the present invention comprises two states of operation for the display device (<NUM>): a first state corresponds when the gesture detector detects that the detected driver's head passes a threshold, e.g., when overtaking, then the second image region as an additional extended view (<NUM>) is displayed (i.e. switched on), as shown in the right picture of <FIG>, in full on the second portion (<NUM>). A second state corresponds when the second image region (<NUM>) is not displayed at all when the driver's head is detected to be back to the original position (i.e. normal driving), i.e. the second portion (<NUM>) is switched off, as shown in the left picture of <FIG>. A difference between the second embodiment and the third embodiment is that the the extended view of the second embodiment is continuously extended or retracted, whereas the third embodiment is entirely displayed (i.e., first state) or not displayed (i.e., second state). Optionally, the ECU may generate a black image for the second state. An ordinary skill person in the art will recognize that dark colours such as black have low power consumption if displayed in LED screens such as an OLED display device.

As explained, since the FOV is being changed (i.e., adjusted), actually by (i) moving the image region (<NUM>) or crop, or alternatively by (ii) expanding and contracting the additional extended view (<NUM>), as explained, based on the movement of the driver's head, another further -fourth- embodiment is related to sensitivity, i.e., the speed of changing the FOV (the speed of moving the crop or increasing the size of the additional extended view). The proposed CMS allows the user to calibrate said sensitivity; for example, if the driver moves his/her head <NUM> centimetres, the crop is moved <NUM> millimetres; but if the driver moves his/her head <NUM> centimetres, the crop is only moved <NUM> millimetres. In another example, by moving his/her head <NUM> centimetres forward, a first driver wants the displayed image to be moved only <NUM> centimetre horizontally within the captured image, whereas by moving his/her head <NUM> centimetres, a second driver may want the displayed image to be moved <NUM> centimetres horizontally within the captured image. That is, not all the head movements produce the same displacement of the crop and the system can be customized according to the driver's preferences for controlling the speed of said displacement/movement (panning). This sensitivity calibration also applies to the second embodiment, in which the aperture (α) of the image region (<NUM>) increases by expanding (extending / lengthen) the additional extended view as the detected relative angle increases.

The movement of the driver's part of the body detected by the sensor is at least in the driving direction of the vehicle, and the panning is at least linear and horizontal accordingly. Preferably, it can also be in the vertical plane (perpendicular to the ground plane), and the panning movement is linear and vertical, or a combination of horizontal and vertical, i.e. diagonal, accordingly.

The movement of the crop, based on the head movements, is performed by comparing a reference position of the driver's head with its current position. Both positions can be calculated for the driver's head or any other part of the driver's body or head, preferably the face. Therefore, the reference position is not calculated with respect to a fixed element (of the car), but with respect to a part of the driver's upper body. To control/customise the system sensitivity in moving the crop, the system is calibrated before the driver starts driving. The system is configured to save the driver's personal settings in order to detect each user and load his/her stored profile. Therefore, while driving, the movement of the displayed image change according to said sensitivity customisation (calibration). That is, by some drivers moving their head a lot (while driving), the FOV will change a few little, whereas other drivers will desire greater sensitivity, in which the FOV can change significantly by moving their head just a little. The user can calibrate (customise the sensitivity of) the camera monitoring or vision system (CMS) of any of the electronic rear-view mirrors (interior and/or any of the side mirrors).

Customisation of the sensitivity of the proposed system comprises, according to a possible example, the following steps:.

In an embodiment, the gesture detector is configured to perform the "calibration", wherein, for initial "calibration", the user/driver customises the desired sensitivity. To that end, the gesture detecting sensor detects a first position (e.g., of the head) and a second position (of the head). The driver selects / determines the FOV associated with the first position (determines the position of the crop) and a second FOV of the second position (determines the adjusted position of the crop). The ECU determines the displacement of the displayed image/crop. While driving, the sensor detects the position of the driver's head with respect to the initial reference position. Based on the comparison of the reference position (first position) with the second position, the displayed image is shifted (moved). Both positions are dynamic, they have not fixed values because the driver can change the initial reference position of his/her head. The driver associates the first position with a given FOV and the second position with another FOV.

The calibration is optional and carried out before driving: the user sits in the driver's seat and in a first position, for example, in his/her usual driving position, selects a first FOV (location of the crop) to be seen on the display. Then a relationship is established between the first position and the associated first FOV (location of the crop) which the driver wants to see when he/she is in said position. Then, the driver changes the position of the body/head, for example, he/she moves his/her head forward, in a second position corresponding to the position that the driver would occupy if he/she were about to change lane to pass (i.e., overtake) the car in front of him/her. Then, the driver selects the second FOV that he/she wants to see on the display when he/she is in said second position. Therefore, the ECU has at least two positions of the driver's body/head, with their respective first and second FOVs. At this point, the ECU establishes a (linear or non-linear) relationship based on these positions input by the driver, i.e., if the driver is in an intermediate position while driving, the crop (displayed image) moves in an intermediate position between the positions corresponding to the first FOV and the second FOV. If this relationship is linear, this means that the variation between the first position and the second position is proportional, but it may also not be proportional.

The reference position (first position) can be calculated as follows:.

Summarizing, the ECU has several ways to determine the movement of the crop (digital panning):.

The sensor or gesture detector is, for example, a camera (although it could be radar or another technology). The ideal situation is for it to always find a representative point of the face / body. If it is a camera, it can detect any facial expression, especially the eyes, so it would find the midpoint between the two eyes. If it is radar, detection could be done by finding the outline of the face, and then finding the midpoint (height of the head divided by two, and width of the head divided by two).

Furthermore, the movement of the crop, the speed of this movement (sensitivity, as explained before) and/or the size of the crop can be changed depending on the yaw angle, pitch angle or roll angle, the turn signal, or a movement of the steering wheel.

A last example of the proposed camera monitoring system, CMS, comprises:.

The adjusted image region (<NUM>) is smaller than the captured image (<NUM>) and, optionally, is not centered on the captured image; instead, the image region (<NUM>), which is displayed by the electronic display device, is located at an inner, right or left, side of the captured image, so as the driver is provided with a path on the display to:.

Claim 1:
A camera monitoring system for a motor vehicle (<NUM>), comprising:
- an image capturing means located at an exterior mounting assembly of the motor vehicle (<NUM>), for capturing a raw image (<NUM>) from an exterior field of view of the vehicle (<NUM>), wherein the field of view, FOV, extends at least rearward outside the vehicle (<NUM>) and encompasses a portion of the exterior part of the vehicle (<NUM>), the image capturing means comprising an image sensor;
- an electronic control unit, ECU, connected to the image capturing means, the ECU (<NUM>) obtaining a captured image (<NUM>) from the raw image (<NUM>);
- an electronic display device connected to the ECU for displaying an image comprising at least one image region (<NUM>) of the exterior FOV, the at least one image region (<NUM>) being selected by the ECU from the captured image (<NUM>), the electronic display device being located inside the vehicle (<NUM>) and to be used by a driver (<NUM>) of the vehicle (<NUM>);
- a gesture detector configured to obtain at least one position of at least one part of the driver's body,
characterized in that the captured image (<NUM>) obtained by the ECU comprises a symmetrical image of the raw image (<NUM>) with respect to a vertical axis of the captured image (<NUM>), the symmetrical image being generated by the image sensor of the image capturing means; and in that the ECU is configured to move the at least one image region (<NUM>) within the captured image (<NUM>) according to a relative angle (X1, X2) defined as the angle between the, at least one, obtained position of the part of the driver's body and the electronic display device,
wherein, for a first relative angle (X1) defined with respect to a first obtained position and a second relative angle (X2) with respect to a second obtained position, when the second relative angle (X2) is greater than first relative angle (X1),
the ECU is configured to move the image region (<NUM>) at least to the left along a horizontal axis of the captured image (<NUM>) if the vehicle (<NUM>) is for right-hand traffic and at least to the right along a horizontal axis of the captured image (<NUM>) if the vehicle (<NUM>) is for left-hand traffic; and
wherein the exterior FOV is adjusted based on the at least one obtained position and the electronic display device is configured to display the adjusted exterior FOV in the image region (<NUM>).