Source: https://patents.google.com/patent/JP5459154B2/en
Timestamp: 2020-08-04 20:49:42
Document Index: 109406940

Matched Legal Cases: ['art 72', 'art 74', 'art 72', 'art 74', 'art 72', 'art 74', 'arts 74', 'arts 72', 'art 74', 'art 72', 'art 72', 'art 74', 'art 74', 'art 74', 'art 74', 'art 78']

JP5459154B2 - Vehicle surrounding image display apparatus and method - Google Patents
Vehicle surrounding image display apparatus and method Download PDF
JP5459154B2
JP5459154B2 JP2010206794A JP2010206794A JP5459154B2 JP 5459154 B2 JP5459154 B2 JP 5459154B2 JP 2010206794 A JP2010206794 A JP 2010206794A JP 2010206794 A JP2010206794 A JP 2010206794A JP 5459154 B2 JP5459154 B2 JP 5459154B2
JP2010206794A
JP2012063918A5 (en
JP2012063918A (en
平司郎 豊田
敏弘 服部
2010-09-15 Application filed by トヨタ自動車株式会社 filed Critical トヨタ自動車株式会社
2010-09-15 Priority to JP2010206794A priority Critical patent/JP5459154B2/en
2012-03-29 Publication of JP2012063918A publication Critical patent/JP2012063918A/en
2012-12-13 Publication of JP2012063918A5 publication Critical patent/JP2012063918A5/ja
2014-04-02 Publication of JP5459154B2 publication Critical patent/JP5459154B2/en
The present invention relates to a vehicle surrounding image display apparatus and method, and the like.
Conventionally, when displaying an image picked up by a camera, a frame display surrounding the recognized attention object (in order for the driver to easily recognize the position of the attention object recognized by the image processing) ( It is known to superimpose an obstacle marking display (see, for example, Patent Document 1).
In addition, as a display method of this kind of frame line display, the frame line display (band-shaped frame display) set around the obstacle is increased as the degree of obstacle danger increases, Depending on the degree, the color of the belt-like frame is also changed (for example, from the one with a high degree of danger, it is colored as red> orange> yellow> blue), or the difference in brightness from the background is increased. A method of changing the brightness is known (for example, see Patent Document 2).
JP 2007-159036 A JP 2009-40108 A
However, when the frame line display is displayed in a single color, the frame line display may become inconspicuous due to the relationship with the background color or the brightness of the background. In this case, there is a problem that the effect of emphasizing the attention object by the frame line display is reduced.
Therefore, an object of the present invention is to provide a vehicle surrounding image display device and method capable of displaying a frame line display in a manner in which the effect of emphasizing the attention object is not reduced.
In order to achieve the above object, according to one aspect of the present invention, there is provided a vehicle surrounding image display device mounted on a vehicle,
A processing device connected to a camera for photographing the periphery of the vehicle;
The processing device processes a captured image from the camera to recognize an attention object and generates a display image in which a frame display at least partially surrounding the recognized attention object is superimposed on the captured image And
The frame display includes a first frame portion drawn with a first color and a second frame portion drawn with a second color different from the first color ,
When the processing device processes a captured image from the camera and recognizes a target object, the processing device temporarily brightens a region that is substantially the entire region of the display image except for the frame line display. There is provided a vehicle surrounding image display device characterized by performing flushing for returning to the original brightness after increasing the height .
According to another aspect of the present invention, there is provided a vehicle surrounding image display method,
Processing a captured image from a camera that captures the periphery of the vehicle and recognizing a target object;
A frame line display that at least partially surrounds the recognized attention object, and a first frame line portion rendered in a first color and a second color rendered in a second color different from the first color A process of generating a display image in which a frame line display including a frame line part is superimposed on the captured image ;
Look including a process for outputting on a display device a display image of said generated
When processing a captured image from the camera and recognizing a cautionary object, the brightness of the entire display image area except for the frame line display is temporarily increased. There is provided a vehicle surrounding image display method characterized by performing flushing to restore the original brightness later .
ADVANTAGE OF THE INVENTION According to this invention, the surrounding image display apparatus for vehicles and a method etc. which can display a frame-line display in the aspect in which the emphasis effect of the attention object is not reduced are obtained.
It is a figure which shows the principal part structure of the surrounding image display apparatus 1 for vehicles by one Example of this invention. It is a flowchart which shows an example of the main processes implement | achieved by the control apparatus 10 of a present Example. It is a figure which shows an example of the frame line display produced | generated by step 204 of FIG. It is an enlarged view of the corner part (X part of FIG. 3) of the frame line display. It is a figure which shows an example of the image for a display containing the frame line display produced | generated by step 204 of FIG. 2 with a comparative example. 6A to 6D are diagrams showing some other examples of the display image including the frame line display generated in step 204 in FIG. FIGS. 7A to 7D are diagrams showing still other examples of display images including a frame line display generated in step 204 of FIG. FIG. 8A and FIG. 8B are diagrams showing some other examples of the display image including the frame line display generated in step 204 of FIG. FIGS. 9A to 9D are diagrams showing still other examples of display images including a frame line display generated in step 204 of FIG. It is a flowchart which shows another example of the main processing implement | achieved by the control apparatus 10 of a present Example. It is a figure which shows an example of the state of the image for a display by a present Example at the time of flushing. It is a figure which shows an example of the state of the image for a display by the comparative example at the time of flushing. It is a figure which shows the principal part structure of the surrounding image display apparatus 2 for vehicles by one Example (Example 2) of this invention. It is a flowchart which shows an example of the main processes implement | achieved by the control apparatus 12 of a present Example. It is a figure which shows an example of the image for a display containing the frame line display produced | generated by step 1404 of FIG.
FIG. 1 is a diagram illustrating a configuration of a main part of a vehicle surrounding image display device 1 according to an embodiment (embodiment 1) of the present invention. The vehicle surrounding image display device 1 is configured around a control device (ECU) 10. The control device 10 is configured as a microcomputer including a CPU, a ROM, a RAM, and the like connected to each other via a bus (not shown). The ROM stores various programs and data executed by the CPU to realize various functions described later. The control apparatus 10 may be implement | achieved by one ECU, for example, and may be implement | achieved in cooperation with one or more ECU.
A near infrared camera 20 is connected to the control device 10. The near-infrared camera 20 is provided corresponding to a near-infrared light irradiation device 42 described later. The near-infrared camera 20 is a camera having a sensitivity distribution up to the near infrared, for example, a CCD (charge-coupled device) camera having a sensitivity distribution up to the near infrared. The near-infrared camera 20 is provided in the vehicle interior so as to image a near-red irradiation region (described later) by the near-infrared light irradiation device 42. For example, the near-infrared camera 20 may be attached to the back side (front surface of the vehicle) of the rearview mirror. The near-infrared camera 20 may acquire a front near-infrared image in real time and supply it to the control apparatus 10 with an NTSC signal while a function selection button 50 described later is turned on.
The control device 10 detects (recognizes) a target object that may exist in front of the vehicle by performing image processing on the front near-infrared image supplied from the near-infrared camera 20. The attention object is typically a person (pedestrian), but may be another animal. Note that there are a wide variety of recognition / determination algorithms for pedestrians and the like by image processing, and any appropriate algorithm may be adopted. For example, a high luminance region in the front near infrared image may be detected, and a pedestrian or the like may be recognized based on the shape, movement, distance, and the like of the high luminance region.
The control device 10 generates a display image for providing various information in front of the vehicle to the driver or the like based on the front near infrared image supplied from the near infrared camera 20. Typically, the control device 10 notifies the driver or the like of the presence or position of an attention object that may exist in front of the vehicle based on the front near infrared image supplied from the near infrared camera 20. A display image is generated.
A display device 30 is connected to the control device 10. The display device 30 may be a liquid crystal display, for example. The display device 30 is provided at a position that is easily visible to the driver in the vehicle interior. For example, the display device 30 may be provided on the instrument panel. The display device 30 displays a display image supplied from the control device 10 (an image generated based on a front near-infrared image from the near-infrared camera 20). For example, the display device 30 may receive a control signal from the control device 10 via a CAN (controller area network) and may receive a video signal from the control device 10 via a GVIF (Gigabit Video InterFace).
A headlamp device 40 is connected to the control device 10. The headlamp device 40 includes a near infrared light irradiation device 42 and a drive circuit 44 for the near infrared light irradiation device 42. The near-infrared light irradiation device 42 is mounted on both sides of the front portion of the vehicle, for example. The near infrared light irradiation device 42 may be incorporated in a headlight assembly including a headlamp. The optical axis of the near-infrared light irradiation device 42 is adjusted so as to irradiate near-infrared light toward the vehicle front area. Hereinafter, the vehicle front area irradiated by the near infrared light irradiation device 42 is referred to as a “near red irradiation area”. The drive circuit 44 operates the near infrared light irradiation device 42 in response to, for example, an ON command from the control device 10. The on command from the control device 10 is typically generated when a function selection button 50 described later is turned on. When the near-infrared light irradiation device 42 is operated, near-infrared light is irradiated to the near-red irradiation region, and the front near-infrared image is captured by the near-infrared camera 20 in real time.
The near-red irradiation area is typically set on the far side from the headlamp (low beam) irradiation area. This is because the near-red illumination area is set to an area where the low beam cannot be illuminated or the illumination amount is insufficient, so that the driver can monitor cautionary objects such as pedestrians in the distance that are difficult for the driver to visually recognize. . For example, the low beam irradiation area may be an area up to about 40 m ahead of the vehicle, and the near red irradiation area may be an area ahead of the vehicle (for example, about 40-100 m ahead) of the low beam irradiation area. Note that the near red irradiation region and the low beam irradiation region may be close to each other and partially overlap.
A function selection button 50 is connected to the control device 10. The function selection button 50 is a switch for switching on / off the night view function. The function selection button 50 is provided at a position where it can be easily operated by the driver in the passenger compartment. The function selection button 50 may be provided on, for example, an instrument panel or a steering column.
FIG. 2 is a flowchart illustrating an example of main processing realized by the control device 10 of the present embodiment.
The processing routine shown in FIG. 2 may be started when the function selection button 50 is turned on by the user.
In step 200, the control device 10 displays a display image on the display device 30 based on the front near infrared image supplied from the near infrared camera 20.
In step 202, the control device 10 performs image processing on the front near-infrared image supplied from the near-infrared camera 20, and determines whether a person (an example of an alerting object) has been detected. If a person is detected, the process proceeds to step 204. On the other hand, if no person is detected, the process returns to step 200. In this case, in step 200, an image for display that does not include an emphasis display (frame line display) described later is displayed on the display device 30.
In step 204, the control device 10 performs an image emphasis display for alerting the user (especially the driver) about the detected person. Specifically, the control device 10 draws (superimposes) a frame line display that at least partially surrounds the detected person, and generates a display image including the frame line display (described later with reference to FIG. 5 and the like). Then, the display image including the frame line display is displayed on the display device 30. Details of the frame display will be described later.
In step 206, the control device 10 determines whether the function selection button 50 is on or off. If the function selection button 50 is on, the process returns to step 202 and the processes in and after step 202 are repeated. On the other hand, if the function selection button 50 is off, the process ends.
FIG. 3 is a diagram showing an example of the frame line display generated in step 204 of FIG. The frame line display 70 shown in FIG. 3 has a substantially rectangular shape as a whole. The frame line display 70 includes a main frame line part 72 and a sub-frame line part 74. The main frame line part 72 is drawn with a first color, and the sub-frame line part 74 is drawn with a second color different from the first color. When the display image is a grayscale image, the first color and the second color are realized by a difference in gradation. For example, the gradation is selected so that the first color is white and the second color is black. In this case, preferably, the first color and the second color are selected so as to be separated from each other by the maximum gradation. For example, in the case of 256 gradations, the first color may be expressed by “255” (white), and the second color may be expressed by “0” (black). When the frame line display 70 is expressed in RGB, the first color is set to an RGB value (R: 255, G: 255, B: 255) (white), and the second color is set to an RGB value ( R: 0, G: 0, B: 0) (black). Alternatively, the first color and the second color are preferably selected so as to have a complementary relationship (that is, the first color and the second color are selected so as not to be similar colors or approximate colors). )
The sub-frame portion 74 is provided in association with the main frame portion 72 as shown in FIG. In the example shown in FIG. 3, the sub-frame portion 74 is drawn in a manner that borders the main frame portion 72. Specifically, the sub-frame portion 74 includes an outer edge portion 74 a adjacent to the outer edge of the main frame portion 72 and an inner edge portion 74 b adjacent to the inner edge of the main frame portion 72. In the example illustrated in FIG. 3, the sub-frame portion 74 is drawn in a manner that realizes edging on the entire circumference of the main frame portion 72.
The thickness (W 1 , W 2 ) of the main frame line portion 72 may be constant or may be changed. Further, the thickness W 2 of the vertical portion of the main frame beam section 72 may be the same as the thickness W 1 of the vertical portion of the main frame beam section 72 may be a different value. The same applies to the sub-frame portion 74. Further, the size of the frame line display 70 may be varied according to the size of the detected image of the alert object.
FIG. 4 is an enlarged view of a corner portion (X portion in FIG. 3) of the frame line display 70. As shown in FIG. 4, the frame line display 70 may be marked with an R at the corner. In the example shown in FIG. 4, the thickness (W 1 , W 2 ) of the main frame line portion 72 is drawn with 5 dots, and the sub frame line portion 74 is 1 dot for both the outer edge portion 74a and the inner edge portion 74b. Has been drawn.
FIG. 5 is a diagram illustrating an example of a display image including a frame line display generated in step 204 of FIG. In FIG. 5, a comparative example is shown on the left side of this example.
In the comparative example, the frame line display is drawn in a single color (in this example, white). In this case, as shown in the figure, the frame line display for the pedestrian detected on the right side may not be noticeable due to the state of surrounding pixels. In the example shown in the drawing, the luminance of the pixels around the pedestrian detected on the right side is increased by the illumination from the oncoming vehicle light, and as a result, the frame line display for the pedestrian detected on the right side is inconspicuous.
On the other hand, according to the present embodiment, as shown in FIG. 5, the frame line display is drawn in two colors (see the main frame line part 72 and the sub frame line part 74 in FIG. 4). Therefore, as shown in the drawing, even when the brightness around the pedestrian detected on the right side is increased by the oncoming vehicle light, the presence of the sub-frame portion 74 having a large color (or brightness) difference from surrounding pixels. Therefore, the frame line display 70 for the pedestrian detected on the right side does not become inconspicuous. That is, the emphasis effect of the frame line display 70 on the pedestrian detected on the right side can be maintained. In this way, according to the present embodiment, by drawing the frame display 70 in two colors, the attention object is emphasized by the frame display 70 regardless of the pixel state around the attention object. Can be effectively maintained.
6A to 6D are diagrams showing some other examples of display images including the frame line display 70 generated in step 204 of FIG.
In the example shown in FIG. 6A, the sub-frame line portion 74 is drawn in a mode in which only the outer edge of the main frame line portion 72 is trimmed. That is, the example shown in FIG. 6A corresponds to the configuration in which the inner edge portion 74b of the example shown in FIG. 4 is eliminated. The sub-frame line portion 74 may be drawn on the main frame line portion 72 near the center in the width direction (thickness direction) of the main frame line portion 72. In this case, the sub-frame portion 74 is bordered by the main frame portion 72, which is substantially equivalent to the case where the sub-frame portion 74 changes the color of the main frame portion 72.
In the example shown in FIG. 6 (B), the sub-frame line portion 74 is drawn in a mode in which only the inner edge of the main frame line portion 72 is trimmed. That is, the example shown in FIG. 6B is different from the example shown in FIG. 4 only in that the outer edge portion 74a is eliminated.
In the example shown in FIG. 6C, the sub-frame portion 74 is drawn with a dotted line. The sub-frame portion 74 may be drawn with other line types such as a one-dot chain line and a broken line. Further, one of the outer edge portion 74a and the inner edge portion 74b may be drawn with a line type different from the other. In this case, one of the outer edge portion 74a and the inner edge portion 74b may be a solid line. The example illustrated in FIG. 6C may be realized in combination with the example illustrated in FIG. 6A and the example illustrated in FIG. That is, in each of the example shown in FIG. 6A and the example shown in FIG. 6B, the sub-frame line portion 74 may be drawn with a line type other than the solid line.
In the example shown in FIG. 6D, the sub-frame portion 74 is drawn so as to realize a dotted line in cooperation with the main frame portion 72. That is, the frame line display 70 is drawn so that the sub-frame line parts 74 and the main frame line parts 72 appear alternately along the longitudinal direction of the line. Similarly, the sub-frame portion 74 may be drawn so as to realize other line types such as a one-dot chain line and a broken line in cooperation with the main frame portion 72.
FIGS. 7A to 7D are diagrams showing still other examples of display images including a frame line display generated in step 204 of FIG.
In the example shown in FIG. 7A, the frame line display 70 is drawn discretely, not continuously over the entire circumference. That is, the frame display 70 is drawn so as to partially surround the detected attention object. Specifically, the frame line display 70 is drawn in a form having a gap at each central part of the four sides of the rectangle. In the example shown in FIG. 7A, the sub-frame line portion 74 is drawn in a mode in which only the outer edge of the main frame line portion 72 is trimmed, but only the inner edge of the main frame line portion 72 is drawn. May be drawn in a manner of bordering, or may be drawn in a manner of bordering both the inner edge and the outer edge of the main frame portion 72. Further, in the example shown in FIG. 7A, only one of the main frame line portion 72 and the sub frame line portion 74 may be drawn discretely, and the other may be drawn continuously over the entire circumference. (See FIGS. 9A and 9B).
In the example shown in FIG. 7B, the frame line display 70 is drawn discretely, not continuously over the entire circumference. That is, the frame display 70 is drawn so as to surround only the upper side of the detected attention object. Specifically, the frame line display 70 is drawn only in a substantially upper half of a quadrangle, and is drawn in a mode having a gap at the center of the upper side. In the example shown in FIG. 7B, the sub-frame line portion 74 is drawn in a mode in which only the outer edge of the main frame line portion 72 is trimmed, but only the inner edge of the main frame line portion 72 is drawn. May be drawn in a manner of bordering, or may be drawn in a manner of bordering both the inner edge and the outer edge of the main frame portion 72. In the example shown in FIG. 7B, only one of the main frame line portion 72 and the sub frame line portion 74 may be drawn discretely, and the other may be drawn continuously over the entire circumference. .
In the example shown in FIG. 7C, the frame line display 70 is not continuous over the entire circumference but is drawn discretely. That is, the frame display 70 is drawn so as to surround only the lower side of the detected attention object. Specifically, the frame line display 70 is drawn only in a substantially lower half of a quadrangle, and is drawn in a mode having a gap at the center of the lower side. In the example shown in FIG. 7C, the sub-frame line portion 74 is drawn in such a manner that only the outer edge of the main frame line portion 72 is outlined, but only the inner edge of the main frame line portion 72 is drawn. May be drawn in a manner of bordering, or may be drawn in a manner of bordering both the inner edge and the outer edge of the main frame portion 72. In the example shown in FIG. 7C, only one of the main frame line portion 72 and the sub frame line portion 74 may be drawn discretely, and the other may be drawn continuously over the entire circumference. .
In the example shown in FIG. 7D, the frame display 70 has a vertically long circular (elliptical) form instead of a quadrangle. As described above, the frame line display 70 is not limited to a quadrangle as long as it is a form that at least partially surrounds the detected attention object, and may be an arbitrary shape. In the example shown in FIG. 7D, the frame display 70 has the same relationship as the rectangular frame display 70 shown in FIG. However, the frame-line display 70 shown in FIG. 7D and the frame-line display 70 having any other shape are a rectangular frame-line display 70 as described with reference to FIGS. It is possible to have variations similar to the various variations.
FIG. 8A and FIG. 8B are diagrams showing some other examples of the display image including the frame line display generated in step 204 of FIG.
In the example shown in FIGS. 8A and 8B, the frame display 70 is drawn so as to look three-dimensional (with a sense of depth). Specifically, the frame line display 70 is drawn at a position where the sub-frame part 74 is offset obliquely in the lower right direction (on the image) with respect to the main frame part 72. In the illustrated example, the sub-frame portion 74 is drawn with an offset corresponding to the thickness of the main frame portion 72 offset from the main frame portion 72 in the downward and right directions. Note that the offset direction of the sub-frame line portion 74 with respect to the main frame line portion 72 is not limited to the diagonally lower right direction, and may be an oblique upper right direction, an oblique lower left direction, an oblique lower left direction, or the like. In addition to the examples shown in FIGS. 8A and 8B, the main frame line portion 72 and the sub-frame line portion 74 may have other shapes (for example, FIG. 7B and FIG. 7). (C) and the form shown in FIG. 7D) may be included.
FIGS. 9A to 9D are diagrams showing still other examples of display images including a frame line display generated in step 204 of FIG.
In the example shown in FIG. 9A, the frame display 70 is drawn in such a manner that the sub-frame portion 74 borders only the four corners of the main frame portion 72 from the outside. In the example shown in FIG. 9B, the frame display 70 is drawn in such a manner that the sub-frame portion 74 borders only the four corners of the main frame portion 72 from the inside. In the example shown in FIG. 9C, the frame display 70 is drawn in such a manner that the sub-frame portion 74 borders only the corners of the four corners of the main frame portion 72 from both the outside and the inside. In the example shown in FIG. 9D, the frame display 70 is drawn in the central part (the central part in the thickness direction) of the corners of the four corners of the main frame part 72 in the sub-frame part 74.
FIG. 10 is a flowchart illustrating another example of the main processing realized by the control device 10 of the present embodiment. The processing routine shown in FIG. 10 differs from the example shown in FIG. 2 mainly in that step 1006 is added. The processing in steps 1000, 1002, 1004, and 1008 may be the same as the processing in steps 200, 202, 204, and 206 shown in FIG.
In step 1006, the control device 10 performs flushing for a certain period of time on the display image displayed on the display device 30. Flushing is an iterative change between the normal display state and the display state in which the brightness of each pixel is increased (hereinafter referred to as a high-brightness display state) for a certain period of time. Including. The high luminance display state may be a state in which the luminance of each pixel in the normal display state is added by a certain luminance or a certain proportion of luminance (however, when the luminance exceeds the maximum luminance), or The brightness of each pixel may be uniformly changed to a constant high brightness (for example, maximum brightness). For example, in the case where the display image is a grayscale image, the gradation of each pixel may be changed to white (“255”) or close to it at a constant value or a constant ratio in the high luminance display state.
In the present embodiment, when performing flushing, the brightness of the frame display 70 is preferably not substantially changed from the brightness of the normal display state. That is, the brightness of the frame display 70 is maintained substantially constant during flushing. For example, when the display image is a grayscale image, the main frame line portion 72 of the frame line display 70 is drawn in white (“255”), and the sub-frame line portion 74 of the frame line display 70 is black (“ In the case of drawing with “0 ″), the black (“ 0 ”) of the sub-frame portion 74 is maintained without being changed during the flushing.
Note that flushing is intended to attract the attention of the user (driver), but the display image itself is difficult to see in the high-intensity display state during flushing, so the fixed time for flushing should be It may be appropriately determined from the viewpoint of the arousal and the visibility of the display image. For example, during the flushing, the high brightness display state may be formed only three or four times.
FIG. 11 is a diagram showing an example of the state of the display image according to the present embodiment at the time of flushing. FIG. 11A shows a normal display state, and FIG. 11B shows a high luminance display state. Show. FIG. 12 is a diagram illustrating an example of a state of a display image according to a comparative example during flushing. FIG. 12A illustrates a normal display state, and FIG. 12B illustrates a high-luminance display state. .
In the comparative example, as shown in FIG. 12A, the frame line display 70 'is drawn in a single color (white in this example). In this case, as shown in FIG. 12B, in the high-intensity display state at the time of flushing, the frame line display 70 ′ for the detected pedestrian may become inconspicuous due to the high-intensity state of surrounding pixels. . In the illustrated example, in the high luminance display state, the gradation of each pixel of the display image is changed to white (“255”) or close to it, and as a result, the frame display 70 for the detected pedestrian is displayed. 'Is not noticeable.
On the other hand, according to the present embodiment, as shown in FIG. 11A, the frame display 70 is drawn in two colors (see the main frame portion 72 and the sub-frame portion 74 in FIG. 4). ing. Therefore, according to the present embodiment, as shown in FIG. 11B, the sub-frame portion 74 (see FIG. 11) having a large color (or brightness) difference from the surrounding pixels even in the high-luminance display state at the time of flushing. 3), the frame line display 70 for the detected pedestrian does not become inconspicuous. That is, it is possible to maintain the emphasis effect of the frame line display 70 on the detected pedestrian even during flushing. In this way, according to the present embodiment, by rendering the frame line display 70 in two colors, the action of emphasizing the attention object by the frame line display 70 can be synergistically enhanced in cooperation with flushing. it can.
In the example shown in FIG. 11, when a person (an example of an alerting object) is detected, in addition to the above-described frame display 70 being displayed, the outer peripheral portion 78 of the display image is brightened. A display image is generated. That is, the outer peripheral portion 78 of the display image is outlined with a bright color. For example, when the display image is a grayscale image, the outer peripheral portion 78 of the display image is drawn in white (“255”). This effectively alerts the user (especially the driver) that a person has been detected.
FIG. 13 is a diagram showing a main configuration of the vehicular ambient image display device 2 according to one embodiment (second embodiment) of the present invention. The second embodiment is different from the first embodiment in that a display image is generated by a normal camera without using the near-infrared camera 20. In the following, the configuration peculiar to the second embodiment will be described in detail, and other configurations will be denoted by the same reference numerals in the drawing, and detailed description thereof will be omitted.
The vehicle surrounding image display device 2 is configured around a control device (ECU) 12 as in the first embodiment.
A camera 22 is connected to the control device 12. The camera 22 may acquire an ambient image outside the vehicle using an image sensor such as a CCD or a CMOS (complementary metal oxide semiconductor). The camera 22 may be a wide angle camera equipped with a fisheye lens. The camera 22 is provided at an appropriate position according to the purpose of imaging. For example, the camera 22 may be a front monitoring camera attached to a rearview mirror or a front seat ceiling, a rear monitoring camera attached to a back door, or a side monitoring camera. The camera 22 may acquire a surrounding image in real time and supply the image to the control device 12 with an NTSC signal while a function selection button 52 described later is turned on.
The control device 12 detects (recognizes) a target object that may exist around the vehicle by performing image processing on the surrounding image supplied from the camera 22. The target object is typically another vehicle, but may be another obstacle, a person (pedestrian), or an animal. There are various recognition / determination algorithms for other vehicles and the like by image processing, and any appropriate algorithm may be employed.
The control device 12 generates a display image for providing various information around the vehicle to the driver or the like based on the surrounding image supplied from the camera 22. Typically, based on the surrounding image supplied from the camera 22, the control device 12 generates a display image for informing the driver or the like of the presence or position of a target object that may exist around the vehicle. To do.
The display device 30 and the function selection button 52 are connected to the control device 12 as in the first embodiment. The function selection button 52 is a switch for switching on / off of a function realized using the camera 22. Here, the function realized using the camera 22 may be, for example, a parking guide function used at the time of parking assistance control, a drive recorder function that records surrounding images for accident analysis, and the like. Here, description is continued as a parking guide function.
FIG. 14 is a flowchart illustrating an example of main processing realized by the control device 12 of the present embodiment.
The processing routine shown in FIG. 14 may be started when the function selection button 52 is turned on by the user.
In step 1400, the control device 12 displays a display image on the display device 30 based on the surrounding image supplied from the camera 22.
In step 1402, the control device 12 performs image processing on the surrounding image supplied from the camera 22, and determines whether or not another vehicle (an example of an alerting object) has been detected. If another vehicle is detected, the process proceeds to step 1404. On the other hand, if no other vehicle is detected, the process returns to step 1400. In this case, in step 1400, a display image that does not include the frame line display 70 is displayed on the display device 30.
In step 1404, the control device 12 performs image emphasis display on the detected other vehicle. Specifically, the control device 12 draws (superimposes) a frame line display 70 that at least partially surrounds the detected other vehicle, and generates a display image including the frame line display 70. The display image including the frame line display 70 is displayed on the display device 30.
In step 1406, the control device 12 determines whether the function selection button 52 is on or off. If the function selection button 52 is on, the process returns to step 1402, and the processes after step 1402 are repeated. On the other hand, if the function selection button 52 is off, the process ends.
FIG. 15 is a diagram illustrating an example of a display image (for example, a color image) including the frame line display 70 generated in step 1404 of FIG. In FIG. 15, the comparative example is shown on the left side with respect to the present example.
In the comparative example, the frame line display is drawn in a single color (for example, yellow). In this case, as shown in the figure, the frame line display for the detected other vehicle may not be noticeable due to the state of the surrounding pixels. In the illustrated example, the color of the peripheral pixels (image of the parking lot wall) around the detected other vehicle is similar to the color of the frame display, and as a result, the frame display for the detected other vehicle is conspicuous. It is gone.
On the other hand, according to the second embodiment, as shown in FIG. 15, the frame display is drawn in two colors (the main frame line portion 72 and the sub frame line portion 74). That is, the main frame portion 72 is drawn with a first color (for example, yellow), and the sub-frame portion 74 is drawn with a second color (for example, blue) different from the first color. The first color and the second color are preferably in a complementary color relationship (that is, selected so as not to be a similar color or an approximate color).
Therefore, as shown in the figure, even when the detected color of surrounding pixels (image of the parking lot wall) around the other vehicle is similar to the color of the main frame 72 in the frame display 70, Due to the presence of the sub-frame portion 74 having a large color difference, the detected frame-line display 70 for other vehicles will not become inconspicuous. That is, the emphasis effect of the frame line display 70 for the detected other vehicle can be maintained. In this way, according to the present embodiment, by drawing the frame display 70 in two colors, the attention object is emphasized by the frame display 70 regardless of the pixel state around the attention object. Can be effectively maintained.
In the second embodiment, flushing as described in the first embodiment may be performed. For example, in step 1404 of FIG. 4, the control device 12 may perform image highlighting on the detected other vehicle and perform flushing for a certain period of time. In this case as well, the color or brightness of the frame display 70 in the normal display state is substantially maintained in the high luminance state, so that the attention object is emphasized by the frame display 70 when linked with flushing. It can be increased synergistically.
For example, in the above-described first embodiment, the near-infrared camera 20 captures an image of the front of the vehicle as the near-red irradiation region. In this case, the near infrared light irradiation device 42 is arranged and configured so that the near red irradiation region corresponds to the imaging region of the near infrared camera 20 accordingly.
Moreover, in the above-mentioned Example 1, the near-infrared image which imaged the near-red irradiation area | region may be displayed on the windshield glass of a vehicle by a head-up display (HUD), for example.
In the above-described embodiment, the color (or brightness / gradation) of the frame line display 70 is not changed from the normal display state in the high luminance state at the time of flushing, but the above-described effect (high luminance) As long as the effect that the frame line display 70 can be conspicuous even in the state is obtained, the color of the frame line display 70 may be changed from the normal display state. For example, when the display image is a grayscale image, the main frame portion 72 is drawn in white (“255”) and the sub-frame portion 74 is drawn in black (“0”) in a normal display state. In this case, if the black (“0”) of the sub-frame portion 74 is substantially black during the flushing, a change to a certain value (for example, “10”) can be allowed. In addition, it is only necessary to make the frame line portion with the lower brightness out of the main frame line portion 72 and the sub-frame line portion 74 of the frame line display 70 stand out even in the high luminance state. The line portion may have any brightness in the high luminance state. From the same viewpoint, the brightness of each of the main frame portion 72 and the sub-frame portion 74 of the frame line display 70 is synchronized with the alternate switching between the high luminance state and the normal display state at the time of flushing. The color (color) may be reversed (alternately replaced).
Further, in the above-described embodiment, the frame line display 70 is drawn in a certain color without depending on the scene, the type of attention object, the alertness level, etc., but the frame line display 70 is as described above. As long as the drawing is performed in two colors, the combination of the two colors of the frame display 70 may be changed. For example, the combination of the two colors of the frame display 70 may be changed according to, for example, the type of the scene or the attention object, the alertness level, or the like.
In the above-described embodiment, the frame line display 70 is drawn with two colors, but may be drawn with three or more colors.
DESCRIPTION OF SYMBOLS 1, 2 Ambient image display device 10, 12 Control device 20 Near infrared camera 22 Camera 30 Display device 40 Head lamp device 42 Near infrared light irradiation device 44 Drive circuit 50, 52 Function selection button 70 Frame line display 72 Main frame Line part 74 Subframe line part 74a Outer edge part 74b Inner edge part 78 Outer peripheral part
A vehicle surrounding image display device mounted on a vehicle,
When the processing device processes a captured image from the camera and recognizes a target object, the processing device temporarily brightens a region that is substantially the entire region of the display image except for the frame line display. A vehicle surrounding image display device that performs flushing for returning to the original brightness after increasing the height .
The second frame part is at least one of an aspect that borders at least a part of the outer edge of the first frame part and an aspect that borders at least a part of the inner edge of the first frame part. The vehicle surrounding image display device according to claim 1, which is drawn in such a manner.
The display image is a grayscale image,
The vehicle surrounding image display device according to claim 1, wherein the first color and the second color are realized by a difference in gradation.
2. The vehicle surrounding image display according to claim 1, wherein the first frame line portion is drawn in a dotted line or broken line manner, and the second frame line portion is drawn in an intermittent portion of the first frame line portion. apparatus.
2. The vehicle surrounding image display device according to claim 1, wherein the second frame line portion is drawn at a position that is obliquely offset on the screen with respect to the first frame line portion.
Further comprising, around the image display apparatus for a vehicle according to any one of claims 1 to 5 display device for outputting the display image generated by the processing unit.
A vehicle surrounding image display method comprising:
When processing a captured image from the camera and recognizing a cautionary object, the brightness of the entire display image area except for the frame line display is temporarily increased. A vehicle surrounding image display method, comprising: performing flushing to restore the original brightness later .
The computer-readable program which makes a computer perform each process as described in the surrounding image display method for vehicles of Claim 7 .
A storage medium storing the computer-readable program according to claim 8 .
JP2010206794A 2010-09-15 2010-09-15 Vehicle surrounding image display apparatus and method Active JP5459154B2 (en)
JP2010206794A JP5459154B2 (en) 2010-09-15 2010-09-15 Vehicle surrounding image display apparatus and method
PCT/IB2011/002031 WO2012035390A1 (en) 2010-09-15 2011-09-02 Surrounding image display system and surrounding image display method for vehicle
DE201111103083 DE112011103083T5 (en) 2010-09-15 2011-09-02 Environment image display system and environmental image display method for a vehicle
US13/822,452 US9123179B2 (en) 2010-09-15 2011-09-02 Surrounding image display system and surrounding image display method for vehicle
CN201180043610.4A CN103098097B (en) 2010-09-15 2011-09-02 Surrounding image display system and surrounding image display method for vehicle
JP2012063918A JP2012063918A (en) 2012-03-29
JP2012063918A5 JP2012063918A5 (en) 2012-12-13
JP5459154B2 true JP5459154B2 (en) 2014-04-02
ID=44801035
JP2010206794A Active JP5459154B2 (en) 2010-09-15 2010-09-15 Vehicle surrounding image display apparatus and method
US (1) US9123179B2 (en)
JP (1) JP5459154B2 (en)
CN (1) CN103098097B (en)
DE (1) DE112011103083T5 (en)
WO (1) WO2012035390A1 (en)
WO2015006935A1 (en) * 2013-07-17 2015-01-22 Harman International Industries, Incorporated Textureless wall detection
KR102069736B1 (en) * 2013-10-11 2020-01-23 현대모비스 주식회사 System and Method of Displaying Virtual Image
DE102014009421A1 (en) * 2014-06-25 2015-12-31 Brose Fahrzeugteile Gmbh & Co. Kommanditgesellschaft, Hallstadt Method for operating a camera of a motor vehicle and motor vehicle with a camera
JP6520668B2 (en) * 2015-02-09 2019-05-29 株式会社デンソー Display control device for vehicle and display unit for vehicle
DE102015002923A1 (en) 2015-03-06 2016-09-08 Mekra Lang Gmbh & Co. Kg Display device for a vehicle, in particular commercial vehicle
JP2018036444A (en) * 2016-08-31 2018-03-08 アイシン精機株式会社 Display control device
US5270688A (en) 1990-12-12 1993-12-14 Apple Computer, Inc. Apparatus for generating a cursor or other overlay which contrasts with the background on a computer output display
JP4060159B2 (en) * 2002-01-18 2008-03-12 本田技研工業株式会社 Vehicle periphery monitoring device
JP2004173048A (en) * 2002-11-21 2004-06-17 Auto Network Gijutsu Kenkyusho:Kk Onboard camera system
JP4217876B2 (en) * 2002-12-20 2009-02-04 財団法人生産技術研究奨励会 Method and apparatus for tracking moving object in image
JP2004325977A (en) * 2003-04-28 2004-11-18 Fuji Wako Kk Attention reminding sheet
JP4389173B2 (en) 2005-12-08 2009-12-24 株式会社デンソー Vehicle display device
DE102006012773A1 (en) 2006-03-17 2006-11-30 Daimlerchrysler Ag Object e.g. animal, image displaying method for e.g. driver assisting system, involves dividing image data of objects into two types of fields whose image data are manipulated in different way, where one field has surrounding field
JP4857974B2 (en) * 2006-07-13 2012-01-18 トヨタ自動車株式会社 Vehicle periphery monitoring device
DE202006010960U1 (en) 2006-07-14 2006-12-28 Donpon Precision Inc., Shituen Chiu Video monitoring device for motor vehicle, has emphasizing unit emphasizing moving object and packing smooth image into frame, which tracks movement of object within detection areas of cameras
JP5023872B2 (en) 2007-08-06 2012-09-12 株式会社デンソー Image display control device and image display control system
CN101742129A (en) * 2008-11-18 2010-06-16 中兴通讯股份有限公司 Method for processing caption
JP5218910B2 (en) * 2009-01-09 2013-06-26 トヨタ自動車株式会社 Night vision system
JP5071743B2 (en) * 2010-01-19 2012-11-14 アイシン精機株式会社 Vehicle periphery monitoring device
2010-09-15 JP JP2010206794A patent/JP5459154B2/en active Active
2011-09-02 US US13/822,452 patent/US9123179B2/en active Active
2011-09-02 DE DE201111103083 patent/DE112011103083T5/en active Pending
2011-09-02 CN CN201180043610.4A patent/CN103098097B/en active IP Right Grant
2011-09-02 WO PCT/IB2011/002031 patent/WO2012035390A1/en active Application Filing
US20130176329A1 (en) 2013-07-11
DE112011103083T5 (en) 2013-08-01
WO2012035390A1 (en) 2012-03-22
US9123179B2 (en) 2015-09-01
JP2012063918A (en) 2012-03-29
CN103098097A (en) 2013-05-08
CN103098097B (en) 2015-05-20
EP2257065B1 (en) 2019-04-10 Vehicle peripheral image display system
JP2012514558A (en) 2012-06-28 Peripheral image generation method and apparatus
CN100413324C (en) 2008-08-20 On-vehicle night vision camera system, display device and display method
Ref document number: 5459154