Source: http://www.google.com/patents/US20030165255?ie=ISO-8859-1&dq=6016038
Timestamp: 2015-01-30 09:36:12
Document Index: 396726547

Matched Legal Cases: ['art 11', 'art 13', 'art 15', 'art 17', 'art 11', 'art 13', 'art 15', 'art 17']

Patent US20030165255 - Peripheral image processor of vehicle and recording medium - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsThe present invention provides a vehicle vicinity image-processing apparatus and a recording medium capable of deducing a scene having left the field of vision of a camera mounted on the rear of a vehicle and drawing an image of the area behind the vehicle which includes this deduced part. An image-processing...http://www.google.com/patents/US20030165255?utm_source=gb-gplus-sharePatent US20030165255 - Peripheral image processor of vehicle and recording mediumAdvanced Patent SearchPublication numberUS20030165255 A1Publication typeApplicationApplication numberUS 10/333,745PCT numberPCT/JP2002/002525Publication dateSep 4, 2003Filing dateMar 15, 2002Priority dateJun 13, 2001Also published asCN1210958C, CN1473433A, DE10292327B4, DE10292327T5, US7317813, WO2002104032A1Publication number10333745, 333745, PCT/2002/2525, PCT/JP/2/002525, PCT/JP/2/02525, PCT/JP/2002/002525, PCT/JP/2002/02525, PCT/JP2/002525, PCT/JP2/02525, PCT/JP2002/002525, PCT/JP2002/02525, PCT/JP2002002525, PCT/JP200202525, PCT/JP2002525, PCT/JP202525, US 2003/0165255 A1, US 2003/165255 A1, US 20030165255 A1, US 20030165255A1, US 2003165255 A1, US 2003165255A1, US-A1-20030165255, US-A1-2003165255, US2003/0165255A1, US2003/165255A1, US20030165255 A1, US20030165255A1, US2003165255 A1, US2003165255A1InventorsHirohiko Yanagawa, Tetsuri Ishikawa, Haruhiko Ogiso, Masayuki ImanishiOriginal AssigneeHirohiko Yanagawa, Tetsuri Ishikawa, Haruhiko Ogiso, Masayuki ImanishiExport CitationBiBTeX, EndNote, RefManReferenced by (18), Classifications (22), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetPeripheral image processor of vehicle and recording mediumUS 20030165255 A1Abstract The present invention provides a vehicle vicinity image-processing apparatus and a recording medium capable of deducing a scene having left the field of vision of a camera mounted on the rear of a vehicle and drawing an image of the area behind the vehicle which includes this deduced part. An image-processing unit 5 first coordinate-converts a picked-up image to create a bird's-eye view image. Two chronologically consecutive bird's-eye view images are created as the bird's-eye view image, and a matching area of the two images is extracted. On the basis of this matching area, a moved area which has left the present field of vision of the camera 1 is extracted. An image of the matching area and the moved area is then drawn on a monitor 3. By this means, it is possible to draw an image including a scene which has left the present field of vision of the camera 1. Images(10) Claims(15)
BEST MODE FOR CARRYING OUT THE INVENTION [0038] Embodiments of a vehicle vicinity image-processing apparatus and recording medium according to the present invention will now be described. [0039] (First Embodiment) [0040] The basic system configuration of this embodiment will be explained using FIG. 1 and FIG. 2. [0041] As shown in FIG. 1, the vehicle vicinity image-processing apparatus of this embodiment has a camera (for example a CCD camera) 1 mounted on the rear of an automobile, an in-vehicle monitor (for example a liquid crystal display) 3 mounted on a dashboard, and an image-processing unit 5 for carrying out image-processing. [0042] As shown in FIG. 2, the image-processing unit 5 is an electronic device, having a microcomputer as a main part, for carrying out processing of image data, and, functionally, it has a coordinate conversion part 11 for performing a coordinate conversion on image data picked up by the camera 1 to create a bird's-eye view image; a matching part 13 for taking in two chronologically consecutive bird's-eye view images and comparing them; an area deduction part 15 for deducing an area having left the field of vision of the camera 1 from a non-matching part of the two bird's-eye view images; and a drawing part 17 for drawing an image to be displayed on the monitor 3. The image-processing unit 5 may be constructed integrally with the camera 1. [0043] An image processing procedure of this embodiment will now be described, on the basis of FIG. 3A through FIG. 5C. Here, the case of reversing (backing) a vehicle into a parking space will be used as an example. [0044] An image outputted from the camera 1 (the output image) is shown in the left side of FIG. 3A. This image is an image of the marking of a parking space drawn on the ground, and the vicinity of the parking space, picked up by the camera 1; and because the camera 1 is mounted on the top of the rear of the vehicle, the actually rectangular parking space marking is displayed distorted in correspondence with the distances and soon between the vehicle (and hence the camera 1) and the positions of the lines of the parking space marking. [0045] Next, as will be discussed in more detail later, the image on the left side of FIG. 3A is coordinate-converted to create a bird's-eye view image, and then to this bird's-eye view image is added an image showing the position of the vehicle and the field of vision angle of the camera 1 to generate a composite image, and this composite image is displayed on the screen of the monitor 3 as shown on the right side of FIG. 3A. That is, because the height at which the camera 1 is mounted and its mounting angle and field of vision angle are known, the output image of the camera 1 can be coordinate-converted by the coordinate conversion part 11 (using known bird's-eye view conversion) to obtain a bird's-eye view image like that shown in the right side of FIG. 3A. [0046] For this processing for converting the image picked up by the camera 1 into a bird's-eye view image and displaying it on the monitor 3, related art technology can be used (for example Japanese Unexamined Patent Publication No. H. 10-211849), as will be explained hereinafter. Here, by carrying out reverse processing of ordinary perspective conversion, the position of an image (for example a parking space marking) on a ground surface is obtained as a bird's-eye view image. In more detail, as shown in FIG. 4, perspective conversion is executed in such a manner that position data of an image on the ground is projected onto a screen plane T which has a focus distance f from the camera position R. [0047] Specifically, it is assumed that the camera 1 is positioned at a point R (0, 0, H) on a Z-axis, monitoring an image on a ground plane (x-y coordinate plane) at a look-down angle τ. And accordingly, here, as shown by the formula below (1), two-dimensional coordinates (α, β) on the screen plane T can be converted into coordinates on the ground plane (bird's-eye view coordinates) (reverse perspective conversion). [ x y ]  [ H � α / ( - β   cos   τ + f   sin   τ ) H � ( β   sin   τ + f   cos   τ ) / ( - β   cos   τ + f   sin   τ ) ] ( 1 ) [0048] That is, by using the above formula (1) it is possible to convert projected image data into image data for the screen of the monitor 3 (which shows a bird's-eye view image) and display it on the monitor 3. [0049] Next, when the vehicle backs into the parking space, as shown in the left side of FIG. 3B and the left side of FIG. 3C, the part of the parking space marking near the vehicle progressively leaves the field of vision of the camera 1; and at this time, the matching part 13 takes in two chronologically consecutive bird's-eye view images and compares them. [0050] Examples of consecutive bird's-eye view images (an image A changing to an image B) are shown in FIGS. 5A and 5B. As will be further discussed later, an area which matches completely in the two images (the matching area) can be extracted by comparing the image A and the image B. In the image B, because the vehicle is closer to the parking space marking than in the case of the image A, a part of the parking space marking is cut off. Consequently, the area deduction part 15 deduces that, of the areas of the images which do not match (the non-matching areas), the V-shaped area near the camera 1 is an area where the image has moved (moved area) due to movement of the vehicle. And by combining the moved area of the image A with the image B, as shown in FIG. 5C, it is possible to make a composite image in which a part of the parking space marking outside the field of vision of the camera 1 (the moved area) is also displayed. [0051] Although a method in which the matching area of the image A and the image B are extracted will be described here in after, various known methods can be employed for this image-processing. For example, predetermined areas (for comparing) of the images A and B can be divided into multiple small areas (pixels), as shown in FIGS. 6A and 6B; the pixels of the images A and B which match to the highest degree detected; and the positional relationship between the image A and the image B (i.e. how far the image A has moved) then obtained from the positional relationship of the matching pixels. [0052] Specifically, as shown in FIGS. 6A and 6B, considering the case where for example the brightness level of the figure (or alternatively the color) in each pixel is expressed with a number, here, the brightnesses of the bottom three rows of pixels in the image A and the brightnesses of the top three rows of pixels in the image B match. Accordingly, in this case, it can be supposed that the image A matches the image B by moving through the width of one row of pixels in the vertical direction of the drawing. In this case, it appears that the figure in the image B has moved upward as a result of the vehicle moving downward in FIGS. 6A and 6B. [0053] That is, when the image A and the image B are compared, both the images are divided into small areas, and when the images match in most of these small areas (when the degree of matching is high), the matching parts are regarded as the same image. Accordingly, when in the image A and the image B the matching parts are shifted, it can be inferred that they have become different due to the image having moved, and the amount and the direction of that movement can also be obtained. Similarly, amounts of movement in the left-right direction and diagonal directions can also be obtained. [0054] When this composite image is displayed on the monitor 3, as shown in the right side of FIG. 3B and the right side of FIG. 3C, the drawing part 17 draws the composite image and also draws an image showing the position of the vehicle and the field of vision angle of the camera 1. In this case, because the camera 1 is fixed to the vehicle in an unchanging position, the image of the vehicle and the field of vision angle of the camera 1 can be displayed in the same position on the screen of the monitor 3 at all times (for example the upper part of the center of the display screen). [0055] Processing carried out by the image-processing unit 5 and so on will now be described, on the basis of the flow chart of FIG. 7. [0056] As shown in FIG. 7, first, instep 100, it is determined whether or not the gear shift position is Reverse. When an affirmative determination is made here processing proceeds to step 110, and when a negative determination is made processing proceeds to step 170. The shift position can be detected on the basis of a signal from a shift position detection sensor (not shown) or a signal from another electronic control unit. [0057] In step 170, the image of an area outside the field of vision of the camera drawn previously is erased; that is, only the present camera image (bird's-eye view image) is displayed, and processing returns to step 100. [0058] In step 110, on the other hand, coordinate conversion (the above-mentioned bird's-eye view conversion) of the image picked up by the camera 1 is carried out, and a bird's-eye view image is obtained. In the following step 120, a matching area of two consecutive bird's-eye view images is extracted. For example from the image A and the image B in FIGS. 5A and 5B, the matching area is extracted. And in the following step 130, a moved area outside the present field of vision of the camera 1 is extracted. [0059] In step 140, whereas the moved area of the image A is later to be drawn outside the field of vision of the camera in the image B, before it is drawn, the image B already drawn is shifted by an amount corresponding to the moved area (downward in the image B), whereby a drawing area is secured. [0060] In the following step 150, as shown for example in FIG. 5C, the moved area is drawn in the drawing area secured in the foregoing step 140 (a composite image is composed). And in the following step 160, as shown for example in the right side of FIG. 3C, the vehicle and a V-shaped line showing the field of vision angle are drawn, and the present processing ends. [0061] Preferably, when a composite image has already been composed and stored, this composite image and the latest bird's-eye view image are compared and a non-matching area thereof is identified, and this non-matching area is added to the stored composite image to make a new composite image. In this way it is possible to synthesize images continuously by suitably adding each post-movement bird's-eye view image to the composite image, and as a result it is possible to understand the situation around the vehicle more clearly. [0062] Thus, in this embodiment, an image picked up by a camera is converted to a bird's-eye view image; a matching area of chronologically consecutive bird's-eye view images is extracted; and a moved area, which is a differing part of the two bird's-eye view images, is extracted. Then, the moved area is added to the bird's-eye view image showing the present image to make a composite image, and when the composite image is displayed on the screen of the monitor 3, an image of the vehicle and the field of vision angle of the camera are added. That is, in this embodiment, when the vehicle moves, an area having left the field of vision of the camera is deduced and combined with the present camera image, whereby it is possible to display the area which is outside the field of vision of the camera and ordinarily could not be displayed; and consequently, it is possible to understand the positional relationship between the vehicle and the parking space clearly. As a result, there is the valuable effect that maneuvering such as reversing into a parking space becomes extremely easy. [0063] (Second Embodiment) [0064] A second embodiment of the invention will now be described, although parts the same as parts in the first embodiment will not be explained again here. In this embodiment, only image of a parking space marking and car stoppers and the like is extracted and drawn, and the image of the background is dispensed with. [0065] As shown in the flowchart of FIG. 8, first, in step 200, it is determined whether or not the gear shift position is Reverse. When an affirmative determination is made here processing proceeds to step 210, and when a negative determination is made processing proceeds to step 280. [0066] In step 280, a previously drawn image of an area outside the field of vision of the camera is erased. In step 210, on the other hand, an image of the parking space marking and car stoppers and the like is extracted by means of binarization processing from the image picked up by the camera. Here, it is assumed that the parking space marking and the car stoppers are painted in white and are easily distinguishable from a ground surface of black asphalt. [0067] In the following step 220, the binarized image is subjected to bird's-eye view conversion to create a bird's-eye view image of the parking space marking and the car stoppers. Here, the background is of a single color. Then in step 230, a matching area of two consecutive bird's-eye view images is extracted. And in step 240, a moved area having left the present field of vision of the camera 1 is extracted. [0068] In the following step 250, before an image of the moved area is drawn, the image B already drawn is shifted (downward in the image B) by the amount of the moved area to secure a drawing area. Then in step 260, an image of the moved area is drawn in the drawing area secured in step 250 (a composite image is composed). Instep 270, the vehicle and a V-shaped line showing the field of vision angle of the camera are drawn, and the processing routine ends. [0069] In this embodiment also, the same benefits as those of the first embodiment are obtained; moreover, in this embodiment, the entire picked-up image is not converted to a bird's-eye view image and displayed, but rather just things necessary for parking, for example only the parking space marking and car stoppers, besides the vehicle and the field of vision angle, are displayed. As a result, the image processing load is lightened; the image displayed on the monitor 3 becomes easy to watch; and there is the merit that maneuvering of the vehicle becomes easy. [0070] (Third Embodiment) [0071] A third embodiment will now be described. First, the basic system configuration of this embodiment will be explained, using FIG. 9 and FIG. 10. [0072] As shown in FIG. 9, the vehicle vicinity image-processing apparatus of this embodiment has a camera (for example a CCD camera) 21 mounted on the rear of the vehicle; an in-vehicle monitor (for example a liquid crystal display) 23 mounted on the dashboard of the vehicle; a vehicle speed sensor 25 for detecting a vehicle speed; a yaw rate sensor 27 for detecting a yaw rate; and an image-processing unit 29 for carrying out image-processing. [0073] The image-processing unit 29, as shown in FIG. 10, is an electronic device, having a microcomputer as a main part, for carrying out processing of image data, and has a coordinate conversion part (bird's-eye view image conversion circuit) 31 for performing coordinate conversion of image data picked up by the camera 21; an image memory A for temporarily storing image data of a created bird's-eye view image; a vehicle speed signal input circuit 33 for inputting a vehicle speed signal; a yaw rate signal input circuit 35 for inputting a yaw rate signal; a CPU 37 for reading in the vehicle speed signal and the yaw rate signal and carrying out computation such as that for moving the bird's-eye view image (for example turning it); and an image memory B for storing data of a bird's-eye view image to be displayed on the monitor 23. [0074] Here, as will be further discussed later, the image data of a bird's-eye view image stored in the image memory B is image data of a bird's-eye view image (composite bird's-eye view image) made by combining an image (moved bird's-eye view image) obtained by moving a bird's-eye view image (first bird's-eye view image) created on the basis of image data picked up by the camera 21 at the previous time (for example at a time T) with a bird's-eye view image (second bird's-eye view image) created on the basis of image data picked up by the camera 21 newly (for example at a time T+1) The image-processing unit 29 may be constructed integrally with the camera 21, and the constituent parts of the image-processing unit 29 may be partially or entirely integrated by an LSI. [0075] A procedure of image processing in this embodiment will now be described, on the basis of FIGS. 11A and 11B and FIG. 12. The case of reversing (backing) a vehicle into a parking space will be used as an example. [0076] First, a procedure by which an image inputted from the camera 21 is converted into a bird's-eye view image and stored in the image memory A will be explained. An unprocessed image (output image) outputted from the camera 21 is shown in FIG. 11A. Because this unprocessed image is an image of the marking of a parking space drawn on the ground and the vicinity thereof, picked up by the camera 21 mounted on the top of the rear of the vehicle, the actually rectangular parking space marking are displayed distorted in correspondence with the distances and so on between the vehicle (and hence the camera 21) and the positions of each line of the parking space marking. [0077] Accordingly, by the same method as that described in the first embodiment, the image data of the unprocessed image is coordinate-converted to generate a bird's-eye view image without distortion, like that shown in FIG. 11B, and the image data of this bird's-eye view image is stored in the image memory A. [0078] Next, a procedure by which, using the vehicle speed signal and the yaw rate signal, the bird's-eye view image stored in the image memory A is moved (for example turned) in correspondence with the amount by which the vehicle has moved (movement amount) will be explained. Here, in FIG. 12 the position of the vehicle at a time T is shown, and a distance through which the vehicle moves by a time T+1 is written L and an angle through which the vehicle moves by the time T+1 is written θ1 (=turn angle θC at turning center). [0079] First, when the sampling time of the camera 21 is 100[ms], using the vehicle speed S [km/h] obtained from the vehicle speed signal, the distance L [m] through which the vehicle has moved can be calculated from the following formula (2). L[m]=S[km/h]�1000[m/km]�3600[S/h]�100[ms]�1000[ms/s] (2) [0080] And, because there is the relationship of the figure shown in FIG. 12, using the yaw rate θ0 [�/s] obtained from the yaw rate signal, the angle through which the vehicle has moved can be calculated from the following formula (3). θ1[�]=θ0[�/s]�100[ms]�1000[ms/s] (3) [0081] Also, here, because it is being assumed that the turning center of the vehicle is on the extension line of the rear axle, if the forward direction of the vehicle is written Y and the direction perpendicular to this is written X, computing with the camera position as the X-direction origin, the turning center position XC [m] in the X-direction can be calculated from the following formula (4). XC[m]=L[m]�TAN (θ1) (4) [0082] On the other hand, the turning center position in the Y-direction is at the rear axle, and the distance from the camera 21 to the rear axle is YC [cm]. And, from the relationship of the figure shown in FIG. 12, the turn angle θC at the turning center is the same as the angle θ1 through which the vehicle has moved. [0083] Next, using the turning center position (XC, YC) and turn angle θC obtained as explained above, the bird's-eye view image is moved as follows. [0084] The image data of the bird's-eye view image stored in the image memory A is also stored once in the image memory B. Accordingly, here, the image data of the bird's-eye view image stored in the image memory B is transferred to internal memory of the CPU 37, and the bird's-eye view image is turned using the turning center position (XC, YC) and the turn angle θC. [0085] A conversion formula for carrying out this turning movement is shown below as formula (5), and by internal processing of the CPU 37 using this formula, with respect to post-movement coordinates (XB, YB) pertaining to after the move through the turn angle θ, corresponding pre-movement points (XA, YA) are calculated, and (XA, YA) pixel values are saved in the (XB, YB) address memory. XA=XB�COS θ+YB�SIN θYA=YB�COS θ−XB�SIN θ (5) [0086] . . . where (XA, YA) are pre-movement coordinates; (XB, YB) are post-movement coordinates; and θ is the turn angle. [0087] Then, by performing these calculations for all the coordinates, from the bird's-eye view image stored in the image memory A (the first bird's-eye view image, at the time T), a moved bird's-eye view image (moved bird's-eye view image at the time T) is created. And along with this, the bird's-eye view image created on the basis of the image newly picked up by the camera 21, that is, the present bird's-eye view image (the second bird's-eye view image, at the time T+1) is newly transferred from the image memory A to the CPU 37. [0088] And accordingly, in the CPU 37, the second bird's-eye view image is written to a position where the positional relationship corresponds with the time T (a position where there is no image misalignment with the moved bird's-eye view image), and by this processing the moved bird's-eye view image and the second bird's-eye view image are combined to create image data of a composite bird's-eye view image and this image data is stored in the image memory B. [0089] In the creation of the composite bird's-eye view image, although an image of the position of the vehicle and the field of vision angle of the camera 21 may be added and stored in the image memory B and this composite bird's-eye view image displayed on the screen of the monitor 23, alternatively, instead of an image of the position of the vehicle and the field of vision angle of the camera 21 being stored in the image memory B, an image showing the position of the vehicle and the field of vision angle of the camera 21 may be added when the composite bird's-eye view image stored in the image memory B is drawn to the monitor 23. [0090] Next, processing carried out by the image-processing unit 29 will be described, on the basis of the flow chart of FIG. 13. [0091] As shown in FIG. 13, first, in step 300, it is determined whether or not the gear shift position is Reverse. When an affirmative determination is made here processing proceeds to step 310, and when a negative determination is made processing proceeds to step 390. The shift position can be detected on the basis of a signal from a shift position detection sensor (not shown) or a signal from another electronic control unit. [0092] In step 390, any previously drawn image of an area outside the field of vision of the camera 21 is erased; that is, only the present image from the camera 21 (bird's-eye view image) is displayed; and processing then returns to step 300. In step 310, on the other hand, coordinate conversion (bird's-eye view conversion) of the image picked up by the camera 21 is carried out, and a bird's-eye view image is created. [0093] In step 320, this bird's-eye view image is stored in the image memory A. At this time, the same bird's-eye view image is also stored in the image memory B. In step 330, on the basis of the vehicle speed signal and the yaw rate signal, an amount of movement of the vehicle (movement amount), expressed by a movement distance L and a turn angle θC, is obtained. [0094] In the following step, step 340, on the basis of the amount of movement of the vehicle, the bird's-eye view image stored in the image memory B (first bird's-eye view image, at the time T) is moved, and a post-movement bird's-eye view image (moved bird's-eye view image from the time T) is obtained. [0095] In step 350, a second bird's-eye view image (at the time T+1), newly picked up and stored in the image memory A, is read in from the image memory A. And in step 360, the moved bird's-eye view image and the second bird's-eye view image are combined to generate a composite bird's-eye view image. Specifically, as shown in FIG. 14, the composite bird's-eye view image is composed by writing to outside the new field of vision of the camera (corresponding to the second bird's-eye view image) a part having left the field of vision of the camera due to the latest movement of the vehicle; that is, a part of the moved bird's-eye view image made by moving the first bird's-eye view image in correspondence with the amount of movement of the vehicle (specifically, the part which is left after a part overlapping with the second bird's-eye view image is removed). In FIG. 14, the vehicle is shown for reference; in practice, a frame or the like showing the vehicle is drawn when the composite bird's-eye view image is drawn on a screen of the monitor. [0096] In the following step 370, the composite bird's-eye view image is stored in the image memory B. In step 380, an image showing the vehicle and an image showing the field of vision angle are added; the composite bird's-eye view image stored in the image memory B is displayed on the monitor 23; and the present processing routine ends. [0097] In this way, in this embodiment, the image picked up by the camera 21 is converted into a bird's-eye view image; an amount by which the vehicle has moved is detected on the basis of vehicle signals; and in correspondence with this vehicle movement amount the bird's-eye view image is moved to create a moved bird's-eye view image. Then, the moved bird's-eye view image and a newly read-in bird's-eye view image are combined to compose a composite bird's-eye view image; images of the vehicle and the field of vision angle are added; and this composite bird's-eye view image is displayed on the screen of the monitor 23. [0098] That is, in this embodiment, when the vehicle moves, by deducing an area having left the camera view and combining it with the present camera view, it is possible to display parts which are outside the field of vision of the camera and ordinarily could not be displayed; and consequently, as shown in FIGS. 15A, 15B and 15C, it is possible to understand the positional relationship between the vehicle and the parking space clearly. As a result, there is the marked benefit that maneuvering such as reversing into a parking space becomes extremely easy. [0099] In FIGS. 15A, 15B and 15C, display pictures of the monitor 23 produced by the method of this embodiment are shown in order, when a vehicle first turns and then moves straight while backing into a parking space are shown in order. [0100] In particular, in this embodiment, because the moved bird's-eye view image is created using vehicle signals obtained from a vehicle speed sensor 25 and a yaw rate sensor 27, rather than being created for example by calculating a movement of the vehicle from change of image with time between first and second bird's-eye view images, there is the merit that the computational processing for this is easy. That is, because image-processing such as matching of images, which necessitates a large amount of computation, can be dispensed with, the processing load is lightened and the necessary image can be displayed on the monitor 23 swiftly. [0101] Consequently, the load on the microcomputer and so on is lightened, and the processing speed of the apparatus as a whole can be increased. Because the processing can be carried out by a microcomputer with a lower processing capacity, a contribution to cost reduction is also made. [0102] The present invention is not limited to the embodiments described above, and various changes can of course be made without deviating from the scope of the invention. [0103] (1) For example, in the first embodiment, preferably, the picked-up image of the parking space marking and the car stoppers and so on is not just simply coordinate-converted and displayed, but rather an image of the parking space marking and the car stoppers, which serve as guides for maneuvering the vehicle, is extracted from the image picked up by the camera, and the parking space marking and the car stoppers in the image are strengthened in color and/or tone. [0104] (2) And, in the third embodiment, although in the above-mentioned processing was started when it was detected that the shift position was Reverse, alternatively, so that processing is carried out also when the vehicle has switched to moving forwards after a reversing movement, starting and continuing of processing may alternatively be decided on the basis of a vehicle signal. For example, processing may be started when the vehicle speed is 10 km/h, and processing executed to display a bird's-eye view image for as long as the vehicle speed remains below this speed. By this means it is possible to display a bird's-eye view image continuously, without being reset, even if the shift position changes in the course of a parking operation. [0105] (3) Also, as another application of the invention, by means of a camera viewing an area in front of a vehicle, it is also possible to display a composite bird's-eye view image of the proximity of the vehicle body including the area having left the field of vision of the camera from in front of the vehicle, and thereby provide information by which it can be judged whether or not it is possible to squeeze through a narrow street or the like. [0106] (4) And, whereas in the first, second and third embodiments a vehicle vicinity image-processing apparatus was discussed, the invention also provides a recording medium on which is recorded means for executing the processing of such an apparatus. This recording medium may be any of various recording media, and may for example be an electronic control unit constructed as a microcomputer; a microchip; a floppy disk; a hard disk; and an optical disk. That is, it maybe any recording medium on which is recorded means, such as for example a program, capable of executing the processing of a vehicle vicinity image-processing apparatus described above. INDUSTRIAL APPLICABILITY [0107] As described above, a vehicle vicinity image-processing apparatus and a recording medium according to the present invention can draw on a monitor an image of the vicinity of a vehicle including a deduced part having left the field of vision of a camera, and accordingly they are highly suitable for use for example in equipment for assisting driving operation such as parking. Referenced byCiting PatentFiling datePublication dateApplicantTitleUS6825880 *Dec 28, 2000Nov 30, 2004Kabushiki Kaisha Toyoda Jidoshokki SeisakushoArrangement for guiding steering to assist parallel parkingUS7237641 *Apr 5, 2004Jul 3, 2007Nissan Motor Co., Ltd.Driving support apparatusUS7365653 *Mar 7, 2006Apr 29, 2008Sanyo Electric Co., Ltd.Driving support systemUS7532975 *Mar 29, 2005May 12, 2009Denso CorporationImaging apparatus for vehiclesUS7659835Sep 14, 2007Feb 9, 2010Mando CorporationMethod and apparatus for recognizing parking slot by using bird's eye view and parking assist system using the sameUS7725225 *Oct 30, 2007May 25, 2010Oshkosh CorporationRefuse vehicle control system and method with footboardUS7756621Oct 29, 2007Jul 13, 2010Oshkosh CorporationSteering control system and methodUS8295644 *Oct 27, 2008Oct 23, 2012Doubleshot, Inc.Birds eye view virtual imaging for real time composited wide field of viewUS8503728 *Oct 1, 2008Aug 6, 2013Nec CorporationRoad marking image processing device, road marking image processing method, and programUS8732592 *Jun 8, 2009May 20, 2014Battelle Energy Alliance, LlcMethods and systems relating to an augmented virtuality environmentUS20080205706 *Feb 26, 2008Aug 28, 2008Sanyo Electric Co., Ltd.Apparatus and method for monitoring a vehicle's surroundingsUS20080266396 *Apr 30, 2008Oct 30, 2008Gideon SteinRear obstruction detectionUS20090092334 *Oct 27, 2008Apr 9, 2009Doubleshot, Inc.Birds eye view virtual imaging for real time composited wide field of viewUS20100215218 *Oct 1, 2008Aug 26, 2010Katsuhiko TakahashiRoad marking image processing device, road marking image processing method, and programUS20100313146 *Jun 8, 2009Dec 9, 2010Battelle Energy Alliance, LlcMethods and systems relating to an augmented virtuality environmentCN1924514BAug 29, 2006Jan 25, 2012歌乐牌株式会社Obstacle detector for vehicleCN101976429A *Oct 27, 2010Feb 16, 2011南京大学Cruise image based imaging method of water-surface aerial viewWO2011047732A1 *Oct 22, 2009Apr 28, 2011Tele Atlas B.V.Method for identifying moving foreground objects in an orthorectified photographic image* Cited by examinerClassifications U.S. Classification382/104, 348/E07.085International ClassificationH04N7/18, G06T3/00, H04N5/225, B60R21/00, G09B29/00, G06T1/00, G06T7/00, B60R1/00Cooperative ClassificationG06T3/00, B60R1/00, B60T2201/10, B60R2300/806, G06T7/0024, B60R2300/302, B60R2300/607, H04N7/18European ClassificationG06T3/00, H04N7/18, B60R1/00, G06T7/00D1Legal EventsDateCodeEventDescriptionJun 8, 2011FPAYFee paymentYear of fee payment: 4Jan 29, 2003ASAssignmentOwner name: DENSO CORPORATION, JAPANOwner name: NIPPON SOKEN, INC., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YANAGAWA, HIROHIKO;ISHIKAWA, TETSURI;OGISO, HARUHIKO;ANDOTHERS;REEL/FRAME:014100/0101;SIGNING DATES FROM 20021223 TO 20021225RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services