Parking assist method and a parking assist apparatus

A parking assist apparatus includes a CPU for obtaining image data from a camera mounted on a vehicle, an image memory for storing the obtained image data as recorded image data, and an image processor for recognizing an object based on the image data. The image processor determines when to display a parking assist image to reveal an area hidden as a blind spot of the camera when the recognized object is in the blind spot of the camera and displays the parking assist image combining both current image data and the recorded image data for the area currently hidden as the blind spot of the camera, together with an indicator indicating the current position of the vehicle superimposed on the display.

INCORPORATION BY REFERENCE

The disclosure of Japanese Patent Application No. 2005-304936 filed on Oct. 19, 2005, including the specification, drawings and abstract thereof, is incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a parking assist method and a parking assist apparatus.

2. Description of the Related Art

One known parking assist apparatus, for assisting in a series of operations in parking a vehicle, obtains image data using an in-vehicle camera which is attached to the rear of the vehicle and outputs the image data on a monitor display mounted near the driver's seat. Although the in-vehicle camera typically provides images of the area behind the vehicle (or more specifically, images of the area behind the rear bumper of the vehicle), the area beneath the body of the vehicle and the area around a rear corner of the vehicle are out of the viewing area of the in-vehicle camera. Therefore, the further the vehicle enters a parking space, the less guidance landmarks, such as a white line marking the parking space, can be seen in the viewing area of the in-vehicle camera, so that it becomes difficult for a driver to park the vehicle because the driver can not determine the relative distance between the vehicle and the parking space or the position of a wheel stop in the monitor.

To resolve such a problem, an apparatus, which stores image data obtained by an in-vehicle camera in a memory and which displays composite image data made up of accumulated past image data and current image data, for example, is disclosed in Japanese Unexamined Patent Application Publication No. 2003-244688. Such an apparatus reads out image data for an area (blind spot) hidden from the in-vehicle camera and outputs a composite image using the image data.

However, in the conventional apparatus described above, the composite image is always displayed upon completion of storing image data, for example, even during the early stage of parking such as when the vehicle has not even begun to enter a parking space. Therefore, even if the driver wants to see a wide view of the area which is currently imaged by the in-vehicle camera, the unwanted composite image may be displayed on the monitor. Accordingly, there is a need for an apparatus which displays the composite image only at a suitable time, i.e. when the driver wants to see the area (blind spot) hidden from the in-vehicle camera.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to provide a parking assist method and a parking assist apparatus for displaying a parking assist image using recorded image data at a suitable time.

To solve the problems described above, the present invention provides a parking assist method for accumulating image data, which is obtained from an imaging device mounted on a vehicle, as recorded image data in an image data storage means, displaying a parking assist image using the recorded image data, wherein an area hidden from the imaging device (blind spot of the imaging device) is shown, recognizing an object in the vicinity of the vehicle, and determining when to display the parking assist image, wherein the area hidden from the imaging device is shown, based on the position of the object in relation to the blind spot of the imaging device.

The present invention also provides a parking assist apparatus for mounting in a vehicle, comprising: image data obtaining means for obtaining image data from an imaging device mounted on the vehicle; image data storage means for storing the obtained image data as recorded image data; recognizing means for recognizing an object; determining means for determining when to display a parking assist image, wherein an area hidden as a current blind spot of the imaging device is shown, based on the position of the recognized object in relation to the blind spot of the imaging device; and display control means for displaying the parking assist image using both the current image data currently obtained at the current position of the vehicle and the stored recorded image data including the area hidden as the current blind spot of the imaging device and for also displaying an indicator indicating the current position of the vehicle within (or superimposed on) the parking assist image, at a suitable time determined by the determining means. Therefore, for example, when the recognized object is located within the field of view of the imaging device, the parking assist apparatus may output a view of the background of the vehicle on a rear monitor screen and switch the screen to the parking assist image at a suitable moment when the recognized object is about to go or is going out of the field of view of the imaging device. More specifically, when the object which the driver wants to check leaves the field of view of the imaging device (goes out of sight), the area hidden as the blind spot, wherein the object is currently located, is displayed so that the driver may continuously check the position of the object, even though the current imaging device is not actually (currently, i.e. in real time) taking an image of the object within the blind spot area. As a result, the driver is assisted in parking the vehicle beginning at an intended position within the parking space. Further the driver may operate the vehicle to reduce impact of contact between the vehicle and the wheel stop, e.g. a low concrete barrier at the rear of a parking space.

The determining means may determine that the moment a part or full view of the recognized object enters the blind spot of the imaging device is the appropriate time to display the parking assist image using the recorded image data. Therefore, for example, when the object is located under the vehicle and therefore is no longer included within the image data, the recorded image data is output instead, so that the driver may still know the relative position and the relative distance between the vehicle and the object.

The imaging device may be mounted on the upper side of the bumper of the vehicle, in which case the determining means may determine that the moment a part or the entire recognized object moves under the bumper of the vehicle is the appropriate time to display the parking assist image. Therefore, the parking assist image may be displayed at the time when the object is located under the bumper in accordance with movement of the vehicle toward the parking space.

The display control means may output the recorded image data including that of the area hidden as the blind spot of the imaging device within a display area of the display means so that the direction of movement in the image displayed is that of the recognized object. Therefore, even if a part or full view of the object can not be seen in the display, the blind spot may be displayed in the direction of movement towards the object, so that the driver may easily understand which part of the road surface is currently displayed.

The recognized object maybe a wheel stop or a white line indicating a parking space, and the recognizing means recognizes the wheel stop or the white line by processing of the image data. Therefore, if the wheel stop or the white line goes out of the field of view of the imaging device, the recorded image data including the area hidden by the blind spot may be displayed at a suitable moment, and may allow the driver to check the relative position and the distance between a wheel of the vehicle and the wheel stop or the relative position and the distance between the rear of the vehicle and the white line.

Alternatively, the recognized object maybe an obstacle located within or around the parking space, and the recognizing means recognizes the obstacle by processing of the image data. Therefore, when the obstacle goes out of the field of view (viewing area) of the imaging device, the recorded image data including the area hidden as the blind spot may be displayed. Thus the driver may check the relative position and distance between the vehicle and the obstacle at a suitable time.

The recognizing means may recognize the object on the basis of an input through operation of an input operation device to specify the object. This operation enables the driver to select the object for determine the position of the vehicle relative thereto.

The parking assist apparatus of the present invention displays, responsive to a determination by the determining means that the parking assist image should not now be displayed, the image data currently obtained at the current position of the vehicle on the display screen, instead of the parking assist image. Therefore, when the object is within the viewing range of the imaging device, a wide-angle view of the background of the vehicle is displayed on the display screen.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention is described below with reference toFIGS. 1 to 13.

As shown inFIG. 1, a parking assist apparatus1mounted in a vehicle C includes a CPU2as an image data obtaining means. The CPU2receives, as input, a speed pulse and a direction detection signal of a speed sensor21and a gyro22, respectively, through a vehicle input I/F section6. Further, the CPU2calculates the relative coordinates of a reference position of the vehicle C based on the input speed pulse and the direction detection signal.

The CPU2also receives as inputs the on/off signals of an ignition system (not shown) from an ignition switch23of the vehicle C via the vehicle input I/F section6. When the CPU2receives the on-signal from the ignition switch23, the parking assist apparatus1is activated. When the CPU2receives the off-signal, the running parking assist apparatus1enters a standby mode and waits for the next on-signal from the ignition switch23. Next, the CPU2receives a shift position signal, indicating the current shift position, from a neutral start switch24, via the vehicle input I/F section6. The CPU2updates the shift position as a variable stored in a RAM3, based on the shift position signal.

When the CPU2receives the on-signal from the ignition switch23, the CPU2activates a camera20(on-board imaging device) mounted on the vehicle C and obtains image data G which including visual information for the area behind the vehicle C at a given time. As shown inFIGS. 2A and 2B, the camera20is positioned roughly in the center of the rear of the vehicle (on the y-axis of the vehicle coordinate system) and the optical axis of the camera faces downward. The camera20is a digital camera capable of taking color images and generating the image data G which undergoes analog/digital conversion. The digital camera may have optics (not shown) such as, for example, a wide-angle lens, a mirror, and so on, and a CCD image sensor (not shown). For example, the camera20may have a 140 degree wide field of view and the viewing area Z of the camera20may extend about several meters from the rear bumper of the vehicle C.

The CPU2stores the image data G as recorded image data15in an image memory7, which functions as an image data storage means, at a suitable time in correlation with position data16indicating an image position as shown inFIG. 3. The recorded image data15may include a white line103marking a parking space102on the road surface behind the vehicle C and a wheel stop101(as shown inFIG. 2).

Further, as shown inFIG. 1, outline data5may be stored in a ROM4of the parking assist apparatus1. The outline data5is data for generating an outline of the vehicle C in which the parking assist apparatus1is installed, as shown inFIG. 2. When the outline data5is output in a display9during a parking assist mode, an additional line35is displayed in a display area9zon the display9as a guide, as shown inFIG. 4. The added line35may include a framework line36indicating the outline of the rear of the vehicle and a wheel line37indicating the position of each rear wheel CR (FIG. 2). The framework line36and the wheel line37are virtual lines indicating, respectively, the positions of the rear of the vehicle and the rear wheels CR, as seen from the center point between the rear wheels CR.

An image processor8, which functions as the recognizing means, the determining means, and the display control means inFIG. 1, manipulates the image data G taken by the camera20. More specifically, the image processor8detects an object behind the vehicle C on the basis of the image data G. In the present embodiment, the recognized object is the wheel stop101behind the vehicle C, the white line103marking the parking space102(inFIG. 2), or an obstacle104.

For example, as shown inFIGS. 5A and 5B, the image processor8detects edges from the image data G. Then the image processor8searches for an edge EG of the recognized object among all detected edges EG, on the basis of pattern information data (not shown) given therefor. Alternatively, the image processor8may abstract feature points other than an edge. As pattern information, the image processor8may obtain data indicating the number of feature points which are known when the edge of the wheel stop101is detected or template data indicating the shape of the wheel stop101. The image processor8determines whether image data pertaining to the recognizable edge EG of the wheel stop101is in the image data G, on the basis of all the detected edges EG and the pattern information. Since the camera20has a wide-angle lens, the actual image data G may be distorted due to lens aberration. However, such image distortion is not shown here for sake of simplicity.

When the image processor8recognizes the white line103, the image processor8detects the edge EG of the white line103from the image data G as shown inFIG. 5A. Then, on the basis of a number of features of the detected edge EG, the image processor8determines whether or not the edge EG is that of the white line103.

The image processor8further detects the obstacle104on the road such as, a relatively large stone, a can, or a piece of trash, based on the image data G. As shown inFIG. 6A, the image processor8detects edges from the image data G and determines edges which form some sort of figure, such as edges in groups, circular edges, and brachymorphic edges, as the edge EG of the obstacle104.

The image processor8tracks the edges EG of each of the wheel stop101, the white line103, and the obstacle104in accordance with the movement of the vehicle C. For example, as shown inFIG. 5A, the image processor8temporarily stores pixel coordinates of each of a small area A1and a small area A2including the edge EG of the detected wheel stop101. Then the image processor8inputs newly obtained image data G from the CPU2. Based on the speed pulse input by the CPU2, the image processor8determines new coordinates for the positions to which the previously input small areas A1and A2in the image data G are moved, in accordance with the direction of movement of the vehicle and the distance moved. As shown inFIG. 5B, a new edge EG, which has the smallest difference from the previously detected edge EG of the wheel stop101is searched for within and around the small areas A1and A2, with movement of the vehicle. Alternatively a new edge EG, which is similar to the previously detected edge EG of the wheel stop101, may be searched for within the display area9z.

Whenever inputting the image data G through the CPU2, the image processor8calculates a relative position between the edge EG corresponding to the recognized object and an area BA behind the vehicle within the image data G, on the basis of the image data G. The term “the area BA behind the vehicle” here means an area including the rear of the vehicle RB and the rear bumper25of the vehicle C. In this case, the position of the camera20and its optical axis are fixed so that the relative position between the camera20and the rear bumper25is fixed as well. Therefore, the coordinates for the area BA behind the vehicle may be calculated in advance.

Next, the image processor8waits for the edge EG to completely enter the area BA behind the vehicle during tracking, the edge EG corresponding to the recognized object such as, for example, the wheel stop101. As shown inFIG. 5A, when the wheel stop101is within the viewing area Z of the camera20, the image processor8outputs the current image data G on the monitor screen9and superimposes a guide line41on the image data G as shown inFIG. 7. The guide line41may include a probable movement locus42, drawn depending on the rudder angle of the vehicle C, and an extended line43indicating the width of the vehicle C.

Then as shown inFIG. 2B, when the vehicle C enters the parking space102and the wheel stop101is located under the rear of the vehicle C, the wheel stop101is no longer within the viewing area Z of the camera20. More specifically, as shown inFIG. 5C, when the edge EG corresponding to the wheel stop101has completely entered the area BA behind the vehicle, the image processor8determines that an image, created by using the previously recorded image data15, should be displayed on the display9. The recorded image data15used by the image processor8at this time is data including the hidden area which is the blind spot of the camera20.

In a similar manner, the image processor8tracks the edges EG of the white line103and the obstacle104in accordance with the movement of the vehicle C. When tracking the edge EG of the white line103, the image processor8tracks, not the entirety of the white lines103, but only the end white line103a(as shown inFIG. 2A), which runs roughly at right angles to the direction of movement of the vehicle C within the parking space102. Then the image processor8waits until the edge EG corresponding to the white line103acompletely enters the area BA behind the vehicle, at which point an image formed using the recorded image data15is output on the monitor screen.

When tracking the obstacle104, as shown inFIG. 6B, when the edge EG corresponding to the obstacle104has completely entered the BA area behind the vehicle BA, the image processor8outputs an image using the recorded image data15on the display9. Note that, when the image processor8detects the edges EG for a number of objects such as, for example, the wheel stop101, the white line103a, and the obstacle104, the image processor8displays the image using the recorded image data15on the monitor screen9when any one of edges EG for the detected objects has completely entered the area BA behind the vehicle.

The display9connected to the parking assist apparatus1is a touch panel. When an input is made on the touch panel, the CPU2receives a predetermined signal corresponding to the input operation through an external input I/F section11of the parking assist apparatus1(as shown inFIG. 1). The CPU2also receives a predetermined signal corresponding to an input operation through the external input I/F section11when an input is made by means of an operation switch10(input operation means) mounted next to the monitor screen. The parking assist apparatus1may include an audio processor12which outputs an alarm and/or guidance from a speaker13connected to the parking assist apparatus1.

Next, the method of the present invention will be described with reference toFIG. 8. As shown inFIG. 8, the CPU2of the parking assist apparatus1waits for an on-signal from the ignition switch23to determine whether the ignition system is activated (Step S1-1). Responsive to receipt of the on-signal from the ignition switch23(Step S1-1=YES), the CPU2sets a bootable flag stored in the RAM3as “1” which means “on-state”. Next, the CPU2determines whether or not the shift position of the current vehicle C is “reverse”, based on the shift position signal from the neutral start switch24(Step S1-2). If the shift position is other than “reverse” (Step S1-2=NO), the CPU2sets a value for the shift position as a variable stored in the RAM3and the execution of the routine returns to Step S1-1. On the other hand, if the shift position is “reverse” (Step S1-2=YES), the CPU2changes the shift position stored in the RAM3as “reverse”, and the new position, established by the shift from “reverse”, is set as an initial position.

When the shift position is changed to “reverse”, the CPU2controls the camera20to obtain the image data G taken at the initial position (Step S1-3). Then the CPU2stores new image data including both image data G and position data16indicating the initial position correlated with the image data G, as the recorded image data15in the image memory7(Step S1-4).

When storing the recorded image data15based on an image captured at the initial position, the CPU2receives as input the speed pulse from the speed sensor21(Step S1-5). In accordance with the number of input pulses, the CPU2updates a distance counter stored in the RAM3, the distance counter accumulating the number of input pulses. Every time pulses are input, the number of input pulses is added to the number of pulses C1accumulated in the counter.

The CPU2determines whether or not the number of the accumulated pulses C1is a number corresponding to a predetermined distance and further determines whether or not the vehicle C has moved in reverse such a predetermined distance (Step S1-6). In the present embodiment, the predetermined distance is set at 200 mm. When the CPU2determines that the vehicle C has not traveled in reverse the predetermined distance (200 mm) in Step S1-4, the routine returns to Step S1-3. The CPU2waits for the vehicle C to move in reverse the predetermined distance (200 mm) during updating of the distance counter.

When the CPU2determines that the vehicle C has moved in reverse the predetermined distance (200 mm) in Step S1-6, the CPU2controls the camera20to obtain the image data G (Step S1-7). The CPU2then calculates the relative coordinates of the current position based on the initial position, where the shift position was changed to “reverse”, based on the speed pulses and the direction detection signal, and correlates the relative coordinates with the obtained image data G as the position data16. Then the CPU2stores the image data G as the recorded image data15in the image memory7(Step S1-8). As the CPU2stores the recorded image data15in the image memory7, the CPU2resets the number of the accumulated pulses C1of the distance counter to the initial value “0” (Step S1-9).

Next, the CPU2controls the image processor8to recognize the objects as described above based on the image data G obtained in Step S1-5(Step S1-10). More specifically, the CPU2detects edges from the image data G and searches the edges EG for the recognized objects among all of the detected edges, based on the pattern information as described above. The CPU2then determines whether any one of the wheel stop101, the white line103a, or the obstacle104is recognized (Step S1-11). When the CPU2does not recognize any of them (Step S1-11=NO), the routine goes to Step S1-14and the CPU2determines whether or not an end trigger has been input. In the current embodiment, the end trigger is a signal indicating any shift position other than “reverse” output from the neutral start switch24. When the CPU2determines that an end trigger has not been input (Step S1-14=NO), the routine returns to Step S1-5.

When the CPU2recognizes that any one of the wheel stop101, the white line103a, or the obstacle104(Step S1-11=YES), the CPU2further determines whether the recognized object is now located under the rear bumper25(Step S1-12). At this time, the image processor8determines whether or not the edge EG of the recognized object has completely entered the area BA behind the vehicle. As shown inFIGS. 5C and 6B, when the image processor8determines that the edge EG corresponding to the wheel stop101or the edge EG corresponding to the obstacle104is not displayed within the display area9zon the display9because the edge EG has completely entered the area BA behind the vehicle (Step S1-12=YES), the image processor8outputs a predetermined interrupt signal to the CPU2(Step S1-13). The interrupt signal is a signal for changing the image display mode to a new image display mode using the recorded image data15. As shown inFIGS. 5A,5B, and6A, when the edge EG of the recognized object has not entered the area BA behind the vehicle (Step S1-12=NO), the routine goes to Step S1-14.

Next, a routine for displaying an image on the display9will be described with reference toFIG. 9. The routine shown inFIG. 9is executed concurrently with the execution of the routine (Step S1-1through S1-14) described above. First, when the CPU2of the parking assist apparatus1receives the on-signal from the ignition switch23(Step S2-1=YES) and determines that the shift position is “reverse” (Step S2-2=YES), the CPU2further determines whether a rear view should be displayed (Step S2-3). The CPU2further determines whether or not the interrupt signal described above has been input. In the case where there has been no input of an interrupt signal, the CPU2determines that the rear view should be displayed (Step S2-3=YES) and the routine goes to Step S2-4. In the case that the interrupt signal has been input, the routine goes to Step S3-1inFIG. 10to display an image using the recorded image data15. More specifically, when the image processor8determines that the recognized object is not located under the vehicle C and when no interrupt signal is output, the image processor8executes the display of the rear monitor. On the other hand, when the image processor8determines that the recognized object is now located under the vehicle C and the interrupt signal is output in Step S1-13, the image processor8executes the display of the composite (synthesized) image using the recorded image data15.

In step S2-3, when determining that the rear view should be displayed, the CPU2controls the camera20to obtain the image data G (Step S2-4). The image data G is obtained at the rate of 30 frames per second. When obtaining the image data G, the image processor8outputs the input image data G and various drawing data onto the display9and displays a rear monitor screen38as shown inFIG. 11A(Step S2-5). As shown inFIG. 11A, a background39and the area behind the vehicle RB are displayed as the rear monitor screen38based on the current image data G. Further, the image processor8superimposes the guide line41on the background39. The guide line41here may be the extended line43indicating the width of the vehicle C and the probable movement locus42.

When the image processor8displays the rear monitor screen38, the CPU2determines whether or not an end trigger as described above has been input (Step S2-6). When the CPU2determines that an end trigger has not been received (input) (Step S2-6=NO), the routine returns to Step S2-2and the CPU2waits for the input of the interrupt signal and the end trigger while updating the rear monitor screen38.

As the vehicle C enters within the parking space102and the rear wheel CR approaches the wheel stop101, the image processor8detects edges within image data G captured each increment of movement of the vehicle C in reverse the predetermined distance (200 mm in this embodiment) and tracks the edges EG corresponding to recognized objects in accordance with the movement of the vehicle C. In Step S1-12, the image processor8determines whether or not the edge EG of an object selected for tracking from among the recognized objects has completely entered the area BA behind the vehicle. For example, when the vehicle C reverses, the rear monitor screen38on the display9is changed from the rear monitor screen38inFIG. 11Ato the screen shown inFIG. 11B. More specifically, when the vehicle C reverses, the image45corresponding to the wheel stop101(or either of the white line103aor the obstacle104) approaches in the image of the area RB behind the vehicle in the rear monitor screen. Then as shown inFIG. 11B, when a part of the image45corresponding to the wheel stop101is no longer completely shown in the image RB, the CPU2does not output the interrupt signal.

When, in the reverse movement of the vehicle C, the wheel stop101is completely located under the rear bumper25, which means the edge EG corresponding to the wheel stop101has completely moved out of the area BA behind the vehicle and out of the viewing area of the camera20as shown inFIG. 11C, the image processor8outputs an interrupt signal to the CPU2in Step S1-13. After the CPU2receives the interrupt signal in Step S2-3, the routine goes to Step S3-1.

As shown inFIG. 10, in Step S3-1, the image processor8is controlled by the CPU2to determine whether predetermined recorded image data15has been accumulated. In this embodiment, the image processor8searches only for the recorded image data15related to the position data16indicating a position, which is a predetermined distance (800 mm), behind the current position of the vehicle C in the direction of movement.

If the image processor8determines that the predetermined recorded image data15is not stored (Step S3-1=NO), the routine goes to Step S2-4and the rear monitor screen38is displayed. When the image processor8determines that the predetermined recorded image data15is stored (Step S3-1=YES), the recorded image data15is read out from the image memory7(Step S3-2). For example, the recorded image data15such as the diagram46ainFIG. 12Amay be read out. The read out image data G is not current data obtained at the current position (the position where the wheel stop101is located under the vehicle) but, rather, is data obtained before the vehicle arrived at the current position. Also, the read out image data15includes an image of the current blind spot of the camera20. More specifically, the recorded image data15may include a rough view of the area under the vehicle at the current position, which extends from the road surface under the rear bumper25to the center axis of the rear wheel CR of the current vehicle C.

After the recorded image data15is read out, the image processor8obtains the current image data18for at the current position of the vehicle C through the CPU2(Step S3-3). At this time, for example, the obtained current image data18may be like that of the diagram46bofFIG. 12B. The image processor8then processes the image data for correction of the obtained recorded image data15or the current image data18(Step S3-4). The image data processing may, for example, include correction of distortion because of the wide-angle lens of the camera20and the view change for the recorded image data15. In this example, only distortion correction is executed. Note that, based on the direction detection signal input from the gyro22or a rudder angle detection signal input from a steering sensor (not shown), the recorded image data15may be rotated in the direction of the current vehicle movement or be rotated with the rudder angle.

After such image processing, the image processor8outputs the parking assist image (Step S3-5). As shown inFIG. 12A, the image processor8abstracts a predetermined area47, within the display area9z, from the read out recorded image data15and generates abstracted data15a. In the present embodiment, the abstracted area47is a brachymorphic area located at the bottom of the display area9z. Further, as shown inFIG. 12B, the image processor8abstracts an area49, which has pre-specified coordinates, from the current image data18and generates abstracted current data18a. The abstracted area49is a brachymorphic area located at the bottom of the display area9z. As shown in the image48ofFIG. 12C, composite data50, which includes the current abstracted data18aon the top of the display area9zand the data15aabstracted from the recorded image data15on the bottom, is generated. The composite data50is output on the display9, along with various drawing data.

As a result, a parking assist image51is shown on the display9as shown inFIG. 11D. As the parking assist image51, a current image52based on the current image data18is displayed at the top of the display and an image revealing the area hidden as the blind spot53, based on the recorded image data15, is displayed at the bottom of the display. In the current image52, the background of the current position of vehicle C is displayed. In the image revealing the area hidden as blind spot53, an image revealing the hidden area (blind spot of the camera20) is displayed, including the road surface from the rear bumper29to the center of the rear wheel CR of the vehicle C, is displayed.

The image processor8also displays a band-like guidance display (messages)54between the current image52and the image revealing the area hidden as the blind spot53. The guidance display54is not only for splitting the screen between the current image52and the image revealing the area hidden by the blind spot53, but also displays guidance messages54to prompt a visual check around the vehicle. Further, the guidance display54includes an operational input section55for changing mode manually and an operational input section56for superimposing the guide line41on the current image52. By touch input at the operational section55, the mode displaying the parking assist image51is switched to the mode displaying the rear monitor screen38. By touch input at the operational section56, the guide line41is superimposed on the current image52.

The image processor8draws the additional line35, depending on the current position of the vehicle C, on the image revealing the area hidden as the blind spot53. For example, the image processor8allows the driver of the vehicle to determine whether the vehicle body is parked straight relative to the white line103or whether the vehicle body is beyond the parking area as defined by the white lines103, by providing a parking assist display showing the position of the framework line36relative to the image59of the white line103displayed as a parking assist image including the blind spot53.

When the recognized object is within the viewing area Z of the camera20, the background of the vehicle (view to the rear) is displayed fully so that the driver may know the position of the recognized object and the area around the object as well. When the recognized object disappears out of sight, the parking assist image51may be displayed, i.e. a virtual image showing the current position of the recognized object relative to the vehicle.

The CPU2outputs the parking assist image51and determines whether or not an end trigger has been received (Step S3-6). In the case where the end trigger has been received (Step S3-6=YES), execution of the routine is terminated. In the case where the end trigger has not been received (Step S3-6=NO), the routine returns to Step S2-2.

When the vehicle has moved to where the recognized object such as the wheel stop101is under the rear of the vehicle C, an image45of the wheel stop101is displayed in a virtual image revealing the area hidden as blind spot53, as shown inFIG. 13, thus enabling the driver to see the position the wheel stop101relative to the wheel37. Therefore, even if the recognized object disappears from the viewing area of the camera20, the position of the object may be continuously known utilizing the recorded image data15, to output a parking image including the blind spot. The driver may anticipate when the rear wheel CR will hit the wheel stop101, based on the relative position between the image45and the wheel37(here, the additional line35), and may park the vehicle in the proper position, with the rear wheel CR against the wheel stop101. Further the driver may slow the vehicle to cushion the shock of bumping into the wheel stop101.

As shown inFIG. 13, when the image59of the white line103ais displayed, revealing the area hidden as the blind spot53, the relative distance between the rear wheel CR and the white line103amay be recognized, so that the driver may park with the vehicle C fully within the marked parking space102.

When an obstacle104is located ahead of the wheel stop101and/or the white line103aduring parking, and/or when either the wheel stop101or the white line103ais not detected, only the obstacle104being detected, the image processor8tracks the edge EG of the obstacle104and determines whether the edge EG is included in the area BA behind the vehicle. As shown inFIGS. 14A and 14B, when the image57of the obstacle104is not completely within the area BA behind the vehicle, the image processor8displays the rear monitor screen38ofFIGS. 14aand14B. As shown inFIG. 14C, the image processor8waits until the image57(the edge EG) of the obstacle104completely within the area BA behind the vehicle before displaying the parking assist image51. As the vehicle C moves further in reverse, the parking assist image51inFIG. 14Dis displayed. The image revealing the area hidden as the blind spot53, includes an image57of the obstacle104. Therefore, the position of the wheel image37relative the image57of the obstacle104can be seen and the driver may steer the vehicle C (the rear wheel CR of the vehicle C) to avoid bumping into the obstacle104.

When the parking operation is terminated and when a shift position other than “reverse” is chosen, the CPU2determines that an end trigger, based on the shift position signal input from the neutral start switch24, has been received and terminates the display of both of the rear monitor (current or “real time” rear view) screen38and the parking assist screen51.

The embodiment described above provides the following advantages.

1) The image processor8determines when to display the parking assist image51on the basis of the relative position of the recognized object within the area BA behind the vehicle. In the case that the recognized object is within the viewing area of the camera20, the rear monitor screen38displays the area behind of the vehicle C without narrowing the view. Thus, the driver may know the position of a recognized object relative to the vehicle C by viewing the rear monitor screen38. When the recognized object goes under the bumper and out of the viewing area from the camera20, the rear monitor screen38displays the parking assist image51. More specifically, when the object which the driver wants to check goes is no longer in sight or viewable in the current image on the rear monitor, the recorded image data15including the hidden area where the object is located is displayed, so that the driver may continuously check the position of the object. Further, along with the parking assist image51, the current image52may also be displayed (concurrently displayed).

2) The image processor8displays the parking assist image51when it has been determined that the edge EG corresponding to the recognized object has completely entered the area behind the vehicle BA. Therefore, while at least a part of the recognized object is within the viewing area Z of the camera20, the current image from the camera20may be displayed on the rear monitor screen38. The driver may check the relative position between the rear of the vehicle C and the recognized object and also a wide view angle view of the current conditions around the vehicle.

The current image52and the image revealing the area hidden as the blind spot53may be displayed one above the other on the display area9zwhen the vehicle C is moving in reverse (x-axis on the screen coordinate system). At this time, the image revealing the area hidden as the blind spot53is displayed so that the direction of movement in the image is opposite that of the vehicle C. That is, in accordance with the movement of the vehicle C in reverse, the image revealing the area hidden as the blind spot53is displayed so that the direction of movement in the image is the same as that of the recognized object within the display area9z(the opposite direction on the x-axis). Therefore, even if the recognized object disappears from sight within the display area9z, the image revealing the area hidden as the blind spot53may be displayed so that the direction of movement in the image corresponds to the direction of movement of the object relative to the vehicle so that the driver may intuitively know the real-time image to which the image revealing the area hidden as the blind spot53corresponds.

4) According to the present embodiment, the image processor8automatically detects the wheel stop101or the white line103aindicating the end of the parking space102by image data processing. Therefore, the driver does not need to specify the object to be recognized manually, thus improving convenience to the driver.

5) The image processor8detects the obstacle104such as a can or a stone on the road surface and subsequently displays the parking assist image51when the obstacle104is located under the vehicle body. Thus, the displayed parking assist image51enables the driver to know the relative position and distance between the rear wheel CR and the obstacle104and the relative position and distance between the rear of the vehicle and the white line103a.

The embodiment described above may be modified as follows.

1) While in the embodiment as described above, the CPU2of the parking assist apparatus1stores the image data G as the recorded image data15every time the vehicle C moves a predetermined distance in reverse, in the alternative, the CPU2may obtain the image data G at regular time intervals.

2) The image processor8may determine whether the obstacle104is a non-movable body or a movable body. This determination may be made, for example, by detecting the difference between two sets of image data G captured at different points in time. If the image processor8determines that the obstacle104is a movable body, the display mode does not need to be changed to display the parking assist image51.

3) While in the above described embodiment the parking assist image51is displayed when the edge EG corresponding to the recognized object has completely entered the area BA behind the vehicle, the parking assist image51does not necessarily need to be displayed at such a time and may be displayed at some other time. For example, as shown inFIG. 11B, the parking assist image51may be displayed when a part of the image45(the edge EG) of the recognized object such as the wheel stop101has entered the area BA behind the vehicle BA or when the relative distance between the image45(the edge EG) of the recognized object and the area BA behind the vehicle has reached a predetermined length in terms of the screen coordinate system.

4) While in the above-described embodiment the parking assist image51is displayed when the image (the edge EG) corresponding to the recognized object such as the wheel stop101enters the area BA behind the vehicle, the parking assist image51may be displayed when the recognized object is no longer within the field of view of the camera20, regardless of whether the object is within or outside of the area BA behind the vehicle. For example, the parking assist image51may be displayed when the image (the edge EG) corresponding to the recognized object moves out of the display area9zwith movement of the vehicle C in reverse. Then the blind spot image including the recognized object may be used as the parking assist image51.

5) While the foregoing embodiment describes the image processor8as detecting the wheel stop101, the white line103a, and/or the obstacle104, the image processor may detect other objects. For example, any object indicating the end and/or side of the parking space102such as, for example, a wall around the parking space102, a guardrail, a cone located on the road or roadside, a fence, and/or a barricade, may be detected.

6) In the above-described embodiment the image processor8displays the parking assist image51when the wheel stop101or the white line103aindicating the end of the parking space102goes out of sight. However, the parking assist image51may be displayed when a white line other than the white line103aexits the field of view of the camera20. For example, the parking assist image51may be displayed when the white line indicating the side of the parking space102or the starting point of the white line103in the shape of a “U” indicating the parking space102goes out of the field of view of the camera20.

7) In the above-described embodiment, the image processor8recognizes the object such as the wheel stop101and displays the parking assist image51when the recognized object has completely entered the area BA behind the vehicle. However, the user may specify the object to be recognized by touch (manual) operation on the display9. More specifically, as in the screen60ofFIG. 15, the user may specify some points on the display9(touch panel) to cause the operating section62to set a small area61such as, for example, a triangle, a rectangle or a polygon, which has the specified points as vertexes. Also the user may specify only one point and a small area61may be set as a circle which has the specified point as the center. The small area61thus set may be tracked based on the speed pulse and the direction of the vehicle C, in accordance with movement of the vehicle C in reverse, so that the parking assist image51is displayed when the small area61has completely entered the area BA behind the vehicle. In this case, the user need specify only the object which he/she wants to track.

8) While in the foregoing embodiment, the current image52and the image revealing the area hidden as the blind spot53are displayed one above the other on the display area9z, as shown in the image66ofFIG. 16, the current image52and the image revealing the area hidden as the blind spot53may be displayed side by side.