Patent Description:
In the conventional art, a car navigation system, an automatic driving system, and the like are widely known as a system using a host vehicle position on a map. A global positioning system (GPS) is known as technology which is used for the systems.

In the convention system, since a host vehicle position on a map has an error of about several meters, for example, there is a problem in that an intersection in which the vehicle has to make a turn is mistaken or a vehicle travels on a lane which does not guide the vehicle to a destination.

In order to solve such a problem, various techniques for estimating a host vehicle position on a map with higher accuracy have been studied. For example, PTL <NUM> discloses a technique of recognizing a landmark (a landmark which is a sign having a predetermined feature) present on a map from images photographed using a camera and estimating a host vehicle position on the map.

A self-position estimating device disclosed in PTL <NUM> includes landmark detecting means that detects a landmark, reliability acquiring means that acquires landmark reliability which is reliability of the landmark detected by the landmark detecting means, and self-position estimating means that estimates a self-position on the basis of a plurality of landmarks detected by the landmark detecting means, and the self-position estimating means sequentially selects the landmarks to be used to estimate the self-position from the landmark having the highest landmark reliability.

According to the self-position estimating device disclosed in PTL <NUM>, when a plurality of landmarks are extracted, it is possible to accurately recognize different landmarks and to estimate a self-position with high accuracy.

However, in the conventional technique disclosed in PTL <NUM>, when no referable landmark is present in an image photographed using a camera, there is a problem in that a host vehicle position is not estimated and stability of the device is not satisfactory.

The invention is made in consideration of the above-mentioned problem and an object thereof is to provide a host vehicle position estimation device that can accurately estimate a host vehicle position on a map even when no referable landmark is present in an image photographed using a camera. Solution to Problem.

The problem mentioned above is solved by a host vehicle position estimation device as described by the appended claims.

In order to solve the above issue, a host vehicle position estimation device according to the present invention is set out in claim <NUM>.

According to the invention, by estimating a host vehicle position on the basis of the intermediate information on a name of a landmark as an intermediate object extracted from an image acquired by photographing the surroundings of the host vehicle and a distance to the landmark, the distance information on a distance to the intermediate object, and the landmark information on names and positions of landmarks which is stored in advance, it is possible to accurately estimate a host vehicle position on a map, for example, even when no referable landmark is present in an image photographed using a camera.

Objects, configurations, and advantageous effects other than described above will be apparent from the following description of embodiments.

Hereinafter, embodiments of a host vehicle position estimation device according to the invention will be described with reference to the accompanying drawings.

<FIG> is a block diagram illustrating an internal configuration of a first embodiment of a host vehicle position estimation device according to the invention.

A host vehicle position estimation device <NUM> illustrated in the drawing is mounted on a vehicle such as an automobile and mainly includes a stereo camera unit <NUM>, a host vehicle position estimating unit <NUM>, a host vehicle position correcting unit <NUM>, and a communication unit <NUM>. The units are connected to each other in an accessible manner. A landmark database <NUM> as a storage unit in which information on types (names) of landmarks and positions thereof on a map (landmark information) is stored in advance is disposed outside the host vehicle position estimation device <NUM>, and the host vehicle position estimation device <NUM> is connected to the landmark database <NUM> via the communication unit <NUM> in an accessible manner.

The stereo camera unit <NUM> constituting the host vehicle position estimation device <NUM> is provided with a plurality of (two in this embodiment) cameras <NUM> and <NUM>, and the stereo camera unit <NUM> performs processes of preparing or recognizing a distance image using images acquired by the cameras <NUM> and <NUM> and outputs the results to the communication unit <NUM> and the host vehicle position estimating unit <NUM>. A communication standard such as a controller area network (CAN) can be applied to connection of the stereo camera unit <NUM> to other units.

The communication unit <NUM> serves to exchange data with the landmark database <NUM>, receives data stored in the landmark database <NUM>, and transmits the received data to the host vehicle position estimating unit <NUM>.

The host vehicle position estimating unit <NUM> estimates a position of the host vehicle on a map using the results acquired from the stereo camera unit <NUM> and the data acquired from the landmark database <NUM> via the communication unit <NUM>, and outputs the estimation result to the host vehicle position correcting unit <NUM>.

The host vehicle position correcting unit <NUM> corrects the host vehicle position on a map acquired by a global positioning system (GPS) using the estimation result transmitted from the host vehicle position estimating unit <NUM>.

<FIG> illustrates an internal configuration of the stereo camera unit <NUM> illustrated in <FIG>.

The stereo camera unit <NUM> serves to recognize an environment around the host vehicle on the basis of image information of an imaging target area on the front side of the vehicle provided with the host vehicle position estimation device <NUM> and, as illustrated in <FIG>, mainly includes a pair of cameras <NUM> and <NUM> arranged in a vehicle width direction (a lateral direction) of the host vehicle to face the front side of the vehicle, an image acquiring unit <NUM>, a distance image preparing unit (distance information acquiring unit) <NUM>, a landmark extracting unit <NUM>, an intermediate object extracting unit <NUM>, and a bus <NUM> capable of accessing images acquired by the two cameras <NUM> and <NUM>.

The image acquiring unit <NUM> of the stereo camera unit <NUM> acquires images which are synchronously photographed using two cameras <NUM> and <NUM> periodically or at an arbitrary timing, and the distance image preparing unit <NUM> prepares an image (a distance image) having depth information (distance information) using the images (synchronous images) acquired by the image acquiring unit <NUM>. The method of preparing the distance image is known well and thus detailed description thereof will not be made herein.

The landmark extracting unit <NUM> extracts a landmark from the images using the image acquired by the distance image preparing unit <NUM> and the images acquired by the two cameras <NUM> and <NUM>, and outputs information on the extracted landmark to the host vehicle position estimating unit <NUM> and the communication unit <NUM>. Here, a landmark in the invention means, for example, an object which can serve as a sign having a predetermined feature among objects installed on a road and examples thereof include a signboard, a speed sign, a signal, and a building. As the method of extracting a landmark, template matching or the like can be used.

Specifically, as illustrated in <FIG>, the landmark extracting unit <NUM> includes a landmark recognizing unit <NUM>, a landmark transverse position information acquiring unit <NUM>, and a landmark distance information acquiring unit <NUM>.

The landmark recognizing unit <NUM> extracts the above-mentioned landmark from the images and specifies a type (name) of an object which is acquired as the landmark. The landmark transverse position information acquiring unit <NUM> acquires a transverse position of the extracted landmark with respect to the host vehicle. Here, a transverse position means a length of a perpendicular line (that is, a distance in a transverse direction perpendicular to the traveling direction of the host vehicle) when a perpendicular line is drawn from the position of the extracted landmark to a straight line extending in the traveling direction of the host vehicle (an photographing direction) from the substantial center of the host vehicle (for example, see L1 and L2 in <FIG> or L3 in <FIG>). In general, a resolution of the transverse position is considered to be more excellent in a stereo camera including a plurality of cameras than in radar and a more accurate transverse position can be acquired using the stereo camera as in this embodiment. The landmark distance information acquiring unit <NUM> acquires a straight-line distance from the host vehicle to the landmark using the transverse position acquired by the landmark transverse position information acquiring unit <NUM> and the image acquired by the distance image preparing unit <NUM> (for example, see D1 and D2 in <FIG> or D3 in <FIG>).

The landmark extracting unit <NUM> outputs information on the landmark acquired by the landmark recognizing unit <NUM>, the landmark transverse position information acquiring unit <NUM>, or the landmark distance information acquiring unit <NUM> to the host vehicle position estimating unit <NUM> and the communication unit <NUM>.

Similarly to the landmark extracting unit <NUM>, the intermediate object extracting unit <NUM> extracts an intermediate object from the images and outputs information on the extracted intermediate object to the host vehicle position estimating unit <NUM> and the communication unit <NUM>. Here, an intermediate object in the invention means an object having information (intermediate information) on the name of the landmark among objects around the host vehicle and a distance to the landmark, and examples thereof include a guide sign of a tollgate of an expressway, a guide display board to a restaurant or a theme park, and an exit, a diverging point, or a junction of an expressway.

Specifically, as illustrated in <FIG>, the intermediate object extracting unit <NUM> includes an intermediate object recognizing unit <NUM>, an intermediate object transverse position information acquiring unit <NUM>, and an intermediate object distance information acquiring unit <NUM>.

The intermediate object recognizing unit <NUM> extracts the above-mentioned intermediate object from the images and recognizes a name of a landmark included in the intermediate object and a distance to the landmark (for example, as character information). The intermediate object transverse position information acquiring unit <NUM> acquires a transverse position of the extracted intermediate object with respect to the host vehicle (for example, see L4 in <FIG>), and the intermediate object distance information acquiring unit <NUM> acquires a straight-line distance from the host vehicle to the intermediate object using the transverse position acquired by the intermediate object transverse position information acquiring unit <NUM> and the image acquired by the distance image preparing unit <NUM> (for example, see D4 in <FIG>).

The intermediate object extracting unit <NUM> outputs information on the intermediate object acquired by the intermediate object recognizing unit <NUM>, the intermediate object transverse position information acquiring unit <NUM>, or the intermediate object distance information acquiring unit <NUM> to the host vehicle position estimating unit <NUM> and the communication unit <NUM>.

As illustrated in <FIG>, the communication unit <NUM> compares the information on the landmark (particularly, a type of an object acquired as the landmark) transmitted from the landmark extracting unit <NUM>, information on the intermediate object (particularly, a name of a landmark included in the intermediate object) transmitted from the intermediate object extracting unit <NUM>, and information (particularly, a name of the landmark) acquired from the landmark database <NUM>, acquires the position of the landmark on a map or the position of the landmark indicated by the intermediate information of the intermediate object (specifically, the landmark having a name coinciding with the name of the landmark included in the intermediate information) on the map, and transmits the acquired position to the host vehicle position estimating unit <NUM>.

The host vehicle position estimating unit <NUM> estimates the position of the host vehicle on the map on the basis of the position of the landmark on the map acquired via the communication unit <NUM> and the information on the distance of the landmark acquired from (the landmark extracting unit <NUM> of) the stereo camera unit <NUM>, or estimates the host vehicle position on the map on the basis of the position of the landmark on the map indicated by the intermediate information of the intermediate object acquired via the communication unit <NUM> and the information on the distance of the intermediate object acquired from (the intermediate object extracting unit <NUM> of) the stereo camera unit <NUM>.

<FIG> illustrates a process flow which is performed by the host vehicle position estimation device <NUM> illustrated in <FIG>. The gist of the invention is to acquire a transverse position of the host vehicle and measure a distance (a straight-line distance) on the basis of the information acquired from the cameras <NUM> and <NUM> (the stereo camera) in a landmark extracting process which is performed by the landmark extracting unit <NUM> (S501) and an intermediate object extracting process which is performed by the intermediate object extracting unit <NUM> (S701) and to measure a distance (straight-line distance), to cause the communication unit <NUM> to perform a database comparing process (S901) and then to perform a landmark map-position information acquiring process (S1301) and to perform a host vehicle position estimating process (S1701) when the extracted landmark is registered in the landmark database <NUM>, and to perform the host vehicle position estimating process (S1701) using the position information of the intermediate object acquired in an intermediate object map-position information acquiring process (S1501) when the extracted landmark is not registered in the landmark database <NUM>.

Specifically, as illustrated in <FIG>, the images acquired by the two cameras <NUM> and <NUM> are input (S101) and a distance image is prepared using the two images as an input (S301).

Then, a landmark is extracted from the images acquired in S101 and S301 (S501). <FIG> illustrates an example of a landmark in one image of the images acquired by the two cameras <NUM> and <NUM>. Since an object and the like installed on a road is extracted as a landmark, a traffic signal LM1 and a speed sign LM2 are extracted as a landmark in the example illustrated in <FIG>.

Similarly to S501, an intermediate object is extracted from the images acquired in S101 and S301 (S701). <FIG> illustrates an example of a landmark and an intermediate object in one image of the images acquired by the two cameras <NUM> and <NUM>. <FIG> includes a speed sign LM3 as a landmark and a tollgate guide sign MM4 as an intermediate object in which a tollgate as a landmark name and <NUM> as a distance to the tollgate are drawn. Accordingly, in the example illustrated in <FIG>, the tollgate guide sign MM4 is extracted and the tollgate as the landmark name and <NUM> as the distance to the tollgate are acquired as character information in S701.

S501 and S701 may be performed in a reversed order in the order illustrated in <FIG>, or may be performed at the same time (that is, in the same extraction process).

Subsequently, the landmark acquired in S501 is compared with the database (S901), and it is checked whether the landmark is registered in the landmark database <NUM> (S1101).

When the landmark is registered in the landmark database <NUM>, the position of the landmark on a map is acquired from the landmark database <NUM> (S1301), and the position of the host vehicle V on the map is estimated (S1701).

<FIG> illustrates a positional relationship between the landmarks in the image illustrated in <FIG> and the host vehicle when viewed from the upside. In <FIG>, lengths L1 and L2 of perpendicular lines drawn from the traffic signal LM1 and the speed sign LM2 to a dotted line C1 drawn in the traveling direction from the substantial center of the host vehicle V indicate transverse positions of the landmarks with respect to the host vehicle which are acquired on the basis of the information acquired from the cameras <NUM> and <NUM> (the stereo camera), and lengths D1 and D2 of straight lines drawn from the host vehicle V to the landmarks indicate distances between the host vehicle V and the landmarks. Since the resolution of the transverse position of the stereo camera is higher than that of the radar as described above, the distances acquired using the transverse positions can be obtained more accurately by using the stereo camera. Since the positions of the traffic signal LM1 and the speed sign LM2 on the map are acquired from the landmark database <NUM>, the position of the host vehicle V on the map can be estimated using the positions of the landmarks on the map, the transverse positions with respect to the host vehicle V, and the distances.

In the examples illustrated in <FIG> and <FIG>, two landmarks appear, but the host vehicle position can be estimated using only one landmark. Here, when a plurality of landmarks are recognized as illustrated in <FIG>, probabilities (scores) of types of the landmarks may be used to recognize the landmarks and the landmark having the greatest score may be used to estimate the host vehicle position. Alternatively, a plurality of estimated positions may be output using the plurality of landmarks and an average position thereof may be determined to be the final estimated position, or the final estimated position may be determined by performing weighting based on the scores.

On the other hand, when it is determined as the database comparison (S901) that the landmark acquired in S501 is not registered in the landmark database <NUM>, the position of the landmark described in the intermediate information of the intermediate object acquired in S701 on the map is acquired (S1401), the position of the intermediate object on the map is acquired (specified) using the acquisition result (S1501), and the position of the host vehicle V on the map is estimated (S1701).

<FIG> illustrates a positional relationship between an intermediate object and the host vehicle in the image illustrated in <FIG> when viewed from the upside, and specifically illustrates a positional relationship between a tollgate LM5, a speed sign LM3, a tollgate guide sign MM4, and the host vehicle V when viewed from the upside. The tollgate LM5 does not appear in the images photographed using the cameras <NUM> and <NUM>. In the example illustrated in <FIG>, the speed sign LM3 is extracted as a landmark in the landmark extracting process (S501) and the tollgate guide sign MM4 is extracted as an intermediate object in the intermediate object extracting process (S701), but when it is determined as the landmark comparison result that the speed sign LM3 is not registered in the landmark database <NUM>, the host vehicle position on the map is estimated using the tollgate guide sign MM4. More specifically, since information (the intermediate information) on the landmark name (the tollgate) and the distance (<NUM>) thereto in the intermediate object extracting process (S701) is acquired from the tollgate guide sign MM4, the position of the tollgate LM5 described in the tollgate guide sign MM4 on the map is acquired from the landmark database <NUM> (S1401) and the position of the tollgate guide sign MM4 on the map is estimated from the position of the tollgate LM5 on the map and the distance (<NUM>) to the tollgate described in the tollgate guide sign MM4 (S1501). Subsequently, the position of the host vehicle V on the map is estimated from the acquired position of the tollgate guide sign MM4 on the map and a distance D4 of the transverse position L4 of the tollgate guide sign MM4 with respect to the host vehicle V which is acquired using the cameras <NUM> and <NUM> (the stereo camera) (S1701).

When the host vehicle position is acquired by a GPS or the like, the host vehicle position on the map acquired by the GPS or the like is corrected using the estimated position acquired in S1701 (S1901).

In this way, the host vehicle position estimation device <NUM> according to the first embodiment can accurately estimate a host vehicle position on a map, for example, even when a referable landmark is not present in an image photographed using a camera, by estimating the host vehicle position on the basis of the intermediate information on a name of a landmark of an intermediate object and a distance to the landmark which is extracted from an image acquired by photographing surroundings of the vehicle, distance information on a distance (a transverse position and a straight-line distance) to the intermediate object, and landmark information on a name and a position of the landmark stored in advance in the landmark database <NUM>.

Since more accurate position or distance of a landmark or an intermediate object can be acquired by acquiring a transverse position of the landmark or the intermediate object with respect to the host vehicle on the basis of the information acquired from the stereo camera (the cameras <NUM> and <NUM>) and measuring a distance thereto (a straight-line distance), it is possible to more accurately estimate a host vehicle position on a map.

<FIG> illustrates an internal configuration of a second embodiment of the host vehicle position estimation device according to the invention.

The functions of the units of a host vehicle position estimation device 101A according to the second embodiment are the same as the functions of the units of the host vehicle position estimation device <NUM> according to the first embodiment, except that an input and output unit 801A (a unit that exchanges data with a landmark database 901A) corresponding to a host vehicle position estimating unit 501A and a communication unit <NUM> is integrated in a stereo camera unit 301A in the second embodiment.

In this way, in the host vehicle position estimation device 101A according to the second embodiment, the entire device configuration can be simplified by integrating the functions in the stereo camera unit 301A.

In the first and second embodiments, a distance to an object appearing in an image is acquired using a plurality of images which are synchronously photographed using two cameras in order to secure estimation accuracy, but the distance to the object appearing in an image may be acquired using a plurality of images which are photographed using a single camera (a monocular camera) in a time series in order to simplify the device configuration. For example, a radar as a distance information acquiring unit may be mounted on the vehicle and a distance (a straight-line distance) to an object appearing in an image may be acquired on the basis of information acquired from the radar.

In the first and second embodiments, the communication unit <NUM> or the input and output unit 801A performs the process of comparing the information on a landmark transmitted from the landmark extracting unit <NUM> or 309A or the information on an intermediate object transmitted from the intermediate object extracting unit <NUM> or 311A with the information acquired from the landmark database <NUM> or 901A, but for example, the communication unit <NUM> or the input and output unit 801A may acquire arbitrary information from the landmark database <NUM> or 901A periodically or at an appropriate timing and transmit the acquired information to the host vehicle position estimating unit <NUM> or 501A and the host vehicle position estimating unit <NUM> or 501A may perform the process of comparing the information on a landmark transmitted from the landmark extracting unit <NUM> or 309A or the information on an intermediate object transmitted from the intermediate object extracting unit <NUM> or 311A with the information acquired from the landmark database <NUM> or 901A.

In the first and second embodiments, the cameras are disposed to face the front side of the vehicle and the host vehicle position on a map is estimated on the basis of image information of an imaging target area on the front side of the vehicle, but the host vehicle position on a map may be estimated on the basis of image information of an imaging target area on the rear side of the vehicle, for example, using cameras disposed to face the rear side of the vehicle.

Claim 1:
A host vehicle position estimation device that estimates a position of a host vehicle using landmark information on names and positions of landmarks which is stored in advance, the host vehicle position estimation device (<NUM>) comprising:
an image acquiring unit (<NUM>) configured to acquire an image by photographing surroundings of the host vehicle,
a distance information acquiring unit (<NUM>) configured to acquire distance information on a distance in a travelling direction of the host vehicle to any object appearing in the image or a straight-line distance to any object;
an intermediate object extracting unit (<NUM>) configured to extract an intermediate object, wherein
the intermediate object is an object having intermediate information on a name of a landmark and a distance to the landmark, from the image and to acquire distance information on a distance to the intermediate object on the basis of the distance information acquired by the distance information acquiring unit (<NUM>), wherein
the distance information on the distance to the intermediate object includes a distance in a transverse direction perpendicular to the travelling direction of the host vehicle to the intermediate object and the distance in the travelling direction of the host vehicle to the intermediate object or the distance in the transverse direction perpendicular to the travelling direction of the host vehicle to the intermediate object and a straight-line distance to the intermediate object; and
a host vehicle position estimating unit (<NUM>) configured to estimate the host vehicle position on the basis of the intermediate information of the intermediate object and the distance information on the distance to the intermediate object which are extracted by the intermediate object extracting unit (<NUM>) and the landmark information.