Patent Description:
Technology to detect an object around an own vehicle and an object traveling around the own vehicle and to estimate the environment around the vehicle is being developed. For example, PTL <NUM> discloses a technique in which observation data from a sensor is sampled and a distribution function of the angle representing the degree of dispersion centered on the azimuth angle from the intensity of the data is calculated, so that the distribution state of the reflection caused by the object over a plurality of grids in the occupied grid map is calculated and the state of the surrounding object is estimated with high accuracy. Citation List.

<CIT> discloses a road boundary detection system in which an occupancy map is the basis for operating a coverage grid separator, a road boundary candidate detector and a road boundary detector.

<CIT> discloses a road structure detection and tracking system in which input images are captured and transformed for performing edge analysis across time-consecutive frames.

<CIT> discloses a map generation device in which a map is continuously updated in accordance with information from an object position detecting means.

<CIT> discloses a driving support device that detects from a captured image a dividing line of the traveling road surface and sets virtual dividing lines where the partitioning line cannot be detected.

<CIT> discloses a steering assistance device that sets a lane width between lane markers.

In the related art of PTL <NUM>, the occupied grid map in consideration of affecting the plurality of grids is created according to the degree of the intensity obtained from the observed data, thereby clarifying the contrast between the occupied region and the unoccupied region of the object and more easily estimating the region where the vehicle can travel. However, there is a problem that the clarity in the position information of the detected object, which is the boundary of the region where the vehicle can travel, is lowered because the occupied region of the object is spread due to dispersion. That is, there is a problem that the traveling region of the vehicle must be set in a narrow area, avoiding an uncertain region with low clarity.

The object of the invention is to provide an in-vehicle control device capable of more clearly and easily determining a region in which a vehicle can travel.

Solution to ProblemThe above object is accomplished by the features of claim <NUM>. An in-vehicle control device includes the features of claim <NUM>, amongst them an object detection unit that detects a position of an object in front of an own vehicle from image information captured by an image pickup device, an object information storage unit that stores a pre-processing grid map in which a position where the object detection unit has detected an object is set as an object occupied region and a position where no object has been detected is as an object unoccupied region, an information processing unit that generates a determination grid map in which a part of the object unoccupied region of the pre-processing grid map is replaced with the object occupied region, and a road surface region determination unit that generates an automatic driving grid map in which a closed space surrounded by the object occupied region of the determination grid map is set as a road surface region. Claim <NUM> recites further features.

In an in-vehicle control device according to the invention, a road surface region in which an own vehicle travels is extracted by using a determination grid map in which an object occupied region has been complemented. Therefore, it is possible to improve the road surface region determination accuracy and reduce the processing load.

Objects, configurations, and effects besides the above description will be apparent through the description of the following embodiments.

Hereinafter, an in-vehicle control device according to the embodiments of the present invention will be described with reference to the drawings and the like. Although the following description illustrates specific examples of the content of the invention, the invention is not limited to the description. Various changes and modifications can be made, by those skilled in the art, within the scope of the technical idea disclosed herein. In all the drawings for describing the invention, components having the same function are designated by the same reference numeral, and the repeated description thereof may be omitted.

An in-vehicle control device <NUM> according to a first embodiment of the invention will be described with reference to <FIG>. The in-vehicle control device <NUM> of this embodiment is called a surrounding environment estimation ECU (Electronic Control Unit) that estimates the road shape around an own vehicle <NUM> and the surrounding environment such as other vehicles, and provides surrounding environment information (specifically, various grid maps to be described later) necessary for executing automatic driving to a driving control ECU (hereinafter, referred to as the "control unit <NUM>") that controls acceleration, steering, braking, and the like of the own vehicle <NUM>. Although <FIG> illustrates a configuration in which the in-vehicle control device <NUM> (surrounding environment estimation ECU) and the control unit <NUM> (driving control ECU) are separated, the two may be integrated.

<FIG> is a functional block diagram of the in-vehicle control device <NUM>. As illustrated in the drawing, the in-vehicle control device <NUM> generates a grid map based on image information captured by an image pickup device <NUM> and provides the map to the control unit <NUM> (driving control ECU). The in-vehicle control device <NUM> is actually a computer including hardware such as an arithmetic device (e.g., a CPU), a main storage device (e.g., a semiconductor memory), an auxiliary storage device (e.g., a hard disk), and a communication device. Then, each function to be described later is performed by the arithmetic device executing a program loaded in the main storage device and, description of such well-known techniques may be omitted, as appropriate, in the following description.

As illustrated in <FIG>, the in-vehicle control device <NUM> includes an object detection unit <NUM>, an object information storage unit <NUM>, a pre-processing grid map <NUM>, an information processing unit <NUM>, a determination grid map 6a, an automatic driving grid map 6b, and a road surface region determination unit <NUM>. Of these, the object detection unit <NUM>, the information processing unit <NUM>, and the road surface region determination unit <NUM> are realized by the above-mentioned arithmetic device executing a predetermined program. Further, the object information storage unit <NUM> is the auxiliary storage device described above, and grid maps, namely, the pre-processing grid map <NUM>, the determination grid map 6a, and the automatic driving grid map 6b are also stored in this auxiliary storage device. Each of the above components will be described in detail below.

The object detection unit <NUM> detects an object position around the own vehicle <NUM> based on the image information captured by the image pickup device <NUM>. The object information storage unit <NUM> stores the position information of the object detected by the object detection unit <NUM> in chronological order. The object information stored in the object information storage unit <NUM> is not limited to the above-mentioned position information, and may include, for example, relative speed information of the object, attribute information such as the color and type of the object, and the like. The object information storage unit <NUM> generates the pre-processing grid map <NUM> by updating the information of each cell of the grid map. Although the size of each cell on the grid map may be determined as appropriate, the following description will proceed assuming a cell with a side of one meter.

An example of the pre-processing grid map <NUM> will be described with reference to <FIG>. The pre-processing grid map <NUM> illustrated in the drawing is a grid map showing the environment in front of the own vehicle <NUM>, and the cell in which the object detection unit <NUM> has detected the object (hereinafter, "object occupied region <NUM>") is colored. In the example of <FIG>, the object occupied regions <NUM> are lined up in a straight line on the left and right sides of the own vehicle <NUM>, and there are also a plurality of cells in which the object detection unit <NUM> has not detected the object on the same straight line (hereinafter, "object unoccupied region <NUM>"). The linear object occupied regions <NUM> on the left and right of the own vehicle <NUM> are specifically structures such as sidewalk curbs, guardrails, guard poles, and fences, and an object unoccupied region 22a on the same straight line corresponds to a place where there are no curbs or fences, such as the entrance of a house.

When the pre-processing grid map <NUM> is updated, the information processing unit <NUM> generates the determination grid map 6a for determining a road surface region <NUM> on which the own vehicle <NUM> can travel. The determination grid map 6a facilitates the extraction of a closed space (that is, the road surface region <NUM> to be described later) by smoothing and simplifying the pre-processing grid map <NUM>.

An example of the determination grid map 6a will be described with reference to <FIG>. The determination grid maps 6a illustrated in the drawing is a grid map in which a plurality of object unoccupied regions <NUM> are continuously aggregated by replacing those satisfying a predetermined condition (object unoccupied region 22a) among the object unoccupied regions <NUM> in the pre-processing grid map <NUM> with the object occupied region <NUM>.

Next, the information processing unit <NUM> that generates the determination grid map 6a based on the pre-processing grid map <NUM> will be described in more detail with reference to <FIG>. As illustrated in the drawing, the information processing unit <NUM> includes an object region extraction unit 5a, an object region distance calculation unit 5b, and an adjacent object region complementing unit 5c. The object region extraction unit 5a extracts a continuous part of the object occupied region <NUM> from the pre-processing grid map <NUM>. In the inter-object region distance calculation unit 5b, when there is a plurality of continuous parts of the object occupied region <NUM>, each the distance between the continuous parts (the continuous distance of the object unoccupied region <NUM> sandwiched between the continuous parts of the object occupied region <NUM>) is calculated. In the adjacent object region complementing unit 5c, when the distance calculated by the object region distance calculation unit 5b is equal to or less than a threshold (for example, two cells or less), both continuous parts are regarded as a continuous object occupied region <NUM>. The adjacent object region complementing unit 5c then generates the determination grid map 6a in which the object unoccupied region 22a (for example, a gap of two cells or less) sandwiched between both continuous parts is replaced with the object occupied region <NUM>.

In this way, the reason for replacing the object unoccupied region 22a with a short continuous distance with the object occupied region <NUM> is that even if there is a gap in a part of the structure such as a fence, if the gap is narrow, the own vehicle <NUM> cannot pass through, and thus the structure is regarded as being continuous and the determination grid map 6a is used which is complemented to keep the continuity of the object occupied region <NUM> as much as possible so as to extract the road surface region <NUM> where the own vehicle <NUM> travels, which is effective in improving the extraction accuracy of the road surface region <NUM> and reducing the extraction processing load.

In the above, an example in which the distance threshold when replacing the object unoccupied region <NUM> with the object occupied region <NUM> is set to a fixed value (two cells) has been described, but this distance threshold may be variable, and the distance threshold may increase as the speed of the own vehicle <NUM> increases. This is because even if the gap is the same (for example, a gap of three cells (three meters)), the own vehicle <NUM> can easily pass through at low speed; however, it is difficult to pass through at high speed when the turning radius is large. Thus, it is appropriate to generate the determination grid map 6a using a high-speed distance threshold at high speed.

When the determination grid map 6a is generated, the road surface region determination unit <NUM> determines the road surface region <NUM>, which is a region in which the own vehicle <NUM> can travel, based on the determination grid map 6a, and generates the automatic driving grid map 6b.

An example of the automatic driving grid map 6b will be described with reference to <FIG>. The automatic driving grid map 6b illustrated in the drawing is a grid map in which, among the object unoccupied regions <NUM> of the determination grid map 6a of <FIG>, those satisfying a predetermined condition are set in the road surface region <NUM>. In the road surface region determination unit <NUM> of this embodiment, a closed space in front of the own vehicle <NUM> and surrounded by the object occupied regions <NUM> in the object unoccupied region <NUM> of the determination grid map 6a is determined as the road surface region <NUM> where the own vehicle <NUM> can travel.

As described above, in the in-vehicle control device <NUM> of this embodiment, after the determination grid map 6a obtained by simplifying the pre-processing grid map <NUM> in which the actual object position is recorded is generated, the road surface region <NUM> in which the own vehicle <NUM> can travel is determined based on this determination grid map 6a, and the automatic driving grid map 6b is generated. Further, the control unit <NUM> plans and controls the traveling state of the own vehicle <NUM> according to the surrounding environment by using the pre-processing grid map <NUM> and the automatic driving grid map 6b.

As described above, in the in-vehicle control device of this embodiment, the road surface region in which the own vehicle travels is extracted by using the determination grid map that has been complemented so as to maintain the continuity of the object occupied region as much as possible. Therefore, it is possible to improve the road surface region determination accuracy and reduce the processing load. Further, it is possible to reduce the communication load for transmitting the road surface region shape to the control unit by optimizing the road surface region shape.

Next, the in-vehicle control device <NUM> according to a second embodiment of the invention will be described with reference to <FIG> and <FIG>. Description of some points in common with the first embodiment will be omitted.

In the first embodiment, it has been assumed that the object occupied region <NUM> is a curb or the like arranged in a straight line, and the determination grid map 6a is generated by using the information processing unit <NUM> of <FIG>. However, in the second embodiment, the determination grid map 6a is generated using the information processing unit <NUM> illustrated in <FIG> so that the object occupied region <NUM> not arranged in a straight line can be dealt with.

As illustrated in <FIG>, the information processing unit <NUM> of this embodiment includes the object region extraction unit 5a, an object region line segment calculation unit 5d, and an adjacent line segment complementing unit 5e. The object region extraction unit 5a extracts a block of the object occupied region <NUM> from the pre-processing grid map <NUM>. The object region line segment calculation unit 5d calculates a representative line segment L from each block when there is a plurality of blocks in the object occupied region <NUM>. The adjacent line segment complementing unit 5e complements the line segments using the calculated representative line segment L. The complement condition for the line segments is that the line segment distance in the adjacent object occupied regions <NUM> is equal to or less than a threshold and the angle formed by the adjacent line segments is equal to or less than a threshold. As a result, complementation is performed not only when the distance between the line segments is short, but also when these line segments, which are the boundary lines of the road surface region <NUM>, become smooth lines having a certain curvature or less. These thresholds may also be changed according to the own vehicle speed for the same reason as in the first embodiment.

Here, an example of a method of obtaining the representative line segment L from a block of object regions xi (i = <NUM> to N) composed of N cells by the object region line segment calculation unit 5d will be described using <FIG>. In the drawing, <NUM> indicates a block of object regions, and <NUM> indicates the representative line segment L. Assuming that the function representing the distance between the line segment L and the cell xi of the object region is d(L, xi), the evaluation formula EL of the line segment L is (Expression <NUM>), and the line segment L that minimizes this evaluation formula EL is the representative line segment of the object region. <NUM>] <MAT>.

As described above, in the in-vehicle control device of this embodiment, even in a situation where the cells in the object occupied region are not arranged in a straight line, the road surface region where the own vehicle travels is extracted using the determination grid map in which the line segment obtained from the object region is processed so as to be continuous. Therefore, it is possible to improve the road surface region determination accuracy and reduce the processing load.

Next, the in-vehicle control device <NUM> according to the third embodiment of the invention will be described with reference to <FIG>. Description of some points in common with the above-described embodiment will be omitted.

In this embodiment, when the information processing unit <NUM> processes information, the information of a high-precision map <NUM> prepared in advance is used together with the processing of the above-described embodiment.

With the progress of automatic driving technology in recent years, an environment in which high-precision maps can be easily used is being established, such as the commercialization of precise maps related to road shapes. Therefore, the information processing unit <NUM> of this embodiment acquires map information from the high-precision map <NUM> regarding the road shape around the own vehicle, and switches the distance threshold between the object occupied regions <NUM>, for determining whether the complementation is performed on the portion where the road boundary on the map is continuous, so as to be longer than usual. That is, even if the distance threshold is set to two cells, the distance threshold is set to four cells or the like in the place where the continuity of the road boundary can be confirmed by the high-precision map <NUM>, and the continuity of the object occupied region <NUM> is more easily maintained.

As a result, even when the object detection unit <NUM> fails to detect an object and the obtained object region information is unclear, a desired complementary processing is performed by using the high-precision map <NUM> together, and an appropriate determination grid map 6a can be obtained.

Next, the in-vehicle control device <NUM> according to a fourth embodiment of the invention will be described with reference to <FIG>. Description of some points in common with the above-described embodiment will be omitted.

In this embodiment, when the information processing unit <NUM> processes information, white line information of the road extracted from the image information captured by the image pickup device <NUM> is used together with the processing of the above-described embodiment. Then, in order to realize this, a white line extraction unit <NUM> is provided in the in-vehicle control device <NUM>.

Due to the high performance of information processing devices in recent years, it has become relatively easy to obtain information on the shape of white lines of the road by image processing. Therefore, when the information on the road shape around the own vehicle is acquired from the shape and position of the white line of the road, and the continuity of the traveling zone on the road can be confirmed, the distance threshold is switched to be longer than usual.

As a result, even when the object detection unit <NUM> fails to detect an object and the obtained object region information is unclear, a desired complementary processing is performed by using the white line information together, and the continuity of the object occupied region <NUM> is easily maintained. In addition to the white line, complementation may be performed in the same manner as the method described above by extracting features and edges on the image such as curbs, guardrails, guard poles, and fences.

Next, the in-vehicle control device <NUM> according to a fifth embodiment of the invention will be described with reference to <FIG>. Description of some points in common with the above-described embodiment will be omitted.

In the above embodiment, only the image pickup device <NUM> has been used to grasp the surrounding environment of the own vehicle <NUM>. However, in this embodiment, in addition to the image pickup device <NUM>, the input information from an object detection device <NUM> using a sensor other than a camera is also stored in the object information storage unit <NUM> so as to generate the pre-processing grid map <NUM> with more accuracy. A plurality of the object detection devices <NUM> may be provided, and in that case, different sensors may be used for each.

As the object detection device <NUM> to be added, for example, a millimeter wave radar, a laser radar, a sonar, etc. can be considered. Since these detect an object by a physical phenomenon different from the detection principle by the image pickup device <NUM>, it is possible to realize the object detection more reliably by increasing redundancy in object detection. Further, it is possible to reduce the blind spot in object detection by shifting the mounting position and viewing angle of the object detection device <NUM> from the image pickup device <NUM>.

Claim 1:
An in-vehicle control device, comprising:
an object detection unit (<NUM>) configured to detect a position of an object in front of an own vehicle from image information captured by an image pickup device (<NUM>);
an object information storage unit (<NUM>) configured to store a pre-processing grid map (<NUM>) in which a position where the object detection unit (<NUM>) has detected an object is set as an object occupied region and a position where no object has been detected is as an object unoccupied region;
an information processing unit (<NUM>) configured to generate a determination grid map (6a) in which a part of the object unoccupied region of the pre-processing grid map (<NUM>) is replaced with the object occupied region; and
a road surface region determination unit (<NUM>) configured to generate an automatic driving grid map (6b) in which a closed space surrounded by the object occupied region of the determination grid map (6a) is set as a road surface region,
wherein the information processing unit (<NUM>) includes
an object region extraction unit (5a) configured to extract a continuous part of the object occupied region from the pre-processing grid map (<NUM>),
an object region distance calculation unit (5b) configured to calculate, when there is a plurality of the continuous parts, a distance between the continuous parts, and
an adjacent object region complementing unit configured to generate, when the calculated distance is equal to or less than a threshold, the determination grid map (6a) in which the object unoccupied region sandwiched between both continuous parts is replaced with the object occupied region,
and/or the information processing unit (<NUM>) includes
an object region extraction unit (5a) configured to extract a block of object occupied regions from the pre-processing grid map (<NUM>),
an object region line segment calculation unit (5d) configured to calculate a representative line segment of each block when a plurality of the blocks exist, and
an adjacent line segment interpolation means configured to generate, when a distance between the calculated representative line segments is equal to or less than a threshold and an angle formed by the calculated representative line segments is equal to or less than a threshold, the determination grid map (6a) in which the object unoccupied region between both representative line segments is replaced with the object occupied region.