Information processing device, imaging device, apparatus control system, movable body, information processing method, and computer program product

An information processing device includes an image analyzing unit. The image analyzing unit includes an object region setting unit configured to set an object region, for image information obtained by capturing an imaging range, the object region corresponding to an object existing in the imaging range; a luminance information acquiring unit configured to acquire luminance information indicating luminance in the imaging range; and a correcting unit configured to correct the object region based on the luminance information in a luminance detection region set in a lower part of the object region.

TECHNICAL FIELD

The present invention relates to an information processing device, an imaging device, an apparatus control system, a movable body, an information processing method, and a computer program product.

BACKGROUND ART

Conventionally, for the safety of automobiles, automotive body structures or the like have been developed in terms of how to save a pedestrian and protect an occupant when the pedestrian and an automobile crash into each other. However, in recent years, with the advancement of an information processing technology and an image processing technology, a technology to detect a person and an automobile at a high speed is being developed. By applying these technologies, an automobile that prevents an occurrence of a crash by automatically putting a brake before the automobile hits an object has already been developed. In automatic vehicle control, it is necessary to accurately measure a distance to an object, such as a person or some other vehicle; therefore, distance measurement using a millimeter wave radar or a laser radar, distance measurement using a stereo camera, and the like have been put to practical use.

If a distance is measured by using a stereo camera, it is possible to generate a disparity image (a distance image) based on an amount of deviation (disparity) between local regions captured by left and right cameras, and measure a distance between an object as a target for collision avoidance or the like and a subject vehicle. In this case, it is possible to recognize a position, a size, or the like of the object through a clustering process of detecting a group of pixels indicating similar distances (with similar disparity values) as a single object. That is, through the clustering process based on a disparity or information that can be handled synonymously with the disparity (for example, information on a distance acquired by using a millimeter wave radar, a laser radar, or the like), a region corresponding to the object is set in a distance image (disparity image) or a luminance image.

For example, to obtain movement information on a target object with high accuracy in an object detection device that detects the target object from a subject vehicle or the like and in a drive support device that performs a collision avoidance assist between the subject vehicle and the target object based on a detection result obtained by the object detection device, a configuration has been disclosed that acquires a position of the target object from a predetermined mounting position, sets a reference point on a road surface around the position of the target object based on a feature amount on the road surface, and calculates movement information on the target object from the position of the target object with reference to the reference point (Japanese Patent No. 5971341).

Furthermore, to improve promptness and accuracy in a road surface estimation device, a configuration has been described that includes a three-dimensional object detecting means for detecting a three-dimensional object in front of a subject vehicle from an image captured by a camera, a lower end position detecting means for detecting a lower end position of the detected three-dimensional object, a temporary road surface calculating means for calculating a temporary road surface from the detected lower end position of the three-dimensional object and a reference position of the subject vehicle, and an actual road surface estimating means for estimating an actual road surface based on the temporary road surface. The three-dimensional object detecting means extracts a longitudinal edge of the three-dimensional object from a longitudinal edge that is extended by a certain length equal to or longer than a predetermined value in the vertical direction within the image captured by the camera. The lower end position detecting means detects the lower end position of the extracted longitudinal edge of the three-dimensional object. The temporary road surface calculating means calculates the temporary road surface by calculating a pitch angle of the subject vehicle from the detected lower end position, the reference position of the subject vehicle, a distance from the camera to the three-dimensional object, and an optical axis of the camera. The actual road surface estimating means estimates the actual road surface based on the calculated pitch angle (Japanese Patent No. 4754434).

SUMMARY OF INVENTION

Technical Problem

When an object region indicating a position, a size, or the like of an object is to be set in a system that recognizes the object based on distance information, such as a disparity, as described above, and if a disparity is detected at a position lower than a position at which the object exists, the object region may be set so as to include a region in which the object does not actually exist. For example, when a vehicle (object) running in front of a subject vehicle passes by a marking on a road surface, and if a disparity is detected in a marking portion, an object region may be set so as to include the marking portion existing below the vehicle in some cases. In this case, an error may occur between an actual object (the vehicle) and a subject (including the vehicle and the marking) handled as the object in the system, and the accuracy of distance measurement, avoidance behaviors, or the like may be reduced. For example, when a distance from a subject vehicle to a vehicle in front of the subject vehicle is calculated based on an average of disparities in the object region, and if the object region is set wider than supposed to be, the calculated distance may be shorter than an actual distance.

The present invention has been made in consideration of the circumstances as described above, and an object is to improve the accuracy of object recognition.

Solution to Problem

According to an embodiment, provided is an information processing device comprising: a setting unit configured to set an object region, for image information obtained by capturing an imaging range, the object region corresponding to an object existing in the imaging range; a luminance information acquiring unit configured to acquire luminance information indicating luminance in the imaging range; and a correcting unit configured to correct the object region based on the luminance information in a luminance detection region that is set in a lower part of the object region.

Advantageous Effects of Invention

According to an aspect of the present invention, it is possible to improve the accuracy of object recognition.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of an information processing device, an imaging device, an apparatus control system, a movable body, an information processing method, and a computer program product according to the present invention will be described in detail below with reference to the drawings. The present invention is not limited by the embodiments below. Configuration elements in the embodiments described below include elements easily conceived by a person skilled in the art, elements substantially the same, and elements within the scope of so-called equivalents. Various omission, replacement, and modifications of the configuration elements may be made within the scope not departing from the gist of the following embodiments.

<Overall Configuration of Apparatus Control System>

FIG. 1is a diagram illustrating an overall configuration of an apparatus control system1according to an embodiment. The apparatus control system1is mounted on a subject vehicle100(an apparatus) such as an automobile as one example of movable bodies, and includes an imaging device101, a display monitor103, and a running control unit104(an apparatus control unit). The apparatus control system1recognizes a road surface and an object as a detection target in front of the subject vehicle100based on captured image data of the front of the subject vehicle100captured by the imaging device101, and causes the running control unit104to control a running state or the like of the subject vehicle100based on an analysis result of the recognized information. The object is an object to be a target for collision avoidance or the like, and may be a vehicle (an automobile, a motorcycle, a bicycle, or the like), a person, an animal, or a structural object (a guardrail, a utility pole, a curb, a bifurcating block, a falling object, or the like), for example. The apparatus control system1of the present embodiment is applicable to not only the automobile but also a movable body, such as an airplane or a robot, and other apparatuses.

The imaging device101includes an imaging unit107and an image analyzing unit108(an information processing device). The imaging unit107is a unit that acquires a plurality of pieces of captured image data (luminance information) for a single object, and may be a stereo camera or the like, for example. The imaging unit107is installed in an upper portion of a windshield105(for example, near a rearview mirror) of the subject vehicle100, for example. Various kinds of data, such as the captured image data, obtained through imaging by the imaging unit107are input to the image analyzing unit108. The image analyzing unit108analyzes data transmitted from the imaging unit107, and generates an analysis result including information indicating a three-dimensional shape of a road surface on which the subject vehicle100is running, information indicating a position, a size, a shape, or the like of the object, or the like. The image analyzing unit108sends the analysis result to the display monitor103and the running control unit104. The display monitor103displays the captured image data, the analysis result, and the information related to the captured image data and the analysis result, which are obtained by the imaging device101. The running control unit104provides a warning to a driver of the subject vehicle100or performs running assist control to automatically control a brake, an accelerator, a steering wheel, or the like of the subject vehicle100based on the analysis result obtained by the image analyzing unit108. Hereinafter, a term “image” will be used; however, the “image” in the present embodiments does not necessarily have to be displayed, and includes a simple aggregate of information that is not displayed on a monitor or the like.

<Overall Configurations of Imaging Unit and Image Analyzing Unit>

FIG. 2is a schematic diagram illustrating overall configurations of the imaging unit107and the image analyzing unit108according to the present embodiment.

The imaging unit107is a stereo camera including two imaging units110aand110b. The two imaging units110aand110bare identical to each other. The imaging units110aand110brespectively include imaging lenses111aand111b, sensor substrates114aand114bincluding image sensors113aand113bon which light-receiving elements are arranged two-dimensionally, and signal processing units115aand115bthat generate and output captured image data (luminance information) by converting analog electrical signals (electrical signals corresponding to the amounts of light received by the light-receiving elements on the image sensors113aand113b) output from the sensor substrates114aand114binto digital electrical signals.

The imaging unit107includes a process hardware unit120constructed by a field-programmable gate array (FPGA) or the like. The process hardware unit120includes a disparity calculating unit121that calculates a disparity value of a corresponding image portion between the captured images captured by the imaging units110aand110b, in order to obtain disparity information from luminance information output from each of the imaging units110aand110b.

The disparity value described herein is obtained such that, by using one of the captured images (luminance images) captured by the imaging units110aand110bas a reference image and using the other one of the captured images as a comparison image, an amount of positional deviation of an image portion of the comparison image with respect to an image portion on the reference image is calculated as the disparity value of the image portions, where the image portions correspond to an identical point (a local region of an object) in an imaging region. A distance from the imaging unit107(the subject vehicle100) to the local region of the object can be calculated from the disparity value by using the principle of triangulation.

FIG. 3is a diagram for explaining the principle of calculation of a distance from the disparity value by using the principle of triangulation. InFIG. 3, f denotes a focal length of each of the imaging lenses111aand111b, D denotes a distance between optical axes, and Z denotes a distance from the imaging lenses111aand111bto an object301(a distance in a direction parallel to the optical axes). In this drawing, imaging positions of a certain point O (a local region) of the object301on a left image and a right image are located at distances Δ1 and Δ2 from imaging centers. A disparity value d in this case is defined such that d=Δ1+Δ2.

Referring back toFIG. 2, the image analyzing unit108includes an image processing board or the like, and includes a storage unit122constructed by a random access memory (RAM), a read only memory (ROM), or the like for storing the luminance information and the disparity information output from the imaging unit107, a central processing unit (CPU)123that executes a computer program for performing a recognition process on a recognition target, disparity calculation control, or the like, a data interface (I/F)124, and a serial I/F125.

The FPGA of the process hardware unit120generates information on a disparity image by performing, on the luminance information, a process required to be executed in real time, such as gamma correction, distortion correction (parallelization of the left captured image and the right captured image), or disparity calculation by block matching, and writes the information to the RAM of the image analyzing unit108. The CPU123of the image analyzing unit108controls an image sensor controller of each of the imaging units110aand110band controls the entire image processing board. Furthermore, the CPU123loads a three-dimensional object detection program for executing a process of detecting a three-dimensional shape of a road surface, a process of detecting an object, or the like from the ROM, executes various processes by using the luminance information or the disparity information stored in the RAM as input, and outputs a processing result to the outside from the data I/F124or the serial I/F125. In the case of executing the processes as described above, it may be possible to input vehicle operation information, such as a vehicle speed, acceleration (mainly acceleration in a front-back direction of the subject vehicle), a steering angle, or a yaw rate of the subject vehicle100, by using the data I/F124, and use the vehicle operation information as a parameter for various processes. The data output to the outside is used as input data for performing various kinds of control (brake control, vehicle speed control, steering control, or warning control) on the subject vehicle100.

<Functional Configuration of Image Analyzing Unit>

FIG. 4is a diagram illustrating an example of a functional configuration of the image analyzing unit108according to the present embodiment. The image analyzing unit108includes a distance information acquiring unit51, a luminance information acquiring unit52, a distance image generating unit53, a road surface shape detecting unit54(a detecting unit), an object region setting unit55(a setting unit), a correcting unit56, an analyzing unit57, and an output unit58.

The distance information acquiring unit51is a functional unit that acquires distance information indicating a distance between the subject vehicle100in which the imaging unit107is mounted and an object existing in an imaging range of the imaging unit107. The distance information in the present embodiment is the disparity information acquired by the imaging unit107that is a stereo camera; however, the distance information is not limited thereto. The distance information acquiring unit51is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

The luminance information acquiring unit52is a functional unit that acquires luminance information indicating luminance in the imaging range of the imaging unit107. The luminance is brightness of a surface of an object with a width, and may be brightness on a road surface, brightness of a surface of an object, such as a vehicle, or the like. The luminance information acquiring unit52is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

The distance image generating unit53is a functional unit that generates distance image data (disparity image data in the present embodiment) indicating a distance image (a disparity image in the present embodiment) indicating a distance distribution in the imaging range, based on the distance information (disparity information in the present embodiment). The distance image generating unit53is constructed by cooperation of the storage unit122, the CPU123, and a program stored in the storage unit122, or the like.

FIG. 5is a diagram illustrating an example of a luminance image61.FIG. 6is a diagram illustrating an example of a disparity image71corresponding to the luminance image61illustrated inFIG. 5. The luminance image61is an image generated based on the luminance information acquired by the imaging unit107(a captured image captured by the imaging unit107). The disparity image71is an image that is generated based on the disparity information generated by the disparity calculating unit121of the imaging unit107and that indicates a distribution of disparities (distances). The luminance image61of the present embodiment includes a road surface62, a vehicle63, and a road marking64representing a crossing. Therefore, the disparity image71includes a road surface corresponding region72corresponding to the road surface62, a vehicle corresponding region73corresponding to the vehicle63, and a road marking corresponding region74corresponding to the road marking64.

The road surface shape detecting unit54is a functional unit that detects a shape of the road surface62on which the subject vehicle100is moving, based on the distance information. The road surface shape detecting unit54is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

A method of detecting the shape of the road surface62is not specifically limited; however, for example, a method using a V-Disparity map (a V map) may be employed.FIG. 7is a diagram illustrating an example of a V map81corresponding to the disparity image71illustrated inFIG. 6.FIG. 8is a diagram illustrating an example of a state in which the shape of the road surface62is detected by using the V map81illustrated inFIG. 7. Hereinafter, those represented as a “map” in the present embodiment means a simple aggregate of information.

The V map81is generated such that the y-axis represents the coordinate of a vertical axis of the disparity image71, the horizontal axis represents a disparity, and a value (x, y, d) of each of pixels of the disparity image71is voted at a corresponding coordinate position on the V map81. That is, assuming that the disparity image is information in which a vertical position, a horizontal position, and a depth position of an object are associated with one another, the V map is information in which the vertical position and the depth position of the object are associated with each other. By generating the V map81as described above, a road surface disparity region82corresponding to the road surface62and a vehicle disparity region83corresponding to the vehicle63appear in the V map81. Therefore, each of pixel values in the V map81indicates a frequency value of a disparity. In the V map81as described above, frequency values are searched for from below, and a candidate point is selected for each column as illustrated inFIG. 8. The shape of the road surface62can be obtained by obtaining an approximate line by applying a least squares method to a group of the selected candidate points. For example, when an equation Y=a×X+b is obtained as an approximate line, and if a disparity value corresponding to a certain distance D from the subject vehicle100is denoted by d, a height Y (a value on the y-axis) of the road surface62in the disparity image71or the luminance image61is obtained such that Y=a×d+b. With repetition of this operation, it is possible to detect the shape of the road surface62.

More specifically, with respect to an object that appears in a portion corresponding to a certain point at a y-axis position of y′ with a certain disparity value d in the luminance image, a height of the object from a road surface can be calculated by (y′−y0), where y0 denotes a y-axis position with the disparity value d on the approximate line. In general, a height H of an object corresponding to the coordinates (d, y′) on the V map from the road surface can be calculated by Equation (1) below. In Equation (1) below, “z” is a distance (z=BF/(d-offset)) calculated from the disparity value d, and “f” is a value obtained by converting the focal length of the camera to a value with the same unit as (y′−y0). Here, “BF” is a value obtained by multiplying the baseline length by the focal length of the stereo camera, and “offset” is a disparity value obtained by capturing the object at infinity.
H=z×(y′−y0)/f(1)

The object region setting unit55is a functional unit that sets an object region corresponding to the object (the vehicle63) in the distance image (the disparity image71) based on the distance information (the disparity information). The object region setting unit55is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

A method of setting the object region is not specifically limited; however, for example, a method using a real U map may be employed. The real U map described herein is a map that represents a real space in a look-down view manner (a bird's eye view image, a look-down view image, or the like), and is one example of look-down view information. A functional unit that generates the look-down view information may be included in the object region setting unit55, or may be configured as an independent functional unit.FIG. 9is a diagram illustrating an example of a real U map91corresponding to the disparity image71illustrated inFIG. 6.FIG. 10is a diagram illustrating an example of a state in which a left end portion and a right end portion of an object region77are set by using the real U map91illustrated inFIG. 9.FIG. 11is a diagram illustrating an example of a state in which an upper end portion and a lower end portion of the object region77are set by using the real U map91illustrated inFIG. 9.

The real U map91is generated from a disparity image, a frequency U map, or a height U map. The frequency U map is a two-dimensional x-y histogram, in which the x-axis represents x, the y-axis represents d, and the z-axis represents a frequency for a combination (x, y, d) of the x-axis position, the y-axis position, and the disparity value d of each of the pixels of the disparity image71. The height U map is a two-dimensional x-y histogram, in which the x-axis represents x, the y-axis represents d, and the z-axis represents a height from the road surface for a combination (x, y, d) of the x-axis position, the y-axis position, and the disparity value d of each of the pixels of the disparity image71. That is, assuming that the disparity image is information in which a vertical position, a horizontal position, and a depth position of an object are associated with one another, the frequency U map and the height U map are information in which the horizontal position and the depth position of the object are associated with each other. The real U map91is a two-dimensional x-y histogram, in which the x-axis (the horizontal axis) represents an actual distance obtained by converting a value on the x-axis of the frequency U map or the height U map into an actual distance, the y-axis (the vertical axis) represents a decimated disparity value obtained by decimating a disparity of the frequency U map or the height U map in accordance with a distance, and the z-axis represents the frequency of the frequency U map or the height of the height U map. The real U map91illustrated inFIG. 9is generated from the frequency U map, and the z-axis thereof represents the frequency. A method of generating the real U map is not limited to the above-described example as long as the real U map is a look-down view image.

In the real U map91as described above, frequency values of a pixel group corresponding to a position in which the object, such as the vehicle63, exists among pixels included in the real U map91are increased. Therefore, an isolated region93corresponding to the object (the vehicle63) appears in the real U map91.

As illustrated inFIG. 10, it is possible to set the left end portion and the right end portion of the object region77in the luminance image61or the disparity image71based on the isolated region93in the real U map91. The object region77in the present embodiment indicates a position and a size of the object (the vehicle63) by a rectangle. That is, the object region77is information in which the central position, the height, and the width of the object are associated with one another, for example. Positions of the left end portion and the right end portion of the object region77can be determined by converting positions of a left end portion and a right end portion of the isolated region93on the real U map91to coordinates of the luminance image61or the disparity image71. The coordinates of the luminance image61and the coordinates of the disparity image71uniquely correspond to each other; therefore, the object region77can be freely converted between both of the images61and71. The isolated region may be detected by detecting a region to which an identical ID (label) is assigned through a well-known labeling process.

As illustrated inFIG. 11, it is possible to set the upper end portion and the lower end portion of the object region77in the disparity image71based on the isolated region93in the real U map91. The upper end portion of the isolated region93detected on the real U map91is a portion with the smallest disparity and the longest distance. The lower end portion of the isolated region93is a portion with the largest disparity and the shortest distance. The longest distance and the shortest distance can be obtained respectively from a disparity of the upper end portion and a disparity of the lower end portion. When positions of the upper end portion and the lower end portion of the object region77on the disparity image71are to be determined, portions in which disparities continue in a range between the longest distance and the shortest distance are detected, and, a disparity continuing portion located in the upper part can be estimated as an upper end position and a disparity continuing portion located in the lower part can be estimated as a lower end position. In this case, when the vehicle63is passing on the road marking64for example, disparities are generated in a portion in which the road marking64exists; therefore, the lower end position of the object region77may be set so as to include the road marking64(the road surface62) in some cases.

FIG. 12is a diagram illustrating an example of a state in which the object region77is temporarily set in the luminance image61. As described above, the object region setting unit55recognizes that portions in which the disparities continue in the vertical direction correspond to the end portions of the object region77. Therefore, when the object region77is set for the vehicle63passing on the road marking64, the object region may be set so as to include a part of the road surface62in which the road marking64is painted, beyond the lowermost portion of the vehicle63(a contact point between wheels and the road surface62).

The object region77may be set by using the luminance information instead of the distance information. For example, there is a method in which a template image of an object determined in advance is used such that a search is performed on the luminance image while changing a size of the template image, and a position at which the degree of match with the template image is the highest is detected and set. However, it is less likely that the template and an actual detection target completely match with each other, and in some cases, the object region77may be set so as to include the periphery of the detection target. That is, even when the object region77is set by using the luminance information, the object region may be set so as to include a part of the road surface62in which the road marking64is painted, beyond the lowermost portion of the vehicle63(a contact point between wheels and the road surface62).

The above-described problem with the lower end position of the object region77significantly occurs especially when the object region77is set by using the look-down view image such as the real U map. This is because information on the vertical position is lost in the look-down view image and an error is likely to occur when the lowermost portion (the vertical position) of the object is determined. The look-down view image is advantageous in terms of a processing speed or the like because the amount of information is smaller than the distance image or the like, but it is likely to cause the above-described problem. The problem with the lower end position of the object region77as described above can be solved by a function of the correcting unit56as described below.

The correcting unit56is a functional unit that corrects the object region77based on luminance information in a luminance detection region set in a lower part within the object region77. The correcting unit56is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

FIG. 13is a diagram illustrating an example of a state in which a luminance detection region78is set in the object region77. The luminance detection region78is a region from the lower end position of the object region77, which is temporarily set based on the disparity information (including information deviated from the disparity information (e.g., the V map81, the U map, the real U map91, or the like)), to an upper position at a predetermined distance D from the lower end position. The distance D is a distance including a predetermined number of pixel rows in the luminance image61, for example. The distance D may be a fixed value or a value that varies depending on predetermined conditions. The correcting unit56corrects the lower end position of the object region77based on the luminance information in the luminance detection region78.

FIG. 14is a diagram illustrating an example of a state in which luminance in the luminance detection region78is detected and the lower end position of the object region77is corrected. The correcting unit56of the present embodiment calculates average luminance (row average luminance) for each pixel row in order from the lowest to the highest pixel row of the luminance detection region78. For example, row average luminance of a first pixel row85illustrated inFIG. 14is larger than row average luminance of a second pixel row86. This is because of an influence of a shadow89formed under the vehicle63. Therefore, the row average luminance decreases as the position of the pixel row goes up.

The correcting unit56corrects the lower end position of the object region77so as to shift to a position of a pixel row in which the row average luminance is smaller than a predetermined value. A method of setting the predetermined value is not specifically limited; however, for example, a method based on comparison with average luminance in the entire luminance detection region78(entire average luminance) may be employed. For example, when a ratio of the row average luminance to the entire average luminance becomes smaller than a predetermined value (for example, 50%), the lower end position of the object region77may be corrected so as to be shifted to the position of the corresponding pixel row. For example, in the example illustrated inFIG. 14, when the row average luminance of the first pixel row85is equal to or larger than a predetermined value (for example, 50% of the entire average luminance) but the row average luminance of the second pixel row86is smaller than the predetermined value, the lower end position of the object region77is corrected so as to be shifted to the position of the second pixel row86.

FIG. 15is a flowchart illustrating a first example of a process of correcting the object region77according to the embodiment. The distance image generating unit53generates disparity image data indicating the disparity image71(one example of the distance image) based on the disparity information acquired from the distance information acquiring unit51(Step S201). Subsequently, the object region setting unit55sets the temporary object region77on the disparity image71and the luminance image61based on the disparity information (Step S202).

Then, the correcting unit56sets the luminance detection region78in the temporary object region77(Step S203). Subsequently, the correcting unit56calculates entire average luminance L1in the luminance detection region78(Step S204). Then, the correcting unit56calculates row average luminance L2for each of pixel rows in order from the lowermost row of the luminance detection region78(Step S205). Subsequently, the correcting unit56determines whether an expression L2×K1<L1is true, that is, whether a ratio of the row average luminance L2to the entire average luminance L1is smaller than a predetermined value (for example, 50%, i.e., K1=0.5) (Step S206).

If the expression L2×K1<L1is true (YES at Step S206), the lower end position of the object region77is corrected so as to be shifted to a position of the current pixel row (Step S207). In contrast, if the expression L2×K1<L1is not true (NO at Step S206), it is determined whether all of the pixel rows in the luminance detection region78are scanned (Step S208). If all of the pixel rows are scanned (YES at Step S208), the routine is terminated. If all of the pixel rows are not scanned (NO at Step S208), the process returns to Step S205.

According to the correction process as described above, it is possible to set the object region77with high accuracy by using the fact that the luminance on the lower side of the object (the vehicle63) is smaller than the luminance of other portions due to the influence of the shadow89of the object itself.

FIG. 16is a flowchart illustrating a second example of the process of correcting the object region77according to the embodiment. In the correction process of the second example, Step S301and Step S302are performed between Step S204and Step S205in the first example of the correction process illustrated inFIG. 15.

The correcting unit56according to the second example calculates the entire average luminance L1in the luminance detection region78(Step S204), and thereafter generates a luminance histogram, in which the number of pixels with luminance larger than the entire average luminance L1is counted for each of the pixel rows in the luminance detection region78(Step S301).

FIG. 17is a diagram illustrating an example of a luminance histogram95. In the luminance histogram95, a threshold T1calculated from a theoretically possible maximum value (a theoretical maximum value) is indicated. When a maximum value of the luminance histogram95is larger than the threshold T1, the correcting unit56according to the second example corrects the lower end position of the object region77.

The correcting unit56according to the second example determines whether the maximum value of the luminance histogram95is larger than the threshold T1(Step S302). If the maximum value of the luminance histogram95is not larger than the threshold T1(NO at Step S302), the routine is terminated. If the maximum value of the luminance histogram95is larger than the threshold T1(YES at Step S302), the processes at Step S205and later are performed.

In the correction process according to second example, the object region77is corrected only when the maximum value of the luminance histogram95is larger than the threshold T1. This is because it is assumed that the lower end position of the object region77is further extended downward with an increase in the number of pixels with high luminance in the lower part of the object region77. When the maximum value of the luminance histogram95is small, it is likely that the amount of downward extension of the lower end position of the object region77is small; therefore, in such a case, the correction process is not performed. With this configuration, it becomes possible to prevent execution of an unnecessary correction process and reduce a calculation load or the like.

The correcting unit56may correct the shape of the road surface62, which is detected by the road surface shape detecting unit54, based on the lower end position of the object region77corrected as described above.

FIG. 18is a diagram illustrating an example of the V map81when it is detected that the road surface62is located at a position lower than a supposed position. InFIG. 18, a state is illustrated in which a noise disparity97caused by the road marking64is detected below the vehicle disparity region83corresponding to the vehicle63, and an estimated line88indicating an estimated shape of the road surface62is deviated downward from the supposed position.

FIG. 19is a diagram illustrating an example of the V map81when the estimated shape of the road surface62illustrated inFIG. 18is corrected. As a method of correcting the road surface shape, a method of correcting the estimated line88so as to pass through a point (d, Y) without changing an intercept, for example. The disparity value d and the y-coordinate position Y are values corresponding to the lower end position of the corrected object region77.

The road surface shape corrected as described above may be used in various ways. For example, it may be possible that a deviation amount between the lower end position of the corrected object region77and a detected road surface shape is stored for each of frames, and when the object region77is continuously corrected for a predetermined number of frames or greater, the road surface shape modified in advance may be used to detect the road surface in a next frame. Furthermore, when a plurality of objects exist in the imaging region, and if the object region77and the road surface shape are corrected for one of the objects, information such as the corrected lower end position of the object region77, the corrected road surface shape, or the like may be used to set the object region77for the other objects, for example. With this configuration, it is possible to reduce a calculation load or the like.

The analyzing unit57is a functional unit that analyzes a pixel value in the object region77based on information on the object region77set as described above, the detected road surface shape, or the like, and generates analysis data indicating an analysis result. The analysis result may be various information; for example, a distance from the subject vehicle100to the object (the vehicle63or the like), a relative moving speed between the subject vehicle100and the object, an expected traveling direction of the object, or the like. The analyzing unit57is constructed by cooperation of the storage unit122, the CPU123, a program stored in the storage unit122, or the like.

The output unit58is a functional unit that outputs the analysis data generated by the analyzing unit57to an external system (the display monitor103, the running control unit104, or the like). The output unit58is constructed by cooperation of the storage unit122, the CPU123, the data OF124, the serial OF125, a program stored in the storage unit122, or the like.

According to the above-described embodiment, it is possible to set the object region77, which indicates a position, a size, or the like of an object to be a target for collision avoidance or the like, with high accuracy. Therefore, it becomes possible to improve the accuracy of running control of the subject vehicle100.

While the embodiments and modifications of the present invention have been described above, the present invention is not limited by the embodiments and modifications. The embodiments and modifications may be changed or modified without departing from the gist and the scope of the present invention.

REFERENCE SIGNS LIST

CITATION LIST

Patent Literature