Living body recognizing device

A living body recognizing device is equipped with a captured image acquiring unit which acquires captured images from infrared cameras having a characteristics that a luminance of an image portion of a target object becomes higher as a temperature of the target object becomes higher than a background, and vice versa, according to a temperature difference between the background and the target object, a living body image extracting unit which executes a first living body image extracting processing of extracting the image portion of the target object assumed as a living body, from a region in the captured image where the luminance is equal to or lower than a first threshold value, and a living body recognizing unit which recognizes an existence of the living body, based on the image portion of the target object extracted by the living body image extracting unit.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is the U.S. National Phase of PCT International Application PCT/JP2012/072212, filed Aug. 31, 2012, which, in turn, claimed priority based on Japanese patent application 2011-212484, filed Sep. 28, 2011. The subject matter of each of these priority documents, including specification, claims, and drawings, is incorporated by reference herein.

TECHNICAL FIELD

The present invention relates to a living body recognizing device which recognizes a living body based on a captured image by an infrared camera.

Conventionally, there is known a device in which a living body existing in surroundings of a vehicle is recognized for example, by extracting an image portion of a living body (a pedestrian, a wild animal and the like) having a luminance equal to or more than a luminance threshold value, by binarization of a captured image (a gray scale image) of the surroundings of the vehicle captured by the infrared camera mounted on the vehicle, with the predetermined luminance threshold value (for example, refer to Patent Literature 1).

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

As the infrared camera, an infrared camera having a characteristics that a luminance of an image portion of a target object becomes higher as a temperature of the target object becomes higher than a background, and the luminance an image portion of the target object becomes lower as the temperature of the target object becomes lower than the background, according to a temperature difference between the background and the target object (a monitor target object such as the living body) (hereinafter referred to as AC characteristics), is used.

In a case where the infrared camera of the AC characteristics is used, for example in a situation where the outside air temperature is higher than the temperature of the living body, the luminance of the image portion of the living body in the captured image by the infrared camera becomes lower than the image portion of the background. And, in such case, it becomes difficult to recognize the living body by extracting the image portion of the living body, from a white region of a binarized image (a high luminance region), which is the binarized captured image of the infrared camera.

The present invention has been made in view of such background, and aims to provide a living body recognizing device which suppresses a state where the recognition of the living body based on the captured image of the infrared camera becomes difficult, arising from the environment surrounding the living body.

Solution to Problem

In order to achieve the above-mentioned object, a living body recognizing device includes a captured image acquiring unit which acquires a captured image having a characteristics that a luminance of an image portion of a target object becomes higher as a temperature of the target object becomes higher than a background, and the luminance of the image portion of the target object becomes lower as the temperature of the target object becomes lower than the background, according to a temperature difference between the background and the target object, using an infrared camera;a living body image extracting unit which executes a first living body image extracting processing of extracting the image portion of the target object assumed to be a living body, from a region in the captured image in which the luminance is equal to or lower than a predetermined first threshold value; anda living body recognizing unit which recognizes an existence of the living body, based on the image portion of the target object extracted by the living body image extracting unit (a first aspect of the invention).

According to the first aspect of the invention, the living body image extracting unit executes the first living body image extracting processing of extracting the image portion of the target object assumed to be the living body, from the region in the captured image in which the luminance is equal to or lower than the first threshold value. By doing so, it becomes possible to extract the image portion of the target body assumed to be the living body, and recognize the existence of the living body with the living body recognizing unit, even in a situation where the temperature of the living body is lower than a temperature of the surroundings, and the luminance of the image portion of the living body in the captured image becomes lower than the background. Therefore, it becomes possible to suppress a state where the recognition of the living body based on the captured image of the infrared camera becomes difficult, arising from the environment surrounding the living body.

The background in the present invention means objects existing behind the target object (wall surface of a structure, a road surface, and the like), when capturing the target object with the infrared camera. Further, as the captured image having the above-mentioned characteristics, a captured image by an infrared camera which outputs a video having the above-mentioned characteristics, or a captured image provided with the above-mentioned characteristics by performing a video filter processing and the like to an image of an ordinary infrared camera, in which a luminance of corresponding pixel becomes higher as the temperature of the target object becomes higher, may be used.

Further, in the first aspect of the invention, the living body image extracting unit generates an inverted captured image in which the luminance of each pixel of the captured image is inverted, and executes the first living body image extracting processing by extracting the image portion of the target object assumed to be the living body, from a region in the inverted captured image in which the luminance is equal to or higher than an inverted threshold value corresponding to the first threshold value (a second aspect of the invention).

According to the second aspect of the invention, the living body image extracting unit is capable of extracting the image portion of the low luminance from the captured image, by performing a general extracting processing of the image portion of extracting the image portion with high luminance, to the inverted captured image. Therefore, it is not necessary to prepare separately a processing program for extracting the image portion of the low luminance from the captured image.

Inverting the luminance of each pixel means subtracting the luminance of each pixel from a maximum value of a luminance range, and replacing the luminance of each pixel representing a height from a minimum value of the luminance range, with a luminance corresponding to a remaining height to the maximum value of the luminance range.

Further, in the first aspect of the invention or the second aspect of the invention, the living body image extracting unit executes the first living body image extracting processing, and a second living body image extracting processing which extracts the image portion of the target object assumed to be the living body, from a region in the captured image in which the luminance is equal to or higher than a predetermined second threshold value (a third aspect of the invention).

According to the third aspect of the invention, the living body image extracting unit is capable of extracting the image portion of the target object assumed to be the living body from the captured image, in both situations of a case where the temperature of the living body is higher than the surrounding temperature, and a case where the temperature of the living body is lower than the surrounding temperature, by executing the first living body image extracting processing and also the second living body image extracting processing.

Further, in the third aspect of the invention, the first threshold value and the second threshold value are set to different values (a fourth aspect of the invention).

According to the fourth aspect of the invention, it becomes possible to set the first threshold value to a value capable of easily extracting the image portion of the target object with low luminance by the first living body image extracting processing, and also to set the second threshold value to a value capable of easily extracting the image portion of the target object with high luminance by the second living body image extracting processing.

Further, in the fourth aspect of the invention, the living body image extracting unit sets the first threshold value and the second threshold value, according to a luminance distribution of the captured image (a fifth aspect of the invention).

According to the fifth aspect of the invention, it becomes possible to judge the first threshold value and the second threshold value capable of extracting the image portion of the target object with low luminance and the image portion of the target object with high luminance, from the luminance distribution of the captured image, with good efficiency.

Further, in any one of the first aspect of the invention through the fifth aspect of the invention, the living body recognizing device further includes an outside air temperature sensor which detects an outside air temperature, and the living body image extracting unit executes the first living body image extracting processing, when a detected temperature of the outside air temperature sensor is equal to or higher than a predetermined temperature (a sixth aspect of the invention).

According to the sixth aspect of the invention, by executing the first living body image extracting processing in a situation in which the outside air temperature is equal to or higher than the predetermined temperature, and the outside air temperature becomes higher than the temperature of the living body, it becomes possible to extract the image portion of the target object assumed to be the living body, from a region of the captured image in which the luminance is equal to or lower than the first threshold value.

Further, in any one of the first aspect of the invention through the sixth aspect of the invention, the living body recognizing device further includes a rainfall detecting unit which detects a rainfall, and the living body image extracting unit executes the first living body image extracting processing, when the rainfall is detected by the rainfall detecting unit (a seventh aspect of the invention).

According to the seventh aspect of the invention, by executing the first living body image extracting processing in a situation in which the temperature of the living body became lower than the surrounding temperature from the rainfall, it becomes possible to extract the image portion of the target object assumed to be the living body, from the region of the captured image in which the luminance is equal to or lower than the first threshold value.

Further, in any one of the first aspect of the invention through the seventh aspect of the invention, the living body image extracting unit executes the first living body image extracting processing, when a contrast of the captured image is equal to or less than a predetermined width (an eighth aspect of the invention).

According to the eighth aspect of the invention, when the contrast of the captured image becomes low to equal to or lower than the predetermined width, it is assumed that the target object and the surroundings are all in a high-temperature situation. In such situation, there is a high possibility that the image portion of the living body, which has high luminance in normal state, has low luminance. Therefore, by executing the first living body image extracting processing in such case, it is expected that the image portion of the living body with low luminance is extracted.

DESCRIPTION OF EMBODIMENTS

An embodiment of a living body recognizing device of the present invention will be explained with reference toFIG. 1throughFIG. 5. With reference toFIG. 1, a living body recognizing device10of the present embodiment is used by being mounted on a vehicle1. The living body recognizing device10recognizes living bodies (pedestrians, wild animals and the like) existing ahead of the vehicle1, based on captured images from a left camera21and a right camera22.

The living body recognizing device10detects a distance between the vehicle1and a target object existing ahead of the vehicle1at each predetermined control cycle, tracks a position of the target object, and when there is a fear that the target object contacts the vehicle1, performs an attention-drawing processing of performing a warning display to an indicator42, and also outputting a warning sound from a loudspeaker41, so as to prompt attention of the driver of the vehicle1to the target object.

The living body recognizing device10performs the attention-drawing processing, when the target object in an approach judging region AR1(exemplified by P1inFIG. 1) is recognized. The approach judging region AR1is set inside a capturing area AR0of the left camera21and the right camera22, to a region closer than Z1of a Z-axis set in a traveling direction of the vehicle1, and has a width obtained by adding a width a of the vehicle1with allowances β on both right and left sides, in an X-axis direction which is set to a width direction of the vehicle1.

Further, the living body recognizing device10performs the attention-drawing processing, also when there is a possibility that the target object existing in entry judging regions AR2, AR3set to both right and left sides of the approach judging region AR1(exemplified by P2, P3inFIG. 1) enters into the approach judging region AR1.

The left camera21and the right camera22are infrared cameras having a characteristics that, according to a temperature difference between a background and the target object, a luminance of an image portion of the target object becomes higher as the temperature of the target object becomes higher than the background, the luminance of the image portion of the target object becomes lower as the temperature of the target object is lower than the background (AC characteristics).

The left camera21and the right camera22are arranged at approximately symmetrical positions with respect to a central axis (the Z-axis) of a lateral direction of the vehicle1(the width direction, the X-axis direction). Further, the left camera21and the right camera22are fixed so that optical axes become parallel with each other, and a height from a road surface becomes identical.

Next, with reference toFIG. 2, the living body recognizing device10is input with video signals of the left camera21and the right camera22. Further, the living body recognizing device10is input with detected signals from a yaw rate sensor31, a vehicle speed sensor32, a brake sensor33, and an outside air temperature sensor34, which are mounted to the vehicle1, and a signal indicating an activating state of a wiper35. Further, the output of the loudspeaker41and the display of the indicator42are controlled by control signals output from the living body recognizing device10.

The living body recognizing device10is an electronic circuit unit configured from a CPU, a memory, an interface circuit and the like, that are not shown, and by executing programs for living body recognition stored in the memory with the CPU, functions as a captured image acquiring unit11which converts the video signals output from the left camera21and the right camera22into digital signals, and which stores the same as data of the captured images by the left camera21and the right camera22to an image memory (not shown), a living body image extracting unit12which extracts the image portion of the target object assumed to be a living body from the captured image, a rainfall detecting unit13which detects existence or non-existence of a rainfall from the activating state of the wiper35, a living body recognizing unit14which recognizes the existence of the living body from the image portion of the target object extracted by the living body image extracting unit12, and a contact judging unit15which determines a contact possibility of the vehicle1and the living body recognized by the living body recognizing unit14.

Next, a recognizing processing of the living body existing ahead of the vehicle1by the living body recognizing device10will be explained in accordance with a flowchart shown inFIG. 3. The living body recognizing device10recognizes the living body existing ahead of the vehicle1, by executing the flowchart ofFIG. 3in each predetermined control cycle.

STEP1inFIG. 3is a processing by the captured image acquiring unit11. The captured image acquiring unit11converts the video signal output from the left camera21and the right camera22into digital signal, and stores the same as the captured image (gray scale image) of the left camera21and the right camera22in the image memory.

Subsequent STEP2through STEP5, and STEP20through STEP22, are processing by the living body image extracting unit12. In the present embodiment, the living body image extracting unit12executes the processing of STEP2through STEP5and STEP20through STEP22to the captured image of the right camera22(hereinafter referred simply as the captured image).

In STEP2, the living body image extracting unit12sets a first threshold value Yth1and a second threshold value Yth2for generating a binary image, from a luminance histogram of the captured image (corresponds to a luminance distribution of the present invention).FIG. 4shows an example of the luminance histogram of the captured image, setting an axis of ordinate to a luminance, and an axis of abscissas to a frequency of appearance of pixels (a number of pixels) having each luminance.

In the luminance histogram ofFIG. 4, there are two peaks of frequency (Y1, Y2), and the living body image extracting unit12sets the first threshold value Yth1, which is a binarization threshold value for extracting the image portion corresponding to a peak Y1of a low luminance side, and the second threshold value Yth2, which is a binarization threshold value for extracting the image portion corresponding to a peak Y2of a high luminance side.

With the first threshold value Yth1, by performing the binarization processing of setting the pixels with the luminance equal to or lower than the first threshold value Yth1to black (0), and setting the pixels with the luminance higher than the first threshold value Yth1to white (1), it becomes possible to extract the image portion corresponding to the peak Y1from a black region in the binarized image. Therefore, in a case where the outside air temperature is higher than a temperature of the living body, and the luminance of the image portion of the living body in the captured image is lower than the luminance of the background, it becomes possible to extract the image portion of the living body.

Further, with the second threshold value Yth2, by performing the binarization processing of setting the pixels with the luminance equal to or lower than the second threshold value Yth2to black (0), and setting the pixels with the luminance higher than the second threshold value Yth2to white (1), it becomes possible to extract the image portion corresponding to the peak Y2from a white region of the binarized image. Therefore, in a general situation in which the outside air temperature is lower than the temperature of the living body, and the luminance of the image portion of the living body is higher than the background, it becomes possible to extract the image portion of the pedestrian.

In subsequent STEP3, the living body image extracting unit12generates a binarized image (a second binarized image) with the second threshold value Yth2. Further, in STEP4, the living body image extracting unit12extracts the image portion of the target object assumed to be the living body, from the white region of the second binarized image.

Now, the image portion of the target object assumed to be the living body is an image portion in which a height, a width, a height from a road surface, a luminance average value, a luminance distribution and the like fall within a preliminarily set range (which is set assuming the living body such as the pedestrian, the wild animal and the like). The processing of STEP3through STEP4corresponds to a second living body image extracting processing of the present invention.

In subsequent STEP5, the living body image extracting unit12determines whether or not the outside air temperature detected by the outside air temperature sensor34is equal to or higher than a predetermined temperature, and whether or not a rainfall is detected by the rainfall detecting unit13. Thereafter, when the outside air temperature is equal to or higher than the predetermined temperature, or when the rainfall is detected, the process branches to STEP20, and when the outside air temperature is lower than the predetermined temperature, and when no rainfall is detected, the process proceeds to STEP6.

In a case where the process branches to STEP20, there is a possibility that the outside air temperature is equal to or higher than the predetermined temperature and the outside air temperature is lower than the temperature of the living body, or the temperature of the living body becomes lower than the surrounding temperature by the rainfall, so that the luminance of the image portion of the living body in the captured image becomes lower than the background.

Therefore, in such case, the living body image extracting unit12extracts the image portion of the target object with low luminance from the captured image, with STEP20through STEP22. In STEP20, the living body image extracting unit12generates an inverted captured image in which the luminance of each pixel of the captured image is inverted.

FIG. 5shows a captured image (original image)50, and an inverted captured image60in which the luminance of each pixel of the captured image50is inverted. In the captured image50, the luminance of an image portion51of the living body (the pedestrian) is lower than the background, whereas in the inverted captured image60, the luminance of an image portion61of the living body is higher than the background.

The inversion of the luminance of each pixel is performed by subtracting the luminance of each pixel from a maximum value of the luminance. For example, in a case of an 8-bit gradation (luminance range: 0 to 255), the luminance is inverted from the following expression (1).
Yc=255−Y(1)where Yc: luminance of each pixel after inversion, and Y: luminance of each pixel in the captured image.

In subsequent STEP21, the living body image extracting unit12generates a binarized image (an inverted binarized image), by binarizing the inverted captured image by an inverted threshold value Ythc in which the first threshold value Yth1is inverted (=the maximum value of the luminance−Yth1). The inverted binarized image is an image in which white pixels and black pixels of the binarized image generated by the binarization of the captured image with the first threshold value Yth1(a first binarized image) are interchanged.

Thereafter, in subsequent STEP22, the living body image extracting unit12extracts the image portion of the target object assumed to be the living body from the white region of the inverted binarized image, in a similar processing as above-mentioned STEP4, and proceeds to STEP6.

As is explained above, by generating the inverted binarized image, it becomes possible to substitute the processing of extracting the image portion of the target object from the black region of the first binarized image, to the processing of extracting the image portion of the target object from the white region of the inverted binarized image. The processing of STEP20through STEP22corresponds to a first living body image extracting processing of the present invention.

In this case, as a program for extracting the image portion of the target object from the binarized image, only the program for extracting the image portion from the white region of the binarized image should be stored to the memory, and it is not necessary to separately prepare a program for extracting the image portion of the target object from the black region of the binarized image. Therefore, it becomes possible to simplify the program for recognizing the living body, and to decrease a program capacity.

STEP6through STEP7are processing by the living body recognizing unit14and the contact judging unit15. The living body recognizing unit14determines whether or not the image portion of the target object is extracted by the living body image extracting unit12, in STEP6. Thereafter, when the image portion of the target object is extracted, the process proceeds to STEP7, and when the image portion of the target object it not extracted, the process branches to STEP8and terminates the processing.

In STEP7, the living body recognizing unit14recognizes a position of the target object in real space, from the image portion of the target object extracted by the living body image extracting unit12. This recognizing processing is performed, for example with a method disclosed in Japanese Patent Application Laid-Open No. 2003-284057 mentioned above, by converting a position of the image portion of the target object in the captured image (camera coordinate) to the real space position (real space coordinate), based on a distance between the vehicle1and the target object, which is calculated from a disparity of the images of the target object between the captured image of the left camera21and the captured image of the right camera22.

The contact judging unit15performs the attention-drawing processing of performing the warning display to the indicator42, and also outputting the warning sound from the loudspeaker41, when the position of the target object is inside the approach judging region AR1(refer toFIG. 1), and when there is a possibility of the target object entering the approach judging region AR1from the entrance determining regions AR2, AR3(refer toFIG. 1), from a moving vector of the target object calculated from the traveling speed of the vehicle1detected by the vehicle speed sensor32, and the yaw rate of the vehicle1detected by the yaw rate sensor31.

The contact judging unit15prompts the attention of the driver of the vehicle1to the target object by the attention-drawing processing, proceeds to STEP8and terminates the process. When a braking operation by the driver is detected by the brake sensor33(refer toFIG. 2), it could be determined that the driver is recognizing the existence of the target object, and is attempting to avoid contact, so that the contact judging unit15will not perform the attention-drawing processing.

In the present embodiment, the living body image extracting unit12generated the inverted binary image, and extracted the image portion of the target object in which the luminance in the captured image is lower than the background, from the white region of the inverted binarized image, by the processing of STEP20through STEP22inFIG. 3. However, the living body image extracting unit12may extract the image portion of the target object from the black region of the first binarized image (the binarized image obtained by binarization of the captured image with the first threshold value Yth1), without generating the inverted binarized image.

Further, in the present embodiment, the living body image extracting unit12branched to STEP20when the outside air temperature is equal to or higher than the predetermined temperature, or when the rainfall is detected, in STEP5ofFIG. 3, and performed the processing of extracting the image portion with low luminance. However, the processing of extracting the image portion with low luminance after STEP20may be constantly performed, without determining the condition of STEP5.

Further, in the present embodiment, the second living body image extracting processing of extracting the image portion of the target object is performed, from the second binarized image generated by binarization of the captured image with the second threshold value Yth2, in STEP3through STEP4inFIG. 3. However, the effect of the present invention may be obtained, even when the second living body image extracting processing is not performed.

Further, in the present embodiment, the living body image extracting unit12sets the first threshold value Yth1and the second threshold value Yth2, based on the luminance histogram of the captured image, in STEP2ofFIG. 3. However, the first threshold value Yth1and the second threshold value Yth2may be set, according to other conditions (the outside air temperature, and the like).

Further, in the present embodiment, the binarized image is generated, and the image portion of the target object is extracted from the binarized image, in STEP3through STEP4, and STEP20through STEP22inFIG. 3. However, the image portion of the target object may be extracted from the region in which the luminance is equal to or lower than the first threshold value, or the image portion of the target object may be extracted from the region in which the luminance is equal to or higher than the second threshold value, directly from the captured image (the gray scale image) without generating the binarized image.

Further, in the present embodiment, the rainfall detecting unit13detected the existence or non-existence of the rainfall from the activating state of the wiper35. However, the rainfall may be detected by a raindrop sensor, or the rainfall may be detected by receiving a weather information via communication.

Further, in the present embodiment, an example where the living body recognizing device10is mounted to the vehicle1is shown. However, the application of the present invention is possible, even when the living body recognizing device is provided to a road, an entrance of a facility, and the like.

Further, in the present embodiment, the process branched to STEP20and the image portion of the target object is extracted using the inverted captured image, when the outside air temperature is equal to or higher than the predetermined temperature, and when the rainfall is detected, in STEP5inFIG. 3. Furthermore, the image portion of the target object may be extracted using the inverted captured image, when a contrast of the captured image is equal to or less than a predetermined width.

Further, in the present embodiment, the infrared cameras21,22having the characteristics that the luminance of the image portion of the target object becomes higher as the temperature of the target object becomes higher than the background, and the luminance of the image portion of the target object becomes lower as the temperature of the target object becomes lower than the background, according to the temperature difference between the background and the target object (the AC characteristics), is used. However, an ordinary infrared camera in which the luminance of the corresponding pixels becomes higher according to the temperature of the target object may be used. In this case, a captured image having similar characteristics as the AC characteristics may be generated, by performing a video filter processing and the like, to the image of the infrared camera.

Further, in the present embodiment, two cameras (stereo camera), that is, the left camera21and the right camera22, are used. However, a configuration of a single camera (a monocular camera) may also be adopted. In this case, the distance between the vehicle and the target object may be detected, based on a rate of change of a size of the image portions of an identical object among time-series images by the monocular camera.

EXPLANATION OF REFERENCES