Information processing apparatus for estimating age and method thereof

An information processing apparatus includes an extraction unit that extracts a feature from an image including a face, a first estimation unit that estimates a likelihood of the face with respect to each generation based on the feature, a storage unit that stores a plurality of samples, the plurality of samples each including a generation-specific combination of likelihoods and a correct age as a pair, a selection unit that selects a sample from the storage unit based on a combination of likelihoods estimated by the first estimation unit, and a second estimation unit that estimates an estimated age of the face and an error range thereof based on the sample selected by the selection unit.

BACKGROUND

Field

The present disclosure relates to an information processing apparatus for estimating age from a captured face image, and a method thereof.

Description of the Related Art

There has recently been a growing trend to estimate age and sex from face images obtained from a camera and using the estimations for marketing or security purposes. Under the circumstances, there are techniques for inputting a face image into generation-specific classifiers and estimating age based on the output results thereof. An example of such a technique is discussed in Japanese Patent Application Laid-Open No. 2014-153815.

It is extremely difficult to estimate the exact age from only the appearance of a face image in image recognition, just like humans have difficulty in estimating other people's age.

SUMMARY

According to an aspect of the present disclosure, an information processing apparatus includes an extraction unit configured to extract a feature from an image including a face, a first estimation unit configured to estimate a likelihood of the face with respect to each generation based on the feature, a storage unit storing a plurality of samples, the plurality of samples each including a generation-specific combination of likelihoods and a correct age as a pair, a selection unit configured to select a sample from the storage unit based on a combination of likelihoods estimated by the first estimation unit, and a second estimation unit configured to estimate an estimated age of the face and an error range thereof based on the sample selected by the selection unit.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will be described below with reference to the drawings.

A first exemplary embodiment will be described.

FIG. 1is a diagram illustrating an example of a hardware configuration of an information processing apparatus10. The information processing apparatus10includes a central processing unit (CPU)11, a storage unit12, a display unit13, an input unit14, and a communication unit15as its hardware configuration. The CPU11controls the entire information processing apparatus10. The storage unit12is, for example, a memory. The storage unit12stores a program, images, and information needed for the CPU11to perform processing. The display unit13displays a result of processing of the CPU11under the control of the CPU11. The input unit14accepts user operations, and inputs information related to the accepted operations into the CPU11. The communication unit15connects the information processing apparatus10to a network, and controls communication with other external apparatuses via the network under the control of the CPU11. A functional configuration of the information processing apparatus10illustrated inFIG. 2and processing of flowcharts ofFIGS. 3, 8, 17, and 19to be described below are implemented by the CPU11performing processing based on the program stored in the storage unit12.

FIG. 2is a diagram illustrating an example of the functional configuration of the information processing apparatus10. The information processing apparatus10includes an image acquisition unit100, a face detection unit110, a face organ detection unit120, a feature extraction unit130, a generation estimation unit140, an age and error estimation unit150, and a display processing unit160as its functional configuration. Functions will be described with reference to the flowchart ofFIG. 3.

In step S1000, the image acquisition unit100obtains a digital image that is obtained through a light collecting element such as a lens, an image sensor for converting light into an electrical signal, and an analog-to-digital (AD) converter for converting an analog signal into a digital signal. Examples of the image sensor include a complementary metal-oxide-semiconductor (CMOS) sensor and a charge-coupled device (CCD) sensor. Through thinning processing, the image acquisition unit100can obtain, for example, an image converted into full high-definition (Full HD) resolution (1920×1080 [pixels]) or HD resolution (1280×720[pixels]).

In step S1100, the face detection unit110detects a face included in the image obtained in step S1000. As a technique for detecting a face (hereinafter, referred to as “face detection”), the face detection unit110uses a technique discussed in P. Viola, M. Jones. “Rapid Object Detection using a Boosted Cascade of Simple Features”, in Proc. Of CVPR, vol. 1, pp. 511-518, December, 2001.

In step S1110, the face detection unit110determines whether a face is detected in step S1100. If a face is detected (YES in step S1110), the processing proceeds to step S1200. If a face is not detected (NO in step S1110), the processing returns to step S1000.

In step S1200, the face organ detection unit120detects face organs, such as eye corners and a mouth, in the face detected in step S1100. As a technique for detecting face organs, the face organ detection unit120uses a technique discussed in Xudong Cao, Yichen Wei, Fang Wen, Jian Sun, “Face Alignment by Explicit Shape Regression”, CVPR, pp. 2887-2894, 2012.

In step S1300, the feature extraction units130sets feature extraction areas, like the rectangles inFIG. 4, based on the positions of the face organs detected in step S1200. The feature extraction unit130then extracts luminance gradient histogram features (luminance gradient histograms) as illustrated inFIG. 5from the respective feature extraction areas. As a technique for extracting a luminance gradient histogram, the feature extraction unit130uses a technique discussed in M. Bertozzi, A. Broggi, M. DelRose, M. Felisa, A. Rakotomamonjy, and F. Suard, “A Pedestrian Detector Using Histograms of Oriented Gradients and a Support Vector Machine Classifier”, IEEE Intelligent Transportation Systems Conference, 2007.

In step S1400, the generation estimation unit140connects the plurality of luminance gradient histograms extracted in step S1300as illustrated inFIG. 6. The generation estimation unit140inputs the resulting connected feature into generation estimators as illustrated inFIG. 7to calculate a likelihood for each generation that the connected feature falls within the generation by generation. As a technique for estimating likelihood by generation, the generation estimation unit140uses a support vector machine (SVM) as discussed in the foregoing paper by Bertozzi et al. For example, a teens estimator can be constructed and trained by luminance gradient histogram features extracted from teens being given a learning label of +1 and by luminance gradient histogram features extracted from other generations being given a learning label of −1.

In step S1500, the age and error estimation unit150determines age and errors by using the outputs of the respective generator estimators in step S1400.FIG. 8illustrates details of the processing for determining age and errors in step S1500. A description thereof will be provided below with reference toFIG. 8.

In step S1501, the age and error estimation unit150generates an N-dimensional feature vector V=[V1, V2, . . . , Vn] from N likelihoods output from the respective generation estimators as illustrated inFIG. 9. The age and error estimation unit150previously determines N-dimensional feature vectors S=[S1, S2, . . . , Sn], like the N-dimensional feature vector V generated in step S1501, for various faces as illustrated inFIG. 10. The age and error estimation unit150stores the N-dimensional feature vectors S and correct ages Agegtin pairs as dictionary samples S in a database. The database is stored, for example, in the storage unit12.

In step S1502, the age and error estimation unit150selects a dictionary sample Si=[Si1, Si2, . . . , Sin] stored in the database. In step S1503the age and error estimation unit150determines a distance Libetween the N-dimensional feature vector of the dictionary sample Siselected in step S1502and the N-dimensional feature vector generated from the input image in step S1501by Eq. (1):
Li=Σj=1n|Vj−Sij|.  Eq. (1)
Instead of the Manhattan distance, the distance Limay be a Euclidean distance given by Eq. (2):
Li=Σj=1n(Vj−Sij)2.  Eq. (2)

In step S1504, the age and error estimation unit150determines whether there is an unselected dictionary sample from among the dictionary samples S stored in the database. If there is an unselected dictionary sample (YES in step S1504), the processing returns to step S1502. In step S1502, the age and error estimation unit150selects the unselected dictionary sample. If all the dictionary samples S have been selected (NO in step S1504), the processing proceeds to step S1505.

In step S1505, the age and error estimation unit150selects M similar dictionary samples by using the distances calculated in step S1503.FIG. 11illustrates an example where two dictionary samples are selected (M=2).

In step S1506, the age and error estimation unit150generates a histogram from the correct ages Agegtof the M dictionary samples selected in step S1505.FIG. 12illustrates a histogram with the correct age Agegton the horizontal axis and frequency on the vertical axis. As illustrated inFIG. 13, a histogram can be generated with the correct age Agegton the horizontal axis and 1/distance on the vertical axis. A description will be provided of a case of using the histogram ofFIG. 12.

In step S1507, the age and error estimation unit150determines an average μ1and standard deviations σ1and σ2of the histogram generated in step S1506as illustrated inFIG. 14. The age and error estimation unit150determines the average μ1to be an estimated age, and the standard deviations σ1and σ2to be errors.

In step S1600ofFIG. 3, the display processing unit160displays the estimated age μ1and the errors σ1and σ2determined in step S1507on the display unit13, such as a display. The left half ofFIG. 15illustrates an example where the estimated age and the errors are separately displayed. The right half ofFIG. 15illustrates an example where the estimated age and the errors are displayed as a range. The range in the right half ofFIG. 15is an example of a range of ages.

According to the present exemplary embodiment, an error range is determined and displayed along with age. The user can thus find out the possible range of ages of the target and also whether the target tends to be determined to be younger or older.

In the first exemplary embodiment, M dictionary samples are selected based on the distances from the dictionary samples prepared in advance, and an estimated age and errors are determined from a histogram of the selected M dictionary samples. However, a feature amount extracted from an image can be affected by a change in an illumination environment and a change in facial expression. The age and error estimation unit150then can determine an estimated age and errors, for example, by analyzing a histogram of the selected M dictionary samples and excluding outliers, if any, as illustrated inFIG. 16.

In step S1508, the age and error estimation unit150analyzes the histogram of the selected M dictionary samples, and excludes outliers if any. As a technique for determining an outlier, the age and error estimation unit150can use the SVM discussed in the foregoing paper by Bertozzi et al.

According to the present exemplary embodiment, the effects of a change in the illumination environment and a change in facial expression can be reduced, and an error range can be determined and displayed along with age.

In the first exemplary embodiment, both an estimated age and errors are always determined. However, such a display with a wide error range is equivalent to displaying “unknown age”, as inFIG. 18where the estimated age is 20 and the errors are −15 to +15 years. Displaying “unknown age” can be easier for the user to understand than unreasonably determining and displaying an estimated age and errors.

FIG. 19is a flowchart illustrating details of the processing for displaying an estimated age in step S1600ofFIG. 3.

In step S1601, the display processing unit160determines a difference between the errors σ1and σ2, and determines whether the absolute value of the difference is greater than or equal to a predetermined threshold. If the absolute value of the difference is greater than or equal to the predetermined threshold (YES in step S1601), the processing proceeds to step S1602. If the absolute value of the difference is less than the predetermined threshold (NO in step S1601), the processing proceeds to step S1603.

In step S1602, the display processing unit160displays “unknown age”. Aside from the display “unknown age”, any character string equivalent to “not known” can be used.

In step S1603, the display processing unit160displays the estimated age and the errors as in the first and second exemplary embodiments. Alternatively, a range of ages can be displayed based on the estimated age and the errors.

According to the present exemplary embodiment, an age and errors are displayed only if reliability is high. If reliability is low, a message “unknown age” is displayed. This enables the user to immediately determine the reliability of the displayed data.

According to the foregoing exemplary embodiments, age and errors can be more accurately estimated from an image including a face. The estimations can be output as well.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2018-012406, filed Jan. 29, 2018, which is hereby incorporated by reference herein in its entirety.