Patent Abstract:
There is provided a non-transitory computer readable medium storing a program causing a computer to execute a process for attribute estimation. The process includes: extracting, for each user, feature quantities of plural pieces of image information that are associated with attributes of the user; integrating the extracted feature quantities for each user; and performing learning, input of the learning being an integrated feature quantity that has been obtained as a result of integration for each user, output of the learning being one attribute, and generating a learning model.

Full Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2014-120377 filed Jun. 11, 2014. 
     
    
     BACKGROUND 
     Technical Field 
       [0002]    The present invention relates to a non-transitory computer readable medium, an information processing apparatus, and an attribute estimation method. 
       SUMMARY 
       [0003]    According to an aspect of the invention, there is provided a non-transitory computer readable medium storing a program causing a computer to execute a process for attribute estimation. The process includes: extracting, for each user, feature quantities of plural pieces of image information that are associated with attributes of the user; integrating the extracted feature quantities for each user; and performing learning, input of the learning being an integrated feature quantity that has been obtained as a result of integration for each user, output of the learning being one attribute, and generating a learning model. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0004]    Exemplary embodiments of the present invention will be described in detail based on the following figures, wherein: 
           [0005]      FIG. 1  is a block diagram illustrating an example of a configuration of an information processing apparatus according to a first exemplary embodiment; 
           [0006]      FIGS. 2A to 2C  are schematic diagrams for describing learning operations performed by the information processing apparatus; 
           [0007]      FIG. 3  is a schematic diagram for describing attribute estimation operations performed by the information processing apparatus; 
           [0008]      FIG. 4  is a flowchart illustrating an example of learning operations performed by the information processing apparatus; 
           [0009]      FIG. 5  is a flowchart illustrating an example of attribute estimation operations performed by the information processing apparatus; 
           [0010]      FIG. 6  is a block diagram illustrating an example of a configuration of an information processing apparatus according to a second exemplary embodiment; 
           [0011]      FIGS. 7A and 7B  are schematic diagrams for describing a method of creating image label information in learning operations performed by the information processing apparatus; 
           [0012]      FIG. 8  is a schematic diagram illustrating a configuration of image label information; 
           [0013]      FIG. 9  is a schematic diagram for describing attribute estimation operations performed by the information processing apparatus; 
           [0014]      FIG. 10  is a flowchart illustrating an example of learning operations performed by the information processing apparatus; and 
           [0015]      FIG. 11  is a flowchart illustrating an example of attribute estimation operations performed by the information processing apparatus. 
       
    
    
     DETAILED DESCRIPTION 
     First Exemplary Embodiment 
     Configuration of Information Processing Apparatus 
       [0016]      FIG. 1  is a block diagram illustrating an example of a configuration of an information processing apparatus according to a first exemplary embodiment. 
         [0017]    An information processing apparatus  1  is constituted by a central processing unit (CPU) and the like, and includes a controller  10  that controls each unit and that executes various programs, a memory  11  that is constituted by a storage medium, such as a flash memory, and that stores information, and a communication unit  12  that performs external communication over a network. 
         [0018]    The controller  10  executes an attribute estimation program  110  described below to thereby function as an image obtaining unit  100 , an image feature quantity extraction unit  101 , a feature quantity integration unit  102 , a learning model generation unit  103 , a user attribute estimation unit  104 , and the like. 
         [0019]    The image obtaining unit  100  obtains learning image information  111  from the memory  11  in a learning stage. The learning image information  111  is image information that has been posted on a social networking service (SNS), and is image information for learning to which attributes of a user typically including sex, age, occupation, and the like have been assigned in advance as verified ones. In addition to information stored in the memory  11 , the learning image information  111  may be information that has been obtained from outside or that has been transmitted and received from outside via the communication unit  12 , or may be information prepared by manually assigning attributes to image information to which attributes have not been assigned in advance. 
         [0020]    Furthermore, the image obtaining unit  100  obtains image information  116  from the memory  11  in an attribute estimation stage. It is assumed that the image information  116  is image information posted on an SNS, but attributes of a user who has posted the image information have not been assigned and are unknown. The image obtaining unit  100  not only obtains the image information  116  from the memory  11 , but may also receive image information obtained from outside or transmitted from outside via the communication unit  12 . 
         [0021]    The image feature quantity extraction unit  101  extracts feature quantities from the learning image information  111  or the image information  116  obtained by the image obtaining unit  100 . The image feature quantity extraction unit  101  stores the feature quantities in the memory  11  as feature quantity information  112 . For example, the image feature quantity extraction unit  101  first extracts a local feature quantity using scale-invariant feature transform (SIFT) when extracting a feature quantity, performs clustering on the extracted local feature quantity using k-means, and takes K cluster centers that have been obtained as codewords. Next, the image feature quantity extraction unit  101  generates a bag-of-features (BoF) histogram for a neighbor codeword using a k-nearest neighbor algorithm and spatial pyramid matching (SPM), and assumes the histogram to be a feature quantity. 
         [0022]    The feature quantity integration unit  102  integrates the feature quantities extracted by the image feature quantity extraction unit  101  for each user, and generates integrated feature quantity information  113 . The feature quantity integration unit  102  adds up, for each user, BoF histograms that are feature quantities, for example, and obtains the integrated feature quantity information  113  by performing normalization using the number of feature quantities. 
         [0023]    The learning model generation unit  103  performs learning, the input of the learning being the integrated feature quantity information  113  that has been generated by the feature quantity integration unit  102  integrating, for each user, the feature quantities extracted from the learning image information  111 , the output of the learning being attributes of the user, and generates a learning model  114 . The learning model generation unit  103  uses an algorithm, such as a support vector machine (SVM), for example, when performing learning. 
         [0024]    The user attribute estimation unit  104  estimates, by using the learning model  114 , attribute information from the integrated feature quantity information  113  that has been generated by the feature quantity integration unit  102  integrating, for each user, the feature quantities extracted from the image information  116 , and generates user attribute information  117  that is associated with the user. 
         [0025]    The memory  11  stores the attribute estimation program  110  that causes the controller  10  to operate as the image obtaining unit  100 , the image feature quantity extraction unit  101 , the feature quantity integration unit  102 , the learning model generation unit  103 , and the user attribute estimation unit  104 , the learning image information  111 , the feature quantity information  112 , the integrated feature quantity information  113 , the learning model  114 , user information  115 , the image information  116 , the user attribute information  117 , and the like. 
         [0026]    The user information  115  is information, such as a user identification (ID), for identifying a user who uses an SNS. 
         [0027]    Note that the learning image information  111 , the user information  115 , the image information  116 , and the user attribute information  117  may be obtained from an external SNS server via the communication unit  12 . 
       Operations Performed by Information Processing Apparatus 
       [0028]    Next, actions in the first exemplary embodiment will be described for (1) learning operations and (2) attribute estimation operations separately. 
       (1) Learning Operations 
       [0029]      FIG. 4  is a flowchart illustrating an example of learning operations performed by the information processing apparatus  1 .  FIGS. 2A to 2C  are schematic diagrams for describing learning operations performed by the information processing apparatus  1 . 
         [0030]    First, the image obtaining unit  100  obtains from the memory  11  the learning image information  111  regarding users who have a specific attribute (step S 1 ). 
         [0031]    For example, in examples illustrated in  FIGS. 2A to 2C , pieces of image information  111   a   1 ,  111   a   2 ,  111   a   3 , and so on are image information posted by a user  115   a , pieces of image information  111   b   1 ,  111   b   2 ,  111   b   3 , and so on are image information posted by a user  115   b , and pieces of image information  111   c   1 ,  111   c   2 ,  111   c   3 , and so on are image information posted by a user  115   c , and the users  115   a  to  115   c  have been assigned in advance respective attributes  111   a   t  to  111   c   t . 
         [0032]    A set of the attributes  111   a   t  and the pieces of image information  111   a   1 ,  111   a   2 ,  111   a   3 , and so on, a set of the attributes  111   b   t  and the pieces of image information  111   b   1 ,  111   b   2 ,  111   b   3 , and so on, and a set of the attributes  111   c   t  and the pieces of image information  111   c   1 ,  111   c   2 ,  111   c   3 , and so on described above are the learning image information  111 . If “male” has been selected as a specific attribute, for example, the image obtaining unit  100  obtains the pieces of image information  111   a   1 ,  111   a   2 ,  111   a   3 , and so on regarding the user  115   a  and the pieces of image information  111   c   1 ,  111   c   2 ,  111   c   3 , and so on regarding the user  115   c . Note that a specific attribute may be selected by an administrator of the information processing apparatus  1 , or the information processing apparatus  1  may select “male”, “female”, and so on in order. 
         [0033]    The image obtaining unit  100  may obtain image information regarding a user to which attributes have not been assigned in advance, the attributes being assigned by the user or an administrator of the information processing apparatus  1  thereafter, and may handle the image information and the attributes as the learning image information  111 . 
         [0034]    Next, the image feature quantity extraction unit  101  extracts feature quantities respectively from the pieces of image information  111   a   1 ,  111   a   2 ,  111   a   3 , and so on and the pieces of image information  111   c   1 ,  111   c   2 ,  111   c   3 , and so on that have been obtained by the image obtaining unit  100  (step S 2 ). The image feature quantity extraction unit  101  stores the feature quantities in the memory  11  as the feature quantity information  112 . 
         [0035]    Next, the feature quantity integration unit  102  integrates the feature quantities extracted by the image feature quantity extraction unit  101  for each user, and generates the integrated feature quantity information  113  (step S 3 ). That is, the feature quantities extracted from the pieces of image information  111   a   1 ,  111   a   2 ,  111   a   3 , and so on are integrated and assumed to be integrated feature quantity information  113   a  regarding the user  115   a , and the feature quantities extracted from the pieces of image information  111   c   1 ,  111   c   2 ,  111   c   3 , and so on are integrated and assumed to be integrated feature quantity information  113   c  regarding the user  115   c.    
         [0036]    Next, the learning model generation unit  103  performs learning, the input of the learning being the integrated feature quantity information  113   a  and  113   c , the output of the learning being an attribute of the users, that is, “male”, generates the learning model  114  (step S 4 ), and stores the learning model  114  in the memory  11  (step S 5 ). 
         [0037]    Next, attribute estimation operations using the above-described learning model  114  will be described. 
       (2) Attribute Estimation Operations 
       [0038]      FIG. 5  is a flowchart illustrating an example of attribute estimation operations performed by the information processing apparatus  1 .  FIG. 3  is a schematic diagram for describing attribute estimation operations performed by the information processing apparatus  1 . 
         [0039]    First, the image obtaining unit  100  refers to the user information  115 , as illustrated in  FIG. 3 , determines a user  115   n  to be a user who is to be a target of attribute estimation, and obtains pieces of image information  116   n   1 ,  116   n   2 ,  116   n   3 , and so on that have been posted by the user  115   n , from the memory  11  (step S 11 ). It is assumed that attributes  111   n   t  of the user  115   n  are unknown. The image obtaining unit  100  may receive plural pieces of image information transmitted from a user for which attribute estimation is desired, and may assume the user to be a target of attribute estimation. That is, the image obtaining unit  100  need not refer to the user information  115 . 
         [0040]    Next, the image feature quantity extraction unit  101  extracts feature quantities respectively from the pieces of image information  116   n   1 ,  116   n   2 ,  116   n   3 , and so on that have been obtained by the image obtaining unit  100  (step S 12 ). 
         [0041]    Next, the feature quantity integration unit  102  integrates the feature quantities extracted by the image feature quantity extraction unit  101 , and generates the integrated feature quantity information  113  (step S 13 ). That is, the feature quantities extracted from the pieces of image information  116   n   1 ,  116   n   2 ,  116   n   3 , and so on are integrated and assumed to be integrated feature quantity information  113   n  regarding the user  115   n.    
         [0042]    Next, the user attribute estimation unit  104  estimates attribute information  117   n  from the integrated feature quantity information  113   n , by using the learning model  114  generated as described in “(1) Learning Operations” (step S 14 ), and, if an attribute “male” is obtained, stores the attribute in the memory  11  as the user attribute information  117  while associating the attribute with the user  115   n  (step S 15 ). 
       Second Exemplary Embodiment 
       [0043]    A second exemplary embodiment is different from the first exemplary embodiment in that learning is performed by taking into consideration not only user attributes but also labels assigned to image information, and an attribute of a user who has posted image information is estimated on the basis of the result of the learning. 
         [0044]      FIG. 6  is a block diagram illustrating an example of a configuration of an information processing apparatus according to the second exemplary embodiment. 
         [0045]    An information processing apparatus  2  is constituted by a CPU and the like, and includes a controller  20  that controls each unit and that executes various programs, a memory  21  that is constituted by a storage medium, such as a flash memory, and that stores information, and a communication unit  22  that performs external communication over a network. 
         [0046]    The controller  20  executes an attribute estimation program  210  described below to thereby function as an image obtaining unit  200 , an image feature quantity extraction unit  201 , an image label assigning unit  202 , a learning model generation unit  203 , an image label estimation unit  204 , a user attribute estimation unit  205 , and the like. 
         [0047]    The image obtaining unit  200  has functions similar to the image obtaining unit  100  in the first exemplary embodiment. The image feature quantity extraction unit  201  has functions similar to the image feature quantity extraction unit  101  in the first exemplary embodiment. The image feature quantity extraction unit  201  stores feature quantities that have been extracted in the memory  21  as feature quantity information  212 . 
         [0048]    The image label assigning unit  202  assigns image label information  213  that is generated by combining user attributes and image contents in accordance with the contents of learning image information  211 . 
         [0049]    The learning model generation unit  203  performs learning, the input of the learning being feature quantities that have been extracted by the image feature quantity extraction unit  201  from the learning image information  211 , the output of the learning being image labels assigned to the learning image information  211 , and generates a learning model  214 . 
         [0050]    The image label estimation unit  204  calculates, by using the learning model  214 , scores of the image labels from feature quantities that have been extracted by the image feature quantity extraction unit  201  from image information  216 , and estimates image labels to be associated with the image information  216  on the basis of the scores. 
         [0051]    The user attribute estimation unit  205  integrates the image labels that have been estimated by the image label estimation unit  204  for each user, estimates an attribute of the user by comparing the scores of respective attributes, and generates user attribute information  217  that is associated with the user. 
         [0052]    The memory  21  stores the attribute estimation program  210  that causes the controller  20  to operate as the image obtaining unit  200 , the image feature quantity extraction unit  201 , the image label assigning unit  202 , the learning model generation unit  203 , the image label estimation unit  204 , and the user attribute estimation unit  205 , the learning image information  211 , the feature quantity information  212 , the image label information  213 , the learning model  214 , user information  215 , the image information  216 , the user attribute information  217 , and the like. 
       Operations Performed by Information Processing Apparatus 
       [0053]    Next, actions in the second exemplary embodiment will be described for (1) learning operations and (2) attribute estimation operations separately. 
       (1) Learning Operations 
       [0054]      FIG. 10  is a flowchart illustrating an example of learning operations performed by the information processing apparatus  2 .  FIGS. 7A and 7B  are schematic diagrams for describing a method of creating the image label information  213  in learning operations performed by the information processing apparatus  2 .  FIG. 8  is a schematic diagram illustrating a configuration of the image label information  213 . 
         [0055]    First, the image label assigning unit  202  accepts selection of an attribute type for which learning (estimation) is desired (step S 31 ). Description will be given below while assuming that, as illustrated in  FIG. 7A , there are attribute types including an attribute type  217   a  that indicates “sex”, an attribute type  217   b  that indicates “age”, and so on, and that the attribute type  217   a  that indicates “sex” has been selected by an administrator or the like. 
         [0056]    Next, the image label assigning unit  202  combines the attribute type  217   a  that has been selected and image contents  213   a  illustrated in  FIG. 7B , and creates the image label information  213  illustrated in  FIG. 8  (step S 32 ). The image label information  213  is obtained by combining attributes included in the attribute type  217   a  and the image contents  213   a , and therefore, 30 image labels are created, the number “30” being obtained by multiplying the number of attributes “3” by the number of labels “10”. 
         [0057]    Next, the image label assigning unit  202  assigns the created image labels of the image label information  213  to the learning image information  211  in accordance with operations performed by an administrator or the like (step S 33 ). Note that the learning image information  211  to which image labels have been assigned in advance may be prepared. Furthermore, a configuration may be employed in which feature quantities of the learning image information  211  are extracted, clustering is performed on the learning image information  211  on the basis of the feature quantities, the image label information  213  is created by using names, such as “class 1”, “class 2”, “class 3”, and so on, that are based on the clustering classification, instead of using the image contents  213   a , and the image labels are automatically assigned. 
         [0058]    Next, the image feature quantity extraction unit  201  extracts feature quantities from the learning image information  211  (step S 34 ). The image feature quantity extraction unit  201  stores the feature quantities in the memory  21  as the feature quantity information  212 . 
         [0059]    Next, the learning model generation unit  203  performs learning, the input of the learning being the feature quantities that have been extracted by the image feature quantity extraction unit  201  from the learning image information  211 , the output of the learning being image labels assigned to the learning image information  211 , generates the learning model  214  (step S 35 ), and stores the learning model  214  in the memory  21  (step S 36 ). 
       (2) Attribute Estimation Operations 
       [0060]      FIG. 11  is a flowchart illustrating an example of attribute estimation operations performed by the information processing apparatus  2 .  FIG. 9  is a schematic diagram for describing attribute estimation operations performed by the information processing apparatus  2 . 
         [0061]    First, the image obtaining unit  200  refers to the user information  215 , determines a user who is to be a target of attribute estimation, and obtains pieces of image information posted by the user, from the memory  21  (step S 41 ). It is assumed that attributes of the user are unknown. 
         [0062]    Next, the image feature quantity extraction unit  201  extracts feature quantities from the pieces of image information obtained by the image obtaining unit  200  (step S 42 ). The image feature quantity extraction unit  201  stores the feature quantities in the memory  21  as the feature quantity information  212 . 
         [0063]    Next, the image label estimation unit  204  calculates, by using the learning model  214  generated as described in “(1) Learning Operations”, scores that are estimation values, each indicating the degree of matching with a corresponding image label, as illustrated in  FIG. 9 , from the feature quantities that have been extracted by the image feature quantity extraction unit  201  from the image information  216 , and obtains score calculation results  204   a  (step S 43 ). In an example illustrated in  FIG. 9 , the score calculation results  204   a  are results of calculation of the scores of all image labels, and items in the score calculation results  204   a  are sorted in descending order of score. 
         [0064]    Next, the user attribute estimation unit  205  integrates the scores of image labels for each attribute on the basis of the score calculation results  204   a  (step S 44 ). For example, the scores of image labels that include “female” are added up, and the score of the attribute “female” is obtained. The scores of image labels that include “male” are added up, and the score of the attribute “male” is obtained. Similarly, the scores of image labels that include “unknown” are added up, and the score of the attribute “unknown” is obtained. Note that a method of integrating scores is not limited to a method using addition, and may be a method in which the highest score is selected for each attribute or may be based on other calculation methods. 
         [0065]    Next, in a case where the score of the attribute “female”, which is 3.56, the score of the attribute “male”, which is 2.11, and the score of the attribute “unknown”, which is 0.22, are obtained, for example, the user attribute estimation unit  205  compares these values, estimates that the attribute “female” that has the highest score is an attribute of the user (step S 45 ), and stores the attribute in the memory  21  as the user attribute information  217  while associating the attribute with the user (step S 46 ). 
         [0066]    In a case where an attribute is not alternatively determined but may have plural values, the user attribute estimation unit  205  estimates each attribute, an integrated value of which exceeds a predetermined threshold, to be an attribute of the user. 
       Other Exemplary Embodiments 
       [0067]    Note that the present invention is not limited to the exemplary embodiments described above, and various modifications may be made without departing from the spirit of the present invention. In the first exemplary embodiment, while the functions of the image obtaining unit  100 , the image feature quantity extraction unit  101 , the feature quantity integration unit  102 , the learning model generation unit  103 , and the user attribute estimation unit  104  of the controller  10  are implemented only by the information processing apparatus  1 , some of the functions may be implemented by other server apparatuses or terminal apparatuses. Similarly, in the second exemplary embodiment, some of the functions of the image obtaining unit  200 , the image feature quantity extraction unit  201 , the image label assigning unit  202 , the learning model generation unit  203 , the image label estimation unit  204 , and the user attribute estimation unit  205  of the controller  20  may be implemented by other server apparatuses or terminal apparatuses. 
         [0068]    The learning image information  111 , the feature quantity information  112 , the integrated feature quantity information  113 , the learning model  114 , the user information  115 , the image information  116 , and the user attribute information  117  need not be stored in the memory  11  of the information processing apparatus  1 , and the learning image information  211 , the feature quantity information  212 , the image label information  213 , the learning model  214 , the user information  215 , the image information  216 , and the user attribute information  217  need not be stored in the memory  21  of the information processing apparatus  2 . These pieces of information may be obtained from an external database or an external apparatus, or may be transmitted and received from an external apparatus without being stored in the memory  11  or the memory  21 , and may be used by each unit. 
         [0069]    In the exemplary embodiments described above, while the functions of the image obtaining unit  100 , the image feature quantity extraction unit  101 , the feature quantity integration unit  102 , the learning model generation unit  103 , and the user attribute estimation unit  104  of the controller  10 , and the functions of the image obtaining unit  200 , the image feature quantity extraction unit  201 , the image label assigning unit  202 , the learning model generation unit  203 , the image label estimation unit  204 , and the user attribute estimation unit  205  of the controller  20  are implemented by the programs, all or some of the units may be implemented by hardware, such as an application-specific integrated circuit (ASIC). The programs used in the above-described exemplary embodiments may be stored in a recording medium, such as a compact disc read-only memory (CD-ROM), and provided. Furthermore, the steps described in the above exemplary embodiments may be interchanged, deleted, or added, for example, without changing the spirit of the present invention. 
         [0070]    The foregoing description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.

Technology Classification (CPC): 6