Abstract:
Classifiers, which are combinations of a plurality of weak classifiers, for discriminating objects included in detection target images by employing features extracted from the detection target images to perform multi class discrimination including a plurality of classes regarding the objects are generated. When the classifiers are generated, learning is performed for the weak classifiers of the plurality of classes, sharing only the features.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The present invention is related to a classifier generating apparatus and a classifier generating method, for generating classifiers having tree structures for performing multi class classification of objects. The present invention is also related to a program that causes a computer to execute the classifier generating method. 
         [0003]    2. Description of the Related Art 
         [0004]    Conventionally, correction of skin tones in snapshots photographed with digital cameras by investigating color distributions within facial regions of people, and recognition of people who are pictured in digital images obtained by digital video cameras of security systems, are performed. In these cases, it is necessary to detect regions (facial regions) within digital images that correspond to people&#39;s faces. For this reason, various techniques for detecting faces from within digital images have been proposed. Among these techniques, there is a known detecting method that employs appearance models constructed by a machine learning technique. The detecting method that employs appearance models employs a plurality of linked weak classifiers which are obtained by learning a great number of sample images by machine learning. Therefore, this method is robust and superior in detection accuracy. 
         [0005]    The detecting method that employs appearance models will be described as a technique for detecting images within digital images. In this method, features of faces are learned by employing a face sample image group consisting of a plurality of sample images of different faces, and a non face sample image group consisting of a plurality of sample images which are known not to be of faces, as learning data to generate classifiers capable of judging whether an image is an image of a face. Then, partial images are sequentially cut out from an image in which faces are to be detected (hereinafter, referred to as “detection target image”), and the aforementioned classifiers are employed to judge whether the partial images are of faces. Finally, the regions of the detection target image corresponding to the partial images which are judged to be faces are extracted, to detect faces within the detection target image. 
         [0006]    Not only forward facing faces, but images in which faces are rotated along the plane of the image (hereinafter, referred to as “in plane rotation”) and images in which faces are rotated within the plane of the image (hereinafter, referred to as “out of plane rotation”) are input to the classifiers. In the case that learning is performed using learning data that include faces of a variety of orientations (faces in multiple views), it is difficult to realize a general use classifier capable of detecting faces in all orientations. For example, the rotational range of faces that a single classifier is capable of classifying is limited to approximately 30 degrees for in plane rotated images, and approximately 30 to 60 degrees for out of plane rotated images. For this reason, classifiers for faces are constituted by a plurality of strong classifiers for respectively discriminating faces in each of a plurality of orientations, in order to efficiently extract statistical features of faces, which are detection targets. Specifically, multi class classifying methods have been proposed, in which a plurality of strong classifiers, which have performed multi class learning to enable classification of images in each orientation, are prepared. Next, all of the strong classifiers are caused to perform classification regarding whether images are faces in specific orientations. Then, it is judged whether the images represent faces, from the ultimate outputs of each of the strong classifiers. 
         [0007]    Here, it is necessary to select features which are optimal for multi class learning from among a plurality of filters for obtaining features, to perform efficient learning of a plurality of weak classifiers that constitutes strong classifiers of each class during multi class learning. For this reason, a technique for selecting optimal features for multi class learning by searching for effective features and sharing relationships of features among classes has been proposed (refer to Japanese Unexamined Patent Publication No. 2006-251955). In addition, a technique in which a predetermined number of weak classifiers are linked such that weak classifiers of a previous step are shared, from among a plurality of weak classifiers that constitute a classifier of each class, and the weak classifiers are branched according to the number of classes, has also been proposed (refer to U.S. Patent Application Publication No. 20090116693). 
         [0008]    Further, the Joint Boost algorithm has also been proposed as a technique for performing multi class learning. The Joint Boost learning algorithm reduces the total number of weak classifiers and improves the classification performance of classifiers, by causing weak classifiers to be shared among classes (refer to A. Torralba et al., “Sharing features: efficient boosting procedures for multiclass object detection”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, pp. 762-769, 2004). In addition, a technique in which positive teacher data are classified when performing learning of weak classifiers in the Joint Boost algorithm, by labeling positive teacher data that belong to a learning target class as 1, and labeling positive teacher data that belong to other classes as 0 or −1 (refer to M. Tsuchiya and H. Fujiyoshi, “A Method for Constructing a High-Accuracy Classifier Using Divide and Conquer Strategy Based on Boosting”, Technical Report of IEICE, Information and Communication Engineers, Vol. 109, No. 182, PRMU2009-66, pp. 81-86, 2009). 
         [0009]    Weak classifiers obtain features from input patterns, and employ the features as materials for judgment for classifying mechanisms included in the weak classifiers to judge whether the input patterns have certain attributes. In the Joint Boost technique, learning is performed by labeling positive learning data for a learning target class as 1 and labeling positive learning data for other classes as −1, and features are selected such that classification lass error obtained by learning becomes minimal. 
         [0010]    However, in the aforementioned Joint Boost technique, not only the features but the weak classifiers are shared among classes. Therefore, an inconsistency that pieces of positive learning data having labels of different values are input to the same weak classifier, exists. Because such an inconsistency is present in the Joint Boost technique, it becomes difficult for learning to converge such that the classification loss error becomes minimal. In addition, the effects of learning are weakened by the presence of this inconsistency, and the performance of strong classifiers constituted by such weak classifiers is limited to the performance of several weak classifiers in the first step. In addition, because the weak classifiers are shared, performing accurate classification regarding objects among classes becomes difficult. Further, in the case that a complex classification structure, such as a tree structure, is constructed, classes cannot be distinguished because the weak classifiers are shared. As a result, designing branches of such tree structures is difficult. 
       SUMMARY OF THE INVENTION 
       [0011]    The present invention has been developed in view of the foregoing circumstances. It is an object of the present invention to solve the deficiencies of the Joint Boost technique when generating classifiers for performing multi class classification, to improve the converging properties of learning and the performance of classifiers. 
         [0012]    A classifier generating apparatus of the present invention generates classifiers, which are combinations of a plurality of weak classifiers, for discriminating objects included in detection target images by employing features extracted from the detection target images to perform multi class discrimination including a plurality of classes regarding the objects, and is characterized by comprising: 
         [0013]    learning means, for generating the classifiers by performing learning of the weak classifiers of the plurality of classes, sharing only the features. 
         [0014]    The “weak classifiers” are classifiers that judge whether features obtained from images represent objects, in order to classify the objects. In the Joint Boost technique described above, not only features, but weak classifiers, more specifically classifying mechanism-s included in the weak classifiers, are shared among classes during learning. The “learning . . . sharing only the features” performed by the classifier generating apparatus of the present invention differs from the Joint Boost technique in that only the features are shared, and the classifying mechanisms within the weak classifiers are not shared. 
         [0015]    The classifier generating apparatus may further comprise: 
         [0016]    learning data input means, for inputting a plurality of positive and negative learning data for the weak classifiers to perform learning for each of the plurality of classes; and 
         [0017]    filter storage means, for storing a plurality of filters that extract the features from the learning data. In this case, the learning means extracts the features from the learning data using filters selected from those stored in the filter storage means, and performs learning using the extracted features. 
         [0018]    The “filters that extract the features” are those that define the positions of pixels which are employed to calculate features within images, the method for calculating features using the pixel values of pixels at these positions, and the sharing relationship of features among classes. In addition, the features are shared among classes in the present invention. Therefore, the filters for extracting features also define sharing data regarding among which classes features are shared. 
         [0019]    In the classifier generating apparatus of the present invention, the learning means may perform labeling with respect to all of the learning data to be utilized for learning according to degrees of similarity to positive learning data of classes to be learned, to stabilize learning. 
         [0020]    In the classifier generating apparatus of the present invention, the learning means may perform learning by: 
         [0021]    defining a total sum of weighted square errors of the outputs of weak classifiers at the same level in the plurality of classes with respect to the labels and input features; 
         [0022]    defining the total sum of the total sums for the plurality of classes as classification loss error; and 
         [0023]    determining weak classifiers such that the classification loss error becomes minimal. 
         [0024]    A classifier generating method of the present invention generates, which are combinations of a plurality of weak classifiers, for discriminating objects included in detection target images by employing features extracted from the detection target images to perform multi class discrimination including a plurality of classes regarding the objects, and is characterized by comprising: 
         [0025]    a learning step, for generating the classifiers by performing learning of the weak classifiers of the plurality of classes, sharing only the features. 
         [0026]    A program of the present invention is characterized by causing a computer to execute the functions of the classifier generating apparatus of the present invention. 
         [0027]    The present invention generates classifiers, by performing learning such that only features are shared by weak classifiers of a plurality of classes, without sharing the weak classifiers. For this reason, learning not converging as in the Joint Boost technique will not occur. As a result, the converging properties of learning can be improved compared to the Joint Boost technique. In addition, because weak classifiers are not shared, classification among classes can be accurately performed. 
         [0028]    Further, because the weak classifiers of classes that share features are different from each other, designing branches of tree structures is facilitated, when classification structures, such as tree structures, are constructed. For this reason, the classifier generating apparatus and the classifier generating method of the present invention are suited for designing classifiers having tree structures. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0029]      FIG. 1  is a block diagram that illustrates the schematic structure of a classifier generating apparatus according to an embodiment of the present invention. 
           [0030]      FIG. 2  is a diagram that illustrates learning data for m+1 classes (C 1  through Cm and a background class). 
           [0031]      FIGS. 3A and 3B  are diagrams that illustrate examples of learning data. 
           [0032]      FIG. 4  is a diagram that illustrates an example of a filter. 
           [0033]      FIG. 5  is a conceptual diagram that illustrates the processes performed by the classifier generating apparatus according to the embodiment of the present invention. 
           [0034]      FIG. 6  is a diagram that illustrates the results of labeling of learning data in the case that there are eight classes (C 1  through C 7  and a background class). 
           [0035]      FIG. 7  is a diagram that schematically illustrates a multi class classifier constructed by the embodiment of the present invention. 
           [0036]      FIG. 8  is a flow chart that illustrates a learning process. 
           [0037]      FIG. 9  is a diagram that illustrates an example of a histogram. 
           [0038]      FIG. 10  is a diagram that illustrates quantization of a histogram. 
           [0039]      FIG. 11  is a diagram that illustrates an example of a generated histogram. 
           [0040]      FIG. 12  is a diagram that illustrates the construction of a classifier generated by the embodiment of the present invention. 
           [0041]      FIG. 13  is a diagram that illustrates sharing of weak classifiers in the Joint Boost technique. 
           [0042]      FIG. 14  is a diagram that illustrates the construction of a classifier generated by the Joint Boost technique. 
           [0043]      FIG. 15A  is a first diagram that illustrates a classifier constructed by the Joint Boost technique and a classifier constructed by the embodiment of the present invention in a comparative manner. 
           [0044]      FIG. 15B  is a second diagram that illustrates a classifier constructed by the Joint Boost technique and a classifier constructed by the embodiment of the present invention in a comparative manner. 
           [0045]      FIG. 16  is a diagram that illustrates the relationship between input to a decision tree and output thereof. 
       
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
       [0046]    Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.  FIG. 1  is a block diagram that illustrates the schematic structure of a classifier generating apparatus  1  according to an embodiment of the present invention. As illustrated in  FIG. 1 , the classifier generating apparatus  1  of the present invention is equipped with: a learning data input section  10 ; a feature pool  20 ; an initializing section  30 ; and a learning section  40 . 
         [0047]    The learning data input section  10  inputs learning data to be utilized for classifier learning into the classifier generating apparatus  1 . Here, the classifiers which are generated by the present embodiment are those that perform multi class classification. For example, in the case that the classification target object is a face, the classifiers are those that perform multi class classification to classify faces which have different orientations along the plane of the image and different facing directions within the images. Accordingly, the classifier generating apparatus  1  of the present invention generates m classes of classifiers, each capable of classifying faces of a different orientation. For this reason, the learning data input section  10  inputs different learning data x i   Cu  (i=1−N Cu , u=1−m, and N Cu  is the number of pieces of learning data corresponding to each class Cu), that is, learning data in which the orientations and facing directions of faces are different, into the classifier generating apparatus  1 . Note that in the present embodiment, the learning data are image data, in which the sizes and the positions of feature points (such as eyes, noses, etc.) are normalized. 
         [0048]    In addition, learning data x i   bkg  (number of data N bkg ) that represent backgrounds that do not belong to any class of the classification target object are also input into the classifier generating apparatus  1  of the present embodiment. Accordingly, learning data for m+1 classes as illustrated in  FIG. 2  are input and utilized to generate classifiers. 
         [0049]      FIGS. 3A and 3B  are diagram that illustrate examples of learning data. Note that  FIGS. 3A and 3B  illustrate learning data to be utilized for classifiers that classify faces. As illustrated in  FIGS. 3A and 3B , the learning data are of a predetermined image size, and include twelve types of in plane rotated images ( FIG. 3A ), in which faces positioned at a set position (the center, for example) within the images are rotated in 30 degree increments, and three types of out of plane rotated images ( FIG. 3B ), in which faces are positioned at a set position within the images are facing directions of 0 degrees, −30 degrees, and +30 degrees. By preparing learning data in this manner, 12·3=36 class classifiers are generated. Note that the classifiers of each class are constituted by a plurality of linked weak classifiers. 
         [0050]    A plurality of filters ft, for extracting features to be employed to judge whether classification target image data belong in a certain class from the learning data, are stored in the feature pool  20 . The filters ft define the positions of pixels which are employed to calculate features within the learning data, the method for calculating features using the pixel values of pixels at these positions, and the sharing relationship of features among classes.  FIG. 4  is a diagram that illustrates an example of a filter. The filter ft illustrated in  FIG. 4  obtains pixel values (α 1  through αk) of k points or k blocks which are determined advance within classification target image data, and defines that calculations are to be performed using a filter function φ among the pixel values obtained for α 1  through αk. Note that the pixel values α 1  through αk are input to the filter ft, and the calculation results of the filter function φ are output by the filter ft. 
         [0051]    The features are shared among classes in the present embodiment. Therefore, in the case that there are three classes C 1  through C 3 , there will be seven types of sharing relationships, (C 1 , C 2 , C 3 ), (C 1 , C 2 ), (C 1 , C 3 ), (C 2 , C 3 ), (C 1 ), (C 2 ), and (C 3 ). The filters define one of these sharing relationships. Note that the learning data and the filters ft within the feature pool  20  are defined and prepared in advance by users. 
         [0052]      FIG. 5  is a conceptual diagram that illustrates the processes performed by the classifier generating apparatus  1  according to the embodiment of the present invention. Hereinafter, each of the processes performed by the initialing section  30  will be described. First, labeling of learning data will be described. Labeling of learning data is performed to indicate whether pieces of learning data belong to a learning target class during learning of weak classifiers for each class. As shown below, labels for all classes are set for each piece of learning data x i   C . Note that setting labels for all classes clarifies whether each piece of learning data x i   C  (belonging to class C) is to be treated as positive teacher data or negative teacher data during learning of each class Cu. Whether each piece of learning data is to be treated as positive teacher data or negative teacher data is determined by the labels. 
         [0000]      x i   C →(z i   C1 , z i   C2 , . . . , z i   Cm )
 
         [0053]    Here, assuming that C∈ [C 1 , C 2 , . . . , Cm, bkg], in the case that C=Cu (u=1 through m, that is, learning data are not background images), the labeling section  30 A of the initializing section  30  sets the value of the label to +1 (z i   Cu =+1). Conversely, in the case that C=bkg (that is, learning data are background images), the value of the label is set to −1 (z i   Cbkg =−1). In addition, in the case that the learning data are not background images, the values of labels are set further, as described below. In cases that the class of the weak classifier which is a learning target and the class of a piece of learning data does not match, for example, a case in which the class of the learning target weak classifier is C 1 , and the class of a piece of learning data to be utilized for the learning is C 3  (a piece of learning data x i   C3 , for example), the value of the label is set according to the degree of similarity within the class of the learning data for the learning target weak classifier and the learning data of different classes. For example, in cases that the class of the learning target weak classifier and the classes of the learning data are similar, such as a case in which the class of the learning target weak classifier is C 3  and the classes of the learning data are C 2  or C 4 , the values of labels are set to 0 (z i   Cu =0) Conversely, in cases that the class of the learning target weak classifier and the classes of the learning data are not similar, such as a case in which the class of the learning target weak classifier is C 3  and the classes of the learning data are C 1  or C 6 , the values of labels are set to −1 (z i   Cu =−1). Note that pieces of learning data having labels valued +1 are positive teacher data, and pieces of learning data having labels valued −1 are negative teacher data. 
         [0054]    Note that judgments regarding whether learning data of a class for a learning target weak classifier and learning data of another class are similar are performed in the following manner. If a class is adjacent to another class, the learning data of the two classes are judged to be similar, and learning data of other classes are judged to not be similar. Accordingly, in the case that the class of a learning target weak classifier is C 3 , the values of labels z i   C3  for learning data of class C 3  are set to +1, the values of labels z i   C2  and z i   C4  for learning data of adjacent classes C 2  and C 4  are set to 0, and the values of labels for learning data of all other classes are set to −1. Accordingly, in the present embodiment, the values of the labels z i   Cu  assume the three values of −1, 0, and +1. By setting the labels as described above, the stability of learning of the weak classifiers of the classes Cu using the learning data x i   C  can be improved. It is necessary to perform learning for seven classes, each of which are assigned faces having different facing directions in 20 degree increments, from a leftward facing face in profile to a rightward facing face in profile, in order to detect faces and to classify the facing directions thereof. The results of labeling for learning data for such a case are illustrated in  FIG. 6 . 
         [0055]    Note that judgments regarding whether learning data are similar to each other may alternatively be performed by calculating correlations among the learning data of the classes, and judging that the learning data are similar if the correlation is a predetermined value or higher. As a further alternative, users may judge whether learning data of different classes are similar, by manual operations. 
         [0056]    Next, the normalizing process for the number of pieces of learning data performed by the normalizing section  30  will be described. As described above, learning data are prepared for each class. However, there are cases in which the numbers of pieces of learning data differ among classes. In addition, in the classifier generating apparatus  1  of the present embodiment, learning data of classes having labels z i   Cu  valued +1 and −1 with respect to the classes of learning target weak classifiers are employed for learning, and learning data of classes having labels z i   Cu  valued 0 are weighted as 0 and are not utilized, as will be described later. Here, learning data having labels z i   Cu  valued +1 with respect to a certain class Cu are employed as positive learning data, and learning data having labels z i   Cu  valued −1 are employed as negative learning data. If the number of pieces of positive learning data is designated as N+ Cu  and the number of pieces of negative learning data is designated as N− Cu  for a certain class Cu, the number of pieces of learning data N tchr   Cu  for the class Cu can be expressed as N+ Cu +N− Cu . 
         [0057]    In the present embodiment, the numbers of pieces of learning data N tchr   Cu  for all classes Cu are normalized such that the number of pieces of learning data N tchr   Cu  for each class is equal to a number  min N tchr   Cu  of pieces of learning data of a class Cu having the smallest number of pieces of learning data. Note that it is necessary to reduce the number of pieces of learning data for classes other than the class having the smallest number of pieces of learning data  min N tchr   Cu . At this time, randomly selected pieces of learning data from among background learning data x i   bkg  may be removed from the negative learning data, to reduce the number of pieces of learning data. The number of pieces of learning data N tchr   Cu  for each class Cu is updated to become the normalized number of pieces of learning data, and the normalizing process with respect to the learning data is completed. 
         [0058]    Next, the weight setting process administered on the learning data by the weight setting section  30 C will be described. Weighting refers to weighting of the learning data during learning of the weak classifiers of each class Cu. As shown below, weighting values for m classes are set for each piece of learning data x i   C . 
         [0000]      x i   C →w i (w i   C1 , w i   C2 , . . . , w i   Cm )
 
         [0059]    Here, assuming that C∈ [C 1 , C 2 , . . . , Cm, bkg], weighting values w i   Cu  are set with respect to pieces of learning data x i   Cu  within a class Cu, based on the values of the labels z i   Cu  thereof. Specifically, weighting w i   Cu  is set as 1/(2N+ Cu ) for positive learning data having labels z i   Cu  with values of +1 for a certain class Cu, set as 1/(2N− Cu ) for negative learning data having labels z i   Cu  with values of −1 for the class Cu, and set as 0 for positive learning data having labels z i   Cu  with values of 0 for the class Cu. Accordingly, the learning data having labels valued 0 are not utilized for learning of the class Cu. Note that N+ Cu  is the number of pieces of positive learning data within a class Cu, and N− Cu  is the number of negative pieces of negative learning data within a class Cu. 
         [0060]    Note that the classifier initializing section  30 D initializes the classifiers of the classes Cu such that the number of weak classifiers is 0 for each class. That is, the classifiers are initialized such that no weak classifiers are present. 
         [0061]    Next, the learning processes performed by the learning section  40  will be described. The multi class classifiers which are generated by the present embodiment are constituted by strong classifiers H Cu  for each class Cu (that is, H C1 , H C2 , . . . , H Cm ). The strong classifier of each class H Cu  is constituted by a plurality of weak classifiers h t   Cu  (t=1˜n, n is the number of weak classifier steps) which are linked.  FIG. 7  is a diagram that schematically illustrates a multi class classifier constructed by the embodiment of the present invention. In  FIG. 7 , the strong classifiers are connected by the sharing relationships with respect to the features. 
         [0000]    
       
                 
         
             
             
         
       
     
         [0062]      FIG. 8  is a flow chart that illustrates the steps of a learning process. Note that the labeling of learning data, the normalization of the numbers of pieces of learning data, setting the weighting of the learning data, and initialization of the classifiers (initialization processes) of step ST 1  are performed by the initializing section  30 . The learning performed by the learning section  40  proceeds by sequentially determining the weak classifiers h t   Cu  for each step of the classifiers H Cu  for each class. First, the learning section  40  selects a filter ft arbitrarily from among the feature pool  20 . Then, the sharing relationship defined by the selected filter ft is referred to, to determine the classes that features are to be shared among. Here, if classifying mechanisms for calculating scores for classification from the features ft(x i ) within the weak classifiers h t   Cu  are designated as g t   Cu , the process that the weak classifiers h t   Cu  perform using the features can be represented as h t   Cu (x i )=g t   Cu (ft(x i )). Note that h t   Cu (x i ) represents scores which are output by weak classifiers h t   Cu  regarding learning data, of which the features are extracted by the selected filter ft. 
         [0063]    In the present embodiment, histogram type classifying functions are utilized as classifying mechanisms. Weak classifiers are determined by generating histograms that determine scores with respect to the values of features obtained from the learning data. In the classifying mechanisms of histogram type classifying functions, the probability that an object is the object of a classification target class increases as the score is greater in the positive direction, and probability that an object is not the object of a classification target class increases as the score is greater in the negative direction. 
         [0064]    Here, the purpose of learning is to determine weak classifiers. For this reason, the learning section  40  employs the labels z i   Cu  and weights w i   Cu  of the learning data x i  of each class and defines weighted square errors of the labels z i   Cu  and the scores as loss error, to determine the weak classifiers. The learning section defines a total sum of loss errors for all pieces of learning data x i  in this manner. For example, the amount of loss error J C1  for class C 1  may be defined by Formula (1) below. Note that in Formula (1), Ntchr is the total number of pieces of learning data. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           
                             C 
                              
                             
                                 
                             
                              
                             1 
                           
                         
                         = 
                           
                          
                         
                           
                             
                               
                                 w 
                                 1 
                                 
                                   C 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     1 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                   - 
                                   
                                     
                                       h 
                                       t 
                                       
                                         C 
                                          
                                         
                                             
                                         
                                          
                                         1 
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         x 
                                         1 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                           + 
                           
                             
                               
                                 w 
                                 2 
                                 
                                   C 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     2 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                   - 
                                   
                                     
                                       h 
                                       t 
                                       
                                         C 
                                          
                                         
                                             
                                         
                                          
                                         1 
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         x 
                                         2 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             
                               
                                 w 
                                 3 
                                 
                                   C 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     3 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                   - 
                                   
                                     
                                       h 
                                       t 
                                       
                                         C 
                                          
                                         
                                             
                                         
                                          
                                         1 
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         x 
                                         3 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                           + 
                           ⋯ 
                           + 
                           
                             
                               w 
                               Ntchr 
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   Ntchr 
                                 
                                 - 
                                 
                                   
                                     h 
                                     t 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                    
                                   
                                     ( 
                                     
                                       x 
                                       Ntchr 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             
                               N 
                               tchr 
                             
                           
                            
                           
                               
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 
                                   C 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                   - 
                                   
                                     
                                       h 
                                       t 
                                       
                                         C 
                                          
                                         
                                             
                                         
                                          
                                         1 
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         x 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0065]    Then, the learning section  40  defines the total sum of loss errors J Cu  for all classes within each branch (or the root) as classification loss error J wse  according to Formula (2) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           wse 
                         
                         = 
                           
                          
                         
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   
                                     C 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                     
                                     - 
                                     
                                       
                                         h 
                                         t 
                                         
                                           C 
                                            
                                           
                                               
                                           
                                            
                                           1 
                                         
                                       
                                        
                                       
                                         ( 
                                         
                                           x 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                           + 
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   
                                     C 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                     
                                     - 
                                     
                                       
                                         h 
                                         t 
                                         
                                           C 
                                            
                                           
                                               
                                           
                                            
                                           2 
                                         
                                       
                                        
                                       
                                         ( 
                                         
                                           x 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                           + 
                           … 
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 Cm 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                   
                                   - 
                                   
                                     
                                       h 
                                       i 
                                       Cm 
                                     
                                      
                                     
                                       ( 
                                       
                                         x 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               u 
                               = 
                               1 
                             
                             m 
                           
                            
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Cu 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                     
                                     - 
                                     
                                       
                                         h 
                                         i 
                                         Cu 
                                       
                                        
                                       
                                         ( 
                                         
                                           x 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0000]    Here, if the number of classes m is 3, and sharing between classes C 1  and C 2  is defined by the filter ft for calculating features, classification loss error is defined as follows. 
         [0000]    
       
         
           
             
               J 
               wse 
             
             = 
             
               
                 
                     
                    
                   
                     
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           Ntchr 
                         
                          
                         
                           
                             
                               w 
                               i 
                               
                                 C 
                                  
                                 
                                     
                                 
                                  
                                 1 
                               
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   i 
                                 
                                 - 
                                 
                                   
                                     h 
                                     t 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                    
                                   
                                     ( 
                                     
                                       x 
                                       i 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                           2 
                         
                       
                       + 
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           Ntchr 
                         
                          
                         
                           
                             
                               w 
                               i 
                               
                                 C 
                                  
                                 
                                     
                                 
                                  
                                 2 
                               
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   i 
                                 
                                 - 
                                 
                                   
                                     h 
                                     t 
                                     
                                       C 
                                        
                                       
                                           
                                       
                                        
                                       2 
                                     
                                   
                                    
                                   
                                     ( 
                                     
                                       x 
                                       i 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                           2 
                         
                       
                     
                      
                   
                 
                 share 
               
               + 
               
                 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       Ntchr 
                     
                      
                     
                       
                         
                           w 
                           i 
                           
                             C 
                              
                             
                                 
                             
                              
                             3 
                           
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                             
                             - 
                             
                               
                                 h 
                                 t 
                                 
                                   C 
                                    
                                   
                                       
                                   
                                    
                                   3 
                                 
                               
                                
                               
                                 ( 
                                 
                                   x 
                                   i 
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                       
                       2 
                     
                   
                    
                 
                 unshare 
               
             
           
         
       
     
         [0066]    Because classes C 1  and C 2  share features, they may be expressed as: 
         [0000]        h   t   C1 ( x   i )= g   t   C1 ( ft ( x   i )) 
         [0000]    h t   C2 (x i )=g t   C2 (ft(x i )). In contrast, because features are not shared with class C 3 , it is necessary to select a filter separately for only class C 3 , which is not preferable because the amount of calculations will increase. For this reason, the present embodiment defines classification loss error J wse  as a constant classifying function for classes that do not share features. The method by which the constant is calculated will be described later. 
         [0067]    Next, the learning section  40  determines weak classifiers h t   Cu  such that the classification loss error J wse  becomes minimal (Step ST 2 ). In the present embodiment, the classifying mechanisms are histogram type classifying functions. Therefore, weak classifiers h t   Cu  are determined by generating histograms to determine scores with respect to features obtained from learning data. Note that the method by which the weak classifiers h t   Cu  are determined will be described later. After the weak classifiers h t   Cu  are determined in this manner, the weights w i   Cu  of the learning data x i   Cu  are updated as shown in Formula (3) below (step ST 3 ). Note that the updated weights w i   Cu  are normalized as shown in Formula (4) below. In Formula (3), h t   Cu  represents scores output by the weak classifiers with respect to the learning data x i   Cu . 
         [0000]    
       
         
           
             
               
                 
                   
                     w 
                     i 
                     Cu 
                   
                   = 
                   
                     
                       
                         w 
                         i 
                         Cu 
                       
                        
                       
                         ( 
                         old 
                         ) 
                       
                     
                     · 
                     
                        
                       
                         
                           - 
                           
                             zi 
                             Cu 
                           
                         
                         · 
                         
                           ht 
                           Cu 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       w 
                       i 
                       Cu 
                     
                      
                     
                       ( 
                       new 
                       ) 
                     
                   
                   = 
                   
                     
                       w 
                       i 
                       Cu 
                     
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         w 
                         i 
                         Cu 
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0068]    Here, in the case that the score output by a weak classifier h t   Cu  with respect to a piece of learning data is positive, the probability that the learning data is an object is the object of a classification target class is high, and if the score output by a weak classifier h t   Cu  with respect to a piece of learning data is negative, the probability that the learning data is an object is the object of a classification target class is low. For this reason, if scores are positive for pieces of learning data having labels h t   Cu  valued +1, the weighting w i   Cu  is updated to become smaller, and if scores are negative, the weighting w i   Cu  is updated to become greater. Meanwhile, if scores are positive for pieces of learning data having labels h t   Cu  valued −1, the weighting w i   Cu  is updated to become greater, and if scores are negative, the weighting w i   Cu  is updated to become smaller. This means that if a weak classifier h t   Cu  classifies a piece of positive learning data and the score output thereby is positive, the weighting of the piece of learning data is updated to become smaller, and if the score output by the weak classifier h t   Cu  is negative, the weighting of the piece of learning data is updated to become greater. Likewise, if a weak classifier h t   Cu  classifies a piece of negative learning data and the score output thereby is positive, the weighting of the piece of learning data is updated to become greater, and if the score output by the weak classifier h t   Cu  is negative, the weighting of the piece of learning data is updated to become smaller. 
         [0069]    Weak classifiers h t   Cu  are determined and the weights w i   Cu  are updated in this manner. Thereafter, the learning section  40  combines the determined weak classifiers h t   Cu  with the strong classifiers H Cu  for each class, to update the strong classifiers H Cu  (step ST 4 ). Note that in a first process, the strong classifiers H Cu  are initialized to equal zero. Therefore, weak classifiers h t   Cu  for the first step of the strong classifiers H Cu  for each class are added in the first process. Newly determined weak classifiers are added thereafter to the strong classifiers H Cu  for each class in a second and subsequent processes. 
         [0070]    After the strong classifiers H Cu  for each class are updated in this manner, it is judged whether the percentage of correct answers of a combination the weak classifiers h t   Cu  which have been determined up to that point exceeds a predetermined threshold value Th1 (step ST 5 ). That is, the weak classifiers h t   Cu  which have been determined up to that point are combined and utilized to classify positive learning data for each class. It is judged whether the percentage of results of classification that match the correct answers as to whether the pieces of learning data actually represent the object of the classification target class exceeds the threshold value Th1. In cases that the percentage of correct answers exceeds the predetermined threshold value Th1, the classification target object can be classified with a sufficiently high probability by using the weak classifiers h t   Cu  which have been determined up to that point. Therefore, the classifiers are set for the classes (step ST 6 ), and the learning process is completed. In cases that the percentage of correct answers is the threshold value Th1 or less, the process returns to step ST 2 , to determine additional weak classifiers h t   Cu  to be linked with the weak classifiers h t   Cu  which have been determined up to that point. Note that the filters ft which are employed for second and subsequent learning steps are arbitrarily selected. Therefore, there are cases in which the same filter ft is selected again before learning is completed. 
         [0071]    The determined weak classifiers h t   Cu  are linked linearly in the order that they are determined. Note that the determined weak classifiers h t   Cu  may be linked in order from those having the highest percentage of correct answers, to construct the strong classifiers. In addition, score tables are generated for calculating scores according to features, based on histograms with respect to each weak classifier h t   Cu . Note that the histograms themselves may be employed as the score tables. In this case, the classification points of the histograms become the scores. The multi class classifiers are generated by performing learning of classifiers for each class in this manner. 
         [0072]    Next, the process by which weak classifiers are determined will be described. The present embodiment utilizes histogram type classifying functions as classifying mechanisms.  FIG. 9  is a diagram that illustrates an example of a histogram type classifying function. As illustrated in  FIG. 9 , a histogram that functions as a classifying mechanism of a weak classifier h t   Cu  has the values of features as its horizontal axis, and probabilities that an object is the object of a classification target class, that is, scores, as its vertical axis. Note that scores assume values within a range from −1 to +1. In the present embodiment, weak classifiers are determined by generating histograms, more specifically, by determining scores corresponding to each feature within the histograms. Hereinafter, generation of a histogram type classifying function will be described. 
         [0073]    In the present embodiment, weak classifiers h t   Cu  are determined by generating histograms which are classifying mechanisms of the weak classifiers h t   Cu , such that the classification loss error J wse  becomes minimal. Here, the weak classifiers h t   Cu  of each step share features. However, a description will be given for a case in which features are not shared among some classes, to describe a general process. Thereby, the classification loss error J wse  of Formula (2) can be modified to a sum of loss error J share  for classes that share features and loss error J unshare  for classes that do not share features, as shown in Formula (5) below. Note that because h t   Cu (x i )=g t   Cu (ft (x i )), the values of the horizontal axis of the histogram is substituted as ft(x i )=r i  in Formula (5). In Formula (5), the “share” and “unshare” beneath the Σ indicate that a total sum of loss error for classes that share features and a total sum of loss error for classes that do not share features are calculated. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           wse 
                         
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               u 
                               = 
                               1 
                             
                             m 
                           
                            
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Cu 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Cu 
                                     
                                     - 
                                     
                                       
                                         h 
                                         t 
                                         Cu 
                                       
                                        
                                       
                                         ( 
                                         
                                           x 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             
                               ∑ 
                               share 
                             
                              
                             
                               
                                 ∑ 
                                 
                                   i 
                                   = 
                                   1 
                                 
                                 Ntchr 
                               
                                
                               
                                 
                                   
                                     w 
                                     i 
                                     Cu 
                                   
                                    
                                   
                                     ( 
                                     
                                       
                                         z 
                                         i 
                                         Cu 
                                       
                                       - 
                                       
                                         
                                           g 
                                           t 
                                           Cu 
                                         
                                          
                                         
                                           ( 
                                           
                                             r 
                                             i 
                                           
                                           ) 
                                         
                                       
                                     
                                     ) 
                                   
                                 
                                 2 
                               
                             
                           
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             ∑ 
                             unshare 
                           
                            
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Cu 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Cu 
                                     
                                     - 
                                     
                                       
                                         g 
                                         t 
                                         Cu 
                                       
                                        
                                       
                                         ( 
                                         
                                           r 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             J 
                             share 
                           
                           + 
                           
                             J 
                             unshare 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0074]    In Formula (5), if the values of both loss error J share  and loss error J unshare  become minimal, a minimal classification loss error J wse  can be achieved. For this reason, assuming that the number of classes that share features is k, the loss error J share  of classes that share features can be represented by Formula (6) below. Note that in Formula (8), s 1  through sk indicate the numerals of classes, which have been renumbered as those that share features, from among the classes of all of the classifiers. In Formula (6), if the items toward the right side of each line are represented as J Cs1   share  through J Cs1   share , Formula (6) may be rewritten as Formula (7). 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           share 
                         
                         = 
                           
                          
                         
                           
                             ∑ 
                             share 
                           
                            
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Cu 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Cu 
                                     
                                     - 
                                     
                                       
                                         g 
                                         t 
                                         Cu 
                                       
                                        
                                       
                                         ( 
                                         
                                           r 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   
                                     C 
                                     
                                       s 
                                        
                                       
                                           
                                       
                                        
                                       1 
                                     
                                   
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       
                                         C 
                                         
                                           s 
                                            
                                           
                                               
                                           
                                            
                                           1 
                                         
                                       
                                     
                                     - 
                                     
                                       
                                         g 
                                         t 
                                         
                                           C 
                                           
                                             s 
                                              
                                             
                                                 
                                             
                                              
                                             1 
                                           
                                         
                                       
                                        
                                       
                                         ( 
                                         
                                           r 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 
                                   C 
                                   
                                     s 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     
                                       C 
                                       
                                         s 
                                          
                                         
                                             
                                         
                                          
                                         2 
                                       
                                     
                                   
                                   - 
                                   
                                     
                                       g 
                                       t 
                                       
                                         C 
                                         
                                           s 
                                            
                                           
                                               
                                           
                                            
                                           2 
                                         
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         ⋮ 
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             
                               + 
                               ∑ 
                             
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 Csk 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     Csk 
                                   
                                   - 
                                   
                                     
                                       g 
                                       t 
                                       Csk 
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
             
               
                 
                   
                     J 
                     share 
                   
                   = 
                   
                     
                       J 
                       
                         Cs 
                          
                         
                             
                         
                          
                         1 
                       
                       share 
                     
                     + 
                     
                       J 
                       
                         Cs 
                          
                         
                             
                         
                          
                         2 
                       
                       share 
                     
                     + 
                     … 
                     + 
                     
                       J 
                       Csk 
                       share 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0075]    In Formula (7), if the values of J Cs1   share  through J Csk   share , which are the items toward the right side of each line of Formula (7) that represent loss errors for classes that share features, become minimal, a minimal classification loss error J share  can be achieved. Here, because the calculation for minimizing loss errors J Cs1   share  through J Csk   share  are the same for all classes, a calculation for minimizing loss error J Csj   share  for a class Csj (j=1˜k) will be described. 
         [0076]    Here, the values that the features can assume are limited to a predetermined range. The present embodiment segments ranges within the horizontal axis of the histogram and quantizes them into sections P 1  through Pv (v=100, for example), as illustrated in  FIG. 10 . The segmentation is performed in order to efficiently express statistical data regarding the features of a great number of pieces of learning data, and in response to requirements with respect to memory, detection speed, and the like when implementing the classifiers. Note that the vertical axis of the histogram is determined by calculating features from all of the learning data, and then calculating statistical data using Formula (11) below. Thereby, the generated histogram reflects statistical data regarding the classification target object, and therefore classification performance is improved. In addition, the amounts of calculation required to generate histograms and during classification can be reduced. The loss error J Csj   share  a total sum of loss errors within each section P 1  through Pv. Therefore, the loss error J Csj   share  can be modified as shown in Formula (8) below. Note that r i ∈P q (q=1˜v) and the like beneath the Σ in Formula (8) indicate that the total sum of loss errors are calculated for cases that features r i  belong to sections P q . 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           Csj 
                           share 
                         
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 Csj 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     Csj 
                                   
                                   - 
                                   
                                     
                                       g 
                                       t 
                                       Csj 
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             
                               ∑ 
                               
                                 
                                   r 
                                   i 
                                 
                                 ∈ 
                                 
                                   P 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Csj 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Csj 
                                     
                                     - 
                                     
                                       
                                         g 
                                         t 
                                         Csj 
                                       
                                        
                                       
                                         ( 
                                         
                                           r 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             ∑ 
                             
                               
                                 r 
                                 i 
                               
                               ∈ 
                               
                                 P 
                                  
                                 
                                     
                                 
                                  
                                 2 
                               
                             
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 Csj 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     Csj 
                                   
                                   - 
                                   
                                     
                                       g 
                                       t 
                                       Csj 
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                           
                          
                         ⋮ 
                       
                     
                   
                   
                     
                       
                           
                          
                         
                           
                             
                               + 
                               ∑ 
                             
                             
                               
                                 r 
                                 i 
                               
                               ∈ 
                               
                                 P 
                                  
                                 
                                     
                                 
                                  
                                 v 
                               
                             
                           
                            
                           
                             
                               
                                 w 
                                 i 
                                 Csj 
                               
                                
                               
                                 ( 
                                 
                                   
                                     z 
                                     i 
                                     Csj 
                                   
                                   - 
                                   
                                     
                                       g 
                                       t 
                                       Csj 
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0077]    Because the histogram is quantized into sections P 1  through PV as illustrated in  FIG. 10 , the score values g t   Csj (r i )) are constant within each section. Accordingly, g t   Csj (r i ) can be expressed as g t   Csj (r i )=θ eq   Csj , and therefore Formula (8) can be modified into Formula (9) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           Csj 
                           share 
                         
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               q 
                               = 
                               1 
                             
                             v 
                           
                            
                           
                             
                               ∑ 
                               
                                 
                                   r 
                                   i 
                                 
                                 ∈ 
                                 Pq 
                               
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Csj 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Csj 
                                     
                                     - 
                                     
                                       
                                         g 
                                         t 
                                         Csj 
                                       
                                        
                                       
                                         ( 
                                         
                                           r 
                                           i 
                                         
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                          
                         
                           
                             ∑ 
                             
                               q 
                               = 
                               1 
                             
                             v 
                           
                            
                           
                             
                               ∑ 
                               
                                 
                                   r 
                                   i 
                                 
                                 ∈ 
                                 Pq 
                               
                             
                              
                             
                               
                                 
                                   w 
                                   i 
                                   Csj 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       z 
                                       i 
                                       Csj 
                                     
                                     - 
                                     
                                       θ 
                                       q 
                                       Csj 
                                     
                                   
                                   ) 
                                 
                               
                               2 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
         [0078]    Here, the values of labels z i   Csj  in Formula (9) are either +1 or −1. Accordingly, (z i   Csj −θ q   Csj ) is either (1−θ q   Csj ) or (−1−θ q   Csj ). Accordingly, Formula (9) can be modified into Formula (10) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           
                             J 
                             Csj 
                             share 
                           
                           = 
                             
                            
                           
                             
                               ∑ 
                               
                                 q 
                                 = 
                                 1 
                               
                               v 
                             
                              
                             
                               
                                 ∑ 
                                 
                                   
                                     r 
                                     i 
                                   
                                   ∈ 
                                   Pq 
                                 
                               
                                
                               
                                 
                                   
                                     w 
                                     i 
                                     Csj 
                                   
                                    
                                   
                                     ( 
                                     
                                       
                                         z 
                                         i 
                                         Csj 
                                       
                                       - 
                                       
                                         è 
                                         q 
                                         Csj 
                                       
                                     
                                     ) 
                                   
                                 
                                 2 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           = 
                             
                            
                           
                             
                               ∑ 
                               
                                 q 
                                 = 
                                 1 
                               
                               v 
                             
                              
                             
                               { 
                               
                                 
                                   
                                     
                                       ( 
                                       
                                         1 
                                         - 
                                         
                                           è 
                                           q 
                                           Csj 
                                         
                                       
                                       ) 
                                     
                                     2 
                                   
                                   · 
                                   
                                     W 
                                     q 
                                     
                                       Csj 
                                       + 
                                     
                                   
                                 
                                 + 
                                 
                                   
                                     
                                       ( 
                                       
                                         
                                           - 
                                           1 
                                         
                                         - 
                                         
                                           è 
                                           q 
                                           Csj 
                                         
                                       
                                       ) 
                                     
                                     2 
                                   
                                   · 
                                   
                                     W 
                                     q 
                                     
                                       Csj 
                                       - 
                                     
                                   
                                 
                               
                               } 
                             
                           
                         
                       
                     
                   
                    
                   
                     
 
                   
                    
                   wherein 
                    
                   
                     
 
                   
                    
                   
                     
                       
                         W 
                         q 
                         
                           Csj 
                           + 
                         
                       
                       = 
                       
                         
                           ∑ 
                           
                             
                               
                                 r 
                                 i 
                               
                               ∈ 
                               Pq 
                             
                             , 
                             
                                 
                             
                              
                             
                               
                                 z 
                                 i 
                                 Csj 
                               
                               = 
                               1 
                             
                           
                         
                          
                         
                           w 
                           i 
                           Csj 
                         
                       
                     
                     , 
                     
                       
                         W 
                         q 
                         
                           Csj 
                           - 
                         
                       
                       = 
                       
                         
                           ∑ 
                           
                             
                               
                                 r 
                                 i 
                               
                               ∈ 
                               Pq 
                             
                             , 
                             
                                 
                             
                              
                             
                               
                                 z 
                                 i 
                                 Csj 
                               
                               = 
                               
                                 - 
                                 1 
                               
                             
                           
                         
                          
                         
                           w 
                           i 
                           Csj 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0079]    If the value calculated by Formula (10) becomes minimal, the loss error J Csj   share  will become minimal. The value of θ q   Csj  may be determined for each section Pq such that the value calculated by Formula (10) partially differentiated by θ q   Csj  becomes 0. Accordingly, θ q   Csj  can be calculated by Formula (11) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       ∂ 
                       
                         J 
                         Csj 
                         share 
                       
                     
                     
                       ∂ 
                       
                         è 
                         q 
                         Csj 
                       
                     
                   
                   = 
                   
                     
                       0 
                        
                       
                         
 
                       
                       ∴ 
                       
                         
                           
                             - 
                             2 
                           
                            
                           
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   è 
                                   q 
                                   Csj 
                                 
                               
                               ) 
                             
                             · 
                             
                               W 
                               q 
                               
                                 Csj 
                                 + 
                               
                             
                           
                         
                         - 
                         
                           2 
                            
                           
                             
                               ( 
                               
                                 
                                   - 
                                   1 
                                 
                                 - 
                                 
                                   è 
                                   q 
                                   Csj 
                                 
                               
                               ) 
                             
                             · 
                             
                               W 
                               q 
                               
                                 Csj 
                                 - 
                               
                             
                           
                         
                       
                     
                     = 
                     
                       
                         0 
                          
                         
                           
 
                         
                         ∴ 
                         
                           è 
                           q 
                           Csj 
                         
                       
                       = 
                       
                         
                           
                             W 
                             q 
                             
                               Csj 
                               + 
                             
                           
                           - 
                           
                             W 
                             q 
                             
                               Csj 
                               - 
                             
                           
                         
                         
                           
                             W 
                             q 
                             
                               Csj 
                               + 
                             
                           
                           + 
                           
                             W 
                             q 
                             
                               Csj 
                               - 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
         [0080]    Here, W q   Csj+  is the total sum of weights w i   Csj  with respect to pieces of learning data x i  having labels valued 1, that is, positive learning data x i , within sections Pq of the histogram. W q   Csj−  is the total sum of weights w i   Csj  with respect to pieces of learning data x i  having labels valued −1, that is, negative learning data x i , within sections Pq of the histogram. Because the weights w i   Csj  are known, W q   Csj+  and W q   Csj−  can be calculated, and accordingly, the vertical axis of the histogram for sections Pq, that is, the scores θ q   Csj , can be calculated by Formula (11) above. 
         [0081]    The weak classifiers h t   Cu  are determined for the class Csj that shares features, by calculating the values of the vertical axis, that is, the scores θ q   Csj , for all sections P 1  through Pv of a histogram which is the classifying mechanism of the weak classifiers h t   Cu  to generate a histogram such that the loss error J Csj   share  becomes minimal, by the steps described above. An example of a generated histogram is illustrated in  FIG. 11 . Note that in  FIG. 11 , the scores of sections P 1 , P 2 , and P 3  are indicated as θ 1 , θ 2 , and θ 3 , respectively. 
         [0082]    Next, how to minimize loss error J unshare  with respect to classes that do not share features will be considered. The loss error J Csj   unshare  for a class Csj which does not share features can be expressed by Formula (12) below. Here, the characteristic of the present embodiment is that features are shared. Therefore, the scores g t   Cu (r i ) for classes that do not share features are designated as a constant ρ Csj  as shown in Formula (13), and a constant ρ Csj  that yields the minimum loss error J Csj   unshare  is determined. 
         [0000]    
       
         
           
             
               
                 
                   
                     J 
                     Csj 
                     unshare 
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       Ntchr 
                     
                      
                     
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                               Csj 
                             
                             - 
                             
                               
                                 g 
                                 t 
                                 Csj 
                               
                                
                               
                                 ( 
                                 
                                   r 
                                   i 
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
             
               
                 
                   
                     J 
                     Csj 
                     unshare 
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       Ntchr 
                     
                      
                     
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                               Csj 
                             
                             - 
                             
                               ρ 
                               Csj 
                             
                           
                           ) 
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
         [0083]    If the value calculated by Formula (13) is minimized, the loss error J Csj   unshare  can be minimized. In order to minimize the value calculated by Formula (13), ρ Csj  may be set to a value such that the value calculated by Formula (13) partially differentiated by ρ Csj  becomes 0. Accordingly, ρ Csj  can be calculated by Formula (14) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       ∂ 
                       
                         J 
                         Csj 
                         unshare 
                       
                     
                     
                       ∂ 
                       
                         
                           n 
                           ~ 
                         
                         Csj 
                       
                     
                   
                   = 
                   
                     
                       0 
                        
                       
                         
 
                       
                       ∴ 
                       
                         2 
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               w 
                               i 
                               Csj 
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   i 
                                   Csj 
                                 
                                 - 
                                 
                                   
                                     n 
                                     ~ 
                                   
                                   Csj 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                     = 
                     
                       
                         0 
                          
                         
                           
 
                         
                         ∴ 
                         
                           
                             n 
                             ~ 
                           
                           Csj 
                         
                       
                       = 
                       
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               w 
                               i 
                               Csj 
                             
                             · 
                             
                               z 
                               i 
                               Csj 
                             
                           
                         
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             w 
                             i 
                             Csj 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
           
         
       
     
         [0084]    The construction of a classifier generated as described above is illustrated in  FIG. 12 . Note that  FIG. 12  illustrates the first three steps of strong classifiers for four classes. As illustrated in  FIG. 12 , features f 1  are shared among all classes C 1  through C 4  for the weak classifiers of the first step, and classifying mechanism g 1   C1 , g 1   C2 , g 1   C3 , and g 1   C4  are generated for the weak classifiers h of all of the classes C 1  through C 4 . The pieces of learning data employed to generate the classifying mechanisms g 1   Cj  (j=1˜4) differ (with respect to label values and weights). Therefore, the classifying functions calculated by Formula (11) also differ. Accordingly, the weak classifiers h 1   C1  through h 1   C4  differ among the classes. The second step of weak classifiers share features f 2  among classes C 1 , C 3 , and C 4 , and classifying mechanisms g 2   C1 , g 2   C3 , and g 2   C4  are generated for the weak classifiers h of classes C 1 , C 3 , and C 4 . Accordingly, weak classifiers h 1   C1 , h 1   C3 , and h 1   C4 , differ among classes C 1 , C 3 , and C 4 . The third step of weak classifiers share features f 3  between classes C 1  and C 3 , and classifying mechanisms g 3   C1  and g 3   C3  are generated for the weak classifiers h of classes C 1  and C 3 . Accordingly, weak classifiers h 1   C1  and h 1   C3  differ between classes C 1  and C 3 . 
         [0085]    Classifiers generated by the present embodiment will be compared against classifiers generated by the Joint Boost technique.  FIG. 13  is a diagram that illustrates sharing of weak classifiers in the Joint Boost technique.  FIG. 14  is a diagram that illustrates the construction of a classifier generated by the Joint Boost technique.  FIG. 14  illustrates the first three steps of strong classifiers for four classes in the same manner as that of  FIG. 12 . As illustrated in  FIG. 14 , features f 1  are shared among all classes C 1  through C 4  for the weak classifiers of the first step, and a classifying mechanism g 1  is also shared for the weak classifiers h of all of the classes C 1  through C 4 . Accordingly, the weak classifiers h 1   C1  through h 1   C4  for classes C 1  through C 4  are the same. The second step of weak classifiers share features f 2  and a classifying function g 2  among classes C 1 , C 3 , and C 4 . Accordingly, weak classifiers h 1   C1 , h 1   C3 , and h 1   C4 , are the same among classes C 1 , C 3 , and C 4 . The third step of weak classifiers share features f 3  and a classifying function g 3  between classes C 1  and C 3 . Accordingly, weak classifiers h 1   C1  and h 1   C3  are the same between classes C 1  and C 3 .  FIGS. 15A and 15B  are diagrams that illustrate classifiers constructed by the Joint Boost technique and classifiers constructed by the embodiment of the present invention in a comparative manner. 
         [0086]    As described above, the present embodiment generates classifiers, by performing learning such that only features are shared by weak classifiers of a plurality of classes, without sharing the weak classifiers. For this reason, learning not converging as in the Joint Boost technique will not occur. As a result, the converging properties of learning can be improved compared to the Joint Boost technique. In addition, because weak classifiers are not shared, classification among classes can be accurately performed. Further, because the weak classifiers of classes that share features are different from each other, designing branches of tree structures is facilitated, when classification structures, such as tree structures, are constructed. As a result, the classifier generating apparatus and the classifier generating method of the present invention are suited for designing classifiers having tree structures. 
         [0087]    As a result of experiments conducted by the present applicant, the stability and flexibility of learning of the classifiers generated by the present invention are higher than those of classifiers generated by the Joint Boost method. In addition, it was also found that the accuracy and detected speed of classifiers generated by the present invention were higher than those of classifiers generated by the Joint Boost method. 
         [0088]    Note that the embodiment described above employs a histogram type classifying function as a classifying mechanism. Alternatively, it is possible to employ a decision tree as a classifying function. Hereinafter, determination of weak classifiers in the case that a decision tree is employed as the classifying function will be described. In the case that a decision tree is employed as the classifying function as well, the weak classifiers h t   Cu  are determined such that classification loss error J wse  becomes minimal. For this reason, in the case that a decision tree is employed as the classifying function as well, calculations for minimizing the loss error J Csj   share  for a class Csj that shares features as shown in Formula (7) will be described. Note that in the following description, a decision tree is defined as shown in Formula (15) below. In Formula (15), Φ t   Csj  is a threshold value, and is defined in the filter for features. In addition, δ( ) is a delta function that assumes a value of 1 when r i &gt;Φ t   Csj , and assumes a value of 0 in all other cases. Further, a t   Csj  and b t   Csj  are parameters. By defining a decision tree in this manner, the relationship between the input and the output of the decision tree becomes that illustrated in  FIG. 16 . 
         [0000]        g   t   Csj ( r   i )= a   t   Csj δ( r   i &gt;φ t   Csj )+ b   t   Csj    (15)
 
         [0089]    In the embodiment in which the classifying mechanism is a decision tree, the loss error J Csj   share  for a class Csj that shares features can be expressed by Formula (16) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     J 
                     Csj 
                     share 
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       Ntchr 
                     
                      
                     
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                               Csj 
                             
                             - 
                             
                               
                                 g 
                                 t 
                                 Csj 
                               
                                
                               
                                 ( 
                                 
                                   r 
                                   i 
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   16 
                   ) 
                 
               
             
           
         
       
     
         [0090]    If the value calculated by Formula (16) is minimized, the loss error J Csj   share  can be minimized. In order to minimize the value calculated by Formula (16), the values of a t   Csj +b t   Csj  and b t   Csj  may be set to a value such that the value calculated by Formula (16) partially differentiated by a t   Csj  and b t   Csj  respectively becomes 0. The value of a t   Csj +b t   Csj  may be determined by partially differentiating the value calculated by Formula (16) by a t   Csj  as shown in Formula (17) below. Note that r i &gt;φ q   Csj  beneath the Σ in Formula (17) indicates that the total sum of weights w i   Csj  when r i &gt;φ q   Csj  and products of the weights w i   Csj  and labels z i   Csj  are calculated. Accordingly, Formula (17) is equivalent to Formula (18). 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         ∂ 
                         
                           J 
                           Csj 
                           share 
                         
                       
                       
                         ∂ 
                         
                           a 
                           t 
                           Csj 
                         
                       
                     
                     = 
                     0 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         ∂ 
                         
                           J 
                           Csj 
                           share 
                         
                       
                       
                         ∂ 
                         
                           a 
                           t 
                           Csj 
                         
                       
                     
                     = 
                     
                       
                         2 
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               w 
                               i 
                               Csj 
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   i 
                                   Csj 
                                 
                                 - 
                                 
                                   
                                     
                                       g 
                                       t 
                                       Csj 
                                     
                                      
                                     
                                       ( 
                                       
                                         r 
                                         i 
                                       
                                       ) 
                                     
                                   
                                   · 
                                   
                                     
                                       a 
                                       ¨ 
                                     
                                      
                                     
                                       ( 
                                       
                                         
                                           r 
                                           i 
                                         
                                         &gt; 
                                         
                                           
                                             o 
                                             ¨ 
                                           
                                           t 
                                           Csj 
                                         
                                       
                                       ) 
                                     
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       = 
                       0 
                     
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         ∑ 
                         
                           
                             r 
                             i 
                           
                           &gt; 
                           
                             
                               o 
                               ¨ 
                             
                             t 
                             Csj 
                           
                         
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                               Csj 
                             
                             - 
                             
                               a 
                               t 
                               Csj 
                             
                             - 
                             
                               b 
                               t 
                               Csj 
                             
                           
                           ) 
                         
                       
                     
                     = 
                     
                       
                         0 
                          
                         
                           
 
                         
                         ∴ 
                         
                           
                             a 
                             t 
                             Csj 
                           
                           + 
                           
                             b 
                             t 
                             Csj 
                           
                         
                       
                       = 
                       
                         
                           
                             ∑ 
                             
                               
                                 r 
                                 i 
                               
                               &gt; 
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                           
                            
                           
                             
                               w 
                               i 
                               Csj 
                             
                             · 
                             
                               z 
                               i 
                               Csj 
                             
                           
                         
                         
                           
                             ∑ 
                             
                               
                                 r 
                                 i 
                               
                               &gt; 
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                           
                            
                           
                             w 
                             i 
                             Csj 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   17 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       a 
                       t 
                       Csj 
                     
                     + 
                     
                       b 
                       t 
                       Csj 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           z 
                           i 
                           Csj 
                         
                          
                         
                           
                             a 
                             ¨ 
                           
                            
                           
                             ( 
                             
                               
                                 r 
                                 i 
                               
                               &gt; 
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                             ) 
                           
                         
                       
                     
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           
                             a 
                             ¨ 
                           
                            
                           
                             ( 
                             
                               
                                 r 
                                 i 
                               
                               &gt; 
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   18 
                   ) 
                 
               
             
           
         
       
     
         [0091]    Meanwhile, the value of b t   Csj  may be set to a value such that the value calculated by Formula (16) partially differentiated by b t   Csj  becomes 0, as shown in Formula (20) below. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         ∂ 
                         
                           J 
                           Csj 
                           share 
                         
                       
                       
                         ∂ 
                         
                           b 
                           t 
                           Csj 
                         
                       
                     
                     = 
                     0 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         ∂ 
                         
                           J 
                           Csj 
                           share 
                         
                       
                       
                         ∂ 
                         
                           b 
                           t 
                           Csj 
                         
                       
                     
                     = 
                     
                       
                         2 
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             
                               w 
                               i 
                               Csj 
                             
                              
                             
                               ( 
                               
                                 
                                   z 
                                   i 
                                   Csj 
                                 
                                 - 
                                 
                                   
                                     g 
                                     t 
                                     Csj 
                                   
                                    
                                   
                                     ( 
                                     
                                       r 
                                       i 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       = 
                       0 
                     
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           ( 
                           
                             
                               z 
                               i 
                               Csj 
                             
                             - 
                             
                               
                                 
                                   a 
                                   t 
                                   Csj 
                                 
                                  
                                 
                                   ( 
                                   
                                     r 
                                     i 
                                   
                                   ) 
                                 
                               
                               · 
                               
                                 
                                   a 
                                   ¨ 
                                 
                                  
                                 
                                   ( 
                                   
                                     
                                       r 
                                       i 
                                     
                                     &gt; 
                                     
                                       
                                         o 
                                         ¨ 
                                       
                                       t 
                                       Csj 
                                     
                                   
                                   ) 
                                 
                               
                             
                             - 
                             
                               b 
                               t 
                               Csj 
                             
                           
                           ) 
                         
                       
                     
                     = 
                     0 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       weights 
                        
                       
                           
                       
                        
                       
                         w 
                         i 
                         Csj 
                       
                        
                       
                           
                       
                        
                       are 
                        
                       
                           
                       
                        
                       normalized 
                     
                     , 
                     
                       
                         and 
                          
                         
                             
                         
                          
                         therefore 
                          
                         
                             
                         
                          
                         
                           
                             ∑ 
                             
                               i 
                               = 
                               1 
                             
                             Ntchr 
                           
                            
                           
                             w 
                             i 
                             Csj 
                           
                         
                       
                       = 
                       
                         
                           1 
                            
                           
                             
 
                           
                           ∴ 
                           
                             
                               
                                 ∑ 
                                 
                                   i 
                                   = 
                                   1 
                                 
                                 Ntchr 
                               
                                
                               
                                 
                                   w 
                                   i 
                                   Csj 
                                 
                                  
                                 
                                   z 
                                   i 
                                   Csj 
                                 
                               
                             
                             - 
                             
                               
                                 a 
                                 t 
                                 Csj 
                               
                                
                               
                                 
                                   ∑ 
                                   
                                     
                                       r 
                                       i 
                                     
                                      
                                     
                                       &gt; 
                                       t 
                                       Csj 
                                     
                                   
                                 
                                  
                                 
                                   w 
                                   i 
                                   Csj 
                                 
                               
                             
                             - 
                             
                               b 
                               t 
                               Csj 
                             
                           
                         
                         = 
                         
                           
                             0 
                              
                             
                               
 
                             
                             ∴ 
                             
                               
                                 
                                   a 
                                   t 
                                   Csj 
                                 
                                  
                                 
                                   
                                     ∑ 
                                     
                                       
                                         r 
                                         i 
                                       
                                       &gt; 
                                       
                                         
                                           o 
                                           ¨ 
                                         
                                         t 
                                         Csj 
                                       
                                     
                                   
                                    
                                   
                                     w 
                                     i 
                                     Csj 
                                   
                                 
                               
                               + 
                               
                                 b 
                                 t 
                                 Csj 
                               
                             
                           
                           = 
                           
                             
                               ∑ 
                               
                                 i 
                                 = 
                                 1 
                               
                               Ntchr 
                             
                              
                             
                               
                                 w 
                                 i 
                                 Csj 
                               
                                
                               
                                 z 
                                 i 
                                 Csj 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
           
         
       
     
         [0092]    The value of b t   Csj  can be calculated from Formula (18) and Formula (19). 
         [0000]    
       
         
           
             
               
                 
                   
                     b 
                     t 
                     Csj 
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           z 
                           i 
                           Csj 
                         
                          
                         
                           
                             a 
                             ¨ 
                           
                            
                           
                             ( 
                             
                               
                                 r 
                                 i 
                               
                                
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                             ) 
                           
                         
                       
                     
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         Ntchr 
                       
                        
                       
                         
                           w 
                           i 
                           Csj 
                         
                          
                         
                           
                             a 
                             ¨ 
                           
                            
                           
                             ( 
                             
                               
                                 r 
                                 i 
                               
                                
                               
                                 
                                   o 
                                   ¨ 
                                 
                                 t 
                                 Csj 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   20 
                   ) 
                 
               
             
           
         
       
     
         [0093]    Note that with respect to classes that do not share features in the case that the classifying mechanism is a decision tree, the values output by the decision tree may be designated as a constant ρ Csj , and a constant ρ Csj  that yields the minimum loss error J Csj   unshare  may be determined in the same manner as in the case that the classifying mechanism is a histogram. In this case, the constant ρ Csj  may be determined in the same manner as shown in Formula (14). 
         [0094]    As described above, in the case that the classifying mechanism is a decision tree as well, the present invention performs multi class learning sharing only the features. Therefore, learning not converging as in the Joint Boost technique will not occur. As a result, the converging properties of learning can be improved compared to the Joint Boost technique. In addition, because weak classifiers are not shared, classification among classes can be accurately performed. 
         [0095]    An apparatus  1  according to an embodiment of the present invention has been described above. However, a program that causes a computer to function as means corresponding to the learning data input section  10 , the feature pool  20 , the initializing section  30 , and the learning section  40  described above to perform the process illustrated in  FIG. 8  is also an embodiment of the present invention. Further, a computer readable medium in which such a program is recorded is also an embodiment of the present invention.