Abstract:
A RBF pattern recognition method for reducing classification errors is provided. An optimum RBF training approach is obtained for reducing an error calculated by an error function. The invention continuously generates the updated differences of parameters in the learning process of recognizing training samples. The modified parameters are employed to stepwise adjust the RBF neural network. The invention can distinguish different degrees of importance and learning contributions among the training samples and evaluate the learning contribution of each training sample for obtaining differences of the parameters of the training samples. When the learning contribution is larger, the updated difference is larger to speed up the learning. Thus, the difference of the parameters is zero when the training samples are classified as the correct pattern type.

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention relates to the technical field of image recognition and, more particularly, to a radial basis function (RBF) neural network pattern recognition method for reducing classification errors. 
   2. Description of Related Art 
   Conventionally, in the field of data processing, a RBF neural network is used in a broad variety of applications such as fault detection, fault diagnosis, speech recognition, image recognition, etc. The RBF neural network is advantageous as compared with an error back-propagation learning algorithm. This is because parameters should be repeatedly updated and trained for increasing a precision of a computer system for pattern recognition. Also, as compared with other networks, the RBF neural network is faster to be trained. It is found that all networks may have the same drawback of incompleteness of data collection in training samples. Advantageously, the RBF neural network provides low output responses to inputs that fall into regions of the data space where there is no training sample. Moreover, the RBF neural network can distinguish different learning contributions among training samples for generating different quantities of learning which are in turn used to update and enhance features of the computer system. That is, the RBF neural network may exploit localized basis functions such that learning of each input sample affects a specialized subset of the network parameters. 
   In a well-known process of training the computer system by the RBF neural network, a plurality of different pattern types are stored in the computer system. Also, samples for training the computer system belong to a specific one of the pattern types. The computer system calculates an output value of each training sample in respective pattern data. In a case that a maximum output value appears in the specific pattern type associated with the training samples, it means a correct recognition of the computer system. In another case that the maximum output value appears in any of other pattern types, it means that a gradient descent optimization algorithm is required to calculate a least means square error (LMSE) between the output value of in the specific pattern type and the maximum output value thereof. Consequently, the LMSE is used to generate parameters for updating the computer system in order to increase recognition accuracy. 
   However, the recognition accuracy of the LMSE is low even though it is advantageous in calculating an approximation function. Further, error may be accumulated as a result of an increase of the number of erroneous classifications in a maximum-likelihood classifier because the LMSE cannot correspond to a classification error calculated by the maximum-likelihood classifier. This can greatly reduce a performance of the computer system. Thus, there is desired to provide a novel pattern recognition method to mitigate and/or obviate the aforementioned problems. 
   SUMMARY OF THE INVENTION 
   An object of the present invention is to provide a radial basis function (RBF) neural network pattern recognition method for reducing classification errors. The invention generates the updated differences of parameters by recognizing a feature vector of training samples. The modified parameters are employed to stepwise adjust a setting of the RBF neural network, resulting in an increase of recognition accuracy. 
   Another object of the present invention is to distinguish different degrees of importance and learning contributions among the training samples and evaluate the learning contribution of each training sample for obtaining differences of the parameters of the training samples. 
   To achieve the object, the present invention provides a pattern recognition method for reducing classification errors by using RBF neural network. The RBF neural network includes an input layer, a kernel layer having a plurality of pattern types, each having a plurality of hidden neurons, each corresponding to a kernel function, and an output layer having a plurality of output neurons, each corresponding to a pattern type of the kernel layer. The pattern recognition method for reducing classification errors comprises the steps of (A) inputting a feature vector formed by a plurality of training samples to the input layer wherein the feature vector comprises a plurality of features each forming a corresponding one of a plurality of input neurons in the input layer; (B) calculating a measured value of the kernel layer relative to the hidden neuron by each of the input neurons based on a kernel function of each hidden neuron; (C) for each output neuron of the output layer, multiplying the measured value of the kernel layer corresponding to the pattern type associated with the output neuron by a weight for obtaining a measured value of the output layer corresponding to the pattern type; (D) finding a maximum one of the measured values of the output layer and an error relative to a correct one of the measured values of the output layer; and (E) modifying values of parameters of the kernel function and calculating a weight of the measured values of the output layer based on the error. 
   Other objects, advantages, and novel features of the invention will become more apparent from the detailed description when taken in conjunction with the accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  schematically illustrates a RBF neural network according to the invention; and 
       FIG. 2  is a flow chart illustrating a training process of pattern recognition according to the invention. 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
   With reference to  FIG. 1 , there is shown a RBF neural network for performing the pattern recognition method for reducing classification errors according to the invention. The RBF neural network comprises an input layer  10 , a kernel layer  20 , and an output layer  30 . The kernel layer  20  comprises K pattern types. Each pattern type has a plurality of hidden neurons  21 , each corresponding to a kernel function as a reference model for representing, for example, a person&#39;s face. The kernel function is a facial expression model of the person, typically an exemplary facial expression. Alternatively, the kernel function is an image reference model of one of a plurality of different taken angles. That is, respective hidden neuron  21  of each pattern type corresponds one of a plurality of different facial expressions taken in different angles. The output layer  30  comprises K output neurons  31 , each corresponding to a pattern type of the kernel layer  20 . 
   With reference to  FIG. 2 , there is shown a flow chart illustrating a training process of pattern recognition of the invention. In step S 201 , a feature vector formed by training samples is inputted to the input layer  10 . The feature vector comprises N features F i , 1≦i≦N, such as pattern area of the training sample, measured length, aspect ratio, image luminance, etc. Moreover, each feature will form a corresponding input neuron x i , 1≦i≦N, in the input layer  10 . 
   In step S 202 , there is determined a measured value of the kernel layer  20  relative to the hidden neuron  21  by each of the input neurons x i  based on a kernel function of each hidden neuron  21 . In this embodiment, the kernel function is a Gaussian function 
               O   kj     =     exp   ⁡     (     -       ∑     i   =   1     N     ⁢         w   kji     (       x   i     -     C   kji       )     2         )         ,         
where k is the k-th pattern type of the kernel layer  20 , j is the j-th hidden neuron  21  of the k-th pattern type, O kj  is the corresponding kernel function of the j-th hidden neuron  21  in the k-th pattern type, C kji  is the corresponding reference model feature of a reference model of each hidden neuron  21 , N is the number of the hidden neurons  21 , and w kji  is a weight of a reference model of C kji . In this embodiment, a clustering approach is employed to fetch a plurality of appropriate clusters from the pattern types. Thus, the initial value of C kji  is a cluster means and the initial value of w kji  is an inverse of cluster variance.
 
   In step S 203 , for each output neuron  31  of the output layer  30 , the measured value O kj  of the kernel layer  20  corresponding to the pattern type associated with the output neuron  31  is multiplied by a weight. As a result, a measured value of the output layer  30  corresponding to the pattern type is calculated, i.e., 
               O   k     =       ∑     j   =   1       k   n       ⁢       λ   kj     ⁢     O   kj           ,         
1≦k≦K, where λ kj  is a weight of the measured value of the kernel layer  20  and k n  is the number of the hidden neurons  21  corresponding to the k-th pattern type.
 
   In step S 204 , a maximum measured value O M  of the output layer  30  is found from all measured values O k  (1≦k≦K) of the output layer  30 , i.e., 
             O   M     =       max     1   ≤   k   ≤   K       ⁢     O   k             
and an error relative to a correct measured value O T  of the output layer  30 . Note that training samples are served for training the RBF neural network. Hence, at the time of inputting training samples, a pattern type associated with each training sample is already known to the RBF neural network. As a result, the measured value of the output layer  30  obtained in the corresponding pattern type by the feature vector of the training sample is the correct measured value O T  of the output layer  30 . An error E is obtained by calculating an error function E(x)=1−e (O     T     −O     M     )A(n) , where 0≦A(n+1)≦A(n) and n is an epoch iteration number. A value of the error E is either one of the following cases when A(n) approaches zero:
   E= 1, if  O   T   &lt;O   M ; and E=0, if O T =O M . 
   Moreover, for a certain A(n), (e.g. A(n)=0.5), when O T &lt;O M  and O T ≈O M , e (O     T     −O     M     )/A(n)  will approach one. That is, a value of 1−E approaches one. As a result, a large quantity of learning is obtained with respect to the RBF neural network. On the contrary, when O T &lt;O M  and O T &lt;&lt;O M , e (O     T     −O     M     )/A(n)  will approach zero. That is, a value of 1−E approaches zero. As a result, a small quantity of learning is obtained with respect to the RBF neural network. 
   In step S 205 , the RBF neural network modifies values of parameters C kji  and w kji  of the kernel function and calculates a weight λ kj  of the measured value of the output layer  30  based on the error E. In this embodiment, a generalized delta rule is carried out to generate updated difference of parameter C kji , w kji , and λ kj : 
               ∇     C   kji       =       -     α   ⁡     (   n   )         ⁢       ∂   E       ∂     C   kji             ,     
     ⁢       ∇     λ   kj       =       -     β   ⁡     (   n   )         ⁢       ∂   E       ∂     λ   kj             ,   and                   ∇     w   kji       =       -     γ   ⁡     (   n   )         ⁢       ∂   E       ∂     w   kji             ,         
where α(n), β(n), and γ(n) are three positive and monotonically-decreasing learning rates. Also, a chain rule is carried out to derive
 
                 δ   ⁢           ⁢   E       δ   ⁢           ⁢     C   kji         =         δ   ⁢           ⁢   E       δ   ⁢           ⁢     O   k         ⁢       δ   ⁢           ⁢     O   k         δ   ⁢           ⁢     O   kj         ⁢       δ   ⁢           ⁢     O   kj         δ   ⁢           ⁢     C   kji             ,     where   ⁢           ⁢       δ   ⁢           ⁢   E       δ   ⁢           ⁢     O   k           ,       δ   ⁢           ⁢     O   k         δ   ⁢           ⁢     O   kj         ,     and   ⁢           ⁢       δ   ⁢           ⁢     O   kj         δ   ⁢           ⁢     C   kji                 
are defined as follows:
 
   
     
       
         
           
             
               δ 
               ⁢ 
               
                   
               
               ⁢ 
               E 
             
             
               δ 
               ⁢ 
               
                   
               
               ⁢ 
               
                 O 
                 k 
               
             
           
           = 
           
             { 
             
               
                 
                   
                     
                       
                         
                           
                             
                               1 
                               A 
                             
                             ⁢ 
                             
                               ( 
                               
                                 1 
                                 - 
                                 E 
                               
                               ) 
                             
                           
                           , 
                         
                       
                       
                         
                           
                             
                               when 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               k 
                             
                             = 
                             M 
                           
                           ; 
                         
                       
                     
                     
                       
                         
                           
                             
                               - 
                               
                                 1 
                                 A 
                               
                             
                             ⁢ 
                             
                               ( 
                               
                                 1 
                                 - 
                                 E 
                               
                               ) 
                             
                           
                           , 
                         
                       
                       
                         
                           
                             
                               when 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               k 
                             
                             = 
                             T 
                           
                           ; 
                         
                       
                     
                     
                       
                         
                           0 
                           , 
                         
                       
                       
                         otherwise 
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         O 
                         k 
                       
                     
                     
                       δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         O 
                         kj 
                       
                     
                   
                 
                 = 
                 
                   λ 
                   kj 
                 
               
               , 
               
                 
                   and 
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         O 
                         kj 
                       
                     
                     
                       δ 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         C 
                         kji 
                       
                     
                   
                 
                 = 
                 
                   2 
                   ⁢ 
                   
                     O 
                     kj 
                   
                   ⁢ 
                   
                     
                       
                         w 
                         kji 
                       
                       ( 
                       
                         
                           x 
                           i 
                         
                         - 
                         
                           C 
                           kji 
                         
                       
                       ) 
                     
                     . 
                   
                 
               
             
           
         
       
     
   
   By combining the above equations, it is able to obtain the updated differences of the following parameters: 
             ∇     C   kji       =     {               -     η   ⁡     (     1   -   E     )         ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,             when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                           ⁢         η   ⁡     (     1   -   E     )       ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,               when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                   0   ,           otherwise   ,                   
where η=2α/A, and η is a learning parameter; likewise,
 
             ∇     λ   kj       =     {                   -     θ   ⁡     (     1   -   E     )         ⁢     O   kj       ,             when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                           ⁢         θ   ⁡     (     1   -   E     )       ⁢     O   kj       ,               when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                   0   ,           otherwise   ,           ⁢     
     ⁢   and   ⁢     
     ⁢     ∇     w   kji         =     {                   ⁢         ρ   ⁡     (     1   -   E     )       ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,               when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                       -     ρ   ⁡     (     1   -   E     )         ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,             when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                   0   ,           otherwise   ,                       
where θ=2β/A, ρ=2γ/A, and both θ and ρ are learning parameters. Moreover, η, θ, and ρ are generally monotonically decreasing.
 
   Note that the above updated differences of the parameters are obtained by assuming O T &lt;O M . In other words, the maximum measured value of the output layer  30  calculated by the RBF neural network is not equal to the correct measured value of the output layer  30 . As a result, differences of the parameters will be generated by the RBF neural network. 
   In a practical training process, there is a need to appropriately update values of the parameters even a correct O T =O M  is calculated by the RBF neural network. As such, a further determination is made as to whether there is a measured value O S  of the output layer  30  less than O T  by a small value, i.e., O T &lt;O S +ε:, where ε is a small positive value (e.g. ε=0.02). At this condition, the updated differences of the parameters C kji , w kji , and λ kj  are as follows: 
             ∇     C   kji   m       =     {                   -       η   1     ⁡     (     1   -   E     )         ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,             when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                           ⁢           η   1     ⁡     (     1   -   E     )       ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,               when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                       -       η   2     ⁡     (     1   -     E   a       )         ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,               when   ⁢           ⁢   k     =   S     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                             ⁢           η   2     ⁡     (     1   -     E   a       )       ⁢     λ   kj     ⁢     O   kj     ⁢       w   kji     ⁡     (       x   i     -     C   kji       )         ,                 when   ⁢           ⁢   k     =   T     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                     0   ,           otherwise   ,           ⁢     
     ⁢     ∇     λ   kj   m         =     {                   -       θ   1     ⁡     (     1   -   E     )         ⁢     O   kj       ,             when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                           ⁢           θ   1     ⁡     (     1   -   E     )       ⁢     O   kj       ,               when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                       -       θ   2     ⁡     (     1   -     E   a       )         ⁢     O   kj       ,               when   ⁢           ⁢   k     =   S     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                             ⁢           θ   2     ⁡     (     1   -     E   a       )       ⁢     O   kj       ,                 when   ⁢           ⁢   k     =   T     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                     0   ,           otherwise   ,           ⁢     
     ⁢   and   ⁢     
     ⁢     ∇     w   kji   m         =     {                         ⁢           ρ   1     ⁡     (     1   -   E     )       ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,               when   ⁢           ⁢   k     =       M   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                       -       ρ   1     ⁡     (     1   -   E     )         ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,             when   ⁢           ⁢   k     =       T   ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;     O   M                           ⁢           ρ   2     ⁡     (     1   -     E   a       )       ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,                 when   ⁢           ⁢   k     =   S     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                         -       ρ   2     ⁡     (     1   -     E   a       )         ⁢     λ   kj     ⁢         O   kj     ⁡     (       x   i     -     C   kji       )       2       ,               when   ⁢           ⁢   k     =   T     ,       O   T     =         O   M     ⁢           ⁢   and   ⁢           ⁢     O   T       &lt;       O   S     +   ɛ                     0   ,         otherwise         ⁢     
     ⁢   where   ⁢           ⁢     η   1       ≥     η   2       ,       θ   1     ≥     θ   2       ,       ρ   1     ≥     ρ   2       ,       E   a     =     1   -     ⅇ       (       O   S     -     O   M       )     /   A           ,       and   ⁢           ⁢     O   S       =       max       k   =   1     ,     k   ≠   M       K     ⁢       O   k     ⁢           .                           
As a result, a possibility of generating an obscure result is greatly reduced. Thus, a recognition correctness of the RBF neural network can be increased.
 
   Based on the above updated differences of the parameters, the parameters C kji , w kji , and λ kj  are modified as follows: 
   
     
       
         
           
             
               C 
               kji 
             
             = 
             
               
                 C 
                 kji 
               
               + 
               
                 ∇ 
                 
                   C 
                   kji 
                   m 
                 
               
             
           
           , 
           
             
 
           
           ⁢ 
           
             
               w 
               kji 
             
             = 
             
               
                 w 
                 kji 
               
               + 
               
                 ∇ 
                 
                   w 
                   kji 
                   m 
                 
               
             
           
           , 
           and 
         
       
     
     
       
         
           
             λ 
             kj 
           
           = 
           
             
               λ 
               kj 
             
             + 
             
               
                 ∇ 
                 
                   λ 
                   kj 
                   m 
                 
               
               ⁢ 
               
                   
               
               . 
             
           
         
       
     
   
   In view of the foregoing, the invention generates the updated differences of parameters by recognizing the inputted training samples, and the modified parameters are employed to stepwise adjust a setting of the RBF neural network, resulting in an increase of recognition accuracy. Furthermore, the degree of importance and learning contribution of one training sample can be distinguished from that of the other. As a result, the training mechanism is enhanced, thereby establishing an optimum RBF neural network for recognizing patterns. 
   Although the present invention has been explained in relation to its preferred embodiment, it is to be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the invention as hereinafter claimed.