Abstract:
An utterance verification method for an isolated word N-best speech recognition result includes: calculating log likelihoods of a context-dependent phoneme and an anti-phoneme model based on an N-best speech recognition result for an input utterance; measuring a confidence score of an N-best speech-recognized word using the log likelihoods; calculating distance between phonemes for the N-best speech-recognized word; comparing the confidence score with a threshold and the distance with a predetermined mean of distances; and accepting the N-best speech-recognized word when the compared results for the confidence score and the distance correspond to acceptance.

Description:
CROSS-REFERENCE(S) TO RELATED APPLICATION(S) 
       [0001]    The present invention claims priority of Korean Patent Application No. 10-2008-0131755, filed on Dec. 22, 2008, which is incorporated herein by reference. 
       FIELD OF THE INVENTION  
       [0002]    The present invention relates to a speech recognition system; and, more particularly, to an utterance verification method and apparatus for providing isolated word N-best recognition results of high reliability in an isolated word speech recognition system. 
       BACKGROUND OF THE INVENTION  
       [0003]    A speech recognition system outputs one or several results of speech recognition based on applications and recognition performance thereof. In general, a speech recognition system having excellent recognition performance uses a method of outputting single data having a likelihood with the highest distance with an input utterance. On the contrary, a speech recognition system having a poor recognition performance provides several output lists so that a user may select a correct answer. 
         [0004]    As such, that a few output lists provided from several outputable lists are referred as N-best and the number of output lists is determined by specification and application of the system. 
         [0005]    In an existing speech recognition system for providing N-best output lists, a user sees output results and determines whether the output result is a correct answer. That is, the existing system does not provide an utterance verification technique but entrusts a user to verify an utterance. 
         [0006]    The N-best output lists are not provided as a vocabulary set similar to utterance data of the user and interaction relation between N-best output lists having no relations. On the other hand, the N-best output lists are provided as a vocabulary set similar to utterance data of the user and interaction relation between N-best output lists having relations. 
         [0007]    The following Example 1 shows 10-best recognition results for a user&#39;s utterance “poongmin mok-yok-tang”. 
       EXAMPLE 1  
       [0008]    Answer: Poongmin mok-yok-tang 
         [0009]    Recognition results: poongmin mokyoktang, munhyen mokyoktang, dongmoon mokyoktang, kwanmin mokyoktang, kwanmoon mokyoktang, bookmoon mokyoktang, tanhyen mokyoktang, yangmoon mokyoktang, younghyen mokyoktang, youngmoon mokyoktang. As shown in the above example, since the recognition results are very similar to the user&#39;s utterance, confusion in recognition of the speech recognition system is acceptable to the user. 
         [0010]    The following Example 2 shows 10-best recognition results for a user&#39;s utterance “chungdahm villa”. 
       EXAMPLE 2  
       [0011]    Answer: chungdahm villa 
         [0012]    Recognition results: hannahm club, chungdahm villa, yongdahm club, hwanggeum bra, handahm minbahk, gongahn joobang, dongahm drum, dongahempura, hannahm color, bohn computer 
         [0013]    However, as shown in the example 2, the recognition results have little relations there between as well as with the user utterance. Thus, the user may doubt the reliability of the speech recognition system. 
       SUMMARY OF THE INVENTION  
       [0014]    In view of the above, the present invention provides an utterance verification method and apparatus for providing isolated word N-best recognition results of high reliability in an isolated word speech recognition system. 
         [0015]    In accordance with one aspect of the present invention, there is provided an utterance verification method for an isolated word N-best speech recognition result, including: 
         [0016]    calculating log likelihood ratio of a context-dependent phoneme and an anti-phoneme model based on an N-best speech recognition result for an input utterance; 
         [0017]    measuring a confidence score of an N-best speech-recognized word using the log likelihoods ratio; 
         [0018]    calculating a distance between phonemes for the N-best speech-recognized word; 
         [0019]    comparing the confidence score with a threshold and the distance with a mean of distances; and 
         [0020]    accepting the N-best speech-recognized word when the compared results for the confidence score and the distance correspond to acceptance. 
         [0021]    In accordance with one aspect of the present invention, there is provided an utterance verification apparatus for an isolated word N-best speech recognition result including: 
         [0022]    a pre-processor for extracting a feature vector of an input utterance and performing endpoint detection; 
         [0023]    an N-best speech recognizer for performing N-best speech recognition through Viterbi search by referring to the context-dependent phoneme model extracted from the feature vector; and 
         [0024]    an N-best utterance verification unit for calculating log likelihood ratio of a context-dependent phoneme and an anti-phoneme model for the N-best speech-recognized word, comparing a confidence score measured for the N-best speech-recognized word with a threshold, comparing a distance measured for the N-best speech-recognized-word with a mean of distances, and accepting the N-best speech-recognized word when the compared results for the confidence score and the distance correspond to acceptances. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS  
         [0025]    The above features of the present invention will become apparent from the following description of embodiments, given in conjunction with the accompanying drawings, in which: 
           [0026]      FIG. 1  shows configuration of a speech recognition system to which an utterance verification apparatus is applied in accordance with an embodiment of the present invention; 
           [0027]      FIG. 2  illustrates a procedure of creating a model which is used to measure a confidence score of utterance verification in accordance with the present invention; and 
           [0028]      FIGS. 3A and 3B  show a flowchart illustrating detailed process of utterance verification in accordance with the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS  
       [0029]    Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. 
         [0030]      FIG. 1  shows configuration of a speech recognition system to which an utterance verification apparatus is applied in accordance with an embodiment of the present invention. The speech recognition system includes a system controller  102 , a pre-processor  104 , an N-best speech recognizer  106 , and an N-best utterance verifier  108 . 
         [0031]    When a user  100  inputs his/her utterance using the speech recognition system having a speech recognition function, the system controller  102  receives the input utterance and delivers the same to the pre-processor  104 . The pre-processor  104  performs an endpoint detection and feature extraction for processing noise and detecting speech interval to deliver the detected endpoint and the extracted features to the N-best speech recognizer  106 . The N-best speech recognizer  106  performs N-best speech recognition through Viterbi search by referring to a context-dependent phoneme/acoustic model  26 , and delivers the speech recognition results to the N-best utterance verifier  108 . 
         [0032]    Then, the N-best utterance verifier  108  measures confidence score for the speech recognition results, using an anti-phoneme model  24  and a phonetic confusion probability  28  obtained from trained phone data. That is, the N-best utterance verifier  108  measures the confidence score based on a log likelihood ratio (LLR) and uses a support vector machine (SVM) as a classifier. In order to measure a distance for the speech recognition results, the N-best utterance verifier  108  measures the distance based on dynamic time warping (DTW) and compares the measured distance with results from the classifier. Then, whether to accept  110  or reject  112  the speech recognition results is determined based on the comparison. If whether to accept or reject is unjudicable, indetermination  114  message is displayed to allow a user to determinate the acceptance  110  or the rejection  112 . 
         [0033]      FIG. 2  illustrates a procedure of creating a model which is used to measure a confidence score of utterance verification in accordance with the present invention. A context-independent phoneme/acoustic model  22  is created using a feature vector  20  as an input and a context-dependent phoneme/acoustic model  26  is created. The context-independent phoneme/acoustic model  22  created during training is classified to create an anti-phoneme model  24 , wherein the anti-phoneme model  24  is used for measurement of the confidence score. The context-dependent phoneme/acoustic model  26  is used by the N-best speech recognizer  106  shown in  FIG. 1 . The anti-phoneme model  24  is used by the N-best utterance verifier  108  shown in FIG.  1 . 
         [0034]      FIGS. 3A and 3B  show a flowchart illustrating detailed process of utterance verification in accordance with the present invention. 
         [0035]    An N-best word and phoneme recognition results from the N-best speech recognizer  106  of  FIG. 1  are provided to the N-best utterance verifier  108  in step S 300 . 
         [0036]    Next, the N-best utterance verifier  108  applies section information of the N-best word and the context-dependent phoneme/acoustic model  26 , obtained from the model creating procedure shown in  FIG. 2 , to the N-best word, to calculate phoneme log likelihood composing the N-best word in step S 302 . 
         [0037]    In this case, the log likelihood of the context-dependent phoneme is calculated from the following Equation 1. 
         [0000]      log likelihood of context−dependent phoneme={(log likelihood of current phoneme)−(mean of base phonemes of current phoneme)}/(standard deviation of base phonemes of current phoneme)   [Equation 1] 
         [0038]    That is, as seen in Equation 1, in order to calculate the log likelihood of the context-dependent phoneme, a mean value of base phonemes of current phoneme is subtracted from the log likelihood of a current phoneme and then is divided by a standard deviation of base phonemes of current phoneme. 
         [0039]    Next, the N-best utterance verifier  108  calculates a log likelihood for the anti-phoneme model  24  using the anti-phoneme model  24  obtained from the procedure shown in  FIG. 2  in step S 304 . In this case, the log likelihood for the anti-phoneme model  24  is calculated from the following Equation 2. 
         [0000]      log likelihood of anti-phoneme model={(log likelihood of anti-phoneme model of current phoneme)−(mean of anti-phoneme model of current phoneme)}/(standard deviation of anti-phoneme model of current phoneme)   [Equation 2] 
         [0040]    That is, as seen in Equation 2, in order to calculate the log likelihood of the anti-phoneme model of the current phoneme, a mean value of the anti-phoneme model of the current phoneme is subtracted from the log likelihood of the anti-phoneme model of the current phoneme and then is divided by a standard deviation of the anti-phoneme model of the current phoneme. 
         [0041]    As such, when the calculation of the log likelihood of the context-dependent phoneme and the log likelihood of the anti-phoneme model is completed, the N-best utterance verifier  108  measures a phoneme-level confidence score in step S 306 . 
         [0042]    Measuring the confidence score normalized by a frame length with respect to a phoneme i is expressed by Equation 3 and a sigmoid function of Equation 4 is used to restrict a range of the phoneme dynamic probability. 
         [0000]        CM   i ={(log likelihood of context-dependent phoneme)−(log likelihood of anti-phoneme model)}/ T   i    [Equation 3] 
         [0000]    
       
         
           
             
               
                 
                   
                     CM 
                     
                       i 
                        
                       
                         - 
                       
                        
                       Sigmoid 
                     
                   
                   = 
                   
                     1 
                     
                       1 
                       + 
                       
                         exp 
                          
                         
                           ( 
                           
                             
                               - 
                               α 
                             
                              
                             
                                 
                             
                              
                             
                               CM 
                               i 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     4 
                   
                   ] 
                 
               
             
           
         
       
     
         [0043]    That is, as known from the above Equations 3 and 4, the measurement of the phoneme-level confidence score is performed by subtracting the likelihood of the anti-phoneme model from the likelihood of the context-dependent phoneme, dividing it by the number of frames of a corresponding phoneme, and by taking a Sigmoid function thereof. 
         [0044]    As such, when the phoneme-level confidence score is obtained, the N-best utterance verifier  108  measures a word-level confidence score in step S 308 . 
         [0045]    In this case, the word-level confidence score is calculated by the following Equation 5. 
         [0000]      word-level confidence score={(log likelihood of context-dependent word)−(log likelihood of anti-phoneme model word)}/(the number of phonemes in word)  [Equation 5] 
         [0046]    As shown in Equation 5, the word level confidence score is calculated by subtracting the log likelihood of the anti-phoneme model word from the log likelihood of the context-dependent word and dividing the calculated value by the number of phonemes in the word. 
         [0047]    The word-level confidence score is expressed by the following Equation 6. 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           CM 
                           am 
                         
                          
                         
                           ( 
                           w 
                           ) 
                         
                       
                       = 
                       
                         
                           1 
                           N 
                         
                         [ 
                         
                           
                             ∑ 
                             
                               t 
                               = 
                               1 
                             
                             N 
                           
                            
                           
                             CM 
                             
                               i 
                                
                               
                                 - 
                               
                                
                               sigmoid 
                             
                           
                         
                         ] 
                       
                     
                     ] 
                   
                   , 
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     6 
                   
                   ] 
                 
               
             
           
         
       
     
         [0000]    where N is the number of phonemes composing a word. 
         [0048]    When the measurement of the word-level confidence score is completed, the N-best utterance verifier  108  sets a threshold by using a classifier or a specific confidence score, and compares the obtained confidence score with the threshold in step S 310 . 
         [0049]    When the word-level confidence score is greater than the threshold, a true value is stored as an acceptance of a corresponding word in step S 312 . However, when the word level confidence score is less than the threshold, a false value is stored as a rejection in step S 314 . In this case, confidence score measuring parameters used as an input of the classifier are listed in the following table 1. 
         [0000]    
       
         
               
               
               
             
           
               
                   
                 TABLE 1 
               
               
                   
                   
               
               
                   
                 Confidence score measuring 
                   
               
               
                   
                 parameters 
                 Description 
               
               
                   
                   
               
             
             
               
                   
                 NBestWordLLR 
                 Sum of log likelihoods of 
               
               
                   
                   
                 N − 1 number of remaining 
               
               
                   
                   
                 words is subtracted from 
               
               
                   
                   
                 log likelihood of current 
               
               
                   
                   
                 Nth word 
               
               
                   
                 MulpyAllMixtureAntiAndNbestLLR 
                 Multiplying LLR of anti- 
               
               
                   
                   
                 model by NbestWordLLR 
               
               
                   
                 WordGaussianDistProb 
                 converting into Gaussian 
               
               
                   
                   
                 distribution using mean and 
               
               
                   
                   
                 standard deviation of 
               
               
                   
                   
                 phoneme duration 
               
               
                   
                   
                 probability 
               
               
                   
                 WordLikelihood 
                 Word-level log likelihood 
               
               
                   
                 AllMixtureMeanOfAntiwordLLR 
                 Measuring word-level 
               
               
                   
                   
                 confidence score by 
               
               
                   
                   
                 arithmetic mean without 
               
               
                   
                   
                 taking sigmoid function 
               
               
                   
                   
                 when obtaining phoneme 
               
               
                   
                   
                 level LLR 
               
               
                   
                 AllMixtureArithmeticAntiwordLLR 
                 Measuring word-level 
               
               
                   
                   
                 confidence score by 
               
               
                   
                   
                 arithmetic mean 
               
               
                   
                 AllMixtureHarmonicAntiwordLLR 
                 Measuring word-level 
               
               
                   
                   
                 confidence score by 
               
               
                   
                   
                 harmonic mean 
               
               
                   
                 AllMixtureGeometricAntiwordLLR 
                 Measuring word-level 
               
               
                   
                   
                 confidence score by 
               
               
                   
                   
                 geometric mean 
               
               
                   
                   
               
             
          
         
       
     
         [0050]    That is, SVM training is performed using the confidence score measuring parameters in table 1, and the confidence score is compared with the threshold using the trained model. 
         [0051]    Next, aside from storing a value determined by the classifier, the N-best utterance verifier  108  calculates distances between phonemes for the recognition results in step S 316  and calculates mean of the distances as in the following Equation 6 in step S 318 . 
         [0000]    
       
         
           
             
               
                 
                   
                     mean 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     distances 
                   
                   = 
                   
                     
                       sum 
                        
                       
                           
                       
                        
                       of 
                        
                       
                           
                       
                        
                       distances 
                        
                       
                           
                       
                        
                       of 
                        
                       
                           
                       
                        
                       N 
                        
                       
                         - 
                       
                        
                       best 
                        
                       
                           
                       
                        
                       recognition 
                        
                       
                           
                       
                        
                       results 
                     
                     
                       the 
                        
                       
                           
                       
                        
                       number 
                        
                       
                           
                       
                        
                       of 
                        
                       
                           
                       
                        
                       N 
                        
                       
                         - 
                       
                        
                       best 
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     7 
                   
                   ] 
                 
               
             
           
         
       
     
         [0052]    In order to calculate mean of distances by Equation 7, the phoneme confusion probability  28  is used as a cost to obtain distances of respective N-best recognition results, sum of the distances is obtained, and the sum of distances of the N-best recognition results is divided by the number of N-best. 
         [0053]    Thereafter, the utterance verifier  108  compares a distance of a corresponding word with mean of N-best word distances in step S 320 . The N-best utterance verifier  108  stores acceptance information when the distance of the corresponding word is less than the mean distance in step S 322 , and stores rejection information when a distance of the corresponding word is greater than the mean distance in step S 324 . 
         [0054]    Then, the N-best utterance verifier  108  combines the value determined by the classifier with a value determined by the distances for the comparison in step S 326 . 
         [0055]    When the confidence score is accepted and the distance is also accepted, the corresponding speech-recognized word is accepted in step S 328 . When the confidence score is rejected and the distance is also rejected, the corresponding speech-recognized word is rejected in step S 330 . Alternately, when the confidence score is rejected and the distance is accepted or the confidence score is accepted and the distance is rejected, the corresponding speech-recognized word is regarded as indetermination in step S 332  and the N-best utterance verifier  108  waits for a user&#39;s action in step S 334   
         [0056]    As described above, in the utterance verification method and apparatus for isolated word N-best speech recognition results, the confidence scores are measured through the N-best utterance verification for the N-best word-level speech recognition results, and distances between phonemes are measured based on dynamic time warping. Thereafter, acceptance/rejection or indetermination of speech recognition are displayed based on the confidence scores and the distances, thereby enabling speech recognition with high reliability. 
         [0057]    While the invention has been shown and described with respect to the embodiments, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims.