Abstract:
A system and method for detecting the recognizability of input speech signal is provided. It is designed in the pre-stage of speech recognition or a dialog system. The invention detects the user&#39;s environmental condition and verifies if the input speech signal can be recognized. It mainly comprises an environment parameter generator, a signal recognition verifier, and a strategy response processor. Through the use of the invention in the pre-stage of speech recognition or a dialog system, it can precisely verify the recognizability of the input speech signal and receives the input speech signals of high recognition probability in a noisy environment. This reduces the impact caused by receiving the input speech signals of low recognition probability. This invention thus increases the recognition probability for a recognizer.

Description:
FIELD OF THE INVENTION 
     The present invention generally relates to the field of speech recognition and more specifically to system and method for detecting the recognizability of input speech signals. 
     BACKGROUND OF THE INVENTION 
     The speech recognition system usually encounters various problems caused by the environments, such as background noise and the channel effect, or other factors of the speakers, such as the accent and the speaking rate, so that the input speech is beyond the recognition capability of the system. Prior researches proposed various improvements over the recognition capability, however, with only limited results. 
     U.S. Pat. No. 6,272,460, “Method for Implementing a Speech Verification System for Use in a Noisy Environment”, disclosed a system including a speech verifier in the front stage of the system. As shown in  FIG. 1 , a speech verifier  100  includes a noise suppressor  110 , a pitch detector  120 , and a confidence determiner  130 . The object is to rid of noises and obtain the pitch. The pitch value is translated into a time-variant confidence index for determining whether the input signal at a certain time is a speech. The confidence index is transferred to the recognizer for assisting the recognition. 
     U.S. Pat. No. 6,272,461 emphasized the speech detection and the assistance in speech recognition of all the input signals regardless of whether the input signals are beyond the acceptable range. 
     The current speech recognition or dialog system does not have the capability for sensing the environment of the usage. This implies that the system will blindly try to recognize the speech and generate an output no matter how harsh the usage environment is and no matter how the task is beyond the system capability. As a result, the user may receive an erroneous answer. This not only wastes the system resource, but also leads to potentially severe outcomes. 
     Take the auto-attendant as an example. When the caller uses the extension number inquiry system from a noisy subway station or on the busy street, the environmental noise will affect the signal-to-noise ratio (SNR) so that the SNR is too low and beyond the system capability. The system will perform the speech recognition process and generates a wrong extension number. At the end, the caller will need to request a customer service representative for the assistance. This scenario shows the waste of system resource and the failure of saving the manpower. 
     On the other hand, if the system can determine whether the input signal is within the recognizable range before the system starts the actual recognition process, the recognizable signals can be passed for recognition while the unrecognizable signals can be responded with appropriate actions. In this manner, the possibility of successful speech recognition will increase. 
     SUMMARY OF THE INVENTION 
     The present invention has been made to overcome the above-mentioned drawback of conventional speech recognition systems that have no capability in sensing the usage environment. The primary object of the present invention is to provide a system and a method for detecting the recognizability of the input speech signals. 
     In comparison with the conventional methods, the present invention includes the following characteristics: (a) The present invention emphasizes the front stage of the recognition system. By using a small amount of system resource to detect whether the input signal can be successfully recognized, the efficiency of the system can be improved. (b) The recognizable signals are passed to the recognizer for recognition, and unrecognizable signals are responded with appropriate actions. (c) The unrecognizable signals are not passed to the system for recognition so that the system resource is saved. 
     To achieve the above object, the present invention provides a system with a front stage for detecting the recognizabiliy of input speech signals, comprising an environment parameter generator, a signal recognition verifier, and a strategy response processor. 
     The system operates as follows. First, the environment parameter generator generates a plurality of parameters in accordance with the environment to represent the environment conditions or the input signal quality. Then, the signal recognition verifier, after the initial training, verifies whether the input signal is recognizable in accordance with the environment parameters. When the input signal is verified as recognizable, the input signal is passed to the recognition device for recognition. On the other hand, when the input signal is verified as unrecognizable, the strategy response processor is triggered to propose a strategy to respond to the environment or signal quality of the user in accordance with the environment parameters. 
     In the embodiment of the present invention, the environment parameter generator selects the SNR of the input signal, the probability of input signal being a speech, and confidence index of the system processing input signal as the environment parameters. The strategy response processor proposes different strategies to guide the user to improve. For example, when the SNR is low, the user is advised to raise the voice or move to a quieter environment. Or, when the confidence index is low, the user is advised to speak more clearly. Then, the user is prompted to input the signal again or is transferred to a customer service representative. 
     The foregoing and other objects, features, aspects and advantages of the present invention will become better understood from a careful reading of a detailed description provided herein below with appropriate reference to the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  shows a schematic view of a conventional speech recognition system and method in a noisy environment. 
         FIG. 2  shows a schematic view of a block diagram of the present invention. 
         FIG. 3  shows a schematic view of the block diagram of the environment parameter generator of the present invention. 
         FIG. 4  shows a schematic view of the block diagram of the signal recognition verifier of the present invention. 
         FIG. 5  shows an embodiment of the strategy response processor of the present invention. 
         FIG. 6  shows the experimental results of the recognition rate for a simulated noise environment with six test sets. 
         FIG. 7  shows the output of the error of the failure and the success of recognition for the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     As mentioned earlier, the speech recognition system for detecting the recognizability of the input speech emphasizes the front stage of the recognition or dialog system.  FIG. 2  shows a schematic view of a block diagram of the present invention. As shown in  FIG. 2 , a speech recognition system  200  comprises an environment parameter generator  210 , a signal recognition verifier  220  and a strategy response processor  230 . The functionality of each component and the system operation is described as follows. 
     First, environment parameter generator  210  generates at least an environment parameter for the input signals. The environment parameter represents the environment conditions for the input signal or the input signal quality. Without the loss of generality, the embodiment of the present invention uses the SNR of the input signal, the probability of input signal being a speech, and the confidence index of system processing input signal as the environment parameters. These environment parameters can be generated by using voice activity detection (VAD) and missing feature imputation (MFI) to obtain a clean speech signal, and then a calculation is performed. The calculation of the environment parameters will be described later. 
     Then, signal recognition verifier  220 , after the initial training with the environment parameters in advance, verifies whether the input signal is recognizable in accordance with the environment parameters. When the input signal is verified as recognizable, the input signal is passed to a recognition device  225  for further recognition. When the input signal is verified as unrecognizable, strategy response processor  230  is triggered to respond with a plurality of strategies to increase the possibility of successful recognition. 
       FIG. 3  shows a block diagram of the environment parameter generator of the present invention. The environment parameter generator includes a SNR calculation unit  310   a , a probability calculator  310   b  for calculating the probability of the signal being a speech, and a confidence calculator  310   c  for calculating the confidence index of the system processing the input signal. The calculation of each calculator is described as follows. 
     In the application in an actual environment, the background noise usually directly affects the recognition rate of the system. Therefore, the present invention uses the SNR as the first environment parameter. 
     First, SNR calculator  310   a  uses the VAD method to detect the speech part x and the non-speech part (noise) u n  from the spectrum feature of the input signal y. Then, the MFI method is used to clear the noise from the speech part x to obtain a clean speech signal {circumflex over (x)}. Based on noise u n  and clean signal {circumflex over (x)}, the SNR of the input signal y, named SNR y  is calculated. In general, the higher the SNR is, the higher the probability that a signal can be recognized successfully. The SNR y  can be expressed as the following equation: 
                 SNR   ⁡     (   t   )       =         1   D     ·       ∑     d   =   0       D   -   1       ⁢       x   ^     ⁡     (     t   ,   d     )               1   D     ·       ∑     d   =   0       D   -   1       ⁢       u   n     ⁡     (   d   )               ,     t   =       0   ~   T     -   1                     SNR   y     =     max   ⁡     (     SNR   ⁡     (   t   )       )             
where SNR(t) is the SNR of the signal y at the time t, and T is the total length of the input signal. D is the total number of the frequency bands of the input signal frequency spectrum. {circumflex over (x)}(t,d) is the clean speech spectrum feature parameter calculated by MFI method at time t and band d. u n (d) is the average of the noise spectrum feature parameter calculated by MFI method at band d. SNR y  is the SNR y  value of the input signal y.
 
     In addition to the SNR, the present invention also uses the probability, P y , of the input signal y being a speech as the second environment parameter. The larger the probability P y  is, the easier the input signal can be recognized successfully. 
     First, probability calculator  310   b  uses the MFI method to calculate the probability that the SNR is greater than 0 when the clean signal spectrum parameter x is at time t and band d. 
               P   ⁡     (       SNR   ⁡     (     t   ,   d     )       &gt;   0     )       =       ∫     -   ∞         (     t   ,   d     )     /   2       ⁢       1         2   ⁢   π       ⁢              σ   ^     n     ⁡     (   d   )                ⁢           ⁢     ⅇ     -     (         (     ω   -         μ   ^     n     ⁡     (   d   )         )     2       2   ⁢         σ   ^     n   2     ⁡     (   d   )           )         ⁢     ⅆ   ω               
where {circumflex over (μ)} n (d) and {circumflex over (σ)} n   2 (d) are the average and the variance of noise spectrum distribution calculated by MFI method, respectively. ω is the value of the noise.
 
     Then, the MFI method is used to calculate the probability that the clean signal spectrum is a speech at time t, as follows: 
                 R   ⁡     (   t   )       =       1   D     ·       ∑     d   =   0       D   -   1       ⁢     P   ⁡     (       SNR   ⁡     (     t   ,   d     )       &gt;   0     )             ,     t   =       0   ~   T     -   1             
where D is the number of the bands of the signal spectrum and T is the total length of the input signals.
 
     Finally, the probability of the input signal y being a speech is calculated as follows: 
     
       
         
           
             
               P 
               y 
             
             = 
             
               
                 1 
                 / 
                 T 
               
               ⁢ 
               
                 
                   ∑ 
                   
                     t 
                     = 
                     0 
                   
                   
                     T 
                     - 
                     1 
                   
                 
                 ⁢ 
                 
                   R 
                   ⁡ 
                   
                     ( 
                     t 
                     ) 
                   
                 
               
             
           
         
       
     
     The present invention uses the confidence index of the system processing the signal as the third environment parameter. The larger the confidence index is, the easier the input signal can be recognized successfully. 
     First, confidence calculator  310   c  measures the divergence between the input signal y and the known system model distribution x on the frequency spectrum, as expressed in the following equation: 
               D   ⁡     (     y   ||   x     )       =     ∫       [       p   ⁡     (   y   )       -     p   ⁡     (   x   )         ]     ⁢     log   ⁡     (       p   ⁡     (   y   )         p   ⁡     (   x   )         )       ⁢     ⅆ   x               
where p(y) is the probability distribution of the spectrum parameter of the signal y, and p(x) is the probability distribution of the spectrum parameter of the system model. The larger the divergence D(y∥x) is, the lower the probability that the input signal can be recognized successfully is.
 
     Then, the divergence D(y∥x) is transformed by a Sigmoid function into a confidence index R y  between 0 and 1: 
               R   y     =     1     1   +     exp   ⁡     (     -     α   ⁡     (     D   +   β     )         )                 
where α and β are the fine-tuning parameters for enlargement and shift, respectively.
 
     After the three environment parameters SNR y , P y  and R y  are calculated, signal recognition verifier  220 , after the initial training with the environment parameters in advance, receives and analyzes the three environment parameters SNR y , P y  and R y  to verify whether the input signal is recognizable, as shown in  FIG. 4 . The training rule with the environment parameters can be the multi-layer perception (MLP) method of the pattern classification. 
     As aforementioned, when the input signal is verified as unrecognizable by the signal recognition verifier  220 , the strategy response processor  230  is triggered to respond with strategies. There are a plurality of possible strategies.  FIG. 5  shows a working example of the strategy. In this working example, the user is informed that the input signal can not be recognized and of what the current usage environment condition and the signal quality are, such as step  501 , to guide the user to improve the environment condition and the signal quality. For example, when the SNR is lower than a threshold, the user is advised to raise the voice or move to a quieter location. When the confidence index of the system processing the signal is lower than a threshold, the user is advised to speak more clearly. Then, the user is prompted to re-enter the input signal or transferred to a customer service representative, as step  502 . 
     In an experiment with 936 clean speech utterances of Chinese names, a babble noise of five different SNR between 0 dB and 20 dB is added to simulate the noise environment and generate six sets of tests, 5616 test signals in total. With the noise interference, the recognition rate of the six sets of tests is shown in  FIG. 6 . In a noise-free environment, the recognition rate is 94.2%. When the babble noise is added, the average recognition rate for the six sets of tests is reduced to 64.8%. 
     It is obvious that the system recognition rate decreases rapidly as the SNR decreases. With the present invention adding the aforementioned detection method in the front stage of the recognition system, the environment parameters are generated for every unrecognizable and recognizable signal.  FIG. 7  shows the output of the error rate for the recognizable and unrecognizable signals, respectively. 
     In  FIG. 7 , A represents the number of the utterances that the recognition device cannot recognize successfully, and B represents the number of the utterances that the present invention mistakenly verifies as recognizable. Similarly, C represents the number of the utterances that the recognition device can recognize successfully, and D represents the number of the utterances that the present invention mistakenly verifies as unrecognizable. The average recognition rate of the recognition device is calculated as the ratio between the number of the correctly recognized utterances and the number of the utterances entering the recognition device, that is, (C−D)/(C−D+B)=(3640−807)/(3640−807+453)=86.2%. 
     As seen in the above results, after the detection method of the present invention is added to the front stage of the recognition system, the recognition rate is improved from 64.8% to 86.2%, and the unrecognizable signals are rejected to prevent further effect of the erroneous recognition. 
     In summary, the present invention provides a system and a method for detecting the recognizability of the input signal. The present invention is to detect the usage environment conditions and the signal quality in the front stage of the recognition system to verify whether the signal can be recognized successfully. In the present invention, three environment parameters, including SNR, the probability of the signal being a speech, and the confidence index of the system processing the signal, are used to represent the environment conditions and the signal quality. The environment parameters are used to train the signal recognition verifier to verify whether the signal can be recognized successfully. When the signal is verified as recognizable, the signal is passed to the recognition device for recognition. When the signal is verified as unrecognizable, a strategy response processor is triggered to inform the user of the environment conditions and prompt the user for inputting better quality signals. 
     Although the present invention has been described with reference to the preferred embodiments, it will be understood that the invention is not limited to the details described thereof. Various substitutions and modifications have been suggested in the foregoing description, and others will occur to those of ordinary skill in the art. Therefore, all such substitutions and modifications are intended to be embraced within the scope of the invention as defined in the appended claims.