Abstract:
Model compression is combined with model compensation. Model compression is needed in embedded ASR to reduce the size and the computational complexity of compressed models. Model-compensation is used to adapt in real-time to changing noise environments. The present invention allows for the design of smaller ASR engines (memory consumption reduced to up to one-sixth) with reduced impact on recognition accuracy and/or robustness to noises.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This application claims the benefit of U.S. Provisional Application No. 60/659,054, filed on Mar. 4, 2005. The disclosure of the above application is incorporated herein by reference n its entirety for any purpose. 
     
    
     FIELD OF THE INVENTION  
       [0002]     The present invention generally relates to automatic speech recognition, and relates in particular to noise robust automatic speech recognition.  
       BACKGROUND OF THE INVENTION  
       [0003]     Embedded noise robust automatic speech recognition (ASR) systems need to conserve memory due to the small size and limited resources of devices such as cell phones, car navigation, digital TVs, and home appliances. However, ASR systems are notorious for consuming large amounts of computational resources, including Random Access Memory (RAM). This tendency of ASR systems can be especially problematic in embedded devices that also need to allocate such resources for other functions that often need to run concurrently with ASR functions. Yet, reducing the amount of memory consumed by a noise robust ASR heavily impacts recognition accuracy and/or robustness to noise.  
         [0004]     Referring to  FIG. 1 , model domain methods try to improve the performance of pattern matching by modifying the acoustic models so that they are adapted to the current noise level, while leaving the input signal  100  unchanged. In particular, a noise estimation module  104  estimates noise in the input signal  100 , and model compensation module  106  adjusts the acoustic models  108  based on these noise estimates. Then, extracted features obtained from the unmodified input signal  100  by feature extraction module  102  are pattern matched to the adjusted acoustic models  108  by pattern matching module  110  to achieve recognition  112 .  
         [0005]     What is needed is a way to reduce the memory requirements of embedded noise robust ASR systems with reduced impact on recognition accuracy and/or robustness to noise. The present invention fulfills this need by making several changes to a noise robustness system employing a model domain method.  
       SUMMARY OF THE INVENTION  
       [0006]     In accordance with the present invention, model compression is combined with model compensation. Model compression is needed in embedded ASR to reduce the size and the computational complexity of compressed models. Model-compensation is used to adapt in real-time to changing noise environments. The present invention allows for the design of smaller ASR engines (memory consumption reduced to up to one-sixth) with reduced impact on recognition accuracy and/or robustness to noises.  
         [0007]     Further areas of applicability of the present invention will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0008]     The present invention will become more fully understood from the detailed description and the accompanying drawings, wherein:  
         [0009]      FIG. 1  is a block diagram illustrating a noise robust ASR system according to the prior art;  
         [0010]      FIG. 2  is a block diagram illustrating real-time implementation of a model compensation distortion function in accordance with the present invention; and  
         [0011]      FIG. 3  is a set of graphs illustrating word error rate percentage in  FIG. 3A , memory consumption in kilobytes in  FIG. 3B , and real time factor in seconds in  FIG. 3C . 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0012]     The following description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.  
         [0013]     In some embodiments, the present invention combines sub-space tying for model compression with alpha-Jacobian model-compensation for noise robustness to achieve a compact noise robust speech recognition system. Unfortunately this combination cannot be accomplished directly as the subspace tying structure does not allow for model-compensation. This difficulty arises because the distortion function used in model compensation requires a full space transformation (full dimensionality) of the acoustic models that invalidates the tying structure.  
         [0014]     One area of interest in the present invention is the present solution to this issue. Specifically, a model compensation distortion function is designed that does not invalidate the tying structure, thus allowing for the coexistence of subspace tying and model-compensation. The design of the model compensation distortion function can be accomplished by making several changes in a noise robust ASR system to the following modules: (a) front-end analysis: the front-end whitening matrix can be block-diagonal to isolate a set independent subspaces (block-diagonal covariance matrix); (b) model-compensation: the model-compensation distortion function can be operating independently on the same subspaces identified by the front-end analysis (and cannot be a full-space transformation); and (c) subspace model compression: the subspaces used for the tying can be aligned with the independent subspaces defined in the front-end.  
         [0015]     One ingredient of this method can be in the definition of the subspaces corresponding to the block-diagonal whitening matrix in the front-end. These subspaces need to be large enough to allow a good coverage of the speech signal correlation structure in the front-end and in the model-compensation step, but small enough to allow a low distortion error from the subspace tying step. In general, the subspace definition is an NP-Complete problem for which there is no computable exact solution, but for which an interactive converging algorithm can be provided.  
         [0016]     The whitening matrix or matrices can take various forms depending on the characteristics of the independent subspaces. For example, in some embodiments, the independent subspaces can span over different time frames, and the whitening matrices include decorrelation across a 2-dimensional time-frequency axis. Also, in additional or alternative embodiments, such 2-D decorrelation matrices are decomposable as discrete cosine transform in the frequency domain and time derivative in the time domain.  
         [0017]     Turning to  FIG. 2 , real-time implementation of a model compensation distortion function in accordance with the present invention includes a number of different components. Such components can include speech input  250 , noise estimation  252 , power spectral energy estimation by Short Time Fourier Transform (STFT) or Wavelets decomposition  254 , band  256 , log compression or power-law compression function  258 , and tying topology (subspace definition structure)  260 . All components in the three main blocks of front-end analysis  200 , model-compensation  202 , and subspace Gaussian distribution computation  204  are split and aligned to follow the subspace definition structure. The decorrelation matrices  206  operate independently on blocks  208  of log filter-bank energies. This architecture allows for the model-compensation to work effectively on each subspace without affecting the subspace tying structure. This capability allows for efficient model-compensation of subspace compressed acoustic models, which in turn allows a considerable reduction in system size and a considerable improvement in speed.  
         [0018]     The model compensation distortion technique of the present invention allows reduction of the acoustic models size by up to ⅙ th  of the initial size, and reduces the computational load to up to ⅓ rd  of the initial while allowing great robustness to noise thanks to the usage of model-compensation. The complexity of the model compensation is also reduced because of the smaller set of distributions to compensate.  
         [0019]     Turning now to  FIG. 3 , results illustrate performance of the noise robust ASR system according to the present invention for a Car-Navigation task in noisy conditions. Jacobian model compensation is applied in all three cases. The “Sub-space” case  300  shows the performance of the proposed invention. The previous embedded model compression method allowing for model compensation, the “Full-space” case  302 , does not provide a good recognition rate, mainly because full-space compression introduces too much distortion in the acoustic models. The proposed method provides better performance, very close to uncompressed models, the “untied” case  304 , but with smaller size and a faster real-time factor.  
         [0020]     It is envisioned that a similar approach can be employed for speaker adaptation, with subspace transformations (such as MLLR constrained to subspaces). For example, subspace tied acoustic model whitening can be employed with model compensation and an additional subspace tying regarding compensated acoustic models for update purposes (store to RAM or flash ROM, etc.).  
         [0021]     The description of the invention is merely exemplary in nature and, thus, variations that do not depart from the gist of the invention are intended to be within the scope of the invention. Such variations are not to be regarded as a departure from the spirit and scope of the invention.