Method and system for recognition of speech in a noisy environment

Abstract: A noise robust speech recognition is provided. In a noise robust speech recognition system (400) a noise suppressor (410) generates a noise-reduced signal spectrum from a noisy input signal. A confidence vector generator (480) generates a confidence vector which modifies the metric used for distance computation in feature space. A feature vector generator (420) generates a feature vector from the noise-reduced signal spectrum. The feature vector is then used in a feature comparator (430) to determine a codebook vector relating to the feature vector by using the modified metric. In a decision unit (440) a recognition result is determined by selecting a best matching word model relating to a sequence of the determined codebook vectors. In another embodiment, a deviation vector generator (570) generates a deviation vector by averaging deviations between feature vectors and said determined codebook vectors. The updated deviation vector is used by a feature vector generator (520) to generate the next feature vectors.

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to speech recognition in a noisy environment, and more particulary to a robust speech recognition method and system. 2. Description of the related Art In recent years the technologies of automatic speech recognition (ASR) have made remarkable progress due to increasing performance of computer and in particular digital signal processing systems and mass memory as well as the development of advanced algorithms. Most ASR systems are designed for office applications running on a PC in a quiet environment. For "real world" applications those systems are often insufficient, since in a typical embedded control application there is not enough computational power and memory available. Furthermore, such applications suffer from noise and acoustic echoes which are big problems in embedded ASR. Many different methods are known which try to cope with the problem of noise and disturbances in speech signals. Noise reduction methods based on filters are known which reduce additive noise from a noisy speech signal. Other systems face the problem of e.g. acoustic echoes in the noisy speech signal. In EP1091349 a noise reduction system is described, which efficiently removes disturbing noise from a speech signal by using a neural network to define a filter function. However, with decreasing signal-to-noise ratio it also causes degradations of the speech signal, which are not known to the ASR system and thus may lead to decreasing recognition results. Kim et al.: Real-Time Noise Suppression Based on Diffusive Gain Factors, ICSP 2001, pp. 703-708, Taejon, Korea, August 2001 have shown that such a noise reduction system used as a pre-processing device in an ASR system can improve the recognition rate in case of noisy input, e.g. inside a running motor vehicle. In DE 19521258 it is suggested to pick up disturbing sounds like noise and acoustic echoes with a separate microphone. The disturbances are than subtracted in the course of feature extraction. This is a costly solution since an additional acoustic path is needed in the system. Moreover, in many situations this method does not apply since it is impossible to get a signal with noise only, i.e. free of the wanted speech signal. In such a case, when the noise cannot be removed from the signal, an alternative idea is to include disturbed speech into the database of the speech engine. In WO 96/27185 a method is described in which noise is artificially added to the clean speech utterances usually used for database generation. Thus it is not needed to record naturally disturbed data. However, it is nearly impossible to foresee all kinds of possible disturbances. Moreover, still a large database is needed to store the plurality of modified utterances. Other approaches work under noisy conditions although the reference data are recorded under noise-free conditions. In J. Ming and F. J. Smith: A probabilistic union model for sub-band based robust speech recognition, ICASSP 2000, pp. 1787-1790, Istanbul, Turkey, June 2000, a statistical model for combining sub-band observations for robust speech recognition is described, in which several Hidden Markov models (HMMs) working on sub-bands of the speech spectrum are used. In this case, noise is expected to disturb speech only in distinct sub-bands, thus HMMs working on low-noise sub-bands are still able to yield good recognition results. However, for each sub-band a separate HMM has to be used which must be stored in a database resulting in high performance and memory requirements for such an ASR. In Barker et al.: Robust ASR Based on Clean Speech Models: An Evaluation of Missing Data Techniques for Connected Digit Recognition in Noise, Eurospeech 2001, pp. 213-216 Denmark, September 2001 another method is described, which uses masking techniques disregarding spectral or temporal parts of the speech signal: masks are calculated with the help of autocorrelation matrices in order to judge spectral or temporal regions of the speech signal as reliable or unreliable. However, such classification techniques are often unreliable and require high computational power and memory. EP-A-1 154 405 discloses a speech recognition method in environments with fluctuating noise levels. When a noise transition has been detected, the parametrization phase includes a step of matching the parameters to the new noise model. Examples of state of the art noise recognition systems are now described in connection to Figs. 11 to 13. Several function blocks of Fig. 11 to 13 may also be used in connection with a noise robust speech recognition system according to the present invention. Figure 11 shows a block diagram of a prior art voice recognition system with noise suppressor 11 that receives samples x(t) of a noisy speech signal from a sampling unit 10 at time t. The noise suppressor outputs at time T noise-reduced speech signal A(f,T) in the spectral domain, that is processed by a feature generator 12 computing from A(f,T) a feature vector U(T) by means of a transformation g k . In a subsequent feature comparator 13 feature vector U(T) is compared with a set of codebook vectors V being the internal data representation of the system. At least one codebook vector V(T) has least Euclidian distance to the feature vector U(T) and is communicated to a decision unit 14 that compares a V(T) sequence with word models 15 representing the words that can be recognized by the system. A discriminator 16 judges the results of said comparison and yields a recognition result or a rejection, i.e. outputs information that the input speech signal x(t) was identified or not identified with a word out of the set of words known by the system. These words may be stored as word models in a storage unit or the like. According to Figure 12, the conventional noise suppressor 11 containing a Fourier transform unit 21 that transforms at time T a frame of n samples of the input signal x(t) into a spectral signal X(f,T). In a filter-function calculator 22 a filter function F(f,T) is generated by means of X(f,T). F(f,T) is applied to the noisy speech signal in a filter unit 23 that computes A(f,T)=X(f,T)F(f,T) and supplies a noise reduced speech spectrum A(f,T) being output of the noise suppressor 11. The computation of a filter function F(f,T) from X(f,T) is shown with reference to Figure 13. A state-of-the-art filter function calculator 22 comprises a minima detect layer 31 which averages X(f,T) in T over m frames and detects minima M(f,T) for each frequency f of the averaged signal within a T-interval of the length of I frames. A reaction layer 32 computes from M(f,T) and X(f,T) a relative spectrum R(f,T) by means of a reaction function r() with argument S(T-1), taking into account a control parameter K controlling the amount of noise suppression. In detail, R(f,T) may be computed by the formula: R ( f,T ) = 1- M ( f , T ) r ( S ( T - 1)) K X ( f,T ) A diffusion layer 33 computes the filter function F(f,T) by means of spectral diffusion with a diffusion constant D using the formula: F ( f,T ) = R ( f,T ) + D ∂ 2 R ( f,T ) ∂ f 2 An integration layer 34 computes from F(f,t) a spectral integral S(T) which is fed back to the reaction layer 32. The spectral integral may be calculated by the formula: S = ∫ F ( f,T ) df Based on the foregoing, embodiments of the present invention provide an improved noise robust speech recognition method and apparatus. SUMMARY OF THE INVENTION The present invention as claimed overcomes the foregoing and other problems with a robust speech recognition in which information generated in noise suppression, disturbance reduction or speech recognition operations are used to improve the recognition result. Methods, apparatuses, systems and articles of manufacture consistent with the present invention use information like noise affection of the speech signal and/or deviations between noise reduced signal spectrum and values of a codebook as parameters in the recognition operation. This improves the recognition result by more relying on components in the speech spectrum which are less affected by noise and/or by averaging deviations between a feature vector generated from the noise reduced signal spectrum and codebook vectors caused by noise, other unforeseen effects or a modified frequency characteristic in the signal path, e.g. by using a different microphone. In accordance with methods consistent with the present invention, a robust speech recognition method is provided in which a noise-reduced signal spectrum and a confidence vector are generated from a noisy speech input signal, a feature vector is generated from the noise-reduced signal spectrum, a codebook vector relating to said feature vector is determined using the confidence vector, and a recognition result is generated by determining a best matching word model relating to a sequence of codebook vectors. The confidence vector may be used to select a codebook vector which is the best matching codebook vector relating to the generated feature vector in a way that components of said feature vector that are more affected by noise are weighted weaker in the distance calculation than those less affected. In accordance with methods consistent with the present invention, a robust automatic speech recognition method is provided in which a noise-reduced signal spectrum is generated from a noisy speech input signal, a feature vector is generated from the noise-reduced signal spectrum, a codebook vector is determined which relates to the generated feature vector, and in a further procedure a deviation vector is generated from the deviation between the feature vector and the codebook vector, and then a modified feature vector is generated by using the deviation vector. Since the deviation vector changes slowly, the generation of the deviation vector may be carried out in not every single recognition cycle but rather in every third, fifth or tenth cycle to e.g. save processing capacity. The methods according to the present invention described above are linked by a common concept that specific features which are influenced by noise in the noisy speech signal and/or other disturbances and/or changed transmission characteristics of devices in the signal path are calculated as vectors defining the signal and/or the system. In the method according to the present invention, these vectors may then be used to improve the determination of a best matching codebook vector. The noise consistency of the speech recognition method and system can be improved or, in other words, the speech to noise ratio can be decreased by still getting usable recognition results. The calculated vectors may represent the degree and the manner of which the speech signal is affected by noise or disturbances in the signal path. With respect to different transmission characteristics of certain input devices, e.g. of a microphone or a A/D converter, the method and system according to the present invention adjusts itself by taking such vectors into account in the recognition process. Highly reliable speech recognition results are obtained with a method or system according to the present invention in which both the confidence vector and the deviation vector are used to improve the quality of the feature vector and the matching result by determining the codebook vector. The deviation vector is calculated for the compensation of disturbances and, specifically of unforeseen effects in the feature vector generation process. By calculating the deviation vector, deviations between the feature vector and the codebook vector caused by noise or other unforeseen effects are averaged so that the deviation vector can be used as a compensation factor by generating a modified feature vector. The deviations between the feature and the codebook vectors can be averaged in time by using a low pass filter filtering the deviation vectors. In accordance with systems consistent with the present invention a robust automatic speech recognition system is provided. The invention also provides a software tool for implementing on a computer a robust speech recognition system and a computer program product comprising computer program code which when executed on a computer enables the computer to carry out the robust speech recognition method according to the present invention. A robust speech recognition system according to the present invention may be realized by a digital signal processing system which performs the signal processing in a program- or state-machine-controlled manner. In accordance with methods, systems and articles of manufacture consistent with the present invention, the noise robust speech recognition is used for recognizing noisy speech signals transmitted acoustically, which speech signals come from a near end user of a device that carries out the noise robust speech recognition method or contains the noise robust speech recognition system, respectively, and the noise recognition is used for controlling that device. The speech recognition method or the speech recognition system may particularly be used for controlling functions of a device in a noisy environment and in a situation where limited processing power is available. Particularly, a speech recognition system and method may be used for controlling functions of devices of e.g. a motor vehicle, e.g. a window winder, a radio receiver, a navigation system, a mobile telephone or even to control the motor vehicle itself. The above-mentioned and other features, utilities and advantages of the invention will become apparent from the following detailed description of preferred embodiments of the invention together with the accompanying drawings. Other systems, methods, features and advantages of the invention will be or become apparent to one skilled in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an implementation of the invention and, together with the description, serve to explain the advantages and principles of the invention. In the drawings: Fig. 1 depicts a block diagram of a noise robust speech recognition system according to first embodiment of the present invention; Fig. 2 depicts a block diagram of a noise robust speech recognition system according to second embodiment of the present invention; Fig. 3 depicts a block diagram which illustrates a digital signal processing system according to an embodiment of the present invention; Fig. 4 depicts a flow diagram which illustrates a method according to an embodiment of the present invention; Fig. 5 depicts a flow diagram illustrating in more detail an operation of generating the confidence vector consistent with a method according to the present invention; Fig. 6 depicts a flow diagram illustrating in more detail an operation of generating the codebook vector consistent with a method according to the present invention; Fig. 7 depicts a flow diagram which illustrates a method according to an embodiment of the present invention; Fig. 8 depicts a flow diagram illustrating in more detail an operation of generating the deviation vector consistent with a method according to the present invention; Fig. 9 depicts a flow diagram illustrating in more detail an operation of generating the feature vector using the deviation vector consistent with a method according to the present invention; Fig. 10 depicts a flow diagram which illustrates a method according to an embodiment of the present invention; Fig. 11 depicts a block diagram of a prior art voice recognition system; Fig. 12 depicts a block diagram of a prior art noise suppressor; Fig. 13 depicts a scheme of a prior art filter function calculator; Fig. 14 depicts a block diagram of a data flow system which illustrates a noise suppressor according to an embodiment of the present invention; Fig. 15 depicts a block diagram of a data flow diagram of a noise robust speech recognition system in accordance with methods, systems and articles of manufacture consistent with the present invention; Fig. 16a depicts a scheme of a prior art feature generator; Fig. 16b depicts a scheme of a feature generator according to an embodiment of the present invention; Fig. 17 depicts a block diagram of a feature comparator according to an embodiment of the present invention; Fig. 18 depicts a block diagram of a deviation calculator or deviation vector calculator according to an embodiment of the present invention. DESCRIPTIONS OF THE PREFERRED EMBODIMENTS The following symbols will be used by describing the preferred embodiments according to the present invention: A(f,T) Signal spectrum, i.e. amplitude of frequency mode f at time T after noise suppression; c(T) Confidence vector computed by the confidence unit of the noise suppressor; c k (T) components of c(T); d(T) Deviation vector computed by the deviation calculator; d k (T) components of d(T); D Diffusion constant determining the amount of smoothing in the diffusion layer; F(f,T) Filter function generating A(f,T) from X(f,T): A(f,T) = X(f,T)F(f,T) for all f at time T; f frequency which distinguishes the modes of a spectrum; g Function set for computing a feature vector U(T) out of a spectrum A(f,T); h Function set for computing confidence vector c(T); k Index denoting the components of vectors d(T), c(T), U(T) and V as well as of the transformations g and h; K Parameter for setting the amount of noise suppression; I Number of frames from which M(f,T) may be obtained as the minimum of the averaged X(f,T); m Number of frames averaged to determine M(f,T); M(f,T) Minimum within I frames of amplitude X(f,T) averaged over m; n Number of samples per frame; q Time constant in deviation calculator; R(f,T) Relative spectrum generated by the reaction layer of the noise suppressor; r[S(T)] Reaction function of the filter function calculator of the noise suppressor; S(T) Integration signal corresponding to the integral of F(f,T) over f at time T; t Time in which the speech signal is sampled; T Time in which the time signal is processed to form frames and spectra are derived therefrom; U(T) Feature vector at time T computed from A(f,T); U k (T) components of U(T); V Set of codebook vectors in feature space; V(i) codebook vector in V; V k (i) components of V(i); V k (T) components of V(T); V(T) codebook vector with least distance to U(T); x(t) Samples of the noisy speech signal; X(f,T) Spectrum of the noisy speech signal. The present invention will now be described with reference to the accompanying drawings. Fig. 1 depicts a block diagram of a robust speech recognition system 400 suitable for practicing methods and impelementing systems consistent with an embodiement of the present invention. The robust speech recognition system 400 comprises a noise suppressor 410 for generating a noise-reduced signal spectrum from a noisy speech input signal. A confidence vector generator 480 is coupled to the noise suppressor and generates a confidence vector which indicates how strong a speech component was affected by noise before the noise suppression. The noise reduced signal spectrum is further processed in a feature vector generator 420 which is also coupled to the noise suppressor and which generates a feature vector from the noise-reduced speech signal spectrum. A feature comparator 430 which receives the confidence vector and the feature vector determines a codebook vector relating to the feature vector influenced by the confidence vector. The feature comparator 430 has access to a codebook vector set 450 in which a set of codebook vectors V in feature space is stored and from which a codebook vector relating to the feature vector is selected. For example, the selected codebook vector is the best matching codebook vector which means that the selected codebook vector is the vector in feature space with the lowest or a minimum deviation to the feature vector. In a decision unit 440 of the noise robust speech recognition system 400 a recognition result is then determined by selecting a word model relating to a sequence of determined codebook vectors. The decision unit selects the word model from a word model database 460 storing the word models to be recognizable by the system. For example, a sequence of determined codebook vectors is compared with the word models and the word model having the most features in common with the sequence is recognized as the input speech signal. The confidence vector generator further improves the recognition result since the confidence vector modifies the metric between the feature vector and the codebook vectors in relation to the noise intensity affecting the speech signal. This noise related modification is achieved by taking vector components more affected by noise less into consideration then those components less affected when calculating distances between feature vectors and codebook vectors. Fig. 2 depicts a block diagram of a noise robust speech recognition system 500 suitable for practicing methods and implementing systems consistent with another embodiement of the present invention. The noise robust speech recognition system 500 comprises also a noise suppressor 410 for generating a noise-reduced signal spectrum from a noisy input speech input signal. The noise reduced signal spectrum is further processed in a feature vector generator 520 which is coupled to the noise suppressor and which generates a feature vector from the noise-reduced speech signal spectrum. The feature vector comparator 530 receives as a further input a deviation vector which is used in the process of generating the feature vector. A feature vector comparator 530 which is similar to the one described in connection with Fig. 1 receiving the feature vector from the feature vector generator determines out of a set of codebook vectors a codebook vector relating to the feature vector. In a deviation vector generator 570 a deviation vector is generated from deviations between feature vectors and relating codebook vectors. This deviation vector is then used in a next recognition cycle in the feature vector generator to generate a further feature vector. Also the feature comparator 530 has access to the codebook vector set 450 in which a set of codebook vectors V in feature space is stored and from which a codebook vector relating to the feature vector is selected. In the decision unit 440 of the noise robust speech recognition system 500 a recognition result is then determined as described with respect to Fig. 1. As well as the confidence vector, the deviation vector is used in the recognition process to improve the recognition result. The deviation vector is generated from deviations of feature vectors and codebook vectors. Since the codebook vectors are fixed in their spectral parameters or components in feature space, the feature comparision becomes difficult if the components of the feature vector are affected by unforeseen effects or drift. Such unforeseen effects could be disturbances such as a change in the signal path, e.g. by a different microphone or a different A/D-converter. The deviation vector generator then determines the deviation between the feature vectors and the codebook vectors and generates a deviation vector for compensation of this deviation. According to an embodiement of the invention, the deviation vector also averages deviations between the feature vectors and the codebook vectors over the time by using a kind of a low pass filter funtionality. One skilled in the art will appreciate that systems and methods consistent with the present invention may by implemented in various data processing systems, like digital signal processing (DSP) systems, microcontroller, or any other computer system. For example, the present invention may be implmented in a digital signal processor (DSP) system 1300 as illustrated schematically in Fig. 3. The DSP system implementing the noise robust speech recognition system according to the invention may comprise a processing unit and one or more storage units. A noisy speech signal is recorded by a microphone 1310 and preprocessed by a sampling unit 1320. The sampling unit 1320 may comprise a sample and hold circuit and an A/D-converter which produces samples x(t) of the noisy speech signal which then are input to the processing unit like a speech recognition processor 1330. The DSP system may further comprise an output device 1360. The functionality of the system as described with respect to Fig. 1 and 2 is realized by a computer program stored in memory 1340 which when executed on the speech recognition processor implements methods and systems consistent with the present invention. The memory 1340 may be a volatile memory and may be used as main memory of the system 1300. As storage units, the system may further comprise a non-volatile memory 1350 storing the codebook vector set V and a non-volatile memory 1370 storing the word models related to words to be recognized. In accordance with an embodiment of the invention, the memory 1340 includes the confidence vector as a data structure 1341. This means that the current confidence vector after generating in the speech recognition processor 1330 is stored in the memory 1340 and fetched by the processor 1330 before the distance between the feature vector and the confidence vector is calculated. In another embodiment of the invention, a set of confidence vectors generated by the system is stored in memory and used in further speech recognition operations. In accordance with another embodiment of the invention, the memory 1340 includes the deviation vector as a data structure 1342. This means that the current deviation vector after generation or updating in the speech recognition processor 1330 is stored in the memory 1340 and fetched by the processor 1330 before a feature vector is generated. In another embodiment of the invention, also the last current deviation vector or a set of deviation vectors generated by the system is stored in memory and used to generate the new deviation vector in the update operation. For example, the present or last current deviation vector enters into the generation of the new the deviation vector to implement a kind of a low pass filter functionality or may be used in further speech recognition operations. The noise robust speech recognition system 1330 according to the present invention is particularly applicable to DSP systems comprising a standard DSP processor like a Texas Instruments TMS 320C5402 or any other fixed-point DSP with a performance range of at least 60 MIPS. Such a performance is sufficient due to the good recognition results of the system also by a relatively low sampling rate in the range of 8 to 16 kHz and a processing width of 16 bit. Good recognition results can be achieved by a sampling rate in the range of 12 kHz to 16 kHz. For example, low cost processors are applicable to the system which may be implemented in an embedded control application like a board computer in a motor vehicle or the like without the need for further processing power. The result of the speech recognition used to control functions of e.g. the vehicle or any other systems by outputting the recognition result in an appropriate format at an output device 1360. The output device may for example be an interface circuit to a radio receiver or a mobile telephone. According to an embodiment of the invention, the speech recognition processor generates in response to instructions stored in the main memory 1340 noise-reduced signal spectra and a confidence vector from a noisy speech input signal recorded by the microphone 1310 and sampled by the sampling unit 1320. Then, the speech recognition processor 1330 further generates in response to instructions stored in the memory a sequence of feature vectors from said noise-reduced signal spectra, determines in relation to said sequence of feature vectors a sequence of codebook vectors from a set of codebook vectors stored in a flash memory 1350 by using said confidence vectors and then a recognition result by calculating from a set of word models stored in the flash memory 1350 a best matching word model relating to said sequence of codebook vectors. An embodiment of the noise robust speech recognition method according to the present invention is shown in Fig. 4 and will now be described. After or together with the generation of a noise-reduced signal spectrum from a noisy speech signal (operation 100) a confidence vector is generated (operation 110) from the spectrum of the noise in the noisy speech signal. In operation 120, a feature vector is generated from the noise reduced signal spectrum. After feature vector generation, a codebook vector relating to the feature vector is determined in operation 130. The feature vector represents an energy distribution in different frequency bands of the noise-reduced signal spectrum after transformation with a function set g. The function set g comprises transformation functions g k for different spectral bands as depicted in Figs. 16a and 16b. With the function set g the signal sectrum is transformed in a manner that the components of the feature vector are suited to qualify the human speech. A set of codebook vectors V in feature space is used as internal data representation representing parts of word models to be recognized. The codebook vectors have components similar to the components of the feature vector. A sequence of codebook vectors relates to a word model. In the determination operation 130 it is now determined which codebook vector is the best matching codebook vector relating to the feature vector by using the confidence vector. The best matching codebook vector may be found by determine a codebook vector which is most similar to the feature vector. According to an embodiment of the invention, the best matching codebook vector is that codebook vector which has a minimum distance from the feature vector. The determination of the distance between the codebook vector and the feature vector is influenced by the confidence vector such that the confidence vector modifies the metric used for the distance determination in feature space. A codebook vector V(T) is selected as that codebook vector out of the set V having least distance to the feature vector U(T) by comparing the feature vector with the set of codebook vectors V under a metric modified by the confidence vector c(T). The confidence vector modifies the metric for the distance calculation in a way that components of the feature vector U k (T) which are more affected by noise are rated weaker than components less affected by noise. Codebook vectors V(T) with such least non-Euclidian distances to feature vectors U(T) are then further processed to determine a recognition result in operation 140. The generation of the confidence vector c(T) is shown with reference to Fig. 5. In operation 111, a function h k out of the function set h is selected. By means of example, the function set h k is depicted in Fig. 14. Filter function F(f,T) is then convoluted with selected function h k in operation 112 to generate a confidence vector component c k (T). With the filter function F(f,T), the noise-reduced signal spectrum was generated from the spectrum of the noisy speech signal. Therefore, the filter function may be used to determine the amount of speech degradation within the respective band and to calculate components of the confidence vector. The operations 111 and 112 are repeatedly carried out by a loop operation until a complete set of confidence vector components c k (T) is calculated and the resulting confidence vector c(T) is output in operation 113. At the beginning of the loop operation, k is initialized and then used as loop index by incrementing k in every loop cycle (operation 114). One skilled in the art will appreciate to use beside incrementing k any other method may be used to select a next possible components to generate a complete confidence vector c(T). To determine the best matching codebook vector V(T), at least one codebook vector V(i) with a least distance to the feature vector U(T) is selected from the set of codebook vectors V as shown in Fig. 6. Starting with any codebook vector V(i) out of the set of codebook vectors V, a component U k (T) of the feature vector U(T) and a respective component V k (i) of the codebook vector V(i) are selected in operation 131. Then, a difference between the components of the feature vector U k (T) and the respective components of the codebook vector V k (i) is calculated in operation 132. The difference is weighted with the respective component of the confidence vector C k (T) in operation 133. Weighted differences for all components are calculated in a loop operation with k as the loop index. At the beginning, as shown in Fig. 6, k is initialized and then incremented in every loop cycle (operation 135). After weighted differences are calculated for all components of V(i) and U(T), the loop is left and the distance between the feature vector U(T) and the codebook vector V(i) is calculated from the weighted differences in operation 134. This means that the distance of codebook vector V(i) to U(T) is determined using a non-Euclidian metric modified by C (T).ln operation 136, the distance of the present codebook vector with least distance to the feature vector V(T) is compared with the distance of the codebook vector V(i). If the distance of V(i) is smaller than the distance of the former best matching codebook vector V(T) or there is no V(T), V(i) becomes the new best matching codebook vector V(T). Then, a next codebook vector V(i) is selected in operation 137 and the operations 131 to 136 are repeatedly carried out until the best matching codebook vector V(T) out of the set of codebook vectors V is determined. In a subsequent operation 138 the best matching codebook vector V(T) is then output and may then be used to determine a recognition result. According to another embodiment of the present invention, a noise robust speech recognition method comprises the generation of a deviation vector which will be now described in connection with Figs. 7 to 9 and 10. In operations 210 a noise-reduced signal spectrum is generated from a noisy speech signal. From this noise-reduced signal spectrum a feature vector is generated by using the deviation vector (operation 220). Then, a codebook vector is determined in operation 230 using a metric which may be modified as described above. Deviations between the feature vector U(T) and the codebook vector V(T) are used to update the deviation vector in operation 240. In subsequent operation 250, a recognition result is yielded depending on the determined codebook vectors. A sequence of these codebook vectors is compared with word models stored in the system and the recognition result is then either a word out of the set of words recognizable by the method or an output that no word could be recognized. The current deviation vector d(T) is used to calculate the feature vector U(T) (in operation 241). By means of a time constant q the deviations between the feature vectors and the codebook vectors caused by disturbances or other unforeseen effects are averaged over the time. In other words, a new deviation vector d(T+1) may be calculated from the present deviation vector d(T) and the deviations between the feature vector and the codebook vector. Depending on the time constant q, the newly calculated deviation vector d(T+1) is mainly determined by either the present deviation vector (q near 0) or mainly by the deviation between the present feature vector U(T) and the best matching codebook vector V(T) (q near 1). In a configuration according to an embodiment of the invention, it may be preferred that the time constant q has a value between 0 and 0.5 which results that the new deviation vector d(T+1) is a good compromise between the low pass function for compensating a disturbance or unforeseen effects like dynamic noise and the sensitivity of the deviation vector for deviations between the feature vector and the codebook vector. With reference to the flow diagram depicted in Fig. 8, a more detailled operation of deviation vector calculation is now described. After calculation of the featue vector U(T) by using d(T) in operation 241, the deviation between the feature vector and the determined codebook vector V(T) is calculated (operation 242). In operation 243 time constant q is selected and in a subsequent operation new deviation vector d(T+1) is calculated from current deviation vector d(T), deviation between V(T) and U(T) and the time constant q. According to an embodiment of the present invention, the deviation vector may be calculated by the following formula: d(T+1) = d(T) + (d(T)V(T)/U(T) - d(T))q Fig. 8 is a flow diagram illustrating in more detail how the feature vector is calculated (operation 220 in Fig. 6 and operation 241 in Fig. 7, respectively). Component index k ist initialized in operation 256. In a following loop operation in which k serves as a loop index components of the feature vector are calculated from the noise-reduced signal spectrum A(f,T) by transformation g k in operation 251. In each loop one component U k (T) of the modified feature vector is calculated. In operation 252, the respective component of the deviation vector is selected with which in operation 253 the calculated feature vector component is multiplied. After incrementing the loop index k in operation 254, operations 251 to 253 are repeated until a complete modified feature vector is generated and output in operation 255. According to another embodiment of the present invention, the deviation vector is updated each time a new codebook vector is determined so that always a current deviation vector is present. Fig. 10 depicts a block diagram of a data flow starting with frames of a noisy speech input signal to a recogntition result which practices methods and implements systems consistent with the present invention. A next frame of the stream of noisy speech input signal including samples of the noisy speech signal is got in operation 310 and may be prepared for further processing, by e.g. quantization. From the noise reduced signal a signal spectrum is generated and in a subsequent step a noise reduced signal spectrum A(f,T) is generated by filter function F(f,T) in operation 320. For noise reduction a method similar to the one described with respect to Fig. 12 may be used. The filter function may be generated by a method similar to the one described with respect to Fig. 13. The noise reduced signal spectrum A(f,T) is then used in operation 330 to generate the feature vector U(T) by using the deviation vector d(T). In operation 340, the confidence vector is generated from the filter function F(f,T) and function set h k . Both, the feature vector and the confidence vector are used to determine the codebook vector relating to the feature vector in operation 350. This determined codebook vector represents the best matching codebook vector in the set of codebook vectors available by the system under the used metric. Both, current best matching codebook vector and current feature vector are then used to update the deviation vector, which means that a new deviation vector is calculated in operation 360. This new deviation vector is used in a next recognition cycle to generate a new feature vector (operation 330 or operation 241: calculating U(T) by using d(T)). After a sequence of codebook vectors is determined, in operation 140, 260 or 350, word models corresponding to these codebook vector sequence are selected and output as recognition result in operation 370. A block diagram of a voice recognition system of an embodiment according to the present invention is shown in Fig. 15. The noise robust speech recognition system comprises a sampling unit 910 which samples, digitizes and divides a speech signal x(t). The speech signal is then used to compute a noise reduced signal spectrum A(f,T) in the noise suppressor 920. The filter function F(f,T) which generates the signal spectrum is also output from the noise suppressor 920 and used by the confidence calculator 960 to calculate the components of the confidence vector c(T). In the feature generator 930 the feature vector U(T) is calculated from signal spectrum A(f,T) and deviation vector d(T) which is calculated in deviation calculator 970. The feature comparator then compares components of the feature vector with components of vectors of a set of codebook vectors and chooses a codebook vector which is nearest to the feature vector under a metric modified by the confidence vector. A decision unit 950 processes a sequence of codebook vectors in order to calculate a recognition result by comparing the sequences of codebook vectors with word models which are preferably stored in a storage unit. A discriminator of the decision unit 950 finds then the best matching word model relating to the codebook vector sequence and outputs it as the recognition result. During the processing, a deviation calculator calculates a deviation between the feature vector and the confidence vector and averages the deviations preferably by means of a low-pass filter so that noise or other unforeseen effects which cause a deviation between the feature vector and the codebook vector can be compensated. Fig. 14 depicts a block diagram showing the noise suppressor 410 which is now described. The noisy speech input signal is input to a Fourier transform unit 810 which outputs a spectrum of the noisy speech signal which is then filtered in the filter unit 830 by the filter function F(f,T) generated in the filter function calculator 820. The filter function is then also used to calculate the confidence vector in a confidence calculation unit 840. The components of the confidence vector are calculated by a discrete integration of the filter function and the function set h k over the frequencies which correspond to the modes of the spectrum. As illustrated in Fig. 14, confidence vector c(T) may be calculated in confidence calulator 840 by using a function set with components h k . Each component of confidence vector c k (T) may be calculated by the fomula: c k ( T ) = ∫ F ( f , T ) h k ( f ) df . The components c k (T) are then assembled to confidence vector c(T) and the noise suppressor 410 outputs the confidence vector and the noise-reduced signal spectrum. The influence of the confidence vector to the feature vector is described in connection with Figs. 16a and 16b. In Fig. 16a, a state of the art feature generator is shown in which a noise-reduced signal spectrum is input and transformed by function g k to generate the feature vector U(T). The transformation g k in Fig. 16a has the same amplitude for all modes, so that the amplitude characteristic of the feature vector corresponds one to one to the one of the noise-reduced signal spectrum A(f,T). In contrast to that, Fig. 16b depicts how a feature vector is generated according to an embodiment of the present invention. In Fig. 16b, a feature generator is shown which has the deviation vector as an additional input. The feature generator convolutes the transformed values of the noise-reduced signal spectrum with components of the deviation vector in order to calculate the components U k (T) of the modified feature vector. As can be seen from Fig. 16b the amplitudes of the modes of the transformation g k are influenced such by the deviation vector that the averaged deviation between the modified feature vector and the codebook vector components are reduced. This means that the noise robust speech recognition system according to the present invention uses not only a static transformation g k calculating a feature vector having the same amplitude characteristic as the noise-reduced signal spectrum but also implements to modify the feature vector by the deviation vector. As an example, if the set of codebook vectors in feature space stored in the system was recorded with a specific microphone with a certain frequency response curve or characteristic and now in the field another microphone with a different frequency characteristic is used, a corresponding deviation resulting from the different characteristics of the microphones between the feature vectors and the codebook vectors would occur resulting in an overall deterioration of the recognition result. According to the present invention, the deviation between the feature vectors and the codebook vectors is reduced by determining the averaged deviation as deviation vector and correcting this deviation by modifying the transformation g k so that the modified feature vector gets nearer to the stored codebook vector in the further processing. The modified feature vector calculation by means of the deviation vector can be expressed by the following formula: U k (T) = g k (A(f,T))d k (T). In Fig. 17 an embodiment of the feature comparator according to the present invention is shown. The confidence vector modifies a metric used for distance computation in feature space in a way that vector components of the feature vector which are more affected by noise are rated weaker than those components that are less affected. The feature comparator 1110 comprises a calculating unit 1120 and a storage unit 1130 storing the codebook vector set V. Feature vector U(T) and confidence vector c(T) are input to the feature comparator and in the feature comparator calculating unit the codebook vector V(T) with least non-Euclidian distance modified by the confidence vector to feature vector U(T) is the output of the feature comparator. In the feature comparator calculating unit the distance between the feature vector U(T) and a codebook vector V(i) of the codebook vector set V influenced by the confidence vector c(T) may be calculated by the formula: In Fig. 18, a deviation calculator 1210 according to an embodiment of the present invention is shown. The derivation calculator comprises a deviation calculator calculating unit 1220 and storage unit 1230 storing the present deviation vector d(T). The present feature vector and the present codebook vector are input to the deviation calculator and the new deviation vector d(T+1) is calculated in the deviation calculator calculating unit by the formula: d(T+1) = d(T) + (d(T)V(T)/U(T) - d(T))q. The new deviation vector d(T+1) is the output from the deviation calculator 1210 and also stored in the storage unit 1230. Both, the feature comparator calculating unit 1120 and the deviation calculating unit 1220 access each the storage unit 1130 or 1230, respectively, in order to fetch a codebook vector or a deviation vector from the respective storage units. However, according to still a further embodiment of the invention, storage units 1130 and 1230 may be implemented in a common memory. The set of codebook vectors in feature space is generated by algorithms which are well known in the art, e.g. the so called k-means algorithm. The set of codebook vectors may be stored in a separate database or also in the common memory. According to an embodiment of the present invention, the noise robust speech recognition method or system is realized as software tool for implementing on a computer a noise robust speech recognition system or method as described above. According to another embodiment of the present invention, a computer program product is provided, which comprises computer program code which when executed on a computer enables the computer to carry out a noise robust speech recognition method as described above. The computer program product may be a computer-readable medium like a CD, a computer disk, DVD, flash memory, EPROM or the like or may also be a downloadable software code portion on a server computer which is downloadable, e.g. via the Internet or the like. It is also within the scope of the invention to implement the noise robust speech recognition system according to an embodiment of the invention in an application specific integrated circuit (ASIC) or an system-on-a-chip in which all functional units of the system are integrated. The control logic to carrying out a method according to the invention may then be implemented in a state machine on the integrated circuit. The described implementation includes software or hardware but methods, systems, and articles of manufacture consistent with the present invention may be implemented as a combination of software and hardware or in software or hardware alone. The invention may be implemented with assembler, object-oriented and non-object-oriented programming systems. While the invention has been particularly shown with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made therein without departing from the scope of the invention as defined by the appended claims.

Claims: Robust speech recognition method comprising: generating (100) a noise-reduced signal spectrum by filtering a noisy speech input signal using a filter function; generating (110) a confidence vector from said filter function which indicates how strong a speech component was affected by noise before the noise suppression; generating (120) a feature vector from said noise-reduced signal spectrum; determining (130) a codebook vector relating to said feature vector by using said confidence vector; determining (140) a recognition result by determining a best matching word model relating to determined codebook vectors. The method of claim 1, wherein said determined codebook vector is the best matching codebook vector having a minimum distance to said feature vector. The method of claim 2, wherein said best matching codebook vector is determined by selecting a codebook vector V(T) with least distance to said feature vector U(T) by comparing said feature vector with a set of codebook vectors V under a metric modified by said confidence vector c(T). The method of claim 3, wherein said confidence vector modifying the metric used for distance calculation in a way that components of said feature vector more affected by noise are weighted weaker then components less affected. The method of one of the preceding claims, wherein said confidence vector c(T) is determined by transforming said filter function F(f,T) generating said noise-reduced signal spectrum, and said generation of said confidence vector comprises repeatedly carrying out: selecting (111) a function h k out of function set h; generating (112) a component c k (T) of said confidence vector by convoluting said filter function F(f,T) with said function h k ; incrementing (114) k; and wherein said method further comprises: outputting (113) said confidence vector c(T). The method of one of the preceding claims, wherein said determination of a codebook vector V(T) further comprises repeatedly carrying out: selecting (131) a component U k (T) of said feature vector U(T) and a component V k (i) of a codebook vector V(i); calculating (132) difference between components U k (T) and V k (i); weighting (133) said difference with c k (T); incrementing (135) k; and wherein said method further comprises: calculating (134) a distance of codebook vector V(i) from said weighted differences; setting (136) codebook vector V(i) as codebook vector V(T) if distance of V(i) is smaller than distance of V(T); selecting (137) next V(i); and wherein said method further comprises: outputting (138) said codebook vector V(T) as best matching codebook vector. The method of one of the preceding claims, wherein the distance calculation is defined by a formula: The method of one of the preceding claims, further comprising: generating (330) said feature vector by using a deviation vector; updating (360) said deviation vector. Robust speech recognition method comprising: generating (210) noise-reduced signal spectra from a noisy speech input signal; generating (220) feature vectors from said noise-reduced signal spectra by using a deviation vector; determining (230) codebook vectors relating to said feature vectors; updating (240) said deviation vector by averaging deviations between said feature vectors U(T) and said relating codebook vectors V(T); determining (250) a recognition result by determining a best matching word model relating to determined codebook vectors. The method of claim 9, wherein said generation (220) of said feature vector further comprises repeatedly carrying out: calculating (251) component U k (T) of said feature vector by transforming said noise-reduced signal spectrum A(f,T) with transformation g k ; selecting (252) component d k (T) of said deviation vector; multiplying (253) said feature vector component U k (T) with said deviation vector component d k (T); incrementing (254) k; and wherein said method further comprises: outputting (255) said feature vector U(T). The method of claim 10, wherein the deviation vector is updated each time a new codebook vector is determined. The method of one of claims 9 to 11, wherein said deviation vector is determined by: calculating (241) U(T) by using current d(T); calculating (242) deviation between V(T) and U(T); selecting (243) time constant q; selecting current deviation vector d(T); calculating (244) new deviation vector d(T+1) from d(T), deviation between V(T) and U(T) and time constant q. The method of claim 12, wherein said deviation vector d(T) is calculated by a formula: d(T+1)=d(T) + (d(T)V(T)/U(T) - d(T))q; wherein said q is a value between zero and one. The method of one of the preceding claims, wherein said recognition result is determined by comparing a sequence of said determined codebook vectors with word models representing words to be recognized. Robust speech recognition system (400) comprising: a noise suppressor (410) for generating a noise-reduced signal spectrum by filtering a noisy input signal using a filter function; a confidence vector generator (480) for generating a confidence vector from said filter function which indicates how strong a speech component was affected by noise before the noise suppression; a feature vector generator (420) for generating a feature vector from said noise-reduced signal spectrum; a feature comparator (430) determining a codebook vector relating to said feature vector by using said confidence vector; a decision unit (440) determining a recognition result by determining a best matching word model relating to a sequence of said determined codebook vectors. Robust speech recognition system (500) comprising: a noise suppressor (410) for generating noise-reduced signal spectra from a noisy input signal; a feature vector generator (520) for generating feature vectors from said noise-reduced signal spectra using a deviation vector; a feature comparator (530) determining codebook vectors relating to said feature vectors; a deviation vector generator (570) for generating said deviation vector by averaging deviations between said feature vectors and said relating codebook vectors; a decision unit (440) determining a recognition result by determining a best matching word model relating to a sequence of said determined codebook vectors. The system of one of the claims 15 or 16, wherein said feature comparator (430, 530, 940) outputs at least one codebook vector with least distance to said modified feature vector by comparing said feature vector with codebook vectors under a metric modified by said confidence vector. The system of one of the claims 16 to 17, wherein said decision unit (440, 950) determines the recognition result by comparing a sequence of said determined codebook vectors with word models representing words to be recognized. A software tool for implementing on a computer a robust speech recognition system, said software tool comprising: a software code portion for generating a noise-reduced signal spectrum by filtering a noisy speech input signal using a filter function; a software code portion for generating a confidence vector from said filter function which indicates how strong a speech component was affected by noise before noise suppression; a software code portion for generating a feature vector from said noise-reduced signal spectrum; a software code portion for determining a codebook vector relating to said feature vector by using said confidence vector; a software code portion for determining a recognition result by determining a best matching word model relating to said determined codebook vectors. The software tool of claim 19, wherein said codebook vector having a minimum distance to feature vector U(T) is determined by a software code portion for selecting a codebook vector with least distance to said feature vector by comparing said feature vector with a set of codebook vectors under a metric modified by said confidence vector. The software tool of claim 20, wherein said metric is modified by a software code portion by weighting components of said feature vector more affected by noise weaker then components less affected. The software tool of one of the claims 20 or 21, further comprising a software code portion for calculating said distance by a formula: The software tool of one of claims 19 to 22, wherein said confidence vector c(T) is generated by transforming said filter function F(f,T) generating said noise-reduced signal spectrum, and said software tool repeatedly carrying out: a software code portion for selecting a function set h k ; a software code portion for generating a component c k (T) of said confidence vector by transforming said filter function F(f,T) with said function set h k ; a software code portion for selecting next parameter k; and wherein said software tool further comprises: a software code portion for determining and outputting said confidence vector c(T). The software tool of one of the claims 19 to 23, further comprising: a software code portion for generating said feature vector by using a deviation vector; a software code portion for updating said deviating vector by calculating deviation between said feature vector and said codebook vector. A software tool for implementing on a computer a robust speech recognition system, said software tool comprising: a software code portion for generating a noise-reduced signal spectra from a noisy speech input signal; a software code portion for generating feature vectors from said noise-reduced signal spectra by using a deviation vector; a software code portion for determining codebook vectors relating to said feature vectors; a software code portion for updating said deviation vector by averaging deviations between said feature vectors and said relating codebook vectors; a software code portion for determining a recognition result by determining a best matching word model relating to said determined codebook vectors. The software tool of claim 25, further comprising a software code portion for generating said feature vector in which first a feature vector is generated by not using said deviation vector and then this feature vector is multiplied with said deviation vector. The software tool of claim 26, further comprising a software code portion in which the deviation vector is generated each time a new codebook vector is determined. The software tool of one of claims 25 to 27, further comprising a software code portion for calculating said deviation vector d(T) by a formula: d(T+1)=d(T) + (d(T)V(T)/U(T) - d(T))q; wherein q is a time constant between zero and one. The software tool of one of the claims 24 to 28, further comprising a software code portion in which said recognition result is determined by comparing said determined codebook vectors with word models representing words to be recognized. A computer program product comprising computer program code which when executed on a computer enables said computer to carry out a method comprising: generating a noise-reduced signal spectra by filtering a noisy speech input signal; generating a confidence vector from said filter function which indicates how strong a speech component was affected by noise before the noise suppression; generating feature vectors from said noise-reduced signal spectra by using a deviation vector; determining codebook vectors relating to said feature vectors by using said confidence vector; updating said deviation vector by averaging deviations between said feature vectors and said relating codebook vectors; determining a recognition result by determining a best matching word model relating to a sequence of said determined codebook vectors. A computer program comprising computer program code which when executed on a computer enables said computer to carry out a method according to one of claims 1 to 14. A digital signal processing system (1300) comprising: a processing unit and a storage unit; said processing unit generates in response to instructions stored in said storage unit a noise-reduced signal spectra by filtering a noisy speech input signal using a filter function; said processing unit generates in response to instructions stored in said storage unit a confidence vector (1341) from said filter function which indicates how strong a speech component was affected by noise before the noise suppression; said processing unit further generates in response to instructions stored in said storage unit feature vectors from said noise-reduced signal spectra using a deviation vector (1342); said processing unit further determines in response to instructions stored in said storage unit codebook vectors relating to said feature vectors from a set of codebook vectors stored in said storage unit by using said confidence vector; said processing unit further updates said deviation vector by averaging deviations between said feature vectors and said relating codebook vectors; said processing unit further determines in response to instructions stored in said storage unit a recognition result by determining a best matching word model relating to said determined codebook vectors from a set of word models stored in said storage unit. The digital signal processing system (1300) of claim 32, further comprising a microphone (1310) and a sampling unit (1320) for receiving and preprocessing said noisy speech input signal. The digital signal processing system (1300) of one of claims 32 or 33, in which said storage unit comprises: a volatile memory (1340) storing said instructions, said confidence vector (1341) and said deviation vector (1342); a first non-volatile memory (1350) storing said codebook vector set; and a second non-volatile memory (1360) storing said word models. A digital signal processing system of one of claims 32 to 34, wherein said system is implemented in a digital signal processor or a customized integrated circuit.

Inventors: RUWISCH DIETMAR DR, RUWISCH, DIETMAR, DR.
Applicants: RUWISCH DIETMAR DR
Application year: 2001
Main classification: G10L15
IPC codes: G10L15, G10L21