Speech feature extraction system

The present invention provides a speech feature extraction system suitable for use in a speech recognition system or other voice processing system that extracts features related to the frequency and amplitude characteristics of an input speech signal using a plurality of complex band pass filters and processing the outputs of adjacent band pass filters. The band pass filters can be arranged according to linear, logarithmic or mel-scales, or a combination thereof.

BACKGROUND OF THE INVENTION

This invention relates to a speech feature extraction system for use in speech recognition, voice identification or voice authentication systems. More specifically, this invention relates to a speech feature extraction system that can be used to create a speech recognition system or other speech processing system with a reduced error rate.

Generally, a speech recognition system is an apparatus that attempts to identify spoken words by analyzing the speaker's voice signal. Speech is converted into an electronic form from which features are extracted. The system then attempts to match a sequence of features to previously stored sequence of models associated with known speech units. When a sequence of features corresponds to a sequence of models in accordance with specified rules, the corresponding words are deemed to be recognized by the speech recognition system.

However, background sounds such as radios, car noise, or other nearby speakers can make it difficult to extract useful features from the speech. In addition, a change in the ambient conditions such as the use of a different microphone, telephone handset or telephone line can interfere with system performance. Also, a speaker's distance from the microphone, differences between speakers, changes in speaker intonation or emphasis, and even a speaker's health can adversely impact system performance. For a further description of some of these problems, see Richard A. Quinnell, “Speech Recognition: No Longer a Dream, But Still a Challenge,” EDN Magazine, Jan. 19, 1995, p. 41–46.

In most speech recognition systems, the speech features are extracted by cepstral analysis, which generally involves measuring the energy in specific frequency bands. The product of that analysis reflects the amplitude of the signal in those bands. Analysis of these amplitude changes over successive time periods can be modeled as an amplitude modulated signal.

Whereas the human ear is a sensitive to frequency modulation as well as amplitude modulation in received speech signals, this frequency modulated content is only partially reflected in systems that perform cepstral analysis.

Accordingly, it would be desirable to provide a speech feature extraction system capable of capturing the frequency modulation characteristics of speech, as well as previously known amplitude modulation characteristics.

It also would be desirable to provide speech recognition and other speech processing systems that incorporate feature extraction systems that provide information on frequency modulation characteristics of the input speech signal.

SUMMARY OF THE INVENTION

In view of the foregoing, it is an object of the present invention to provide a speech feature extraction system capable of capturing the frequency modulation characteristics of speech, as well as previously known amplitude modulation characteristics.

It also is an object of this invention to provide speech recognition and other speech processing systems that incorporate feature extraction systems that provide information on frequency modulation characteristics of the input speech signal.

The present invention provides a speech feature extraction system that reflects frequency modulation characteristics of speech as well as amplitude characteristics. This is done by a feature extraction stage which, in one embodiment, includes a plurality of complex band pass filters arranged in adjacent frequency bands according to a linear frequency scale (“linear scale”). The plurality of complex band pass filters are divided into pairs. A pair includes two complex band pass filters in adjacent frequency bands. For every pair, the output of the filter in the higher frequency band (“primary frequency”) is multiplied by the conjugate of the output of the filter in the lower frequency band (“secondary filter”). The resulting signal is low pass filtered.

In another embodiment, the feature extraction phase includes a plurality of complex band pass filters arranged according to a logarithmic (or exponential) frequency scale (“log scale”). The primary filters of the filter pairs are centered at various frequencies along the log scale. The secondary filter corresponding to the primary filter of each pair is centered at a predetermined frequency below the primary filter. For every pair, the output of the primary filter is multiplied by the conjugate of the output of the secondary filter. The resulting signal is low pass filtered.

In yet another embodiment, the plurality of band pass filters are arranged according to a mel-scale. The primary filters of the filter pairs are centered at various frequencies along the mel-scale. The secondary filter corresponding to the primary filter of each pair is centered at a predetermined frequency below the primary filter. For every pair, the output of the primary filter is multiplied by the conjugate of the output of the secondary filter. The resulting signal is low pass filtered.

In still another embodiment, the plurality of band pass filters are arranged according to a combination of the linear and log scale embodiments mentioned above. A portion of the pairs of the band pass filters are arranged in adjacent frequency bands according to a linear scale. For each of these pairs, the output of the primary filter is multiplied by the conjugate of the output of the secondary filter. The resulting signal is low pass filtered.

The primary filters of the remaining pairs of band pass filters are centered at various frequencies along the log scale and the secondary filters corresponding to the primary filters are centered a predetermined frequency below the primary filters. For every pair, the output of the primary filter is multiplied by the conjugate of the output of the secondary filter. The-resulting signal is low pass filtered.

For the embodiments described above, each of the low pass filter outputs is processed to compute two components: a FM component that is substantially sensitive to the frequency of the signal passed by the adjacent band pass filters from which the low pass filter output was generated, and an AM component that is substantially sensitive to the amplitude of the signal passed by the adjacent band pass filters. The FM component reflects the difference in the phase of the outputs of the adjacent band pass filters used to generate the lowpass filter output.

The AM and FM components are then processed using known feature enhancement techniques, such as discrete cosine transform, mel-scale translation, mean normalization, delta and acceleration analysis, linear discriminant analysis and principal component analysis, to generate speech features suitable for statistical processing or other recognition or identification methods. In an alternative embodiment, the plurality of complex band pass filters can be implemented using a Fast Fourier Transform (FFT) of the speech signal or other digital signal processing (DSP) techniques.

In addition, the methods and apparatus of the present invention may be used in addition to performing cepstral analysis in a speech recognition system.

DETAILED DESCRIPTION OF THE INVENTION

Referring toFIG. 1, a generalized depiction of illustrative speech recognition system5is described that incorporates the speech extraction system of the present invention. As will be apparent to one of ordinary skill in the art, the speech feature extraction system of the present invention also may be used in speaker identification, authentication and other voice processing systems.

System5illustratively includes four stages:pre-filtering stage10, feature extraction stage12, statistical processing stage14, and energy stage16.

Pre-filtering stage10, statistical processing stage14and energy stage16employ speech processing techniques known in the art and do not form part of the present invention. Feature extraction stage12incorporates the speech feature extraction system of the present invention, and further includes feature enhancement techniques which are known in the art, as described hereinafter.

Audio speech signal is converted into an electrical signal by a microphone, telephone receiver or other device, and provided as an input speech signal to system5. In a preferred embodiment of the present invention, the electrical signal is sampled or digitized to provide a digital signal (IN) representative of the audio speech. Pre-filtering stage10amplifies the high frequency components of audio signal IN, and the prefiltered signal is then provided to feature extraction stage12.

Feature extraction stage12processes pre-filtered signal X to generate a sequence of feature vectors related to characteristics of input signal IN that may be useful for speech recognition. The output of feature extraction stage12is used by statistical processing stage14which compares the sequence of feature vectors to predefined statistical models to identify words or other speech units in the input signal IN. The feature vectors are compared to the models using known techniques, such as the Hidden Markov Model (HMM) described in Jelinek, “Statistical Methods for Speech Recognition,” The MIT Press, 1997, pp. 15–37. The output of statistical processing stage14is the recognized word, or other suitable output depending upon the specific application.

Statistical processing at stage14may be performed locally, or at a remote location relative to where the processing of stages10,12, and16are performed. For example, the sequence of feature vectors may be transmitted to a remote server for statistical processing.

The illustrative speech recognition system ofFIG. 1preferably also includes energy stage16which provides an output signal indicative of the total energy in a frame of input signal IN. Statistical processing stage14may use this total energy information to provide improved recognition of speech contained in the input signal.

Referring now toFIG. 2, pre-filtering stage10and feature extraction stage12are described in greater detail. Pre-filtering stage10is a high pass filter that amplifies high frequency components of the input signal. Pre-filtering stage10comprises one-sample delay element21, multiplier23and adder24. Multiplier23multiplies the one-sample delayed signal by constant Kf, which typically has a value of −0.97. The output of pre-filtering stage10, X, is input at the sampling rate into a bank of band pass filters301,302, . . .30n.

In one embodiment, the band pass filters301,302, . . .30nare positioned in adjacent frequency bands. The spacing of the band pass filters301,302. . .30n, is done according to a linear frequency scale (“linear scale”)68as shown in graph72ofFIG. 5. The term “linear frequency scale” is used in this specification in accordance with its ordinary and accustomed meaning, i.e., the actual frequency divisions are uniformly spaced. The plurality of complex band pass filters301,302, . . .30nare divided into pairs P1-2. A pair (P1or P2) includes two complex band pass filters (301-2or303-4) respectively in adjacent frequency bands. For every pair (P1or P2), the output of the filter in the higher frequency band (302or304) (referred to hereinafter as the “primary filter”) is multiplied by the conjugate of the output of the filter in the lower frequency band (301or303) (referred to hereinafter as the “secondary filter”). The resulting signal is low pass filtered.

The number of band pass filters301,302. . .30nand width of the frequency bands preferably are selected according to the application for the speech processing system. For example, a system useful in telephony applications preferably will employ about forty band pass filters301,302, . . .30nhaving center frequencies approximately 100 Hz apart. For example, filter301may have a center frequency of 50 Hz, filter302may have a center frequency of 150 Hz, filter303may have a center frequency of 250 Hz, and so on, so that the center frequency of filter3040is 3950 Hz. The bandwidth of each filter may be several hundred Hertz.

In another embodiment, as illustrated in the graph70ofFIG. 6, the band pass filters301,302. . .30108are arranged according to a non-linear frequency scale such as a logarithmic (or exponential) frequency scale74(“log scale”). The term logarithmic frequency scale is used in this specification according to its ordinary and accustomed meaning.

Empirical evidence suggests that using log scale74ofFIG. 6instead of linear scale68ofFIG. 5improves voice recognition performance. That is so because the human ear resolves frequencies non-linearly across the audio spectrum. Another advantage using log scale74instead of linear scale68is that log scale74can cover a wider range of frequency spectrum without using additional band pass filters301,302. . .30n.

Pairs P1-54of band pass filters301-108are spaced according to log scale74. Pair P1includes filters301, and302, pair P10includes filters3019and3020, and pair P54includes filters30107and30108. In this arrangement, filters302,3020and30108are the primary filters and filters301,3019and30107are the secondary filters.

In one preferred embodiment, primary filters302,3020. . .30108are centered at various frequencies along log scale74, while secondary filters301,303. . .30107are centered 100 hertz (Hz) below corresponding primary filters302,304. . .30108respectively. An exemplary MATLAB code to generate graph70ofFIG. 6is shown below.

v=(2.{circumflex over ( )}([26.715:0.25:40]/3.345));% Generate center frequencies for bandpass filter pairsf(2:2:2*length(v))=v+50;% Primary filters center frequenciesf(1:2:2*length(v))=v−50;% Secondary filters center frequenciessemilogy([v′+50 v′−50],′.′);grid% Plot center frequencies on a logarithmic scale′

In another embodiment, the center frequencies of primary filters302,3020. . .30108and secondary filters301,303. . .30107may be placed using separate and independent algorithms, while ensuring that secondary filters301,303. . .30107are centered 100 hertz (Hz) below their corresponding primary filters302,304. . .30108, respectively.

In one embodiment, band pass filters301-108are of triangular shape. In other embodiments, band pass filters301-108can be of various shapes depending on the requirements of the particular voice recognition systems.

Log scale74is shown to range from 0–4000 Hz. For every pair P1-54, the output of the primary filter302,304. . .30108is multiplied by the conjugate of the output of secondary filter301,303. . .30107. The resulting signal is low pass filtered.

The pairs P1-54are arranged such that the lower frequencies include a higher concentration of pairs P1-54than the higher frequencies. For example, there are 7 pairs (P16-22) in the frequency range of 500–1000 Hz while there are only 3 pairs (P49-51) in the frequency range of 3000–3500 Hz. Thus, although there is over sampling at the lower frequencies, this embodiment also performs at least some sampling at the higher frequencies. The concentration of pairs P1-54along log scale74can be varied depending on the needs of a particular voice recognition system.

As will be apparent to one of ordinary skill in the art of digital signal processing design, the band pass filters of the preceding embodiments may be implemented using any of a number of software or hardware techniques. For example, the plurality of complex filters may be implemented using a Fast Fourier Transform (FFT), Chirp-Z transform, other frequency domain analysis techniques.

In an alternative embodiment, depicted inFIG. 7, band pass filters301,302, . . .30nare arranged according to a non-linear frequency scale, such as mel-scale80. Mel-scale80is well known in the art of voice recognition systems and is typically defined by the equation,Mel⁡(f)=2595⁢⁢log10⁡(1+f700)
where f represents frequency according to the linear scale68and Mel(f) represents its corresponding mel-scale80frequency.

FIG. 7illustrates one embodiment of a graph84showing band pass filters301-9spaced according to mel-scale80. The center frequencies (CF1-9) are the Mel(f) values calculated by using the above equation. Typically, filters301-9are spread over the whole frequency range from zero up to the Nyquist frequency. In one embodiment, filters301-9have the same band width. In another embodiment, filters301-9may have different bandwidths.

In yet another embodiment, depicted inFIG. 8, the band pass filters301-8are spaced according to a combination of linear68and non-linear74frequency scales. Band pass filters301-4(P1-2) are arranged in adjacent frequency bands according to linear frequency68.

Primary filters306and308are centered along log scale74. Secondary filters305and307are centered at frequencies of 100 Hz below the center frequencies of306and308respectively. For each of these pairs (P1or P2), the output of the primary filter (306or308) is multiplied by the conjugate of the output of the secondary filter (305and307), respectively and the resulting signal is low pass filtered.

Referring again toFIG. 2, blocks401-20provide the complex conjugate of the output signal of band pass filter301,303, . . .30n−1. Multiplier blocks421-20multiply the complex conjugates by the outputs of an adjacent higher frequency band pass filter302,304,306, . . .3040to provide output signals Z1-20. Output signals Z1-20then are passed through a series of low pass filters441-20. The outputs of the low pass filters typically are generated only at the feature frame rate. For example, at a input speech sampling rate of 8 kHz, the output of the low pass filters is only computed at a feature frame rate of once every 10 msec.

Each output of low pass filters441-20is a complex signal having real component R and imaginary component I. Blocks461-20process the real and imaginary components of the low pass filter outputs to provide output signals A1-20and F1-20as shown in equations (1) and (2):Ai=log⁢⁢Ri(1)fi=IiRi2+Ii2(2)
wherein Riand Iiare the real and imaginary components of the corresponding low pass filter output. Output signals Aiare a function of the amplitude of the low pass filter output and signals Fiare a function of the frequency of the signal passed by the adjacent band pass filters from which the low pass filter output was generated. By computing two sets of signals that are indicative of the amplitude and frequency of the input signal, the speech recognition system incorporating the speech feature extraction system of the present invention is expected to provide reduced error rate.

The amplitude and frequency signals A1-20and F1-20then are processed using conventional feature enhancement techniques in feature enhancement component12b,using, for example, discrete cosine transform, mel-scale translation, mean normalization, delta and acceleration analysis, linear discriminant analysis and principal component analysis techniques that are per se known in the art. A preferred embodiment of a speech recognition system of the present invention incorporating the speech extraction system of the present invention employs a discrete cosine transform and delta features technique, as described hereinafter.

Still referring toFIG. 2, feature enhancement component12breceives output signals A1-20and F1-20, and processes those signals using discrete cosine transform (DCT) blocks50and54, respectively. DCTs50and54attempt to diagonalize the co-variance matrix of signals A1-20and F1-20. This helps to uncorrelate the features in output signals B0-19of DCT50and output signals C0-19of DCT54. Each set of output signals B0-19and C0-19then are input into statistical processing stage14. The function performed by DCT50on input signals A1-20to provide output signals B0-19is shown by equation (3), and the function performed by DCT54on input signals F1-20to provide output signals C0-19is shown by equation (4).Br=D⁡(r)⁢∑n=0N-1⁢An+1·cos⁢⁢(2⁢n+1)⁢π⁢⁢r2⁢N(3)Cr=D⁡(r)⁢∑n=0N-1⁢Fn+1·cos⁢⁢(2⁢n+1)⁢π⁢⁢r2⁢N(4)

In equations (3) and (4), N equals the length of the input signal vectors A and F (e.g., N=20 inFIG. 2), n is an index from 0 to N−1 (e.g., n=0 to 19 in the embodiment ofFIG. 2), and r is the index of output signals B and C (e.g., r=0 to 19 in the embodiment ofFIG. 2). Thus, for each vector output signal Br, each vector of input signals A1-20are multiplied by a cosine function and D(r) and summed together as shown in equation (3). For each vector output signal Cr, each vector of input signals S1-20are multiplied by a cosine function and D(r) and summed together as shown in equation (4). D(r) are coefficients that are given by the following equations:D⁡(r)=1N⁢⁢for⁢⁢r=0(5)D⁡(r)=2N⁢⁢for⁢⁢r>0(6)

Output signals B0-19and C0-19also are input into delta blocks52and56, respectively. Each of delta blocks52and56takes the difference between measurements of feature vector values between consecutive feature frames and this difference may be used to enhance speech recognition performance. Several difference formulas may be used by delta blocks52and56, as are known in the art. For example, delta blocks52and56may take the difference between two consecutive feature frames. The output signals of delta blocks52and56are input into statistical processing stage14.

Energy stage16ofFIG. 2is a previously known technique for computing the logarithm of the total energy (represented by E) of each frame of input speech signal IN, according to the following equation:E⁡(n⁢⁢T)=log(∑i=0K-1⁢IN(n-i)⁢T2K)(7)

Equation 7 shows that energy block16takes the sum of the squares of the values of the input signal IN during the previous K sampling intervals (e.g., K=220, T= 1/8000 seconds), divides the sum by K, and takes the logarithm of the final result. Energy block16performs this calculation every frame (e.g., 10 msec), and provides the result as an input to statistical processing block14.

Referring now toFIG. 3, illustrative complex bandpass filter30′ suitable for use in the feature extraction system of the present invention is described. Filter30′ comprises adder31, multiplier32and one-sample delay element33. Multiplier32multiples the one-sample delayed output Y by complex coefficient G and the resultant is added to the input signal X to generate an output signal Y.

An alternative embodiment of the feature extraction system of the present invention is described with respect toFIG. 4. The embodiment ofFIG. 4is similar to the embodiment ofFIG. 2and includes pre-filtering stage10, statistical processing stage14, and energy stage16the operate substantially as described above. However, the embodiment ofFIG. 4differs from the previously described embodiment in that feature extraction stage12′ includes additional circuitry within feature extraction system12a,so that the feature vectors include additional information.

For example, feature extraction stage12a′includes a bank of 41 band pass filters301-41and conjugate blocks401-40. The output of each band pass filter is combined with the conjugate of the output of a lower adjacent band bass filter by multipliers421-40. Low pass filters441-40, and computation blocks461-40compute vectors A and F as described above, except that the vectors have a length of forty elements instead of twenty. DCTs50and54, and delta blocks52and56of feature enhancement component12b′each accept the forty element input vectors and output forty element vectors to statistical processing block14. It is understood that the arrangement illustrated inFIG. 4is not applicable if the band pass filters301-41arranged according to a non-linear frequency scale such as a log scale or a mel-scale.

The present invention includes feature extraction stages which may include any number of band pass filters30, depending upon the intended voice processing application, and corresponding numbers of conjugate blocks40, multipliers42, low pass filters44and blocks46to provide output signals A and F for each low pass filter. In addition, signals A and F may be combined in a weighted fashion or only part of the signals may be used. For example, it may be advantageous to use only the amplitude signals in one frequency domain, and a combination of the amplitude and frequency signals in another.

While preferred illustrative embodiments of the invention are described above, it will be apparent to one skilled in the art that various changes and modifications may be made therein without therein without departing from the invention, and it is intended in the appended claims to cover all such changes and modifications which fall within the true spirit and scope of the invention.