A COMPUTATIONAL APPROACH TO RECOGNITION OF SPEECH FEATURES USING MODELS OF AUDITORY SIGNAL PROCESSING

We present a computational approach to the detection of important speech features, such as fonnants and pitch, based on a model of auditory signal processing. Algorithms have been designed to be computationally simple, physiologically reasonable and to emulate human psychophysical performance. INTRODUCTION Most current approaches to computer speech recognition are based on a spectrographic approach to feature extraction[1]. In this approach, the energy of speech is measured as a function of frequency, and parameters derived from the resulting spectrum are compared to a template or rule. Spectrographic techniques include computation of HTS. extraction of LPC and cepstral coefficients and processing by filter banks. » Spectrographic approaches suffer from well—known problems. Because spectrograms are sensitive to anything that changes the relative magnitude of in-band energies, their performance is often severely degraded in situations of practical interest; for example, in conditions of reduced spectral bandwidth (over the phone) or in the presence of background or line noise. In our approach, we have sought to understand the fundamental strategy used by the auditory system to process speech Signals and apply this understanding to the desrgn of improved algorithms for detection of speech features. We have: . 0 developed a comprehensive model of Signal processing by the peripheral and eariy central auditory system, studied the response of this model to speech and other stimuli, and o distilled what we believe are important signal processing techniques of the auditory system into practical algorithms for feature extraction that provide noise-immune, speech-specific detection of forrnants and pitch pulses in sonorant parts of speech. RESULTS A model of auditory signal processing The model of auditory signal processing[2] includes components describing the external and middle car. a detailed three-dimensional hydromechanical model of the cochlea, a biophysical model of mechano-electric transduction by the cochlear hair cells, a description of the time—dependent synaptic chemistry of hair cells and auditory-nerve fibers including models of the hair cell‘s calcium channel and synapse and a ‘microneural-net' description of signal processing in the cochlear nucleus. A comparison of the predictions of this model with experimental physiological data in response to both simple stimuli (i.e. tones) and complex stimuli (i.e. speech) suggests that the model adequately describes essential features of auditory signal processrng. The response of the model to In! Figure 1 shows the response of the auditory model to a voiced utterance. /8/. spoken by a male speaker. The model response to this utterance comprises two distinct spatio-temporal patterns occurring in alternation. We term these patterns the impulsive epoch and the synchrm epoch. The impulsive epoch occurs in response to the glottal pulse. In this epoch. rnostfibersrtespondatar’atelhat ICPhS 95 Stockholm Session 41.3 Vol. 3 Page 51 . l . III at) so so a h("I“) Figure I. A. The response of 120 model fibers with characteristic frequencies (CFs) spanning the range from 250 Hz (bottom trace) to 3.4 kHz (top trace). _Each waveform represents the probability density function ofneural discharge for an audrtory nerve fiber innervating one location along the cochlea which is maximally sensitive to a particular CF. The responses offibers have been time aligned to remove the delay that results from the transit time of sound along the basilar membrane and the delay of neural response. The impulsive (I). and synchronous (S) epochs are marked B. The times at which the nerve fiber ensemble in A is most likely tofire. This plot resultsfrom processing the wavefonns o] A with a threshold—crossing algorithm that places a tick mark at the times at which‘eoch fiber is most likely to fire. The plot gives a stylized description of the pattern of trmrng information that this ensemble offibers delivers to the brain in response to /a[ conesponds with their best or characteristic frequency (CF), giving the pattern of response of the ensemble of fibers a splayed appearance. In the synchronous epoch which follows, several groups of fibers respond distinctly at a rate that comesponds to the frequency of a proximal fomtant. We pcetically term each group of fibers entrained to one formant an “island of synchrony". There appear to be at least three sharply delineated islands of synchrony: fibers with CFs between approximately 500 and 800 Hz are synchronized to F1; fibers with CFs between 1000 and 1400 Hz are synchronized to F2; fibers with CFs above 2000 Hz are synchronized to F3. The alternation of an impulse-like pattern with a synchronous pattern is highly characteristic of the response to voiced speech. These observations suggest that the alternately impulsive and synchronous nature of the model‘s response could be used to locate and track linguistically interesting quantities such as the times of occurrence of pitch pulses and the frequencies of the fomiants. Our approach has been to build separate physiologically motivated “detectors" for the impulse-like first epoch and the synchronous second epoch and then use these detectors to identify fonnants and pitch pulses. The response of the auditory model to