Abstract:
The restoration of melody perception is a key remaining challenge in cochlear implants. A novel sound coding strategy is proposed that converts an input audio signal into time-varying electrically stimulating pulse trains. A sound is first split into several frequency sub-bands with a fixed filter bank or a dynamic filter bank tracking harmonics in sounds. Each sub-band signal is coherently downward shifted to a low-frequency base band. These resulting coherent envelope signals have Hermitian symmetric frequency spectrums and are thus real-valued. A peak detector or high-rate sampler of half-wave rectified coherent envelope signals in each sub-band further converts the coherent envelopes into rate-varying, interleaved pulse trains. Acoustic simulations of cochlear implants using this new technique with normal hearing listeners, showed significant improvement in melody recognition over the most common conventional stimulation approach used in cochlear implants.

Description:
RELATED APPLICATIONS 
       [0001]    This application is based on a prior copending provisional application Ser. No. 61/057,994, filed on Jun. 2, 2008, the benefit of the filing date of which is hereby claimed under 35 U.S.C. § 119(e). 
     
    
     GOVERNMENT RIGHTS 
       [0002]    This invention was made with government support under grant numbers FA 9550-06-1-0191 and FA 9550-09-1-0060 awarded by the Air Force Office of Scientific Research, and grant numbers R01 DC007525 and P30 DC 004661 awarded by the National Institutes of Health. The government has certain rights in the invention. 
     
    
     BACKGROUND 
       [0003]    More than 100,000 patients worldwide with profound hearing loss have received cochlear implants as a clinical treatment to regain partial hearing. In current cochlear implants, most speech coding strategies extract and deliver a small number of temporal envelope cues via pulsatile electrical stimulation. 
         [0004]    Unfortunately, cochlear implants are limited in that the patient can only perceive relatively low frequency signals induced by the cochlear implant. Natural speech and music include both relatively high frequency and relatively low frequency components, and existing cochlear implant signal processing techniques do not extract useful information from the relatively high frequency portions of acoustical inputs. As a result, cochlear implants have relatively poor performance in noisy environments and in regard to the perception of music. 
         [0005]    In the widely used continuous interleaved sampling (CIS) coding scheme, sounds are split into a few sub-bands, and the slowly varying envelopes are extracted with a half-wave or full-wave rectifier followed by a low-pass filter in each sub-band. This technique provides a signal that can be used to successfully control cochlear implants to enable users to perceive speech in relatively quiet environments. However, the CIS encoding scheme does not extract much useful information from the relatively high frequency portions of acoustical inputs. Other prior art signal processing techniques for cochlear implants have calculated envelopes from the magnitude of the Fast Fourier Transform (FFT) or the Hilbert transform. Again, such techniques do not extract much useful information from the relatively high frequency portions of acoustical inputs. 
         [0006]    This issue can be better understood by examining the following sum-of-product model for any given sound signal x(t), as shown in Eq. (1): 
         [0000]    
       
         
           
             
               
                 
                   
                     x 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           k 
                           = 
                           1 
                         
                         N 
                       
                        
                       
                         
                           x 
                           k 
                         
                          
                         
                           ( 
                           t 
                           ) 
                         
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           k 
                           = 
                           1 
                         
                         N 
                       
                        
                       
                         
                           
                             a 
                             k 
                           
                            
                           
                             ( 
                             t 
                             ) 
                           
                         
                         · 
                         
                           
                             c 
                             k 
                           
                            
                           
                             ( 
                             k 
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where k is a sub-band index, x k (t) is the output for each of N sub-bands, a k (t) is a slowly varying envelope, and c k (t) is a higher-frequency carrier. Some type of detection rule is used to determine the product decomposition of each sub-band output (x k (t)=a k (t)·c k (t)) into slowly varying amplitude and higher frequency carrier signals, respectively. 
         [0007]    The envelope signal a k (t) can be derived from the amplitude of Fourier transform, or by incoherent demodulations, e.g., half-wave rectification, full-wave rectification, and the Hilbert transform. In current cochlear implants, only the positive envelope signal a k (t) is coded in each sub-band, resulting in significant loss of information contained in the carrier signal or temporal fine structure c k (t). 
         [0008]    For example, a detection rule used in existing cochlear implants decomposes each sub-band signal x k (t) into a Hilbert envelope and an associated carrier. This approach begins with the determination of the analytic signal as shown in Eq. (2): 
         [0000]        {tilde over (x)}   k ( t )= x   k ( t )+ jH{x   k ( t )}  (2) 
         [0000]    where H {x k (t)} is the Hilbert transform of x k (t). The amplitude portion of the signal is non-negative and the real magnitude of the analytic signal is as shown in Eq. (3): 
         [0000]        a   k ( t )=| {tilde over (x)}   k ( t )|.  (3) 
         [0009]    The result from Eq. (3) is commonly referred to as the “Hilbert envelope.” The carrier portion of the signal is the remaining uni-modular phase of the analytic signal, as shown in Eq. (4): 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       c 
                       k 
                     
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       cos 
                        
                       
                         { 
                         
                           
                             tan 
                             
                               - 
                               1 
                             
                           
                            
                           
                             
                               Im 
                                
                               
                                   
                               
                                
                               
                                 
                                   
                                     x 
                                     ~ 
                                   
                                   k 
                                 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                             
                             
                               Re 
                                
                               
                                   
                               
                                
                               
                                 
                                   
                                     x 
                                     ~ 
                                   
                                   k 
                                 
                                  
                                 
                                   ( 
                                   t 
                                   ) 
                                 
                               
                             
                           
                         
                         } 
                       
                     
                     = 
                     
                       cos 
                        
                       
                           
                       
                        
                       
                         
                           
                             ϕ 
                             k 
                           
                            
                           
                             ( 
                             t 
                             ) 
                           
                         
                         . 
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0010]    Thus, in current cochlear implants, only the non-negative and real envelope a k (t) is delivered to the selected stimulating electrode at a fixed stimulation rate. The conventional envelope extraction process eliminates the temporal fine structure cues (cos φ k (t)) in each sub-band, yielding a coarse spectral and temporal representation of speech and music sounds. Psychoacoustic experiments have shown that, with a limited number of envelopes, most patients are still able to understand speech relatively well and they can even converse over the phone. However, among the majority of cochlear implant users, the lack of temporal fine structure has led to poor speech recognition in noisy environments, near-chance level of melody recognition, poor Mandarin tone recognition and production, and inability to use ITD (Inter-aural Timing Difference) cues to localize sounds. 
         [0011]    The encoding of temporal fine structure in cochlear implants is ultimately restricted by the ability of temporal pitch perception in electrical stimulation. Studies have shown that cochlear implant patients can only perceive stimulated rate variations up to about 1000 Hz. However, the frequency content of the temporal fine structure (cos φ k (t)) in speech and music can be up to 10,000 Hz at higher spectral sub-bands and it is not a band-limited signal. 
         [0012]    It would therefore be desirable to provide an acoustical signal processing technique that extracts useful information from the frequency content of the temporal fine structure, to provide enhanced implant performance to users of cochlear implants. 
       SUMMARY 
       [0013]    This application specifically incorporates by reference the disclosures and drawings of each patent application and issued patent identified above as a related application. 
         [0014]    As noted in the discussion above, cochlear implant patients can only perceive stimulated rate variations up to about 1000 Hz, while the frequency content of the temporal fine structure (cos φ k (t)) in speech and music can be up to 10,000 Hz at higher spectral sub-bands. In broad terms, the concepts disclosed herein can be used to extract useful information from relatively high frequency portions of an acoustic input (i.e., the temporal fine structure), and to convert that information into a relatively low frequency, slowly varying signal that is compatible with cochlear implants, to provide enhanced implant performance for users of cochlear implants. In a particularly exemplary, but not limiting embodiment, the enhanced signal processing disclosed herein can be used with existing cochlear implant hardware. 
         [0015]    In more detailed terms, one aspect of the concepts disclosed herein uses a single sideband demodulation approach to coherently shift a sub-band signal to its base band, generating a low-frequency, real coherent envelope signal. Such a signal will encode both temporal envelope and fine structure cues in a slowly varying manner that is compatible with the frequency limitations of cochlear implant technology. This sideband demodulation makes it feasible to deliver perceivable temporal cues originating in the temporal fine structure of higher frequency portions of the original audio spectrum. 
         [0016]    One aspect of the concepts disclosed herein is directed to a method for controlling a cochlear implant for a hearing impaired patient. An exemplary method includes the steps of providing an auditory input signal, determining a pitch of the auditory input signal over time, and separating the auditory input signal into a plurality of harmonics over time as a function of the pitch over time. For each of the plurality of harmonics, the frequency of the harmonic is shifted downward, generating a plurality of frequency shifted harmonics. For each of the frequency shifted harmonics, an amplitude modulation is performed, generating a plurality of frequency shifted and amplitude modified harmonics, and each frequency shifted and amplitude modified harmonic is mapped to at least one of a plurality of cochlear implant stimulation electrodes. 
         [0017]    Another aspect of the concepts disclosed herein is directed to the use of coherent envelope separation in signal processing for cochlear implants. Prior art signal processing for cochlear implants (such as the CIS encoding and Hilbert envelope encoding discussed above) employ non-coherent envelope separation. A novel coherent envelope separation technique for cochlear implants using a time-varying carrier estimate chosen as a center of sub-band gravity is disclosed herein. The resulting coherent envelope is a complex quantity, which is not suitable to be coded in electrical stimulation. Additional processing is required to enable the extracted information from the temporal fine detail to be included in a relatively low frequency, slowly varying signal that is compatible with cochlear implants. To achieve such a relatively low frequency, slowly varying signal, it is assumed that the carrier w k (t) is fixed at the lower boundary of each sub-band. Such a coherent envelope extraction approach has minimal information loss, and the corresponding coherent envelope is a real-valued signal as a result of direct spectral shifting. 
         [0018]    This Summary has been provided to introduce a few concepts in a simplified form that are further described in detail below in the Description. However, this Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. 
     
    
     
       DRAWINGS 
         [0019]    Various aspects and attendant advantages of one or more exemplary embodiments and modifications thereto will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein: 
           [0020]      FIG. 1  is a flow chart of exemplary high level method steps for using coherent demodulation in an enhanced signal processing paradigm for use with cochlear implants, which provides enhanced cochlear implant performance by extracting useful information from relatively high frequency portions of an acoustic input (i.e., the temporal fine structure) and including that information in a relatively low frequency, slowly varying signal compatible with cochlear implants; 
           [0021]      FIG. 2A  is a flow chart of exemplary method steps in a first exemplary technique disclosed herein for using coherent demodulation in an enhanced signal processing paradigm for use with cochlear implants, which employs bandpass filtering; 
           [0022]      FIG. 2B  is a flow chart of exemplary method steps of a second technique, for using coherent demodulation in an enhanced signal processing paradigm that employs harmonic filtering and is usable with cochlear implants; 
           [0023]      FIG. 3  schematically illustrates coherent fixed carrier demodulation; 
           [0024]      FIG. 4A  is a functional block diagram of the CIS (Prior Art) cochlear implant signal processing technique; 
           [0025]      FIG. 4B  is a functional block diagram of an exemplary single-sideband encoder implementing the first exemplary technique of  FIG. 2A ; 
           [0026]      FIG. 5A  graphically illustrates exemplary waveforms of sub-band signals of a short segment of music sound; 
           [0027]      FIG. 5B  graphically illustrates exemplary single sideband demodulated real envelopes obtained using the processing technique of block  22  in  FIG. 2A ; 
           [0028]      FIG. 6  graphically illustrates two exemplary notes of a melody, the Hilbert transform of the two exemplary notes as implemented by prior art signal processing techniques for cochlear implants, and demodulated real envelopes obtained using the processing technique of block  22  in  FIG. 2A ; 
           [0029]      FIGS. 7A and 7C  illustrate exemplary pulse train patterns produced by processing two music notes using the CIS (Prior Art) cochlear implant signal processing strategy; 
           [0030]      FIGS. 7B and 7D  illustrate exemplary pulse train patterns produced by the processing two music notes using the novel first exemplary coherent demodulation technique of  FIG. 2A ; 
           [0031]      FIG. 8  graphically illustrates an exemplary impulse response of a Butterworth low-pass filter; 
           [0032]      FIG. 9  graphically illustrates a comparison of melody recognition with acoustic simulations of the CIS (Prior Art) cochlear implant signal processing strategy and the novel first exemplary coherent demodulation technique of  FIG. 2A ; 
           [0033]      FIG. 10  graphically illustrates ratings from cochlear implant patients whose cochlear implants were tested using both the CIS (Prior Art) cochlear implant signal processing strategy and the novel first exemplary coherent demodulation technique of  FIG. 2A ; 
           [0034]      FIG. 11  is a functional block diagram schematically illustrating an exemplary technique for extracting the harmonic components of an audio signal with a series of harmonic separators, to implement the steps corresponding to blocks  30  and  32  in  FIG. 2B ; 
           [0035]      FIG. 12  is a functional block diagram providing exemplary details for implementing the steps corresponding to block  34  in  FIG. 2B ; 
           [0036]      FIG. 13  is a functional block diagram for a first exemplary technique to implement the steps corresponding to block  36  in the novel second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B ; and 
           [0037]      FIG. 14  is a functional block diagram for a second exemplary technique to implement the steps corresponding to block  36  in the novel second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B . 
       
    
    
     DESCRIPTION 
     Figures and Disclosed Embodiments are Not Limiting 
       [0038]    Exemplary embodiments are illustrated in referenced Figures of the drawings. It is intended that the embodiments and Figures disclosed herein are to be considered illustrative rather than restrictive. No limitation on the scope of the technology and of the claims that follow is to be imputed to the examples shown in the drawings and discussed herein. 
       Coherent Versus Incoherent Demodulation 
       [0039]    Incoherent demodulation utilizes an energetic envelope detector with no phase reference (i.e., no direct knowledge of the carrier). Coherent demodulation explicitly detects well-behaved carriers for modulation filtering, and is better able to maintain signal elements present in the original signal. Significantly, coherent demodulation is more computationally intensive than incoherent demodulation. Because incoherent demodulation is more computationally efficient, and thus, easier to implement for real time signal processing applications, prior art signal processing techniques for cochlear implants have utilized incoherent demodulation. 
         [0040]      FIG. 1  is a flow chart of exemplary high level method steps for using coherent demodulation in an enhanced signal processing paradigm for use with cochlear implants, which provides enhanced cochlear implant performance by extracting useful information from relatively high frequency portions of an acoustic input (i.e., the temporal fine structure) and including that information in a relatively low frequency, slowly varying signal that is compatible with cochlear implants. 
         [0041]    In a block  10 , the audio input is filtered. As discussed in detail below, different exemplary embodiments employ different filtering strategies. In a block  12 , coherent demodulation filtering is employed to extract useful information from relatively high frequency portions of the audio input to achieve a frequency shifted signal. In a block  14 , the frequency shifted signal is processed to include that useful information in a relatively low frequency, slowly varying signal that is compatible with cochlear implants. In a block  16 , the slowly varying signal is mapped to electrodes in the cochlear implant. 
         [0042]      FIG. 2A  is a flow chart showing exemplary method steps that are employed in a first exemplary technique disclosed herein, for using coherent demodulation in an enhanced signal processing paradigm employing bandpass filtering, which is suitable for use with cochlear implants. 
         [0043]      FIG. 2B  is a flow chart of illustrating exemplary method steps that are used in a second exemplary technique disclosed herein. This second technique employs coherent demodulation in an enhanced signal processing paradigm, which employs harmonic filtering, for use with cochlear implants. 
         [0044]    Referring to  FIG. 2A , in a block  20 , the audio input is filtered using bandpass filtering. In a block  22 , coherent demodulation filtering is employed to extracting useful information from relatively high frequency portions of an acoustic input to achieve a frequency shifted signal. In a block  24  the frequency shifted signal is processed, using peak detection or half-wave rectification modulated with a high-rate pulse train, to include that information in a relatively low frequency, slowly varying signal compatible with cochlear implants. In a block  26  the slowly varying signal is mapped to electrodes in the cochlear implant. 
         [0045]    Referring to  FIG. 2B , in a block  30  the pitch of the audio input is determined. In a block  32  the audio input is separated into a plurality of harmonics as a function of pitch. The number of resulting harmonics should be at least two, although more harmonics can employed. In general, the use of additional harmonics is likely to result in greater performance, although it should be recognized that a specific number of harmonics may work better (or worse) with a specific type of cochlear implant. In a block  34 , coherent demodulation is used to shift the frequency of each harmonic downward, to extract useful information from relatively high frequency portions of the acoustic input, producing a frequency shifted signal. Those of ordinary skill in the art will recognize that the input signal likely includes relatively lower frequency portions not relevant to human perception of speech signals or music. Thus, in at least one exemplary embodiment, in addition to separating the input signal into a plurality of harmonics in block  32 , the signal processing technique can also employ bandpass filtering to remove such low-frequency content. In other words, in such an embodiment, in block  32  the input signal is separated into portions to be downshifted, and portions to be discarded. However, in at least one further embodiment, the entire input signal is separated into a plurality of harmonics without bandpass filtering to remove such low-frequency content, such that the entire input signal is spectrally shifted downwards. 
         [0046]    In a block  36 , for each frequency shifted harmonic, amplitude modulation of the pulse train is performed, to include the information from the relatively high frequency portions of the input signal in a relatively low frequency, slowly varying signal compatible with cochlear implants. In a block  38 , the slowly varying signal is mapped to electrodes in the cochlear implant. 
         [0047]      FIGS. 3-10  relate to the first exemplary cochlear implant signal processing technique of  FIG. 2A . Empirical studies based on the first exemplary technique led to modifications to provide even further performance enhancements, resulting in the second exemplary cochlear implant signal processing technique of  FIG. 2B .  FIGS. 11-14  relate to the second exemplary cochlear implant signal processing technique of  FIG. 2B . 
       First Exemplary Embodiment Using Fixed-Carrier Coherent Demodulation 
       [0048]    Referring to block  22  of  FIG. 2A , a fixed carrier was utilized to coherently demodulate each sub-band signal as shown in Eq. (5): 
         [0000]    
       
         
           
             
               
                 
                   
                     x 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           k 
                           = 
                           1 
                         
                         N 
                       
                        
                       
                         
                           x 
                           k 
                         
                          
                         
                           ( 
                           t 
                           ) 
                         
                       
                     
                     = 
                     
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           N 
                         
                          
                         
                           
                             
                               a 
                               k 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                           · 
                           
                             
                               c 
                               k 
                             
                              
                             
                               ( 
                               t 
                               ) 
                             
                           
                         
                       
                       = 
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             1 
                           
                           N 
                         
                          
                         
                           
                             
                               
                                 a 
                                 k 
                               
                                
                               
                                 ( 
                                 t 
                                 ) 
                               
                             
                             · 
                             cos 
                           
                            
                           
                               
                           
                            
                           
                             ω 
                             k 
                           
                            
                           t 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where ω k  is a fixed carrier frequency at the lower edge of each sub-band. The envelope a k (t) is now a real (positive- and negative-going) signal, yet intentionally is not a positive-only Hilbert envelope. This fixed-carrier demodulation is similar to that used in early single sideband receivers; however, the post processing required to generate a usable signal for cochlear implants is significantly different. 
         [0049]      FIG. 3  is a functional block diagram illustrating how the fixed carrier demodulation is performed in the first exemplary embodiment of  FIG. 2A , wherein “Conj.” indicates “conjugate” and “H {.}” represents the Hilbert transform. It is assumed that the fixed carrier resides on the lower edge of the sub-band. Each sub-band was considered to be an upper sideband signal generated from an envelope signal by single sideband modulation. To perform coherent demodulation, an analytic signal was formed for each sub-band by the Hilbert transform. The analytic signal has a one-sided spectrum, and it was then multiplied with a complex carrier at the lower edge of that sub-band, producing a spectrum replica at the base band. The demodulated complex signal was conjugated and summed to construct a real-valued signal. 
         [0050]    Note Eqs. 6 and 7 below for each sub-band: 
         [0000]        g   k ( t )={ x   k ( t )+ jH[x   k ( t )]}· e   −jω     k     t   (6) 
         [0000]        a   k ( t )= g   k ( t )+ g   k *( t )  (7) 
         [0000]    where the symbol, *, signifies the complex conjugate. 
         [0051]    Alternatively, the demodulation can be performed with a product detector, which mixes a sub-band signal with a carrier and then low-passes the mixture. 
         [0052]    The coherent envelope signal has a maximum frequency equivalent to its bandwidth. Normally, when a sufficient number of sub-bands is used, the maximum signal bandwidth can be lower than 1000 Hz. Such low bandwidth signals would be within the perceivable range of rate pitch elicited by electrical stimulation in a cochlear implant. 
       First Exemplary Embodiment of a Coherent Envelope (CE) Encoder 
       [0053]      FIG. 4A  is a functional block diagram of the CIS (Prior Art) cochlear implant signal processing technique, while  FIG. 4B  is an exemplary block diagram showing how the exemplary method steps of  FIG. 2A  are implemented using single-sideband encoding, wherein BPF is an acronym for band-pass filter and CE is an acronym for a coherent envelope demodulator. 
         [0054]    A sound signal is first filtered into N bands with equal bandwidth on a logarithmic scale. For instance, exemplary cutoff frequencies are: 300, 462, 687, 996, 1423, 2013, 2827, 3950 and 5500 Hz, when the number of bands is eight. Each band-passed signal is coherently demodulated by single sideband demodulation, as discussed above.  FIG. 5A  graphically illustrates the waveforms of sub-band signals of an exemplary short segment of music, in response to a 22.7 ms-long melody signal.  FIG. 5B  graphically illustrates the exemplary music signal after processing using coherent envelopes demodulated with fixed carriers. The bottom trace on each panel corresponds to the lowest frequency sub-band. The coherent envelopes convey both temporal envelope and temporal fine structure cues embedded in each sub-band. It is possible to deliver these analog signals directly to current cochlear implants. One of the advantages of using coherent envelopes is that the demodulated analog signals would significantly reduce channel interactions due to the nature of low rates. 
         [0055]    To reduce channel interactions, analog signals should be transformed into interleaved pulse trains in cochlear implants, meaning different stimulating electrodes are sequentially activated within one stimulation cycle. 
         [0056]      FIG. 6  graphically illustrates how the coherent demodulation of the input signal disclosed herein differs from the Hilbert transform employed in prior art cochlear implant signal processing techniques. Portion A of  FIG. 6  corresponds to two melody notes before processing. Portion B of  FIG. 6  corresponds to the Hilbert envelope for the two melody notes, while Portion C of  FIG. 6  corresponds to the real coherent envelope of the two melody notes. 
         [0057]    For pulsatile stimulation, each analog waveform should be converted into a pulse train. To perform this conversion, pulses are generated in synchrony with the positive peaks in the analog waveforms. The inter-pulse interval then carries zero-crossing cues or phase information. The pulse heights are equal to the amplitudes at the peaks. It has been found that auditory neurons phase lock to temporal peaks only up to 4-5 kHz. Here, a similar mechanism is used for generating pulses. 
         [0058]    Finally, these pulses might overlap each other in time. An interleaved pulse train generator is used to detect the overlapping pulses and force them to be interleaved. Within one stimulating cycle, all bands are sequentially scanned to locate a peak. If a peak is found, a flag is raised, indicating that the corresponding electrode should be activated subsequently. A biphasic pulse will be generated to stimulate that electrode and the flag is thereafter cleared. This pulse selection procedure ensures only one pulse at a time is applied during a defined period. Alternatively, stimulating pulses can be generated by sampling the half-wave rectified real coherent envelope with a high-rate pulse train (see  FIG. 13  and the related text). 
         [0059]    The exemplary processing technique of  FIG. 2A  (referred to herein as the first CE strategy) has been implemented using a research interface provided by the Cochlear Corporation. This system provides the needed flexibility to generate arbitrary interleaved electrical pulse stimulation patterns, as is required for patients with an implanted cochlear stimulator. 
         [0060]      FIGS. 7A and 7C  graphically illustrate a pulse train pattern created with the CIS (Prior Art) cochlear implant signal processing technique, and  FIGS. 7B and 7D  graphically illustrate a pulse train pattern created with the first CE cochlear implant signal processing technique strategy (i.e., the first exemplary embodiment shown in  FIG. 2A ). All parameters for  FIGS. 7A and 7B  are identical except that the stimulation rate of the CIS strategy was set at the typically constant  800  pulses per channel, while the parameter for  FIGS. 7C and 7D  are also the same. The pulse pattern of the first CE strategy intentionally contains detailed temporal cues via time-variable yet independent stimulation rates on each electrode. Electrode  22  of  FIGS. 7A and 7B  corresponds to a low frequency sub-band. 
       First Exemplary Embodiment Acoustic Simulations 
       [0061]    As a preliminary test of the relative efficacy of the conventional CIS and first proposed CE strategies, acoustic sounds were reconstructed from the above pulse trains. Pulse trains were convolved with the impulse response of a 2nd-order Butterworth low-pass filter at 300 Hz, as shown in  FIG. 8 , mimicking the deteriorating temporal pitch discrimination ability in electrically-induced hearing. Each filtered signal was modulated by a carrier identical to the demodulation carrier. All signals were summed together to re-synthesize an acoustic sound. 
         [0062]    Melody recognition was performed to assess whether the first CE coding strategy could improve music recognition performance. Subjects were asked to identify a closed set of twelve common melodies, e.g. “Happy Birthday,” “Frere Jacques,” “Jingle Bells,” etc. Rhythmic cues were removed, and all 12 melodies were isochronous. During these listening experiments, the subject was allowed to practice twice prior to the test. Each melody was presented twice, and no feedback was given. The stimuli and user interface are part of a battery of tests for assessing cochlear implant users&#39; melody recognition performance. 
       First Exemplary Embodiment Results 
       [0063]    The melody recognition scores from four (4) normal hearing subjects are presented in  FIG. 9 , which graphically illustrates melody recognition with acoustic simulations of the CIS and SSE strategies (error bars indicate standard errors). The mean score of the SSE is 92%, whereas the mean for CIS is only 57%. A paired-t test suggests that the recognition performance of the SSE strategy is significantly different than that of the CIS strategy. 
         [0064]    In a follow up study, the hearing of patients using the Nucleus™ cochlear implant was tested after modifying the cochlear implant to process audio input using the first exemplary cochlear implant signal processing embodiment shown in  FIG. 2A . This study indicated improved implant performance using the first exemplary cochlear implant signal processing embodiment; however, the desired melody recognition was not achieved. Additional research lead to the development of the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B . The results of the follow-up study are graphically illustrated in  FIG. 10 . The rating scores are for sound quality, sound clarity and music perception (1 for the worst, 10 for the best). The test subjects indicated that the CIS coding provided a monotone, with input often being unrecognizable. The test subjects clearly preferred the first exemplary cochlear implant signal processing embodiment shown in  FIG. 2A , indicating that they perceived multi-pitch, distinguishable, and pleasant sounds. They did report perceiving some mistuning. 
       First Exemplary Embodiment Discussion and Conclusions 
       [0065]    The SSE strategy presents a faithful representation of a high-frequency signal at the lower rates required for successful cochlear implant electrical stimulation. Higher rates saturate the patient&#39;s perception. This signal processing technique can be easily implemented in real-time. Most importantly, it appears that the coherent envelopes provide usable temporal cues to implant users. Since lower-frequency channels normally have sparse pulse trains, implementing interleaved pulse train stimulation is also feasible. The analog version of the SSE strategy provides low-rate analog stimulation comparable to other cochlear implant strategies, with less simultaneous channel interaction. 
         [0066]    The fixed-carrier demodulation approach also provides a potentially useful tool for extracting slowly varying features from speech and music. 
       Second Exemplary Embodiment Using Coherent Demodulation 
       [0067]      FIGS. 11-14  and the related descriptive text provide additional details about the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B . 
         [0068]      FIG. 11  is a functional block diagram schematically illustrating an exemplary technique for extracting the harmonic components of an audio signal with a series of harmonic separators, to implement the steps corresponding to blocks  30  and  32  in the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B  (in which harmonics are extracted from the audio input signal). Referring to  FIG. 2B , in block  30 , the pitch of the audio input is determined, while in block  32 , the audio input is separated into a plurality of harmonics, as a function of pitch. 
         [0069]    Referring to  FIG. 11 , a pre-emphasis filter  50  is first applied to the input audio signal for energy equalization. A pitch tracker  52  is then used to estimate a time-varying fundamental frequency, F 0 (t). Based on F 0 (t), a harmonic locator  54  splits the audio single into N different harmonics, each different harmonic being output to a harmonic separator (see blocks  56   a ,  56   b , . . .  56   n ). Each harmonic separator is implemented as a bandpass filter specifically customized for that particular harmonic. For example, harmonic separator  56   k  is implemented as a bandpass filter centered around K th  multiples of F 0 (t), and its bandwidth is equal to F 0 (t). The outputs are a number of selected harmonics, based on the pitch tracker. 
         [0070]    It should be noted that the frequency shifting for block  34  (in the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B ) can be implemented in at least two different ways (both of which are discussed in detail below). In broad terms, one has the choice of frequency shifting analytic subbands or real-valued subbands. To frequency shift analytic subbands, one first converts a real-valued signal to an analytic signal using the Hilbert transform, and then multiplies the result with a complex carrier. To frequency shift a real-valued signal, one first multiplies the real-valued signal directly with a real-valued carrier, and then performs low-pass filtering. The latter technique does require additional low-pass filtering. Both broad techniques produce the same signal. Note that frequency shifting the analytic subband requires just one filter operation, whereas shifting the real-valued subband requires at least two. 
         [0071]      FIG. 12  is a functional block diagram providing exemplary details for implementing the steps corresponding to block  34  in the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B , in which each harmonic is processed such that relatively higher frequency portions of the audio input are frequency shifted downward by manipulating an analytic signal. The information from those relatively higher frequency portions of the audio input are incorporated in the signal generated for controlling the cochlear implant. Referring to  FIG. 2B , in a block  34 , coherent demodulation is used to shift the frequency of each harmonic downward, to extract useful information from relatively high frequency portions of the acoustic input to achieve a frequency shifted signal. 
         [0072]    Referring to  FIG. 12 , the step of block  34  is implemented using two-sided frequency shifting on selected harmonics output from  FIG. 11 . For each channel, i.e., each harmonic, the Hilbert transform (see blocks  58   a ,  58   b ,  58   n ) is first applied to obtain a one-sided analytic signal. Note that while the Hilbert transform is employed in prior art cochlear implant signal processing techniques, in the present novel technique, the Hilbert transform is applied to a harmonically separated audio input, whereas in prior art cochlear implant signal processing techniques, the audio input is not separated harmonically, but simply filtered using a bandpass filter. Next, each one-sided analytic harmonic signal is multiplied by a complex exponential signal (as indicated by operators  60   a ,  60   b , . . .  60   n ). The phase term of the complex exponential signal is determined by the fundamental frequency F 0 (t), and the targeted temporal pitch trajectory {circumflex over (F)} 0 (t). Specifically, {circumflex over (F)} 0 (t) is obtained by linearly (or nonlinearly) mapping F 0 (t), 2F 0 (t), and NF 0 (t) to the frequency range of about 50˜300 Hz (i.e., the perceivable temporal pitch range of cochlear implant users). {circumflex over (F)} 0 (t) can be smaller than or equal to F 0 (t). The phase term φ k (t) of the K th  complex exponential function is determined by Eq. (8) as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                        
                       
                         
                           ϕ 
                           k 
                         
                          
                         
                           ( 
                           t 
                           ) 
                         
                       
                     
                     
                        
                       t 
                     
                   
                   = 
                   
                     g 
                      
                     
                       ( 
                       
                         
                           
                             kF 
                             0 
                           
                            
                           
                             ( 
                             t 
                             ) 
                           
                         
                         , 
                         
                           
                             
                               F 
                               ^ 
                             
                             0 
                           
                            
                           
                             ( 
                             t 
                             ) 
                           
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where g(kF 0 (t), {circumflex over (F)} 0 (t)) is a linear function. 
         [0073]    After multiplying each one-sided analytic harmonic signal with the complex exponential signal, each result is frequency shifted to {circumflex over (F)} 0 (t) to maintain the harmonic relationship among all channels. Finally, the real part of the complex signal for each harmonic is extracted and output (as indicated by blocks  62   a ,  62   b , . . .  62   n ). 
         [0074]    Alternatively, the above two-sided frequency shifting operation can be implemented without using the Hilbert transform. In such an embodiment (frequency shifting a real-valued signal), the processing includes two steps. The first step is to multiply each harmonic with a cosine function cos(φ k (t)), whose phase term φ k (t) is determined as described above. The second step is to apply a low pass filter to remove double-frequency components. 
         [0075]      FIG. 13  is a functional block diagram for a first exemplary technique that can be employed to implement the steps of block  36  in the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B . An amplitude modulation of the pulse train is performed for each frequency shifted harmonic, to include the information from the relatively high frequency portions of the input signal in a relatively low frequency, slowly varying signal that is compatible with cochlear implants. 
         [0076]    Referring to  FIG. 13 , for each channel output from  FIG. 12 , a half-wave rectifier (see blocks  64   a ,  64   b , . . .  64   n ) is used to obtain a non-negative envelope signal that carries both amplitude and periodicity information of the frequency-shifted harmonic. A stimulation pulse train is further generated by modulating the envelope signal with a biphasic pulse train at a constant rate (see operators  66   a ,  6   b , . . .  66   n ). Across all channels, the biphasic pulses are interleaved in time to avoid channel interaction. 
         [0077]    It should be noted that harmonic and inharmonic sounds produce noticeably different pulses, which have not been seen in prior art cochlear implant signal processing strategies. In the case of a harmonic input, each amplitude-modulated pulse train is a strong periodic signal with a temporal pitch below 300 Hz. Significantly, the temporal pitch change is within the perceivable range of most cochlear implant users. The periodic signal encodes information about the fundamental frequency (F 0 ) of the harmonic input, which is critical for music perception, voice gender identification, tonal language understanding and speech recognition in noisy environments. In the case of an inharmonic or noise-like input, each pulse train exhibits the properties of an aperiodic signal, which is crucial to consonant recognition and musical instrument identification. 
         [0078]      FIG. 14  is a functional block diagram for a second exemplary technique to implement the steps of block  36  in the second exemplary cochlear implant signal processing embodiment shown in  FIG. 2B . In this second exemplary technique, a pulse train can be generated with a peak detector, which produces pulses only at the local maxima of the half-wave rectified envelope signal. Pulse sequences are also forced to be interleaved to avoid channel interaction. In this second exemplary technique, the produced pulse trains have variable stimulation rates, rather than a constant rate as discussed with respect to the first exemplary technique of  FIG. 13 . However, the temporal pattern of variation in each pulse rate encodes either a periodic/harmonic input signal with a regular pulse sequence, or encodes an aperiodic/inharmonic signal with a random/irregular pulse train. This type of pulse train stimulation has been proven effective for cochlear implant patients as well. Note the dashed box indicating that the high-rate pulse trains are interleaved in time. 
         [0079]    Referring to  FIG. 14 , for each channel output from  FIG. 12 , a half-wave rectifier (see blocks  68   a ,  68   b , . . .  68   n ) is used to obtain a non-negative envelope signal that carries both amplitude and periodicity information of the frequency-shifted harmonic. In this embodiment, the pulse train is generated using peak detection (see blocks  70   a ,  70   b , . . .  70   n ), the output of which is interleaved (see block  72 ). 
         [0080]    Referring once again to  FIG. 2B , once the pulse train has been produced in block  36 , the slowly varying signal is mapped to electrodes in the cochlear implant in block  38 . An exemplary mapping technique is described below. 
       Electrode Assignment 
       [0081]    A limited number (N) of harmonics are dynamically tracked according to the F 0  contour of the input sound. These harmonics span a wide range of frequency in the spectral domain. However, the actual number of electrodes (M) that can be stimulated varies in a typical patient map. Each electrode corresponds to a specific range of frequency in relation to its tonotopic placement in the cochlea. Typically, the basal electrodes have wider bandwidths centered around higher frequencies, whereas the apical electrodes have narrow bandwidths centered around lower frequencies. 
         [0082]    In order to assign N harmonics to M electrodes, the frequency location of the i th  harmonic is correlated with the specified frequency range of each individual electrode. The frequency of the i th  harmonic is first calculated, and it is then compared with the frequency mapping of a cochlear implant patient. If the i th  harmonic falls in the spectral coverage of one electrode, the i th  amplitude-modulated pulse train is assigned to this corresponding electrode. When two or more harmonics compete for the same electrode, those pulse trains are summed together to produce a single stimulation signal. 
         [0083]    Although the concepts disclosed herein have been described in connection with the preferred form of practicing them and modifications thereto, those of ordinary skill in the art will understand that many other modifications can be made thereto within the scope of the claims that follow. Accordingly, it is not intended that the scope of these concepts in any way be limited by the above description, but instead be determined entirely by reference to the claims that follow.