Encoding Temporal Fine Structure for Cochlear Implants
Phase: NA
Conditions: Hearing Loss, Cochlear Implants
Sponsor: University of Southern California

SIGNIFICANCE 
Cochlear implants provide hearing to people with severe to profound hearing loss. Recipients generally achieve 
high levels of speech comprehension with their implants, but music perception and speech comprehension in 
noise is often poor compared to normal hearing. For historical and technological reasons, most cochlear implants 
in use today discard the temporal fine structure of sound and only transmit slowly varying envelopes. Evidence 
indicates that temporal fine structure is important for music perception and speech comprehension in noise1–8. 
Historically, some of the earliest cochlear implants used analog stimulation, which conveyed temporal fine 
structure, but in a manner that caused interference across electrodes that was difficult to control. Consequently, 
interleaved pulsatile stimulation gained prominence since it mitigated effects of electrical interference9. While 
this solution addressed aspects of electrical interference, it came at the expense of discarding temporal fine 
structure. Several attempts have since been made to restore temporal fine structure into cochlear implant 
stimulation while using pulsatile stimulation. The most successful attempt to date is Fine Spectral Processing 
used with MEDEL devices, which uses dynamic pulse timings, phase-locked to the temporal fine structure of 
sound, when stimulating the most apical electrode(s). This strategy has shown promise for improving pitch 
perception for cochlear implant users, particularly when recipients are provided time to learn how to use the new 
information10,11. 
The extent that restoring temporal fine structure into cochlear implant stimulation will improve outcomes for 
cochlear implant users is unknown. Psychophysical and physiological studies indicate that temporal fine 
structure is used in normal hearing for frequencies up to 1500 Hz and perhaps as high as 10 kHz12,13. In contrast, 
studies indicate that cochlear implant users are generally insensitive to stimulation rate, particularly so for rates 
above 300 Hz14,15. In our previous work, we showed that sensitivity to stimulation rate can be improved with 
training16. This is a critical finding. It suggests that the impoverished temporal sensitivity in cochlear implant users 
might be a result of the simple fact that cochlear implants discard temporal fine structure and only weakly encode 
temporal periodicity using amplitude modulation for a limited range of frequencies. If so, then it is quite possible 
that temporal sensitivity can be refined to what is generally observed for normal-hearing listeners, perhaps even 
improved upon because cochlear implant stimulation can be temporally more precise than normally provided by 
healthy hearing. The potential impact is profound. At the very least, with better encoding and auditory 
rehabilitation, voice pitch perception could be enhanced for frequencies above 300 Hz, allowing better perception 
of children’s voices. Beyond, enhancing temporal coding in cochlear implants could broadly improve music 
perception and speech recognition. 
INNOVATION 
This proposal brings together psychophysics, electrophysiology, and neural modeling to examine perceptual 
learning for stimulation cues that are poorly encoded by clinical devices. Studies have shown that sensitivity to 
pitch can be improved with training even in normal hearing listeners17–20. Our work is unique in that we explore 
the effects of training for a pitch cue, stimulation rate, that is not used or is poorly encoded by existing devices. 
This key innovation for examining the perceptual plasticity of underlying psychophysical cues is coupled with a 
comprehensive approach for characterizing mechanisms and models of pitch perception. By considering the 
effects of learning, our approach challenges existing dogma regarding the limits of temporal sensitivity provided 
by cochlear implant stimulation. The uniting theme across aims is the power of plasticity. 
• 
Test the limits of temporal pitch perception when provided as a clear and consistent cue. This 
innovation is at the core of our proposal. The extent that fine timing of stimulation can improve music and 
speech comprehension in noise for cochlear implant users depends on how well they can learn to hear timing 
differences. We provide extensive laboratory training for pitch cues while working with implant manufacturers 
to extend fine timing of stimulation into commercial devices and rehabilitation. 
• 
Develop physiological methods and models to optimize encoding of temporal fine structure. Encoding 
temporal fine structure of sound into fine timing of cochlear implant stimulation is challenging because it 
requires microsecond decisions of pulse timings to be made in a way that transmits the timing information 
effectively to the auditory nerve. We outline the development of physiological methods and models for refining 
how cochlear implant stimulation is transmitted to auditory nerve activity. The models established here will 
guide the development of new stimulation strategies that use fine timing of stimulation to enhance hearing. 
Encoding Temporal Fine Structure for Cochlear Implants (NCT04708717) 1/1/2020 

 
 
1.1 
  Background & Rationale 
Cochlear implants provide a sense of pitch using place and rate of stimulation. Place of stimulation is controlled 
by progressively filtering lower frequencies to more apical electrodes. Presently, most cochlear implants do not 
use stimulation rate to convey information but use amplitude modulation to transmit slowly varying envelopes21. 
The upper modulation frequency that is encoded depends on the device, but most devices only encode 
modulation frequencies up to 300 Hz, well below what has been shown to be possible in psychophysical and 
physiological studies. Further, filtering sound into frequency bands can diminish modulation depth, which tends 
to weaken pitch salience. Thus, most cochlear implants in use today make poor use of stimulation timing and 
only provide envelope fluctuations with limited modulation depth and frequency range. 
In contrast to the impoverished temporal coding provided by clinical cochlear implants, temporal fine structure is 
exceptionally well encoded and apparently utilized in normal hearing. Studies of mammalian physiology have 
shown that the auditory nerve phase locks to acoustic frequencies up to 3 kHz and psychophysical studies of 
normal hearing typically conclude that temporal fine structure can be used for pitch and spatial hearing tasks at 
least up to 1500 Hz, perhaps as high as 10 kHz22. Given the physiological and psychophysical evidence 
indicating the importance of temporal fine structure, it has been discouraging to find that cochlear implant users 
struggle to perceive differences between stimulation rates above 300 Hz. Studies have shown that temporal 
encoding of electrical stimulation into auditory-nerve activity is not limited by physiology23–26. Phase locking of 
the auditory nerve has been reported to be stronger for electrical than for acoustic stimulation12,27,28. Overall, 
there is strong psychophysical and physiological evidence that temporal fine structure can be well encoded into 
synchronous auditory-nerve activity, both by acoustic and electric stimulation. We therefore hypothesize that the 
poor temporal sensitivity of cochlear implant users often reported in the literature is not a fundamental limit but 
one that could be improved if temporal fine structure was provided in a clear and consistent manner. 
In Goldsworthy and Shannon, 201416, we showed that pitch discrimination based on stimulation rate can be 
improved with training. There have been several studies of the plasticity in normal-hearing listeners20,29, but our 
approach is unique in that we study plasticity for a cue that is not typically provided during everyday exposure. 
Because it is not typically provided, we speculate that the time course and potential for improvements with 
training will differ from studies of normal-hearing listeners. 
Figure 1 shows cochlear implant rate discrimination thresholds for 
studies with and without training. The shaded gray region shows 
the range of thresholds reported by Zeng in 2002. For rates less 
than 200 Hz, discrimination thresholds without training were 
between 10 and 25% and deteriorated for rates above 300 Hz. 
These results are representative of studies in which subjects are 
not provided training for the rate cue. In contrast, we found in our 
2014 study that with 32 hours of training, average discrimination 
thresholds were about 3% for rates less than 200 Hz and 
discrimination could be consistently measured as high as 
1760 Hz. This is impressive given that aside from our study, the 
literature contains only a few mentions of star subjects who can 
pitch rank based on stimulation rates up to 900 Hz. The effect of 
training is large (Cohen’s d > 1) and we want to know the extent 
that further training will produce further improvements. 
Participants reported that the task was difficult at first, but with 
practice at higher rates, they could consistently discriminate pitch. 
Given the strong evidence that neural encoding of timing is important for music and speech perception and our 
strong preliminary evidence that temporal sensitivity can be improved if stimulation timing is provided in a clear 
and consistent manner, we propose to study the fundamental limits of pitch perception based on place and rate 
within the context of perceptual learning. Our primary hypothesis is that sensitivity to both place and rate as cues 
for pitch will improve with training, but that the time course and trajectory of improvements will differ for these 
cues because stimulation rates are not presently used in clinical devices to convey information. Because the 
rate cue is not presently encoded, we believe the potential for improvements is greater than for place. 
AIM 1: TEST THE PERCEPTUAL AND PHYSIOLOGICAL PLASTICITY  
OF PLACE AND RATE CUES IN COCHLEAR IMPLANT STIMULATION 
Figure 1: Pitch discrimination based on pulsatile stimula-
tion rate improves with training. 

1.2 
Research Design 
The proposed research examines the plasticity of pitch perception in adult cochlear implant users as conveyed 
by place and rate of stimulation. Adult cochlear implant users will take part in a crossover study to compare 
changes in pitch sensitivity across two training periods that alternate between training of pitch discrimination 
based on place and rate of stimulation. Training periods will be 4 weeks in duration with exercises completed 
4 times per week during 2-hour training sessions. Psychophysical measures will include pitch discrimination 
conveyed by place and rate in isolation and as conveyed congruently or incongruently. Psychophysical measures 
will also include pitch discrimination as conveyed through the clinical processor to gauge the extent that existing 
devices use the underlying psychophysical potential. Electrophysiological measures will be used to characterize 
individual differences and changes that occur with training. Electrophysiological measures will include electrically 
evoked compound action potentials (eCAPs) and electrically evoked auditory change complex (eACC). Partici-
pants will be randomly assigned to first train either on place or rate discrimination and will subsequently crossover 
to train on the other cue. All psychophysical and electrophysiological measures will be conducted at baseline, 
midpoint, and endpoint. Neural modeling will be used to quantify and predict pitch salience evoked by place and 
rate of stimulation. Models will be calibrated using the results from psychophysical and electrophysiological 
measures. The modeling process will inform and facilitate new hypotheses regarding individual differences and 
pitch processing. 
1.2.1 Participants 
This study will be completed by 24 adult cochlear implant users. The nature of the study requires substantial 
psychophysical training and assessment that younger children would struggle with, so while future work should 
consider perceptual learning in pediatric patients, we focus the initial characterization in adults. Any adult 
cochlear implant user who can complete the procedures may participate. We have the necessary equipment and 
expertise to conduct the described procedures with Advanced Bionics, Cochlear Corporation, and MEDEL 
implants. Equal numbers of men and women will complete the protocol. Both prelingually and postlingually 
deafened adults will be enrolled and the effects of age and hearing-loss history will be considered. Participants 
will be scheduled for 2-hour training sessions with 24 to 72 hours between sessions. We will actively recruit 
participants who use single-electrode cochlear implants as they are of special interest. They are unique in that 
by having only a single electrode they likely receive and learn to perceive temporal patterns with more resolution. 
We will also active recruit participants who have residual hearing in the implanted ear. These participants are of 
special interest because we want to consider how duration of deafness affects temporal sensitivity. 
1.2.2 Psychophysics 
Psychophysics include computer-controlled electrode psychophysics and through-the-processor psychophysics. 
Computer-controlled electrode psychophysics will be used to probe pitch salience as conveyed by place and 
rate of stimulation. Through-the-processor psychophysics will be used to probe how well pitch is conveyed by 
clinical devices. Assessments are designed to determine how much training on place and rate improves 
sensitivity, and the extent that benefits transfer to sensitivity measured through clinical devices. Analysis will 
consider the extent that existing sound processing makes use of available psychophysical sensitivities. 
a) Computer-controlled electrode psychophysics. Pitch discrimination will be measured as provided inde-
pendently by place and rate of stimulation, as well as by their congruent and incongruent combinations. For 
congruent combinations, higher stimulation rates will be presented with more basal stimulation (and with more 
apical stimulation for incongruent combinations). Stimuli will be dual-electrode pulse trains as described by 
McDermott and McKay30, which probe place-pitch perception with greater resolution than possible with single-
electrode stimulation. Two-alternative forced-choice procedures will be used in which participants judge which 
of two stimuli is higher in pitch. Frequency discrimination will be measured near condition frequencies of 110, 
220, 440, 880, and 1760 Hz for each of the 4 stimulation conditions (place, rate, congruent, incongruent). 
Adaptive procedures will be used to measure 75% discrimination accuracy. There are 24 conditions in this 
assessment with each condition frequency mapped to a corresponding dual-electrode configuration. Thresh-
olds and comfort levels will be measured as a function of stimulation rate and used to balance loudness as 
described in Goldsworthy and Shannon, 201416. Stimulation levels will be roved between 70 and 80% of the 
dynamic range and the reference frequencies will be roved ¼-octave around the condition frequency. 

Figure 2 shows preliminary results from 7 subjects (10 ears), for independent and incongruent conditions. 
Participants were naïve in that they did not receive training for either pitch cue. Discrimination threshold based 
on place of stimulation was roughly flat as a function of frequency, which indicates that electrodes were roughly 
equally discriminable. The dashed line indicates the discrimina-
tion threshold corresponding to a place difference of 1 electrode. 
Discrimination thresholds based on stimulation rate were better 
resolved than for place at low frequencies but deteriorated 
above 220 Hz, as expected for naïve participants. Discrimination 
thresholds were generally better when place and rate were con-
gruently combined than for either cue alone. 
The proposed research tests the independence and plasticity of 
place and rate cues. Independence will be examined using con-
gruent and incongruent combinations of the two cues. Independ-
ence of these cues has been sparsely explored, with most stud-
ies underpowered and with no studies considering plasticity of 
the underlying cues. Importantly, the shaded yellow region of 
Figure 2 shows the rate discrimination thresholds after 32 hours 
of training. Rate discrimination with training is better than even 
the combined cue condition in naïve subjects. This comparison 
demonstrates the remarkable plasticity and resolution of rate 
discrimination and its potential for enhancing the overall pitch 
percept. 
b) Through-the-processor psychophysics. Pitch sensitivity and speech comprehension as conveyed by clinical 
processors will be measured to test how well existing devices take advantage of underlying psychophysical 
cues and to consider transfer of learning. Pure tone frequency discrimination will be measured for frequencies 
from 125 Hz to 8 kHz in octave steps. Harmonic complex fundamental frequency (f0) discrimination will be 
measured for f0s of 110, 220, and 440 Hz for high-pass filtered complexes and low levels of background noise 
to mask low frequency components. This is done to avoid access to a place-of-excitation cue associated with 
the fundamental. In previous work6,31, we showed that correlations between these measures of pitch perception 
were correlated with consonant and vowel identification in fluctuating noise, which suggests that improving 
pitch discrimination might partially transfer to phoneme identification. Consonant and vowel identification will 
be measured in quiet and in multi-talker background noise as an adaptive speech reception threshold proce-
dure. These measures are as described in Goldsworthy, 201531, with the following specifications. Frequency 
discrimination will be measured with ¼-octave roving about the condition frequency and 6 dB level roving about 
65 dB SPL with sound presented through an audio speaker in the free field. Consonant and vowel identification 
will be measured in quiet and in two-talker maskers as described in Goldsworthy, 201531, but using the web-
enabled technology described for pediatric assessment in Goldsworthy and Markle, 201932. 
1.2.3  
Electrophysiology 
We will measure eCAPs and eACCs to characterize individual differences and the effects of training at different 
levels of the auditory pathway. eCAPs will be measured to characterize spatial tuning and temporal synchronicity 
at the auditory nerve. eACCs will be measured to characterize changes in cortical response evoked by place 
and rate of stimulation. These measures will be used to consider individual differences, to examine plasticity of 
the neural response, to refine models of cochlear implant pitch perception, and to encourage the development 
of new hypotheses. 
a) eCAPs. Spread of excitation will be measured using a two-pulse forward-masking paradigm in which the probe 
electrode is fixed, and the masker is varied across the electrode array33–35. Stimuli will be as described by 
Hughes and Abbas, 200635, presented at 80% of the behavioral dynamic range. Measures will be collected 
and used to calculate the channel separation index as described by Hughes. Temporal responsiveness and 
synchronicity will be measured based on the composite response to individual pulses of a pulse train as de-
scribed in He et al., 2016, for a 400 ms pulse train. Temporal responsiveness will be characterized by refrac-
toriness and recovery, neural adaptation and adaptation recovery23,26,33. Temporal synchrony will be charac-
terized by the autocorrelation and power spectral density of composite eCAP responses across pulse presen-
tations. This measure of temporal synchrony will connect the electrophysiology to the physiology studies of 
Cariani and Delgutte36,37, which explore models of pitch perception based on interspike interval information. 
 
Figure 2: Pitch discrimination is generally better with 
place and rate of stimulation provided in a congruent 
manner compared to either cue alone. 

b) eACCs. eACCs will be measured for changes in place and rate of stimulation. Spatial resolution will be probed 
in response to dual-electrode stimulation as specified for the computer-controlled electrode psychophysics. 
Scheperle and Abbas38 found that subjects had large eACCs associated with place of stimulation changes of 
a single electrode. Dual-electrode stimulation allows intermediate steps to probe spatial resolution with better 
resolution30. In control conditions, the ratio of charge delivered to the dual-electrode configuration remains 
constant throughout an 800 ms recording interval. In experimental conditions, the eACC will be elicited by 
introducing a change in the dual-electrode charge configuration 400 ms after the onset of the pulse train. 
Scheperle and Abbas38 found that eACCs typically saturate with a separation of 2 electrodes; consequently, 
we will measure eACCs for electrode spacings from ±2 electrodes in ¼-electrode steps. eACCs will also be 
measured for changes in stimulation rate. For control conditions, stimulation rate remains constant throughout 
the 800 ms recording interval. For experimental conditions, the eACC will be elicited by introducing a change 
in rate 400 ms after the onset of the pulse train. Probed contrasts will be based on measured psychophysical 
rate discrimination thresholds (i.e., higher stimulation rates require larger rate changes to elicit a response). 
Probed conditions will typically span an octave but with narrower ranges used as needed. 
1.2.4  
Neural Modeling 
Models of the auditory periphery and midbrain will be used to first characterize neural response to pure and 
complex tones in healthy physiology. Models of electrical current spread and action potential generation will be 
used to produce a comparison set of auditory-nerve responses predicted for cochlear implant stimulation. Using 
these models of normal auditory-nerve response and of response to electrical stimulation, we will characterize 
relationships between neural responses and psychophysics. Our initial approach will follow the work of Co-Inv. 
Dr. Carney, as described in Bianchi et al., 201939, where they examined the effects of hearing loss on 
f0 discrimination and temporal fine structure processing. In that study, auditory nerve responses were simulated 
for normal and impaired hearing for complex tones and used to estimate frequency and modulation encoding at 
the level of the inferior colliculus. Distance metrics will be used to characterize differences in place and rate 
coding of frequency at both the level of the auditory nerve and inferior colliculus. Neural modeling will be adjusted 
based on measured electrophysiology. For example, measured eCAPs characterizing channel interaction will be 
used to adjust the estimated electrode-to-neural distance in a current spread model. Likewise, measured eCAPs 
characterizing neural synchrony will be used to adjust membrane time constants in a charge integration model. 
The adjusted model will be used to calculate corresponding distance metrics derived from changes in place and 
rate of implant stimulation and relationships will be examined with measured discrimination thresholds. Modeled 
thresholds will be estimated using the same stimuli used for psychophysical and physiological measures, 
allowing direct comparison of modeled and measured thresholds. This approach will allow systematic tests of 
specific functional relationships between neural and perceptual responses, as follows. 
One functional relationship that will be examined is the pooled interspike interval distributions of the estimated 
auditory-nerve response. Cariani and Delgutte demonstrated that periodic stimuli that evoke a strong sense of 
pitch produce pooled interval distributions with relatively high peak-to-mean ratios. It is expected that pulsatile 
stimulation and modeling of such will produce higher peak-to-mean ratios compared to normal physiological 
encoding and we will consider individual differences based on electrophysiological measures. Further, we will 
use the predicted neural response to acoustic and electric stimulation as inputs to the temporal pitch processing 
model described by Bahmer and colleagues40,41. As described by Cariani and Delgutte36,37, evidence for a 
physiological mechanism for decoding temporal oscillations into place-of-excitation would provide a unified 
explanation of different aspects of pitch perception. The modeling work of Bahmer and colleagues40,41 clarifies 
how networks of cells in the cochlear nucleus could perform such temporal processing. Our work will combine 
the modeling efforts of Dr. Carney and collaborator Dr. Bahmer and will characterize the response of the cochlear 
nucleus oscillatory network models to auditory-nerve response estimated for both acoustic and electric 
stimulation. 
1.2.5  Data Analysis Plan 
The psychophysical measures are continuous variables and will be described as means and standard deviations. 
Of primary interest is whether measures change with training. The difference in means will be analyzed using a 
repeated-measures analysis of variance. The study is overpowered on the main hypothesis that pitch ranking 
will improve with experience. For comparison, the data published in Goldsworthy and Shannon 201442 is of 
similar design and can be used to guide expectations. That data set consisted of 6 subjects who trained on pitch 
ranking for a total of 32 hours across 4 weeks. In the proposed design, training periods are 16 hours in duration. 
Analysis of variance based on the earlier data set over the initial 16 hours indicates a large training effect 

(F4,74 = 17.3, p < 0.001). The corresponding thresholds improved by more than a factor of two (from 4.2 to 
1.8 semitones, dCohens > 1). The experimental design is overpowered on the main effect of training to provide 
power to detect transfer of learning to untrained measures, which are expected to be smaller. Power analysis of 
this design with 24 subjects and a 0.05 significance criterion indicates a 95% likelihood of detecting effects as 
small as 0.19, corresponding to a 10% improvement in thresholds. This level of resolution will be useful when 
testing secondary hypotheses associated with the salience and time course of learning when comparing pitch 
evoked by place and rate of stimulation. 
Electrophysiological measures will be collapsed into summary statistics and analyzed for relationships with psy-
chophysical measures. Planned summary statistics were briefly described in the electrophysiology section and 
include measures of spectral tuning and temporal responsiveness. Correlation between electrophysiological and 
psychophysical measures will be tested. Power analysis of this design with 24 subjects and a 0.05 significance 
criterion indicates an 80% likelihood of detecting correlations of 0.48 or greater (explaining 23% of the variance). 
As multiple stimulation sites will be examined (e.g., apical/basal), correlations will also be examined within par-
ticipants across stimulation sites. For example, correlation between the change in psychophysical and electro-
physiological measures across stimulation sites. Such correlations predicting within-subject changes in 
measures has been used in cochlear implant psychophysics to characterize individual differences with much 
smaller participant population than generally required for across subject analyses. 
Planned modeling analyses include variations of the approach described by Co-Inv. Dr. Carney and colleagues 
in Bianchi et al., 201939. For example, spectral tuning will be modeled for normal physiological response using 
the model of normal physiology developed by Dr. Carney. Spectral tuning of cochlear implant stimulation will 
be modeled using the computational model of auditory-nerve response to electrical stimulation developed by 
Dr. Frijns, which has been validated with animal data for modulated and unmodulated pulse trains. Modeled 
response to electrical stimulation will be simulated for the conditions probed for measured eCAPs and will be 
analyzed in terms of spread of excitation. Simulated neural responses will be described by summary statistics; 
for example, the psychophysical channel separation index described by Hughes34 but based on simulated re-
sponse properties. Direct comparisons of thresholds, as well as trends with place and rate of stimulation, will 
be examined to characterize relationships between modeled and measured sensitivity. 
1.3 
Expectations, Contingencies, and Explorations 
We expect to learn the extent that pitch ranking based on place and rate of cochlear implant stimulation can be 
improved through psychophysical training. We will clarify the extent that place and rate of stimulation are inde-
pendent. We will characterize relationships between such measures of pitch perception and electrophysiological 
measures of neural response at the level of the auditory nerve and cortex. We will refine models of pitch percep-
tion by comparing modeled and neural responses to acoustic and electric stimulation. We expect these efforts 
to inform the unexplored limits of stimulation rate to convey information in the next generation of sound pro-
cessing strategies for cochlear implants. 
Aim 1 is supported by strong preliminary data and there is a high likelihood that we will successfully characterize 
the plasticity of pitch perception provided by place and rate of stimulation, and that we will characterize the 
dependence/independence of these cues. Primary concerns/obstacles are associated with the efficacy of the 
planned psychophysical training. The psychophysical training presently must be performed in the laboratory 
since we use computers and special hardware to provide specific stimulation patterns. Psychophysical training 
at home, rather than in the laboratory, would avoid practical issues associated with optimal training. Training at 
home would facilitate training durations that have generally been shown to be more effective, such as using 
shorter training blocks around 30 to 40 minutes a day. Short sessions are impractical for laboratory training since 
participants often have long commutes. We have developed web-based training software that participants can 
access at home, but we need to develop customized maps for participants that would allow for carefully controlled 
electrode psychophysics to be completed at home. For example, for Advanced Bionics devices, participants 
would need to be able to enter a research program on their clinical devices that would be configured to having a 
single electrode configured for a processing strategy such as HiRes that provides temporal envelopes with high 
precision. Likewise, single-electrode configurations for Cochlear devices would be developed using FAST and 
PDT algorithms to allow single-electrode and dual-electrode psychophysics to be trained and evaluated re-
motely. Long-term plans for developing electrode psychophysics to be conducted at home will allow our research 
to flexibly respond to contingencies associated with optimal paradigms for perceptual learning including: opti-
mized stimulus presentation, multisensory facilitation, and consistent reinforcement of training stimuli43, which 
have individually contributed to increasing the speed44, magnitude44,45, and generality of learning46,47. 

 
 
 
2.1  Background & Rationale 
Most cochlear implants use amplitude modulation of constant stimulation rates, an exception being Fine Spectral 
Processing for MED-EL devices, which uses a restricted form of dynamic rates on the most apical electrode(s). 
In terms of stimulation control, dynamic-rate stimulation is temporally more precise than amplitude modulation, 
yet few studies have examined differences produced by these two stimulation paradigms. Baumann and Nobbe48 
found that dynamic rates provide better pitch resolution than provided by amplitude modulation, with larger 
effects observed above 200 Hz. Vandali and van Hoesel49 found an advantage of dynamic rates for pitch ranking 
at 275 Hz. In contrast, Kong and colleagues50 did not find differences between the two for pitch ranking. Our 
preliminary results, summarized within the research design, indicate that dynamic rates provide better pitch 
discrimination and that the resulting sense of pitch is more robust to interference. 
While only a few studies have compared amplitude-modulated and dynamic-rate stimulation, other studies have 
examined general effects of modulation depth and waveshape in both acoustic and electric hearing. Bernstein 
and Trahiotis showed that temporally compact waveshapes provide better interaural timing sensitivity51–59. In 
200257, they showed benefits of transposed envelopes compared to sinusoidally amplitude-modulated (SAM) 
envelopes; in 200960, they showed increased spatial hearing resolution driven by sharper temporal envelopes 
along a continuum of waveshapes. Several studies have extended this work to bilateral cochlear implants and 
have shown that interaural timing sensitivity generally improves with sharper temporal envelopes. Comparable 
studies have considered the effect of waveshape on pitch by considering the extent that pitch can be evoked by 
unresolved harmonics. Kaernbach and Bering, 200161, showed that discrimination thresholds for unresolved 
harmonics can be as low as 1.2% when represented using multiple harmonics, which would provide better 
temporal precision in the decoded neural envelope. A few studies have considered the effect of modulation 
waveshape on cochlear implant pitch perception. Kreft and colleagues, 201562, found that SAM and transposed 
envelopes provide similar pitch discriminability, but several other studies have shown that more aggressive 
envelope sharpening improves pitch sensitivity49,63–65. In summary, there is strong evidence that temporal 
envelope sharpening improves pitch and spatial hearing for normal-hearing listeners when provided unresolved 
harmonics and for cochlear implant users when listening to amplitude modulation. Dynamic-rate stimulation is in 
many ways an extreme form of envelope sharpening in which envelope periodicity is replaced by singular pulses 
representing the pitch period. Consequently, if any advantage can be derived from sharper temporal 
representation, we expect it to be most pronounced when using dynamic rates. 
Physiologically, there is clear evidence that sharper temporal envelopes evoke a more compact response from 
the auditory nerve. Dreyer and Delgutte66 showed that phase locking of the auditory nerve is better for transposed 
compared to SAM tones, and that phase locking is better for pure tones than either form of amplitude modulation. 
Further, Jeng et al., 200967, characterized auditory-nerve response to amplitude-modulated electrical pulse trains 
in guinea pigs and found that modulations can be distorted when presented at low presentation levels. These 
results indicate that temporal coding in the auditory nerve is generally more precise with increasing sharpness 
of the acoustic/electric envelope. Phase locking is more precise for pure tones in acoustic stimulation than for 
modulated tones, even when using transposed stimuli66, which is consistent with psychophysical results 
indicating that resolution of pitch and spatial hearing is best for low frequency tones but can be transmitted by 
modulation of high frequency tones. In summary, psychophysical and physiological studies have shown that 
temporal pitch perception tends to be better when conveyed by temporally compact or sharpened waveforms. 
Acoustically, performance is better for low frequency tones compared to modulated high frequency tones; 
electrically, we hypothesize better performance for dynamic rates compared to modulation of constant rates. 
There is strong evidence that cochlear implant stimulation excites a highly synchronous response in the auditory 
nerve. Studies of normal and impaired hearing suggest that temporal periodicity is a fundamental pitch cue. 
Normal-hearing listeners can discriminate tones based on period differences as small as 1.2% when provided 
temporal cues associated with unresolved harmonics. The primary goal of Aim 2 is to determine the extent that 
dynamic-rate stimulation can be used to engage this temporal periodicity mechanism to provide implant users 
with better pitch perception on par with normal-hearing performance when restricted to unresolved harmonics. 
We hypothesize that dynamic-rate stimulation provides better resolution and more tolerance to interference than 
provided by modulation of constant-rate stimulation. Aim 2 is designed to test this hypothesis and to characterize 
the limits of temporal processing in the auditory system. 
AIM 2: DETERMINE IF DYNAMIC-RATE STIMULATION IMPROVES  
PITCH PERCEPTION AND PHYSIOLOGICAL ENCODING OF PERIODICITY 

2.2 
Research Design 
Aim 2 is designed to examine differences between amplitude-modulated and dynamic-rate stimulation. The study 
protocol compares pitch salience and tolerance provided by sinusoidal amplitude-modulated (SAM) stimulation 
with that provided by dynamic-rate stimulation. The study protocol examines salience and tolerance of pitch in 
the context of perceptual learning. Since conventional devices do not typically use dynamic rates, participants 
may require experience to learn to use the more precise representation of periodicity provided by dynamic-rate 
stimulation. The salience and tolerance of pitch provided by SAM and dynamic-rate stimulation will be examined 
across training periods that provide participants with experience with temporal periodicity. A crossover design 
will be used for Aim 2 with psychophysical and physiological measures designed to probe modulation and rate-
based encoding of periodicity. Participants will be randomly assigned to train first either on SAM or on dynamic-
rate stimulation and will subsequently crossover to train on the other method of stimulation. Training periods, as 
for Aim 1, will be 4 weeks in duration with on average 4, 2-hour, training sessions provided each week. Assess-
ment measures will be conducted at baseline, midpoint, and endpoint. This research is designed to determine if 
dynamic-rate stimulation provides better pitch perception for cochlear implant users and to test the extent that 
pitch sensitivity can be progressively refined through experience. 
2.2.1 Participants 
This study will be completed by 24 adult cochlear implant users. Subjects will be recruited from the Aim 1 cohort 
as well as outside of that cohort. Prior experience with psychophysical measures conducted both in our lab and 
outside studies will be considered in data analysis. Subjects will include AB, Cochlear, MEDEL, and single-
electrode device users. The single-electrode device users are a special population in that they receive 
information purely through temporal cues, though some might argue a small spatial effect can be controlled using 
single-electrode stimulation, but that effect if any would be small. Cochlear implant users who have substantial 
residual hearing in the non-implanted ear are also a special population as they allow for pitch-matching 
comparisons to be made with the residual acoustic hearing. 
2.2.2 Psychophysics 
Our primary hypotheses are that dynamic-rate stimulation provides a sense of pitch that is better resolved and 
more tolerant to interference than SAM stimulation. These hypotheses are connected to the Aim 1 hypothesis 
concerning the plasticity of pitch provided by dynamic-rate stimulation. If dynamic-rate stimulation provides a 
better sense of pitch, then it is likely that the degraded representation provided by amplitude modulated 
stimulation will result in physiological deterioration. Consequently, we are not only interested in differences 
between amplitude modulated and dynamic-rate stimulation in naïve subjects, but also in the differences that 
emerge with dedicated psychophysical training. 
a) Computer-controlled electrode psychophysics. As for Aim 1, computer-controlled psychophysics will be used 
to characterize basic sensitivities to electrical stimulation. These measures by-pass clinical sound processing 
and the associated limitations related to discarding temporal 
fine structure and reduced modulation encoding. A battery of 
measures has been collected to probe pitch sensitivity and 
tolerance to interference for amplitude modulated and dynamic-
rate stimulation: 
 
1) Beat-frequency distortion. This measure tests sensitivity to 
beat-frequency distortions that occur with amplitude modulation 
of pulsatile stimulation. The standard stimulus is a SAM pulse 
train with the carrier rate an integer multiple of the f0 (no 
beating). The target stimulus is identically defined except having 
a small (~10 Hz) shift in the carrier rate. For low harmonic 
multiples (equivalently, pulses per period), the shift in carrier 
rate produces audible beating. For rates approaching 8 to 10 
pulses per pitch period, this beating becomes weak and 
eventually inaudible. Figure 3 shows preliminary results from 8 
subjects (10 ears). The carrier rate needed to avoid artifactual 
beating between carrier and envelope is plotted as a function of 
modulation frequency. Results indicate that higher carrier rates 
are needed to encode higher modulation frequencies to avoid 
Figure 3: Cochlear implant users require higher rate pul-
satile carriers to convey higher modulation frequencies to 
avoid beat-frequency distortions. 

distortion. Two subjects were particularly sensitive to beat-frequency distortions for f0s near 440 Hz and 
required carrier rates above 5 kHz to avoid distortions. Consequently, subsequent psychophysical measures 
of modulation sensitivity will use carrier rates of 6400 Hz when possible (N22 users use 4 kHz carrier rates). 
 
2) Pitch discrimination. This measure is designed to test differences in pitch discrimination provided by SAM 
and dynamic-rate stimulation. The standard stimulus is either a SAM or dynamic-rate pulse train and the target 
stimulus is identically defined as the corresponding standard but having an adaptively higher modulation 
frequency or carrier rate. Protocol conditions include f0s of 110, 220, 440, and 880 Hz with dual-electrode 
stimulation of apical and basal locations for a total of 8 
conditions that will be repeated 3 times within an hour. Stimuli 
will be 400 ms in duration and presented at comfortable 
listening levels controlled as a function of stimulation rate16. 
Figure 4 shows preliminary results from 7 subjects (11 ears). 
Pitch discrimination was better for dynamic-rate compared to 
SAM stimulation (F1,138 = 13.15, p < 0.01). The difference 
between discrimination thresholds provided by dynamic-rate 
and SAM stimulation was large (dCohens = 2.2) when measured 
for f0s near 220 Hz and medium (dCohens = 0.7) when measured 
near 440 Hz. It is exciting to compare these results with the 
results reported in Goldsworthy and Shannon, 201416. The 
shaded yellow region of Figure 4 indicates average rate 
discrimination thresholds achieved after 32 hours of training. 
Even though dynamic rates provide better discrimination than 
SAM stimulation, these benefits could be further improved with 
training. By considering perceptual learning of temporal pitch 
provided by these two stimulation methods, we will determine 
the extent that pitch sensitivity is limited by the stimulating 
waveform as opposed to an underlying physiological limitation. 
3) Tolerance to interference. Tolerance to interference will be measured for SAM and dynamic-rate stimulation. 
Conditions will include f0s of 220 Hz with single-electrode stimulation of apical and basal electrodes with 
interfering electrodes 0, 2, or 4 electrodes away, as well as a 
condition without interference. Tolerance will be measured as 
the lowest target-to-interference level at which subjects can 
discriminate stimuli that differ by an octave in f0. The 8 
conditions (f0 = 220 Hz, apex and base, and 4 interference 
conditions) will be repeated 3 times within an hour. Figure 5 
shows preliminary results from 5 subjects (8 ears). As for pitch 
discrimination in quiet, the target sound was defined identically 
as 
the 
standard, 
but 
having 
a 
higher 
modulation 
frequency/stimulation rate. For this tolerance measure, the rate 
difference is held constant at 1 octave, but the relative target 
level is adaptively controlled. Notably, for dynamic rates, 
subjects achieved average discrimination thresholds for which 
the interference was louder than the target. Analysis of 
variance and post-hoc multiple comparisons indicated that the 
relative target level was significantly lower for dynamic-rate 
stimulation compared to SAM stimulation both as a main effect 
and for each masker separation (p < 0.01 for all comparisons). 
4) Pitch discrimination with interference. Pitch discrimination will be measured for the same conditions as the 
tolerance measure but f0 discrimination thresholds will be measured at a set relative target level. Similar 
measures have been reported in the literature for SAM pulse trains tested at a relative target levels of 0 (i.e., 
target and interference at the same level). Kreft et al., 201368, found that subject’s struggled to perform the 
task at relative target level with SAM stimulation. Our method for first measuring tolerance across conditions 
allows us to adjust relative target levels to avoid floor effects encountered by Kreft et al., 201368. Figure 6 
shows preliminary results from 5 subjects (8 ears) for discrimination thresholds measured with interference. 
Figure 5: Pitch ranking using dynamic-rate stimulation is 
more tolerant to interference. 
Figure 4: Pitch discrimination thresholds are better for 
dynamic-rate compared to SAM stimulation; the differ-
ence is significant and most pronounced for rates near 
220 Hz where modulation encoding diminishes. 

On average, discrimination thresholds were better with 
dynamic-rate compared to SAM stimulation (p < 0.001). Multiple 
comparisons for each masker separation condition indicates 
that thresholds were significantly lower for dynamic-rate 
stimulation for every condition tested (p < 0.05). For 
comparison, results are plotted with the average thresholds 
measured after training for the 220 Hz condition from 
Goldsworthy and Shannon, 201416. Comparison with the 
previous data indicates that the trained thresholds are better, a 
pertinent remaining question is to what extent training can not 
only enhance discrimination thresholds, but also tolerance to 
interference. 
5) Synthetic vowel pitch discrimination in quiet. Pitch 
discrimination of synthetic vowels will be measured using 
carefully controlled multi-electrode complexes. A synthetic 
vowel will be characterized by a f0 and 3 formant frequencies. 
The formant frequencies will be mapped to electrode position 
using each subject’s clinical frequency allocation. For both SAM 
and dynamic-rate stimulation, the formant frequency would be 
used to control an electrode dyad comprised of stimulation of 
adjacent electrodes. For example, if a specific formant 
frequency corresponded to a virtual electrode location of 4.2, 
then electrodes 4 and 5 would comprise the dyad with the ratio 
of stimulation levels controlled by the relative location of the 
virtual electrode. Note, that only for AB devices is such 
stimulation simultaneous, with other devices the stimulation 
across electrodes will be done with as small as delay as 
possible, typically less than 100 µs. For stimulation of 
3 formants, stimuli would typically be represented with activity 
presented across 6 electrodes. Standard stimulation is 
presented with a nominal f0 and target stimulation is identically 
defined in terms of place-of-excitation but having an adaptively 
higher f0. Figure 7 shows preliminary results from 4 subjects 
(5 ears). Average discrimination thresholds were lower for 
dynamic-rate compared to SAM stimulation (p < 0.001). With 
4 subjects tested, multiple comparisons of dynamic-rate versus 
SAM stimulation for each condition do not significantly differ. 
These preliminary results indicate that the better pitch salience 
provided by dynamic-rate stimulation seems to be maintained 
for multi-electrode stimulation in a manner that mimics vowel excitation. 
b) Through-the-processor psychophysics. Through-the-processor psychophysics will be used to test the extent 
that clinical devices make use of underlying psychophysical cues. These methods generally use SAM tones or 
high-pass filtered pulse trains as way of producing acoustic analogs of electrode psychophysics. 
1) Modulation detection thresholds. This measure probes sensitivity to amplitude modulations. The standard 
stimulus is a 400-ms pure tone and the target is identically defined but is a SAM tone. Adaptive procedures 
are used to measure modulation detection threshold corresponding to the minimum modulation depth that can 
be detected with 75% accuracy. Modulation frequencies of 10, 110, 220, and 440 Hz will be tested for carrier 
frequencies of 500 and 2000 Hz. 
2) Pitch discrimination. This measure is an acoustic analog of the pitch discrimination task outlined for electrode 
psychophysics and probes sensitivity to temporal pitch. The standard stimulus is a 400-ms bandpass-filtered 
harmonic complex and the target is identically defined except having an adaptively higher f0. Conditions will 
include modulation frequencies of 110, 220, and 440 Hz for bandpass center frequencies of 500 and 2000 Hz. 
Stimuli will be presented in pink background noise to reduce the likelihood that participants attend to place-of-
excitation cues for performing the task. 
Figure 6: Pitch discrimination is better for dynamic-rate 
stimulation compared to amplitude modulated stimula-
tion when tested in the presence of electrical interference. 
Figure 7: Pitch discrimination is better for dynamic-rate 
stimulation compared to amplitude modulated stimula-
tion when tested using multi-electrode synthetic vowels. 

3) Pitch discrimination with interference. This measure is an acoustic analog of the electrode psychophysical 
task and probes tolerance to interference. The standard and target stimuli are as defined for pitch discrimina-
tion but having an additional harmonic complex with independently controlled f0 and filter center frequency. 
Conditions will include the same conditions as for the pitch discrimination task but for combinations of interfer-
ing tones. 
4) Melodic contour identification. This measure does not have an electrode psychophysical analog, but more 
generally probes how well pitch cues are available for performing a melodic contour identification task. This 
measure will allow us to test whether focused psychophysical training on periodicity cues transfers to this more 
complex representation of melody. 
5) Phoneme identification in quiet and competing speech. Consonant and vowel identification will be measured 
in quiet and in competing speech as background noise. The consonant database consists of 20 vowels spoken 
by five male and five female talkers in /a/–C–/a/ context6,69. The vowel database consists of 12 vowels spoken 
by five male and five female talkers in a /h/-V-/d/ context6,70. Listeners will respond using a computer interface 
with alternatives for the appropriately labeled phonemes. For the measures in noise, the signal-to-noise ratio 
will be adaptively controlled to converge to 50% recognition accuracy. 
2.2.3  
Electrophysiology 
As for Aim 1, we will measure eCAPs and eACCs to characterize individual differences and the effects of training 
but with emphasis on modulation encoding. eCAPs will be measured to characterize modulation encoding in the 
auditory nerve as functions of modulation depth and stimulation level. eACCs will be measured to characterize 
changes in cortical response evoked by changes in modulation frequency and by rate of stimulation. As for 
Aim 1, These measures will be used to consider individual differences, to examine neural plasticity, to refine 
models of cochlear implant perception, and encourage the development of new hypotheses. 
a) eCAPs. Aim 2 centers on understanding psychophysical and physiological differences between amplitude 
modulated and dynamic-rate stimulation. eCAPs will be measured using modulated and unmodulated stimuli 
and will be analyzed in terms of periodicity encoding. Stimuli will be as described by Tejani et al.71 for SAM 
stimuli and first-order analysis based on the modulation response amplitude. Temporal responsiveness and 
synchronicity will be further characterized based on the composite response to individual pulses of a pulse 
train as described in He et al., 201623, for a 400 ms pulse train. Temporal responsiveness will be characterized 
by refractoriness and recovery, neural adaptation and adaptation recovery. Temporal synchrony will be char-
acterized by the autocorrelation and power spectral density of composite eCAP responses across pulse 
presentations. These measures will be used to determine if any of the perceived differences in pitch salience 
and tolerance can be predicted by observed differences in periodicity encoding of the eCAPs. 
b) eACCs. eACCs will be measured for changes in modulation frequency for amplitude modulated stimulation 
and for changes in stimulation rate for dynamic-rate stimulation. In control conditions, the modulation/dynamic 
rate remains constant throughout an 800 ms recording interval. In experimental conditions, the eACC will be 
elicited by introducing a change in the modulation/dynamic rate 400 ms after the onset of the pulse train. 
Probed conditions will be based on measured psychophysical thresholds but will typically span an octave with 
narrower spacings as needed. eACCs will be measured for psychophysically derived spacings for sinusoidally 
amplitude modulated stimulation will be examined as a function of modulation depth and presentation level. 
eACCs will be measured for dynamic-rate stimulation as a function of level. For both stimulation methods, 
eACCs will be measured near modulation frequency or stimulation rates of 110, 220, 440, and 880 Hz.   
2.2.4  Data Analysis Plan 
Of primary interest is characterizing differences between pitch salience and tolerance provided by amplitude 
modulation of constant-rate stimulation and by dynamic-rate stimulation. Analysis of the preliminary data guides 
expectations. Considering preliminary data shown in Figure 4, the associated effect sizes comparing the better 
pitch discrimination thresholds provided by dynamic-rate stimulation to that provided by amplitude modulation 
are: dCohens = 0.20, 1.8, and 0.71, for the 110, 220, and 440 Hz conditions, respectively. With 11 ears tested, 
these effects were significant at the 0.05 level for the comparisons at 220 and 440 Hz. Power analysis of this 
design with 24 subjects and a 0.05 significance criterion indicates a 95% likelihood of detection effects as small 
as 0.19, which would provide needed power to detect effects at 110 Hz and for training improvements of 10% or 
greater. This level of resolution will be useful when testing secondary hypotheses associated with the 
interference tolerance and plasticity when comparing amplitude modulation and dynamic-rate stimulation. 

3  
Study Team and Timeline 
The proposed work brings together researchers with multiple lines of independent research into a cohesive team 
to examine the plasticity of pitch in cochlear implant users. As the PI, my own research has focused on cochlear 
implant pitch perception, the effects of experience through psychophysical training, and relationships between 
pitch perception and speech recognition6,31,42,72,73. Co-investigator Dr. Eisenberg leads research on childhood 
development after cochlear implantation and her team at USC has developed a hierarchy of speech recognition 
measures and they have published on the responsiveness of this hierarchy for tracking emergent skills after 
cochlear implantation72,74–82. Co-investigator Dr. He leads multiple projects to characterize neural encoding of 
auditory perception in pediatric and adult cochlear implant users. Dr. He has expertise in how the temporal 
response properties of the auditory-nerve response and ascending pathway affect cochlear implant out-
comes23,24,83–86. Co-investigator Dr. Carney leads research that bridges physiology and psychophysics through 
computational modeling of the auditory system. Dr. Carney’s expertise in neural modeling of the auditory nerve 
and ascending brainstem will guide analyses of the proposed research and will inform the development of new 
hypotheses based on predicted neural response. Together, the project team includes a broad range of expertise 
overlapping the disciplines of psychophysics, physiology, and neural modeling. 
The proposed aims will run concurrently. The psychophysical procedures described for both aims have been 
piloted and the preliminary results indicate large effect sizes for psychophysical training and for the comparison 
of amplitude modulated and dynamic-rate stimulation. During the first six months of the project period, we will 
focus on setting up the proposed electrophysiological measures at USC. Drs. Goldsworthy, Eisenberg, and Loeb 
will meet twice a month with Drs. He and Carney joining remotely to discuss amendments to procedures such 
as details of stimulus design and recording analysis. During this period, Drs. He and Carney will visit USC to 
help establish the electrophysiological procedures and neural modeling. The goal of neural modeling during the 
first 6 months will be to provide a common framework for comparing the auditory nerve and ascending circuitry 
model developed by Dr. Carney with the auditory-nerve response model developed by Dr. Frijns. At the end of 
the initial six-month period, the psychophysical and electrophysiological procedures will be finalized, and sub-
jects will be recruited through emails, newsletters, and direct patient contact (as IRB approved). Results will be 
analyzed as acquired and active enrollment is expected through the second year of the project period, at which 
time emphasis will be given to retainment and consolidation as opposed to new enrollment. 
The proposed research for Aim 2 will also be further piloted during the first six months of the project. All psycho-
physical training and assessment measures (or close analogs) have been used in our previous work6,72. During 
piloting, we will enroll participants and evaluate initial results to consider modifications to the protocol. A focal 
point that we intent to examine more thoroughly before commencing the crossover design is the effect of stimu-
lation level and modulation depth on the salience of amplitude modulated stimulation. Once piloting is complete, 
participants will be enrolled on a rolling basis throughout the project period. Once the core protocols described 
in this proposal are underway, key variations will be considered as new experiments. For example, new experi-
ments investigating the salience of pitch as conveyed by different modulation waveshapes, and new experiments 
on the salience of auditory-tactile and auditory-visual integration. These experimental variations will be started 
once the primary experimental line has consolidated. 
Across each aim, electrophysiology will be used to examine cortical activation in response to key auditory con-
trasts relevant to the respective aims. The combined use of EEG and eCAP methodologies is a new direction 
for my lab and we have assembled a strong team to facilitate integration of these methods into our research 
program. During this first six months, we will arrange for multiple site visits with Dr. He to work closely with the 
USC team to set up a new EEG system. By the middle to end of the first year, we expect to transition into formal 
testing of activation contrasts associated with the aural contrasts outlined in this proposal. Electrophysiology 
would then run concurrently with the psychometrics and would be used as part of the longitudinal characterization 
of the plasticity of pitch perception. 
A potential obstacle that could arise in this research is that it may difficult to obtain high completion rates because 
of the substantial amount of laboratory training and assessment that is required. The proposed research has 
built-in contingencies at a core level. We will work closely with implant manufacturers to develop methods for 
providing psychophysical training for electrode contrasts at home. The project team is balanced and flexible 
enough to modify our approach as needed to achieve the goal of characterizing the psychophysical potential of 
periodicity encoding in a rigorous and informative manner. The proposed research will thus contribute to scientific 
understanding of how perceptual learning affects brain and cognitive development, and will clarify the potential 
of temporal fine structure for improving music and speech perception for cochlear implant users. 

REFERENCES 
 
1.  
Miyazono H, Moore BCJ. Implications for pitch mechanisms of perceptual learning of fundamental 
frequency discrimination: Effects of spectral region and phase. Acoust Sci Technol. 2013;34(6):404-412. 
doi:10.1250/ast.34.404 
2.  
Laneau J, Wouters J, Moonen M. Improved music perception with explicit pitch coding in cochlear 
implants. Audiol Neurootol. 2006;11(1):38-52. doi:10.1159/000088853 
3.  
Henry KS, Abrams KS, Forst J, et al. Midbrain Synchrony to Envelope Structure Supports Behavioral 
Sensitivity to Single-Formant Vowel-Like Sounds in Noise. JARO - J Assoc Res Otolaryngol. 
2017;18(1):165-181. doi:10.1007/s10162-016-0594-4 
4.  
Brown CA, Bacon SP. Fundamental frequency and speech intelligibility in background noise. Hear Res. 
2010;266(1-2):52-59. doi:10.1016/j.heares.2009.08.011 
5.  
Cedolin L, Delgutte B. Spatiotemporal representation of the pitch of harmonic complex tones in the 
auditory nerve. J Neurosci. 2010;30(38):12712-12724. doi:10.1523/JNEUROSCI.6365-09.2010 
6.  
Goldsworthy RL, Delhorne L a., Braida LD, Reed CM. Psychoacoustic and Phoneme Identification 
Measures in Cochlear-Implant and Normal-Hearing Listeners. Trends Amplif. 2013;17(1):27-44. 
doi:10.1177/1084713813477244 
7.  
Froemke RC, Schreiner CE. Synaptic plasticity as a cortical coding scheme. Curr Opin Neurobiol. 
2015;35:185-199. doi:10.1016/j.conb.2015.10.003 
8.  
Rader T, Fastl H, Baumann U. Speech perception with combined electric-acoustic stimulation and 
bilateral cochlear implants in a multisource noise field. Ear Hear. 2013;34(3):324-332. 
doi:10.1097/AUD.0b013e318272f189 
9.  
Wilson BS, Finley CC, Lawson DT, Wolford RD, Eddington DK, Rabinowitz WM. Better speech 
recognition with cochlear implants. Nature. 1991;352(6332):236-238. doi:10.1038/352236a0 
10.  
Hochmair I, Hochmair E, Nopp P, Waller M, Jolly C. Deep electrode insertion and sound coding in 
cochlear implants. Hear Res. 2015;322:14-23. doi:10.1016/j.heares.2014.10.006 
11.  
Kleine Punte A, De Bodt M, Van De Heyning P. Long-term improvement of speech perception with the 
fine structure processing coding strategy in cochlear implants. Orl. 2014;76(1):36-43. 
doi:10.1159/000360479 
12.  
Dynes SB, Delgutte B. Phase-locking of auditory-nerve discharges to sinusoidal electric stimulation of 
the cochlea. Hear Res. 1992;58(1):79-90. http://www.ncbi.nlm.nih.gov/pubmed/1559909. 
13.  
Henry KS, Kale S, Heinz MG. Distorted Tonotopic Coding of Temporal Envelope and Fine Structure with 
Noise-Induced Hearing Loss. 2016;36(7):2227-2237. doi:10.1523/JNEUROSCI.3944-15.2016 
14.  
Zeng F-G. Temporal pitch in electric hearing. Hear Res. 2002;174(1-2):101-106. 
http://www.ncbi.nlm.nih.gov/pubmed/12433401. 
15.  
Kong Y-Y, Deeks JM, Axon PR, Carlyon RP. Limits of temporal pitch in cochlear implants. J Acoust Soc 
Am. 2009;125(3):1649-1657. doi:10.1121/1.3068457 
16.  
Goldsworthy RL, Shannon R V. Training improves cochlear implant rate discrimination on a 
psychophysical task. J Acoust Soc Am. 2014;135(1):334-341. doi:10.1121/1.4835735 
17.  
Habibi A, Cahn BR, Damasio A, Damasio H. Neural correlates of accelerated auditory processing in 
children engaged in music training. Dev Cogn Neurosci. 2016;21:1-14. doi:10.1016/j.dcn.2016.04.003 
18.  
Galvin JJ, Fu Q-J, Shannon R V. Melodic contour identification and music perception by cochlear 
implant users. Ann N Y Acad Sci. 2009;1169:518-533. doi:10.1111/j.1749-6632.2009.04551.x 
19.  
Parbery-Clark A, Skoe E, Lam C, Kraus N. Musician enhancement for speech-in-noise. Ear Hear. 
2009;30(6):653-661. doi:10.1097/AUD.0b013e3181b412e9 
20.  
Micheyl C, Delhommeau K, Perrot X, Oxenham AJ. Influence of musical and psychoacoustical training 
on pitch discrimination. Hear Res. 2006;219(1-2):36-47. doi:10.1016/j.heares.2006.05.004 
21.  
Svirsky MA. Cochlear implants and. 2017;52. doi:10.1063/PT.3.3661 
22.  
Verschooten E, Shamma S, Oxenham AJ, et al. The upper frequency limit for the use of phase locking 
to code temporal fine structure in humans: A compilation of viewpoints. Hear Res. 2019;377:109-121. 
doi:10.1016/j.heares.2019.03.011 
23.  
He S, Abbas PJ, Doyle DV, McFayden TC, Mulherin S. Temporal Response Properties of the Auditory 
Nerve in Implanted Children with Auditory Neuropathy Spectrum Disorder and Implanted Children with 
Sensorineural Hearing Loss. Vol 37.; 2016. doi:10.1097/AUD.0000000000000254 
24.  
He S, Teagle HFB, Buchman CA. The Electrically Evoked Compound Action Potential: From Laboratory 
to Clinic. Front Neurosci. 2017;11(June):1-20. doi:10.3389/fnins.2017.00339 

25.  
Carlyon RP, Deeks JM. Combined neural and behavioural measures of temporal pitch perception in 
cochlear implant users. J Acoust Soc Am. 2015;138(5):2885-2905. doi:10.1121/1.4934275 
26.  
Hughes ML, Castioni EE, Goehring JL, Baudhuin JL. Temporal response properties of the auditory 
nerve:... Hear Res. 2013;285:46-57. doi:10.1016/j.heares.2012.01.010.Temporal 
27.  
Hartmann R, Topp G, Klinke R. Discharge patterns of cat primary auditory fibers with electrical 
stimulation of the cochlea. Hear Res. 1984;13(1):47-62. http://www.ncbi.nlm.nih.gov/pubmed/6546751. 
28.  
Joris PX, Verschooten E. On the Limit of Neural Phase Locking to Fine Structure in Humans. :101-108. 
doi:10.1007/978-1-4614-1590-9 
29.  
Wright BA, Zhang Y, Wright BA, Zhang Y. Linked references are available on JSTOR for this article : A 
review of the generalization of auditory learning. 2019;364(1515):301-311. doi:10.1098/rstb.2008.0262 
30.  
Mckay CM, McDermott HJ. The perceptual effects of current pulse duration in electrical stimulation of 
the auditory nerve. J Acoust Soc Am. 1999;106(2):998-1009. 
http://www.ncbi.nlm.nih.gov/pubmed/10462805. 
31.  
Goldsworthy RL. Correlations Between Pitch and Phoneme Perception in Cochlear Implant Users and 
Their Normal Hearing Peers. J Assoc Res Otolaryngol. 2015. doi:10.1007/s10162-015-0541-9 
32.  
Goldsworthy RL, Markle KL. Pediatric Hearing Loss and Speech Recognition in Quiet and in Different 
Types of Background Noise. J Speech, Lang Hear Res. 2019;62(3):758-767. doi:10.1044/2018_jslhr-h-
17-0389 
33.  
He S, Mcfayden TC, Shahsavarani BS, et al. The Electrically Evoked Auditory Change Complex Evoked 
by Temporal Gaps Using Cochlear Implants or Auditory Brainstem Implants in Children With Cochlear 
Nerve Deficiency. 2018:482-494. 
34.  
Hughes ML. A re-evaluation of the relation between physiological channel interaction and electrode 
pitch ranking in cochlear implants. J Acoust Soc Am. 2008;124(5):2711-2714. doi:10.1121/1.2990710 
35.  
Hughes ML, Abbas PJ. The relation between electrophysiologic channel interaction and electrode pitch 
ranking in cochlear implant recipients. J Acoust Soc Am. 2006;119(3):1527-1537. 
doi:10.1121/1.2163273 
36.  
Cariani P a, Delgutte B. Neural correlates of the pitch of complex tones. I. Pitch and pitch salience. J 
Neurophysiol. 1996;76(3):1698-1716. 
37.  
Cariani PA, Delgutte B. Neural correlates of the pitch of complex tones. II. Pitch shift, pitch ambiguity, 
phase invariance, pitch circularity, rate pitch, and the dominance region for pitch. J Neurophysiol. 
1996;76(3):1717-1734. doi:10.1152/jn.1996.76.3.1717 
38.  
Scheperle RA, Abbas PJ. Peripheral and central contributions to cortical responses in cochlear implant 
users. Ear Hear. 2015;36(4):430-440. doi:10.1097/AUD.0000000000000143 
39.  
Bianchi F, Carney LH, Dau T, Santurette S. Effects of Musical Training and Hearing Loss on 
Fundamental Frequency Discrimination and Temporal Fine Structure Processing: Psychophysics and 
Modeling. JARO - J Assoc Res Otolaryngol. 2019;277:263-277. doi:10.1007/s10162-018-00710-2 
40.  
Bahmer A, Gupta DS. Role of Oscillations in Auditory Temporal Processing: A General Model for 
Temporal Processing of Sensory Information in the Brain? Front Neurosci. 2018;12(October):1-12. 
doi:10.3389/fnins.2018.00793 
41.  
Bahmer A, Langner G. A simulation of chopper neurons in the cochlear nucleus with wideband input 
from onset neurons. Biol Cybern. 2009;100(1):21-33. doi:10.1007/s00422-008-0276-3 
42.  
Goldsworthy RL, Shannon R V. Training improves cochlear implant rate discrimination on a 
psychophysical task. J Acoust Soc Am. 2014;135(1):334-341. doi:10.1121/1.4835735 
43.  
Seitz AR, Watanabe T. The phenomenon of task-irrelevant perceptual learning. Vision Res. 
2009;49(21):2604-2610. doi:10.1016/j.visres.2009.08.003 
44.  
Seitz AR, Kim R, Shams L. Sound Facilitates Visual Learning. Curr Biol. 2006;16(14):1422-1427. 
doi:10.1016/j.cub.2006.05.048 
45.  
Vlahou EL, Protopapas A, Seitz AR. Implicit training of nonnative speech stimuli. J Exp Psychol Gen. 
2012;141(2):363-381. doi:10.1037/a0025014 
46.  
Green CS, Bavelier D. Video-Game Experience Alters the Spatial Resolution of Vision. 2014;18(1):88-
94. 
47.  
Xiao LQ, Zhang JY, Wang R, Klein SA, Levi DM, Yu C. Complete Transfer of Perceptual Learning 
across Retinal Locations Enabled by Double Training. Curr Biol. 2008;18(24):1922-1926. 
doi:10.1016/j.cub.2008.10.030 
48.  
Baumann U, Nobbe A. Pulse rate discrimination with deeply inserted electrode arrays. Hear Res. 
2004;196(1-2):49-57. doi:10.1016/j.heares.2004.06.008 

49.  
Vandali AE, van Hoesel RJM. Development of a temporal fundamental frequency coding strategy for 
cochlear implants. J Acoust Soc Am. 2011;129(6):4023-4036. doi:10.1121/1.3573988 
50.  
Kong Y, Deeks JM, Carlyon RP. Limits of temporal pitch in cochlear implants. 2009;(March):1649-1657. 
doi:10.1121/1.3068457 
51.  
Bernstein LR, Trahiotis C. Converging measures of binaural detection yield estimates of precision of 
coding of interaural temporal disparities. J Acoust Soc Am. 2015;138(5):EL474-EL479. 
doi:10.1121/1.4935606 
52.  
Bernstein LR, Trahiotis C. Measures of extents of laterality for high-frequency “transposed” stimuli under 
conditions of binaural interference. J Acoust Soc Am. 2005;118(3):1626-1635. doi:10.1121/1.1984827 
53.  
Bernstein LR, Trahiotis C. Accounting for binaural detection as a function of masker interaural 
correlation: Effects of center frequency and bandwidth. J Acoust Soc Am. 2014;136(6):3211-3220. 
doi:10.1121/1.4900830 
54.  
Bernstein LR, Trahiotis C. Sensitivity to envelope-based interaural delays at high frequencies: Center 
frequency affects the envelope rate-limitation. J Acoust Soc Am. 2014;135(2):808-816. 
doi:10.1121/1.4861251 
55.  
Bernstein LR, Trahiotis C. An interaural-correlation-based approach that accounts for a wide variety of 
binaural detection data. J Acoust Soc Am. 2017;141(2):1150-1160. doi:10.1121/1.4976098 
56.  
Bernstein LR, Trahiotis C. The apparent immunity of high-frequency “transposed” stimuli to low-
frequency binaural interference. J Acoust Soc Am. 2004;116(5):3062-3069. doi:10.1121/1.1791892 
57.  
Bernstein LR, Trahiotis C. Enhancing sensitivity to interaural delays at high frequencies by using 
“transposed stimuli.” J Acoust Soc Am. 2002;112(3):1026-1036. doi:10.1121/1.1497620 
58.  
Bernstein LR, Trahiotis C. The normalized correlation: Accounting for binaural detection across center 
frequency. J Acoust Soc Am. 2005;100(6):3774-3784. doi:10.1121/1.417237 
59.  
Dietz M, Bernstein LR, Trahiotis C, Ewert SD, Hohmann V. The effect of overall level on sensitivity to 
interaural differences of time and level at high frequencies. J Acoust Soc Am. 2013;134(1):494-502. 
doi:10.1121/1.4807827 
60.  
Bernstein LR, Trahiotis C. How sensitivity to ongoing interaural temporal disparities is affected by 
manipulations of temporal features of the envelopes of high-frequency stimuli. J Acoust Soc Am. 
2009;125(5):3234. doi:10.1121/1.3101454 
61.  
Kaernbach C, Bering C. Exploring the temporal mechanism involved in the pitch of unresolved 
harmonics. J Acoust Soc Am. 2001;110(2):1039-1048. doi:10.1121/1.1381535 
62.  
Kreft HA, Oxenham AJ, Nelson DA. Modulation rate discrimination using half-wave rectified and 
sinusoidally amplitude modulated stimuli in cochlear-implant users Modulation rate discrimination using 
half-wave rectified and sinusoidally amplitude modulated stimuli in cochlear-implant users. 
2015;656(2010). doi:10.1121/1.3282947 
63.  
Vandali AE, Sucher C, Tsang DJ, McKay CM, Chew JWD, McDermott HJ. Pitch ranking ability of 
cochlear implant recipients: a comparison of sound-processing strategies. J Acoust Soc Am. 
2005;117(5):3126-3138. doi:10.1121/1.1874632 
64.  
Canneyt J Van, Hofmann M, Wouters J, Francart T. The effect of stimulus envelope shape on the 
auditory steady-state response. bioRxiv. 2019:541359. doi:10.1101/541359 
65.  
Riss D, Hamzavi JS, Blineder M, et al. Effects of Stimulation Rate with the FS4 and HDCIS Coding 
Strategies in Cochlear Implant Recipients. Otol Neurotol. 2016;37(7):882-888. 
doi:10.1097/MAO.0000000000001107 
66.  
Dreyer A, Delgutte B. Frequency Sounds : Implications for Sound Localization. J Neurophysiol. 
2006;96(5):2327-2341. 
67.  
Jeng FC, Abbas PJ, Hu N, Miller CA, Nourski K V., Robinson BK. Effects of temporal properties on 
compound action potentials in response to amplitude-modulated electric pulse trains in guinea pigs. 
Hear Res. 2009;247(1):47-59. doi:10.1016/j.heares.2008.10.007 
68.  
Kreft HA, Nelson DA, Oxenham AJ. Modulation frequency discrimination with modulated and 
unmodulated interference in normal hearing and in cochlear-implant users. JARO - J Assoc Res 
Otolaryngol. 2013;14(4):591-601. doi:10.1007/s10162-013-0391-2 
69.  
Shannon R V., Jensvold A, Padilla M, Robert ME, Wang X. Consonant recordings for speech testing. J 
Acoust Soc Am. 1999;106(6):L71-4. http://www.ncbi.nlm.nih.gov/pubmed/10615713. 
70.  
Hillenbrand J, Getty LA, Clark MJ, Wheeler K. Acoustic characteristics of American English vowels. J 
Acoust Soc Am. 1995;97(5 Pt 1):3099-3111. http://www.ncbi.nlm.nih.gov/pubmed/7759650. 
71.  
Tejani VD, Abbas PJ, Brown CJ. Relationship Between Peripheral and Psychophysical Measures of 

Amplitude Modulation Detection in Cochlear Implant Users. Ear Hear. 2017;38(5):e268-e284. 
doi:10.1097/AUD.0000000000000417 
72.  
Zaltz Y, Goldsworthy RL, Kishon-Rabin L, Eisenberg LS. Voice Discrimination by Adults with Cochlear 
Implants: the Benefits of Early Implantation for Vocal-Tract Length Perception. J Assoc Res Otolaryngol. 
2018. doi:10.1007/s10162-017-0653-5 
73.  
Hossain S, Goldsworthy RL. Factors Affecting Speech Reception in Background Noise with a Vocoder 
Implementation of the FAST Algorithm. JARO - J Assoc Res Otolaryngol. 2018;19(4):467-478. 
doi:10.1007/s10162-018-0672-x 
74.  
Eisenberg LS, Shannon R V., Martinez AS, Boothroyd JW, Wygonski J. Speech recognition with 
reduced spectral cues as a function of age Speech recognition with reduced spectral cues as a function 
of age. J Acoust Soc Am. 2012;2704(2000). doi:10.1121/1.428656 
75.  
Niparko JK, Tobey EA, Thal DJ, et al. Spoken language development in children following cochlear 
implantation. JAMA - J Am Med Assoc. 2010;303(15):1498-1506. doi:10.1001/jama.2010.451 
76.  
Fink NE, Wang NY, Visaya J, et al. Childhood development after Cochlear Implantation (CDaCI) study: 
Design and baseline characteristics. Cochlear Implants Int. 2007;8(2):92-116. doi:10.1002/cii.333 
77.  
Eisenberg LS, Johnson KC, Martinez AS, et al. Comprehensive evaluation of a child with an auditory 
brainstem implant. Otol Neurotol. 2008;29(2):251-257. doi:10.1097/mao.0b013e31815a352d 
78.  
Desjardin JL, Ambrose SE, Martinez AS, Eisenberg LS. Relationships between speech perception 
abilities and spoken language skills in young children with hearing loss. Int J Audiol. 2009;48(5):248-
259. http://www.ncbi.nlm.nih.gov/pubmed/19842800. 
79.  
Markman TM, Quittner AL, Eisenberg LS, et al. Language development after cochlear implantation: An 
epigenetic model. J Neurodev Disord. 2011;3(4):388-404. doi:10.1007/s11689-011-9098-z 
80.  
Boothroyd A, Eisenberg LS, Martinez A. An On-Line Imitative Test of Speech-Pattern Contrast 
Perception (OlimSpac): Developmental Effects in Normally Hearing Children. … Speech, Lang Hear …. 
2010. http://jslhr.pubs.asha.org/article.aspx?articleid=1781555. Accessed July 18, 2014. 
81.  
Eisenberg LS, Johnson KC, Martinez AS, Visser-Dumont L, Ganguly DH, Still JF. Studies in pediatric 
hearing loss at the House Research Institute. J Am Acad Audiol. 2012;23(6):412-421. 
doi:10.3766/jaaa.23.6.4 
82.  
Eisenberg LS, Johnson KC, Martinez AS, et al. Speech recognition at 1-year follow-up in the childhood 
development after cochlear implantation study: Methods and preliminary findings. Audiol Neurootol. 
2006;11(4):259-268. doi:10.1159/000093302 
83.  
He S, Teagle HFB, McFayden TC, et al. Longitudinal changes in electrically evoked auditory event-
related potentials in children with auditory brainstem implants: Preliminary results recorded over 3 years. 
Ear Hear. 2018;39(2):318-325. doi:10.1097/AUD.0000000000000488 
84.  
He S, Grose JH, Buchman CA. Auditory discrimination: The relationship between psychophysical and 
electrophysiological measures. Int J Audiol. 2012;51(10):771-782. doi:10.3109/14992027.2012.699198 
85.  
McFayden TC, He S, Ewend M, et al. The Electrically Evoked Auditory Change Complex Evoked by 
Temporal Gaps Using Cochlear Implants or Auditory Brainstem Implants in Children With Cochlear 
Nerve Deficiency. Ear Hear. 2017:1. doi:10.1097/aud.0000000000000498 
86.  
He S, Teagle HFB, Roush P, Grose JH, Buchman CA. Objective hearing threshold estimation in children 
with auditory neuropathy spectrum disorder. Laryngoscope. 2013;123(11):2859-2861. 
doi:10.1002/lary.24137