PI: Abel, Taylor John Title: Flexible representation of speech in the supratemporal plane.

Received: 07/15/2020 Opportunity: PA-20-196 Council: 01/2021

Competition ID: FORMS-F FOA Title: NIH Exploratory/Developmental Research Grant Program (Parent R21 

Basic Experimental Studies with Humans Required)

1R21DC019217-01A1 Dual: Accession Number: 4471894

IPF: 2059802 Organization: 

Former Number: 1R21DC019217-01 Department: Neurological Surgery

IRG/SRG: LCOM AIDS: N Expedited: N

Subtotal Direct Costs 

(excludes consortium F&A)

Year 1: 

Animals: N

Humans: Y

Clinical Trial: Y

Current HS Code: 30

HESC: N

HFT: N

New Investigator: 

Early Stage Investigator: 

Senior/Key Personnel: Organization: Role Category: 

Taylor Abel MD PD/PI

Lori Holt PhD MPI
OMB Number: 4040-0010

Expiration Date: 12/31/2022

APPLICATION FOR FEDERAL ASSISTANCE

SF 424 (R&R)

3. DATE RECEIVED BY STATE State Application Identifier

1. TYPE OF SUBMISSION* 4.a. Federal Identifier

DC019217

❍ Pre-application ● Application ❍ Changed/Corrected

Application

b. Agency Routing Number

2. DATE SUBMITTED Application Identifier c. Previous Grants.gov Tracking Number

5. APPLICANT INFORMATION Organizational DUNS*: 

Legal Name*:

Department: Office of Sponsored Programs

Division:

Street1*:

Province:

Country*: USA: UNITED STATES

ZIP / Postal Code*:

Person to be contacted on matters involving this application

Prefix: First Name*: Jennifer Middle Name: E. Last Name*: Woodward Suffix: PhD

Position/Title: Vice Chancellor for Sponsored Programs

Street1*:

Province:

Country*: USA: UNITED STATES

ZIP / Postal Code*:

Fax Number: Email: 

6. EMPLOYER IDENTIFICATION NUMBER (EIN) or (TIN)*

7. TYPE OF APPLICANT* X: Other (specify)

Other (Specify): Private, non-profit, state-related, educ inst

Small Business Organization Type ❍ Women Owned ❍ Socially and Economically Disadvantaged

8. TYPE OF APPLICATION* If Revision, mark appropriate box(es).

❍ New ● Resubmission ❍ A. Increase Award ❍ B. Decrease Award ❍ C. Increase Duration

❍ Renewal ❍ Continuation ❍ Revision ❍ D. Decrease Duration ❍ E. Other (specify) :

Is this application being submitted to other agencies?* ❍Yes ●No What other Agencies?

9. NAME OF FEDERAL AGENCY*

National Institutes of Health

10. CATALOG OF FEDERAL DOMESTIC ASSISTANCE NUMBER

TITLE:

11. DESCRIPTIVE TITLE OF APPLICANT'S PROJECT*

Flexible representation of speech in the supratemporal plane.

12. PROPOSED PROJECT

Start Date* Ending Date*

04/01/2021 03/31/2023

13. CONGRESSIONAL DISTRICTS OF APPLICANT

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00 Page 1
SF 424 (R&R) APPLICATION FOR FEDERAL ASSISTANCE Page 2

14. PROJECT DIRECTOR/PRINCIPAL INVESTIGATOR CONTACT INFORMATION

Prefix: First Name*: Taylor Middle Name: Last Name*: Abel Suffix: MD

Position/Title: Assistant Professor

Organization Name*:

Department: Neurological Surgery

Division: School of Medicine

Street1*:

Province:

Country*: USA: UNITED STATES

ZIP / Postal Code*:

Fax Number: Email*:

15. ESTIMATED PROJECT FUNDING

a. Total Federal Funds Requested*

16. IS APPLICATION SUBJECT TO REVIEW BY STATE

EXECUTIVE ORDER 12372 PROCESS?*

a. YES ❍ THIS PREAPPLICATION/APPLICATION WAS MADE

AVAILABLE TO THE STATE EXECUTIVE ORDER 12372

PROCESS FOR REVIEW ON:

DATE:

b. NO ● PROGRAM IS NOT COVERED BY E.O. 12372; OR

❍ PROGRAM HAS NOT BEEN SELECTED BY STATE FOR

REVIEW

17. By signing this application, I certify (1) to the statements contained in the list of certifications* and (2) that the statements herein

are true, complete and accurate to the best of my knowledge. I also provide the required assurances * and agree to comply with

any resulting terms if I accept an award. I am aware that any false, fictitious, or fraudulent statements or claims may subject me to

criminal, civil, or administrative penalties. (U.S. Code, Title 18, Section 1001)

● I agree*

* The list of certifications and assurances, or an Internet site where you may obtain this list, is contained in the announcement or agency specific instructions.

18. SFLLL or OTHER EXPLANATORY DOCUMENTATION File Name: 

19. AUTHORIZED REPRESENTATIVE

Prefix: First Name*: Jennifer Middle Name: E. Last Name*: Woodward Suffix: PhD

Position/Title*: Vice Chancellor for Sponsored Programs

Organization Name*: University of Pittsburgh

Department: Office of Sponsored Programs

Division:

Street1*:

Province:

Country*: USA: UNITED STATES

ZIP / Postal Code*:

Phone Number*: Fax Number: Email*:

Signature of Authorized Representative*

Jennifer.Woodward

Date Signed*

07/15/2020

20. PRE-APPLICATION File Name: Mime Type:

21. COVER LETTER ATTACHMENT File Name:2020_NIH_R21_PerceptualWeights_CoverLetter_fin.pdf Mime Type: application/pdf

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 2
424 R&R and PHS-398 Specific 

Table Of Contents

SF 424 R&R Cover Page 1

Table of Contents 3

Performance Sites 4

Research & Related Other Project Information 5

Project Summary/Abstract(Description) 6

Project Narrative 7

Facilities & Other Resources 8

Equipment 10

Research & Related Senior/Key Person 11

PHS398 Cover Page Supplement 22

PHS 398 Modular Budget 24

Personnel Justification 27

Consortium Justification 28

PHS 398 Research Plan 29

Introduction to Application 30

Specific Aims 31

Research Strategy 32

PHS Human Subjects and Clinical Trials Information 38

Study 1: Flexible Representation of Speech 40

Inclusion Enrollment Reports 49

Multiple PD/PI Leadership Plan 67

Bibliography & References Cited 69

Resource Sharing Plan(s) 72

Contact PD/PI: Abel, Taylor

Table of Contents Page 3
OMB Number: 4040-0010

Expiration Date: 12/31/2022

Project/Performance Site Location(s)

Project/Performance Site Primary Location ❍ I am submitting an application as an individual, and not on behalf of

a company, state, local or tribal government, academia, or other type of

organization.

Organization Name:

Additional Location(s) File Name:

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 4
OMB Number: 4040-0010

Expiration Date: 12/31/2022

RESEARCH & RELATED Other Project Information

1. Are Human Subjects Involved?* ● Yes ❍ No

1.a. If YES to Human Subjects

Is the Project Exempt from Federal regulations? ❍ Yes ● No

If YES, check appropriate exemption number: 1 2 3 4 5 6 7 8

If NO, is the IRB review Pending? ● Yes ❍ No

IRB Approval Date:

Human Subject Assurance Number 00006790

2. Are Vertebrate Animals Used?* ❍ Yes ● No

2.a. If YES to Vertebrate Animals

Is the IACUC review Pending? ❍ Yes ❍ No

IACUC Approval Date:

Animal Welfare Assurance Number

3. Is proprietary/privileged information included in the application?* ❍ Yes ● No

4.a. Does this project have an actual or potential impact - positive or negative - on the environment?* ❍ Yes ● No

4.b. If yes, please explain:

4.c. If this project has an actual or potential impact on the environment, has an exemption been authorized or an

environmental assessment (EA) or environmental impact statement (EIS) been performed?

❍ Yes ❍ No

4.d. If yes, please explain:

5. Is the research performance site designated, or eligible to be designated, as a historic place?* ❍ Yes ● No

5.a. If yes, please explain:

6. Does this project involve activities outside the United States or partnership with international

collaborators?*

❍ Yes ● No

6.a. If yes, identify countries:

6.b. Optional Explanation:

Filename

7. Project Summary/Abstract* 2020_NIH_R21_PerceptualWeights_ProjectSummary_.pdf

8. Project Narrative* 2020_NIH_R21_PerceptualWeights_ProjectNarrativ.pdf

9. Bibliography & References Cited 2020_NIH_R21_PerceptualWeights_References_fina.pdf

10.Facilities & Other Resources 2020_NIH_R21_PerceptualWeights_Resources_final.pdf

11.Equipment 2020_NIH_R21_PerceptualWeights_Equipment_final.pdf

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 5
PROJECT SUMMARY

Speech communication plays a crucial role in conveying our thoughts to others, maintaining social ties, and 

supporting educational achievement. As a result, communication disorders that impact speech perception like 

autism, dyslexia, and hearing loss can be costly to both individuals and society. Understanding the 

neurobiological bases of speech processing is an important goal that has been hastened by invasive intracranial 

electrophysiology in neurosurgical contexts. Yet, substantial behavioral evidence demonstrates dynamic, flexible 

aspects of the mapping of speech input to phonemes that is not yet accounted for in neurobiological models. 

This Exploratory/Developmental R21 project pursues the central hypothesis that listening context systematically 

impacts cortical response to speech and therefore affects the diagnosticity of acoustic dimensions in signaling 

phonemes. A newly established cross-disciplinary research team will use intracerebral recording via 

stereoelectroencephalography (sEEG) obtained in a neurosurgical context to pursue this hypothesis. Like 

electrocorticography (ECoG), sEEG offers high spatiotemporal resolution and can target the cortical surface, 

including superior temporal gyrus (STG). Owing to the intracortical electrode placement, sEEG electrodes record 

through the supratemporal plane, specifically targeting both deep sulcal and gyral grey matter including superior 

temporal sulcus (STS) and Heschl’s gyrus (HG). Simultaneous scalp electroencephalography (EEG) will be 

acquired to link these intracortical measures with noninvasive approaches appropriate in studies of healthy 

listeners. Aim 1 will establish neural response to two acoustic-phonetic dimensions as a function of the perceptual 

weight with which they signal phoneme identity. This will provide a baseline response for each participant for 

comparison as experimental manipulations to listening context shift perceptual weights in Aim 2, and will 

establish how individual differences in perceptual weighting strategies predict cortical electrophysiological 

response. Aim 2 will introduce two well-established manipulations that, behaviorally, shift perceptual weights 

relative to baseline: introduction of noise and introduction of an ‘accent’ for which the short-term speech input 

deviates from distributional regularities of the native language. Examination of experimental manipulations 

within-participant will provide a sensitive means by which to assay changes in neural response as a function of 

changes in perceptual weights arising across listening contexts. Participants will be sampled across later 

adolescence (15-25 years), a period during which perceptual weights provide informative heterogeneity. The 

project will compound its impact by filling an important gap in understanding of speech processing, building a 

bridge from invasive electrophysiological studies with patients to scalp EEG measures of human listeners 

through combined sEEG+EEG, wedding classic and state-of-the-art computational approaches to inform 

mechanisms, and delivering an understanding of the dynamic, flexible nature of speech processing with 

substantial implications for communication disorders. 

Contact PD/PI: Abel, Taylor 

Project Summary/Abstract Page 6
PROJECT NARRATIVE

The goal of this research is to discover the fundamental mechanisms that support listeners’ ability to flexibly 

perceive speech even as listening contexts change to include foreign accents or background noise. An 

understanding of how speech perception flexibly adapts will have important implications for developing new 

rehabilitative strategies for communication disorders like aphasia, dyslexia and autism that impact speech 

perception. 

Contact PD/PI: Abel, Taylor 

Project Narrative Page 7
FACILITIES & OTHER RESOURCES

Laboratory

Children’s Hospital of Pittsburgh (CHP) provides a child-friendly environment that is ideal for care of pediatric 

patients and pediatric clinical / translational research. CHP has a fully equipped 8-bed epilepsy-monitoring unit 

(EMU) where intracranial electrophysiology experiments will take place. CHP has a neuroimaging research 

facility equipped with a 3T Siemens MR scanner. All data acquisition from patient populations will take place at 

CHP.

• Magnetic Resonance Imaging Room: CHP has a Siemens 3 Tesla MR scanner available for research. 

The magnet rooms are magnetically, acoustically, and RF shielded. 

• MRI Simulator: CHP has an MRI simulator that mimics the scanner room both in appearance and noise. 

This serves to acclimate research participants to the scanner environment. A stimulus presentation system 

similar to the one used in the scanning environment has been installed in the simulator for training children 

on experimental tasks. 

Animal: None

Office: PI Abel has a closed-door laboratory space in the basement of CHP that provides direct elevator access 

to the EMU for quick transport of research electrophysiology equipment (via cart). Additionally, within the Division 

of Neurosurgery at CHP, the PI’s lab has four cubicles equipped with computers that are allocated for his 

research team (currently a postdoctoral fellow, a research assistant, and two medical students). PI Abel also has 

his own clinical office with a computer that is on the same floor and in close proximity to laboratory research 

cubicles. 

PI Abel hosts weekly lab meetings in a conference room on the same floor as his office and research cubicles. 

This conference room is equipped with a white board and projector to facilitate data presentations. 

PI Holt has offices a short distance from CHP, on the University campus. These rooms have 

designated space for files, manuscript preparation, and overall organizational duties. The offices are equipped 

with locking file cabinets, a desk, a computer equipped with all the software available on laboratory computers, 

a telephone, and a printer to perform these functions. 

PI Holt hosts weekly lab meetings in a conference room in the same building as the offices. This room has 

presentation equipment, white boards, a projector and videoconferencing equipment. 

Other:

Scientific and Intellectual Environment

There is a vibrant, highly collaborative intellectual environment at the

 area available to the PI, Co-I, and personnel. The University of Pittsburgh is ranked 

5th among US schools in NIH funding and 1st in National Institute of Mental Health funding. Additionally, the 

 has the Clinical and Translational Science Institute which helps researchers develop and 

execute innovative programs. It benefits from being immediately adjacent to Carnegie Mellon University, the 

academic home of PI Holt. 

Children’s Hospital of Pittsburgh: Patient recruitment and patient electrophysiology experiments will take 

place at the at Children’s Hospital As a world-class academic children’s hospital ranked in 

the top 10 for NIH funding among children’s hospitals and in the top 10 by US World and News for children’s 

hospital. 

Contact PD/PI: Abel, Taylor 

Facilities & Other Resources Page 8
Department of Psychology, Carnegie Mellon University: PI Holt has a laboratory at Carnegie Mellon 

University, a short bus ride or drive from Children’s Hospital Her laboratory is suitable to 

accommodate pilot testing and paradigm development and provides ample resources to support the 

development of the graduate student trainee. 

Neuroscience Institute, Carnegie Mellon University: PI Holt is affiliated with the Neuroscience Institute of 

Carnegie Mellon University (https://www.cmu.edu/ni/), which provides resources to support collaborative 

neuroscience endeavors across academic departments and centers. 

Department of Otolaryngology: The PIs are affiliated with the Pittsburgh Hearing Research Center (PHRC 

http://www.phrc.pitt.edu/), based within the Department of Otolaryngology. The PHRC focuses on the 

interdisciplinary study of basic and clinical aspects of hearing and sound perception, in health and disease. The 

PHRC includes members from different Medical School, School of Health and Rehabilitation Sciences, and Arts 

and Sciences Departments as well as faculty from Carnegie Mellon University. The 

mission of these centers is to facilitate interdisciplinary research to provide data-driven knowledge to physicians 

and patients and to offer treatments. 

Department of Neurobiology: The department of Neurobiology currently has a T32 that supports two 

predoctoral and two postdoctoral trainees per year. The focus of this T32 is auditory and vestibular neuroscience. 

Fellows receive training in a wide range of neuroscience methodologies (cell level to systems level) and 

education in clinical populations. PI Holt is a preceptor on this training grant. 

Behavioral-Brain Training Program: PI Holt is co-Director of a T32 predoctoral training program supporting 8 

trainees/year at Carnegie Mellon University and This training program supports students 

working at the interface of brain and behavior. This program offers hands-on experiences with different 

neuroscience methods and technology and training from an interdisciplinary group of preceptors.

Center for the Neural Bases of Cognition (CNBC http://www.cnbc.cmu.edu/): The CNBC, an interinstitutional center that leverages the complementary strengths of the (in basic and 

applied neuroscience) and Carnegie Mellon in cognitive and computational neuroscience. The CNBC fosters 

cross-university research and education (affiliates can train and take coursework at both Universities) with 

experience using state-of-the-art functional neuroimaging technology, access to human and animal populations 

and research facilities. In addition to excellent resources, the center has a successful record of getting training 

grants that attract strong predoctoral and postdoctoral applicants. The postdoctoral associate and graduate 

student associated with this project will be associated with the CNBC.

Contact PD/PI: Abel, Taylor 

Facilities & Other Resources Page 9
EQUIPMENT

Children’s Hospital of Pittsburgh (CHP) provides a child-friendly environment that is ideal for care of pediatric 

patients and pediatric clinical / translational research. CHP has a fully equipped 8-bed epilepsy-monitoring unit 

(EMU) where intracranial electrophysiology experiments will take place. CHP has a neuroimaging research 

facility equipped with a 3T Siemens MR scanner. All data acquisition from patient populations will take place at 

CHP. CHP is a facility equipped to conduct cutting-edge clinical research in a medical facility, including with 

patient populations. CHP has the following equipment available for use by the MPIs: 

• Magnetic Resonance Imaging System: CHP has a Siemens 3 Tesla MR scanner available for research. 

There is associated instrumentation to handle high rates of storage required by fMRI scanning. The magnet 

rooms are magnetically, acoustically, and RF shielded. The research MR at CHP has administrative and 

technical support, including MR physicists who continually monitor and optimize the functioning of the MR 

scanners. Staff physicists and radiologists are also available for the development of new pulse sequences 

tailored to the needs of the individual investigator. The scanner has an integrated stimulus delivery system 

that has been installed for the presentation of stimuli and the detection of responses in the fMRI environment 

on each scanner. The MR scanner is also equipped with an eye tracker. 

• Electrophysiology Recording System: Nomad (Ripple, Inc., Salt Lake City, UT) 512-channel Neural 

Interface Processor (30kHz sampling rate), Grapevine touchproof adaptors (4), Grapevine digital I/O, 

Grapevine analogue I/O, and head stages (4); handgrip force transducers (2) and associate software and 

hardware (MIE medical research ltd., Leeds, UK). 

• Eye Tracking: EyeLink Portable Duo eye tracking system (SR Research, Ottawa, Ontario, Canada), with 

sampling rates of 2,000 Hz (head-stabilized mode) or 1,000 Hz (free-to-move mode). Experiment Builder and 

Data Viewer software (SR Research, Ottawa, Ontario, Canada). 

• Computing: All laboratory personnel have a personal computer. All computers are networked and have 

access to a laser printer. Analysis computers will be integrated with the Pittsburgh Supercomputing Center 

(PSC). All computers are password-protected, with data backed up daily. 

• Software: Investigators have access to MATLAB mathematical analysis software (MATLAB, The Mathworks, 

Inc., Natick, MA); JupyterHub server for Python development (Jupyter, Worldwide). Offline Sorter software 

(Plexon, Inc., Dallas, Texas); Microsoft SQL Server 2012 (Microsoft, Inc., Redmond, WA); R (R Foundation 

for Statistical Computing, Vienna, Austria). FreeSurfer MRI imaging analysis software (FreeSurfer, The 

General Hospital Corporation, Boston, MA, USA). MRIcron cross-platform NIfTI format image viewing 

software (Neuropsychology Lab, University of South Carolina, Columbia, SC, USA). Brainstorm neuroimage 

recording analysis software (Biomedical Imaging Group, University of South Carolina, Columbia, SC, USA). 

• Data Storage: Combined computational and storage machine; storage includes three 10 TB hard drives in 

RAID 5 configuration (20 TB effective storage, single drive fault tolerance) and remote backup for triply 

redundant storage. Liquid-cooled Intel Core i9-9900K CPU with 16 MB cache (3.6 GHz, 8 physical cores, 16 

threads). 64 GB HyperX DDR4 RAM at 2800/3000 MHz. 2 NVLinked Nvidia Titan RTX CUDA-enabled 

graphics cards, totaling 48 GB VRAM. 2 TB Samsung 970 EVO M.2 solid-state hard drive for I/O-intensive 

operations. Laptop computers. 

Carnegie Mellon University’s Pittsburgh Cognitive Auditory Neuroscience (PCAN) research laboratories, 

under the direction of Dr. Holt, provide full access to state-of-the-art acoustic recording, signal processing, and 

auditory experimentation facilities including sound-isolating booths. Though data collection will take place 

through CHP, the project will have full access to the resources of Dr. Holt’s CMU laboratory. 

 

Contact PD/PI: Abel, Taylor 

Equipment Page 10
OMB Number: 4040-0010

Expiration Date: 12/31/2022

RESEARCH & RELATED Senior/Key Person Profile (Expanded)

PROFILE - Project Director/Principal Investigator

Prefix:  First Name*: Taylor Middle Name  Last Name*: Abel Suffix: MD

Position/Title*:

Province:

Country*: USA: UNITED STATES

Zip / Postal Code*:

Phone Number*: 412-692-8142 Fax Number: 

E-Mail*: abelt@pitt.edu

Credential, e.g., agency login:

Project Role*:  PD/PI Other Project Role Category: 

Degree Type:  MD Degree Year:  2010

Attach Biographical Sketch*:             File Name: 2020_NIH_R21_PerceptualWeights_Biosketch_ABEL_.pdf

Attach Current & Pending Support:   File Name:

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 11
PROFILE - Senior/Key Person

Prefix:  First Name*: Lori Middle Name  Last Name*: Holt Suffix: PhD

Position/Title*: Professor

Organization Name*:

Province:

Country*: USA: UNITED STATES

Zip / Postal Code*:

Phone Number*:  Fax Number: 

E-Mail*:

Credential, e.g., agency login:

Project Role*:  PD/PI Other Project Role Category: 

Degree Type:  PhD Degree Year:  1999

Attach Biographical Sketch*:             File Name: 2020_NIH_R21_PerceptualWeights_Biosketch_HOLT_.pdf

Attach Current & Pending Support:   File Name:

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 12
OMB No. 0925-0001 and 0925-0002 (Rev. 09/17 Approved Through 03/31/2020)

BIOGRAPHICAL SKETCH

Provide the following information for the Senior/key personnel and other significant contributors.

Follow this format for each person. DO NOT EXCEED FIVE PAGES.

NAME: Taylor John Abel

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Assistant Professor of Neurological Surgery

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

include postdoctoral training and residency training if applicable. Add/delete rows as necessary.)

INSTITUTION AND LOCATION

DEGREE

(if applicable) 

Completion 

Date

MM/YYYY

FIELD OF STUDY

University of Washington, Seattle WA B.S. 6/2004 Neurobiology

University of Washington, Seattle WA M.D. 6/2010 Medicine 

University of Iowa, Iowa City IA 

University of Iowa, Iowa City IA

Residency 

Postdoctoral 

6/2016

6/2015

Neurosurgery

Cognitive 

Neuroscience 

CHU de Grenoble, France Fellow 6/2017 Epilepsy Surgery 

The Hospital for Sick Children, Toronto, Canada Fellow 6/2018 Pediatric Neurosurgery 

A. Personal Statement

I am an Assistant Professor of Neurological Surgery at the University of Pittsburgh, a pediatric neurosurgeon, 

and Surgical Director of the Pediatric Epilepsy Surgery Program at UPMC Children’s Hospital of Pittsburgh 

(CHP). I direct the Pediatric Brain Electrophysiological Laboratory (PBEL) at CHP. The goals of this project 

are to use human sEEG and EEG recordings to understand flexible speech representation in adolescent 

auditory cortex in neurosurgery patients. I have the experience, skills, and expertise necessary to successfully

perform this multi-disciplinary and multi-institutional collaborative research project. This will be supported, as 

well, by the geographic proximity and intellectual engagement of collaborators spanning auditory cognitive 

neuroscience and speech communication who have essential domain expertise and also bring project 

management expertise. My background and training are in pediatric neurosurgery, epilepsy surgery, and 

human brain neurophysiology. In my role, I work with my colleagues to design and perform experiments and to 

publish our work in the peer-reviewed literature. I have access to departmental and institutional resources 

sufficient to ensure that the needs of the PBEL and our research collaboration are met. My research laboratory 

offers the equipment and facilities necessary to conduct the clinical aims of the proposed research. The 

combination of investigators and specialized research capabilities of the PBEL and the Pittsburgh Cognitive 

Auditory Neuroscience group are uniquely suited to pursue the scientific aims of this application. 

B. Positions and Honors

Positions and Employment:

2010 – 2016 Intern, Resident, Chief Resident, Neurological Surgery, University of Iowa 

2013 – 2015 Postdoctoral Fellow, Human Brain Research Laboratory and Benton Neuropsychology 

Laboratory, University of Iowa (Mentors: Dan Tranel and Matthew Howard, III) 

2016 - 2017 Fellow Associate, Neurological Surgery, University of Iowa 

2017 Clinical Fellow, Epilepsy and Functional Neurosurgery, CHU Grenoble 

2017 – 2018 Robin and Judith Humphreys Fellow (Chief Fellow) in Pediatric Neurosurgery, The Hospital for 

Sick Children and University of Toronto 

2018 - Assistant Professor of Neurological Surgery, University of Pittsburgh 

2018 - Surgical Director, Pediatric Epilepsy Surgery Program, UPMC Children’s Hospital of Pittsburgh 

2019 - Adjunct Faculty, Department of Bioengineering, University of Pittsburgh 

Contact PD/PI: Abel, Taylor 

Biosketches Page 13
Other Experience and Professional Memberships:

2010 – 2018 Member, Congress of Neurological Surgeons 

2010 - Member, American Association of Neurological Surgeons 

2013 – 2016 Member, American Society of Functional and Stereotactic Neurosurgery 

2014 - Member; Membership Committee (2016 – present), American Epilepsy Society 

2014 - Member; North American Regional Representative, ILAE Young Epilepsy Section (2018 - ); ExOfficio Member of Executive Board of ILAE North America (2018 - ); ILAE Young Epilepsy 

Section Executive Board (2018 - ); Task Force on Research Advocacy and Priorities (2019 - ), 

International League Against Epilepsy 

2017 - Member; Clinical Guideline Committee (2019 – present), AANS / CNS Joint Section on Pediatric 

Neurosurgery

2018 - Member, Flux Society for Developmental Cognition Neuroscience 

2018 - Member, Society for Neuroscience 

2019 - Neurosurgery Representative, University of Pittsburgh Institutional Review Board (IRB) 

Honors

2005 Mary Gates Research Scholar, University of Washington 

2007 Medical Student Research Training Award, University of Washington School of Medicine 

2009 Functional Neurosurgery Top Ten Abstract, Congress of Neurological Surgeons 

2010 Third Place Abstract, Functional and Stereotactic Neurosurgery, AANS 

2014 American Epilepsy Society Fellow Travel Award 

2014 NIH Loan Repayment Award (Clinical Science) 

2014 NIH Ruth L. Kirschstein National Research Service Award 

2014 Stereotactic and Functional Neurosurgery Resident Award, Congress of Neurological Surgeons 

2014 First Place Abstract, Iowa Medical Society 

2016 NIH Loan Repayment Award Renewal 

2017 Robin and Judith Humphreys Fellow in Pediatric Neurosurgery 

C. Contributions to Science

H-index: 15, Citations: 725

Google Scholar Profile: https://scholar.google.com/citations?user=1cBCgOIAAAAJ&hl=en

1. Cortical physiology of auditory and visual convergence for person and object identification in the

human anterior temporal lobe (ATL). I have a long-standing interest in understanding the neural

mechanisms of person and object identification in the human temporal lobe, particularly the anterior temporal

regions. Lesions studies show that damage to the ATL is associated with impaired visual and auditory naming

of people, places, and things. I conducted work to describe the large-scale physiologic mechanisms

associated with visual and auditory naming in the human ATL. First, in collaboration with Dr. Hiroto Kawasaki

and Dr. Matthew Howard, we developed a specialized electrode array that provides dense coverage of the ATL

in epilepsy surgery patients. Using this array, I described convergent beta band responses to voice and face

identification in the human ATL. I further demonstrated that these beta band responses occur regardless of

ATL laterality and uniqueness of stimulus being identified. In collaboration with Dr. Chris Petov, we developed

a model of person and object-identification in the primate and human brain that involves multisensory

convergence and amodal cortical representation in the superior temporal sulcus (STS). I am continuing this

work now examining neural mechanisms of speaker identification in the human STS.

a. Abel TJ, Rhone A, Nourski KV, Granner MA, Oya H, Tranel DT, Kawasaki H, Howard MA III.

Electrodcorticographic monitoring of the temporal pole with a specialized electrode array: technique

and preliminary results. Physiol Meas, Mar 35(3): 323-37, 2014.

b. Abel TJ, Rhone AE, Nourski NV, Kawasaki H, Oya H, Griffiths TD, Howard III, MA, Tranel D. Direct

physiologic evidence of a heteromodal convergence region for proper naming in the human left anterior

temporal lobe. J Neurosci 35(4): 1513 – 1520, 2015

c. Abel TJ, Rhone AE, Nourski KV, Howard III MA, Tranel D. Investigating the anterior temporal lobe with

direct intracranial recordings. Neurosurgery 62 (suppl 1): 185 – 9, 2015.

Contact PD/PI: Abel, Taylor 

Biosketches Page 14
d. Perrodin C, Kayser C, Abel TJ, Logothetis NK, Petkov CI. Who is that? Brain Networks and 

Mechanisms for Identifying Individuals. Trends Cogn Sci. 19 (12):783-96, 2015. 

e. Abel TJ, Rhone AE, Nourski KV, Ando TK, Oya H, Kovach CK, Kawasaki H, Howard III MA, Tranel D. 

Beta modulation reflects name retrieval in the human anterior temporal lobe: An intracranial recording 

study. J Neurophysiol, April 16, 2016. 

2. Clinical outcomes of stereoelectroencephalography (sEEG), subdural grids (SDE), and other

invasive monitoring methods in epilepsy surgery. An ongoing research interest of mine is to understand 

the safest and most effective methods for invasive recording in epilepsy surgery patients. Approximately 50% 

of epilepsy surgery patients require invasive monitoring to localize the epileptic focus, but the optimal 

technique (i.e. sEEG or SDE) remains unknown. We recently published the largest systematic review 

comparing clinical outcomes of sEEG and SDE, which suggests that sEEG is associated with higher rates of 

post-resection seizure freedom and lower rates of invasive monitoring morbidity. I have also published results 

of sEEG clinical outcomes in pediatric patients. The relatively efficacy of sEEG and SDE continues to be a 

matter of debate, which I plan to continuing exploring in my clinical research studies. 

a. Remick M, Ibrahim GM, Mansouri A, Abel TJ. Patient phenotypes and clinical outcomes in invasive 

monitoring for epilepsy: An individual patient data meta-analysis. Epilepsy & Behavior. 2020 Jan 

1;102:106652.

b. Yan H, Katz JS, Anderson M, Mansouri A, Remick M, Ibrahim GM, Abel TJ. Method of invasive 

monitoring in epilepsy surgery and seizure freedom and morbidity: A systematic review. Epilepsia. 

2019 Sep;60(9):1960-1972.

b. Katz JS, Abel TJ. Stereoelectroencephalography Versus Subdural Electrodes for Localization of the 

Epileptogenic Zone: What Is the Evidence? Neurotherapeutics. 2019 Jan;16(1):59-66.

c. Abel TJ, Varela Osorio R, Amorim-Leite R, Mathieu F, Kahane P, Minotti L, Hoffmann D, Chabardes S. 

Frameless robot-assisted stereoelectroencephalography in children: technical aspects and comparison 

with Talairach frame technique. J Neurosurg Pediatr. 2018 Jul;22(1):37-46. 

e. Abel TJ, Woodroffe RW, Moritani T, Capizzano A, Kirby P, Howard MA III, Kawasaki H, Werz MA. The 

Role of the Temporal Pole in Temporal Lobe Seizure Networks: An Intracranial Electrode Investigation. 

Oct 13: 1-9, J Neurosurg, 2017

3. The role of ventromedial prefrontal cortex (vmPFC) in value based decision-making, personality, 

and quality of life in neurosurgery patients. Using a combination of lesion mapping techniques and 

experimental neuropsychology, I have examined the role of vmPFC lesions in impairment to decision making 

and personality disturbance in neurosurgical patients. Specifically, we investigated patients with olfactory 

groove meningioma who often present with subjective complaints of personality change and often have 

impairments in adaptive functioning out of proportion to the impairment measured by routine 

neuropsychological testing. We showed that patients with meningioma lesions involving the vmPFC have 

impairment in both adaptive functioning and value based decision-making (as measured by the Iowa Gambling 

Task), while patients with meningioma lesions in other brain areas do not exhibit these characteristics. We 

further showed that personality disturbance as measured by the Iowa Scales of Personality Change (ISPC) is 

associated with impairment in adaptive functioning. I have also investigated surgical lesions of the vmPFC in 

children who have undergone epilepsy surgery. 

a. Abel TJ, Manzel K, Bruss J, Belfi AM, Howard MA 3rd, Tranel D. The cognitive and behavioral effects 

of meningioma lesions involving the ventromedial prefrontal cortex. J Neurosurg. 2016 

Jun;124(6):1568-77. 

b. Barrash J, Abel TJ, Okerstrom-Jezewski K, Zanaty M, Manzel K, Bruss J, Howard MA 3rd, Tranel D. 

Acquired personality disturbances after meningioma resection are strongly associated with impaired 

quality of life. In Press, Neurosurgery. 

c. Stewart E, Abel TJ, Davidson B, Smith ML. Behaviour outcomes in children with epilepsy 1 year after 

surgical resection of the ventromedial prefrontal cortex. Neuropsychologia. 2019 Aug 6;133:107155. 

d. Abel TJ, Barrash J, Tranel D. Letter to the Editor. Neuropsychological impairment and quality of life 

after skull base meningioma resection: size and location matter. J Neurosurg. 2017 Dec;127(6):1467-

1468.

Contact PD/PI: Abel, Taylor 

Biosketches Page 15
D. Additional Information: Research Support

Prior Research Grants / Awards:

1. Principal Investigator, NIH Loan Repayment Award Renewal (2016 - 2017) $35,000/yr for loans 

“Physiologic mechanisms of naming in the anterior temporal lobe” Mentor: Matthew Howard III, MD

2. Principal Investigator, NIH NINDS F32 NRSA Training Grant (2014 - 2016) $59,888/yr 

“Electrophysiology of proper naming in the human left anterior temporal lobe” 

Mentors: Daniel Tranel, PhD and Matthew Howard III, MD 

3. Principal Investigator, NIH Loan Repayment Award (2014 - 2016) $35,000/yr for loans

“Physiologic mechanisms of naming in the anterior temporal lobe” Mentor: Matthew Howard III, MD

4. Principal Investigator, Medical Student Research Training Award (2007) $4,500 

“Electrocorticographic analysis of motor speech” Mentor: Jeffrey Ojemann, MD

5. Principal Investigator, Mary Gates Research Grant, University of Washington (2005) $3,000 

“Alterations in pial arteriolar reactivity following subarachnoid hemorrhage” Mentor: Gavin W. Britz, MD, MPH 

Contact PD/PI: Abel, Taylor 

Biosketches Page 16
BIOGRAPHICAL SKETCH

NAME: Holt, Lori L.

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Professor

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

include postdoctoral training and residency training if applicable. Add/delete rows as necessary.)

INSTITUTION AND LOCATION

DEGREE

(if 

applicable) 

Completion 

Date

MM/YYYY

FIELD OF STUDY

University of Wisconsin - Madison B.S. 1995 Psychology 

University of Wisconsin - Madison Ph.D. 1999 Psychology/Neurosci 

A. Personal Statement

As a Professor of Psychology at Carnegie Mellon University, I am affiliated with Pittsburgh’s Center for the Neural 

Basis of Cognition and CMU’s Neuroscience Institute, and University of Pittsburgh’s Center for Neuroscience. 

For the last 20 years, my laboratory has investigated human auditory cognitive neuroscience, with a focus 

understanding how humans interpret the complexity of spoken language. This research program has developed 

theory and evidence demonstrating that human speech recognition can be considered to arise from general, and 

not uniquely human or speech-specific, mechanisms. The work extends to understanding of learning and 

attention in perception of complex sounds, generally. My training includes single-unit electrophysiology, animal 

behavioral models of audition, and computational modeling, in addition to human behavioral methods across 

development. My current research program capitalizes on human psychophysics and learning paradigms in 

adults and children and incorporates human electrophysiology (EEG), neuroimaging (fMRI), and acoustic 

analyses. My research has implications for critical periods in development, for developmental disabilities 

involving language, for communication disorders, for advancing language learning in adulthood, and for research 

on computer understanding of speech. The proposed project will leverage my expertise in auditory processing 

across complex sound, cortical auditory and learning attention, and speech processing. 

In the context of the proposed project, my laboratory can lend deep expertise on perceptual weighting strategies 

in speech processing which has been a focus of our research for more than a decade. The stimuli and behavioral 

paradigms described in the proposed project were developed in my lab and have been the subject of extensive 

empirical research, computational modeling, and theoretical development. 

Over the course of my career, I have mentored more than 40 undergraduates, 8 Ph.D. students (serving on 

committees of an additional 10 students), and 5 post-doctoral researchers. As a mentor I encourage trainees to 

engage with researchers outside the laboratory, to develop a wide arsenal of methods, and to direct their skills 

to furthering science whether in or outside the academy. I am deeply committed to supporting diversity in science. 

I have developed and taught undergraduate courses in Research Methods (with hands-on laboratories), the 

Biological Foundations of Behavior, and Auditory Cognitive Neuroscience. My leadership in graduate education 

demonstrates my commitment to training and diversity. Since 2007, I have served as co-Director of the 

Predoctoral Training Program in Behavioral Brain Research (T32GM081760, with co-Director Dr. Julie Fiez, 

University of Pittsburgh), an NIH-supported initiative to train the next generation of behavioral researchers to 

employ biomedical techniques in their research. 

B. Positions and Honors

Positions 

Assistant Professor, Carnegie Mellon University, 1999-2004

Associate Professor, Carnegie Mellon University, 2004-2010

Contact PD/PI: Abel, Taylor 

Biosketches Page 17
Professor, Carnegie Mellon University, 2010-present

Honors

James McKeen Cattell Sabbatical Award, 2015 

National Academy of Sciences Troland Award, 2013

Virginia W. Toomey Award in Auditory Science, 2008 

Evie and Ron Krancer Award in Auditory Science, 2007 

Association for Psychological Science, Rising Star, 2007 

American Speech Language and Hearing Association, Dennis Klatt Research Award, 2002

National Academy of Sciences, 14th Annual Beckman Frontiers of Science Symposium, 2002 

James S. McDonnell Foundation 21st Century Scientist Award - Bridging Mind, Brain & Behavior, 2000 

Acoustical Society of America, Young Investigator Award, 1999 

C. Contribution to Science

(1) Speech perception is founded on general cognitive and perceptual mechanisms rather than 

specialized processes. My theoretical perspective derives from the idea that speech perception developed 

among speakers already in possession of perceptual and cognitive systems equipped to deal with natural 

environments. I suggest that rather than developing novel neural mechanisms with which to accommodate sound 

produced by a human vocal tract, speech perception exploits existing general perceptual and cognitive 

processes. This leads to the conclusion that the challenge of perception of speech sounds shares much with 

perception of other complex events. The essence of this approach is to determine to what extent speech employs 

general processes before postulating specialized mechanisms. This approach is unorthodox in the field of 

speech perception, for which the dominant theory has long posited a unique set of representations and processes 

to accommodate speech signals. It departs somewhat, as well, from traditional study of the auditory system. In 

particular, there has been a tradition in auditory science to work from the bottom-most levels to understand, in 

great detail, the input and output relationships of the system under highly controlled circumstances. This 

approach has provided considerable insight, particularly with respect to peripheral processing. However, a byproduct has been the implicit treatment of the auditory system as a passive decoder, implying that if we can 

understand limits of auditory resolution at various levels, we will have understood sound perception. Practically, 

this has meant that the tradition has been to examine very simple sounds in isolation. From my theoretical 

perspective, the representation and processing of speech are not fundamentally different from other complex 

auditory signals. But, neither can speech processing be accounted for by simple psychoacoustic principles. 

Instead, we must begin to integrate auditory science into a domain better described as auditory cognitive 

neuroscience, considering the richness of the acoustic (and, in fact, cross-modal) perceptual environment, its 

regularity and structure, the influences of short- and long-term experience, and the effects of higher-order 

knowledge and processing.

• Holt, L. L. & Lotto, A. J. (2010). Speech perception as categorization. Attention, Perception & 

Psychophysics, 72, 1218-1227, PMC2921848.

• Diehl, R. L., Lotto, A. J. & Holt, L. L. (2004). Speech perception. Annual Review of Psychology, 55, 149-

179.

• Holt, L. L. & Lotto, A. J. (2008). Speech perception within an auditory cognitive science framework. 

Current Directions in Psychological Science, 17, 42-46. PMC2593873.

• Lotto, A. J., & Holt, L. L. (2015). Speech perception: The view from the auditory system. Invited 

contribution to appear in G. Hickok and S. Small (Eds). The Neurobiology of Language. 

(2) Understanding auditory categorization. Categorization of complex signals is a central problem for 

cognitive science; speech categories, with their inherent complexity and multidimensionality, provide an 

exemplary opportunity to investigate categorization. Speech categories are defined by multiple acoustic cues, 

none of which is typically necessary or sufficient for marking category membership. Moreover, these cues are 

usually not perceptually equivalent; some acoustic dimensions are more perceptually significant, or more heavily 

perceptually weighted, than others. Moreover, both the set of cues signaling the category and the perceptual 

weighting of these cues is native-language-dependent. Surprisingly (given the vast literature on visual 

categorization), we know little about auditory categorization to guide understand speech categorization. My work 

is filling this gap with development of novel methods, empirical behavioral research in speech and nonspeech 

auditory category learning, examination of impaired populations, and neuroimaging. This is important because 

Contact PD/PI: Abel, Taylor 

Biosketches Page 18
many disorders of communication are characterized by poor speech categorization. Understanding the learning 

mechanisms that underlie auditory category learning contributes more refined models of speech communication, 

and has transparent clinical implications. 

• Gabay, Y., Dick, F., Zevin, J. D., & Holt, L. L. (2015). Incidental auditory category learning. Journal of 

Experimental Psychology: Human Perception & Performance, 41, 1124-1138. PMC4516559. 

• Lim, S, -J., Lacerda, F., & Holt, L. L. (2015). Discovering functional units in continuous speech. Journal 

of Experimental Psychology: Human Perception & Performance, 41, 1139-1152. PMC4601578. 

• Lim, S. –J. & Holt, L. L. (2011). Learning foreign sounds in an alien world: video game training improves 

non-native speech categorization. Cognitive Science, 35, 1390-1405. PMC3166392.

• Leech, R., Holt, L. L., Devlin, J. T., Dick, F. (2009). Expertise with artificial non-speech sounds recruits 

speech-sensitive cortical regions. Journal of Neuroscience, 29, 5234 –5239. PMC2747609.

(3) Context dependence in auditory processing. Complex sounds like speech are not perceived simply as a 

sum total of their frequency and loudness limens. Moreover, sounds presented in sequence interact in perceptual 

processing, so understanding the representation of each in isolation does little to describe how they are 

perceived in context. In my view, perceptual systems seek out structure in the world and respond to changes in 

the environment. As a consequence, context must be considered a critical component in perceptual processing. 

The “interference” introduced by context is not disruptive of perceptual processing, but fundamental to its 

adaptive operation in developing and constraining a percept from information that lasts for mere moments. I have 

contributed to understanding the multiple levels at which auditory processing rapidly tunes to incoming 

information, and applied these insights to central phenomena of speech perception, such as compensation for 

coarticulation and talker normalization. 

• Holt, L. L. (2005). Temporally non-adjacent non-linguistic sounds affect speech categorization. 

Psychological Science, 16, 305-312.

• Huang, J. & Holt, L. L. (2009). General perceptual contributions to lexical tone normalization. Journal of 

the Acoustical Society of America, 125, 3983-3994. PMC2806435.

• Holt, L. L. (2006). The mean matters: Effects of statistically-defined non-speech spectral distributions on 

speech categorization. Journal of the Acoustical Society of America, 120, 2801-2817. PMC1635014.

• Laing, E. J. C., Liu, R., Lotto, A. J., & Holt, L. L. (2012). Tuned with a tune: talker normalization via general 

auditory processes. Frontiers in Psychology, 3, 203. PMC3381219.

(4) Dissociable computational principles of subcortical regions contribute to speech learning.

Contemporary behavioral research makes clear that learning is a fundamental aspect of speech comprehension. 

Listeners learn to map highly variable acoustic speech sounds to representations that capture long-term 

regularities of the language community, such as the inventory of native-language speech sounds (phonemes) 

and words. But, they also maintain enough flexibility to adapt when speech deviates from community norms, like 

foreign-accented or noise-distorted speech. Although research increasingly emphasizes the dynamic, flexible 

nature of speech perception, core theoretical issues remain unresolved. Neurobiological models of spoken 

language have been largely focused on relatively stable aspects of cerebral cortical processing, and theoretical 

models detailing dynamic aspects of speech learning have employed learning mechanisms that are incompatible 

with biological or behavioral data. My recent research is contributing to next-generation neurobiological models 

that fully incorporate the dynamic, adaptive nature of speech perception. 

• Lim, S.-J., Fiez, J. & Holt, L. L. (2019). Role of the striatum in incidental learning of sound categories. 

Proceedings of the National Academy of Sciences, https://doi.org/10.1073/pnas.1811992116.

• Guediche, S., Holt, L. L., Laurent, P., Lim, S. J., & Fiez, J. (2015). Evidence for cerebellar contributions 

to adaptive plasticity in speech perception. Cerebral Cortex, 27, 1867-1877. PMC4481605. 

• Lim, S.-J., Fiez, J. & Holt, L. L. (2014). How may the basal ganglia contribute to auditory categorization 

and speech perception? Frontiers in Neuroscience, 8, 230. PMC4117994.

• Guediche, S., Blumstein, S., Fiez, J. A. & Holt, L. L. (2014). Speech perception under adverse conditions: 

Insights from behavioral, computational, and neuroscience research. Frontiers in Systems Neuroscience, 

7, 126. PMC3879477.

Contact PD/PI: Abel, Taylor 

Biosketches Page 19
(5) The mapping from acoustic input to auditory categories is dynamically adjusted. The dual nature of 

categorization is quite apparent in speech categorization. Efficient categorization systems must come to stably 

represent long-term environmental regularities in order to generalize effectively to new information. Yet, they 

must also maintain enough flexibility to adapt to short-term deviations from the norm. Balance is essential as it 

would be disadvantageous to be either entirely plastic or entirely inflexible. How this crucial balance is achieved 

in perceptual categorization is not well understood. In general, little attention has been directed to understanding 

how established category representations interact with local environmental regularities to influence online 

categorization, in any domain or modality. The dual nature of categorization presents itself clearly in speech. In 

recent work, I have made contributions to understanding the establishment of long-term perceptual weights for 

phonetic categories and to understanding how listeners dynamically adjust these weights in response to shortterm input that deviates from long-term norms. 

• Zhang, X. & Holt, L. L. (2018). Simultaneous tracking of co-evolving distributional regularities in speech. 

Journal of Experimental Psychology: Human Perception & Performance, 44, 1760-1779.

• Idemaru, K. & Holt, L. L. (2013). The long developmental trajectory of children’s perception and 

production of English /r/-/l/. Journal of the Acoustical Society of America, 133, 4232-4246. PMC3689790.

• Idemaru, K. Holt, L. L., & Seltman, H. (2012). Individual differences in cue weights are stable across time: 

the case of Japanese stop lengths. Journal of the Acoustical Society of America, 132, 3950-3964. 

PMC3528741.

• Idemaru, K. & Holt, L. L. (2011). Word recognition reflects dimension-based statistical learning. Journal 

of Experimental Psychology: Human Perception & Performance, 37, 1939-1956. PMC3285244.

Complete List of Published Work in MyBibliography: 

http://www.ncbi.nlm.nih.gov/sites/myncbi/lori.holt.1/bibliography/41159142/public/?sort=date&direction=ascending

D. Research Support

Ongoing Research Support

T32GM081760 (Fiez/Holt) 07/01/2007-06/30/2022 

National Institutes of Health (NIGMS) 0.5 Calendar

Predoctoral Training Program in Behavioral Brain Research (Institutional T32) $1,485,792

The overall goal of this training program is to train the next generation of behavioral science researchers who 

can skillfully incorporate neuroscience perspectives and methods into their programs of research, based on an 

understanding of brain structure and function that bridges across traditional areas of behavioral research. 

Overlap: None 

BCS_xxx (Holt/Dick) 

National Science Foundation 03/01/2020-02/28/2024

Incidental learning across statistically-structured input in active tasks. 

Recommended for funding/pending

This project investigates how listeners acquire auditory and speech categories incidentally in the course of 

performing other tasks. 

Overlap: None 

BCS_xxx (Holt) 

National Science Foundation 03/01/2020-02/28/2024

Doctoral Dissertation Research: Mechanisms of adaptive plasticity in speech perception $10,800

Recommended for funding/pending 

This project supports dissertation research related to dimension-based statistical learning carried out by 

Charles Yunan Wu. 

Overlap: None

R01DC017734 (Holt)

National Institutes of Health (NIDCD) 01/01/2020-12/31/2025

Contact PD/PI: Abel, Taylor 

Biosketches Page 20
Dimension-based auditory selective attention $2,334,776

This project investigates human listeners ability to direct attention to specific frequency bands in complex 

sounds. 

Overlap: None 

R13DC018243 (Holt)

National Institutes of Health (NIDCD) 01/01/2020-12/31/2022 

Symposium on Cognitive Auditory Neuroscience $74,131

This grant supports a conference on cognitive auditory neuroscience, to be hosted in Pittsburgh in 2020 and 

2022.

Overlap: None 

Binational Science Foundation (Holt) 05/01/2016-04/30/2019

0.25 Calendar

$151,200

The dynamics of procedural auditory category learning in developmental dyslexia 

The main project goal is examination of online learning, consolidation and retention in children with 

developmental dyslexia and matched controls 

Overlap: None 

BCS1655126 (Holt) 04/15/2017-03/31/2021

National Science Foundation 2.0 Calendar

 $974,755

Trajectories of acquisition and consolidation in incidental auditory category learning 

The main project goal is examination of the developmental course of incidental auditory category learning in 

typical children and those with dyslexia 

Overlap: None 

4500002827 09/01/2018-08/31/2020

Department of Defense (Tager-Flusberg) 0.25 Calendar

 $79,986

A novel intervention for training auditory attention in adolescents with autism spectrum disorder (ASD). 

The main project goal is development of software to support auditory attention training in ASD. 

Overlap: None 

Contact PD/PI: Abel, Taylor 

Biosketches Page 21
PHS 398 Cover Page Supplement

OMB Number: 0925-0001

Expiration Date: 02/28/2023

1. Vertebrate Animals Section

Are vertebrate animals euthanized? ❍ Yes ❍ No

If "Yes" to euthanasia

Is the method consistent with American Veterinary Medical Association (AVMA) guidelines?

❍ Yes ❍ No

If "No" to AVMA guidelines, describe method and provide scientific justification

2. *Program Income Section

*Is program income anticipated during the periods for which the grant support is requested?

❍ Yes ● No

If you checked "yes" above (indicating that program income is anticipated), then use the format below to reflect the amount and

source(s). Otherwise, leave this section blank.

*Budget Period *Anticipated Amount ($) *Source(s)

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 22
PHS 398 Cover Page Supplement

3. Human Embryonic Stem Cells Section

*Does the proposed project involve human embryonic stem cells? ❍ Yes ● No

If the proposed project involves human embryonic stem cells, list below the registration number of the specific cell line(s) from the

following list: http://grants.nih.gov/stem_cells/registry/current.htm. Or, if a specific stem cell line cannot be referenced at this time,

check the box indicating that one from the registry will be used:

Specific stem cell line cannot be referenced at this time. One from the registry will be used.

Cell Line(s) (Example: 0004):

4. Human Fetal Tissue Section

*Does the proposed project involve human fetal tissue obtained from elective abortions? ❍ Yes ● No

If "yes" then provide the HFT Compliance Assurance

If "yes" then provide the HFT Sample IRB Consent Form

5. Inventions and Patents Section (Renewal applications)

*Inventions and Patents: ❍ Yes ● No

If the answer is "Yes" then please answer the following:

*Previously Reported: ❍ Yes ❍ No

6. Change of Investigator/Change of Institution Section

❏

Change of Project Director/Principal Investigator

Name of former Project Director/Principal Investigator

Prefix:

*First Name:

Middle Name:

*Last Name:

Suffix:

❏ Change of Grantee Institution

*Name of former institution:

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 . Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 23
OMB Number: 0925-0001

Expiration Date: 02/28/2023

PHS 398 Modular Budget

Budget Period: 1

Start Date: 04/01/2021 End Date: 03/31/2022

A. Direct Costs Funds Requested ($)

Direct Cost less Consortium Indirect (F&A)*

B. Indirect (F&A) Costs

Indirect (F&A) Type Indirect (F&A) Rate (%) Indirect (F&A) Base ($) Funds Requested ($)

1.

2.

3.

4.

Cognizant Agency

(Agency Name, POC Name and Phone Number)

U.S. Department of Health and Human Services Steven

Zuraf 301-492-4855

Indirect (F&A) Rate Agreement Date 05/13/2020 Total Indirect (F&A) Costs

C. Total Direct and Indirect (F&A) Costs (A + B) Funds Requested ($)

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date: 2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 24
PHS 398 Modular Budget

Budget Period: 2

Start Date: 04/01/2022 End Date: 03/31/2023

A. Direct Costs Funds Requested ($)

Direct Cost less Consortium Indirect (F&A)*

B. Indirect (F&A) Costs

Indirect (F&A) Type Indirect (F&A) Rate (%) Indirect (F&A) Base ($) Funds Requested ($)

1.

2.

3.

4.

Cognizant Agency

(Agency Name, POC Name and Phone Number)

U.S. Department of Health and Human Services Steven

Zuraf 301-492-4855

Indirect (F&A) Rate Agreement Date 05/13/2020 Total Indirect (F&A) Costs

C. Total Direct and Indirect (F&A) Costs (A + B) Funds Requested ($)

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date: 2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 25
PHS 398 Modular Budget

Cumulative Budget Information

1. Total Costs, Entire Project Period

Section A, Total Direct Cost less Consortium Indirect (F&A) for Entire Project Period ($)

2. Budget Justifications

Personnel Justification Personnel_Justification.pdf

Consortium Justification NIH_R21_CMU_Budget_Justification.pdf

Additional Narrative Justification

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date: 2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 26
BUDGET JUSTIFICATION

SENIOR / KEY PERSONNEL

Multiple Principal Investigator, Taylor J. Abel, MD, 5% Effort / 0.60 Calendar Months

Dr. Abel is an Assistant Professor and Surgical Director of the Pediatric Epilepsy Surgery Program at

 Medical Center (UPMC) Children’s Hospital of Pittsburgh. His clinical expertise is in pediatric 

neurosurgery, with a focus on epilepsy surgery and steroelectroencephalograpy (sEEG). Dr. Abel’s primary 

research interest is understanding how human auditory cortex mediates voice and speech perception. For this 

project, Dr. Abel will oversee all sEEG research activities, including project development, simultaneous EEG and 

sEEG data acquisition, processing, analyses, interpretation, and dissemination of results via presentations and 

manuscripts. He will also supervise postdoctoral researchers and project personnel at UPMC Children’s Hospital 

of Pittsburgh.

OTHER PERSONNEL

Postdoctoral Fellow, TBA, 100% Effort / 12 Calendar Months

A postdoctoral research associate with training in sEEG and EEG methods will be hired to support the PI and 

the research team on implementation of planned research activities at UPMC Children’s Hospital of Pittsburgh. 

Specifically, the postdoctoral research associate will be responsible for supporting data acquisition, processing, 

and analyses of sEEG and EEG data. Additional responsibilities will include assisting with participant recruitment, 

ensuring ethical and consistent implementation of research activities, and dissemination of study findings. 

Research Coordinator, TBA, 50% Effort / 6.00 Calendar Months

A Bachelor’s level research coordinator will be hired to oversee administrative management of the project at 

UPMC Children’s Hospital of Pittsburgh. The research coordinator will oversee recruitment of patients, IRB 

management, including ensuring adherence to IRB regulations for all research procedures, and ensuring 

compliance will all research regulations at UPMC Children’s Hospital of Pittsburgh. 

SUPPLIES

Funds are requested for supplies both year 1 and year 2, with lower supply costs in year 2 anticipated with 

completion of patient-subject enrollment before the end of the year. 

Contact PD/PI: Abel, Taylor 

Personnel Justification Page 27
BUDGET JUSTIFICIATION 

CARNEGIE MELLON UNIVERSITY 

SENIOR / KEY PERSONNEL 

Principal Investigator, Lori L. Holt, Ph.D. 

Dr. Lori Holt is a Professor of Psychology at Carnegie Mellon University with an appointment in 

the university’s Neuroscience Institute and the Department of Modern Languages, a courtesy 

faculty appointment at the Department of Neurosciences and Center for 

Neuroscience, graduate training program. Her expertise is in human 

auditory cognitive neuroscience, with emphasis on speech communication. Along with Dr. Abel, 

Dr. Holt will be responsible for managing all aspects of the project, including crafting and executing 

the experiments, supervising the junior researchers and staff, managing data, and dissemination 

of results. She will work closely with Dr. Abel to supervise all aspects of the project, including 

scientific and management oversight and manuscript preparation. Dr. Holt will be the contact PI 

for grant reporting. The CMU fringe benefit rate is 24.5%, 5% Effort. 

OTHER PERSONNEL 

Graduate Student, TBA 

Partial stipend support (29%) is requested to support the involvement of a graduate student 

pursuing the PhD in the research. The student will meet one-on-one with Drs. Holt and Abel to 

discuss project goals, with additional meetings as necessary to support her/his training. The 

student will work closely with the postdoctoral scholar toward project aims. The student will be 

recruited through the CMU Department of Psychology PhD, the Cognitive Neuroscience PhD, or 

the Program in Neural Computation PhD graduate programs. 100% appointment with stipend, 

tuition and fees, 24.5% fringe benefits. 

Indirect Costs

Overhead on this proposal has been calculated at our current proposed or negotiated rate for all 

fiscal years in accordance with OMB Circular A-21, Section G.7.The modified total direct cost 

base (MTDC) amount used in calculating the indirect costs is the total direct costs, excluding 

capital equipment (items over $5,000 per unit), graduate student tuition remission, and 

individual subcontract costs in excess of $25,000.

Overhead Rate: 55.4% 

Contact PD/PI: Abel, Taylor 

Consortium Justification Page 28
 

PHS 398 Research Plan

OMB Number: 0925-0001

Expiration Date: 02/28/2023

 Introduction

1. Introduction to Application

(for Resubmission and Revision applications)

2020_NIH_R21_PerceptualWeights_Introduction_fi.pdf

Research Plan Section

2. Specific Aims 2020_NIH_R21_PerceptualWeights_SpecificAims_fi.pdf

3. Research Strategy* 2020_NIH_R21_PerceptualWeights_ResearchStrateg.pdf

4. Progress Report Publication List

Other Research Plan Section

5. Vertebrate Animals

6. Select Agent Research

7. Multiple PD/PI Leadership Plan 2020_NIH_R21_MultiPIPLAN_final.pdf

8. Consortium/Contractual Arrangements

9. Letters of Support

10. Resource Sharing Plan(s) 2020_NIH_R21_PerceptualWeights_DataSharing_fin.pdf

11. Authentication of Key Biological and/or

Chemical Resources

Appendix

12. Appendix

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 29
Contact PD/Pl: Abel, Taylor 

INTRODUCTION TO THE RESUBMISSION. We wish to thank the panelists for their thoughtful reviews and the 

constructive criticism that has led us to thoroughly revise the proposal. We are very pleased the panel found our 

proposal to be "excellent" with the "potential for high impact." Panelists collectively appreciated the strength of 

the design, the strong behavioral support for the hypotheses, and the targeted tasks. They found combined 

stereoelectroencephalography (sEEG) and scalp electroencephalography (EEG) to be an innovation that will 

build a bridge to non-invasive human studies. These project elements remain at the center of the revised 

proposal. Reviewers expressed some concern about project feasibility. With sincere appreciation for the detailed 

communication of this concern to us, we have revised the application in the following ways: 

        (1) New EEG pilot data demonstrate feasibility. W

collected new scalp EEG data from teenagers and young adults 

     using the Aim 2 stimuli/task. Behavioral results demonstrate t

     the diagnosticity of acoustic dimensions like fundamental      frequency (F0) in speech categorization is modu

 statistical regularities across short-term speech input (e.g.,  foreign accent, Reverse block, Figure 2). Figure A shows tha

Figure A, Pilot EEG. Difference in event-related potentials there is corresponding modulation of cortical auditory-stimulus­ for two stimuli varying in F0 as a function of short-term speech 

regularities (Canonical, Reverse). Differences in MMN (A), evoked responses to voice onset time (VOT)-ambiguous Test 

N100 amplitude (B), and P300 amplitude (C), are apparent in stimuli differentiated only by fundamental frequency (F0; Canonical, but not Reverse blocks. N=23, mean 20 yrs old. blue/green symbols, Figure 2a) across three EEG measures: 

mismatch negativity (MMN, passive listening), N100 amplitude (overt categorization), and P300 amplitude (overt 

categorization). Test stimuli varying in F0 were less well-differentiated in neural response in the context of a 

short-term regularity with an 'accent' (Reverse), compared to when short-term regularities mirrored English 

(Canonical), a result consistent with behavioral down-weighting of F0 in the Reverse block (Figure 2b). Thus, 

scalp EEG with relatively low SNR (relative to sEEG) is sensitive to dynamic change in perceptual weights across 

55 evoked potentials widely held to originate from distinct cortical generators_ 53

- These pilot results underscore 

project feasibility and suggest that the greater sensitivity and localization afforded by intracerebral sEEG will 

important yield important insights in how speech is flexibly represented across the supratemporal plane (STP). 

(2) Stimulus specificity. By design, we test our seven hypotheses across a simple, but powerful, behavioral 

model backed by extensive prior research.24-

29 Our prior and ongoing behavioral and EEG research 

demonstrates that these effects scale to other speech contrasts, continuous speech, and to passive listening. 

Thus, the approach balances careful control with evidence for future scalability. This 'sweet spot' is chosen 

intentionally to deliver foundational data with the potential to deeply inform future research. 

(3) New sEEG pilot data demonstrate sensitivity to F0 and VOT acoustic dimensions, supporting project 

feasibility. This particular stimulus testbed also pairs particularly well with sEEG. In new sEEG pilot data (Figure 

3) we demonstrate that 7 of 7 patients exhibited at least one STP channel with significant graded variation in 

high-gamma activity to both VOT and F0 dimensions. We note that this is likely to be a conservative estimate as 

these pilot data were collected during passive listening to continuous speech: pre-lexical representations tend to 

be amplified in overt speech categorization tasks like those proposed here. Finally, we note that prior research 

59 demonstrates graded scalp-recorded N100 responses to stimuli varying across VOT56

•

57 and F0.58

•

(4) Cortical Sampling with sEEG. Reviewers expressed concern about sparse cortical sampling with sEEG, 

but electrodes will be situated in stereotyped, constrained positions that always include an individual's planum 

polare, Heschl's gyrus, and planum temporale along the STP. This provides a consistent assay of anatomofunctional regions across subjects that, as evident in new sEEG pilot data (Figure 3), exhibit sensitivity to F0 

and VOT dimensions. This presents an ideal testbed for our seven hypotheses, especially leveraging the power 

of within-subject/within-electrode responses to the same speech stimuli presented across distinct experimental 

contexts that have been shown to influence perceptual weights. It is important to note that the project also will 

yield rich data from sEEG electrodes placed across cortex in the same subjects (Figure 4) that, while not the 

specific focus of this 2-year-long R21 project, will support rich secondary hypotheses sure to inform future work. 

(5) Under-specification. Aspects of our approach were underspecified in the original proposal. Here, we provide 

much more detail about Inclusion/Exclusion Criteria (see Human Subjects) and Patients Meeting Criteria Over 

the Last 2 Years (see Human Subjects, Table I), Statistical Analysis (see Statistical Design and Approach), 

Clinician/Scientist Conflict of Interest (see Data Safety and Monitoring Plan), Age Range (see Approach, 

Human Subjects, Inclusion of Women, Minorities and Children), Clinical Rationale for STP Electrodes 

(Human Subjects), and Power (Approach) that attest to the feasibility of the proposed research. 

Introduction to Application Page 30 
SPECIFIC AIMS. Understanding spoken language is predicated on clarifying how the detailed features of speech 

influence neural representation. In recent years, intracranial electrophysiology has supported rapid advancement 

in understanding human cortical speech processing to reveal that fundamental acoustic-phonetic dimensions 

are represented in neural response across superior temporal gyrus (STG). This moves us forward in ‘cracking 

the speech code.’ Yet, contemporary behavioral research demonstrates that acoustic-phonetic speech 

dimensions do not stably map to phonemes. Instead, the multiple acoustic dimensions that signal phonemes 

carry different perceptual weights that rapidly and flexibly shift according to listening demands and context. The 

very input dimensions that inform speech recognition are flexible, not fixed, and we do not yet know how this 

relates to cortical response. Filling this gap is especially crucial since nonnative speakers of English outnumber 

native speakers 3 to 1 worldwide. Thus, flexible mapping across the systematic variability of accents and dialects 

is the norm – not an exception – and is an important factor in communication disorders. Our scientific premise is 

that next-generation neurobiological models must account for adaptive plasticity – the dynamic, flexible mapping 

of speech acoustics to perception. 

Our central objective is to break new ground in understanding how human cortical response to speech represents 

perceptual weights of acoustic dimensions signaling phonemes, and how the neural code changes as perceptual 

weights flexibly adjust under different listening demands. We take a unique approach that marries classic and 

cutting-edge methods and the complementary strengths of a collaborative research team. Using intracerebral 

recordings via stereoelectroencephalography (sEEG) obtained in neurosurgical procedures, we will assess 

neural representation not just across STG but also (owing to the depth of sEEG electrodes) through the 

supratemporal plane (STP) that includes superior temporal sulcus (STS) and Heschl’s gyrus (HG). Additionally, 

we will acquire simultaneous scalp electroencephalography (EEG) allowing us to link intracerebral recording with 

long-standing approaches appropriate for healthy listeners. Using a well-studied suite of behavioral paradigms, 

we will precisely manipulate acoustic dimensions, and the short-term speech regularities in which they are 

experienced, to influence the diagnosticity, or perceptual weight, of acoustic-phonetic dimensions in signaling 

phonemes. 

Aim 1 will establish whether neural responses across the supratemporal plane (STP) reflect the 

perceptual weight of acoustic dimensions. Although multiple acoustic dimensions contribute to speech 

recognition, they do not contribute equally. Some carry more perceptual weight than others and these baseline 

perceptual weights appear to be established by long-term experience with speech input distributional regularities. 

Prior studies demonstrate the neural validity of acoustic-phonetic dimensions across the STG, but have not 

addressed changes in representation as a function of perceptual weight. We predict that response across the 

STP will be modulated as a function of the perceptual weight of a dimension in signaling a particular phoneme. 

Aim 1 tests this by examining the neural representation of two acoustic-phonetic dimensions as a function of 

their behaviorally-assessed perceptual weight across the STP, including superior temporal sulcus (STS) and 

Heschl’s gyrus (HG). Examining behavior, sEEG, and scalp EEG simultaneously will leverage the sensitivity and 

localization afforded by intracerebral sEEG to better understand neural representation of perceptual weights in 

speech categorization, while also building doing crucial groundwork to link back to noninvasive approaches 

suitable outside a neurosurgical context. 

Aim 2 will test whether neural responses in the supratemporal plane flexibly adjust. Changes in context, 

background noise, and even statistical regularities of speech across time (e.g., a foreign accent) influence the 

perceptual weights of acoustic dimensions, demonstrating the dynamic nature of the mapping of input to speech 

representations. Aim 2 will examine behavioral speech categorization, sEEG, and scalp EEG in the participants 

for whom baseline perceptual weights are measured in Aim 1. Our approach will be to use two well-established 

behavioral manipulations that lead to shifts in perceptual weights relative to baseline: introduction of noise and 

introduction of an ‘accent’ for which the short-term speech input deviates from distributional regularities of the 

native language. We predict these manipulations will shift neural responses relative to those observed at 

baseline in Aim 1 in a manner directionally predictable according to shifts in behavioral perceptual weights. Here, 

as in Aim 1, simultaneous scalp EEG will complement sEEG to connect invasive methods in clinical populations 

and noninvasive measures available in studying healthy listeners. 

Across aims, our pilot behavioral, EEG, and sEEG data support project feasibility. This 

Exploratory/Developmental R21 project will be to establish a means by which to fill a crucial gap in understanding 

speech representation in human cortex, with the possibility of radically shifting existing conceptual models and 

substantially advancing current understanding. As well, the research will build a bridge between invasive and 

noninvasive human electrophysiology. Further, it will establish a new research team with complementary 

strengths spanning neurosurgery, speech perception, and auditory cognitive neuroscience. 

Contact PD/PI: Abel, Taylor 

Specific Aims Page 31
SIGNIFICANCE. A simple utterance like beach is distinguished from its near-neighbor peach by as many as 16 

acoustic input dimensions.1

 The details of how these dimensions are expressed varies as a function of whether 

beach is part of a story told by John or Mary, whether the talker speaks British or American English, and even 

whether the storytelling venue is quiet or noisy. Research directed at understanding speech comprehension has 

long grappled with how complex acoustic input relates to native-language representations for phonemes, the 

linguistically distinct units of sound that differentiate meaning like the [b] and [p] in beach versus peach. 

Intracranial neurosurgical techniques have rapidly advanced understanding of human cortical response to 

speech. There is now robust evidence that superior temporal gyrus (STG) plays a crucial role in extracting 

meaningful linguistic features from speech input, with consonant and vowel acoustic-phonetic dimensions 

apparent in the tuning of neural populations.2,3

 This success can give the impression, at least implicitly, of a 

stable mapping from acoustic input dimensions to native-language speech representations, much like had been 

the starting point of traditional theoretical accounts.4

 However, a rich behavioral research literature now

demonstrates that the mapping of acoustic input dimensions to speech representations varies substantially 

across listeners and is malleable according to listening context. This adaptive plasticity – the dynamic, flexible 

mapping of speech acoustics to perception – is not yet well integrated into models of cortical speech processing. 

This proposed Exploratory/Developmental R21 project will lay the groundwork for developing new 

neurobiological models that accommodate the following well-established characteristics of speech processing:

(1) Acoustic dimensions do not signal phonemes with equal ‘perceptual weight’. Although the simple 

distinction between beach and peach can involve covariation among as many as 16 acoustic dimensions, these 

dimensions to not contribute equally. Some more robustly signal speech category identity than others: they carry 

more perceptual weight.

5–11 For example, among the dimensions signaling beach vs. peach, voice onset time 

(VOT, the time between consonant release and the onset of voicing in the vowel) tends to carry greater 

perceptual weight in the sense that it is more strongly associated with category identity than fundamental 

frequency (F0, the frequency of voicing in the vowel), which also covaries with [b] vs. [p] and somewhat less 

strongly signals category identity. This is apparent when speech acoustics are parametrically manipulated across 

VOT and F0 to vary perceptually between beach and peach and the contribution of each dimension to 

categorization responses is estimated using regression models. On average, listeners rely predominantly upon 

VOT in quiet, with F0 signaling category membership to a lesser extent (Figure 1a,b). Our understanding of 

cortical response to speech does not yet reflect perceptual weight, the relative 

strength of the influence of sensory input on speech categorization. 

(2) Individuals exhibit stable perceptual weights within a listening context, but 

weights shift substantially across contexts. Whereas VOT carries greater 

perceptual weight for beach vs. peach in quiet, F0 more effectively signals category 

identity in modest noise (Figure 1c,d).12 Perceptual weights are labile; perception 

relies on different input dimensions in different listening contexts. As a result, 

understanding the representation of VOT (or any acoustic dimension) in a single 

context takes us only part of the way to understanding how the auditory system maps 

the complex acoustics of speech to meet the communicative demands of everyday 

listening. A better understanding of the dynamic, flexible nature of speech 

processing will advance understanding of communication in listening conditions that 

are typical of natural environments and will be especially important for understanding 

listeners with hearing loss and communication disorders, for whom perceptual 

weights often differ from healthy listeners at baseline.13

(3) Individuals differ in baseline perceptual weights. An added complication is 

that individuals differ in baseline perceptual weights (Figure 1e). These differences 

are stable across time suggesting that they reflect underlying processing rather than 

measurement fluctuation.14 For example, even among listeners who consistently 

weight VOT more than F0 there is considerable individual variation in the extent to 

which individuals rely upon F0 in signaling contrasts like beach vs. peach.

15–19 Thus, 

careful examination of variation in baseline perceptual weights can inform 

understanding of cortical response to speech, especially in samples with a good 

degree of heterogeneity in perceptual weights, like those shown in Figure 1e. 

(4) Teenagers and young adults exhibit heterogeneity in baseline perceptual 

weights. Phonetic category development continues well into later adolescence20,21

such that perceptual weights across acoustic dimensions are not yet fully adultFigure 1. Pilot Data. Clear 

Speech (left) and Speech-inNoise (right) varying acoustically 

across F0 and VOT and 

perceptually from beach (white)

to peach (blue). (A, C), 

Categorization over orthogonal 

acoustic dimensions provides 

data from which to calculate 

baseline perceptual weights, 

shown as relative perceptual 

weights in (B, D). Listening 

context (clear, speech-in-noise) 

causes the same listeners to rely 

upon different acoustic 

dimensions to categorize 

speech; VOT is dominant in 

clear speech whereas F0 is 

dominant in noise. (E), These 

same teenage and young-adult 

subjects (mean 20yrs) exhibit

substantial heterogeneity in 

weights in each context.

Contact PD/PI: Abel, Taylor 

Research Strategy Page 32
like.14,20 In this regard, examination of speech processing among adolescents offers an opportunity to observe 

informative perceptual weight heterogeneity (as in Figure 1e, mean 20 yrs). The proposed research targets 15- 

to 25-year-old listeners to capitalize on this heterogeneity, with the premise that graded differences in perceptual 

weight will relate to graded differences in neural representation. In the proposed project, the richness of this 

testing ground is amplified by pairing natural heterogeneity in baseline perceptual weights (Aim 1) with 

experimental manipulation of context in the same listeners to evoke dynamic adjustments to perceptual weight 

(Aim 2, Figure 1b,d) to examine the impact on neural response. 

(5) Short-term input regularities also affect perceptual weights. We regularly encounter ‘non-standard’ 

pronunciations. A talker may have a strong accent, or a stuffy nose. Although comprehension initially suffers 

when speech departs from the norms of a listener’s language-community, the perceptual system rapidly adapts 

and comprehension improves. The mapping from acoustics to linguistic representations flexibly accommodates 

foreign accents22, signal distortions23, and audio-visual mismatch24. This adaptive plasticity appears to happen 

at a pre-lexical level of processing.23 In a broad sense, the very acoustic dimensions that signal speech 

representations are dynamically, and rapidly, adjusted in online speech processing to accommodate regularities 

in the ambient speech environment. The mapping of acoustic dimensions to speech representations appears 

not to be rigidly fixed by long-term experience. Rather, the ‘feature space’ serving speech recognition adapts to 

listening context. These effects showcase the need for more dynamic, flexible neurobiological models of the 

mapping from acoustics to cortical response. 

Our scientific premise is that next-generation models of human cortical speech processing must 

account for adaptive plasticity -- the dynamic, flexible mapping of speech input to perception.

Dimension-based statistical learning (DBSL) is an example of adaptive plasticity 

in speech perception with a number of properties attractive for pursuing mechanistic 

questions regarding how the perceptual weight of acoustic dimensions is reflected 

in cortical response.25–29 For example, when participants respond to stimuli that vary 

orthogonally across VOT and F0 dimensions signaling beach vs. peach, it is possible 

to measure the relative effectiveness of each dimension in signaling phoneme 

category responses (Figure 1a). In these judgments, it is evident that VOT carries 

a stronger perceptual weight in quiet and that VOT and F0 exhibit a correlation in 

their pattern of influence (higher F0s and longer VOTs are each associated with 

peach, upper right quadrant Figure 1a). As noted above, these baseline perceptual 

weights shift upon the introduction of noise (Figure 1c,d) and there are stable 

individual differences underneath these group averages (Figure 1e).

The DBSL paradigm selectively samples stimuli to manipulate short-term speech 

regularities. Instead of sampling equiprobably across the full 2-d VOTxF0 acoustic 

space to estimate baseline perceptual weights (Figure 1a), stimulus sampling is 

selective. Across Exposure trials (Figure 2a, light yellow) this short-term regularity 

can match the typical VOTxF0 correlation in English (Canonical Block, with higher 

F0s and longer VOTs for peach) or introduce an ‘accent’ with a short-term VOTxF0 

correlation that is opposite that typically experienced in English (Reverse Block, 

lower F0s with longer VOTs for peach). This artificial accent is subtle and introduced 

unbeknownst to listeners. Yet, it rapidly produces a marked shift in the perceptual 

weight of F0 in beach-peach categorization decisions. This is evident in listeners’ 

responses to Test stimuli (blue, green larger symbols in Figure 2a) that occur 

infrequently and are randomly intermixed with Exposure stimuli. Test stimuli have a 

neutral, perceptually ambiguous VOT thereby removing this dominant dimension 

(Figure 1b) from adjudicating a category identity decision. But, F0 varies across the Test stimuli. Therefore, the 

proportions of Test stimuli categorized as beach vs. peach provide a metric of the extent to which F0 is 

perceptually weighted in categorization as a function of the different short-term speech input regularities 

experienced across Canonical and Reverse blocks. Figure 2b shows that listeners rapidly re-weight F0 reliance 

upon introduction of the accent (Reverse block). Thus, the effectiveness of acoustic dimensions in signaling 

speech categories, their perceptual weight, dynamically adjusts according to short-term experience across 

acoustic input dimension regularities. 

Flexible speech representation in human cortex. Building from these well-established characteristics of 

speech processing, we propose to undertake an Exploratory/Developmental R21 project to measure cortical 

Figure 2. Dimension-based 

statistical learning (DBSL)

Pilot Data. (A) Stimuli are 

sampled selectively from the 

VOTxF0 acoustic space (yellow 

shading) to align with English 

(Canonical) or to introduce a 

short-term ‘accent’ (Reverse). 

Test stimuli (blue/green) hold 

VOT ambiguous and vary in F0, 

measuring reliance on F0 in 

categorization responses. (B) F0 

signals category identity in 

Canonical block. But, in Reverse 

block it is down-weighted in its 

utility and does not support 

differential categorization. 

Contact PD/PI: Abel, Taylor 

Research Strategy Page 33
response using invasive human neurosurgical sEEG techniques and simultaneous noninvasive scalp EEG to 

establish baseline perceptual weights across orthogonally varying VOT and F0 input dimensions in quiet (for

which VOT is expected to be dominant; Figure 1a,b), in noise (for which F0 is predicted to be dominant; Figure 

1c,d) and also in Canonical and Reverse blocks that manipulate the short-term input regularities that have been 

shown to modulate perceptual weight (Figure 2b). 

INNOVATION. We take a dynamic approach to speech processing that is not yet well-represented in 

human cortical models. Although there have been major innovations and advances in understanding the 

neurobiological bases of human speech perception since the advent of intracranial electrophysiology in human 

listeners2,3,32–35, this foundational work has tended to take a relatively static perspective regarding the mapping 

from speech input to cortical representations (see 32,36 for notable exceptions). However, as emphasized above, 

these mappings are highly task-dependent and dynamically molded by listening demands. The proposed 

research is innovative in directly examining dynamic aspects of speech processing under two contexts: in the 

adverse listening conditions of speech-in-noise and in the context of an ‘accent’ to which listeners must adapt in 

DBSL, each of which has important implications for communication disorders. 

Our use of sEEG provides unprecedented access to the supratemporal plane (STP). Ground-breaking 

intracranial EEG research has deeply informed speech coding in human cortex. With some notable 

exceptions40,41, this work has focused mostly on extra-pial intracranial recordings on the cortical surface, usually 

across the STG (electrocorticography, ECoG). However, much of auditory cortex hides deep within the STP

where important processing related to selective attention42 and learning43–45 that may support adaptive plasticity 

in speech perception cannot be accessed by ECoG 

grids. The proposed research will marry the 

advantages of well-characterized adaptive plasticity 

tasks with neurosurgical intracerebral sEEG 

recordings along the STP including electrodes that 

specifically target both sulcal and gyral grey matter 

including superior temporal sulcus (STS) and 

Heschl’s gyrus (HG). As Figure 3 illustrates, when

electrodes are placed in the STP, they are situated 

in stereotyped, constrained positions such that their 

multiple channels span an individual’s Heschl’s 

gyrus, planum polare, and planum temporale to 

provide a consistent assay of anatomo-functional 

regions of cortex across subjects (see Data Safety 

and Monitoring Plan for details). 

Testing whether speech is flexibly represented 

in STP addresses crucial theoretical debates. There are central theoretical debates of the nature of top-down 

versus bottom-up processing schemes, and whether these information sources interact bidirectionally in speech 

processing (e.g., interactive vs. feedforward37–39). Importantly, because we will take an intracerebral approach 

that provides access to early auditory cortical regions like Heschl’s gyrus (HG) this study will provide data to 

inform where in the auditory cortical hierarchy adaptive plasticity effects are apparent. Our pilot sEEG data in 

Figure 3 illustrate the feasibility of examination of speech categorization across VOT and F0 acoustic 

dimensions. High-γ (high-gamma frequency band, ~70-150Hz, a prominent neural signature in intracranial data 

that has been associated with neuron population level firing rate) activity (HGA) along the STP is graded across 

stimulus changes in F0 and VOT acoustic dimensions (Figure 3c,d). Indeed, each of 7 patients had at least one 

STP channel with HGA separable according to graded modulations in F0 and VOT (Figure 3e). By examining 

modulation of HGA across experimental contexts we have powerful, and feasible, tests within-subject/withinelectrode that will establish the extent to which regions along the STP flexibly represent speech input. 

sEEG assists in the ‘searchlight’ challenge. Inasmuch as human intracranial research must be driven first 

and foremost by clinical necessity, research faces a ‘searchlight’ challenge: examination of neural representation 

where ‘the light is shining’ by virtue of electrode coverage. In the context of this 2-year project, we direct our 

hypotheses to the STP because it has high potential for adjudicating theoretically-significant questions. However, 

it is worth noting that sEEG electrodes cover diverse cortical sites (Figure 4), presenting an opportunity to pursue 

secondary hypotheses. For example, we can investigate whether electrodes in inferior frontal cortex, implicated 

in the adaptive plasticity of speech processing in adverse listening conditions, exhibit context-sensitive HGA.60

The broader data set we collect also will establish an informative means by which to test hypotheses across 

Figure 3. New sEEG Pilot Data: 

supratemporal plane (STP) highgamma activity (HGA) reflects 

graded acoustic speech dimensions

in 7 of 7 patients. (A) Open dots show 

channels on two sEEG electrodes in 

STP. (B) Two channels near Heschl's 

gyrus (HG, diamond/square) are 

highlighted (also A). (C) F0-sensitive 

HGA, aligned to vowel midpoint for all

[ə] vowels across a natural speech 

passage, with F0 divided into quartiles 

(1-4=low-high F0). (D) Voicing-tuned 

HGA, aligned to phoneme onset. 

Unvoiced vs. voiced ([p],[t],[k] vs. 

[b],[d],[g]) tracks long vs. short voice 

onset time. (E) Each of 7 patients had at 

least one STP channel with separable 

HGA for both F0 and VOT (sliding 

ANOVAs, Bonferonni-corrected across 

patients x channels x time). 

Contact PD/PI: Abel, Taylor 

Research Strategy Page 34
‘control’ electrode sites, for example even in STP where, in our data,

planum polare channels have shown little F0- or VOT-sensitivity. 

Combined sEEG+EEG is a major innovation. The placement of 

sEEG electrodes allows for scalp electrodes to be placed 

successfully on the central sites most responsive in auditory tasks. 

This makes it possible to create an enormously informative link 

across the measurements possible with typical listeners, and 

listeners undergoing neurosurgery. Pragmatic solutions to 

combined sEEG+EEG developed in partnership with Carnegie Mellon University aid 

this innovation (Figure 5). With EEG and behavioral alignment with sEEG data, the 

present project will establish an approach to using EEG as a ‘Rosetta Stone’ with which, 

in future larger-scale projects, we aim to be able to affirm – even at the level of 

individuals – whether patient-participants conform to ‘typical’ auditory behavior, a vital 

comparison not yet well-represented in contemporary neurosurgical studies. Feasibility 

is supported by our new pilot EEG data (Figure A).

An adolescent sample supports the research aims, and opens new opportunities. Here, the study of

teenagers and young adults is a pragmatic choice not motivated by development per se, but instead chosen to

capitalize on good heterogeneity in patterns of perceptual weights in this sample, as observed in our pilot data 

(Figure 1d, Figure 3). Nonetheless, we note that from a lifespan perspective adolescence is the great 

unexplored period of auditory and speech processing despite its importance in many communication disorders. 

The approaches we develop will support future research directed specifically at developmental questions. 

The formation of a new cross-disciplinary research team is a major strength. This project unites unique, 

and complimentary, strengths of Multiple Principal Investigators (MPIs) that span disciplines at adjacent 

institutions in Pittsburgh (Taylor Abel, University of Pittsburgh Medical Center (UPMC) Children's Hospital of 

Pittsburgh (CHP); Lori Holt, Carnegie Mellon University). The project is a natural extension of the MPIs’ 

complementary expertise. All the pieces are in place to assure an integrated project, including frequent in-person 

meetings and cross-lab mentoring synergies (see MPI Plan). 

APPROACH. Experiments will be conducted at CHP where MPI Abel is the Surgical Director of the Pediatric 

Epilepsy Surgery Program. MPI Holt’s laboratory is located a short distance from CHP. The requisite patient 

population, equipment, IRB approval, and local expertise are in place for the proposed work. Human Subjects

provides a detailed plan for cross-site project management, including a timeline. 

Participants and Consent. Human Subjects provides full protocol details. In brief, our approach will be to test 

25 neurosurgical patients ages 15-25 years, among whom it is possible to measure intracerebral neural response 

using sEEG with paired scalp EEG. Patient permitting, individuals will complete all tasks across Aims 1 and 2. 

The age range of our sample overlaps with the typical young adults sampled by the larger 

behavioral literature, and our pilot data indicate desirable baseline heterogeneity in perceptual 

weights with this sample size and age range (Figures A, 1, 2, 3). We expect to enroll an equal 

male/female patient distribution. Power and Sample Size. Robust behavioral effects will allow 

examination of perceptual weighs in neural representation. Using behavioral effect sizes from 

Figure 2 pilot data to estimate the sample size required for a predicted power of 0.8 (two-tailed 

alpha at .05) yields a sample of N=25. This leaves open the issue of power for neural measures. 

Our pilot EEG data with the same task/stimuli revealed robust effects with N=23 (Figure A), 

reassuring that N=25 is a reliable target for EEG. The SNR advantages of sEEG versus EEG, 

and our sEEG pilot data in Figure 3, suggest that this sample size will be more than sufficient 

for sEEG measures, especially as they will be utilized in a within-patient/within-electrode 

experimental design. Enrollment. Based on MPI Abel’s neurosurgical enrollment with similar 

inclusion criteria across 2019 we anticipate no enrollment challenges across this two-year 

project (see Table I and Recruitment & Retention). Inclusion/Exclusion Criteria are detailed 

in Human Subjects. 

Neurosurgical Approach. Electrode implantation is planned based on clinical necessity by epileptologists not 

involved in the research, protecting against conflict of interest.46 No electrode is implanted solely for research 

purposes. Data Safety & Monitoring provides further details and the clinical rationale for STP electrodes. Postsurgery, patients typically spend a night in the pediatric intensive care unit and then transfer to the epilepsy 

monitoring unit the following day. Once feeling well (1-2 days postop) patients may consent/assent to participate. 

Figure 4. sEEG electrode

sites across 5 patients 

illustrating coverage across

the supratemporal plane

(STP), superior temporal

sulcus (STS) and other

brain regions including

frontal cortex. 

Figure 5. Model of 

sEEG+EEG approach.

sEEG

(pins)

scalp

EEG

(bands)

Contact PD/PI: Abel, Taylor 

Research Strategy Page 35
Acoustic Stimuli and Behavioral Approach. The acoustic stimuli have been extensively tested in MPI Holt’s 

research,25–27,29 with feasibility for our sample attested in pilot data (Figures A, 1, 2, 3). Stimuli are derived from 

natural speech, and subtly manipulated to parametrically vary across the VOT and F0 dimensions. During the 

post-op period, patients will complete self-paced blocks of trials that simply require listening to a sound diotically 

over insert earphones and deciding whether the initial consonant is a [b] or a [p]. To facilitate response, we will 

use minimal pair English words (beach vs. peach, e.g.) that can be illustrated so that patients can tap a button 

to indicate which picture matches the word heard. Subjects will categorize stimuli across the 2-d VOTxF0 

acoustic grid for clear speech (Aim 1, Figure 1a) and speech-in-noise (Aim 2, Figure 1c) and across the 

Canonical-Reverse block structure that introduces an artificial accent in the DBSL paradigm (Aim 2, Figure 2a).

All manipulations are within-patient, lending considerable power to test our predictions. Crucially, the same Test

stimuli are present in each block (blue/green symbols, Figure 2a, also mirrored in Figure 1a), providing a ‘gold 

standard’ against which to examine how neural response to identical stimuli is modulated as a function of 

listening contexts that robustly influence perceptual weights in behavioral tasks. 

Hypotheses. We hypothesize that (H1) broadly, the relative perceptual weight of VOT and F0, as measured 

behaviorally, will be reflected in cortical response, (H2) with modulation as a function of baseline perceptual 

weights, (H3) shifts experimentally invoked by a change in listening context by presenting speech in noise, (H4) 

and by introducing an ‘accent’ that shifts short-term input regularities across VOT and F0. In the latter case, our 

approach will allow us to test the specific directional hypothesis (H5) that F0 perceptual weights in the DBSL 

paradigm are both exaggerated by Canonical input regularities that cleanly convey a VOTxF0 correlation 

consistent with English and that F0 perceptual weights are down-weighted upon introduction of a regularity that 

violates the typical pattern of English (supported by scalp EEG pilot, Figure A). Our use of sEEG allows us to 

evaluate these hypotheses across the supratemporal plane thereby testing the strong, and falsifiable, hypothesis 

(H6) that adaptive plasticity effects are present in HG versus (H7) apparent only at higher levels of the cortical 

hierarchy. Our ability to test these hypotheses is complemented by sEEG electrode placement in cortical regions 

outside STP (see Figure 3) which will support secondary hypotheses and serve as control electrode sites. 

Evaluating the Hypotheses: Behavioral Analyses. We will evaluate the behavioral impact of the VOT and F0 

acoustic dimensions on classification using mixed-effects logistic regression (with patient as a random effect, 

stimulus VOT and F0 as fixed effects and classification responses as the outcome). Following our prior work,

14,25–

29 perceptual weights for the dimensions will be computed for each patient as the correlation between dimension 

values and the proportion of peach classifications across all stimuli in the VOTxF0 stimulus grid with absolute 

values of the correlation coefficients normalized to sum to one as an index of relative perceptual weight in quiet 

(Aim 1) and in noise (Aim 2). To examine the impact of a change in ‘accent’ in the DBSL paradigm, we will use 

mixed logit models with responses as a function of patient, block, test stimulus F0 and the interaction between 

block and F0 (Aim 2). Patient will be modeled as a random effect, with the other factors as fixed effects. 

Evaluating the Hypotheses: Neural Analyses. We will take a multi-pronged analysis approach. Our pilot data 

in Figure 3 demonstrate that high-γ activity (HGA) is modulated by graded acoustic details across the VOT and 

F0 dimensions across electrodes placed in the STP. Following the analysis pipeline used in our pilot data 

analyses, we will specifically examine stimulus-time-locked HGA to the Test stimuli, which possess consistent, 

perceptually-ambiguous VOT and differentiated F0. We have specific, directional predictions (detailed above) 

regarding how HGA to F0-differentiated Test stimuli will vary according to perceptual weight in behavior. To 

briefly recap, we expect that HGA to Test stimuli will be better differentiated (1) in the Speech-in-Noise compared 

to the Clear baseline context (because F0 carries greater perceptual weight); (2) in the Canonical, compared to 

the Reverse, context; (3) in the Canonical, compared to the Baseline Clear Speech context (because stimulus 

statistics exaggerate the dimension regularity in behavior). We will use least-squares linear regression neural 

encoding models to investigate relationships between acoustic stimulus dimensions and STP neural responses 

during the baseline quiet context, in which stimuli sample a 2-d F0 x VOT grid. This approach will allow us to 

identify the subset of electrodes and temporal windows that encode at least one of these 2 dimensions at 

baseline; targeted analysis on this subset (described below) will then be used to compare listening contexts. 

Encoding model inputs will consist of F0 and VOT, with an output of channel- and time-specific HGA. For a given 

electrode, individual models will be built using single trial data and a sliding window, allowing us to identify the 

temporal window relative to stimulus onset that yields significant models. Model quality will be assessed in two

ways. First, using models built on all trials, we will calculate the regression F-statistic, which determines if any 

coefficients are significant. This will be compared to null distributions estimated with permutation tests that shuffle 

data across trials. Second, goodness-of-fit will be assessed for significant models by performing leave-one-out 

cross-validation and calculating R2

, the proportion of variance in neural activity explained by the model. Finally, 

Contact PD/PI: Abel, Taylor 

Research Strategy Page 36
we will assess the relative encoding strength of each acoustic dimension using model coefficient t-statistics. In 

summary, this approach will allow us to identify the timing and anatomical location of F0 and VOT encoding 

during baseline quiet listening conditions for which dimensions are sampled orthogonally (as in Figure 1a). 

To characterize shifts in neural encoding across listening contexts, we will investigate Noise using encoding 

models and Canonical/Reverse contexts using cluster-based approaches.

32,49 Encoding models will be built from 

the Noise context for all channels and timepoints, using the same approach as Baseline. If a channel/timepoint 

is significant in either Baseline or Noise, the F0 and VOT coefficient t-statistics will be compared across contexts. 

We hypothesize that encoding of F0 will strengthen and VOT will weaken in Noise, as measured by changes in

t-statistic magnitudes across multiple models. Next, we will compare neural responses for each of the two Test

stimuli (which were embedded in the F0 x VOT grids, Figure 1a) between Baseline vs. Canonical and Canonical 

vs. Reverse. The clustering approach will look only at Baseline significant channels and time windows and 

involves randomly assigning listening context labels to single-trial data followed by a t-test at each time step. 

Across all permutations, a criterion value will be established for each timepoint (>95% of absolute value of t). 

For each of these permutations, we will next determine whether its value exceeds criterion across timepoints, 

and for how many timepoints it exceeds criterion (a ‘cluster’). For each cluster, t values will be summed and 

assigned to all points in the cluster, with the largest summed cluster value stored for each permutation. This will 

create a null distribution of 1000 cluster values. We then will establish whether the cluster size calculated across 

real neural data (organized according to listening context) exceeds the 95% permutation-based cluster values 

such that p<.001 indicates an observed cluster is greater than all permutation-based clusters. Using this 

approach, we will identify context-dependent shifts in HGA responses, which we predict will reflect observed 

shifts in perceptual weights. Namely, we hypothesize that HGA responses in F0-encoding channels will be 

enhanced in the Canonical context relative to both Baseline and the Reverse context. Rigor and Reproducibility. 

Analyses will be controlled for multiple comparisons, with sex as a co-variate in our analyses. We will use preregistration and provide access to all the deidentified source data.

Potential Pitfalls and Alternative Strategies. Clinical Sample. Our goal is the understand normative brain 

function, but sEEG electrodes are only implanted in patients. We note this strategy has been incredibly 

informative in prior work. Here, we go further to include scalp EEG as a means of establishing commonalities 

with healthy listeners, a pursuit still quite rare in intracranial research. Patient performance. Presurgical tests will 

assure basic sensory and language function in patients prior to participation. Hemisphere. There is not yet 

sufficient data to support explicit hemisphere predictions so we will collect data across left and right hemispheres. 

Adolescents. Our participants will include teenagers and young adults (15-25 yrs), as in our behavioral (Figures 

1, 2), EEG (Figure A) and sEEG (Figure 3) pilot data. This range overlaps substantially with the large literature 

of speech perception studies examining undergraduate students. We have established (Figure 1e) this sample 

exhibits sufficient heterogeneity in baseline perceptual weights to evaluate perceptual weights as a function of 

EEG and sEEG (Aim 1), especially in pairing with within-patient/within-electrode experimental manipulations 

(Aim 2). Electrode Placement. Epileptologists not involved in the research determine the clinical necessity of 

electrode placement, eliminating conflict of interest. When electrodes are placed in the STP (approximately 70% 

of 2019 caseload), they are situated in stereotyped, constrained anatomic positions that always include an 

individual’s Heschl’s gyrus, planum polare, and planum temporale. Sensitivity to Acoustic Dimensions. We 

investigate F0 and VOT acoustic dimensions because our pilot sEEG data (Figure 3) demonstrate graded 

responses across these dimensions in the STP, and in the N100 measured with EEG.56-59 New EEG pilot data 

further demonstrate that short-term speech regularities modulate neural response to these dimensions (Figure 

A). This gives us confidence in the feasibility of using this testbed to investigate the flexible representation of 

speech in the STP. Focus on the STP. In this 2-year R21, we intentionally direct our primary focus to the STP 

because prior research56-59 and our new sEEG pilot data (Figure 3) demonstrate a graded response to F0 and 

VOT acoustic dimensions that we can examine across experimental manipulations to listening context to 

understand flexible representation in speech processing, with theoretically-significant implications. Thus, our 

focus on STP is in the interest of delivering data to answer our seven theoretically-motivated hypotheses in the 

framework of a two-year R21 project. Nonetheless, it is important to note that the project will yield rich data from 

electrodes placed across cortex in the same subjects (Figure 4). We are enthusiastic about the secondary 

hypotheses and comparison of STP response to control electrodes that this will support to motivate future work 

founded on the results from this R21. 

Contact PD/PI: Abel, Taylor 

Research Strategy Page 37
PHS Human Subjects and Clinical Trials Information

OMB Number: 0925-0001

Expiration Date: 02/28/2023

Use of Human Specimens and/or Data

Does any of the proposed research in the

application involve human specimens and/or data * ● Yes ❍ No

Provide an explanation for any use of human

specimens and/or data not considered to be

human subjects research.

Are Human Subjects Involved ● Yes ❍ No

Is the Project Exempt from Federal regulations? ❍ Yes ● No

Exemption Number ❏ 1 ❏ 2 ❏ 3 ❏ 4 ❏ 5 ❏ 6 ❏ 7 ❏ 8

Other Requested Information

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 38
Human Subject Studies

Study# Study Title Clinical Trial?

1 Flexible Representation of Speech Yes

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 39
OMB Number: 0925-0001

Expiration Date: 02/28/2023

Section 1 - Basic Information (Study 1)

1.1. Study Title *

Flexible Representation of Speech

1.2. Is this study exempt from Federal

Regulations * ❍ Yes ● No

1.3. Exemption Number ❏ 1 ❏ 2 ❏ 3 ❏ 4 ❏ 5 ❏ 6 ❏ 7 ❏ 8

1.4. Clinical Trial Questionnaire *

1.4.a. Does the study involve human participants? ● Yes ❍ No

1.4.b. Are the participants prospectively assigned to an intervention? ● Yes ❍ No

1.4.c. Is the study designed to evaluate the effect of the intervention on the

participants? ● Yes ❍ No

1.4.d. Is the effect that will be evaluated a health-related biomedical or

behavioral outcome? ● Yes ❍ No

1.5. Provide the ClinicalTrials.gov Identifier (e.g.

NCT87654321) for this trial, if applicable

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 40
Section 2 - Study Population Characteristics (Study 1)

2.1. Conditions or Focus of Study

❍ sEEG Implantation

2.2. Eligibility Criteria

Patients undergoing sEEG implantation to the STP for localization of epileptic foci or clinical language mapping are candidates

 for inclusion. Both epilepsy and brain tumor patients are candidates for inclusion. All participants will be fluent English

 speakers within the normal range on cognitive, speech-language, and hearing tests as tested by a speech language

 pathologist or neuropsychologist in advance of neurosurgery. Patient participants will have nonverbal IQ and receptive and

 expressive language scores within the normal range, normal or corrected-to-normal visual acuity, and normal hearing acuity in

 each ear (as determined during a full audiometric assessment), with no history of autism or ADHD.

2.3. Age Limits Min Age: 15 Years Max Age: 25 Years

2.3.a. Inclusion of Individuals Across the Lifespan 2020_NIH_R21_IndividualsAcrossLifespan_final.pdf

2.4. Inclusion of Women and Minorities 2020_NIH_R21_PerceptualWeights_WomenMinorities.pdf

2.5. Recruitment and Retention Plan 2020_NIH_R21_PerceptualWeights_RecruitmentRete.pdf

2.6. Recruitment Status Not yet recruiting

2.7. Study Timeline 2020_NIH_R21_PerceptualWeights_ProjectTimeline.pdf

2.8. Enrollment of First Participant 02/01/2021 Anticipated

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 41
INCLUSION OF INDIVIDUALS ACROSS THE LIFESPAN

This proposal includes participants from ages 15 to 25 years of age. This age range was picked because it 

overlaps with the typical young adults sampled by the larger behavioral literature on perceptual weight in speech 

perception. Additionally, we have access to this age range at our site of enrollment: Children’s Hospital of 

Pittsburgh. Therefore, our sample will be a mix of children (<18 yrs) and adults (>18 yrs), by the NIH definition. 

As described in the Research Strategy there is heterogeneity in baseline perceptual weights (Figure 1e) that 

is stable across time suggesting that differences reflect underlying processing rather than measurement 

fluctuation.14 For example, even among listeners who consistently weight VOT more than F0 there is 

considerable variation in the extent to which individuals rely upon F0 in signaling contrasts like beach vs. 

peach.

15–19 Therefore, careful examination variation baseline perceptual weights can inform understanding of 

cortical response to speech. Heterogeneity is desirable. Our pilot results demonstrate that teenagers and young 

adults (mean age 20 years) exhibit a good degree of heterogeneity in perceptual weights (Figure 1e). This 

affords an opportunity to observe informative perceptual weight heterogeneity and to relate it to neural response. 

In the proposed project, the richness of this developmental period as a testing ground is amplified by pairing 

natural heterogeneity in baseline perceptual weights (Aim 1) with experimental manipulation of context in the 

same listeners, which evokes dynamic, flexible adjustments to perceptual weight (Aim 2, Figure 1b, d) to 

examine the relationship to neural response. The age range of our sample overlaps with the typical young adults 

sampled by the larger behavioral literature, and in our preliminary data (Figures A, 1, 2, 5).

The children included in this study will be undergoing clinically-necessary neurosurgery. Electrode implantation 

is planned based on clinical necessity by epileptologists not involved in the research, thereby protecting against 

surgeon-scientist conflict of interest. No electrode is implanted solely for research purposes. 

The research team has extensive experience acquiring data from children, adolescents, and young adults. All 

minor participants (< 18 years of age) will be required to provide assent in addition to parental consent. Children’s 

Hospital of Pittsburgh (CHP) is a highly child-friendly environment. There is a sitting area for parents and families 

where they can view their child at all times, via closed circuit video capture.

Contact PD/PI: Abel, Taylor 

Inclusion of Individuals Across the Lifespan Page 42
INCLUSION OF WOMEN, MINORITIES, AND CHILDREN

1. INCLUSION OF WOMEN AND MINORITIES

Recruitment of the patient group will consist of teenagers and young adults (ages 15-25 years) evaluated in the 

Children’s Hospital of Pittsburgh epilepsy surgery program, with all efforts made to include equal numbers of 

males and females in the patient group. We do not expect gender to have any significant influence on any of the 

study results, and gender comparisons are not part of the study design; however, we cannot exclude the 

possibility of gender differences and so we will include gender as a covariate in our analyses. 

Every effort is always made to include a diverse sample. We will also attempt to include minority participants 

among patients, reflecting the surrounding communities of the proposed research site. In Pittsburgh and Western 

Pennsylvania, the representation of minority populations is as follows: White/Caucasian 66.0%, Black/African 

American 26.1%, Asian 4.4%, American Indian/Alaskan Native ~1%, two or more races ~3%. Importantly, our 

studies are of equal benefit to males, females, and minority populations. Ethnic and racial data, using the Federal 

guidelines, will be collected for all participants. 

2. INCLUSION OF CHILDREN

This proposal includes participants from ages 15 to 25 years of age. Therefore, our sample will be a mix of 

children (<18 yrs) and adults (>18 yrs), by the NIH definition. 

As described in the Research Strategy there is heterogeneity in baseline perceptual weights (Figure 1e) that 

is stable across time suggesting that differences reflect underlying processing rather than measurement 

fluctuation.14 For example, even among listeners who consistently weight VOT more than F0 there is 

considerable variation in the extent to which individuals rely upon F0 in signaling contrasts like beach vs. 

peach.

15–19 Therefore, careful examination variation baseline perceptual weights can inform understanding of 

cortical response to speech. Heterogeneity is desirable. Our pilot results demonstrate that teenagers and young 

adults (mean age 20 years) exhibit a good degree of heterogeneity in perceptual weights (Figure 1e). This 

affords an opportunity to observe informative perceptual weight heterogeneity and to relate it to neural response. 

In the proposed project, the richness of this developmental period as a testing ground is amplified by pairing 

natural heterogeneity in baseline perceptual weights (Aim 1) with experimental manipulation of context in the 

same listeners, which evokes dynamic, flexible adjustments to perceptual weight (Aim 2, Figure 1b, d) to 

examine the relationship to neural response. The age range of our sample overlaps with the typical young adults 

sampled by the larger behavioral literature, and in our preliminary data (Figures A, 1, 2, 5).

The children included in this study will be undergoing clinically-necessary neurosurgery. Electrode implantation 

is planned based on clinical necessity by epileptologists not involved in the research, thereby protecting against 

surgeon-scientist conflict of interest. No electrode is implanted solely for research purposes.

The research team has extensive experience acquiring data from children, adolescents, and young adults. All 

minor participants (< 18 years of age) will be required to provide assent in addition to parental consent. Children’s 

Hospital of Pittsburgh (CHP) is a highly child-friendly environment. There is a sitting area for parents and families 

where they can view their child at all times, via closed circuit video capture. 

Contact PD/PI: Abel, Taylor 

Inclusion of Women and Minorities Page 43
RECRUITMENT AND RETENTION PLAN

This document provides a detailed plan for execution of the Aims of this research project at the University of 

Pittsburgh and UPMC Children’s Hospital of Pittsburgh. The organization of the document is as follows: 

1. Engagement 

2. Planning and timeline

3. Staff

4. Power & sample size 

5. Recruitment strategies

6. Retention strategies

7. Coordinating the research project 

8. Benefits to participation 

9. Barriers to participation 

10. Informational materials

11. Ensuring a diverse study sample 

12. When the study has been completed 

1. Engagement

• The research project will recruit patients from UPMC Children’s Hospital of Pittsburgh and neurotypical 

subjects from the greater Pittsburgh community. 

• MPI Abel has been at the and UPMC Children’s Hospital of Pittsburgh since July 

2018, where he is an Assistant Professor of Neurological Surgery (with adjunct appointment in 

Bioengineering) and Surgical Director of the Pediatric Epilepsy Surgery Program. 

• Together MPIs Holt and Abel will actively work throughout the tenure of this research project to maintain 

a high level of engagement. The team will have a standing bi-weekly meeting, or more frequently as 

needed, to review study progression, discuss roadblocks and develop strategies for moving forward. 

2. Planning and Timeline

• Suitable IRB approvals are already in place 

 Modifications to the IRB protocol 

will be made upon funding to accommodate the full scope 

of the proposed research. 

• Research tools for collection of pilot data, including 

button boxes, stimulus display monitors, sound booths 

and computer workstations are in place and functional in 

the MPIs’ respective labs. 

• The tasks and stimuli required for this project are in place, 

having been used to generate the pilot data described in 

the proposal. Protocol refinement for this study 

commence immediately Year 1 along with participant 

recruitment and enrollment. Participant recruitment and 

enrollment continue through Years 2, as needed. 

3. Staff

• A Postdoctoral Researcher and Graduate student will be recruited in Year 1 of the study and will assist 

with aspects of engagement, study recruitment, IRB approval, data management, and research 

coordination. 

• Regular (typically bi-weekly, although often more frequent) in-person meetings with the team will provide 

an opportunity to review past enrollment and retention, as well as discuss research findings and 

preparation of results for professional meetings and publication in peer-reviewed journals. These 

meetings will foster study organization across senior personnel and project staff/trainees. 

• It is expected that some trainees or staff may be replaced during the tenure of the study. If this occurs, 

the replacement trainee or staffer will be recruited in anticipation of the end date of the exiting study 

personnel to ensure sufficient training time and knowledge transfer. 

 

 

 

 

 

 

 

 

 

 

 

Contact PD/PI: Abel, Taylor 

Recruitment and Retention Plan Page 44
4. Power & Sample Size

• Behavioral Effects. Robust behavioral effects will support examination of their impact on neural 

representation. Using effect sizes from Figure 2 data to estimate the sample size required for a predicted 

power of 0.8 (two-tailed alpha at .05) yields N=25. 

• Neural EEG Effects. Our pilot EEG data with the same task/stimuli revealed robust effects with N=23 

(Figure A), reassuring that N=25, estimated from the behavioral effect, is a reliable sample for EEG. 

• Neural sEEG Effects. The SNR advantages of sEEG versus EEG suggest that this sample size (N=25) 

will be sufficient for sEEG measures, as well, especially as they will be utilized in a within-patient (and 

within-electrode) experimental design. sEEG studies are analogous to invasive neurophysiology in 

animal models and electrocorticography (ECoG) in humans, for which sample sizes are limited 

(sometimes to <10), but data collection from individuals is extremely rich. Our projected patient population 

sample size (N=25) will therefore provide a large enough sample size for a hold-out sample that would 

be useful in assessing the robustness of the proposed encoding and decoding models. 

• Sample size will allow for robust and replicable effects. 

5. Recruitment Strategies

• Patients will be recruited prior to sEEG implantation at UPMC Children’s Hospital of Pittsburgh. Patients 

will be introduced to the project by MPI Abel prior to surgery and, when interested, consented by PI Abel. 

• Any changes that are necessary to the recruitment materials, study protocol, additional study personnel, 

or consent forms will be accomplished through review by the IRB at the 

• Recruitment strategies will be assessed and approved by the Institutional Review 

Board (with Carnegie Mellon University as an sIRB site). Participants under age 18 will assent to 

participation, with parent’s informed consent. 

Exclusion Criteria. Exclusion criteria include intellectual disability, abnormal epileptiform activity in the 

supratemporal plane, or a lack of fluent English comprehension/production. Patients are also excluded when 

preoperative cognitive testing demonstrates severe language or auditory specific cognitive dysfunction. 

Patients with autism or ADHD will also be excluded. 

Inclusion Criteria. Patients undergoing sEEG implantation to the STP for localization of epileptic foci or 

clinical language mapping are candidates for inclusion. Both epilepsy and brain tumor patients are candidates 

for inclusion. All participants will be fluent English speakers within the normal range on cognitive, speechlanguage, and hearing tests as tested by a speech language pathologist or neuropsychologist in advance of 

neurosurgery.

Approach to Avoiding Conflicts of Interest. Research electrodes are placed solely for clinical purposes. 

As described elsewhere, targets for sEEG electrode exploration are chosen by the pediatric epileptology 

team who are not involved in this research and do not benefit from this research program. STP electrodes 

are commonly implanted to rule-out or establish the involvement of insulo-opercular cortex in temporal lobe 

epilepsy (‘temporal plus’ epilepsy), which has been shown to be a major determinant of failure after temporal 

lobe resection for epilepsy. Additionally, our IRB requires that a pediatric epileptologist caring for the 

research subject approves of all research activity on a daily basis to ensure that research will not obstruct 

clinical care or adversely impact the child’s seizure threshold. 

Projected Enrollment. Current caseload for MPI Abel at Children’s Hospital of 

Pittsburgh will support successful recruitment of patient participants at the proposed 

sample size. Based on MPI Abel’s neurosurgical enrollment with similar inclusion 

criteria across 2019 we anticipate no enrollment challenges across this two-year 

project (see Table I). Approximately 70% of his 2019 cases involved electrode 

placement in the STP. The planned study enrollment is realistic given the clinical 

volume and research history at UPMC Children’s Hospital of Pittsburgh. However, 

should recruitment be lower or should attrition rates be higher than anticipated, we 

will extend the period of recruitment further into Year 2 than anticipated. 

Fourteen patients were enrolled in research during the last 12 months at CHP, 

of whom 12 had electrodes in the STP. This period included 2 months of halted 

elective surgery due to the pandemic. In the next two months, we anticipate 

Contact PD/PI: Abel, Taylor 

Recruitment and Retention Plan Page 45
enrolling three patients with planned STP coverage. On this basis, we do not expect difficulty achieving the 

enrollment numbers for this proposal. 

6. Retention Strategies

• Studies with sEEG occur in the acute setting when sEEG electrodes are implanted and patients are 

hospitalized. Optimizing patient comfort will be important for facilitating retention during this short period 

of implantation. Patient comfort is essential. 

• Throughout the experiment, the researchers repeatedly check in with the participant to ensure they are 

comfortable, providing regular breaks between block/tasks with additional breaks and snacks as needed. 

If the participant does not appear comfortable at any time, even if they do not state it aloud, we terminate 

the experiment immediately. While we do not anticipate stressful reactions using our protocols, if they 

occur, we would offer comfort by offering other activities the participant enjoys, including snacks, reading 

a book, videos/games, watching part of a movie, etc., to help the participant relax. Our primary concern 

if a participant were to be upset is to ensure the participant is calm and happy before the experimenter 

leaves the patient. 

• Retention will be monitored by examining the percentage of participants who complete a first session 

who go on to complete all sessions. If average retention levels drop from our expectations, determined 

through quarterly reviews by the research team, the research team will determine a new course of action 

to encourage greater retention and participation levels. We have noticed that patient participation is 

higher now that patients are remunerated $15/hr for their participation. 

7. Coordinating the Research Project

• MPI Abel’s laboratory at UPMC Children’s Hospital of Pittsburgh will serve as the coordinating center for 

all aspects of this study. As noted above, the entire research team (MPIs, postdoctoral associate, 

graduate student) will have quarterly meetings to discuss the research project as a whole. At least one 

of the MPIs will meet at least weekly with each member of the research team. The MPIs will have biweekly meetings, or more frequently as needed, to discuss project progress. Electronic communication 

(e.g., email, text messaging) will be ongoing throughout the project. 

8. Benefits to Participation

• The principal benefit for enrolling in this study, from the perspective of potential participants, is that they 

will have the opportunity to advance scientific knowledge. Many participants have noted, upon 

participating in our research team’s similar studies, that they perceive a benefit to be contributing to 

knowledge and science, with the hope that the knowledge gained may be used in the future to help 

others. 

• Care is taken to ensure that participation in these research studies is as easy as possible for participants. 

• Participants are compensated $15/hr for research participation. 

9. Barriers to Participation

• For sEEG patients, medical factors (fatigue, seizures, surgical pain) can sometimes be barriers to 

research participation. As a leader of the surgical team, PI Abel is intimately aware of these concerns 

and works to mitigate these barriers when possible. 

• Should we encounter barriers, the research team will meet to consider adaptations to the protocol that 

can meet the scientific needs while broadening participation. 

10. Informational Materials

• All participants (or, for minors, their parents) will be provided a detailed explanation of all aspects of the 

project, encouraged to ask questions, and given a copy of the written consent form that describes the 

experiment for a lay audience. The information has been developed with a wide audience in mind, and 

wide literacy levels, indicating the components of participation. Minor children will give their assent to 

participate in the study. 

• Patients and families who are interested can request to receive information about the study outcomes 

written for a lay audience.

11. Ensuring a Diverse Study Sample

• Recruitment into the study follows consecutive enrollment for participants meeting the eligibility criteria. 

Contact PD/PI: Abel, Taylor 

Recruitment and Retention Plan Page 46
• sEEG patients will be recruited from neurosurgical patients who are undergoing evaluation and treatment 

for epilepsy or brain tumor, and who require chronic (i.e. >7 d) sEEG implantation and recording of the 

temporal lobe for seizure localization or brain mapping. 

• Every effort will be made to recruit a diverse sample from within this patient population. We attempt to 

include minority participants among patients, reflecting the surrounding communities of the proposed 

research site. In Pittsburgh and Western Pennsylvania, the representation of minority populations is as 

follows: White/Caucasian 66.0%, Black/African American 26.1%, Asian 4.4%, American Indian/Alaskan 

Native ~1%, two or more races ~3%. Importantly, our studies are of equal benefit to males, females, and 

minority populations. Ethnic and racial data, using the Federal guidelines, will be collected for all 

participants. 

• All efforts made to include equal numbers of males and females in the patient group. We do not expect 

gender to have any significant influence on any of the study results, and gender comparisons are not part 

of the study design; however, we cannot exclude the possibility of gender differences and so we will 

include gender as a covariate in our analyses. 

• The research team aims to have a diverse group of trainees and staff working on the study (and related 

studies), including racial and ethnic minorities and a gender balanced group of individuals. The research 

team will have bi-annual meetings with all trainees and staff to ensure that everyone is following best 

practices when interacting with study participants, and that all team members maintain a high-level of 

sensitivity to cultural, racial and ethnic differences, as well as socioeconomic disparities. 

12. When the Study has been Completed

• Participants will be given debriefing information including the MPIs’ email and office phone number, as 

well as contact information for the IRB. 

• Additionally, participants will have the opportunity to sign up for study updates including publication or 

presentation of study findings. This ensures that participants understand they can still reach out to the 

study team to learn about results even after they are no longer actively participating in the research. 

Contact PD/PI: Abel, Taylor 

Recruitment and Retention Plan Page 47
PROJECT TIMELINE

The timeline initiates immediately upon funding. In the chart below Aim 1 and Aim 2 proceed in parallel

throughout the project, with recruitment and testing proceeding into Year 2. 

sEEG, EEG, and behavioral data will be collected in patient populations by the PIs and study personnel at 

Children’s Hospital of Pittsburgh throughout the project. Data will be analyzed continuously and added to the 

larger dataset at quarterly intervals. Preparation of research products corresponding to core and secondary 

hypotheses will initiate in Year 1. Each Aim is expected to yield at least two major publications, and additional 

shorter papers corresponding to interim findings appropriate for conference proceedings. 

Table 1. Project Timeline C = Conference, P = Publications 

Milestones Y1 Y2

Aim 1

Personnel Recruitment  ̈

Stimulus/paradigm development  ̈

Pilot testing  ̈

EEG set-up and piloting  ̈  ̈

Recruitment & testing  ̈  ̈  ̈

Analyses  ̈  ̈

Dissemination C P 

Aim 2

Pilot testing  ̈

EEG set-up and piloting  ̈  ̈

Recruitment & testing – 

neurotypical & patient participants 

 ̈  ̈  ̈

Analyses  ̈  ̈

Dissemination C P

Contact PD/PI: Abel, Taylor 

Study Timeline Page 48
2.9. Inclusion Enrollment Reports

IER ID# Enrollment Location Type Enrollment Location

Study 1, IER 1 Domestic UPMC Children's Hospital of Pittsburgh

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 49
Inclusion Enrollment Report 1

1. Inclusion Enrollment Report Title* : Anticipated Enrollment

2. Using an Existing Dataset or Resource* : ❍ Yes ● No

3. Enrollment Location Type* : ● Domestic ❍ Foreign

4. Enrollment Country(ies): USA: UNITED STATES

5. Enrollment Location(s): UPMC Children's Hospital of Pittsburgh

6. Comments:

Planned

Racial Categories

Ethnic Categories

Not Hispanic or Latino

Female Male

Hispanic or Latino

Female Male

Total

American Indian/

Alaska Native 0 0 0 0 0

Asian 1 1 0 0 2

Native Hawaiian or

Other Pacific Islander 0 0 0 0 0

Black or African

American 2 2 0 0 4

White 9 8 1 1 19

More than One Race 0 0 0 0 0

Total 12 11 1 1 25

Cumulative (Actual)

Racial Categories

Ethnic Categories

Not Hispanic or Latino

Female Male

Unknown/

Not

Reported

Hispanic or Latino

Female Male

Unknown/

Not

Reported

Unknown/Not

Reported Ethnicity

Female Male

Unknown/

Not

Reported

Total

American Indian/

Alaska Native 0 0 0 0 0 0 0 0 0 0

Asian 0 0 0 0 0 0 0 0 0 0

Native Hawaiian or

Other Pacific Islander 0 0 0 0 0 0 0 0 0 0

Black or African

American 0 0 0 0 0 0 0 0 0 0

White 0 0 0 0 0 0 0 0 0 0

More than One Race 0 0 0 0 0 0 0 0 0 0

Unknown or

Not Reported 0 0 0 0 0 0 0 0 0 0

Total 0 0 0 0 0 0 0 0 0 0

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 50
Section 3 - Protection and Monitoring Plans (Study 1)

3.1. Protection of Human Subjects 2020_NIH_R21_PerceptualWeights_HumanSubjects_f.pdf

3.2. Is this a multi-site study that will use the same protocol to

conduct non-exempt human subjects research at more than one

domestic site?

● Yes ❍ No ❍ N/A

If yes, describe the single IRB plan 2020_NIH_R21_PerceptualWeights_sIRB_final.pdf

3.3. Data and Safety Monitoring Plan 2020_NIH_R21_PerceptualWeights_DataSafetyMonit.pdf

3.4. Will a Data and Safety Monitoring Board be appointed for

this study?

● Yes ❍ No

3.5. Overall structure of the study team 2020_NIH_R21_PerceptualWeights_StudyTeam_final.pdf

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 51
PROTECTION OF HUMAN SUBJECTS

1. Risks to the Subjects 

a. Human subjects involvement, characteristics, and design. Our scientific premise is that next-generation 

models of human cortical speech processing must account for adaptive plasticity -- the dynamic, flexible mapping 

of speech input to perception. Our approach marries behavioral manipulation of listening context in speech 

categorization with simultaneous EEG and sEEG measurement of neural response. Our pilot data establish the 

behavioral (Figures 1, 2), EEG (Figure A), and sEEG (Figure 3) approaches, all of which have been reviewed 

by Carnegie Mellon University and/or Institutional Review Boards.

Recruitment and Sample: Patient participants will be recruited from pediatric patients who 

undergo implantation of sEEG electrodes at CHP. This will predominantly (95% based on 

previous enrollment) be epilepsy patients undergoing sEEG implantation for localization of 

epileptic foci, but occasionally brain tumor patients who are not candidates for awake craniotomy 

undergo sEEG implantation for localization of language function. Our participants across two

years will include 25 patient participants, ages 15-25 years who are undergoing evaluation and 

treatment for epilepsy or brain tumor, and who require chronic (i.e. >3 d) sEEG implantation and 

recording of the temporal lobe for seizure localization or brain mapping. Based on the historical 

volume of epilepsy surgery at CHP and the volume from the PI’s clinical practice at CHP, this 

level of participant recruitment is highly feasible (see Table I). Please see Inclusion of Women, 

Minorities, and Children and Approach for a full justification of the scientific rationale for 

examining this age range. 

Exclusion Criteria. 

In general, patients with significant medical or neuropsychological impairment, who would be unable to 

participate for the proposed research, will not be considered for inclusion in the proposed studies. Exclusion 

criteria include intellectual disability, abnormal epileptiform activity in the supratemporal plane, or a lack of fluent 

English comprehension/production. Patients are also excluded when preoperative cognitive testing 

demonstrates severe language or auditory specific cognitive dysfunction. Patients with autism or ADHD will also 

be excluded. 

Inclusion Criteria. Patients undergoing sEEG implantation to the STP for localization of epileptic foci or clinical 

language mapping are candidates for inclusion. Both epilepsy and brain tumor patients are candidates for 

inclusion. All participants will be fluent English speakers within the normal range on cognitive, speech-language, 

and hearing tests as tested by a speech language pathologist or neuropsychologist in advance of neurosurgery. 

Patient participants will have nonverbal IQ and receptive and expressive language scores within the normal 

range, normal or corrected-to-normal visual acuity, and normal hearing acuity in each ear (as determined during 

a full audiometric assessment), with no history of autism or ADHD. 

The population in this project will be recruited equally from both sexes and all ethnic or racial groups (see 

Inclusion of Women, Minorities, and Children).

Power: Robust behavioral effects will support examination of their impact on neural representation. Using effect 

sizes from Figure 2 (Approach) data to estimate the sample size required for a predicted power of 0.8 (twotailed alpha at .05) yields N=25.This leaves open the issue of power for neural measures. Our pilot EEG data

with the same task/stimuli revealed robust effects with N=23 (Figure A, Introduction), reassuring that N=25 is 

a reliable sample for EEG. The SNR advantages of sEEG versus EEG suggest that this sample size will be 

sufficient for sEEG measures, as well, especially as they will be utilized in a within-patient (and within-electrode) 

experimental design. sEEG studies are analogous to invasive neurophysiology in animal models and 

electrocorticography (ECoG) in humans, for which sample sizes are limited (sometimes to <10), but data 

collection from individuals is extremely rich. Our sEEG pilot data demonstrate that each of the 7 patients 

examined exhibited significant graded high-gamma activity across VOT and F0 dimensions for channels on an 

electrode placed along the STP. 

Contact PD/PI: Abel, Taylor 

Protection of Human Subjects Page 52
Retention. Our retention plan involves ensuring that participants are comfortable before and during sessions. 

We will offer breaks and snacks as needed. Please see Recruitment and Retention Plan for details. 

Confidentiality: Data will be acquired from patient participants at Children’s Hospital of Pittsburgh (CHP), under 

the direction of MPI Abel. All data will be obtained directly from patients and/or their caregivers and stored at 

CHP, respectively. Only consent forms will contain participant names. All other forms will be labeled by an 

assigned participant identification number. One digital form will contain the participant’s name and assigned 

participant identification number. This form will be individually locked, only accessible by the MPIs and trained 

research personnel. It will be stored on a locked computer. All participant files will be stored in a locking file 

cabinet or on a locked computer accessible only by researchers affiliated with the project. 

Neurosurgical Protocol: In brief, individual electrode trajectories are planned using a 3D MPRAGE postcontrast MRI fused to a 3D CT angiogram (to ensure avascular trajectories). Robot-assisted (Meditech ROSA) 

stereotactic implantation of each planned trajectory is then performed in the operating room under general 

anesthesia, per clinical protocols. Electrode localization is confirmed using intraoperative CT imaging (Medtronic 

O-Arm). After surgery, patients typically spend one night in the pediatric intensive care unit and then transfer to 

the epilepsy monitoring unit the following day. Once feeling well (1-2 days postop) patients may consent/assent 

to participate. 

Behavioral Protocol: The acoustic stimuli have been extensively tested in MPI Holt’s prior research25–27,29, as 

in pilot data in Figures A, 1 and 2. They are derived from natural speech, and subtly manipulated to 

parametrically vary across the VOT and F0 dimensions. During the post-op period, patients will complete selfpaced blocks of trials that simply require listening to a sound diotically over insert earphones and deciding

whether the initial consonant is a [b] or a [p]. To facilitate response, we will use minimal pair English words 

(beach vs. peach, e.g.) that can be illustrated. Patients will tap a button to indicate which picture matches the 

word heard. Each patient will categorize stimuli across the 2-d VOTxF0 acoustic grid for clear speech (Aim 1, 

Figure 1a) and speech-in-noise (Aim 2, Figure 1c) and across the Canonical-Reverse block structure that 

introduces an artificial accent in the DBSL paradigm (Aim 2; Figure 2a). All manipulations are within-patient, 

lending considerable statistical power to test our predictions. Crucially, the same Test stimuli are present in each 

block (blue/green symbols, Figure 2a), providing a ‘gold standard’ against which to examine how neural 

response to identical stimuli is modulated as a function of listening contexts that robustly influence perceptual 

weights in behavioral tasks. These methods are classified as minimal risk by our IRB. Our approach is to make 

the task as interesting and engaging as possible to make patient participants comfortable.

b. Sources of Materials. The neurophysiological and behavioral data acquired in these studies is specifically 

and only used for research purposes. Behavioral and neurophysiologic data will be collected as well as personal 

information from answers on questionnaires. Only the MPIs and research personnel who are actively working 

on this project will have access to data collected for this project. All researchers have completed the 

 and NIH Protection of Human Research Protection computer-based training programs. All data will 

be stored in a locking file cabinet or on servers protected by a password, which are accessible only by password 

protected computers. All electrophysiological and behavioral data will be labeled with a participant identification 

number. The link between participant name and identification number will be housed in one electronic document 

that is individually password protected (a different password from lab computers and servers). Only the trained 

researcher personnel will have access to this file. 

c. Potential Risks. Participants will incur minimal additional risk by participating in this study. Electrode 

implantation strategies for localizing epileptic foci are based solely on each patient’s clinical needs and 

are not dictated by research. Furthermore, surgical technique would be the same for research and nonresearch participants (there will be no special “research electrodes” implanted for the purpose of this study). The 

risks of electrode implantation are discussed with the parents and patients during the surgical consent process, 

which is completely separate from the research, consent process. STP electrodes are commonly implanted to 

rule-out or establish the involvement of insulo-opercular cortex in temporal lobe epilepsy (‘temporal plus’ 

epilepsy), which has been shown to be a major determinant of failure after temporal lobe resection for epilepsy.

Contact PD/PI: Abel, Taylor 

Protection of Human Subjects Page 53
On that basis, while electrodes are not implanted solely for research purposes, we will frequently enroll 

participants with STP electrode coverage as this region is important for localization of insulo-opercular seizure 

involvement in temporal lobe epilepsy. Please also see Data Safety and Monitoring Plan. 

Our neurosurgical research protocol is currently approved by the University of Pittsburgh IRB and we 

are actively enrolling research subjects on highly similar protocols. 

We have had no research related adverse events.

Psychological risks are also minimal since no pressure or stress is applied at any time, or specifically as a result 

of performance on the tasks. The MPIs have extensive experience acquiring behavioral and brain data from 

hundreds of participants, including from vulnerable populations, and we have found that the vast majority of 

participants are comfortable with our procedures. Adolescents and parents, and adults (> 18 years) are informed 

that they may stop participation at any time. Throughout the experiment, the researchers repeatedly check in 

with the participant to ensure they are comfortable, providing regular breaks between block/tasks with additional 

breaks and snacks as needed. If the participant does not appear comfortable at any time, even if they do not 

state it aloud, we terminate the experiment immediately. While we do not anticipate stressful reactions using our 

protocols, if they occur, we would offer comfort by offering other activities the participant enjoys, including snacks, 

reading a book, videos/games, watching part of a movie, etc., to help the participant relax. Our primary concern 

if a participant were to be upset is to ensure the participant is calm and happy before the experimenter leaves 

the patient. 

There are no social or legal risks. 

2. Adequacy of Protection Against Risks

a. Recruitment and Informed Consent. Patient participants will be recruited from pediatric patients who 

undergo invasive localization of epileptic foci at the CHP and are paid $15/hour. 

Patients and parents will be given extensive explanations about the potential risks and benefits of the proposed 

research during the research consent process. Patients and parents will be told about the aims of the study and 

about the confidentiality of the data obtained. It will be made clear to patients and parents that their participation 

is completely voluntary at all stages and is distinct from clinical care. Each patient (or their parent) is free to 

disengage the patient from research involvement at any time. 

Experimenters will obtain consent by asking patients and parents or legal guardians to sign consent forms after 

answering any questions and addressing any concerns that they may have. Patients <18 years will be read an 

age-appropriate assent form aloud and any questions will be answered before obtaining written assent via the 

Child Assent Form. Parents will be required to read and sign a separate Parent Consent Form. 

All personnel involved in any aspect of the research program are certified as completing the University of 

Pittsburgh and the NIH Protection of Human Research Subjects computer-based training programs. 

b. Protection Against Risk. Physical risks will be minimized using standard laboratory and equipment 

procedures whose main elements are outlined above. No adverse effects specific to the proposed research that 

would require medical intervention are expected. Potential adverse effects of sEEG specific to the surgical 

process, which is completely separate from the research process. 

For the proposed studies, risks of electrical shock during EEG recording are minimized by proper electrical 

grounding of participants and by isolation of electrical recording and stimulation equipment from ground. 

Participants’ records are kept completely confidential under control of the investigators and associates. A 

participant’s data are only identified by a code and could not be used adversely to his or her interests. 

Contact PD/PI: Abel, Taylor 

Protection of Human Subjects Page 54
If a participant becomes upset at any time during a session, the task will be discontinued immediately and every 

effort will be made to comfort the participant, with the parent’s help, and engage him/her in an activity that is not 

distressing (for example, playing a game) so that s/he will leave with a positive experience. In all cases 

participants are informed repeatedly that they can decide to halt participation at any time. 

3. Potential Benefits of the Proposed Research to the Subjects and Others 

There will be no direct benefit to the patient beyond payment for participation in the experiment ($15/hr). The 

research described holds the promise of contributing to the refinement of our understanding of the 

neurobiological basis of speech processing. 

Since the risks to the participants are very low, the risk-benefit ratio seems quite acceptable in light of the 

potential for improving understanding of the neurobiological basis of speech processing. 

4. Importance of the Knowledge to be Gained 

The proposed research will increase understanding of the neural basis of speech processing. The findings from 

this study will enhance knowledge regarding foundational auditory processes available to speech 

communication. Long-term implications of this research include the potential for translational contributions to 

tackling the many communication disorders that impact listeners.

The research risks to the participant are very low; therefore, the risk-benefit ratio seems quite acceptable in light 

of the potential for improving models of auditory processing as well as potential for refining interventions targeting 

adolescents. 

Contact PD/PI: Abel, Taylor 

Protection of Human Subjects Page 55
sIRB PLAN

This study will comply with the NIH Policy on the Use of sIRB for Multi-Site Research. The

Institutional Review Board (IRB) has agreed to serve as the sIRB of record for human subjects research under 

this grant. Carnegie Mellon University (CMU), the participating site, will rely on the proposed sIRB and any sites 

added after award will rely on the sIRB. , the proposed sIRB will maintain records of the 

reliance agreements and the communication plan. The participating sites will, prior to initiating the study, sign an 

authorization/reliance agreement that will clarify the roles and responsibilities of the Reviewing IRB and 

participating sites. 

We will comply with the NIH Policy on the Use of sIRB for Multi-Site Research. Awardees will comply with the 

policy pursuant to the sIRB plan. 

The team will be responsible for ensuring ongoing communication with all participating 

sites via regular in-person meetings and electronic communication throughout the study. Key communication 

points will occur to: 

• Disseminate IRB determinations and IRB-approved documents 

• Educate study teams regarding the approved study and amendments to the study 

• Alert study teams to problems that may affect the conduct of the study or the rights and welfare of 

research participants, such as unanticipated problems. 

• Inform study teams of any changes in study status or new information 

o Facilitate submissions to the Reviewing IRB, including: 

o Inclusion of site-specific requirements in consent documents 

o Identification of any variability in study implementation across sites that must be communicated 

to the University of Pittsburgh IRB

o Collection of information from participating sites to include in continuing review reports to the 

IRB

o Site-specific amendments 

o Personnel updates (as required by the IRB)

o Reportable events (e.g., noncompliance, unanticipated problems) 

o Closure reports 

o Ensure revisions to applicable conflict of interest management plans are provided to the 

IRB 

This project will use the SMART IRB Master Common Reciprocal Institutional Review Board Authorization 

Agreement (SMART IRB Agreement) to support single IRB review across and Carnegie 

Mellon University. SMART IRB is an online reliance platform designed to streamline and advance reliance 

through a common IRB authorization agreement. The SMART IRB Agreement outlines the responsibilities of all 

Participating Institutions, the Reviewing IRB, and Relying Institutions, and provides templates for communication 

between the Reviewing IRB and Relying Institutions. All sites in this study are members of SMART IRB and will 

indicate reliance and willingness to invoke the SMART IRB agreement prior to IRB review. 

Contact PD/PI: Abel, Taylor 

IRB Plan Page 56
DATA SAFETY AND MONITORING PLAN

The MPIs will assume mutual responsibility for data safety and monitoring. The data monitoring and management 

plan covers all aspects of this project. 

1. Safety in a Neurosurgical Context: Clinical Rationale

Electrodes trajectories are chosen purely on the basis of clinical necessity as determined by pediatric 

epileptologists that are not involved in the research protocols. This includes placement of the STP electrodes 

that are utilized for the purposes of this study. Our STP electrode trajectories mimic those that have been used 

at experienced sEEG centers in Europe for over 50 years, which have standardized trajectories to explore the 

1) anterior STG, planum polare, and anterior insula (electrode ‘T’), 2) middle STG, Heschl’s gyrus, and posterior 

insula (electrode ‘U’) and 3) posterior STG, planum temporale, and the posterior cingulate (electrode ‘V’). These 

trajectories are used at experienced sEEG centers across France, Italy, and at some centers in the United States 

as they maximize the number of brain regions examined by single electrode trajectories in a safe and established 

manner. Figure 1 depicts all of the standard electrode trajectories we use and provides an example of a deidentified schematic implantation plan formulated by our pediatric epileptology team (who are not involved in the 

research). 

Clinically, STP electrodes are often placed for two 

reasons: 1) to rule-out or establish temporal plus 

epilepsy (temporal lobe seizures that spread 

rapidly from the mesial temporal lobe to the insuloopercular region; Barba et al. 2007; Barba et al. 

2016) and 2) for clinical language stimulation 

mapping with sEEG electrodes. As outlined by 

Barba et al. (2016), temporal lobe epilepsy 

patients with temporal plus epilepsy are 5x more 

likely to not be seizure free after epilepsy surgery, 

so exploration of the insulo-opercular region is 

essential when clinical evaluation implicates 

temporal plus epilepsy. Furthermore, since the 

rationale for exploration of the STP and insuloopercular region is to rule-out temporal plus 

epilepsy or perform language mapping, often the 

interictal activity from STP electrodes is normal 

(though STP electrodes can be removed from 

analysis when interictal abnormalities preclude 

research investigation of normative brain function). 

2. Safety in a Neurosurgical Context: Avoiding Clinician/Scientist Conflict of Interest

Electrode trajectories are proposed entirely by the pediatric epileptology team on the basis of clinical hypotheses 

of the localization of each individual’s epileptogenic zone. PI Dr. Abel’s clinical role is the safe implantation of 

sEEG electrodes once the pediatric epileptology team has provided a schematic plan (see Figure 1b) for what 

brain regions each sEEG electrode should explore. 

3. Expected data. This project will produce the following types of data: interview responses from background 

questionnaires from participants to assess eligibility, behavioral data standardized and non-standardized tests, 

questionnaires, psychoacoustic/perceptual tasks, scalp electroencephalography (EEG), and intracerebral 

stereoelectroencephalography (sEEG).

• Interview responses will record basic demographic information. Responses will be entered in a password 

protected and encrypted file on a secured laboratory computer. 

• Experimental behavioral data will consist of keypress responses made during the course of the experiment 

using E-prime, Psychophysics Toolbox 3 in Matlab, Unity, tablet/online videogames, or similar. Custom 

Figure 1. Clinical sEEG electrode planning schematics. (A) Standard 

electrode trajectories used at UPMC Children’s Hospital of 

Pittsburgh. (B) Example of electrode schematic provided by pediatric 

epileptology team utilizing standard clinical electrode trajectories. 

Contact PD/PI: Abel, Taylor 

Data and Safety Monitoring Plan Page 57
analysis scripts will be used to summarize the data. All data is stored with a study ID, independent of 

participant identity. 

• Electrophysiology data will consist of auditory cortical responses evoked via sound and recorded with a Brain 

Vision actiCHamp system. 

All data will be de-identified at the time of collection to a pre-assigned subject ID code. An encrypted and 

password protected spreadsheet will be the only recorded link between specific subjects and their IDs. This is 

the only recorded link between participant identity and their identification number. 

All interview responses will be recorded into electronic spreadsheets and paper notes will be stored in a locked 

file cabinet in the MPIs’ laboratories. All electronic data (behavioral data files, EEG files, etc.) will be stored on a 

secured computer server in the MPIs’ laboratory and backed up onto the servers at the CHP. This way, two full 

copies of all of the recorded data are preserved in two separate locations at all times. Files and folders will be 

encrypted and password protected. 

4. Period of data retention. All data will be for a minimum of 7 years after completing the project.

5. Data publication. The investigators will promptly prepare and submit all significant findings from work 

conducted under the award for publication, with authorship that accurately reflects the contributions of those 

involved and support from NIH. The investigators will pursue open access approaches to publication (as 

budgeted) and will deposit manuscripts in PubMed Central (PMC), which may be accessed online.

6. Standards to be used for data and metadata format and content. Data and metadata will be stored using 

the simplest possible formats to assure data longevity. The lowest level of individual trial-level data will be 

available in a de-identified format. Each type of data will be linked to detailed notes regarding how the data were 

generated, the metadata, and the analytical methods used to summarize and analyze them. During the 

conversion process, any potential identifying information is stripped from the file so that all data made for analysis 

is anonymous. 

7. How the data are accessible and data-sharing practices and policies. All data will be stored in the MPIs’ 

laboratories. The MPIs will maintain the datasets privately until publication of their results. Once published, all 

datasets will be opened to secondary research by both internal and external collaborators. Should the MPIs

leave the institution, or should there be a change to their roles and responsibilities, the senior personnel 

remaining at University of Pittsburgh will assume responsibility for data management. 

8. Policies and provisions for re-use, re-distribution, and production of derivatives. During the project 

lifecycle, data set access will be restricted to the core research team. This includes the MPIs and lab personnel. 

At the conclusion of the project, the EEG datasets and related experimental materials will be made public and 

accessible through a publicly available web server and/or through sources such as NeuroVault (depending on 

analysis type). In addition, all analysis scripts built as part of this project will be made publicly available via open 

source code development websites. Attracting researchers to perform secondary analysis on these data sets 

will occur on a lab-by-lab basis, based on presentation of works in scientific publications and professional 

meetings. 

Once published, datasets may be opened to secondary research by both internal and external collaborators by 

submission to data sharing websites. There is a data-sharing clause in our IRB consent forms that participants 

can opt out of if desired. 

9. Reusability of Data. Research groups who are likely to be interested in using these data include researchers 

who study auditory cognitive neuroscience and speech processing. 

10. Ethics and Protection of Privacy - Human Subjects Research. All data collected or analyzed will be solely 

for research purposes and for the proposed studies. During the project lifecycle, dataset access will be restricted 

to the core research team. This includes the MPIs’ senior lab personnel (postdocs, lab manager, and graduate 

Contact PD/PI: Abel, Taylor 

Data and Safety Monitoring Plan Page 58
students) and IRB-approved collaborators. Extensive precautions are in place to ensure subject privacy and 

maintain confidentiality of the data. These precautions include restricted identification of subjects by de-linking 

their data ID number to any identifiable information. Only research staff will have access to this information. In 

addition, all data will be stored on encrypted and password protected servers behind the University of Pittsburgh 

firewall. 

11. Responsibility. All research personnel will be instructed on data collection, data analysis, data archiving 

and data sharing by the MPIs. The MPIs and the first author of subsequent publications will be responsible for 

archiving the data. The MPIs will oversee the archival process and will ultimately be responsible for data archiving 

and sharing. 

12. Intellectual Property Rights. All intellectual property, including data, generated under this project will be 

managed in accordance with institutional policies. 

13. Safety Monitoring. The protocols used in the present studies are all classified as “Greater than Minimal 

Risk” by the University of Pittsburgh Institutional Review Board. This classification is because our IRB enables 

us to perform brain stimulation experiments, but no such experiments are part of this proposal. The occurrence 

of any adverse events associated with this study and its procedures, as well as any changes in risk level will be 

monitored by the MPIs of this study. Participants will be monitored for the duration of their participation in the 

proposed studies. The investigators of the study will promptly report to the IRB any unexpected adverse reactions 

of serious severity or unexpected adverse reactions of moderate or greater severity that are associated with the 

research and observed in conjunction with the conduct of this research study. Non-fatal or non-life-threatening 

adverse reactions associated with this research study will be reported to the IRB as soon as possible and no 

later than 10 days. Any major disputes between the research investigators and a research participant, or between 

research investigators, will be promptly reported to the IRB. The study investigators will closely monitor all 

participants for any such adverse events. The study investigators will meet annually to review the occurrence of 

any adverse events, as well as discuss recruitment, confidentiality, and/or relevant information that may impact 

on the safety of study participants and ethics of the research study, participant confidentiality, and data collection 

and data analysis issues pertinent to this study. A summary detailing the frequency of data and safety monitoring 

and the information gathered from these meetings will be submitted to the IRB in writing at the time that the 

research study is submitted for renewal. 

These experiments will provide basic knowledge that, in addition to developing theories of speech processing, 

may extend to communication disorders, phonological disorders, developmental disorders such as autism, 

hearing impairment, and assistive technologies such as hearing aids and automatic speech recognition. The 

research involves minimal risk. 

14. References

Barba, C., et al. "Ictal clinical and scalp-EEG findings differentiating temporal lobe epilepsies from temporal ‘plus’ 

epilepsies." Brain 130.7 (2007): 1957-1967.

Barba, Carmen, et al. "Temporal plus epilepsy is a major determinant of temporal lobe surgery failures." Brain

139.2 (2016): 444-451.

Contact PD/PI: Abel, Taylor 

Data and Safety Monitoring Plan Page 59
STRUCTURE OF THE STUDY TEAM

Multiple PI Leadership Plan

PI Abel and PI Holt have established an organizational structure for the conduct of all aspects of the research. 

Although this is a new research team, there is a clear structure that has already demonstrably produced excellent 

results, as exemplified by the preliminary data collected over the last year. Consistent with team-based science 

PI Abel arrived in Pittsburgh, PA last year to join the Children’s Hospital of Western Pennsylvania (Neurological 

Surgery) which is in the vicinity of Carnegie Mellon University, where PI Holt directs her laboratory. Abel and Holt 

have collaborated closely in developing the scientific premise and the research plan described in the proposal. 

The work draws synergistically from their complementary research expertise. The collaboration is supported by 

the geographic adjacency of their laboratories (a short drive or bus ride). Weekly joint research meetings through 

the Pittsburgh Cognitive Auditory Neuroscience research network and meetings specific to this project will foster 

continued intellectual exchange and co-mentorship of junior members of the research team. The MPIs will be 

involved in all scientific aspects of the proposed project; decision-making will be joint. They will collaborate to 

achieve the specific aims and to ensure that both data collection sites comply with US laws, DHHS and NIH 

policies. Both sites will have human participant research approvals to cover the scope of the proposed projects, 

as detailed in the sIRB Plan. 

Leadership: The MPIs will oversee the project jointly, with PI Abel serving as the NIH contact who will assume, 

with his institution, the fiscal and administrative management responsibilities. The MPIs 

will share the responsibility for the scientific direction of the research, its progress, theoretical interpretation, and 

dissemination. The MPIs will jointly oversee the entire Research Program, the implementation of the scientific 

agenda and the leadership plan as equals. 

Communication: Communication among the MPIs occurs at least bi-weekly by electronic conference/phone or 

in person; there is more frequent contact by email, online messaging, and electronic file transfer as dictated by 

project demands. In these meetings, the MPIs discuss project progress, student training, experimental design, 

integration of the research efforts, data analysis, manuscript preparation and administrative responsibilities. 

Additionally, key personnel and students within the MPIs’ laboratories maintain regular contact about research 

progress through joint lab meetings as well as through electronic data sharing, email, meet-ups at local events, 

and phone conversations. The MPIs will assume the responsibility for informing the other about research 

progress in his/her own laboratory. 

Conflict Resolution: Should a conflict arise the MPIs shall meet to attempt to resolve any dispute. If this meeting 

fails to resolve the dispute, the appropriate Department administrators from the MPIs’ home departments will 

attempt to settle the dispute in good faith. However, if the Department administrators fail to resolve the conflict 

within 30 business days, an impartial arbitration committee will be formed consisting of an impartial senior 

executive and a third impartial member mutually agreed upon by both the MPIs. This committee will assist in 

resolving the dispute. No members of the arbitration committee will be directly involved in the proposed research 

or in the disagreement. 

Data Sharing will be open between the laboratories. The MPIs will share their respective research results with 

the each other, key personnel, staff and students. To facilitate data sharing, the MPIs have established a policy 

of electronic data sharing that serves to support the ongoing collaboration.

Publication Authorship will reflect the relative scientific contributions of the MPIs, key personnel, and students.

Any Intellectual Property developed as part of the proposed project will be negotiated by the Technology 

Transfer Offices at and Carnegie Mellon University to create a cross-institution 

agreement. In the event of such a negotiation, an Intellectual Property Committee of representatives from each 

institution will coordinate to protect intellectually property developed by the MPIs according to policies 

established in the agreement.

Contact PD/PI: Abel, Taylor 

Overall Structure of the Study Team Page 60
Should there be a Change in MPI Location, attempts will be made to transfer the relevant portion of the grant 

to the new institution.

A Post-doctoral Scholar will be involved in the research. Based at Children’s Hospital, s/he will be involved in 

all aspects of the project, providing continuity. S/he will have full access to training in the PIs’ laboratories, as 

well. S/he will spend time training with each PI, which will further support project continuity across sites. The 

post-doctoral scholar will be involved in all aspects of the work. The MPIs will work with the post-doctoral scholar 

to develop a post-doctoral mentoring plan, including an individual development plan. A PhD student based in 

PI Holt’s laboratory will be involved in all aspects of the project, with mentoring by the senior personnel with 

primary responsibility by PI Holt with substantial input from PI Abel. A part-time Clinical Research Coordinator

will assist with testing patients under the supervision of PI Abel. 

Contact PD/PI: Abel, Taylor 

Overall Structure of the Study Team Page 61
Section 4 - Protocol Synopsis (Study 1)

4.1. Study Design

4.1.a. Detailed Description

Epilepsy or brain tumor patients who undergo sEEG implantation for clinical purposes are approached for potential

 participation in this research study. Consented patients then participate in an array of behavioral tasks during sEEG

 recording from the STP.

4.1.b. Primary Purpose Basic Science

4.1.c. Interventions

Type Name Description

Other Behavioral Tasks Behavioral tasks are administered during sEEG recordings to examine

 changes in brain activity.

4.1.d. Study Phase N/A

Is this an NIH-defined Phase III Clinical Trial? ❍ Yes ● No

4.1.e. Intervention Model Single Group

4.1.f. Masking ❍ Yes ● No

❏ Participant ❏ Care Provider ❏ Investigator ❏ Outcomes Assessor

4.1.g. Allocation N/A

4.2. Outcome Measures

Type Name Time Frame Brief Description

Primary sEEG Recording Acute Neural signals are acquired from sEEG recordings

 during behavioral tasks that vary F0 and VOT.

4.3. Statistical Design and Power 2020_NIH_R21_PerceptualWeights_StatisticalDesi.pdf

4.4. Subject Participation Duration Up to 7 days after sEEG implantation.

4.5. Will the study use an FDA-regulated intervention? ❍ Yes ● No

4.5.a. If yes, describe the availability of Investigational

Product (IP) and Investigational New Drug (IND)/

Investigational Device Exemption (IDE) status

4.6. Is this an applicable clinical trial under FDAAA? ❍ Yes ● No

4.7. Dissemination Plan 2020_NIH_R21_PerceptualWeights_DisseminationPl.pdf

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196 Received Date:

2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 62
STATISTICAL DESIGN AND POWER ANALYSIS

Statistical Design, Power and Analytic Approach. We will test neurosurgical patients (ages 15-25 years) 

undergoing evaluation and treatment who require chronic (>7 d) sEEG implantation and recording of the temporal 

lobe for seizure localization or brain mapping. One set of behavioral tasks, and corresponding 

electrophysiological measures, will allow us to address Aim 1. Another will address Aim 2. Patient permitting, 

our goal is to have the same patient complete all tasks. All participants will undergo neuropsychological tests 

and will have detailed history/demographic information as part of their clinical battery. 

Power and Sample Size. Robust behavioral effects will allow examination of perceptual weighs in neural 

representation. Using behavioral effect sizes from Figure 2 pilot data to estimate the sample size required for a 

predicted power of 0.8 (two-tailed alpha at .05) yields a sample of N=25. This leaves open the issue of power for 

neural measures. Our pilot EEG data with the same task/stimuli revealed robust effects with N=23 (Figure A), 

reassuring that N=25 is a reliable target for EEG. The SNR advantages of sEEG versus EEG, and our sEEG 

pilot data in Figure 3, suggest that this sample size will be more than sufficient for sEEG measures, especially 

as they will be utilized in a within-patient/within-electrode experimental design. 

Analyses. Specific pre-registered analyses will assess the hypotheses outlined in the Approach section of the 

proposal. Specifically, we will test the following hypotheses: 

(H1) broadly, the relative perceptual weight of VOT and F0, as measured behaviorally, will be reflected 

in cortical response, (H2) with modulation as a function of baseline perceptual weights, (H3) shifts 

experimentally invoked by a change in listening context by presenting speech in noise, (H4) and by 

introducing an ‘accent’ that shifts short-term input regularities across VOT and F0. In the latter case, our 

approach will allow us to test the specific directional hypothesis (H5) that F0 perceptual weights in the 

DBSL paradigm are both exaggerated by Canonical input regularities that cleanly convey a VOTxF0 

correlation consistent with English and that F0 perceptual weights are down-weighted upon introduction 

of a regularity that violates the typical pattern of English (supported by scalp EEG pilot, Figure A). Our 

use of sEEG allows us to evaluate these hypotheses across the supratemporal plane thereby testing the 

strong, and falsifiable, hypothesis (H6) that adaptive plasticity effects are present in HG versus (H7) 

apparent only at higher levels of the cortical hierarchy. Our ability to test these hypotheses is 

complemented by sEEG electrode placement in cortical regions outside STP (see Figure 3) which will 

support secondary hypotheses and serve as control electrode sites. 

The study design is justified our extensive behavioral research demonstrating the feasibility of the project 

rationale. On the electrophysiological side, our pilot data (Figures A,3) demonstrate the feasibility of recording 

robust sEEG and EEG signals responsive to the acoustic dimensions we manipulate. This will provide clear, 

informative, interpretable data with which to evaluate the hypotheses listed above. 

Evaluating the Hypotheses: Behavioral Analyses. We will evaluate the behavioral impact of the VOT and F0 

acoustic dimensions on classification using mixed-effects logistic regression (with patient as a random effect, 

stimulus VOT and F0 as fixed effects and classification responses as the outcome). Following our prior work,14,25–

29 perceptual weights for the dimensions will be computed for each patient as the correlation between dimension 

values and the proportion of peach classifications across all stimuli in the VOTxF0 stimulus grid with absolute 

values of the correlation coefficients normalized to sum to one as an index of relative perceptual weight in quiet 

(Aim 1) and in noise (Aim 2). To examine the impact of a change in ‘accent’ in the DBSL paradigm, we will use 

mixed logit models with responses as a function of patient, block, test stimulus F0 and the interaction between 

block and F0 (Aim 2). Patient will be modeled as a random effect, with the other factors as fixed effects. 

Evaluating the Hypotheses: Neural Analyses. We will take a multi-pronged analysis approach. Our pilot data 

in Figure 3 demonstrate that high-γ activity (HGA) is modulated by graded acoustic details across the VOT and 

F0 dimensions across electrodes placed in the STP. Following the analysis pipeline used in our pilot data 

analyses, we will specifically examine stimulus-time-locked HGA to the Test stimuli, which possess consistent, 

perceptually-ambiguous VOT and differentiated F0. We have specific, directional predictions (detailed above) 

Contact PD/PI: Abel, Taylor 

Statistical Design and Power Page 63
regarding how HGA to F0-differentiated Test stimuli will vary according to perceptual weight in behavior. To 

briefly recap, we expect that HGA to Test stimuli will be better differentiated (1) in the Speech-in-Noise compared 

to the Clear baseline context (because F0 carries greater perceptual weight); (2) in the Canonical, compared to 

the Reverse, context; (3) in the Canonical, compared to the Baseline Clear Speech context (because stimulus 

statistics exaggerate the dimension regularity in behavior). 

High-gamma amplitude will be calculated using an approach that utilizes the Hilbert transform. Specifically, the 

signal will be filtered into 8 subbands, logarithmically spaced between 70-150 Hz. For each subband, we will 

calculate the amplitude (absolute value) of the analytic signal, which is estimated using the Hilbert transform. 

Each subband will be normalized to its baseline mean and standard deviation, estimated across trials. The HGA 

estimate is then the mean across these subbands. 

We will use least-squares linear regression neural encoding models to investigate relationships between acoustic 

stimulus dimensions and STP neural responses during the baseline quiet context, in which stimuli sample a 2-d 

F0 x VOT grid. This approach will allow us to identify the subset of electrodes and temporal windows that encode 

at least one of these 2 dimensions at baseline; targeted analysis on this subset (described below) will then be 

used to compare listening contexts. 

Encoding model inputs will consist of F0 and VOT, with an output of channel- and time-specific HGA. For a given 

electrode, individual models will be built using single trial data and a sliding window, allowing us to identify the 

temporal window relative to stimulus onset that yields significant models. Model quality will be assessed in two 

ways. First, using models built on all trials, we will calculate the regression F-statistic, which determines if any 

coefficients are significant. This will be compared to null distributions estimated with permutation tests that shuffle 

data across trials. Second, goodness-of-fit will be assessed for significant models by performing leave-one-out 

cross-validation and calculating R2

, the proportion of variance in neural activity explained by the model. Finally, 

we will assess the relative encoding strength of each acoustic dimension using model coefficient t-statistics. In 

summary, this approach will allow us to identify the timing and anatomical location of F0 and VOT encoding 

during baseline quiet listening conditions for which dimensions are sampled orthogonally (as in Figure 1a). 

To characterize shifts in neural encoding across listening contexts, we will investigate Noise using encoding 

models and Canonical/Reverse contexts using cluster-based approaches.

32,49 Encoding models will be built from 

the Noise context for all channels and timepoints, using the same approach as Baseline. If a channel/timepoint 

is significant in either Baseline or Noise, the F0 and VOT coefficient t-statistics will be compared across contexts. 

We hypothesize that encoding of F0 will strengthen and VOT will weaken in Noise, as measured by changes in 

t-statistic magnitudes across multiple models. Next, we will compare neural responses for each of the two Test

stimuli (which were embedded in the F0 x VOT grids, Figure 1a) between Baseline vs. Canonical and Canonical 

vs. Reverse. The clustering approach will look only at Baseline significant channels and time windows and 

involves randomly assigning listening context labels to single-trial data followed by a t-test at each time step. 

Across all permutations, a criterion value will be established for each timepoint (>95% of absolute value of t). 

For each of these permutations, we will next determine whether its value exceeds criterion across timepoints, 

and for how many timepoints it exceeds criterion (a ‘cluster’). For each cluster, t values will be summed and 

assigned to all points in the cluster, with the largest summed cluster value stored for each permutation. This will 

create a null distribution of 1000 cluster values. We then will establish whether the cluster size calculated across 

real neural data (organized according to listening context) exceeds the 95% permutation-based cluster values 

such that p<.001 indicates an observed cluster is greater than all permutation-based clusters. Using this 

approach, we will identify context-dependent shifts in HGA responses, which we predict will reflect observed 

shifts in perceptual weights. Namely, we hypothesize that HGA responses in F0-encoding channels will be 

enhanced in the Canonical context relative to both Baseline and the Reverse context. 

Rigor and Reproducibility. Analyses will be controlled for multiple comparisons, with sex as a co-variate in our 

analyses. We will use pre-registration and provide access to all the deidentified source data.

Contact PD/PI: Abel, Taylor 

Statistical Design and Power Page 64
DISSEMINATION PLAN

The structure of this research activity is staged to meet the Specific Aims. We anticipate that work directed 

toward each aim will yield a major publication and with an additional paper corresponding to the collected data 

and the materials developed during the research program. Additionally, we anticipate 2-3 conference 

presentations per aim across the project period. 

We will target the publications to researchers including those in cognitive neuroscience, auditory cognitive 

neuroscience, and speech processing. Other papers will be directed toward the neurosurgery community 

regarding benefits, challenges and outcomes of our work with stereoelectroencephalography (sEEG). Together 

the MPIs will seek to disseminate research through peer-reviewed publications, colloquium talks, and conference 

talks. They will be responsible for ensuring that all reportable findings are published in a peer-reviewed format 

in a prompt manner.

The MPIs also will endeavor to make the results broadly available to the general public supporting the work. 

Project personnel will be available to speak to the lay public; both the PIs have a strong history of public 

engagement. 

The results of the studies will also be made available to the participants. This will be accomplished through layaudience descriptions of research articles and conference proceedings on the PIs’ websites and by maintaining 

contact with all participants (should they choose to provide their contact information). This ensures that 

participants are aware of the results of the studies in which they volunteered as research participants. 

Additionally, our publications will comply with the NIH Public Access Policy assuring the general public has 

access to the research they fund through their tax dollars. 

Finally, we will make the data accessible through a public database as detailed in our Data Management Plan. 

Contact PD/PI: Abel, Taylor 

Dissemination Plan Page 65
Delayed Onset Studies

Delayed 

Onset Study#

Study Title Anticipated Clinical 

Trial?

Justification

The form does not have any delayed onset studies

Tracking Number: GRANT13169283 Funding Opportunity Number: PA-20-196. Received Date: 2020-07-15T15:11:48.000-04:00

Contact PD/PI: Abel, Taylor 

Page 66
STRUCTURE OF THE STUDY TEAM

Multiple PI Leadership Plan

PI Abel and PI Holt have established an organizational structure for the conduct of all aspects of the research. 

Although this is a new research team, there is a clear structure that has already demonstrably produced excellent 

results, as exemplified by the preliminary data collected over the last year. Consistent with team-based science 

PI Abel arrived in Pittsburgh, PA last year to join the Children’s Hospital of Western Pennsylvania (Neurological 

Surgery) which is in the vicinity of Carnegie Mellon University, where PI Holt directs her laboratory. Abel and Holt 

have collaborated closely in developing the scientific premise and the research plan described in the proposal. 

The work draws synergistically from their complementary research expertise. The collaboration is supported by 

the geographic adjacency of their laboratories (a short drive or bus ride). Weekly joint research meetings through 

the Pittsburgh Cognitive Auditory Neuroscience research network and meetings specific to this project will foster 

continued intellectual exchange and co-mentorship of junior members of the research team. The MPIs will be 

involved in all scientific aspects of the proposed project; decision-making will be joint. They will collaborate to 

achieve the specific aims and to ensure that both data collection sites comply with US laws, DHHS and NIH 

policies. Both sites will have human participant research approvals to cover the scope of the proposed projects, 

as detailed in the sIRB Plan. 

Leadership: The MPIs will oversee the project jointly, with PI Abel serving as the NIH contact who will assume, 

with his institution, the fiscal and administrative management responsibilities. The MPIs 

will share the responsibility for the scientific direction of the research, its progress, theoretical interpretation, and 

dissemination. The MPIs will jointly oversee the entire Research Program, the implementation of the scientific 

agenda and the leadership plan as equals. 

Communication: Communication among the MPIs occurs at least bi-weekly by electronic conference/phone or 

in person; there is more frequent contact by email, online messaging, and electronic file transfer as dictated by 

project demands. In these meetings, the MPIs discuss project progress, student training, experimental design, 

integration of the research efforts, data analysis, manuscript preparation and administrative responsibilities. 

Additionally, key personnel and students within the MPIs’ laboratories maintain regular contact about research 

progress through joint lab meetings as well as through electronic data sharing, email, meet-ups at local events, 

and phone conversations. The MPIs will assume the responsibility for informing the other about research 

progress in his/her own laboratory. 

Conflict Resolution: Should a conflict arise the MPIs shall meet to attempt to resolve any dispute. If this meeting 

fails to resolve the dispute, the appropriate Department administrators from the MPIs’ home departments will 

attempt to settle the dispute in good faith. However, if the Department administrators fail to resolve the conflict 

within 30 business days, an impartial arbitration committee will be formed consisting of an impartial senior 

executive and a third impartial member mutually agreed upon by both the MPIs. This committee will assist in 

resolving the dispute. No members of the arbitration committee will be directly involved in the proposed research 

or in the disagreement. 

Data Sharing will be open between the laboratories. The MPIs will share their respective research results with 

the each other, key personnel, staff and students. To facilitate data sharing, the MPIs have established a policy 

of electronic data sharing that serves to support the ongoing collaboration.

Publication Authorship will reflect the relative scientific contributions of the MPIs, key personnel, and students.

Any Intellectual Property developed as part of the proposed project will be negotiated by the Technology 

Transfer Offices at Carnegie Mellon University to create a cross-institution 

agreement. In the event of such a negotiation, an Intellectual Property Committee of representatives from each 

institution will coordinate to protect intellectually property developed by the MPIs according to policies 

established in the agreement.

Contact PD/PI: Abel, Taylor 

Multiple PD/PI Leadership Plan Page 67
Should there be a Change in MPI Location, attempts will be made to transfer the relevant portion of the grant 

to the new institution.

A Post-doctoral Scholar will be involved in the research. Based at Children’s Hospital, s/he will be involved in 

all aspects of the project, providing continuity. S/he will have full access to training in the PIs’ laboratories, as 

well. S/he will spend time training with each PI, which will further support project continuity across sites. The 

post-doctoral scholar will be involved in all aspects of the work. The MPIs will work with the post-doctoral scholar 

to develop a post-doctoral mentoring plan, including an individual development plan. A PhD student based in 

PI Holt’s laboratory will be involved in all aspects of the project, with mentoring by the senior personnel with 

primary responsibility by PI Holt with substantial input from PI Abel. A part-time Clinical Research Coordinator

will assist with testing patients under the supervision of PI Abel. 

Contact PD/PI: Abel, Taylor 

Multiple PD/PI Leadership Plan Page 68
REFERENCES

1 Lisker L. “Voicing” in English: A Catalogue of Acoustic Features Signaling /b/ Versus /p/ in Trochees. 

Lang Speech 1986. https://doi.org/10.1177/002383098602900102.

2 Yi HG, Leonard MK, Chang EF. The Encoding of Speech Sounds in the Superior Temporal Gyrus. 

Neuron 2019. https://doi.org/10.1016/j.neuron.2019.04.023. 

3 Mesgarani N, Cheung C, Johnson K, Chang EF. Phonetic feature encoding in human superior temporal 

gyrus. Science (80- ) 2014. https://doi.org/10.1126/science.1245994. 

4 Blumstein SE, Stevens KN. Acoustic Invariance In Speech Production: Evidence From Measurements 

Of The Spectral Characteristics Of Stop Consonants. J Acoust Soc Am 1979. 

https://doi.org/10.1121/1.383319.

5 Francis AL, Kaganovich N, Driscoll-Huber C. Cue-specific effects of categorization training on the 

relative weighting of acoustic cues to consonant voicing in English. J Acoust Soc Am 2008. 

https://doi.org/10.1121/1.2945161. 

6 Lotto AJ, Holt LL. Putting phonetic context effects into context: A commentary on Fowler (2006). Percept 

Psychophys 2006. https://doi.org/10.3758/BF03193667.

7 Iverson P, Kuhl PK. Mapping the perceptual magnet effect for speech using signal detection theory and 

multidimensional scaling. J Acoust Soc Am 1995. https://doi.org/10.1121/1.412280. 

8 Toscano JC, McMurray B. Cue integration with categories: Weighting acoustic cues in speech using 

unsupervised learning and distributional statistics. Cogn Sci 2010. https://doi.org/10.1111/j.1551-

6709.2009.01077.x. 

9 Scharinger M, Henry MJ, Erb J, Meyer L, Obleser J. Thalamic and parietal brain morphology predicts 

auditory category learning. Neuropsychologia 2014. 

https://doi.org/10.1016/j.neuropsychologia.2013.09.012. 

10 Nittrouer S, Chapman C. The Effects of Bilateral Electric and Bimodal Electric—Acoustic Stimulation on 

Language Development. Trends Amplif 2009. https://doi.org/10.1177/1084713809346160. 

11 Wellmann C, Holzgrefe J, Truckenbrodt H, Wartenburger I, Höhle B. How each prosodic boundary cue 

matters: Evidence from German infants. Front Psychol 2012. https://doi.org/10.3389/fpsyg.2012.00580. 

12 Winn MB, Chatterjee M, Idsardi WJ. Roles of voice onset time and F0 in stop consonant voicing 

perception: Effects of masking noise and low-pass filtering. J Speech, Lang Hear Res 2013. 

https://doi.org/10.1044/1092-4388(2012/12-0086).

13 Hedrick MS, Younger MS. Perceptual weighting of stop consonant cues by normal and impaired 

listeners in reverberation versus noise. J Speech, Lang Hear Res 2007. https://doi.org/10.1044/1092-

4388(2007/019).

14 Idemaru K, Holt LL, Seltman H. Individual differences in cue weights are stable across time: The case of 

Japanese stop lengths. J Acoust Soc Am 2012. https://doi.org/10.1121/1.4765076. 

15 Kong EJ, Edwards JR. Individual Differences in Speech Perception: Evidence from Visual Analogue 

Scaling and Eye-Tracking. 

16 Kong EJ, Edwards J. Individual differences in categorical perception of speech: Cue weighting and 

executive function. J Phon 2016. https://doi.org/10.1016/j.wocn.2016.08.006.

17 Schertz J, Cho T, Lotto A, Warner N. Individual differences in phonetic cue use in production and 

perception of a non-native sound contrast. J Phon 2015. https://doi.org/10.1016/j.wocn.2015.07.003. 

18 Schertz J, Cho T, Lotto A, Warner N. Individual differences in perceptual adaptability of foreign sound 

categories. Attention, Perception, Psychophys 2016. https://doi.org/10.3758/s13414-015-0987-1.

19 Shultz AA, Francis AL, Llanos F. Differential cue weighting in perception and production of consonant 

voicing. J Acoust Soc Am 2012. https://doi.org/10.1121/1.4736711.

20 Hazan V, Barrett S. The development of phonemic categorization in children aged 6-12. J Phon 2000. 

https://doi.org/10.1006/jpho.2000.0121. 

Contact PD/PI: Abel, Taylor 

References Cited Page 69
21 Zevin JD. A sensitive period for shibboleths: The long tail and changing goals of speech perception over 

the course of development. Dev Psychobiol 2012. https://doi.org/10.1002/dev.20611. 

22 Bradlow AR, Bent T. Perceptual adaptation to non-native speech. Cognition 2008. 

https://doi.org/10.1016/j.cognition.2007.04.005. 

23 Greenspan SL, Nusbaum HC, Pisoni DB. Perceptual Learning of Synthetic Speech Produced by Rule. J 

Exp Psychol Learn Mem Cogn 1988. https://doi.org/10.1037/0278-7393.14.3.421.

24 Bertelson P, Vroomen J, De Gelder B. Visual Recalibration of Auditory Speech Identification: A McGurk 

Aftereffect. Psychol Sci 2003. https://doi.org/10.1046/j.0956-7976.2003.psci_1470.x. 

25 Lehet M, Holt LL. Dimension-Based Statistical Learning Affects Both Speech Perception and 

Production. Cogn Sci 2017. https://doi.org/10.1111/cogs.12413. 

26 Zhang X, Holt LL. Simultaneous tracking of coevolving distributional regularities in speech. J Exp 

Psychol Hum Percept Perform 2018. https://doi.org/10.1037/xhp0000569. 

27 Idemaru K, Holt LL. Specificity of dimension-based statistical learning in word recognition. J Exp Psychol 

Hum Percept Perform 2014. https://doi.org/10.1037/a0035269.

28 Liu R, Holt LL. Dimension-based statistical learning of vowels. J Exp Psychol Hum Percept Perform

2015. https://doi.org/10.1037/xhp0000092. 

29 Idemaru K, Holt LL. Word Recognition Reflects Dimension-based Statistical Learning. J Exp Psychol 

Hum Percept Perform 2011. https://doi.org/10.1037/a0025641.

30 Mesgarani N, Chang EF. Selective cortical representation of attended speaker in multi-talker speech 

perception. Nature 2012. https://doi.org/10.1038/nature11020.

31 Steinschneider M, Volkov IO, Fishman YI, Oya H, Arezzo JC, Howard MA. Intracortical responses in 

human and monkey primary auditory cortex support a temporal processing mechanism for encoding of 

the voice onset time phonetic parameter. Cereb Cortex 2005. https://doi.org/10.1093/cercor/bhh120. 

32 Sjerps MJ, Fox NP, Johnson K, Chang EF. Speaker-normalized sound representations in the human 

auditory cortex. Nat Commun 2019. https://doi.org/10.1038/s41467-019-10365-z. 

33 Cheung C, Hamiton LS, Johnson K, Chang EF. The auditory representation of speech sounds in human 

motor cortex. Elife 2016. https://doi.org/10.7554/eLife.12577.

34 Akbari H, Khalighinejad B, Herrero JL, Mehta AD, Mesgarani N. Towards reconstructing intelligible 

speech from the human auditory cortex. Sci Rep 2019. https://doi.org/10.1038/s41598-018-37359-z. 

35 Khalighinejad B, da Silva GC, Mesgarani N. Dynamic encoding of acoustic features in neural responses 

to continuous speech. J Neurosci 2017. https://doi.org/10.1523/JNEUROSCI.2383-16.2017.

36 Holdgraf CR, De Heer W, Pasley B, Rieger J, Crone N, Lin JJ, et al. Rapid tuning shifts in human 

auditory cortex enhance speech intelligibility. Nat Commun 2016. https://doi.org/10.1038/ncomms13654. 

37 McClelland JL, Mirman D, Holt LL. Are there interactive processes in speech perception? Trends Cogn 

Sci 2006. https://doi.org/10.1016/j.tics.2006.06.007. 

38 Norris D, McQueen JM, Cutler A. Merging information in speech recognition: Feedback is never 

necessary. Behav Brain Sci 2000. https://doi.org/10.1017/S0140525X00003241. 

39 McClelland JL, Elman JL. The TRACE model of speech perception. Cogn Psychol 1986. 

https://doi.org/10.1016/0010-0285(86)90015-0.

40 Nourski K V., Reale RA, Oya H, Kawasaki H, Kovach CK, Chen H, et al. Temporal envelope of timecompressed speech represented in the human auditory cortex. J Neurosci 2009. 

https://doi.org/10.1523/JNEUROSCI.3065-09.2009.

41 Howard MA, Volkov IO, Abbas PJ, Damasio H, Ollendieck MC, Granner MA. A chronic microelectrode 

investigation of the tonotopic organization of human auditory cortex. Brain Res 1996. 

https://doi.org/10.1016/0006-8993(96)00315-0.

42 Dick FK, Lehet MI, Callaghan MF, Keller TA, Sereno MI, Holt LL. Extensive tonotopic mapping across 

Contact PD/PI: Abel, Taylor 

References Cited Page 70
auditory cortex is recapitulated by spectrally directed attention and systematically related to cortical 

myeloarchitecture. J Neurosci 2017. https://doi.org/10.1523/JNEUROSCI.1436-17.2017.

43 Bajo VM, Nodal FR, Moore DR, King AJ. The descending corticocollicular pathway mediates learninginduced auditory plasticity. Nat Neurosci 2010. https://doi.org/10.1038/nn.2466. 

44 Dahmen JC, King AJ. Learning to hear: plasticity of auditory cortical processing. Curr Opin Neurobiol

2007. https://doi.org/10.1016/j.conb.2007.07.004. 

45 Yin P, Fritz JB, Shamma SA. Rapid spectrotemporal plasticity in primary auditory cortex during 

behavior. J Neurosci 2014. https://doi.org/10.1523/JNEUROSCI.2799-13.2014.

46 Abel TJ, Osorio RV, Amorim-Leite R, Mathieu F, Kahane P, Minotti L, et al. Frameless robot-assisted 

stereoelectroencephalography in children: Technical aspects and comparison with Talairach frame 

technique. J Neurosurg Pediatr 2018. https://doi.org/10.3171/2018.1.PEDS17435. 

47 Roark CL, Holt LL. Auditory information-integration category learning in young children and adults. J Exp 

Child Psychol 2019. https://doi.org/10.1016/j.jecp.2019.104673. 

48 Hufnagle DG, Holt LL, Thiessen ED. Spectral information in nonspeech contexts influences children’s 

categorization of ambiguous speech sounds. J Exp Child Psychol 2013. 

https://doi.org/10.1016/j.jecp.2013.05.008. 

49 Maris E, Oostenveld R. Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods

2007. https://doi.org/10.1016/j.jneumeth.2007.03.024.

50 Di Liberto GM, O’Sullivan JA, Lalor EC. Low-frequency cortical entrainment to speech reflects phonemelevel processing. Curr Biol 2015. https://doi.org/10.1016/j.cub.2015.08.030. 

51 Guediche S, Blumstein SE, Fiez JA, Holt LL. Speech perception under adverse conditions: Insights from 

behavioral, computational, and neuroscience research. Front Syst Neurosci 2014. 

https://doi.org/10.3389/fnsys.2013.00126. 

52 Fritz J, Shamma S, Elhilali M, Klein D. Rapid task-related plasticity of spectrotemporal receptive fields in 

primary auditory cortex. Nat Neurosci 2003. https://doi.org/10.1038/nn1141. 

53 Näätänen R, Picton T. The N1 wave of the human electric and magnetic response to sound: A review 

and an analysis of the component structure. Psychophys 1987. https://doi.org/ 10.1111/j.1469-

8986.1987.tb00311.x

54 Linden DEJ. The P300: Where in the brain is it produced and what does it tell us? The Neuroscientist

2005. https://doi.org/10.1177/1073858405280524

55 Ishishita Y, Kunii N, Shimada S, Ibayashi K, Tada M, Kirihara K, Kawai K, Uka T, Kasai K, Nobuhito S. 

Devaiance detection is the dominant component of auditory contextual processing in the lateral superior 

temporal gyrus: a human ECoG study. Hum Brain Mapp 2019. https://doi.org/ 10.1002/hbm.24438

56 Toscano J, McMurray B, Dennhardt J, Luck SJ. Continuous perception and graded categorizaiton: 

Electrophysiological evidence for a linear relationship between the acoustic signal and perceptual 

encoding of speech. 2010. Psychol Sci https://doi.org/ 10.1177/0956797610384142

57 Pereira O, Gao YA, Toscano JC. Perceptual encoding of natural speech sounds revealed by the N1 

event-related potential response. 2018. Aud Percept Cogn https:// doi.org/ 10.1080/ 25742442.2018.

1545106

58 Mäkelä AM, Alku P, Mäkinen V, Valtonen J, May P, Titinen H. Human cortical dynamics determined by 

speech fundamental frequency. 2002. Neuroimage https://doi.org/ 10.1006/nimg.2002.1279

59 Frank J, Muhlack B, Zebe F, Scharinger M. Contributions of pitch and spectral information to cortical 

vowel categorization. 2020 J Phonetics https://doi.org/10.1016/j.wocn.2020.100963 

60 Peelle J, Johnsrude IS, Davis MH. Hierarchical processing for speech in human auditory cortex and 

beyond. 2010 Front Hum Neurosci https://doi.org/ 10.3389/fnhum.2010.00051

Contact PD/PI: Abel, Taylor 

References Cited Page 71
DATA SHARING PLAN

The MPIs will assume mutual responsibility for data sharing between laboratories. The research team 

appreciates the importance of sharing research NIH-sponsored research and will aim to make the data as widely 

and freely available as possible while safeguarding participant confidentiality and privacy. We seek to provide a 

stable, reliable and cost-effective means for distributing data, with appropriate protections for confidentiality and 

data longevity. Regarding the former, all data will be de-identified at the time of collection to a pre-assigned 

participant ID code. Data and metadata will be stored using the simplest possible formats to assure data 

longevity. The lowest level of individual trial-level data will be available in a de-identified format. Each type of 

data will be linked to detailed notes regarding how the data were generated, the metadata, and the analytical 

methods used to summarize and analyze them. 

The MPIs will maintain the datasets privately until publication of their results. Once published, all datasets will 

be opened to secondary research by both internal and external collaborators. Metadata for the data sets, which 

includes information about the quantity of data, the domain, and research objectives (if entered by the MPIs), 

are always public, even if the data set is not. The data should be particularly useful for secondary analyses by 

multiple groups. During the project lifecycle, data set access will be restricted to the core research team. This 

includes the MPIs and project personnel. At the conclusion of the project, the datasets and related experimental 

materials will be made public and accessible through publicly available web server and/or through sources such 

as NeuroVault (depending on analysis type). In addition, all analysis scripts built as part of this project will be 

made publicly available via open source code development websites. Attracting researchers to perform 

secondary analysis on these data sets will occur on a lab-by-lab basis, based on presentation of works in 

scientific publications and professional meetings. Once published, datasets may be opened to secondary 

research by both internal and external collaborators by submission to data sharing websites. 

Contact PD/PI: Abel, Taylor 

Resource Sharing Plan(s) Page 72