PI: Munson, Benjamin R Title: Race, Ethnicity, and Speech Intelligibility in Normal Hearing and Hearing 

Impairment 

Received: 10/15/2018 Opportunity: PA-18-489 

Clinical Trial:Not Allowed 

Council: 05/2019 

Competition ID: FORMS-E FOA Title: NIH Exploratory/Developmental Research Grant Program ( Parent R21 

Clinical Trial Not Allowed) 

1R21DC018070-01 Dual: MD Accession Number: 4227645 

IPF: 1450402 Organization: 

Former Number: Department: Speech-Language-Hearing Sci. 

IRG/SRG: LCOM AIDS: N Expedited: N 

Subtotal Direct Costs 

(excludes consortium F&A) 

Year 1: 

Animals: N 

Humans: Y 

Clinical Trial: N 

Current HS Code: 30 

HESC: N 

New Investigator: 

Early Stage Investigator: 

Senior/Key Personnel: Organization: Role Category: 

Benjamin Munson PD/PI 

Matthew Winn Consultant 

JEFFRY SIMPSON Consultant 

Andrew Oxenham Co-Investigator 

Peggy Nelson Co-Investigator 

Evelyn DaviesVenn Consultant 

Molly Babel Consultant 
OMB Number 4040-0001 

Expiration Date 10/31/2019 

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

APPLICATION FOR FEDERAL ASSISTANCE 

SF 424 (R&R) 

3. DATE RECEIVED BY STATE State Application Identifier 

1. TYPE OF SUBMISSION* 4.a. Federal Identifier 

❍ Pre-application ● Application ❍ Changed/Corrected 

Application 

b. Agency Routing Number 

2. DATE SUBMITTED Application Identifier 

PRF 962034 NIH

c. Previous Grants.gov Tracking Number 

5. APPLICANT INFORMATION Organizational DUNS*: 

Legal Name*: REGENTS OF THE UNIVERSITY OF MINNESOTA

Department: 

Division: 

Street1*: 

Person to be contacted on matters involving this application 

Prefix: First Name*: Victoria Middle Name: Last Name*: Troxler Suffix: 

Position/Title: Grants Administrator

Street1*: 

6. EMPLOYER IDENTIFICATION NUMBER (EIN) or (TIN)* 

7. TYPE OF APPLICANT* H: Public/State Controlled Institution of Higher Education

Other (Specify): 

Small Business Organization Type ❍ Women Owned ❍ Socially and Economically Disadvantaged 

8. TYPE OF APPLICATION* If Revision, mark appropriate box(es). 

● New ❍ Resubmission ❍ A. Increase Award ❍ B. Decrease Award ❍ C. Increase Duration

❍ Renewal ❍ Continuation ❍ Revision ❍ D. Decrease Duration ❍ E. Other (specify) :

Is this application being submitted to other agencies?* ❍Yes ●No What other Agencies? 

9. NAME OF FEDERAL AGENCY* 

National Institutes of Health

10. CATALOG OF FEDERAL DOMESTIC ASSISTANCE NUMBER 

TITLE: 

11. DESCRIPTIVE TITLE OF APPLICANT'S PROJECT* 

Race, Ethnicity, and Speech Intelligibility in Normal Hearing and Hearing Impairment

12. PROPOSED PROJECT 

Start Date* Ending Date* 

06/01/2019 05/31/2021

13. CONGRESSIONAL DISTRICTS OF APPLICANT 

 Page 1 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

SF 424 (R&R) APPLICATION FOR FEDERAL ASSISTANCE Page 2 

14. PROJECT DIRECTOR/PRINCIPAL INVESTIGATOR CONTACT INFORMATION 

Prefix: First Name*: Benjamin Middle Name: R Last Name*: Munson Suffix: 

Position/Title: Professor

Organization Name*: 

Division: 

Street1*: 

15. ESTIMATED PROJECT FUNDING 

a. Total Federal Funds Requested* 

.00 

16. IS APPLICATION SUBJECT TO REVIEW BY STATE 

EXECUTIVE ORDER 12372 PROCESS?* 

a. YES ❍ THIS PREAPPLICATION/APPLICATION WAS MADE 

AVAILABLE TO THE STATE EXECUTIVE ORDER 12372 

PROCESS FOR REVIEW ON: 

DATE: 

b. NO ● PROGRAM IS NOT COVERED BY E.O. 12372; OR 

❍ PROGRAM HAS NOT BEEN SELECTED BY STATE FOR 

REVIEW 

17. By signing this application, I certify (1) to the statements contained in the list of certifications* and (2) that the statements herein 

are true, complete and accurate to the best of my knowledge. I also provide the required assurances * and agree to comply with 

any resulting terms if I accept an award. I am aware that any false, fictitious, or fraudulent statements or claims may subject me to 

criminal, civil, or administrative penalties. (U.S. Code, Title 18, Section 1001) 

● I agree* 

* The list of certifications and assurances, or an Internet site where you may obtain this list, is contained in the announcement or agency specific instructions. 

18. SFLLL or OTHER EXPLANATORY DOCUMENTATION File Name: 

19. AUTHORIZED REPRESENTATIVE 

Prefix: First Name*: Kevin Middle Name: Last Name*: McKoskey Suffix: 

Position/Title*: Director, Sponsored Projects Administration

Organization Name*: 

Signature of Authorized Representative* 

April Coon

Date Signed* 

10/15/2018

20. PRE-APPLICATION File Name: 

21. COVER LETTER ATTACHMENT File Name: 

Page 2 
Contact PD/PI: Munson, Benjamin R 

Table of Contents 

424 R&R and PHS-398 Specific 

Table Of Contents 

SF 424 R&R Cover Page............................................................................................................1 

Table of Contents...................................................................................................................3 

Performance Sites.....................................................................................................................4 

Research & Related Other Project Information......................................................................5 

Project Summary/Abstract(Description)..............................................................................6 

Project Narrative....................................................................................................................7 

Facilities & Other Resources................................................................................................8 

Equipment...............................................................................................................................9 

Research & Related Senior/Key Person................................................................................10 

PHS398 Cover Page Supplement...........................................................................................45 

PHS 398 Modular Budget........................................................................................................47 

Personnel Justification........................................................................................................50 

PHS 398 Research Plan...........................................................................................................52 

Specific Aims........................................................................................................................53 

Research Strategy................................................................................................................54 

PHS Human Subjects and Clinical Trials Information......................................................60 

Study 1: Race, Ethnicity, and Speech Intelligibility in Hearing Impairment and Normal 

Hearing..............................................................................................................................62 

Inclusion Enrollment Reports......................................................................................68 

Bibliography & References Cited.......................................................................................76 

Letters of Support................................................................................................................79 

Resource Sharing Plan(s)...................................................................................................80 

 Page 3 
Contact PD/Pl: Munson, Benjamin R 

Project/Performance Site Location(s) 

0MB Number: 4040-0010 

Expiration Date: 10/31/2019 

Project/Performance Site Primary Location O I am submitting an application as an individual, and not on behalf of 

Organization Name: 

-

-

Additional Location(s) 

Tracking Number: GRANT1 2729780 

File Name: 

a company, state, local or tribal government, academia, or other type of 

organization. 

-

Page4 

Funding Opportunity Number: PA-18-489. Received Date: 

2018-10-15T15:01 :25.000-04:00 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489. Received Date: 

2018-10-15T15:01:25.000-04:00 

OMB Number: 4040-0001

Expiration Date: 10/31/2019

RESEARCH & RELATED Other Project Information

1. Are Human Subjects Involved?* ● Yes ❍ No 

1.a. If YES to Human Subjects 

Is the Project Exempt from Federal regulations? ❍ Yes ● No 

If YES, check appropriate exemption number: 1 2 3 4 5 6 7 8 

If NO, is the IRB review Pending? ● Yes ❍ No 

IRB Approval Date: 

Human Subject Assurance Number 00000312

2. Are Vertebrate Animals Used?* ❍ Yes ● No 

2.a. If YES to Vertebrate Animals 

Is the IACUC review Pending? ❍ Yes ❍ No 

IACUC Approval Date: 

Animal Welfare Assurance Number 

3. Is proprietary/privileged information included in the application?* ❍ Yes ● No 

4.a. Does this project have an actual or potential impact - positive or negative - on the environment?* ❍ Yes ● No 

4.b. If yes, please explain: 

4.c. If this project has an actual or potential impact on the environment, has an exemption been authorized or an 

environmental assessment (EA) or environmental impact statement (EIS) been performed? 

❍ Yes ❍ No 

4.d. If yes, please explain: 

5. Is the research performance site designated, or eligible to be designated, as a historic place?* ❍ Yes ● No 

5.a. If yes, please explain: 

6. Does this project involve activities outside the United States or partnership with international 

collaborators?* 

● Yes ❍ No 

6.a. If yes, identify countries: Canada

6.b. Optional Explanation: 

Filename 

7. Project Summary/Abstract* AVGrant_R21_ProjectSummaryAbstract_Final.pdf

8. Project Narrative* AVGrant_R21_ProjectNarrative_Final.pdf

9. Bibliography & References Cited AVGrant_R21_Bibliography_Final.pdf

10.Facilities & Other Resources AVGrant_R21_Facilities_Final.pdf

11.Equipment AVGrant_R21_Equipment_Final.pdf

 Page 5 
Project Summary/Abstract 

Hearing impairment (HI) affects a large proportion of older adults: 25% of 65-74 year olds, and 50% of people 

75 and older have a HI that is sufficiently severe to be disabling [36,42]. The primary complaint of individuals 

with HI is difficulty understanding speech. Suboptimal speech communication by people with HI is a likely 

reason why they suffer from broader behavioral and physical health problems [35,53]. Understanding the 

factors that affect speech perception by people with HI, and finding methods to overcome the speech 

perception difficulties that these individuals face, is of great societal importance. Because of changing US 

demographics, communication between older individuals with HI and younger individuals—including those in 

caregiving and service-delivery roles—is increasingly likely to be between individuals of different races and 

ethnicities [44,54]. This presents a potential unique problem to individuals with HI. Research has shown that a 

talker's speech can become less intelligible to normal hearing (NH) individuals when they become aware of a 

talker's race or ethnicity by seeing a picture of the talker [6,37,48]. Given ongoing demographic changes, these 

findings have potentially profound implications for our understanding of speech perception by older adults 

with HI, as their communication with younger individuals is likely to be across lines of race and ethnicity. This 

may provide an additional challenge to speech perception beyond the challenge posed by the sensory loss itself. 

The proposed project seeks to remove two barriers to conducting large-scale studies of effects of race and 

ethnicity on speech perception by older adults with HI. One barrier is that there are no existing audiovisual 

corpora of speech stimuli that are produced by ethnically and racially diverse individuals. In specific aim 1, we 

will build a new corpus of audiovisual speech stimuli produced by individuals from diverse races and 

ethnicities. This corpus will include both standard-of-care sentences used in research on HI [26], and a new set 

of sentence materials designed specifically for this project. This corpus will be made available to the public at 

the conclusion of funding. The second barrier is that the previous studies have not determined the specific 

mechanism that explains why talker race and ethnicity affect speech intelligibility. In specific aim 2, we will use 

stimuli from specific aim 1 in a series of intelligibility experiments with younger listeners with NH, older 

listeners with NH, and older listeners with HI, using both behavioral and eye-tracking responses. We will also 

collect information on individuals' attitudes toward individuals of different races and ethnicities, as well as the 

ethnic and racial diversity in their peer groups. Results from this specific aim will help us better understand the 

mechanisms that underlie effects of race and ethnicity on speech intelligibility, and whether the speech 

perception of older adults with HI is disproportionately poorer in interactions with racially and ethnically 

diverse interlocutors. Given ongoing demographic changes and the high incidence of hearing impairment, 

these results will be of high public health relevance. 

Project Summary/Abstract 

 Page 6 
Project Narrative 

Recent research has shown that speech intelligibility can change when a listener becomes aware of a talker's 

race and ethnicity. This project will build a corpus of audiovisual speech stimuli produced by ethnically and 

racially diverse talkers, determine the locus of effects of talker race and ethnicity effects on speech 

intelligibility, and determine the extent to which talker race and ethnicity affect speech perception by older 

listeners with hearing impairment (HI). Given the large percentage of the population that has HI (50% of 

people over 75), and the large and growing percentage of the US population of the US that is Latinx (~17%) and 

non-white (~37%), these findings are of great public health relevance. 

Project Narrative 

 Page 7 
Physical Facilities: 

Dr. Munson has a 270 ft2 private office in 205A Shevlin Hall. This space has locked file cabinets and a 

computer that is used for data analysis and writing. Dr. Munson's laboratory is a 666 ft2 space in 5 Shevlin 

Hall. In addition, Dr. Munson has access to the Multisensory Perception (MSP) lab, 

which has facilities to collect perception data. 

Intellectual Environment: 

The Center for Translational and Applied Sensory Science (CATSS) is a collective of researchers interested in 

sensory perception and sensory loss. PI Munson and consultants Winn and Davies-Venn are members of 

CATSS; co-I Oxenham is co-scientific director, and co-I Nelson is executive director. CATSS houses the MSP 

lab, in which data collection will take place, along with space for working-group meetings. It also hosts 

periodic colloquia, and opportunities for members to present research in progress. Many CATSS researchers 

not named in this application work on issues that are peripherally related to this project, such as work on visual 

perception, virtual reality, and visual impairment. In short, CATSS provides an intellectual environment that 

will support this work as it progresses. Moreover, CATSS will provide a rich and supportive intellectual 

environment for the TBA postdoctoral fellow who will be employed on this project. CATSS provides a potential 

resource for additional staff support, as CATSS is home to an NSF-funded NRT training program in sensory 

science. Components of the project described in this application will be of interest to trainees in that program. 

The has numerous services for statistical consulting and collaboration, including both 

free and fee-for-service consultation. The University of Minnesota's School of Statistics is housed in the same 

college as PI Munson, and he has used these services previously to develop novel statistical analyses for 

complex research designs. 

Other Staff Support Mechanisms: A potential source of additional staff support is from the University of 

Minnesota's Undergraduate Research Opportunities Program (UROP). UROP grants provide undergraduate 

students with 15 weeks of funding to work on research projects. PI Munson has advised numerous UROP 

grants, and anticipates UROP being a source of additional staff support for this project. University of 

Minnesota also has two mechanisms to engage under-represented groups in research: the Multicultural 

Summer Research Program (MSROP), and the Trio/McNair scholars program. PI Munson has advised 

students in both of these programs previously, and has already reached out to the MSROP and Trio/McNair 

leadership to recruit students for Summer 2019 and Summer 2020 to work on the project described in this 

application. 

Liaisons to Diverse Twin Cities Communities: One important goal of this project is to recruit a diverse 

population of talkers in specific aim 1, and a diverse group of listeners in specific aim 2. The University of 

 Urban Outreach and Research Center (UROC) facilitates connections between university 

researchers and diverse communities in the Twin Cities metropolitan areas. Moreover, the University 

collaborates with a number of agencies and organizations who serve racially and ethnically diverse 

communities. These resources will help us to achieve our goal of recruiting a diverse group of participants. 

Facilities & Other Resources 

 Page 8 
Equipment 

All of the equipment needed for this grant is housed in PI Munson's laboratory, and in the University of 

Multisensory Perception (MSP) Laboratory, which is located in a building adjacent to the one that 

houses PI Munson's lab. 

Dr. Munson's laboratory is equipped with an 8' x 8' double-walled sound-treated room that is configured to 

collect perception data, and to record audio-visual stimuli. The sound-treated room has a touch-screen 

monitor, a boom microphone, professional-grade lighting, and the capacity to present sounds either in a 

soundfield (in either mono or stereo) or over headphones. Dr. Munson's lab also has a separate area for data 

analysis, a separate area for data collection via standardized tests (i.e., not requiring a sound-treated room), 

and numerous locked cabinets for storing hard copies of data. Dr. Munson's laboratory will be used to record 

audiovisual stimuli for specific aim 1, and for data analysis for both specific aims. Dr. Munson's lab is equipped 

with two GigE Prosilica 680 cameras with Kowa C Mount lenses to make high-resolution, high-frame rate 

video recordings. These are synched with professional-grade microphones and solid-state recorders for 

making high-quality audio recordings. 

Dr. Munson's laboratory is equipped with 4 desktop computers that are replaced on a four-year cycle. These 

have the software needed for this project (R, Praat, Avidemux). 

The MSP Lab has a 150 ft2 sound-treated room with a 48-speaker surround-sound array, and with the 

capability to project video images on three walls. The room is equipped with a Tobii T60 XL eye-tracker, which 

will be used to collect eye-tracking data. The MSP will be used to collect the speech-perception data for specific 

aim 2 of this project. 

The MSP has 4 computers, and is in the process of purchasing many others as their space develops. 

Dr. Munson has access to a private, secured server that houses data and materials for his research projects. 

The server can be accessed only by people that Dr. Munson grants access to, including individuals at other 

sites. This server will be used to house the data for this project. 

Equipment 

 Page 9 
OMB Number: 4040-0001 

Expiration Date: 10/31/2019 

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

RESEARCH & RELATED Senior/Key Person Profile (Expanded)

PROFILE - Project Director/Principal Investigator

Prefix:  First Name*: Benjamin Middle Name R Last Name*: Munson Suffix: 

Position/Title*: Professor

Organization Name*:

Credential, e.g., agency login: 

Project Role*:  PD/PI Other Project Role Category: 

Degree Type:  Degree Year: 

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Munson.pdf

Attach Current & Pending Support:   File Name:

 Page 10 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

PROFILE - Senior/Key Person

Prefix:  First Name*: Matthew Middle Name Brandon Last Name*: Winn Suffix: 

Position/Title*: Assistant Professor

Organization Name*:

Phone Number*:  Fax Number: 

E-Mail*: 

Credential, e.g., agency login

Project Role*:  Consultant Other Project Role Category: 

Degree Type:  AUD,PHD,BA Degree Year:  2010,2011,2005

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Winn.pdf

Attach Current & Pending Support:   File Name:

PROFILE - Senior/Key Person

Prefix:  First Name*: JEFFRY Middle Name Alan Last Name*: SIMPSON Suffix: 

Position/Title*: Professor

Organization Name*:

Phone Number*:  Fax Number: 

E-Mail*: 

Credential, e.g., agency login: 

Project Role*:  Consultant Other Project Role Category: 

Degree Type:  PHD,AB Degree Year:  1986,1981

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Simpson.pdf

Attach Current & Pending Support:   File Name:

Contact PD/PI: Munson, Benjamin R 

 Page 11 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

PROFILE - Senior/Key Person

Prefix:  First Name*: Andrew Middle Name J. Last Name*: Oxenham Suffix: 

Position/Title*: Prof.

Organization Name*:

Phone Number*: 

Credential, e.g., agency login

Project Role*:  Co-Investigator Other Project Role Category: 

Degree Type:  PHD Degree Year:  1995

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Oxenham.pdf

Attach Current & Pending Support:   File Name:

PROFILE - Senior/Key Person

Prefix:  First Name*: Peggy Middle Name B Last Name*: Nelson Suffix: 

Position/Title*: Professor

Organization Name*:

E-Mail*: 

Credential, e.g., agency login: 

Project Role*:  Co-Investigator Other Project Role Category: 

Degree Type:  PHD,MA,BA Degree Year:  1991,1982,1974

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Nelson.pdf

Attach Current & Pending Support:   File Name:

Contact PD/PI: Munson, Benjamin R 

 Page 12 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

PROFILE - Senior/Key Person

Prefix:  First Name*: Evelyn Middle Name EMO Last Name*: DaviesVenn Suffix: 

Position/Title*: Assistant Professor

Organization Name*:

Zip / Postal Code*:

Phone Number*: 

Credential, e.g., agency login

Project Role*:  Consultant Other Project Role Category: 

Degree Type:  AUD,PHD,MS,BA Degree Year:  2005,2010,2004,2002

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Davies-Venn.pdf

Attach Current & Pending Support:   File Name:

PROFILE - Senior/Key Person

Prefix:  First Name*: Molly Middle Name  Last Name*: Babel Suffix: 

Position/Title*: Associate Professor

Organization Name*: University of British Columbia

Department: Linguistics

Division:

Street1*: 2613 West Mall

Street2:

City*: Vancouver

County:

State*:

Province: British Columbia

Country*: CAN: CANADA

Zip / Postal Code*: V6T 1Z4

Phone Number*: 1 604 822 2976 Fax Number: 

E-Mail*: molly.babel@ubc.ca

Credential, e.g., agency login: mollybabel

Project Role*:  Consultant Other Project Role Category: 

Degree Type:  PhD Degree Year:  2009

Attach Biographical Sketch*:             File Name: AVGrant_R21_Biosketch_Babel.pdf

Attach Current & Pending Support:   File Name:

Contact PD/PI: Munson, Benjamin R 

 Page 13 
OMB No. 0925-0001 and 0925-0002 (Rev. 09/17 Approved Through 03/31/2020) 

BIOGRAPHICAL SKETCH 

Provide the following information for the Senior/key personnel and other significant contributors. 

Follow this format for each person. DO NOT EXCEED FIVE PAGES. 

NAME: Benjamin Ray Munson, Jr. 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Professor and Chair 

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

include postdoctoral training and residency training if applicable. Add/delete rows as necessary.) 

INSTITUTION AND LOCATION 

DEGREE 

(if 

applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

State University of New York, Buffalo, NY B.A. 1992 Linguistics 

University of California, Los Angeles, CA (Graduate 

Coursework) 

1992-1994 Linguistics 

The Ohio State University, Columbus, OH M.A. 1997 Speech-Language 

Pathology 

The Ohio State University, Columbus, OH Ph.D. 2000 Speech and Hearing 

Science 

A. Personal Statement 

In my 18+ years I have developed a diverse research program that examines the 

interplay between speech production and speech perception in individuals with communication disorders. 

Most relevant to this grant, my previous work has included adults with hearing impairment (Munson et al., 

2003; Collison, Munson, & Carney, 2004; Munson & Nelson, 2005) and a recent grant that included children 

with hearing impairment (NIH grant R01 DC002932). My research has focused on the interplay between 

'bottom-up' knowledge of sounds' articulatory and auditory characteristics, and 'top-down' knowledge of how 

sounds pattern in words, and how speech sound variation conveys socially meaningful information about 

talkers. The latter topic is central to this grant: how do listeners' knowledge and expectations of socially 

meaningful variation affect speech communication by individuals with hearing impairment? This topic is vital 

to our understanding of impaired communication. Communication across lines of race and ethnicity is fast 

becoming the norm in the US, and there is growing evidence that these social factors provide an additional 

challenge to communication that has not been considered in previous research. This grant will develop a 

valuable resource to study this topic: a corpus of audio-visual speech produced by racially and ethnically 

diverse talkers. That resource builds on my expertise in corpus development (Beckman, Plummer, Munson, & 

Reidy, 2017). It will be of great use to my future research on this topic, and to the community of individuals 

who study hearing loss, amplification, and speech perception. Moreover, this grant will use this resource to 

study speech intelligibility in individuals with and without hearing impairment, to better understand the scope 

and nature of effects of talker ethnicity and race on speech perception. 

B. Positions and Honors 

Positions 

Department of Speech-Language-Hearing Sciences (renamed from Department of Communication Disorders, 

7/2004), University of Minnesota, Minneapolis, MN 

Assistant Professor, 8/28/2000-8/27/2006. 

Associate Professor with Tenure, 8/28/2006-8/26/2012 

Full Professor with Tenure, 8/27/2012-6/10/2018 

Full Professor with Tenure and Chair, 6/11/2018-current 

Biosketches 

 Page 14 
Honors 

1. Young Investigator Travel Award, Acoustical Society of America Committee on Women in Acoustics, 2002 

2. Students' Choice Outstanding Faculty Member of the Year, University of Minnesota College of Liberal Arts 

Student Board, 2002-2003. 

3. Editor's Award for Outstanding Article in Journal of Speech Language, and Hearing Research (Hearing 

section). Co-awarded to Elizabeth Collison, Benjamin Munson, and Arlene E. Carney, 2004. 

4. Award for Early Career Contributions in Research, American Speech-Language-Hearing Association, 2005. 

5. Arthur "Red" Motley Outstanding Teaching Award, University of Minnesota College of Liberal Arts, 2006 

6. McKnight Foundation Presidential Fellowship, University of Minnesota Graduate School, 2006-2009. 

7. John Erskine Fellowship, University of Canterbury, Christchurch, New Zealand, 2008 

8. Fellow, Acoustical Society of America, 2011 

9. Visiting Faculty Fellowship, University of Newcastle upon Tyne, 2011 

10. Fellow, American Speech-Language-Hearing Association, 2013 

11. Visiting Scholar, Center for Advanced Studies, Ludwig-Maximillians University, Munich, 2014 

12. Visiting Scholar, Laboratoire Parole et Langage, Aix-Marseille University, September, 2013, November, 

2015 

13. Visiting Scholar, University of Hong Kong, June, 2016 

Other Experience and Professional Memberships 

Member of: American Speech-Language-Hearing Association, Acoustical Society of America, Linguistic 

Society of America 

2005-2007 Associate Editor, Journal of Speech, Language, and Hearing Research 

2010-2013 Chair, Speech Communication technical committee, Acoustical Society of America 

2011-2014 Associate Editor, Journal of the Acoustical Society of America 

2014-2017 Member, Language and Communication (LCOM) review panel, Center for Scientific Review, 

National Institutes of Health 

2011-current Chair, University of Minnesota Institutional Review Board (IRB), Human Subjects’ Protection 

Program, Panel J (Faculty Social and Behavioral Science Research). 

C. Contributions to Science 

1. Social and other Top-Down Influences on Speech Perception 

Speech perception involves both bottom-up and top-down process. My first scientific contribution has been to 

examine top-down influences on speech perception. The focus of much of this work has been on how social 

information affects speech perception, which is the topic of the current proposal. 

Evans, K., Munson, B., & Edwards, J. (2018). Does speaker race affect the assessment of children's speech 

accuracy? A comparison of speech-language pathologists and clinically untrained listeners. Language, 

Speech, and Hearing Services in Schools. E-Publication ahead of print, doi:10.1044/2018_LSHSS-17-

0120. PMID: 29971346 

Munson, B., & Nelson, P.B. (2005). Phonetic identification in quiet and in noise by listeners with cochlear 

implants. Journal of the Acoustical Society of America, 118, 2607-2617. PMID: 16266181 

Munson, B., Jefferson, S.V., & McDonald, E.C. (2006). The influence of perceived sexual orientation on 

fricative identification. Journal of the Acoustical Society of America, 119, 2427-2437. PMID: 16642855 

Munson, B. (2011). The Influence of Actual and Imputed Talker Gender on Fricative Perception, Revisited. 

Journal of the Acoustical Society of America, 130, 2631-2634. PMID: 22087888 

2. Sexuality and Phonetic Variation across the Lifespan 

A key fact about speech is that it varies along multiple dimensions. My second scientific contribution has been 

to examine how listeners convey one socially meaningful variable, sexuality, through phonetic variation. 

Sexuality is a particularly interesting social variable to examine because one’s sexuality is typically not 

established until late in adolescence. Hence, the acquisition of this speech style a potentially illuminating case 

study of late-life language acquisition. My studies have shown that there is a distinctive speech style used by a 

subset of self-identified gay men and lesbian women, and that this style is salient enough to allow naïve 

listeners to identify a person’s sexuality from content-neutral speech. Rudiments of this speech style can be 

seen in children with non-canonical patterns of gender identity as young as five years of age. 

Mack, S., & Munson, B. (2012). The influence of /s/ quality on ratings of men's sexual orientation: Explicit and 

implicit measures of the 'gay lisp' stereotype. Journal of Phonetics, 40, 198–212. 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 15 
Munson, B. (2007). The acoustic correlates of perceived sexual orientation, perceived masculinity, and 

perceived femininity. Language and Speech, 50, 125-142. PMID: 17518106 

Munson, B., Crocker, L., Pierrehumbert, J., Owen-Anderson, A., & Zucker, K. (2015). Gender typicality in 

children's speech: A comparison of the speech of boys with and without gender identity disorder. 

Journal of the Acoustical society of America, 137. PMID: 25920850 

Munson, B., McDonald, E.C., DeBoe, N.L., & White, A.R. (2006). Acoustic and perceptual bases of judgments 

of women and men's sexual orientation from read speech. Journal of Phonetics, 34, 202-240. 

3. Lexical Influences on Phonological Development and Disorders 

Traditional models of phonological development posited that knowledge of the sound structure influences 

vocabulary development only insomuch as early perception and production impairments might impede wordlearning. My work has shown that expansion in the lexicon has a top-down influence on phonological 

development, such that children with larger vocabularies are better able to make implicit generalizations about 

the sound structure of language. These relationships hold across children with a variety of vocabulary sizes due 

to variation in age and in overall language ability. 

Munson, B. (2001). Phonological pattern frequency and speech production in children and adults. Journal of 

Speech, Language, and Hearing Research, 44, 778-792. PMID: 11521771 

Edwards, J., Beckman, M.E., & Munson, B. (2004). The interaction between vocabulary size and phonotactic 

probability effects on children’s production accuracy and fluency in nonword repetition. Journal of 

Speech, Language, and Hearing Research, 47, 421-436. PMID: 15157141 

Munson, B., Edwards, J., & Beckman, M.E. (2005). Relationships between nonword repetition accuracy and 

other measures of linguistic development in children with phonological disorders. Journal of Speech, 

Language, and Hearing Research, 48, 61-78. PMID: 15938060 

Munson, B., Kurtz, B.A., & Windsor, J. (2005). The influence of vocabulary size, phonotactic probability, and 

wordlikeness on nonword repetitions of children with and without specific language impairment. 

Journal of Speech, Language, and Hearing Research, 48, 1033-1047. PMID: 16411794 

4. Lexical Influences on Phonetic Variation 

Traditional models of phonological knowledge posited a strict modular demarcation between the size and 

composition of the mental lexicon and knowledge of the sound structure of language. My work in this area has 

shown that lexical characteristics of words (specifically, their frequency of use and their similarity to other 

known real words) affect their phonetic structure. Words that are frequently used and which are dissimilar to 

other real words are produced with a less distinct articulation than words with the opposite characteristics. 

This line of research has important implications for models of speech production. 

Munson, B., & Solomon, N.P. (2004). The effect of phonological neighborhood density on vowel articulation. 

Journal of Speech, Language, and Hearing Research, 47, 1048-1058. PMID: 15605431 

Munson, B. (2007). Lexical access, lexical representation, and vowel articulation. In J. Cole & J. Hualde 

(Eds.), Laboratory Phonology 9 (p. 201-228). New York: Mouton de Gruyter. 

Watson, P., & Munson, B. (2008). Parkinson's disease and the effect of lexical factors on vowel articulation. 

Journal of the Acoustical Society of America, 5, EL291-EL295. PMID: 19045680 

Yoneyama, K., & Munson, B. (2017). The influence of lexical characteristics and talker accent on the 

recognition of English words by speakers of Japanese. Journal of the Acoustical Society of America, 

141, 1308-1320. PMID: 28253637 

5. Improving Methods for Assessing Phonetic Development 

The most commonly used tool to study speech sound development is phonetic transcription. Transcription is 

also used in almost every assessment tool that has been developed for assessing speech sound disorders. 

Phonetic transcription has the weakness that it is poorly equipped to assess variation within phonemic 

categories. Research using articulatory and acoustic measures has shown that such variation is pervasive. My 

research has developed a variety of novel perceptual measures for assessing continuous variation. This work 

has shown that such measures are sensitive to cross-linguistic differences in acquisition. They also reveal 

effects of clinical experience in speech-language pathology on ratings. 

Munson, B., Edwards, J., Schellinger, S.K., Beckman, M.E., & Meyer, M.K. (2010). Deconstructing phonetic 

transcription: covert contrast, perceptual bias, and an extraterrestrial view of Vox Humana. Clinical 

Linguistics and Phonetics, 24, 245-260. PMID: 20345255 

Li, F., Munson, B., Edwards, J., Yoneyama, K., & Hall, K.C. (2011). Language specificity in the perception of 

voiceless sibilant fricatives in Japanese and English: Implications for cross-language differences in 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 16 
speech-sound development. Journal of the Acoustical Society of America, 129, 999-1011. PMID: 

21361456 

Munson, B., Johnson, J., & Edwards, J. (2012). The role of experience in the perception of phonetic detail in 

children's Speech: a comparison of speech-language pathologists with clinically untrained listeners. 

American Journal of Speech-Language Pathology, 24, 124-139. PMID: 22230182 

Schellinger, S.K., Munson, B., & Edwards, J. (2017). Gradient Perception of Children’s Productions of /s/ and 

/θ/: A Comparative Study of Rating Methods. Clinical Linguistics and Phonetics, 31, 80-103. PMID: 

27552446 

A full set of publications can be found at https://scholar.google.com/citations?user=WG_AOVIAAAAJ&hl=en

D. Research Support

Completed 

NIH Grant R01 DC002932 Munson (MPI), 4/1/2011-3/31/2018 

(multi-PI grant. J. Edwards, M. 

Beckman, B. Munson, MPIs, David 

Kaplan, co-I) 

Longitudinal Study of Lexical and Phonological Development 

This study examines the relationships among components of phonological knowledge (including the ability to 

perceive speech sounds in challenging listening tasks, the ability to robustly differentiate among speech sounds 

in production, and the ability to make inferences about how sounds function in the language being acquired) 

and lexical acquisition in children with a wide range of vocabulary sizes, where size varies for different reasons 

across groups. Specifically, this is a longitudinal study from 30 to 60 months of age of approximately 200 

children with a wide variety of initial vocabulary sizes resulting from a range of advantages or deficits in the 

types of phonological knowledge that support word learning. The specific research question addressed is: what 

are the developmental relationships among vocabulary growth and three types of phonological knowledge 

(speech production knowledge, speech perception knowledge, and higher-level categorical knowledge) for 

children with normal hearing from middle-SES and low-SES families, and for children with cochlear implants 

(CIs)? The study also includes a statistical modeling component to determine optimal interventions for 

children with small-sized vocabularies. Structural equation modeling, based on data from this rich longitudinal 

sample, will allow us to test what interventions might be most effective to increase vocabulary for these 

different groups of children. Ultimately, the knowledge obtained from this study will enable us to develop 

efficacious and targeted early interventions for young children with atypically small vocabularies 

NSF Grant BCS0729277 Munson (PI) 1/1/08-12/31/2010 

NSF DHB/Collaborative Research: Using machine learning to model the interplay of production dynamics 

and perception dynamics in phonological acquisition 

The goal of this collaborative research grant (including components at the lead institution, Ohio State 

University [Mary E. Beckman, PI, Eric Fosler-Lussier co-I] and University of Wisconsin-Madison [Jan 

Edwards] is to apply acoustic models that are currently being developed for robust Automatic Speech 

Recognition (ASR) will be applied to a large multi-language database to explore how cognitive representations 

relevant to speech production and perception in any given speech community come to be internalized by 

normally developing children. 

NSF Grant BCS074644 Munson (PI) 1/1/08-12/31/2009

Supporting Young Investigator Travel to the Eleventh Conference on Laboratory Phonology 

Through this grant, the NSF provided funding for ten early-career scholars, including students, to attend the 

Eleventh Conference on Laboratory Phonology. Travel grant applications were evaluated by a committee 

comprising early-career investigators from U.S. and New Zealand universities. They were disbursed by the 

University of Minnesota. 

NIH Grant R03 DC005702 Munson (PI) 6/1/03-5/31/2007 

Speech Production in Phonological Impairment 

The overall goal of this project is to assess the extent to which lexical representation and processing (referred to 

from here forward as lexical skills), phonological representation and processing (phonological skills), and the 

linking between them (linking skills) are compromised in children with phonological impairments (PhI) aged 4 

Contact PD/PI: Munson, Benjamin R 

Biosketches

 Page 17 
to 7. Three experiments are proposed to determine the locus of phonological impairment. In addition, 

measures of lexical, phonological, and linking skill will be related to phonological patterns in spontaneous 

speech and single-word naming, to assess the extent to which these skills predict error patterns of children with 

PhI. 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 18 
OMB No. 0925-0001 and 0925-0002 (Rev. 10/15 Approved Through 10/31/2018) 

BIOGRAPHICAL SKETCH 

NAME: Matthew Brandon Winn 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Assistant Professor of Speech-Language-Hearing Sciences 

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

include postdoctoral training and residency training if applicable. Add/delete rows as necessary.) 

INSTITUTION AND LOCATION 

DEGREE 

(if 

applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

University of Delaware B.A. 8/2005 Psychology, Philosophy 

University of Maryland College Park Au.D. 5/2010 Clinical Audiology 

Veterans Affairs (fourth-year audiology residency) 7/2010 Clinical Audiology 

University of Maryland College Park Ph.D. 11/2011 Hearing and Speech 

Sciences 

University of Wisconsin-Madison 8/2015 Communication 

Sciences and Disorders 

A. Personal Statement 

I am an audiologist and hearing scientist with training in speech science and linguistics. My research 

focuses on speech perception and how it is affected by hearing loss. In my lab we use measures of eye 

tracking and pupillometry to measure perception of speech and the effort involved in listening. 

I am interested in measuring various effects of hearing loss that are not well described by conventional 

testing, including listening effort, adjustment to different talker styles, reliance on non-auditory cues, 

spectral resolution, perceptual gradiency and binaural hearing. Additionally, I have developed opensource solutions for speech stimulus generation for experiments involving phonetic categorization and 

cue weighting, as well as analysis tools for pupillometry. My goal is to make a positive contribution to 

auditory science, the cognitive sciences, and the care of audiology patients. I aim to embody the 

mission of the NIH in terms of mentoring students in the area of bridging clinical practice and research, 

as well as incorporating a multidisciplinary approach to speech, language and hearing sciences. 

B. Positions and Honors 

Positions: 

7/2009 – 2/2012 Clinical audiologist, Washington D.C. Veterans Affairs Medical Center 

4/2012 – 8/7/2015 Postdoctoral researcher, Waisman Center, Univ. of Wisconsin-Madison 

6/2013 – 8/7/2015 Research associate, Department of Surgery, Univ. of Wisconsin-Madison 

9/2015 – 8/2018 Assistant Professor, Speech & Hearing Sciences, Univ. of Washington 

8/2015 – present Assistant Professor, Speech-Language-Hearing Sciences, Univ. of Minnesota 

Professional memberships: 

Biosketches 

 Page 19 
Acoustical Society of America 

Association for Research in Otolaryngology 

American Auditory Society 

Honors: 

2010, 2011: Pre-Doctoral Training Grant, Center for Comparative Evolution and Biology of Hearing 

2010; Mentored Doctoral Student Research Poster grant, American Auditory Society 

2010, 2011: Distinguished Teaching Award, Univ. of Maryland Center for Teaching Excellence 

2011: Dean’s Scholar Mentorship Award, Univ. of Maryland College of Behavioral Social Sciences 

2011: Student travel award, Conference on Implantable Auditory Prostheses 

2013: Student travel award, Conference on Implantable Auditory Prostheses 

2014: Young Investigator award, Conference on Objective Measures in Auditory Implants 

C. Contribution to Science 

1). Measuring listening effort using pupillometry 

The pupillary response is a sensitive metric of listening effort that is driven by changes in auditory 

spectral resolution and linguistic processing. Specifically, the pupil shows greater dilation as effort 

increases, and this measurement can capture dynamics of effort as they happen over time. 

I am part of a small group of investigators worldwide who use pupillometry to assess listening 

effort. In a recent international workshop on pupillometry in Amsterdam, I joined with a small number of 

others to initiate a joint effort to collect our shared experiences and contribute a “best-practices” paper 

to raise standards and avoid duplicate work of pupillometry research in the field of hearing science. 

The influence of this work can be described in two categories. First, it emphasizes the importance 

of listening effort as complementary to traditional intelligibility scores. In light of considerable literature 

demonstrating the public health and economic impact of elevated listening effort in people with hearing 

impairment, the scope of this work extends beyond basic auditory processing. Second, the nature of 

pupillometry as a time-varying measurement means that sentence processing can be measured as it 

unfolds over time, which is a novel contribution in the field of clinical hearing science (where speech 

perception measurements are almost uniformly recorded after perception has taken place. By 

capitalizing on this aspect of the measurement, I have shown that the timing of sentence 

comprehension can be delayed in people with cochlear implants, despite excellent intelligibility scores. 

The importance of this work is that delayed or disrupted language processing might not show up 

in intelligibility scores, but still could play a major role in the functional performance of a listener who 

has hearing impairment. In all of these studies, my role has been the primary investigator. 

a. Winn, M.B., Edwards, J.R., and Litovsky, R.Y. (2015). The impact of auditory spectral 

resolution on listening effort revealed by pupil dilation. Ear and Hearing, 36(4):e153-65. 

b. Winn, M.B. (2016) Rapid release from listening effort resulting from semantic context, 

and effects of spectral degradation and cochlear implants. Trends in Hearing, 20, 1-17. 

c. Winn, M.B., Wendt, D., Koelewijn, T., Kuchinsky, S. Best practices in using 

pupillometry to measure listening effort: an introduction for those who want to get 

started. In Press, Trends in Hearing. 

2. Phonetic cue weighting and how it is affected by hearing loss 

My earliest and ongoing publications address the manner in which people perceive speech sounds 

under various conditions of impaired or simulated impaired hearing. The central finding of this branch of 

work is that people with hearing impairment can rely on acoustic cues to identify speech sounds in ways 

that differ from patterns observed in people who have normal hearing. Specifically, people who use 

cochlear implants show greater reliance upon temporal aspects of speech sounds and relatively less 

reliance on spectral components, consistent with the nature of implant speech signal processing. 

The importance of this work is that it demonstrates the fact that patterns of speech cue perception 

and weighting as measured in people with normal hearing do not necessarily carry over to populations of 

people with hearing impairment. Because of the nature of speech as a complex and multi-dimensional 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 20 
signal, there are many opportunities to mistake performance in one domain (i.e. temporal) as 

performance in another domain (i.e. spectral). 

The philosophy behind this work has spawned a new line of research (by myself and other labs) 

that uses phonetic cue weighting as a diagnostic outcome measure or a predictor of other auditory 

abilities. In all of these studies, my role has been the primary investigator. 

a. Winn, M.B., Chatterjee, M., & Idsardi, W.J. (2013). The roles of voice onset time and F0 

in stop consonant voicing perception: Effects of masking noise and low-pass filtering. 

Journal of Speech, Language and Hearing Research, 56, 1097-1107. 

b. Winn, M.B., Rhone, A.E., Chatterjee, M., & Idsardi, W.J. (2013). Auditory and visual 

context effects in phonetic perception by normal-hearing listeners and listeners with 

cochlear implants. Frontiers in Psychology: Auditory Cognitive Neuroscience, 4, article 

824, 1-13. 

c. Winn, M.B., Won, J.H., Moon, I.J. (2016). Assessment of spectral and temporal 

resolution in cochlear implant users using psychoacoustic discrimination and speech cue 

categorization. Ear and Hearing, 37(6), e377– e390. 

d. Kapnoula, E., Winn, M.B., Kong, E.J., Edwards, J., McMurray, B. (2017) Evaluating the 

sources and functions of gradiency in phoneme categorization: An individual differences 

approach. Journal of Experimental Psychology: Human Perception and Performance, 43, 

1594-1611. doi: 10.1037/xhp0000410 

3). Measuring spectral resolution in cochlear implant patients using speech sounds 

Stemming from the aforementioned work on phonetic cue weighting, this branch of research 

focuses on speech cues that are specifically spectral in nature (i.e. are cued by frequency differences 

rather than timing or overall amplitude differences). This decision was made to specifically tailor 

experiments to the needs and abilities of people who use cochlear implants, which are devices that 

restore hearing, albeit with particular challenges in spectral resolution. By developing a test of spectral 

phonetic cue perception, this line of work has produced a simple yet powerful diagnostic test of 

functional performance that bears relationship to speech perception without relying on abstract nonlinguistic psychophysical stimuli, and while still avoiding the historical confounds of using multidimensional speech stimuli that confound the measurement of a single auditory dimension. 

The importance of this work is that the paradigm used to test functional spectral resolution can be 

used to evaluate new sound processing strategies in cochlear implants and hearing aids. Importantly, 

the test is sensitive specifically to the domain that is the target of improvement in cochlear implant 

technology. 

a. Winn, M.B. & Litovsky, R.Y. (2015) Using speech sounds to test functional spectral 

resolution in listeners with cochlear implants. Journal of the Acoustical Society of 

America, 137, 1430-1442. 

b. Kong, Y.-Y., Winn, M.B., Poellmann, K., Donaldson, G. (2016). Discriminability and 

perceptual saliency of temporal and spectral cues for final fricative consonant voicing in 

simulated cochlear-implant and bimodal hearing. Trends in Hearing, 20, 1-15. 

c. DiNino, M., Wright, R., Winn, M.B., Bierer, J.A. (2016). Vowel and consonant confusion 

patterns resulting from spectral manipulations in vocoded stimuli designed to replicate 

poor electrode-neuron interfaces in cochlear implants. Journal of the Acoustical Society 

of America, 140, 4404–4418. 

4). Development of freely available software and other resources 

Most of the aforementioned research contributions depend on the generation of controlled 

acoustic stimuli that sound like natural speech and reflect the complexity and cues important to natural 

speech, and are yet controlled with precision sufficient for rigorous auditory experimentation. My 

contribution has been to develop software that enables an experimenter to generate such speech 

sounds or simulating cochlear implants using a freely available platform (Praat). This software includes 

numerous options to customize sound output in ways that are valuable to auditory scientists but often 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 21 
extremely tedious to control without extensive experience in speech acoustics and programming. By 

wrapping these experiences into a user-friendly program, I have produced a tool that has been used at 

several universities to test perception of speech sounds in a finely controlled fashion. 

The software is available at: 

http://www.mattwinn.com/praat.html 

and the specific speech manipulation tools are at: 

http://www.mattwinn.com/praat.html#formantContinuum 

http://www.mattwinn.com/praat.html#votContinuum 

http://www.mattwinn.com/praat.html#fricativeContinuum 

http://www.mattwinn.com/praat.html#durationContinuum

a. Winn, M.B. & Litovsky, R.Y. (2015) Using speech sounds to test functional spectral

resolution in listeners with cochlear implants. Journal of the Acoustical Society of

America, 137, 1430-1442.

b. Stilp, C.E., Anderson, P.W., Winn, M.B. (2015) Predicting contrast effects following

reliable spectral properties in speech perception. Journal of the Acoustical Society of

America, 137, 3466-3476.

Complete List of Published Work: http://www.ncbi.nlm.nih.gov/pubmed/?term=matthew+B+winn

D. Research Support

Ongoing Research Support 

R01 DC 017114 8/2018 – 7/2023 

Listening effort in cochlear implant users 

The goal of this project is to explore factors that make listening effortful for people with hearing 

impairment, with special focus on the need to repair perceptual mistakes by relying on context. 

Role: PI 

R03 DC 014309 01/11/2016 – 12/31/2018 

Measuring listening effort and spectral resolution in cochlear implant patients 

The goal of this study is to evaluate the impact of spectral resolution (i.e. sound quality) with regard to its 

effects on listening effort and also the perception of fine-grained detail in speech sounds. 

Role: PI 

Completed Research Support 

5R01DC002932-12 Jan R. Edwards (PI) 2012-2013 

Goals: Longitudinal study of vocabulary growth and phonological development 

My role: Postdoctoral researcher 

5R01DC003083-15 Ruth Y. Litovsky (PI) 2012-2015 

Goals: Exploring binaural sensitivity and spatial hearing in people with bilateral cochlear implants 

My role: Postdoctoral researcher 

Biosketches

 Page 22 
OMB No. 0925-0001 and 0925-0002 (Rev. 09/17 Approved Through 03/31/2020) 

BIOGRAPHICAL SKETCH 

Provide the following information for the Senior/key personnel and other significant contributors. 

Follow this format for each person. DO NOT EXCEED FIVE PAGES. 

NAME: Jeffry A. Simpson 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Professor and Chair 

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, include 

postdoctoral training and residency training if applicable. Add/delete rows as necessary.) 

INSTITUTION AND LOCATION 

DEGREE 

(if 

applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

University of Illinois, Champaign-Urbana, IL B.A. 05/1981 Political Science & 

Psychology 

University of Minnesota, Minneapolis, MN Ph.D. 07/1986 Psychology 

A. Personal Statement 

In recent years, our lab has been assessing how implicit (unconscious) attitudes affect different kinds of social 

behavior. Thus, as a consult on this R21 application, I bring skills and knowledge sets that will facilitate this 

team effort. First, I have conducted several prior research projects that have measured implicit attitudes to 

determine how they predict various outcomes. As a result, I have a great deal of experience with research 

different implicit assessment methods and techniques, including both data reduction and data analytic 

techniques needed to contribute to this part of the proposed research. Second, I am the PI on a current NSF 

grant focusing on how implicit relative to explicit attitudes toward children are related to different measures 

of parental investment. Many of the central aims this R21, therefore, are well aligned with my expertise on 

measuring implicit attitudes, and I know how to test between competing models. This will help the current 

research team tie the findings more clearly and tightly with other related work in this area. 

B. Positions and Honors 

Positions 

1986–1987 Visiting Assistant Professor, Department of Psychology, Texas A&M University, College 

Station, TX 

1987–1992 Assistant Professor, Department of Psychology, Texas A&M University, College Station, TX 

1992–1997 Associate Professor, Department of Psychology, Texas A&M University, College Station, TX 

1997–2004 Professor, Department of Psychology, Texas A&M University, College Station, TX 2004– 

present Professor, Department of Psychology, University of Minnesota, Minneapolis, MN 

2006-present Director, Doctoral Minor in Interpersonal Relationships (IREL), University of Minnesota 

2008-present Adjunct Professor, Institute of Child Development, University of Minnesota 

2012-present Adjunct Professor, Department of Communication, University of Minnesota 

2018-present Chair, Department of Psychology, University of Minnesota 

Other Experience and Professional Memberships 

1996-1998 Social Psychology Panel (member), National Science Foundation 

1998-2001 Editor, Personal Relationships, 1998-2001 

2002-2004 Social Psychology, Personality, and Interpersonal Processes Panel (ad hoc), National 

Institutes of Health 

2002-2008 Associate Editor, Journal of Personality and Social Psychology: Interpersonal Relations 

and Group Processes 

Biosketches 

 Page 23 
2004-2006 Social Psychology, Personality, and Interpersonal Processes Panel (member), National 

Institutes of Health 

2006-2008 Social Psychology, Personality, and Interpersonal Processes Panel (chair), National 

Institutes of Health 

2008-2014 Editor, Journal of Personality and Social Psychology: Interpersonal Relations and 

Group Processes 

2014-2018 Special Emphasis Panel (member), National Institute of Aging 

Fellow American Psychological Association 

Fellow Association for Psychological Science 

Fellow Society for Personality and Social Psychology 

Fellow Society of Experimental Social Psychology 

President International Association for Relationship Research (2014-2016) 

Honors 

1981 Phi Beta Kappa, University of Illinois 

1984-1985 Eva O. Miller Fellowship, University of Minnesota 

1985-1986 Doctoral Dissertation Fellowship, University of Minnesota 

1990-1991 University Honors Program Teacher/Scholar Award, Texas A&M University 

1992 Young Investigator Award, Society of Experimental Social Psychology, Relationships 

Group 1992 Sigma Xi, Texas A&M University 

1995 Erskine Fellowship, University of Canterbury, New Zealand 

1995 Distinguished Teaching Award, Association of Former Students, Texas A&M 

 University 

1998 New Contribution Award (best scientific article published in 1995-1996), International 

Society for the Study of Personal Relationships 

1999 Distinguished Research Award, College of Liberal Arts, Texas A&M University 

2001-2004 University Faculty Fellow, Texas A&M University 

2003 University Distinguished Research Award, Texas A&M University 

2004 Berscheid-Hatfield Award for Distinguished Mid-Career Achievement, International Association 

for Relationship Research 

2009 Deiner Award for Distinguished Mid-Career Achievement, Society of Personality and 

Social Psychology 

2011-2014 Scholar of the College, College of Liberal Arts, University of Minnesota 

2012 Seelye Charitable Trust Fellowship, University of Auckland, New Zealand 

2015Honorary Fellow, The Royal Society of New Zealand 

2017 Distinguished University Teaching Professor Award, University of Minnesota 

2018 Distinguished Mentoring Award, International Association for Relationship Research 

 C. Contributions to Science 

The Impact of Early Life Stress/Threat on Physical, Mental Health, and Personal Outcomes. 

Recent theory and research suggests that exposure to stressful (threatening) events early in life may lead to 

biological and mental “calibrations” that shunt individuals down different developmental trajectories over the 

life course. For example, individuals who are exposed to unpredictable forms of stress early in life learn that 

they must take risks and be opportunistic in order to obtain the outcomes they want or need, given the 

unstable, less predictable environments they are likely to inhabit as adults. Conversely, individuals exposed to 

less stressful, more stable early environments learn that making long-term investments in themselves, in 

others, and in high- quality, long-term relationships typically result in obtaining the best outcomes. During the 

past decade, my colleagues and I have tested various models grounded in some of these key life history ideas by 

studying how different types and amounts of stress experienced at different points of development 

prospectively predict an array of important life outcomes, ranging from physical and mental health, to risky 

sexual behavior, and to risky financial decision-making. Five recent sample publications are listed below: 

Biosketches 

 Page 24 
• Simpson, J. A., Griskevicius, V., Kuo, S. I., Sung, S., & Collins, W. A. (2012). Evolution,

stress, and sensitive periods: The influence of unpredictability in early versus late childhood on

sex and risky behavior. Developmental Psychology, 48, 674-686. doi: 10.1037/a0027293

• Puig, J., Englund, M. M., Simpson, J. A., & Collins, W. A. (2013). Predicting adult physical illness

from infant attachment: A prospective longitudinal study. Health Psychology, 32, 409-417. doi:

10.1037/a0028889

• Griskevicius, V., Ackerman, J. M., Cantú, S. M., Delton, A. W., Robertson, T. E., Simpson, J. A.,

Thompson, M. E., & Tybur, J. M. (2013). When the economy falters do people spend or save?

Responses to resource scarcity depend on childhood environments. Psychological Science, 24, 197-

205. doi: 10.1177/0956797612451471

• Farrell, A. K., Simpson, J. A., Englund, M. M., Carlson, E. A., & Sung, S. (2017). The impact of stress

at different life stages on physical health and the buffering effects of maternal sensitivity. Health

Psychology, 36, 35-44. doi: 10.1037/hea0000424

• Johnson, W., Huelsnitz, C., Carlson, E. A., Roisman, G. I., Englund, M. M., Miller, G. E., & Simpson, J. 

A. (2017). Childhood abuse and neglect and physical health at midlife: Prospective, longitudinal

evidence. Development and Psychopathology, 29, 1935-1946. doi: 10.1017/S095457941700150X

The Developmental Antecedents of Adult Romantic Relationship Outcomes. For years, many 

psychologists assumed that the key predictors of the quality and satisfaction of adult romantic relationships 

existed within the interpersonal context of the relationship (e.g., each partner’s personality, how well each 

partner behaved toward the other, how much each partner cared about the other). Even though these 

contemporaneous variables are important predictors of many basic romantic relationship outcomes, they do 

not tell the entire story. Working with my colleagues on the MLSRA project, we have identified how early-life 

attachment status, the quality of early care received from mothers, and other forms of stress and support 

provision systematically predict how longitudinally studied adults think, feel, and behave when they are 

observed having conflict discussions with their romantic partners and when they are observed trying to recover 

emotionally from these discussions. Five sample publications from this line of research are listed below: 

• Simpson, J. A., Collins, W. A., Tran, S., & Haydon, K. C. (2007). Attachment and the experience

and expression of emotions in adult romantic relationships: A developmental perspective. Journal of

Personality and Social Psychology, 92, 355-367. doi: 10.1037/0022-3514.92.2.355

• Salvatore, J. E., Kuo, S. I., Steele, R. D., Simpson, J. A., & Collins, W. A. (2011). Recovering from

conflict in romantic relationships: A developmental perspective. Psychological Science, 22, 376-383. 

doi: 10.1177/0956797610397055

• Oriña, M. M., Collins, W. A., Simpson, J. A., Salvatore, J. E., Haydon, K. C., & Kim, J. S. (2011).

Developmental and dyadic perspectives on commitment in adult romantic relationships.

Psychological Science, 22, 908-915. doi: 10.1177/0956797611410573

• Raby, K. L., Roisman, G. I., Simpson, J. A., Collins, W. A., & Steele, R. D. (2015). Maternal

insensitivity in childhood predicts greater electrodermal reactivity during conflict discussions with

adult romantic partners. Psychological Science, 26, 348-353. doi: 10.1177/0956797614563340

• Labella, M. H., Johnson, W. F., Martin, J., Ruiz, S. K., Shankman, J. L., Egeland, B., Englund, M. M.,

Collins, W. A., & Roisman, G. I., & Simpson, J. A. (2018). Multiple dimensions of childhood abuse

and neglect prospectively predict poorer romantic functioning in adulthood. Personality and Social

Psychology Bulletin, 44, 238-251. doi: 10.1177/0146167217736049

Attachment Orientations and the Quality/Functioning of Romantic Relationships. For the past 

30 years, I (along with several collaborators, especially Steven Rholes) have investigated how attachment 

working models (which are assessed with attachment measures) affect the quality and functioning of romantic 

relationships, both immediately (while partners are engaged in stressful lab tasks) and across time. Most of this 

research, which is reviewed and synthesized in Simpson and Rholes (2012), has examined how acute stressors 

such as fear, the stress of trying to resolve major relationship conflicts, or the challenge of supporting a highly 

distressed partner influence the way in which secure, anxious, and avoidant people think, feel, and especially 

behave in these challenging situations. We have also studied how the partners of insecurely attached people 

can—and often do—buffer their partners from displaying the relationship-damaging thoughts, feelings, and 

behaviors that insecurely attached people often enact when in these stressful situations. Five recent sample 

publications are listed below: 

Contact PD/PI: Munson, Benjamin R 

Biosketches

 Page 25 
• Simpson, J. A., & Rholes, W. S. (2012). Adult attachment orientations, stress, and romantic

relationships. In P. G. Devine, A. Plant, J. Olson, & M. Zanna (Eds.), Advances in Experimental Social

Psychology, 45, 279-328. doi: 10.1016/B978-0-12-394286-9.00006-8

• Haydon, K. C., Collins, W. A., Salvatore, J. E., Simpson, J. A., & Roisman, G. I. (2012). Shared and

distinctive origins and effects of adult attachment representations: The developmental organization

of romantic functioning. Child Development, 83, 1689-1702. doi: 10.1111/j.1467-8624.2012.01801.x

• Simpson, J. A., & Overall, N. C. (2014). Partner buffering of attachment insecurity. Current

Directions in Psychological Science, 23, 54-59. doi: 10.1177/0963721413510933

• Overall, N. C., Fletcher, G. J. O., Simpson, J. A., & Fillo, J. (2015). Attachment insecurity, biased

perceptions of romantic partners’ negative emotions, and hostile relationship behavior. Journal of

Personality and Social Psychology, 108, 730-749. doi: 10.1037/a0038987

• Girme, Y. U., Agnew, C. R., Harvey, S. M., VanderDrift, L. W., Rholes, W. S., & Simpson, J. A. (2018).

The ebbs and flows of attachment: Within-person fluctuations in attachment undermine secure

individuals’ relationship wellbeing across time. Journal of Personality and Social Psychology, 114,

397- 421. doi: 10.1037/pspi0000115

The Transition to Parenthood, Chronic Stress, and Adaptation. The transition to parenthood is one 

of the most intensely and chronically stressful life events that the majority of people in our society encounter at 

some point during their lives. In general, most new parents experience sharp downturns in relationship quality 

and upturns in physical and/or mental health problems during the first 1-3 years of the transition period. Not 

everyone, however, experiences relationship declines or more personal problems; some partners remain 

steady, and a few actually report improvements in well-being. Who are the individuals who experience declines 

in marital satisfaction and/or increases in depression, and which individuals tend to fare better? Our 

longstanding research on the transition to parenthood, much of which has adopted an attachment perspective, 

has shed clarifying light on these important questions. Five sample publications from this program of research 

are listed below: 

• Rholes, W. S., Simpson, J. A., Campbell, L., & Grich, J. (2001). Adult attachment and the transition to

parenthood. Journal of Personality and Social Psychology, 81, 421-435. doi: 10.1037/0022-3514.81.3.421

• Simpson, J. A., Rholes, W. S., Campbell, L., Tran, S., & Wilson, C. L. (2003). Adult attachment, the

transition to parenthood, and depressive symptoms. Journal of Personality and Social Psychology, 84,

1172-1187. doi: 10.1037/0022-3514.84.6.1172

• Rholes, W. S., Simpson, J. A., Kohn, J. L., Wilson, C. L. Martin, A. M., Tran, S., & Kashy, D. A. (2011).

Attachment orientations and depression: A longitudinal study of new parents. Journal of Personality and

Social Psychology, 100, 567-586. doi: 10.1037/a0022802

• Fillo, J., Simpson, J. A., Rholes, W. S., & Kohn, J. L. (2015). Dads doing diapers: Individual and

relational outcomes associated with the division of childcare across the transition to parenthood. Journal

of Personality and Social Psychology, 108, 298-316. doi: 10.1037/a0038572

• Simpson J. A. & Rholes, W. S. (in press). Adult attachment orientations and well-being during the

transition to parenthood. Current Opinion in Psychology, 25, 47-52. doi:

10.1016/j.copsyc.2018.02.019

Human Mating from an Evolutionary/Life History Perspective. In recent years, growing attention 

has focused on the evolutionary pressures that our ancestors most likely faced during evolutionary history and 

how these pressures may have affected the way in which we think, feel, and behave in different types of 

situations and life domains. One domain that has been of key interest is mating patterns in humans. In 2000, 

Steve Gangestad and I develop what has become one of the 3 or 4 major theoretical models of human mating in 

the biological, social, and life sciences—strategic pluralism theory. This model, which has launched hundreds of 

published studies, suggests the human females evolved to make trade-offs between a prospective male 

partner’s willingness and ability to invest (parental investment) and his evidence of having “good genes” 

(associated with better health and higher reproductive success). Women are conjectured to make these tradeoffs while weighing specific features of their current environments (e.g., the prevalence of pathogens, the 

availability of caregiving help from kin, the environmental need for biparental care). Strategic pluralism theory 

also helped launch research on how the female ovulatory cycle intersects with short-term and long-term mating 

decisions in women. Five sample publications from this line of research are listed below: 

Contact PD/PI: Munson, Benjamin R 

Biosketches

 Page 26 
• Gangestad, S. W., & Simpson, J. A. (2000). The evolution of human mating: Trade-offs and strategic

pluralism. Behavioral and Brain Sciences, 23, 573-587. doi: 10.1017/ S0140525X 0000 337X

• Gangestad, S. W., Garver-Apgar, C. E., Simpson, J. A., & Cousins, A. J. (2007). Changes in women’s

mate preferences across the ovulatory cycle. Journal of Personality and Social Psychology, 92, 151-

163. doi: 10.1037/0022-3514.92.1.151

• Cantú, S. M., Simpson, J. A., Griskevicius, V., Weisberg, Y. J., Durante, K. M., & Beal, D. (2014).

Fertile and selectively flirty: Women's behavior toward men changes across the ovulatory cycle.

Psychological Science, 25, 431-438. doi: 10.1177/0956797613508413

• Fletcher, G. J. O., Simpson, J. A., Campbell, L., & Overall, N. C. (2015). Pair-bonding, romantic love,

and evolution: The curious case of Homo sapiens. Perspectives on Psychological Science, 10, 20-36.

doi: 10.1177/1745691614561683

• Szepsenwol, O., & Simpson J. A. (in press). Attachment within life history theory: An evolutionary

perspective of individual differences in attachment. Current Opinion in Psychology, 25, 65-70. doi:

10.1016/j.copsyc.2018.03.005

D. Research Support

Ongoing

Australian Research Council (Karantzas) 01/01/16-12/31/18

Caring for older parents: Determining the role of attachment security (ARC) 

This research is investigating: (1) how attachment bonds between adult children and older parents influence 

the parental care provided by children in later life, caregiver burden, and parents’ satisfaction with care; and 

(2) the effects of enhancing caregivers’ attachment security on caregiving interactions and caregiver burden.

This project will provide insights into strategies to strengthen family bonds and services required to support

caregivers and care recipients and enhance the sustainability of aged care in the community.

Role: International co-Investigator

National Science Foundation (Simpson) 09/1/17-08/30/20

Developmental and interpersonal antecedents of parenting orientations and behavior: A 

life history perspective (NSF) 

This research is: (1) examining whether certain types/amounts of perceived early-life stress are associated with 

parenting orientations and behavior in a family context, (2) investigating whether certain aspects of the current 

romantic relationship buffer exposure to early-life stress, (3) identifying whether different types/amounts of 

perceived early-life stress are associated with fathers’ and mothers’ implicit (vs. explicit) parenting attitudes, 

and (4) determining whether implicit parenting attitudes mediate the link between early-life stress and explicit 

parenting orientations. 

Completed 

R01 AG039453 (Simpson) 09/30/11-06/30/18 

Early Life Stress, Developmental Processes, and Adult Health (NIA) 

This research examined the mechanisms responsible for associations between psychosocial stress earlier in life 

and health outcomes as individuals age. More specifically, using the Minnesota Longitudinal Study of Risk and 

Adaptation, this research tested whether, when, and how exposure to different types and amounts of stress 

early in life statistically interacted with life stress in adulthood to predict negative health outcomes, including 

standard biomarkers of health and aging. 

Role: Principal Investigator 

1057482 (Simpson) 06/01/11-05/30/15 

Interaction of current and childhood environment on risky decision-making: An experimental 

and longitudinal life-history theory approach (NSF) 

This research investigated the way in which an individual’s early childhood environment works together with 

(statistically interacts with) his or her current environment in adulthood to influence risk-taking behavior in 

different life domains. In several laboratory experiments and analyses with the MLSRA data, we found that 

people who were exposed to more unpredictable stress early in life—especially during the first 5 years—were 

more likely to take different kinds of risk (e.g., financial, sexual, interpersonal), but only when they were also 

exposed to current (immediate) threat/stress. This work reveals how early unpredictable stress combines with 

current environmental stress to lead certain people toward greater risk-taking in adulthood 

Role: Principal Investigator 

Biosketches

 Page 27 
OMB No. 0925-0001 and 0925-0002 (Rev. 09/17 Approved Through 03/31/2020) 

BIOGRAPHICAL SKETCH 

Provide the following information for the Senior/key personnel and other significant contr butors. 

Follow this format for each person. DO NOT EXCEED FIVE PAGES. 

NAME: Oxenham, Andrew, J. 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Distinguished McKnight University Professor 

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

include postdoctoral training and residency training if applicable. Add/delete rows as necessary.) 

INSTITUTION AND LOCATION 

DEGREE 

(if 

applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

University of Surrey, UK B.Mus. 06/1992 Music and Sound 

Recording 

University of Cambridge, UK Ph.D. 09/1995 Experimental 

Psychology 

Institute for Perception Research (IPO), 

Eindhoven, The Netherlands Postdoc 09/1997 Psychoacoustics 

A. Personal Statement

I have a strong track record in studying auditory and speech perception and the effects of hearing loss and

cochlear implantation on speech perception abilities. I am excited to be collaborating with the PI, Dr. Munson,

on this proposal, as I believe our experience complement each other’s in ways that will lead to synergy in the

resulting scientific approaches. In particular, I am keen to be involved in studying the interactions between the

talker and perceiver in measures of speech intelligibility. This important component has not received sufficient

attention, and could play a particularly important role in the speech understanding of people with hearing

difficulties. The references below are examples of my work in speech perception in populations with hearing

loss and cochlear implants.

References: 

Feng, L., Oxenham, A. J. (2018). “Effects of spectral resolution on speech context effects in cochlear-implant 

users,” Journal of the Acoustical Society of America 143, EL468-473. 

Oxenham, A. J., Kreft, H. (2014). “Speech perception in tones and noise via cochlear implants reveals 

influence of spectral resolution on temporal processing,” Trends in Hearing 18, 1-14. 

Ruggles, D. R., Freyman, R. L., Oxenham, A. J. (2014). “Influence of musical training on understanding voiced 

and whispered speech in noise,” PLOS One 9, e86980, 1-8. 

Gregan, M. J., Nelson, P. B., Oxenham, A. J. (2013). “Behavioral measures of cochlear compression and 

temporal resolution as predictors of speech masking release in hearing-impaired listeners,” Journal of the 

Acoustical Society of America 134, 2895-2912. 

B. Positions and Honors

Positions and Employment

1995-1997 Wellcome Trust International Prize Research Fellow at the Institute for Perception Research 

(IPO), Eindhoven, The Netherlands. 

1997-1999 Research Scientist, Northeastern University, Department of Speech-Language Pathology and 

Audiology. 

1999-2002 Research Scientist, Research Laboratory of Electronics, MIT. 

2003-2006 Principal Research Scientist, Research Laboratory of Electronics, MIT. 

Biosketches

 Page 28 
2006-2007 Assistant Professor, Department of Psychology, University of Minnesota 

2007-2010 Associate Professor, Department of Psychology, University of Minnesota 

2008-2010 Adjunct Associate Professor, Department of Otolaryngology, University of Minnesota 

2010-present Professor, Department of Psychology, University of Minnesota 

2010-present Professor, Department of Otolaryngology, University of Minnesota 

2013-present Distinguished McKnight University Professor, University of Minnesota 

Other Experience and Professional Memberships 

1992- Member, Acoustical Society of America, Association for Research in Otolaryngology, American 

Auditory Society 

2001-2004 Psychological and Physiological Acoustics Technical Committee, Acoustical Society of America 

2004 - Ad hoc reviewer for NIH panels, including R01s, R03s, Training and Fellowship grants. 

2004-2007 Associate Editor, Journal of the Acoustical Society of America 

2006-2009 Associate Editor, Journal of the Association for Research in Otolaryngology 

2008-2011 Chair, Technical Committee for Psychological and Physiological Acoustics, Acoustical Society 

of America 

2010-2014 Standing member of the NIH CDRC review panel 

2013-2014 Chair, NIH CDRC review panel 

2013-2014 Section Editor, Ear and Hearing 

2015-2018 Executive Council Member, Acoustical Society of America 

2014-present Editor in Chief, Trends in Hearing 

Honors 

2001 R. Bruce Lindsay Award, Acoustical Society of America 

2003 Ear and Hearing Editor’s Award for Best Paper of 2003 (Oxenham and Bacon, 2003). 

2003 Fellow of the Acoustical Society of America 

2005 McMahon Mentoring Award, Harvard–MIT Division of Health Sciences and Technology 

2006 Kavli Fellow, National Academy of Sciences (Invitee to Frontiers of Science Symposium) 

2009 Troland Award, National Academy of Sciences 

2013 Distinguished McKnight University Professor, University of Minnesota 

2014 Elected to the Collegium Oto-Rhino-Laryngologicum Amicae Sacrum (CORLAS). One of six 

non-clinician members from the US. 

C. Contributions to Science 

1) Behavioral measures and consequences of cochlear compression 

This work emanated from my PhD dissertation with Prof. Brian C. J. Moore, where I studied the additivity of 

non-simultaneous masking in normal-hearing and hearing-impaired listeners. I discovered that hearingimpaired listeners tended to show less masking summation than normal-hearing listeners, and that this 

reduced additivity could be described in terms of reduced dynamic range compression at the front-end of an 

auditory model. The finding of reduced compression came at a time when the highly compressive nature of the 

basilar-membrane input-output function was being generally accepted, after many years of controversy in the 

field of auditory physiology. My follow-on work was to link cochlear compression to the compression in the 

model, and (together with Chris Plack) to derive a novel psychoacoustic measure of compression that was 

found to be in good agreement with invasive measures in other mammals. The work was used to explain 

several aspects of the perceptual consequences of hearing loss in terms of loss of cochlear compression, 

including aspects of temporal processing and speech perception, and it led to a large number of studies that 

continue to explore behavioral measures and consequences of cochlear nonlinearity in normal and impaired 

hearing. 

References: 

Oxenham, A. J., and Plack, C. J. (1997). “A behavioral measure of basilar-membrane nonlinearity in listeners 

with normal and impaired hearing,” Journal of the Acoustical Society of America 101, 3666-3675. 

Moore, B. C. J., and Oxenham, A. J. (1998). “Perceptual consequences of compression in the peripheral 

auditory system,” Psychological Review 105, 108-124 

Oxenham, A. J., and Bacon, S. P. (2003). “Cochlear compression: Perceptual measures and implications for 

normal and impaired hearing,” Ear and Hearing 24, 352-366. 

Biosketches 

 Page 29 
Gregan, M. J., Nelson, P. B., Oxenham, A. J. (2013). “Behavioral measures of cochlear compression and 

temporal resolution as predictors of speech masking release in hearing-impaired listeners,” Journal of the 

Acoustical Society of America 134, 2895-2912. PMCID: PMC3799689. 

2) Behavioral and otoacoustic emission measures of human cochlear filtering 

In collaboration with Chris Shera and John Guinan, I worked on a project to compare human cochlear tuning 

with that in smaller mammals (cat, guinea-pig, chinchilla) that are typically used for auditory research. We 

combined stimulus-frequency otoacoustic emission (SFOAE) measurements with a new variant of the 

behavioral notched-noise technique to estimate human tuning and compared it with SFOAE measurements 

and auditory-nerve tuning curves in the other mammals. We found that the pattern of tuning (increasing 

sharpness with increasing center frequency) was very similar across the species, but that human tuning 

appeared to be sharper than that in the smaller mammals, by a factor of about 2. Although the finding remains 

controversial, it has received recent support from other groups, who have reported intermediate tuning in oldworld monkeys (Joris et al., 2011, PNAS). The work has important implications for any conclusions about 

human auditory perception and processing, based on animal physiology and auditory models. In collaboration 

with Torsten Dau, we also devised methods to estimate the phase curvature of the auditory filters across the 

frequency spectrum. In combination the data from these collected studies provides the necessary data to 

derive a comprehensive description of human auditory filtering. 

References: 

Oxenham, A. J., and Dau, T. (2001). “Towards a measure of auditory-filter phase response,” Journal of the 

Acoustical Society of America 110, 3169-3178 

Shera, C. A., Guinan, J. J., and Oxenham, A. J. (2002). “Revised estimates of human cochlear tuning from 

otoacoustic and behavioral measurements,” Proceedings of the National Academy of Sciences USA 99, 

3318-3323. 

Oxenham, A. J., and Simonson, A. M. (2006). “Level dependence of auditory filters in nonsimultaneous 

masking as a function of frequency,” Journal of the Acoustical Society of America 119, 444-453. 

Shera, C. A., Guinan, J. J., and Oxenham, A. J. (2010). “Otoacoustic estimation of cochlear tuning: Validation 

in the chinchilla,” Journal of the Association for Research in Otolaryngology 11, 343-365. PMCID: 

PMC2914235. 

3) Pitch perception and processing in normal and impaired hearing 

Our work on pitch perception has focused on questions regarding the basic mechanisms of F0 coding in the 

auditory periphery and its role in speech understanding in complex environments. Our early work confirmed 

that resolved harmonics were necessary for accurate pitch perception, but found that artificially resolving 

higher-order harmonics did not lead to better pitch perception, suggesting a role of templates formed only of 

normally resolved harmonics. We have also examined the role of different temporal cues for pitch and speech 

perception. Our work on ‘auditory chimeras’, which attempted to dissociate the role of temporal fine structure 

from temporal envelope has been highly cited (over 500 according to Google Scholar) and has inspired many 

follow-up studies. Using ‘transposed stimuli’ that present low-frequency temporal information to high-frequency 

cochlear locations, we demonstrated that timing information was not sufficient to induce complex pitch 

perception. Later, using very high-frequency harmonics (> 6 kHz), we showed that phase-locked information 

was also not necessary to induce complex pitch perception. Overall, the results have moved forward the field 

of pitch perception research and have overturned some ‘textbook facts’, such as the absence of pitch when all 

components were higher than 5 or 6 kHz, which in turn have changed how we view the basic mechanisms of 

pitch. 

References: 

Smith, Z. M., Delgutte, B., and Oxenham, A. J. (2002). “Chimaeric sounds reveal dichotomies in auditory 

perception,” Nature 416, 87-90 

Oxenham, A. J., Bernstein, J. G. W., and Penagos, H. (2004). “Correct tonotopic representation is necessary 

for complex pitch perception,” Proceedings of the National Academy of Sciences USA 101, 1421-1425. 

Oxenham, A. J., Micheyl, C., Keebler, M. V., Loper, A., Santurette, S. (2011). “Pitch perception beyond the 

traditional existence region of pitch,” Proceedings of the National Academy of Sciences USA. 108, 7629-

7634. PMCID: PMC3088642. 

Lau, B. K., Mehta, A. H., Oxenham, A. J. (2017). “Superoptimal Perceptual Integration Suggests a PlaceBased Representation of Pitch at High Frequencies,” Journal of Neuroscience 37, 9013-9021. PMCID: 

PMC5597982. 

Biosketches 

 Page 30 
4) Auditory streaming and perceptual organization

We provided the first demonstration that auditory stream segregation can occur in the absence of tonotopic 

(cochlear) differences between stimuli by inducing streaming of complex harmonic tones, comprising of only 

unresolved harmonics. We have also manipulated the interactions between simultaneous and sequential 

grouping constraints to induce an interesting illusion, whereby a component potentially belonging to two 

objects is allocated to neither and perceptually ‘disappears’. More recently, we combined human behavioral 

and animal single-unit data to show how streaming depends on temporal coherence and how it is represented 

in A1. We have also demonstrated MEG correlates of auditory awareness, showing that early cortical 

responses to pure tones are robustly present, whether or not they are detected, whereas later responses 

(latency of ~100 ms) seem entirely absent when the tones remain undetected or unattended. 

References: 

Gutschalk, A., Micheyl, C., and Oxenham, A. J. (2008). “Neural correlates of auditory perceptual awareness 

under informational masking,” PLOS Biology 6, 1156-1165 (e138). PMCID: PMC2422852. 

Elhilali, M., Ma, L., Micheyl, C., Oxenham, A. J., and Shamma, S. (2009) “Temporal coherence in the 

perceptual organization and cortical representation of auditory scenes,” Neuron 61, 317-329. PMCID: 

PMC2673083. 

McDermott, J. H., Wrobleski, D., Oxenham, A. J. (2011). “Recovering sound sources from embedded 

repetition,” Proceedings of the National Academy of Sciences USA 108, 1188-1193. PMCID: PMC3024660. 

Lu, K., Xu, Y., Yin, P., Oxenham, A. J., Fritz, J., Shamma, S. A. (2017). “Temporal coherence structure rapidly 

shapes neuronal interactions,” Nature Communications 7, 13900 doi:10.1038/ncomms13900. PMCID: 

PMC5206281 

5) Effects of spectro-temporal interactions in speech perception by cochlear-implant users

Our lab has made a number of discoveries concerning auditory perception via cochlear implants, and have 

developed new tests for rapid assessment of thresholds and spectral tuning. One potentially important finding 

relates to the influence of inherent temporal-envelope fluctuations in noise on the ability to understand speech. 

Earlier work in normal-hearing listeners showed that speech perception was limited by the presence of the 

temporal envelope fluctuations inherent in noise. Our surprising results showed that this was not the case for 

cochlear-implant users, despite the fact that they rely on envelope information to understand speech 

(Oxenham and Kreft, 2014). We explained the results in terms of poorer spectral resolution leading to 

spectrally overlapping temporal envelopes, which in turn smoothed out the envelope fluctuations of the noise. 

We were able to reproduce the results from cochlear-implant users in a group of normal-hearing listeners by 

simulating the expected effects of spectral spread. The results also suggested new and more sensitive ways to 

test spectral resolution in cochlear-implant users. More recently, we have discovered that some, but not all, 

aspects of auditory enhancement are found in cochlear-implant users (Kreft and Oxenham, 2017), suggesting 

that the missing aspects could be restored via signal processing. 

Wang, N., Kreft, H. A., Oxenham, A. J. (2015). “Loudness context effects in normal-hearing listeners and 

cochlear-implant users,” Journal of the Association for Research in Otolaryngology 16, 535-545. PMCID: 

PMC4488167 

Oxenham, A. J., Kreft, H. (2014). “Speech perception in tones and noise via cochlear implants reveals 

influence of spectral resolution on temporal processing,” Trends in Hearing 18, 1-14. PMCID: 

PMC4227666. 

Kreft, H. A., Nelson, D. A., Oxenham, A. J. (2013). “Modulation frequency discrimination with modulated and 

unmodulated interference in normal hearing and in cochlear-implant users,” Journal of the Association for 

Research in Otolaryngology 14, 591-601. PMCID: PMC3705089. 

Wang, N., Kreft, H., and Oxenham, A. J. (2012). “Vowel enhancement effects in cochlear-implant users,” 

Journal of the Acoustical Society of America 131, EL421-426. PMID: 22713016. 

Complete list of published works in MyBibliography: 

My bibliography is available here (total of 155+ peer-reviewed journal articles): 

http://www.ncbi.nlm.nih.gov/sites/myncbi/andrew.oxenham.1/bibliography/40507433/public/?sort=date&directio 

n=ascending 

Biosketches

 Page 31 
D. Additional Information: Research Support and/or Scholastic Performance 

NSF-BCS 1840818 Oxenham (PI) 09/15/2018-08/31/2020 

NSF 

NeuroDataRR. Collaborative Research: Testing the relationship between musical training and enhanced neural 

coding and perception in noise 

This multi-site project seeks to replicate findings relating musical training to enhanced neural coding and 

auditory and speech perception in noise. 

Role: Lead PI 

R01 DC 012262 Oxenham (PI) 03/01/2013-02/28/2019 

NIH/NIDCD 

Spectro-temporal interactions in electric and acoustic processing and auditory perception 

This project studies auditory and speech perception in cochlear-implant users and hearing-impaired listeners 

and is designed to develop new processing algorithms to aid speech perception in complex and changing 

acoustic environments. 

Role: PI 

R01 DC 005216 Oxenham (PI) 02/01/2002-01/31/2022 

NIH/NIDCD 

Complex pitch perception in complex environments 

This study uses behavioral methods to investigate the mechanisms of pitch perception in normal and impaired 

hearing and their role in understanding speech in complex backgrounds. 

Role: PI 

R01DC015987 M. Wojtczak (PI) 04/01/2017-03/31/2022 

NIH/NIDCD 

Physiological measures and perceptual consequences of noise-induced auditory synaptopathy in humans 

This study investigates the newly discovered phenomenon of “hidden hearing loss” in humans to develop 

behavioral and non-invasive physiological tests and to determine whether this is a widespread problem in our 

population. 

Role: Co-investigator 

R01DC015462 M. Wojtczak (PI) 07/01/2017-06/30/2022 

NIH/NIDCD 

Perceptual and functional role of medial olivocochlear efferents in humans 

This study investigates the potential role of efferents in auditory and speech perception, and explores novel 

ways to measure efferent activation in humans. 

Role: Co-investigator 

Past research support 

R01 DC 007657 S. Shamma (PI) 04/01/2006-05/31/2017 

NIH/NIDCD 

Neural Correlates of Streaming of Complex Sounds 

This study combines human behavioral and animal neural studies in awake behaving ferrets to study the 

neural correlates of auditory scene analysis. 

Role: PI of University of subcontract 

Biosketches 

 Page 32 
OMB No. 0925-0001 and 0925-0002 (Rev. 10/15 Approved Through 10/31/2018) 

BIOGRAPHICAL SKETCH 

NAME: Peggy B. Nelson 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Professor 

EDUCATION/TRAINING 

INSTITUTION AND LOCATION DEGREE 

(if applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

St Olaf College, Northfield, MN B.A. 05/1974 Psychology, Mathematics 

Kansas State University, Manhattan, KS M.A. 05/1982 Speech (Audiology) 

University of Kansas, Lawrence, KS Ph.D. 04/1991 Audiology 

A. Personal Statement 

I am very enthused and I believe well positioned to contribute to the proposed project on social variation in 

audiovisual speech perception, in particular as it pertains to studying older persons and people with 

hearing loss. We are just completing a study of self-adjusting hearing aids, and the results are promising 

and exciting. I have extensive experience evaluating listeners’ speech understanding in background noise. 

I have direct experience in measuring the diversity of speech recognition abilities and other outcome 

measures seen in groups of listeners with similar hearing loss. I bring audiological expertise in devising 

appropriate speech tests for our subjects. I direct a multisensory research laboratory (Center for Applied 

and Translational Sensory Science, CATSS) and have extensive experience collaborating with other 

investigators who study vision and vision loss. We currently have initiated a study on the audiovisual 

perception of spatial cues. I believe we have an outstanding team and strong preliminary data to launch 

this important project. 

I have worked in translational projects with industry partners on research and training projects. Some of 

our recent findings have implications for clinical practice, including the following from the past five years: 

1. Nie, Y., Zhang, Y., and Nelson, P.B. (2014) Auditory stream segregation using bandpass noises: 

evidence from event-related potentials. Frontiers in Neuroscience. Vol. 8 August, doi: 10.3389 

2. Miller, S., Zhang, Y. and Nelson, P (2015) Efficacy of multiple-talker phonetic identification 

training in postlingually deafened cochlear implant listeners. Journal of Speech, Language, and 

Hearing Research doi:10.1044/2015_JSLHR-H-15-0154 

3. Svec, A., Nelson, P. and Dubno, J. (2016) "Inherent envelope fluctuations in forward maskers: 

Effects of masker-probe delay for listeners with normal and impaired hearing" J Acoust. Soc. Am. 

139, 1195 (2016); http://dx.doi.org/10.1121/1.4944041 

4. Nelson, P., Gregan, M., Perry, T., and VanTasell, D. (2018) "Self-adjusted amplification parameters 

produce large between-subject variability and preserved speech intelligibility in noise”, Trends in 

Hearing. 22. 1 – 13. doi.org/10.1177/2331216518798264 

B. Positions and Honors: 

1974 - 1975: Teacher aide, interpreter, Central NC School for the Deaf 

1975 - 1977: Director, Guilford County (NC) Communications Center for the Deaf 

1977 - 1979: Sign interpreter, sign language instructor, Pima County (AZ) Community College 

1979 - 1982: Graduate Assistant, Purdue University and Kansas State University 

1982 - 1988: Instructor, Audiology, Speech Pathology/Audiology Program, Kansas State Univ. 

1988 - 1992: Instructor/Clinic Director, Hearing and Speech Dept, Univ. of Kansas Medical Center 

Biosketches 

 Page 33 
1992 - 1996: Research scientist, Gallaudet University Center for Auditory and Speech Sciences, 

1996 - 1999: Assistant Professor and Director of Pediatric Audiology, Div. of Otolaryngology, UMB 

1997 - 1999: Adjunct Assistant Professor, Dept. of Audiology & SLP, Gallaudet University 

2000 – 2005: Assistant Professor, Dept. of Communication Disorders, University of Minnesota 

2005 – 2012: Associate Professor, Department of Speech-Language-Hearing Sci., Univ. of Minnesota 

2008 – 2015: Chair, Department of Speech-Language-Hearing Sciences, Univ. of Minnesota 

2013 – Present: Professor, Department of Speech-Language-Hearing Sciences, Univ. of Minnesota 

2015 – Present: Director, Center for Applied and Translational Sensory Science, Univ. of Minnesota 

National Board Memberships: 

1999 – 2002 & 2011 - present: Member, Physiological and Psychological Acoustics Technical Cmte, ASA 

1999 – present: Member, Acoustical Society of America Committee on Women in Acoustics 

1997 – present: Member, Acoustical Society of America (ASA) Task Force on Classroom Acoustics 

2002 – 2007: Member, American Speech-Language-Hearing Assoc Research and Scientific Affairs 

2002 – 2009: Member, Speech Communication Technical Committee, Acoustical Society of America 

2006 – 2009: Member, executive council, Acoustical Society of America 

2007 – present: Member, American Academy of Audiology Task Force on classroom acoustics 

2013 – present: Vice Chair of ANSI Standards S3 Committee 

2014 – present: Vice President for Scholarship and Research, Council of Academic Programs of 

 Communication Sciences and Disorders 

2018 – present: Vice President elect, Acoustical Society of America 

Honors: 

2002, Editor’s award, American Journal of Audiology 

2003, Editor’s award, American Journal of Audiology 

2005, Fellow of the Acoustical Society of America 

2010, Fellow of the American Speech-Language-Hearing Association 

2011, Editor’s award, ASHA Leader 

C. Contributions to Science 

1. My early projects investigated factors affecting speech perception in listeners with hearing loss. These 

focused on speech-like psychoacoustic tasks that were closely related to speech perception tasks, in 

listeners with cochlear hearing loss (hearing aid users) and those with severe loss who use cochlear 

implants. Together with doctoral students, we have investigated the effects of increased signal level, 

audibility of the signal, spectral resolution, and the resulting speech abilities in quiet. Relationships 

revealed considerable variability in speech recognition unexplained by psychoacoustic abilities, 

audibility, or level. Publications include: 

a. Nelson, P., & Dwyer-Thomas, S. (1997). Gap detection as a function of stimulus loudness for 

listeners with and without hearing loss. Journal of Speech, Language, and Hearing Research, 

40, 1387-1394. 

b. DiGiovanni, J., Nelson, P., & Schlauch, R. (2005). A psychophysical evaluation of spectral 

enhancement. Journal of Speech, Language, and Hearing Research. 48 (5), 1121-1135. 

16411801 

c. Munson, B., & Nelson, P. (2005). Phonetic identification in quiet and in noise by listeners with 

cochlear implants. Journal of Acoustical Society of America 118, 2607-2617. 

d. Miller, S., Zhang, Y. and Nelson, P. (2016) Neural correlates of phonetic learning in 

postlingually deafened cochlear implant listeners, Ear and Hearing 37 (3), 247-374. DOI: 

10.1097/AUD.0000000000000287 

2. A second area focused on the effects of room acoustics on listener performance, including listeners 

with hearing loss, and listening in second languages. Some work was coordinated with the Acoustical 

Society of America classroom acoustics coalition. Study of hearing devices and second language 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 34 
learning revealed complex negative effects of background noise on learning. Studies were highly 

collaborative with colleagues and graduate students in multiple universities. Publications include: 

a. Knecht, H., Nelson, P. Whitelaw, G., & Feth, L. (2002) Background noise levels and 

reverberation times in unoccupied classrooms: Predictions and measurements, American 

Journal of Audiology, 11, 65-71. 

b. Miller-Hansen, D.R., Nelson, P., Widen, J. E., & Simon, S. D. (2003). Evaluating the benefit 

of speech recoding hearing aids in children. American Journal of Audiology, 12, 106-113. 

c. Nelson, P., Kohnert, K., Sabur, S., & Shaw, D. (2005). Classroom noise and children 

learning through a second language: Double jeopardy? Language, Speech, and Hearing 

Services in Schools, 36, 219 – 229. 

d. Koerner, T., Zhang Y, Nelson P, Wang, B. and Zou, H. (2017) Neural indices of phonemic 

discrimination and sentence-level speech intelligibility in quiet and noise: A P3 study. 

Hearing Research, 350: 58-67. https://doi.org/10.1016/j.heares.2017.04.009 

3. A third scientific area investigated the role and nature of masking release in speech perception 

performance for listeners with hearing aids and cochlear implants. These were among the first papers 

to identify the nearly complete lack of masking release in listeners with cochlear implants, and to 

investigate possible causes. Several doctoral students and I collaborated on these projects, as seen 

in the list below. Publications include: 

a. Nelson, P., Jin, S.-H., Carney, A.E., & Nelson, D. A. (2003). Understanding speech in 

modulated interference: Cochlear implant users and normal-hearing listeners. Journal of the 

Acoustical Society of America, 113, 961-968. 

b. Nelson, P., & Jin, S.-H. (2004). Factors affecting speech understanding in gated interference: 

Cochlear implant users and normal-hearing listeners. Journal of Acoustical Society of America, 

114, 2286-2294. 

c. Jin, S. & Nelson, P. (2006) Speech perception in gated noise: the effects of temporal 

resolution, Journal of Acoustical Society of America 119, 3097 – 3108. (2, 4, 5, 6, 7) 

d. Jin, S. and Nelson, P. (2010) Interrupted speech perception: The effects of hearing sensitivity 

and frequency resolution, Journal of the Acoustical Society of America. 128, 881 – 889. 

PMC293326 (2, 4, 5, 6, 7) 

4. A fourth area focused on the role of cochlear compression for performance of hearing-impaired 

listeners in noise. These included measures of spectral resolution and cochlear compression and their 

predictive ability for understanding speech in noise for listeners who use acoustic hearing and hearing 

aids. This work was accomplished collaboratively with colleagues and doctoral students Anderson and 

Gregan. No direct relationships between measures of cochlear compression and performance in noise 

were seen. Considerable variability in HI listeners’ performance in noise pointed us in other directions 

for an explanation. Publications include: 

a. Gregan, M.J., Nelson, P. B. and Oxenham, A. J, and Nelson, P. (2010) Effects of background 

noise level on behavioral estimates of basilar-membrane compression, Journal of the 

Acoustical Society of America, 127, 3018 – 3025. PMID: 21117751 (2, 4, 5, 6, 7) 

b. Anderson ES, Nelson DA, Kreft H, Nelson PB, Oxenham AJ. (2011) Comparing spatial tuning 

curves and spectral ripple resolution in cochlear implant users, Journal of the Acoustical 

Society of America, 130, 364-375. PMID: 21786905 (2, 4) 

c. Gregan, M., Nelson, P. and Oxenham, A. (2011) Behavioral estimates of basilar-membrane 

compression: Additivity of forward masking in noise-masked normal-hearing listeners, Journal 

of the Acoustical Society of America 130, 2835-2844 (2, 4) 

d. Gregan, J. Nelson, P. and Oxenham, A (2013) Behavioral measures of cochlear compression 

and temporal resolution as predictors of speech intelligibility and masking release in hearingimpaired listeners, JASA 134, 2895-2904.(2, 4, 6, 7) 

5. A fifth scientific area is the role of amplitude modulation confusion in the problem of speech 

recognition in noise. This research follows up on the variability noted and further explains some 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 35 
problems on listeners with hearing loss, and has implications for sensory aid use. Publications 

include: 

a. Jin, S.H., Nie, Y., and Nelson, P. (2013) Modulation interference and masking release in 

cochlear implant users, American Journal of Audiology 135-46. doi: 10.1044/1059-0889. (2, 4, 

6) 

b. Nie, Y., Zhang, Y., and Nelson, P.B. (2014) Auditory stream segregation using bandpass 

noises: evidence from event-related potentials. Frontiers in Neuroscience. Vol. 8 August, doi: 

10.3389 

c. Svec, A., Nelson, P. and Dubno, J. (2015) Inherent fluctuations in forward maskers: Effects of 

temporal envelope, age, and hearing loss, JASA 137, 1336–1343. 

dx.doi.org/10.1121/1.4908567 

d. Miller, S., Nelson, P. and Zhang, Y. (2015) Efficacy of multiple-talker phonetic identification 

training in postlingually deafened cochlear implant listeners. Journal of Speech, Language, and 

Hearing Research doi:10.1044/2015_JSLHR-H-15-0154 

D. Research Support 

Ongoing research support: 

Principal investigator, Improving Amplification Outcomes in Noise by Self-Directed Hearing Aid Fitting 1 

R01 DC 13276, July1, 2013 - June 30, 2018. This grant focuses on fitting and follow-up for amplification 

systems in noisy situations. We are currently in a no-cost extension 

Investigator, National Science Foundation training grant 

Graduate Training Program in Sensory Science: Optimizing the Information Available for Mind and Brain, 

awarded to Center for Cognitive Sciences and Center for Applied and Translational Sensory Science, 

August 2017 

Co-investigator 

Xcel Energy Public Utilities Commission, 2014 - 2019 

Title: Wind turbine generated sound: Targeted research to improve measurement, analysis and annoyance 

thresholds based on measured human response 

PI: Jeffrey Marr, St Anthony Falls Laboratories 

Co-investigator 

Department of Energy DE-FOA-0001554: Wind energy 00 Eagle impact minimization technologies and 

field testing opportunities. 

Detection and perception of sound by eagles and surrogated raptors 

Award notice received Nov 29, 2016 

Co-Investigator 

Andrew Oxenham, Principal Investigator NIH R01 

Spectro-temporal interactions in cochlear implant processing and perception 

Spring 2013 – 2018 

Co-Investigator 

Magda Wojtczak, Principal Investigator NIH R01 

Physiological measures and perception 

Awarded Spring 2018 

Completed research support: 

Principal investigator, Masking release in hearing loss: Cochlear compression and effective audibility, 

National Institute on Deafness and Other Communication Disorders 1 R01 DC008306, period 4/1/08 – 

3/31/13. The current grant focuses on speech understanding by listeners with hearing loss and their 

performance in steady and gated noise. 

Biosketches 

 Page 36 
For the following subcontracts, which are representative of others, I serve as the local principal 

investigator, testing the effectiveness of new technologies for the improvement of speech understanding in 

noise by listeners with hearing loss: 

Subcontractor, NIDRR (PI American Medical Electronics), “Connectivity to Modern Electronics for the 

Hearing Disabled” Oct 2010 – August 2012 

Subcontract, AME Ph I NIDRR "Enhancing Conversation Intelligibility for Hearing Aid Users in Noisy 

Environments." Mar 2012 – Feb 2013 

Subcontractor, NIH SBIR Advanced Medical Electronics, Head mounted magnetometer hearing aid. 

Submitted April 2012. Funded Summer 2012. 

Subcontractor, NIH SBIR Phase II (PI American Medical Electronics), submitted April 1010, “Visual aide to 

hearing aids” Phase II funded May 2012 - October 2013 

Principal investigator, NIH NIDCD 5R01DC8306 Diversity Supplement, “Masking release in hearing loss: 

Cochlear compression and effective audibility,” submitted May, 2009, September 2009 – May 2011 

Co-investigator, Mechanisms of auditory dysfunction, National Institute on Deafness and Other 

Communication Disorders, P01-DC00110-24A2, in collaboration with D. Nelson, Department of 

Otolaryngology, Multi-project grant, for period 6/1/99 – 5/31/04. 

Biosketches 

 Page 37 
OMB No. 0925-0001 and 0925-0002 (Rev. 09/17 Approved Through 03/31/2020) 

BIOGRAPHICAL SKETCH 

Provide the following information for the Senior/key personnel and other significant contr butors. 

Follow this format for each person. DO NOT EXCEED FIVE PAGES. 

NAME: Evelyn Davies-Venn 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Assistant Professor 

EDUCATION/TRAINING (Begin with baccalaureate or other initial professional education, such as nursing, 

INSTITUTION AND LOCATION DEGREE 

(if 

applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

University of Minnesota B.A 05/2002 Communication Sciences 

and Disorders 

University of Washington M.S. 12/2004 Audiology 

University of Washington Au.D. 08/2005 Audiology 

University of Washington Ph.D. 12/2010 Audiology 

Purdue University Post-Doc 04/2014 Audiology 

University of Minnesota Post-Doc 08/2015 Audiology 

A. Personal Statement 

This project will evaluate the effect of listeners' knowledge and expectations of socially meaningful variation 

affect speech communication by individuals with hearing loss. This is a timely project that aims to determine 

quantitative (i.e. signal processing) and qualitative (attitudes) that will guide our efforts on the use of emerging 

technology to improve access and success with hearing aids. I have been greatly interested in translational 

research that determine behavioral factors that influence outcomes with hearing aid amplification. My current 

research program measures behavioral and electrophysiological factors that govern individual variance in 

amplification outcomes in real-world listening environments. I am excited to be contributing my expertise in 

measuring hearing and hearing-aid related factors that influence social effects on speech perception for older 

adults with hearing loss. 

B. Positions and Honors 

Positions 

2002-2009 Research Assistant University of Washington 

2009-2010 NIH pre-doctoral Research Fellow University of Washington 

2010-2010 Visiting pre-doctoral Research Fellow Northwestern University 

2015 - Assistant Professor University of Minnesota 

Honors 

2003 McNair Scholar Graduate Fellow 

2004 Graduate Opportunity Award 

2004 Eugene Beebee Scholarship 

2010 American Academy of Audiology Student Award 

2011 ASHA Lessons for Success: Emerging Scientist Awardee 

2014 International Hearing Aid Convention (IHCON) Travel Award 

2015 NIH Future Leaders 

2016 NIH-National Research Mentoring Network (NRMN) P3 mentee 

Biosketches 

 Page 38 
2018 TRIO achievers award 

Other Experience and Professional Memberships 

2002- Member: American Academy of Audiology, Association for Research in Otolaryngology 

2002- Board member: Graduate student advisory council, Diversity Recruitment Advisory Board, 

University of Washington. ARO Diversity and Minority Affairs Committee. 

2010 - Ad-hoc Reviewer: Journal of the Acoustical Society of America, Journal of the American Academy 

of Audiology, Journal of Communication Sciences and Disorders, Ear and Hearing, Journal of 

Speech, Language, Hearing Research, IEEE signal processing. 

2015- Faculty Fellow, Center for Translational and Sensory Sciences 

C. Contributions to Science 

My main contribution to science can be summarized as translational research designed to alleviate the speech 

perception difficulties experienced by listeners with hearing loss. My research focuses on the interaction 

between sensory aids and fundamental aspects of auditory processing. I seek to determine factors that could 

be personalized to optimize amplification outcomes for individuals with hearing loss. 

For human psychophysics, my main contributions relate to determining the role of audibility, spectral resolution, 

and temporal distortion on amplification outcomes with non-linear hearing aids and devising clinical 

recommendations based on our research findings. 

1. Dirks, C., Shahin, A., Kwan, T., and Davies-Venn, E. (2018). Effect of musical training on EEG 

measures of spectro-temporal processing. The Journal of the Acoustical Society of America 143 

(3), 1962-1963. 

2. Davies-Venn, E., Oeding, K., and Haug, A. (2018). The effects of carrier bandwidth and intensity 

on spectral ripple perception in listeners with hearing loss. The Journal of the Acoustical Society 

of America 143 (3), 1748-1748. 

3. Nie, Y., Davies-Venn, E., Svec, A., & Nelson, P. Detection of Frequency-Glides in Gate and 

Steady Noise by Listeners with Normal Hearing and Sensorineural Hearing Loss. In revision for 

Ear and Hearing. 

4. Nelson, P., and Souza, P. (2015). Comparing auditory filter bandwidths, spectral ripple detection, 

spectral ripple discrimination and speech recognition: normal and impaired hearing. Journal of 

the Acoustical Society of America, 138: 492–503. 

5. Davies-Venn, E., and Souza, P. (2014). The role of audibility, spectral resolution, and cognition 

in explaining variance in susceptibility to temporal envelope distortion for listeners with mild to 

moderate and moderate-to-severe hearing loss. Journal of the American Academy of Audiology, 

25-29. 

For speech perception assessment, my main contribution relates to developing speech stimuli for laboratory 

testing and determining how non-linear amplification can be used to improve speech understanding for 

individuals with hearing loss. 

1. Hossain, E., Davies-Venn, E., and Zilany, M (in-press). A Nonintrusive Predictor of Speech 

Intelligibility Using a First-order Spectral Moment of the Bispectrum of the Spectrogram. 

Computer, Speech and Language. 

2. Alam, S., Zilany, M., and Davies-Venn, E. (2017). Effects of speech-shaped noise on consonant 

confusions in Malay. IEEE Region 10 Humanitarian Technology Conference (R10-HTC:)22-25. 

Biosketches 

 Page 39 
3. Gibson PL, Hedin DS, Davies-Venn E, Nelson P, Kramer K. (2012). Multi-microphone adaptive 

array augmented with visual cueing. Engineering in Medicine and Biology Society (EMBC), 

IEEE:1000-1003. 

4. Davies-Venn, E., Souza P., Brennan, M., and Stecker, G. (2009). Effects of audibility and 

multichannel wide dynamic range compression on consonant recognition for listeners with severe 

hearing loss. Ear and Hearing. 30:494-504. 

5. Davies-Venn, E., Souza, P., and Fabry, D. (2007). Speech and Music Quality Ratings for Linear 

and Non-linear Hearing Aid Circuitry. Journal of the American Academy of Audiology, 1 690-70. 

Complete List of Published Work in MyBibliography: here and here 

D. Additional Information: Research Support and/or Scholastic Performance 

On-going 

Hearing Health Foundation Davies-Venn (PI) 2018-2019 

Neural correlates of variance in amplification outcomes. For this project, we measure spectral processing 

to evaluate how lateral inhibition is enhanced or reduced at respective regions along the auditory pathway in 

listeners with normal auditory function. We aim to discover mechanisms that could be exploited to enhance the 

functional benefit of lateral inhibition in hearing aid amplification. 

Brain Imaging Davies-Venn (PI) 2017-2018 

Neural correlates of amplification outcomes for speech in loud and noisy environments. For this project, 

we measure concurrent subcortical and cortical processing of non-speech stimuli to evaluate how lateral 

inhibition is enhanced or reduced at respective regions along the auditory pathway in listeners with normal 

auditory function. We aim to discover mechanisms that could be exploited to enhance the functional benefit of 

lateral inhibition in individuals with enhances versus reduced inhibition. 

Grant-in-aid Davies-Venn (PI) 2016-2018 

Improving auditory performance variability in real-world environments 

For this project, we measure concurrent speech and noise features during a real-time dialogue in a clinical 

population of hearing aid users. We aim to discover high-intensity mechanisms that could be enhanced in 

hearing aid signal processing to improve amplified speech perception in noise. 

Completed Research Support 

F31DC010127-02 Davies-Venn (PI) 2009 – 2010 

Using spectral resolution to explain performance variability with severe loss. 

This translational research study examined mechanisms that explain the listening difficulties and amplification 

outcomes for listeners with severe hearing loss. 

Biosketches 

 Page 40 
OMB No. 0925-0001 and 0925-0002 (Rev. 10/15 Approved Through 10/31/2018) 

BIOGRAPHICAL SKETCH 

NAME: Molly E. Babel 

eRA COMMONS USER NAME (credential, e.g., agency login): 

POSITION TITLE: Associate Professor 

EDUCATION/TRAINING 

INSTITUTION AND LOCATION DEGREE 

(if applicable) 

Completion 

Date 

MM/YYYY 

FIELD OF STUDY 

University of Minnesota, Twin Cities, 

Minneapolis, MN 

B.A. 2004 Linguistics, Anthropology, 

Spanish Studies 

University of California, Berkeley, CA M.A. 2006 Linguistics 

University of California, Berkeley, CA Ph.D. 2009 Linguistics 

A. Personal Statement 

I have been a faculty member at the University of British Columbia for 9 years. In that time span, I have 

developed an interdisciplinary research program, which intertwines methods and theories from linguistics, 

speech science, experimental psychology, and social psychology. With this arsenal, I explore fundamental 

issues related to the role of experience in language knowledge, how social knowledge is manifested 

phonetically, and the mental representation of linguistic knowledge. In my research, experience and social 

cognition encompass attributes as basic as native language to more complex issues like how social 

expectations play out in both speech production and speech perception. Broadly, my research agenda has a 

primary focus on issues of phonetic theory and the processing and representation of phonetic variation. More 

specifically, my research examines perception and variation in the speech signal with particular attention on 

the relationship between speech perception and production and how social cognition moderates speech 

processing. I maintain multiple lines of research which investigate several aspects of this topic. 

B. Positions and Honors 

Positions 

Department of Linguistics, University of British Columbia, Vancouver, BC Canada. 

Assistant Professor, 7/1/2009-6/30/2015. 

Associate Professor with Tenure, 7/1/2015-current 

Honours 

1. Member of the Royal Society of Canada’s College of New Scholars, Artists and Scientists (2016) 

2. Visiting Professor, Department of Speech, Language, and Hearing Sciences, University of Minnesota – 

Twin Cities. September 1, 2015 – May 31, 2016. 

Other Experience and Professional Memberships 

Member of: Acoustical Society of America, Linguistic Society of America, Association of Laboratory 

Phonology 

2012-2015 Guest Associate Editor for Frontiers in Psychology – Cognitive Science, March 2012-2015. 

2018-present Member of the Executive Committee for the Association of Laboratory Phonology 

C. Contribution to Science 

Biosketches 

 Page 41 
1. Spontaneous Phonetic Accommodation 

Spontaneous phonetic imitation or accommodation is the process by which a speaker unintentionally adopts 

acoustic characteristics of the speech patterns of a voice they are exposed to. My research in this area has 

had a large impact on the field. I initiated work in this area at a pivotal time, at the onset of a burst in work the 

area. In addition to providing novel empirical insights, my work provides a unique perspective on this work by 

integrating phonetic, sociolinguistic, and psycholinguistic perspectives. Babel (2012) reports on the selectivity 

of the phonetic accommodation process. I have extended my research on accommodation across dialects in 

Babel (2010) and Babel, McAuliffe, & Haber (2013), which demonstrated how structure of a language’s vowel 

system and listener attitude affected the degree to which speech perception and production processes 

interact. We have continued to explore the mechanisms of phonetic accommodation through the role 

of surprisal in spontaneous imitation by testing the role of voice novelty (Babel, McGuire, Walters, & Nichols, 

2014) and by through manipulations of cognitive load (Abel & Babel, 2016). 

Abel, J., & Babel, M. (2016). Cognitive Load Reduces Perceived Linguistic Convergence Between 

Dyads. Language and Speech, 0023830916665652. 

Babel, M., McGuire, G., Walters, S., & Nichols, A. (2014). Novelty and social preference in phonetic 

accommodation. Journal of Laboratory Phonology, 5, 1, 123-150. 

Babel, M., McAuliffe, M. & Haber, G. (2013). Can mergers-in-progress be unmerged in speech 

accommodation? Frontiers in Psychology, 4, 653, 1-14. 

Babel, M. (2012). Evidence for phonetic and social selectivity in spontaneous phonetic imitation. Journal of 

Phonetics 40, 177-189. 

Babel, M., & Bulatov, D. (2012). The role of fundamental frequency in phonetic accommodation. Language and 

Speech, 55(2), 231-248. 

Babel, M. (2010). Dialect convergence and divergence in New Zealand English. Language in Society, 4(3), 

437-456. 

2. Perception of Unfamiliar, Atypical, and Stereotyped Speech 

How perceptual processes remain flexible (allowing us to understand a wide range of speakers and accents) 

yet stable (not showing dramatic changes with exposure to each unique voice) is a question that has dogged 

the discipline for years. In this line of work, I examine how listeners attend and adapt to unfamiliar accents or 

pronunciation patterns or speakers who have unanticipated pronunciation patterns. Using a range of 

behavioural paradigms that range from transcription tasks to categorization paradigms, we show that the 

nature and position of the phonetic variation affects how listeners attend to the incoming speech stream. These 

results have implications for theories of phonetic attention and provide practical insights for improving spoken 

language communication in diverse real-world settings. 

Babel, M., McAuliffe, M., Norton, C., Senior, B., & Vaughn, C. (in press). The Goldilocks Zone of Perceptual 

Learning. Phonetica. 

Senior, B. & Babel, M. (2018). The role of unfamiliar accents in competing speech. Journal of the Acoustical 

Society of America, 143 (2), 931-942. 

McAuliffe, M., & Babel, M. (2016). Stimulus-directed attention attenuates lexically-guided perceptual 

learning. The Journal of the Acoustical Society of America, 140(3), 1727-1738. 

Babel, M. & Russell, J. (2015). Expectations and Intelligibilty. Journal of Acoustical Society of America, 137(5), 

2823-2833 

3. Biases and Stereotypes in Speech Perception and Spoken Language Processing 

An ongoing debate in spoken language processing is how social information affects spoken language 

processing. For example, a hot topic is whether a loss of intelligibility can stem from a lack of listener effort. 

Babel and Russell (2015) showed that Asian Canadians’ speech was less intelligible that the speech of White 

Canadians only when listeners knew they were listening to the speech of Asian Canadians. Using standard 

instruments to measure social and racial attitudes, we argued that these results appear to be the results of 

erroneous linguistic predictions and not a lack of listener effort. Associations between speaker populations and 

linguistic forms inhibited accurate identification of the stimuli. Similarly, Szakay, Babel, and King (2016) show 

Contact PD/PI: Munson, Benjamin R 

Biosketches 

 Page 42 
that for Maori-English bilinguals in New Zealand, listeners associations between languages facilitate crosslinguistic processing. 

Senior, B., Hui, J. & Babel, M. (2018). Liu vs. Luke? Name influence on voice recall. Applied 

Psycholinguistics, 1-30. doi:10.1017/S0142716418000267. 

Wong, P., & Babel, M. (2017). Perceptual identification of talker ethnicity in Vancouver English. Journal of 

Sociolinguistics, 21(5), 603-628. 

 

 

Szakay, A., Babel, M., & King, J. (2016). Social categories are shared across bilinguals' lexicons. Journal of

Phonetics, 59, 92-109. 

Babel, M. & McGuire, G. (2015). Perceptual fluency and judgments of vocal aesthetics and stereotypicality. 

Cognitive Science, 39, (4), 766-787. 

Babel, M., McGuire, G., & King, J. (2014). Towards a More Nuanced View of Vocal Attractiveness. PLOS ONE, 

9, 2, 1-10. 

4. The interaction of speech perception and phonology.

Our language teaches us how to listen. My work in this area has demonstrated how experiences with language 

shape our perceptual space (Johnson & Babel, 2010). We have also shown, however, that listener sensitivity 

to psychoacoustic contrasts can be increased by “tricking” participants into exploiting less language specific 

processing strategies (Babel & Johnson, 2010). For challenging auditory contrasts that are low in perceptual 

salience, my colleagues and I have found that listeners exploit auditory general contrast effects to predict 

phoneme categories (Babel & McGuire, 2013). Additionally, we have shown that ambiguity in visual speech 

information feeds cross-linguistic patterns of sound change (McGuire & Babel, 2012). This line of research has 

improved our understanding of how cross-linguistic differences in sound structure affect communication break 

downs in adult populations. 

Babel, M. & McGuire, G. (2013). Listener expectations and gender bias in nonsibilant fricative perception. 

Phonetica, 70, 117-151. 

McGuire, G. & Babel, M. (2012) A cross-modal account for synchronic and diachronic patterns of /f/ and /θ/ in 

English. Journal of Laboratory Phonology, 3, 251-272. 

Babel, M. & Johnson, K. (2010). Accessing psycho-acoustic perception with speech sounds. Laboratory 

Phonology, 1 (1), 179-205. 

Johnson, K. & Babel, M. (2010). On the perceptual basis of distinctive features: Evidence from the perception 

of fricatives by Dutch and English speakers. Journal of Phonetics, 38 (1), 127-136. 

A full set of publications can be found at https://scholar.google.ca/citations?user=NTm7MhAAAAAJ&hl=en

D. Research Support

Ongoing:

Social Sciences and Humanities Research Council Insight Development Grant, Babel (PI), 6/1/2018-

5/31/2020 

Documenting Urban and Regional Accents in BC 

The languages we speak and the accents we speak with provide insight into our identities and histories. In this 

project we are creating an accent map of British Columbia (BC) to describe the regional varieties of English 

that comprise the linguistic landscape of the province. Using computational techniques in linguists’ and

musicians’ toolkits, we have developed an online system to allow for the recording of urban and rural BC 

English speakers from their own homes. With a custom-written story as our elicitation material, we will create 

an accessible repository for researchers and communities. The proposed work will permit an understanding of 

how social and regional identities intersect with language, which is important in a globalized society. 

Natural Sciences and Engineering Research Council Discovery Grant, Babel (PI), 4/1/2018-3/31/2023 

Biosketches

 Page 43 
Competition and evaluation in perceptual learning of speech 

The focus of this research program is to test the weighting of signal-based and knowledge-based information in 

spoken language recognition by exploring the limits and constraints of perceptual learning in non-clinical adult 

listeners. With this population, we study lexically-guided perceptual learning: that is, how known linguistic 

information (in the form of known words) facilitates the recognition of a deviant or ambiguous acoustic-phonetic 

signal (e.g., hearing a known word with a controlled mispronunciation). This particular type of perceptual 

learning is critical as it is assumed to be exploited in daily communication when we interact with speakers or 

machines whose pronunciation deviates from the expected. 

Social Sciences and Humanities Research Council Insight Grant, Babel (PI), 4/1/2017-3/31/2022 

Socially Selective Phonetic Memory 

This project examines the role of social group membership on how we perceive and process spoken language. 

For example, language users acquire the language varieties spoken around them; infants who grow up in a 

French-speaking environment learn to speak French, not German. We tend to mirror patterns of our linguistic 

input in our own linguistic output. In this SSHRC award we explore whether this is always the case. There is 

evidence in language acquisition that children start to craft their speech in gender-specific ways before the 

onset of puberty-induced anatomical differences. That is, children start to sound like little boys and little girls 

before it is an anatomically inevitability. What is the mechanism underlying the use of these selective language 

patterns? This award supports studies that explore the hypothesis that selective acquisition of speech patterns 

is driven by a socially selective attentional mechanism determined by group membership in the form 

of ingroup biases (e.g., preferences for those who are members of the same social group as oneself). 

Completed: 

Social Sciences and Humanities Research Council, Babel (PI) 4/1/11-03/31/2012 

Phonetic Imitation: Units and Mechanisms 

The results of the research funded by this SSHRC award examined phonetic imitation. Phonetic imitation is the 

unintentional and subconscious process of adopting the speech patterns of others. The research probed both 

the mechanisms and units behind phonetic imitation. The mechanisms of phonetic imitation were examined by 

testing two theories: one which predicts that cognitively novel or atypical voices would be subject to increased 

imitation, and another which predicts imitation is driven largely by social cognition such that one's social 

preferences would drive imitative behavior. To test these theories we first conducted a series of experiments 

asking independent groups of listeners to assess typicality and attractiveness of a corpus of voices. Based on 

these measures we selected voices which were most and least typical (less and more cognitively novel voices, 

respectively) and most and least attractive (more and less socially preferred, respectively) to our listener 

population for use in the phonetic imitation experiments. The results of our phonetic imitation tasks provided 

support for the theory that phonetic imitation is supported by cognitive novelty, as participants imitated the 

atypical voices the most. Female participants, however, imitated the attractive voices more than male 

participants, thus providing some support to the theory that phonetic imitation is facilitated by social 

preferences. In publications stemming from this SSHRC-funded research, we connect these findings to 

reported in differences in exemplar memory for females and males, along with females' purported stronger 

feeling of group membership. These results of this research are impacting theories in a diverse range of 

disciplines, including linguistics and the cognitive sciences, more broadly. More specifically, this SSHRCfunded research is shaping theories of speech perception and production, new dialect acquisition, and 

language change. 

Biosketches 

 Page 44 
PHS 398 Cover Page Supplement OMB Number: 0925-0001

Expiration Date: 03/31/2020

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

1. Vertebrate Animals Section

Are vertebrate animals euthanized? ❍ Yes ● No

If "Yes" to euthanasia

Is the method consistent with American Veterinary Medical Association (AVMA) guidelines?

❍ Yes ❍ No

If "No" to AVMA guidelines, describe method and provide scientific justification

2. *Program Income Section

*Is program income anticipated during the periods for which the grant support is requested?

❍ Yes ● No

If you checked "yes" above (indicating that program income is anticipated), then use the format below to reflect the amount and

source(s). Otherwise, leave this section blank.

*Budget Period *Anticipated Amount ($) *Source(s)

 Page 45 
PHS 398 Cover Page Supplement

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 . Received Date: 

2018-10-15T15:01:25.000-04:00 

3. Human Embryonic Stem Cells Section

*Does the proposed project involve human embryonic stem cells? ❍ Yes ● No

If the proposed project involves human embryonic stem cells, list below the registration number of the specific cell line(s) from the

following list: http://grants.nih.gov/stem_cells/registry/current.htm. Or, if a specific stem cell line cannot be referenced at this time,

check the box indicating that one from the registry will be used:

❏ Specific stem cell line cannot be referenced at this time. One from the registry will be used.

Cell Line(s) (Example: 0004):

4. Inventions and Patents Section (Renewal applications)

*Inventions and Patents: ❍ Yes ● No

If the answer is "Yes" then please answer the following:

*Previously Reported: ❍ Yes ❍ No

5. Change of Investigator/Change of Institution Section

❏ Change of Project Director/Principal Investigator

Name of former Project Director/Principal Investigator

Prefix:

*First Name:

Middle Name:

*Last Name:

Suffix:

❏ Change of Grantee Institution

*Name of former institution:

Contact PD/PI: Munson, Benjamin R 

 Page 46 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489. Received Date: 2018-10-15T15:01:25.000-04:00 

OMB Number: 0925-0001

Expiration Date: 03/31/2020

PHS 398 Modular Budget

Budget Period: 1

Start Date: 06/01/2019 End Date: 05/31/2020

A. Direct Costs Funds Requested ($)

Direct Cost less Consortium Indirect (F&A)*

B. Indirect (F&A) Costs

Indirect (F&A) Type Indirect (F&A) Rate (%) Indirect (F&A) Base ($) Funds Requested ($)

1.

2.

3.

4.

Cognizant Agency

(Agency Name, POC Name and Phone Number)

DHHS: Arif Karim, (214) 767-3600

Indirect (F&A) Rate Agreement Date 05/11/2018 Total Indirect (F&A) Costs

C. Total Direct and Indirect (F&A) Costs (A + B) Funds Requested ($)

Contact PD/PI: Munson, Benjamin R 

 Page 47 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489. Received Date: 2018-10-15T15:01:25.000-04:00 

PHS 398 Modular Budget

Budget Period: 2

Start Date: 06/01/2020 End Date: 05/31/2021

A. Direct Costs Funds Requested ($)

Direct Cost less Consortium Indirect (F&A)*

B. Indirect (F&A) Costs

Indirect (F&A) Type Indirect (F&A) Rate (%) Indirect (F&A) Base ($) Funds Requested ($)

1

2.

3.

4.

Cognizant Agency

(Agency Name, POC Name and Phone Number)

DHHS: Arif Karim, (214) 767-3600

Indirect (F&A) Rate Agreement Date 05/11/2018 Total Indirect (F&A) Costs

C. Total Direct and Indirect (F&A) Costs (A + B) Funds Requested ($)

Contact PD/PI: Munson, Benjamin R 

 Page 48 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489. Received Date: 2018-10-15T15:01:25.000-04:00 

PHS 398 Modular Budget

Cumulative Budget Information

1. Total Costs, Entire Project Period

Section A, Total Direct Cost less Consortium Indirect (F&A) for Entire Project Period ($)

2. Budget Justifications

Personnel Justification _Personnel_Justification.pdf

Consortium Justification

Additional Narrative Justification

Contact PD/PI: Munson, Benjamin R 

 Page 49 
Budget Justification 

Senior/Key Personnel 

Benjamin Munson, Ph.D. (Principal Investigator): PI Dr. Munson will devote 12.5% academic year effort (1.125 

calendar months) in both years of the project. Munson will coordinate all phases of the project. Munson's 

position in the Department of Speech-Language-Hearing Sciences gives him protected research time to devote 

to this study, and the salary funds in this grant will provide him with an additional one-course teaching release. 

Dr. Munson will supervise the postdoctoral fellow for this project. He will also be responsible for writing and 

managing the IRB protocol for this project. 

Peggy Nelson, Ph.D. (co-Investigator). Co-I Dr. Nelson will devote 5% academic year effort (0.45 calendar 

months) in both years of this project. Nelson will assist in the recruitment and audiometric testing of 

individuals with hearing impairment for specific aim 2 of this study. Nelson will also assist in the development 

of the final protocol for determining the presentation of the audio stimuli (i.e., determining a signal-to-noise 

ratio that is appropriate for the entire range of individuals in the study, and verifying the implementation of the 

HAL protocol for providing uniform amplification to individuals with hearing impairment). Nelson (and co-I 

Oxenham) will meet with PI Munson monthly during all phases of this project, and will coauthor selected 

presentations and publications from this study. 

Andrew Oxenham, Ph.D. (co-Investigator). Co-I Dr. Oxenham will devote 5% academic year effort (0.45 

calendar months) to this project. Oxenham will assist in the recruitment of older adults with and without 

hearing impairment for specific aim 2 of this study. He will assist in the preparation of the audio stimuli. 

Oxenham will be centrally involved in the analysis of the speech perception data from older adults for specific 

aim 2 of this study, and in particular in the comparison of intelligibility performance between the novel 

sentences developed in this study, and the standard-of-care sentences that will be recorded. Oxenham (and coI Nelson) will meet with PI Munson monthly during all phases of this project, and will coauthor selected 

presentations and publications from this study. 

Evelyn Davies-Venn, PhD (Consultant): Consultant Dr. Davies-Venn will devote 2% academic year effort (0.18 

calendar months) in both years of this project. Davies-Venn has considerable expertise on health and wellness 

outcomes in individuals with hearing impairment, and on modeling perception by individuals with hearing 

impairment. She will provide assist Munson, Nelson, and Oxenham on recruitment of older adults with hearing 

impairment, and on the interpretation of the individual-differences measures of the older adults with and 

without hearing impairment. Davies-Venn will meet with PI Munson and co-Is Nelson and Oxenham in a full 

project team meeting quarterly. 

Jeffery Simpson, Ph.D. (Consultant). Consultant Dr. Simpson will devote 1% academic year effort (0.09 

calendar months) in both years of this project. Simpson has extensive expertise in social psychology. He will 

provide consultation on the selection and interpretation of the IAT measures for this study, and on the 

interpretation of information about listeners' social networks. Simpson will meet with PI Munson and co-Is 

Nelson and Oxenham in a full project team meeting quarterly. 

Matthew Winn, Ph.D. (Consultant). Consultant Dr. Winn will devote 2% academic year effort (0.18 calendar 

months) in both years of this project. Winn has considerable expertise on the acquisition and analysis of eyetracking data in individuals with hearing impairment and normal hearing. He will assist in the development of 

the eye-tracking components of the experiment protocol, and in analyzing the eye-tracking data. He will also 

provide guidance on the analysis of individual differences in the older adults with and without hearing 

impairment. Winn will meet with PI Munson and co-Is Nelson and Oxenham in a full project team meeting 

quarterly. 

Other Personnel 

Personnel Justification 

 Page 50 
TBA (Postdoctoral fellow): The TBA postdoctoral fellow (PF) will dedicate 100% of her/his time (12 personmonths) in both years of this project. The PF will be responsible for the day-to-day implementation of this 

project, including creation of the stimuli, running subjects, and analyzing data. The PF and PI Munson will 

meet weekly during both years of this project, and will participate in all monthly and quarterly team meetings. 

The PF will supervise the work of the undergraduate RA on this project. The PF will be given the opportunity to 

take the lead in giving presentations and writing manuscripts summarizing selected topics from this project. 

The The PF position in this project can hold a PhD in a number of different academic fields, including 

audiology, speech and hearing science, psychology, and linguistics. The PF will also work with PI Munson, coIs Nelson and Oxenham, and the collaborators to develop skills outside of her/his primary expertise (i.e., 

interpreting audiometric results if the PF does not have training in audiology, administering and scoring IATs 

if the PF does not have training in social psychology) 

TBA (Undergraduate Research Assistant): The TBA undergraduate research assistant (URA) will dedicate 10 

hours a week (10 months) in both years of this project. The URA will work under the direction of the PF. 

She/he will be responsible for assisting in stimulus preparation, subject testing, and data archiving. The URA 

will meet with the PF weekly, and will attend the quarterly full project team meeting. 

Consultant 

Molly Babel, Ph.D. (Consultant). Consultant Dr. Babel will dedicate 100 hours of effort in each year at $50/hr 

(in Year 2, an additional $5000 is also requested for consultant travel costs). Babel has extensive expertise on 

audiovisual speech perception and on social influences on phoneme perception and speech intelligibility. She 

will provide guidance and feedback on the video stimuli, and on developing the parts of the protocol that relate 

specifically to AV perception. Babel will also serve as a co-author on selected presentation and publications 

from this study, particularly those related to individual differences among the younger, normal-hearing adults. 

Personnel Justification 

 Page 51 
PHS 398 Research Plan OMB Number: 0925-0001

Expiration Date: 03/31/2020

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489. Received Date: 

2018-10-15T15:01:25.000-04:00 

Introduction

1. Introduction to Application

(for Resubmission and Revision applications)

Research Plan Section

2. Specific Aims AVGrant_R21_SpecificAims_Final.pdf

3. Research Strategy* AVGrant_R21_ResearchPlan_Final.pdf

4. Progress Report Publication List

Other Research Plan Section

5. Vertebrate Animals

6. Select Agent Research

7. Multiple PD/PI Leadership Plan

8. Consortium/Contractual Arrangements

9. Letters of Support AVGrant_R21_Support_Letter_Babel.pdf

10. Resource Sharing Plan(s) AVGrant_R21_DataSharingPlan_Final.pdf

11. Authentication of Key Biological and/or

Chemical Resources

Appendix

12. Appendix

 Page 52 
A. Specific Aims 

Hearing impairment (HI) affects a large percentage of older adults: 8.5% of 55-64 year olds, 25% of 65-74 

year olds, and 50% of people 75 and older have a HI that is sufficiently severe to be disabling [36,42]. HI is 

associated with negative social and health outcomes [35,53]. One of the most significant consequences of HI is 

a reduced ability to perceive speech and, consequently, a reduction in social interaction. This decrease in social 

interaction may be the cause of many of the broader impacts that HI has on health and well-being. Improving 

speech perception by individuals with HI is of great public health relevance. Because of demographic shifts in 

the US, interactions between older adults and the younger adults (including those in caregiving roles) are 

increasingly likely to be between individuals with different racial and ethnic backgrounds: older adults in the 

US are more likely than younger adults to be white and non-Latinx [44,54]. These population dynamics are 

likely to provide an additional challenge to speech perception by individuals with HI beyond those posed by the 

sensory deficit itself, as recent research has found evidence that speech intelligibility can decrease when 

a listener becomes aware of a talker's race and ethnicity by seeing a picture of a talker, especially 

when the talker is a person of color (i.e., Latinx, non-white, or both). That is, some intelligibility decrements 

are due to social factors alone, and are not due to the acoustic and linguistic characteristics of the speech being 

perceived, or the listener's auditory acuity [6,37,48]. Given demographic shifts, effects of speaker race and 

ethnicity on speech intelligibility are likely to be of great significance to the population of older 

adults with HI. Understanding when and why these race and ethnicity effects on intelligibility (REI) occur is 

a critical first step to mitigating or removing them. 

However, there are two barriers to studying REI that must be removed before large-scale studies on this 

topic can be conducted. The first barrier concerns the fact that the speech stimuli that are used in clinical 

assessments and in research on individuals with HI are inadequate for studying REI. These are generally 

audio-only (A-only) materials. The visual component of audio-visual (AV) signals is a rich source of social 

information about talkers, including their race and ethnicity—features that are not conveyed robustly through 

A-only signals. The stimuli needed to study REI are by necessity AV, and existing corpora of AV speech stimuli 

[16] do not have the racial and ethnic diversity needed to study REI. The second barrier is that there are no 

data on REI in individuals with HI. It is possible that listeners with HI show similar or larger REI than 

individuals with normal hearing [NH], if they ignore or avoid critical visual speech cues—ones that are needed 

to compensate for the cues that their auditory impairment renders imperceptible—when presented AV speech 

by people of color. This hypothesis is suggested by work on face perception across race and ethnicity [ , ]. 

An alternative hypothesis stems from the finding that decrements in hearing ability prompt individuals with HI 

to attend to visual speech cues even more robustly than NH individuals [ , ]. If this increased visual 

attention occurs regardless of talkers' race and ethnicity, then individuals with HI should show 

14 59

14 58

smaller REI 

than NH individuals, because the NH individuals would ignore critical visual speech cues produced by racially 

and ethnically diverse talkers, while HI individuals would attend to these cues. More generally, the lack of 

large-scale experiments on REI means that we know relatively little about the perceptual and cognitive 

mechanisms that underlie REI across the lifespan and across levels of hearing acuity. This R21 grant will 

exploit our research team's expertise on corpus development and annotation, hearing impairment, auditory 

perception, social psychology, and sociolinguistics to move past the barriers identified above and set the stage 

for larger-scale work on this vitally important topic. 

Specific Aim 1: To build a large corpus of AV speech stimuli produced by racially and ethnically 

diverse individuals. The speech materials will include standard-of-care sentences commonly used in clinical 

research (IEEE Sentences, [ ]), as well as a new set of sentences designed to elicit socially meaningful 

phonetic variation. These will be produced by a set of men and women who are diverse in race and ethnicity. 

We will share this corpus in a digital archive, along with de-identified data on the intelligibility of these stimuli 

(collected for specific aim 2). These speech stimuli will be more appropriate to assess the abilities of the 

increasingly diverse population of individuals in the US with HI than a corpus of decontextualized speech 

produced by a small, homogenous set of talkers. 

26

Specific Aim 2: To determine the extent and nature of effects of talker race and ethnicity on the 

intelligibility of speech presented to NH and HI listeners. This will be achieved by using the stimuli 

developed in specific aim 1 (both the standard-of-care sentences and the newly developed ones) in a series of 

AV and A-only intelligibility experiments with both behavioral responses and eye tracking. Younger NH, older 

NH, and older HI participants will serve as listeners. These data, along with a set of individual-differences 

measures, will be used to test different hypotheses about the locus of REI, and to determine whether REI is 

larger or smaller in HI listeners than in NH listeners. 

Contact PD/PI: Munson, Benjamin R 

Specific Aims 

 Page 53 
B. Significance 

B.1. Social influences on speech perception 

Problems understanding speech are a hallmark of hearing impairment (HI), a communication impairment that 

affects numerous Americans, including a strikingly high percentage of older adults [36,42]. Understanding the 

nature of speech perception challenges by adults with HI is a pressing public health challenge, as untreated HI 

has far-reaching and potentially devastating effects on health and well-being [35,53]. The focus of this grant is 

on an aspect of speech perception that has traditionally been ignored in speech perception research and which 

is of great importance to society: the influence of talkers' race and ethnicity on their speech intelligibility to 

individuals with and without HI. 

Speech communication is a quintessentially social act. The premise underlying this application is that 

standard-of-care measurements of speech perception, such as those for individuals with HI, are suboptimal 

because they ignore social factors. For example, assessment of speech perception by individuals with HI 

generally use single words or contrived sentences (Bring your problems to the wise chief, Ducks fly north but 

lack a compass) [ . Moreover, the talkers and speech styles of these materials are generally not varied 

systematically. They tend to be produced in a careful speech style by a small number of talkers whose 

demographic characteristic are unspecified. The decontextualized nature of speech perception assessments is 

greatly at odds with the inherently social nature of speech communication, and with the growing body of 

literature showing that speech perception is sensitive to 

26]

social information [ ]. Social information is 

expansive [ ]; in this application we use this term more narrowly to refer to attributes about the people whose 

speech is being perceived, such as their gender identity, their social class, and, most especially, their race and 

ethnicity. Social information and speech perception interact in a bidirectional manner. Listeners use patterns 

of phonetic variation to infer attributes about talkers like age, gender, race, ethnicity, social class, emotion, and 

attitudes toward topics being discussed [ ]. In turn, listeners' assumptions about the social characteristics of 

talkers affects lower-level aspects of speech perception. For example, the labeling of an ambiguous phoneme 

can differ when different attributes about the talker who produced it are suggested to a listener. A fricative that 

is ambiguous between /s/ and /

8

5

18

ʃ/ is more likely to be labeled as [s] when listeners believe it was produced by a 

man, and [ʃ] when thought to be produced by a woman [ . This is true even when the speaker sex is cued very 

subtly, for example, by combining the ambiguous fricative with a vowel whose f0, voice quality, and formant 

frequencies are gender-neutral, and then pairing this doubly ambiguous stimulus with a picture or video of a 

man or a woman's face [ , . These effects are particularly interesting because they cannot be attributed to 

peripheral auditory processes: the very same acoustic signal is perceived differently when the talker's gender is 

suggested by a visual prime, albeit in ways that are consistent with actual population-level differences between 

men and women’s voices. Similar visual-priming effects on phoneme perception have been found for a variety 

of vowel and consonant contrasts and a variety of social categories other than gender (i.e., age [ ], race 

[ , ], geographic origin ). For example, recently showed that listeners label children's productions 

of /ld/-final words with a completely or partially elided /d/ as more-accurate when paired with AfricanAmerican child's face than when paired with a white child's face. This presumably reflects listeners' knowledge 

that some African American children speak African American Vernacular English (AAVE), a variety of English 

in which the normative production of a word like 

20 25 [43] [20] 

19

39 41]

52]

cold is [khoʊl] rather than [khoʊld]. Social factors can affect 

performance on a variety of tasks beyond perception. For example, [ showed that listeners imitate less 

phonetic detail in speech when spoken stimuli are paired with African-American people’s faces (suggesting that 

the speech was produced by them) than when paired with white people’s faces; this tendency was strongest for 

individuals whose performance on an implicit association test (IAT) suggested implicit biases against African 

Americans. 

4]

Social effects on speech perception extend more broadly to language comprehension, as measured by 

sentence intelligibility tasks. showed that Mandarin-accented English was more intelligible when 

presented with an ethnically Chinese person’s face than when paired with a white person’s face. The magnitude 

of the differences between these conditions and a third condition using a race-neutral silhouette was mitigated 

by listeners’ experience interacting with Mandarin speakers. Again, these effects are not driven by aspects of 

the signal: the intelligibility of the same speech varied depending on which visual stimulus it was paired with. 

Research by suggests that race effects might be grounded in audiovisual integration: audiovisual 

integration was less effective for ethnically Korean faces paired with Korean-accented English than for white 

faces paired with unaccented English. found that speaker attitudes about talkers mitigated social effects on 

intelligibility: measures of implicit and explicit attitudes toward accented talkers uniquely predicted their 

speech intelligibility, beyond other signal- and listener-rated factors (i.e., the language variety used in the 

[27]

 [62] 

[37] 

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 54 
stimuli, the listeners’ executive function and age). Social effects on intelligibility extend beyond accented 

speech. In a seminal study, presented connected speech (a university lecture) produced by a native 

Midwestern American speaker (i.e., someone without a foreign accent), paired with either a white person's face 

or an ethnically East Asian person's face. Undergraduate students remembered less information from the 

lecture, and judged the unaccented speaker to be more highly accented when the audio sample was paired with 

the East Asian person's face than when paired with a white person's face. showed that the intelligibility of 

sentences produced by 12 native speakers of Canadian English (6 of whom were white, 6 of whom were 

ethnically Chinese) differed depending on whether the sentences were presented along with pictures of the 

speakers' faces. For the ethnically Chinese speakers, intelligibility was lower when these sentences were 

presented with their faces as compared to a condition where no face was presented. For white speakers, there 

was no intelligibility cost associated with revealing the speaker's face. 

 [6]

[48] 

In the remainder of this application, we refer to the findings in the previous paragraph as race and ethnicity 

effects on intelligibility [REI]. We use both the terms race and ethnicity so not to conflate differences between 

the two major ethnic groups in the US (Latinx, non-Latinx) and the major racial categories in the US (AfricanAmerican, East Asian, South Asian, Native American, white, etc.), as is consistent with NIH reporting 

requirements. The existence of REI has strong, pressing implications for our understanding of the challenges 

faced by individuals for whom speech perception is most difficult: people with hearing impairment (HI), as 

described in detail in section B.2, below. Prior to studying REI in individuals with HI, a number of key barriers 

in this area of study must be removed. We must better understand the specific mechanism(s) that drive(s) REI, 

and conduct rigorous and reproducible experiments that evaluate them directly. There are at least three 

hypotheses for why REI occur, none of which is mutually exclusive from the other two. One hypothesis is that 

images of speakers elicit expectations about the language variety that the speaker uses. For example, seeing an 

ethnically East Asian person's face might lead someone to expect to hear an accented variety of English, despite 

the fact that not all ethnically East Asian people speak with accents. Seeing an African American person's face 

might prompt people to expect to hear a non-mainstream variety of English like AAVE. This expectationdriven hypothesis is invoked specifically by , 62], but has not been tested directly. Furthermore, this 

hypothesis is surprisingly very challenging to test rigorously. It predicts that qualitatively different perception 

errors should occur when perceiving speakers of different ethnicities. However, the nature and extent of those 

errors is likely to be mitigated by the amount of experience that listeners have interacting with people of 

different races and ethnicities. For people with less experience interacting with diverse populations, 

expectations are likely not to be a viable source of information. A second hypothesis is that REI occur because 

people have negative attitudes toward the individuals whose speech they are perceiving, and that these negative 

attitudes make processing more effortful, and consequently less accurate. This hypothesis is supported by the 

results of [27], and by other work using related paradigms, like [4]. This 

[19 20,

attitude-driven hypothesis calls 

for larger-scale studies of the relationship between attitudes toward different groups and the intelligibility of 

those people's speech in A-only and AV conditions. A third hypothesis relates to the nature of AV perception of 

different speakers: talkers whose race and ethnicity are different from that of the listener might elicit different 

patterns of visual speech perception from same-race, same-ethnicity talkers. It is well documented that 

individuals perceive the faces of people who do not share their race and ethnicity much more coarsely than 

those of people who share their race and ethnicity , . This may lead to differences in visual attention 

during AV speech perception: an individual who is listening to and viewing a racially or ethnically different 

speaker might not attend to phonetically relevant visual signals, especially mouth movement, as when viewing 

a person whose race and ethnicity are the same as theirs. This is explanation was invoked by but has 

never been tested systematically by looking at eye gaze during perception. This explanation is unlikely to 

explain the results of studies using static visual images, but may explain why it could be hard to 

[1,62],

 [14 58]

learn the 

phonetic characteristics of ethnically and racially diverse talkers. This attention-driven hypothesis 

suggests that studies of REI should include measures of visual search from eye-tracking during speech 

perception, to examine whether attention to phonetically relevant visual signals differs across races and 

ethnicities. These three mechanisms might interact. For example, the processing challenges associated with 

negative attitudes might reduce the efficiency of visual search, as was found by [1]. 

B.2. Sources of variation in speech perception by people with hearing impairment 

One of the most commonly occurring communication disorders is hearing impairment (HI). When compared 

to normal hearing (NH) individuals of the same age, individuals with HI interact socially less, have less 

participation in the workforce, and are at a greater risk for other health problems like dementia 

[33,35,36,53,55]. Maximizing the communication of people with HI is of enormous relevance to public health, 

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 55 
as better communication can mitigate or alleviate the negative social, cognitive, and vocational consequences of 

HI. The REI described in section B.1 are of particular importance to individuals with HI. Changing 

demographics in the US mean that there are likely to be racial mismatches between older Americans and 

younger ones: each successive generation of Americans contains fewer non-Latinx white people, and more 

racial and ethnic diversity overall, than the one prior to it. Moreover, the workforce of individuals providing 

day-to-day caregiving and health care support for older adults (including home health care assistants, certified 

nurses' assistants, audiologists, speech-language pathologists, etc.) is younger than the people they care for, 

and hence is more likely to comprise people of color (i.e., people who are Latinx, non-white, or both) [ . 

In short, older adults with HI are more likely than the general US population not to be people of color, and the 

individuals that they interact with in critical roles are more likely than the general US population to be people 

of color. These demographics sets the stage for many real-world interactions that mirror those in 

27, , , . This motivates the current project, the ultimate goal of which to build a large set of carefully 

constructed speech stimuli to study REI in NH and HI adults, to use these stimuli in a series of experiments 

designed to measure the extent of REI in A-only and AV tasks by three groups (older adults with HI, older 

adults with NH, and a large, diverse cohort of younger adults with NH), and to use the results of these studies 

to determine the mechanism underlying REI more generally. 

[6, 37 48 62]

44,54]

Very little is known about how social factors like those described in section B.1 affect speech perception by 

adults with HI. One exception is [ ], who found that phoneme identification by listeners with CIs was more 

strongly affected than NH listeners by pictures that revealed talker gender, using a task like that in [ ,

Examining REI in listeners with HI accomplishes a number of important goals. The first speaks to the external 

validity of standard-of-care speech perception assessments. If REI is larger in HI listeners than in NH listeners, 

then we can conclude that the assessment of speech intelligibility using a small set of talkers likely underpredicts the speech perception success of individuals with HI in real-world interactions across races and 

ethnicities. Studying REI in older adults also helps us better understand the mechanisms that drive REI. As a 

group, older adults in the US are likely to have less interaction with racially and ethnically diverse peers than 

younger adults, at least among their same-age peers. Consistent with this, research has shown them to have 

more negative views of individuals in other races and ethnicities, at least in the cases of white adults' 

evaluations of nonwhite individuals The increase in the range of implicit attitudes toward different races 

and ethnicities will allow us to make a more robust test of the attitude-driven hypothesis. The changes in visual 

attention during speech perception that occur with age and hearing impairment allow us to make a morerobust test of the attention-driven hypothesis. In short, including older adults with HI in this project has both 

practical and theoretical motivations. 

 [15] 

[24]. 

39 41,52].

59

B.3. Barrier: The lack of a resource to study REI. 

There is at least one substantial but surmountable barrier to conducting large-scale studies of REI: the lack of 

an audiovisual database of speech stimuli produced by individuals of diverse races and ethnicities. The first aim 

of this grant is to build such a database. In specific aim 2 of this project, this new database will be used in a 

series of studies that test the hypotheses outlined in section B.1 about the locus of REI, and those in B.2 about 

the magnitude and locus of REI differences between older adults with and without HI. While specific aim 2 is a 

solidly exploratory aim, it is important to emphasize that specific aim 1 will have a lasting and sustained impact 

on science regardless of the outcomes of specific aim 2. The database will be made available to the scientific 

community through a digital archive. This will allow researchers to use these stimuli in any future clinical or 

research studies. The ethnic and racial diversity in this corpus makes it a socially valid instrument for assessing 

speech perception in an increasingly more-diverse population of individuals with HI. This is particularly 

important in light of the lack of racial and ethnic diversity within the profession that measures speech 

perception in HI in the US, audiology, in which only 8% of professionals are non-white and 3% are Latinx [47]. 

C. Innovation 

The innovation in this project is at least threefold. First, the end result of this project will be a new corpus of 

audiovisual speech stimuli that is radically different from all existing corpora (e.g., . It will include both 

standard-of-care sentences, and sentences that are designed specifically to elicit more natural speech styles. 

These will be produced by a group of talkers that is ethnically and racially diverse. Second, the results of this 

project—and the results of the work that inspired it 48]—suggest that the fields of speech perception and 

audiology should reconsider the long-standing assumption that AV speech perception is always better than Aonly speech perception (e.g., [ , , . That hypothesis is based on studies of A-only and AV speech 

perception using stimuli produced by a homogeneous set of talkers. Research with more-diverse speaker 

populations may show that AV perception can be significantly poorer than A-only perception in cases of racial 

10 17 21])

[6,37,

[16])

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 56 
and ethnic mismatch between talker and listener. Finally, this project will evaluate a potentially important 

source of variation in speech perception that has hitherto not been considered in studies of HI. 

D. Approach 

D.1. Speech Materials 

The first specific aim is to develop a corpus of AV speech stimuli produced by a racially and ethnically diverse 

group of talkers. From each talker, we will record two types of sentences. One set will be a subset of the 

standard-of-care IEEE sentences [ . These will be recorded because they are the standard sentences used in 

much speech intelligibility research. We will also develop a new set of sentence materials to be recorded. The 

overall goal in developing this new list is to craft sentences that speakers might use in real-world 

communication, and which consequently would be more likely to elicit a more natural, real-world speech style. 

26]

To make the materials socially meaningful, we will mine repositories of internet blogs and select all 

declarative sentences that have 5-7 content words, as is true of the IEEE sentences. We will then eliminate the 

sentences that contain words that are not in the English Lexicon Project (ELP, [ ). This will remove proper 

names (includes places and people), slang, taboo words, and very low-frequency words. We will then randomly 

select 1000 sentences that a researcher deems to be (a) syntactically well-formed, (b) appropriate for use in an 

experiment (i.e., they contain no offensive content), and (c) free from first-person statements that would imply 

that the sentence represents the talker's views. A second individual will then review these 1000 sentences. In all 

instances where there is not perfect agreement on the neutrality and research-appropriateness of the sentence, 

it will be discarded. We anticipate the final set will include 750 sentences. 

7]

The phonemic balance of the lists will not be controlled, but the phonemic composition of the lists will be 

measured and reported, as will the frequency and potential confusability (estimated from the phonological 

neighborhood density) of the words comprising the sentences, taken from the ELP. We will be able to use these 

measures to examine post-hoc whether they account for sentence-level intelligibility differences. 

Each talker will produce 50 of the 750 sentences, including a set of 10 sentences that all speakers will 

produce, and 40 other sentences. Each talker will also produce 50 IEEE sentences, including 10 that all talkers 

will produce, and 40 other IEEE sentences. All of these sentences will be made available in the public archive, 

even if they are not ultimately used in specific aim 2. 

D.2. Talkers 

We will recruit and record a group of 50 18-40 year old talkers to produce the stimuli. Half of the talkers will be 

men and half will be women. We will recruit equal numbers of talkers from five different race/ethnicity 

categories: (1) non-Latinx white, (2) Latinx of any race [most of whom we anticipate to be white Latinx 

individuals], (3) African American/Black, (4) South Asian, and (5) East Asian. Our choice of these five 

categories is motivated by a combination of sensitivity to US and world demographics, and pragmatic reasons. 

The first three categories are the dominant race/ethnicity categories in the US, comprising approximately 84% 

of the population. While South Asian and East Asian individuals make up only 5% of the US population, they 

make up the nearly half of the world’s population. We anticipate none of our talkers will produce an accented 

variety of English. All of the talkers will have begun to acquire English in the US no later than age 7. This age is 

within the range that is broadly agreed to be the critical period for language acquisition, at which time 

languages can be acquired with little or no foreign accent. Talkers will be recruited from the University of 

Minnesota undergraduate and graduate-student populations, and will leverage the social networks of the 

diverse students that work in the PI and co-I's laboratories. 

D.3. Recording 

We will use a high-resolution, high frame-rate camera (GigE Prosilica 680 cameras with Kowa C Mount lenses) 

to video-record the talkers for specific aim 1. Sentences will be elicited in a reading task. The newly developed 

sentences will be read in the original contexts in which they occurred in blogs. We anticipate that the reading in 

context, particularly in the context of a blog, will elicit a more natural speech style than reading a single 

sentence in isolation. Talkers will be recorded from the neck upward using professional-grade video lighting, a 

consistent and neutral background, and at a consistent distance from the camera. Individuals' necks, faces, and 

hair will be visible in the recording. Individuals will be asked to pull their hair back from their faces so that the 

eyebrows can be seen. We will instruct individuals to dress and use makeup as they normally would, so that any 

semiotic markers of social-group membership and individual identity are present, as is true in real-life social 

communication. Headwear will be permitted only it is something the talkers habitually wear for cultural or 

religious reasons (e.g,. hijab, kippah, taqiyah, dastaar), as these items are generally worn habitually in public 

and are hence encountered in real-world social communication. Individuals who wear face-covering clothing 

(e.g., a niqab or burqa) will not be eligible to participate as talkers. During the video recording, a separate audio 

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 57 
recording will be made, using a professional-grade boom microphone and a digital recorder. Hence, the audio 

recordings for both the A-only and AV sentences will be very high quality. 

D.4. Listeners 

The listeners will be 120 individuals, divided into three groups, based on age and hearing status: younger (18-

40]) with NH (NYH, n=60), older (60-90) with HI (OHI, n=30), and older with NH (ONH, n=30). Equal 

numbers of men and women will be recruited within each group. The older adults with HI will all use hearing 

aids (HAs). The absence of a younger HI group is due to the lower incidence of hearing loss in younger people. 

The criteria for normal hearing in the YNH group will be audiometric thresholds of no more than 20 dB HL at 

octave frequencies between 250 and 8000 Hz. For the ONH group, the criteria will be relaxed to allow for 

thresholds up to 60 dB at frequencies of 4 and 8 kHz. However, the degree of high-frequency hearing loss will 

be recorded and used as a covariate in analyses, if needed. For the OHI group, inclusion criteria include 

sensorineural hearing loss with no conductive component, with pure-tone average audiometric thresholds 

between 500 and 2000 Hz between 40 and 65 dB HL, and no pure-tone thresholds exceeding 75 dB HL at 4 

and 8 kHz. All participants will be native speakers of English. 

This project endeavors to recruit a racially and ethnically diverse population of listeners. This will be 

achieved by using University of Minnesota community engagement mechanisms, as well as partnerships that 

PI Munson built during a previous grant that examined low-income children (grant R01 DC02932). Given 

demographic changes, we anticipate that the YNH group will be more diverse than the ONH and ONI groups. 

D.4. Listening Tasks 

Prior to conducting any intelligibility tasks, we will run a series of baseline tasks in which we will play A-only 

stimuli to listeners and ask them to guess the race and ethnicity of the talkers. These data will be used to 

estimate the extent to which race and ethnicity can be garnered from voices alone. It is unclear what we will 

find. Some previous work has found that certain racial and ethnic categories can be estimated from A-only 

signals [ but those studies used very different stimuli from those proposed in the current project. If we find 

that race and ethnicity can be estimated from A-only signals, we will use the proportion of accurate 

identification of race and ethnicity in the A-only condition as a covariate in all of our analyses of the effect of 

race on intelligibility. 

45],

In the main experiment, listeners will complete two listening tasks: an A-only intelligibility task, and an AV 

intelligibility task. In the A-only condition, listeners will hear sentences over headphones and will type their 

response in a GUI. Each listener will be presented with 10 different sentences each from a quasi-random subset 

of 20 of the 50 talkers (2 men and 2 women from each of the race/ethnicity categories). The 10 sentences will 

include 5 of the standard-of-care sentences, and five of the sentences from the new corpus developed for this 

study. No sentence will be repeated twice in the experiment. The decision to use only a subset of the talkers is 

based on largely practical grounds: our previous work has found that 200 sentences is the maximum that 

participants can tolerate in a listening task. In the AV task, listeners will view and listen to the same 20 talkers 

producing a different set of sentences. The same response GUI will be used. In this task, we will capture the 

subjects' eye movements using Tobii T60 XL eye-tracker during the entire interval during which the talker's 

face is present (i.e., the sentence, and 500 ms intervals prior to the sentence). The order of the A-only and AV 

tasks will be randomized across participants. 

The masking noise will be a 4-talker babble, created via random concatenation and mixing of talkers and 

sentences not used in the experiment. The SNR will be based on pilot experiments and will be different for the 

three groups so that the percentage of keywords repeated is in a similar range for all subjects. In both tasks, the 

auditory stimuli will be presented diotically over headphones. For the NH groups, the level of the target will be 

set to 65 dB SPL. For the HI listeners, the stimuli will be spectrally shaped and amplified according to the NALNL2 formula for each listener individually to ensure audibility. Gain will be adjusted if necessary based on 

subject complaint using the procedures described in

[32]

[28]. 

D.5. Individual-Differences Measures 

For YNH, ONH, and ONI listeners, we will collect or measure pure-tone audiograms. We will also collect aided 

audiograms for OHI listeners. For all three groups, we will administer a series of IATs 49], which measure 

individuals' preconscious attitudes toward the five racial and ethnic categories in this grant. The selection of 

the specific IAT measures will be guided by consultant Prof. Jeff Simpson. We will also estimate the racial and 

ethnic diversity in individuals' peer groups using methods from : asking the names of up to 15 people who 

would help the participant with a significant chore requiring group participation. This will allow us to estimate 

the racial and ethnic diversity in each listener's social group, and to use these measures in further exploratory 

analyses of factors that mitigate REI. Finally, we will include brief measures of cognition from the NIH toolkit, 

[13]

[20,

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 58 
as was done by This will allow us to test whether individual listeners’ differences in REI are mitigated by 

processing fluency. 

[27].

D.6. Stimulus Annotation and Dissemination 

Each of the AV sentences that develop in specific aim 1 will be segmented and made available in a public 

archive, similar in design to other speech-stimulus archives (i.e., the ALLSSTAR corpus, ). The metadata 

will include the speakers' age, sex, and self-reported race and ethnicity, the proportion correct ethnicity 

perception in the A-only condition, and the proportion of each of the content words in the sentences that were 

correctly reported, separated by listener group (YNH, ONH, OHI). Given the high audio quality of these data, 

they will be potentially useful to speech production researchers or speech processing research (i.e., for studies 

of automatic classification of speakers' race and ethnicity from acoustic signals alone, [2]). 

[12]

D.7. Intelligibility measures. For the listening task, we will measure each stimulus's intelligibility as the 

proportion of keywords (i.e., content words) correct per sentence. Listeners will be asked to type what they 

heard, via a computer keyboard. Scores will be first automatically tallied, and then all errors will be manually 

checked for potential misspellings or homophones, which will be marked as correct. 

D.8. Eye-Tracking Measures 

Following we will define five areas on each talker's face: the mouth, the left eye, the right eye, the left 

forehead, and the right forehead. For each trial, we will calculate the proportion of eye fixations on each of the 

five areas of the face, on the remaining non-face areas of the person, both during the 500 ms interval prior to 

the sentence onset, and during the sentence itself. 

 [57],

D.9. Statistical Analyses 

The statistical analyses will involve a combination of frequentist [ 8,9,34] and Bayesian [ ] multilevel logistic 

modeling. Mixed-effects models will be used, as the specific talkers and items will vary across the individual 

listeners. Given that the goal of specific aim 2 is exploratory in nature, there is not a large body of literature on 

which to base a power analysis that would be needed for frequentist statistics to be used exclusively. In all of 

our analyses, the dependent measure will be the whether each of the content words was correctly reported. 

2, 56

The first set of analyses compares the two sets of speech materials. To determine whether the standard-ofcare sentences and the newly created sentences have different intelligibility levels, we will run logit mixedeffects models predicting the probability of keyword recognition in A-only conditions as a function of sentence 

type. 

The next set of analyses examines the mechanisms that underlie REI. To replicate [ ], we will examine 

whether a model with condition (A-only vs. AV) fits data better than a model with talker race/ethnicity, both 

alone and in interaction with race/ethnicity. To evaluate the attention-driven hypothesis, we will examine 

whether a model that includes the proportion of time spent looking at the mouth improves fit compared to a 

model with only talker race/ethnicity and condition. If the model fits are significantly different, we can 

conclude that attentional factors are at least partly responsible for REI. In this case, we will run a series of 

models that examines the relationship between visual attention and accuracy across different race and ethnicity 

categories. To evaluate the attitude-driven hypothesis, we will examine whether a model that includes listeners' 

IAT scores fits the data better than a model with talker race/ethnicity and condition only. If the fits are 

significantly different, then we can conclude that attitudes drive REI, at least partly. We will follow up on this 

finding by running a series of models that examines the relationship between IAT and race/ethnicity in 

different conditions. To examine whether expectations drive REI, we will compare a model that includes the 

proportion of the person's peer group that is people of color (using the measures described in D.5) to a model 

that includes talker race/ethnicity and condition only. If the fits are significantly different, then we can 

conclude that expectations drive REI, at least partly. Finally, we will build a set of progressively more complex 

models that use multiple predictor variables to examine whether attention, attitudes, and expectations predict 

intelligibility in combination with one another. 

6

These analyses will be followed by a series of analyses that include a variable coding the mismatch in 

race/ethnicity between the listeners and talkers in place of a variable coding talker race. This will allow us to 

assess whether REI reflects mismatches between talkers and listeners, or if it relates specifically to the 

perception of the speech of people of color. 

The final set of analyses examines whether the effects from the previous paragraph are statistically 

equivalent across the three listener groups (YNH, ONH, OHI). In each of these analyses, group will be added to 

the most-complex models above. In cases where the addition of listener group results in significantly better 

model fit, we will re-run models examining the interaction between race/ethnicity and condition separately in 

the different listener groups. 

Contact PD/PI: Munson, Benjamin R 

Research Strategy 

 Page 59 
PHS Human Subjects and Clinical Trials Information OMB Number: 0925-0001 and 0925-0002

Expiration Date: 03/31/2020

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Are Human Subjects Involved ● Yes ❍ No 

Is the Project Exempt from Federal regulations? ❍ Yes ● No 

Exemption Number ❏ 1 ❏ 2 ❏ 3 ❏ 4 ❏ 5 ❏ 6 ❏ 7 ❏ 8 

Other Requested Information 

 Page 60 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Human Subject Studies 

Study# Study Title Clinical Trial? 

1 Race, Ethnicity, and Speech Intellig bility in Hearing Impairment and Normal Hearing No 

 Page 61 
OMB Number: 0925-0001 and 0925-0002

Expiration Date: 03/31/2020

Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Section 1 - Basic Information (Study 1) 

1.1. Study Title * 

Race, Ethnicity, and Speech Intelligibility in Hearing Impairment and Normal Hearing 

1.2. Is this study exempt from Federal 

Regulations * ❍ Yes ● No 

1.3. Exemption Number ❏ 1 ❏ 2 ❏ 3 ❏ 4 ❏ 5 ❏ 6 ❏ 7 ❏ 8 

1.4. Clinical Trial Questionnaire * 

1.4.a. Does the study involve human participants? ● Yes ❍ No 

1.4.b. Are the participants prospectively assigned to an intervention? ❍ Yes ● No 

1.4.c. Is the study designed to evaluate the effect of the intervention on the 

participants? ❍ Yes ● No 

1.4.d. Is the effect that will be evaluated a health-related biomedical or 

behavioral outcome? ❍ Yes ● No 

1.5. Provide the ClinicalTrials.gov Identifier (e.g. 

NCT87654321) for this trial, if applicable 

 Page 62 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Section 2 - Study Population Characteristics (Study 1) 

2.1. Conditions or Focus of Study

❍ Specific Aim 1: Record, annotate, and disseminate a large set of ethnically and racially diverse talkers producing a set of

standard audiometric sentences, and sentences specifically created to elicit socially meaningful phonetic variation.

❍ Specific Aim 2: Determine whether talker race and ethnicity affect the magnitude the AV benefit to speech intelligibility in

individuals with and without hearing impairment.

2.2. Eligibility Criteria

Specific Aim 1: Individuals who are recruited as talkers will be between 18 and 30 years old. They will have no past or current

 speech, language, or hearing impairment, per self-report. Given our goal of recruiting an ethnically and racially diverse group

 of talkers, we anticipate that some of our potential participants will not be monolingual speakers of English. We will limit

 participation to individuals who began to acquire English in the US, who use English as the primary language in the home

 and in their daily lives, and who began to acquire English no later than age 7. This age is within the range that is broadly

 agreed to be the critical period for language acquisition, at which time languages can be acquired with little or no foreign

 accent. Hence, we anticipate that none of our talkers will produce speech with a foreign accent. Given that the focus of

 this corpus is on audio-visual speech perception, people who wear face-covering clothing (e.g., a niqab or burqa) will not

 be eligible to participate as talkers. Headwear will be permitted only it is something the talkers habitually wear for cultural or

 religious reasons (e.g,. hijab, kippah, taqiyah, dastaar), as these items are generally worn habitually in public and are hence

 encountered in real-world social communication.

Specific Aim 2: Individuals in the younger, normal-hearing (YNH) group will be between 18 and 40 years old. They will have

 no past or current speech, language, or hearing impairment, per self-report. We will limit participation to individuals who

 began to acquire English in the US, no later than age 7. The criteria for normal hearing in the young group will be audiometric

 thresholds of no more than 20 dB HL at octave frequencies between 250 and 8000 Hz. The older hearing impairmed (OHI)

 and normal hearing (ONH) groups will be between 60 and 90 years old. For the older NH group, the criteria for normal

 hearing will be to have thresholds no more than 20 dB HL at octave frequencies between between 250 and 2000 Hz, and to

 have thresholds no more than 40 dB HL at 4000 and 8000 Hz. This relaxed criterion is due to the high incidence of threshold

 shifts in older adults due to normal aging processes. The thresholds for 2000 Hz and 4000 Hz will be recorded for the ONH

 group, and will be used as a covariate in analyses of differences between ONH and YNH groups. For the HI groups, inclusion

 criteria include sensorineural hearing loss with no conductive component with pure-tone average audiometric thresholds

 between 500 and 2000 Hz between 40 and 65 dB HL, and no pure-tone thresholds exceeding 75 dB HL at 4 kHz or above.

 Individuals in all three groups will be fluent English speakers, defined as individuals who began to acquire English no later

 than age 7, in the US, who use English as the primary language in the home and in their daily lives.

2.3. Age Limits Min Age: 18 Years Max Age: 90 Years

2.4. Inclusion of Women, Minorities, and Children AVGrant_R21_HumanSubjects_InclusionOfWomenMinoritiesChildren.pdf

2.5. Recruitment and Retention Plan AVGrant_R21_HumanSubjects_RecruitmentAndRetention.pdf

2.6. Recruitment Status Not yet recruiting

2.7. Study Timeline AVGrant_R21_HumanSubjects_StudyTimeline.pdf

2.8. Enrollment of First Subject 07/01/2019 Anticipated

 Page 63 
Inclusion of Women and Minorities 

Specific Aim 1: We will recruit equal numbers of talkers who identify as one of five different race/ethnicity 

categories (N=50 [25F, 25M] overall, N=10 [5F, 5M] in each category), as determined by self-report: (1) nonLatinx white, (2) Latinx of any race [most of whom we anticipate to be white Latinx individuals], (3) African 

American/Black, (4) South Asian [defined as people who identify as having ancestral roots in Afghanistan, 

Pakistan, India, Bhutan, Nepal, Bangladesh, Sri Lanka, or Maldives], and (5) East Asian [defined as people who 

identify as having ancestral roots in Mongolia, P.R. China, Taiwan, Korea, or Japan]. Our choice of these five 

demographic categories is motivated by a combination of demographics and pragmatic considerations. The 

first three categories are the dominant race/ethnicity categories in the US, comprising approximately 84% of 

the population. While South and East Asian individuals make up only 5% of the US, they make up the nearly 

half of the world’s population. The inclusion of South and East Asian individuals will make this database more 

reflective of racial and ethnic diversity across the world. It is also broadly consistent with the race/ethnicity 

distributions in databases of static images of faces created previously, such as those described in [46]. In 

addition to demographic factors, the decision to limit the talkers to these five categories is partly for feasibility. 

These categories are well-represented in the population from which we will recruit the talkers. Talkers will be 

recruited from the University of Minnesota undergraduate and graduate-student populations, and will leverage 

the social networks of the ethnically and racially diverse students that work in the PI and co-Is' laboratories. 

These mechanisms for talker recruitment have been successful in previous efforts to recruit diverse talkers. 

This is not an exhaustive list of race/ethnicity categories in the US. It does not include culturally significant, 

smaller ethnic groups in the US, such as Native Americans, Native Alaskans, and Pacific Islanders. Moreover, 

the groups themselves comprise considerable internal ethnic variation. Increasing the size and diversity of this 

database is an important future endeavor, beyond the period of funding. 

Specific Aim 2: This project will recruit a racially and ethnically diverse population of younger and older 

listeners. Our enrollment goals for different racial and ethnic categories was determined by taking the most 

recent figures from the US Census Bureau, and doubling the percentage of white, non-Latinx individuals that 

we seek to enroll. This was motivated by our goal of generating findings that will be appropriate for the morediverse future generations, given the ongoing demographic changes in the US. Recruiting a diverse set of 

listeners is important to the success of this project. This will be accomplished by using University of Minnesota 

community engagement mechanisms, as well as community partnerships that PI Munson built during a 

previous grant that examined low-income children (R01 DC02932). Given demographic changes, we anticipate 

that the YNH (n=60 [30F, 30M]) group will be more diverse than the ONH (n=30 [15F, 15M]) and ONI (n=30 

[15F, 15M]) groups. 

Inclusion of Children 

No individuals below age 18 will be included in this study. This exclusion is justified because this research has 

no direct therapeutic benefit, and because the materials and the experimental procedures are not appropriate 

for children. 

Inclusion of Women, Minorities, and Children 

 Page 64 
Human Subjects Recruitment and Retention Plan 

Participants for Specific Aim 1 will be recruited using two methods. The first of these will be advertisements in 

different University of Minnesota venues (i.e., newspapers, listservs for organizations of students of color, 

flyers, etc.) and in the community, particularly in university-affiliated organizations that serve communities of 

color (i.e., the Urban Research and Outreach-Engagement Center [UROC]). Younger, normal-hearing 

participants for Specific Aim 2 will be recruited through these same mechanisms. Older, normal-hearing 

participants will be recruited through University of Minnesota retiree groups, and by distributing flyers at 

University of Minnesota events that cater to older adults, such as the Osher Lifelong Learning Institute (OLLI). 

Older adults with hearing loss will be recruited through a participant database curated by the Julia M. Davis 

Speech-Language-Hearing Center, which is housed in the PI's home department, and by contacting individuals 

who have participated previously in co-I Nelson and Oxenham's research, and who have given consent to be 

contacted for future studies. These mechanisms have been used successfully in previous studies with these 

populations at the University of Minnesota. 

The one-time nature of this study obviates the need for an explicit retention plan. However, we will write a 

biannual newsletter that we will share with participants and partnering organizations (i.e, UROC, OLLI). The 

newsletter will document the progress on the grant, and will highlights research on the topic of this grant in our 

lab and in others' labs. We will also create a project website, on which we will post similar information. These 

two tactics were used previously in PI Munson and colleagues' work with children, with great success. The 

project website has the additional benefit that it serves as a hub for students who are interested in contributing 

to this project through volunteering or conducting student research (as described in section 10, Facilities and 

Other Resources). 

Recruitment and Retention Plan 

 Page 65 
Recruitment and Retention Plan Page 66
Contact PD/Pl: Munson, Benjamin R 

Hu1nan Subjects Study Timeline 

Month of funding in year 1 (1- 12) and prior to the start of funding (- 1) 

Activity -1 1 2 3 4 5 6 7 8 9 10 11 12 

Meet with stakeholders at potential recruitment sites, 

secure their agreement to adve1tise the study 

Develop new sentences for specific aim 1 

Develop recording and annotation protocol 

Recruit talkers 

Record talkers 

Segment stimuli 

Finalize data-collection protocol for specific aim 2 

Pilot-test protocol for specific aim 2 to determine optimal 

signal-to-noise ratio 

Recruit participants for specific aim 2 

Month of funding in year 2 (12- 24) 

Activity 13 14 15 16 17 18 19 20 2 1 22 23 24 

Test participants for specific aim 2 

Analyze data from specific aim 2 

Finalize design of the digital archive to be used to 

disseminate the corpus from specific aim 1 

Write manuscripts summarizing results of this proiect 

Release the digital archive of stimuli for specific aim 1 

Page 67 

Study Timeline 

x

x

x x

x x x x x x

x x x x x x x x

x x x x x x x x x

x x x

x

x x

x

x x x x x x x x x

x x x x x x

x x x

x x x

x

x
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Inclusion Enrollment Reports 

IER ID# Enrollment Location Type Enrollment Location

Study 1, IER 1 Domestic University of Minnesota, Twin Cities

 Page 68 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Inclusion Enrollment Report 1 

Using an Existing Dataset or Resource* : ❍ Yes ● No 

Enrollment Location Type* : ● Domestic ❍ Foreign 

Enrollment Country(ies): USA: UNITED STATES

Enrollment Location(s): University of Minnesota, Twin Cities

Comments: 

Planned 

Racial Categories 

Ethnic Categories 

Not Hispanic or Latino 

Female Male 

Hispanic or Latino 

Female Male 

Total 

American Indian/ 

Alaska Native 0 0 0 0 0 

Asian 13 12 0 0 25 

Native Hawaiian or 

Other Pacific Islander 1 1 0 0 2 

Black or African 

American 8 9 1 0 18 

White 24 23 7 8 62 

More than One Race 1 2 0 0 3 

Total 47 47 8 8 110 

Cumulative (Actual) 

Racial Categories 

Ethnic Categories 

Not Hispanic or Latino 

Female Male 

Unknown/ 

Not 

Reported 

Hispanic or Latino 

Female Male 

Unknown/ 

Not 

Reported 

Unknown/Not 

Reported Ethnicity 

Female Male 

Unknown/ 

Not 

Reported 

Total 

American Indian/ 

Alaska Native 0 0 0 0 0 0 0 0 0 0 

Asian 0 0 0 0 0 0 0 0 0 0 

Native Hawaiian or 

Other Pacific Islander 0 0 0 0 0 0 0 0 0 0 

Black or African 

American 0 0 0 0 0 0 0 0 0 0 

White 0 0 0 0 0 0 0 0 0 0 

More than One Race 0 0 0 0 0 0 0 0 0 0 

Unknown or 

Not Reported 0 0 0 0 0 0 0 0 0 0 

Total 0 0 0 0 0 0 0 0 0 0 

 Page 69 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Section 3 - Protection and Monitoring Plans (Study 1) 

3.1. Protection of Human Subjects AVGrant_R21_HumanSubjects_ProtectionOfHumanSubjects.pdf

3.2. Is this a multi-site study that will use the same protocol to

conduct non-exempt human subjects research at more than one

domestic site?

❍ Yes ● No ❍ N/A 

If yes, describe the single IRB plan 

3.3. Data and Safety Monitoring Plan AVGrant_R21_HumanSubjects_DataSafetyAndMonitoring.pdf

3.4. Will a Data and Safety Monitoring Board be appointed for

this study?

❍ Yes ● No 

3.5. Overall structure of the study team

 Page 70 
Protection of Human Subjects 

1. Risks to Human Subjects 

a. Human Subjects Involvement, Characteristics, and Design 

For Specific Aim 1, the procedures involve video- and audio-recording individuals producing a standard set of 

sentences used in audiological research [26], and a newly created set of sentences designed to have socially 

meaningful content, and to elicit a more natural, socially meaningful speech style than is generally elicited in 

productions of standard sentences. The standard sentences will be elicited with a reading task. The new 

sentences will also be elicited with a reading task, but the target sentences will be embedded in a longer stretch 

of text. The content of that text (i.e., the word choice, the topic) will be designed to elicit a natural speaking 

style (i.e., by using more colloquialisms, and by including topics that are socially engaging). Both types of 

sentences will be segmented, and used as stimuli for specific aim 2. Upon the completion of this project, the 

audiovisual and audio-only sentences created in specific aim 1 will be made available to the research 

community through a digital archive. 

For Specific Aim 2, participants will complete a sentence intelligibility task. In this task, individuals will be 

presented with audio-only and AV sentences (created in specific aim 1) in the presence of background noise. 

They will type what they hear in a GUI. During the AV task, participants' eye movements will be tracked using 

a Tobii T60 XL eye-tracker, so that we can measure whether talker race and ethnicity affect the areas of the face 

that participants gaze at both before and during the production of target sentences. In addition, participants 

will complete (1) a series of implicit association tests to measure preconscious attitudes about race and 

ethnicity, (2) a series of brief nonverbal cognitive measures, and (3) a brief survey to estimate the racial and 

ethnic diversity in participants' peer groups. 

For Specific Aim 1, participants will be healthy young adults aged 18-40. For Specific Aim 2, the participants 

will be healthy younger adults (aged 18-30) without hearing impairment, and older adults (60-90) with or 

without sensorineural hearing loss. 

All of the research will take place at a single site, University of Minnesota. 

b. Study Procedures, Materials, and Potential Risks 

Specific Aim 1: 

The procedures for this specific aim are entirely specific to research (i.e., none of these procedures is part of 

standard care). No existing records will be accessed. For this specific aim, the primary risk is that someone 

(i.e., a participant in specific aim 2, or a participant in any future study using the archive developed in this 

specific aim) will recognize one of the talkers who is recorded in specific aim 1, and subsequently ask the 

person something about their participation in this study that is inappropriate or uncomfortable. There is no 

way to prevent this from happening. The same risk applies to individuals who consented to be in corpora of 

faces. To mitigate this risk, we will provide extensive information to potential participants about this potential 

problem prior to recording. We will point participants to existing corpora (both audio-only corpora, like those 

in TalkBank, and static-image corpora, like those in the Tarr Lab's Face Place) as examples of the type of data 

that they will provide. 

Specific Aim 2: 

The only previously collected records that will be used in this study are audiological assessment data for the 

older, hearing-impaired participants. These will be collected only after participants have provided 

authorization to access HIPAA-protected health information. The procedures for specific aim 2 are commonly 

used in research and in audiological assessment. They pose no potential risk to participants, beyond the mild 

boredom and frustration that accompany any task requiring sustained attention. To mitigate that risk, we will 

provide frequent breaks. 

Protection of Human Subjects 

 Page 71 
2. Adequacy of Protection Against Risks 

a. Informed Consent and Assent 

Informed consent will be secured from all participants prior to testing. All individuals who secure consent (PI 

Munson, TBA Postdoctoral Fellow, and TBA Undergraduate Research Assistant) will have completed CITI 

training in the protection of human subjects in research. Prior to being scheduled to participate in Specific Aim 

1, participants will affirm that they understand that the research involves creating archival recordings of their 

faces and voices. The consent form will only be signed after the potential participants have successfully 

answered comprehension questions about the study. In addition, the consent form for Specific Aim 1 will have 

a separate section, developed in conjunction with the University of Minnesota IRB and the office of external 

relations, in which participants will initial that they understand the different potential consequences of 

consenting to have their audio and video data shared in a public archive. 

Prior to being scheduled in Specific Aim 2, participants will affirm that they do not have a legally authorized 

representative, and that they are able to consent to participate themselves. All consent-securing staff will be 

trained on the use of the University of California, San Diego Brief Assessment of Capacity to Consent (UBACC). 

In any cases where a researcher suspects that the potential participant might not have the capacity to consent, 

the UBACC will be administered. Participants who fail this screening will be excluded from the study. The 

consent form will only be signed after the potential participants have successfully answered comprehension 

questions about the study. 

b. Protections Against Risk 

All research assistants who administer the protocol will have been trained by PI Munson on its proper 

administration. PI Munson will develop the protocol-training procedures in collaboration with co-Is Nelson 

and Oxenham (whose expertise is in auditory perception) and consultants Babel and Simpson (whose expertise 

is in social perception and implicit attitudes). 

All data will be stored on the PI's project server, access to which is limited to researchers on this project, and 

which is accessed through two-factor authentication. Paper data (i.e., consent forms) will be stored in a locked 

cabinet in the PI's faculty office. 

c. Vulnerable Subjects, if relevant to your study 

No vulnerable populations will be included in this research. 

3. Potential Benefits of the Proposed Research to Research Participants and Others 

We do not anticipate any direct therapeutic benefit to participants. The risks to the participants are minimal. 

4. Importance of the Knowledge to be Gained 

The knowledge to be gained in this study is of vital importance to society. Communication between older and 

younger adults often crosses lines of race and ethnicity. Preliminary work suggests that race differences 

introduce a barrier to speech communication. This study will be the first to examine whether these barriers 

disproportionately influence older adults with hearing impairment. It will also examine the mechanisms that 

might explain why race and ethnicity affect speech intelligibility in both older and younger adults. A better 

understanding of the mechanisms of these effects is needed before we can develop methods to mitigate or 

remove them. Moreover, the corpus of speech produced by racially and ethnically diverse individuals will be of 

great use to basic science research, and to clinical investigations of speech intelligibility. 

Protection of Human Subjects 

 Page 72 
Data and Safety Monitoring Plan 

This project does not require a data safety monitoring plan, as it does not meet NIH's definition of a clinical 

trial. Hence, there is no need to appoint a monitoring board. 

The only risk associated with this study is a loss of confidentiality if there is a data breach. To protect against 

this, all data will be stored on the PI Munson's project server, access to which he alone controls. The PI reviews 

access to the server on a quarterly basis, and removes access to individuals who have left the project. 

Data and Safety Monitoring Plan 

 Page 73 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Section 4 - Protocol Synopsis (Study 1) 

4.1. Brief Summary

4.2. Study Design

4.2.a. Narrative Study Description

4.2.b. Primary Purpose

4.2.c. Interventions

Type Name Description 

4.2.d. Study Phase

Is this an NIH-defined Phase III Clinical Trial? ❍ Yes ❍ No 

4.2.e. Intervention Model

4.2.f. Masking ❍ Yes ❍ No 

❏ Participant ❏ Care Provider ❏ Investigator ❏ Outcomes Assessor 

4.2.g. Allocation 

4.3. Outcome Measures

Type Name Time Frame Brief Description 

4.4. Statistical Design and Power

4.5. Subject Participation Duration

4.6. Will the study use an FDA-regulated intervention? ❍ Yes ❍ No 

4.6.a. If yes, describe the availability of Investigational

Product (IP) and Investigational New Drug (IND)/

Investigational Device Exemption (IDE) status

4.7. Dissemination Plan 

 Page 74 
Tracking Number: GRANT12729780 Funding Opportunity Number: PA-18-489 Received Date: 

2018-10-15T15:01:25.000-04:00 

Delayed Onset Studies 

Delayed 

Onset 

Study# 

Study Title Anticipated Clinical 

Trial? 

Justification 

The form does not have any delayed onset studies 

 Page 75 
References 

[1] Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters

under high attention demands. Current Biology, 15, 839-843.

[2] Arjmandi, M., Dilley, L., & Wagner, S. (2018). Acoustic cues to linguistic profiling? Machine learning of

phonetic features of African American English [A]. Journal of the Acoustical Society of America, 143,

1969.

[3] Baayen, R.H., Davidson, D.J., & Bates, D.M. (2008). Mixed-effects modeling with crossed random

effects for subjects and items, Journal of Memory and Language, 59, 390–412.

[4] Babel, M. (2012). Evidence for phonetic and social selectivity in spontaneous phonetic imitation.

Journal of Phonetics, 40, 177-189.

[5] Babel, M., & Munson, B. (2014). Producing socially meaningful linguistic variation. In M. Goldrick, V.

Ferreira, & M. Miozzo (Eds.), Oxford Handbook in Language Production. Oxford University Press.

[6] Babel, M. & Russell, J. (2015). Expectations and speech intelligibility. Journal of the Acoustical Society

of America, 137, 2823-2833.

[7] Balota, D.A., Yap, M.J., Cortese, M.J., Hutchison, K.A., Kessler, B., Loftis, B., Neely, J.H., Nelson, D.L.,

Simpson, G.B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39,

445-459.

[8] Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory

hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255-278.

[9] Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). lme4: Linear mixed-effects models using Eigen

and S4. R package version 1.1-9, https://CRAN.R-project.org/package=lme4.

[10] Bernstein, L., Auer, E., & Takayanagi, S. (2004). Auditory speech detection in noise enhanced by

lipreading. Speech Communication, 44, 5-18.

[11] Blair, I. (2002). The malleability of automatic stereotypes and prejudice. Personality and Social

Psychology Review, 6, 242-261.

[12] Bradlow, A. R., Ackerman, L., Burchfield, L. A., Hesterberg, L., Luque, J. S., & Mok, K. (2010).

ALLSSTAR: Archive of L1 and L2 Scripted and Spontaneous Transcripts And Recordings,

http://groups.linguistics.northwestern.edu/speech_comm_group/allsstar/index.html.

[13] Chesley, P. (2011). You know what it is: learning words through listening to hip-hop. PLoS ONE 6(12):

e28248. https://doi.org/10.1371/journal.pone.0028248.

[14] Chiroro, P., & Valentine, T. (1995). An investigation of the contact hypothesis of the own-race bias in

face recognition. Quarterly Journal of Experimental Psychology: Human Experimental Psychology,

48A, 879–894.

[15] Cienkowski, K., & Carney, A. (1999). Audiovisual speech perception and aging. Ear & Hearing, 23, 

439-449.

[16] Cooke, M., Barker, J., Cunningham, S., & Shao, X. (2006). An audio-visual corpus for speech perception

and automatic speech recognition (L). Journal of the Acoustical Society of America, 120, 2421-2424.

[17] Demorest, M., & Bernstein, L. (1992). Sources of variability in speechreading sentences: a

generalizability analysis. Journal of Speech and Hearing Research, 35, 876-891.

[18] Drager, K. (2010). Sociophonetic variation in speech perception. Linguistics and Language Compass,

4/7, 473–480.

[19] Drager, K. (2011). Speaker age and vowel perception. Language and Speech, 54, 99-121

[20] Evans, K.E., Munson, B., & Edwards, J. (2018). Does speaker race affect the assessment of children’s

speech accuracy? a comparison of speech-language pathologists and clinically untrained listeners.

Language, Speech, and Hearing Services in Schools, E-Publication ahead of print,

doi:10.1044/2018_LSHSS-17-0120. PMID: 29971346

[21] Gagne, J., Masterson, V., Munhall, K., Bilida, N., & Querengesser, C. (1994). Across talker variability in

auditory, visual, and audiovisual speech intelligibility for conversational and clear speech. Journal of the 

Academy of Rehabilitative Audiology, 27, 135-158.

[22] Goldinger, S., & Azuma, T. (2004). Episodic memory reflected in printed word naming. Psychonomic

Bulletin & Review, 11, 716-722.

[23] Gregan, M., Nelson, P., & Oxenham, A. (2013).Behavioral measures of cochlear compression and

temporal resolution as predictors of speech masking release in hearing-impaired listeners. The Journal

of the Acoustical Society of America, 134, 2895-2912.

Contact PD/PI: Munson, Benjamin R 

References Cited

 Page 76 
[24] Hess, T. Hinson, J. & Statham, J. (2004). Explicit and implicit stereotype activation effects on memory:

do age and awareness moderate the effect of priming? Psychology and Aging, 19, 495-505.

[25] Jannedy, S., Weirich, M., & Brunner, J. (2011). The Effect of Inferences on the Perceptual Categorization 

of Berlin German Fricatives. In Proceedings of the International Congress of Phonetic Sciences (ICPhS 

2011), 962-965.

[26] IEEE Subcommittee (1969). IEEE recommended practice for speech quality measurements. IEEE

Transactions on Audio and Electroacoustics, Au-17, 225–246.

[27] Ingvalson, I., Lansford, L., Federova, V., & Fernandez, G. (2017). Listeners' attitudes toward accented

talkers uniquely predicts accented speech perception. Journal of the Acoustical Society of America, 141,

EL234-EL238.

[28] Jenstad, L., VanTasell, D. and Ewert, C. (2003). Hearing aid troubleshooting based on patients'

descriptions. Journal of the American Academy of Audiology, 14, 347-360.

[29] Jiang, J., Auer, E., Alwan, A., Keating, P., & Bernstein, L. (2007). Similarity structure in visual speech

perception and optical phonetic signals. Perception & Psychophysics, 69, 1070-1083.

[30] Johnson, J. A., Cox, R. M., & Alexander, G. C. (2010). Development of APHAB norms for WDRC hearing 

aids and comparisons with original norms. Ear and Hearing, 31, 47-55.

[31] Johnsrude, I. S., Mackey, A., Hakyemez, H., Alexander, E., Trang, H. P., & Carlyon, R. P. (2013).

Swinging at a cocktail party: Voice familiarity aids speech perception in the presence of a competing

voice. Psychological Science, 24, 1995-2004.

[32] Keidser, G., Dillon, H., Carter, L., and O’Brien, A. (2012) NAL-NL2 Empirical Adjustments. Trends in

Amplification, 16, 211–223

[33] Kramer, S., Kapteyn, T., Kuik, D., & Deeg, D. (2002). The association of hearing impairment and

chronic diseases with psychosocial health status in older age. Journal of Aging and Health, 14, 122–137

[34] Kuznetsova, A., Brockhoff, P.B., & Christensen, R.H.B. (2013). lmerTest: Tests for random and fixed

effects for linear mixed effect models (lmer objects of lme4 package). R-Version:1.1-0.

http://cran.rproject.org/web/packages/lmerTest/index.htm.

[35] Lin, F., Metter, E., O’Brien, R., Resnick, S., Zonderman, A., Ferrucci, L. (2011). Hearing loss and

incident dementia. Archives of Neurology, 68, 214-220

[36] Lin, F., Niparko, J., & Ferrucci, L. (2011). Hearing loss prevalence in the United States. Archives of

Internal Medicine, 171, 1851-1852.

[37] McGowan, K. B. (2015). Social expectation improves speech perception in noise. Language and Speech,

58, 502-521.

[38] Munhall, K., Jones, J., Callan, D., Kuratate, T., & Vatekiotis-Bateson, E. (2004). Visual prosody and

speech intelligibility: head movement improves auditory speech perception. Psychological Science, 15,

133-137.

[39] Munson, B. (2011). The influence of actual and imputed talker gender on fricative perception, revisited.

Journal of the Acoustical Society of America, 5, 2631-2634. PMID: 22087888

[40] Munson, B., Jefferson, S. V., & McDonald, E. C. (2006). The influence of perceived sexual orientation on 

fricative identification. Journal of the Acoustical Society of America, 119, 2427-2437. PMID: 16642855

[41] Munson, B., Ryherd, K., & Kemper, S. (2017). Implicit and explicit gender priming in English lingual

sibilant fricative perception. Linguistics, 55 ,1073-1108.

[42] NIDCD (2016). Quick Statistics About Hearing. Retrieved from

https://www.nidcd.nih.gov/health/statistics/quick-statistics-hearing, September 25, 2018.

[43] Niedzielski, N. (1999). The effect of social information on the perception of sociolinguistic variables.

Journal of Language and Social Psychology, 18, 62-85.

[44] Pew Research Center (2013). King’s Dream Remains an Elusive Goal; Many Americans See Racial

Disparities. Downloaded from http://www.pewsocialtrends.org/2013/08/22/kings-dream-remainsan-elusive-goal-many-americans-see-racial-disparities/ on September 25, 2018.

[45] Purnell, T., Idsardi, W., & Baugh, J. (1999). Perceptual and phonetic experiments of American English

dialect identification. Journal of Language and Social Psychology, 18, 10–30.

[46] Righi, G., Peissig, J. J., & Tarr, M. J. (2012). Recognizing disguised faces. Visual Cognition, 20, 143-169.

[47] Rodriguez, J. (2016). Our clients are diverse: why aren't we? ASHA Leader, 21, 40-42.

[48] Rubin, D. (1992). Nonlanguage factors affecting undergraduates' judgments of nonnative Englishspeaking teaching assistants. Research in Higher Education, 33, 511-531.

Contact PD/PI: Munson, Benjamin R 

References Cited

 Page 77 
[49] Rudman, L., Ashmore, R., & Gary, M. (2001). ‘Unlearning' automatic biases: the malleability of implicit 

stereotypes and prejudices. Journal of Personality and Social Psychology, 81, 856-868. 

[50] Scarborough, R, Keating, P., Mattys, S., Cho, T., & Alwan, A. (2009). Optical phonetics and visual 

perception of lexical and phrasal stress in English. Language and Speech, 51, 135-175. 

[51] Senior, B., & Babel, M. (2018). The role of unfamiliar accents in competing speech. Journal of the 

Acoustical Society of America, 143, 931-942. 

[52] Strand, E., & Johnson, K. (1996). Gradient and visual speaker normalization in the perception of 

fricatives. Natural Language Processing and Speech Technology: Results of the 3rd KONVENS 

Conference, Bielfelt (D. Gibbon, Ed.), 14-26. 

[53] Strawbridge, W., Wallhagen, M., Shema, S., & Kaplan G. (2000). Negative consequences of hearing 

impairment in old age: a longitudinal analysis. Gerontologist, 40, 320–326. 

[54] Taylor, P., & Pew Research Center (2016). The Next America. New York: PublicAffairs. 

[55] Uhlmann, R., Larson, E., Rees, T., Koepsell, T., & Duckert, L. (1989). Relationship of hearing 

impairment to dementia and cognitive dysfunction in older adults. Journal of the American Medical 

Association, 261, 1916–1919. 

[56] Vasishth, S., Nicenboim, B., Beckman, M., Li, F., & Kong, E. (2018). Bayesian data analysis in the 

phonetic sciences: A tutorial introduction. Journal of Phonetics, 71, 147-161. 

[57] Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., & Munhall, K. (1998). Eye movements of perceivers 

during audiovisual speech perception. Perception & Psychophysics, 60, 926-940. 

[58] Walker, P., & Tanaka, J. (2003). An encoding advantage for own-race versus other-race faces. 

Perception, 32, 1117-1125. 

[59] Winn, M., Rhone, A., Chatterjee, M., & Idsardi, W. (2013). The use of auditory and visual context in 

speech perception by listeners with normal hearing and listeners with cochlear implants. Frontiers in 

Psychology, 4, 824 

[60] Yehia, H., Kuratate, T., & Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion and 

speech acoustics. Journal of Phonetics, 30, 555–568. 

[61] Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial 

behavior. Speech Communication, 26, 23-43. 

[62] Yi, H.-G., Phelps, J., Smiljanic, R., & Chandrasekaran, B. (2013). Reduced efficiency of audiovisual 

integration for nonnative speech. Journal of the Acoustical Society of America, 134, EL387-EL393. 

References Cited 

 Page 78