Objective Response Detection to Natural Speech Stimuli for Optimisation of Hearing Aid Fitting Evaluation
Conditions: Hearing Loss, Hearing Abnormality
Sponsor: University of Southampton

 
 
 
1 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
Protocol 
 
Study Title: Objective response detection to natural speech stimuli for 
optimisation of hearing aid fitting evaluation 
 
Researcher(s): Dr Frederique J Vanheusden, Dr Katie Ireland, Dr Antonio E Forte, 
Dr James Eaton, Dr Frederic Marmel, Dr Emanuele Perugia, Constantina Georga 
 
Supervisors: Dr Steven Bell, Professor David Simpson, Dr Ben Lineton, Dr Patrick 
Naylor, Dr Tobias Reichenbach and Dr Karolina Kluk-de Kort 
 
Funder: Engineering and Physical Sciences Research Council (EPSRC) 
Sponsor (if known): University of Southampton 
 
 
Background 
 
For people with hearing deficiencies, a specific and well-perceived choice of 
treatment is of necessity to maintain an optimal quality of life. This step is usually 
performed by accurately setting up a hearing aid apparatus based on subjective 
measurements of the hearing impaired’s response (e.g. the participant has to 
provide a physical response when hearing a sound). This assessment cannot be 
performed with hearing impaired people unwilling or incapable of giving a voluntary 
response, such as infants or elderly people. For this reason, a myriad of objective 
assessment tools measuring brain responses to speech stimuli have been proposed 
(Easwar et al., 2015; Hosseini et al., 2015; Jafarpisheh et al., 2015). A comparative 
study on how robust these tools would be for hearing aid optimisation has however 
not yet been performed. Besides this, some tools have only been tested on speech-
like stimuli (Easwar et al., 2015; Jafarpisheh et al., 2015), and it would be of interest 
to assess their applicability to more complex speech stimuli such as sentences or 
running speech (e.g. from an audiobook). This study proposes to compare the 
robustness of objective measurements tools for brain responses to speech on a 
single cohort of hearing impaired volunteers under aided and unaided conditions. 
The study will be performed by measuring brain responses with a multichannel 
electroencephalogram (EEG) system whilst the participant is listening to simple or 
complex speech stimuli. Responses will be measured to clean speech stimuli as well 
as distorted speech stimuli, e.g. speech in noise. Results of this study could shed 
further light on how the brain processes speech and give a clear view on the quality 
of these objective brain response measurements. This could improve the current 
standards for optimally setting up hearing aid devices. 
 
Research questions: 
 
1. Are speech-evoked objective brain responses sensitive to hearing aid 
amplification? 
2. Are speech-evoked brain responses sensitive to distortions in speech and can 
hearing aid amplification resolve issues with brain responses to distorted 
speech? 
3. Which (statistical) tests are optimal for detection of objective brain responses 
to speech?  
4. Are realistic speech stimuli (words or running speech) able to robustly detect 
brain responses compared to current clinical standards (clicks and tones)? 
 

 
 
 
2 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
Method 
 
In this exploratory study focussing on the feasibility of obtaining objective 
measurements of responses to speech under aided and unaided conditions, the 
research questions will be addressed through a series of experiments in a group of 
up to 40 participants (in total) with mild to moderate hearing loss in the better 
hearing ear. Participants will undergo a maximum of two test procedures in which 
they are exposed to speech or speech-like stimuli (clicks, tones, short phonemes, 
words, repeated sentences and/or continuous running speech such as extracts from 
audiobooks). Testing will be performed in a sound treated room. 
 
Materials 
 
Recording of the brain responses will be performed using a 32-channel EEG system 
(ActiveTwo, BioSemi BV, Amsterdam, Netherlands). System components will be 
applied as per the manufacturer’s recommendations to ensure subject safety and 
optimal brain response recording. The system has obtained EU health and safety 
approval to be used for conducting research with human subjects. The 
experimenters will receive prior training from a member of academic staff 
experienced with the equipment and closely related protocols. The EEG system uses 
active electrodes, which means scraping of the scalp is unnecessary. This reduces 
the discomfort for the participants.  
 
Participants 
 
Participants will be 40 native English speaking subjects aged between 18 and 70 
years with mild to moderate hearing loss in their better ear. Participants will be 
recruited from the hearing impaired cohort at the Royal Berkshire NHS Foundation 
Trust. Participants that might be suitable for the study will be identified from their 
medical records. The chief investigator, who is part of their direct healthcare team, 
will approach the potential participant to offer information about the study. If a 
participant shows interest in participating after reading the participant information 
sheet, an audiologist who is part of the participant’s medical team will go through 
the participant information sheet to ensure the participant has no further questions 
and understands the experiment. Informed consent to participate in the study will be 
sought by the audiologist. Participation in the study will be entirely voluntary and 
subjects can be withdrawn at any time without any reason required. Subjects will be 
informed that they are contributing to research and that they will receive standard 
care in accordance with current National Institute for Health and Clinical Excellence 
(NICE) guidelines. Travel reimbursements up to £20 will be made. 
 
Procedure 
 
Experiments will take place at the Royal Berkshire NHS Foundation Trust. As a 
preparation, subjects will be asked to wash their hair. Before the procedure starts, 
the researcher performing the experiment will ask if the participant has read the 
appropriate Participant Information Sheet. Further to this, participants will be asked 
to complete a questionnaire about their hearing function and any history related to 
hearing dysfunction. The questionnaire will be sent along with the Participant 
Information Sheet and will help the researcher to assess if the participant can be 
included in the study and/or if specific precautions have to be made during the 
experiment. In the questionnaire, the participant will be asked for his gender, age, 
native language, current use of medication, presence of neurological diseases, 

 
 
 
3 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
learning deficiencies, a family history of hearing loss and previous musical training 
as previous studies have shown that responses to sound stimuli are affected by these 
characteristics (Hall, 2007). We would also like to ask the participant if he/she hears 
better from a specific ear as taking measurements from the preferred ear could 
potentially optimise the responses to speech (Hall, 2007). We will further ask if the 
participant is aware of skin conditions or allergies as these could increase the chance 
of an allergic reaction to the electode gel applied on the head for performing the EEG 
measurement. In case the participant is aware of severe conditions of the skin or 
allergic reactions, the participant will be withdrawn from the study. In cases of light 
skin diseases or allergic reactions the researcher will check the skin every 15 
minutes to assess if an allergic reaction is occurring. We will ask the participant if 
he/she is suffering from tinnitus (ringing sound in the ear), current ear discomfort or 
infection, or has been exposed to noise in the last 24 hours, as these could influence 
the brain responses or make the test discomfortable for the participant. When 
deemed necessary, the researcher will delay the test to a later date. Lastly, it will be 
asked if the participant had any ear surgery as this could lead to incompatibility with 
the inclusion criteria, and therefore the participant would have to be withdrawn from 
the study.  
 
Informed consent will be taken from the participant when the researcher has assured 
the participant understands the experiment and the participant feels any questions 
related to the study have been answered satisfactorily. 
 
In case their hearing function was not tested within the last 3 months before the first 
experiment, participants will undergo a standard clinical check-up. We will perform 
otoscopy on all participants and test the hearing aid of each of the participants using 
the test battery available at the Royal Berkshire NHS Foundation Trust. The rooms 
will be set up with appropriate temperature and lighting to ensure subject’s comfort. 
In addition, participants will undergo a speech-in-noise test using the Bamford-
Kowal-Bench (B.K.B.) sentence list to obtain a behavioural (subjective) response for 
detection of speech (Bench et al., 1979). Results of this behavioural test will be 
compared to objective responses measured as explained below as part of the answer 
to question 4 of the research questions. 
 
Application of EEG electrodes 
An electrode cap appropriate to the subject’s head size will be placed on the 
participant’s head whilst ensuring correct electrode placement. After applying an 
electrolyte gel (recommended by the EEG system’s manufacturer) to each electrode 
holder on the head cap using a clean gel dispenser, pin-type active electrodes will be 
inserted into these holders. The electrode ribbon cable will be draped over the 
participants shoulder (to avoid damage and trip-hazards from stepping onto it) and 
plugged into the Analog-to-Digital Converter (A-D) box. Additionally, individual flat 
active electrodes will be placed at the forehead and neck to obtain a reference signal. 
Individual flat electrodes for measuring eye movement will also be applied around 
the eyes, and one electrode will be placed under the chin to measure muscular 
activity due to swallowing. Eye and chin electrodes will be used to identify EEG signal 
segments which contain artefact and need to be removed for further analysis. The 
individual electrodes will be plugged into individual sockets available on the EEG A-D 
box.  
 
After checking the impedance of all electrodes are in range of a reference (Common 
Mode Rejection-Driven Right Leg) electrode to ensure participant safety and data 
quality, measurement will start. Participants will be asked to sit in a reclining sofa for 
the measurements and a rolled towel can be provided for neck comfort. Data will be 
recorded with the software provided by the EEG system manufacturer. 

 
 
 
4 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
Stimuli 
 
Types of stimuli 
Participants will be exposed to three types of stimuli during the test: 
 
1. Short stimuli (tones, clicks or phonemes such as /da/ in Cunningham et al., 
2001). These stimuli have been generated for previous research studies at the 
ISVR to elicit brain responses. 
2. Words or Short sentences taken from the B.K.B (Bench et al., 1979), the 
Hearing in Noise Test (HINT, Nilsson et al., 1994) sentence list or the matrix 
test developed at Imperial College London by the research group and based 
on previous research (for review, see Kollmeier et al., 2015). These tests are 
generally used for assessment of speech perception in noise during hearing 
tests. 
3. Segments 
of 
auditory 
recording 
from 
free 
audiobooks 
(found 
on 
https://librivox.org/ or taken from previous studies at the ISVR and the 
Institute of Hearing Research) will be used. Audiobook segments have been 
used in various published research related to speech perception and speech 
intelligibility analysis (Ding and Simon, 2013). 
 
Stimuli will be presented as clear stimuli or degraded by adding noise at different 
signal-to-noise ratios (SNRs), or by applying signal processing methods that emulate 
hearing impairment or speech processing in hearing aids. 
 
Stimulus presentation 
Stimuli will be presented at maximally 80 dB (A) through loudspeakers. According to 
the ISVR Task report on noise and vibration ethics (ISVR Technical Memorandum No. 
808, 1996), sound levels must be less than 76 dB (A) for an 8 hour exposure during 
a 24 hour period, to be considered ‘usual’. Participants will undergo sessions where 
the listening time will be maximally 2 hours at a maximum intensity of 80 dB (A). 
Stimuli can therefore be considered as usual. 
 
Stimulus levels will be controlled using the calibration procedural approach as per 
ISO 389-8:2004. These settings of the calibration signal directly control the level of 
the test stimuli. Calibration will be performed at the beginning of the experimental 
period and every 4 weeks thereafter. 
 
Test procedure 
For this study, participants will be asked to participate in two tests. One of the tests 
will focus on detecting responses to short stimuli (clicks, phonemes and words). A 
second test will focus on detecting responses to more complex stimuli (tones, 
sentences and running speech). Tests will be randomised, and be performed with an 
interval of at least 24 hours between tests. 
 
Test 1: Short stimuli 
 
After participants are sat in the chair and the EEG electrode are fitted, participants 
will be asked to listen to click stimuli played from a loudspeaker placed 1 meter 
directly in front of them. Participants will not wear their hearing aid (unaided 
condition) for this part of the procedure. This part of the study will take about 10 
minutes. Clicks would be considered the gold standard for robustness of response 
detection and used to answer question 4 of the research questions. 
 
After collecting the click stimuli, participants will be asked to listen to repeated word 
stimuli under aided and unaided conditions. Word stimuli have been used in previous 

 
 
 
5 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
experiments with normal hearing participants and showed good detection rate of 
brain responses. Words will be played as normal and distorted words. Distortions will 
include high-pass filtered (frequencies above 1000 Hz only) and low-pass filtered 
(frequencies below 1000 Hz only) versions of the words and words mixed with noise 
at different signal-to-noise ratios. Each participant will be exposed to 6 conditions 
listed below, each lasting 7 minutes, in aided and unaided conditions. This means 
that the total test time will be 12 conditions of 7 minutes, or 84 minutes. Conditions 
will be randomised during the study. Breaks will be offered between the tests, but 
researchers will make participants aware they can ask for a break at any time during 
the procedure. Participants will also be allowed to sleep during this part of the study. 
 
To summarise: the different word conditions are 
1. Clean words in quiet 
2. Clean words in noise (SNR = 0dB) 
3. Clean words in noise (SNR = 3dB) 
4. Low-pass filtered words in quiet (0-1000 Hz) 
5. High-pass filtered words in quiet (>1000 Hz) 
6. High-pass filtered words in quiet (>2000 Hz) 
These will be repeated in aided and unaided conditions. This would help in 
answering questions 1 and 2 of the research questions. 
 
Test 2: Complex stimuli 
 
For this part of the study, participants will be asked to listen to tones and running 
speech presented from a loudspeaker at 1 meter distance directly in front of them. 
Running speech will be a segment of about 20 minutes. Participants will be asked to 
remain attentive during this tests, as we will be measuring cortical responses to 
speech. To ensure this, they will be asked to answer multiple choice questions 
regarding the content of the speech presented at random intervals. Participants will 
perform the experiment twice under aided and unaided conditions. Therefore, at 
total of 80 minutes of EEG signal to running speech will be collected per participant. 
Results would be used to answer question 1 and 2 of the research questions. 
 
Lastly, modulated tones will be presented at frequencies of 1000 Hz under aided and 
unaided conditions at intensities of 40, 60.and 80 dB (A) Tones will be modulated 
with a modulation frequency of 40 Hz for 6 minutes. Total test time over the entire 
test 2 procedure will therefore be 92 minutes. For each of the tones, we will ask the 
participant to rate its loudness on a seven-point scale ranging from very soft to very 
loud (Castro, et al. 2008). This could be used to help answering question 4 of the 
research questions. 
 
End of experiment 
By the end of the recording session all of the equipment, electrodes, insets and cap 
will be removed gently from the participant by the researcher. Equipment will be 
cleaned following the manufacturer’s recommendations. At the end of the 
procedures, the experimenter will give information to the participant about the 
possibility to become part of a database collecting contact information for 
participants for future studies at the Royal Berkshire NHS Foundation Trust or 
University of Southampton. 
 
Data analysis 
Data analysis will be performed off-line. Pre-processing and artefact rejection will be 
performed according to current standards in the research field. Data from tests 
consisting of repeated stimulus sweeps will be averaged over all sweeps to allow 
analysis of brain responses. For all tests, the relationship of the response to the 

 
 
 
6 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
speech stimulus will be performed by comparing the stimulus envelope with spectro-
temporal features in the (averaged) brain response. Further analysis will include 
assessment of coherence and delay measurements between EEG channels during 
brain responses, as well as temporal and spectral assessment tools and source 
separation techniques. 
 
Statistical analysis 
 
Power calculation 
This is a feasibility study, and the research team is only aware of 1 study by Easwar 
et al (Ear and Hearing. 2015) comparing detection success rate and detection time 
for words stimuli during aided and unaided listening conditions is available, with 
only one word stimulus used in this study. The smallest significant difference in 
response amplitude between aided and unaided conditions at conversational level 
was 35 nano-volts (nV) with a within-group standard deviation of 30 nV). Based on 
these findings, the current study would need to include 29 participants for an 
ANOVA study to have a power of 80% at an alpha level of 0.01. From analysis of 
cortical responses to running speech in normal hearing participant, we expect larger 
variability compared to the results found for words, which led us to decide on a 
participant number of 40.  
 
Statistical analysis of data 
Prior to analysing the results, their normality will be tested. This will guide decision 
on using parametric or non-parametric statistical tests. Temporal relations between 
responses will be quantified with correlation analysis. Robustness of responses to 
noise level will be assessed with an F-ratio test. Statistical significant relationships 
between EEG channels and the stimulus will be tested using analysis of variance 
measures. Statistical signal processing methods (including bootstrap methods and 
confidence limits) will be used in detecting responses. 
 
References 
 
Bench, J., Kowal, A., Bamford, J. 1979. The BKB (Bamford-Kowal-Bench) Sentence Lists for 
Partially-Hearing Children, British Journal of Audiology, 13:3; 108-112. 
 
Castro, F.Z., Barajas de Prat, J.J., Zabala, E.L. 2008. Loudness and auditory steady-state 
responses in normal-hearing subjects. International Journal of Audiology, 47:5, 269-275. 
 
Cunningham, J., Nicol, T., Zecker, S., et al. 2001. Neurobiologic Responses to Speech in Noise 
in Children with Learning Problems: Deficits and Strategies for Improvement, Clinical 
Neurophysiology, 112:5; 758-767. 
 
Ding, N., Simon, J.Z. 2013. Adaptive temporal encoding leads to a background-insensitive 
cortical representation of speech. Journal of Neuroscience, 33:5728-5735. 
 
Easwar, V., Purcell, D., Aiken, S., et al. 2015. Evaluation of Speech-Evoked Envelope Following 
Responses as an Objective Aided Outcome Measure: Effect of Stimulus Level, Bandwidth, and 
Amplification in Adults With Hearing Loss, Ear and Hearing, 36:6; 635-652. 
 
Hosseini, P., Bell, S., Wang, S., et al. 2015. Induced Activity in EEG in Response to Auditory 
Stimulation, Biomedical Signal Processing and Control, 22;31-43. 
 
Jafarpisheh, A., Jafari, A., Abolhassani, M., et al. 2016. Nonlinear Feature Extraction for 
Objective Classification of Complex Auditory Brainstem Responses to Diotic Perceptually 
Critical Consonant-Vowel Syllables, Auris Nasus Larynx, 46:1; 37-44. 
 

 
 
 
7 
14/08/2017 Version 1.0 
 
ERGO Reference: 30123 
 
Nilsson, M., Soli, S.D., Sullivan, J.A. 1994. Development of the Hearing in Noise Test for the 
measurement of speech reception thresholds in quiet and in noise, Journal of the Acoustical 
Society of America, 95:2; 1085-1099. 
 
Kollmeier B., Warzybok A., Hochmuth S., et al. 2015. The multilingual matrix test: Principles, 
applications, and comparison across languages: A review. International Journal of Audiology, 
54:s2; 3-16. 
 
Hall, JW. 2007. The new handbook of auditory evoked potentials. Pearson.