PROCEEDINGS of the
23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

Does the method matter? A review of the main testing methods forthe subjective evaluation of room acoustics through listening tests
Daniel DE LA PRIDA1; Antonio PEDRERO1; Luis A. AZPICUETA-RUIZ2; María A.
NAVACERRADA1; César DÍAZ1;
1

Grupo de Investigación en Acústica Arquitectónica, Universidad Politécnica de Madrid, 28040 Madrid, Spain

2

Department of Signal Theory and Communications, Universidad Carlos III de Madrid, 28911 Leganés, Spain

ABSTRACTThe listening tests are experimental procedures in which a group of participants are asked to express theiropinion about certain questions regarding some stimuli that they have been urged to listen to. The listening
tests have proved their effectiveness as methods for the subjective assessment of perception in different fieldsof acoustics, and particularly in room acoustics and sound insulation. The way in which the listening test is
conducted, how the participant's response is expressed and how the stimuli are presented to the participants
define what is known as the testing/query method. There is not yet a common methodological framework thatdefines which testing method is the best for each purpose and under what conditions the experiments are to be
carried out. The purpose of this communication is to review the most commonly used testing methods in thesubjective perception of sound and to present the advantages and disadvantages of each of them when they
are used to evaluate the subjective perception of a sample of population.
Keywords: listening test, room acoustics

1. INTRODUCTION
Sensory evaluation tests allow to obtain the subjective perception of human beings regarding
certain stimuli that are presented to them. This type of test has been found to be very useful in thecommercial sector, when aiming to improve, consolidate or renew products, so that the population
better perceives them. They are also used to assess how reducing costs on certain materials orprocesses affects the perceived quality of the final product. In this sense, these types of tests are used
regularly in the fields of food (1), fragrances (2) and design (3), among others.
Although the use of subjective tests is not new in the field of acoustics, this kind of testing has
gained great importance in recent decades. A paradigm shift in acoustics is taking place, centred on thefact that the objective descriptors used to assess acoustic quality should be more representative of thesubjective perception of human beings.
In particular, in the field of environmental acoustics, concepts such as "soundscape" (4) andpsychoacoustic parameters have taken on an important relevance in re cent years. Listening tests arebeing used to evaluate how human beings perceive soundscapes and how this perception can affecttheir actions (5,6).
In the field of building acoustics, the most recent research related to the assessment of soundinsulation is also based on listening tests. These tests are mainly focused on evaluating the degree ofagreement between the SNQs (Single Number Quantities), commonly used to assess the protection
contributed by a sound insulation element, and the subjective perception of loudness and annoyance(7,8).
12

arquilav.arquitectura@upm.es
lazpicueta@tsc.uc3m.es

7871

In the field of room acoustics, in recent years, listening tests have also gained great interest. They
are mainly aimed to determine the just noticeable differences (JND) of the room acoustic descriptors
that are frequently used to evaluate the acoustic quality of venues. Among these descriptors
reverberation time, clarity or definition, as well as spatial parameters such as apparent source width(ASW) or listener envelopment (LEV) are of great interest (9,10).
Finally, since the term auralization was coined and, in particular, since this term is supported by therise of new virtual reality applications, listening tests are of great importance in this field. Main
research is related with the model calibration requirements as well as the precision of the simulation
software, models and HRTFs (Head-Related Transfer Functions). The common aim is that differences
between real situations and simulations are as small as possible and that auralizations are always
perceived as plausible (11–13).
In other completely different fields, such as food research, great efforts are being made to develop
subjective tests that are as reproducible as possible (14,15). Therefore, numerous studies have been
conducted in the last years in order to determine the best testing method and design of the experiment,so that the tests are comparable and accurate. In the field of acoustics, although numerous listening
tests are being performed, there are not yet clear methodologies for their design and performance. This
fact can make the comparison between the results of different investigations difficult and tedious.
The purpose of this communication is to review the listening tests that have been recently
undertaken in the field of room acoustics, in order to assess to what extent, the different works arefollowing similar or different methodologies in each of the stages of the process. The main differences
between researches are discussed and certain conclusions are drawn, which may be useful for
improving the subjective evaluation in the field of room acoustics.

2. METHODOLOGY
The methodology followed in this review was divided into several steps: First, a detailed search forreferences in the fields of study was performed. Then, the references found were filtered in order toselect only those that were relevant. Finally, a series of common concepts were defined and collectedfrom each reference. Each of these steps is described individually below:
2.1 Literature search
In order to carry out this review, an extensive search has been carried out for the most recent
bibliographical references (mostly between 2009 and 2017) in the field of subjective assessment ofroom acoustics descriptors, although some older references (between 1993 and 2005) have also been
considered due to their relevance in the field under study. This search for the state of the art was done
during the months of December 2018 and February 2019, although some of the references wereoriginally collected in August 2017.
In order to perform this search, scientific databases have been used, as well as scientific social
networks and specific search engines. In particular, most of the references have been found through
Google Scholar, ScienceDirect and ResearchGate. Different procedures were followed for the search:
1. Search by keywords: This was the main search method. Scientific search engines and databases
were used for keywords such as "perception", "perceptual", "subjective", "room acoustics","reverberation time", "clarity", "JND", "just-noticeable-difference" or "virtual”.
2. Search by affinity: Databases such as ScienceDirect suggest similar references to the lastconsulted by affinity between concepts.
3. Search by references: Consultation of the bibliography of the most recent articles allows, onsome occasions, to obtain references to previous related articles.
4. Search by author: Some authors focus all their research activity on the same field, so that onsome occasions several articles by the same researcher were found in the field of study.
2.2 Filtering of the literature
From the conducted search, 15 references were obtained (9,10,16–28), which were selected, in a7872

preliminary way, only by reviewing the title or the abstract. After completing the search tasks, a deeper
filtering of the articles was done in order to use only those that were directly related to the purpose of
the review. Therefore, only journal and conference papers, in which a subjective study was performedby means of listening tests and in which the methodological design and the analysis of the listening test
results had an important contribution were used. This task reduced the number of papers to 8
(9,10,16,20,22,23,25,28).
2.3 Definition of common concepts
As already mentioned in the introduction, listening tests are tools for the subjective
characterization of sound in which a series of participants are presented with various stimuli and theirjudgement about these stimuli is collected through some system. Since a listening test is a tool thatinteracts with the human being, there are several sources of bias that can reduce the relevance of theexperiment or even lead to erroneous conclusions. Therefore, when reviewing listening test studies, it
is important to evaluate concepts such as the following:
1. Purpose of the experiment: What is the experiment intended to study?
2. Testing method: How is the consultation and collection of results addressed?
3. Selection and preparation of stimuli: What sound samples were used? How were they
processed?
4. Method of analysis: Which statistical method was employed for the analysis of the results?
5. Quality of the design: Is the performed design robust? How many participants did theexperiment have? Is it possible to identify design actions that were presumably aimed to reduce bias?
For the sake of simplicity, the particular results of each of these studies are not taken into account,as this is not within the scope of this review, which relates to the design of the listening test methods.

3. RESULTS
In this section, each of the eight articles presented in the previous section is described in detail,considering each of the five common concepts. A summary of this analysis is shown in Table 1.
In reference (22), the purpose of the research was to measure the smallest perceivable change in theobjective room acoustic parameters associated with spatial impression and clarity. For that purpose,the difference limens for ELEF (early lateral energy fraction), IACC (inter-aural cross correlation), Ts(centre time) and C80(clarity) were addressed. In order to find the difference limens, the “Method of
Minimal Changes” was used with 7 to 10 participants. The experiment was conducted in an anechoic
environment where simulated variations of a concert hall were played through a loudspeaker system ,comprising delays, effects and attenuators. Two sound samples were used for the listening test, a 8s.
excerpt of “Handel’s Watermusic” and a 5s. excerpt of “Mendelsohn’s Symphony no.3 Opus 56 The
Scottish”. The analysis of the results was mainly based on the calculation of the difference limens as
the smallest perceivable changes that are perceived by at least 50% of the participants, which is theregular method of analysis for a Minimal Changes testing method. Confidence intervals at 95% werealso given. Presumably, a couple of actions were aimed to the reduction of bias. First, only normal
hearing participants could perform the test, even though their hearing was not tested. Fatigue effects
were checked through F-test analysis.
The suitability of various objective measures as predictors of subjective ratings was examinedin(23). In particular the relation between the EDT (early decay time), Ts, C80, G (loudness), early bass
level and treble ratio, and the subjective perception was addressed. For that purpose, a sample of 10
subjects performed 8 different listening tests ranking several sound fields in terms of loudness, clarity,
reverberance, envelopment, apparent source width and overall preference. The testing method wasbased on double blind paired comparisons (attribute-related 2-AFC). The sound fields were generated
through near-field loudspeakers, equipped with mechanical barriers to avoid crosstalk. Just one soundsample was used during the listening test, a 15s. passage of “Mozart’s La Nozze di Figaro”. The
analysis of the results was based on the regression analysis between the preference counts of thesubjective perception attributes and the room acoustic objective parameters described. Regarding bias
control, it is said that being a difference test, any imperfection in the reproduction chain would be
minimized. A description of the parameters to evaluate was given before the test and a brief training7873

sequence performed.
The purpose of (9) was to find the just noticeable difference for C50. For that purpose simulated
sound fields representing realistic situations were played through a 8 -loudspeaker system, similar tothat of (22), in an anechoic environment. 10 participants took part in a listening test, which consisted
in deciding whether there was difference between pairs of sound fields, with step changes in clarity of
0.5, 1.0, 1.5, 2.0, 2.5 and 4 dB. This test was repeated for three different sound fields with different
reverberation times. A sound sample was played through the loudspeakers to allow the participants toperform the test, an anechoic male modified rhyme test at 3.0 syllables per second. To determine the
JND for C50, the percentage of participants that perceived a difference was accounted for each clarity
step, similar to the procedure described in (22). Also, regression was calculated between the subjective
results and the reverberation time, to evaluate the change in perceived clarity with reverberation ti me.
Two actions can be found, which presumably are aimed to reduce bias. First, the pairs of sounds wererandomized. Also, a training was given before the test, in which the participants tested the extreme
cases.
In (25), an experiment was developed to examine the discrimination threshold for the
direct-to-reverberant energy ratio (v). For that purpose, 6 participants took part in a test following an
adaptive 3-down 1-up two-alternative forced choice procedure through headphones. The
individualized BRIR (binaural room impulse response) was measured for 5 out of the 6 participants
and the different stimuli were obtained by scaling the reverberant energy of the BRIR. Four soundsamples were convolved with the BRIR: a short duration noise burst of 50 ms with 2.5 ms onset/offset
times, a long duration burst of 300 ms with 150 onset/offset times, a male speech sample of syllable/da/ of 300 ms and an impulsive signal, which was the BRIR itself. Several trials were repeated before
the procedure stopped. Runs terminated after 20 reversals and the thresholds were determined as theaverage of the last 5 reversals. Also, psychometric functions were fitted for each stimulus andevaluated by Chi-square test. The reduction of bias was presumably the aim of different actions, suchas the performance of an audiometry for each participant, the individualized measurement of the
BRIRs as well as the individualized headphone equalization (HpTF).
A research was performed in (10), with the aim of determining the difference limen for single
number parameters relating the ASW and LEV. For that purpose, the BRIRs of different venues were
measured with a dummy head in positions with different LF (lateral energy fraction) and LG (lateralsound level). These BRIRs were later convoluted with a sound sample and presented to the participants
through a crosstalk cancellation loudspeaker system. The sound sample was a 14s. excerpt of “Haydn’s
Trumpet Concerto played with an E flat Cornet”. Different participants samples were used inindividual listening tests for LF changes (12 participants) and LG changes (7 participants). The testing
method was similar to the others, were the participants listened to stimuli in pairs and had to make a
judgment about the stimuli being the same or different. From the results of the listening test , thedifference limens were calculated by means of the fitting of the psychometric function a Monte Carlosimulation for the goodness of fit. Several actions could be found, which presumably aimed to reducebias. In particular, the participants might not repeat the hearing of the samples. That way, it was
assured that identical audio presentation is given to each participant. Also, all participants received an
audiometry, to check the hearing. A training and familiarization process was performed by theparticipants prior to the test, where extreme cases could be listened. Instructions were given by meansof a written guide. The experimenter collected data about the difficulty perce ived by the participants.
The approach of (16) was to investigate the possible dependence of the JND of the Ts and C80 as afunction of the reverberation time in large reverberant enclosures. Impulse responses were measured in
B-format and later varied in clarity parameters. These impulse responses were then convolved with
two sound samples and played in a silent room through four loudspeakers. A 5s. excerpt of “Pange
Lingua” and a 9s. excerpt from “Weber’s Theme for Cello” were used. The “Method of Minimal
Changes” was then used, in which around 40 participants were presented with two stimuli and had todecide whether they were the same or different. If the participants identified the stimuli as different,
they had also to report which was the clearer. As the “Method of Minimal Changes” was employed, theJND was defined as the smallest perceivable change detected by 50% of the participants. Regarding
bias, the participants were not test for hearing but asked about hearing issues. Sound samples were7874

presented randomly. To avoid any learning or memory effect, listening test for each stimulus werecarried in different days. Participants were asked about the design of the experiment and some reported
difficulties regarding the concept of clarity, with graphical interface and with failing to find an
effective listening method.
The determination of the necessary change of the primary cues, such as ITD (interaural timedifference), ILD (interaural level difference) and IACC for subjects to detect them was investigated in(20) . For that purpose, the ITD and ILD of the early part of measured BRIRs, corresponding to threedifferent environments, were manipulated. Eight participants conducted an adaptive three-interval
2-AFC test, following a 1-up 2-down staircase procedure. A 2s. excerpt of three different soundsamples was randomly selected. Sound samples were recordings of a guitar, a violin and snare drum.
The results were presented as mean values and standard errors of detection for the different rooms andinstruments. Regarding bias control, each test is repeated four times. However, th ere is no mention in
the article regarding training or familiarization processes, the environment, equipment or equalizationof the playback chain.
Finally, in (28), the focus was on the evaluation of the JND of C80 and the assessment of whether
a variation in the way the stimuli are presented to the participants gave the same or different results for
the JND. For that purpose, several simulated impulses responses were used, and sound samples
convoluted with these IR (impulse responses) used as stimuli. The stimuli were presented to theparticipants, in an anechoic environment, through a 8-channel loudspeaker system. 51 participants
were presented with the first variant of the test, where the participants were presented with stimuli inpairs and had to listening to both stimuli, in order and in full, and judge whether the they were the sameor not. They might listen to the stimuli as much as they wanted but they had to listen both stimuli,
completely and following a fixed order, each time. 11 subjects were presented with the second variantof the test, where the participants might switch between stimuli during the reproduction of the soundsample as much as they wanted and didn’t have to listen to the stimuli in full. Three different soundsamples were used for the first listening test. A 10.9s. passage of the “Third movement of Bizet’s
L’Arlésienne Suite No. 2”, a 10.3s. excerpt of “Weber’s Theme” and a 10.3s. motif of the beginning of“Handel’s Water Music”. For the second test, the Bizet piece was used again. In addition, a sample
which length was not reported was used of the “Debussy’s Prelude to the Afternoon of the Faun”. The
analysis of the results followed the same as for the method of Minimal Changes. Regarding the controlof bias, the impulses responses and sound samples were randomized. Training was given prior to thelistening test, with samples and impulses responses not used later during the test. Hearing level wasmeasured for each participant.

4. DISCUSSION
Several research articles are presented, in which the main purpose is the study of the JND for
several room acoustic parameters. Even though some trends can be found regarding the testing method
used for the presentation of the stimuli and consultation of the participants, three main different
methods are identified; the Method of Minimal Changes, the attribute-related 2-AFC and adaptive
up-down approaches based on non-attributive 2-AFC procedures. In particular, the third is a more
precise and efficient method of the first. Both the first and the third are based on a non-attributive
2-AFC also called same-different method. Being this method prone to tendency bias among others (29),
it is considered that further research is necessary, to find newer testing methods that are more efficientand less prone to bias. Not only how the participants are consulted but also how the participantsinteract with the stimuli should be considered for deeper analysis. It can be found that some works(9,16,23) prefer to leave the participants freedom of choice, allowing them to control the reproduction,
while others prefer to fix how the stimuli are presented (10,25). Specially interesting are the results of(28), where performing the listening test with different procedures regarding this matter reported
different results.

7875

Table 1 – Summary of common concepts for all the reviewed articles
Reference

(22)

(23)

(9)

(25)

(10)

(16)

(20)

(28)

Purpose of the

ELEF, IACC,

EDT, Ts, C80, G,

study

Ts, C80

treble, bass

C50

v

ASW, LEV

Ts, C80

ITD, ILD, IACC

C80

Participants

7 - 1010

10

6

7 - 1240

8

11 - 51

Testing method

Minimal Changes

Attribute 2-AFC

Same - different

Adaptive 2-AFC

Same - different

Minimal Changes

Adaptive
3I/2-AFC

Same-different
S1. L’Arlésienne

S1. Water Music

S1. Burst - 50 ms

Stimuli and– 8s

S1. Nozze Figaro

S1. Male rhyme

S2. Burst - 300 ms

duration

S2. The Scottish -– 15stest - Loop

S3. /da/ - 300 ms

5s

S4. Impulse

S1. Trumpet
concerto Cornet– 15s

S1. Pange Lingua

S1. Guitar – 2s

– 5s

S2. Violin - 2s

S2. Theme for

S3. Snare drum –Cello – 9s

2s

Suite n.2 – 10.9s
S2. Theme for
Cello – 10.3s
S3. Water Music– 10.3s
S4. Prelude…

Analysis method

Diff. limen 50%
F-test analysis

Regression% difference andPsychometric

regressionfunction.
Chi-square fit test

Diff. limen 50%

Mean andfunction. Monte

Psychometricstandard errors of

Carlo fit

function.

detection

Diff. limen 50%
Regression

Playback system

8 Loudspeakers

2 Loudspeakers

8 Loudspeakers

Headphones

2 Loudspeakers

4 Loudspeakers

Not mentioned

8 Loudspeakers

Training

Not mentioned

Yes

Yes

Not mentioned

Yes

Yes

Not mentioned

Yes

Playback method

Not mentioned

Free switching

Free switching

Fixed

Fixed

Free switching

Not mentioned

Both (fixed/free)

Audiometry

No

No

No

Yes

Yes

No

Not mentioned

Yes

WrittenRandom order of

Individual BRIR

WrittenParticipant

Random choice

Random order of

instructions

stimuliand HpTF

instructions

feedbackof sound sample

stimuli

Additional bias
control

7876

Psychometric-Regarding the sound samples, each research uses different samples and different durations. Even
though in some occasions the papers justify why certain sound samples are selected, no clear
background is given sustaining the selection. It is not clear for the authors whether unified soundsamples should be described for all listening test regarding room acoustics. However, it is considerednecessary that further deeper and detailed research is performed to evaluate the importance of thesound samples and their duration in the subjective perception of room acoustics. It is also considered,that the excerpts used by published works should be available for other researchers, so that they canused them for reproducible research.
Regarding the analysis of the results, a base descriptor should be selected among existing methodsof analysis or research should be done to describe a new descriptor that can be extracted,
independently of the nature of the difference and the purpose of the experiment that can make theresults of different researches comparable.
Bias is an important factor that must be kept under control when performing subjective tests. Somesources of bias are related with the testing method and might be reduced by using better testing
methods. However, other sources of bias can be related to the design of the experiment, the interaction
between the participants and the experimenter, the participants themselves and the environment inwhich the listening test is to be performed. In this review, several actions that are presumably aimed tocontrol and reduce bias could be found, which was not applied by all the works. The last four rows oftable 1 exemplify this fact easily. In those cases where a mention was made regarding the training andfamiliarization of the participants, the playback method, the hearing level audiometry and playback
chain equalization, different procedures were followed. More explanations should be given in thepapers about the background sustaining the taken actions and further analysis must be done in thosecases where no relevant background can be given.
In summary, citing Weinzierl and Vorländer (30) “In room acoustics, we look back on more than 50
years of research on developing psychological measuring instruments for the concept of ‘room
acoustical impression’, and on more than 100 years of research on the development of physicalmeasures which could serve as technical predictors for these perceptual qualities. On both sides, thestate of the art is surprisingly unsatisfactory.”. Therefore research in the field of subjective perceptionof room acoustics should not only focus on the application of listening tests to assess perceptible
differences, but also on the evaluation, understanding and improvement of methods, so that clear
methodologies can be described, which are followed by all research es in this field. In other words, it
must be pursued that the study of the subjective perception of room acoustics is carrie d out throughreproducible research.

REFERENCES
1. Rousseau B, Rogeaux M, O’Mahony M. Mustard discrimination by same–different and triangle tests:
aspects of irritation, memory and τ criteria. Food Qual Prefer. 1999;10(3):173–84.
2. Ward P, Davies BJ, Kooijman D. Olfaction and the retail environment: examining the influence of
ambient scent. Serv Bus. 2007;1(4):295–316.
3. Seckler M, Opwis K, Tuch AN. Linking objective design factors with subjective aesthetics: An
experimental study on how structure and color of websites affect the facets of users’ visual aesthetic
perception. Comput Human Behav. 2015;49:375–89.
4. Kang J, Schulte-Fortkamp B. Soundscape and the Built Environment. CRC press; 2016.
5. de la Prida D, Pedrero A, Navacerrada MÁ, Díaz C. Relationship between the geometric profile of thecity and the subjective perception of urban soundscapes. Appl Acoust [Internet]. 2019 Jun;149:74–84.
Available from: https://linkinghub.elsevier.com/retrieve/pii/S0003682X18308168
6. Guastavino C, Katz BFG, Polack J-D, Levitin DJ, Dubois D. Ecological validity of soundscape
reproduction. Acta Acust united with Acust. 2005;91(2):333–41.
7. Hongisto V, Oliva D, Rekola L. Subjective and objective rating of the sound insulation of residentialbuilding façades against road traffic noise. J Acoust Soc Am. 2018;144(2):1100–12.
8. Monteiro C, Machimbarrena M, de la Prida D, Rychtarikova M. Subjective and objective acoustic
performance ranking of heavy and light weight walls. Appl Acoust [Internet]. 2016 Sep [cited 2016 Apr13];110:268–79.
Available
from:

7877

http://www.sciencedirect.com/science/article/pii/S0003682X1630041X
9. Bradley JS, Reich R, Norcross SG. A just noticeable difference in C50 for speech. Appl Acoust.
1999;58(2):99–108.
10. Witew IB, Behler GK, Vorländer M. About just noticeable differences for aspects of spatial impressions
in concert halls. Acoust Sci Technol. 2005;26(2):185–92.
11. Lindau A, Weinzierl S. Assessing the plausibility of virtual acoustic environments. Acta Acust united
with Acust. 2012;
12. Postma BNJ, Katz BFG. Perceptive and objective evaluation of calibrated room acoustic simulation
auralizations. J Acoust Soc Am. 2016;
13. Rychtáriková M, Chmelík V, Roozen NB, Glorieux C. Front-back localization in simulated rectangularrooms. Appl Acoust. 2015;14. Stocks MA, van Hout D, Hautus MJ. Cognitive decision strategies adopted by trained judges inreminder difference tests when tasting yoghurt, mayonnaise, and ice tea. Food Qual Prefer. 2014;34:14–
23.
15. O’Mahony M. Sensory difference and preference testing: the use of signal detection measures. In:
Applied Sensory Analy of Foods. Routledge; 2017. p. 145–75.
16. Martellotta F. The just noticeable difference of center time and clarity index in large reverberant spaces.
J Acoust Soc Am. 2010;128(2):654–63.
17. Bradley JS. Review of objective room acoustics measures and future needs. Applied Acoustics. 2011.
18. Lokki T, Pätynen J, Kuusinen A, Tervo S. Disentangling preference ratings of concert hall acoustics
using subjective sensory profiles. J Acoust Soc Am. 2012;
19. Klockgether S, van Dorp Schuitman J, Van de Par S. Perceptual limits for detecting interaural-cue
manipulations measured in reverberant settings. In: Proceedings of Meetings on Acoustics ICA2013.
ASA; 2013. p. 15004.
20. Klockgether S, van de Par S. Just noticeable differences of spatial cues in echoic and anechoic
acoustical environments. J Acoust Soc Am. 2016;140(4):EL352–7.
21. Klumpp RG, Eady HR. Some measurements of interaural time difference thresholds. J Acoust Soc Am.
1956;28(5):859–60.
22. Cox T, Davies W, Lam Y. The sensitivity of listeners to early sound field changes in auditoriums.
Acustica. 1993;
23. Soulodre GA, Bradley JS. Subjective evaluation of new room acoustic measures. J Acoust Soc Am.
1995;
24. Bradley JS, Reich RD, Norcross SG. On the combined effects of signal-to-noise ratio and room
acoustics on speech intelligibility. J Acoust Soc Am. 1999;106(4):1820–8.
25. Zahorik P. Direct-to-reverberant energy ratio sensitivity. J Acoust Soc Am. 2002;
26. Okano T. Judgments of noticeable differences in sound fields of concert halls caused by intensity
variations in early reflections. J Acoust Soc Am. 2002;111(1):217–29.
27. Bradley JS. Using ISO 3382 measures, and their extensions, to evaluate acoustical conditions in concert
halls. Acoust Sci Technol. 2005;
28. Vigeant MC, Celmer RD. Effect of Experimental Design on the Results of Clarity-Index
Just-Noticeable-Difference Listening Tests. In: Proceedings of 20th International Congress on Acoustics,
ICA 2010, Paper. 2010.
29. Prins N. Psychophysics: a practical introduction. Academic Press; 2016.
30. Weinzierl S, Vorländer M. Room acoustical parameters as predictors of room acoustical impression:
What do we know and what would we like to know? Acoust Aust. 2015;43(1):41–8.

7878