PROCEEDINGS of the
23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

Questions applied in audio and visual environment assessment
Jan FELCYN1, 2; Anna PREIS2; Marcin PRASZKOWSKI2, Małgorzata WRZOSEK3
2

Institute of Acoustics, Adam Mickiewicz University, Poznań, Poland
3

Szczecin University, Szczecin, Poland

ABSTRACTThe aim of the study was to examine how the way in which a question regarding audio, visual andaudio-visual stimuli is formulated could influence environment assessment. To get quantitative ratings inboth modalities, a numerical scale coupled with precisely formulated questions was used for all theinvestigated audio, visual, and audio-visual stimuli. The participants were asked either about the comfort or
the discomfort caused by the perceived stimuli. The assessments of audio and visual comfort and discomfort
were carried out in a laboratory settings. In all cases an 11-point numerical scale (from 0 to 10, derived fromICBEN recommendations) was used to rank the comfort and discomfort ratings. The results show that there
are no statistically significant differences between the comfort and discomfort assessment of visual samples,with one exception. Actually, the comfort and discomfort ratings are equivalent in such a way that a
discomfort rating can be represented as the opposite of the comfort rating, i.e. discomfort rating is equal to 10– the comfort rating. For the audio stimuli, whether the comfort and the discomfort scales are reversible
depends on both the sound level at which the stimuli were presented and on the type of stimulus.
Keywords: Environment assessment, noise annoyance, audio-visual interaction

1. INTRODUCTION
In the modern world, people are surrounded by sounds. Some of them could be harmful andinfluence human health. As (1, 2) have shown, noise evokes annoyance and disturbs sleep; it can also
increase the occurrence of hypertension or cardiovascular diseases.
As this relation between noise and health has been observed from many years, some law regulations
were made to protect people from exposure to loud noises. One of the most important regulations is the
EU directive from 2002 (3) which obliges cities with more than 100 000 inhabitants to draw noisemaps of the area and provide action plans in places where noise limits are exceeded.
However, as many papers have proved (4), no more than 1/3 of the variance in noise annoyanceassessments made by people could be explained by values of noise factors. That is why there is a strong
tendency to not rely only on them, but also to take into account other non -acoustical factors.
Nevertheless, this research needs people to listen to and rate the annoyance they experience in
response to different noises.
At this point it is worth describing sounds around us not only as noises and single pass -bys, but, in
as more complex manner, as soundscapes. A soundscape is strongly related to human perception andshould be perceived in the context of a particular time, place and activity (5). Soundscapes and theirperception is a matter of interest to many researchers (6, 7). In recent years, the notion of thesoundscape has become one of the crucial factors in the planning and modernization of modern cities(8–13).
With every passing year it is becoming increasingly clear that soundscapes cannot be rated only
through the hearing process. People in an environment perceive it as a whole, not only with their ears
but also with other senses, particularly sight. On the basis of information from every modality humans
can experience their environment most effectively (14). On the other hand, more modalities lead tomore sophisticated research plans and methods which have to be carefully controlled, as someinteractions can be observed between the senses (15, 16).
Some experiments were conducted to evaluate sound in relation to sight (17–19). Measurement of
the relationship between audio and visual components in environment assessment usually requires
1

7932

janaku@amu.edu.plquantitative scales for both modalities. To get quantitative ratings in both modalities, a numerical scale
coupled with precisely formulated questions should be used (20, 21). It is a matter of discussion
exactly what questions should be applied in the case of soundscape or landscape evaluation (22, 23).
The next problem is whether the scale/question which is appropriate for soundscape as sessment is also
suitable as a tool for landscape assessment. For example, one can assume that it is more natural to ask
about the discomfort associated with a soundscape, than to ask about the discomfort associated with alandscape. In the case of a landscape case, questions about the comfort associated with a given
landscape seem more natural than those about discomfort. Finally, how to compare audio and visualcomponents in total environment assessment when two different questions/scales are used?
We assumed that soundscape and landscape could be rated on two opposite scales: discomfort forsoundscape and comfort for landscape assessment. These scales could be reversible, as discomfort canbe expressed by 10-comfort. To our knowledge, it is the first trial to use both scales and check theirconvertibility in the case of audio-visual environment assessment.
The article is organized as follows: first we define the aim of the study, next the method,
participants, stimuli and procedure used in the experiments are described, then the results of the two
experiments are shown and discussed. Finally, we provide our conclusions.

2. MATERIAL AND METHODS
2.1 Aim
The hypothesis tested in the study is: the comfort and discomfort ratings are equivalent in such a
way that a comfort rating can be represented as the opposite of the discomfort rating, i.e. discomfort
rating is equal to 10-the comfort rating. This hypothesis was tested in a laboratory setting, in four
different assessment conditions: (a) audio samples only, (b) visual samples only, (c) consistentaudio-visual samples, and (d) non-consistent audio-visual samples. In all cases an 11-point numerical
ICBEN (20, 21) scale (from 0 to 10) was used to rank the comfort and discomfort. Both comfort anddiscomfort assessments were collected in the same experiments. The comfort ratings have alreadybeen used in our previous publication (24). The discomfort data is the new data. This means, however,that the participants and the stimuli are the same as in the previous and the present studies. The comfort
data published previously are used here only as reference data to compare the ratings as they are, andin order to test the hypothesis of this study.
2.2 ParticipantsTwo groups of participants took part in two experiments. Seventeen university students, comprising
8 males and 9 females, aged between 22 and 30, took part in Experiment I. Fifteen participants,
comprising 9 males and 5 females, aged between 22 and 26, took part in Experiment II. All had normal
hearing (inclusion criterion was 15 dB maximum allowable hearing loss at hearing threshold) .
2.3 Stimuli
The audio-visual samples of seven different city places (Fig. 1), each of 10 -minute duration wererecorded using a Sony HDR-XR200 camera and a TEDS 4101 binaural microphone, together with a
Bruel & Kjaer PULSE v.12.6.0.255 system. From these recordings, audio-visual samples, each of 10s
duration were created. To test the possible influence of the sound level on the environment comfort anddiscomfort assessments, additional soundscape samples were created with the sound level value lower(−6 dB) and higher (+6 dB) than the original recorded sound level.

MarketBoardwalkBusy Street

7933

Park Near Busy Street

Park

Roundabout

Side Street
Figure 1. Snapshots from seven different places within Poznań area where audio and video samples werecollected.
The audio-visual samples were created with the help of Adobe Premiere Pro CS5 software. In thisway, 21 soundscape samples (7 points × 3 sound level) and a combination of 21 (7 points × 3 soundlevel) original audio-visual samples were used in Experiment I. Each soundscape and audio -visualsample was presented three times. All together 126 different stimuli (42 audio and audio -visualsamples × 3 repetition) were assessed by the participants of Experiment I. In E xperiment II, thecomfort and discomfort caused by 7 different “landscape samples only” and 21 mixed combinations ofaudio-visual samples at three different sound levels, were assessed. In this case each sample was alsopresented three times. All together 210 stimuli (7 points plus 21 × 3 sound levels, all presented threetimes) were assessed by the participants of Experiment II.
2.4 Procedure
The audio and audio-visual samples were presented to the participants of Experiment I. They wereasked to rate the degree of comfort and discomfort on a numerical scale from 0 to 10 while imagining
they were in such an environment. The participants were sitting in front of a computer screen (PCcomputer with 17’ LCD screen), watching and listening (Beyerdynamic DT-150 headphones) to theaudio and audio-visual samples. The assessments of comfort and discomfort caused by the recorded
samples were carried out in two different conditions: (a) audio samples only, (b) original audio -visualsamples. Experiment II used the same setup, this time participants assessed the comfort and discomfort
created by recorded samples in the other two conditions: (c) the video samples only, and (d)non-consistent combinations of audio-visual samples.
The instruction given to the participants in Experiment I was as follows: “Soundscape andaudio-visual samples coming from different places in the city of Poznań will be presented to you.
Imagine that you are in this place and would like to relax. Assess your comfort/discomfort feelingusing the scale from 0 to 10, where 0 means total lack of comfort/discomfort, 10 means total
comfort/discomfort. Please concentrate on the positive/negative feelings connected to a given
environment”.
In Experiment II the same instruction was given to the participants, with a change at the beginning.
The words “Soundscape and audio-visual sample (…)” were replaced by “Landscape andaudio-visual samples (…)”. This means that in Experiment II the comfort/discomfort assessment of
the investigated places was based on visual and non-consistent audio-visual samples. In both7934

experiments each sample was presented three times to each participant.

3. THE RESULTS OF THE COMPARISON OF DISCOMFORT AND 10-COMFORTRATINGS FOR SOUNDSCAPE AND LANDSCAPE ASSESSMENTS
3.1 The results for soundscapes only (Ex. I) and landscapes only (Ex. II)
In order to test if the discomfort and comfort ratings are equivalent in such a way that a discomfort
rating can be represented as the opposite of the comfort rating, the comfort data from our previous
experiment were recalculated as “10-the comfort rating”. In Figs. 2a and 2b the averaged results
obtained for both scales: “discomfort ratings” and “10-comfort ratings” for soundscape stimuli(without any video) and landscape stimuli (without any audio) are presented . The results were
calculated in R environment using bootstrapped confidence intervals (functions from the package‘boot’). The differences between the means were calculated using a ‘WRS2’ package and bootstrappedversion of t-tests.
In Fig. 2a the annoyance ratings of soundscape stimuli for three different sound levels, marked withdifferent colours, for -6dB, 0dB and +6dB are presented. The “*” means that there is a statisticallysignificant difference between the two analyzed ratings. It can be seen that for all the soundscapes
presented at the original sound level, there are no differences between these two ratings. This is also
the case for all the investigated landscapes presented in Fig. 2b with one exception – Boardwalk.

Figure 2. Averaged assessment of “discomfort scale” and “10-comfort” scale for 7 soundscapes (a) and 7
landscapes (b). Statistically significant differences between both scales are marked with ‘*’.
When the soundscape stimuli were presented at the +6dB sound level only for the “Park” stimulus
there was a significant difference between the discomfort and 10-comfort ratings. In contrast,
significant differences occurred for five out of seven soundscapes at the -6dB sound level.
This means that the hypothesis that discomfort and 10-comfort ratings are equivalent can beverified differently, depending on the sound level at which the stimuli were assessed: positively for theoriginal stimuli and those presented at the +6dB sound level (with one exception) and neg atively forthe softer stimuli presented at the -6dB sound level. We could also try to verify this hypothesis for eachstimulus independently of the sound level. As can be seen in Fig. 2, there are no significant differences7935

between the discomfort and 10-comfort ratings for two stimuli “Park near busy street “ and“Roundabout” for all three sound levels.
3.2 The results of consistent mixes (Experiment I)
Based on the results obtained for the case of single stimulus assessment, soundscape only andlandscape only, our expectation was that a similar tendency should be observed in the originallyrecorded audio-visual stimuli. In the original stimuli, called in the present study consistent mixes, thesoundscape and the landscape information comes from the same environmental sample. Thus, the lackof differences between the type of scale for the original sound level, 0dB, and the +6dB sound level,means that a similar tendency should appear in consistent mixes. The comparison of the mean valuesof two annoyance scales is presented in Fig. 3.

Figure 3. Mean values of discomfort and 10-comfort scales for investigated consistent mixes in 7 places in
the city of Poznań. Different colours signify the sound level of the presented audio part of the mix, ‘*’ meansa significant difference between annoyance assessment performed when using the discomfort and 10comfort scales.
As can be seen in Fig. 3, it is only when original stimuli were presented at the +6dB sound level that
there are no significant differences for either scale, with the “Side Street” stimulus as one exception. In
contrast there are statistically significant differences between two scales for 4 out of 7 stimuli and for
5 out of 7 stimuli when stimuli were presented at 0 dB and the -6dB sound level respectively. Wheneach consistent mix is analyzed separately, it is only with the “Busy street” stimulus that there are no
significant differences between the discomfort and 10-comfort ratings. For three stimuli there is only
one significant difference, at the relative level of presentation equals -6dB for “Park near busy
street“ and “Roundabout” and for “Market” at the original level of 0dB.
3.3 Results of non-consistent mixes (Experiment II)
The non-consistent mix means that the audio information does not match the visual information in
the investigated sample of an environment. In Table 1 all variants of non-consistent stimuli arepresented regarding differences between ‘discomfort’ and ’10-comfort’ scale for each case. Similarly

7936

to the results of the consistent mixes, the largest number of differences between two annoyance scales
were observed when the stimuli were presented at the -6dB sound level (19 cases). The number ofdifferent pairs decreases when the sound level of the stimuli presentation increases. For the original
sound level of the stimuli presentation there were 11 different pairs, while for the +6dB sound levelstimuli presentation there were only 4 such cases. The sound level at which the stimuli were presented
to the participants of the experiments has an influence on the equivalence of these two annoyancescales in the following way: the louder the stimuli, more similar the discomfort and 10 -comfort ratings.
In contrast, the softer the stimuli, the larger the differences between those two annoyance scales.
Table 1. All non-consistent combinations of audio and visual stimuli used in the experimental procedure. All
cases when a statistically significant difference between both scales (discomfort and 10-comfort) wereobserved are marked with ‘*’.
Video
Boardwalk

Busy
StreetMarketPark*

BoardwalkPark nearbusy street

Roundabout

Side
Street****Busy Street-6dB

Market

*

Park
Park
near
busy street

*

*

*
*

*

*

*

*

*
*

Roundabout
Side Street**

Boardwalk

*

*
*

*

Busy Street*

Market
Audio

0
dB

Park

*

*

*

*

*

Park
near
busy street*

Roundabout*

Side Street
BoardwalkBusy Street*

Market
+6
dB

Park

*

*

Park
near
busy street
Roundabout
Side Street*

4. DISCUSSION
Regarding the results from the analyses it seems that with an increase in the relative sound level,
there is a drop in the number of cases when both scales (Discomfort and 10-Comfort) are significantly
different.
To examine if that drop is statistically significant, we did a chi square test to find out if the numberof cases changes significantly. The test confirmed our findings, with chi = 15.565 and p = .0004. It is7937

also worth noticing that when the relative sound level was -6dB, only 25 out of 49 cases were found tobe equal on both scales. While the level increased to 0 dB, the proportions changed to 34:15 and finallybecame 43:6 when the relative sound level was equal to +6 dB.
To sum up, these two experiments showed that generally it is possible to treat comfort anddiscomfort ratings as the same in the way that discomfort is equal to 10-comfort. This convertibility isobserved for 6 out of 7 video stimuli presented without any audio. It is also true for audio stimuli only
when they are presented with the original sound level. However, when the sound level is artificiallychanged (especially to a softer level), this tendency is not clear and the scale becomes less reversible.
When it comes to consistent audio-video mixes, it seems reasonable to say that while the soundlevel increases, the scales become more similar and more mixes are rated convertible with regard toboth scales – the scales were statistically different 5 times when the sound level was -6dB, 4 times with
0dB and only once when the sound level was 6dB higher than originally.
The same tendency could be observed for non-consistent mixes, the louder the audio part is, the lessambiguous both scales are.

5. CONCLUSIONSIn this article we simply show that both scales can be convertible with the exceptions, depending on
both sound level of the stimuli presentation and on the type of stimuli. The louder the audio part of astimulous is, the more agreement between both scales is observed. Finally, it is worth to mention thatin almost all observed cases ‘pure’ discomfort ratings were slightly lower than ‘10-comfort’ ratings. It
can be interpreted that, theoretically, when a stimulous represents the same comfort and discomfort,
comfort would be rated with lower values than discomfort.

ACKNOWLEDGEMENTS
This research was funded(UMO-2017/25/N/HS6/01164).

by

National

Science

Centre

with

the

grant

Preludium

REFERENCES
1. Lercher P, De Greve B, Botteldooren D, et al. Health effects and major co-determinants associated with
rail
and
road
noise
exposure
alongtransalpine
traffic
corridors,
http://www.icben.org/2008/pdfs/lercher_et_al_corridors.pdf (2008, accessed 26 April 2017).
2. Basner M, Babisch W, Davis A, et al. Auditory and non-auditory effects of noise on health. Lancet
2014; 383: 1325–1332.
3. European Union. DIRECTIVE 2002/49/EC OF THE EUROPEAN PARLIAMENT AND OF THE
COUNCIL of 25 June 2002 relating to the assessment and management of environmental noise,
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32002L0049&from=en(2002,
accessed 31 May 2017).
4. Marquis-Favre C, Premat E, Aubrée D. Noise and its Effects – A Review on Qualitative Aspects ofSound. Part II: Noise and Annoyance. Acta Acust united with Acust 2005; 91: 626–642.
5. Brown AL. Soundscapes and environmental noise management. Noise Control Eng J 2010; 58: 493.
6. Axelsson Ö, 54 I 43/SC 1/WG. The ISO 12913 series on soundscape: An update, May 2012. J Acoust
Soc Am 2012; 131: 3381–3381.
7. Raimbault M. Qualitative Judgements of Urban Soundscapes: Questionning Questionnaires andSemantic Scales. Acta Acust united with Acust 2006; 92: 929–937.
8. Craig A, Moore D, Knox D. Experience sampling: Assessing urban soundscapes using in-situ
participatory methods. Appl Acoust 2017; 117: 227–235.
9. Ou D, Mak CM, Pan S. A method for assessing soundscape in urban parks based on the service quality
measurement models. Appl Acoust 2017; 127: 184–193.
10. Velardi L, Hermand J-P, D’Autilia R. On timbre in urban soundscapes: The role of fountains. J Acoust
Soc Am 2017; 141: 4017–4017.
11. Watts G. The effects of “greening” urban areas on the perceptions of tranquillity. Urban For Urban
Green 2017; 26: 11–17.
12. Woodcock J, Davies WJ, Cox TJ. A cognitive framework for the categorisation of auditory objects in

7938

urban soundscapes. Appl Acoust 2017; 121: 56–64.
13. Zhang Y, Kang J, Kang J. Effects of Soundscape on the Environmental Restoration in Urban Natural
Environments. Noise Health 2017; 19: 65.
14. Driver J, Spence C. Cross-modal links in spatial attention. Philos Trans R Soc B Biol Sci 1998; 353:
1319–1331.
15. Santangelo V, Fagioli S, Macaluso E. The costs of monitoring simultaneously two sensory modalities
decrease when dividing attention in space. Neuroimage 2010; 49: 2717–2727.
16. Talsma D, Senkowski D, Soto-Faraco S, et al. The multifaceted interplay between attention andmultisensory integration. Trends Cogn Sci 2010; 14: 400–410.
17. Carlesa JL, López Barrioa I, Luciob JV. Sound influence on landscape values. Landsc Urban Plan 1999;
43: 191–200.
18. Gidlöf-Gunnarsson A, Öhrström E. Noise and well-being in urban residential environments: The
potential role of perceived availability to nearby green areas. Landsc Urban Plan 2007; 83: 115–126.
19. Hong JY, Jeon JY. Designing sound and visual components for enhancement of urban soundscapes. J
Acoust Soc Am 2013; 134: 2026–36.
20. Fields JM, De Jong RG, Gjestland T, et al. STANDARDIZED GENERAL-PURPOSE NOISEREACTION QUESTIONS FOR COMMUNITY NOISE SURVEYS: RESEARCH AND A
RECOMMENDATION. J Sound Vib 2001; 242: 641–679.
21. Preis A, Kaczmarek T, Wojciechowska H, et al. POLISH VERSION OF STANDARDIZED NOISEREACTION QUESTIONS FOR COMMUNITY NOISE SURVEYS. IJOMEH Int J Occup Med
Environ Heal 2003; 16: 155–159.
22. Bodin T, Björk J, Ohrström E, et al. Survey context and question wording affects self reported
annoyance due to road traffic noise: a comparison between two cross-sectional studies. Environ Health2012; 11: 14.
23. Brink M, Schreckenberg D, Vienneau D, et al. Effects of Scale, Question Location, Order of Response
Alternatives, and Season on Self-Reported Noise Annoyance Using ICBEN Scales: A Field Experiment.
Int J Environ Res Public Health 2016; 13: 1163.
24. Preis A, Kociński J, Hafke-Dys H, et al. Audio-visual interactions in environment assessment. Sci Total
Environ 2015; 523: 191–200.

7939