## 3 Listening Test 1
### Stimuli, Subjects and Test Methods  
Listening test 1 showed that loudness and sharpness have a significant effect on annoyance perception. However, it was not clear from this experiment if there was an interaction effect between these two parameters, i.e., higher-frequency content might influence the annoyance estimations as a function of the overall loudness of the stimulus. Multicollinearity is an important problem in statistical modeling that could lead to redundant input parameters in developed quality models. Moreover, the mathematical definitions of loudness and sharpness have a strong correlation from a purely acoustical point of view [40]. These two facts are particularly critical in sound quality evaluations of vacuum cleaners, where the loudness and sharpness play important roles. First, it can be interpreted from verbal  
Figure 14: Mean annoyance evaluations vs. single value tonality calculations, based on (**a**) DIN 45681—Low; (**b**) DIN 45681—Mid; (**c**) DIN 45681—High; (**d**) Hearing Model—Low; (**e**) Hearing Model—Mid; (**f**) Hearing Model—High. descriptions of the participants that, when the stimuli are louder, the stimuli with stronger high-frequency content are perceived as more annoying. However, we know from the definition that an increase in higher frequencies increases loudness as well as sharpness but at a different rate of change. For this reason, it is necessary to investigate whether there is an interaction effect between these two parameters. To analyze this possible interaction effect, a series of listening tests was designed, in each of which the loudness and sharpness values were varied in the context of a factorial design. Listening test 2 was divided into four parts. Part 1 included a slider scale experiment with a 3 \(\times\) 3 factorial experiment design for loudness and sharpness. Part 2 included a magnitude estimation test with a similar 3 \(\times\) 3 factorial design. Part 3 of the listening test had the same 3 \(\times\) 3 factorial design, but this time, a random access method was used. In Part 4, the factorial design was changed to 5 \(\times\) 3 for loudness and sharpness, and the random access method was used. Moreover, in addition to the possible loudness-sharpness interaction effect, this section also compares the different test methods to discuss the advantages and shortcomings in factorial design experiments. Mainly, for Parts 1, 2 and 3, the slider scale, magnitude estimation and random access methods were applied for the same stimuli under the same reproduction conditions. Finally, in Part 4, the variability in the loudness was extended in both directions so that the possible interaction effects can also be observable in the quieter and louder stimuli. For Parts 1, 2 and 3, a 3 \(\times\) 3 factorial design was used for the loudness and sharpness values. For Part 4, a 5 \(\times\) 3 factorial experimental design was used for loudness and sharpness values. The only difference in Part 4 was that the maximum and minimum values of the loudness values were extended. The values used for each factorial design can be found in Table 7. Here, stimuli 4-12 were used for the 3 \(\times\) 3 design (numbers with an asterisk), and stimuli 1, 2, 3, 13, 14 and 15 were added for the 5 \(\times\) 3 design. The loudness values, calculated according to the standard ISO 532-1 [22], were selected to be approximately 16 sone, 20 sone and 25 sone for the 3 \(\times\) 3 design. These values were selected such that they are in the limits measured for each vacuum cleaner given in Table 3. For the 5 \(\times\) 3 factorial design, the loudness values were extended to 13, 16, 20, 25 and 30 sone. Meanwhile, the sharpness values, calculated according to the calculation method of Aures ([25] with [22]), were selected as 2, 4, 29 and 3.3 acum. To obtain vacuum cleaner noise with different sharpness values, a parametric IIR low-pass filter was applied to a selected vacuum cleaner recording. The cutoff frequency of the low-pass filter was set to 4000 Hz.