## 3 Listening Test 1
### Results  
The results of Parts 1, 2 and 3 can be seen together in Figure 17. For Part 1 and Part 3, the average annoyance evaluations and standard deviations were calculated from the individual ratings of participants. The results of Part 2 are shown on a second axis in the same graph. The main reason for this visualization was because the evaluations from a magnitude estimation are ratio-scaled quantities. The evaluations of each participant were linearized by taking the log10 of each value. Since the first stimulus was an anchor stimulus with a reference value of 100, participants evaluated this signal as 100, which was shown in a linearized way as the number \"2\" in this figure. Similarly, if a participant rated a stimulus as \"200\" (two times more annoying than the anchor stimulus), this was represented approximately as 2.30 on this graph. A similar trend was observed between the three different methodologies, whereas the 'drops' between the different loudness levels (stimuli 3 to 4 and stimuli 6 to 7) were more obvious in the magnitude estimation. The slider scale and the random access methods had similar trends. These evaluation methods used the same scale and eventually showed similar standard deviations. In both cases, participants had a limited response scale, where they had to provide answers between the predefined numbers (i.e., 0-100), which reflect the categorizations with verbal anchors. Depending on the number of stimuli used for a listening experiment in annoyance evaluations, both methods can be used interchangeably. However, for an experiment with a large number of stimuli, the random access method can be disadvantageous for a participant, since it might be overwhelming to place many sound samples on the evaluation surface at the same time. From a similar perspective, access to all stimuli encourages a participant to play back every possible pair, which might lead the multiple-stimulus evaluation method to become a pairwise evaluation method. In contrast, in the slide scale method, where participants evaluated a single stimulus in each round,  
Figure 17: Results of Parts 1, 2 and 3 with the same stimuli and three different testing methods (slider scale, magnitude estimation and random access method, respectively). Results of Part 1 and Part 3 were gathered by calculating the arithmetic mean values of the results, as well as standard deviations. For Part 2, on the other hand, since the magnitude estimation test provides ratio-scaled data, results are linearized by taking the log10 of the values and given in the second y-axis. Averages and standard deviations were calculated after the linearization. they usually reported that they were not sure at the beginning of the test, so they wanted to change their previous evaluations depending on the newly available stimulus. The slider scale method does not provide participants an opportunity to go \"back\" and \"correct\" their response. However, with a proper training session and randomization of the order of the stimuli for each participant, we can eliminate this possible bias, which we call \"beginning bias\". Eventually, for the case with nine stimuli, both methods showed similar tendencies. However, a magnitude estimation has its own advantages and disadvantages. The main disadvantage of using a magnitude estimation in annoyance evaluations is the question itself. The main feedback from the participants was that they could not estimate \"what was two/three times more/less annoying\". These estimations are more suitable for evaluating better scalable quantities, for example, \"two times longer\" or \"three times larger surface area\". For a line with a given length, participants can better \"estimate\" the length of a second line; however, the same approach is not always clear for participants of annoyance evaluations. The second disadvantage can be seen in the selection of the anchor stimulus. Here, stimulus 1, which had the lowest loudness and sharpness values, was selected as the anchor stimulus.