# A General Framework for Visualization of Sound Collections in Musical Interfaces
## 3 Proposed Framework
### Relation to Prior Work  
As described in earlier sections, our aim is to generalize previous work on the visualization of sound collections. In this section, we analyze previous work in relation to the dimensions of the framework presented in the previous section. Our analysis is presented in Table 1. Our focus is on the potential of visualization for creative applications. For this reason, we have selected works where a working prototype that can be used in this context is presented. We describe 10 relevant systems ranging from 2007 until present:  
1. [noitemsep]
2. Mused, an interactive scatter plot system for browsing a large database of sounds by [PERSON] [10]. 3. SoundTorch, a virtual flashlight system for browsing large collections of audio by [PERSON] et al. [15]. 4. A study on interactive navigation-based search in large sound databases by [PERSON] et al. [37]. 5. A visualization approach to represent the perceptual qualities of large collections of textural sounds based on tile maps by [PERSON] and [PERSON] [7]. 6. AudioQuilt, a system that flexibly arranges audio clips on a layout based on the user taste by [PERSON] et al. [38]. 7. AudioMetro, a system that organizes sounds on an evenly distributed grid by [PERSON] et al. [39]. 8. Floop, a system based on rhythm and timbre analysis for exploring loops in the Freesound database by [PERSON] and [PERSON] [8]. 9. Freesound Explorer, a visual interface system for exploring sounds from Freesound based on their timbral properties by [PERSON] and [PERSON] [5]. Figure 3: Synthetic example of the grid layout algorithm showing values of oversampling from 1 to 5. A study on visualization methods based on perceptual representations of timbre suitable for large libraries of sound samples by [PERSON] and [PERSON] [21]. * AudioStellar, a system that enables the generation of interactive visualizations based on large collections of sounds by [PERSON] et al. [6]. It can be seen from Table 1 that some features are notably rare. For example, most systems do not include segmentation as part of the pipeline. However, in our view, adding segmentation opens up the possibilities for using longer recordings. While segmentation can be carried out with other tools, integrating it into the same framework allows interactive adjustment of the parameters while seeing the resulting visualization. Thus, we consider the lack of segmentation an important limiting factor of most prior work. Controlling the layout is also relatively rare. This is necessary to avoid overlaps when icons are used. In [7], the sounds are mapped to a tiled grid, but it is unclear if an automatic algorithm was used, so we would assume the mapping was carried out manually. [PERSON] et al. [38] specifically addressed this issue, but using a significantly expensive method (e.g., they mention the process taking 30 min for 176 samples). They later applied IsoMatch [18] to the same problem, which presumably is faster. We use the same approach as IsoMatch although with different dimensionality reduction algorithms. In [39], an algorithm named Proximity Grid is used [40]. Although the work in [21] is more focused on the evaluation than on the prototype, we included it here because they assess the use of icons, which again is rare among the compared systems. However, their prototype was mostly focused on this experiment and included only 30 samples, so they do not address the issue of overlaps. In addition to this work, only [7; 39] feature visualizations for the sound beyond color. The system by the first author in [8] used spectrogram images from the Freesound database, but this approach would be difficult to scale to many sounds due to the complexity of the images. The system in [39] added a unique capability of synthesizing image texture using supervised learning. Our system focuses on waveforms derived from descriptors, which is similar to their mapping to shape, but we allow using different descriptors. The shapes used in [7] were again based on a manual approach.