DOI: 10.4119/unibi/2920448
title: Supplementary Material for "Sonification for Exploratory Data Analysis"

description: ### Sonification for Exploratory Data Analysis #### Chapter 8: Sonification Models In Chapter 8 of the thesis, 6 sonification models are presented to give some examples for the framework of Model-Based Sonification, developed in Chapter 7. Sonification models determine the rendering of the sonification and possible interactions. The "model in mind" helps the user to interprete the sound with respect to the data. ##### **8.1 Data Sonograms** Data Sonograms use spherical expanding shock waves to excite linear oscillators which are represented by point masses in model space. * Table 8.2, page 87: Sound examples for Data Sonograms File: Iris dataset: started in plot (a) at S0 (b) at S1 (c) at S2 <br>10d noisy circle dataset: started in plot (c) at S0 (mean) (d) at S1 (edge) <br>10d Gaussian: plot (d) started at S0 <br>3 clusters: Example 1 <br>3 clusters: invisible columns used as output variables: Example 2 Description: Data Sonogram Sound examples for synthetic datasets and the Iris dataset Duration: about 5 s ##### **8.2 Particle Trajectory Sonification Model** This sonification model explores features of a data distribution by computing the trajectories of test particles which are injected into model space and move according to Newton's laws of motion in a potential given by the dataset. * Sound example: page 93, [PTSM-Ex-1](https://pub.uni-bielefeld.de/download/2920448/2920459/trj_singleparticle.mp3) Audification of 1 particle in the potential of phi(x). * Sound example: page 93, [PTSM-Ex-2](https://pub.uni-bielefeld.de/download/2920448/2920460/trj_seq1_15_2clust.mp3) Audification of a sequence of 15 particles in the potential of a dataset with 2 clusters. * Sound example: page 94, [PTSM-Ex-3](https://pub.uni-bielefeld.de/download/2920448/2920461/traj_25_1_2cl.mp3) Audification of 25 particles simultaneous in a potential of a dataset with 2 clusters. * Sound example: page 94, [PTSM-Ex-4](https://pub.uni-bielefeld.de/download/2920448/2920462/traj_25_1_1cl.mp3) Audification of 25 particles simultaneous in a potential of a dataset with 1 cluster. * Sound example: page 95, [PTSM-Ex-5](https://pub.uni-bielefeld.de/download/2920448/2920463/PTSM-Ex-5-3clusters.mp3) sigma-step sequence for a mixture of three Gaussian clusters * Sound example: page 95, [PTSM-Ex-6](https://pub.uni-bielefeld.de/download/2920448/2920464/PTSM-Ex-6-1Gaussian.mp3) sigma-step sequence for a Gaussian cluster * Sound example: page 96, [PTSM-Iris-1](https://pub.uni-bielefeld.de/download/2920448/2920465/PTSM-ExIris.mp3) Sonification for the Iris Dataset with 20 particles per step. * Sound example: page 96, [PTSM-Iris-2](https://pub.uni-bielefeld.de/download/2920448/2920466/PTSM-ExIris3P.mp3) Sonification for the Iris Dataset with 3 particles per step. * Sound example: page 96, [PTSM-Tetra-1](https://pub.uni-bielefeld.de/download/2920448/2920467/PTSM-ExTetra4D.mp3) Sonification for a 4d tetrahedron clusters dataset. ##### **8.3 Markov chain Monte Carlo Sonification** The McMC Sonification Model defines a exploratory process in the domain of a given density p such that the acoustic representation summarizes features of p, particularly concerning the modes of p by sound. * Sound Example: page 105, [MCMC-Ex-1](https://pub.uni-bielefeld.de/download/2920448/2920516/MCMCampl.mp3) McMC Sonification, stabilization of amplitudes. * Sound Example: page 106, [MCMC-Ex-2](https://pub.uni-bielefeld.de/download/2920448/2920517/mcmc_audification_3cl.mp3) Trajectory Audification for 100 McMC steps in 3 cluster dataset * McMC Sonification for Cluster Analysis, dataset with three clusters, page 107 * Stream 1 [MCMC-Ex-3.1](https://pub.uni-bielefeld.de/download/2920448/2920518/E2a_stream1.mp3) * Stream 2 [MCMC-Ex-3.2](https://pub.uni-bielefeld.de/download/2920448/2920519/E2a_stream2.mp3) * Stream 3 [MCMC-Ex-3.3](https://pub.uni-bielefeld.de/download/2920448/2920520/E2a_stream3.mp3) * Mix [MCMC-Ex-3.4](https://pub.uni-bielefeld.de/download/2920448/2920521/E2a_mix.mp3) * McMC Sonification for Cluster Analysis, dataset with three clusters, T =0.002s, page 107 * Stream 1 [MCMC-Ex-4.1 (stream 1)](https://pub.uni-bielefeld.de/download/2920448/2920522/E2b_stream1.mp3) * Stream 2 [MCMC-Ex-4.2 (stream 2)](https://pub.uni-bielefeld.de/download/2920448/2920523/E2b_stream2.mp3) * Stream 3 [MCMC-Ex-4.3 (stream 3)](https://pub.uni-bielefeld.de/download/2920448/2920524/E2b_stream3.mp3) * Mix [MCMC-Ex-4.4](https://pub.uni-bielefeld.de/download/2920448/2920525/E2b_mix.mp3) * McMC Sonification for Cluster Analysis, density with 6 modes, T=0.008s, page 107 * Stream 1 [MCMC-Ex-5.1 (stream 1)](https://pub.uni-bielefeld.de/download/2920448/2920526/E2c_stream1.mp3) * Stream 2 [MCMC-Ex-5.2 (stream 2)](https://pub.uni-bielefeld.de/download/2920448/2920527/E2c_stream2.mp3) * Stream 3 [MCMC-Ex-5.3 (stream 3)](https://pub.uni-bielefeld.de/download/2920448/2920528/E2c_stream3.mp3) * Mix [MCMC-Ex-5.4](https://pub.uni-bielefeld.de/download/2920448/2920529/E2c_mix.mp3) * McMC Sonification for the Iris dataset, page 108 * [MCMC-Ex-6.1](https://pub.uni-bielefeld.de/download/2920448/2920530/mcmc_iris1_bw0p5.mp3) * [MCMC-Ex-6.2](https://pub.uni-bielefeld.de/download/2920448/2920531/mcmc_iris1_bw0p25.mp3) * [MCMC-Ex-6.3](https://pub.uni-bielefeld.de/download/2920448/2920532/mcmc_iris1_bw0.mp3) * [MCMC-Ex-6.4](https://pub.uni-bielefeld.de/download/2920448/2920533/mcmc_iris1_bw-0p25.mp3) * [MCMC-Ex-6.5](https://pub.uni-bielefeld.de/download/2920448/2920534/mcmc_iris1_bw-0p5.mp3) * [MCMC-Ex-6.6](https://pub.uni-bielefeld.de/download/2920448/2920535/mcmc_iris1_bw-0p75.mp3) * [MCMC-Ex-6.7](https://pub.uni-bielefeld.de/download/2920448/2920536/mcmc_iris1_bw-1.mp3) * [MCMC-Ex-6.8](https://pub.uni-bielefeld.de/download/2920448/2920537/mcmc_iris1_bw-1p25.mp3) ##### **8.4 Principal Curve Sonification** Principal Curve Sonification represents data by synthesizing the soundscape while a virtual listener moves along the principal curve of the dataset through the model space. * Noisy Spiral dataset, [PCS-Ex-1.1](https://pub.uni-bielefeld.de/download/2920448/2920588/PCS_Ex-1_1-noisyspiral.mp3) , page 113 * Noisy Spiral dataset with variance modulation [PCS-Ex-1.2](https://pub.uni-bielefeld.de/download/2920448/2920589/PCS_Ex-1_2-noisyspiralmod.mp3) , page 114 * 9d tetrahedron cluster dataset (10 clusters) [PCS-Ex-2](https://pub.uni-bielefeld.de/download/2920448/2920590/PCS_Ex-2-tetra9d.mp3) , page 114 * Iris dataset, class label used as pitch of auditory grains [PCS-Ex-3](https://pub.uni-bielefeld.de/download/2920448/2920591/PCS_Ex-3-iris.mp3) , page 114 ##### **8.5 Data Crystallization Sonification Model** * Table 8.6, page 122: Sound examples for Crystallization Sonification for 5d Gaussian distribution File: DCS started at center, in tail, from far outside Description: DCS for dataset sampled from N{0, I_5} excited at different locations Duration: 1.4 s * Mixture of 2 Gaussians, page 122 * DCS started at point A [DCS-Ex1A](https://pub.uni-bielefeld.de/download/2920448/2920595/Ex2gs_1.mp3) * DCS started at point B [DCS-Ex1B](https://pub.uni-bielefeld.de/download/2920448/2920596/Ex2gs_2.mp3) * Table 8.7, page 124: Sound examples for DCS on variation of the harmonics factor File: h_omega = 1, 2, 3, 4, 5, 6 Description: DCS for a mixture of two Gaussians with varying harmonics factor Duration: 1.4 s * Table 8.8, page 124: Sound examples for DCS on variation of the energy decay time File: tau_(1/2) = 0.001, 0.005, 0.01, 0.05, 0.1, 0.2 Description: DCS for a mixture of two Gaussians varying the energy decay time tau_(1/2) Duration: 1.4 s * Table 8.9, page 125: Sound examples for DCS on variation of the sonification time File: T = 0.2, 0.5, 1, 2, 4, 8 Description: DCS for a mixture of two Gaussians on varying the duration T Duration: 0.2s -- 8s * Table 8.10, page 125: Sound examples for DCS on variation of model space dimension File: selected columns of the dataset: (x0) (x0,x1) (x0,...,x2) (x0,...,x3) (x0,...,x4) (x0,...,x5) Description: DCS for a mixture of two Gaussians varying the dimension Duration: 1.4 s * Table 8.11, page 126: Sound examples for DCS for different excitation locations File: starting point: C0, C1, C2 Description: DCS for a mixture of three Gaussians in 10d space with different rank(<b>S</b>) = {2,4,8} Duration: 1.9 s * Table 8.12, page 126: Sound examples for DCS for the mixture of a 2d distribution and a 5d cluster File: condensation nucleus in (x0,x1)-plane at: (-6,0)=C1, (-3,0)=C2, ( 0,0)=C0 Description: DCS for a mixture of a uniform 2d and a 5d Gaussian Duration: 2.16 s * Table 8.13, page 127: Sound examples for DCS for the cancer dataset File: condensation nucleus in (x0,x1)-plane at: benign 1, benign 2 <br>malignant 1, malignant 2 Description: DCS for a mixture of a uniform 2d and a 5d Gaussian Duration: 2.16 s ##### **8.6 Growing Neural Gas Sonification** * Table 8.14, page 133: Sound examples for GNGS Probing File: Cluster C0 (2d): a, b, c <br>Cluster C1 (4d): a, b, c <br>Cluster C2 (8d): a, b, c Description: GNGS for a mixture of 3 Gaussians in 10d space Duration: 1 s * Table 8.15, page 134: Sound examples for GNGS for the noisy spiral dataset File: (a) GNG with 3 neurons 1, 2 <br>(b) GNG with 20 neurons end, middle, inner end <br>(c) GNG with 45 neurons outer end, middle, close to inner end, at inner end <br>(d) GNG with 150 neurons outer end, in the middle, inner end <br>(e) GNG with 20 neurons outer end, in the middle, inner end <br>(f) GNG with 45 neurons outer end, in the middle, inner end Description: GNG probing sonification for 2d noisy spiral dataset Duration: 1 s * Table 8.16, page 136: Sound examples for GNG Process Monitoring Sonification for different data distributions File: Noisy spiral with 1 rotation: sound <br>Noisy spiral with 2 rotations: sound <br>Gaussian in 5d: sound <br>Mixture of 5d and 2d distributions: sound Description: GNG process sonification examples Duration: 5 s #### Chapter 9: Extensions #### In this chapter, two extensions for Parameter Mapping Sonification are introduced. The first is the Auditory Legend, which provides the limits and an average sonification for the attributes of an instrument. The second is Multidimensional Perceptual Scaling, a technique to combine MDS and PCA in such a way that sonifications are obtained that reveal groupings in the data without the need for extensive parameter tuning. ##### **9.1: Auditory Legends for Parameter Mapping Sonification** * Table 9.1, page 140: Sound examples for Auditory Legends and Sonic Scatter Plots File: Iris dataset: <br>raw sonification, with frame markers, Auditory Legend <br> Cancer dataset: <br>raw sonification, with frame markers, Auditory Legend Description: Auditory Legend, Frame and Tick marks Duration: ##### **9.2: Multi-dimensional Perceptual Scaling** * Table 9.2, page 142: Sound examples for MPS Sonifications File: FM instrument: Auditory Legend Iris Dataset: MPS (PCA), MPS (MDS/PCA) <br>Cancer dataset: MPS (PCA), MPS (MDS/PCA) <br>7d tetrahedron: MPS (PCA), MPS (MDS/PCA) <br>10d noisy circle: MPS (PCA), MPS (MDS/PCA) Description: MPS Sonification using MDS and PCA Duration: ##### Chapter 10: Applications #### In this chapter, sonification is applied in 3 domains. Two of them are concerned with the analysis of data which is ordered in time. Sonification is here devoloped to assist the monitoring of dynamical features. The third application is concerned with identification and comparison of high-dimensional patterns. Here sonifications are developed that allow a user to interact with an auditory map by browsing elements of a visual display to query an auditory presentation. ##### **10.1 Sonification of Psychotherapeutic Sessions** * Sound example, page 146, Marker for abstract words: cymbal * Sound example, page 146, Marker for emptional words: claves. * Table 10.1, page 146: Sound examples for Session Transcript Monitor Sonification File: Session 11, Session 12, Session 24, Session 26 Description: Monitoring Session Transcripts Duration: 15 s * Table 10.2, page 147: Sound examples for Cycle Sonifications using AIB streams for the attributes abstract, emotion and CRA. File: Session 12 (10 s), Session 24 (10 s), <br>Session 12 (5 s), Session 24 (5 s) Description: AIB sonification of session transcripts for Session 12,24 Duration: 10 s, resp. 5 s <br> * Table 10.3, page 149: Sound examples for Cycle Sonification using AIB streams and spoken markers. File: Session 11, Session 12, Session 24, Session 26 Description: AIB sonification of session transcripts using speech marker Duration: 10 s <br> ##### **10.2 Sonification for EEG Data Analysis** * Table 10.4, page 151: Audification of EEG data (2 electrodes, 3 subjects and all 3 conditions). Besides the dominating noise, pitched spikes can be heard File: S1 EEGr Fp1, T5, Fp1/T5 S1 speech Fp1, T5, Fp1/T5 S1 pseudo speech Fp1/T5 S2 EEGr Fp1, T5, Fp1/T5 S2 speech Fp1, T5, Fp1/T5 S2 pseudo speech Fp1/T5 S3 EEGr Fp1, T5, Fp1/T5 S3 speech Fp1, T5, Fp1/T5 S3 pseudo speech Fp1/T5 Description: Audification of EEG Data for 2 electrodes (Fp1 and T5) for subjects S1-S3 Duration: 1 s * Table 10.5, page 153: Sound examples for Spectral Mapping Sonification of EEG data, for the conditions pseudo speech and speech with a short gap in between (2.5 s in real-time). File: delta=0.4, [0-30Hz], left T3,T5, right T4,T6 S1 S2 S3 S4 S5 S6 delta=0, [0-30Hz], left T3,T5, right T4,T6 S1 S2 S3 S4 S5 S6 delta=0, [0-30Hz], <br>left: F7,F3,T3,C3,T5,P3, right: F4,F8,C4,T4,T6 S1 S2 S3 S4 S5 S6 Description: Spectral Mapping Sonification of EEG data Duration: 4 s <br> * Table 10.6, page 154: Sound examples for Distance Matrix Sonification of spectral vectors. High-pitched loud tones indicate frequency selective couplings of long-range. File: frequency range 8-20 Hz, subjects: S1 S2 S3 S4 S5 S6 Description: Distance Matrix Sonification for concatenated datasets for the pseudospeech and speech condition. A noise burst separates the two parts Duration: 5 s Sound example, page 156: Ex-DiffSon-1 , Differential Sonification for the example shown as spectrogram. * Table 10.7, page 157: Sound examples for Differential Sonifications File: pseudospeech-speech (1s): S1, S2, S3, S4, S5, S6 same with higher threshold (1s): S1, S2, S3, S4, S5, S6 speech - EEGr (1s): S1, S2, S3, S4, S5, S6 pseudospeech-speech (4s): S1, S2, S3, S4, S4, S6 Description: Differential Sonification of EEG Data Duration: 1 s and 4 s ##### **10.3 Sonification of Multi-Channel Image Data** * Table 10.8, page 164: Sonification of cell fluorescence patterns. Sound examples for rhythmical, melodic, harmonic and multitimbral sonifications are given. File: Rhythmical: b1 b2 b3 b4 b5 b6 b7 b8 b9 Harmonical: b1 b2 b3 b4 b5 b6 b7 b8 b9 Harmonical2: b1 b2 b3 b4 b5 b6 b7 b8 b9 Melodic: b1 b2 b3 b4 b5 b6 b7 b8 b9 Multi-timbral: b1 b2 b3 b4 b5 b6 b7 b8 b9 Description: Marker Pattern Sonifications Duration: 1.4 s