Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

COLLABORATIVE STUDY OF INTERACTIVE SEISMIC ARRAY SONIFICATION FORDATA EXPLORATION AND PUBLIC OUTREACH ACTIVITIES
Masaki Matsubara

Yota Morimoto

Takahiko Uchide

University of Tsukuba,
Japan

Royal Conservatory of the Hague,
The Netherlands

Geological Survey of Japan,
AIST, Tsukuba, Japan

masaki@slis.tsukuba.ac.jp

yota@tehis.net

t.uchide@aist.go.jp

ABSTRACT
Earthquakes are studied on the basis of seismograms. Whenseismologists review seismograms, they plot them on a screen
or paper after preprocessing. Proper visualisations help them
determine the nature of earthquake source processes and/or theeffects of underground structures through which the seismic
wave propagates. Audification is another method to obtain an
overview of seismic records. Since the frequency of seismic
records is generally too low to be audible, the audification
playback rate needs to be increased to shift frequencies into the
audible range. This method often renders the playback of soundtoo fast to perceive the nature of earthquake rupture andseismic propagation. Furthermore, audified sounds are often
perceived as fearful and hence unsuitable for distribution to the
public. Hence, we aim to understand spatio-temporal wave
propagation by sonifying data from a seismic array and todesign a pleasant sound for public outreach. In this research, asonification researcher, a composer and a seismologist
collaborated to propose an interactive sonification system forseismologists. An interactive sonification method for multiple
seismic waves was developed for data exploration. To
investigate the method, it was applied to a seismic array of the
wave propagation from the 2011 Tohoku-oki earthquake overJapanese islands. As the playback rate is only 10 times in theinvestigation, it is easy to understand the propagation ofseismic waves. The sonified sound shapes show somecharacteristics and distributions such that seismologists caneasily determine the time span and frequency band to befocused on. The case study showed how a seismologist
explored the data with visualisation and sonification and how
he discovered triggered earthquake by using the sonified sound.

1.

INTRODUCTION

Earthquakes are energetic natural events in the Earth and huge
earthquakes cause disasters in many ways, such as seismic
shaking, surface rupture, and tsunami. Because of the impact ofhuge earthquakes on society, studies on earthquake phenomena,
seismic hazard, and disaster prevention as well as public
outreach activities are quite important. Seismologists usually
analyse seismograms1 to extract information including seismic
source processes. The seismograms also represent the shaking
felt by people. Therefore, seismograms must be useful for bothresearch and public outreach activities.
1

When seismologists review seismograms, they usually plotthem on a screen or paper after preprocessing. Proper
visualisations help them determine the nature of earthquake
source processes and/or the effects of underground structures
through which the seismic wave propagates. Based on suchobservations, the seismologists design data analyses to extract
the findings objectively and quantitatively.
Audification is another method to obtain an overview of theseismic records [1]. Many seismic waveforms have been
audified by seismologists and artists [2, 3, 4, 5]. Since thefrequencies of seismic records are generally too low to be
audible, the audification playback rate needs to be increased toshift frequencies into the audible range. However, this often
renders the playback of sound too fast to perceive the nature ofearthquake rupture and seismic propagation. Although the
fearful sounds of such audification is reminiscent of the threatof earthquakes, it hampers the objective understanding ofearthquake phenomena.
Sonification is another way to represent seismic waves, andvarious methods have been investigated by sonification
researchers and composers [6, 7, 8]. Since sonified sounds havemore variety compared to audified sounds, seismic sonification
is used for not only musical works, but also for public outreachactivities and data explorations.
The problem in interdisciplinary sonification research, asGoudarzi pointed out [9], is that sonification researchers oftenknow little about seismology and the seismologists are notfamiliar with sonification methodology. Nevertheless, some of
the interdisciplinary sonification studies succeeded incontributing to the domain of science [10, 11, 12]. For
seismologists to use sonification, it is necessary to focus on theissues significant to seismology. Therefore, it is beneficial to
distribute sonification tools and to describe the advantages ofthe sonification over traditional visualization in seismology.
In this study, we adopt an interdisciplinary design process[9], namely a sonification researcher, a composer, and aseismologist collaborate to propose an interactive sonification
method for seismologists. The benefits of collaborative study
are as follows. (1) We can focus on significant seismologicalproblems or data. Collaboration with a seismologist can
prevent the sonification researcher from processing the data
inappropriately. (2) We can adapt sonified sound for thedomain requirements. Collaboration with a composer can makesound easier to understand naturally based on music theory.
We aim to understand spatio-temporal wave propagation bysonifying data from seismic array and to design a pleasant,sound for public outreach. Since seismic waves fromneighbouring stations are correlated, we use the audio gestalt
strategy [13] for the sonification method. Thus, we can hearsimilar waves in the same stream to recognize salient
unexpected events easily when they occur.

Records produced by seismographs at seismic stations.

56
ISon2016-1

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

Each signal in every blocks (playback rate)

LeakDC

ZeroCrossing

LPF (cutoﬀ frequency)

DegreeToKey

Frequency

SinOscPeakFollower

…

…

…

Amplitude

Seismic array signals

Pan2(Scale, Range, Interval)(posi"on)LPF (cutoﬀ frequency)

Figure 1. Flow of seismic datafor sonification. Each box represents common functions of SuperCollider, and italic words
represent parameter variables.
2.

METHOD

We implemented audification and sonification designs for aseismic array to achieve the primary aims of our collaboration,
which are to understand seismic wave propagation by sonifying
data from multiple stations, and to design a possible pleasantsound for public outreach. The sonification method was builtwith common functions of SuperCollider. We also
implemented custom classes that provide functions to parse theseismic data, to perform simple signal processing tasks such as
DC removal and normalization, and to audify seismic data bothin real-time and non-real-time (rendering to a disk as fast as
possible).

Algorithm 1 Audification of seismic array
1: seismic_array.do {|start_time id|
Routine {2:start_time.wait;
3:
4:{
var rate, sig, freq, peak;5:
sig = PlayBuf.ar(1, ~bufs[id], rate, doneAction: 2);
6:
sig = LeakDC.ar(sig);
7:
DetectSilence.ar(sig, doneAction: 2);
8:
Pan2.ar(sig, id/(seismic_array.size-1)*2-1);
9:}.play;10:}.play;11:
12: }Figure 2. SuperCollider pseudocode for audification

2.1 Audification and Sonification Method
As an example, we have chosen to use the data from the 2011
Tohoku-oki, Japan earthquake for designing the sonification.
The famous sonification of the seismic array in the Tohoku-oki
earthquake is available on YouTube [14], and its playback rateis greater than 1000 because the target span is a few days fromearthquake occurrence and they presented the aftershock
activity. In this study, we focused on seismic propagation fromthe mainshock; therefore, the target duration was 5 min.
starting from onset of the mainshock. Seismologists require amaximum duration of approximately 1 min. Therefore, we haveto design a playback rate less than 10.
2.1.1. Audification of Seismic Array
In this system, we audify seismic data from multiple stations
such that the spatio-temporal wave propagation becomes
audible. Figure 2 shows a SuperCollider pseudocode for
audification. The locations of stations are mapped for
localization within the stereophonic image. The recorded onsetof each data point determines the temporal alignment of eachaudified event. The global temporal scale, as well as the
playback rate of each seismic wave, is transposed by a factor of
10. The transposition and the consequently reduced time scale
enabled us to listen to how a seismic wave propagates and tofind earthquakes potentially triggered remotely.
2.1.2. Sonification of Seismic Array
Even after transposing the playback rate of each seismic wave
by a factor of 10, the sound is rather rumbling and unclear.
Although it is possible to transpose the seismic waves further,
the global time scale of seismic wave propagation will beconsequently reduced, which was not desired.

Algorithm 2 Sonification of seismic array
1: seismic_array.do {|start_time id|
2:Routine {start_time.wait;
3:
{
4:
5:
var rate, sig, freq, peak;
sig = PlayBuf.ar(1, ~bufs[id], rate, doneAction: 2);
6:
sig = LeakDC.ar(sig);
7:
freq = ZeroCrossing.ar(sig);
8:
freq = LPF.ar(freq, 100);9:freq = DegreeToKey.ar(scale.as(LocalBuf),10:
freq.cpsmidi.round, 12, 1, 5).midicps;
11:
peak = PeakFollower.ar(sig, 0.7);
12:peak = LPF.ar(peak, 1);13:
sig = SinOsc.ar(freq, 0, peak);14:
DetectSilence.ar(sig, doneAction: 2);
15:
Pan2.ar(sig, id/(seismic_array.size-1)*2-1);
16:}.play;17:}.play;18:
19: }Figure 3. SuperCollider pseudocode for sonification
With this sonification design, we sought to overcome theproblem of intelligibility while maintaining the transposition
rate. We have also addressed our aim of producing a more
pleasant sound with this sonic design.
Figure 1 shows the schematic data flow and Figure 3 showsa SuperCollider pseudocode for sonification. In order tounderstand spatio-temporal wave propagation, we need to focus
on the change of dominant frequency and amplitude envelope.

57
ISon2016-2

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

Thus, we applied zero-crossing over a moving window with the
ZeroCrossing function and an amplitude follower with the
PeakFollower function to derive the dominant frequency andamplitude envelope estimations in each wave. Since seismic
waves generally include high-frequency components, we needto remove them with the LPF function to make the soundpleasant. We also used discrete frequency mapping with the
DegreeToKey function to utilize various musical scales. Then,
we synthesized parameter-mapped sinusoid wave with the
SinOsc function and panned with the Pan2 function according
to location of seismic stations.
In contrast to audification, this sonification method enables
the seismic waves to provide parameters for sound synthesis.
This allows us to maintain a coherent playback rate while thedominant frequency dynamically changes. Since each seismic
wave contained a similar component, the waves were formed as
one large sound object based on gestalt cognition.
Consequently, the sonified sounds become bubble-like timbre
so that this method satisfies the aim of a “more pleasant sound.”

2.2 Interface of Interactive Sonification Tool
For seismologists, we have implemented an interactive
sonification system with a graphical user interface (GUI). With
the GUI, users can interactively specify the geographic region,
method (audification or sonification with choices of musicalscales), and global time scale of sonification (Figure 4). The
system intends to provide an interactive means forseismological explorations (from global spatio-temporalobservation to regional selective listening) and for a real-timedemonstration of outreach purpose.

Figure 5. Peak ground accelerations (PGA) in verticalcomponents and examples of seismic waves from the 2011
Tohoku-oki earthquake. Densely distributed symbols indicate
locations of seismic stations coloured by PGA. Contours in thegrey-scale image show the slip distributions of the fault
inferred from seismic data [16]. The contour interval is 10 m.
The seismograms displayed on the right-hand side are verticalaccelerograms from K-NET stations in well-known cities.

3.

CASE STUDY

In this section, we describe how the seismologist analyses andexplores the seismogram data with the proposed method.
3.1 Earthquake, Data description
We first applied the sonification method to the 2011 Tohokuoki, Japan earthquake (magnitude 9.0) that impacted the society
because of the disaster caused by the strong ground shaking
and the devastating tsunami that hit the Pacific coast of Japan
Islands and propagated across the Pacific Ocean (Figure 5). It
is worth sharing the seismic records with the public to draw
people’s attention to earthquake disasters and the preparedness
for them.
This huge earthquake was recorded by seismometers all over
the world in addition to the dense seismic networks in Japan.
K-NET and KiK-net are nationwide strong-motion seismic
networks composed of seismometers [15]2 . K-NET stations
have instruments only on ground surface, while KiK-net haveborehole and surface seismometers. We use the up-down
component of acceleration recorded at the surface to observe
seismic shaking often amplified by near-surface soil, as we feelon the surface. They both provide accelerations of threedimensional ground motions. The record is started at the time
when the amplitude exceeds a threshold; therefore, stations
away from the earthquake source region may have missed thefirst arrivals of seismic waves with small amplitudes.

3.1 Nation-wide sonification
First, we sonify the seismic data from all over Japan. We haddata from more than 600 stations, and we picked 116
representative stations from them evenly in space. The
playback speed is 10 times the actual speed. The synthesizedsound spatializes the recorded seismic waves from different
Figure 4. GUI of interactive seismic array sonification system.
Users can specify the geographic region.

2

K-NET and KiK-net data by NIED are available at
http://www.kyoshin.bosai.go.jp/

58
ISon2016-3

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

Figure 6. Seismic data in Hida area, central Japan. The inverted
triangles denote the location of stations used for the areal
sonification. The seismograms displayed on the right-hand side
are vertical accelerograms from K-NET and KiK-net stations.

earthquake [e.g., 17]. The stations for sonification are shown in
Figure 6. The sonified sound contains a high-pitched sound at
the end similar to what we hear in the sound fromthe nationwide sonification. In fact, the timing agrees with alocal earthquake seen in the seismic waveforms. Since this isrecorded by nearby stations, the anelastic attenuation effect isweak and the high-frequency components in seismic waves arepreserved.
Therefore, we conclude that the high-pitched sound in thenationwide sonification is due to this dynamically triggered
earthquake.
Another example is east Hokkaido area. We hear some highpitched sounds by the regional sonification using the stations
shown in Figure 7. The seismograms contain high-frequency
waves at one station and other waves at another station.
Though it is difficult to draw conclusions from the seismic dataof only one station, it is possible that the high-frequency waves
originate from local earthquakes or some other phenomena that
occurred near the station.
Although we can find the causes of the curious sounds by thevisualisation of seismograms, the sonification makes it easy tofind them.

4.

Figure 7. Seismic data in eastern Hokkaido area, nothern
Japan. The inverted triangles denote the location of stations
used for the regional sonification. The seismograms displayedon the right-hand side are vertical accelerograms from K-NET
stations.
stations so that the overall sonic impression conveys the feelingof seismic waves propagating all over Japan. At first the soundis high-pitched and loud, and the pitch and volume become
progressively lower. This reflects the elastic attenuation at a
distance and anelastic attenuation especially in high-frequency
waves due to scattering and absorption by the medium (i.e.,underground soil and rock). It is interesting that the end ofsynthesized sound (around 23 s, equivalent to 230 s in theactual time after the origin time) contains a short series of highpitched sounds, which will be investigated in the next
subsection.

DISCUSSION

Figure 8 shows a schematic of the work flow in thecollaboration among a seismologist, a composer, and asonification researcher in this study. Each node indicates a role.
This schematic is quite similar to that of a conventional study[9].
From the case study, the sonified sounds satisfied therequirement, which means that the sonification design was
successful. From the sonification researcher’s point of view,
this sonification could not succeed without the composer andseismologist. In the data exploration process, when thesonification researcher and the composer recognized the salient
unexpected event, they could not interpret the meaning of thephenomenon. On the other hand the seismologist sometimes
felt it difficult to listen to the auditory stream separately. When
we listened to the sonified sounds together, we could notice thesalient acoustic event and interpret the significance ofcharacteristics from a seismological aspect.

3.2 Regional sonification
We have explored the cause of the high-pitch sounds bysonifying seismograms from stations in particular regions. Here
we show two examples that seem to be related to the highpitched sound series. In regional cases, we sonify all stations
available, while in the nationwide case, we reduced the numberof stations in use.
One example is Hida area, Gifu prefecture, where amagnitude-4 earthquake was dynamically triggered by thestrong seismic waves from the mainshock of the Tohoku-oki

Figure 8. Schematic work flow of the collaboration.

59
ISon2016-4

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

In addition, during the use of interactive sonification tool,
the seismologist could guess the seismic propagation process sothat he could explore the data efficiently. If the user does nothave any domain knowledge, the interactive data exploration
process would be random and futile.
For public outreach activity, we presented both thevisualisation and sonification simultaneously (Figure 9).
Although the audience generally did not have seismologicalknowledge, they could recognize the high-pitched sound event.
It is difficult to understand this event with only visualisation.
Therefore, the sonification seemed to help the interpretation.

[2] C. Davison, “Earthquake sounds,” Bulletin of the
Seismological Society of America, vol. 28, no. 3, pp. 147–
161, 1938.
[3] S. Speeth, “Seismometer sounds,” Journal of the AcousticalSociety of America, vol. 33, pp. 909–916, 1961.
[4] C. Hayward, “Listening to the earth sing,” in Auditory
Display: Sonification, Audification, and Auditory Interfaces.
Addison-Wesley, 1994.
[5] F. Dombois, “Using audification in planetary seismology,”
Proceedings of the International Conference on Auditory
Display, 2001.
[6] M. Meier and A. Saranti: “Sonic explorations withearthquake data,” Proceedings of the 14th International
Conference on Auditory Display, 2008.
[7] M. Quinn and L. D. Meeker: “Research set to music: Theclimate symphony and other sonifications of ice core, radar,dna, Seicmic and solar wind data,” Proceedings of the 7thInternational Conference on Auditory Display, 2001.
[8] R. McGee and D. Rogers: “Musification of seismic data,”
Proceedings of the 22nd International Conference on
Auditory Display, 2016.

Figure 9. A snapshot of visualisation with sonification for
public outreach.
5.

CONCLUSION

In this paper, we proposed an interactive seismic arraysonification method using collaborative design strategy. In
order to design the sonification method for significant
discovery in the domain, both domain knowledge and musictechniques were required. We chose Tohoku-oki earthquake as
an example and the sonified sound of its seismic data indicateda dynamically triggered earthquake. The sounds also satisfied
the domain needs for public outreach purpose.
From the case study, we found that the interpretation of
sonified sound needs not only seismological knowledge but
also listening ability. From the seismologist viewpoint, theinteractive system has potential for helping domain experts inthe analysis of seismic propagation of a newly occurred
earthquake. For future work, we will investigate how thesystem scaffolds domain experts’ analyses and reduces their
cognitive load.
6.

ACKNOWLEDGMENT

We thank three anonymous reviewers for their constructive
comments. We used the seismograms from K-NET and KiKnet operated by the National Research Institute of Earth
Science and Disaster Resilience (NIED), Japan. Some of the
figures were prepared using Generic Mapping Tools (GMT)
[18].

7.

REFERENCES[1] F. Dombois and G. Eckel, Audification, in T. Hermann, A.
Hunt and J. G. Neuhoff, Eds. The Sonification Handbook,
Berlin, Germany: Logos Verlag, 2011.

[9] V. Goudarzi, K. Vogt and R. Hoeldrich: “Observations on
an Interdisciplinary Design Process using a sonification
Framework,” Proceedings of the 18th International
Conference on Auditory Display, 2015.
[10] F. Dombois, O. Brodwolf, O. Friedli, I. Rennert and T.
Koenig: “Sonifyer: A concept, a software, a platform,”
Proceedings of the 14th International Conference on
Auditory Display, pp. 1 – 4, 2008.
[11] Z. Peng, C. Aiken, D. Kilb, D. R. Shelly and B. Enescu:“Listening to the 2011 Magnitude 9.0 Tohoku-Oki, Japan,
Earthquake,” Seismological Research Letters, Vol. 83, No.2,
pp. 287 – 293, 2012.
[12] V. Goudarzi: “Designing an Interactive Audio Interface for
Climate Science,” IEEE Multimedia, pp. 41- 47, 2013.
[13] H. Terasawa, J. Parvizi and C. Chafe: “Sonifying ECOG
seizure data with overtone mapping: A strategy for creating
auditory gestalt from correlated multichannel data,”
Proceedings of the 18th International Conference on
Auditory Display, pp. 129 – 134, 2012.
[14] Sonification of Tohoku Earthquake / Sendai Coast, Japan,
2011/03/11,
https://www.youtube.com/watch?v=3PJxUPvz9Oo,(2016.9.15 Accessed)[15] Y. Okada, K. Kasahara, S. Hori, K. Obara, S. Sekiguchi, H.
Fujiwara, and A. Yamamoto, “Recent progress of seismic
observation networks in Japan - Hi-net, F-net, K-NET and
KiK-net”, Earth Planets Space, vol. 56, pp. xv-xxviii, 2004.
[16] T. Uchide: “High-speed rupture in the first 20 s of the 2011
Tohoku earthquake, Japan,” Geophysical Research Letter,
40, 2993-2997, 2013.
[17] M. Miyazawa: Propagation of an earthquake triggering
front from the 2011 Tohoku-Oki earthquake, GeophysicalResearch Letter, 38, L23307, 2011.
[18] P. Wessel and W. H. F. Smith: “Free software helps map
and display data,” EOS, vol. 72, p. 441, 1991.

60
ISon2016-5