Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016

SONIFICATION OF FLUIDITY AN EXPLORATION OF PERCEPTUAL CONNOTATIONS OF A PARTICULAR
MOVEMENT FEATURE
Emma Frid, Ludvig Elblaus, Roberto Bresin
KTH Royal Institute of Technology
Stockholm, Sweden{emmafrid, elblaus, bresin}@kth.se
ABSTRACT

integrate” [2]. Fluidity could therefore be considered a meaningful feature for characterizing both audio and movement [2]. Furthermore, fluidity has been found to be one of the properties thatappears to contribute significantly to perception of emotions indance [3], suggesting that it could serve as a meaningful parameter which could be mapped to sound in interactive sonification of
bodily movement.

In this study we conducted two experiments in order to investigate
potential strategies for sonification of the expressive movement
quality “fluidity” in dance: one perceptual rating experiment (1)in which five different sound models were evaluated on their ability to express fluidity, and one interactive experiment (2) in which
participants adjusted parameters for the most fluid sound model in(1) and performed vocal sketching to two video recordings of contemporary dance. Sounds generated in the fluid condition occupied
a low register and had darker, more muffled, timbres compared to
the non-fluid condition, in which sounds were characterized by ahigher spectral centroid and contained more noise. These results
were further supported by qualitative data from interviews. The
participants conceptualized fluidity as a property related to water,
pitched sounds, wind, and continuous flow; non-fluidity had connotations of friction, struggle and effort. The biggest conceptual
distinction between fluidity and non-fluidity was the dichotomyof “nature” and “technology”, “natural” and “unnatural”, or even“human” and “unhuman”. We suggest that these distinct connotations should be taken into account in future research focusing on
the fluidity quality and its corresponding sonification.

2. FLUIDITY AND ENERGY
The fluidity data used for sonification in this study was extractedusing the method described in [1]. In this context of body movement, a fluid movement is smooth and coordinated, such as forexample a wave-like propagation through body joints [2]. Here,fluidity is estimated by comparing the mean jerk values (i.e. thethird derivative of the position) of the shoulders, elbows and handsfor both original measurements of a dancer and simulated data of amass-spring model. The mass-spring model is defined so that eachjoint of the body is modeled as a mass connected to springs thatsimulate muscle tension. The human body is thus represented as aset of interconnected masses, where each mass represents a joint.
The mass-spring model consists of two types of springs: longitudinal springs l, which connect joints, and rotational springs r, which
impress rotational forces on body segments. By tuning parameters(e.g. mass of the joints, spring stiffness or damping coefficients)
this model can be used to simulate very fluid conditions generating
very smooth trajectories. By calculating the distance between thejerk of the recorded movement data and the fluid simulated massspring model data, an estimate of fluidity of a dance movement
can be obtained for a given trajectory segment. This is done bycalculating the fluidity index (FI) at frame k of a motion segment
recorded with a motion capture system (MoCap) as follows:

1. BACKGROUNDThis study is part of the EU H2020 Project ICT DANCE1 focusing on investigating how affective and social qualities of humanfull-body movement can be expressed, represented and analyzed
through sound and music. The purpose of the DANCE project is toinvestigate if it is possible to perceive expressive movement qualities in dance solely through the auditory channel, i.e. to capture
expressive qualities of dance movements and convey them throughsounds. In a general sense, the ability to translate the finer qualities of some information from one modality to another has many
practical implications and use cases. Communicating movement
qualities through audition can be of great use, e.g. for the blind.
While the DANCE project explores artistic practice, the findings
will be useful not only in that domain but also in everyday applications. In this paper, we focus on one particular expressive
movement quality belonging to the third layer of the four-layered
conceptual framework proposed by Camurri et al. in [2]: Fluidity.
Third level features such as equilibrium, coordination, repetitiveness, and fluidity “are at a level of abstraction such that they represent amodal descriptors, i.e., the level where perceptual channels

F Ik = JIkl + JIkr

(1)

where JIkl is the distance of the overall jerk for the longitudinal
spring and JIkr the distance of the overall jerk for the rotational
spring according to the following equations:... ls... le... lhJIkl = |Xk + Xk + Xk |... ls ... le ... lh|Y k + Y k + Y k |(2)... rs... re... rh... ls ... re ... rh
JIkr = |Xk + Xk + Xk | |Yk + Yk + Yk | (3)...
where Xk is the third derivative of a set of 3D coordinates measured at frame k for the shoulders (s), elbows (e) and hands (h),1 http://dance.dibris.unige.it/

ISon2016-1

11

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016...
respectively, and Yk the third derivative of the corresponding simulated set of coordinates for the spring model. After evaluation of
F Ik , averaging is done to compute the estimated fluidity. For a
more detailed description of the computation of the fluidity estimation and algorithm, see [1].
Apart from the fluidity parameter, the sound models describedin this paper also make use of kinetic energy in the sonificationof movement data. Kinetic energy is computed as the productbetween body joint masses and velocities. Given that we havea three-dimensional space, we define velocity of the i-th tracked
joint at frame k as:
vik = |X˙ik |SM2 A sinusoidal carrier oscillator with a variable amount ofphase modulation from three sinusoidal modulators in intervals of unison, two octaves, and two octaves and a fifth.
An increase in energy increased the pitch of all oscillatorsand the modulation index. A decrease in fluidity injected
a white noise component into the sum of the modulators,
destabilizing the pitch of the carrier as well as introducing
noise in the final output.
SM3 A complex source-filter-model that aims to simulate a windsound. A mix of noise sources is filtered through a set ofband pass and low pass filters that both respond to changes
in the energy parameters but also individually vary theircut off frequencies using low frequency random signals.
An increase in energy increased the frequency of the random modulations to the filters cut off frequencies as wellas their center frequencies. An increase in fluidity resulted
in smoother shapes in the modulation signals, as well as asmoother, less coarse, mix of noise sources.

(4)

and then compute the kinetic energy index EI as:
N

EIk =

1X
2
mi · vik
2 i=1

(5)

where N is the number of tracked joints of the dancer’s body (inour case shoulders, hands and elbows).

SM4 A simple source-filter-model using a mix of smooth andcoarse noise sounds filtered through a 24dB per octave lowpass filter. An increase in energy increased the cut off frequency of the filter. An increase in fluidity increased theamount of smooth noise in the source mix.

3. SOUND MODELSIn previous research on sonification of continuous body gestures,
researchers have identified sound properties that can be found insounds associated to fluent or irregular movements. For example, in a study on the sonification of handwriting it was foundthat sound models characterized by low frequency components
were more suitable for both aiding and communicating fluency of
movements, while sound models characterized by high frequency
crackling sounds (sounding like small impacts) were suitable forportraying jerky hand movements lacking fluency [4]. In anotherstudy in which sound was used for learning movement kinematics, researchers found that sounds which were more noisy or withlouder high-frequency components could help users identify motion behavior [5]. These results were taken into account when designing the sound models used in the present study.
As described in the previous section, all sound models were
designed to respond to two movement feature parameters: fluidity
and energy. In the descriptions below, when smooth or coarse isused to describe noise, smooth noise stands for spectrally brightand continuous noise, e.g. white noise. Coarse noise signifies
more varied and less spectrally even noise, e.g. the noise obtainedby randomly switching sample values between +1 and -1, respectively. In all sound models, an increase in energy was mapped to anincrease in amplitude, starting from complete silence, given zeroenergy. All sound models were created using the SuperCollider2
programming language. The five sound models were:

SM5 White noise processed by a bank of parallel band pass filters,with variable tuning quantized to semi tone steps in a equal
temperament scale, and variable resonance. An increase inenergy increased the cut off frequency of the filters, maintaining their harmonic relationship. An increase in fluidity
increased the q-value of the filter, making it narrower, approaching a sinusoidal wave. A decrease in fluidity madethe filter wider, resulting in a noisier output, and also added
a detuning component independently to all filters, making
the result less harmonious.
The sound models3 draw on different strategies to express thequalities of the incoming control data. SM1 and SM2 exploits
the tension of inharmonicity, chorusing and beating effects to express lack of fluidity, contrasted with harmonicity and stability to
express the opposite. SM3 and SM4 both borrow qualities fromphysical every day interaction, using amplitude and spectral tilt to
express energy, e.g. a spectrally brighter and louder sound represents a more forceful and fast movement. Furthermore, they bothchange their source material to express the presence or lack of fluidity, by varying the coarseness and variability of the source material. In the case of SM4 this is further embellished to approach therealistic sounds of wind gusts, whereas SM3 is decidedly artificial
and electronic in its nature. SM5 is a combination of all of theseapproaches and exploits harmonic pitch sensitivity, physical interpretation of spectral slope and amplitude, as well as a noise generator that can move from coarse, gravel-like sounds, to smoothersound reminiscent of a water stream or light wind. Spectrogramsof excerpts of the five sound models are seen in Fig. 1.
The ability of the five sound models to portray fluidity wastested in two experiments presented in the following sections. Our
hypothesis was that some of these models would be more suitable
than others when sonifying fluidity.

SM1 An open chord, Eb, in five octaves with one added fifth in
the middle of the octave stack, made of saw wave oscillators with variable pitch stability, from perfectly stable toa random modulation of 50Hz centered around the pitch,summed and filtered using a 24dB per octave low pass filter with variable cut off frequency. Increasing the energy
increased the filter cut off frequency and increasing the fluidity decreased the amount and speed of the pitch modulation.

3 Examples of sonified dance data using the five models can be found athttps://kth.box.com/v/ison2016.

2 http://supercollider.github.io/

ISon2016-2

12

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016sults showed that there was a significant effect of sound modelon perceived fluidity, F(2.536, 46.878), F = 3.333, p = 0.029. Post
hoc comparisons using the Bonferroni correction indicated that themean score for sound model SM5 (M = 3.274, SD = 0.557) was
significantly different from sound model SM2 (M = 2.971, SD =0.487) and sound model SM3 (M = 2.872 , SD = 0.686). Boxplotsof each sound model is seen in Fig. 2, descriptive statistics foreach model is seen in Tab. 1.

3.5
3.0
1.5

2.0

Figure 1: Spectrograms of 9 second excerpts from sonified movement data using SM1-SM5 in Experiment 1.

2.5

Fluidity Rating

4.0

Perceptual Rating of Fluidity

4. EXPERIMENT 1
Experiment 1 was a web-based perceptual rating experiment inwhich participants rated the presence of fluidity in a recording. In
order to avoid bias due to different interpretations of the word “fluidity”, participants were asked to rate the presence of the following
property (slightly modified from the one suggested in [1]):“A dancer’s energy of movement (energy of muscles) is free toflow between the regions of the body (e.g. from torso to arms, fromhead to torso to feet) in the same way that a wave propagates ina fluid (such as the wave propagation caused by a stone which isthrown into water).”
The participants listened to sonifications of five segments ofmovement data, recorded in a studio setting with a dancer, that had
previously been rated as very fluid in a previous study [1]. The fivedifferent sound models described in Sec. 3 were used, providing
a total of 5 sound models times 5 segments of movement data, i.e.
25 stimuli. The stimuli were presented to the participants in a randomized order. For each stimuli, the participants provided theiranswer to the statement “The above described movement property
is present in the sonic representation” using a continuous slider labeled from “I completely disagree” to “I completely agree”. The
continuous slider was coded in such a manner that “I completely
disagree” corresponded to value of 1 and “I completely agree” corresponded to a value of 5 (these values were not visible to the participants).
We collected data from 41 participants (F = 20 and M = 21,
Mean = 26.610 yrs, SD = 7.141) from 6 nationalities (the twobiggest were Swedish, 76 %, and Italian, 10 %). For all participants, the average rating for each sound model was obtained by
calculating the mean of all five segments. A one-way repeatedmeasures ANOVA was conducted to investigate if there was a significant difference in perceived level of fluidity for the different
sound models. Mauchly’s test of sphericity indicated that the assumption of sphericity had been violated, 2 (9) = 60.468, p =1.142 · 10 9 , therefore degrees of freedom were corrected using
Greenhouse-Geisser estimates of sphericity (" = 0.634). The re-

SM1

SM2

SM3

SM4

SM5

Sound Model

Figure 2: Boxplots of fluidity ratings for each sound model in Experiment 1.

Table 1: Perceptual Rating of Fluidity - Descriptive Statistics per
Sound ModelSound Model
SM1
SM2
SM3
SM4
SM5

Mean
2.958
2.971
2.872
2.959
3.274

Median
2.994
2.930
2.902
3.080
3.284

SD
0.699
0.487
0.686
0.757
0.557

SEMa
0.109
0.076
0.107
0.118
0.087

a) Standard error of the mean.

5. EXPERIMENT 2
Experiment 2 was an interactive experiment. In total, 9 participants (M=7, F=2, Mean=30.444 yrs, SD=6.697) took part in theexperiment. The sound model ranked as the most fluid one in Experiment 1 (SM5) was used to sonify the movement of a dancer
performing a fluid movement sequence. This movement sequence
was presented in a video that was recorded in a previous study [1].
The synchronized playback of the video and the data that was sonified was done using a custom video playback software written in

ISon2016-3

13

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016
C++ using the openFrameworks4 environment. The sound model
was controlled by the energy and fluidity values extracted from thedancer’s movement, as described in Sec. 2. The participants were
instructed to adjust 6 sliders (with 8 bit resolution) that controlled
the following aspects of the sound model in real-time:

Final Settings for Fluid vs. Non−fluid
1.00

0.75

Value

• Slider 1: The quantization step of the center frequencies ofthe band-pass filters, ranging from continuous to steps of aminor third.
• Slider 2: The amount of high frequency content in the noise
source.

Condition
Fluid

0.50

Non−fluid

0.25• Slider 3: Scaling of the fluidity parameter mapping to the
bandwidth of the band-pass filters.

0.00• Slider 4: Scaling of the energy parameter mapping to thecenter frequencies of the band-pass filters.

S1

S2

S3

S4

S5

S6

Slider• Slider 5: The presence of an echo effect.

Figure 3: Final slider settings for the fluid versus the non-fluid
condition in Experiment 2.

• Slider 6: Manipulation of the center frequencies of the bandpass filters, ranging from harmonic to inharmonic.
The parameters were chosen so that all parts of the model
could be modified, in a bi-polar fashion, from the original parameter setting. The echo effect was added to provide the option of temporal diffusion or smearing to investigate whether distinct clarity
over time was a factor in the sonification of fluidity.
Instructions for the experiment were read from a pre-written
manuscript. Initially, the participants were instructed to perform
two tasks; T1: “Adjust sliders so that the sound corresponds wellto the movement performed in the video”, and T2: “Adjust sliders
so that the sound does not correspond well to the movement performed in the video”. Values from the sliders were continuously
logged and the audio output was recorded. Each task was finishedwhen the participant stated that (s)he was satisfied with the audible
result.
After the real-time adjustment of the sound model (T1 and T2)
the participants were given the following task (T3): “You will now
see a video of a movement. After the video, try to describe how
you believe that a sound portraying this movement would sound.
You are encouraged to use metaphors when describing the sound.
If you would use your voice to sketch the sound that would portray
this movement, what would it sound like?” In task T3 participants
were presented with a video of a fluid movement, different fromthe video used in T1 and T2. Finally, task T3 was repeated, but thistime with a video of a non-fluid movement (T4). All videos usedas stimuli in Experiment 2 were perceived as very fluid versus very
non-fluid in the previous study [1] by Camurri et al. The purpose ofincluding vocal sketching in the study was to allow the participants
to explore whatever sounds they thought were appropriate, even
if the sound model used wasn’t able to produce those particularsounds.

conducted a paired-sampled t-test to investigate if the six meanslider values were significantly different for the two conditions(fluid and non-fluid). Rescaling the values of the sliders to 0-1,
the paired-samples t-test indicated that values were significantly
lower for the fluid condition (M = 0.314 , SD = 0.081) than for thenonfluid condition (M = 0.507 , SD = 0.121), t(8) = -2.829, p =0.037, d = 0.193.
For each respective slider controlling a certain aspect of thesound, we carried out pair-wise comparisons to investigate if there
was a significant difference between the fluid versus non-fluid condition. Data for S1, S4 and S5 did not meet the assumption of normality and was therefore examined using a Wilcoxon signed-ranks
test. Since the data for slider S2, S3 and S6 met the assumptionof normality, paired-samples t-tests were conducted for these sets.
The only observed significant effect for the pair-wise comparisons
was for S2: the t-test indicated significantly higher values for the
non-fluid condition (M = 0.609, SD = 0.332) than for the fluid
condition (M = 0.228, SD = 0.215), t(7) = -2.986, p = 0.020, d =0.381. This suggests higher amount of high frequency content in
the noise source for the non-fluid condition compared to the fluidone.
When examining the recorded sounds6 , we observed that therecordings from the fluid condition were more homogeneous than
the recordings from the non-fluid one. In general, the fluid recordings occupied a lower register and were characterized by a darker
or more muffled timbre. In contrast, many of the non-fluid recordings were characterized by a high spectral centroid.
5.2. Interviews
After the interviews had been transcribed, the words and phrases
used by the participants to describe the observed movements andthe imagined sounds were compared and categorized. Below, sevencategories are presented together with a few quotes from the interviews that are emblematic of the category. The categories also
have a brief description that contextualizes how the participants
discussed the sounds and movements in respective condition (fluidor non-fluid). Each category also has a fraction in parenthesis that
represents how many of the participants that stated something that

5.1. Real-time Adjustment
Boxplots of the final slider settings from all participants are seen in
Fig. 3. We computed mean values for all sliders for the fluid versus non-fluid condition, thereby obtaining an estimation of slider
values for the averaged fluid versus non-fluid sound model 5 . Spectrograms of these two models are seen in Fig. 4. We subsequently
4 http://openframeworks.cc

5 Sound examples of these averaged models can be found at https://kth.box.com/v/ison2016.

6 See footnote 3 for sound examples.

ISon2016-4

14

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016
Wind (3/9)
Air and wind, and how an air stream sounds and feels.
• “Something airy, [. . . ], it sounds like wind.”• “An airy feeling, like you feel when you move your arms
around yourself, you feel the air.”• “Looks like the movement of the wind.”
Natural (3/9)
Actions and sounds that are expected, natural, and follow somelogic of uninterrupted flow.
• “Something stable.”• “A flow and continuum.”• “Comfortable, natural.”• “Flowing, [. . . ] a comfortable movement.”
Categories describing non-fluid sounds and movements
Friction (6/9)
The creaks and scrapings of rusty joints or old doors.
• “Something very creaky, like a creaking door.”• “It is like some rusty door [. . . ] metal, the shaft parts, theconnection parts.”

Figure 4: Spectrograms generated from recordings of sound modelSM5 when sliders were set to mean values for the fluid versus nonfluid condition in Experiment 2.

• “Creaking sound that shows how the joints may sound ifthey are so hard to move.”• “Friction and strong interaction between parts that dont really want to interact.”

can be sorted into that category: e.g. a “(9/9)” implies that all
participants used words or expressions from the category.

• “Like wood creaking.”• “The sound of an old metal thing that needs to be greased[. . . ], a joint that needs oil.”Categories describing fluid sounds and movements
Water (6/9)

Unnatural (5/9)
A category where technology, industrial machines, and robots are
put in contrast to nature and an idea of natural behavior.

The flowing qualities of water itself, but also the way things underwater move, constantly affected by their surroundings.

• “The movement reminds me a bit of a robot, [. . . ], something more industrial [. . . ], a factory line.”• “The movement makes me think of sort of liquids.”• “Seaweed in the ocean.”• “A bit robotic.”• “A sound for water.”• “Abnormal, [. . . ] robotic movement, but not a proper robot,
a normal robot would not move like this.”• “Like waves.”• “A flowing sound.”• “Unhuman, coarse, metallic.”• “A very unnatural movement, [. . . ], [a sound] you don’t expect to come from a human.”

Pitch (5/9)
On the qualities of pitch and related metaphors such as vertical
position and speed.

Effort (4/9)
A struggle against constraints, containment, weight.

• “Connected to the floor, heavy and slow, [. . . ], something
stable that always comes down to the ground.”• “Something that feels restricted to some extent [...] it ismore confined.”• “Pitched, not very noisy.”• “It is like he is stuck, and can’t move properly.”• “I feel that it is pretty low, [. . . ] like a melody that is easy
to listen to.”• “The person is unable to move, and he or she is trying very
hard to move legs and hands”, ”probably [. . . ] the soundthat comes when you move your bones, something like thebones cracking.”• “Varying pitch.”• “Not a high pitch.”

ISon2016-5

15

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016
• “A strained feeling.”

possibilities. In general, significantly larger values were found forthe non-fluid condition compared to the fluid one for slider S2(which controlled the high frequency content in the noise source).
Nevertheless, as described in Section 3, all parameters had a complex interdependent relationship and one should take into accountthat the combined slider settings influenced the extent to which
adjustment of the S2 slider had an effect.

• “Struggling and making a real effort.”

In general, the participants spent more time talking about the
non-fluid sounds and movements, and they seemed to have a harder
time finding the words to describe the fluid movements. The fluid
movements were seen as “natural” and somehow appropriate andwere therefore hard to pinpoint (they were as things “ought to”
be). The non-fluid movements on the other hand evoked a set ofrather well defined metaphors of the “industrial”, “unnatural” andimages of painful struggle and malfunctioning machines.
To summarize the results, the ideal sound to express fluidity
is continuous, pitched and maybe even melodic, in a comfortably
low register and pleasing to listen to. The ideal sound to express
non-fluidity on the other hand is creaky, metallic, unhuman androbotic. By taking the opposites of the qualities connected to fluidity and non-fluidity and adding them to the other category, we
can also add organic (opposite of metallic) and smooth (opposite
of creaky) to fluidity, and stuttering, non-pitched, and high pitched
to non-fluid (the opposites of continuous, pitched and low-pitched,respectively).

The main conclusion drawn from examining the logged data
in Experiment 2 is that participants managed to create two distinct
sonic representations. In general, the fluid recordings occupied alower register and were characterized by a darker or more muffled
timbre, whereas many of the non-fluid recordings were characterized by a higher spectral centroid and more noise.
In the qualitative interviews in Experiment 2, the participants
conceptualized fluidity (both in movement and in sound) as a property related to water, pitched sounds, wind, and continuous flow.
Non-fluidity on the other hand had connotations of friction, struggle and effort. However, the biggest conceptual distinction between fluidity and non-fluidity was the dichotomy of nature andtechnology, natural and unnatural, or even human and unhuman.
We believe that it is important to take these distinct connotations
into account when performing perceptual studies focusing on thefluidity parameter.

5.3. Vocal Sketching
Vocal sketching can be effective when describing sounds that don’t
have clear agreed upon symbols in language (e.g. when the sourceof the sound cannot be identified), or when communicating characteristics of a sound that are ambiguous, such as pitch or temporal
qualities (e.g. how low is “low”, how fast is “fast”) [6, 7].
While all participants carried out some vocal sketching, thesounds produced varied from a few seconds to up to a minute fordifferent participants. Some participants felt uncomfortable doing
the vocal sketching, while others didn’t give it a second thought.
However, even when taking this into account, some general characteristics can be observed in the sketching across most participants.
The vocal sketching of fluid movements was continuous, somewhat softer and sometimes lower pitched than the one of non-fluid
movements. It used an uninterrupted air flow, whistling, whooshing and whispering sounds, and tended towards darker timbres.
The vocal sketching of non-fluid movements was louder, strained,
used vocal creaks and often contained bursts of sound, or shortseries of completely separated staccato sounds. Generally, it contained much more high frequency energy than the vocal sketchingof fluid movements.
Even though the material collected in this study was too small
and varied to serve as a basis for any conclusions on its own, it isevident that the vocal sketching supports the analysis of the interviews presented above.

Some general differences could also be observed in the vocal
sketching in the fluid and non-fluid conditions: fluidity was expressed using continuous, softer, and lower pitched sounds with adarker timbre; non-fluidity was vocalized with louder, more strained,creaks and bursts of sound, with more high frequency content. The
vocal sketching served two important functions. Firstly, it corroborated the analysis of the interview. Secondly, it provided a possible
explanation for the heterogeneity in the resulting audio recordings
for the non-fluid condition, as the participants sketched sounds that
simply were not possible to arrive at given the possibilities offeredby SM5.
Finally, the participants tendency to interpret fluid/non-fluid
as natural/unnatural meant that they generally had a harder time
providing descriptions or sketches of fluidity as it was seen as the“correct” way of moving and sounding. It was the internalized natural state and therefore somewhat invisible. On the other hand, byconnecting non-fluidity to otherness and the unnatural, it could be
more clearly described, being something external that could easily
be pointed to.

7. FUTURE WORKOne disadvantage of the approach employed in Experiment 2 wasthat only a small set of actual videos (not point-light displays),
were used as stimuli. During the qualitative interviews, it was evident that some of the descriptive key words used related more to
the dancer’s personal impersonation of the movement property, i.e.
to the theatrical aspects of the dance performance, rather than theproperties innate in the actual high-level movement feature. We
propose future investigations involving a larger data set. Furthermore, building on the findings from the vocal sketching, we suggest a follow-up experiment in which the parameter-space of thesound models is expanded so as to allow exploration into both fluidand non-fluid sounds.

6. CONCLUSIONSIn Experiment 1, we observed a significant effect of sound modelon perceived fluidity, with sound model SM5 being perceived as
significantly more fluid than SM2 and SM3. We believe this to bea result of SM5 representing a complex and layered approach thatcombines several strategies to express the variations in fluidity.
In Experiment 2, when the participants manipulated the internal parameters of SM5, they tuned the model in significantly
different ways in the fluid and non-fluid conditions. This indicatesthat the parameter space provided offered distinctly different sonicISon2016-6

16

Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016
8. SUPPLEMENTARY MATERIAL
The movement data, the source code for the software that reads the
movement data and generates the sonification, as well as examples
of sonified movement data using the five models can be found athttps://kth.box.com/v/ison2016.
9. ACKNOWLEDGMENTS
The work presented in this paper was funded by the European
Unions Horizon 2020 research and innovation programme under
grant agreement No 645553 (DANCE).
10. REFERENCES[1] Stefano Piana, Paolo Alborno, Radoslaw Niewiadomski,
Maurizio Mancini, Gualtiero Volpe, and Antonio Camurri,“Movement fluidity analysis based on performance and perception,” in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems,
New York, NY, USA, 2016, CHI EA ’16, pp. 1629–1636,
ACM.
[2] Antonio Camurri, Gualtiero Volpe, Stefano Piana, Maurizio
Mancini, Radoslaw Niewiadomski, Nicola Ferrari, and Corrado Canepa, “The dancer in the eye: Towards a multilayered computational framework of qualities in movement,”in Proceedings of the 3rd International Symposium on Movement and Computing. ACM, 2016, p. 6.
[3] Ginevra Castellano, Santiago D Villalba, and Antonio Camurri, “Recognising human emotions from body movement
and gesture dynamics,” in International Conference on Affective Computing and Intelligent Interaction. Springer, 2007,
pp. 71–82.
[4] Jérémy Danna, Vietminh Paz-Villagrán, Charles Gondre,
Mitsuko Aramaki, Richard Kronland-Martinet, Sølvi Ystad,and Jean-Luc Velay, “Let me hear your handwriting! Evaluating the movement fluency from its sonification” PloS one,
vol. 10, no. 6, pp. e0128388, 2015.
[5] Eric O Boyer, Quentin Pyanet, Sylvain Hanneton, andFrédéric Bevilacqua, “Learning movement kinematics witha targeted sound,” in International Symposium on Computer
Music Modeling and Retrieval. Springer, 2013, pp. 218–233.
[6] Inger Ekman and Michal Rinott, “Using vocal sketching fordesigning sonic interactions,” in Proceedings of the 8th ACM
conference on designing interactive systems. ACM, 2010, pp.
123–131.
[7] Guillaume Lemaitre, Arnaud Dessein, Patrick Susini, andKarine Aura, “Vocal imitations and the identification ofsound events,” Ecological psychology, vol. 23, no. 4, pp.
267–307, 2011.

ISon2016-7

17