Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Mixed Writing with Karlax and Acoustic Instruments:
Interaction Strategies from Computer Music
Benjamin Lavastre1 and Marcelo M. Wanderley2
1

DCS, IDMIL, CIRMMT, McGill University
2
IDMIL, CIRMMT, McGill University
benjamin.lavastre@mail.mcgill.ca; marcelo.wanderley@mcgill.ca

Abstract. The karlax is a gestural controller developed around 2010. Since its
inception, it arose substantial interest among composers and continues to be commonly used in solo and group performances. One of the reasons for its longevity
is its great adaptability especially in interaction with acoustic instruments. This
article analyses six chamber music pieces for karlax and acoustic instruments bycomparing the sound and visual results and the writing process (scores, patches,and mapping). We discuss different composition strategies through the use of interaction metaphors from the computer music literature. These metaphors prove
to be powerful analysis tools that allow describing the use of a digital music instrument (DMI), such as the karlax, in a chamber music context.
Keywords: Mixed pieces, Computer Music, Digital Music Instruments (DMI),Electronic Chamber Music, Input Devices, Mapping

1

Introduction

Though several hundred interfaces for musical expression have been developed anddescribed in a variety of venues, most notably in the last two decades at the International
Conference on New interfaces for Musical Expression (NIME)3 , relatively few articles
discuss how these interfaces ares used in actual musical contexts, for instance [1], [2],[3] and [4]. Indeed, the use of DMIs is not often discussed from the perspective ofartistic and musical composition. In other words, the ”M” in NIME: why don’t we talkmore about music performance with musical interfaces, beyond sound control? In part,
this is the consequence that most of the interfaces described in the literature have shortlife spans and/or are mainly used by their designers [5]. In this sense, the karlax offers aparticularly rich subject of study with an existence of more than ten years, a communityof regular users from different musical cultures and several significant creations, notably
with acoustical instruments, incorporating some form of music notation.
The karlax is an input device that combines several sensors: continuous keys, velocity pistons, axis, switches, and three axes of accelerometers and gyroscopes (Fig. 1)
4. ”Its ability to detect subtle as well as larger gestures, continuous as well as eventbased control, its low latency and high bandwidth, its reliability and portability” has3
4

www.nime.org
www.dafact.com

31

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

been praised [6]. Like many musical interfaces that output sensor data but which do
not have a pre-defined sound, the karlax is defined by its control characteristics, i.e., its
gestural identity instead of a given sonic identity. This opens up unlimited musical possibilities but requires the composer to describe the sounds controlled and the mapping
between sensor data and sound generation to be used in each context. A digital musical instrument (DMI) is composed of the group: control interface + mapping + soundgeneration [7].

Fig. 1. Front and rear views of the karlax (www.dafact.com)

In this study, we analyze a corpus of six pieces for karlax and acoustic instrumentsfrom audio and video recordings, scores, Max/MSP patches, articles, presets, etc. We
have identified three compositional models that allow us to define the main areas ofinspiration for each of the pieces. In a second step, we will discuss excerpts in the corpus
by detailing the action of the karlax and its interaction with the acoustic instruments
thanks to interaction metaphors from Computer Music.

2

Objectives

The objectives of this article are:
1. Study of six pieces for karlax and acoustic instrument(s) including analysis ofsound synthesis, mapping, gestures, and scores.
2. Among these pieces describe the ”role” of the karlax by identifying compositionalmodels.
3. Analyse the use of the karlax and its interaction with acoustic instruments in excerpts of these pieces thanks to Computer Music metaphors.

32

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

3

Corpus of Pieces

We have selected 6 pieces written between 2013 and 2018 that combine the karlax
controller with one or two acoustic instruments among a flute, a violin, and a cello.
Five of the six pieces of the corpus have been commissioned by the Fabrique Nomade
ensemble and have been performed by it. This ensemble is an ”electronic chamber music ensemble that wishes to rediscover the gestures and listening of classical chambermusic”5 . In this regard, ”each musician is independent and has total control over theiracoustic or electronic instrument” (each instrumentalist has their own laptop and theirown sound broadcasting system). This means that acoustic instruments performers trigger their own electronic part (most of the time real-time processing) thanks to a midi
pedal and that the karlax cannot process in real-time the acoustic sound of an instrumentalist. This is not the case for the sixth piece where the karlax transforms the soundof the violin in real-time.
A Fogg by Lorenzo Bianchi for violin, cello and karlax, 2013 (performed by Fabrique
Nomade ensemble)
B Frottement, Bourdon, Craquement by Francis Faber for cello, karlax and electronic,
2013 (performed by Fabrique Nomade ensemble)
C Le Patch Bien Tempéré III by Tom Mays, for flute, karlax and real time electonic,
2013 (performed by Fabrique Nomade ensemble)
D Ripples Never Come Back by Michele Tadini for violin, cello and karlax, 2013
(performed by Fabrique Nomade ensemble)
E Discontinuous Devices ”In-between” by Michele Tadini for cello and karlax, 2015
(performed by Fabrique Nomade ensemble)
F Le Violon, l’Oeillet et le Bambou by Raphaël-Tristan Jouaville, for violin and karlax, 2018

4

Composition models

Among these pieces, we have identified three compositional models that represent threemain sources of inspiration for the composers: model based on acoustic sounds, model
based on electronic sounds and karlax as model. These allow describing the main “role”
of this controller in relation to the other instruments.
Model based on acoustic instruments sounds
For several pieces in the corpus, the acoustic sound of the instrument(s) withwhich the karlax plays is used as the basic composition material. For example, in thepiece Fogg (A), the sound synthesis of the karlax is realized through an additive synthesis from the spectral analysis of several violin pizzicati with different ”preparations”(addition of objects like pegs attached to the string). The karlax triggers and controls
processes related to the spectral content of pizzicato sounds by pressing continuous
keys (control of the spectral envelope) (Fig. 2).
Other examples are pieces where the karlax plays sounds very close to the sounds
played by the instrument(s) it interacts with. In this way, the acoustic instrument is5

www.fabriquenomade.com

33

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

“augmented” by the action of the karlax. For example, in the third part of Discontinuous
Devices (E), the karlax activates flautando and harmonics cello samples by pressing thecontinuous keys. Shorter samples of the same type are also triggered by the pistons. This
forms a harmonic environment for the cello, which performs more percussive figureslike jettatti and glissandi that let the natural harmonics of the open strings resonate. With
the same idea, in Jouaville’s piece (F), the karlax plays a physical model of a string byactivating the pistons in a consecutive way whose pitches are previously set up (String
Studio module). In most of the piece, the karlax highlights and develops the melodic
contour of the violin and/or creates a harmonic accompaniment (Fig. 3).

Fig. 2. ”Shaping” of the spectral envelope with karlax continuous keys in Fogg by Lorenzo
Bianchi (mes. 68-69, karlax part) (with the permission of the composer). Each staff line represents the activation of a continuous key that will control the volume of a group of oscillators.

Fig. 3. Results of pitches played by the karlax pistons with the corresponding fingerings in Le
Violon, l’Oeillet et le Bambou, by Raphaël-Tristan Jouaville (mes. 7) (with the permission of thecomposer). See video from 00:30 to 00:32 www.youtube.com/watch?v=IrCmiwwFSUs

Model based on electronic sounds
This type of composition model is the most common in the selected pieces. In thiscategory are represented the treatments and manipulations associated with electronic
music such as filtering, delay, granular synthesis, additive synthesis, ring modulation,
arpeggiators, freeze, etc. Also, this control interface is often associated with the processing of electronic synthesis. By assigning certain parameters of the sound synthesis to34

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

different sensors, the karlax can “drive” processes in real-time and bring an expressive
dimension to the transformations. In this model, the sound of the karlax is perceived as
independent from the acoustic sound of the instruments. For example, in the piece Le
Patch bien tempéré III (C), the composer focuses on complementary electronic techniques such as harmonizers, delays, and ”paf” synthesis based on voice formants 6 . In
this piece, the input device activates different synthetic voices and modifies parameters.
In general, the accelerometer data corresponding to the forward, backward movements
are correlated with dynamics (brightness and intensity) and the left-right movements
are correlated with pitch (glissandi) while the central axis applies a speed tremolo [8].
In the score are noted the part of the flute, the karlax movements laid out on four staves,and the acoustic results (Harmonizers and Synthesis staves) (Fig. 4).

Fig. 4. General score of Le Patch bien tempéré III by Tom Mays (mes.6) (with the permission of the composer). The karlax part combines -movements (“Gesture” staff with circle symbols) which controls intensity, brightness, and pitch-bend of the sound synthesis, -rotation of
the axis (dotted lines) which control a speed tremollo and -continuous keys depression (”Right
Hand” and ”Left Hand” staves with thick lines) which activates “paf” synthesis voices. The
numbers at the top of the score represent the time in seconds. See video from 01:44 to 02:00
https://vimeo.com/80464641

6

Phase Aligned Formant developed by Miller Puckette in 1995

35

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Karlax as model
The design of the karlax can also inspire the composition and constitute a modelin itself. Indeed, this controller is conceived by being inspired by the keys system ofwind and keyboard instruments (pistons and continuous keys) enriched with an axis
(with bends) and movement sensors (accelerometer and gyroscope). The instrumental
aspect of the karlax is developed among others in the introduction of the Faber’s piece(B). Indeed, the instrumentalist performs a “call” thanks to the pistons produced byshort harmonic synthetic sounds. The play of the karlax can be compared to the play ofpistons of a trumpet (Fig. 5). Also, the possibilities of the karlax can inspire the ”trajectory” of the piece. For instance, Discontinuous Devices (E) starts with an extensive
use of the pistons and then in the second section the karlax triggers and controls longsequences through the accelerometer and gyroscope data, making the karlax gestures
more and more expressive.

Fig. 5. “Call” played by the Karlax pistons in Frottement, Bourdon, Craquement (mes. 1-2) (withthe composer permission). See video from 00:00 to 00:04 https://vimeo.com/118148219

5Interaction Metaphors from Computer MusicIn this part, we analyze excerpts of the corpus pieces thanks to metaphors from Computer Music. We have selected five metaphors from three articles: [9], [10], [11], fortheir relevance to describe the action of a gestural controller such as the karlax (particularly in interaction with acoustic instruments) and for their capacity to give an overview
of compositional strategies.
“Shaping” [Caramiaux et al., 2014]
Shaping “refers to scenarios where performers control sound morphologies by“tracing” in the air those salient sound features they desire to control”[9]. This metaphor
is described as the “transfer of variations into a gestural morphology” and as synchronization of sound with movement. It is widely used in the pieces thanks to Karlax motion sensors but also with continuous keys. For example, in (C), the karlax imitates the
distortions of the flute sound (created by harmonizers, flatterzunge, etc.) by “shaping”
the “paf” synthesis. At the same time, the ancillary gestures of the flutist seem to imitate the gestures of the controller (Fig. 4). With a more reduced gestural expression,36

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

the continuous key activation allows the karlax performer in (A) to “shape” the spectralenvelope in a differentiated way to provide a harmonic accompaniment to the violin andthe cello (Fig. 2).
“Catch and Throw” [Wessel & Wright, 2002]
This strategy of interaction “involves the notion of selectively trapping musicalphrases from an ongoing performance, transforming them in some way, and then sending these transformed materials back into the performance”[10]. This way of interaction, which could be defined as delayed real-time processing, is exploited in improvisational situations by Tom Mays in the early 2010’s, where the direct sound of the acoustic
instrument is captured, transformed by the karlax and broadcast in real-time7 . This typeof interaction is also employed at the end of Jouaville’s piece (G) where the acoustic sound of the violin is processed by resonator, delay, and pitch shift modules (GRM
Tools) whose parameter nodes are controlled by the karlax movements. This brings asonic halo to the violin8 .
“Fishing” [Caramiaux et al., 2014]
This metaphor is related to the learning stage in gesture recognition. When a gesture is recognized by the dedicated program, a sound will be “fished” out to be played.
One can compare this scenario of interactions with certain compositional strategies. For
example, at the beginning of (A), several violin and cello actions with obvious gesturalcharacteristics such as jettato, glissandi, strokes on the body of the instrument seemto be “recognized” by the karlax, which reacts by imitating gestures, triggering andtransforming nearby sounds9 .
Musical tasks [Wanderley & Orio, 2002]
In the same idea as the composition model based on instrumental playing presented above (see Karlax as model), the article [11] proposes two levels of metaphors:
Musical Instrument Manipulation Metaphor and Other Metaphor. In the first category
are listed the interactions metaphors that refer to traditional instrumental playing (isolated notes, basic musical gestures like glissandi, vibrato, musical phrases, rhythmic
playing, etc.) that appear for example in Faber’s piece with the ”call” (Fig. 5). In thesecond category, the authors evoke the actions of triggering of sequences but also theirorganization in time: synchronization, envelope control, continuous modulation features, etc.
“Space” [Wessel & Wright, 2002]
The purpose of using a control interface like karlax in this type of strategy is to“suggest musically interesting trajectories for gesture [10]”. Moreover, the article emphasizes the importance of proximity and timbre in the perception of these trajectories.
7

In this video, the karlax controls the transformations of the acoustic sound of a Sheng, a mouthblown free reed instrument: https://www.youtube.com/watch?v=fg9TgbI4gTM
8
See video from 05:43 to 06:42 https://www.youtube.com/watch?v=IrCmiwwFSUs
9
See video from 00:00 to 01:10 https://vimeo.com/67049071

37

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

In addition, various strategies to suggest movements and trajectories are employed by
the composers of the corpus. For example, in Ripples Never Come Back (D), the composer evokes a distancing through repeated sequences where the violin and cello instruments begin a quasi homorhythmic figure which is “taken up” by the electronic part
performed by the karlax in the form of arpeggios towards the high register. The karlax
controls a flow of notes produced by a subtractive synthesis: the axis controls the pitchof the arpeggio, the continuous keys control parameters like volume, filtering or speed
while the inclination combined with a key activation controls the envelope (Fig. 6).

Fig. 6. Sequence that evokes a distancing in Ripples Never Come Back by Michele Tadini (mes.
32) (with the composer permission). See video from 00:48 to 01:00 https://vimeo.com/72995021

38

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

6

Discussion

The use of compositional models and Computer Music metaphors provide a framework
and powerful analytical tools to apprehend pieces that appear at first sight very complex.
It allows to categorize certain roles of the karlax in this precise context, with a small
number of acoustic instruments, and allows to discuss situations.
For example, the piece (A) seemed to us to belong to both the first and the second
composition model, depending on whether one considers the process of composition or
the sound result. Indeed, the process of additive synthesis and the fact that the “target”
sounds are prepared (with the addition of pegs) make the sound synthesis played by
the karlax particularly distant from the acoustic sound of the violin. From a perceptual
perspective, we would then need to determine whether or not the timbre of the sounds
played by the karlax “blends” with the sound of the instrument and determine whatallows us to assert this. For the other examples given for the first model: (E) and (F),we can use the terminology of “timbral augmentation” as presented in [12].
The selected metaphors are thought in real-time interactions context. While thecomposition process necessarily evolving in a delayed time, we have seen that these
metaphors are proper to comment on typical situations of the pieces of the corpus.
Firstly, because they offer situations of real-time transformations and secondly because
the composition strategies in terms of dramaturgy can be compared to situations of improvisations. Moreover, the setup chosen by the Fabrique Nomade ensemble influences
these strategies. As the instrumentalists are independent and trigger more or less random
processes (for example delays), the composer tends to opt for ”encompassing” strategies, highly describable by the metaphors [13]. On the other hand, these metaphors arelimited to comment precisely on temporal and rhythmic aspects as specified in the article [11]. In addition, metaphors that qualify the action of a controller such as Shaping,or Musical Tasks facilitate the interaction with the instrumentalist(s) and the “reading”
of the piece by the spectator/listener as they help to identify acoustically and gesturally
the part played by the karlax.
Another important aspect to qualify the action of the karlax is its notation. Depending on the project of each piece, composers adopt a prescriptive (oriented on theaction of the karlax player) and/or descriptive method of notation (which reports theacoustic result)[14]. As a reference point, the composers of the corpus use the basics
of karlax notation presented in the article [6]. We can mention however the more pragmatic approach described in the Jouaville’s piece (G) which consists in assigning eventsin order of appearance to a simple range of fingerings and allows to visualize the pitches
played by the karlax and movements on a single staff (Fig. 3). Also, it is particularlyinteresting to relate the approach of the composer Andrew Stewart notably in his piece
Ritual (2015) for karlax solo, based among others on gestures categorization and a spatial representation of space in the form of a grid [15]. In general, composers add rarely
information related to mapping and sound synthesis, which would allow performers tofurther appropriate the karlax instrument. Simultaneously, the notation must be practical and represent the composer’s intention in a precise and concise way. As such, anindication in the score of the metaphorical context, as presented above, would provide
valuable information about the way(s) the karlax is played and how it interacts withother instrument(s).

39Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

7

Conclusions

In this article we presented an analysis of six pieces for karlax and acoustic instruments.
Three models of compositions have been identified and five metaphors from Computer
Music have been proposed to characterize typical musical situations. To go further, it
seems particularly interesting to deepen the analysis of these pieces by providing a detailed description of their conception and by comparing them both in terms of soundsynthesis, mapping, gestures, notation, and interactions. In addition, it would be interesting to compare the use of the karlax with other DMIs like T-Stick in the same typechamber music context.
Acknowledgments. The authors would like to warmly thank Rémi Dury, Francis Faber,
Tom Mays, Andrew Stewart, Michele Tadini, Lorenzo Bianchi, Raphaël-Tristan Jouavilleand Richard McKenzie for sharing their resources and their time.

References
1. Dobrian, C., Koppelman, D.: The E in NIME: Musical Expression with New Computer Interfaces. Proc. Int. Conf. on New Interfaces for Musical Expression, pp. 277–282 (2006)
2. Palacio-Quintin, C.: Eight Years of Practice on the Hyper-Flute : Technological and Musical
Perspectives. Proc. Int. Conf. on New Interfaces for Musical Expression, pp. 293–298 (2008)
3. Morreale, F., McPherson, A.: Design for Longevity: Ongoing Use of Instruments from NIME
2010-14. Proc. Int. Conf. on New Interfaces for Musical Expression, pp. 192–197 (2017)
4. Hödl, O.: ’Blending Dimensions’ when Composing for DMI and Symphonic Orchestra. Proc.
Int. Conf. on New Interfaces for Musical Expression, pp. 198–203 (2019)
5. Ferguson, S., Wanderley, M. M.: The McGill Digital Orchestra: An Interdisciplinary Projecton Digital Musical Instruments. Journal of Interdisciplinary Music Studies (2010)
6. Mays, T., Faber, F.: A Notation System for the Karlax Controller. Proc. Int. Conf. on New
Interfaces for Musical Expression, pp. 553-556 (July, 2014)
7. Miranda, E., Wanderley, M.: New Digital Musical Instruments: Control and Interaction Beyond the Keyboard. A-R Editions, Inc. (2006)
8. Mays, T.: L’Harmoniseur augmenté : Le dispositif d’écriture mixte dans l’oeuvre Le Patch
Bien Tempéré III”. Proc. Journées d’Informatique Musicale (July 2015)
9. Caramiaux, B., Françoise, J., Scnell, N., Bevilacqua F.:Mapping Through Listening. Computer Music Journal, 38(3):34-48 (Fall, 2014)
10. Wessel, D., Wright, M.: Problems and Prospects for Intimate Musical Control of Computers.
Computer Music Journal, 26(3):11-22 (Fall, 2002)
11. Wanderley, M., Orio, N.: Evaluation of Input Devices for Musical Expression: Borrowing
Tools from HCI. Computer Music Journal, 26(3):62-76 (Fall, 2002)
12. Touizrar, M., McAdams, S.: Aspects perceptifs de l’orchestration dans Angel of Death deRoger Reynolds: Timbre et groupement auditif. Dans P. Lalitte (Ed.), Musique et cognition :Perspectives pour l’analyse et la performance musicales, (pp. 55-78). Editions universitaires
de Dijon (2019)13. Dahl, L., Wang, Ge.: Sound Bounce: Physical Metaphors in Designing Mobile Music Performance Proc. Int. Conf. on New Interfaces for Musical Expression, 15-18 (June, 2010)
14. Kanno, M.: Prescriptive notation: Limits and challenges. Contemporary Music Review,26(2):231-254 (2007)
15. Stewart, A.: Karlax Performance Techniques: It Feels Like... Proc. Int. Computer Music
Conference (2016)40