# Reflecting on research-creation methodologies in the development of distributed applications for live performance

Jean-Brice Godet, Aliénor Golvet, Suzanne Saint-Cast, Benjamin Matuszewski

## ► To cite this version:

Jean-Brice Godet, Aliénor Golvet, Suzanne Saint-Cast, Benjamin Matuszewski. Reflecting on research-creation methodologies in the development of distributed applications for live performance. Web Audio Conference 2025, Ircam, Nov 2025, Paris, France. 10.5281/zenodo.17642070. hal-05502536

HAL Id: hal-05502536

https://hal.science/hal-05502536v1

Submitted on 10 Feb 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Reflecting on research-creation methodologies in the development of distributed applications for live performance

Jean-Brice Godet

Musician, Compagnie LpLp
jeanbrice.godet@gmail.com

Suzanne Saint-Cast

Ircam-STMS UMR9912
suzanne.saintcast@ircam.fr

Aliénor Golvet

Ircam-STMS UMR9912
alienor.golvet@ircam.fr

Benjamin Matuszewski

Ircam-STMS UMR9912
benjamin.matuszewski@ircam.fr

## ABSTRACT

In this paper, we propose to report and reflect on a long-term collaboration between researchers and a musician from the improvisation scene, to design and implement a Web-based distributed system - composed of 8 Raspberry Pi nano-computers equipped with speakers and batteries connected together through a local WiFi network - dedicated to co-located settings and live performance. The paper first presents the different applications developed during this collaboration. Then we propose to reflect on some of the aspects of the collaboration that appear salient during the project and that we think have important implications both for design and for the successful implementation of research-creation methodologies. More precisely, we propose to unfold the different socio-economical questions and constraints that we noticed during the collaboration. We examine the learning and appropriation process and analyse it in the framework of the Instrumental Genesis Theory. Finally, we reflect on how artistic practice and design considerations interact in the creation of a Digital Music Instrument (DMI) as a co-evolutionary process.

## 1. INTRODUCTION

Recent development of Web and Internet of Things (IoT) technologies – by democratizing the access to both software (e.g. large JavaScript ecosystem) and hardware (e.g. nano-computers) – have enabled novel possibilities for artists, developers and researchers experimenting with distributed multimedia environments. In the musical field, the adoption of the Web Audio API [1], and the larger appropriation of electronic devices through Makers communities, opens up possibilities of profound changes in music practices, from ideation to production [2]. However, the inherent complexity of such distributed systems – considered as “a collection of autonomous computing elements that appears to its users as a single coherent system” [26] – also opens novel questions

regarding the ways in which practitioners can interact with, and moreover, can appropriate them.

In this general context, the present paper proposes to report and reflect on a long-term collaboration between researchers and a musician from the improvisation scene, to design and implement a Web-based distributed and wireless system dedicated to co-located settings and live performance. Drawing from previous research and developments, the system is composed of 8 Raspberry Pi nano-computers equipped with speakers and batteries connected together through a local WiFi network [15], and running applications developed using Web technologies [14, 16]. Such a light setup allows in particular to deploy the system in unusual places, forms and contexts (i.e. indoor, outdoor, with or without power supply), promoting a relocalization and a dissemination of experimental music. The collaboration has been held in the framework of two research projects: 1) the PhD of one of the authors and 2) the DOTS research project funded by the French National Research Agency (ANR), leading to the creation of *MurMures* by Jean-Brice Godet, a piece for clarinets and ensemble of 8 nano-computers (cf. Fig. 1).

<figure><img src="image_2.png" /></figure>

Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Attribution: owner/author(s).

Web Audio Conference WAC-2025, November 19–21, 2025, Paris, France.

© 2025 Copyright held by the owner/author(s).

<figure><img src="image_3.png" /><figcaption>**Figure 1: The whole setup used by Jean-Brice Godet in the MurMures project**</figcaption></figure>
Such an art/science collaborative project conducted in the very demanding context of live music performance, comes with a number of questions from different perspectives. How to reduce the inherent complexity of such distributed systems to foster appropriation, operability and ease of use? How to reconcile the conflicting requirements for robustness and stability necessary for public concerts on the one hand, and for openness to evolution necessary to research and experimentation on the other hand? On a larger perspective, how to address requirements from the different stakeholders with their own needs, constraints and timelines? These linked and intertwined questions, which goes beyond the strict context of distributed music systems, seem to require, beyond purely technological lenses, an interdisciplinary approach mixing design, development and methodological questions, recognizing and taking into account given socio-economic and socio-technical contexts.

Our goal in this paper is to modestly contribute to these questions regarding some aspects that appeared salient during this on-going collaboration. As such our contributions are twofold: 1) to present the different applications developed during this collaboration and 2) to reflect on different aspects of the collaboration that we think may have important implications in terms of design and in the successful implementation of art/science collaborations and research-creation methodologies.

The remainder of the paper is structured as follows. After a related work section (cf. Section 2), we present in Section 3 the projects that structured the collaboration and the different Web-based applications that were implemented. Then, in Section 4, we discuss three main points which arise during the collaboration. First, we attempt to unfold the different socio-economical questions and constraints that we noticed during the collaboration and that we think have important implications in the design process. Second, we examine the learning and appropriation process and analyse it in the framework of the dual process of instrumentation and instrumentalization. Finally, we reflect on how improvisation practice and design considerations interact in the creation of a Digital Music Instrument (DMI) as a coevolutionary process.

# 2. RELATED WORKS

## 2.1 Co-located Distributed Music Systems

The works presented in this article were influenced by earlier examples of co-located distributed music systems. One of the more emblematic examples is *The Hub*, founded in 1986 following the *League of Automatic Music Composers* [10] (see. Fig. 2). Although each musician in the group played a computer programmed of his own making, their performances were based on the exchange of information via a local network according to a communication protocol defined for each piece. Over time, the group's dynamics and aesthetics evolved through each player's development in their individual practice and instrument creation but also as the group developed new pieces influenced by new technologies (e.g. MIDI, Internet) [9].

While *The Hub* was based on the interconnection of computer programs, the *Ambiguous Devices* ecosystem by Stapletton and Davis [23] aims to build a network of physical musical objects as part of the improvised musical practice of the two authors and with the objective to “better under-

stand how the technology used to facilitate networking could actually play a role in reconfiguring improvised music”. For instance they describe how the signal picked up by a magnet mounted on a turntable would be sent over the network to control a solenoid that would strike strings on another instrument. *Ambiguous Devices* was gradually extended by adding modular artefacts to the system and perpetually reconfigured through the musicians’ practice as new networked connections were created between these artifacts. In particular, the authors insist on improvisation’s “radical openness to the present” as a resource for design and the positive role of ambiguity in the behaviour of instruments [7].

Weinberg proposed the theoretical framework of *Interconnected Musical Networks* (IMN) [24] to describe distributed music systems in which participants are able to directly influence the musical output of their peers using communication channels. An example of IMN is *Voice Network* (2003)[25], a collaborative music installation that allows non-expert users to take part in a collective music-making experience, highlighting the social aspect of group play. The installation comprises four stations arranged in a square, facing each other, with a screen in the middle. Each station is equipped with a microphone, a touchpad controller and loudspeakers. Participants can record sound loops with their microphones and apply sound transformations to them using the touchpad which controls a Max/MSP patch. The network is used to exchange sound loops created between participants. The technical configuration of *Voice Network* and the prominent use of vocal loops were some of the main inspirations behind early versions of our project *Simone* (cf. Sec. 3.1) which was at the time intended to be played by multiple players [8].

The conceptual framework of IMN was later refined by Matuszewski et al. with what they call “Situated Networked Music Systems” using a concept of “interaction topologies” that describe possible interactions between actors [18]. These different topologies were implemented in multiple applications and installations. Among those is *CoLoop* (2017) [22] in which a physical speaker is playing loop sequences that may be created collectively by participants using a web interface available on their smartphone.

<figure><img src="image_4.png" /><figcaption>Figure 2: The League of Automatic Music Composers</figcaption></figure>
## 2.2 Research-Creation Methodologies

Collaboration and synergies between researchers and artists have seen significant growth in the 2000s. In the French-speaking context, the alliance of academic research and artistic creation gave rise to "recherche-création", with "a dual objective: the production of a work (material or immaterial), artifact or original performance, as well as the production of knowledge<fnref n="1" />" [19]. The institutionalisation of recherche-création was initiated in Quebec in the 1990s as a consequence of the transfer of artistic teaching from art schools to university, and spread to Europe, going hand in hand with the entry of artistic practice into higher education and the restructuring of national higher education standards resulting from the Bologna Process. The arrival of artists in university and the introduction of practice-based PhDs prompted debate about the methods and forms of knowledge derived from artistic research, and raised various questions regarding the publication and evaluation of such research. Canadian philosopher Erin Manning suggests that these new forms of knowledge are "extra-linguistic" and "may have no means of evaluation within current disciplinary models" [13]. Such novel approaches enjoin us to "find ways of activating thought that is experienced rather than known", and disclose the embodied, material and affective aspects of practical knowledge.

In the English-speaking context, Frayling proposed in 1993 to distinguish between research into art, research through art, and research for art [5]. Numerous terms have later been proposed: practice-based research, practice-led research, practice as research, research-through-practice, artistic research, creative research, art-based research, and so on. This growing interest for practice as research comes along with a shift of perspective on science. With the emergence of Science and Technology Studies (STS), which have revealed the "constitutive role of material and social practices in the production of knowledge and technologies" [4], science is no longer seen as a "self-contained body of knowledge made of empirical and theoretical propositions about the world" [20], but as practice [21].

Practice-based research modes have reached the field of Computer Science and Human-Computer Interaction (HCI), taking multiple forms, as design practice has become more prevalent within the HCI research community. An influential framework is research through design [5, 6, 28], which employs methods and processes from design practice, in particular its interdisciplinarity and its iterative and dialogic approach, as a "legitimate method of inquiry" [29]. Design is seen as a way to explore and produce knowledge about "situations chosen for their topical and theoretical potential", with the resulting design artifacts "embodying designers' judgments about valid ways to address the possibilities and problems implicit in such situations" and allowing "a range of topical, procedural, pragmatic, and conceptual insights to be articulated" [6].

In the NIME community, technical practice research, introduced by Pelinski et al. [20], "translates a practice research approach to technical practice, focussing on the first-person and real-time nature of practice", instead of conceiving technical practice from a science and engineering perspective as it is usually done. Pelinski et al. propose to locate practice-based research not only in musical performance, but also in technical development and plead for real-time accounting of practice, as a posteriori reports are often made to fit preestablished linear narratives which "neglect the material agencies that enter into play instrument building".

Benford et al. [3] proposed the concept of performance-led research, as a form of research in the wild, focussing on real-life interaction with technology in the context of performance. They highlighted the numerous challenges raised by this research methodology, from contrasting approaches, requirements and temporalities between researchers and artists, to difficulties in documentation or archiving.

Referring to the roots of the concept of technè, Kang et al. [12] put forward "techno-aesthetic encounters" as a theoretical and methodological framework drawing from three modes of inquiry: engineering (linear and problem-solving approach), art (sensitive and first-person approach), and humanities (reflexive and analytic approach). According to the authors, "it intends to explore undiscovered, underdeveloped, and more creative HCI inquiries through alliance, competition, and occasional moments of unity or complementarity between these now largely separated approaches". This epistemological framework is accompanied by three methodological recommendations: 1) cultivate a trustful and supporting environment for people involved in collective experiments, 2) secure 'error-engaged studios' as spaces of safe and open inquiry, and 3) include art-based ethnography to collect rich and multidimensional traces of the interactions and collective explorations.

We find these proposals promising and try to fit our approach into this line of research. Our aim is to acknowledge and reflect on the materiality and non-linearity of research/practice collaboration in real-life contexts. We think that long-term engagement and dialogue can foster richer interactions – enabling to alternate “performance-led” and “research-led” projects, and mitigating the extractivist tendencies of research.

## 3. PROJECTS AND APPLICATIONS

<figure><img src="image_5.png" /><figcaption>Figure 3: Chronology of the collaboration</figcaption></figure>

The collaboration started in 2022 when Jean-Brice Godet was commissioned to design the sound for a performance called Espace de gratuité, taking place in a 40 x 5 meter installation containing 5854 objects selected by stage director Ema Drouin. In this context he reached the Sound Music Movement Interaction (ISMM) team from Ircam to have a state of the art of the technical possibilities to deploy a wireless portable multichannel system. In return, the team

<footnote n="1">¹translated from French</footnote>
offered Jean-Brice a collaboration with Aliénor Golvet in as part of her PhD project. He then started to use *Simone* (cf. Sec. 3.1), a distributed instrument using audio mosaicing synthesis, to test and improve it, as well as to include it in live improvised performance.

For the premiere of *Espace de gratuité* in 2023, Jean-Brice needed a simple application that could play pre-recorded sound files (text reading, field recordings...) with or without delay effect, which led to the development of *Nina* (cf. Sec. 3.2). In 2024, during the development of the mixed composition *MurMures* for double-bass clarinet and electronics, the set of available applications was extended with a Step Sequencer (cf. Sec. 3.3), to introduce time based and rhythmic patterns. Additionally, a re-implementation of *Grainfield* (cf. Sec. 3.4) [17], which allows to create a sort of granular reverberation of the musician's performance, appeared an interesting compromise to connect the acoustic and electronic worlds. Finally, to allow the use of these four applications within a single performance, we developed a mixer application, the *Metamixette*, which allows the performer to control the relative volume of each application and to create transitions between them from one single interface.

During the lifetime of the collaboration, the system and the different applications were used in different contexts, such as concerts, installations or pedagogical settings.

## 3.1 Simone

<figure><img src="image_6.png" /><figcaption>Figure 4: The *Simone* controller interface</figcaption></figure>

*Simone*<fnref n="2" /> is a distributed instrument controlled through a web interface meant to be played by a single instrumentalist. The sound is produced by a network of any number of Raspberry Pi computers (each called *satellite*) equipped with loudspeakers. Sound synthesis in *Simone* is done using a technique called audio mosaicing[27, 11]. This technique, which can be seen as the audio analog to the more widespread concept of *photomosaics*, consists in reconstructing an audio signal (thereafter named the *model* signal) with elements from another signal (the *generator* signal). To do this, the *model* signal is cut into small audio segments (called *grains*). For each grain, we look for the most similar grain in the *generator* file, according to a chosen metric and audio descriptors. The resulting sound is the concatenation of grains from the *generator* signal. The synthesized sound then follows the temporal evolution of the *model* signal but with the timbre of the *generator* signal. *Simone* is operated through a web interface that offers the option to: 1) record a sound with the microphone and use it (either in real

time or by looping a section of a pre-recorded sound) as the *model* sound that controls audio mosaicing on all *satellites* simultaneously. 2) control various synthesis parameters on user-defined groups of *satellites*.

## 3.2 Nina

<figure><img src="image_7.png" /><figcaption>Figure 5: The *Nina* controller interface</figcaption></figure>

*Nina*<fnref n="3" /> is an application (cf. Fig. 5) that aims at providing simple ways to playback sound files on an arbitrary Raspberry Pi, in an approach inspired by *acousmatic* systems. For each connected client (on top of the interface), the user can select a sound file, start and stop its looped playback, control its volume and optionally pipe it through a feedback delay effect. The playback of the audio file is done using a granular synthesis technique, where the position of each consecutive grain is chosen in such a way that the source audio file is reconstructed without distortion of time. This allows the user to either play the file such that it is almost identical to the original, but also to be able to alter it via the manipulation of the grains' period and duration. A second part of the control interface (on the bottom of the interface) allows to trigger playback of a list of audio files on every connected client.

## 3.3 Step Sequencer

<figure><img src="image_8.png" /><figcaption>Figure 6: The *Step Sequencer* controller interface</figcaption></figure>

The *Step Sequencer* application allows the user to create rhythmic patterns that can be arbitrarily distributed on the connected Raspberry Pi devices. The application enables the creation of multiple sequencers running in parallel. A sequencer is defined by a given number of steps and a BPM. The user can add and remove tracks and define on which client the track is played through a diffusion matrix. The score as well as the diffusion matrix can be saved as presets that can be recalled at runtime, allowing the creation of evolving rhythmic structure in time. Future evolution of

<footnote n="2">²https://github.com/ircam-ismm/simone-v2</footnote>

<footnote n="3">³https://github.com/ircam-ismm/nina</footnote>
the application will allow the scheduling of preset change to be triggered on a defined beat, as well as the possibility to synchronize the playback start of sequencer on a predefined beat of another sequencer, opening the door for precise and complex polymetrical and polyrhythmatic patterns.

## 3.4 GrainField (embedded)

<figure><img src="image_9.png" /><figcaption>Figure 7: The GrainField (embedded) controller interface</figcaption></figure>

GrainField (embedded) is a reimplementation of the GainField described in [17]. In GainField, the sound of the performer is constantly recorded and cut into overlapping chunks that are sent round-robin to each of the 8 Raspberry Pi segment groups (see Fig. 8).

<figure><img src="image_10.png" /><figcaption>Figure 8: The segmentation of the recording into chunks that are sent round-robin to each Raspberry Pi.</figcaption></figure>

The sound segments received on the Raspberry Pi are then played back through a granular synthesis engine whose parameters (e.g. grain period and duration, detune) can be controlled from the controller interface shown in Fig. 7. The overall sonic result can be described as a sort of constantly evolving granular reverberation of what is played by the instrumentalist.

Additionally to these general principles, the application also allows the performer to start and stop the recording at runtime, enabling a possibility of freezing the memory of the system and thus the audio rendering at runtime. Finally, the application also allows the user to script and automate some parameters of the application (e.g. fade-in, fade-out) though an embedded code editor.

## 3.5 Méta-Mixette

The Méta-Mixette application (cf Fig. 9) provides a way for the user to control, and thus mix, the volume of all clients of the other applications. It also provides a scripting ability that allows to prepare transitions and presets, e.g. to cross-fade between two applications, that can be triggered at any moment using MIDI controllers.

<figure><img src="image_11.png" /><figcaption>Figure 9: The Méta-Mixette controller interface. On the left, the mixing tables that allow to control volumes of every client of each application. On the right, the scripting interface that allows the user to prepare automations that can be dynamically launched at runtime.</figcaption></figure>

# 4. DISCUSSION

In the section, we propose to report and reflect on different aspects of our collaboration that we think can be of interest in the design of new instruments and in such a recherche-création setup.

## 4.1 Socio-economic contexts and constraints

From a general perspective, we think it is important to first report on some of the socio-economic constraints that can occur in art / science collaborations, as we think they are generally overlooked while they can be of great importance regarding some design choices and constraints.

To demonstrate our point, it's interesting to consider a performance context as a form of contract between a performer or a set of performers and an audience. Hence, if the contract is not considered as fulfilled by the audience, the direct consequence for the performers might be that there is no payment for the performance, probably no other chance to perform, and therefore, no possibility to make performance as a living income for the performer(s). From this simple consideration, consequences will flow that will impact the process of design and appropriation of a new instrument.

The first obvious one is that professional musicians cannot bring a prototype that they do not master on stage, as there is no second chance in a live performance (at least if not explicitly written in the contract with the audience). This basically means that during a performance only a very few number of "crashes", be them musical or technical, are allowed. Hence, the performer needs to invest an important amount of time to practice and co-develop a tool that cannot be immediately used on stage. However, and somehow contradictory, developing a practice and furthermore a mastery of an instrument requires to use it in multiple contexts (e.g. in concert halls or outdoor, in solo or within collaborations). Each specific context allows the performer to discover specific use, possibilities or limits of the instrument. Only by multiplying the contexts can they test different usages and adapt the system to specific needs. This leads us to the question, that may have institutional consequences, of how to create concert situations in which the possibility of experimenting is accepted and recognised as valuable by the public?

Second, mastering an instrument on stage means developing a deep embodied knowledge of the gestures to control the instrument, so that the technique can somehow disap-
pear during the performance to give the place to the artistic
discourse. This learning phase is necessarily long (see for
example the years of practice that are necessary to mas-
ter a traditional instrument), and maybe more specifically
with an instrument that is being designed and is thus evolv-
ing in real time. Indeed, the performer will have to teach
herself without any existing repertoire or help from an exist-
ing community of practice. Additionally, this learning and
development of a specific gestural practice may have to
be co-constructed alongside a specific artistic vision and dis-
course enabled by the instrument. Finally, the interaction
between the development team and the performer during
the co-development of the instrument is also defined by a
number of constraints, e.g. feedback, bug report (all sorts
of things that have to be learned too) that can slow down
further the appropriation process. For example, in some sit-
uations one or the other can be stuck for days because of a
bug, a missing element in the reporting or misunderstand-
ing, just because the other end is not available.

While these considerations may appear relatively trivial,
we think they must be fully acknowledged and recognised in
the implementation of research / creation methodologies.

## 4.2 Learning and appropriation

For Jean-Brice the first steps of appropriation – beside
the mere fact of being able to plug and run the system
– consisted in trying to get a basic understanding of the
sound synthesis process and to infer which sound is going
to be produced by a determined input or gesture. As he
specifically started working with Simone, he needed to un-
derstand and appropriate the behavior of mosaicing on a
distributed system composed of 8 devices. Therefore, he
started by selecting a set of basic *generator* sounds that he
describes as “simple musical elements” to understand how
the system would react to them. Making up what he de-
scribes as a “base grammar”, these sounds, which could be
qualified as “test sounds”, include: sinus, clicks, white noise,
piano chords, clarinet sounds, etc. We can describe Jean-
Brice’s process of appropriation as an additive, combinato-
rial and inferential approach, described by him as follows
“starting from simple elements [...] and to see how I could
combine them, how I could transform them to try to make
sound pieces with them”. This first phase of the learning
process may be comparable to instrumentalists practicing
sound emission and scales. Over time, this soundbank of
“simple elements” was supplemented with sounds originally
coming from the rest of Jean-Brice’s music practice (clarinet
solos, rehearsal recordings, personal recordings, ...).

In the following steps of his appropriation process, Jean-
Brice focused on the development of instrumental gestures.
The support of MIDI controllers (i.e. an 8 channel mixing
table) was seen by Jean-Brice as a definite “step” that had a
significant change in his way to play the instrument. With
this novel feature, he was able to get over the limitations
of mouse-based web interfaces to define new gestures, to
change multiple parameters at the same time and to define
control groups of several Raspberry Pi computers. The em-
bodied and gestural relationship with the system, where the
technique and the gestures can be forgotten in favor of the
musical ideas, contributed to make it a (real) instrument.

During this process of development, one idea that guided
the design was that one application should offer enough pos-
sibilities and have the potential to generate enough music

materials to cover the needs of the creation of a full concert.
However, after numerous upgrades in this direction, Simone
became bloated, too complicated to use and over-specific. To
avoid that problem, we decided to keep Simone as simple as
possible and to adopt the strategy of having several simple
and generic applications running in parallel, which led to
the implementation of the other applications presented in
Section 3. Those five applications are used simultaneously
by Jean-Brice in *MurMures*.

This learning and appropriation process resonates with
Instrumental Genesis Theory where the development of in-
struments is considered as “a dual process comprising in-
strumentation (i.e., the user’s adaptation to the artifact’s
constraints) and instrumentalization (i.e., the attribution of
functions to the artifact and the technical transformation
of the artifacts by the user)” [?]. Indeed, the first step of
appropriation where Jean-Brice used simple test sound to
create a "base grammar" relates to instrumentation where
the user adapts to the artifact by creating a sort of men-
tal map of its possibilities and constraints. While the phase
of development of his own mappings and gesture vocabu-
lary through MIDI interfaces relates to instrumentalization.
Hence a complex feedback loop appears between the under-
standing of the possibilities and constraints of the system
which in turn allows us to manipulate it and reconfigure it
to adapt it to specific needs, or in other words a dynamic
spiral between what is "doable" and what is "thinkable".

## 4.3 A co-evolutionary process

This period of learning and appropriation never happened
with a fixed state of the system and of the different ap-
plications. Instead, we were constantly engaging in a co-
evolutionary process between engineering, design and artis-
tic questions. Indeed, as improvised music is a very demand-
ing context in terms of adaptability and instant reactions it
is an interesting playground to test and challenge the lim-
its of a system. By observing Jean-Brice practice with the
systems, using his feedback from his usage and discussing
with him, we gradually made changes to the design of the
applications, but also improved lower level engineering ele-
ments such as the audio engine performance or GUI compo-
nents usability. In return, each new feature would provide
more options for Jean-Brice and would have an influence on
the development of playing techniques. Numerous feedbacks
from Jean-Brice can be described as “the repetition of a task
or gesture triggers the need for (of the implementation of)
a new button”. However such an approach in the design,
exemplified above by the proliferating phase in the develop-
ment of Simone, can lead to two possible and not exclusive
problems in the development of an instrument. The first one
is that the instrument ends-up being completely tailored to
one’s practice or even to one’s piece of music, preventing
the reuse of the instruments by others or for other projects.
The second one is that the instrument tries to handle every
possible idea proposed by the user, ending in an application
that somehow loses its "identity". In both cases, these "de-
sign traps" can be a hindrance to the learnability and for
the development of an instrumental practice with the sys-
tem. Over time, it therefore became important for us to set
limits and declare which elements of the design were fixed
and not open to rediscussion.

From another perspective, an important point to discuss
is that the creation of performance (or musical piece) is a
moment of crystallization of the design due to the need for
the performer to have a fixed state of her practice space.
Indeed, this crystallization of the embodied knowledge of
the interface is paired with the crystallisation of the musical
ideas. Hence, any evolution that changes this practice space,
even aimed at simplifying a given task for the performer, can
impede his/her capacity to use the system and therefore to
perform the piece, as a new learning and embodied appro-
priation of the new features is required. We could observe
this issue when, after an update of *Simone* in order to sim-
plify its usage with more than 8 devices, Jean-Brice was no
longer able to play his piece *Clarissime Iris*<fnref n="4" /> that he created
using the old interface.

These two examples tend to show an interesting property
in the articulation between the different aspects, e.g. engi-
neering, design, art, etc, involved in such research-creation
projects. Indeed, while each of these parts of the projects
tends to run in their own feedback loops, with their own
questions and temporalities, sometimes one or the other re-
quires another one to stop its process and/or to crystalize
an aspect of its problem so that the other one can continue
to iterate (cf. Fig 10).

<figure><img src="image_12.png" /><figcaption>Figure 10: Process by which one aspect of the project re-
quires a crystallization and a suspension of the feedback loop
of another aspect of the project</figcaption></figure>

5. CONCLUSION

In this paper, we have reported and reflected on a long-
term collaboration between researchers and a musician from
the improvisation scene, to design and implement a Web-
based distributed system (composed of 8 Raspberry Pi nano-
computers equipped with speakers and batteries connected
together through a local WiFi network) dedicated to co-
located settings and live performance. After a presentation
of the different applications developed during this collabo-
ration, we discussed some of the aspects of the collaboration
that appeared salient during the project and that we think
may have important implications for design, as well as for
the successful implementation of research-creation method-
ologies. More precisely, we attempted to unfold the different
socio-economical questions and constraints that we noticed
during the collaboration. We examined the learning and
appropriation process and analysed it in the framework of
the dual process of instrumentation and instrumentalization.
Finally, we reflected on how artistic practice and design con-
siderations can be seen as coevolutionary processes that in-
teract and sometimes conflict. Importantly, we think that

these findings are closely linked to the long-term engage-
ment and dialogue between the different persons involved in
the project, fostering richer interactions and enabling to al-
ternate “performance-led” and “research-led” projects, hence
mitigating the extractivist tendencies of research.

6. ACKNOWLEDGEMENTS

We would like to thank our colleagues at Ircam for con-
stant support and suggestions in these different projects.
This work has received support from the DOTS research
project funded by the French National Research Agency
(ANR- 22-CE33-0013-01).

7. REFERENCES

[1] Web Audio API Specification, 2021.

[2] B. Bacot and C. Canonne. Musique et hacking : de l'éthique aux pratiques. *Volume !. La revue des musiques populaires*, (16 : 1):7–14, Dec. 2019.

[3] S. Benford, C. Greenhalgh, A. Crabtree, M. Flintham, B. Walker, J. Marshall, B. Koleva, S. Rennick Egglestone, G. Giannachi, M. Adams, N. Tandavanitj, and J. Row Farr. Performance-Led Research in the Wild. *ACM Transactions on Computer-Human Interaction*, 20(3):1–22, July 2013.

[4] H. Borgdorff, P. Peters, and T. Pinch, editors. *Dialogues Between Artistic Research and Science and Technology Studies*. Routledge, New York, NY : Routledge, 2020. |, 1 edition, Oct. 2019.

[5] C. Frayling. *Research in Art and Design*. Royal College of Art research papers. Royal College of Art (Great Britain), 1993.

[6] W. Gaver. What should we expect from research through design? In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, CHI '12, pages 937–946, New York, NY, USA, May 2012. Association for Computing Machinery.

[7] W. W. Gaver, J. Beaver, and S. Benford. Ambiguity as a resource for design. In *Proceedings of the Conference on Human Factors in Computing Systems - CHI '03*, page 233, Ft. Lauderdale, Florida, USA, 2003. ACM Press.

[8] A. Golvet, B. Matuszewski, and F. Bevilacqua. Designing a distributed musical instrument for collective improvised interaction. In *Audio Mostly 2024 - Explorations in Sonic Cultures*, pages 405–420, Milan Italy, Sept. 2024. ACM.

[9] S. Gresham-Lancaster. The Aesthetics and History of the Hub: The Effects of Changing Technology on Network Computer Music. *Leonardo Music Journal*, 8:39, 1998.

[10] S. Gresham-Lancaster. A Personal Reminiscence on the Roots of Computer Network Music. *Leonardo Music Journal*, 27:71–77, Dec. 2017.

[11] J. Janer and M. de Boer. Extending voice-driven synthesis to audio mosaicing. In *SMC'08*, Berlin, Germany, 2008.

[12] L. Kang, S. Jackson, and T. Pinch. The Electronicists: Techno-aesthetic Encounters for Nonlinear and Art-based Inquiry in HCI. In *CHI Conference on Human Factors in Computing Systems*, pages 1–17, New Orleans LA USA, Apr. 2022. ACM.

<footnote n="4">⁴https://youtu.be/10uxYHGUzGw?si=ZMNtASsOw4lhoNGI</footnote>
[13] E. Manning. Ten Propositions for Research-Creation. In *Collaboration in Performance Practice*, pages 133–141. Palgrave Macmillan, London, 2016.

[14] B. Matuszewski. A Web-Based Framework for Distributed Music System Research and Creation. *Journal of Audio Engineering Society*, 68(10):717–726, Oct. 2020. Number: 10.

[15] B. Matuszewski and A. Golvet. Rapid Prototyping of Distributed Musical Things using Web Technologies. In *Proceedings of the 4th International Symposium on the Internet of Sounds*, Pise, Oct. 2023. IEEE.

[16] B. Matuszewski and O. Rottier. The Web Audio API as a standardized interface beyond Web browsers. *Journal of Audio Engineering Society*, (Special Issue on Fostering Creativity through Web Audio), 2023.

[17] B. Matuszewski and N. Schnell. GrainField. In *Proceedings of the Audio Mostly Conference*, London, UK, 2017. ACM.

[18] B. Matuszewski, N. Schnell, and F. Bevilacqua. Interaction Topologies in Mobile-Based Situated Networked Music Systems. *Wireless Communications and Mobile Computing*, 2019:1–9, Mar. 2019.

[19] L.-C. Paquin and C. Noury. Petit récit de l'émergence de la recherche-création médiatique à l'UQAM et quelques propositions pour en guider la pratique. *Communiquer : revue de communication sociale et publique*, 2020. Publisher: Communiquer.

[20] T. Pelinski, A. McPherson, and R. Fiebrink. Ways of knowing, ways of writing: technical practice research in new musical instrument design. *Journal of New Music Research*, 53(1-2):79–92, Mar. 2024.

[21] A. Pickering. 1. From Science as Knowledge to Science as Practice. In A. Pickering, editor, *Science as Practice and Culture*, pages 1–26. University of Chicago Press, Nov. 2010.

[22] N. Schnell, B. Matuszewski, J.-P. Lambert, S. Robaszkiewicz, O. Mubarak, D. Cunin, S. Bianchini, X. Boissarie, and G. Cieslik. Collective Loops: Multimodal Interactions Through Co-located Mobile Devices and Synchronized Audiovisual Rendering Based on Web Standards. In *Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction*, pages 217–224, Yokohama Japan, Mar. 2017. ACM.

[23] P. Stapleton and T. Davis. *Ambiguous Devices*: Improvisation, agency, touch and feedthrough in distributed music performance. *Organised Sound*, 26(1):52–64, Apr. 2021.

[24] G. Weinberg. Interconnected Musical Networks: Toward a Theoretical Framework. *Computer Music Journal*, 29(2):23–39, 2005.

[25] G. Weinberg. Voice Networks: The Human Voice as a Creative Medium for Musical Collaboration. *Leonardo Music Journal*, 15:23–26, Dec. 2005.

[26] A. Wirfs-Brock and B. Eich. JavaScript: the first 20 years. *Proceedings of the ACM on Programming Languages*, 4(HOPL):77:1–77:189, June 2020. Number: HOPL.

[27] A. Zils and F. Pachet. Sony Computer Science Laboratory, Paris. In *DAFx-01*, Limerick, Ireland, 2001.

[28] J. Zimmerman, J. Forlizzi, and S. Evenson. Research through design as a method for interaction design research in HCI. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, CHI ’07, pages 493–502, New York, NY, USA, Apr. 2007. Association for Computing Machinery.

[29] J. Zimmerman, E. Stolterman, and J. Forlizzi. An analysis and critique of Research through Design: towards a formalization of a research approach. In *Proceedings of the 8th ACM Conference on Designing Interactive Systems*, DIS ’10, pages 310–319, New York, NY, USA, Aug. 2010. Association for Computing Machinery.