# Culture and Politics of Machine Learning in NIME: A Preliminary Qualitative Inquiry

Théo Jourdan, Baptiste Caramiaux

► To cite this version:

Théo Jourdan, Baptiste Caramiaux. Culture and Politics of Machine Learning in NIME: A Preliminary Qualitative Inquiry. New Interfaces for Musical Expression (NIME), May 2023, Mexico, Mexico. hal-04075438

HAL Id: hal-04075438

https://hal.science/hal-04075438

Submitted on 20 Apr 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Culture and Politics of Machine Learning in NIME: A Preliminary Qualitative Inquiry

Théo Jourdan

Sorbonne Université, CNRS, ISIR

Paris, France

jourdan@isir.upmc.fr

Baptiste Caramiaux

Sorbonne Université, CNRS, ISIR

Paris, France

caramiaux@isir.upmc.fr

## ABSTRACT

For several years, the various practices around ML techniques have been increasingly present and diversified. However, the literature associated with these techniques rarely reveals the cultural and political sides of these practices. In order to explore how practitioners in the NIME community engage with ML techniques, we conducted interviews with seven researchers in the NIME community and analysed them through a thematic analysis. Firstly, we propose findings at the level of the individual, resisting technological determinism and redefining sense making in interactive ML. Secondly, we propose findings at the level of the community, revealing mitigated adoption with respect to ML. This paper aims to provide the community with some reflections on the use of ML in order to initiate a discussion about cultural, political and ethical issues surrounding these techniques as their use grows within the community.

## Author Keywords

Cultural Studies, Science and Technology Studies, NIME, Machine learning, Interview, Artificial Intelligence

## CCS Concepts

*Human-centered computing → Human computer interaction (HCI); *Applied computing → Sound and music computing*;

## 1. INTRODUCTION

Machine Learning (ML) is among the technologies that have received the greatest attention in the past decade in various academic fields. ML research has grown exponentially. Several scientific fields have appropriated the technology as a necessary tool for inquiry and humanities and social sciences have helped to situate the technology in a historical, cultural, societal and political context. In this landscape, the musical appropriation of ML has contributed to its technological and cultural development. However, while this contribution is explicit in terms of scientific and artistic dis-

semination, it remains implicit from a cultural and political point of view within the community.

ML is primarily seen as a technique and the community has explored its use as a tool to respond to its specific needs. Early works employed ML to build gesture-sound mappings. It was firstly used in early 1990s to create complex mappings from high-level control parameters to sound synthesis parameters [32, 16]. Since then, building input-output mappings with ML remains an important practice for gestural sound control, facilitated through the creation of toolkits [18, 21, 9, 10]. Recently, applications and needs have diversified with improvements in ML-based generative algorithms. Applied to audio synthesis [42, 45, 22, 44] or melodic sequences generation [4, 41, 39], they became a tool to explore new sonic landscapes and develop rich interaction styles. In addition, recent advances in ML have allowed researchers to delegate a degree of autonomy to the system in musical creation, thus exploring forms of musical co-expression [4, 22, 14].

Although ML has become an important research interest for the community, previous work does not yet provide a basis for understanding the culture and politics of ML within NIME. In this paper, our objective is to propose a preliminary study to fill this gap. To do so, we conducted seven semi-structured interviews of researchers or artists of NIME community working with ML. Our main contributions are of two kinds. Firstly, we analyse, based on the experience and practices of the interviewees, the multidimensional relationship between humans and ML-based artefacts, and highlight the progress and obstacles to these specific interactions. Secondly, we explore the socio-cultural implications behind the use of this technology, the ethical issues it raises, which may potentially mitigate its use. More broadly, the critical study of ML through the experiences of NIME researchers can help the community to discuss the benefits, challenges, and good practices related to this technology. As a preliminary qualitative investigation based on seven interviews, the analysis does not intend to cover exhaustively these topics and discussions related to ML in the NIME community, but gives first reading keys and future research directions.

## 2. RELATED WORK

In this section we review previous work in cultural studies of science and technology within the NIME community and then focus on reflections on the use of ML in arts and technology.

### 2.1 Cultural and Science Studies and NIME

So far, NIME community has been looking at his own research under the lens of cultural and science studies. The community is scrutinized, the scientific and artistic objects

<figure><img src="image_2.png" /></figure>

Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s).

NIME'23, 31 May–2 June, 2023, Mexico City, Mexico.
are situated [26, 24]. In doing so, these works propose to reflect on matters that go beyond the themes of the field: authors excavate political stances taken and intended [25, 37], the social and environmental impact of NIME outcomes [35], epistemological concerns about the field as a dual scientific and artistic venue [25], or under-representation and gender-related issues [2, 15, 20, 46]. In the following, we present different research that highlight the intertwining of societal, political, cultural and technical issues, brought to light by a variety of analytical methods.

Hayes and Marquez-Borbon [25] conducted an analysis of the field, involving their own first-person perspective as researchers, teachers and artists. By grounding their essay in the lineage between NIME and the Human-Computer Interaction (HCI) field, they discussed socio-political concerns (e.g. diversity, gender, implicit power structures) and epistemological concerns (e.g. methodology of inquiry, assessment). The lineage with HCI can be sometimes seen as problematic (due to its overwhelming neo-liberal agenda [17]), but also be an ally as similar concerns have been identified in HCI literature (e.g. [3, 29]). Supporting their analysis with works engaging feminist theory and cultural studies, the authors identified axes for action around a certain number of themes (such as diversity and inclusion, quantification or interdisciplinarity).

In addition, the NIME community does not evolve in isolation, and sociopolitical account of its scientific and artistic research can be analysed through external influences. In a recent article, Morreale et al. [37] described musical instruments as cultural artefacts, inherently political. They inspect the influence that NIME can have on other fields, promoting, for instance, a non-techno-solutionist ethos. The impact of NIME outside the community can also be analysed in terms of its impact on the environment and the sustainability of the contributions it enables [35, 38].

This previous work shows the increasing need for reflecting about NIME, as field and community. In this paper, we contribute to this effort by bringing such methods into the case study of ML, which helps us to draw out broader issues and challenges for the community.

## 2.2 ML Studies in Arts and Technology

NIME has a number of works involving ML and AI at its core, and some of them have been mentioned in the introduction. Few works, however, discussed the technology as the main object of inquiry. In this section, we report previous works that are concerned with reflections about the use of ML in NIME and Arts and Technology in general. In a recent article, Fiebrink and Sonami [19] shared their long-term personal experiences to better understand what makes ML valuable for their research, performances and compositions. The article was presented as a series of questions submitted to the artist and a second series submitted to the technology creator. This article highlights the strategies elaborated to use ML in creative practice on the long run and informs design implications for ML-based musical systems. In parallel, Caramiaux and Donnarumma [11] adopted a reflexive approach on their own practice through the description of a long term researcher-artist collaboration. The authors chose to adopt a mixed approach between research-through-practice and artistic intervention in scientific research. They discussed the epistemological implications of their approach, and its influence on the involvement of ML, first as tool, then as an actor of the performance.

In the field of dance and technology, Fdili Alaoui [1] described the collaboration between choreographer, dancer, musician and developer to create a choreographic dance

piece. Through an artistic research methodology, the author analysed the interaction between technology (involving ML) and the act of choreography. She discussed the tensions emerging from the contradictory agenda of technology-driven research often seen in HCI and art-making, provoking the field of HCI on an epistemological opening of its approaches. Finally, Yoshida and Fukasawa [47] self-reflected on the collaboration between an AI engineer and an artist in the context of dance creation. For them, such collaboration goes beyond the simple use of technology in dance and thus beyond a certain technological determinism. They point out the incompatibility between the culture of AI as a problem solver and the requirements of dance creation.

In summary, these works have made explicit often invisible elements of the use of ML in art and music making, through first-person inquiries. In this paper we propose to broaden the spectrum by adopting an interview-based method used in science and technology studies to make the culture of ML in NIME more legible.

## 3. METHOD

To explore the culture and practice of NIME practitioners with ML, we conducted a series of interviews of researchers and artists of the community. In this section, we present the method used and how the analysis has been conducted.

### 3.1 Participants

We interviewed seven researchers and artists being part of the NIME community. All the persons interviewed are anonymized in this article, they are differentiated by a number between one and seven (additional information on the socio-demographic situation of the interviewees is given in Section 5.3 and 6). Some of these practitioners were present in the early days of the creation of NIME conference and are particularly renowned for their work with ML in the field, leading to seminal publications at NIME. Two of them are newer in the community recently published in the conference. We contacted each interviewee via emails. Nine researchers were contacted. Seven could be interviewed, two were not available. Each interviewee received a consent form authorizing us to record their voice, use the transcripts of their interviews for research purposes. All the interviews were conducted online using a secure video conferencing tool. Each interview lasted approximately one hour.

### 3.2 Semi-structured interview

We conducted semi-structured interviews designed to address the researchers' personal experiences of using ML in the context of NIME, their perceptions of the culture of ML in the community, and the ethical issues that this technology may engender. More precisely, the interviews were structured according to the following items:

* **Introduction** After introducing the project by the interviewer, the interviewees were invited to give a general introduction of themselves, explaining the role of ML in their work and, if possible, describe an example of projects that they consider important.

* **Motivations and background** We asked the interviewees what motivated them in using ML, and how this has evolved over time. We also asked about their background (e.g if they have training in ML or related technical fields), what research literature on ML they are interested in (i.e. technical, HCI, ethical, art, ...).
* **Culture of the community** We asked the researchers about the use of ML at NIME, and the specificity of this technology within the NIME community, compared to other artistic and non-artistic communities. We also asked them whether they considered ML to be a common practice within the NIME community and what might be the most significant bottlenecks in the use of ML and, if so, whether they are on a technical or cultural level.

* **Practices and methodology** We questioned the interviewees if they consider having a practice of ML, then asked to describe their workflow. We also focus on the way they build or reuse datasets for their ML system.

* **Actors around the development** We asked who were the other actors involved in the design of systems that include ML in the research (e.g. artists, research collaborators, designers, etc.). The interviewees were invited to describe the type of interaction they have with each of them.

* **Ethical concerns** We asked interviewees if they follow discussions, actions, and published works that may be critical of ML or raise questions about its use. Then we asked them how they include this consideration in their research and, if so, what forms it takes. At last, we asked how this discussion is addressed within the NIME community.

The transcriptions of the interviews were carried out in two stages. First, the interviews were automatically transcribed using Whisper<fnref n="1" /> running locally. Then each automatically transcribed interview was manually corrected using the corresponding audio recording.

## 3.3 Data analysis

Based on the transcriptions, we conducted a thematic analysis, relying on the methodology proposed by Braun and Clark [8]. Analysis of qualitative data began with familiarisation with the data by reading the transcripts twice. Then the authors separately highlighted the relevant parts of the interview and assigned a code to each highlighted part. A code represents "the most basic segment, or element, of the raw data or information that can be assessed in a meaningful way regarding the phenomenon" [7]. Then the codes of both authors were put together and discussed. Codes common to both authors were retained. Codes identified by only one of the two authors were discussed and retained if they were collegially considered relevant. Then, the authors organised separately the codes into themes. The development of the themes represented the interpretation of the authors made from the codes. Finally these themes were put together and discussed until finding a consensus. We converged toward two themes for this analysis, divided into five sub-themes presented into two sections reported in Section 4.

# 4. INTERVIEW RESULTS

We identified two themes from the thematic analysis that we report in this section. The themes are organized in two sections. Firstly, we analyse the considerations made at the level of the individual, researcher or artist. Secondly, we report findings on the cultural perspective at the level of the community.

## 4.1 Individual level

The first theme deals with the technological culture at the level of the individual, whether a researcher or an artist. We present three sub-themes that analyse the subjective practices, the importance of the notion of embodiment and sense making with the technology.

### 4.1.1 Between subjective and normative practices

The use of ML techniques is usually associated to standardised processes for building model architecture, training models, and evaluating model performance. We found that in the NIME community when this rigid process is brought back into musical and interactive practices, it is transformed and reappropriated according to the desires and needs of the practitioners. P1 describe this process in his words:

> "We can have an anti-program to the technology, we take it and we misuse it and we reject the script of the technology. And that's where artists come in, in a very interesting way and misuse the technology."

Anti-program refers to the terminology used by Latour [31] to describe the ways in which users can bypass the use of an innovation that does not conform to the original prescription for its use. So there is no formal way of using ML techniques in the NIME context, "it's a fairly organic thing" as P6 said. P2 talks about instead of focusing on technical aspects, the goal is to "reuse and hack stuff that was already kind of built and customize those to the specific use".

Datasets to train ML systems are also subject to these constraints. Indeed, the way the dataset is structured, what information is given, implicitly prescribes how this dataset should be used and may prevent a certain non-conforming use. So practitioners do not necessarily have the same point of view about re-using existing datasets or even sharing datasets in the community. Firstly re-using and playing with existing datasets can be seen as a form of re-appropriation for personal purposes as describe P3 : "So it would be interesting to have a kind of a pool, a NIME pool of data, of different types of data and audio and other stuff that people can use, reuse, and make their own data sets".

Other practitioners prefer to build their own database to avoid normative behaviour stemming from data aggregation. They promote more personal practices as developed P4: "So specifically in NIME, that's why I'm sort of interested to do what I do in the project is to specifically say this is not big data. You know, this is small data. This is my data. This is myself as one person, not a whole population." Indeed, storing and sharing a dataset raises the question on which norms, biases or standards this dataset has been constructed. Choosing to work with their own, and rather small, datasets, practitioners avoid the problem of normativity and external biases that would stem from big data, at the cost of not taking fully advantage of the latest advances in ML and AI.

### 4.1.2 The quest of embodiment

Embodiment generally refers to how our bodies and active experiences shape our perception, feelings and thinking. Dourish [13] built the foundation of embodiment concept for human-computer interaction. This concept allows analyzing forms of participation and the settings in which interactions occur. Describing how artifacts of everyday interaction play different roles in different contexts and situations. Few studies have attempted to analyze the aspect of

<footnote n="1">¹https://github.com/openai/whisper</footnote>
embodiment of machine learning for musical performance. In music performance, embodiment can be seen as a particular relationship between the body and the instrument, through the action-perception cycle, which makes the instrument an extension of the body [33]. This notion has a certain importance at NIME, as P3 put it: “I think the clear specificity is again, the interaction and embodiment aspect. Embodiment aspect again, is very important in NIME.” But the recent development of ML seems to show that this form of interaction is not sufficiently developed according to P3:

“That’s kind of one of my main critiques about recent developments [of ML], this lack of interaction, or lack of interactive resources of these new models and systems. It’s so distant and disembodied”

Ihde [28] describes an embodied technology as transparent, in a sense that such a technology does not draw attention to itself but to the world it gives access through it. The ability to expressively shape a musical performance, when mediated by technology, implies real-time interaction capabilities between the musician and sound objects [23]. In the context of ML, this may mean the use of simple, lightweight algorithms. As P4 puts it:

“A very simple regression model suddenly becomes very musical, very powerful and satisfies for me because what was interesting was the continuous interaction and the ability to train on the fly. And so these algorithms are light enough, you can literally use them live on stage.”

The “liveliness and spontaneity” described by P4 does not seem to have been achieved with the use of more complex, higher-capacity, models such as deep learning methods, from the point of view of the practitioners. As P3 says for example:

“I think, especially like this new, like deep learning, it’s still a kind of pain [...] to work with it in real time. So it’s not that easy to make it in as part of your daily practice.”

An embodied relation, that refers to an instrument as an extension of the body, implies that the practitioners do not focus on the instrument as a distinct object but rather on the music itself [43]. To achieve that, practitioners seek to better control the behavior of ML systems.

### 4.1.3 Sense making through interactive experience

The approach engaged by the practitioners when working with ML systems is often by practicing it first. Practitioners' backgrounds are diverse, such as HCI, music, philosophy, computer science, but rarely ML. So they develop sense making through experiencing the technology, as explained P5 through the analogy with street musicians:

“We are like street musicians, I would say. You know, we do not really have any training. We just learn, I mean, we just actually really explore the technology and we just dive in. [...] But we have this intention to, try to learn as much as that could be useful for us to do what we want to do. So street musician once more. You learn on the streets.”

Therefore, the usefulness of technology necessarily involves experimenting with it and not necessarily anticipating what could make sense for them. P1 explicitly describe this sense making by practice in their words:

“There’s so many things that we can do, but to really make something interesting and useful, that has to be done. We have to do it. We can not just sit and philosophize about it, I think.”

Experimentation sometimes involves the difficulty of understanding or predicting the behavior of a complex ML model in use, which is sometimes well-received because it can trigger creative processes as explained P1: “And they start to suddenly become very creative. And then you start to get forms and playing with the sound that you put in, your guitar sound, your expression, but it goes in directions that are very surprising.”. P6 uses the term of serendipity to describe this unplanned and sort of autonomous process :“There’s lots of kind of serendipity, really interesting things happening, it seemed like they had a life of their own.”.

Sense making is also important for practitioners to provide to the listeners or the audience an artistic creation that is meaningful for them, as having a relation between what you see and what you hear, as explained P3: “So when we see a mechanical machine, we can see that certain things turning, making sounds, so we can kind of make sense of the process. But when there’s a software that generate things, it’s very hard to make sense.”. P3 goes further by arguing that this lack of sense-making can generate a form of reluctance:

“But then when you start using a more generative or agent-based processes, when you’re not able to make that sense directly, then I realize that some critics can arise.”

## 4.2 Community level

This section explores how the interviewees see ML at the community level, especially they express their opinions about the way ML is received and what critical and political reflections it provokes.

### 4.2.1 Tension between enthusiasm and skepticism

It is part of the culture of NIME to explore new technologies to redefine what it means to make music:

“You can see every two or three years some new technology emerges and you get a lot of new projects around that technology in NIME. [...] I don’t want to use the word fashion because it has negative connotations. It’s more like curiosity of what is this new technology? What people are thinking? What can it do for me? Let’s try. Let’s explore it.” (P1)

However, the use of ML techniques can raise concerns as they are now used in many fields and application areas, with sometimes dramatic societal impacts when they lead to discrimination of minorities or marginalised communities, or to mass surveillance. It is therefore by default full of meaning, as said P2 : “people have a lot of opinions, even if they don’t use it. And some very strong opinions and some people don’t, of course, they really care. But I think most people are thinking about it.” Specifically to the NIME community, one of the criticisms is the use of the ML as an agent with some autonomy that can be interacted with musically: “The more the traditional art musicianship is more careful about the ownership of the music, of the product. [...] And I think tend to be a little careful about not sharing the ownership with other entities like AI or something.” (P3).
The black-box effect when practitioners start working with larger models pushes them to try to introduce more transparency and explainability into the decisions made and the musical interactions. This is a difficult process to conduct and sometimes criticise when applied to the subjective process of musical expression:

> "I'm also using explainability. But when I explain these things to other people, they also get mad. The word explainability is like, you know, no, there's nothing can be explainable. They say that it's kind of like subjective, that yeah, it is subjective. That's true. But more at least we are aiming to make these things to be more controllable, but more interpretable by human musicians, so that they could understand or have an idea how these systems response." (P5).

### 4.2.2 Critical perspectives

The community has always been keen to reflect on its own practices in terms of methodology and practice. But it is only in the last few years that political issues have been addressed. Workshops have been held, for example, on accessibility and gender inequality. The community has developed a strong ethical policy. However, it was noted that issues specifically related to ML tools are rarely made explicit. For example P1 specifies:

> "NIME has got an ethics committee, I'm not sure if they have anything specific about machine learning and their ethics policy, I think it would be a useful thing to do, but it also depends how much machine learning is part of NIME, I wouldn't say it's the dominant force at the moment, it probably will become more dominant."

That being said, one of the first areas of criticism regarding the use of ML tools is energy consumption. Indeed, recent models developed for the general public such as GPT-3, ChatGPT or Stable Diffusion have shown that they are capable of consuming an extremely large amount of energy to operate. This issue also needs to be addressed in the NIME community in terms of how models are designed, trained and then used: "There is a more general discussion about environmental impact of technology, or this technology for marginalized people, or people that don't have. And of course, AI and machine learning are inevitable, you know, and probably most critical. It's not really frugal." as expressed P3. Moreover, P7 refocuses the question around the usefulness of these tools, and what purposes they are used for: "Does it make sense in terms of resources, is it a good use of resources at present to make music ?".

As already pointed by P3 above, ML techniques raise also the issue of unequal access to technologies. For some years there has been a discourse that ML techniques, by automating tasks, would make certain applications more accessible because systems could be more customisable to users' specifications. Now this discourse has been widely criticised, notably on accessibility but not only. P7 discuss this issues related to the inequality of access:

> "There's a bit of an idea that finally we're developing some sort of tools that could be ways of bringing in musical or listening practices, in a broader way. But perhaps we're still recreating an unequal system in relation to the people who really have access to it. It's rather at this level that I think the questions lie."

As a matter of fact, the resources needed to train latest advances in ML research are beyond reach. P6 raised this issue by saying: "So we've got a lot of kind of big style, deep learning stuff going on at the moment, which is really wonderful. But it's also completely inaccessible for artists and people without GPU farms.". This trend raises questions about the interest of using ML techniques for some projects specific to the NIME community: "Would simpler systems without machine learning would allow a better appropriation to a wider audience, that's more of a question." (P7).

## 5. DISCUSSION

We propose to discuss the objective of researchers to break out of the norms imposed by the technologies and focus on the conflicting adoption of ML in the community.

### 5.1 Resisting normative technology

NIME practitioners have elicited ways they engage with datasets involved in training ML models. Certain artists and researchers may prefer to work on personal small data that they can curate, in order to shape the behaviour of the model on stage. Others may prefer to train high-capacity models with aggregated data from their own archives. Authoring is a fundamental element of their practice and authoring stems from the need to avoid norms induced by the technology. These norms conveyed by pre-trained models do not come from the technology itself, as technology as an isolated entity has no reality, but from the various cultural and political human decisions that led to the final artefact (the model). It seems that NIME practitioners resist this and find their way through hacking and misuages. This approach was also taken by the artists using AI in visual arts in the early days of Deep Learning [12], where the use of this technology in an artistic context fell within the hacking and crafting practices of ML as design material.

Through the practice of hacking and misusages, NIME practitioners seem also to be free from a technological determinism dictated by ML. This norm tends to be hijacked by NIME practitioners, avoiding a techno-solutionist tendency which wants that to a given problem, the answer must be technological. However, this perspective is rarely made explicit in their research writings, while it has a strong political implication that is consistent with recent work in the community [37]. NIME practitioners put the emphasis on the interaction and the musical experience rather than the technique used. In other words, the epistemology of NIME does not stricly align with the one promoted in ML research. There is therefore an interesting tension between the use of technology that is by design normative and statistical, and artistic and research needs that can be subjective and open-ended.

### 5.2 Machine learning is political

We reported that there was a tension in the NIME community between, on the one hand, enthusiasm and curiosity about a new technology and, on the other hand, scepticism related to the socio-political forces underlying the technology. Machine learning is a transformative technology that can provide the community with ways to explore new forms of sound expression and artistic creation. At the same time, this technology has shown its downsides, in terms of the inequalities it can create, and its growing ecological footprint [30]. Thus, beyond the fact that the technology is not neutral, its use is not neutral either.

The problem of developing a complex ML system is re-
lated to the black box effect which implies that potential biases, included in the system, are not necessarily explicit and visible. For example, if a population category is not represented when building or reusing datasets, the behaviour of the model will be affected. The development of more transparent and documented systems makes it possible to make explicit the researcher's responsibility for the choices made in training an ML system with specific datasets. Although the consequences may seem less serious in music than in critical applications such as health or justice, it is an issue that is present in the NIME community, highlighting its underlying ethics. NIME members have actively begun to unpack and address some of these issues and tensions through a workshop started in 2020 [34].

Finally, previous works in Science and Technology Studies (STS) and cultural studies have demonstrated that technologies are inherently value-laden, and these values are encoded in technological artifacts (see for instance [5, 40]). These values have been criticized within the ML community, pointing out, for example, how these technologies, originally situated in Western socio-economic and socio-political contexts, could reproduce colonial power relations [36, 6]. Ethical principles and values that guide thinking in Western culture can be challenged or reinterpreted, especially in the way ML is used for music. In this perspective, Huang et al. [27] propose to illuminate ML practices with different East Asian philosophical traditions to decentralise the debate from Western philosophical thinking and advocate for ethical pluralism in the field of AI applied to music. In a complementary way, some authors from the NIME community have noted the importance of a pluralism of perspectives beyond Western thought [2].

## 5.3 Limitations

Our study has limitations that may open up opportunities for further research. Firstly, we have intentionally selected interviewees who chose to integrate ML in their practice, biasing the discussion on more critical views on this technology. It therefore deprives us of the discourse of the majority of NIME researchers that do not use ML and maybe some of them that intentionally chose to not use it for explicit reasons. We are also conscious that gender equality is not respected between the seven interviewees due to deferred interviews that could not be included in this study, biasing also the topics covered in the analysis. This contribution is therefore a first attempt to analyse the cultural underpinnings of ML in NIMEs and should therefore be extended to other interviewees for a better representation of practices and cultures.

# 6. ETHICAL STATEMENTS

Before conducting the interviews, we sought ethical clearance from the Research Ethics Committee of Sorbonne University. The committee considered the methods, recruitment strategies and treatment of personal data. A consent sheet was signed by each interviewee, including the information about the project and the data processing. It was also specified to the interviewees that they could withdraw from the project at any point in time. All personal information is kept safely and anonymously. As a preliminary inquiry, this paper analyzes a first series of interviews presented in this paper that does not fully respect gender equality. The paper also does not claim universality in terms of demographic and socio-cultural representation as the interviewees are either researchers or artists working in Western research centers.

# 7. CONCLUSION

Through this contribution, we sought to better understand the culture and research practices in the NIME community that feature ML technologies. Our results show, at the level of the individual, a subjective engagement, resistant to the normative constraints of the technology and seeking to make sense of the technology through interaction. Furthermore, our findings suggest that at the community level, the adoption of ML is not a given. This technology is divided between enthusiasm and skepticism, which leads to the question of the socio-political implications of its use in music making. This article therefore contributes to a critical study of the field with a focus on ML. In this way, it contributes to the emerging work on the cultural, social and political aspects of our research practice at the interface between art and technology.

# 8. ACKNOWLEDGMENTS

This research was supported by the ARCOL project (ANR-19-CE33-0001) – Interactive Reinforcement Co-Learning, from the French National Research Agency. We want to acknowledge and thank everyone involved in each stage of the research. We want to express our sincere gratitude to the anonymous reviewers for their constructive comments and the interviewees for their time and their thoughts.

# 9. REFERENCES

[1] S. F. Alaoui. Making an interactive dance piece: tensions in integrating technology in art. In *ACM Designing Interactive Systems*, pages 1195–1208, San Diego, United States, 2019. ACM.

[2] J. P. M. Avila, J. Tragtenberg, F. Calegario, X. Alarcon, L. P. C. Hinojosa, I. Corintha, T. Dannemann, J. Jaimovich, A. Marquez-Borbon, M. M. Lerner, et al. Being (a) part of nime: Embracing latin american perspectives. In *NIME 2022*. PubPub, 2022.

[3] S. Bardzell. Feminist hci: taking stock and outlining an agenda for design. In *Proceedings of the SIGCHI conference on human factors in computing systems*, pages 1301–1310, 2010.

[4] C. Benetatos, J. VanderStel, and Z. Duan. BachDuet: A Deep Learning System for Human-Machine Counterpoint Improvisation. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, 2020.

[5] R. Benjamin. *Race after technology: Abolitionist tools for the new Jim code*. Polity, 2019.

[6] A. Birhane, P. Kalluri, D. Card, W. Agnew, R. Dotan, and M. Bao. The values encoded in machine learning research. *CoRR*, 2021.

[7] R. Boyatzis. Transforming qualitative information: Thematic analysis and code development. 01 1998.

[8] V. Braun and V. Clarke. Using thematic analysis in psychology. *Qualitative Research in Psychology*, 3:77–101, 01 2006.

[9] J. Bullock and A. Momeni. MI. lib: robust, cross-platform, open-source machine learning for max and pure data. In *NIME*, pages 265–270, 2015.

[10] B. Caramiaux, A. Altavilla, J. Françoise, and F. Bevilacqua. Gestural sound toolkit: Reflections on an interactive design project. In *International Conference on New Interfaces for Musical Expression*. PubPub, 2022.

[11] B. Caramiaux and M. Donnarumma. Artificial
Intelligence in Music and Performance: A Subjective Art-Research Inquiry. In *Handbook of Artificial Intelligence for Music: Foundations, Advanced Approaches, and Developments for Creativity*. 2020.

[12] B. Caramiaux and S. Fdili Alaoui. "explorers of unknown planets": Practices and politics of artificial intelligence in visual arts. *Proceedings of the ACM on Human-Computer Interaction*, 6(CSCW2):1–24, 2022.

[13] P. Dourish. *Where the Action is: The Foundations of Embodied Interaction*. MIT Press, 2001.

[14] C. Erdem, B. Wallace, and A. R. Jensenius. CAVI: A coadaptive audiovisual Instrument–Composition. In *NIME*, 2022.

[15] G. Essl. On gender in new music interface technology. *Organised Sound*, 8(1):19–30, 2003.

[16] S. Fels and G. Hinton. Glove-talkii: An adaptive gesture-to-formant interface. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, CHI '95, 1995.

[17] T. Feltwell, S. Lawson, E. Encinas, C. Linehan, B. Kirman, D. Maxwell, T. Jenkins, and S. Kuznetsov. "grand visions" for post-capitalist human-computer interaction. In *Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems*, CHI EA '18. Association for Computing Machinery, 2018.

[18] R. Fiebrink and P. Cook. The wekinator: A system for real-time, interactive machine learning in music. *Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)*, 2010.

[19] R. Fiebrink and L. Sonami. Reflections on eight years of instrument creation with machine learning. In *International Conference on New Interfaces for Musical Expression (NIME)*, 2020.

[20] E. Frid. The gender gap and the computer music narrative-on the under-representation of women at computer music conferences. *array.*, (1):43–49, 2021.

[21] N. Gillian and J. A. Paradiso. The gesture recognition toolkit. *Journal of Machine Learning Research*, 2014.

[22] J. Gillick and D. Bamman. What to Play and How to Play it: Guiding Generative Music Models with Multiple Demonstrations. In *NIME*, 2021.

[23] A. Guidi and A. McPherson. Quantitative evaluation of aspects of embodiment in new digital musical instruments. In *NIME 2022*, jun 16 2022.

[24] L. Hayes and A. Marquez-Borbon. Addressing NIME's Prevailing Sociotechnical, Political, and Epistemological Exigencies. *Computer Music Journal*, 2020.

[25] L. Hayes and A. Marquez-Borbon. Addressing nime's prevailing sociotechnical, political, and epistemological exigencies. *Computer music journal*, 44(2-3):24–38, 2020.

[26] L. Hayes and A. Marquez-Borbon. Addressing nime's prevailing sociotechnical, political, and epistemological exigencies. *Computer Music Journal*, 2021.

[27] R. Huang, B. Sturm, and A. Holzapfel. De-centering the west: East asian philosophies and the ethics of applying artificial intelligence to music. In *ISMIR*, 2021.

[28] D. Ihde. Technology and the lifeworld: From garden to earth. 1990.

[29] O. Keyes, J. Hoy, and M. Drouhard. Human-computer insurrection: Notes on an anarchist hci. In

*Proceedings of the 2019 CHI conference on human factors in computing systems*, pages 1–13, 2019.

[30] A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres. Quantifying the carbon emissions of machine learning, 2019.

[31] B. Latour. Technology is society made durable. *The Sociological Review*, 38:103–131, 1990.

[32] M. A. Lee, A. Freed, and D. Wessel. Real-time neural network processing of gestural and acoustic signals. In *Proceedings of the 1991 International Computer Music Conference, ICMC*. Michigan Publishing, 1991.

[33] M. Leman. *Embodied Music Cognition and Mediation Technology*. 09 2007.

[34] C. Martin, F. Morreale, B. Wallace, and H. Scurto. Critical perspectives on ai/ml in musical interfaces. In *Workshop at the New Interfaces for Musical Expression*, 2020.

[35] R. Masu, A. P. Melbye, J. Sullivan, and A. R. Jensenius. Nime and the environment: toward a more sustainable nime practice. In *NIME 2021. PubPub*, 2021.

[36] S. Mohamed, M. Png, and W. Isaac. Decolonial AI: decolonial theory as sociotechnical foresight in artificial intelligence. *CoRR*, 2020.

[37] F. Morreale, S. Bin, A. Mcpherson, P. Stapleton, and M. Wanderley. A nime of the times: Developing an outward-looking political agenda for this community. 07 2020.

[38] F. Morreale and A. McPherson. Design for longevity: Ongoing use of instruments from nime 2010-14. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, 2017.

[39] T. R. Naess and C. P. Martin. A physical intelligent instrument using recurrent neural networks. NIME, 2019.

[40] S. U. Noble. Algorithms of oppression. In *Algorithms of oppression*. New York University Press, 2018.

[41] R. Proctor and C. P. Martin. A laptop ensemble performance system using recurrent neural networks. NIME, 2020.

[42] K. Tahiroğlu, M. Kastemaa, and O. Koli. AI-terity 2.0: An autonomous NIME featuring GANSpaceSynth deep learning model. 2021.

[43] K. Tahiroğlu, T. Magnusson, A. Parkinson, I. Garrelfs, and A. Tanaka. Digital musical instruments as probes: How computation changes the mode-of-being of musical instruments. *Organised Sound*, 2020.

[44] N. Warren and A. Çamcı. Latent drummer: A new abstraction for modular sequencers. 2022.

[45] A. Weber, L. N. Alegre, J. Torresen, and B. C. da Silva. Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise. In *Proceedings of the International Conference on New Interfaces for Musical Expression*, pages 174–179, 2019.

[46] A. Xambo Sedo. Who are the women authors in nime?—improving gender balance in nime research. In *Proceedings of the International Conference on New Interfaces for Musical Expression*. Virginia Tech Blacksburg, VA, 2018.

[47] S. Yoshida and N. Fukasawa. How artificial intelligence can shape choreography: The significance of techno-performance. *Performance Paradigm*, (17):67–86, 2022.