# Design and Evaluation of Accessible Digital Musical Instruments for Pupils with Neurodevelopmental Disorders

Matteo Olivo, Florence Carrouel, Emily Darlington, Laurent Pottier

► To cite this version:

Matteo Olivo, Florence Carrouel, Emily Darlington, Laurent Pottier. Design and Evaluation of Accessible Digital Musical Instruments for Pupils with Neurodevelopmental Disorders. Sound and Music Computing, ESMAE, Jul 2024, Porto, Portugal. 10.5281/zenodo.14362970. hal-04940944

HAL Id: hal-04940944

https://hal.science/hal-04940944v1

Submitted on 11 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# DESIGN AND EVALUATION OF ACCESSIBLE DIGITAL MUSICAL INSTRUMENTS FOR PUPILS WITH NEURODEVELOPMENTAL DISORDERS

Matteo OLIVO (matteo.olivo@univ-st-etienne.fr)<fnref n="1" />,<fnref n="2" />,

Florence CARROUEL (florence.carrouel@univ-lyon1.fr)<fnref n="2" />,

Emily DARLINGTON (emily.darlington@univ-lyon1.fr)<fnref n="2" />, and

Laurent POTTIER (laurent.pottier@univ-st-etienne.fr)<fnref n="1" />

## ABSTRACT

Although considerations have been proposed by some researchers, little is still known on the design and the evaluation of Accessible digital musical instruments (ADMIs) for pupils with neurodevelopmental disorders (NDDs), and more generally for pupils with special educational needs (SEN). This article reports on the design process and evaluation of a series of ADMIs, specifically developed for pupils with NDDs. The series consists of 5 web applications (a Max/MSP version is also available for some of them) aimed to play a musical composition evoking a natural soundscape. Through a participatory and iterative design process, some modifications to the application interfaces were made, to cope with the users' impairments. The evaluation was carried out through a qualitative approach in a special educational class of a French middle school: 4 pupils with NDDs took part in some musical sessions aimed to test the applications, and in interviews. A special education teacher and a teaching assistant were also involved in the musical sessions and in interviews. Some considerations emerged from the evaluation that will guide the future steps of the design process.

## 1. INTRODUCTION

According to Frid [1], Accessible digital musical instruments (ADMIs) are “accessible musical control interfaces used in electronic music, inclusive music practice and music therapy settings”. These digital instruments are based on different types of control interfaces, such as touchscreen controllers, gaze controllers or mouth-operated interfaces. This variety of devices allows to adapt the instruments to the needs of individuals with disabilities, making music more accessible. The development of ADMIs is grounded on the theory of the Social Model of Disability [2], which argued that people with disabilities are not disabled by their impairments, but by the disabling factors they face in society [3]. According to this theory, it is necessary to implement assistive technologies to overcome the barriers

that disabled people face in music-making [4], and to foster their empowerment process [1].

The design method and the evaluation [5] are key aspects to be considered in the development of ADMIs [6]. Some attempts were made to provide design considerations [7,8] and evaluation frameworks for ADMIs, based on different use cases [9, 10]. However, there are still no established and commonly accepted design and evaluation frameworks for this kind of musical instruments [6, 11].

With regards to ADMIs design, the literature suggests that participatory method is essential [12]. The review carried out by Frid [1] showed that most of the ADMIs are developed following participatory and iterative design methods. Participatory design involves all users in the problem-solving process, allowing an understanding of holistic features of the person-user's interests, needs and preferences [13]. Participatory design takes inspiration from the motto of the disability rights movement “Nothing about us without us” [14]. This means that the design process should ensure that users are in control of the technology, and not just passive recipients of it [15]. Iterative design is a method for developing user interfaces by refining them iteratively, over several versions. Each iteration is subjected to an evaluation, through user testing or other methods [16], to determine whether or not the iteration achieved its goals [17].

Concerning the evaluation of ADMIs, some researchers [18,19] stressed that it cannot be based on frameworks borrowed from Human-Computer Interaction, such as the task-based framework proposed by Wanderley and Orio [20]. Various authors highlighted that the evaluation of Digital Musical Instruments (DMIs) should take in consideration the sociocultural context in which musical interactions take place [6, 18, 21]; Jack et al. suggested that the evaluation of DMIs should be based on a qualitative and reflective process [21].

School music programs can play an important role for the empowerment process of pupils with neurodevelopmental disorders (NDDs). NDDs are intellectual and cognitive disorders that have an onset in the developmental period. They affect a large percentage of the school-age population<fnref n="1" /> and include intellectual disability, autism spectrum

<footnote n="1">¹Laboratory ECLLA, Jean Monnet University, France</footnote>

<footnote n="2">²Laboratory Health Systemic Process (P2S) UR4129, University Claude Bernard Lyon 1, France</footnote>

<footnote>Copyright: © 2024. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</footnote>

<footnote n="1">¹ According to the United States Environmental Protection Agency, NDDs affect the 15% of children between 3 and 17 years old (see https://www.epa.gov/).</footnote>
trum disorder, attention-deficit/hyperactivity disorder, neu-
rodevelopmental motor disorders (dyspraxia, stereotypic
movement) and specific learning disorders (dyslexia, dys-
graphia, dyscalculia) [22]. There is growing research in-
terest in the impact of school-based music activities on the
wellbeing of children with NDDs [23, 24]. However, little
is still known on the actual use of ADMIs in school set-
tings by pupils with NDDs (and, more generally, by pupils
with SEN), and well-grounded design considerations are
rare [11]. The last review on the use of music technologies
in SEN schools was published in 2011 [25]. More recent
publications provided considerations regarding the design
of ADMIs in SEN settings: Förster [11] highlighted the
importance of including students and SEN teachers in the
design process. Frid et al. [15] also stressed the need to
involve different stakeholders in ADMIs design, such as
parents and SEN teachers. Förster et al. [26] and Ward
and Davis [7] highlighted the importance of considering
the context (the school environment).

This article reports on the participatory and iterative de-
sign process and evaluation of some ADMIs, aimed at mid-
dle school pupils with NDDs. The evaluation led to some
considerations that will guide the next iterations of the de-
sign process, and may benefit further research in this field.

## 2. THE ACCESSIBLE MUSICAL APPLICATIONS FOR PUPILS WITH NDDS

2.1 Educational Goals and Technical Features

The ADMIs described and evaluated in this study are 5
musical applications<fnref n="2" />. They are part of a set of 8 appli-
cations, developed by the first author (MO) for his PhD
research. These ADMIs are designed for a school-based
training program, based on writing and performing a mu-
sical composition, aimed to develop life skills (communi-
cation, teamworking, empathy, etc.) [27] of pupils with
NDDs. The composition has to evoke a natural sound-
scape; it is written on the basis of a story invented by the
pupils, which is then sonified using the applications. For
this reason, the applications simulate sounds of a natural
soundscape, or musical instruments (such as the Digeridoo
or the Djembé) that are reminiscent of a natural environ-
ment. The aim of these ADMIs is to provide pupils with
accessible and intuitive instruments that they can quickly
play, without previous musical training. In this way, it
is possible to quickly start the collective work that facil-
itates the life skills development. For each application
a web version exists, developed in Javascript using the
Web Audio API<fnref n="3" />; some customized web audio nodes (Au-
dioWorkletNodes) are developed in Faust [28]. For some
applications a second version is available, implemented in
Max/MSP. Below we provide a description of the 5 appli-
cations (for more technical details, please refer to [29]):

- Soundscape: web application that allows the user to
play recorded sounds from a natural soundscape (birds,
rain, etc.), by clicking on buttons. Some sliders allow the
user to add reverb, delay and a low-pass filter to the sounds.

A Max/MSP version of the application is also available,
which can be used with the AKAI MIDImix controller:
the line faders of the controller trigger the sounds and the
knobs adjust their volumes, the amount of effects and the
cut-off frequency of the filters.

- The Sea: web application that simulates the sounds of the sea and waves through subtractive synthesis techniques. On smartphones and tablets, the sound of the waves can be triggered by clicking on a button or by rotating the device; this system is based on the mapping of the accelerometer, developed with a Javascript Sensor API<fnref n="4" />.

- Didgeridoo: web application for smartphone or tablet that simulates the sound of a didgeridoo through additive synthesis techniques. The movement of the device allows to adjust the angle of a Wah pedal applied to the sound; this system, based on accelerometer mapping, simulates the sound modulations that the musician produces through the vibrations of the lips when blowing into the didgeridoo.

- Wind: web application for smartphone or tablet that simulates the sound of the wind using subtractive synthesis techniques. The mapping of the accelerometer allows to modulate the wind force, according to the movement of the device.

- Djembé: web application that simulates the sounds of the djembe (tonic, slap and bass). A 2-bar sequencer, based on buttons to press, allows the creation of basic rhythms; a slider adjusts the bpm of the sound. A Max/MSP version is also available, that can be used with the AKAI pro MPD 2018 Midi controller: the controller pads trigger the sounds of the djembé and a knob adjusts their volume.

The Midi controllers to be used with Soundscape and Djembé were chosen because they are intuitive and easy to carry, thus suitable for SEN settings.

## 2.2 Design Process

The design of the applications described in section 2.1 is
based on a participatory and iterative process, carried out
in the context of the MO's PhD. In 2021 some early-stage
prototypes were realized. They were mainly developed in
Max/MSP, with only some simplified web versions. Their
design process was based on the analysis of the literature
on ADMIs, and in particular on the works of Ward and
Davis [7] and Graham-Knight and Tzanetakis [8], which
provided some key-considerations: the importance of ap-
plying the participatory method and of providing feedback
of the interaction; the importance of the instrument's ro-
bustness and sound quality; the fact that ADMIs should
be small, cheap, easy to use and should produce sound
quickly. Between February and July 2021 the applications
were tested by a group of 4 adolescents with intellectual
disability (Williams and Beuren syndrome) and a group
of 3 adults with cognitive impairments, outside the school
context: 4 musical sessions were carried out with the fist
group and 5 with the second. These tests allowed a pre-
liminary evaluation of the accessibility of the applications
and their impact on music composition. The results of this
evaluation are presented in this article [29]. In 2022/2023
some prototypes of the applications were further improved,

<footnote n="2">² matteoolivo.com</footnote>

<footnote n="3">³ https://www.w3.org/TR/webaudio/</footnote>

<footnote n="4">⁴ https://www.w3.org/TR/generic-sensor/</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1. Example of web application interface (Djembé).</figcaption></figure>

and others were abandoned. 8 web applications prototypes
are currently available; for 2 of them the Max/MSP ver-
sion has been developed. The 5 applications evaluated in
this study have been chosen because they are considered
to be the most relevant for the composition work; how-
ever, some considerations that emerge from this study are
equally applicable to the other applications.

In 2022 it was decided that the musical training program
(see section 2.1) should be focused on school settings, as
schools play a major role in life skills development [30].
This led to a shift in focus from cognitive and intellec-
tual disabilities in general, to NDDs: indeed, NDDs are
cognitive and intellectual disorders (listed in the Introduc-
tion) that emerge in the developmental period, affecting a
significant proportion of the school-age population. Some
adaptations (see Figure 1) have been made to the applica-
tions to make them more accessible to pupils with NDDs,
compared to other soundscape web apps<sup>5</sup> or educational
softwares [31], and to help pupils use these ADMIs au-
tonomously. The adaptations are based on the tests con-
ducted in 2021, guidelines on accessibility [32] and infor-
mal conversations carried out in 2022 with a SEN teacher:

- adaptations to cope with impairments in movement co-
ordination (dyspraxia): the Max/MSP version of the appli-
cations can be handled by MIDI controllers. In Didgeri-
doo, Wind and The Sea, the triggering of sounds or the
modification of the sound parameters can be controlled by
the device movement. The responsive interfaces of the web
applications automatically adapt to all screen sizes. This
variety of options allows the users to choose the applica-
tion, the device (MIDI controller, smartphone, tablet, com-
puter) and the interaction mode (mouse, rotation, MIDI
controller, touchscreen) that best suit their motor coordi-
nation skills.

- Adaptations to facilitate the comprehension of the operation mode (for web versions), for children with intellectual disability: the functionalities of the applications have been simplified, and the number of UI elements has been reduced as much as possible. The button in the top right-hand corner of the interfaces allows to access a downloadable file, containing simplified instructions on how to use

<figure><img src="image_3.png" /><figcaption>Figure 2. Interface of the application and text explaining the operation mode (The Sea).</figcaption></figure>

the applications (see Figure 2). Labels and small icons ex-
plain the functions of the UI elements. When buttons are
pressed, their colour changes to provide visual feedback of
the interaction.

- Adaptations for children with dyslexia or other reading impairments (for web versions): the text on the interfaces and the instructions (see Figure 2) are written in the OpenDyslexic font <fnref n="6" />, specifically designed for dyslexic people. Other adaptations are made to the texts (font size, line spacing, colour contrast, separation of syllables), to make them easier to read. By clicking on a button in the top left-hand corner of the interfaces, a voice synthesis system (developed using the Javascript Web speech API <fnref n="7" />) allows the user to listen to the instructions on the operation mode of the applications. The voice reading speed can be adjusted according to users' needs.

3. EVALUATION OF THE APPLICATIONS

**3.1 Study Design and Setting**

The evaluation of the 5 musical applications described in
section 2.1 was based on a qualitative method, and was car-
ried out by MO in the Waldeck-Rousseau middle school
of Firminy (France): between November and December
2023, 4 8th grade middle school pupils with NDDs took
part in 4 one-hour musical sessions aimed to test the appli-
cations. The pupils attended the ULIS (Unitée Localisée
pour l'Inclusion Scolaire) class, a specific class for chil-
dren with SEN.

3.2 Participants

4 pupils participated in the study (A, B, C, D). The in-
clusion criteria for the pupils were: (i) 8th grade middle
school pupils, (ii) with NDDs, (iii) attending the ULIS
class, (iv) who agreed to participate in the study by in-
formed consent, (v) whose parents provided informed con-
sent to their involvement in the study. The 4 pupils partici-
pated in the musical sessions and in interviews. They were
affected by the following NDDs:

<footnote n="5">⁵ See an example here: https://soundscape.world/</footnote>

<footnote n="6">⁶ https://opendyslexic.org/</footnote>

<footnote n="7">⁷ https://wicg.github.io/speech-api/</footnote>
- A: autism and attention-deficit/hyperactivity disorders.

- B: moderate intellectual disability, dyslexia, memory and communication impairments.

- C: dyslexia.

- D: moderate intellectual disability, dyslexia.

Also the teacher and the teaching assistant [33] in charge of the ULIS class (henceforth referred to as “the professionals”) took part in the musical sessions and in an interview.

MO conducted the evaluation. He was a PhD student at the Jean Monnet University of Saint-Etienne, with a background and professional experience in the fields of disability, assistive technologies and computer music.

## 3.3 The Musical Sessions for the Evaluation of the Applications

The 4 one-hour musical sessions took place in the ULIS classroom. A, B and C participated in all the sessions; D did not take part in the third one. The teacher in charge of the ULIS class was involved in all the sessions, whether the teaching assistant only in the third one. The professionals supported the pupils in the activities and helped them to maintain the concentration. The following applications were assigned to the pupils, on the base of the preferences expressed during the first interview:

- A: Didgeridoo (tested on smartphone), The Sea and Soundscape (web version tested on computer, and Max/MSP version tested on AKAI MIDImix controller).

- B: Soundscape and Djembé (tested on computer).

- C: The Sea (tested on smartphone), Soundscape (web version tested on computer, and Max/MSP version tested on AKAI MIDImix controller).

- D: Djembé (tested on computer), Didgeridoo and Wind (tested on smartphone).

During the first session, MO presented the aims of the study and provided pupils with basic knowledge on soundscape and electro-acoustic music. Afterward, an activity was carried out: firstly, the pupils had to listen to the explanations provided by the voice synthesis system, and to read the instructions in the downloadable file (see Figure 2). Then, they had to test the assigned applications on the basis of the instructions received. This activity continued throughout the second session. During the third and the fourth sessions the pupils created an imaginary story, through a brainstorming process based on the use of sticky notes. The story was then sonified with the applications: with the help of MO, the pupils wrote a musical score using a simplified scheme based on blocks of different colors. Each block represented a sound. The piece was played several times by the pupils using the applications.

## 3.4 Data Collection and Analysis

The pupils were involved in 2 face-to-face semi-structured interviews, before the beginning (questions 1-6, Table 1) and at the end of the session series (questions 7-11). For question 8 some visual aids<fnref n="8" /> were used, to help pupils express their feelings. The professionals participated in a

face-to-face semi-structured interview (questions 12-29), at the end of the session series. All interviews were audio-recorded. MO conducted the interviews and a participant observation of the musical sessions.

The evaluation framework used in this study assessed the following properties:

- pleasure generated by the use of the applications (assessed through observation and interviews with pupils);

- impact of the applications on the creation and performance of the musical composition (assessed through interviews with pupils);

- accessibility of the interfaces (assessed through observation and interviews with the professionals);

- adaptability of the applications to the context and to the users' needs (assessed through observation and interviews with professionals);

- playability (assessed through observation);

- sound quality (assessed through observation).

This framework was conceived on the basis of the literature [7,9,10] and the results of the evaluation conducted in 2021.

Data were analyzed using qualitative content analysis. The interviews were transcribed verbatim and the transcripts were returned to the professionals for feedback. The transcriptions and observation field notes were analyzed and sorted into categories and sub-categories.

## 3.5 Ethics Statement

The study protocol was approved by the ethics committee "Terre d'Éthique", University Hospital Center of Saint-Etienne, France (ref. IRBN882023/CHUSTE), on August the 21st, 2023. Informed consent was obtained from the pupils and their parents, as well as from the professionals.

# 4. RESULTS

## 4.1 Pleasure

All pupils expressed the wish to try musical applications (question 5, Table 1), because of the desire to discover something new and to learn music. All pupils liked making music with the applications (question 7), mainly because of the sounds they produced (2 answers). The most recurrent feelings generated by the use of these ADMIs were "excited, joyful, delighted, satisfied" (question 8). A asked several times to continue using the applications after the session series.

Initially, all the pupils wanted to use the smartphone (question 6); the reasons given by 3 of them were: "I have the habit of using a smartphone", "4G makes me want to use it" and "it's easier". 2 pupils mentioned the computer, and only 1 the MIDI controller, as second choice. 2 participants changed their opinion during the sessions, and expressed the wish to test the MIDI controller. At the end of the musical session series, 2 pupils affirmed (question 9) that they preferred to use digital applications for the music composition, rather than traditional musical instruments.

<footnote n="8">⁸ The “emotions cards”, see https://www.cres-paca.org/.</footnote>
<table><caption>Table 1. Interview guides</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Pleasure (interviews with pupils)</strong></td><td>1. Have you ever played music in your life?<br>2. What musical instruments did you play?<br>3. Have you ever used software or apps to play or create music? Which ones?<br>4. Did you enjoy using these software or apps?<br>5. If you never used them, would you like to try them?<br>6. If you could choose between playing music with a smartphone, a computer or a MIDI controller, which device/s would you choose?<br>7. Did you enjoy playing with these apps?<br>8. Can you describe the emotions you felt playing music with these applications?<br>9. What would you have preferred to use between apps and traditional musical instruments?</td></tr><tr><td><strong>Impact of technologies (interviews with pupils)</strong></td><td>10. Do you think that digital technologies made the composition easier to play, or would it have been easier to play traditional instruments?<br>11. How did technologies help you?</td></tr><tr><td><strong>Accessibility (interviews with professionals)</strong></td><td>12. Do the speed settings on the voice synthesis device fit users' needs?<br>13. What kind of changes would you make to improve the voice synthesis device?<br>14. Is the text on the interfaces legible for the pupils on computers and smartphones? Can you explain why?<br>15. What improvements would you make, to make the text easier to understand?<br>16. Is the text on the operation mode readable for the pupils on computer and smartphone?<br>17. What changes would you make, to make the text easier to read?<br>18. Are the explanations in the text understandable for the pupils? Can you explain why?<br>19. What changes would you make, to make the explanations easier to understand?<br>20. Do the labels and pictograms on interfaces help to understand the functions of the elements?<br>21. What changes are needed to make the functions of the elements easier to understand?</td></tr><tr><td><strong>Adaptability (interviews with professionals)</strong></td><td>22. Are the sizes of the interfaces and the elements suitable for the users' needs?<br>23. Which interaction mode (mouse, touchscreen, smartphone movement, Midi controller) best suits the pupils' needs? Why?<br>24. What are the potential barriers and strengths of each interaction mode?<br>25. Which type of device (computer, smartphone, Midi controller) should be promoted for this kind of activities in schools?<br>26. Do you think web applications are appropriate for the school setting?<br>27. What are the advantages and disadvantages of the web applications used?<br>28. What other types of applications would you recommend?<br>29. Would you like to add any elements?</td></tr></tbody></table>

## 4.2 Impact of the Applications on the Musical Composition

The 4 pupils affirmed that the applications made the work on the musical composition easier, compared to traditional musical instruments (question 10, Table 1). The reasons of these answers concerned the possibility of triggering sound by clicking on a button or moving the smartphone, and the fact of having the habit of using a smartphone (question 11).

## 4.3 Accessibility

- Voice synthesis system (questions 12-13, Table 1): for the professionals, the interface of the voice synthesis system was accessible and the voice was understandable; the playback settings allowed the speed to be adapted to a wide range of user profiles, but at minimum speed, the voice became unintelligible to pupils.

- Labels/icons on UI elements (questions 14-15): according to the professionals, the texts of the labels and the icons on the UI elements of the web applications were legible for the pupils, even on small screens.

- Instructions (see Figure 2) on the operation mode of the web applications (questions 16-19): for the teaching assistant "the text, with the adaptations for dyslexic pupils and the colors, is very accessible". The content of the text was "understandable", but pupils "may not be able to decode some terms" (such as "filter" or "effect"); this was confirmed by the observation. For the professionals, small images should have been added to explain what certain words referred to. In their opinion, the presence of an adult was necessary to help pupils understand and maintain the concentration.

- User interfaces (questions 20-21): on a general level, the professionals found the web applications interfaces accessible, and highlighted the importance of icons and labels to help understanding the UI functionalities.

## 4.4 Adaptability

### 4.4.1 Adaptability to the Users' Needs

- Interfaces adaptability (question 22, Table 1): for the professionals, the size of the interfaces and of their elements allowed a proper interaction with the web applications on all screen types.

- Interaction mode/gestures (question 23-24): for the professionals, touchscreen-based devices are the most suitable for pupils. Pupils use smartphones every day and can move their fingers very quickly on touchscreens. However, for the teaching assistant the choice of the interaction mode "depends on pupils' individual desires" and "it is beneficial to be able to propose the four options".

### 4.4.2 Adaptability to the Context

For the professionals, tablets and computers are the most suitable devices for school settings (question 25, Table 1). Tablets, in particular, are more suitable as they allow pupils to move according to the characteristics of the room. Moreover, for the professionals every pupil should have the same material, to avoid social inequalities. French schools
are normally equipped with tablets and computers, whether
not all the pupils have the same smartphone or the same
type of internet connection. This also emerged from the
observation: only 1 pupil had an unlimited connection, 2
had a limited number of gigabytes and 1 had no connec-
tion. To use the applications, 2 participants had to ask
the others to share the connection, causing distraction and
quarrels. The professionals added that the Akai MIDImix
controller could be another suitable device for pupils with
NDDs: the movements of the faders and of the knobs are
easy to control, because of their tangible dimension, and
the object is attractive. However, schools could not pro-
vide MIDI controllers for every student.

According to the professionals, web applications are suit-
able for school settings (question 26), because no instal-
lation is required. However, some technical constraints
emerged during the sessions: the access to some websites
is not authorized by the French Education System. There-
fore, an authorization was asked by the teacher to access
the MO's website, where the applications were located. On
the pupils' smartphones the access to some websites was
forbidden by the parents. In addition, not all web appli-
cation features are compatible with all browsers. Google
Chrome is the only one that supports all the functional-
ities of the applications tested, but it was not installed on
the school's computers: as they were Microsoft computers,
only Microsoft Edge was installed by default.

4.5 Playability

Concerning Djembé, B showed difficulties in understand-
ing how the sequencer worked. The concept of time se-
quencing was perhaps too abstract and complex for a pupil
with intellectual disability. As regards The Sea, the move-
ment to trigger the waves appeared not intuitive for A and
C: they did not understand that the smartphone had to ro-
tate around only 1 axis, and rotated it on multiple axes.
In Didgeridoo and Wind, D succeeded in modulating the
sound parameters as the multi-axis rotation allowed to do
it. Soundscape appeared the easiest application to use, as
sound triggering was based on buttons to press. No par-
ticular issues emerged on the use of the Akai MIDImix
controller.

4.6 Sound Quality

As described in section 4.1, the sounds generated by
the applications seemed attractive to pupils (question 7,
Table 1). The only issue concerned Didgeridoo: the
amount of distortion should be reduced, to make the sound
more pleasant and realistic. The smartphone has a very low
volume and needs to be amplified to be balanced with the
other devices (amplified by speakers during the sessions).
In Soundscape, not all the sound levels were balanced.

5. DISCUSSION

This article reports on the participatory and iterative design
process and evaluation of some musical applications for
pupils with NDDs. The involvement of the pupils and the

professionals made it possible to collect relevant insights,
that may benefit other researchers in this field.

The use of the applications seemed to elicit a feeling of
pleasure in the 4 pupils: they liked playing with these AD-
MIs and expressed the wish to test them. The smartphone
and the MIDI controller were their favourite devices.

As affirmed by the pupils, the applications had a bene-
ficial impact on the music composition work, making it
easier compared to traditional musical instruments. This
is coherent with the results of the evaluation carried out
in 2021 (section 2.2) [29]: indeed, all the participants in-
volved in this study stated that the use of ADMIs greatly
facilitated the work on the musical composition.

For the professionals, the adaptations described in sec-
tion 2.2 made the interfaces more accessible for the pupils,
but the presence of an adult is still necessary to let
them use these ADMIs. This consideration highlights the
role of teaching assistants [33] in musical activities with
pupils with NDDs. This topic was addressed by another
study [34], which underlined the crucial support provided
by teaching assistants in SEN settings (particularly, in un-
derstanding students' needs and engaging children in the
musical interaction).

Regarding adaptability, for the professionals the
touchscreen-based devices, and in particular the tablets,
are the most suitable for school settings. Pupils have the
habit to use touchscreens and their use reduces inequal-
ities: as French schools are equipped with tablets, every
pupil would have the same material. This is consistent
with another study [11], which confirmed the availability
of tablets in SEN schools and the growing trend of using
touchscreen devices. The professionals also emphasized
the accessibility of the AKAI MIDImix controller, be-
cause of its tangible dimension. This is in line with what
affirmed by Förster and Komesker [35], who highlighted
the power of tangible interfaces to facilitate embodied
interaction and reduce the cognitive load of users with
SEN. However, for the professionals, schools could not
provide MIDI controllers for every pupil, due to monetary
constraints. The issue concerning schools' financial
constraints for the purchase of ADMIs was also raised
by other researchers [11, 34]. The professionals stressed
the importance to provide pupils with several options, to
fit different users' needs. This consideration seems in
line with another study [11]: indeed, one of the teachers
interviewed in this study underlined the need to have a
multitude of ADMIs at school, so that students have the
chance to find the instrument suited their abilities best.
The professionals added that web applications are suitable
for school settings; however, some technical issues
emerged from observation: the browser compatibility of
the Web Audio API, stressed in the literature [36], and
the fact that the French Education System does not allow
access to the websites where applications are located.

As regards playability, one of the main difficulties ob-
served concerned the use of the sequencer on Djembé.
This seems not consistent with what affirmed by Förster
and Komesker [35], who emphasized the accessibility of
sequencer-based instruments: for them, the loop process
reduces the stress of children with SEN, who are not obliged to perform exactly in time. Another difficulty concerned the 1-axis rotation of the smartphone in The Sea. This also seems inconsistent with what affirmed by other authors [25, 37], who underlined the potential of accelerometer-based applications in SEN settings. To overcome these difficulties, more than 4 sessions should have been perhaps implemented for the evaluation.

The interviews with pupils also highlighted that the sound quality influenced the pupils' desire to use the applications. Also this issue was addressed by other authors: Ward and Davis [7] stressed the importance of ADMIs sound quality; moreover, in the above-mentioned study of Förster [11], the teachers interviewed highlighted that ADMIs should sound as pleasant as possible, so that it worst listening to for the students.

## 6. LIMITATIONS

A first limitation was due to the setting and the audience of the study: the ADMIs evaluated in this article are used in a school program aimed to pupils with NDDs. Because of the schools' constraints, and the variety of profiles that children with NDDs can have, it is not possible to develop customized instruments that fit individual needs [11, 34]. Another limitation concerned the fact that the pupils were not involved in the design process since the ideation phase [13]. As explained in section 2.2, the audience of the project and the context were identified more precisely at a later stage of the research. The participation of the pupils in the ideation phase might have provided further insights for the adaptation of the prototypes. Other limitations concerned the evaluation method: due to logistical constraints, the researcher who conducted the musical sessions and data collection was the same who carried out data analysis. This may have influenced the objectivity of the evaluation. In addition, the way interviews were structured did not seem appropriate for pupils with intellectual disability or communication impairments: indeed, B did not answer to several questions.

## 7. CONCLUSIONS AND FUTURE WORK

The considerations emerged from this study will guide the next iterations of the design process of the applications. The sound of The Sea will be improved, and the volume levels of the applications on all devices will be balanced. The playback settings of the voice synthesis will be adjusted, and more images will be added to the instructions (see Figure 2). Wherever possible, web applications should also be controllable via a MIDI controller. In the next evaluation the web applications will be tested on tablets and computers, and more than 4 sessions will be implemented; the participation of teaching assistants will be encouraged and to promote social inclusion and empowerment, groups of pupils with NDDs and neurotypical children will be involved. There should be a division of tasks between the persons carrying out data collection and analysis. More visual aids will be used in interviews with pupils.

## Acknowledgments

The authors would like to thank all the pupils and the professionals of the Waldeck-Rousseau middle school, involved in the study. The authors would also like to thank the Maison des Sciences de l'Homme de Lyon-Saint-Etienne, the IXXI Rhône-Alpes Institute for Complex Systems and the Byosil - Systems Biology Network of the University of Lyon, for funding this study.

## 8. REFERENCES

[1] E. Frid, "Accessible Digital Musical Instruments-A Review of Musical Interfaces in Inclusive Music Practice," in *Multimodal Technol. Interact.*, 2019, p. 57.

[2] E. G. Duarte, I. Cossette, and M. Wanderley, "Analysis of Accessible Digital Musical Instruments through the lens of disability models: a case study with instruments targeting d/Deaf people," in *Front. Comput. Sci., sec. Human-Media Interaction*, 2023.

[3] M. Oliver, "The social model of disability: thirty years on," in *Disability and Society*, 2013, pp. 1024-1026.

[4] K. Samuels, "The Meanings in Making: Openness, Technology and Inclusive Music Practices for People with Disabilities," in *Leonardo Music J.*, 2015, pp. 25-29.

[5] J. Barbosa, J. Malloch, and M. Wanderley, "What does "Evaluation" mean for the NIME community," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Baton Rouge, LA, 2015, pp. 156-161.

[6] F. Lindetorp, M. Svahn, J. Hölling, K. Falkenberg, and E. Frid, "Collaborative music-making: special educational needs school assistants as facilitators in performances with accessible digital musical instruments," in *Front. Comput. Sci., Sec. Human-Media Interaction*, 2023.

[7] A. Ward and T. Davis, "Design Considerations for Instruments for Users with Complex Needs in SEN Settings," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Copenhagen, 2017.

[8] K. Graham-Knight and G. Tzanetakis, "Adaptive Musical Technology: History and Future Perspectives," in *Proc. 41st Int. Computer Music Conference*, Denton, 2015, pp. 416-419.

[9] N. Davanzo and F. Avanzini, "A Dimension Space for the Evaluation of Accessible Digital Musical Instruments," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Birmingham, 2020, pp. 214-220.

[10] E. Frid, *Diverse Sounds: Enabling Inclusive Sonic Interaction*. PhD Thesis, 2019.

[11] A. Förster, "Accessible Digital Musical Instruments in Special Educational Needs Schools - Design Considerations Based on 16 Qualitative Interviews with Music Teachers," in *Int. J. of Human-Computer Interaction*, 2023, pp. 863-873.
[12] E. Zayas-Garin and A. McPherson, "Dialogic Design of Accessible Digital Musical Instruments: Investigating Performer Experience," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Auckland, 2022.

[13] C. Quintero, "A review: accessible technology through participatory design," in *Disability and Rehabilitation: Assistive Technology*, 2022, pp. 369-375.

[14] J. Charlton, *Nothing about us without us*. University of California Press, 1998.

[15] E. Frid, C. Panariello, and C. Núnez-Pacheco, "Customizing and Evaluating Accessible Multisensory Music Experiences with Pre-Verbal Children-A Case Study on the Perception of Musical Haptics Using Participatory Design with Proxies," in *Multimodal Technol. Interact.*, 2022, p. 55.

[16] J. Nielsen, "Iterative user-interface design," in *Computer*, 1993, pp. 32-41.

[17] P. Reimer and M. Wanderley, "Embracing Less Common Evaluation Strategies for Studying User Experience in NIME," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Shangai, 2021.

[18] A. Lucas, J. Harrison, F. Schroeder, and M. Ortiz, "Cross-Pollinating Ecological Perspectives in ADMI Design and Evaluation," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Shangai, 2021.

[19] M. Rodger, P. Stapleton, M. van Walstijn, M. Ortiz, and L. Pardue, "What Makes a Good Musical Instrument? A Matter of Processes, Ecologies and Specificities," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Birmingham, 2020.

[20] M. Wanderley and N. Orio, "Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI," in *Computer Music J.*, 2002, pp. 62-76.

[21] R. Jack, J. Harrison, and A. McPherson, "Digital Musical Instruments as Research Products," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Birmingham, 2020.

[22] D. Morris-Rosendhal and M. Crocq, "Neurodevelopmental disorders-the history and future of a diagnostic concept," in *Dialogues in Clinical Neuroscience*, 2020, pp. 65-72.

[23] L. Kossyvaki and S. Curran, "The role of technology-mediated music-making in enhancing engagement and social communication in children with autism and intellectual disabilities," in *J. of Intellectual Disabilities*, 2020, pp. 118-138.

[24] S. Campbell-Philips, "The Effects of Music On The Outcome of Learning On Children With Disability," in *The Int. J. of Humanities & Social Studies*, 2020, pp. 53-65.

[25] B. Farrimond, D. Gillard, D. Boutt, and D. Lonie, *En-gagement with technology in special educational & disabled music settings*. Youth Music, 2011.

[26] A. Förster, C. Komesker, and N. Schnell, "SnoeSky and SonicDive - Design and Evaluation of Two Accessible Digital Musical Instruments for a SEN School," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Birmingham, 2020, pp. 83-88.

[27] WHO, *Skills for Health: skills-based health education including life skills: an important component of a child-friendly/health-promoting school*. WHO, 2003.

[28] S. Ren, S. Letz, Y. Orlarey, R. Michon, D. Fober, M. Buffa, and J. Lebrun, "Using Faust DSL to Develop Custom, Sample Accurate DSP Code and Audio Plugins for the Web browser," in *Audio Eng. Soc.*, 2020, pp. 703-716.

[29] M. Olivo, "Les instruments de musique numériques pour les personnes en situation de handicap cognitif et mental : exemples d'applications via le web," in *Proc. Int. Conf. Sound and Music Computing*, Saint-Etienne, 2022, pp. 643-652.

[30] WHO, *Promoting health through schools. Report of a WHO Expert Committee on Comprehensive School Health Education and Promotion*. WHO Technical Report Series N°870. WHO, 1997.

[31] J. Tejada, A. Murillo, and M. Berenguer, "Acouscapes: A software for ecoacoustic education and soundscape composition in primary and secondary education," in *Organised Sound*, 2023, pp. 1-9.

[32] C. Allaire, J. Ruel, and A. Moreau, *Communiquer pour tous. Guide pour une information accessible*. Santé Publique France, 2018.

[33] U. Sharma and S. Salend, "Teaching Assistants in Inclusive Classrooms: A Systematic Analysis of the International Research," in *Australian J. of Teacher Education*, 2016, pp. 118-134.

[34] M. Svahn, J. Hölling, F. Curtson, and N. Nokelainen, "The Rullen Band," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Shangai, 2021.

[35] A. Förster and M. Komesker, "Loopblocks: Design and Preliminary Evaluation of an Accessible Tangible Musical Step Sequencer," in *Proc. Int. Conf. New Interfaces for Musical Expression*, Shangai, 2021.

[36] B. Matuszewski and O. Rottier, "The Web Audio API as a Standardized Interface Beyond Web Browsers," in *J. Audio Eng. Soc.*, 2023, pp. 790-801.

[37] J. Quian and J. Sahagun, "An Application of Bluetooth-Enabled Accelerometer-Based Movement Detection for Musical Interactions with Mobile Devices," in *Proc. 3rd Int. Conf. Advances in Computing & Information Technologies*, Dubai, 2023, pp. 26-27.