# MusicTraces: Combining music and painting to support adults with neurodevelopmental conditions and intellectual disabilities

Valentin Bauer, Giacomo Caslini, Marco Mores, Mattia Gianotti, Franca Garzotto

<figure><img src="image_1.png" /></figure>

## To cite this version:

Valentin Bauer, Giacomo Caslini, Marco Mores, Mattia Gianotti, Franca Garzotto. MusicTraces: Combining music and painting to support adults with neurodevelopmental conditions and intellectual disabilities. International Journal of Human-Computer Studies, 2025, 207, pp.103675. 10.1016/j.ijhcs.2025.103675. hal-05452250

HAL Id: hal-05452250

https://hal.science/hal-05452250v1

Submitted on 10 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Highlights

MusicTraces: Combining Music and Painting to Support Adults with Neurodevelopmental Conditions and Intellectual Disabilities

Valentin Bauer, Giacomo Caslini, Marco Mores, Mattia Gianotti, Franca Garzotto

* Co-design of a collaborative music and paint app for neurodevelopmental conditions.

* Two field studies showing benefits for social, emotional, and well-being aspects.

* A categorization of participants’ music and painting experiences.
MusicTraces: Combining Music and Painting to
Support Adults with Neurodevelopmental Conditions
and Intellectual Disabilities

Valentin Bauer<sup>a,b,*</sup>, Giacomo Caslini<sup>a</sup>, Marco Mores<sup>c</sup>, Mattia Gianotti<sup>a,d</sup>,
Franca Garzotto<sup>e,a</sup>

<footnote><sup>a</sup>Politecnico di Milano, Department of Electronics, Information and Bioengineering, Via Giuseppe Ponzio, 34, Milan, 20133, Italy</footnote>

<footnote><sup>b</sup>Sciences et Technologies de la Musique et du Son (STMS Lab), IRCAM, CNRS, Sorbonne Université, Ministère de la Culture, 1 Place Igor Stravinsky, Paris, 75004, France</footnote>

<footnote><sup>c</sup>Fraternità e Amicizia Società Cooperativa Sociale ONLUS, Via Giorgio Washington, 59, Milan, 20146, Italy</footnote>

<footnote><sup>d</sup>KTH Royal Institute of Technology, Brinellvägen 8, Stockholm, 114 28, Sweden</footnote>

<footnote><sup>e</sup>University of Milano-Bicocca, Department of Psychology, Piazza dell'Ateneo Nuovo, 1, Milan, 20126, Italy</footnote>

<footnote>*Correspondence should be addressed to bauer.valentin123@gmail.com.
Email addresses: bauer.valentin123@gmail.com (Valentin Bauer),
giacomo.caslini@polimi.it (Giacomo Caslini), marco.mores@email.com (Marco
Mores), mattiagi@kth.se (Mattia Gianotti), franca.garzotto@unimib.it (Franca
Garzotto)</footnote>
Abstract

People with NeuroDevelopmental Conditions (NDC) and associated Intel-
lectual Disability (ID) often face social and emotional issues affecting their
well-being. Although art therapies like music-making or painting can be ben-
eficial, they often lack adaptability for those with moderate-to-severe ID or
limited language abilities. Multisensory creative technologies combining art
practices offer promising solutions but remain under-explored. This research
investigates the impact of integrating music-making and painting within an
interactive multisensory environment to support social communication, emo-
tional regulation, and well-being in adults with moderate-to-severe NDCs
and ID. The two-player music and painting activity called *MusicTraces* was
first created through co-design process involving caregivers and people with
NDCs and ID. Two field studies were conducted. The first study compared
*MusicTraces* with traditional group painting with music for ten adults, while
the second examined its impact over two sessions on eight adults with more
severe conditions. Both studies showed improved social communication, so-
cial regulation and well-being. Insights are discussed to enhance collaboration
between people with NDCs and ID through interactive creative technologies.

*Keywords:* Neurodevelopmental Conditions, Art therapy, Music, Paint, Interactive Multisensory Environment, Social communication, Emotional regulation

1. Introduction

People with NeuroDevelopmental Conditions (NDCs)<fnref n="1" />, such as autism,
ADHD, or intellectual disability (ID), often face social and emotional chal-
lenges that affect their daily lives (e.g., social isolation, depression) [2, 3].
Creative activities like music-making or painting, commonly used in care
centers, can support their well-being and social engagement [4, 5]. However,
people with ID or low verbal abilities, especially adults, remain underrepre-
sented in research due to methodological challenges (e.g., in data collection
and analysis) and design challenges (e.g., resources lacking the flexibility to

<footnote n="1">¹This paper adopts autism stakeholders’ language preferences [1], e.g., identity first-language (e.g., autistic people), no offending terms (e.g., “disorder”.)</footnote>
accommodate specific cognitive and sensory needs) [6, 7]. In particular, as
these individuals can show narrow interests, struggle to express preferences
(e.g., for painting or music), or with sustained attention, active open-ended
approaches are advised to keep them involved [8, 5], as well as combinations
of art forms to offer varied modes of expression [9]. Interactive multisens-
sory technologies could help meet these needs by enabling engaging creative
experiences.

Previous studies have combined art practices for people with NDCs to support social communication, emotional regulation, and well-being, with promising outcomes, using different interactive multisensory technologies, such as Interactive Multisensory Environments (iMSEs) [10, 11, 12], virtual reality [13], or other types of multimodal systems [14, 15, 16, 17, 18]. Indeed, digital tools are flexible, predictable, appealing [19, 20], already used in care centers [21], and promising to extend creative practices [22, 23, 24, 25]. However, challenges remain. Assessing the projects' impact remains hard, as they are often not readily comparable with traditional approaches. Adults with ID or limited verbal abilities remain underrepresented [26], and most studies target a single NDC, even though art activities in care centers target various NDCs. Surrounding sensory stimuli often remain uncontrolled - except for iMSEs - presenting risks of over-arousal. Finally, most projects focus on training specific behaviors to help individuals cope in a non-autistic "dominant environment" (e.g., emotional recognition) [27, p. 20]. Though, end-users value broader dimensions such as well-being or non-verbal communication with projects granting more room for their agency [27, 28, 29].

To address existing limitations, it is essential to involve stakeholders through participative design [30], and to consider people’s strengths in the design process [5]. This approach encourages open-ended, active activities, allowing participants to choose how to interact, rather than adapting to pre-defined correct interactions [18, 10, 14]. It also calls for collecting direct input from participants themselves (e.g., through their creative outputs), rather than relying solely on proxies or external observations [31, 32].

This study explores how combining music and painting through an active, open-ended activity in an iMSE can facilitate collaboration among adults with moderate-to-severe NDCs and ID. Three main research questions address the extent to which such activity can promote social communication (RQ1), emotional regulation (RQ2), and well-being (RQ3). A secondary research question concerns its potential to support social play (RQ4), which is a prerequisite for the development of the other aspects. To address these ques-
tions, the two-player activity called *MusicTraces* was co-designed through a four-step process, and then tested in two complementary field studies. The first study examined how *MusicTraces* differed from a traditional creative activity, with ten adults with NDCs. The second investigated the use of *MusicTraces* over multiple sessions with eight adults with more severe NDCs. This work builds on a previous study focused on the design process [33].

Our work offers three contributions. As the first combination of music and painting for adults with NDCs within an iMSE to our knowledge, our design could inform the creation of future interactive creative technologies to support collaboration among people with NDCs and ID. Second, we provide evidence of *MusicTraces'*s potential to support social communication, emotional regulation, and well-being. Third, we propose a categorization of participants' experiences, which could inform the methodologies of future HCI studies involving people with NDCs and ID.

After reviewing existing art therapy practices and creative iMSEs approaches, we present the co-design process and the two field studies. At last, we discuss our findings and draw future directions.

## 2. Related Work

### 2.1. Art therapy practices for people with neurodevelopmental conditions

Art therapy practices for people with NDCs and ID are promising to support engagement and social abilities [9, 34]. Many studies rely on passive approaches [35, 5]. However, active approaches are more and more advised, as this population is increasingly considered active [5, 35], in line with the neurodiversity movement [36]. Indeed, active approaches support motivation and self-expression, which in turn benefit emotional regulation and well-being [26, 37, 38]. Therefore, our study leverages the potential of active approaches.

Active music practices often include singing and instrumental play [35]. Improvisational practices are commonly reported [35, 39], and advised for people with more severe NDCs and ID [8, 40]. This approach falls under the informal music learning framework, as defined in [41]. Instruments often include readily available resources in care centres: untuned percussion (e.g., drums) or pitched instruments (e.g., guitar, keyboard) [35]. While sound preferences of people with NDCs often remain unknown, low-pitch and soothing sounds seem more advised for autistic people (e.g., marimba sounds) [42].

Paint practices often have an overarching goal (e.g., creating a portrait), with significant autonomy for participants to improvise [43, 44]. Such ap-
approaches allow participants to experiment with various tools and textures (e.g., paint with their hands). Considering these existing practices, our study grants participants much autonomy to improvise together.

Methodological challenges arise when working with people with severe NDCs and ID (e.g., understanding their emotional state) [35, 45]. However, during art therapy sessions, caregivers can sometimes interpret users' performances (e.g., music-making or painting) to understand their experiences. This potential was explored in *MusicTraces*, with caregivers observing users' performances in real-time, or at the audiovisual traces left after they finished.

Art therapy practices often involve multiple users with various NDCs, collaborating. As such, they have ties with social play, a topic which has recently received more attention in HCI [46, 47]. Research showed that social play is key to acquiring fundamental developmental abilities (e.g., relational, emotional) and promoting well-being [48]. It is often promoted through open-ended activities, which allow users to play in their own way [49], according to their needs and interests [26]. As such, social play is particularly related to collaborative improvised art therapy, and was investigated in *MusicTraces*.

## 2.2. Multisensory creative projects for people with disabilities

In last decades, various interactive technologies have shown promise in supporting people with NDCs [50]. Among them, Interactive MultiSensory Environments (iMSEs) are particularly appropriate for collaborative and expressive play. Drawing upon the principles of embodied cognition theory [51] and sensory integration theory [52], iMSEs enable full-body interaction [53], object manipulation [54], and multisensory stimulation [55, 56], in safe physical spaces, without requiring users to wear uncomfortable equipment (e.g., head mounted displays). As such, they are ideal for artistic expression [11, 12, 57] and social play [56, 53, 10, 54]. An example of social-play oriented iMSE is *Land of Fogs*, consisting of a magical world visible on a floor projection that two users must explore together, using interactable butterfly nets [53, 56]. Art-oriented iMSEs include: *Sensory Paint*, where users can throw balls onto a canvas to paint, their actions being sonified, with promising outcomes regarding attention and social abilities [11]; and *RHYME*, a multi-user project involving musical haptic interfaces (e.g., puppets), with benefits over social abilities and well-being [12]. While all these projects have inspired the design of *MusicTraces*, our project is the only one that equally balances music-making and painting while leveraging social play.
Some multisensory projects, based on the same theoretical principles, have been designed outside of iMSEs, with promising outcomes over social interaction and well-being for people with NDCs [17, 14, 15]. Some include expressive interfaces generating sounds and/or visuals, such as *Bendable Sound*, a one-user elastic tactile display enabling music-making with visual feedback; *Mazi*, a multi-user big music tangible ball [14]; or *Reactable*, a multi-user tabletop project for collaborative music-making[15, 58]. Such projects have inspired the design of *MusicTraces* in terms of artistic expressivity and materiality, considering the variety of tangible interactions used.

In all the above-mentioned projects, transactional support is fundamental to motivate the participants with NDCs and ID and support their understanding. It can be provided by the caregiver (e.g., verbally, physically) [59, 60] and the system (e.g., providing hints, supporting diverse ways of playing) [61, 60, 53]. For instance, *Lands of Fog* system automatically redirect the participants towards the middle of the space using visual cues [53]. While most projects only rely on caregivers’ support, we devised *MusicTraces* as an active contributor to support actions [62, 60].

In current HCI research, capturing the first-hand experiences of people with limited language abilities or ID remains challenging, often leading to relying on third-hand accounts, potentially being misleading [31]. Some alternative data collection methods have been proposed: using cards [63, 64], closed questions, or long-term design methods [31]. In our research, the design of *MusicTraces* draws upon the paradigm of probes [65] (or musical probes [66]), which intends to capture first-hand experiences (e.g., users’ preferences) while users paint and make music with the system. The traces left (e.g., visuals and sounds created) can then be analyzed to probe into users’ experiences after the session.

### 3. Co-design process of *MusicTraces*

The co-design process includes four phases that are successively detailed: a workshop, the initial design, remote iterations, and expert interviews.

#### 3.1. *Phase 1 - Co-design workshop*

This one-hour workshop included five caregivers and three people with NDC and ID. The goal was to brainstorm ideas about blending music and painting within the iMSE called the *Magic Room* [55, 67, 68] to foster collaboration between people with NDCs and ID.
### 3.1.1. Participants

Eight people participated (5 males and 3 females): seven from a music association for people with disabilities and one from our network. They include: the head of the association (H), two educators (Ed1 and Ed2), two psychologists (P1, P2), and three people with NDC and ID (PwD1, PwD2, PwD3). PwD1 (23 year-old male) has an Adams-Oliver Disorder, inducing ID and lacks autonomy. PwD2 (44-year-old male) and PwD3 (32-year-old female) have an ID and social and motor issues. Participants participated voluntarily without being paid after signing consent forms.

### 3.1.2. Protocol

The participants were introduced to the authors who ran the session: the animator, the secretary, and an observer. Then, they tried some non-creative activities within the *Magic Room* (e.g., storytelling) for twenty minutes, to introduce them to a new technology with which they had no familiarity, so that they could better understand its potential and limitations. After that, they went to a meeting room where the animator outlined the workshop's rationale, organization, and rules: (1) generate as many ideas as you can, (2) do not judge ideas, and (3) feel free to express unusual ideas. Participants used “concept sheets” to write or draw their ideas, as well as (if possible) their best and worst aspects to flesh them out and promote creativity. Paint materials were set aside for disposal to support idea generation (i.e., brushes).

The brainstorming session included two 30-minute rounds, first without cards and then with cards. Each time, participants started by taking notes individually, and then shared them with others. The secretary organized the emerging ideas on a Miro board<fnref n="2" /> projected on the wall. During the second round, twenty cards were used that we designed based on previous studies [69, 64]. Each card presented one design principle and gave one example of how to apply it. Ten cards were design heuristics [70] (e.g., “Allow user to rearrange”), and the others were activity-related (e.g., “Think about other music possibilities”). Card examples are visible in Figure 1.

At the end, the participants were thanked, asked if agreeing on participating in future testing, and given gift bags. Then, they could share a snack with the lab members and try other applications.

<footnote n="2">²Miro application: https://miro.com/fr/</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 1: Material used during the co-design workshop. (a.) Example of activity-related card, (b.) Example of heuristic card, (c.) Example of answers on a “concept sheet” from a participant with a NDC (PwD1).</figcaption></figure>

### 3.1.3. Data collection and analysis

Two authors analyzed participants’ ideas using thematic analysis [71]. The emerging themes were discussed with two other authors, one of whom (MM) is a clinical psychologist with over ten years of experience working with people with NDCs. The theme identifiers and the number of participants mentioning them are written as Tx and y/8 (where x and y are numbers).

### 3.1.4. Findings

In addition to the need to individualize the design based on users’ tastes (e.g., music taste), expressed by all participants, four themes were built: *Creative and accessible self-expression* (T1, 5/8), *Collaboration* (T2, 2/8), *Support* (T3, 4/8), and *Gamification* (T4, 3/8).

*Creative and accessible self-expression* is expressed by four caregivers and PwD2. Proposing accessible interactions is advised, with hand and arm gestures due to their connection with painting gestures (i.e., users painting on a canvas) and music gestures (i.e., users conducting an orchestra), while still allowing users with motor issues to participate. Then, multisensory stimuli are advised (H, P1, P2, PwD2), with straightforward relationships between the music and visuals, and tactile stimuli with accessible controllers. Finally, supporting self-expression is advised (e.g., using different brush colors).

Fostering *collaboration* is expressed by two caregivers (H, P2). The activity should support turn-taking (H) or simultaneous play (H, P2). It may be task-oriented (competing against each other) (P2, H) or open-ended (e.g., creating together) (H). However, social anxiety may hinder collaboration (H).

*Support* is expressed by four participants (H, PwD2, MM, and P2). Care-
givers may give prompts (e.g., verbally). Showing music or visuals during the experience may foster creativity (e.g., painting over a background image).

*Gamification* is evoked by two caregivers and PwD2. It consists of promoting engagement, for instance through task-oriented activities (Ed2, P2) (e.g., the discovery of a drawing (P2)) or using levels and challenges (PwD2).

## 3.2. Phase 2: Initial design of MusicTraces

The initial design was proposed based on the workshop's themes (referred to as Tx), on previous studies, and on the experience of the first author, who is both a musician and an artist. It consists of a music and painting activity within an iMSE that aims to foster collaboration, inspired by improvised art therapy practices [62]. As such, it intends to accommodate multiple ways of acting, and be accessible to people with various kinds of abilities.

## 3.3. Activity

*MusicTraces* is a two-user activity designed within our iMSE called the *Magic Room* [72, 67, 68]. The experience includes audio (from two speakers), visual (with smart lights, frontal and floor projections), tactile (with smart objects), and proprioceptive (when drawing and moving) stimuli [T1]. Each user can move hand-held smart objects (i.e., brush and eraser, see Figure 2) [T1] to create music shapes on the paper-like front user interface based on their hand position (see Figure 3.b). Users can also step on a specific floor area to activate the background music [T1,T3]. This design aims to cater to people with severe motor issues (e.g., those who cannot use their arms). Information is structured in terms of its role to promote understanding [T3].

Regarding the music shapes, each visual has a music counterpart and conversely, drawing upon [T1] and [58], allowing users to focus on sounds, visuals, or both, depending on their interests. Paint spots correspond to music notes (e.g., the higher the spot with respect to the floor, the higher the note pitch). Lines are melodies, being composed when users connect several spots. Internal composition rules help make the resulting music sound harmonious (e.g., when to hear each note), informed by previous HCI studies and contemporary music practices [73, 74]. All sounds are designed to be soothing (i.e., marimba and handpan). Visuals are minimalistic to prevent risks of over-arousal [42, 13].

*MusicTraces* aims to support collaboration in various ways [T2, T3]. Drawing upon past studies [10, 62, 60, 53], some design features play that role: users have different colors and musical instruments, their colors blend when
they cross, and what one user creates is available to the other to continue the
creation. Caregivers can give prompts (verbal or physical) to support col-
laboration, and use a tablet to make adjustments if needed (e.g., removing
lines). Although users can act as they wish, a hint system helps prevent idle-
ness, isolation, or repetitive actions, stemming from *Mediate* [10], [T2], and
[T3]. Thus, when the system detects such behaviors, lines are automatically
drawn to point at unexplored screen areas and encourage actions.

3.4. User Experience

Users can perform three types of actions: *Explore*, *Create*, and *Play* [T1,T4]. *Explore* consists in exploring the music space by leaving temporary traces (that quickly disappear), which trigger sounds based on the hand position. *Create* is about leaving permanent spots or lines or erasing them. *Play* consists in touching spots or lines to play the related sounds. When users play lines, a cursor navigates them starting from the interaction point. This metaphor stems from the previous projects *Iannix* [73] and *Upic* [75].

Each user uses two controllers [T1]: a smart brush to explore or create, and a smart eraser to erase the permanent traces (see the first iteration in Figure 2). These controllers are distinct in shape and grip, facilitating association between each action and its corresponding tool [T3]. To create permanent traces, users must move their hand to the initial point where they want to draw, press the single button on the brush (with any finger), and move the objects. If users press and release the brush button quicker than one second, they start to explore, i.e., leave temporary traces when moving the brush. To stop exploring, they must press the button again. At last, to erase, users must take the eraser, press the button on this object, and move the eraser over the permanent traces on the canvas. For simplicity, hovering any part of a line will delete the whole line.

Before playing with the activity, a video tutorial teaches the users how to
play, using video modeling techniques [76]. In particular, users can see and
try every action of the activity separately.

3.4.1. *Apparatus*

*MusicTraces* is developed under Unity Engine (version 2021.2.14f1). A custom package gives access to the devices of the *Magic Room* by communicating with ad-hoc servers, and to the tablet. The hardware components include two projectors, a Microsoft Kinect v2, smart lights, a tablet, audio speakers, and a windows PC. The setup is fully described in previous works
<figure><img src="image_4.png" /><figcaption>Figure 2: Design iterations for the smart objects. **First iteration** smart objects are used for the first study. Each user uses two objects: one brush and one eraser. **Second iteration** smart objects are an intermediate smaller design to facilitate pressing the buttons. They are not used in our studies. **Third iteration** smart objects are used for the second study. Each user uses one small object to draw and erase.</figcaption></figure>

[72, 67, 68]. The smart objects are built using parts from commercial objects or being 3D printed, magnets, a module ESP8266 Witty Cloud ESP-12F WiFi, external batteries, and LED strips. All sounds are created using virtual instruments on LogicProX software. The paint visuals are made using Unity Shader graph, and the textures with Inkscape software.

## 3.5. Phase 3 - Remote agile process

To then perform testings in care centers managed by the organization called Fraternità e Amicizia (FeA), we pursued our design process with a psychologist who had worked there for more than ten years (M).

### 3.5.1. Method

An agile method of working was used with (M), using small iterative design cycles [77]. Three email exchanges occurred within two weeks. Each time, we asked (M) feedback about our design. Videos were used for the first two exchanges to present our design. Textual feedback was analyzed according to the activity features using a deductive qualitative analysis [78].
<figure><img src="image_5.png" /><figcaption>Figure 3: Main design features of *MusicTraces*. (a.) Front User Interface (UI). 1: Temporary Line. 2: Permanent line (with a sound). 3: Permanent line (without sound). 4: Permanent dot (i.e., sound). 5: Animated cursor navigating the line. 6: Hand of blue player (b.) Floor UI: 7: player position. 8: Floor area to trigger the background music.</figcaption></figure>

Overall, some features we were unsure of were validated, while others were modified, or added. They are summarized below.

### 3.5.2. Findings

Three elements were validated. Using hint lines seemed promising to support understanding. Adapting the line thickness based on the user's distance from the screen seemed to support engagement and expressivity. The smart objects were also validated to support usability and understanding.

Regarding modifications, to avoid visual clutter, (M) suggested that after users draw the first line, caregiver should manually add the hint lines from the tablet, rather than being automatically spawned.

Three elements were added to promote agency: automatically filling the closed shapes with color, and enabling users to draw lines without notes (to cater for the users not wanting to make music).

## 3.6. Phase 4 - Expert Interviews

To further adapt our design for FeA's care centers, and validate its acceptability among caregivers, interviews were led with caregivers at FeA: three educators who had not participated in the previous design phases and (M).

### 3.6.1. Method

The three educators (Ed3, Ed4, Ed5) have been conducting weekly creative activities at FeA, for four years, more than ten years, and five years. Ed3 (male) and Id4 (male) do group-based manual activities (e.g., painting, sculpting). Ed5 (female) is a trained art therapist. In particular, she conducts group-based painting activities over a music background.
Individual 45-minute interviews were organized in one of FeA's care centers. We started by presenting the activity (using a video to show every action), followed by questions. One week later, (M) came to our laboratory. After presenting the activity, he tested *MusicTraces* while commenting on his actions aloud. Interviews were audio recorded, with caregivers' agreement. Data were analyzed using thematic analysis [71].

### 3.6.2. Findings

All educators were positive about using *MusicTraces* at FeA's care centers. Ed5 said that it could benefit her activities. Id3 emphasized the unique combination of music and drawing, and its potential to capture users' attention and help escape from repetitive behaviors that arise from playing.

Interactions using the smart objects seemed easy to understand (all). Some precision issues in the body tracking were noted (M). We then fixed them through coding. Following (M)'s suggestion, we used a different color for the brushes and smart lights to avoid confusing users.

Hint lines seemed promising (all), especially in relation to detecting isolation and repetitiveness. To differ from the players' lines, and guide users' attention toward specific areas, wavy lines were suggested and adopted (Ed3).

All caregivers agreed to use a video tutorial with audio-recorded instructions. Id3 advised including all steps, and Id4 advised a maximum of three steps. Thus, we decided to use the tablet to choose the steps to include.

## 4. Study 1: Effect of *MusicTraces* compared with a traditional art therapy activity

After this co-design process, this exploratory study aimed to compare the impact of using *MusicTraces* with a traditional art therapy activity consisting of painting as a group on a music background, over how adults with NDCs and ID could collaborate. To that end, a within-group experimental design was conducted with ten participants in one of FeA's care centers.

### 4.1. Method

The Ethical Committee at our university approved the study protocol. The issues of security, privacy, and confidentiality were addressed according to the guidelines provided by the European Data Protection Supervisor (EDPS).
### 4.1.1. Participants

Ten participants with NDCs and ID (5F/5M) aged between 20 and 47 (M: 33.8, SD: 7.68) were recruited by caregivers from the care center. Inclusion criteria focused on adults with NDCs and associated ID presenting social difficulties. Exclusion criteria included aggressive behaviors, epilepsy, blindness, deafness, or severe sensory issues. For recruitment, the caregivers explained the study's rationale to the participants and their relatives. Informed consent was collected from parents or legal tutors. The informed consent forms included information about the procedures, goals, and data treatment. Due to scheduling issues, among the ten participants, seven participated in both *MusicTraces* and the traditional art activity, one participated in *MusicTraces* only, and two in the traditional activity only. Participant profiles are summarized in Table 1, which can be read two rows at a time to indicate the participants who played together (e.g., P1.1 and P1.2 are the first pair). To refer to ID, we used severity levels given by the caregivers, which referred to intellectual quotient: mild (70-55), moderate (55-40), severe (40-20), and profound (< 20).

Most participants presented little to non-language abilities (7/10), moderate to severe (or more) ID (7/10), and low attention spans (less than five minutes for five participants). Some participants (P1.6, P1.7, and P1.9) displayed atypical sensory experiences (see Table 1 for further data). Most participants (except P1.1) were expected to face difficulties in understanding *MusicTraces* or integrating the physical and digital stimuli. Four participants could easily get frustrated (e.g., when not completing a task). Most participants were not curious toward new activities (M: 2.5/5, SD: 0.93; see Section 4.1.3 for details about the ratings), except for P1.7 (4/5). Moreover, context-switching could often be anxiety-provoking (M:2.88, SD:1.13), especially for P1.2, P1.3, and P1.8 (4/5). Most participants were not used to digital tools (M: 1.75, SD: 0.89) - except for three who sometimes used social networks. None had used iMSEs in the past. P1.7 and P1.3 had tried virtual reality once: P1.3 largely refused it and P1.7 enjoyed it much. At last, according to the caregivers, P1.4 could be “extremely hard to engage in traditional activities” and P1.9 often “invested in relationships”.

### 4.1.2. Protocol

Every participant experienced *MusicTraces* and the traditional art therapy activity. To mitigate the learning effect, the order of the conditions was randomized and a delay of two to three weeks was set between the conditions.
<table><caption>Table 1: Participant profiles for two studies. Legend: *St.* = Study; *Ses* = sessions completed; *MT* = MusicTraces experience; *TA* = Traditional Activity experience; *Gend* = Gender; *OCD* = Obsessive-Compulsive Disorder; *HL* = bilateral Hearing Loss; *ASC* = Autism Spectrum Condition; *ADHD* = Attention Deficit Hyperactivity Disorder; *ID* = Intellectual Disability; *mod* = moderate; *sev* = severe; *prof* = profound; *Communic.* = Communication; *Min. verbal* = minimally-verbal; *Exp. Diff.* = expected difficulties with *MusicTraces*; *Spec. Behav.* = specific behaviors; *anx* = anxiety; *frust* = frustration; *sw.* = difficulties with context-switching.</caption><thead><tr><th>St. Ses</th><th>ID</th><th>Age</th><th>Gender</th><th>Main condition</th><th>Intellectual disability</th><th>Communic. abilities</th><th>Attention span (mn)</th><th>Exp. Diff.</th><th>Spec. Behav.</th></tr></thead><tbody><tr><td>1</td><td>MT+TA</td><td>P1.1</td><td>20</td><td>F</td><td>Dyspraxia with mild ID</td><td>mild</td><td>Min. verbal</td><td>10-15</td><td>No</td><td></td></tr><tr><td>1</td><td>TA+MT</td><td>P1.2</td><td>41</td><td>F</td><td>Down syndrome with ID</td><td>sev-prof</td><td>Min. verbal</td><td>5-10</td><td>Yes</td><td>sw</td></tr><tr><td>1</td><td>MT+TA</td><td>P1.3</td><td>37</td><td>F</td><td>ID due to marfonoid habitus</td><td>sev-prof</td><td>Min. verbal</td><td>0-5</td><td>Yes</td><td>anx, sw</td></tr><tr><td>1</td><td>MT</td><td>P1.4</td><td>36</td><td>M</td><td>Down syndrome</td><td>prof</td><td>Non-verbal</td><td>0-5</td><td>Yes</td><td></td></tr><tr><td>1</td><td>TA+MT</td><td>P1.5</td><td>47</td><td>M</td><td>Down syndrome with moderate ID</td><td>sev</td><td>Verbal</td><td>0-5</td><td>Yes</td><td>frus</td></tr><tr><td>1</td><td>MT+TA</td><td>P1.6</td><td>24</td><td>M</td><td>Focal epilepsy and moderate ID</td><td>sev</td><td>Min. verbal</td><td>0-5</td><td>Yes</td><td>frus</td></tr><tr><td>1</td><td>TA+MT</td><td>P1.7</td><td>33</td><td>M</td><td>Moderate ID, deafness, motor issues</td><td>mod</td><td>Verbal</td><td>15-20</td><td>Yes</td><td>frus</td></tr><tr><td>1</td><td>MT+TA</td><td>P1.8</td><td>37</td><td>F</td><td>Epilepsy and ID</td><td>prof</td><td>Min. verbal</td><td>0-5</td><td>Yes</td><td>frus, sw</td></tr><tr><td>1</td><td>TA</td><td>P1.9</td><td>36</td><td>M</td><td>Down syndrome, HL</td><td>mod-sev</td><td>Verbal</td><td>10-15</td><td>Yes</td><td></td></tr><tr><td>1</td><td>TA</td><td>P1.10</td><td>27</td><td>F</td><td>Moderate ID, OCD</td><td>mild-mod</td><td>Min. verbal</td><td>10-15</td><td>Yes</td><td>atypical.</td></tr><tr><td>2</td><td>MT+MT</td><td>P2.1</td><td>34</td><td>F</td><td>ASC and ID</td><td>prof</td><td>Verbal</td><td>15-20</td><td>Yes</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.2</td><td>35</td><td>F</td><td>sensorineural HL; microcephaly</td><td>prof</td><td>Non-verbal</td><td>5-10</td><td>Yes</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.3</td><td>20</td><td>M</td><td>ID, developmental verbal dyspraxia</td><td>sev</td><td>Verbal</td><td>10-15</td><td>Yes</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.4</td><td>30</td><td>M</td><td>pachygyria</td><td>mod-sev</td><td>Verbal</td><td>15-20</td><td>No</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.5</td><td>22</td><td>F</td><td>mild-moderate ID, emotional fragility</td><td>mod-sev</td><td>Verbal</td><td>15-20</td><td>No</td><td>frus</td></tr><tr><td>2</td><td>MT+MT</td><td>P2.6</td><td>19</td><td>M</td><td>Moderate ID, ASC, ADHD</td><td>mod</td><td>Verbal</td><td>15-20</td><td>No</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.7</td><td>20</td><td>M</td><td>ASC</td><td>prof</td><td>Verbal</td><td>5-10</td><td>Yes</td><td></td></tr><tr><td>2</td><td>MT+MT</td><td>P2.8</td><td>20</td><td>M</td><td>ASC</td><td>prof</td><td>Verbal</td><td>10-15</td><td>Yes</td><td></td></tr></tbody></table>

*Protocol for MusicTraces activity.* One research investigator dims the lights and starts the video recording. Another investigator makes the participants enter the room with their caregiver, introduces them to the game, and makes them step into the interaction space. Both investigators are authors of this article. Another caregiver is also in the room to guide or observe the session.

The session starts with a 5-to-10-minute tutorial. The tutorial successively presents the main game actions, using video modeling techniques [76] with short videos (5-10 seconds): explore, draw spots and lines, play objects, erase objects, and start the background music. The participants must go through each step, one after the other. Success for each step is notified with audio rewards. If they fail, the caregivers can restart or skip the step. Once the tutorial is completed, the two participants start playing with *MusicTraces*. Sessions are expected to last around 10 minutes, but caregivers adjust the duration following the users' request or state. In case of a premature ending, caregivers must tell the participants to do one last action one minute before stopping. During the entire session, the caregiver and research
investigator can give prompts (e.g., verbal, physical) to support users.

Once ended, lights are switched on. The caregiver asks the participants closed questions about what they liked or disliked, their preference between music-making and painting, and how they feel now compared to before the session. After that, the participants leave the room with one caregiver. Then, the other caregiver answers a semi-directed 10-minute questionnaire, focusing on the various aspects of the experience (see Section 4.1.3).

*Protocol for traditional art therapy activity.* The caregivers welcome the participants. Then, the psychologist runs a 15-minute meditation time, using relaxing music. After that, the participants take a short break. Then, the caregivers prepare the paint material on the table with the participants, introduce it, and select the background music. The participants start painting when the music starts, for around 15 minutes. To stop, they must agree as a group when they deem their painting finished. After the end, they gather to share their experiences. At last, the caregivers answer the questionnaire we prepared for each participant. During the entire session, the caregivers can give verbal and physical prompts.

### 4.1.3. Data collection

Mixed data (quantitative and qualitative) are collected at various stages to mitigate the bias due to each source. They are similar for both experimental conditions.

Each session is recorded using one video camera. Before the session, caregivers complete a profiling questionnaire to understand the participants’ profiles. Questions address under/over sensitivities, expected difficulties with the activity, anxiety-provoking stimuli, behaviors accounting for anxiety and engagement, usual attention span and curiosity, ways of communicating, difficulties with context-switching, use of digital tools and iMSEs. Some answers use 1-5 Likert-type scales (5 always being the most extreme value). After the session ends, participants are asked to point at what they like and dislike in their final drawing and indicate whether they preferred making music or painting. Then, they must indicate how they felt before and after the session, whether they enjoyed the activity, and whether they would like to participate again, using a 1-5 analog visual scale with emojis. Overall, such questions help collect first-hand insights on social aspects, emotional regulation, and engagement. Second, the caregivers complete a semi-directed questionnaire, using 1-5 Likert-type scales and optional comments. The first
seven questions focus on social aspects. Six of them come from the Music Therapy Session Assessment Scale (MT-SAS) [79], and ask about: body reciprocity, emotional involvement (concerning harmonious productions with their partner), refusal/disturbed behaviors, audiovisual productions, harmonized audiovisual productions, and dynamism/variations (we removed the item about eye contact considering its low impact [79]). The respective level of support from the caregiver and system for each question is also asked. Then, a seventh question asks if caregivers observed more social interaction than usual. About regulation, two questions inquire about opportunities for emotional regulation and self-regulation. For engagement, two questions ask if the participants looked engaged and proud of their achievements. Finally, six questions from the NASA-TLX - a short questionnaire that targets the perceived workload [80] - were adapted so that the caregiver could answer on behalf of the users.

#### 4.1.4. Data analysis

The data were analyzed using quantitative and qualitative methods, following three main steps.

First, quantitative data were analyzed to identify trends and potential discrepancies between the ratings and comments. When discrepancies emerged, we looked for confirmation or refutation by watching the respective videos with one of the caregivers. Once confirmed, the caregivers re-rated the respective items. Due to the low number of participants, data were analyzed using descriptive statistics, and tests of significance could not be used.

Second, two of the authors coded the videos, looking for behaviors accounting for Social Communication (SC), Emotional Regulation (ER), Transactional Support (TS), and Play (PL). We used this coding scheme following our research questions, and specifically the SCERTS framework [81]. Indeed, SCERTS is a well-established assessment method stemming from a multidisciplinary theory [81] that was used in recent HCI studies [61, 14], and aligned well with our research questions. While SC and ER are self-explanatory, TS consists of support provided by the caregiver (e.g., verbally, physically) or the system (e.g., supporting various ways of playing) to the users while performing. For PL, we differentiated between various social play states [82, 46]: non-play, solitary, parallel, associative, cooperative. While non-play and solitary play are self-explanatory, parallel play refers to participants playing side by side without interacting, associative play refers to some level of social interaction, and cooperative play refers to actively engaging in
social play sometimes. In addition to these four main categories (SC, ER, TS, PL), to not overlook significant insights, critical incidents (e.g., events that were surprising, unexpected) were coded, following the critical incident technique [83]. The inter-coder agreement between the two coders during this step was high (80,2%). Discrepancies mainly concerned codes related to TS. They were all discussed between the two coders to reach an agreement.

Third, to build the respective concepts for the four main categories, we inductively analyzed the codes obtained during the second step by using inductive methods inspired by the Grounded Theory [84, 85]. The behaviors for each category were iteratively grouped into concepts, that were gradually refined through constant comparisons with the initial codes of the behaviors, and by watching the videos again. We also investigated the causes, context, and consequences for each behavior, following cognitive event analysis methods [86, 46]. To that end, we specifically watched and analyzed the ten to twenty seconds before and after the events corresponding to each behavior.

## 4.2. Results

Results are presented following the four categories coming from the qualitative analysis: Play (PL), Social Communication (SC), Emotional Regulation (ER), and Transactional Support (TS). Aggregate measures of the quantitative ratings are computed and reported for each participant in Figure 4. The engagement ratings reported by the participants and caregivers were aggregated as they were similar. The TS ratings between the caregivers and the system were kept separate as they were different. Details about the SC items are shown in Figure 5. The categorization resulting from the qualitative analysis process is summarized in Table 2. The table includes the four main categories (SC, ER, TS, PL), their related concepts, associated observed behaviors, and the corresponding number of participants for each, for *MusicTraces* (MT) and the traditional activity (TA). All concepts are written in bold in the text to facilitate the reading.

### 4.2.1. *Play*

Regarding the **possibility of play**, all participants tried *MusicTraces* (8/8). Most participants also tried the traditional art activity (8/9), except for P1.8 who drew apart from the group. Most participants were engaged in both conditions (see Figure 4). Such engagement with *MusicTraces* surprised the caregivers since most participants were expected to face difficulties (see Table 1). Regarding the two participants who were not engaged: P1.4
<table><caption>Table 2: Categories, concepts, and related behaviours coming from the qualitative analysis.</caption><thead><tr><th rowspan="2">Cat</th><th rowspan="2">Concept</th><th rowspan="2">Observed behaviours</th><th colspan="2">Study1</th><th colspan="2">Study2</th></tr><tr><th>MT</th><th>TA</th><th>s1</th><th>s2</th></tr></thead><tbody><tr><td rowspan="6">SC</td><td rowspan="3">Direct SC</td><td>Looking at the other</td><td>8/8</td><td>8/9</td><td>7/8</td><td>7/8</td></tr><tr><td>Talking to their partner</td><td>0/8</td><td>4/9</td><td>3/8</td><td>2/8</td></tr><tr><td>Talking to the caregiver</td><td>8/8</td><td>3/9</td><td>6/8</td><td>6/8</td></tr><tr><td rowspan="2">Mediated SC</td><td>Attuned productions</td><td>3/8</td><td>8/9</td><td>4/8</td><td>4/8</td></tr><tr><td>Sharing the space</td><td>2/8</td><td>8/9</td><td>0/8</td><td>0/8</td></tr><tr><td rowspan="6">ER</td><td rowspan="3">Mediated ER</td><td>Being happy</td><td>7/8</td><td>7/9</td><td>5/8</td><td>5/8</td></tr><tr><td>Being focused</td><td>8/8</td><td>7/9</td><td>6/8</td><td>6/8</td></tr><tr><td>Being relaxed</td><td>3/8</td><td>2/9</td><td>4/8</td><td>4/8</td></tr><tr><td rowspan="2">Self regulation</td><td>Acting repetitively</td><td>5/8</td><td>2/9</td><td>2/8</td><td>2/8</td></tr><tr><td>Unloading</td><td>1/8</td><td>0/9</td><td>1/8</td><td>1/8</td></tr><tr><td rowspan="6">TS</td><td rowspan="4">Caregivers' prompts</td><td>Promoting understanding</td><td>7/8</td><td>0/9</td><td>7/8</td><td>7/8</td></tr><tr><td>Encouraging actions</td><td>7/8</td><td>8/9</td><td>3/8</td><td>3/8</td></tr><tr><td>Supporting movements</td><td>3/8</td><td>0/9</td><td>3/8</td><td>3/8</td></tr><tr><td>Helping to refocus</td><td>1/8</td><td>1/9</td><td>1/8</td><td>1/8</td></tr><tr><td rowspan="3">Levels of autonomy</td><td>Exploring</td><td>3/8</td><td>8/9</td><td>6/8</td><td>6/8</td></tr><tr><td>Acting intentionally</td><td>7/8</td><td>8/9</td><td>4/8</td><td>6/8</td></tr><tr><td>Asking for support</td><td>7/8</td><td>1/9</td><td>7/8</td><td>7/8</td></tr><tr><td>Peer support</td><td>Verbal & physical</td><td>0/8</td><td>1/9</td><td>0/8</td><td>0/8</td></tr><tr><td rowspan="10">PL</td><td rowspan="2">Possibility of play</td><td>Trying</td><td>8/8</td><td>8/9</td><td>7/8</td><td>7/8</td></tr><tr><td>Being able to play</td><td>7/8</td><td>8/9</td><td>7/8</td><td>7/8</td></tr><tr><td rowspan="6">Play state</td><td>Non-play</td><td>1/8</td><td>1/9</td><td>1/8</td><td>1/8</td></tr><tr><td>Solitary play</td><td>1/8</td><td>1/9</td><td>2/8</td><td>2/8</td></tr><tr><td>Parallel play</td><td>7/8</td><td>7/9</td><td>7/8</td><td>7/8</td></tr><tr><td>Associative play</td><td>4/8</td><td>8/9</td><td>4/8</td><td>6/8</td></tr><tr><td>Cooperative play</td><td>0/8</td><td>2/9</td><td>2/8</td><td>2/8</td></tr><tr><td>Preferring the music</td><td>4/8</td><td>na</td><td>3/8</td><td>3/8</td></tr><tr><td rowspan="3">Preference</td><td>Preferring to paint</td><td>2/8</td><td>na</td><td>2/8</td><td>2/8</td></tr><tr><td>Preferring both</td><td>1/8</td><td>na</td><td>1/8</td><td>1/8</td></tr><tr><td>Staying still</td><td>3/8</td><td>2/9</td><td>1/8</td><td>1/8</td></tr><tr><td rowspan="3">Motion range</td><td>Moving</td><td>3/8</td><td>4/9</td><td>4/8</td><td>6/8</td></tr><tr><td>Moving a lot</td><td>2/8</td><td>2/9</td><td>0/8</td><td>1/8</td></tr></tbody></table>

Legend: *Cat* = Category; *SC* = Social Communication; *ER* = Emotional Regulation; *TS* = Transactional Support. In the *first* study: *MT* = *MusicTraces* activity; *TA* = Transactional Activity experience; *8* = number of participants in *MT*; *9* = number of participants in *TA*. In the *second* study: *s1* = first session; *s2* = second session; *8* = number of participants in *s1* and *s2*.
<figure><img src="image_6.png" /><figcaption>Figure 4: Study 1: Results about the participants’ state (*Before* and *After*), engagement (*Eng.*), Social Communication (*SC*), Emotional Regulation (*ER*), and Transactional Support from the caregiver (*TS Car.*) and system (*TS Sys.*), represented in a radar plot (scale 1-5)</figcaption></figure>

tried some features before exiting the room (see Figure 6.b), but caregivers described him as “*difficult to engage*”, and P1.3 felt like having failed in both activities. Participants stayed longer than their usual attention spans (see Table 1) for both conditions (MusicTraces: 25’00, SD: 1’19; Traditional activity M: 16’30, SD: 06’24), with shorter sessions during the traditional activity. In both conditions, participants’ states remained stable throughout the session (see Figure 4), even though two participants expressed being slightly tired by the end of the session *MusicTraces* due to holding the smart objects. The perceived workload was slightly higher with *MusicTraces* (NASA-TLX - M: 70.5, SD: 15.4) than with the traditional activity (NASA-TLX - M: 54.5, SD: 07.07). High perceived levels of success and difficulty were reported in both conditions, with lower levels of mental and physical demands during the traditional activity. This insight was expected since no participant knew *MusicTraces* before playing. In particular, most participants faced slight usability issues with the smart objects (e.g., pressing the buttons), due to their large size (6/8). Though, most participants understood how to play with *MusicTraces* (7/8) and the traditional activity (8/9). This outcome sometimes surprised the caregivers, mainly for P1.6, who seemed highly focused,
<figure><img src="image_7.png" /><figcaption>Figure 5: Study 1: Results about the SC items, including the six MT-SAS items, and the question asking if the participants socially communicated more with *MusicTraces* in comparison with their daily life (*More social*), represented in a radar plot (scale 1-5)</figcaption></figure>

and P1.7, who drew circles despite his motor issues (see Figure 6.c and d).

Regarding **play states**, parallel play was most common with *MusicTraces* (7/8). Associative play emerged more during the traditional activity, with participants who started drawing in front of them and then extended their creations towards those of their peers, filling the space separating them, until they had sometimes to physically move closer to one another (8/9). With *MusicTraces*, it consisted of spontaneously crossing each others' lines (P1.1, P1.5, e.g., see Figures 6.a), making music together (P1.1 and P1.2), or sometimes sharing their spaces (P1.5, P1.6, e.g., see Figures 6.c). Cooperative play only emerged during the traditional activity, between P1.7 and P1.9 who were friends, to discuss what to paint. In both conditions, P1.6 was sometimes an onlooker due frequent attention losses. Finally, P1.5 always played in parallel or associatively with *MusicTraces*, while always drawing solitarily during the traditional activity.

In terms of **preferences** with *MusicTraces*, half of the participants preferred the music (P1.2, P1.5, P1.7, P1.8), two favored the painting (P1.3, P1.6), P1.1 enjoyed both, and P1.4 did not express a preference.

Similar **motion ranges** emerged in both conditions. With *MusicTraces*, three participants stayed still (P1.4, P1.6, P1.8), three performed regular
<figure><img src="image_8.png" /><figcaption>Figure 6: Specific events for the first study that occurred during the experimental (a,b,c,d) and control (e,f) conditions. (a.) P1.1 (right) and P1.2 (left) crossing their lines. (b.) P1.3 (left) painting and P1.4 (right) still trying to play before exiting the room. (c.) P1.5 almost swapping positions with P1.6. (d.) P1.7 repetitively making spots. (e.) Participants drawing together. (f.) P1.9 (front center) and P1.7 (front right) having moved to draw with others.</figcaption></figure>

movement amplitude (P1.1, P1.2, P1.3), and two made ampler movements (P1.5, P1.7). With the traditional activity, two stayed still (P1.5, P1.6), four performed regular movement amplitude (P1.1, P1.3, P1.7, P1.9), and two made ampler movements (P1.2, P1.10). The participants spatially moved more during the traditional activity, partly due to caregivers' incentives (e.g., see Figure 6.e and f). Surprisingly, P1.5 performed ample movements with *MusicTraces* (see Figure 6.c) but stayed still during the traditional activity.

### 4.2.2. Social Communication (SC)

Participants showed moderate-to-high levels of SC with *MusicTraces* and the traditional activity (see Figure 4), the levels of the MT-SAS items being equally spread (see Figure 5). While the participants painted more during the traditional activity, their productions were slightly more dynamic and harmonious with the other player with *MusicTraces*. In particular, caregivers said that P1.1, P1.5, and P1.8 socially interacted more when playing with *MusicTraces* than in their daily life.

**Direct SC** often emerged by looking at what their partner was doing, with *MusicTraces* (8/8) and the traditional activity (8/9). With *Music-
*Traces*, the participants only talked to the caregiver (8/8), sometimes to describe what they were doing (e.g., P1.5 said “*This is grass*”). During the traditional activity, they talked less to the caregiver (3/9) and more to the other players (4/9). In particular, P1.9 and P1.7, who were friends, often discussed what to paint, and P1.9 sometimes talked to P1.3 and P1.6. Moreover, in both conditions, P1.6 also often uttered sounds, which the psychologist interpreted as intended to catch others’ attention.

**Mediated SC** mainly occurred with *MusicTraces* through attuned productions (3/8, e.g., spontaneously crossing each other’s lines), or sharing the physical space (2/8, e.g., P1.5 almost swapped position with P1.6, see Figure 6.c). In the traditional activity, more attuned productions emerged (8/9, e.g., P1.1 imitated P1.10’s drawing), due to sharing more their spaces (8/9). In particular, P1.2, P1.7, P1.9, and P1.10 moved around the table to play with the others who stayed still and agreed to share their spaces.

### 4.2.3. Emotional Regulation (ER)

The levels of ER were moderate-to-high in both conditions (see Figure 4). However, P1.3 often feared committing mistakes even though she enjoyed both activities. In terms of **mediated ER**, most participants felt happy when playing with *MusicTraces* (7/8): smiling (7/8, even P1.3), expressing it with their body (3/8), comparing the activity to something else they liked (2/8, e.g., P1.8 said the brush felt like a guitar), or feeling proud (P1.5 proudly said “*I did art school*”). Likewise, most participants felt happy during the traditional activity (7/9): saying it (P1.2, P1.9), smiling (8/9), or laughing (3/9). Most participants were focused with *MusicTraces* (8/8) and the traditional activity (7/8). However, while P1.6 often lost his focus during the traditional activity, caregivers described him as “into the game” with *MusicTraces* without the intervention of caregivers of system facilitators. As a result, most participants ignored unexpected events in both conditions, such as some body tracking artifacts with *MusicTraces* or the music stopping during the traditional activity. Participants could also relax with *MusicTraces* (3/8) and during the traditional activity (2/9). In particular, P1.2 spontaneously said *MusicTraces* was “relaxing”, even more than the other activities at the care centre.

Some participants adopted **self-regulation** strategies. With *MusicTraces*, they mainly included repetitive actions (5/8). Some were unrelated to the activity (3/8, e.g., P1.1 looked up when excited, P1.6 often uttered sounds), and others were supported by *MusicTraces* (P1.7 and P1.5 repetitively performed
repetitive circular gestures, e.g., see Figure 6.d). During the traditional activity, repetitive actions were unrelated to the activity (2/9), with P1.1 and P1.6 showing the same repetitive actions as they did with *MusicTraces*.

#### 4.2.4. Transactional Support (TS)

Support was significant with *MusicTraces*, but scarce during the traditional activity, as the participants already knew how to paint (see Figure 4).

With *MusicTraces*, **caregivers' prompts** aimed to promote understanding of how to play (7/8), mainly being verbal with mild (6/8) or high (P1.4 and P1.6) physical support. Incentives aimed to promote exploration, spontaneous actions, and line-crossing (7/8). Some sensory prompts were intended to help users move more (P1.2, P1.4, P1.6). Since caregivers were busy checking the session, the hint system was only used once for P1.3 with positive effects, allowing him to draw elsewhere on the canvas. Yet, caregivers still expressed that the hint system could help some users (particularly for P1.6). In the traditional activity, prompts mainly included incentives (8/9) to encourage collaboration (e.g., “it's beautiful to work as a solid group”), spontaneous behaviors (8/9, e.g., “bravo P1.7!”), or to take hindsight (e.g., “try to take hindsight as P1.2 does”).

Various **levels of autonomy** emerged in both conditions. With *MusicTraces*, most participants could act intentionally without much help (7/8, e.g., drawing lines), or ask for support when needed (7/8), either verbally or non-verbally, based on their profiles. Most participants also performed intentional acts during the traditional activity (8/9), but they tended to explore more (8/9), maybe due to higher usability.

Interestingly, P1.9 (who only participated in the traditional activity), provided some **peer support** to P1.3 and P1.6 during the traditional activity, to help them draw or refocus.

### 5. Study 2: Effect of *MusicTraces* over time

The first study's results informed the redesign of some features of *MusicTraces* to improve its usability and cater to people with more severe NDCs. In particular, the smart objects were iteratively redesigned, leading to only using one object per participant for paint and erasing (see Section 5.1). Thus, this second study aimed to complement the first study by testing the revised version of *MusicTraces* with people have more severe NDCs and over a longer timescale to obtain more robust results. The protocol is similar to the first
study's. Still, as the study was conducted in a different care center to work
with adults who did not participate in the first study, this situation prevented
us from using a control group. Moreover, based on the insights from the first
study and to adapt to this new care center, a few minor methodological
differences were made, which are detailed in Section 5.2.

5.1. Redesign of the smart objects

The smart objects were redesigned to address the usability issues en-
countered during the first study, due to their large size and the complexity
of switching between the brush and eraser. This process involved two new
iterations (see Figure 2). First, we made the objects smaller while improving
the grip. Second, we kept only one object to paint and erase. The new de-
sign stems from two metaphors: the pencil eraser, which allows one to draw
from one end and erase from the other, and the whiteboard eraser, which en-
ables erasing from the side of the object with synthetic felt. All iterations of
the smart objects were 3D-printed using UV-resistant, shock-absorbing resin.
Similar electronic components were used compared to the first iteration.

5.2. Method

The experiment was conducted in a different care center managed by FeA.
Only the methodological differences compared with the first study are high-
lighted below. The protocol received approval from the ethical committee at
our university under the same reference as the first study.

5.2.1. Participants

Eight participants with NDCs and ID (3F/5M) were recruited by the
caregivers of the care center using convenience sampling. They aged between
19 and 35 (M: 25.0, SD: 6.38). Their profiles are described in Table 1. Most
participants had verbal abilities (7/8), with P2.1 and P2.4 often doing short
sentences (respectively, yes-no answers and due to his shyness). P2.2 could
only vocalize, point, and mimic. Participants displayed more severe ID than
during the first study (“profound” for six of them) and slightly higher atten-
tion spans (more than 15 minutes for four participants). However, P2.6 still
showed good autonomy (coming to the care center alone). Some participants
displayed atypical sensory experiences: P2.2 had a hearing impairment, P2.5
and P2.7 disliked being touched, P2.7 hated high-pitched sounds, and P2.3
was easily anxious toward sensory stimuli. Three participants often displayed
repetitive verbal behaviors (P2.6, P2.7, P2.8). Most participants were curious
about new activities (M: 3.93/5, SD: 1.46), except for P2.2 (1/5). Context-
switching did not generate much anxiety (M: 1.79, SD: 0.91), except for P2.7
(3.5/5) and P2.8 (4/5). Most participants often used digital tools (M: 4.07;
SD: 1.27), like mobile phones for social media, television, or computers. Only
P2.5 previously experienced other iMSEs and greatly enjoyed it.

5.2.2. *Adjustments to the protocol*

Participants completed two sessions with *MusicTraces* separated by one
week. Sessions were structured similarly to the first study, except for four
changes. First, each participant only used one smart object to create and
erase (see Section 5.1). Second, to adapt to caregivers’ requests in this new
care center, all participants involved in this study stayed in the room while
two were using *MusicTraces*. Third, the digital tutorial was replaced by
a “physical” tutorial, with the research investigator explaining each step.
Indeed, physical support was key to achieve a complete understanding during
the first study, and explanations from the caregiver or investigator were easier
to understand than the digital tutorial for the first participants in the second
study (P2.1, P2.2). Finally, due to technical difficulties in the network setup,
the tablet could not be used. However, since the caregivers only used it once
during the first study due to being busy helping the participants, and since
they were expected to support the participants even more during this study
given their more severe conditions, it was not considered an issue.

5.2.3. *Data collection and analysis*

The second study employed data collection and analysis methods similar
to those of the first study. Caregivers completed a profiling questionnaire for
all participants, whose questions are identical to those described in Section
4.1.3 in both formulation and scale. At the end of each session, each par-
ticipant and their caregiver were asked a semi-directed questionnaire. The
questions and scale, again, are identical to the ones reported in Section 4.1.3.
In particular, they include the Music Therapy Session Assessment Scale (MT-
SAS) and the NASA TLX-Short form questionnaire. These data were ana-
lyzed using the same descriptive statistics as for the first study to guarantee
comparability between the two studies. Each session was recorded with a sin-
gle camera. The videos were coded using a predefined coding scheme coming
from the SCERTS framework, as well as inductive methods stemming from
the Grounded Theory, as described in Section 4.1.4.
<figure><img src="image_9.png" /><figcaption>Figure 7: Study 2: Results about the participants’ state (Before and After), engagement (Eng.), Social Communication (SC), Emotional Regulation (ER), and Transactional Support from the caregiver (TS Car.) and system (TS Sys.), represented in a radar plot (scale 1-5, 5 external value)</figcaption></figure>

## 5.3. Results

As for the first study, results are presented following the four categories coming from the qualitative analysis: Play (PL), Social Communication (SC), Emotional Regulation (ER), and Transactional Support (TS). Aggregate measures of the quantitative ratings are computed and reported in Figure 7 and Figure 8. Across both studies, TS ratings are similar for caregivers and the system, as well as the engagement levels reported by the participants and caregivers. Thus, in the figure, the levels of engagement are combined, but TS levels are kept separate to be consistent with the first study. The number of participants corresponding to each behavior identified through the qualitative analysis for each session is summarized in Table 2. The first and second sessions are abbreviated s1 and s2 in the text.

### 5.3.1. Play

Regarding **possibility of play**, most participants (7/8) played for more than ten minutes during both sessions (s1 - M: 16'30, SD: 6'06; s2 - M: 15'23, SD: 06'08), whose durations were similar. However, P2.8 refused to enter the room. Some participants stayed longer than their usual attention spans during s1 (P2.1, P2.2, P2.3, P2.7) and s2 (P2.1, P2.2, P2.7). Six participants were engaged and focused, including three (P2.1, P2.2, P2.3) who were expected to face difficulties (see Table 1). P2.7 showed some refusal behaviors: asking to leave (s1, s2). However, he was fatigued upon arrival at s1 and still did more actions during s2, a positive outcome given his condition. The participants’ states were stable throughout the sessions (see Figure 7), even though three reported feeling more tired by the end (e.g., due to holding the objects). The perceived workload was high during s1 (NASA-TLX - M: 51.9, SD: 20.5) and s2 (NASA-TLX - M: 47.4, SD: 12.3), despite a high
<figure><img src="image_10.png" /><figcaption>Figure 8: Study 2: Results about the *SC* items, including the six MT-SAS items, and the question asking if the participants socially communicated more with *MusicTraces* in comparison with their daily life (*More social*) represented in a radar plot (scale 1-5, 5 external value)</figcaption></figure>

perceived level of success and low frustration. However, it decreased during s2, being comparable with the levels observed with the traditional activity in the first study. By the end of s2, most participants had understood how to play (7/8), with caregivers' support often fading over time.

In terms of **play states**, most participants played in parallel during both sessions while often glancing at each other (7/8). More associative play emerged during s2 (6/8), e.g., four participants imitated each others' drawings during s2 (see Figures 9.b and d). Cooperative play concerned two friends (P2.5 and P2.6), e.g., making music together during s1 (see Figure 9.a). P2.2 and P2.7 sometimes played alone, maybe due to P2.2's understanding issues and P2.7's disengagement.

Regarding participants' **preferences**, three favored the music (P2.1, P2.6, P2.7), two the painting (P2.4, P2.5), P2.3 enjoyed both, and P2.2 did not express any preference. In particular, P2.3 kept saying the name of his favorite singer during both sessions. Three participants who enjoyed painting tended to paint more precisely (P2.1, P2.4, P2.5), e.g., depicting the Little Red Riding Hood (P2.5 during s1, see Figure 9.a) or closed shapes (P2.4 during s2, see Figure 9.d). The four others, who often preferred the music, produced more scribbles (see P2.3 and P2.6 on Figures 9.a and c).

In terms of **motion range**, four participants could move from the start, P2.1 and P2.3 made ampler movements from s2, and P2.2 always stayed still.

### 5.3.2. *Social Communication (SC)*

All participants showed moderate SC levels (see Figure 7), with more body reciprocity and emotional involvement during s2 (see Figure 8). The levels of harmonized productions and dynamism were stable over time. In particular, P2.6 socially interacted more than in his daily life according to
<figure><img src="image_11.png" /></figure>

Figure 9: Specific events that occurred during the second study. (a.) P2.5 (right) joining P2.6 (left) in making music after having drawn the Little Red Riding Hood (session 1 - s1). (b.) P2.5 (right) and P2.6 (left) both drawing flower-like shapes (session 2). (c, d) P2.3 (right) and P2.4 (left) drawing in parallel (s1). (e.) P2.3 (right) and P2.4 (left) drawing similar shapes (s2). (f.) P2.1 becoming autonomous while P2.2 is supported after having repetitively drawn in the same position (s2). (g.) P2.7 agreeing to make music (s2).

caregivers during s1.

**Direct SC** consisted in observing each other for all participants, often to understand how to play (e.g., P2.6 erased after having looked at P2.5 during s2). Verbal interactions were often addressed to the caregivers (6/8 for both tests), when answering questions or spontaneously speaking (e.g., P2.1 said "*beautiful*" during s1). Nevertheless, a few participants did speak to their partner (3/8 and 2/8 during s1 and s2), mainly to share their joy (P2.1 to P2.2 during s1) or to discuss their drawings (P2.5 and P2.6 during s1).

Regarding **mediated SC**, some attuned productions emerged (4/8 during s2), by imitating each other's drawings (4/8 s2 P2.3 and P2.4 made similar drawings, see Figure 9.b and d) or making music together (P2.5 joined P2.6 after having finished her drawing during s1, see Figure 9.a). Imitating each other's drawings emerged more during s2, with four participants against two (P2.3 and P2.4) during s1 (see Figures 9.a, b, c, d). P2.4 also expressed he preferred his partner's drawing during s1, making his caregiver say that the activity aligns with their day-to-day objectives to interact with others.
### 5.3.3. Emotional Regulation (ER)

Caregivers' ratings suggest that ER was moderate with a slight decrease during s2 (see Figure 7). In particular, low levels of ER concerned P2.7 who was disengaged (see above), and P2.2 who faced some bugs during s2 (due to staying very still), leading to be slightly puzzled (see Figure 7).

Regarding **mediated ER** (by the activity), most participants felt happy (6/8), saying it (4/8), smiling and laughing (4/8), or asking to continue playing (P2.5 s2). In particular, P2.3 said he was “*moved by the music*”. As a result, most participants were focused (6/8), ignoring unexpected events (3/8, e.g., body tracking issues or external sounds coming from the care center). Some participants stayed longer than their usual attention span (4/8, see Table 1). Also, four users relaxed (4/8s2), and six remained calm (P2.1 during s1). Indeed, P2.5 said it was *relaxing* (s1), and one caregiver said the “*sounds helped P2.3 take his mind off his anxious state*” (s2).

Several **self-regulation** strategies emerged. Three participants performed repetitive actions during both sessions: verbal repetitions (2/8, e.g., P2.7 repeated “like we did last time” during s2) or through painting (P2.2 and P2.3). About the latter, P2.2 and P2.3 (both having hearing impairments) sometimes filled small areas with music notes (see Figure 9.c and e). Some participants also unloaded their energy, such as P2.7 who performed wide circular gestures to make music during s2 (see Figure 9.f). P2.1 also said she was “*energized*” during s1.

### 5.3.4. Transactional Support (TS)

The level of support was high but heterogeneous among the participants, and faded over time in relation to their increased autonomy (see Figure 7).

Caregivers' prompts mainly aimed to promote understanding about how to play (7/8 for both tests), being predominantly verbal with high (P2.5, P2.6, P2.7) or mild (P2.1, P2.2, P2.3, P2.7) physical support. Verbal incentives aimed to spur motivation (7/8 for both tests), by encouraging actions through compliments (e.g., “*bravo P2.1!*”). Sensory prompts aimed to help P2.1, P2.2, and P2.3 move more during both tests. It benefited P2.1 and P2.3, while P2.2 always stayed still (which is understandable given her condition). At last, caregivers tried to help P2.7 refocus on the canvas during both tests, e.g., turning his head (s1).

Various **levels of autonomy** emerged. Six participants explored by themselves during s1 (except P2.2), including P2.7, who was also curious about the controller (e.g., approaching it to his ear). Four participants per-
formed intentional acts alone from s1 (P2.3, P2.4, P2.5, P2.6), and six during
s2 (P2.1, P2.7). However, P2.2 and P2.7 always required significant support.
All participants except P2.7 could ask for support when needed, either ver-
bally (5/8) or non-verbally (P2.2). The caregivers checked more on the par-
ticipants with limited verbal abilities (P2.2, P2.6). No peer support emerged.
Yet, P2.2 turned to P2.1 during s2 when the sound got loud, maybe looking
for support. At last, higher autonomy was often related to increased usability
and facing fewer bugs during gameplay.

6. Discussion

This work investigated whether a two-users music-making and painting
activity within an iMSE could support social communication (RQ1), emo-
tional regulation (RQ2), and well-being (RQ3) among adults with moderate-
to-severe NDCs and ID, as well as social play (RQ4), considered a prerequisite
for the other outcomes. To that end, *MusicTraces* was created through a co-
design process. Two complementary field studies were then conducted: the
first compared the experiences of ten adults using *MusicTraces* with a tradi-
tional art therapy activity; the second explored its impact over two sessions
for eight adults with more severe NDCs.

Findings from both studies — whether quantitative ratings from ques-
tionnaires or qualitative findings derived from observations — provide initial
supporting evidence for our three primary research questions. In particular,
the qualitative analysis, which drew on video coding and combined inductive
and deductive approaches, offers detailed insights into participants’ behav-
iors. Notably, in the first study, observed behaviors with *MusicTraces* were
often comparable to, and at times surpassed, those from the traditional ac-
tivity in terms of social interaction, emotional regulation, and well-being. In
the second study, slight increases in social interaction and autonomy were
observed over time. Caregivers were surprised by the high engagement of
some participants with severe ID or motor issues, several of whom interacted
more than they typically do in their daily lives. These positive outcomes
are related to *MusicTraces'* support for social play (RQ4). Indeed, most
participants quickly grasped how to play, and while parallel play was most
common, associative play and cooperative play emerged in the second study,
with joint actions being frequent (e.g., crossing each other's lines).

In what follows, we examine the main similarities and differences across
both studies, then discuss the limitations, and finally outline perspectives for
the HCI community working with individuals with NDCs and ID

## 6.1. Potential of combining music and painting

### 6.1.1. Social communication (SC)

*MusicTraces* positively supported SC across both studies. Moderate-to-high SC levels were observed with *MusicTraces* and the traditional activity in the first study. This finding is positive, as caregivers reported that it took more than a year to reach such SC levels with their traditional art activity. Moreover, compared with the traditional activity, more harmonious and dynamic productions appeared with *MusicTraces*. In the second study, SC levels slightly increased over time (e.g., body reciprocity). Four participants also socially interacted more than in their daily lives (P1.2, P1.5, P1.8, P2.6). Three interacted more than in the traditional activity (P1.2, P1.5, P1.8), including P1.8, who even refused to participate in the traditional activity, or P1.5, who drew alone during the traditional activity. In contrast, P1.7 and P1.1, who socially interacted the most during the traditional activity, also interacted much with *MusicTraces*.

Our findings highlight the potential of combining music and painting within open-ended activities to accommodate multiple ways of playing together for participants with varying conditions. In particular, associative play sometimes emerged through various means (e.g., one user making music while the other painted). Only participants who were friends cooperated in both studies. Except for instances of associative play, participants often played in parallel and looked at their partner to understand how to play. They also sometimes spontaneously shared their joy with their partner (e.g., P2.1 talked to P2.2) and with their caregiver (e.g., describing their painting). However, participants moved more to share their spaces during the traditional activity, maybe due to more caregivers' prompting.

### 6.1.2. Emotional regulation (ER)

*MusicTraces* positively supported ER, with ER levels being slightly higher than in the traditional activity, and heterogeneous in the second study (see Figures 4 and 7). Most participants had fun and remained focused longer than their usual attention spans. Sessions' durations were similar to those of the traditional activity. In particular, P1.6 was described as "into the game", while he often lost focus during the traditional activity. As a result, most participants could ignore unexpected events (e.g., body tracking issues). In both studies, some participants also felt relaxed, echoing observations made
in the past project called *SensoryPaint* [11]. In particular, one caregiver said that “the sounds momentarily stopped anxious behaviors for P2.3.”

Four participants showed low ER levels (P1.3, P1.4, P2.2, P2.7), maybe related to their profiles: not being curious (P1.4, P2.7), showing understanding issues (P1.4, P2.2), or being afraid of committing mistakes (P1.3). Still, they all agreed to try the activity, with no signs of anxiety. In particular, P1.3 said she still had fun and behaved similarly during the traditional activity. P2.7 still performed more actions during the second session. For P2.7 and P2.8, conducting more sessions over a longer time frame may have helped.

Some participants developed self-regulation strategies with *MusicTraces*. For instance, three participants performed repetitive circular gestures (in both studies), and two with hearing impairments crowded canvas areas with musical spots (in the second study). In contrast, during the traditional activity, self-regulation strategies were often unrelated to the activity (e.g., looking up). Such outcomes are promising and reinforce previous claims in the HCI literature to design systems that support creative uses, i.e., not initially intended by the designers [61, 14]. Indeed, participants could play as they wished, supporting the concept of *neurodivergent play* as defined in [26]. Such outcomes also echo a caregiver’s comment during the design process, who suggested to “play with repetitive behaviors”, and previous claims in the literature [26, p. 29] (e.g., using “fidgeting as a resource for play”).

### 6.1.3. *Well-being*

Participants’ well-being was high in both studies due to several reasons. First, most participants had fun playing with *MusicTraces* (see above). They felt like succeeding, even though caregivers’ support was required to fully understand the activity (which was expected given their profiles). This fundamental role of caregivers aligns with previous HCI literature [59, 60]. Participants’ actions sometimes surprised the caregivers, e.g., P1.8’s engagement, P1.7’s extensive range of motion, or P1.5’s ample movements (see Figure 6.c). Such engagement is related to the fact that the activity facilitated social play in various ways (parallel, associative, or cooperative), while enabling multiple ways of acting (e.g., self-regulation strategies). Finally, *MusicTraces* often matched participants’ interests, with around half preferring the music and the others favoring the painting or both activities. Such preferences influenced how they played, scribbling more when they preferred the music or being more precise when they enjoyed painting.
## 6.2. Limitations

The first limitations of our research are related to the difficulties in conducting our studies in care centers, specifically with participants who have NDCs with or without ID. During the first study, *MusicTraces* was compared with a group-based art therapy activity, since no two-user activity existed at the care center we worked with. However, the comparison still holds relevance, as both approaches aim to foster collaboration and rely on music and painting. Additionally, we could recruit only a few participants (respectively ten and eight for the two studies), and scheduling issues during the first study prevented some participants from playing with *MusicTraces* and the traditional activity. Furthermore, users' physical and emotional state prior to entering the care center could not be controlled or fully accounted for, despite its potential impact on their behavior during the experience. Such difficulties are common when conducting field studies with vulnerable populations [35, 23, 11]. To mitigate this limitation, participants were asked questions during the experiment, and the caregiver verified that they felt sufficiently well to participate before beginning, thereby preventing them from participating if they felt unwell. Despite these challenges, our studies generated rich qualitative insights that could potentially inform future HCI research involving people with NDCs, both with and without ID.

The second type of limitation concerns the presence of caregivers during the experience, acting as potential facilitators. While their involvement was mandatory to ensure users' safety and security, as well as to comply with the center's internal regulations, further research should investigate the impact of caregivers' presence on users' performance, engagement, and social behavior.

Lastly, although we intended to capture participants' views through closed questions, the interpretations of researchers and caregivers prevailed due to participants' limited verbal and intellectual abilities. Developing methodologies to involve participants more directly is therefore desirable, for instance by drawing on participatory design methods [31], such as cards [63, 64] or the concept of *probes* [65]. We particularly encourage the latter, as the visual or auditory *traces* left by *MusicTraces* could serve as *probes* into participants' experiences and be interpreted post-session by participants, caregivers, and researchers.

## 6.3. Perspectives

Caregivers remarked that, overall, people with NDCs can struggle to paint together when they have to cross each others' lines. During the traditional
activity, P1.1 also expressed that it was harder to cross lines when she did
not know the other participants well. Two potential redesigns could facili-
tate such interactions. First, *MusicTraces* could rely on thick textures that
would blend to make up an overall texture without seeing boundaries. To
our knowledge, no previous research has addressed such objectives. Second,
rather than using an iMSE, *MusicTraces* could rely on physical painting ma-
terials (e.g., sponges) that would be sonified in real time. Another co-design
process with stakeholders is required to decide on such design possibilities.

In both studies, many participants were highly focused, staying longer
than their usual attention spans. Future research should investigate how
combining music and painting can help increase participants’ attention and
how long such effects last. Indeed, *MusicTraces* could then be used to calm
participants and prepare them for other clinical activities.

*MusicTraces* was designed for an iMSE to precisely control sensory stimuli
and thus cater to our participants’ sensory needs. However, iMSEs induce
portability issues that we want to address in future works to facilitate working
in different spaces. To that end, using creative tangibles seems promising,
drawing upon projects such as *RHYME* [12]. However, tangibles prevent
filtering out unwanted surrounding stimuli in the experimental space (e.g.,
wall colors), and large-size tangibles are challenging to move. Thus, we plan
on porting *MusicTraces* in augmented reality, given its potential for people
with severe NDCs, due to keeping contact with their familiar environment
while augmenting it with sensory stimuli [87, 88]. After implementing the
respective design changes, we plan to compare this new version with the
iMSE version in terms of the same research questions.

7. Conclusion

This paper explored how combining music-making and painting within
an interactive multisensory environment could support social communica-
tion, emotional regulation, and well-being for adults with Neurodevelopmen-
tal Conditions (NDCs) and associated Intellectual Disability (ID). To that
end, the two-player music and painting activity called *MusicTraces* was first
created through a four-step co-design process involving caregivers and people
with NDCs and ID. Two complementary field studies were then conducted
within care centers based on *MusicTraces*. The first study compared it with
traditional group painting with music for ten adults, while the second exam-
ined its impact over two sessions on eight adults with more severe conditions.
Our research presents several contributions. First, the co-design process of *MusicTraces* is presented, which could inform the design of future HCI studies. Second, findings from both studies suggest positive evidence concerning our three research questions. In particular, the caregivers were often positively surprised by our findings regarding the participants with the most severe conditions. A categorization of participants’ experiences with *MusicTraces* was also derived from our data analysis, drawing upon quantitative and qualitative methods. This categorization could serve as a methodological resource for future studies. Lastly, perspectives are drawn from comparing the findings of both studies, which could inform the design and methodologies of future HCI studies involving people with NDCs and ID.

## 8. Research data

The raw data supporting the conclusions of this article cannot be made publicly available by the authors, due to patient confidentiality restrictions. Requests to access the datasets should be directed to the corresponding author.

## 9. Author contributions

**Valentin Bauer:** Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft. **Giacomo Caslini:** Investigation, Software, Writing – review and editing. **Marco Mores:** Resources, Investigation, Writing – review and editing. **Mattia Gianotti:** Methodology, Supervision, Writing – review and editing. **Franca Garzotto:** Methodology, Funding acquisition, Resources, Supervision, Writing – review and editing.

## 10. Funding sources

This research was carried out within the MUSA – Multilayered Urban Sustainability Action – project, funded by the European Union – NextGenerationEU, under the National Recovery and Resilience Plan (NRRP) Mission 4 Component 2 Investment Line 1.5: Strengthening of research structures and creation of R&D “innovation ecosystems”, set up of “territorial leaders in R&D”. G. Caslini’s Ph.D. grant is supported by TIM S.p.A., Innovation Department. This research was also pursued thanks to the funding from the Fondation John Bost pour la Recherche.
# 11. Acknowledgements

The authors would like to thank all the participants who participated in this research, and all the caregivers working at Fraternità e Amicizia Società Cooperativa Sociale ONLUS. Without them, conducting these experiments would not have been possible.

# References

[1] K. Bottema-Beutel, S. K. Kapp, J. N. Lester, N. J. Sasson, B. N. Hand, Avoiding Ableist Language: Suggestions for Autism Researchers, Autism in Adulthood 3 (1) (2021) 18–29. doi:10.1089/aut.2020.0014.

[2] A.-M. DePape, S. Lindsay, Lived Experiences From the Perspective of Individuals With Autism Spectrum Disorder: A Qualitative Meta-Synthesis, Focus on Autism and Other Developmental Disabilities 31 (1) (2016) 60–71. doi:10.1177/1088357615587504.

[3] N. Humphrey, S. Lewis, ‘Make me normal’: The views and experiences of pupils on the autistic spectrum in mainstream secondary schools, Autism 12 (1) (2008) 23–46. doi:10.1177/1362361307085267.

[4] D. Fancourt, D. Finn, What is the evidence on the role of the arts in improving health and well-being? A scoping review., Health Evidence Network synthesis report 67, World Health Organization Regional Office for Europe, Copenhagen (2019).

[5] J.-P. Després, F. Julien-Gauthier, F. Bédard-Bruyère, M.-C. Mathieu, Music and young people with intellectual disability: A scoping review, Psychology of Music 52 (3) (2024) 362–379. doi:10.1177/03057356231203697.

[6] G. Russell, W. Mandy, D. Elliott, R. White, T. Pittwood, T. Ford, Selection bias on intellectual ability in autism research: a cross-sectional review and meta-analysis, Molecular Autism 10 (1) (2019) 9. doi:10.1186/s13229-019-0260-x.

[7] H. Tager-Flusberg, D. Plesa Skwerer, R. M. Joseph, B. Brukilacchio, J. Decker, B. Eggleston, S. Meyer, A. Yoder, Conducting research with minimally verbal participants with autism spectrum disorder, Autism 21 (7) (2017) 852–861. doi:10.1177/1362361316654605.
[8] H. Mayer-Benarous, X. Benarous, F. Vonthron, D. Cohen, Music Therapy for Children With Autistic Spectrum Disorder and/or Other Neurodevelopmental Disorders: A Systematic Review, Frontiers in Psychiatry 12 (2021) 643234. doi:10.3389/fpsyt.2021.643234.

[9] A. Bernier, K. Ratcliff, C. Hilton, P. Fingerhut, C.-Y. Li, Art Inter-ventions for Children With Autism Spectrum Disorder: A Scoping Re-view, The American Journal of Occupational Therapy 76 (5) (2022) 7605205030. doi:10.5014/ajot.2022.049320.

[10] N. Pares, P. Masri, G. van Wolferen, C. Creed, Achieving Dialogue with Children with Severe Autism in an Adaptive Multisensory Inter-action: The "MEDIATE" Project, IEEE Transactions on Visualization and Computer Graphics 11 (6) (2005) 734–743.

[11] K. E. Ringland, R. Zalapa, M. Neal, L. Escobedo, M. Tentori, G. R. Hayes, SensoryPaint: a multimodal sensory intervention for children with neurodevelopmental disorders, in: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp '14 Adjunct, ACM Press, Seattle, Washington, 2014, pp. 873–884, sensoryPaint. doi:10.1145/2632048.2632065.

[12] B. Cappelen, A.-P. Andersson, Health Improving Multi-Sensorial and Musical Environments, in: Proceedings of the Audio Mostly 2016, ACM, Norrköping Sweden, 2016, pp. 178–185. doi:10.1145/2986416.2986427.

[13] V. Bauer, A. Adjorlu, L. B. Pedersen, T. Bouchara, S. Serafin, Music Therapy in Virtual Reality for Autistic Children with Severe Learning Disabilities, in: 29th ACM Symposium on Virtual Reality Software and Technology, ACM, Christchurch New Zealand, 2023, pp. 1–9. doi:10.1145/3611659.3615713.

[14] A. Nonnis, N. Bryan-Kinns, Mazi: Tangible Technologies as a Channel for Collaborative Play, in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, ACM Press, Glasgow, Scotland Uk, 2019, pp. 1–13. doi:10.1145/3290605.3300670.

[15] L. Villafuerte, S. Jordà, M. S. Markova, Acquisition of Social Abilities Through Musical Tangible User Interface: Children with Autism Spectrum Condition and the Reactable (2012) 16.
[16] G. Ragone, K. Howland, E. Brulé, Evaluating Interactional Syn-
chrony in Full-Body Interaction with Autistic Children, in: Interac-
tion Design and Children, ACM, Braga Portugal, 2022, pp. 1–12.
doi:10.1145/3501712.3529729.

[17] F. L. Cibrian, O. Peña, D. Ortega, M. Tentori, BendableSound: An elastic multisensory surface using touch-based interactions to assist children with severe autism during music therapy, International Journal of Human-Computer Studies 107 (2017) 22–37. doi:10.1016/j.ijhcs.2017.05.003.

[18] A.-P. Andersson, B. Cappelen, Vocal and tangible interaction in RHYME, in: K. Stensæth (Ed.), Music, Health, Technology and Design, no. 8 in Series from the Centre for Music and Health, NMH-publications, 2014, pp. 21–38.

[19] M. H. Laurie, P. Warreyn, B. V. Uriarte, C. Boonen, S. Fletcher-Watson, An International Survey of Parental Attitudes to Technology Use by Their Autistic Children at Home, Journal of Autism and Developmental Disorders 49 (4) (2019) 1517–1530. doi:10.1007/s10803-018-3798-0.

[20] M. O. Mazurek, C. R. Engelhardt, K. E. Clark, Video games from the perspective of adults with autism spectrum disorder, Computers in Human Behavior 51 (2015) 122–130. doi:10.1016/j.chb.2015.04.062.

[21] M. Sandbank, K. Bottema-Beutel, S. Crowley, M. Cassidy, K. Dunham, J. I. Feldman, J. Crank, S. A. Albarran, S. Raj, P. Mahbub, T. G. Woynaroski, Project AIM: Autism intervention meta-analysis for studies of young children., Psychological Bulletin 146 (1) (2020) 1–29.
doi:10.1037/bul0000215.

[22] L. Albrecht, H. Starnes, K. Benton, A. Bol, E. Gettings, P. Dagenhard-Trainer, Interactive digital art for individuals with intellectual disabilities, Journal of Intellectual Disabilities 26 (2) (2022) 442–454. doi:10.1177/1744629520975554.

[23] G. Ragone, J. Good, K. Howland, How Technology Applied to Music-Therapy and Sound-Based Activities Addresses Motor and Social Skills in Autistic Children, Multimodal Technologies and Interaction 5 (3) (2021) 11. doi:10.3390/mti5030011.
[24] K. R. Agres, R. S. Schaefer, A. Volk, S. van Hooren, A. Holzapfel, S. Dalla Bella, M. Müller, M. de Witte, D. Herremans, R. Ramirez Melendez, M. Neerincx, S. Ruiz, D. Meredith, T. Dimitriadis, W. L. Magee, Music, Computing, and Health: A Roadmap for the Current and Future Roles of Music Technology for Health Care and Well-Being, Music & Science 4 (2021) 1-32. doi:10.1177/2059204321997709.

[25] D. Johnston, H. Egermann, G. Kearney, Innovative computer technology in music-based interventions for individuals with autism moving beyond traditional interactive music therapy techniques, Cogent Psychology 5 (1) (Nov. 2018). doi:10.1080/23311908.2018.1554773.

[26] K. Spiel, K. Gerling, The Purpose of Play: How HCI Games Research Fails Neurodivergent Populations, ACM Transactions on Computer-Human Interaction 28 (2) (2021) 1-40. doi:10.1145/3432245.

[27] K. Spiel, C. Frauenberger, O. Keyes, G. Fitzpatrick, Agency of Autistic Children in Technology Research—A Critical Literature Review, ACM Transactions on Computer-Human Interaction 26 (6) (2019) 1-40. doi:10.1145/3344919.

[28] V. Bauer, T. Bouchara, P. Bourdot, Extended Reality Guidelines for Supporting Autism Interventions Based on Stakeholders' Needs, Journal of Autism and Developmental Disorders 53 (5) (2023) 2078-2111. doi:10.1007/s10803-022-05447-9.

[29] E. Cage, C. J. Crompton, S. Dantas, K. Strachan, R. Birch, M. Robinson, S. Morgan-Appel, C. MacKenzie-Nash, A. Gallagher, M. Botha, What are the autism research priorities of autistic adults in Scotland?, Autism (2024) 13623613231222656 doi:10.1177/13623613231222656.

[30] S. Parsons, N. Yuill, J. Good, M. Brosnan, 'Whose agenda? Who knows best? Whose voice?' Co-creating a technology research roadmap with autism stakeholders, Disability & Society 35 (2) (2020) 201-234.

[31] K. Spiel, C. Frauenberger, E. Hornecker, G. Fitzpatrick, When Empathy Is Not Enough: Assessing the Experiences of Autistic Children with Technologies, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17, ACM, Denver, CO, USA, 2017, pp. 2853-2864. doi:10.1145/3025453.3025785.
[32] K. Spiel, L. Malinverni, J. Good, C. Frauenberger, Participatory Evaluation with Autistic Children, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, Denver Colorado USA, 2017, pp. 5755–5766. doi:10.1145/3025453.3025851.

[33] V. Bauer, T. Padovano, M. Gianotti, G. Caslini, F. Garzotto, MusicTraces: A collaborative music and paint activity for autistic people, in: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, ACM, Honolulu HI USA, 2024, pp. 1–7. doi:10.1145/3613905.3651101.

[34] J. Mino-Roy, J. St-Jean, O. Lemus-Folgar, K. Caron, O. Constant-Nolett, J. Després, C. Gauthier-Boudreault, Effects of music, dance and drama therapies for people with an intellectual disability: A scoping review, British Journal of Learning Disabilities 50 (3) (2022) 385–401. doi:10.1111/bld.12402.

[35] R. Rushton, L. Kossyvaki, E. Terlektsi, Music-based interventions for people with profound and multiple learning disabilities: A systematic review of the literature, Journal of Intellectual Disabilities 27 (2) (2023) 370–387. doi:10.1177/17446295221087563.

[36] S. K. Kapp, K. Gillespie-Lynch, L. E. Sherman, T. Hutman, Deficit, difference, or both? Autism and neurodiversity., Developmental Psychology 49 (1) (2013) 59–71. doi:10.1037/a0028353.

[37] B. Stige, On a Notion of Participation in Music Therapy, Nordic Journal of Music Therapy 15 (2) (2006) 121–138. doi:10.1080/08098130609478159.

[38] E. L. Deci, R. M. Ryan, Self-determination theory: A macrotheory of human motivation, development, and health., Canadian Psychology / Psychologie canadienne 49 (3) (2008) 182–185. doi:10.1037/a0012801.

[39] E. Özcan, L. Frankel, J. Stewart, Uncommon music making: The functional roles of music in design for healthcare, Music and Medicine 11 (4) (2019) 245–255.

[40] Ł. Bieleninik, M. Geretsegger, K. Mössler, J. Assmus, G. Thompson, G. Gattino, C. Elefant, T. Gottfried, R. Igliozzi, F. Muratori, et al.,
Effects of improvisational music therapy vs enhanced standard care on symptom severity among children with autism spectrum disorder: The time-a randomized clinical trial, Jama 318 (6) (2017) 525-535.

[41] G. Storsve, Formal, informal, and non-formal learning as analytic categories for research in music education, International Journal of Music Education 0 (0) (2024) 02557614241250035. arXiv:https://doi.org/10.1177/02557614241250035, doi:10.1177/02557614241250035. URL https://doi.org/10.1177/02557614241250035

[42] F. L. Cibrian, J. Mercado, L. Escobedo, M. Tentori, A Step towards Identifying the Sound Preferences of Children with Autism, in: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare, ACM, New York NY USA, 2018, pp. 158-167. doi:10.1145/3240925.3240958.

[43] C. Schweizer, E. J. Knorth, T. A. Van Yperen, M. Sreen, Consensus-based typical elements of art therapy with children with autism spectrum disorders, International Journal of Art Therapy 24 (4) (2019) 181-191. doi:10.1080/17454832.2019.1632364.

[44] C. Schweizer, E. J. Knorth, M. Sreen, Art therapy with children with Autism Spectrum Disorders: A review of clinical case descriptions on 'what works', The Arts in Psychotherapy 41 (5) (2014) 577-593. doi:10.1016/j.aip.2014.10.009.

[45] N. Spiro, G. Tsiris, C. Cripps, A Systematic Review of Outcome Measures in Music Therapy, Music Therapy Perspectives 36 (1) (2018) 67-78. doi:10.1093/mtp/mix011.

[46] K. Werner, K. Kender, L. Scheepmaker, C. Frauenberger, Technologies Supporting Social Play in Neurodiverse Groups of Children, in: Proceedings of the 23rd Annual ACM Interaction Design and Children Conference, ACM, Delft Netherlands, 2024, pp. 218-231. doi:10.1145/3628516.3655791.

[47] C. Frauenberger, K. Kender, L. Scheepmaker, K. Werner, K. Spiel, Designing Social Play Things, in: Proceedings of the 11th Nordic
Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, ACM, Tallinn Estonia, 2020, pp. 1–12.
doi:10.1145/3419249.3420121.

[48] D. Whitebread, D. Neale, H. Jensen, C. Liu, S Lynneth Solis, E. Hopkins, K. Hirsh-Pasek, J. Zosh, The role of play in children’s development: a review of the evidence (2017). doi:10.13140/RG.2.2.18500.73606.
URL http://rgdoi.net/10.13140/RG.2.2.18500.73606

[49] P. Wolfberg, M. DeWitt, G. S. Young, T. Nguyen, Integrated Play Groups: Promoting Symbolic Play and Social Engagement with Typical Peers in Children with ASD Across Settings, Journal of Autism and Developmental Disorders 45 (3) (2015) 830–845. doi:10.1007/s10803-014-2245-0.

[50] J. Kientz, G. Hayes, M. Goodwin, M. Gelsomini, G. Abowd, Interactive Technologies and Autism, Second Edition, Synthesis Lectures on Assistive, Rehabilitative, and Health-Preserving Technologies 9 (1) (2020) i–229. doi:10.2200/S00988ED2V01Y202002ARH013.

[51] M. Wilson, Six views of embodied cognition, Psychonomic bulletin & review 9 (2002) 625–636.

[52] M. C. Smith, Sensory integration: Theory and practice, FA Davis, 2019.

[53] J. Mora-Guiard, C. Crowell, N. Pares, P. Heaton, Sparking social initiation behaviors in children with Autism through full-body Interaction, International Journal of Child-Computer Interaction 11 (2017) 62–71.
doi:10.1016/j.ijcci.2016.10.006.

[54] S. A. Brown, D. Silvera-Tawil, P. Gemeinboeck, J. McGhee, The case for conversation: a design research framework for participatory feedback from autistic children, in: Proceedings of the 28th Australian Conference on Computer-Human Interaction - OzCHI '16, ACM Press, Launceston, Tasmania, Australia, 2016, pp. 605–613. doi:10.5014/ajot.2010.09077.

[55] F. Garzotto, E. Beccaluva, M. Gianotti, F. Riccardi, Interactive Multisensory Environments for Primary School Children, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, Honolulu HI USA, 2020, pp. 1–12. doi:10.1145/3313831.3376343.
[56] C. Crowell, B. Sayis, J. P. Benitez, N. Pares, Mixed reality, full-body interactive experience to encourage social initiation for autism: Comparison with a control nondigital intervention, Cyberpsychology, Behavior, and Social Networking 23 (1) (2020) 5–9.

[57] E. Frid, H. Lindetorp, K. F. Hansen, L. Elblaus, R. Bresin, Sound Forest: Evaluation of an Accessible Multisensory Music Installation, in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow Scotland Uk, 2019, pp. 1–12. doi:10.1145/3290605.3300907.

[58] S. Jordà, G. Geiger, M. Alonso, M. Kaltenbrunner, The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces, in: Proceedings of the 1st international conference on Tangible and embedded interaction, ACM, Baton Rouge Louisiana, 2007, pp. 139–146. doi:10.1145/1226969.1226998.

[59] H. Lindetorp, M. Svahn, J. Hölling, K. Falkenberg, E. Frid, Collaborative music-making: special educational needs school assistants as facilitators in performances with accessible digital musical instruments, Frontiers in Computer Science 5 (2023) 1165442. doi:10.3389/fcomp.2023.1165442.

[60] K. Stensæth, Music therapy and interactive musical media in the future: Reflections on the subject-object interaction, Nordic Journal of Music Therapy 27 (4) (2018) 312–327. doi:10.1080/08098131.2018.1439085.

[61] C. Frauenberger, J. Good, A. Alcorn, H. Pain, Conversing through and about technologies: Design critique as an opportunity to engage children with autism and broaden research(er) perspectives, International Journal of Child-Computer Interaction 1 (2) (2013) 38–49. doi:10.1016/j.ijcci.2013.02.001.

[62] K. Stensæth, “Musical co-creation”? Exploring health-promoting potentials on the use of musical and interactive tangibles for families with children with disabilities, International Journal of Qualitative Studies on Health and Well-being 8 (1) (2013) 20704. doi:10.3402/qhw.v8i0.20704.

[63] E. Frid, Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice, Multimodal Technologies and Interaction 3 (3) (2019) 57. doi:10.3390/mti3030057.
[64] L. Scheepmaker, K. Kender, C. Frauenberger, G. Fitzpatrick, Leaving the Field: Designing a Socio-Material Toolkit for Teachers to Continue to Design Technology with Children, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, ACM, Yokohama Japan, 2021, pp. 1–14. doi:10.1145/3411764.3445462.

[65] B. Gaver, T. Dunne, E. Pacenti, Design: Cultural probes, Interactions 6 (1) (1999) 21–29. doi:10.1145/291224.291235.

[66] J. Harrison, A. Chamberlain, A. P. McPherson, Accessible Instruments in the Wild: Engaging with a Community of Learning-Disabled Musicians, in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow Scotland Uk, 2019, pp. 1–6. doi:10.1145/3290607.3313037.

[67] M. Gelsomini, G. Cosentino, M. Spitale, M. Gianotti, D. Fisicaro, G. Leonardi, F. Riccardi, A. Piselli, E. Beccaluva, B. Bonadies, L. Di Terlizzi, M. Zinzone, S. Alberti, C. Rebourg, M. Carulli, F. Garzotto, V. Arquilla, M. Bisson, B. Del Curto, M. Bordegoni, Magika, a Multisensory Environment for Play, Education and Inclusion, in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow Scotland Uk, 2019, pp. 1–6.

[68] F. Garzotto, E. Beccaluva, M. Gianotti, F. Riccardi, Interactive Multisensory Environments for Primary School Children, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, 2020, pp. 1–12. doi:10.1145/3313831.3376343.

[69] L. Benton, A. Vasalou, R. Khaled, H. Johnson, D. Gooch, Diversity for design: a framework for involving neurodiverse children in the technology design process, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Toronto Ontario Canada, 2014, pp. 3747–3756. doi:10.1145/2556288.2557244.

[70] J. Kramer, S. Daly, S. Yilmaz, C. Seifert, A case-study analysis of Design Heuristics in an upper-level cross-disciplinary design course., in: Annual Conference of American Society of Engineering Education (ASEE), Indianapolis, 2014.
[71] V. Braun, V. Clarke, Using thematic analysis in psychology, Qualitative Research in Psychology 3 (2) (2006) 77-101. doi:10.1191/1478088706qp063oa.

[72] F. Garzotto, M. Gelsomini, Magic Room: A Smart Space for Children with Neurodevelopmental Disorder, IEEE Pervasive Computing 17 (1) (2018) 38-48. doi:10.1109/MPRV.2018.011591060.

[73] G. Jacquemin, T. Coduys, M. Ranc, IANNIX 0.8, Mons, France, 2012, p. 10. doi:hal-03041777.

[74] W. Thompson, Soundpainting: the art of live composition. Workbook I, Walter Thompson, New York, N.Y., 2006, oCLC: 65221853.

[75] J.-B. Thiebaut, P. G. T. Healey, N. B. Kinns, Drawing Electroacoustic Music, in: International Conference on Mathematics and Computing (ICMC), 2008, p. 8.

[76] S. Bellini, J. Akullian, A Meta-Analysis of Video Modeling and Video Self-Modeling Interventions for Children and Adolescents with Autism Spectrum Disorders, Exceptional Children 73 (3) (2007) 264-287, publisher: SAGE Publications Inc. doi:10.1177/001440290707300301.

[77] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, D. Thomas, Manifesto for agile software development (2001). URL http://www.agilemanifesto.org/

[78] S. Elo, H. Kyngäs, The qualitative content analysis process, Journal of Advanced Nursing 62 (1) (2008) 107-115. doi:10.1111/j.1365-2648.2007.04569.x.

[79] A. Raglio, M. Gnesi, M. C. Monti, O. Oasi, M. Gianotti, L. Attardo, G. Gontero, L. Morotti, S. Boffelli, C. Imbriani, C. Montomoli, M. Imbriani, The Music Therapy Session Assessment Scale (MT-SAS): Validation of a new tool for music therapy process evaluation, Clinical Psychology & Psychotherapy 24 (6) (2017) 01547-01561. doi:10.1002/cpp.2115.

[80] S. G. Hart, L. E. Staveland, Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, in: P. A. Hancock,
N. Meshkati (Eds.), Advances in Psychology, Vol. 52 of Human Mental Workload, North-Holland, 1988, pp. 139–183. doi:10.1016/S0166-4115(08)62386-9.

[81] B. M. Prizant, A. M. Wetherby, E. Rubin, A. C. Laurent, The SCERTS Model: A Transactional, Family-Centered Approach to Enhancing Communication and Socioemotional Abilities of Children With Autism Spectrum Disorder, Infants & Young Children 16 (4) (2003) 296–316. doi:10.1097/00001163-200310000-00004.

[82] W. Farr, N. Yuill, H. Raffle, Social benefits of a tangible user interface for children with Autistic Spectrum Conditions, Autism 14 (3) (2010) 237–252, publisher: SAGE Publications Ltd. doi:10.1177/1362361310363280.

[83] J. C. Flanagan, The Critical Incident Technique, Psychological Bulletin 51 (4) (1954) 327–358. doi:10.1037/h0061470.

[84] B. G. Glaser, A. L. Strauss, The discovery of grounded theory: strategies for qualitative research, 4th Edition, Chicago: Aldine, New Brunswick, 1967, oCLC: 553535517.

[85] K. Charmaz, Constructing grounded theory: a practical guide through qualitative analysis, SAGE, London, 2006, oCLC: 255483218.

[86] S. V. Steffensen, F. Vallée-Tourangeau, G. Vallée-Tourangeau, Cognitive events in a problem-solving task: a qualitative method for investigating interactivity in the 17 Animals problem, Journal of Cognitive Psychology 28 (1) (2016) 79–105. doi:10.1080/20445911.2015.1095193.

[87] V. Bauer, T. Bouchara, O. Duris, C. Labossière, M.-N. Clément, P. Bourdot, Head-mounted augmented reality to support reassurance and social interaction for autistic children with severe learning disabilities, Frontiers in Virtual Reality 4 (2023) 1106061. doi:10.3389/frvir.2023.1106061.

[88] C. Berenguer, I. Baixauli, S. Gómez, M. d. E. P. Andrés, S. De Stasio, Exploring the Impact of Augmented Reality in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review, International Journal of Environmental Research and Public Health 17 (17) (2020) 6143. doi:10.3390/ijerph17176143.
URL https://www.mdpi.com/1660-4601/17/17/6143