# A Biofeedback System to Compose Your Own Music While Dancing

Carmelo Ardito, Tommaso Colafiglio, Tommaso Di Noia, Eugenio Di Sciascio

<figure><img src="image_1.png" /></figure>

## To cite this version:

Carmelo Ardito, Tommaso Colafiglio, Tommaso Di Noia, Eugenio Di Sciascio. A Biofeedback System to Compose Your Own Music While Dancing. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.309-312, 10.1007/978-3-030-85607-6_27. hal-04291228

HAL Id: hal-04291228

https://inria.hal.science/hal-04291228v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# A Biofeedback System to Compose Your Own Music
while Dancing

Carmelo Ardito[0000-0001-8993-9855], Tommaso Colafiglio[0000-0001-7184-310X], Tommaso Di Noia[0000-0002-0939-5462], Eugenio Di Sciascio[0000-0002-5484-9945]

Politecnico di Bari – Via E. Orabona 4, Bari (I-70125), Italy
{name.surname}@poliba.it

**Abstract.** Brain Computer Interfaces can enable engaging interactions between different art forms such as music, dance, painting. Building on this, we present a demo of a biofeedback system: a dancer wearing a NeuroSky headset adapts her performance according to the music she listens to. The same music has been generated by a music-composition software depending on her own real-time mental status represented by different fluctuations of some EEG parameters.

**Keywords:** Brain Computer Interface, Biofeedback, Music.

## 1 Introduction

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** An actual implementation of the proposed biofeedback system.</figcaption></figure>

The relationship between various art forms is one of the fields of investigation that crosses all artistic disciplines, from musicians/composers to choreographers and theater or cinema directors. The interplay between music and dance has profoundly influenced composers of every era by creating new technical-compositional models both in the
world of music and in the world of dance itself. At the same time, in Computer Science, the interest in Brain Computer Interfaces (BCIs) has significantly increased in recent years, because it represents a possibility for implementing a more intrinsic human-machine relationship. Also biofeedback systems [1] are gaining momentum, characterized by the continuous loop between users wearing a BCI and computers.

In this paper, we propose the demonstration of a biofeedback system that puts in a cho-
reographic/compositional relationship a dancer with a music composition software de-
veloped on purpose and two musicians. The dancer wears a NeuroSky MindSet device
[2] that detects some neurologic parameters, in particular her attention values [3]. De-
pending on these values, the software generate a music polyphony that is presented on
an electronic score to musicians, who play it. In turn, the music thus produced influ-
ences the mental status of the dancer, who will adapt her choreography according to
what she hears, generating new polyphonic musical bars.

<figure><img src="image_5.png" /><figcaption>Fig. 2. Dancer wearing the BCI headset during a live performance. <fnref n="1" /></figcaption></figure>

## 2 System Architecture and Operation

The hardware part of the system is based on the NeuroSky MindSet, i.e. a device that
can detect the brain EEG signal through a dry electrode placed on the forehead at the
Fp1 position (see [4] for further details). It translates electrical impulses related to brain
neuronal activity into digital numerical signals that can then be processed by a com-
puter. The electrical signals can be measured by placing the electrode of the device on
the forehead. The NeuroSky MindSet headset returns two parameters, namely attention
(similar to concentration) and meditation (similar to relaxation) [3], [5].

<footnote n="1">¹ The cables connected to the dancer's wrists, as well as the light strips on the headset are only for choreographic reasons and do not have any further technical implication.</footnote>
<figure><img src="image_6.png" /><figcaption>**Fig. 3.** The NeuroSky MindWave Mobile 2 used in the system.</figcaption></figure>

The general workflow of the system usage is schematized in **Fig. 4**. The *dancer*, wearing the NeuroSky MindSet headset, starts dancing. The device transmits every second her values of attention. The algorithm of *Polyphony Generator* module, depending on the intensity of the attention value, generates strings of rhythmic musical figures. For an high attention status, polyphonic voices are generated with smaller musical values such as quarter note, eighth note and sixteenth note and relative pause value. For low attention status, polyphonic voices with larger musical values such as whole note, half note and quarter note and relative pause values are generated. Given the attention value, the algorithm selects a set of rhythmic values and generates strings of rhythmic figures in a randomized modality.

<figure><img src="image_7.png" /><figcaption>**Fig. 4.** System workflow.</figcaption></figure>

The software is provided with a graphical interface that permits to choose on which musical scales model the polyphony is generated (see **Fig. 5**). All possible scales that can be calculated within an octave interval can be selected. Finally, the generated rhythmic sequences are superimposed on the previously selected scalar model. The process is repeated for all four voices of the polyphony texture. The music score reporting the resulting polyphony is displayed and continuously updated on a screen, so that musicians can play it.
## 3 Conclusion

We presented a biofeedback system useful for the interaction between music and dance in a BCI context. A possible evolution of this work could be to implement emotion recognition techniques to further improve the interaction between different art forms.

**Acknowledgments.** The authors acknowledge partial support of the following projects: PON Casa delle tecnologie emergenti di Matera "CTEMT" (CUP I14E20000020001), Servizi Locali 2.0, PON ARS01_00876 Bio-D, PON ARS01_00821 FLET4.0, PON ARS01_00917 OK-INSAID, H2020 PASSPARTOUT.

<figure><img src="image_8.png" /><figcaption>Fig. 5. User interface of the Polyphony Generator module.</figcaption></figure>

## References

1. Michael J. Wagner: Brainwaves and BioFeedback. A Brief History-Implication for Music Research. In: Journal of Music Therapy, Volume 12, Issue2, Summer 1975, Pages 46-58.

2. NeuroSky EEG device, http://neurosky.com/biosensors/eeg-sensor

3. Neurosky: NeuroSky's eSense™ Meters and Detection of Mental State. 2009

4. Tyagi, A., Semwal, S., Shah, G, A Review of EEG Sensors Used for Data Acquisition, International Journal of Computer Application, (2012)

5. Bronzino J.D. (2000) - *Principles of Electroencephalography*. The Biomedical Engineering - Handbook: Second Edition. Ed. Joseph D. Bronzino, Boca Raton: CRC Press LLC.