# Let's Dance: An Exergame to Lead the Way in Parkinson Rehabilitation

Pierpaolo Di Bitonto, Angela La Forgia, Giovanni Melone, Ada Potenza, Luca Rossetti, Antonio Ulloa Severino, Felice Vitulano

<figure><img src="image_1.png" /></figure>

## To cite this version:

Pierpaolo Di Bitonto, Angela La Forgia, Giovanni Melone, Ada Potenza, Luca Rossetti, et al.

HAL Id: hal-04291263

https://inria.hal.science/hal-04291263v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Let's dance: an exergame to lead the way in Parkinson rehabilitation

Pierpaolo Di Bitonto<fnref n="1" />, Angela la Forgia², Giovanni Melone², Ada Potenza<fnref n="1" />, Luca Rossetti², Antonio Ulloa Severino<fnref n="1" />, Felice Vitulano²

**Abstract.** Healthy Ageing represents one of the most crucial topics that our society will face in the next decades, due to the progressive demographic ageing of the population, (e.g., in Italy old-age dependency ratio is estimated to reach 59,7% by 2065). These statistics highlights the need of a new paradigm for active and healthy ageing, as well as a healthcare system able to support elderly population in age-related diseases management.

In this work we present Let's dance, an innovative exergame that aims to provide a new way to deliver physiotherapy for mild or moderate Parkinson's disease patients. Within the game, the patients are asked to reproduce the choreography proposed, whose dance moves are aimed toward the achievement of specific therapeutic goals (e.g., optimize motor coordination, physical endurance, etc.). The game is part of a broader integrated platform that through sensors and actuators located on a robot device in the clinical environment and on patients themselves, can track in real-time both the game performance and the clinical KPIs and determine the assessment of patient's therapeutic progress.

In this paper we present the first prototype of Let's dance and a use-case scenario.

**Keywords:** Parkinson, Exergame, Rehabilitation.

## 1 Introduction

According to the World Health Organization (WHO), noncommunicable diseases (NCDs) are responsible for almost 70% of all deaths worldwide [1].

The epidemic of NCDs poses devastating health consequences for individuals, families and communities, and threatens to overwhelm health systems. The socioeconomic costs associated with NCDs make the prevention and control of these diseases a major development imperative for the 21st century.

NCD management is the way to improve the quality of life for patients and their families and to reduce the costs of the treatment for health systems. Scientists, in fact, have proven that good management of NCD at home reduce drastically hospitalizations.

NCD management implies adopting healthy lifestyles, eating healthy foods and practicing physical activity in order to reduce the risk of complication and prevent the problems related to the disease. The difficulty lies in how to persuade the patients and their

<footnote n="1">¹ Grifo multimedia srl, Via B. Zaccaro, 17-19, 70126 Bari (BA) Italy
{p.dibitonto, a.potenza, a.ulloa}@grifomultimedia.it</footnote>

<footnote n="1">¹ Exprivia spa, Via A. Olivetti, 11 70056 Molfetta (BA) Italy
{angela.laforgia, giovanni.melone, luca.rossetti, felice.vitulano}@exprivia.com</footnote>
families to change their habits. Patients need to be continuously motivated, monitored and supported during the therapy.

In order to support patient empowerment and improve compliance with the medical prescription, game dynamics can help. Games are very useful because they give further motivation to reach the therapeutics goals even if patients lack intrinsic motivations connected to their health. In other words, patients that usually resist adopting healthy behaviors, if involved in a game tend to be more willing to adopt the same behaviors.

"Let's Dance" is an exergame used in rehabilitation for patients affected by Parkinson's Disease (P.D.). Usually, the rehabilitation activity for P.D. merges physical and cognitive activity: "Let's dance" game uses a rehabilitation protocol based on Irish dance [2] to involve patients in a joyful and engaging rehabilitation activity in two different game settings: at home and in the hospital. Game dynamics are inspired by the famous Ubisoft game "Just dance".

In this paper we present the game Let's dance and specifically section two describes how the motivational levers are used to engage patients and convince them to comply with the medical prescriptions; section three describes the game dynamics and software architecture; section four reports some conclusion and future work.

## 2 Engagement design

One of the first problems faced in the design phase was how to engage patients in the hospital or at home. Elderly people often reject technology and tend to think that it is a further and useless complication in everyday life. The challenge needed to be overcome (1) how to make the game appealing in the eyes of the patient and (2) how to convince the patient to play it repeatedly.

The first challenge was approached by presenting to the patient the game as digital therapy. It is not just a game but also a way to undergo rehabilitation. The game, during the exercises, can "observe" patients, gaining implicit feedback and calculating scores about their health. At the end of each game, session feedbacks are released. They are used to show health improvement or to suggest to the therapist new exercises that best fit the patient's condition. The second challenge was approached including a game dynamic where two couples of patients face each other to win the title of best dancers. The collaborative/competitive dynamics is very useful to include patients who are not very competitive and play just for fun.

## 3 Game description

"Let's Dance" is an exergame that aims to support patients suffering from mild or moderate Parkinson's disease in the rehabilitation process, by introducing a new way of approaching the therapy based on Irish dance. The game is used both at the hospital (within a specific clinical environment used for rehabilitation treatments) and at home, and it has been designed to ensure a rehabilitation protocol completely safe as regards to traumas and falls that may occur during physical activities.
## 3.1 Game session at the hospital

The game session at the hospital involves the participation of 1 to 4 users that can share similar therapeutic goals about motor and cognitive abilities. Prior to the game session, the users carry out a warm-up phase, preparatory to the correct execution of rehabilitation activities. Based on the specific users involved, the therapist can set a number of initial parameters, such as the music track, the exercise duration and, most importantly, the therapeutic goals on which to focus the session (see Fig. 1).

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Setting of initial parameter for the therapy.</figcaption></figure>

Once the game level starts, each player has two key elements on the screen: a visual representation of steps (hereinafter the “Virtual Dance Teacher”) that mimics the movements that the player will have to perform (each of which represents a game task), and a humanoid avatar (hereinafter the “Player”), that replicates each gesture made by the user in real space. During each game level, the Virtual Dance Teacher shows to the user a number of steps to perform: the user has to recreate precisely each movement, both in terms of correct physical performance and timing of execution. For each task performed, the game provides audio-visual (positive or negative) feedback relative to the player’s performance (see Fig. 2).

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Game scene.</figcaption></figure>
At the same time, as the users carry out the exercise, the range of sensors set up in the game environment (both artificial vision setting and biofeedback sensors) track the users' key kinematic and clinical parameters. These data, elaborated by the Data Analysis component, are used by the AI reasoner (alongside the game performance) to plan a new and subsequent task sequence to be performed by the user. In this way, the game is able to dynamically adapt both to the players' performance and clinical status, generating levels suitable for the correct users' rehabilitation and safety.

After the last task, users perform cool-down exercises, in order to allow a correct physical and energy recovery.

## 3.2 Game session at home

The exergame will be available also at home, where the patients will be able to execute rehabilitation sessions designed specifically for the domestic environment. This activity has been created for those patients that find it difficult to join the clinical activities, but may also be used as a strengthening activity between two clinical sessions. The exergame can be used under remote medical supervision

The game session may be realized in two different modes: single player (the player carries out a game session without other participants); 2) multi-player (the player carries out a game session with other users remotely connected). The latter approach may be used to encourage the players' interaction and engagement, as well as to stimulate the competition among the participants. As it will happen in the clinic, the game session will be recorded, in order to be shared among the users and provided to the therapist for further analysis.

## 3.3 Personalization logic

Each game session can be set up and personalized according to the type of users involved. A preliminary customization of the level may be carried out by the therapist, and concerns the setup of parameters such as music track, exercise duration and therapeutic goals. These settings, alongside with the user data, will lay the ground for the AI reasoner to plan a first sequence of tasks to provide to the users.

The sequence setup will be realized starting from a pool of approved steps, that have been identified and enumerated (i.e. Forward step, Backward step, Alternated steps forward, etc.). Each of these movements is self-consistent, i.e., it brings back the user to a rest position, and therefore is potentially linkable with other movements. Steps can involve both the use of a single limb (e.g. upper limb or lower limb) or both limbs.

In each level, the Virtual Dance Teacher provides to the player a sequence of steps. The player will have to perform the same movements as closely as possible.

The key parameters analyzed during the game session are listed below:

1. Game performance: step execution, rhythm (meant as step execution in the expected time frame in terms of bpm).

2. Kinematic parameters: gait analysis, centre of gravity position, upper and lower limb symmetry, degree of torso angle, cadence, etc.

3. Physiological parameters: heart rate, respiratory rate, blood pressure, etc.
## 3.4 A Middleware to Facilitate Module Interaction

During the game, the kinematic and clinical parameters of the patient are monitored by a middleware platform that reads the values transmitted by the environmental and wearable sensors, and processes the data through artificial intelligence algorithms, thus obtaining information on the state of health with a high degree of accuracy, supporting the physiotherapist in evaluating the patient's playing performance.

In more infrastructural terms, the platform can be defined as IoT Cloud and Edge combined with IoT Smart Gateway and IoT Device. Instead of sending information back to the cloud for analysis, IoT devices, using the Edge processing architecture, send information directly to IoT Smart Gateway to effectively reduce response time, while also saving cloud resources and bandwidth and providing better connection security.

In Let's dance scenario, the middleware platform allows the integration of the exergame with the following components: (1) *Environment Camera Gateway*: captures video frames from the cameras placed in the environment and sends them, via TCP protocol, to other middleware components for their processing. (2) *Wearable Sensor Gateway*: collects data from meters worn by patients and sends them to the module in charge of an initial processing through the TCP protocol. (3) *Event Bus Interoperability Framework*: through a system of connectors and topics and the adoption of the publish /subscribe architectural pattern, it manages the messages that circulate within the robotic platform, redistributing them to other components that express the need. (4) *Stream & Batch Processing*: allows to execute data analysis scripts/jobs in streaming or batch mode. The events produced can be lost-stored as data as well as sent as new events to the Event Bus module so that they can be consumed by other modules interested in them. (5) *Persistence Layer*: it allows to provide persistence to the circulating information: contextualized sensor data (internal and external to the robot), events produced by streaming and batch processing video.

# 4 Conclusion and future works

The present paper describes the first prototype of the exergame developed for the treatment of PD. A great effort has been spent to improve the appeal of the game for the target users. The first prototype will be tested in a trial in order to evaluate the usability. The project's ambition is to certify the game as a medical device.

## References

1. WHO page, https://www.who.int/health-topics/noncommunicable-diseases#tab=tab_1, last accessed 2021/04/23.

2. Petzinger, G. M., Fisher, B. E., McEwen, S., Beeler, J. A., Walsh, J. P., & Jakowec, M. W. (2013). Exercise-enhanced neuroplasticity targeting motor and cognitive circuitry in Parkinson's disease. The Lancet. Neurology, 12(7), 716-726. doi:10.1016/S1474-4422(13)70123-6