# A Pilot Study on Mixed-Reality Approaches for Detecting Upper-Limb Dysfunction in Multiple Sclerosis: Insights on Cerebellar Tremor

Miriam Moschetta, Etty Sabatino, Andrea Lucaroni, Giacinto Barresi, Carlo Ferraresi, Jessica Podda, Erica Grange, Giampaolo Brichetto, Anna Bucchieri

<figure><img src="image_1.png" /></figure>

## To cite this version:

Miriam Moschetta, Etty Sabatino, Andrea Lucaroni, Giacinto Barresi, Carlo Ferraresi, et al. A Pilot Study on Mixed-Reality Approaches for Detecting Upper-Limb Dysfunction in Multiple Sclerosis: Insights on Cerebellar Tremor. Virtual Worlds, 2025, 4 (1), pp.4. 10.3390/virtualworlds4010004. hal-05038188

HAL Id: hal-05038188

https://hal.science/hal-05038188v1

Submitted on 17 Apr 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
Article

# A Pilot Study on Mixed-Reality Approaches for Detecting Upper-Limb Dysfunction in Multiple Sclerosis: Insights on Cerebellar Tremor

Miriam Moschetta ¹,‡, Etty Sabatino ¹,‡, Andrea Lucaroni ²*‡, Giacinto Barresi ³,⁴, Carlo Ferraresi ¹, Jessica Podda ⁵, Erica Grange⁵, Giampaolo Brichetto⁵, and Anna Bucchieri ⁶

1. PDIMEAS, Politecnico di Torino, Turin, Italy;

2. Rehab Technologies Lab, Italian Institute of Technology (IIT), Genoa, Italy;

3. Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom;

4. Knowledge Media Institute, The Open University, Milton Keynes, United Kingdom;

5. Scientific Research Area, Italian Multiple Sclerosis Foundation (FISM), Genoa, Italy;

6. Inria, Université de Lorraine, CNRS, Loria, Nancy, France;

* Correspondence: andrea.lucaroni@iit.it

† These authors contributed equally to this work.

**Abstract:** The assessment and rehabilitation of upper limb functionality are crucial for addressing motor disorders in individuals with Multiple Sclerosis (PwMS). Traditional methods often lack the sensitivity to quantify subtle motor impairments, with cerebellar tremor diagnosis typically based on subjective visual inspections by clinicians. This study explored the feasibility of using Microsoft HoloLens2 for motion capture to assess upper-limb function in PwMS. Using the ROCKapp application, kinematic metrics such as movement quality and oculomotor coordination were recorded during pick-and-place tasks. Data from twelve healthy individuals served as benchmarks, while nine PwMS, including three with cerebellar tremor and one with ataxia, were tested to evaluate the tool's diagnostic potential. Clustering algorithms applied to the kinematic data classified participants into distinct groups, showing that PwMS without cerebellar symptoms sometimes displayed behavior similar to healthy controls. However, those with cerebellar conditions, like tremor and ataxia, were more easily differentiated. While the HoloLens2 shows promise in detecting motor impairments, further refinement is required to improve sensitivity for those without overt cerebellar symptoms. Despite these challenges, this approach offers potential for personalized rehabilitation, providing detailed feedback that could improve interventions and enhance quality of life for PwMS. In conclusion, these findings highlight the potential of mixed-reality tools to refine diagnostic accuracy, suggesting future studies to validate their integration in clinical rehabilitation programs.

**Keywords:** Multiple sclerosis, upper-limb, mixed-reality, rehabilitation

**Citation:** Moschetta M., Sabatino E., Lucaroni A.; Barresi, G.; Ferraresi C.; Podda J.; Grange E.; Brichetto G.; Bucchieri, A. Title. *Virtual Worlds* **2024**, *1*, 1–20. https://doi.org/

Received:

Revised:

Accepted:

Published:

**Copyright:** © 2025 by the authors. Submitted to *Virtual Worlds* for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## 1. Introduction

Multiple sclerosis (MS) is a chronic autoimmune disease primarily affecting the central nervous system, including the brain and spinal cord [1]. It is commonly diagnosed in young adults aged 20 to 50, affecting around 2.8 million people globally [2], including 130,000 in Italy [3–5]. Symptoms like optic neuritis, muscle weakness, coordination issues, and cognitive impairments significantly impact quality of life [6–8]. Severe fatigue and motor dysfunctions such as spasticity and sensory disturbances, including numbness and pain, can make daily tasks difficult [1,9,10]. Visual issues like double vision also affect hand-eye coordination, further complicating daily activities [11]. Overall, the disease impacts physical, cognitive, sensory, and emotional aspects of daily life, emphasizing the need for interventions to improve independence and quality of life [12,13].

PwMS may exhibit cerebellar symptoms, including tremor and ataxia, with intention tremor, truncal ataxia, and nystagmus often indicating cerebellar dysfunction [14–17]. Cerebellar tremor, characterized by involuntary limb oscillations, affects 25% to 58% of PwMS
and is distinct from essential and parkinsonian tremors [18–22]. A prevalent subtype is intentional tremor, worsening during targeted movements, linked to cerebellar lesions and impaired coordination [23–25]. Ataxia may manifest as dysmetria, dysdiadochokinesia, and nystagmus, often leading to an unstable gait [14,26]. Eye-hand coordination is particularly compromised, impacting motor tasks [27–30]. Cerebellar tremor and ataxia in PwMS highlight the need for comprehensive management approaches to address the significant disabilities they cause.

Impairments in upper limb function affect daily activities, with occupational therapy focusing on functional recovery. Customized rehabilitation regimens aimed at improving motor coordination and preserve functional autonomy in individuals with Multiple Sclerosis. Diagnosis of cerebellar conditions is currently operator-dependent as it is carried out through visual inspections [31–34].

In this context, cutting-edge technologies like motion capture systems and Virtual Reality platforms offer new ways to assess and rehabilitate motor abilities in objective ways. These tools offer immediate feedback and adaptive environments, potentially improving motor learning and compensatory techniques [35].

The integration of immersive technologies, such as Virtual Reality and Mixed Reality, enhances rehabilitation by creating stimulating environments, improving motivation and treatment outcomes [36–40]. Personalized feedback—visual, auditory, or tactile—is key to maintaining patient engagement, a critical factor in rehabilitation success [41–43]. More in details:

* Virtual Reality (VR) offers immersive, real-life scenarios for safe motor skills practice [44–46].

* Augmented Reality (AR) overlays digital information onto real-world scenes, providing real-time guidance during exercises [47,48].

* Mixed Reality (MR) combines AR with the physical world, allowing interaction with virtual objects for more engaging rehabilitation [49–52].

Another advantage of such technologies is the possibility to exploit cameras, sensors, and additional instruments for biometric measurements for the evaluation of both motor and hand-eye coordination abilities in people affected by a neurological disorder. Motion capture techniques have undergone substantial evolution, enabling the precise and detailed measurement of limb movements in various applications [53]. Recent advancements in AR and MR technologies have introduced head-mounted visors as a state-of-the-art tool for motion capture [54]. Devices like the Microsoft HoloLens2 seamlessly integrate MR capabilities with advanced motion tracking functionalities, presenting a range of significant advantages [55,56]:

* Markerless Tracking: Head-mounted visors utilize internal sensors and cameras to monitor upper limb movements without the requirement for reflective markers, enhancing user comfort and operational ease.

* Enhanced Interaction: MR environments overlay digital data onto the real-world environment, facilitating interactive and immersive rehabilitation exercises and training scenarios. This feature enhances patient engagement and allows for dynamic therapy sessions.

* Portability and Convenience: These devices are compact and simple to deploy, making them adaptable for diverse settings, including clinical facilities and patients' homes.

However, challenges like reduced tracking accuracy, limited field of view, and battery life need to be addressed to maximize their impact in rehabilitation. In [55] Hololens 2 and ROCKapp limitations are discussed extensively.

VR-based interventions show promising results specifically applied to PwMS rehabilitation, altering their sense of presence [40]. Saladino and colleagues [57] demonstrated the advantages in boosting daily functioning and satisfaction, with respect of traditional therapy procedures. The use of serious games and gamification in VR-based rehabilitation helps patients engage in repetitive training with greater motivation, improving clinical
adherence [58]. VR exergames are particularly effective in MS rehabilitation, especially for enhancing upper limb movements [59–61]. Despite XR technologies show less employment compared to VR solutions, they presented promising results in term of engagement and rehabilitation outcomes [62]. Even less studies in literature cover the usage of XR technologies as assessment and diagnostic tool for the quality of movement of upper-limb activity in PwMS.

Despite the growing body of research on rehabilitation methods for individuals with neurological disorders, current techniques often fail to provide precise feedback on the motor and coordination deficits experienced by patients. MS is particularly challenging due to its complex symptoms affecting motor skills and hand-eye coordination. Traditional methods for assessing these impairments can be labor-intensive and lack the nuanced data needed for highly personalized rehabilitation strategies. Thus, integrating advanced motion capture systems and immersive technologies like AR and MR into rehabilitation could fill this gap by offering detailed and adaptive assessments. These technologies can provide clinicians with objective metrics that are essential for tailoring more effective interventions.

The primary objective of this research was to enable clinicians to make a detailed categorization between healthy individuals, PwMS, those with cerebellar tremor and those with other motor impairments, considering the quality of upper limb movement. Thereby improving the understanding of upper-limb impairments and facilitating tailored interventions. Using the ROCKapp application for HoloLens2, previously introduced in [55], the study integrated holographic elements with physical objects to assess pick-and-place tasks.

Data from twelve healthy subjects was collected to calculate clinically relevant kinematic metrics—such as movement quality and coordination—which serve as benchmarks for assessing upper-limb impairments in individuals with MS. Subsequently, 9 PwMS (three with cerebellar tremor and one with ataxia) were recruited to evaluate the effectiveness of these metrics as both an assessment and diagnostic tool. A clustering algorithm was implemented to determine whether these metrics could accurately identify the severity of impairment among the participants, demonstrating the potential of this approach in clinical settings.

## 2. Materials and Methods

### 2.1. Participants

In this study a population of healthy subjects (control group) was recruited to perform kinematic acquisition of both hand and gaze displacements while performing a pick-and-place task with HoloLens2 (4 males, 8 females, average age of 42.15 ± 8.13). Nine PwMS were recruited including 4 males and 5 females, with an average age of 43.22 ± 13.71 years. PwMS group was characterised by different levels of disability both in the upper and lower limb. In particular, Three individuals had been diagnosed with cerebellar tremor while one with ataxia (Table 1).

<table><caption>**Table 1.** Demographic and clinical characteristics of the study participants. Data are expressed as study ID, Sex, Age, Side (Handedness), EDSS (Expanded Disability Status Scale), 9-HPT (9 Hole Peg Test), BBT (Box and Block Test) and clinic cerebellar condition, if present.</caption><thead><tr><th>ID</th><th>SEX</th><th>AGE</th><th>SIDE</th><th>EDSS</th><th>9-HPT</th><th>BBT</th><th>CEREBELLAR CONDITION</th></tr></thead><tbody><tr><td>S1</td><td>M</td><td>29</td><td>R</td><td>1</td><td>20.43</td><td>66</td><td>N/A</td></tr><tr><td>S2</td><td>F</td><td>28</td><td>R</td><td>1.5</td><td>15.53</td><td>71</td><td>N/A</td></tr><tr><td>S3</td><td>F</td><td>49</td><td>R</td><td>4.5</td><td>42.28</td><td>31</td><td>Cerebellar tremor</td></tr><tr><td>S4</td><td>M</td><td>49</td><td>R</td><td>2</td><td>25.30</td><td>49</td><td>N/A</td></tr><tr><td>S5</td><td>F</td><td>38</td><td>L</td><td>6</td><td>24.70</td><td>42</td><td>Ataxia</td></tr><tr><td>S6</td><td>M</td><td>34</td><td>R</td><td>3.5</td><td>35.50</td><td>52</td><td>N/A</td></tr><tr><td>S7</td><td>M</td><td>35</td><td>R</td><td>1</td><td>17.60</td><td>59</td><td>N/A</td></tr><tr><td>S8</td><td>F</td><td>64</td><td>L</td><td>6.5</td><td>N/A</td><td>12</td><td>Cerebellar tremor (severe)</td></tr><tr><td>S9</td><td>F</td><td>63</td><td>L</td><td>6.5</td><td>66.34</td><td>33</td><td>Cerebellar tremor</td></tr></tbody></table>
All subjects were recruited according to Declaration of Helsinki and under ENACT01 protocol (229/2022) approved by the Ethical Committee of Liguria Region (Italy) on November 14th, 2022. Participants signed a written informed consent after being introduced to the objectives of the study.

## 2.2. ROCKapp and experimental set-up

In this study ROCKapp [55] – an app developed for Microsoft HoloLens2 – was employed to provide a novel approach to functional MS assessment. This application integrates holographic elements with physical objects and markers to create interactive environments where PwMS can perform specific upper limb tasks, such as pick-and-place exercises. The Mixed-Reality environment was constructed using Unity 2021.2.16f1, with integration of both Mixed Reality Toolkit 3 (MRTK3) and PTC Vuforia extensions. The MRTK package offers essential components for spatial interactions and user interface elements. This framework equips developers with the necessary APIs to utilize the user's hands as interactive tools. It can accurately compute the position and orientation of each hand joint, including the fingers (with each phalanx), knuckles, palm, and wrist. PTC Vuforia, an Augmented Reality software, enabled holographic interaction by exploiting image-based recognition algorithms. Upon detection of specific image targets by the HoloLens2 camera, the program would anchor holograms to these targets, enhancing the user experience.

For ROCKapp, a cylindrical image target representing a rocket was placed on top of a 500 ml plastic bottle (Figure 1). When the target was recognized by HoloLens2 cameras, the user could visualize the digital version of the same image on top of the physical marker. The bottle was chosen as transparent for allowing the optical hand tracking system to see the hand even when the finger joints were occluded by the object during the grabbing phase.

Each participant was expected to move the object based on four different cues positioned as cardinal points: in front close (S), in front distant (N), on the left (W), and on the right (E). Virtual target N was placed in front of the user at a distance of maximum arm extension. S, W, and E coincided with vertices of a square of side 28 cm, and were automatically generated according to the chosen North coordinates. Position N served as a reference upon which the bottle had be placed back once moved to either S, W, or E. Please refer to Table 2 for the nomenclature adopted in this manuscript.

<table><caption>**Table 2.** Nomenclature of discrete movements to be performed within the game</caption><thead><tr><td>Movement</td><td>Label</td></tr></thead><tbody><tr><td>North-South</td><td>NS</td></tr><tr><td>South-North</td><td>SN</td></tr><tr><td>North-East</td><td>NE</td></tr><tr><td>East-North</td><td>EN</td></tr><tr><td>North-West</td><td>NW</td></tr><tr><td>West-North</td><td>WN</td></tr></tbody></table>

The order of appearance of cues S, E, W was randomized (Figure 1). Placing the object on an activated target triggered the launch of the holographic rocket. This was possible employing Unity Engine's colliders. For additional details please refer to the work of Bucchieri and colleagues [55]. The ROCKapp task was conducted in a seated position to accommodate individuals with significant lower limb impairments. Participants were instructed to repeat the task 30 times, distributed across 5 trials with 6 movements each. During runtime, hand and eye tracking data were recorded through HoloLens2 at a frequency of 50 Hz.
<figure><img src="image_3.png" /><figcaption>**Figure 1.** (Left) Schematic of ROCKapp, (Right) First and third-view of a person performing ROCKapp.</figcaption></figure>

## 2.3. *Experimental protocol*

Each participant of both control and PwMS group signed an informed consent before proceeding with the experimentation. PwMS previously underwent clinical evaluation to assess their level of motor and cognitive disability. The experimenter then started the application installed on HoloLens2 and, by using a custom menu, selected the side (right or left) the participant would carry out the pick and place tasks. The environment was manually placed by the experimenter in front of the participant so that target N would result at the subject's arm length. Participants then donned the HoloLens2 device and performed ocular calibration. Ocular calibration is a built-in feature of the Microsoft HoloLens 2 and was conducted only once for each participant. This step was essential to enhance both the accuracy and reliability of clue placement and the assessment of eye-hand coordination.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Experimental protocol</figcaption></figure>

Upon starting ROCKapp, participants were asked to execute movements in a natural manner to simulate an activity of daily living. Moreover, they were required to keep their hand on the table, grasp the bottle, place it on target (N, S, E, or W according to the movement to be performed), and place the hand back on the table. Participants were given the option to take breaks between repetitions to prevent the introduction of bias in movement execution caused by fatigue.

Following the completion of the session, participants were requested to complete the NASA Task Load Index (NASA-TLX) [63], a widely used subjective questionnaire that measures perceived workload across six dimensions—mental demand, physical demand, temporal demand, performance, effort, and frustration—during task performance (Figure 2).
Each experimental session lasted approximately 30 minutes, with a minimum 1-hour interval between subjects for recharging HoloLens2 battery.

## 2.4. Data processing

Hand and eye-tracking data were processed using Matlab 2021b. Kinematic data from the HoloLens2, recorded relative to a world reference frame set up between the subject's eyes, were initially not directly comparable across subjects. To standardize the data, the following steps were taken:

1. **Coordinate System Adjustment:** Data were first roto-translated from the HoloLens2's left-handed coordinate system to a standard right-handed system.

2. **Centroid Calculation:** The centroids of Target Areas N and S were computed from the Hand dataset by averaging x and y coordinates.

3. **Reference System Alignment:** The centroid of Target Area N was translated to the origin (0,0), and all other points were adjusted accordingly.

4. **Angle Correction:** The inclination angle of the line between centroids N and S relative to the y-axis was calculated, and a rigid rotation around the z-axis was applied to correct this inclination.

Absolute positions and velocities of the hand were computed for each participant. Before derivation, positions were filtered with a fourth order low-pass Butterworth filter at cut-off frequency of 6-Hz, according to other upper-limb kinematics analysis found in literature. [64,65].

Outlines were exploited to isolate the pick-and-place movements (NS, SN, NE, EN, NW, WN) from the reach-to-grasp and back-to-resting position phases. After a rough manual cut of the outlines, the starting and ending point of the interested movements were defined by setting a threshold of 5% of the maximum velocity to the velocity profile.

Some trajectories experienced data loss while recording with HoloLens2. For each subject it was calculated the percentage of loss of data-points as

$$ \text{Data loss\%} = \frac{\text{Number of data points lost}}{\text{Total number of data points}} \cdot \% $$

As Subject 2 (PwMS) experienced a percentage exceeding 80%, the participant was excluded from further analyses. For the same reason, Subject 5 (control group) was excluded.

Moreover, signals containing more than 25% undetected points by HoloLens2 were excluded to ensure the dataset accurately represented the performed movements, without significant distortions or artifacts introduced by interpolation. Not considering S2 (PwMS) and S14 (control group), 3.77% of the available dataset was discarded. Hand positions presenting lower than 25% of data loss were interpolated with a 5th order polynomial following minimum jerk theory [54].

## 2.5. Kinematic metrics

Five kinematic metrics red and one proprioceptive were calculated to quantify the quality and efficiency of movement execution, providing valuable insights into the motor performance of PwMS during rehabilitation exercises. Specifically, metrics related to hand and eye displacements and velocities, focusing on smoothness, efficiency, planning, and hand-eye coordination. The kinematic metrics were carefully selected from a previous study in which their clinical relevance was investigated [64].

### 2.5.1. Smoothness

* **Spatial Arc Length (SPARC):** SPARC is defined as the Arch Length of the frequency spectrum derived from the Fourier Transform of the velocity profile [65–70].

* **Number of Velocity Peaks (NVP):** NVP denotes the number of submovements required to complete an action. Hand motion patterns showing multiple peaks in the velocity curve signify impaired smoothness, while a bell-shaped velocity profile is characteristic of normal, healthy movement [68–74].
### 2.5.2. Efficiency

* **Movement Time (MT):** MT is defined as the duration of the movement from the moment the object is picked to when it is placed on the target [69,70,74–76].

### 2.5.3. Planning

* **Symmetry:** Symmetry is defined as the ratio between the duration of the acceleration phase and the duration of the deceleration phase while performing a kinematic movement [77]. Clinically, high Symmetry in movements indicates effective motor control and suggests proper functioning of the patient's neuromuscular system [78]. In contrast, movement asymmetry can signify motor impairments or compensatory strategies, potentially necessitating further therapeutic intervention.

* **Kurtosis:** Kurtosis offers insights into the distribution of velocity throughout a movement [77]. High Kurtosis indicates that the movement features more frequent extreme values (peaks), while low Kurtosis suggests a more consistent distribution of velocities. Clinically, Kurtosis can aid in evaluating the smoothness and control of a patient's movements. Higher Kurtosis values may point to abrupt or jerky motions, which are often associated with motor control issues or neurological disorders. Conversely, lower Kurtosis values suggest smoother and more controlled movements.

### 2.5.4. Hand-Eye Coordination

* **Gaze Accuracy–Number of Zero Crossing Points (N0C):** This a novel metric introduced in this manuscript to evaluate the hand-eye coordination ability of a person. N0C, calculated as the distance between the eye path and the hand path instantaneously, refers to the measurement of how closely the gaze trajectory follows the hand trajectory at each moment in time. Particularly, the number of zero crossing points of the first derivative of the distance between the eye path and the hand path (N0C) was calculated. This approach provides a quantitative measure of how often the gaze trajectory intersects or deviates from the hand trajectory over time. By analyzing the derivative's zero crossings, we can capture changes in gaze accuracy dynamically throughout the task or experimental session. Ideally, a low number of N0C would indicate healthy hand-eye coordination, suggesting that the person can smoothly execute the movement without frequent shifts in gaze. Conversely, a high number may suggest uncertainty in movement execution, with the person needing to frequently shift their gaze between the hand and the task. Since this metric is task-dependent, the baseline was first computed using a control group and then compared to the recruited PwMS.

## 2.6. *Metrics-based clustering: PwMS and control group*

In this study, the K-means clustering algorithm was used to evaluate whether the proposed kinematic metrics could be utilized to classify PwMS according to the severity of their impairment. The K-means clustering algorithm is a widely utilized tool for classification tasks in research focused on motor-related studies. Its simplicity, efficiency, and ability to group data based on similarity make it a preferred choice for researchers aiming to identify distinct patterns or categories within complex datasets [79].

The analysis involved calculating indicators for both PwMS and a control group. Two different classification approaches were applied based on statistical analysis of the dataset and based on clinician directives:

* **Four-Class Classification Based on Clinical Evaluation:** This approach identified four clusters corresponding to: the control group, PwMS, PwMS with Intention Tremor, and PwMS with Ataxia.

* **Six-Class Classification Based on the Silhouette Method:** This method tested on the complete dataset prior to classification: This method determined the optimal number of clusters to be six.
Kinematic metrics were pooled together regardless of movement direction.

To ensure consistency across the dataset, each metric was normalized relative to the average metric calculated from the control group across all movements. This normalization step was essential for standardizing the dataset.

## 2.7. Statistical analysis

A statistical analysis was performed to evaluate differences in kinematic behavior between the control group and PwMS without cerebellar symptoms (S1, S4, S6, S7, as shown in Table 1). The hypothesis tested was that PwMS and the control group exhibit statistically significant differences in kinematic behavior, specifically in terms of smoothness, efficiency, planning, and hand-eye coordination.

To test this hypothesis, the non-parametric Friedman test was employed with a significance level set at 0.05. If the test indicated significant differences, pairwise comparisons were conducted using the Mann-Whitney test to contrast the behavior of each PwMS subject with that of the control group. For these comparisons, the significance level was adjusted using the Bonferroni correction ($\alpha = 0.05/4 = 0.0125$) [80].

An additional statistical test was performed to evaluate, if present, differences in perceived workload between PwMS and control group, based on the results of NASA-TLX questionnaire. As the requested task was a pick-and-place one with limited cognitive and physical effort required, the hypothesis tested was that PwMS and control group do not exhibit statistically significant differences in terms of mental demand, physical demand, temporal demand, performance, effort, and frustration. Pair-wise Mann-Whitney test was employed to test such hypothesis, with a significance level $\alpha$ of 0.05.

# 3. Results

## 3.1. Clustering at 4 classes based on clinical evaluation

<figure><img src="image_5.png" /><figcaption>**Figure 3.** K-means clustering for k number of clusters=4, based on clinical evaluation. The subject's bar color follows the clustering result, and therefore the bar height level. S2 subject was omitted from the classification because of the elevated percentage of data loss during the acquisition.</figcaption></figure>

The classification for the subjects conditions' established by the testS realized by clinicians consisted of 4 classes:

1. PwMS affected by Ataxia

2. PwMS affected by cerebellar tremor

3. PwMS with no cerebellar tremor

4. Control group

The results of our analysis, based on a preliminary sample of subjects, showed that PwMS subjects with no cerebellar conditions (S1, S4, S6, S7) were grouped together with the control group (green bars in Figure 3).
Subjects S3, S8, and S9, all affected by intention tremor, were expected to cluster together. This was true for S3 and S9, while S8 was classified separately (red bar). Finally, S5, who was affected by ataxia, was correctly classified in its own group.

Since some PwMS subjects were grouped with the control group, a statistical analysis was conducted to assess the differences between these individuals (S1, S4, S6, S7) and the healthy group across various kinematic metrics. The Friedman test demonstrated statistically significant differences among all groups across each indicator, with p-values less than 0.001 for SPARC, NVP, MT, Kurtosis, and N0C, and a p-value of 0.011 for Symmetry (Table 3).

These results suggest that the hypothesis that PwMS and the control group exhibit different kinematic behaviors in terms of movement smoothness, planning, and efficiency cannot be rejected.

A pairwise Mann-Whitney test (significance level 0.0125) further highlighted significant differences between the control group and each PwMS subject (Table 3, Figure 4):

* **S1 vs. Control:** Significant differences in 4 out of 6 metrics (NVP, MT, Symmetry, and N0C).

* **S4 vs. Control:** Significant in 2 out of 6 metrics (MT and Kurtosis).

* **S6 vs. Control:** Significant in 1 out of 6 metrics (SPARC).

* **S7 vs. Control:** Significant in 3 out of 6 metrics (NVP, MT, and N0C).

## 3.2. Clustering at 6 classes based on Silhouette method

The K-means algorithm, applied to six classes, revealed that S4 and S6 were grouped with the control group, while S1 and S7 formed their own cluster. S5 and S8 remained in individual classifications, while S3 and S9 were assigned to separate clusters, indicating distinct kinematic behaviors between the two subjects affected by intention tremor. (Figure 5).

<table><caption>**Table 3.** Non-parametric Friedman tests (significance level 0.05) and pair-wise Mann-Whitney tests (significance level 0.05/4 = 0.0125) performed only between control group and each PwMS classified as control group from the K-means (k = 4). \* indicates statistical significance.</caption><thead><tr><th>Metric</th><th>Friedman test</th><th>S1</th><th>S4</th><th>S6</th><th>S7</th></tr></thead><tbody><tr><td>SPARC</td><td>&lt; 0.001*</td><td>0.03</td><td>0.63</td><td><strong>0.002*</strong></td><td>0.06</td></tr><tr><td>NVP</td><td>&lt; 0.001*</td><td>&lt; <strong>0.001*</strong></td><td>0.057</td><td>0.91</td><td>&lt; <strong>0.001*</strong></td></tr><tr><td>MT</td><td>&lt; 0.001*</td><td>&lt; <strong>0.001*</strong></td><td>&lt; <strong>0.001*</strong></td><td>0.62</td><td>&lt; <strong>0.001*</strong></td></tr><tr><td>SYMMETRY</td><td>0.011*</td><td><strong>0.008*</strong></td><td>0.11</td><td>0.318</td><td>0.034</td></tr><tr><td>KURTOSIS</td><td>&lt; 0.001*</td><td>0.044</td><td>&lt; <strong>0.001*</strong></td><td>0.33</td><td>0.06</td></tr><tr><td>N0C</td><td>&lt; 0.001*</td><td><strong>0.002*</strong></td><td>0.40</td><td>0.13</td><td>&lt; <strong>0.001*</strong></td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>**Figure 4.** Boxcharts of patient's distributions classified with healthy subjects for clusters number k=4, and the control group distributions. Each graph represents one of the metrics considered for the classifier (SPARC, N0C, MT, SIMMETRY, KURTOSIS, NVP). Statistical significance is reported with an horizontal line and \* symbol above.</figcaption></figure>

### 3.3. *Movement-based kinematic evaluation of PwMS*

343

<figure><img src="image_7.png" /><figcaption>**Figure 6.** Dotplot of SPARC metric: Blue dots represent the mean value across repetitions for each movement in healthy subjects, while red dots represent the mean value across repetitions for each movement in a pathological subject.</figcaption></figure>

In the previous section, metrics for each movement were grouped without distinguishing between trajectory directions. However, it may be useful to differentiate the kinematic motor abilities of PwMS.

344

345

346
<figure><img src="image_8.png" /><figcaption>**Figure 5.** K-means clustering for k number of clusters=6. The subject's bar color follow the clustering result, and therefore the bar height level. S2 subject was omitted from the classification because of the elevated percentage of data loss during the acquisition</figcaption></figure>

For example, Figure 6 shows the average Spectral Arc length metric (SPARC) for each PwMS (in red), superimposed on the average for the control group (in blue). The x-axis shows different movement directions (NS, SN, NE, EN, NW, WN), while the y-axis shows the SPARC values. Similar pattern observed for SPARC was also present for the other kinematic metrics, which were omitted for brevity.

It is observed that for S1, S4, S6, and S7, who do not have tremor, SPARC values are generally comparable to the control group for all movements. In contrast, S5, affected by ataxia, exhibits patterns similar to PwMS with intention tremor (S3, S9), who show smooth movements in one direction (WN) but impairments in others. S8, with severe intention tremor, demonstrates significantly abnormal SPARC values around -4, indicating pronounced motor difficulties.

**Figure 7** shows the value of Number of Zero Crossings (N0C), a novel metric that highlights differences in hand-eye coordination between PwMS and control groups. While controls show low, stable N0C values across movement directions, PwMS exhibit significantly higher values, indicating greater coordination deviations, especially in participants like S5 and S8. S8 shows extreme difficulty, with N0C values exceeding 60. PwMS without cerebellar issues, such as S4 and S6, align more closely with the control group, unlike S1 and S7, who show greater deviation. This metric provides clinicians with valuable insights to target specific movements and coordination challenges in therapy.
<figure><img src="image_9.png" /><figcaption>**Figure 7.** Dotplot of N0C metric: Blue dots represent the mean value across repetitions for each movement in healthy subjects, while red dots represent the mean value across repetitions for each movement in a pathological subject.</figcaption></figure>

### 3.4. NASA-TLX questionnaire

Figure 8 presents boxplots comparing subjective workload scores between PwMS and the control group across six dimensions of the NASA Task Load Index (NASA-TLX) questionnaire: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration.

In terms of *mental demand*, PwMS show a slightly higher median compared to the control group, with both distributions being similar (p = 0.55). For *physical demand*, both groups exhibit low scores, though the control group shows a slightly higher median, with an outlier present in the PwMS group (p = 0.69). A significant difference is observed in *temporal demand*, where the Mann-Whitney test resulted in p = 0.0117 between the two groups. In *performance*, PwMS report lower scores compared to the control group, although their distributions overlap (p = 0.079). PwMS report expending more *effort* than the control group, as reflected by a wider distribution and higher median. However the two distributions do not present statistically significant differences (p = 0.63). Lastly, *frustration* levels are similar between the two groups, with PwMS showing a marginally higher median and fewer outliers in comparison to the control group (p = 0.81).

Considering these findings, the hypothesis that PwMS and control group show similar workload demand for ROCKapp is not rejected for all categories but *temporal demand*.

### 4. Discussion and conclusions

This study introduces an innovative use of the Microsoft HoloLens2 device and the immersive application ROCKApp for motion capture of upper limb kinematics in people with Multiple Sclerosis (PwMS). Offering this way an objective and data-driven method for diagnosing motor impairments. This contrasts with current clinical approaches, which rely heavily on subjective assessments by clinicians [31]. The primary objective of this study was to evaluate the feasibility of using the Microsoft HoloLens2 for calculating kinematic features that support the assessment and diagnosis of PwMS. Considering the varied severity of the disease among the PwMS subjects recruited (Table 1), the K-means algorithm was initially tested with four classes, aiming to differentiate between PwMS, healthy individuals (control group), PwMS with intention tremor, and PwMS with ataxia. The clustering algorithm was applied based on six performance metrics related to movement smoothness (Spectral Arc Length, Number of Velocity Peaks), motor planning (Kurtosis,
<figure><img src="image_10.png" /><figcaption>**Figure 8.** Boxchart representing distribution of PwMS and healthy subjects control group responses to each NASA-TLX questionnaire metric. Significance between distributions are shown with horizontal line and \* symbol above.</figcaption></figure>

Symmetry), efficiency (Movement Time), and oculomotor coordination (Number of Zero Crossings). However, the clustering results revealed that PwMS subjects S1, S4, S6, and S7 were grouped with the control group, contradicting the expectation that PwMS without cerebellar conditions would display kinematic behaviors distinct from healthy individuals (Figure 3). The Friedman test, applied to S1, S4, S6, S7, and the control group, indicated significant differences across multiple metrics ($p < 0.001$ for all but Symmetry, where $p = 0.011$, Table 3), suggesting that some aspects of motor performance differ markedly between PwMS and healthy controls. Pairwise comparisons revealed that only in certain instances did subjects show significantly different kinematic behaviors compared to the control group ($p < 0.0125$), with S1 and S7 exhibiting at least three statistically significant differences out of six metrics. These findings suggest that the HoloLens2, in combination with ROCKapp specifically, may not be suitable for diagnosing PwMS without cerebellar conditions from healthy individuals. PwMS with no cerebellar impairments may retain certain motor functions that are not detected by this tool. Clinical tests such as the box-and-block test (BBT) [81] and the 9-hole peg test (9-HPT) [82] may be more appropriate for identifying subtle motor impairments, likely due to the temporal constraints inherent in these tasks, which make kinematic deficiencies more evident. About PwMS affected by cerebellar conditions, S3 and S9 – both affected by intention tremor – were clustered together, affirming the sensitivity of the kinematic metrics in identifying motor deficiencies in individuals with this condition. S8, who was expected to cluster with the other tremor-affected subjects, was classified separately, which may point to the unique severity or manifestation of the tremor in this individual. The correct classification of S5 (ataxia) highlights the model's ability to differentiate the nature of motor impairments.

In Figures 6 and 7 we presented a clear depiction of the average behavior of each PwMS (in red) compared to the control group (in blue) across various movement sets (NS, SN, NE, EN, NW, WN). This graphical representation allows clinicians to easily identify:

1. The overall differences in motor abilities compared to the control group.

2. The specific movements for which the subject may require focused rehabilitation.

For example, S1 and S7 showed deviations in N0C compared to healthy participants, while S4 and S6 exhibited behavior more similar to the control group, with fewer gaze-hand coordination deviations.
The silhouette method identified six optimal clusters, with K-means clustering (k = 6) distinguishing S1 and S7 from S4 and S6, which resembled the control group. S5 and S8 formed separate clusters, while S3 and S9, both affected by intention tremor, were also classified separately. Despite similar Box-and-Block Test (BBT) scores, S3 and S9 differed significantly on the 9-Hole Peg Test (9-HPT), reflecting variations in dexterity and hand-eye coordination. Additionally, S3 exhibited cognitive impairments not present in S9.

In conclusion, this pilot study explored the potential of clustering algorithms to assess motor performance in PwMS and highlighted the utility of advanced kinematic metrics and mixed-reality assessments. The comparison between clustering at four and six classes underscores the importance of refining analytical approaches for evaluating motor impairments. Specifically, the initial K-means clustering with four classes demonstrated promise as a diagnostic tool for cerebellar conditions in PwMS by successfully differentiating individuals with intention tremor from those with ataxia and severe tremor. However, it failed to distinguish PwMS without cerebellar conditions from healthy individuals, as evidenced by the grouping of S1, S4, S6, and S7 with the control group.

The application of the silhouette method revealed that a six-class model more effectively captured the nuances in kinematic behaviors, providing a more accurate representation of motor skill variability among PwMS. This refinement was particularly notable in distinguishing S1 and S7 from S4 and S6, as well as S3 and S9, who, despite similar Box-and-Block Test (BBT) scores, exhibited significant differences in fine motor skills as measured by the 9-Hole Peg Test (9-HPT). These findings align with prior studies — such as those by Carpinella et al. [83] — which utilized inertial sensor signals to classify motor impairments and successfully differentiate PwMS with intention tremor from healthy individuals during tasks like finger-to-nose.

Nevertheless, the pilot nature of this study must be emphasized. While the findings suggest that advanced clustering techniques can enhance diagnostic capabilities, the results remain preliminary and require validation in larger cohorts. For example, S5, affected by ataxia, was successfully classified into a distinct group, indicating the tool's potential to differentiate ataxia from intention tremor. However, this distinction remains speculative without further investigation and a larger sample size to confirm its diagnostic utility. Another critical consideration is the inherent challenge in distinguishing between overlapping motor conditions such as ataxia and intention tremor, which can lead to potential misdiagnoses [21,84]. Addressing this challenge would benefit from integrating evaluations based on activities of daily living (ADLs), offering a more comprehensive understanding of motor performance in real-world settings. Traditional clinical tests like the 9-HPT and BBT provide valuable insights, but advanced kinematic metrics add depth to the assessment of motor function, particularly for PwMS with cerebellar impairments. From a cognitive perspective, the use of ROCKapp and a mixed reality solution was well-received by both PwMS and the control group, with low perceived workload overall.

Interestingly, pairwise comparisons of workload metrics revealed no statistically significant differences, except for temporal demand, where the control group reported higher perceived pressure. This unexpected result suggests that the control group may have experienced a greater sense of urgency to complete tasks quickly, possibly due to competitiveness feelings, potentially influencing their perceived workload. In contrast, individuals with PwMS might have approached the test with a clinical mindset, prioritizing careful task execution over speed and exerting extra effort to perform the tasks thoroughly. Nevertheless, PwMS reported low cognitive demand, reinforcing the feasibility of employing HoloLens2 in rehabilitative settings.

Overall, MR, compared to VR, has proven to be beneficial in the rehabilitation setting as it provides more effective hand-eye coordination experience compared to typical applications on fully immersive or two-dimensional displays [85–88]. While rehabilitative applications of Microsoft HoloLens have been explored for recording kinematic data in upper-limb assessments (e.g., kinematic hand tracking and hand-eye coordination) [89,90],
these studies have not thoroughly investigated the potential of this technology for both the assessment and diagnosis of individuals with neurological disorders.

In summary, this exploratory study provides preliminary evidence supporting the utility of mixed-reality based assessments and advanced clustering techniques for monitoring PwMS with diverse motor impairments. While the results are promising, further research with larger, more diverse cohorts is essential to validate these findings and to fully realize the potential of these tools in improving diagnosis, monitoring, and rehabilitation for PwMS.

This study highlights the potential of using Microsoft HoloLens2 and the ROCKapp application to assess upper-limb functionality in PwMS, particularly those with cerebellar conditions like tremor and ataxia. Kinematic metrics related to movement quality, coordination, and accuracy showed promise for identifying motor impairments in PwMS, although further refinement is necessary to improve sensitivity for those without overt cerebellar symptoms. The ability to differentiate between PwMS with cerebellar impairments and healthy controls supports the potential of this approach as a valuable tool for both assessment and diagnosis.

Despite its potential, the current study has limitations. The small sample size does not fully represent the variability in motor impairments experienced by PwMS. Future work should involve testing a larger population of both PwMS and healthy individuals to validate the generalizability of the results. The recruited sample size should encompass a diverse population, considering both cultural and environmental factors, to ensure representation of a broadly applicable, general population. Additionally, developing an automatic algorithm to identify pick-and-place tasks from other movement phases will streamline data analysis and improve real-time diagnostics. This could be useful to create a desktop application that allows clinicians to directly visualize kinematic data from the HoloLens2, enhancing the practical utility of this tool in clinical settings.

By offering a low-workload, objective assessment tool, the proposed framework holds great promise for real-time clinical use, potentially revolutionizing how motor impairments are diagnosed and treated in PwMS.

Future developments should prioritize advancing MR-based applications to explore techniques for reducing intention tremor in affected individuals, with the proposed framework serving as a valuable tool for evaluating the effectiveness of these therapies. It would be valuable to explore additional tasks related to ADL (e.g. eating, cooking, self-hygiene) using our classification approach, to further evaluate their respective effectiveness in highlighting differences in impairment levels. Such advancements have the potential to greatly enhance the quality of care and independence for people with Multiple Sclerosis.

## Author Contribution

Conceptualization, Barresi G., Lucaroni A. and Bucchieri A.; methodology, Barresi G., Lucaroni A. and Bucchieri A.; software, Lucaroni A.; validation, Lucaroni A. and Bucchieri A., Moschetta M. and Sabatino E.; formal analysis, Lucaroni A. and Bucchieri A.; investigation, Giacinto B.; resources, Grange E. and Podda J.; data curation, Moschetta M. and Sabatino E.; writing—original Bucchieri A. and Lucaroni A.; writing—review and editing, Grange E., Podda J., Moschetta M., Sabatino E. Ferraresi C., Barresi G., Brichetto G.; visualization, Lucaroni A. and Bucchieri A., Moschetta M. and Sabatino E., Grange E. and Podda J.; supervision, Barresi G.; project administration, Barresi G. and Brichetto G.; funding acquisition, Barresi G. and Brichetto G.. All authors have read and agreed to the published version of the manuscript.

## Funding

This study is supported by ENACT Project, a FISM-IIT project (https://www.enactproject.eu) in synergy with RAISE - Robotics and AI for Socio-economic Empowerment (https://www.raiseliguria.it/). ENACT is supported by FISM - Fondazione Italiana Sclerosi Multipla - cod. 2021/Special/003 and financed or co-financed with the '5 per mille' public
funding. RAISE is funded by the European Union - NextGenerationEU. However, the views and opinions expressed are those of the authors alone and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the European Commission can be held responsible for them.

**Institutional Review Board Statement**

Nine subjects affected by Multiple Sclerosis and twelve healthy subjects were recruited according to Declaration of Helsinki and under ENACT01 protocol (229/2022) approved by the Ethical Committee of Liguria Region (Italy) on November 14th, 2022.

**Informed Consent Statement**

Written informed consent was obtained from all subjects involved in the study after being introduced to the objectives of the research.

**Data Availability Statement**

Experimental data are not published along with this study because are intended to be used in a following publication.

**Conflicts of Interest**

The authors declare no conflict of interest.

**References**

1. Goldenberg, Marvin M. Multiple sclerosis review. *Pharmacy and Therapeutics*, vol. 37, no. 3, pp. 175, 2012.

2. Walton, C., King, R., Rechtman, L., Kaye, W., Leray, E., Marrie, R., Robertson, N., La Rocca, N., Uitdehaag, B., Der Mei, I. & Others Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS. *Multiple Sclerosis Journal*. **26**, 1816-1821 (2020)

3. Lane, J., Ng, H., Poyser, C., Lucas, R. & Tremlett, H. Multiple sclerosis incidence: a systematic review of change over time by geographical region. *Multiple Sclerosis And Related Disorders*. **63** pp. 1039-1052 (2022)

4. Sospedra, Mireia, and Roland Martin. Immunology of multiple sclerosis. *Annual Review of Immunology*, vol. 23, no. 1, pp. 683-747, 2005.

5. Mosconi, Paola, Tommaso Guerra, Pasquale Paletta, Antonio D'Ettorre, Michela Ponzio, Mario Alberto Battaglia, Maria Pia Amato, Roberto Bergamaschi, Marco Capobianco, Giancarlo Comi, et al. Data monitoring roadmap. The experience of the Italian Multiple Sclerosis and Related Disorders Register. *Neurological Sciences*, vol. 44, no. 11, pp. 4001-4011, 2023.

6. Dobson, R. & Giovannoni, G. Multiple sclerosis – a review. *European Journal Of Neurology*. **26**, 27-40 (2018)

7. Chiaravalloti, Nancy D., and John DeLuca. Cognitive impairment in multiple sclerosis. *The Lancet Neurology*, vol. 7, no. 12, pp. 1139-1151, 2008.

8. Sparaco, M., Lavorgna, L. & Bonavita, S. Psychiatric disorders in multiple sclerosis. *Journal Of Neurology*. **268**, 45-60 (2021).

9. Krupp, L., Alvarez, L., LaRocca, N. & Scheinberg, L. Fatigue in multiple sclerosis. *Archives Of Neurology*. **45**, 435-437 (1988)

10. Induruwa, I., Constantinescu, C. S. & Gran, B. Fatigue in multiple sclerosis—a brief review. *Journal Of The Neurological Sciences*. **323**(1-2), 9-15 (2012).

11. Helsen, W., Feys, P., Heremans, E. & Lavrysen, A. Eye-hand coordination in goal-directed action: normal and pathological functioning. (Human Kinetics; Champaign,2010)

12. Berrigan, L. I., et al. Health-related quality of life in multiple sclerosis: direct and indirect effects of comorbidity. *Neurology*. **86**(15), 1417-1424 (2016).

13. Zwibel, Howard L. Contribution of impaired mobility and general symptoms to the burden of multiple sclerosis. *Advances in Therapy*, vol. 26, pp. 1043-1057, 2009.

14. Klockgether, T. Sporadic ataxia with adult onset: classification and diagnostic criteria. *The Lancet Neurology*. **9**(1), 94-104 (2010)

15. Tornes, L., Conway, B. & Sheremata, W. Multiple sclerosis and the cerebellum. *Neurologic Clinics*. **32**, 957-977 (2014)
16. Koch, Marcus, Jop Mostert, Dorothea Heersema, and Jacques De Keyser. Tremor in multiple sclerosis. *Journal of Neurology*, vol. 254, pp. 133–145, 2007.

584

585

17. Weier, Katrin, Brenda Banwell, Antonio Cerasa, D. Louis Collins, Anne-Marie Dogonowski, Hans Lassmann, Aldo Quattrone, Mohammad A. Sahraian, Hartwig R. Siebner, and Till Sprenger. The role of the cerebellum in multiple sclerosis. *The Cerebellum*, vol. 14, pp. 364–374, 2015.

586

587

588

589

18. Hess, C. & Pullman, S. Tremor: clinical phenomenology and assessment techniques. *Tremor And Other Hyperkinetic Movements. 2* (2012)

590

591

19. Labiano-Fontcuberta, A. & Benito-León, J. Understanding tremor in multiple sclerosis: prevalence, pathological anatomy, and pharmacological and surgical approaches to treatment. *Tremor And Other Hyperkinetic Movements. 2* (2012)

592

593

594

20. Hallett, Mark. Tremor: pathophysiology. *Parkinsonism & Related Disorders*, vol. 20, pp. S118–S122, 2014.

595

596

21. Alusi, S., Glickman, S., Aziz, T. & Bain, P. Tremor in multiple sclerosis. *Journal Of Neurology, Neurosurgery & Psychiatry*. **66** pp. 131-134 (1999)

597

598

22. Alusi, S. H., J. Worthington, S. Glickman, and P. G. Bain. A study of tremor in multiple sclerosis. *Brain*, vol. 124, no. 4, pp. 720–730, April 2001. Oxford University Press.

599

600

23. Deuschl, G. Movement disorders in multiple sclerosis and their treatment. *Neurodegenerative Disease Management*. **6**, 31-35 (2016)

601

602

24. McCreary, J., Rogers, J. & Forwell, S. Upper limb intention tremor in multiple sclerosis: an evidence-based review of assessment and treatment. *International Journal Of MS Care*. **20**, 211-223 (2018)

603

604

605

25. Feys, Peter, Frederik Maes, Bart Nuttin, Werner Helsen, Veerle Malfait, Guy Nagels, Ann Lavrysen, and Xuguang Liu. Relationship between multiple sclerosis intention tremor severity and lesion load in the brainstem. *Neuroreport*, vol. 16, no. 12, pp. 1379–1382, 2005.

606

607

608

26. Oakes, P., Srivatsal, S., Davis, M. & Samii, A. Movement disorders in multiple sclerosis. *Physical Medicine And Rehabilitation Clinics*. **24**, 639-651 (2013)

609

610

27. Baril, F. Multiple sclerosis, vision problems and visual impairment interventions. (2011).

611

612

613

28. Versino, M., Hurko, O. & Zee, D. Disorders of binocular control of eye movements in patients with cerebellar dysfunction. *Brain*. **119**, 1933-1950 (1996)

29. Feys, P., Helsen, W., Lavrysen, A., Nuttin, B. & Ketelaer, P. Intention tremor during manual aiming: a study of eye and hand movements. *Multiple Sclerosis Journal*. **9**, 44-54 (2003)

614

615

30. Feys, P., et al. Interaction between eye and hand movements in multiple sclerosis patients with intention tremor. *Movement Disorders: Official Journal of the Movement Disorder Society*. **20**(6), 705-713 (2005).

616

617

618

31. Ayache, S., Chalah, M., Al-Ani, T., Farhat, W., Zouari, H., Créange, A. & Lefaucheur, J. Tremor in multiple sclerosis: The intriguing role of the cerebellum. *Journal Of The Neurological Sciences*. **358**, 351-356 (2015)

619

620

621

32. Alusi, S., Worthington, J., Glickman, S., Findley, L. & Bain, P. Evaluation of three different ways of assessing tremor in multiple sclerosis. *Journal Of Neurology, Neurosurgery Psychiatry*. **68**, 756-760 (2000)

622

623

624

33. Feys, P., Helsen, W., Prinsmel, A., Ilsbroukx, S., Wang, S. & Liu, X. Digitised spirography as an evaluation tool for intention tremor in multiple sclerosis. *Journal Of Neuroscience Methods*. **160**, 309-316 (2007)

625

626

627

34. Wastensson, G., Holmberg, B., Johnels, B. & Barregard, L. Quantitative methods for evaluating the efficacy of thalamic deep brain stimulation in patients with essential tremor. *Tremor And Other Hyperkinetic Movements*. **3** (2013)

628

629

630

35. Pernalete, N., Raheja, A., Segura, M., Menychta, D., Wieczorek, T. & Carey, S. Eye-hand coordination assessment metrics using a multi-platform haptic system with eye-tracking and motion capture feedback. *2018 40th Annual International Conference Of The IEEE Engineering In Medicine And Biology Society (EMBC)*. pp. 2150-2153 (2018)

631

632

633

634

36. Sánchez-Herrera-Baeza, P., Cuerda, R., Oña-Simbaña, E., Palacios-Ceña, D., Pérez-Corrales, J., Cuenca-Zaldivar, J., Gueita-Rodriguez, J., Quirós, C., Jardón-Huete, A. & Cuesta-Gomez, A. The impact of a novel immersive virtual reality technology associated with serious games in Parkinson's disease patients on upper limb rehabilitation: a mixed methods intervention study. *Sensors*. **20**, 2168 (2020)

635

636

637

638

639

37. Cuesta-Gómez, A., Sánchez-Herrera-Baeza, P., Oña-Simbaña, E., Martinez-Medina, A., Ortiz-Comino, C., Quirós, C., Jardón-Huete, A. & Cuerda, R. Effects of virtual reality associated with

640

641
serious games for upper limb rehabilitation in patients with multiple sclerosis: Randomized controlled trial. *Journal Of Neuroengineering And Rehabilitation.* **17** pp. 1-10 (2020)

38. Patil, V., Narayan, J., Sandhu, K. & Dwivedy, S. Integration of virtual reality and augmented reality in physical rehabilitation: a state-of-the-art review. *Revolutions In Product Design For Healthcare: Advances In Product Design And Design Methods For Healthcare*. pp. 177-205 (2022)

39. Ficarra, B. Virtual reality, augmented reality, and mixed reality. *Emerging Technologies For Nurses: Implications For Practice*. 95-1267 (2020).

40. Albanese, G., Bucchieri, A., Podda, J., Tacchino, A., Buccelli, S., De Momi, E., Laffranchi, M., Mannella, K., Holmes, M., Zenzeri, J. & Others Robotic systems for upper-limb rehabilitation in multiple sclerosis: a SWOT analysis and the synergies with virtual and augmented environments. *Frontiers In Robotics And AI.* **11** pp. 133-147 (2024)

41. Nahum, M., Lee, H. & Merzenich, M. Principles of neuroplasticity-based rehabilitation. *Progress In Brain Research.* **207** pp. 141-171 (2013)

42. Taylor, Matthew J. D., and Murray Griffin. The use of gaming technology for rehabilitation in people with multiple sclerosis. *Multiple Sclerosis Journal*, vol. 21, no. 4, pp. 355-371, 2015.

43. Alawieh, A., Zhao, J. & Feng, W. Factors affecting post-stroke motor recovery: implications on neurotherapy after brain injury. *Behavioural Brain Research.* **340** pp. 94-101 (2018)

44. Sveistrup, Heidi. Motor rehabilitation using virtual reality. *Journal of NeuroEngineering and Rehabilitation*, vol. 1, pp. 1-8, 2004.

45. Valentina, M., Ana, Š., Valentina, M., Martina, Š., Željka, K. & Mateja, Z. Virtual Reality in Rehabilitation and Therapy. *Acta Clinica Croatica.* **52**(4), 453-457 (2013)

46. Mubin, O., Alnajjar, F., Jishtu, N., Alsinglawi, B. & Al Mahmud, A. Exoskeletons with virtual reality, augmented reality, and gamification for stroke patients' rehabilitation: systematic review. *JMIR Rehabilitation And Assistive Technologies.* **6**, e12010 (2019)

47. Carmigniani, J., Furht, B., Anisetti, M. & et al. Augmented reality technologies, systems and applications. *Multimedia Tools And Applications.* **51**, 341-377 (2011).

48. Burke, James William, M. D. J. McNeill, Darryl K. Charles, Philip J. Morrow, Jacqui H. Crosbie, and S. M. McDonough. Augmented reality games for upper-limb stroke rehabilitation. In *Proceedings of the 2010 Second International Conference on Games and Virtual Worlds for Serious Applications*, pp. 75-78, 2010.

49. Guo, H.-J. & Prabhakaran, B. Hololens 2 technical evaluation as mixed reality guide. *arXiv Preprint arXiv:2207.09554* (2022).

50. Colomer, Carolina, Roberto Llorens, Enrique Noé, and Mariano Alcañiz. Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke. *Journal of NeuroEngineering and Rehabilitation*, vol. 13, pp. 1-11, 2016.

51. Lehrer, N., Chen, Y., Duff, M. & et al. Exploring the bases for a mixed reality stroke rehabilitation system, Part II: Design of Interactive Feedback for upper limb rehabilitation. *Journal Of NeuroEngineering And Rehabilitation.* **8**, 54 (2011).

52. Bucchieri, A., Buccelli, S., Barresi, G., Tessari, F., DE MOMI, E., Laffranchi, M., De Michieli, L. & Others Design of a mixed reality environment for the extrapolation of reference trajectories in upper-limb rehabilitation. *International Society For Virtual Rehabilitation (ISVR)*. (2022)

53. Yahya, Muhammad, Jawad Ali Shah, Kushsairy Abdul Kadir, Zulkhairi M. Yusof, Sheroz Khan, and Arif Warsi. Motion capture sensing techniques used in human upper limb motion: A review. *Sensor Review*, vol. 39, no. 4, pp. 504-511, 2019.

54. Menolotto, M., Komaris, D., Tedesco, S., O'Flynn, B. & Walsh, M. Motion capture technology in industrial applications: A systematic review. *Sensors.* **20**, 5687 (2020)

55. Bucchieri, A., Lucaroni, A., Moschetta, M., Ricci, L., Sabatino, E., Grange, E., Tacchino, A., Podda, J., De Momi, E., Ferraresi, C. & Others Exploring the Potential of Mixed Reality for Functional Assessment in Multiple Sclerosis. *2024 IEEE Gaming, Entertainment, And Media Conference (GEM)*. pp. 1-6 (2024)

56. Palumbo, A. Microsoft HoloLens 2 in medical and healthcare context: state of the art and future prospects. *Sensors.* **22**, 7709 (2022)

57. Saladino, M. L., Gualtieri, C., Scaffa, M., Lopatin, M. F., Kohler, E., Bruna, P., et al. (2023). Neurorehabilitation effectiveness based on virtual reality and tele rehabilitation in people with multiple sclerosis in Argentina: reavitelem study. *Multiple Scler. Relat. Disord.* 70, 104499.

58. Godfrey, S. B., and Barresi, G. (2022). "Video games for positive aging: playfully engaging older adults," in *Internet of things for human-centered design: application to elderly healthcare* (Springer), 375-404.
59. Webster, A., Poyade, M., Rooney, S., and Paul, L. (2021). Upper limb rehabilitation interventions using virtual reality for people with multiple sclerosis: a systematic review. Multiple Scler. Relat. Disord. 47, 102610

700

701

702

60. Pau, M., Porta, M., Bertoni, R., Mattos, F. G. M., Cocco, E., and Cattaneo, D. (2023b). Effect of immersive virtual reality training on hand-to-mouth task performance in people with multiple sclerosis: a quantitative kinematic study. Multiple Scler. Relat. Disord. 69, 104455.

703

704

705

61. Chadali, A., Trevlaki, E., Zarra, E., and Trevlakis, E. (2023). Virtual reality in upper extremity rehabilitation of multiple sclerosis patients

706

707

708

62. Pruszyńska, M., Milewska-Jędrzejczak, M., Bednarski, I., Szpakowski, P., Głąbiński, A., and Tadeja, S. K. (2022). Towards effective telerehabilitation: assessing effects of applying augmented reality in remote rehabilitation of patients suffering from multiple sclerosis. ACM Trans. Accessible Comput. (TACCESS) 15, 1–14.

709

710

711

712

63. Hart, S.G., & Staveland, L.E. (2019). Development of NASA-TLX (Task LoadIndex): Results of empirical and theoretical research. *In P.A. Hancock & N. Meshkati (Eds.), Human mental workload* (pp. 139–183). (1988)

713

714

715

64. Bucchieri, A., Tessari, F., Buccelli, S., De Momi, E., Laffranchi, M. & De Michieli, L. The effect of gravity on hand spatio-temporal kinematic features during functional movements. *PloS One*. 19, e0310192 (2024)

716

717

718

65. Bayle, N., Lempereur, M., Hutin, E., Motavasseli, D., Remy-Neris, O., Gracies, J. & Corne, G. Comparison of Various Smoothness Metrics for Upper Limb Movements in Middle-Aged Healthy Subjects. *Sensors*. 23, 1158 (2023)

719

720

721

722

66. Hailey RO, De Oliveira AC, Ghonasgi K, Whitford B, Lee RK, Rose CG, et al. Impact of gravity compensation on upper extremity movements in harmony exoskeleton. In: 2022 International Conference on Rehabilitation Robotics (ICORR). IEEE; 2022. p. 1–6.

723

724

725

67. Saes M, Mohamed Refai MI, van Kordelaar J, Scheltinga BL, Van Beijnum BJF, Bussmann JB, et al. Smoothness metric during reach-to-grasp after stroke: Part 2. Longitudinal association with motor impairment. Journal of neuroengineering and rehabilitation. 2021;18:1–10.

726

727

728

729

68. Mohamed R, Mohamed I, Saes M, Scheltinga BL, van Kordelaar J, Bussmann JB, et al. Smoothness metrics for reaching performance after stroke. Part 1: which one to choose? Journal of neuroengineering and rehabilitation. 2021;18:1–16.

730

731

732

69. Hajihosseinali M, Behzadipour S, Taghizadeh G, Farahmand F. Direction-dependency of the kinematic indices in upper extremities motor assessment of stroke patients. Medical Engineering & Physics. 2022;108:103880.

733

734

735

70. Schwarz A, Kanzler CM, Lambercy O, Luft AR, Veerbeek JM. Systematic review on kinematic assessments of upper limb movements after stroke. Stroke. 2019;50(3):718–727.

736

737

71. Brandon Rohrer, Susan Fasoli, Hermano Igo Krebs, Richard Hughes, Bruce Volpe, Walter R. Frontera, Joel Stein and Neville Hogan *Journal of Neuroscience* 15 September (2002)

738

739

72. Goffredo M, Mazzoleni S, Gison A, Infarinato F, Pournajaf S, Galafate D, et al. Kinematic parameters for tracking patient progress during upper limb robot-assisted rehabilitation: an observational study on subacute stroke subjects. Applied Bionics and Biomechanics. 2019;2019.

740

741

742

73. Panarese A, Colombo R, Sterpi I, Pisano F, Micera S. Tracking motor improvement at the subtask level during robot-aided neurorehabilitation of stroke patients. Neurorehabilitation and neural repair. 2012;26(7):822–833.

743

744

745

74. Clark L, Riggs S. VR-Based Kinematic Assessments: Examining the Effects of Task Properties on Arm Movement Kinematics. In: CHI Conference on Human Factors in Computing Systems Extended Abstracts; 2022. p. 1–8.

746

747

748

75. Valevicius AM, Boser QA, Lavoie EB, Murgatroyd GS, Pilarski PM, Chapman CS, et al. Characterization of normative hand movements during two functional upper limb tasks. PLoS One. 2018;13(6):e0199549

749

750

751

752

76. Zollo, L., Gallotta, E., Guglielmelli, E. & Sterzi, S. Robotic technologies and rehabilitation: new tools for upper-limb therapy and assessment in chronic stroke. *European Journal Of Physical And Rehabilitation Medicine*. 47, 223-236 (2011,6), http://europepmc.org/abstract/MED/21445028

753

754

77. Ponsiglione, A., Ricciardi, C., Amato, F., Cesarelli, M., Cesarelli, G. & D'Addio, G. Statistical analysis and kinematic assessment of upper limb reaching task in Parkinson's disease. *Sensors*. 22, 1708 (2022)

755

756

78. Jaric, S. Changes in Movement Symmetry Associated With Strengthening and Fatigue of Agonist and Antagonist Muscles. *Journal of Motor Behavior* (2000).

757
79. Raouafi, S., Achiche, S., Begon, M., Sarcher, A. & Raison, M. Classification of upper limb disability levels of children with spastic unilateral cerebral palsy using K-means algorithm. *Medical Biological Engineering Computing.* **56** pp. 49-59 (2018) 757

80. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics. 1979; p. 65-70. 760

81. Mathiowetz, V., Volland, G., Kashman, N. & Weber, K. Adult norms for the Box and Block Test of manual dexterity. *The American Journal Of Occupational Therapy.* **39**, 386-391 (1985) 762

82. Mathiowetz V, Weber K, Kashman N, Volland G. Adult Norms for the Nine Hole Peg Test of Finger Dexterity. The Occupational Therapy Journal of Research. 1985;5(1):24-38. doi:10.1177/153944928500500102 764

83. Carpinella, I., Cattaneo, D. & Ferrarin, M. Hilbert-Huang transform based instrumental assessment of intention tremor in multiple sclerosis. *Journal Of Neural Engineering.* **12**, 046011 (2015) 767

84. Alusi, S., Worthington, J., Glickman, S. & Bain, P. A study of tremor in multiple sclerosis. *Brain.* **124**, 720-730 (2001) 770

85. V. Karunakaran, S. B. E. Raj, P. L. Jancy et al., "Investigation of in-home augmented reality assisted rehabilitation therapies for disabled patient," in *2021 International Conference on ICT for Smart Society (ICISS)*, 1-7 (IEEE, 2021). 772

86. A. Alamri, J. Cha, M. Eid, and A. El Saddik, "Evaluating the post-stroke patients progress using an augmented reality rehabilitation system," in *2009 IEEE International Workshop on Medical Measurements and Applications*, 89-94 (IEEE, 2009). 775

87. M. Speicher, B. D. Hall, and M. Nebeling, "What is mixed reality?" in *Proceedings of the 2019 CHI conference on human factors in computing systems*, 1-15 (2019). 778

88. M. Khademi, H. M. Hondori, L. Dodakian, S. Cramer, and C. V. Lopes, "Comparing "pick and place" task in spatial augmented reality versus non-immersive virtual reality for rehabilitation setting," in *2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)*, 4613-4616 (IEEE, 2013). 780

89. S. Condino, G. Turini, R. Viglialoro, M. Gesi, and V. Ferrari, "Wearable augmented reality application for shoulder rehabilitation," *Electronics*, 8, 1178 (2019). 784

90. K. Tada, K. Kutsuzawa, D. Owaki, and M. Hayashibe, "Quantifying motor and cognitive function of the upper limb using mixed reality smartglasses," in *2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)*, 2556-2559 (IEEE, 2022). 786