# Quantitative Assessment of ADL: A Pilot Study of Upper Extremity Reaching Tasks

Authors: Saiyi Li; Pubudu N. Pathirana; Mary P. Galea; Goetz Ottmann; Fary Khan
Journal: Journal of Sensors
 (2015)

---

## Abstract

Effective telerehabilitation technologies enable patients with certain physiological disabilities to engage in rehabilitative exercises for performing Activities of Daily Living (ADLs). Therefore, training and assessment scenarios for the performance of ADLs are vital for the promotion for telerehabilitation. In this paper we investigate quantitatively and automatically assessing patient’s kinematic ability to perform functional upper extremity reaching tasks. The shape of the movement trajectory and the instantaneous acceleration of kinematically crucial body parts, such as wrists, are used to compute the approximate entropy of the motions to represent stability (smoothness) in addition to the duration of the activity. Computer simulations were conducted to illustrate the consistency, sensitivity and robustness of the proposed method. A preliminary experiment with kinematic data captured from healthy subjects mimicking a reaching task with dyskinesia showed a high degree of correlation (Cohen’s kappa 0.85 withp
<
0.05) between a human observer and the proposed automatic classification tool in terms of assigning the datasets to various levels to represent the subjects’ kinematic abilities to perform reaching tasks. This study supported the use of Microsoft Kinect to quantitatively evaluate the ability of individuals with involuntary movements to perform an upper extremity reaching task.


---

## Body

## 1. Introduction

In recent decades, with the advancements in telerehabilitation and associated motion capture technologies, an increasing number of research and development activities are focusing on the development of automated quantitative measures of patient performance in Activities of Daily Living (ADL) [1–6]. Due to the important role played by the upper extremity in ADL [7, 8], an automated approach for measuring and assessing the ability of upper extremity to perform certain tasks is vital for telerehabilitation systems to deliver their full potential.In the past few decades, a number of approaches have been proposed for assessing upper extremities, the majority of which are questionnaires. In musculoskeletal movement disorders of the extremities, most scales are generic. For instance, the self-reported Musculoskeletal Function Assessment (MFA) instrument [9], Short Musculoskeletal Function Assessment (SMFA) questionnaire [10], and self-administered measure of disabilities of the arm, shoulder, and hand [11] were developed but few of them are condition-specific. More examples can be found in [8]. However, for neurological movement disorders, assessments are more disease-specific and rarely focus on upper extremities. For instance, the Fahn-Marsden rating scale (F-M) [12], Global Dystonia Rating Scale (GDRS) [13], Unified Dystonia Rating Scale (UDRS) [14], and so on [15] were developed for dystonia. The Parkinson’s Disease Questionnaire (PDQ-39) [16] and its shorter version (PDQ-8) [17], as well as Parkinson’s Disease Quality of Life questionnaire (PDQL) [18], Webster [19], and Unified Parkinson’s Disease Rating Scale (UPDRS) [20], were developed for Parkinson’s disease. More relevant to this paper, Lane et al. [21] developed Abnormal Involuntary Movement Scale (AIMS) to assess patients with dyskinesia. In this scale, the amplitude of involuntary movements was taken into consideration. In addition, Goetz et al. [22] proposed to use the Objective Dyskinesia Rating Scale (also known as the Rush Dyskinesia scale) to assess the severity of dyskinesia.Although these assessment tools have been utilised pervasively by clinicians, they are not suitable for the telerehabilitation environment. One reason is that the use of the majority of these tools requires clinicians, who are usually absent in the telerehabilitation sessions. Furthermore, some self-report questionnaires may lead to biased results. Therefore, in telerehabilitation, it is critical to develop an automated approach to objectively assess the ability of patients in order to assist therapists in making further clinical decisions.In this paper, we conducted a preliminary investigation of the feasibility of utilising an automatic approach to assess the ability of patients suffering from dyskinesia, to perform an upper extremity reaching task in their daily living. This is assessed by measuring the smoothness of motion trajectories and the duration to finish the task. As is pointed out by Daneault et al. [23], dyskinesia is one of the factors that adversely influence voluntary movement since some involuntary movements would be performed. Therefore, they proposed that dyskinesia in Parkinson’s disease could be seen as a factor in the signal-to-noise ratio (SNR) equation with voluntary movements as the numerator (motion input) and dyskinesia as one element in the denominator. Furthermore, to assess the severity of dyskinesia, the amplitude of involuntary movements is one of the important elements that has been utilised in Abnormal Involuntary Movement Scale (AIMS) [21, 23]. In addition, dyskinesia may be associated with some degree of jerk in the extremities [24, 25]. Due to inaccurate motion trajectories, patients with dyskinesia are more likely to reduce their speed and take a longer time to finish a task in comparison to healthy subjects [26]. Therefore, it is reasonable to infer that by looking at the submovements and jerks (smoothness) in motion trajectories, as well as the motion duration, we are able to evaluate the ability of the subject to perform reaching tasks in daily life from an action kinematic standpoint. In this paper, we used submovements and jerks to infer involuntary movements with large and small amplitudes, which are defined in Table 1 for the experiments.Table 1
The definition of three kinematic severity levels of involuntary movements and jerks, as well as their corresponding abilities in performing reaching tasks in daily living.
Level
1
2
3
Number of submovements
0
≤3
>3
Amplitude of submovements
0
≤0.3 m
>0.3 m
Number of jerks
0
≤3
>3
Amplitude of jerks
0
≤0.03 m
>0.03 m
Duration
≤5 s
5 s~10 s
>10 s
Ability in performing reaching task
High
Medium
LowIn line with our work, a number of studies have been conducted to evaluate automated performance measurements or kinematics relating to upper extremities [27] (some commonly utilised features are given in Figure 1). One of the most obvious critical factors negatively impacting the quality of upper extremity movements is the smoothness of motion trajectories. Zariffa et al. [28] considered directional changes, mean velocity, ratio of mean and maximum velocity, and mean jerk to measure the smoothness of trajectories in patients’ upper limb movements collected from a robotic rehabilitation device. In addition, Rohrer et al. [29] compared five features, namely, jerk, speed, mean arrest period ratio, number of peaks in speed, and “tent” metric, to evaluate the smoothness of an arm motion trajectory performed by stroke patients. Moreover, Lum et al. [30] counted the number of times the tangential acceleration of hand passed zero to measure the smoothness of upper extremity movements in stroke patients. Apart from smoothness in motion trajectories, the duration of a specific task is also important for upper extremity performance evaluation. Murphy et al. [31] took duration of drinking into account to analyse the kinematic aspect of drinking with a cohort of healthy subjects. Similarly, duration was also considered as a factor of upper extremity movement assessment in [32, 33]. In addition, Balasubramanian et al. [32] highlighted the disadvantages in some existing approaches and proposed utilising the spectral arc-length metric of the movement speed profile’s Fourier magnitude spectrum to evaluate the smoothness of the movements.Figure 1
Examples of commonly used techniques with features considered for ADL performance measurement. Feature-based performance evaluation (FPE) is primarily based on kinematic or kinetic measurements. Dynamic measurements such as number of velocity peaks [34], number of zero-crossing tangential accelerations [35], dimensionless jerk metrics [29], and spectral arc-length [32] are used in techniques such as VP, ZCA, DJ, and SAL, respectively, in the literature. Our proposed entropy of shape and instantaneous acceleration (ESA) introduced in this paper uses both shape and kinematic based measurements. Kinetic-related features, such as moment [36], force [37], and torque [38], have also been investigated.The contributions of this paper are threefold:(i)
Using the shape of the trajectory and instantaneous acceleration to extract kinematic smoothness via the concept of motion entropy.
(ii)
Utilising smoothness and duration as criteria to evaluate the performance of an upper extremity reaching task.
(iii)
Investigating the possibility of using an affordable, noninvasive consumer device for evaluation of an upper extremity reaching task performance evaluation on a regular basis.In Section2, the methodology used in this paper is discussed, including the experimental setup, data collection, analysis, and feature extraction approaches, followed by the description of the protocol of the experiments. The results of computer simulation and the real-data experiment are stated in Section 4. Concluding remarks of the paper are given in Section 5.


## 2. Methodology

### 2.1. Severity Levels Definition

In order to perform computer simulations as well as obtain data from healthy subjects mimicking the underlying involuntary movements, it is important to precisely specify the severity levels of involuntary movements in a kinematic standpoint. Since the frequency and amplitude of involuntary movements in addition to the duration of the specific task are important factors in assessments [21, 25, 39], we defined three severity levels of involuntary movements by assessing the kinematic performances of the upper extremity in a reaching task as indicated in Table 1. As this paper is a preliminary study to investigate the feasibility of using kinematic measurements to evaluate the severity of involuntary movements, indeed a more focused exercise to describe each level and a larger set of levels can be used [21]. Nevertheless, our proposed approach can simply be used with an improved distinction of severity levels and hence consider this aspect if not the primary focus of this work.


### 2.2. Feature Extraction

In order to quantitatively evaluate the ability to perform the experimental task, various features need to be extracted from a raw 3D trajectory (γ
(
k
)
=
[
x
(
k
)
,
y
(
k
)
,
z
(
k
)
]
⊤, where x
(
k
), y
(
k
), and z
(
k
) are the joint position in a Cartesian coordinate frame at time k
∈
1,2
,
…
,
K with a temporal interval of δ
k) captured from the Kinect (δ
k
=
30 ms). In this paper, we considered the concept of approximate entropy of motion trajectory and the duration associated with the motion. The approximate entropy of the trajectory is related to the shape and the dynamics (i.e., instantaneous acceleration) computed from the trajectory. The feature extraction process is depicted as follows.(i) Shape Model. Apart from the dynamics in a trajectory, its shape is also taken into account since the involuntary movements are usually associated with randomly moving joint positions, which are represented as unknown uncertainties in the shape. Therefore, it is important to extract the shape of the trajectory for evaluation [40].Curvature (κ) and torsion (τ) of a trajectory can be computed as(1)
κ
=
V
×
A
V
3
;
τ
=
V
·
A
×
J
V
×
A
2
,where V is velocity, A is acceleration, and J is jerk.Since the normal approach to compute numerical differentiation is very sensitive to noise, we utilised the approach introduced in [41] to estimate V, A, and J.(ii) Instantaneous Acceleration. The instantaneous acceleration is the magnitude of the acceleration throughout the trajectory computed as(2)
A
k
i
=
A
k
,where k
=
1,2
,
…
,
K and A
k was estimated in the previous step.(iii) Approximate Entropy. Since the uncertainty in the features, including acceleration and shape model, is implicitly captured to determine how smoothly or not a person performs a task, approximate entropy [42] can be computed based on the previous features.To compute the approximate entropy of a variable, that is, instantaneous acceleration, we denote it as(3)
A
=
a
1
,
a
2
,
…
,
a
Kfor a trajectory with the length K. By defining a constant m
∈
Z for the length of the captured sequence of data, the vector format is given as (4)
B
i
=
a
1
,
a
2
,
…
,
a
i
+
m
-
1
,where i
=
1,2
,
…
,
L
-
m
+
1.A given constantr
>
0 indicates the filtering level and, for each i, C
i
m
(
r
)
∈
R is calculated by finding the number of a
js that satisfies the condition d
[
a
i
,
a
j
]
⩽
r and dividing it by K
-
m
+
1, where j
=
1,2
,
…
,
L
-
m
+
1 and d
[
a
i
,
a
j
]
=
m
a
x
p
=
1,2
,
…
,
m
(
|
a
i
+
p
-
1
-
a
j
+
p
-
1
|
). Then the approximate entropy of the variable in (3) can be computed as(5)
H
A
=
lim
r
→
0
⁡
lim
m
→
+
∞
⁡
lim
K
→
+
∞
⁡
Φ
m
r
-
Φ
m
+
1
r
,where(6)
Φ
m
r
=
K
-
m
+
1
-
1
∑
i
=
1
K
-
m
+
1
log
2
⁡
C
i
m
r
.However, for a 3D trajectory shape, there are two variables of significance, curvature (κ) and torsion (τ). Here we compute the approximate entropy of the trajectory considering joint approximate entropy of κ and τ as H
κ
,
τ. However, the following remark is vital for computational simplicity.Remark. Curvature and torsion are independent variables (κ
⫫
τ
)
.According to the Frenet-Serret formulas [43, 44](7)
d
T
d
N
d
B
=
0
κ
0
κ
0
τ
0
-
τ
0
T
N
B
.
κ and τ describe the relationship between T, N, and B, where T
⫫
N
⫫
B. From(8)
d
T
d
t
=
κ
N
,we can see that κ is the amplitude of the projection of the change of tangent vector on the normal vector. Similarly,(9)
d
B
d
t
=
-
τ
Nshows that τ is the amplitude of the projection of the change of binormal vector on the normal vector. Since κ and τ indicate the change in two independent vectors, they are independent of each other. Therefore, H
κ
,
τ
=
H
κ
+
H
τ.


## 2.1. Severity Levels Definition

In order to perform computer simulations as well as obtain data from healthy subjects mimicking the underlying involuntary movements, it is important to precisely specify the severity levels of involuntary movements in a kinematic standpoint. Since the frequency and amplitude of involuntary movements in addition to the duration of the specific task are important factors in assessments [21, 25, 39], we defined three severity levels of involuntary movements by assessing the kinematic performances of the upper extremity in a reaching task as indicated in Table 1. As this paper is a preliminary study to investigate the feasibility of using kinematic measurements to evaluate the severity of involuntary movements, indeed a more focused exercise to describe each level and a larger set of levels can be used [21]. Nevertheless, our proposed approach can simply be used with an improved distinction of severity levels and hence consider this aspect if not the primary focus of this work.


## 2.2. Feature Extraction

In order to quantitatively evaluate the ability to perform the experimental task, various features need to be extracted from a raw 3D trajectory (γ
(
k
)
=
[
x
(
k
)
,
y
(
k
)
,
z
(
k
)
]
⊤, where x
(
k
), y
(
k
), and z
(
k
) are the joint position in a Cartesian coordinate frame at time k
∈
1,2
,
…
,
K with a temporal interval of δ
k) captured from the Kinect (δ
k
=
30 ms). In this paper, we considered the concept of approximate entropy of motion trajectory and the duration associated with the motion. The approximate entropy of the trajectory is related to the shape and the dynamics (i.e., instantaneous acceleration) computed from the trajectory. The feature extraction process is depicted as follows.(i) Shape Model. Apart from the dynamics in a trajectory, its shape is also taken into account since the involuntary movements are usually associated with randomly moving joint positions, which are represented as unknown uncertainties in the shape. Therefore, it is important to extract the shape of the trajectory for evaluation [40].Curvature (κ) and torsion (τ) of a trajectory can be computed as(1)
κ
=
V
×
A
V
3
;
τ
=
V
·
A
×
J
V
×
A
2
,where V is velocity, A is acceleration, and J is jerk.Since the normal approach to compute numerical differentiation is very sensitive to noise, we utilised the approach introduced in [41] to estimate V, A, and J.(ii) Instantaneous Acceleration. The instantaneous acceleration is the magnitude of the acceleration throughout the trajectory computed as(2)
A
k
i
=
A
k
,where k
=
1,2
,
…
,
K and A
k was estimated in the previous step.(iii) Approximate Entropy. Since the uncertainty in the features, including acceleration and shape model, is implicitly captured to determine how smoothly or not a person performs a task, approximate entropy [42] can be computed based on the previous features.To compute the approximate entropy of a variable, that is, instantaneous acceleration, we denote it as(3)
A
=
a
1
,
a
2
,
…
,
a
Kfor a trajectory with the length K. By defining a constant m
∈
Z for the length of the captured sequence of data, the vector format is given as (4)
B
i
=
a
1
,
a
2
,
…
,
a
i
+
m
-
1
,where i
=
1,2
,
…
,
L
-
m
+
1.A given constantr
>
0 indicates the filtering level and, for each i, C
i
m
(
r
)
∈
R is calculated by finding the number of a
js that satisfies the condition d
[
a
i
,
a
j
]
⩽
r and dividing it by K
-
m
+
1, where j
=
1,2
,
…
,
L
-
m
+
1 and d
[
a
i
,
a
j
]
=
m
a
x
p
=
1,2
,
…
,
m
(
|
a
i
+
p
-
1
-
a
j
+
p
-
1
|
). Then the approximate entropy of the variable in (3) can be computed as(5)
H
A
=
lim
r
→
0
⁡
lim
m
→
+
∞
⁡
lim
K
→
+
∞
⁡
Φ
m
r
-
Φ
m
+
1
r
,where(6)
Φ
m
r
=
K
-
m
+
1
-
1
∑
i
=
1
K
-
m
+
1
log
2
⁡
C
i
m
r
.However, for a 3D trajectory shape, there are two variables of significance, curvature (κ) and torsion (τ). Here we compute the approximate entropy of the trajectory considering joint approximate entropy of κ and τ as H
κ
,
τ. However, the following remark is vital for computational simplicity.Remark. Curvature and torsion are independent variables (κ
⫫
τ
)
.According to the Frenet-Serret formulas [43, 44](7)
d
T
d
N
d
B
=
0
κ
0
κ
0
τ
0
-
τ
0
T
N
B
.
κ and τ describe the relationship between T, N, and B, where T
⫫
N
⫫
B. From(8)
d
T
d
t
=
κ
N
,we can see that κ is the amplitude of the projection of the change of tangent vector on the normal vector. Similarly,(9)
d
B
d
t
=
-
τ
Nshows that τ is the amplitude of the projection of the change of binormal vector on the normal vector. Since κ and τ indicate the change in two independent vectors, they are independent of each other. Therefore, H
κ
,
τ
=
H
κ
+
H
τ.


## 3. Experiment Setup

### 3.1. Simulation Data Collection

The simulations were conducted with Matlab 2013a to ensure that the proposed approach for smoothness measurement met the consistency, sensitivity, and robustness requirements given in [32], according to which(i)
smoothness measurement should change consistently (either increase or decrease) with respect to the increasing numbers of submovements and jerky movements;
(ii)
smoothness measurement should be sensitive to any variation of smoothness in the trajectory so that more information regarding the smoothness characteristics can be shown;
(iii)
the calculation of smoothness measurement should be robust to measurement noise so that the result of smoothness is less likely to be affected by noise.To simulate the reaching movement, we utilised the following process.(1) Voluntary Movement. A noiseless, free reaching involuntary movement is simulated as a smooth arc with a duration K,(10)
γ
k
=
cos
⁡
π
k
3
×
K
+
π
3
,
sin
⁡
π
k
3
×
K
+
π
3
,
0
,where k
=
1,2
,
…
,
K.(2) Submovements. The submovements are generated by the sum of multiple Gaussian models for each axis (X, Y, and Z) as(11)
I
t
=
∑
n
i
=
1
N
i
A
n
i
exp
⁡
-
t
-
μ
n
i
2
2
σ
n
i
2
,where N
i is the number of involuntary movements and A
n
i, μ
n
i, and σ
n
i are the amplitude, mean time, and standard deviation of the duration of the n
ith involuntary movement. These variables can be different for various axes. By adding the I
(
t
)
x, I
(
t
)
y, and I
(
t
)
z to γ
t, we are able to create a trajectory with involuntary movements.(3) Jerk. Jerks are simulated by adding Gaussian noise with normal distribution and various amplitudes into the motion trajectories. There are four parameters to determine the jerk, including the number of jerks (N
j), starting time (S
n
j), duration (D
n
j), and amplitude of jerks (A
n
j).To ascertain the consistency and sensitivity of the proposed approach, 50 trajectories were generated to simulate a reaching movement with various numbers and amplitudes of involuntary movements and jerks. The specifications for these trajectories are shown in Table2.Table 2
Parameters used to simulate two groups of trajectories. These two groups of trajectories correspond to two severity levels of involuntary movements. The first 25 trajectories belong to the second level with two involuntary movements and two jerks. The last 25 trajectories are in the third level with four involuntary movements and jerks. To simulate the severity in one level, the amplitudes of involuntary movements and jerks increase with the index.
Level 2
Level 3
Submovements
Index
1,2
,
…
,
25
26,27
,
…
,
50
Duration
4.5 seconds (135 frames)
7.5 seconds (210 frames)
N
i
2
4
μ
n
i
/
σ
n
i
45/50
40/50
90/50
90/60
—
140/60
—
190/30
A
n
i
(
m
)
Index× 0.009
Index× 0.009
Index× 0.01
Index× 0.01
—
Index× 0.011
—
Index× 0.012
Jerks
N
j
2
4
A
n
j
(
m
)
Index× 0.001
Index× 0.001
Index× 0.0011
Index× 0.0011
—
Index× 0.0012
—
Index× 0.0013To simplify the simulation without losing the effects, we only simulated the trajectories in levels two and three. Furthermore, we assumed the numbers and amplitudes of submovements in three axes of the trajectories are the same, which meansI
(
t
)
x
=
I
(
t
)
y
=
I
(
t
)
z. It is the same for jerks.In addition, to evaluate the robustness of different approaches, we generated 100 noiseless trajectories in level two and the numbers and amplitudes of involuntary movements were randomly generated in the given range (refer to Table2). For each trajectory, we generated 100 noisy trajectories by adding Gaussian noise with zero mean and 0.33
v
p
e
a
k standard deviation following the experiment in [32].


### 3.2. Real-Data Experiment Data Collection

For real-data experiment, no film recordings of subjects were made in this study. The Kinect camera provided numerical data that directly related to arm movements. Only deidentified numerical data, representing motion vectors, were stored in the database. Volunteers were researchers and students at Deakin University. Ethics for conducting the experiments in this paper have been approved by Deakin University.The real-data experiment was conducted with four healthy subjects mimicking three severity levels of involuntary movements (refer to Table1) while performing an upper limb task, that is, moving a book from one location to another and bringing it back to the original location. Before recording the data, subjects were required to practice the tasks to make sure that their movements for different levels involved the required involuntary movements and durations.In the experiment, a table, a chair, a book, a second version Kinect, and a laptop were used (refer to Figures2 and 3). The chair had no arms and its height was adjustable to suit the subjects. During the experiment, the subject was about 20 cm away from the front of the table so that the book could easily be reached with pure arm movements (without moving his/her trunk). The dimensions of the book were 23.5 × 15.5 × 1 cm and it weighed 0.25 kg. At the same time, the book was placed near the edge of the table so that the subject could hold the book easily.Figure 2
A diagrammatic view of the experimental setup.Figure 3
Real-data experiment setup image. The top image shows the setup of the Kinect and the subject. The bottom left and right are the RGB and depth images taken from the Kinect. Note that the marker was on the right wrist of the subject (the depth and RGB camera in the Kinect reversed the image).To accurately track the involuntary movements and jerk, we attached a small infrared reflective marker on the tracking joint (wrist) and made sure the marker always faced the Kinect so that the Kinect could capture the position of the wrist. The data collection program was written with Kinect SDK v2.0-1409 with C# under Windows 8.1. Although there was no precision evaluation on the second version of Kinect, according to [45], the precision of the new version of Kinect was close to its predecessor (less than 5 mm for the distance between the Kinect and the object less than 1.5 meter [46]).Four healthy subjects were recruited for the experiment. Their demographic data can be seen in Table3. They were required to perform the task of taking a book from one predefined location to another location and then returning it to the original location. The original and target locations were marked on the table in advance and the subjects were required to place the book at the designated position accurately. Further, the subjects were required to mimic three different severities of involuntary movements with various durations (10 iterations for each subject). These involuntary movements were mimicking patients experiencing dyskinesia so that the motion trajectories involved jerkiness and uncontrollable submovements. The numbers and amplitudes of involuntary movements generally followed the specification in Table 1. In addition, we played music with three different durations (4, 7, and 15 seconds for levels one to three) so that the motion duration of the subjects could be generally controlled in three levels. Eventually, we expected deterioration in smoothness from the first to the third level and increased duration. Therefore, the ability to perform reaching tasks decreased. These criteria would be used by a human observer to classify the ability to perform the reaching task into three levels manually for the purpose of validation.Table 3
Demographic data of subjects.
Age
Weight (kg)
Height (cm)
Gender
Subject 1
28
55
172
Male
Subject 2
29
70
175
Male
Subject 3
27
60
173
Male
Subject 4
22
58
160
FemaleDuring the experiment, the subject initially held the book with his/her dominant hand (right hand for all subjects) and kept it steady. At the same time, the system operator checked whether the Kinect could capture the marker. If the Kinect could capture the marker and the subject was ready, the system operator gave the subject an instruction to start moving the book and played the music. In the meantime, the Kinect system started recording the position information of the marker into a database for offline analysis. As soon as the subject finished the task (replacing the book in the original position), the system operator stopped the system. Apart from the system operator and the subject, another researcher classified the task (one of the three levels). The manual classification criteria include the duration of finishing the task and numbers and amplitudes of involuntary movements listed in Table1.Each subject was required to perform the task at least 30 times in total to ensure that there were at least 10 trials at each level. All the 30 trials were conducted over three days with 10 to 15 trials per day depending on the subject availability. Between each trial, the subject could rest for thirty seconds.


## 3.1. Simulation Data Collection

The simulations were conducted with Matlab 2013a to ensure that the proposed approach for smoothness measurement met the consistency, sensitivity, and robustness requirements given in [32], according to which(i)
smoothness measurement should change consistently (either increase or decrease) with respect to the increasing numbers of submovements and jerky movements;
(ii)
smoothness measurement should be sensitive to any variation of smoothness in the trajectory so that more information regarding the smoothness characteristics can be shown;
(iii)
the calculation of smoothness measurement should be robust to measurement noise so that the result of smoothness is less likely to be affected by noise.To simulate the reaching movement, we utilised the following process.(1) Voluntary Movement. A noiseless, free reaching involuntary movement is simulated as a smooth arc with a duration K,(10)
γ
k
=
cos
⁡
π
k
3
×
K
+
π
3
,
sin
⁡
π
k
3
×
K
+
π
3
,
0
,where k
=
1,2
,
…
,
K.(2) Submovements. The submovements are generated by the sum of multiple Gaussian models for each axis (X, Y, and Z) as(11)
I
t
=
∑
n
i
=
1
N
i
A
n
i
exp
⁡
-
t
-
μ
n
i
2
2
σ
n
i
2
,where N
i is the number of involuntary movements and A
n
i, μ
n
i, and σ
n
i are the amplitude, mean time, and standard deviation of the duration of the n
ith involuntary movement. These variables can be different for various axes. By adding the I
(
t
)
x, I
(
t
)
y, and I
(
t
)
z to γ
t, we are able to create a trajectory with involuntary movements.(3) Jerk. Jerks are simulated by adding Gaussian noise with normal distribution and various amplitudes into the motion trajectories. There are four parameters to determine the jerk, including the number of jerks (N
j), starting time (S
n
j), duration (D
n
j), and amplitude of jerks (A
n
j).To ascertain the consistency and sensitivity of the proposed approach, 50 trajectories were generated to simulate a reaching movement with various numbers and amplitudes of involuntary movements and jerks. The specifications for these trajectories are shown in Table2.Table 2
Parameters used to simulate two groups of trajectories. These two groups of trajectories correspond to two severity levels of involuntary movements. The first 25 trajectories belong to the second level with two involuntary movements and two jerks. The last 25 trajectories are in the third level with four involuntary movements and jerks. To simulate the severity in one level, the amplitudes of involuntary movements and jerks increase with the index.
Level 2
Level 3
Submovements
Index
1,2
,
…
,
25
26,27
,
…
,
50
Duration
4.5 seconds (135 frames)
7.5 seconds (210 frames)
N
i
2
4
μ
n
i
/
σ
n
i
45/50
40/50
90/50
90/60
—
140/60
—
190/30
A
n
i
(
m
)
Index× 0.009
Index× 0.009
Index× 0.01
Index× 0.01
—
Index× 0.011
—
Index× 0.012
Jerks
N
j
2
4
A
n
j
(
m
)
Index× 0.001
Index× 0.001
Index× 0.0011
Index× 0.0011
—
Index× 0.0012
—
Index× 0.0013To simplify the simulation without losing the effects, we only simulated the trajectories in levels two and three. Furthermore, we assumed the numbers and amplitudes of submovements in three axes of the trajectories are the same, which meansI
(
t
)
x
=
I
(
t
)
y
=
I
(
t
)
z. It is the same for jerks.In addition, to evaluate the robustness of different approaches, we generated 100 noiseless trajectories in level two and the numbers and amplitudes of involuntary movements were randomly generated in the given range (refer to Table2). For each trajectory, we generated 100 noisy trajectories by adding Gaussian noise with zero mean and 0.33
v
p
e
a
k standard deviation following the experiment in [32].


## 3.2. Real-Data Experiment Data Collection

For real-data experiment, no film recordings of subjects were made in this study. The Kinect camera provided numerical data that directly related to arm movements. Only deidentified numerical data, representing motion vectors, were stored in the database. Volunteers were researchers and students at Deakin University. Ethics for conducting the experiments in this paper have been approved by Deakin University.The real-data experiment was conducted with four healthy subjects mimicking three severity levels of involuntary movements (refer to Table1) while performing an upper limb task, that is, moving a book from one location to another and bringing it back to the original location. Before recording the data, subjects were required to practice the tasks to make sure that their movements for different levels involved the required involuntary movements and durations.In the experiment, a table, a chair, a book, a second version Kinect, and a laptop were used (refer to Figures2 and 3). The chair had no arms and its height was adjustable to suit the subjects. During the experiment, the subject was about 20 cm away from the front of the table so that the book could easily be reached with pure arm movements (without moving his/her trunk). The dimensions of the book were 23.5 × 15.5 × 1 cm and it weighed 0.25 kg. At the same time, the book was placed near the edge of the table so that the subject could hold the book easily.Figure 2
A diagrammatic view of the experimental setup.Figure 3
Real-data experiment setup image. The top image shows the setup of the Kinect and the subject. The bottom left and right are the RGB and depth images taken from the Kinect. Note that the marker was on the right wrist of the subject (the depth and RGB camera in the Kinect reversed the image).To accurately track the involuntary movements and jerk, we attached a small infrared reflective marker on the tracking joint (wrist) and made sure the marker always faced the Kinect so that the Kinect could capture the position of the wrist. The data collection program was written with Kinect SDK v2.0-1409 with C# under Windows 8.1. Although there was no precision evaluation on the second version of Kinect, according to [45], the precision of the new version of Kinect was close to its predecessor (less than 5 mm for the distance between the Kinect and the object less than 1.5 meter [46]).Four healthy subjects were recruited for the experiment. Their demographic data can be seen in Table3. They were required to perform the task of taking a book from one predefined location to another location and then returning it to the original location. The original and target locations were marked on the table in advance and the subjects were required to place the book at the designated position accurately. Further, the subjects were required to mimic three different severities of involuntary movements with various durations (10 iterations for each subject). These involuntary movements were mimicking patients experiencing dyskinesia so that the motion trajectories involved jerkiness and uncontrollable submovements. The numbers and amplitudes of involuntary movements generally followed the specification in Table 1. In addition, we played music with three different durations (4, 7, and 15 seconds for levels one to three) so that the motion duration of the subjects could be generally controlled in three levels. Eventually, we expected deterioration in smoothness from the first to the third level and increased duration. Therefore, the ability to perform reaching tasks decreased. These criteria would be used by a human observer to classify the ability to perform the reaching task into three levels manually for the purpose of validation.Table 3
Demographic data of subjects.
Age
Weight (kg)
Height (cm)
Gender
Subject 1
28
55
172
Male
Subject 2
29
70
175
Male
Subject 3
27
60
173
Male
Subject 4
22
58
160
FemaleDuring the experiment, the subject initially held the book with his/her dominant hand (right hand for all subjects) and kept it steady. At the same time, the system operator checked whether the Kinect could capture the marker. If the Kinect could capture the marker and the subject was ready, the system operator gave the subject an instruction to start moving the book and played the music. In the meantime, the Kinect system started recording the position information of the marker into a database for offline analysis. As soon as the subject finished the task (replacing the book in the original position), the system operator stopped the system. Apart from the system operator and the subject, another researcher classified the task (one of the three levels). The manual classification criteria include the duration of finishing the task and numbers and amplitudes of involuntary movements listed in Table1.Each subject was required to perform the task at least 30 times in total to ensure that there were at least 10 trials at each level. All the 30 trials were conducted over three days with 10 to 15 trials per day depending on the subject availability. Between each trial, the subject could rest for thirty seconds.


## 4. Data Analysis and Results

### 4.1. Computer Simulation Data Analysis

Three simulations were conducted to assess the performance of the proposed approach in terms of motion smoothness, which was evaluated with five approaches, namely, the number of tangential velocity peaks (VP) [34], number of zero-crossing tangential acceleration (ZCA) [35], dimensionless jerk metrics (DJ) [29], spectral arc-length (SAL) [32], and entropy of shape model and instantaneous acceleration (ESA). Except for SAL, the increase of metrics computed by the other approaches illustrates the deterioration of smoothness. Simulation results were analysed from three aspects, namely, consistency, sensitivity, and robustness. For consistency, the raw metrics computed with these five approaches were compared to see if the metrics could keep the same trend with the change of smoothness of trajectories. Secondly, the sensitivity of these approaches with respect to the change of smoothness was analysed. The improvements are computed as(12)
Q
i
=
M
i
-
M
1
M
1
,where M
i is the metric value of trajectories with index of i
=
1,2
,
…
,
50. The approach which had the most significant improvement with respect to the trajectory index was the most sensitive to the change of smoothness since the smoothness deteriorates with the increase of trajectory index (refer to Figure 5). Eventually, the robustness of these approaches was analysed. Firstly, the noisy trajectories were processed by a low pass filter with cut-off frequency of 14 Hz since the simulated frequency of the trajectories was 30 Hz. The normalised difference metrics were computed between the noisy and noiseless trajectories with approach a (a can be VP, ZCA, DJ SAL, and ESA) as follows:(13)
d
i
j
a
=
M
i
j
-
m
i
max
⁡
m
-
min
⁡
m
a
,where m is the collection of the metrics (we call them ground truths in the rest of this paper) of the 100 noiseless trajectories and m
i is the metric of ith, i
=
1,2
,
…
,
100 noiseless trajectory. In addition, M
i
j is the metric of the jth (j
=
1,2
,
…
,
100) noisy trajectories generated based on ith noiseless trajectory. Moreover, m
a
x
(
·
) and m
i
n
(
·
) selected the maximum and minimum data from the collection m. Lastly, the probability density functions of d (the collection of the normalised differences) were estimated by usingksdensity function in Matlab. The whole evaluation process followed that introduced in [32].


### 4.2. Computer Simulation Result

Examples of generated trajectories are depicted in Figure4. Before comparing the metrics from various approaches, the change of smoothness with respect to trajectory indices was shown in Figure 5.Figure 4
These three graphs show trajectories in three levels. (a) is in level one for natural movements without involuntary movements and jerks. (b) Trajectory is in the second level with two involuntary movements (with amplitudes of 0.225 and 0.25 meter) and jerks (with amplitudes of 0.025 and 0.0275 meter). (c) is in the third level with four involuntary movements (with amplitudes of 0.45, 0.5, 0.55, and 0.6 meter) and jerks (with amplitude of 0.05, 0.055, 0.06 and 0.065 meter). The red circles are examples of jerks in the second and third level.
(a)
(b)
(c)Figure 5
The smoothness level of trajectories, which is represented by one over signal-to-noise ratio (SNR) since SNR is nonlinear with respect to the linear change of smoothness, while1
/
S
N
R is linear. It is easier for the following comparison.Figure6 illustrates the consistency characteristics of different approaches. Since the simulated trajectories were classified into two levels of severity of involuntary movements, the trend in individual levels were analysed first. For the results in level one (with blue colour), VP, SAL, and ESA showed a consistent trend with the metrics where VP and ESA kept increasing from around 5 and 0.013 to approximately 17 and 0.1, while SAL decreased from −4 to around −5. The fluctuations were caused by the randomly generated values in the simulation (the same as follows). However, the consistent trend was hardly observed in ZCA and DJ. The metrics of the former fluctuate between 10 and 15, while those of the latter decreased from −25 to around −75 and then increased to approximately −10. For the metrics of trajectories belonging to the third level, DJ, SAL, and ESA show consistent trend, where DJ and ESA increased from about −25 and 0.2 to approximately −5 and 0.3, while SAL decreased from around −5.5 all of these approaches to −6. As for the other two approaches, the consistency was not obvious. Lastly, according to Section 2.1, the trajectories in level three were less smooth than those in level two. Therefore, metrics from these two levels should show the differences, which can be observed in the result for all these approaches. For example, the average metrics from VP, ZCA, DJ, and ESA in level three were higher than the average metrics in the second level, while SAL showed lower metrics in level three than level two. All in all, SAL and ESA outperformed VP, ZCA, and DJ in terms of consistency.Figure 6
The metric given by these approaches tends to illustrate the consistency characteristics in various approaches used to evaluate the smoothness of trajectories in two severity levels. With the increase of numbers and amplitudes of both involuntary movements and jerks, the smoothness of the trajectories deteriorates. The first half (with blue from 1 to 25) is in the second level and the last half (with red from 26 to 50) is in the third level. The VP was computed with a threshold value of 0.01 meter/s and the temporal gap between two consecutive peaks was 100 ms.The second aspect was sensitivity (refer to Figure7), which analysed the change rate of metrics from various methods with respect to the change of smoothness. From the result, it is not hard to observe that ESA was the most sensitive approach since the improvement percentages changed from 0% to around 1000% for the second level and from 2000% to about 2300%. By comparison, metrics of SAL increased from 0% to around 50% from trajectory indices of 1 to 50. As for VP, it was quite sensitive to the change of the severity in the first 15 trajectories (increased from 0% to about 100%) but less sensitive for the rest. For ZCA, the improvement percentages were very minimal. Although DJ was very sensitive in the second level, the sensitivity gradually reduced in the third level. In addition, since the smoothness decreased linearly with respect to the trajectory index, we fitted a line (orange lines) to the metrics generated by each method and computed the gradient of each line indicating the general sensitivity. This confirmed our conclusion that ESA was the most sensitive with the highest gradient.Figure 7
Sensitivity comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique is preferred to be sensitive to the small changes in the severity of involuntary movements and the change rate metric should be proportional to the change rate of severity. The blue and red lines show the improvement of metrics of the second and third severity levels with respect to the metric of the first trajectory computed with various approaches (refer to (12)). The orange lines are the regression lines of the corresponding metrics, showing the approximate improvement rate of each method. The value at the right bottom of each graph is the gradient of the regression line.Lastly, the robustness of the proposed approach was investigated. From Figure8, the performance of ESA was very close to SAL since they had a similar value (around 0.05) with the highest density. However, the range of ESA (0 to 0.75) was a little bigger than SAL (−0.1 to 0.5). By comparison, VP and DJ were not able to maintain the metrics with the influence of measurement noise since they had a large metrics range and large difference from the ground truth. Although the metric for ZCA was in a reasonable range (from 0.1 to 0.5), the offset from the ground truth was large (around 0.3).Figure 8
Robustness comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique needs to be limited to a smaller range (on the horizontal axis) in terms of the differences in metrics between the noisy trajectories and their corresponding noiseless ones, and the value with the highest amplitude in vertical axis should be close to zero.Eventually, ESA outperformed SAL as it was more sensitive to the change of smoothness and also met the requirement of dimensionless, consistency, and robustness.


### 4.3. Healthy Subjects Simulation Data Analysis

In the real-data experiment, firstly, the smoothness of all the trajectories was evaluated using the same approaches considered in the computer simulation section. Additionally, by taking the duration into consideration, all these trials were classified into three levels of ability to perform the task by using three commonly used clustering methods, namely,k-means clustering, Gaussian mixture model (GMM), and fuzzy clustering [47] so as to determine which clustering method suits the purpose of classifying motion trajectories into different levels of ability to perform upper extremity reaching tasks (severity levels of involuntary movements). Since the trials had also been classified by a human observer, a Cohen’s kappa correlation coefficient was computed between the human observer and the computer to indicate the assessment agreement. Higher coefficient values indicate a greater level of agreement between the utilised approach and the human observer. Except for the proposed method, the other four approaches, including VP, ZCA, DJ, and SAL, were used for comparison.


### 4.4. Healthy Subjects Simulation Result

Here we present the results of our preliminary real-data experiment with healthy subjects mimicking different severity levels of involuntary movements with their upper extremities while seated.First of all, some examples of simulated trajectories and features are shown in Figure9. The first three rows are the trajectories in three axes, thereby showing the submovements (with red circles) and jerks (with green rectangles) more clearly. The third and fourth rows were shape models of these trajectories with curvature and torsion, while the last row is the instantaneous acceleration. As for the columns, levels one to three (refer to Section 2.1) are in the first to the third columns. As can be observed, from the first to the third level, the number of submovements and jerks increased from 0 to 4 and 5, respectively. Correspondingly, the curvature, torsion, and instantaneous acceleration were increasingly noisy.Figure 9
Examples of trajectories (first three rows), shape models, including curvatures (fourth row) and torsions (fifth row), and instantaneous accelerations (sixth row) are illustrated for three levels of the ability to perform an upper extremity reaching task (columns one to three corresponding to levels one to three of the severity of involuntary movements). The red circles show examples of submovements and green rectangles are examples of jerks.Secondly, the distributions of various features, including the duration of the task, as well as metrics computed with various approaches, are shown in Figure10. The first box plot was the distribution of durations for three severity levels of involuntary movements. As can be seen, although there was some overlap between two consecutive levels, the interquartile ranges followed the definition of severity levels (refer to Table 1). The remaining five graphs show the metrics computed with five approaches. First of all, it is obvious that, in VP, ZCA, DJ, and ESA, the more severe involuntary movements were associated with higher values, while it was opposite in SAL (the more severe involuntary movements had lower metrics). However, this trend between the first and the second levels in DJ was very small. Secondly, although every approach showed a certain degree of overlap between two consecutive levels, DJ showed the worst situation since the metrics of its first and second levels were very similar. As for the other four approaches, VP and ZCA showed a larger overlap between the first and the second levels than SAL and ESA and VP had the largest overlap between the second the third levels. Generally speaking, the differences between these three levels of ESA were more obvious than in the other four approaches.Figure 10
The distributions of durations utilised to finish the reaching task, as well as metrics computed by five approaches, for three severity levels. The threshold utilised to compute VP was 0.25 m/s and the temporal gap was 100 ms for all subjects.Table4 shows the Cohen’s kappa between five automated approaches and the human observer generated by three clustering approaches. Generally speaking, ESA always gave the highest agreement (0.8250 for K-means and GMM, with 0.85 for fuzzy clustering). In addition, SAL and ZCA had a similar performance with kappa values slightly smaller than 0.8 in the majority of cases. By comparison, VP was a little better than DJ. But these two approaches were worse than the other three methods. Moreover, the comparison of these three clustering approaches suggested that the fuzzy clustering was more suitable for automated assessment of the severity of involuntary movements, thereby evaluating the ability to perform reaching tasks in daily living in terms of kinematics.Table 4
Cohen’s kappa (p
<
0.05) between various automated approaches and the human observer.
Approach
K-means
GMM
Fuzzy clustering
Average
VP
0.6875
0.6926
0.7250
0.7017
ZCA
0.7625
0.8000
0.7500
0.7708
DJ
0.5500
0.5631
0.5500
0.5544
SAL
0.7875
0.7875
0.7500
0.7750
ESA
0.8250
0.8250
0.8500
0.8333


## 4.1. Computer Simulation Data Analysis

Three simulations were conducted to assess the performance of the proposed approach in terms of motion smoothness, which was evaluated with five approaches, namely, the number of tangential velocity peaks (VP) [34], number of zero-crossing tangential acceleration (ZCA) [35], dimensionless jerk metrics (DJ) [29], spectral arc-length (SAL) [32], and entropy of shape model and instantaneous acceleration (ESA). Except for SAL, the increase of metrics computed by the other approaches illustrates the deterioration of smoothness. Simulation results were analysed from three aspects, namely, consistency, sensitivity, and robustness. For consistency, the raw metrics computed with these five approaches were compared to see if the metrics could keep the same trend with the change of smoothness of trajectories. Secondly, the sensitivity of these approaches with respect to the change of smoothness was analysed. The improvements are computed as(12)
Q
i
=
M
i
-
M
1
M
1
,where M
i is the metric value of trajectories with index of i
=
1,2
,
…
,
50. The approach which had the most significant improvement with respect to the trajectory index was the most sensitive to the change of smoothness since the smoothness deteriorates with the increase of trajectory index (refer to Figure 5). Eventually, the robustness of these approaches was analysed. Firstly, the noisy trajectories were processed by a low pass filter with cut-off frequency of 14 Hz since the simulated frequency of the trajectories was 30 Hz. The normalised difference metrics were computed between the noisy and noiseless trajectories with approach a (a can be VP, ZCA, DJ SAL, and ESA) as follows:(13)
d
i
j
a
=
M
i
j
-
m
i
max
⁡
m
-
min
⁡
m
a
,where m is the collection of the metrics (we call them ground truths in the rest of this paper) of the 100 noiseless trajectories and m
i is the metric of ith, i
=
1,2
,
…
,
100 noiseless trajectory. In addition, M
i
j is the metric of the jth (j
=
1,2
,
…
,
100) noisy trajectories generated based on ith noiseless trajectory. Moreover, m
a
x
(
·
) and m
i
n
(
·
) selected the maximum and minimum data from the collection m. Lastly, the probability density functions of d (the collection of the normalised differences) were estimated by usingksdensity function in Matlab. The whole evaluation process followed that introduced in [32].


## 4.2. Computer Simulation Result

Examples of generated trajectories are depicted in Figure4. Before comparing the metrics from various approaches, the change of smoothness with respect to trajectory indices was shown in Figure 5.Figure 4
These three graphs show trajectories in three levels. (a) is in level one for natural movements without involuntary movements and jerks. (b) Trajectory is in the second level with two involuntary movements (with amplitudes of 0.225 and 0.25 meter) and jerks (with amplitudes of 0.025 and 0.0275 meter). (c) is in the third level with four involuntary movements (with amplitudes of 0.45, 0.5, 0.55, and 0.6 meter) and jerks (with amplitude of 0.05, 0.055, 0.06 and 0.065 meter). The red circles are examples of jerks in the second and third level.
(a)
(b)
(c)Figure 5
The smoothness level of trajectories, which is represented by one over signal-to-noise ratio (SNR) since SNR is nonlinear with respect to the linear change of smoothness, while1
/
S
N
R is linear. It is easier for the following comparison.Figure6 illustrates the consistency characteristics of different approaches. Since the simulated trajectories were classified into two levels of severity of involuntary movements, the trend in individual levels were analysed first. For the results in level one (with blue colour), VP, SAL, and ESA showed a consistent trend with the metrics where VP and ESA kept increasing from around 5 and 0.013 to approximately 17 and 0.1, while SAL decreased from −4 to around −5. The fluctuations were caused by the randomly generated values in the simulation (the same as follows). However, the consistent trend was hardly observed in ZCA and DJ. The metrics of the former fluctuate between 10 and 15, while those of the latter decreased from −25 to around −75 and then increased to approximately −10. For the metrics of trajectories belonging to the third level, DJ, SAL, and ESA show consistent trend, where DJ and ESA increased from about −25 and 0.2 to approximately −5 and 0.3, while SAL decreased from around −5.5 all of these approaches to −6. As for the other two approaches, the consistency was not obvious. Lastly, according to Section 2.1, the trajectories in level three were less smooth than those in level two. Therefore, metrics from these two levels should show the differences, which can be observed in the result for all these approaches. For example, the average metrics from VP, ZCA, DJ, and ESA in level three were higher than the average metrics in the second level, while SAL showed lower metrics in level three than level two. All in all, SAL and ESA outperformed VP, ZCA, and DJ in terms of consistency.Figure 6
The metric given by these approaches tends to illustrate the consistency characteristics in various approaches used to evaluate the smoothness of trajectories in two severity levels. With the increase of numbers and amplitudes of both involuntary movements and jerks, the smoothness of the trajectories deteriorates. The first half (with blue from 1 to 25) is in the second level and the last half (with red from 26 to 50) is in the third level. The VP was computed with a threshold value of 0.01 meter/s and the temporal gap between two consecutive peaks was 100 ms.The second aspect was sensitivity (refer to Figure7), which analysed the change rate of metrics from various methods with respect to the change of smoothness. From the result, it is not hard to observe that ESA was the most sensitive approach since the improvement percentages changed from 0% to around 1000% for the second level and from 2000% to about 2300%. By comparison, metrics of SAL increased from 0% to around 50% from trajectory indices of 1 to 50. As for VP, it was quite sensitive to the change of the severity in the first 15 trajectories (increased from 0% to about 100%) but less sensitive for the rest. For ZCA, the improvement percentages were very minimal. Although DJ was very sensitive in the second level, the sensitivity gradually reduced in the third level. In addition, since the smoothness decreased linearly with respect to the trajectory index, we fitted a line (orange lines) to the metrics generated by each method and computed the gradient of each line indicating the general sensitivity. This confirmed our conclusion that ESA was the most sensitive with the highest gradient.Figure 7
Sensitivity comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique is preferred to be sensitive to the small changes in the severity of involuntary movements and the change rate metric should be proportional to the change rate of severity. The blue and red lines show the improvement of metrics of the second and third severity levels with respect to the metric of the first trajectory computed with various approaches (refer to (12)). The orange lines are the regression lines of the corresponding metrics, showing the approximate improvement rate of each method. The value at the right bottom of each graph is the gradient of the regression line.Lastly, the robustness of the proposed approach was investigated. From Figure8, the performance of ESA was very close to SAL since they had a similar value (around 0.05) with the highest density. However, the range of ESA (0 to 0.75) was a little bigger than SAL (−0.1 to 0.5). By comparison, VP and DJ were not able to maintain the metrics with the influence of measurement noise since they had a large metrics range and large difference from the ground truth. Although the metric for ZCA was in a reasonable range (from 0.1 to 0.5), the offset from the ground truth was large (around 0.3).Figure 8
Robustness comparison of the five approaches with respect to the change in the severity of involuntary movements. A better evaluation technique needs to be limited to a smaller range (on the horizontal axis) in terms of the differences in metrics between the noisy trajectories and their corresponding noiseless ones, and the value with the highest amplitude in vertical axis should be close to zero.Eventually, ESA outperformed SAL as it was more sensitive to the change of smoothness and also met the requirement of dimensionless, consistency, and robustness.


## 4.3. Healthy Subjects Simulation Data Analysis

In the real-data experiment, firstly, the smoothness of all the trajectories was evaluated using the same approaches considered in the computer simulation section. Additionally, by taking the duration into consideration, all these trials were classified into three levels of ability to perform the task by using three commonly used clustering methods, namely,k-means clustering, Gaussian mixture model (GMM), and fuzzy clustering [47] so as to determine which clustering method suits the purpose of classifying motion trajectories into different levels of ability to perform upper extremity reaching tasks (severity levels of involuntary movements). Since the trials had also been classified by a human observer, a Cohen’s kappa correlation coefficient was computed between the human observer and the computer to indicate the assessment agreement. Higher coefficient values indicate a greater level of agreement between the utilised approach and the human observer. Except for the proposed method, the other four approaches, including VP, ZCA, DJ, and SAL, were used for comparison.


## 4.4. Healthy Subjects Simulation Result

Here we present the results of our preliminary real-data experiment with healthy subjects mimicking different severity levels of involuntary movements with their upper extremities while seated.First of all, some examples of simulated trajectories and features are shown in Figure9. The first three rows are the trajectories in three axes, thereby showing the submovements (with red circles) and jerks (with green rectangles) more clearly. The third and fourth rows were shape models of these trajectories with curvature and torsion, while the last row is the instantaneous acceleration. As for the columns, levels one to three (refer to Section 2.1) are in the first to the third columns. As can be observed, from the first to the third level, the number of submovements and jerks increased from 0 to 4 and 5, respectively. Correspondingly, the curvature, torsion, and instantaneous acceleration were increasingly noisy.Figure 9
Examples of trajectories (first three rows), shape models, including curvatures (fourth row) and torsions (fifth row), and instantaneous accelerations (sixth row) are illustrated for three levels of the ability to perform an upper extremity reaching task (columns one to three corresponding to levels one to three of the severity of involuntary movements). The red circles show examples of submovements and green rectangles are examples of jerks.Secondly, the distributions of various features, including the duration of the task, as well as metrics computed with various approaches, are shown in Figure10. The first box plot was the distribution of durations for three severity levels of involuntary movements. As can be seen, although there was some overlap between two consecutive levels, the interquartile ranges followed the definition of severity levels (refer to Table 1). The remaining five graphs show the metrics computed with five approaches. First of all, it is obvious that, in VP, ZCA, DJ, and ESA, the more severe involuntary movements were associated with higher values, while it was opposite in SAL (the more severe involuntary movements had lower metrics). However, this trend between the first and the second levels in DJ was very small. Secondly, although every approach showed a certain degree of overlap between two consecutive levels, DJ showed the worst situation since the metrics of its first and second levels were very similar. As for the other four approaches, VP and ZCA showed a larger overlap between the first and the second levels than SAL and ESA and VP had the largest overlap between the second the third levels. Generally speaking, the differences between these three levels of ESA were more obvious than in the other four approaches.Figure 10
The distributions of durations utilised to finish the reaching task, as well as metrics computed by five approaches, for three severity levels. The threshold utilised to compute VP was 0.25 m/s and the temporal gap was 100 ms for all subjects.Table4 shows the Cohen’s kappa between five automated approaches and the human observer generated by three clustering approaches. Generally speaking, ESA always gave the highest agreement (0.8250 for K-means and GMM, with 0.85 for fuzzy clustering). In addition, SAL and ZCA had a similar performance with kappa values slightly smaller than 0.8 in the majority of cases. By comparison, VP was a little better than DJ. But these two approaches were worse than the other three methods. Moreover, the comparison of these three clustering approaches suggested that the fuzzy clustering was more suitable for automated assessment of the severity of involuntary movements, thereby evaluating the ability to perform reaching tasks in daily living in terms of kinematics.Table 4
Cohen’s kappa (p
<
0.05) between various automated approaches and the human observer.
Approach
K-means
GMM
Fuzzy clustering
Average
VP
0.6875
0.6926
0.7250
0.7017
ZCA
0.7625
0.8000
0.7500
0.7708
DJ
0.5500
0.5631
0.5500
0.5544
SAL
0.7875
0.7875
0.7500
0.7750
ESA
0.8250
0.8250
0.8500
0.8333


## 5. Discussion and Conclusion

From the simulation and real-data experiment, the proposed approach has shown its superior performance in terms of capturing nonsmooth movement patterns due to involuntary movements during performance of a specific upper extremity task. The reasons for this are threefold. Firstly, dimensionless and duration-independent entropy was utilised as a metric for evaluation. It is well known that entropy can be utilised to analyse the regularity of variables, which makes it suitable for analysing the ability to perform a task in terms of motion smoothness because motions involving involuntary movements tend to have an irregular trajectory shape. Secondly, apart from considering the dynamics (instantaneous acceleration) of the motion trajectory, the shape of the trajectory was also taken into account, thereby not only meeting the requirement of dimensionless, consistency, and robustness but also more sensitivity than other approaches compared in the computer simulation (refer to Section4.2). Thirdly, the shape model, including the curvature and torsion, is sensitive to noise in trajectories. Therefore, the involuntary movements in motion trajectories significantly introduce noise, thereby increasing the entropy of the shape model. At the same time, by considering the instantaneous acceleration, the dynamics of the trajectories were utilised. Eventually, the trajectories containing involuntary movements were evaluated comprehensively.However, since this was a preliminary study, there are some areas that require further attention. Firstly, being an affordable device, the Microsoft Kinect is not as accurate as other more expensive commercially available products, such as VICON. Therefore, the lower resolution (especially inZ axis) hinders the Kinect in identifying small movements of human joints. In other words, jitters or tremors with minimum amplitude may not be captured by the Kinect. Therefore, at present, it can only be utilised to detect involuntary movements with relatively large amplitude. In real applications, the Kinect should be optimised so that it can detect involuntary movement of a few millimetres. However, this paper illustrates the ability of the proposed approach to evaluate a person performing reaching tasks particularly in a nonclinical environment. It is noteworthy that although the real-data experiment consisted of a seated activity involving the arms, the proposed approach is not limited to upper body movements. Other applications will be further investigated in future work. Secondly, healthy subjects, instead of patients with involuntary movements, were evaluated. Therefore, the proposed approach should be further validated and analysed with people who have involuntary movements and other movement impairments.This paper presents a novel approach for the quantitative evaluation of the ability of individuals with involuntary movements to perform reaching tasks involving the upper extremity. We based our approach on the smoothness of the movement trajectory and also the duration of the activity. In particular, the entropy of the shape model and the instantaneous acceleration were used to capture the appropriate performance indices. Experiments with computer simulation and professional role playing mimicking involuntary movements were conducted to provide preliminary validation of the feasibility and performance of using the proposed approach with an affordable Microsoft Kinect. The computer simulation showed the effectiveness of using entropy of the shape and instantaneous acceleration for motion smoothness evaluation in terms of their consistency, sensitivity, and robustness. The real-data experiment results showed that agreement (Cohen’s kappa correlation coefficient) between a human observer and the proposed automated approach with fuzzy clustering in the experiment was 0.8500, compared to 0.7250, 0.7500, 0.5500, and 0.7500 by using the number of tangential velocity peak, number of zero-crossing tangential acceleration, dimensionless jerk, and spectral arc-length, respectively. Further studies involving patients with movement disorders will be conducted in the future to validate the feasibility of the proposed method.



---

*Source: 236474-2015-06-17.xml*