# Gait Motion Classification for Neurodegenerative Diseases by Recurrence Structure Analysis

Gauthier Debes, Diwei Wang, Peter beim Graben, Frédéric Blanc, Candice Muller, Hyewon Seo, and Axel Hutt

**Objective:** Gait analysis plays a significant role in clinical assessments to discriminate neurological disorders from healthy controls, to grade disease severity, and to further differentiate dementia subtypes. **Methods:** In this paper, we propose to apply recurrence structure analysis (RSA) as a method to classify pathological gait tasks from 3D skeleton pose sequences. **Results:** For each dataset, RSA yields symbolic sequences, whose complexity reflects the subject's gait movement complexity. A new gait movement model permitted to derive novel complexity measures, which serve as classification features. Applying a Multi-Layer Perceptron classification to healthy and pathological subjects suffering from Alzheimer Disease (AD) or Dementia with Lewy Bodies (DLB) permits to distinguish subjects with regular and irregular gait and AD and DLB patients. Moreover, the performed analysis indicates that arms movement is more informative in the distinction of AD and DLB than legs movement. A final comparison to previous studies of similar data demonstrates that the proposed model-based feature classification outperforms some previous data-based feature classification methods. **Conclusion:** Model-based data features permit to discriminate patients suffering from Alzheimer Disease and Dementia with Lewy Bodies based on gait videos. **Significance:** Complexity model features derived from gait videos permit the classification of neurological patients, which outperforms some data-based feature classification methods.

**Index Terms**—Recurrent structure analysis, Pathological gait classification, 3D human pose sequence.

## I. INTRODUCTION

ALZHEIMER'S Disease (AD) and Dementia with Lewy Bodies (DLB) are two of the most prevalent neurodegenerative diseases affecting the elderly. Numerous studies have demonstrated that quantitative gait impairment analysis is a well-established method for assessing these conditions and evaluating their severity, even in the early, prodromal phases [1]. By analyzing gait patterns, clinicians can facilitate earlier detection and ensure effective monitoring of disease progression.

Gait data have been extensively used to assess patients with neurodegenerative diseases. For instance, Mannini et al.[2] applied Hidden Markov Models and Support Vector Machines to classify gaits in healthy elderly, post-stroke, and Huntington's disease patients using inertial measurement units data. Other studies focused on differentiating dementia subtypes, such as AD and DLB, leveraging data from pressure-sensitive electronic walkways to calculate key gait parameters. Muller [3] identified walking speed and step length asymmetry as key factors for distinguishing dementia subtypes using decision trees. Merory et al. [4] further demonstrated that AD and DLB patients exhibit distinct gait patterns compared to the general population, while Mc Ardle [5] observed greater step variability in DLB compared to AD.

These studies were based on observations from wearable sensors and electronic walkways. Recent studies have shown

that 3D pose sequences of AD and DLB patients can provide new insights. For instance, recent deep-learning based methods deploy neural network models specially designed to consume time-series data, such as temporal CNN (Convolutional Neural Networks)[6] or Transformer[7]. Some extract 3D pose sequences from gait videos and work on those 3D data for the pathological gait analysis[6], [7], while others develop methods that directly work on 2D videos[8].

While promising, deep learning methods require a large amount of training data with a considerable amount of computational resources. In this paper, we show that our lightweight method based on a gait model and utilizing recurrence analysis can achieve performance comparable to many state-of-the-art deep-learning models.

## II. METHODS

### A. Data

Gait movement represents a three-dimensional motion and its classification necessitates three-dimensional recordings. Since movement data from patients should be recorded without constraining their mobility to gain a realistic classification, non-invasive recording techniques are necessary, such as wearable devices [9], [10], ground reaction force [11] or videos [12]. Studies of video-sequences can extract the three-dimensional movement of body joints and analyze these data. The data in the present work originates from videos of walking subjects, whose joint locations have been reconstructed from 2D video by using the sequence of image features and the subsequently estimated gait parameters from it. The group of subjects under study comprised 50 persons yielding 133 recorded movement videos. Typically, sets of videos were recorded from the same subject, e.g. from different observation angles or while walking back and forth. These videos were taken from healthy subjects and AD patients and DLB patients. The observed subjects included 25% males and 75% females.

Gauthier Debes (e-mail: gauthier.debes@etu.unistra.fr), Diwei Wang (e-mail: d.wang@unistra.fr), Frédéric Blanc (e-mail: f.blanc@unistra.fr), Hyewon Seo (e-mail: seo@unistra.fr) and Axel Hutt (e-mail: axel.hutt@inria.fr) are with the ICube laboratory, University of Strasbourg - CNRS - Inria, 67000 Strasbourg, France.

Peter beim Graben (e-mail: Peter.beimGraben@b-tu.de) is with Bernstein Center for Computational Neuroscience, 10115 Berlin, Germany.

Frédéric Blanc and Candice Muller (e-mail: candice.muller@chru-strasbourg.fr) are with CM2R (Research and Resources Memory Centre), Geriatric Day Hospital and Neuropsychology Unit, Geriatrics Department and Neurology Service, University Hospitals of Strasbourg, 67000 Strasbourg, France.
They were elderly individuals with a mean age of 80.8 ± 8.8 years (77.3 ± 10.8 years for the males and 82.0 ± 8.0 years for the females).

These data were categorized by a physician according to a gait score and a diagnosis. The gait score was based on the MDS-UPDRS Gait III classification [13] and quantified gait impairment. It was categorized into four classes ranging from Normal: No impairment (67 videos), Slight: Independent Walking with minor gait impairment (42 videos), over Mild: Independent Walking with substantial gait impairment (14 videos) to Moderate: Requires an assistance device for safe walking (walking stick, walker) but not a person (10 videos). The diagnosis distinguished the disease severity of subjects, which were classified as Healthy (47 videos), Mild Dementia with Lewy Bodies (19 videos), Mild Alzheimer's Disease (37 videos), Severe Dementia with Lewy Bodies (12 videos) and Severe Alzheimer's Disease (18 videos), see more information in the Supplementary Material [14].

The present work merged the patient's videos into specific groups to classify these groups using the video-based data. Considering the gait score, in a first regular/irregular setting, subjects were grouped into a set of normal and non-normal walking subjects. An additional setting called GaitScoring distinguished three subject groups including normal walking subjects, subjects with slight gait impairments and all subjects with mild to moderate gait impairments.

Complementary to the gait score, we also introduced groups based on the subjects' diagnosis. The group pair AD/DLB distinguishes Alzheimer Disease and Lewy Body Disease patients, the group set Healthy/Patients permits to compare healthy subjects and patients, the group TernaryDiagnosis splits data into the three groups of healthy subjects, Alzheimer patients and Lewy Body Disease patients.

The datasets represented the position vector time series of 25 joints recorded with a video sampling rate of 30Hz. The data at each joint represented sets of points in a 3-dimensional phase space. The duration of the videos was different for each subject and comprised a diverse number of gait steps (between 2 and 19 full steps). Hence each dataset represented the time- dependent positions of 25 joints of an individual subject while walking, see Fig. 1 for illustration of the joints' positions. The

<figure><img src="image_1.png" /><figcaption>Fig. 1: **Relative location of the 25 joints.** The coordinates in the datasets were given relative to the SpineBase and thus were translational invariant in the walking person.</figcaption></figure>

joints coordinates were given relative to the root position of the walking person. This allowed to interpret the coordinates in

one, two, or three dimensions as translation-invariant position of the subject's body dependent of time.

## B. Recurrence structure analysis

Recurrence structures in high-dimensional systems have been observed for over a century, beginning with the work of Poincaré in the 19th century [15] and are well-established in the study of complex natural systems today [16]. Figure 2 illustrates an example of a multivariate time series and its corresponding phase space trajectory. Figure 2 (A) displays the two time series of a right ankle's signal on the anteroposterior (top) and craniocaudal (bottom) axes positions of a person's right ankle, representing just two dimensions of the three-dimensional position trajectory. Given that the points are sampled at constant intervals, the movement is slow, where data points are close to each other in phase space and fast, where data points are more distant [17], cf. also Fig. 2(C). In the remainder of this article, we will call the slow domains metastable states and the fast domains transient states [18]. Moreover, we point out that Fig. 2(A,C) just illustrates the phase space in two dimensions, while of course its dimension may be much larger. For instance, assuming a single joint its phase space would be three-dimensional, while all joints together would span a 3 × 25 = 75-dimensional space.

An important parameter for our RSA<fnref n="1" /> is the symbolic sequence representing metastable and transient states [18], [20], see Fig. 2(B). In such a symbolic sequence, each color represents a unique state, being either a metastable state or a transient state. All transient states are color-coded in red. Since the sequence is ordered in time, it also reflects the temporal location of each transient and each metastable state. The symbolic sequence may enter the corresponding recurrence matrix as shown in Fig. 2(D).

The resulting symbolic segmentation of the trajectory can be eventually employed to identify metastable states in the recurrence plot and in the original time series. Figure 2(c) and (d) visualizes the phase space trajectories and the recurrence matrix, respectively, with the colored symbols of the sequence. In phase space, metastable states (non-red colored) accumulate in small clusters, whereas transient states (colored in red) are more spread in phase space.

Finally, we mention that we optimized the distance $\varepsilon$ by maximising the Shannon entropy of state occurrences without normalization as in a previous study [18]. As a result of entropy maximization, the dwell times of the system within its metastable and transient domains are uniformly distributed. This ensures an optimal balance of metastability and transients and, consequently, allows for a detailed analysis of gait irregularities through complexity measures.

## C. Complexity measures

Symbolic sequences exhibit a certain complexity or regularity and multiple complexity measures are already known [21], [22], [23]. For instance, the alphabet cardinality [24], [23] $M(\varepsilon)$ counts the number of distinct states that appear in the

<footnote n="1">¹https://github.com/peterbeimgraben/rsa</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 2: Elements of the symbolic recurrence structure analysis. (A) Time series of the anteroposterior (top) and craniocaudal (bottom) axes positions of a person's right ankle (joint #18 in Fig. 1); sampling points are color-coded with respect to metastability and transients. (B) Corresponding resulting symbolic sequence. (C) Color-coded 2D trajectory of the ankle's joint with respect to anteroposterior (X) and craniocaudal (Z) coordinates. (D) Corresponding color-coded recurrence plot [19].</figcaption></figure>

sequence. Moreover, the *number of words* may also express the degree of complexity. It represents the number of distinct states with different size in the sequence. The Lempel-Ziv Complexity, $C_{Lempel-Ziv}$ [25], [26], evaluates sub-sequences in the global sequence and counts how many distinct ones exist.

The data under study comprises a set of $N_{data} = 3 \times 25 = 75$ time series reflecting the three-dimensional movement of all joints. Now a selection of certain components of certain joints, e.g. the anteroposterior and craniocaudal component of the right ankle joint as shown in Fig. 2, of number $N_{joints} \le N_{data}$ is possible and this subset of data spans a phase space of dimension $N_{joints}$. For this dataset, one estimates a certain number of complexity measures $N_{cm}$ choosing a subset of the complexity measures available. This number of complexity measures represents the dimension of the feature space of a subsequent classification procedure.

In general, it is possible to combine several subsets of joints of number $N_{subsets}$ described in respective phase spaces of different dimensions. For instance, one may choose all components of three leg joints (yielding $3 \times 3 = 9$ dimensional phase space) and of two hand joints (yielding $2 \times 3 = 6$ dimensional space), i.e. $N_{subsets} = 2$. In this example, for the leg and hand phase space one may estimate $N_{cm}^{leg} = 4$ and $N_{cm}^{hand} = 6$ complexity measures. Consequently the final data space has dimension $N_{cm}^{leg} + N_{cm}^{hand} = 10$ for each subject under study.

We point out that these measures are applicable for general symbolic sequences and are unspecific to systems. However, complexity measures taking into account specific properties of specific systems may better capture and describe the system's

complexity. This will be shown in detail in section III.

### D. Classification techniques

At the end, we have a dataset composed of $X = 342$ subsets, and for each subject, we have $N$ biomarkers that quantify the gait impairments (see section III-B). These $N$ biomarkers are used as inputs for the classification employing Multi-Layer Perceptron [27] (MLP) composed of four layers. The first three layers of the neural network employ the ReLU as activation function, while the final layer uses a softmax activation function. We trained the four-layer MLP to minimize cross-entropy loss using gradient descent, utilizing the Adamax optimizer to adjust the learning rate based on the moments of the gradients calculated through backpropagation. Finally, to evaluate the performance of our model, we use a 5-fold cross-validation [28]. Cross-validation involves partitioning the dataset into five distinct subsets. The model is trained on four of these subsets and validated on the remaining subset. This process is repeated five times, with each subset used exactly once as the validation data. The overall performance metric is obtained by averaging the results from each fold. The metric used to assess the reliability of the model is the accuracy, defined as: $A = \#(\text{Correct predictions})/\#(\text{Total data})$.

To ensure that a detected classification can be trusted to distinguish healthy subjects and different patient groups, a mandatory test controls that the classification of subjects yields different results than a random group of subjects. To generate such a surrogate group of random subjects, we randomized the labels of the data sets [29], [30]. In this procedure, we drew randomly from the set of subject labels and mapped randomly these labels to subjects. Repeating this procedure
100 times and classifying each new surrogate dataset yielded a distribution of accuracy values. To be as close as possible to the original classification, we performed 20 times a 5-fold cross-validation each time with different cross-validation folds. If an applied Mann-Whitney U test between the surrogate accuracy distribution and the original accuracy distribution led to a p-value *p* < 0.05, the originally gained classification was distinguishable significantly from a random classification and hence was more trustworthy.

### III. RESULTS

It is well-known that the size of a symbolic sequence determines the range of the derived corresponding complexity measure. Consequently, it is necessary to normalize the duration of the movement recording and we adapted this size to the single subject. This specific analysis for each single-subject takes into account the natural variability in human subjects. Novel complexity measures adapted to each subject characterize the single-subjects gait. A final classification gathering all subject's complexity measures permits to distinguish well healthy from pathological subjects and AD from DLB patients.

#### A. Patient gait as a complex recurrent pattern

The recurrence analysis features shown in Figure 2 provide a precise visualization of the gait dynamics, including the temporal locations of transient and metastable states. This method may be applied on the three-dimensional positions of one joint, or in a high-dimensional space considering multiple joints' positions moving together through time. In this latter case each time instance corresponds to a pose and the recurrence analysis provides a sequence of metastable states and transients for the entire pose. In the subsequent paragraphs, we detail this novel link between recurrent dynamics and medical gait analysis.

Considering a person walking, a regular gait corresponds to a regular recurrent symbolic sequence. However, if the person exhibits motion troubles, its symbolic sequence will be more complex, not showing recurrent patterns or many patterns of short duration. In fact, an individual affected by a neurodegenerative disease presents disorder and random moves in her/his gait. To this end, an estimation of the symbolic sequences' complexity or regularity [31], [32], [33] promises to provide biomarkers for the single subject's movement behavior.

Since conventional complexity measures depend on the symbolic sequence length, i.e. the duration of the video, we constrained the analysis by focusing on three gait movement steps only. We are using the maximum and minimum peaks of the ankle's anteroposterior axis position (right or left does not change anything) to spot the start and the end of the steps (see the Supplementary Material [14] for illustration). In fact, since the positions of the joints are relative to the position of the root, considering the anteroposterior axis implies a highest ankle position while the foot is attacking the floor. This is the beginning of the so-called stance phase. Correspondingly the minimum ankle reflects the position, where the foot is leaving the floor. This is the beginning of the swing phase. Using the peaks of the ankle's position we can segment the data to keep four peaks corresponding to three steps. Hence the

datasets analyzed include identical numbers of steps, yielding a normalization of subsequently estimated complexity measures. This segmentation lead to 342 datasets in total.

The normalization provides the opportunity to evaluate whether a gait is regular. Healthy and regular gait imply that one metastable state should appear in the symbolic sequence in each step. If a state appears more or less often than the number of steps, an irregularity is present. As a consequence, for three steps three appearances of each metastable states should be expected. An exception occurs if a metastable state appears at both the beginning and the end of the three-step segmentation. This means that the metastable state corresponds to the area where the ankle's anteroposterior position is at the peak used to normalize the three steps. In this case, the corresponding metastable state should appear four times.

#### B. Biomarkers

Now we introduce several different complexity measures, which serve as biomarkers for the gait sequence:

* As stated before, a regular gait exhibits a single metastable state in each step and we evaluate the degree of regularity as the difference between the number of metastable states' appearances $n_k$ in segment $k$ and the number of steps. Then, the average over all metastable states $n$ yields $C_{\text{appearance}} = \frac{\sum_{k=1}^{N} |n_k - 3|}{3n}$. For illustration, also see the Supplementary Material [14].

* The previous complexity measure estimates the deviation of the optimal number of metastable states. The symbolic sequence of a perfectly regular gait segmented in three steps is supposed to contain three times the same states' pattern. Now we quantify the states' pattern and count the number of different sub-sequences with only distinct metastable states. For a perfectly regular gait, the sequence exhibits three times a unique sub-sequence and the corresponding complexity measure should be equal to 1. The resulting number of sub-sequences $C_1 = \#(\text{Sub-sequences})$ is an additional complexity measure. New sub-sequences start with a metastable state only and transient states at the beginning are neglected. In addition, the first state at the end of a sequence is neglected. For illustration, also see the Supplementary Material [14].

* Extending the previous complexity measure, one searches for the longest sub-sequence that occurs more than once in the symbolic sequence. Once identified, this sub-sequence is removed from the sequence. Then, the algorithm aims to find the second longest recurrent sub-sequence, and continues in this manner. When only non-recurrent sub-sequences remain, the algorithm identifies the longest sub-sequence composed only of distinct metastable states, neglecting transient states. This iterative process continues until no symbols remain in the symbolic sequence. Similar to the algorithm of the previous complexity measure $C_1$, the first transient state and the last metastable state are removed, if necessary, yielding another complexity measure, $C_2 = \#(\text{Iterated Sub-sequences})$. For more details and illustration, also see the Supplementary Material [14].
* Metastable states may occur regularly indicating a certain regularity of the full sequence. We calculate the deviation of the sequence position distance between the occurrences of the states. To this end, we segment the symbolic sequence step by step and keep only those metastable states, which appear only once in each step. Considering the center sequence position of each state, computing the distance between the three center positions allows to compute the average deviation of the center positions. For illustrative examples, please see the Supplementary Material [14].

* It is known that neurodegenerative disease patients show gait impairment, such as slow walk steps, irregularities during the stance or the swing phase, or a small amplitude of arm movement [5]. We can easily identify these characteristics thanks to the joints position. Since we segmented the data in three steps, we defined the step period as the number of data time steps per segment, normalized by the leg's length, $C_{\text{step period}} = \#(\text{data points})/3 \times \text{step length}$.

* By considering the ankle joint position on the anteroposterior axis, we can determine the beginning and the end of the swing and stance phases. Consequently, we estimate the deviation of the number of data points between the minimum and maximum peaks $C_{\text{swing}}$ to quantify the irregularity of the swing phase.

* Similarly, the deviation $C_{\text{stance}}$ of the number of data points between the maximum and minimum peaks in the stance phase quantifies its irregularity.

* Finally, by calculating the mean of the maximum distance between two positions of the wrist joint at each step, we quantify the mean maximum amplitude of the arms $C_{\text{arms}}$.

<figure><img src="image_3.png" /><figcaption>Fig. 3: **Classification results for one fold.** Left: The gait score distinguishes subjects with regular from irregular gait, i.e. neurotypicals from patients. Right: The diagnosis classification differentiates individual subjects suffering from Alzheimer's disease (AD) and Dementia with Lewy Bodies (DLB). The numbers denote the number of trials in each class out of 68 (left) and 64 (right) datasets. The data considered four body parts, cf. dataset 4BPs in Fig. 4.</figcaption></figure>

In sum, the first four biomarkers are complexity measures, which evaluate symbolic sequences. The last four biomarkers

are gait characteristics estimated from the joint trajectories. Together with the alphabet cardinality, the Number of Words and the Lempel-Ziv complexity (see section III-B), eleven biomarkers reflect different degrees of the data complexity (see the Supplementary Material [14]).

### C. Classification

In sum, we consider 11 biomarkers including 7 complexity measures and 4 gait characteristics measures. These biomarkers permit to classify the data of individuals assuming two different criteria. The first criterion focuses on gait impairment. It distinguishes healthy subjects with regular gait and subjects with gait impairments. This gait impairment merges all MDS-UPDRS Gait III scores, cf. section II-A). The second criterion differentiates the diagnosis of individual AD and DLB patients. The classification is performed for each criterion separately using a supervised learning algorithm with the biomarkers as inputs, see section II-D. The outputs are classes corresponding to the scores for either the gait score or the diagnosis.

At first, we considered the three-dimensional positions of multiple joints in space to analyze the gait of an individual and identify, if present, recurrent patterns. After supervised learning and subsequent cross-validation, we gained the confusion matrix to distinguish binary gait scores. The question now is: which joints do we consider, and how do we combine them to extract the most relevant gait classification? Since the movement of hands, fingers, and feet tips appear to be rather irregular even in healthy individuals (analysis not shown), we decided to exclude them from the analysis. Additionally, the head, neck, and entire spine are rather static and do not contribute any relevant information on the gait movements (analysis not shown). Therefore, we analyze only the movements of both legs (ankle, knee, and hip) and both arms (wrist, elbow, and shoulder) joints in agreement with previous studies [34] (see Figure 1).

Moreover, multiple classification tests (not shown) indicated that the more input information is considered, the better the classification results are. However, analyzing joint by joint to have a set of measures for each joint is time consuming. Moreover, the movement of certain joints alone is not relevant for the general gait's dynamic. Thus, the best compromise was to perform the analysis on the four body parts (2 legs and 2 arms) independently. For the legs, we estimated the seven complexity measures and the three gait characteristics specific to the legs (step speed, occurrence deviation of stance, and swing phase lengths). This represented ten measures per leg. For the arms, we estimated six complexity measures only and added the arm amplitude for gait characteristics. In fact, $C_{\text{deviation}}$ works better with the legs because they move more regularly than the arms. This resulted in 7 measures per arm. Consequently, we had 34 measures in total when considering the analysis of the four body parts separately, which we then merged into one dataset. With these 34 measures, we could classify the gait score (regular or impaired gait) for all subject groups. The cross-validation resulted to three folds of 68 data sets and two folds of 69 datasets. In details, we have 277
irregular datasets against 65 regular datasets. Consequently, we add a weight of 4 on the regular class during the classification to compensate the lack of data of this class. For the diagnosis classification (distinguishing data from AD and DLB patients), we exclude the healthy datasets, leaving us with 320 datasets resulting in 64 datasets per fold. We have here 139 DLB datasets against 181 AD datasets. In Fig. 3 below, we can observe an example of a confusion matrix for one fold of each classification problem (gait score and diagnosis). We observe a clear distinction between regular and irregular gait (Fig. 3(left)) and between AD and DLB patients (Fig. 3(right)).

Figure 4 shows a comparison between the mean classification accuracy according to different joint combinations for the RSA. At first, we observe that just three combinations of body parts yield a *regular/irregular* classification, which is significantly different from random classification (red stars). These combinations utilize four body parts separately (*4BPs*), two legs separately (*2Ls*) and two legs in one dataset (*2Lo*) and these also yield the best *regular/irregular* classification results. In the *AD/DLB* classification, all but two combinations yield non-random classifications. These two exceptions utilize two legs in one dataset (*2Lo*) and all joints simultaneously (*All*) and they also yield the lowest classification accuracy. For all tests on random classifications, see [14].

Moreover, utilizing the four body parts (*left/right arm* and *left/right leg*) analyzed separately (*4BPs*) yields the best results for both classification problems (colored light blue), cf. also Fig. 3. More precisely, this combination yields a mean cross-validation accuracy of 85.5% for the gait score classification and a mean cross-validation accuracy of 75.6% for the diagnosis classification. These results are significantly different to all other results (marked as black star).

The previous analysis considered recordings from both arms and legs joints yielding good classification accuracy. This raised the question whether arms and legs decode different gait features and thus reflect specific gait patterns. To this end, we grouped arm joints and leg joints together and classified subjects. In fact, Table I shows that just considering legs classified the gait score significantly better than with arm joints only (*p* < 0.05). Conversely, utilizing arms led to a better accuracy in the diagnosis classification (*p* < 0.05).

<table><caption>TABLE I: Classification based on arms and legs. The data comprised 5×2 = 10 classification accuracy values from the two arms datasets (*Ao* and *As*) and the two legs datasets (*Lo* and *Ls*) for the Gait score (*regular/irregular*) and diagnosis (*AD vs. DLB*) classifications. The p-values were estimated by two-sided Mann-Whitney U tests and indicated statistically different classification results between data utilizing arms and legs.</caption><thead><tr><th>regular/irregular</th><th>Median accuracy (in %)</th><th>p-value</th></tr></thead><tbody><tr><td>Arms</td><td>73.91 (±9.73)</td><td rowspan="2">0.005</td></tr><tr><td>Legs</td><td>80.56 (±4.82)</td></tr></tbody></table><table><thead><tr><th>AD vs DLB</th><th>Median accuracy (in %)</th><th>p-value</th></tr></thead><tbody><tr><td>Arms</td><td>69.13 (±5.68)</td><td rowspan="2">0.005</td></tr><tr><td>Legs</td><td>62.50 (±8.20)</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Fig. 4: Mean classification accuracy for multiple joints combinations. Left: Gait score classification. Right: Diagnosis classification. *BP*: body parts (arms and/or legs); *A*: arms; *L*:legs; *s*:separated; *o*:in one. The term *separated* indicates that we analyzed the body parts in separate phase spaces and then merged all the complexity measures into one dataset. The term *in one* describes the approach to perform the recurrence analysis considering all the joints of the body parts at the same time. *All* represents the results for the recurrence analysis of all 25 joints simultaneously. The black stars indicate that the corresponding joints combination is significantly different from all the others (two-sided Mann-Whitney U test, *p* < 0.05). The red stars denote the combinations, whose classification was significantly different from a random classification (two-sided Mann-Whitney U test, *p* < 0.05). All surrogate tests are given in [14].</figcaption></figure>

## D. Importance of biomarkers

All biomarkers quantify the regularity of the subjects' gait. To understand better which of these biomarkers reflect most the gait regularity, we performed an ablation feature importance test[35] on both classification criteria. To this end, we considered four body parts (*4BPs* in Fig. 4 utilizing 11 biomarkers) and estimated the classification accuracy removing one feature. Table II indicates that the feature *C*<sub>step period</sub> considering the inverse gait speed is important in both the *regular/irregular* and *AD/DLB* classification. Moreover, the feature *C*<sub>deviation</sub> encoding the gait complexity is more important in *regular/irregular* than in *AD/DLB*, whereas the arm feature *C*<sub>arms</sub> is much more important in *AD/DLB* than in *regular/irregular*.

## E. Comparison to previous studies

We conducted three distinct classification tasks on the *4BPs* dataset to provide a qualitative comparison with previous studies on pathological gait analysis. These tasks in-
<table><thead><tr><th colspan="4">regular/irregular</th></tr><tr><th>Rank</th><th>Feature</th><th>ΔA (in %)</th><th>Accuracy (in %)</th></tr></thead><tbody><tr><td>1</td><td>C<sub>step period</sub></td><td>2.44</td><td>85.58</td></tr><tr><td>4</td><td>C<sub>deviation</sub></td><td>1.32</td><td>86.70</td></tr><tr><td>10</td><td>C<sub>arms</sub></td><td>0.55</td><td>87.47</td></tr><tr><td colspan="2">Reference accuracy</td><td colspan="2">88.02%</td></tr></tbody></table>

<table><caption>TABLE II: **Feature ablation test.** Selected results of a feature ablation test utilizing dataset 4BPs (cf. Fig. 4). This test removes single features and computes the accuracy difference of a single feature *i* defined by $ΔA_i = A_{ref} - A_{without feature i}$. This difference and reflects the feature importance. The larger $ΔA$ is the lower is the resulting accuracy and the more important is the feature.</caption><thead><tr><th colspan="4">AD/DLB</th></tr><tr><th>Rank</th><th>Feature</th><th>ΔA (in %)</th><th>Accuracy (in %)</th></tr></thead><tbody><tr><td>1</td><td>C<sub>arms</sub></td><td>2.54</td><td>72.61</td></tr><tr><td>2</td><td>C<sub>step period</sub></td><td>1.76</td><td>73.39</td></tr><tr><td>11</td><td>C<sub>deviation</sub></td><td>-1.21</td><td>76.36</td></tr><tr><td colspan="2">Reference accuracy</td><td colspan="2">75.15%</td></tr></tbody></table>

<table><caption>TABLE III: **Comparison of performance with other methods.** The performance of our method across various classification tasks is presented along with comparable previous studies. The given performance represents the median accuracy obtained through cross-validation. Previous methods were VIBE: VIBE [36] + OF-DDNet [6]; MAX-GRNet 1: MAX-GRNet [7] + ST-GCN [37]; MAX-GRNet 2: MAX-GRNet [7] + KShapeNet [38].</caption><thead><tr><th>Methods</th><th>Healthy/Patients</th><th>Diagnosis</th><th>Gait</th></tr></thead><tbody><tr><td>VIBE</td><td>89.6</td><td>72.78</td><td>60.43</td></tr><tr><td>MAX-GRNet 1</td><td>94.24</td><td>66.67</td><td>72.31</td></tr><tr><td>MAX-GRNet 2</td><td>96.22</td><td>75.39</td><td>65.41</td></tr><tr><td>ours</td><td><b>93.85</b></td><td><b>70.16</b></td><td><b>66.01</b></td></tr></tbody></table>

include: distinguishing between healthy subjects and patients (Healthy/Patients); differentiating diagnostic groups of healthy controls, Alzheimer's Disease, and Dementia with Lewy Bodies patients (TernaryDiagnosis); assessing gait impairment severity among subjects based on MDS-UPDRS Gait III scores, categorized into no impairment, slight impairment, and mild to moderate impairment (GaitScoring). See also section II-A for more data details.

Our method achieved a notably high accuracy of 93.85% in the Healthy/Patients classification, whereas the other tasks of TernaryDiagnosis and GaitScoring exhibited less accuracy (70.16% and 66.01%, respectively). These results are comparable to previous deep learning-based approaches developed for similar tasks, whose results are presented in Table III. Note however that all other methods perform the classification based on subsequences cutted from videos, mainly due to computational resource limitation, and the classification accuracies are calculated by averaging the validation results across 10 folds.

## IV. DISCUSSION

A smooth regular gait indicates a healthy body movement, while irregular body movement is supposed to reflect a pathological motor state. The proposed RSA permits to derive complexity measures, which are optimized by the best choice of the ball size $ε$. The complexity measures decode the

regularity of body movements together with newly developed gait-specific movement features. We showed that both the complexity measures based on recurrent symbolic dynamics and gait movement contribute to the classification (Fig. II) and hence carry important information on the body movement.

Moreover, the present work employs the recurrence complexity measures and the gait-relevant movement features in a feature space. This high-dimensional feature space permits to classify subjects by machine learning. A similar combination of recurrence analysis and machine learning classification has recently been employed in [39]. Our method permits to choose combinations of different joints, which revealed optimal, i.e. most informative, sets of joints, cf. Fig. 4. In this context, surrogate data analysis identified the non-informative joint combinations, whose classification can not be distinguished from random classification.

The importance of complexity in human motor action has been shown in previous studies by recurrence analysis techniques in non-clinical setups [40]. Complexity has also been estimated without utilizing recurrence analysis to quantify the regularity of gait movement in clinical settings without [41], [42] and by utilizing recurrence analysis [43], [44]. In most clinical studies, the gait is observed by feet movement. Few previous studies took into account observations from distant body parts [45], [46] and applied recurrence analysis. To the best of our knowledge, all such previous studies based on recurrence analysis chose a heuristic ball size $ε$ independently from the signal dynamics. Our recurrence structure analysis extended these studies by optimising the ball size specific for each dataset.

The proposed analysis extracts biomarkers, which span a high-dimensional feature space. This feature space is specific for the observations gained during human gait movement. By virtue of these observations of the body movement and the system-specific feature space, we found that arms movement is important to distinguish Alzheimer and Lewy Body Disease, but much less informative to differentiate regular from irregular gait movement, cf. Table I and Table II.

In fact, it is well-known that the arms movement is an informative feature of AD patients [47]. In addition, AD and DLB patients are known to show different movement asymmetries and degrees of movement variability [5]. The present work adds to these results the hypothesis, that arms movement may represent a further movement feature supporting the distinction between AD and DLB patients.

The proposed method assumes a model of regular gait based on previous physiological knowledge of healthy subjects. Corresponding recurrence complexity measures and gait timing indices quantify the gait of healthy subjects. These biomarkers are distinguished to pathological gait yielding the classification of patients. This approach contrasts with previous deep-learning techniques, which are purely based on video sequences without assuming any gait model. The performance of our proposed model-based classification resembles the deep-learning classification, while it is faster in computation and consumes much less resource power.

The employed machine learning classification used a multi-layer perceptron as supervised learning technique. This spe-
cific choice may have limited the results. Future work will
explore whether alternative machine learning techniques, such
as more complex model architectures based on advanced deep
learning algorithms, can improve classification performance
compared to the current four-layer MLP approach.

The present work points out, that both body movement
complexity features and gait-specific movement features repre-
sent biomarkers and contribute to the classification of healthy
subjects and patients. Moreover, the data represent a combined
set of joints of free choice. The final subject classification
depends heavily on the choice of these biomarkers and the
choice of joints. Future work will further analyze in detail,
which features and which joint combinations are essential for
the classification. This analysis will yield deeper insights into
the importance of body parts in the distinction of AD and DLB
patients.

V. CONCLUSION

Data features extracted from 3D pose video-observations
permit to distinguish Alzheimer Disease patients and patients
suffering from Dementia with Lew Bodies. We show that
gait model-based feature classification utilizing recurrence
structure analysis performs similar to high-performance deep-
learning classification based on the observation times series.
Our model-based approach reveals that arms and legs con-
tribute differently to the classification of Alzheimer Disease
and Dementia with Lewy Bodies patients.

ACKNOWLEDGMENT

This work of the Interdisciplinary Thematic Institute HealthTech, as part of the ITI 2021-2028 program of the University of Strasbourg, CNRS and Inserm. Further it was supported by IdEx Unistra (ANR-10-IDEX-0002) and SFRI (STRAT'US project, ANR-20-SFRI-0012) under the framework of the French Investments for the Future Program.

REFERENCES

[1] I. McKeith, T. Ferman, A. Thomas, F. Blanc, B. Boeve, H. Fujishiro, K. Kantarci, C. Muscio, J. O'Brien, R. Postuma, D. Aarsland, C. Ballard, L. Bonanni, P. Donaghy, M. Emre, J. Galvin, D. Galasko, J. Goldman, S. Gomperts, L. Honig, M. Ikeda, J. Leverenz, S. Lewis, K. Marder, M. Masellis, D. Salmon, J. Taylor, D. Tsuang, Z. Walker, and P. Tira-boschi, "Research criteria for the diagnosis of prodromal dementia with lewy bodies," *Neurology*, vol. 94, no. 17, pp. 743–755, 2020.

[2] A. Mannini, D. Trojaniello, A. Cereatti, and A. M. Sabatini, "A machine learning framework for gait classification using inertial sensors: Application to elderly, post-stroke and huntington's disease patients," *Sensors*, vol. 16, no. 1, p. 134, 2016.

[3] C. Muller, J. Perisse, F. Blanc, M. Kiesmann, C. Astier, and T. Vogel, "Corrélation des troubles de la marche au profil neuropsychologique chez les patients atteints de maladie d'alzheimer et maladie à corps de lewy," *Revue Neurologique*, vol. 174, pp. S2-S3, 2018.

[4] J. Merory, J. Wittwer, C. Rowe, and K. Webster, "Quantitative gait analysis in patients with dementia with lewy bodies and alzheimer's disease," *Gait & posture*, vol. 26, pp. 414–9, 10 2007.

[5] R. Mc Ardle, B. Galna, P. Donaghy, A. Thomas, and L. Rochester, "Do alzheimer's and lewy body disease have discrete pathological signatures of gait?" *Alzheimer's & Dementia*, vol. 15, no. 10, pp. 1367–1377, 2019.

[6] M. Lu, K. Poston, A. Pfefferbaum, E. Sullivan, L. Fei-Fei, K. Pohl, J. Niebles, and E. Adeli, "Vision-based estimation of md5-updrs gait scores for assessing parkinson's disease motor severity," in *International Conference on Medical Image Computing and Computer-Assisted Intervention*. Springer, 2020, pp. 637–647.

[7] D. Wang, C. Zouaoui, J. Jang, H. Drira, and H. Seo, "Video-based gait analysis for assessing alzheimer's disease and dementia with lewy bodies," *Lect. Notes Comp. Sci.*, vol. 313, pp. 72–82, 2024.

[8] D. Wang, K. Yuan, and H. Seo, "Enhancing gait video analysis in neu-
rodegenerative diseases by knowledge augmentation in vision language
model," *Lect. Notes Comp. Sci.*, vol. XX, pp. XX–XX, 2024.

[9] A. Borbély, T. Rusterholz, and P. Achermann, "Three decades of continuous wrist-activity recording: analysis of sleep duration," *Journal of Sleep Research*, vol. 26, no. 2, pp. 188–194, 2017.

[10] H. Terashi, T. Taguchi, Y. Ueta, Y. Okubo, H. Mitoma, and H. Aizawa, "Analysis of non-invasive gait recording under free-living conditions in patients with parkinson's disease: relationship with global cognitive function and motor abnormalities," *BMC Neurol.*, vol. 20, no. 1, p. 161, 2020.

[11] C. Pandey, D. Roy, R. Poonia, A. Altameem, S. Nayak, A. Verma, and A. Saudagar, "Gaitrec-net: A deep neural network for gait disorder detection using ground reaction force," *PPAR Res.*, vol. 2022, p. 9355015, 2022.

[12] J. Stenum, M. Hsu, A. Pantelyat, and R. Roemmich, "Clinical gait analysis using video-based pose estimation: Multiple perspectives, clinical populations, and measuring change," *PLoS Digital Health*, vol. 3, no. 3, p. e0000467, 2024.

[13] C. Goetz, B. Tilley, S. Shaftman, G. Stebbins, S. Fahn, P. Martinez-Martin, W. Poewe, C. Sampaio, M. Stern, R. Dodel, B. Dubois, R. Holloway, J. Jankovic, J. Kulisevsky, A. Lang, A. Lees, S. Leurgans, P. LeWitt, D. Nyenhuis, C. Olanow, O. Rascol, A. Schrag, J. Teresi, J. van Hilten, and N. LaPelle, "Revision of the unified parkinson's disease rating scale (mds-updrs): Scale presentation and clinimetric testing results," *Mov. Dis.*, vol. 23, no. 15, pp. 2129-2170, 2008.

[14] "Supplementary material," https://github.com/axelhutt/publications/blob/main/IEEE_TBME_Debes_etal_Supplementary_Material.pdf.

[15] H. Poincaré, "Sur le problème des trois corps et les équations de la dynamique," *Acta Mathematica*, vol. 13, pp. 1 – 270, 1890.

[16] N. Marwan, M. Romano, M. Thiel, and J. Kurths, "Recurrence plots for the analysis of complex systems," *Phys. Rep.*, vol. 438, no. 5-6, p. 237, 2007.

[17] A. Hutt, M. Svensen, F. Kruggel, and R. Friedrich, "Detection of fixed points in spatiotemporal signals by a clustering method," *Phys. Rev. E*, vol. 61, no. 5, pp. R4691–R4693, 2000.

[18] P. beim Graben and A. Hutt, "Detecting recurrence domains of dynamical systems by symbolic dynamics," *Phys. Rev. Lett.*, vol. 110, p. 154101, 2013.

[19] P. Faure and A. Lesne, "Recurrence plots for symbolic sequences," *Int. J. Bifurc. Chaos*, vol. 20, p. 1731, 2010.

[20] P. beim Graben and A. Hutt, "Detecting event-related recurrences by symbolic analysis: applications to human language processing," *Phil. Trans. R. Soc. A*, vol. 373, p. 20140089, 2015.

[21] C. Adami and N. Cerf, "Physical complexity of symbolic sequences," *Physica D*, vol. 137, no. 1-2, pp. 62–69, 2000.

[22] F. Von Wegner, M. Wiemers, G. Hermann, I. Tödt, E. Tagliazucchi, and H. Laufs, "Complexity measures for eeg microstate sequences: Concepts and algorithms," *Brain Topogr.*, vol. 37, pp. 296–311, 2024.

[23] P. beim Graben, K. Sellers, F. Fröhlich, and A. Hutt, "Optimal estimation of recurrence structures from time series," *Europhys. Lett.*, vol. 114, no. 3, p. 38003, 2016.

[24] S. Janson, S. Lonardi, and W. Szpankowski, "On average sequence complexity," *Theoret. Comp. Sci.*, vol. 326, pp. 213–227, 2004.

[25] J. Ziv and A. Lempel, "A universal algorithm for sequential data compression," *IEEE Transactions on Information Theory*, vol. 22, pp. 75–81, 1976.

[26] A. Lempel and J. Ziv, "On the complexity of finite sequences ieee transactions on information theory," *IEEE Transactions on Information Theory*, vol. 23, pp. 337–343, 1977.

[27] C. Bishop, *Pattern Recognition and Machine Learning*. New York: Springer. ISBN 978-0-387-31073-2., 17 August 2006.

[28] P. Refaeilzadeh, L. Tang, and H. Liu, *Cross-Validation*. In: LIU, L., ÖZSU, M.T. (eds) Encyclopedia of Database Systems. Springer, Boston, MA., 2009.

[29] J. Theiler, S. Eubank, A. Longtin, B. Goldrikian, and J. Farmer, "Testing for nonlinearity in time series: the method of surrogate data," *Physica*, vol. D 58, pp. 77–94, 1992.

[30] P. beim Graben, S. Frisch, A. Fink, D. Saddy, and D. Kurths, "Topographic voltage and coherence mapping of brain potentials by means of the symbolic resonance analysis," *Physical Reviews E*, vol. 72, p. 051916, 2005.
[31] V. Gusev, L. Nemytikova, and N. Chuzhanova, "On the complexity measures of genetic sequences," *Bioinformatics*, vol. 15, no. 12, pp. 994-999, 1999.

[32] P. Diamond, P. Kloeden, V. Kozyakin, and A. Pokrovskii, "On the fragmentary complexity of symbolic sequences," *Th. Comp. Sci.*, vol. 148, pp. 1-17, 1995.

[33] F. Von Wegner, M. Wiemers, G. Hermann, I. Tödt, E. Tagliazucchi, and H. Laufs, "Complexity measures for eeg microstate sequences: Concepts and algorithms," *Brain Topogr.*, vol. 37, pp. 296-311, 2024.

[34] N. Mizuta, N. Hasui, T. Kai, Y. Inui, M. Sato, S. Ohnishi, J. Taguchi, and T. Nakatani, "Characteristics of limb kinematics in the gait disorders of post-stroke patients," *Sci. Rep.*, vol. 14, p. 3082, 2024.

[35] L. Merrick, "Randomized ablation feature importance," *arXiv:1910.00174*, October 3, 2019.

[36] M. Kocabas, N. Athanasiou, and M. Black, "Vibe: Video inference for human body pose and shape estimation," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 5253-5263.

[37] A. Sabo, S. Mehdizadeh, A. Iaboni, and B. Taati, "Estimating parkinsonism severity in natural gait videos of older adults with dementia," *IEEE journal of biomedical and health informatics*, vol. 26, no. 5, pp. 2288-2298, 2022.

[38] R. Friji, H. Drira, F. Chaieb, H. Kchok, and S. Kurtek, "Geometric deep neural network using rigid and non-rigid transformations for human action recognition," in *Proceedings of the IEEE/CVF international conference on computer vision*, 2021, pp. 12611-12620.

[39] T. Chomiak, N. P. Rasiah, L. A. Molina, B. Hu, J. S. Bains, and T. Füzesi, "A versatile computational algorithm for time-series data analysis and machine-learning models," *npj Parkinson's Disease*, vol. 7, no. 1, p. 97, 2021.

[40] Z. Laudańska, D. López Pérez, A. Radkowska, K. Babis, A. Malinowska-Korczak, S. Wallot, and P. Tomalski, "Changes in the complexity of limb movements during the first year of life across different tasks," *Entropy (Basel)*, vol. 24, no. 4, p. 552, 2022.

[41] T. Warlop, C. Detrembleur, G. Stoquart, T. Lejeune, and A. Jeanjean, "Gait complexity and regularity are differently modulated by treadmill walking in parkinson's disease and healthy population." *Front. Physiol.*, vol. 9, p. 68, 2018.

[42] D. Nohelova, L. Bizovska, N. Vuillerme, and Z. Svoboda, "Gait variability and complexity during single and dual-task walking on different surfaces in outdoor environment." *Sensors*, vol. 21, p. 4792, 2021.

[43] O. Afsar, U. Tirnakli, and N. Marwan, "Recurrence quantification analysis at work: Quasi-periodicity based interpretation of gait force profiles for patients with parkinson disease," *Sci. Rep.*, vol. 8, p. 9102, 2018.

[44] M. Fedotenkova, P. beim Graben, J. W. Sleigh, and A. Hutt, "Time-frequency representations as phase space reconstruction in symbolic recurrence structure analysis," in *Advances in Time Series Analysis and Forecasting: Selected Contributions from ITISE 2016*, I. Rojas, H. Pomares, and O. Valenzuela, Eds. Springer, 2017, pp. 89-102.

[45] F. Labini, A. Meli, Y. Ivanenko, and D. Tufarelli, "Recurrence quantification analysis of gait in normal and hypovestibular subjects," *Gait & Posture*, vol. 35, no. 1, pp. 48-55, 2012.

[46] C. Kao, B. Liau, and F. Kuo, "Measures of gait complexity during the timed up-and-go test in older adults with vertebral compression fracture," *J. Med. Biol. Eng.*, pp. 2199-4757, 2024.

[47] B. Ott, S. Ellias, and M. Lannon, "Quantitative assessment of movement in alzheimer's disease," *J. Geriatr. Psychiatry Neurol.*, vol. 8, no. 1, pp. 71-75, 1995.