# Improved markerless gait kinematics measurement using a biomechanically-aware algorithm with subject-specific geometric modeling

Mehran Hatamzadeh, Laurent Busé, Katia Turcot, Raphael Zory

► To cite this version:

Mehran Hatamzadeh, Laurent Busé, Katia Turcot, Raphael Zory. Improved markerless gait kinematics measurement using a biomechanically-aware algorithm with subject-specific geometric modeling. Measurement - Journal of the International Measurement Confederation (IMEKO), 2024, 234, pp.114857. 10.1016/j.measurement.2024.114857. hal-04574589

HAL Id: hal-04574589

https://hal.science/hal-04574589

Submitted on 14 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Improved markerless gait kinematics measurement using a biomechanically-
aware algorithm with subject-specific geometric modeling

Mehran Hatamzadeh a, b *, Laurent Busé b, Katia Turcot c, Raphael Zory a, d

a Université Côte d'Azur, LAMHESS, Nice, France

b Université Côte d'Azur, Inria, Sophia Antipolis, France

c Université Laval, Department of Kinesiology, Faculty of Medicine, Centre interdisciplinaire de recherche en
réadaptation et intégration sociale (Cirris), Quebec, Canada

d Institut Universitaire de France (IUF), Paris, France

* Corresponding author: Mehran Hatamzadeh

Address: Université Côte d'Azur, Campus STAPS – Sciences du Sport, 261 Boulevard du Mercantour, 06205 Nice, France.

E-mail: mehran.hatamzadeh@inria.fr, hatamzadeh.mehran@gmail.com (M. Hatamzadeh).

**Abstract:**

Despite the advancements in developing markerless gait analysis systems, they still demonstrate lower
accuracy compared to gold-standard systems. Hence, in this research, a novel approach is presented to
improve the lower limb kinematics accuracy in markerless gait analysis. This approach refines the 3D
lower-limb skeletons obtained by AI-based pose estimation algorithms in a subject-specific geometric
manner, preserves skeleton links' length, benefits from gait phases information that adds
biomechanical awareness to the algorithm, and utilizes an embedded trajectory smoothing. Validation
of the proposed method shows that it reduces 12.6%-43.5% of root mean square error (RMSE) and
significantly improves kinematic curves' similarity to the gold-standard ones. Results also prove the
feasibility of more accurate lower limb kinematics calculation using a single (2.02°-7.57° RMSE) or
dual RGB-D camera (1.66°-7.25° RMSE). Development of such algorithms could result in
requirement of fewer cameras that deliver comparable or even superior measurement accuracy
compared to multi-camera approaches.

**Keywords:** Markerless gait analysis, Improved kinematics, Geometric lower limb model, Refined pose estimation, Stance phase constraints, RGB-D camera
Improved markerless gait kinematics measurement using a biomechanically-
aware algorithm with subject-specific geometric modeling

Mehran Hatamzadeh a, b *, Laurent Busé b, Katia Turcot c, Raphael Zory a, d

a Université Côte d'Azur, LAMHESS, Nice, France

b Université Côte d'Azur, Inria, Sophia Antipolis, France

c Université Laval, Department of Kinesiology, Faculty of Medicine, Centre interdisciplinaire de recherche en
réadaptation et intégration sociale (Cirris), Quebec, Canada

d Institut Universitaire de France (IUF), Paris, France

* Corresponding author: Mehran Hatamzadeh

**Address:** Université Côte d'Azur, Campus STAPS – Sciences du Sport, 261 Boulevard du Mercantour, 06205 Nice, France.

E-mail: mehran.hatamzadeh@inria.fr, hatamzadeh.mehran@gmail.com (M. Hatamzadeh).

**Abstract:**

Despite the advancements in developing markerless gait analysis systems, they still demonstrate lower
accuracy compared to gold-standard systems. Hence, in this research, a novel approach is presented to
improve the lower limb kinematics accuracy in markerless gait analysis. This approach refines the 3D
lower-limb skeletons obtained by AI-based pose estimation algorithms in a subject-specific geometric
manner, preserves skeleton links' length, benefits from gait phases information that adds
biomechanical awareness to the algorithm, and utilizes an embedded trajectory smoothing. Validation
of the proposed method shows that it reduces 12.6%-43.5% of root mean square error (RMSE) and
significantly improves kinematic curves' similarity to the gold-standard ones. Results also prove the
feasibility of more accurate lower limb kinematics calculation using a single (2.02°-7.57° RMSE) or
dual RGB-D camera (1.66°-7.25° RMSE). Development of such algorithms could result in
requirement of fewer cameras that deliver comparable or even superior measurement accuracy
compared to multi-camera approaches.

**Keywords:** Markerless gait analysis, Improved kinematics, Geometric lower limb model, Refined
pose estimation, Stance phase constraints, RGB-D camera
# 1. Introduction:

Gait analysis is a clinical assessment that provides quantitative data on various gait parameters, utilizable to diagnose walking impairments. Gold-standard systems for this purpose are 3D marker-based motion analysis (3DMA) systems [1]. However, technological progression has led to the emergence and development of markerless approaches with fewer drawbacks compared to 3DMAs [2-5]. Expensiveness, unportability, and large dedicated space requirement are drawbacks of 3DMAs, whereas the markerless systems do not offer them, which subsequently increases their deployment possibility in a broader array of clinics [6,7]. Nevertheless, a notable downside in them is their lower accuracy in estimating gait parameters when compared to 3DMAs. [8-11]. Depth sensing precision and pose estimation accuracy are the two main factors influencing the accuracy of markerless analysis.

Commonly used cameras in markerless approaches utilize either stereoscopic algorithms, structured light, or time-of-flight (ToF) technologies for depth sensing [12-16]. Among them, ToF-based cameras present higher precision and outperform compared to other variants, hence, could perform better when utilized for movement analysis [17]. Nonetheless, there exist some noise origins affecting the generated point cloud by ToF-based cameras, which ultimately influence markerless analysis accuracy, including objects and ground reflectivity, room illumination, and clothes' color [12]. The inaccuracy also originates from the human pose estimation algorithms utilized for body joints' localization. Many approaches have already been developed to obtain a more realistic skeleton with more accurate joints' position, which among them, artificial intelligence (AI)-based algorithms outperform the others [18-26]. In them, a deep neural network is supervisory trained to track body joints on an image or in 3D [13,27]. Despite their efficiency, they do not conserve the skeleton links' length when used on frame sequences and are prone to error. This is due to the exhibition of jitter between the frames, a consequence of inaccurate labeling when training the network [27].

The inaccuracy of pose estimation algorithms coupled with the point clouds' noise, affects both spatio-temporal and kinematic measurements of the gait. To partially fulfill these gaps, many applications have been recently developed including Theia3D [2], KinaTrax [28], OpenCap [29], and Pose2Sim [30,31]. In the majority of them, the primary strategy to tackle the inaccuracy problem of markerless
# 1. Introduction:

Gait analysis is a clinical assessment that provides quantitative data on various gait parameters, utilizable to diagnose walking impairments. Gold-standard systems for this purpose are 3D marker-based motion analysis (3DMA) systems [1]. However, technological progression has led to the emergence and development of markerless approaches with fewer drawbacks compared to 3DMAs [2-5]. Expensiveness, unportability, and large dedicated space requirement are drawbacks of 3DMAs, whereas the markerless systems do not offer them, which subsequently increases their deployment possibility in a broader array of clinics [6,7]. Nevertheless, a notable downside in them is their lower accuracy in estimating gait parameters when compared to 3DMAs. [8-11]. Depth sensing precision and pose estimation accuracy are the two main factors influencing the accuracy of markerless analysis.

Commonly used cameras in markerless approaches utilize either stereoscopic algorithms, structured light, or time-of-flight (ToF) technologies for depth sensing [12-16]. Among them, ToF-based cameras present higher precision and outperform compared to other variants, hence, could perform better when utilized for movement analysis [17]. Nonetheless, there exist some noise origins affecting the generated point cloud by ToF-based cameras, which ultimately influence markerless analysis accuracy, including objects and ground reflectivity, room illumination, and clothes' color [12]. The inaccuracy also originates from the human pose estimation algorithms utilized for body joints' localization. Many approaches have already been developed to obtain a more realistic skeleton with more accurate joints' position, which among them, artificial intelligence (AI)-based algorithms outperform the others [18-26]. In them, a deep neural network is supervisory trained to track body joints on an image or in 3D [13,27]. Despite their efficiency, they do not conserve the skeleton links' length when used on frame sequences and are prone to error. This is due to the exhibition of jitter between the frames, a consequence of inaccurate labeling when training the network [27].

The inaccuracy of pose estimation algorithms coupled with the point clouds' noise, affects both spatio-temporal and kinematic measurements of the gait. To partially fulfill these gaps, many applications have been recently developed including Theia3D [2], KinaTrax [28], OpenCap [29], and Pose2Sim [30,31]. In the majority of them, the primary strategy to tackle the inaccuracy problem of markerless
analysis has been refining the obtained AI-based skeletons with the aim of lengths preservation.
Inaccurate rigid body lengths lead to errors in joints' angle calculations and consequently affect the
interpretation of motion patterns and their characteristics. Hence, length preservation plays a crucial
role in the correct computation of biomechanical measurements and has proven to be the first effective
strategy in enhancing pose estimation accuracy [32]. Accordingly, algorithms such as OpenCap and
Pose2Sim have adopted this strategy by utilizing AI-driven skeletons to scale a generic body model
and ensuring lengths' preservation throughout the task [29-31]. Despite the advancements, markerless
approaches still exhibit accuracy lacks compared to the 3DMAs, which highlights the necessity of
integrating additional strategies alongside length preservation to further improve the accuracy. One
such strategy could be incorporating task-related information when refining the skeletons, as for
instance gait phases information when walking [33]. Additionally, it's important to note that the
advantages of body point clouds, even if are contaminated by noise, should not be overlooked.
Benefiting from its advantages requires constructing geometric models that could facilitate length
preservation and assist in refining the skeletons according to the orientation of the limbs' point cloud
data. Another advantage of utilizing body surface point clouds could be the facilitated elimination of
the jitters that appear in AI-based skeleton trackers, which could result in stability of tracking over
time and robustness to occlusions.

Therefore, in this research, a novel approach is presented to improve the kinematic measurements'
accuracy in markerless gait analysis by combining both lengths' preservation and utilization of gait
phases information. In this approach, the body point clouds are employed and the AI-driven skeletons
are refined with a geometric model of the lower limb. To address the effectiveness of such a
combinatorial strategy with a geometric modelization, the obtained kinematics in pre- and post-
application of the proposed method are concurrently validated against a gold-standard system.

## 2. Materials and methods:

### 2.1. Subjects:
analysis has been refining the obtained AI-based skeletons with the aim of lengths preservation.
Inaccurate rigid body lengths lead to errors in joints' angle calculations and consequently affect the
interpretation of motion patterns and their characteristics. Hence, length preservation plays a crucial
role in the correct computation of biomechanical measurements and has proven to be the first effective
strategy in enhancing pose estimation accuracy [32]. Accordingly, algorithms such as OpenCap and
Pose2Sim have adopted this strategy by utilizing AI-driven skeletons to scale a generic body model
and ensuring lengths' preservation throughout the task [29-31]. Despite the advancements, markerless
approaches still exhibit accuracy lacks compared to the 3DMAs, which highlights the necessity of
integrating additional strategies alongside length preservation to further improve the accuracy. One
such strategy could be incorporating task-related information when refining the skeletons, as for
instance gait phases information when walking [33]. Additionally, it's important to note that the
advantages of body point clouds, even if are contaminated by noise, should not be overlooked.
Benefiting from its advantages requires constructing geometric models that could facilitate length
preservation and assist in refining the skeletons according to the orientation of the limbs' point cloud
data. Another advantage of utilizing body surface point clouds could be the facilitated elimination of
the jitters that appear in AI-based skeleton trackers, which could result in stability of tracking over
time and robustness to occlusions.

Therefore, in this research, a novel approach is presented to improve the kinematic measurements'
accuracy in markerless gait analysis by combining both lengths' preservation and utilization of gait
phases information. In this approach, the body point clouds are employed and the AI-driven skeletons
are refined with a geometric model of the lower limb. To address the effectiveness of such a
combinatorial strategy with a geometric modelization, the obtained kinematics in pre- and post-
application of the proposed method are concurrently validated against a gold-standard system.

## 2. Materials and methods:

### 2.1. Subjects:
Twelve healthy subjects were recruited to participate, each providing informed written consent before the testing (Age: 26.4±2 years old, weight: 71.2±9.2 kg, height: 171.3±7.5 cm). The study was approved by the local ethics committee and conducted according to the Declaration of Helsinki.

## 2.2. Equipment:

OptiTrack 3D motion capture system (NaturalPoint, Corvallis, OR, USA) equipped with 9 cameras (PrimeX13) and set at 120Hz, is used as the reference 3DMA. Besides, two Microsoft Azure Kinect cameras, in front and back, are utilized in two setups including single-camera setup (utilizing only the front camera) and dual-camera setup (Fig. 1 (a)). Azure Kinects were both placed at 80 cm height and 9 meters apart (Resolution: 1920×1080, FPS: 30). To record simultaneously and prevent IR pulses interference, a synchronization device (eSync 2) was used. The back Azure Kinect was connected to the front one in daisy-chain configuration, and the front one was plugged into the eSync 2 device, which provided the triggering signal and enabled simultaneous recordings with OptiTrack and Azure Kinects without interference.

## 2.3. Data acquisition:

Thirty-two reflective markers were placed on each subject's body landmarks using adhesive double-sided tape. Then, the static pose, and 3 gait trials at a comfortable speed (toward the front camera) were captured with 3DMA and Azure Kinects. Afterward, markers' trajectories were low-pass filtered using a 4th-order zero-lag Butterworth filter (Cut-off: 6 Hz). To calculate the kinematics with the 3DMA, joints center for the knees, the ankles, and the toes, were obtained using the midpoint between their medial and lateral markers [2]. Hip joints center were obtained using regression equations utilizing pelvis width [34-36]. To find joints' position on videos, first, an image segmentation algorithm (Pascalvoc model) is used to detect a 2D silhouette mask of the human body. The detected mask in each frame is utilized for surrounding elimination and creating a bounding box around the subject. The images inside the boxes were then used to estimate the joints' position using OpenPose [37]. The joints'
Twelve healthy subjects were recruited to participate, each providing informed written consent before
the testing (Age: 26.4±2 years old, weight: 71.2±9.2 kg, height: 171.3±7.5 cm). The study was
approved by the local ethics committee and conducted according to the Declaration of Helsinki.

**2.2. Equipment:**

OptiTrack 3D motion capture system (NaturalPoint, Corvallis, OR, USA) equipped with 9 cameras (PrimeX13) and set at 120Hz, is used as the reference 3DMA. Besides, two Microsoft Azure Kinect cameras, in front and back, are utilized in two setups including single-camera setup (utilizing only the front camera) and dual-camera setup (Fig. 1 (a)). Azure Kinects were both placed at 80 cm height and 9 meters apart (Resolution: 1920×1080, FPS: 30). To record simultaneously and prevent IR pulses interference, a synchronization device (eSync 2) was used. The back Azure Kinect was connected to the front one in daisy-chain configuration, and the front one was plugged into the eSync 2 device, which provided the triggering signal and enabled simultaneous recordings with OptiTrack and Azure Kinects without interference.

**2.3. Data acquisition:**

Thirty-two reflective markers were placed on each subject's body landmarks using adhesive double-sided tape. Then, the static pose, and 3 gait trials at a comfortable speed (toward the front camera) were captured with 3DMA and Azure Kinects. Afterward, markers' trajectories were low-pass filtered using a 4^th^-order zero-lag Butterworth filter (Cut-off: 6 Hz). To calculate the kinematics with the 3DMA, joints center for the knees, the ankles, and the toes, were obtained using the midpoint between their medial and lateral markers [2]. Hip joints center were obtained using regression equations utilizing pelvis width [34-36]. To find joints' position on videos, first, an image segmentation algorithm (Pascalvoc model) is used to detect a 2D silhouette mask of the human body. The detected mask in each frame is utilized for surrounding elimination and creating a bounding box around the subject. The images inside the boxes were then used to estimate the joints' position using OpenPose [37]. The joints'
position in all frames was smoothed using the Savitzky-Golay low pass filter and then were mapped on their corresponding point clouds to obtain their 3D position [6]. In single-camera setup, OpenPose results obtained from the front camera were used to calculate the kinematics. In dual-camera setup, first, the point clouds were merged. To do so, identical pixels in front and back view and their corresponding positions were identified. Then, the optimal rotation and translation matrices, used for point clouds’ merging, were computed through least-squares fitting of two data sets [38]. Afterward, position of the toes was obtained using the front camera, heels using the back camera, and other joints by finding the mid-point between their front and back positions, which were then used for kinematics calculation. Kinematics were calculated in the frontal plane (abduction-adduction) and the sagittal plane (flexion-extension), inside a 6-meter-long walking path (Fig. 1(a)). The static pose offsets were subtracted from the kinematics and all curves were time-normalized to 100% of the gait cycle.

## 2.4. Proposed method:

The proposed method is a combination of two algorithms. Using the intra-frame modelization, it refines lower limb skeletons estimated by AI-based algorithms in a subject-specific geometric manner and preserves skeleton links’ length when walking. Using the between-frames adjusting algorithm, it benefits from gait phases information and applies joints’ trajectory smoothing. This algorithm proceeds in 3 steps: (1) adaptive point cloud segmentation, (2) frame-by-frame optimization, and (3) whole video optimization.

### 2.4.1. Point clouds segmentation:

Point clouds are first cleared by eliminating the surroundings, and only data of the body is preserved. Within the pertinent data, the orthogonal projection of each point on each bone line is first calculated. Considering $P_i$ a 3D point, its orthogonal projection on the bone line $\overrightarrow{AB}$ is as follows:

$$P_i^* = A + \frac{\overrightarrow{AB} \cdot \overrightarrow{AP_i}}{\|\overrightarrow{AB}\|^2} \overrightarrow{AB} \quad (1)$$
position in all frames was smoothed using the Savitzky-Golay low pass filter and then were mapped on their corresponding point clouds to obtain their 3D position [6]. In single-camera setup, OpenPose results obtained from the front camera were used to calculate the kinematics. In dual-camera setup, first, the point clouds were merged. To do so, identical pixels in front and back view and their corresponding positions were identified. Then, the optimal rotation and translation matrices, used for point clouds’ merging, were computed through least-squares fitting of two data sets [38]. Afterward, position of the toes was obtained using the front camera, heels using the back camera, and other joints by finding the mid-point between their front and back positions, which were then used for kinematics calculation. Kinematics were calculated in the frontal plane (abduction-adduction) and the sagittal plane (flexion-extension), inside a 6-meter-long walking path (Fig. 1(a)). The static pose offsets were subtracted from the kinematics and all curves were time-normalized to 100% of the gait cycle.

## 2.4. Proposed method:

The proposed method is a combination of two algorithms. Using the intra-frame modelization, it refines lower limb skeletons estimated by AI-based algorithms in a subject-specific geometric manner and preserves skeleton links’ length when walking. Using the between-frames adjusting algorithm, it benefits from gait phases information and applies joints’ trajectory smoothing. This algorithm proceeds in 3 steps: (1) adaptive point cloud segmentation, (2) frame-by-frame optimization, and (3) whole video optimization.

### 2.4.1. Point clouds segmentation:

Point clouds are first cleared by eliminating the surroundings, and only data of the body is preserved. Within the pertinent data, the orthogonal projection of each point on each bone line is first calculated. Considering $P_i$ a 3D point, its orthogonal projection on the bone line $\overrightarrow{AB}$ is as follows:

$$P_i^* = A + \frac{\overrightarrow{AB} \cdot \overrightarrow{AP_i}}{\|\overrightarrow{AB}\|^2} \overrightarrow{AB} \quad (1)$$
Then, the shortest distance of $P_i$ to the bone [AB] is calculated as follows:

$$ \text{Shortest Distance} = \begin{cases} \|P_i - P_i^*\|_2 & P_i^* \in [AB] \\ \min\{\|P_i - A\|_2, \|P_i - B\|_2\} & P_i^* \notin [AB] \end{cases} \qquad (2) $$

Where $\|\cdot\|_2$ refers to the L2 norm. Afterward, a label is assigned to each point depending on the nearest bone (i.e. skeleton link), and the data are segmented [39]. Following segmentation, two adjustments were implemented as follows: I) Only the data below the truncation planes positioned at the hips are utilized (Fig. 2(a)). II) Data in the forefoot area are eliminated using the truncation planes positioned at the toes (Fig. 2(b,c)), due to possible deformation of the foot around the toes. In the segmentation process, utilizing imprecise joints' positions will lead to inaccurate segments, and hence, segmentation should be done adaptively. Given that the joints' position in each optimization iteration evolves, the segmentation starts with the joints obtained by the OpenPose in the first iteration and is repeated in each iteration using the updated joints' position.

<figure><img src="image_1.png" /><figcaption>**Fig. 1.** (a) Experimentation setups for markerless gait analysis, (b) the proposed geometric model for the lower limb, made transparent near the joints for a better illustration of the internal skeleton. [Single column]</figcaption></figure>
Then, the shortest distance of $P_i$ to the bone [AB] is calculated as follows:

$$ \text{Shortest Distance} = \begin{cases} \|P_i - P_i^*\|_2 & P_i^* \in [AB] \\ \min\{\|P_i - A\|_2, \|P_i - B\|_2\} & P_i^* \notin [AB] \end{cases} \qquad (2) $$

Where $\|\cdot\|_2$ refers to the L2 norm. Afterward, a label is assigned to each point depending on the nearest bone (i.e. skeleton link), and the data are segmented [39]. Following segmentation, two adjustments were implemented as follows: I) Only the data below the truncation planes positioned at the hips are utilized (Fig. 2(a)). II) Data in the forefoot area are eliminated using the truncation planes positioned at the toes (Fig. 2(b,c)), due to possible deformation of the foot around the toes. In the segmentation process, utilizing imprecise joints' positions will lead to inaccurate segments, and hence, segmentation should be done adaptively. Given that the joints' position in each optimization iteration evolves, the segmentation starts with the joints obtained by the OpenPose in the first iteration and is repeated in each iteration using the updated joints' position.

<figure><img src="image_1.png" /><figcaption>**Fig. 1.** (a) Experimentation setups for markerless gait analysis, (b) the proposed geometric model for the lower limb, made transparent near the joints for a better illustration of the internal skeleton. [Single column]</figcaption></figure>
### 2.4.2 Intra-frame modelization:

A 6-part geometric model is designed (Fig. 1(b)), in which the thighs and the shanks are modeled with a portion of a cone, truncated by two orthogonal planes to the axis (Fig. 2(a)), and feet are modeled with a section of such cones that are also truncated by a plane containing the axis (Fig. 2(b,c)). To obtain a human leg-like structure, the conic models in each leg should be connected as follows: thigh and shank cones' axes always intersect each other at the knee joint, shank cone's axis always intersects the surface of the foot model and provides the ankle joint. Additionally, to connect the legs, a length constraint between the hips is considered. Each model part has two radius values for the circles obtained on truncation planes, equal to the corresponding joints' radii, and has its axis length equal to the corresponding body limb's length.

<figure><img src="image_2.png" /><figcaption>**Fig. 2.** (a) geometric model for the thighs and the shanks, (b) geometric model for the feet in single-camera setup, and (c) geometric model for the feet in dual-camera setup. [1.5 column]</figcaption></figure>
### 2.4.2 Intra-frame modelization:

A 6-part geometric model is designed (Fig. 1(b)), in which the thighs and the shanks are modeled with a portion of a cone, truncated by two orthogonal planes to the axis (Fig. 2(a)), and feet are modeled with a section of such cones that are also truncated by a plane containing the axis (Fig. 2(b,c)). To obtain a human leg-like structure, the conic models in each leg should be connected as follows: thigh and shank cones' axes always intersect each other at the knee joint, shank cone's axis always intersects the surface of the foot model and provides the ankle joint. Additionally, to connect the legs, a length constraint between the hips is considered. Each model part has two radius values for the circles obtained on truncation planes, equal to the corresponding joints' radii, and has its axis length equal to the corresponding body limb's length.

<figure><img src="image_2.png" /><figcaption>**Fig. 2.** (a) geometric model for the thighs and the shanks, (b) geometric model for the feet in single-camera setup, and (c) geometric model for the feet in dual-camera setup. [1.5 column]</figcaption></figure>
To fit the model to the data, the cones' orientation relative to the data is considered and segment's data are separated into different areas. Defining $N_A$ and $N_B$ as unit normal vectors of truncation planes in thigh or shank segments, where:

$$N_A = \frac{\overline{A - B}}{\|A - B\|_2}, \quad N_B = -N_A, \quad R_A > R_B; \qquad (3)$$

the areas in these segments (Fig. 2(a)) are defined as follows:

$$\text{Thigh or Shank: } \begin{cases} \text{Area1} & (P_i - A) \cdot N_A \ge 0 \\ \text{Area2} & (P_i - A) \cdot N_A < 0 \\ \text{Area3} & (P_i - B) \cdot N_B \ge 0 \end{cases} \quad \& \quad (P_i - B) \cdot N_B < 0 \qquad (4)$$

Also, defining unit normal vectors of truncation planes in feet models as $N_{\text{Toe}}$, $N_{\text{Ankle}}$, $N_{\text{Heel}}$, where:

$$N_{\text{Toe}} = \frac{\text{Toe - Heel}}{\|\text{Toe - Heel}\|_2}, \quad N_{\text{Heel}} = N_{\text{Ankle}} = -N_{\text{Toe}}; \qquad (5)$$

the areas in feet segments (Fig. 2(b,c)) are defined as follows:

$$\text{Foot: } \begin{cases} \text{Area1} & (P_i - \text{Toe}) \cdot N_{\text{Toe}} \ge 0 \\ \text{Area2} & (P_i - \text{Toe}) \cdot N_{\text{Toe}} < 0 \\ \text{Area3}_{\text{Single Camera}} & (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} \ge 0 \\ \text{Area3}_{\text{Dual Camera}} & (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} \ge 0 \\ \text{Area4}_{\text{Single Camera}} & (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} \ge 0 \\ \text{Area4}_{\text{Dual Camera}} & (P_i - \text{Heel}) \cdot N_{\text{Heel}} \ge 0 \\ \text{Area5} & (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} \ge 0 \end{cases} \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \quad \& \quad (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} < 0 \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \quad \& \quad (P_i - \text{Heel}) \cdot N_{\text{Heel}} < 0 \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \qquad (6)$$

After defining the areas, semi-opening angle $\psi_{\text{Segment}}$ and apex for each segment's cone, the angle $\omega_i$ between the axis line and the line connecting $P_i$ to the apex, and the orthogonal projection of $P_i$ on the corresponding truncation plane $P_i^{**}$, should be calculated. These parameters, assuming a truncated cone between two joints (J1, J2) with radii of ($R_{J1}$, $R_{J2}$), where:

$$R_{J1} > R_{J2}, \quad N_{J1} = -N_{J2}; \qquad (7)$$

are calculated as follows:
To fit the model to the data, the cones' orientation relative to the data is considered and segment's data are separated into different areas. Defining $N_A$ and $N_B$ as unit normal vectors of truncation planes in thigh or shank segments, where:

$$N_A = \frac{\overline{A - B}}{\|A - B\|_2}, \quad N_B = -N_A, \quad R_A > R_B; \qquad (3)$$

the areas in these segments (Fig. 2(a)) are defined as follows:

$$\text{Thigh or Shank: } \begin{cases} \text{Area1} & (P_i - A) \cdot N_A \ge 0 \\ \text{Area2} & (P_i - A) \cdot N_A < 0 \\ \text{Area3} & (P_i - B) \cdot N_B \ge 0 \end{cases} \quad \& \quad (P_i - B) \cdot N_B < 0 \qquad (4)$$

Also, defining unit normal vectors of truncation planes in feet models as $N_{\text{Toe}}$, $N_{\text{Ankle}}$, $N_{\text{Heel}}$, where:

$$N_{\text{Toe}} = \frac{\overline{\text{Toe} - \text{Heel}}}{\|\text{Toe} - \text{Heel}\|_2}, \quad N_{\text{Heel}} = N_{\text{Ankle}} = -N_{\text{Toe}}; \qquad (5)$$

the areas in feet segments (Fig. 2(b,c)) are defined as follows:

$$\text{Foot: } \begin{cases} \text{Area1} & (P_i - \text{Toe}) \cdot N_{\text{Toe}} \ge 0 \\ \text{Area2} & (P_i - \text{Toe}) \cdot N_{\text{Toe}} < 0 \\ \text{Area3}_{\text{Single Camera}} & (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} \ge 0 \\ \text{Area3}_{\text{Dual Camera}} & (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} \ge 0 \\ \text{Area4}_{\text{Single Camera}} & (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} \ge 0 \\ \text{Area4}_{\text{Dual Camera}} & (P_i - \text{Heel}) \cdot N_{\text{Heel}} \ge 0 \\ \text{Area5} & (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} \ge 0 \end{cases} \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \quad \& \quad (P_i - \text{Ankle}) \cdot N_{\text{Ankle}} < 0 \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \quad \& \quad (P_i - \text{Heel}) \cdot N_{\text{Heel}} < 0 \quad \& \quad (P_i - \text{Toe}) \cdot N_{\text{ShoeSole}} < 0 \qquad (6)$$

After defining the areas, semi-opening angle $\psi_{\text{Segment}}$ and apex for each segment's cone, the angle $\omega_i$ between the axis line and the line connecting $P_i$ to the apex, and the orthogonal projection of $P_i$ on the corresponding truncation plane $P_i^{**}$, should be calculated. These parameters, assuming a truncated cone between two joints (J1, J2) with radii of $(R_{J1}, R_{J2})$, where:

$$R_{J1} > R_{J2}, \quad N_{J1} = -N_{J2}; \qquad (7)$$

are calculated as follows:
$$
\begin{align*}
\psi_{Segment} &= \operatorname{Arctan}\left(\frac{R_{J1} - R_{J2}}{\|J1 - J2\|_2}\right) \\
Apex_{Segment} &= J1 + \left(\frac{R_{J1}}{\tan(\psi_{Segment})} N_{J2}\right) \\
\omega_i &= \operatorname{Arccos}\left(\frac{(\overline{J1 - Apex_{Segment}}) \cdot (\overline{P_i - Apex_{Segment}})}{\|\overline{J1 - Apex_{Segment}}\|_2 \|\overline{P_i - Apex_{Segment}}\|_2}\right) \\
P_i^{**} &= P_i - \underbrace{\left(\frac{(P_i - J1) \cdot N_{J1} N_{J2}}{\|N_{J1}\|_2^2}\right)}_{\text{On truncation plane positioned at J1}}
\end{align*}
\tag{8}
$$

Afterward, the distance error for each point, in each area, is calculated. If the point $P_i$ is in between two truncation planes, the orthogonal distance to the cone's surface is calculated [40]. A penalty is also considered for those points falling inside the cone, to ensure fitting the model to the internal side of the point cloud. These areas include Area 2 of all parts, and Area 3 in dual-camera model of the foot, in which the error is calculated as follows:

$$
\begin{equation}
\begin{aligned}
\text{Error} &= \|P_i - \text{Apex}_{\text{Segment}}\|_2^2 \sin^2(\omega_i - \psi_{\text{Segment}}) \text{Penalty}^2 \\
\text{where} \quad \begin{cases} \text{Penalty} = 1 & \text{if } \omega_i \ge \psi_{\text{Segment}} \\ \text{Penalty} = 10 & \text{if } \omega_i < \psi_{\text{Segment}} \end{cases}
\end{aligned}
\tag{9}
\end{equation}
$$

If $P_i$ belongs to Area 1 in all models, or Area 3 in thighs or shanks or single-camera model of the foot, depending on whether its projection falls inside the circular shapes on the truncation planes or not, the shortest distance error is calculated as follows:

$$
\text{Error} = \begin{cases} \|P_i - P_i^{**}\|_2^2 & \text{if } \|\text{Joint} - P_i^{**}\|_2 \le R_{\text{Joint}} \\ \|P_i - P_i^{**}\|_2^2 + (\|P_i^{**} - \text{Joint}\|_2 - R_{\text{Joint}})^2 & \text{if } \|\text{Joint} - P_i^{**}\|_2 > R_{\text{Joint}} \end{cases} \tag{10}
$$

For those $P_i$ belonging to Area 4 in single-camera foot model, or Area 5 in dual-camera foot model, if $P_i^{**}$ on the shoe sole plane is inside the trapezoid or hexagon, the distance between $P_i$ and $P_i^{**}$ is calculated. But, if the projection falls outside, minimum distance to the shape sides is computed. Sum of distance errors for all data points in segment $s$, and frame number $f$, provides $E_G_{Geo_{f,s}}$, and sum of them in all segments provides the geometric error term as follows:

$$
\begin{equation}
\begin{aligned}
\text{Error\_Geometric}_f &= \sum_{s \in \text{segment}} E_{\text{Geo}_{f,s}} \\
\text{segment} \in \{\text{Thigh}, \text{Shank}, \text{Foot}\}_{\text{Left} \ \& \ \text{Right}}
\end{aligned}
\tag{11}
\end{equation}
$$
$$
\begin{align*}
\psi_{Segment} &= \operatorname{Arctan}\left(\frac{R_{J1} - R_{J2}}{\|J1 - J2\|_2}\right) \\
Apex_{Segment} &= J1 + \left(\frac{R_{J1}}{\tan(\psi_{Segment})} N_{J2}\right) \\
\omega_i &= \operatorname{Arccos}\left(\frac{(\overline{J1 - Apex_{Segment}}) \cdot (\overline{P_i - Apex_{Segment}})}{\|\overline{J1 - Apex_{Segment}}\|_2 \|\overline{P_i - Apex_{Segment}}\|_2}\right) \\
P_i^{**} &= P_i - \underbrace{\left(\frac{(P_i - J1) \cdot N_{J1} N_{J2}}{\|N_{J1}\|_2^2}\right)}_{\text{On truncation plane positioned at J1}}
\end{align*}
$$

Afterward, the distance error for each point, in each area, is calculated. If the point $P_i$ is in between two truncation planes, the orthogonal distance to the cone's surface is calculated [40]. A penalty is also considered for those points falling inside the cone, to ensure fitting the model to the internal side of the point cloud. These areas include Area 2 of all parts, and Area 3 in dual-camera model of the foot, in which the error is calculated as follows:

$$
\begin{equation}
\begin{aligned}
\text{Error} &= \|P_i - \text{Apex}_{\text{Segment}}\|_2^2 \sin^2(\omega_i - \psi_{\text{Segment}}) \text{Penalty}^2 \\
\text{where} \quad \begin{cases} \text{Penalty} = 1 & \text{if } \omega_i \ge \psi_{\text{Segment}} \\ \text{Penalty} = 10 & \text{if } \omega_i < \psi_{\text{Segment}} \end{cases}
\end{aligned}
\tag{9}
\end{equation}
$$

If $P_i$ belongs to Area 1 in all models, or Area 3 in thighs or shanks or single-camera model of the foot, depending on whether its projection falls inside the circular shapes on the truncation planes or not, the shortest distance error is calculated as follows:

$$
\text{Error} = \begin{cases} \|P_i - P_i^{**}\|_2^2 & \text{if } \|\text{Joint} - P_i^{**}\|_2 \le R_{\text{Joint}} \\ \|P_i - P_i^{**}\|_2^2 + (\|P_i^{**} - \text{Joint}\|_2 - R_{\text{Joint}})^2 & \text{if } \|\text{Joint} - P_i^{**}\|_2 > R_{\text{Joint}} \end{cases} \tag{10}
$$

For those $P_i$ belonging to Area 4 in single-camera foot model, or Area 5 in dual-camera foot model, if $P_i^{**}$ on the shoe sole plane is inside the trapezoid or hexagon, the distance between $P_i$ and $P_i^{**}$ is calculated. But, if the projection falls outside, minimum distance to the shape sides is computed. Sum of distance errors for all data points in segment $s$, and frame number $f$, provides $E_G_{Geo_{f,s}}$, and sum of them in all segments provides the geometric error term as follows:

$$
\begin{equation}
\begin{aligned}
\text{Error\_Geometric}_f &= \sum_{s \in \text{segment}} E_{\text{Geo}}_{f,s} \\
\text{segment} \in \{\text{Thigh}, \text{Shank}, \text{Foot}\}_{\text{Left} \ \& \ \text{Right}}
\end{aligned}
\tag{11}
\end{equation}
$$
In addition, skeleton links' lengths are also preserved in the proposed algorithm. Assuming $L_{f,m}$ as the length of link $m$ in frame number $f$, and $L_m^*$ as its desired length, intended to be preserved while walking, the lengths' error term is calculated as follows:

$$ 
\begin{aligned}
\text{Error\_Lengths}_f &= \sum_{m=\text{link}} (L_{f,m} - L_m^*)^2 \\
\text{link}_{\text{Dual\ Camera}} &\in \{\text{HipToHip}, \{\text{Thigh}, \text{Shank}, \text{MidFoot}, \text{RareFoot}\}_{\text{Left} \ \& \ \text{Right}}\} \\
\text{link}_{\text{Single\ Camera}} &\in \{\text{HipToHip}, \{\text{Thigh}, \text{Shank}, \text{MidFoot}\}_{\text{Left} \ \& \ \text{Right}}\}
\end{aligned}
\qquad (12) $$

Sum of the geometric and the lengths' errors, normalized with their initial value, yields the total intra-frame error term as follows:

$$ \text{Error\_Intra\_Frame}_f = \frac{\text{Error\_Geometric}_f}{\text{Error\_Geometric}_{f(\text{Initial})}} + \frac{\text{Error\_Lengths}_f}{\text{Error\_Lengths}_{f(\text{Initial})}} \qquad (13) $$

### 2.4.3 Between-frames adjusting algorithm:

To connect each frame's skeleton to its adjacent skeletons and perform a global optimization that adds dependency among the frames, between-frames' adjustments are considered. The first consideration is the incorporation of gait phases information, which adds biomechanical awareness to the algorithm according to each subject's walking pattern. In terms of gait events, the discriminative factor distinguishing between various walking patterns is the events' temporal sequencing order. On this basis, walking patterns are classified into three categories including those contacting the ground on rearfoot, those with midfoot contact, and those contacting the ground on forefoot. The biomechanical error term is then considered such that it is applicable to diverse walking patterns by encompassing all aforementioned categories. To do so, first, gait events are detected for each leg [6]. Then, in each gait cycle represented by lower index $c$, the frame number in which heel contacts the ground $f_c^{\text{Heel-On}}$, toe contacts the ground $f_c^{\text{Toe-On}}$, heel goes off the ground $f_c^{\text{Heel-Off}}$, and toe goes off the ground $f_c^{\text{Toe-Off}}$, are extracted. Two events matrices (EM) are created for each leg, in which the rows represent the events' frame numbers and the columns represent the cycle sequences with $N_C$ gait cycles, as follows:
In addition, skeleton links' lengths are also preserved in the proposed algorithm. Assuming $L_{f,m}$ as the length of link $m$ in frame number $f$, and $L_m^*$ as its desired length, intended to be preserved while walking, the lengths' error term is calculated as follows:

$$ 
\begin{aligned}
\text{Error\_Lengths}_f &= \sum_{m=\text{link}} (L_{f,m} - L_m^*)^2 \\
\text{link}_{\text{Dual\ Camera}} &\in \{\text{HipToHip}, \{\text{Thigh}, \text{Shank}, \text{MidFoot}, \text{RareFoot}\}_{\text{Left} \ \& \ \text{Right}}\} \\
\text{link}_{\text{Single\ Camera}} &\in \{\text{HipToHip}, \{\text{Thigh}, \text{Shank}, \text{MidFoot}\}_{\text{Left} \ \& \ \text{Right}}\}
\end{aligned}
\qquad (12) $$

Sum of the geometric and the lengths' errors, normalized with their initial value, yields the total intra-frame error term as follows:

$$ \text{Error\_Intra\_Frame}_f = \frac{\text{Error\_Geometric}_f}{\text{Error\_Geometric}_{f(\text{Initial})}} + \frac{\text{Error\_Lengths}_f}{\text{Error\_Lengths}_{f(\text{Initial})}} \qquad (13) $$

### 2.4.3 Between-frames adjusting algorithm:

To connect each frame's skeleton to its adjacent skeletons and perform a global optimization that adds dependency among the frames, between-frames' adjustments are considered. The first consideration is the incorporation of gait phases information, which adds biomechanical awareness to the algorithm according to each subject's walking pattern. In terms of gait events, the discriminative factor distinguishing between various walking patterns is the events' temporal sequencing order. On this basis, walking patterns are classified into three categories including those contacting the ground on rearfoot, those with midfoot contact, and those contacting the ground on forefoot. The biomechanical error term is then considered such that it is applicable to diverse walking patterns by encompassing all aforementioned categories. To do so, first, gait events are detected for each leg [6]. Then, in each gait cycle represented by lower index $c$, the frame number in which heel contacts the ground $f_c^{\text{Heel-On}}$, toe contacts the ground $f_c^{\text{Toe-On}}$, heel goes off the ground $f_c^{\text{Heel-Off}}$, and toe goes off the ground $f_c^{\text{Toe-Off}}$, are extracted. Two events matrices (EM) are created for each leg, in which the rows represent the events' frame numbers and the columns represent the cycle sequences with $N_C$ gait cycles, as follows:
$$ EM_{Heel} = \begin{bmatrix} f_1^{Heel-On} & f_2^{Heel-On} & f_3^{Heel-On} & \dots \\ f_1^{Heel-Off} & f_2^{Heel-Off} & f_3^{Heel-Off} & \dots \end{bmatrix}_{2 \times N_c} \qquad (14) $$ $$ EM_{Toe} = \begin{bmatrix} f_1^{Toe-On} & f_2^{Toe-On} & f_3^{Toe-On} & \dots \\ f_1^{Toe-Off} & f_2^{Toe-Off} & f_3^{Toe-Off} & \dots \end{bmatrix}_{2 \times N_c} $$

Afterward, the midpoints among positions of each foot landmark in their corresponding frame intervals listed inside the EMs, are calculated as follows:

$$ \overline{Heel}_c = \frac{\sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Heel-Off}} Heel_{f,c}}{\frac{f_c^{Heel-Off} - f_c^{Heel-On}}{f_c^{Toe-On} + 1}} $$ $$ \overline{Toe}_c = \frac{\sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Toe-Off}} Toe_{f,c}}{\frac{f_c^{Toe-Off} - f_c^{Toe-On}}{f_c^{Toe-On} + 1}} $$ $$ \overline{Ankle}_c = \frac{\min_{c}^{f_c^{Heel-Off}, f_c^{Toe-Off}} \sum_{f=\max\{f_c^{Heel-On}, f_c^{Toe-On}\}}^{f_c^{Toe-Off}} Ankle_{f,c}}{\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\} - \max\{f_c^{Heel-On}, f_c^{Toe-On}\} + 1} $$

Then, the variation of each foot landmark from its midpoint in the corresponding frame interval is minimized as follows:

$$ E_{Bio} = \sum_{c=1}^{N_c} \left( \sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Heel-Off}} \|Heel_{f,c} - \overline{Heel}_c\|_2^2 + \sum_{f=\max\{f_c^{Heel-On}, f_c^{Toe-On}\}}^{f_c^{Toe-Off}} \|Toe_{f,c} - \overline{Toe}_c\|_2^2 + \sum_{f=\max\{f_c^{Heel-On}, f_c^{Toe-On}\}}^{f_c^{Heel-Off}, f_c^{Toe-Off}} \|Ankle_{f,c} - \overline{Ankle}_c\|_2^2 \right) \qquad (16) $$ $$ \text{Error\_Biomechanical} = E_{Bio}_{Left} + E_{Bio}_{Right} $$

The second consideration is obtaining smooth joints' trajectories. To do so, a batch of W frames (Window size) and their corresponding time stamps are considered. Inside the window, 3 quadratic curves are fitted, each on one of the coordinates and their time stamps. Then, the trajectory samples' errors with respect to their direct projection on the quadratic curves are calculated. Afterwards, the window is shifted by one frame to forward and the procedure is repeated till the moving window reaches the last frame. Considering $(x_{h,j}, y_{h,j}, z_{h,j})$ the coordinates of sample number h in the trajectory of joint j, and $(x_{h,j}^*, y_{h,j}^*, z_{h,j}^*)$ as their direct projection on the quadratic curves fitted on each coordinate, the error $D_{h,j}$ is calculated as follows:

$$ D_{h,j} = (x_{h,j} - x_{h,j}^*)^2 + (y_{h,j} - y_{h,j}^*)^2 + (z_{h,j} - z_{h,j}^*)^2 \qquad (17) $$

Considering the total number of frames as $N_F$, the trajectory smoothing error term in joint j is calculated as follows:
$$ EM_{Heel} = \begin{bmatrix} f_1^{Heel-On} & f_2^{Heel-On} & f_3^{Heel-On} & \dots \\ f_1^{Heel-Off} & f_2^{Heel-Off} & f_3^{Heel-Off} & \dots \end{bmatrix}_{2 \times N_c} \qquad (14) $$ $$ EM_{Toe} = \begin{bmatrix} f_1^{Toe-On} & f_2^{Toe-On} & f_3^{Toe-On} & \dots \\ f_1^{Toe-Off} & f_2^{Toe-Off} & f_3^{Toe-Off} & \dots \end{bmatrix}_{2 \times N_c} $$

Afterward, the midpoints among positions of each foot landmark in their corresponding frame intervals listed inside the EMs, are calculated as follows:

$$ \overline{Heel}_c = \frac{\sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Heel-Off}} Heel_{f,c}}{\frac{f_c^{Heel-Off} - f_c^{Heel-On}}{f_c^{Toe-On} + 1}} $$ $$ \overline{Toe}_c = \frac{\sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Toe-Off}} Toe_{f,c}}{\frac{f_c^{Toe-Off} - f_c^{Toe-On}}{f_c^{Toe-On} + 1}} $$ $$ \overline{Ankle}_c = \frac{\sum_{f=\max\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}} Ankle_{f,c}}{\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\} - \max\{f_c^{Heel-On}, f_c^{Toe-On}\} + 1} $$

Then, the variation of each foot landmark from its midpoint in the corresponding frame interval is minimized as follows:

$$ E_{Bio} = \sum_{c=1}^{N_c} \left( \sum_{f=\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Heel-Off}} \|Heel_{f,c} - \overline{Heel}_c\|_2^2 + \sum_{f=\max\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{f_c^{Toe-Off}} \|Toe_{f,c} - \overline{Toe}_c\|_2^2 + \sum_{f=\max\{f_c^{Heel-Off}, f_c^{Toe-Off}\}}^{\min\{f_c^{Heel-Off}, f_c^{Toe-Off}\}} \|Ankle_{f,c} - \overline{Ankle}_c\|_2^2 \right) \qquad (16) $$ $$ \text{Error\_Biomechanical} = E_{Bio}_{Left} + E_{Bio}_{Right} $$

The second consideration is obtaining smooth joints' trajectories. To do so, a batch of W frames (Window size) and their corresponding time stamps are considered. Inside the window, 3 quadratic curves are fitted, each on one of the coordinates and their time stamps. Then, the trajectory samples' errors with respect to their direct projection on the quadratic curves are calculated. Afterwards, the window is shifted by one frame to forward and the procedure is repeated till the moving window reaches the last frame. Considering $(x_{h,j}, y_{h,j}, z_{h,j})$ the coordinates of sample number $h$ in the trajectory of joint $j$, and $(x_{h,j}^*, y_{h,j}^*, z_{h,j}^*)$ as their direct projection on the quadratic curves fitted on each coordinate, the error $D_{h,j}$ is calculated as follows:

$$ D_{h,j} = (x_{h,j} - x_{h,j}^*)^2 + (y_{h,j} - y_{h,j}^*)^2 + (z_{h,j} - z_{h,j}^*)^2 \qquad (17) $$

Considering the total number of frames as $N_F$, the trajectory smoothing error term in joint $j$ is calculated as follows:
$$ Error\_Smoothing\_j = \sum_{f=1}^{N_F-W} \left( \sum_{h=f}^{f+W} D_{h,j} \right) \qquad (18) $$

Here, the window size is selected according to the FPS of the RGB-D cameras ($\frac{FPS}{5} \le W \le \frac{FPS}{3}$). The window size must be small enough to maintain fundamental information and meanwhile, large enough to remove redundant fluctuations and noise.

### 2.4.4 Optimization:

In this algorithm, obtaining the desired skeleton links' lengths is an offline process. They are obtained by manually marking the joints on a single frame of the static pose, which are then utilized in the optimization of gait trials. Static pose optimization involves using intra-frame errors, with joints' location and cones' radii as optimization variables. It provides cones' subject-specific radii and refined static skeleton. Gait trials optimization involves using both intra-frame and between-frames adjusting errors, with the joints' location as optimization variables. Considering $N_J$ as the number of joints and $Q_i$ as the weight coefficients for each error term, the total error functions are as follows:

$$ 
\begin{aligned}
Total\_Error_{Static\ Pose} &= Q_1 \left( \frac{\sum_{s=segment} E_{Geo}_{f,s}}{Error\_Geometric_{f(Initial)}} \right) + Q_2 \left( \frac{\sum_{m=link} (L_{f,m} - L_m^*)^2}{Error\_Lengths_{f(Initial)}} \right) \\
Total\_Error_{Gait\ Trial} &= Q_1 \left( \sum_{f=1}^{N_F} \frac{\sum_{s=segment} E_{Geo}_{f,s}}{Error\_Geometric_{f(Initial)}} \right) + Q_2 \left( \sum_{f=1}^{N_F} \frac{\sum_{m=link} (L_{f,m} - L_m^*)^2}{Error\_Lengths_{f(Initial)}} \right) + \frac{Q_3 N_F}{N_J} \left( \sum_{j=joint} \frac{\sum_{f=1}^{N_F-W} \left( \sum_{h=f}^{f+W} D_{h,j} \right)}{Error\_Smoothing_{f(Initial)}} \right) \\
&\qquad + Q_4 N_F \left( \frac{E_{Bio}_{Left} + E_{Bio}_{Right}}{Error\_Biomechanical_{Initial}} \right)
\end{aligned}
\qquad (19) $$

$$ 
\text{where} \left\{ \begin{array}{ll} \text{joint}_{\text{Dual\ Camera}} \in \{\text{Hip, Knee, Ankle, Toe, Heel}\}_{\text{Left \& Right}} & , \quad N_J = 10 \\ \text{joint}_{\text{Single\ Camera}} \in \{\text{Hip, Knee, Ankle, Toe}\}_{\text{Left \& Right}} & , \quad N_J = 8 \end{array} \right. $$

To obtain the same scale errors in all terms, regularization parameters are added to the smoothing term ($N_F/N_J$) and biomechanical term ($N_F$) in Equation (19). Both static pose and gait trials were optimized using sequential quadratic programming (SQP) [41]. Following application of the proposed method in each setup, which some examples are shown in Fig. 3, the kinematics are calculated and used for comparison.
$$ Error\_Smoothing\_j = \sum_{f=1}^{N_F-W} \left( \sum_{h=f}^{f+W} D_{h,j} \right) \qquad (18) $$

Here, the window size is selected according to the FPS of the RGB-D cameras ($\frac{FPS}{5} \le W \le \frac{FPS}{3}$). The window size must be small enough to maintain fundamental information and meanwhile, large enough to remove redundant fluctuations and noise.

### 2.4.4 Optimization:

In this algorithm, obtaining the desired skeleton links' lengths is an offline process. They are obtained by manually marking the joints on a single frame of the static pose, which are then utilized in the optimization of gait trials. Static pose optimization involves using intra-frame errors, with joints' location and cones' radii as optimization variables. It provides cones' subject-specific radii and refined static skeleton. Gait trials optimization involves using both intra-frame and between-frames adjusting errors, with the joints' location as optimization variables. Considering $N_J$ as the number of joints and $Q_i$ as the weight coefficients for each error term, the total error functions are as follows:

$$ 
\begin{aligned}
Total\_Error_{Static\ Pose} &= Q_1 \left( \frac{\sum_{s=segment} E_{Geo}_{f,s}}{Error\_Geometric_{f(Initial)}} \right) + Q_2 \left( \frac{\sum_{m=link} (L_{f,m} - L_m^*)^2}{Error\_Lengths_{f(Initial)}} \right) \\
Total\_Error_{Gait\ Trial} &= Q_1 \left( \sum_{f=1}^{N_f} \frac{\sum_{s=segment} E_{Geo}_{f,s}}{Error\_Geometric_{f(Initial)}} \right) + Q_2 \left( \sum_{f=1}^{N_f} \frac{\sum_{m=link} (L_{f,m} - L_m^*)^2}{Error\_Lengths_{f(Initial)}} \right) + \frac{Q_3 N_F}{N_J} \left( \sum_{j=joint} \frac{\sum_{f=1}^{N_F-W} \left( \sum_{h=f}^{f+W} D_{h,j} \right)}{Error\_Smoothing_{f(Initial)}} \right) + Q_4 N_F \left( \frac{E_{Bio}_{Left} + E_{Bio}_{Right}}{Error\_Biomechanical_{Initial}} \right)
\end{aligned}
\qquad (19) $$

$$ \text{where} \begin{cases} \text{joint}_{\text{Dual\ Camera}} \in \{\text{Hip, Knee, Ankle, Toe, Heel}\}_{\text{Left \& Right}} , & N_J = 10 \\ \text{joint}_{\text{Single\ Camera}} \in \{\text{Hip, Knee, Ankle, Toe}\}_{\text{Left \& Right}} , & N_J = 8 \end{cases} $$

To obtain the same scale errors in all terms, regularization parameters are added to the smoothing term ($N_F/N_J$) and biomechanical term ($N_F$) in Equation (19). Both static pose and gait trials were optimized using sequential quadratic programming (SQP) [41]. Following application of the proposed method in each setup, which some examples are shown in Fig. 3, the kinematics are calculated and used for comparison.
<figure><img src="image_3.png" /><figcaption>**Fig. 3.** Frame examples extracted from a gait video representing the images, masks, bounding boxes, and the fitted geometric model on the lower limb point clouds in both setups. Thigh segments' data are illustrated in blue, shanks data in red, and feet data are illustrated in green. **[Double column]**</figcaption></figure>

**2.5 Statistical analysis:**

To assess ensemble angle curves' similarities in identical gait cycles, Pearson correlation coefficient (R) and coefficient of multiple correlation (CMC) were used. Pearson correlation assesses linear relationship strength, whereas CMC takes into account correlation, gain, and offset, and is designed to compare different measurement methods [42]. CMCs are described as excellent similarity (0.95–1), very good (0.85–0.94), good (0.75–0.84), moderate (0.6–0.74), and poor (CMC<0.59). To compare the differences, root mean square error (RMSE) was calculated [31]. Significant differences were detected using either paired-samples t-test or Wilcoxon-signed rank test, following the normality check by Shapiro-Wilk test [29,31]. Also, to show the proposed method's ability in reducing error, RMSE differences were expressed as percentages (Diff%). Additionally, the absolute error of the kinematics compared to the 3DMA were calculated to find the improved areas over the entire gait cycles. To do so, the normality was checked on the absolute errors and then, one-dimensional statistical parametric (SPM-1D) or nonparametric (SNPM-1D) mapping was used [43-45].
<figure><img src="image_3.png" /><figcaption>**Fig. 3.** Frame examples extracted from a gait video representing the images, masks, bounding boxes, and the fitted geometric model on the lower limb point clouds in both setups. Thigh segments' data are illustrated in blue, shanks data in red, and feet data are illustrated in green. **[Double column]**</figcaption></figure>

**2.5 Statistical analysis:**

To assess ensemble angle curves' similarities in identical gait cycles, Pearson correlation coefficient (R) and coefficient of multiple correlation (CMC) were used. Pearson correlation assesses linear relationship strength, whereas CMC takes into account correlation, gain, and offset, and is designed to compare different measurement methods [42]. CMCs are described as excellent similarity (0.95–1), very good (0.85–0.94), good (0.75–0.84), moderate (0.6–0.74), and poor (CMC<0.59). To compare the differences, root mean square error (RMSE) was calculated [31]. Significant differences were detected using either paired-samples t-test or Wilcoxon-signed rank test, following the normality check by Shapiro-Wilk test [29,31]. Also, to show the proposed method's ability in reducing error, RMSE differences were expressed as percentages (Diff%). Additionally, the absolute error of the kinematics compared to the 3DMA were calculated to find the improved areas over the entire gait cycles. To do so, the normality was checked on the absolute errors and then, one-dimensional statistical parametric (SPM-1D) or nonparametric (SNPM-1D) mapping was used [43-45].
## 3. Results:

Statistical results on the obtained kinematics are detailed in Table.1. The kinematic curves, absolute errors, and SPM results are illustrated in Fig. 4. As detailed in Table.1, the proposed method significantly reduced the kinematics error compared to 3DMA, in both setups. In single-camera setup, it significantly reduced the kinematics error in the ankle joint by 42.3% and 43.5% in the frontal (abduction-adduction) and sagittal (dorsi flexion-plantar flexion) planes respectively (P<0.001). In this setup, application of the proposed method resulted in 16.1% (P=0.003) and 36.4% (P<0.001) lesser error for the knee joint kinematics, and 12.6% (P=0.006) and 33.1% (P<0.001) lesser error for the hip joint kinematics in the frontal (abduction-adduction) and sagittal (flexion-extension) planes.

In dual-camera setup, the proposed method significantly reduced the kinematics error in the ankle joint by 36% and 43%, in the frontal and sagittal planes respectively (P<0.001). Application of the proposed method in dual-camera setup resulted in 16.8% (P=0.005) and 28.3% (P<0.001) lesser error for the knee joint kinematics, and 14.9% (P=0.05) and 15.3% (P=0.012) lesser error for the hip joint kinematics in the frontal and sagittal planes (Table.1).

As a consequence of reducing errors, the similarity of angle curves significantly improved in both setups compared to the 3DMA. Despite the effect of the proposed method on all joints' kinematics (Fig. 4), it showed the biggest impacts on the ankle kinematics by turning the poor results of OpenPose (0.3≤CMC≤0.58) into moderate-to-good (0.7≤CMC≤0.81) results. For instance, in single-camera setup (Table.1), the similarity of the ankle's flexion angle compared to the 3DMA, was poor with the OpenPose (R=0.56, CMC=0.55), while the proposed method improved it to good similarity (R=0.81, CMC=0.78).

Comparing the setups revealed that having a second camera in the back of the subject improves only some of the kinematics obtained by the proposed method while affecting more joints' kinematics when OpenPose is used. The main effect of having a second camera in the proposed method was mostly on the frontal plane kinematics, where it resulted in 17.86% (P<0.001) and 18.62% (P=0.011) lesser error for the hip and the knee abduction-adduction angle (Table.1). The proposed approach improved sagittal plane kinematics to the extent that utilizing a single-camera provides equal results to multi-camera
## 3. Results:

Statistical results on the obtained kinematics are detailed in Table.1. The kinematic curves, absolute errors, and SPM results are illustrated in Fig. 4. As detailed in Table.1, the proposed method significantly reduced the kinematics error compared to 3DMA, in both setups. In single-camera setup, it significantly reduced the kinematics error in the ankle joint by 42.3% and 43.5% in the frontal (abduction-adduction) and sagittal (dorsi flexion-plantar flexion) planes respectively (P<0.001). In this setup, application of the proposed method resulted in 16.1% (P=0.003) and 36.4% (P<0.001) lesser error for the knee joint kinematics, and 12.6% (P=0.006) and 33.1% (P<0.001) lesser error for the hip joint kinematics in the frontal (abduction-adduction) and sagittal (flexion-extension) planes.

In dual-camera setup, the proposed method significantly reduced the kinematics error in the ankle joint by 36% and 43%, in the frontal and sagittal planes respectively (P<0.001). Application of the proposed method in dual-camera setup resulted in 16.8% (P=0.005) and 28.3% (P<0.001) lesser error for the knee joint kinematics, and 14.9% (P=0.05) and 15.3% (P=0.012) lesser error for the hip joint kinematics in the frontal and sagittal planes (Table.1).

As a consequence of reducing errors, the similarity of angle curves significantly improved in both setups compared to the 3DMA. Despite the effect of the proposed method on all joints' kinematics (Fig. 4), it showed the biggest impacts on the ankle kinematics by turning the poor results of OpenPose (0.3≤CMC≤0.58) into moderate-to-good (0.7≤CMC≤0.81) results. For instance, in single-camera setup (Table.1), the similarity of the ankle's flexion angle compared to the 3DMA, was poor with the OpenPose (R=0.56, CMC=0.55), while the proposed method improved it to good similarity (R=0.81, CMC=0.78).

Comparing the setups revealed that having a second camera in the back of the subject improves only some of the kinematics obtained by the proposed method while affecting more joints' kinematics when OpenPose is used. The main effect of having a second camera in the proposed method was mostly on the frontal plane kinematics, where it resulted in 17.86% (P<0.001) and 18.62% (P=0.011) lesser error for the hip and the knee abduction-adduction angle (Table.1). The proposed approach improved sagittal plane kinematics to the extent that utilizing a single-camera provides equal results to multi-camera
utilization, without significant differences. The observed improvements in both sagittal and frontal plane kinematics are mostly spread throughout the entire gait cycle, as revealed by the SPM analysis (Fig. 4). Although the biomechanical errors constrain one leg during the stance phase in each gait cycle, their impact extends to the contralateral leg which is in the swing as well, due to skeleton lengths' preservation in the algorithm. Hence, the whole gait cycles are impacted and improved.

<caption>**Table 1**</caption>

Statistical results for the OpenPose and the proposed method in single and dual-camera setups compared to 3DMA. Statistics include Pearson correlation coefficient (R), coefficient of multiple correlation (CMC), mean ± standard deviation for the root mean square error (RMSE), percentage of reduction in RMSE following application of the proposed method (Between methods Diff%), and following addition of the second camera to the setup (Between setups Diff%).

<table><thead><tr><th rowspan="2">Joint/Statistics</th><th rowspan="2">Setup</th><th colspan="2">R</th><th colspan="2">CMC</th><th colspan="3">RMSE</th></tr><tr><th>OpenPose</th><th>Proposed method</th><th>OpenPose</th><th>Proposed method</th><th>OpenPose</th><th>Proposed method</th><th>Between methods Diff%</th><th>Between methods P-value</th></tr></thead><tbody><tr><td rowspan="4">Hip flexion-extension</td><td>Single-camera setup</td><td>0.97</td><td>0.99</td><td>0.96 Excellent</td><td>0.99 Excellent</td><td>5.02 ± 2.88</td><td>3.36 ± 1.30</td><td>33.1% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.98</td><td>0.99</td><td>0.97 Excellent</td><td>0.99 Excellent</td><td>4.04 ± 3.25</td><td>3.42 ± 0.94</td><td>15.30%*</td><td>0.012</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>19.64% *</td><td>-1.80%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.03</td><td>0.217</td><td></td><td></td></tr><tr><td rowspan="4">Hip abduction-adduction</td><td>Single-camera setup</td><td>0.76</td><td>0.82</td><td>0.74 Moderate</td><td>0.81 Good</td><td>2.31 ± 0.80</td><td>2.02 ± 0.70</td><td>12.6% *</td><td>0.006</td></tr><tr><td>Dual-camera setup</td><td>0.87</td><td>0.92</td><td>0.82 Good</td><td>0.91 Very Good</td><td>1.95 ± 1.12</td><td>1.66 ± 0.49</td><td>14.9% *</td><td>0.05</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>15.89% *</td><td>17.86% *</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.024</td><td>&lt;0.001</td><td></td><td></td></tr><tr><td rowspan="4">Knee flexion-extension</td><td>Single-camera setup</td><td>0.93</td><td>0.98</td><td>0.91 Very Good</td><td>0.97 Excellent</td><td>9.46 ± 4.36</td><td>6.02 ± 2.02</td><td>36.4% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.94</td><td>0.99</td><td>0.93 Very Good</td><td>0.97 Excellent</td><td>7.38 ± 3.40</td><td>6.14 ± 1.42</td><td>16.8% *</td><td>0.005</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>22.01% *</td><td>-1.94%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.045</td><td>0.093</td><td></td><td></td></tr><tr><td rowspan="4">Knee abduction-adduction</td><td>Single-camera setup</td><td>0.64</td><td>0.75</td><td>0.62 Moderate</td><td>0.71 Moderate</td><td>3.78 ± 1.47</td><td>3.17 ± 1.18</td><td>16.1% *</td><td>0.003</td></tr><tr><td>Dual-camera setup</td><td>0.75</td><td>0.82</td><td>0.64 Moderate</td><td>0.8 Good</td><td>3.60 ± 1.96</td><td>2.58 ± 0.89</td><td>28.3% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>4.91%</td><td>18.62% *</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.74</td><td>0.011</td><td></td><td></td></tr><tr><td rowspan="4">Ankle dorsi/plantar flexion</td><td>Single-camera setup</td><td>0.56</td><td>0.81</td><td>0.55 Poor</td><td>0.78 Good</td><td>13.40 ± 6.26</td><td>7.57 ± 3.08</td><td>43.5% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.60</td><td>0.84</td><td>0.58 Poor</td><td>0.81 Good</td><td>12.73 ± 5.15</td><td>7.25 ± 2.08</td><td>43% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>5.01%</td><td>4.24%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.62</td><td>0.8</td><td></td><td></td></tr><tr><td rowspan="4">Ankle abduction-adduction</td><td>Single-camera setup</td><td>0.40</td><td>0.74</td><td>0.3 Poor</td><td>0.7 Moderate</td><td>9.66 ± 3.95</td><td>5.57 ± 2.15</td><td>42.3% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.48</td><td>0.79</td><td>0.4 Poor</td><td>0.74 Moderate</td><td>7.96 ± 2.65</td><td>5.03 ± 2.46</td><td>36% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>17.62% *</td><td>9.75%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.002</td><td>0.28</td><td></td><td></td></tr></tbody></table>
utilization, without significant differences. The observed improvements in both sagittal and frontal plane kinematics are mostly spread throughout the entire gait cycle, as revealed by the SPM analysis (Fig. 4). Although the biomechanical errors constrain one leg during the stance phase in each gait cycle, their impact extends to the contralateral leg which is in the swing as well, due to skeleton lengths' preservation in the algorithm. Hence, the whole gait cycles are impacted and improved.

<caption>**Table 1**</caption>

Statistical results for the OpenPose and the proposed method in single and dual-camera setups compared to 3DMA. Statistics include Pearson correlation coefficient (R), coefficient of multiple correlation (CMC), mean ± standard deviation for the root mean square error (RMSE), percentage of reduction in RMSE following application of the proposed method (Between methods Diff%), and following addition of the second camera to the setup (Between setups Diff%).

<table><thead><tr><th rowspan="2">Joint/Statistics</th><th rowspan="2">Setup</th><th colspan="2">R</th><th colspan="2">CMC</th><th colspan="3">RMSE</th></tr><tr><th>OpenPose</th><th>Proposed method</th><th>OpenPose</th><th>Proposed method</th><th>OpenPose</th><th>Proposed method</th><th>Between methods Diff%</th><th>Between methods P-value</th></tr></thead><tbody><tr><td rowspan="4">Hip flexion-extension</td><td>Single-camera setup</td><td>0.97</td><td>0.99</td><td>0.96 Excellent</td><td>0.99 Excellent</td><td>5.02 ± 2.88</td><td>3.36 ± 1.30</td><td>33.1% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.98</td><td>0.99</td><td>0.97 Excellent</td><td>0.99 Excellent</td><td>4.04 ± 3.25</td><td>3.42 ± 0.94</td><td>15.30%*</td><td>0.012</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>19.64% *</td><td>-1.80%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.03</td><td>0.217</td><td></td><td></td></tr><tr><td rowspan="4">Hip abduction-adduction</td><td>Single-camera setup</td><td>0.76</td><td>0.82</td><td>0.74 Moderate</td><td>0.81 Good</td><td>2.31 ± 0.80</td><td>2.02 ± 0.70</td><td>12.6% *</td><td>0.006</td></tr><tr><td>Dual-camera setup</td><td>0.87</td><td>0.92</td><td>0.82 Good</td><td>0.91 Very Good</td><td>1.95 ± 1.12</td><td>1.66 ± 0.49</td><td>14.9% *</td><td>0.05</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>15.89% *</td><td>17.86% *</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.024</td><td>&lt;0.001</td><td></td><td></td></tr><tr><td rowspan="4">Knee flexion-extension</td><td>Single-camera setup</td><td>0.93</td><td>0.98</td><td>0.91 Very Good</td><td>0.97 Excellent</td><td>9.46 ± 4.36</td><td>6.02 ± 2.02</td><td>36.4% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.94</td><td>0.99</td><td>0.93 Very Good</td><td>0.97 Excellent</td><td>7.38 ± 3.40</td><td>6.14 ± 1.42</td><td>16.8% *</td><td>0.005</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>22.01% *</td><td>-1.94%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.045</td><td>0.093</td><td></td><td></td></tr><tr><td rowspan="4">Knee abduction-adduction</td><td>Single-camera setup</td><td>0.64</td><td>0.75</td><td>0.62 Moderate</td><td>0.71 Moderate</td><td>3.78 ± 1.47</td><td>3.17 ± 1.18</td><td>16.1% *</td><td>0.003</td></tr><tr><td>Dual-camera setup</td><td>0.75</td><td>0.82</td><td>0.64 Moderate</td><td>0.8 Good</td><td>3.60 ± 1.96</td><td>2.58 ± 0.89</td><td>28.3% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>4.91%</td><td>18.62% *</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.74</td><td>0.011</td><td></td><td></td></tr><tr><td rowspan="4">Ankle dorsi/plantar flexion</td><td>Single-camera setup</td><td>0.56</td><td>0.81</td><td>0.55 Poor</td><td>0.78 Good</td><td>13.40 ± 6.26</td><td>7.57 ± 3.08</td><td>43.5% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.60</td><td>0.84</td><td>0.58 Poor</td><td>0.81 Good</td><td>12.73 ± 5.15</td><td>7.25 ± 2.08</td><td>43% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>5.01%</td><td>4.24%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.62</td><td>0.8</td><td></td><td></td></tr><tr><td rowspan="4">Ankle abduction-adduction</td><td>Single-camera setup</td><td>0.40</td><td>0.74</td><td>0.3 Poor</td><td>0.7 Moderate</td><td>9.66 ± 3.95</td><td>5.57 ± 2.15</td><td>42.3% *</td><td>&lt;0.001</td></tr><tr><td>Dual-camera setup</td><td>0.48</td><td>0.79</td><td>0.4 Poor</td><td>0.74 Moderate</td><td>7.96 ± 2.65</td><td>5.03 ± 2.46</td><td>36% *</td><td>&lt;0.001</td></tr><tr><td>Between setups Diff%</td><td></td><td></td><td></td><td></td><td>17.62% *</td><td>9.75%</td><td></td><td></td></tr><tr><td>Between setups P-value</td><td></td><td></td><td></td><td></td><td>0.002</td><td>0.28</td><td></td><td></td></tr></tbody></table>
Single-Camera Setup

<figure><img src="image_4.png" /></figure>

Dual-Camera Setup

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Mean ± standard deviation for each joints' kinematics obtained by the 3DMA (green), the OpenPose (red), and the proposed method (blue), absolute errors compared to the 3DMA, and SPM analysis showing the significantly improved areas (black) following application of the proposed method in each setup. **[Double column]**</figcaption></figure>
Single-Camera Setup

<figure><img src="image_4.png" /></figure>

Dual-Camera Setup

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Mean ± standard deviation for each joints' kinematics obtained by the 3DMA (green), the OpenPose (red), and the proposed method (blue), absolute errors compared to the 3DMA, and SPM analysis showing the significantly improved areas (black) following application of the proposed method in each setup. **[Double column]**</figcaption></figure>
## 4. Discussion:

The results of validations in this study confirmed that the combined application of length preservation and gait phases information significantly improves the accuracy of gait kinematics measurements in markerless analysis. The RMSEs obtained for the lower-limb kinematics following application of the proposed method are between 2.02°-7.57° and between 1.66°-7.25° in single and dual-camera setups, which outperform the RMSEs reported in the literature.

In comparison to other research that established similar setups for gait kinematics calculation, Ma and her colleagues reported RMSEs between 12.5°-23° with a single camera, and 7.2°-15.1° using two cameras, for the lower limb kinematics during gait [46,47]. Another research utilized one camera and reported RMSEs of 11.7° and 28° for the hip and the knee flexion-extension respectively [48]. Yeung and his colleagues reported RMSEs between 4°-16.1° when utilizing a single Azure Kinect placed at 0-degree viewing angle [49]. The higher errors obtained in [46,47,48,49] compared to the results of current research are due to the lack of lengths' preservation or any other skeleton refining strategy in those studies. Meanwhile, it reveals the importance of such strategies to calculate joint kinematics with higher accuracy in markerless approaches.

In comparison to the multi-camera approaches, Theia3D and KinaTrax, two commercially available markerless solutions utilizing 8 cameras, calculate hip kinematics with 11° and 2.6° RMSE error (Theia3D), and with 8.21° and 3.16° RMSE error (KinaTrax), in sagittal and frontal planes respectively [2,28]. However, the proposed method in this research showed superior results for hip kinematics by utilizing only one or two cameras, reflecting the efficiency of a correct modelization rather than utilizing more cameras. Results for the knee and the ankle joints are also similar to Theia3D and KinaTrax, with differences of about 1°-3°. Pose2Sim, a 4-camera-based algorithm, uses OpenPose skeletons as input and transfers them into OpenSim software to create physically consistent skeletons during movement and to compute joint angles through inverse kinematics [30,31]. It provides lower limb kinematics with 3.1°-5.6° RMSE, which is similar to the results of this research. Another algorithm is OpenCap, an open-source application for markerless movement analysis using two or more smartphones [29]. OpenCap algorithm is similar to Pose2Sim, and provides lower limb
## 4. Discussion:

The results of validations in this study confirmed that the combined application of length preservation and gait phases information significantly improves the accuracy of gait kinematics measurements in markerless analysis. The RMSEs obtained for the lower-limb kinematics following application of the proposed method are between 2.02°-7.57° and between 1.66°-7.25° in single and dual-camera setups, which outperform the RMSEs reported in the literature.

In comparison to other research that established similar setups for gait kinematics calculation, Ma and her colleagues reported RMSEs between 12.5°-23° with a single camera, and 7.2°-15.1° using two cameras, for the lower limb kinematics during gait [46,47]. Another research utilized one camera and reported RMSEs of 11.7° and 28° for the hip and the knee flexion-extension respectively [48]. Yeung and his colleagues reported RMSEs between 4°-16.1° when utilizing a single Azure Kinect placed at 0-degree viewing angle [49]. The higher errors obtained in [46,47,48,49] compared to the results of current research are due to the lack of lengths' preservation or any other skeleton refining strategy in those studies. Meanwhile, it reveals the importance of such strategies to calculate joint kinematics with higher accuracy in markerless approaches.

In comparison to the multi-camera approaches, Theia3D and KinaTrax, two commercially available markerless solutions utilizing 8 cameras, calculate hip kinematics with 11° and 2.6° RMSE error (Theia3D), and with 8.21° and 3.16° RMSE error (KinaTrax), in sagittal and frontal planes respectively [2,28]. However, the proposed method in this research showed superior results for hip kinematics by utilizing only one or two cameras, reflecting the efficiency of a correct modelization rather than utilizing more cameras. Results for the knee and the ankle joints are also similar to Theia3D and KinaTrax, with differences of about 1°-3°. Pose2Sim, a 4-camera-based algorithm, uses OpenPose skeletons as input and transfers them into OpenSim software to create physically consistent skeletons during movement and to compute joint angles through inverse kinematics [30,31]. It provides lower limb kinematics with 3.1°-5.6° RMSE, which is similar to the results of this research. Another algorithm is OpenCap, an open-source application for markerless movement analysis using two or more smartphones [29]. OpenCap algorithm is similar to Pose2Sim, and provides lower limb
kinematics with 2°-10.2° RMSE, similar to the results of this research [29]. Even in some joints, the proposed approach showed lesser errors than the ones reported by OpenCap, as for instance in the hip joint kinematics. Developers of OpenCap have stated that their results did not improve significantly when utilizing more than two cameras [29]. In line with this statement, the results of the proposed method also indicate that utilizing a correct modelization could result in the sufficiency of only one camera to reach a certain accuracy in markerless gait analysis.

There exist some differences between the proposed approach and existing algorithms including the lack of utilizing task-related information in the literature, different methods of skeleton refining, and lengths' preservation strategy. For instance, Pose2Sim and OpenCap utilize AI-driven skeletons and preserve the lengths by scaling a generic human body model and utilizing it for the whole task [29-31]. However, the proposed approach benefits from the advantages of the body point cloud and refines the AI-driven skeletons with a geometric model that takes into account the orientation of the limbs during the fitting process. Using the point cloud in refining AI-driven skeletons ensures eliminating possible jitters that appear in AI-based pose estimators and results in maintaining stable and reliable tracking over time. Additionally, it enhances robustness to occlusions, allowing for more consistent performance even if some part of a limb is temporarily obscured and only a portion of a limb's point cloud is available. The proposed method can also be utilized in single-camera approaches by transferring the skeletons obtained on the surface of the body to the inside of the body and reducing the offsets in the joints' position. In addition, the proposed approach utilizes gait phases information which combined with the length's preservation leads to a better localization of the internal skeleton and consequently, more accurate computation of joint kinematics.

This approach is not without limitations. Firstly, it is validated only on healthy subjects. Hence, future studies are required to validate such approaches on different pathologies and adapt them to diverse populations. Secondly, fitting shapes to point clouds involves computational complexities and may require substantial storage, especially for high-density point clouds. Hence, it is recommended that future studies consider the real-time applications of such geometric approaches for practical implementation. Lastly, due to the utilized conic shapes, the algorithm only provides frontal and
kinematics with 2°-10.2° RMSE, similar to the results of this research [29]. Even in some joints, the proposed approach showed lesser errors than the ones reported by OpenCap, as for instance in the hip joint kinematics. Developers of OpenCap have stated that their results did not improve significantly when utilizing more than two cameras [29]. In line with this statement, the results of the proposed method also indicate that utilizing a correct modelization could result in the sufficiency of only one camera to reach a certain accuracy in markerless gait analysis.

There exist some differences between the proposed approach and existing algorithms including the lack of utilizing task-related information in the literature, different methods of skeleton refining, and lengths' preservation strategy. For instance, Pose2Sim and OpenCap utilize AI-driven skeletons and preserve the lengths by scaling a generic human body model and utilizing it for the whole task [29-31]. However, the proposed approach benefits from the advantages of the body point cloud and refines the AI-driven skeletons with a geometric model that takes into account the orientation of the limbs during the fitting process. Using the point cloud in refining AI-driven skeletons ensures eliminating possible jitters that appear in AI-based pose estimators and results in maintaining stable and reliable tracking over time. Additionally, it enhances robustness to occlusions, allowing for more consistent performance even if some part of a limb is temporarily obscured and only a portion of a limb's point cloud is available. The proposed method can also be utilized in single-camera approaches by transferring the skeletons obtained on the surface of the body to the inside of the body and reducing the offsets in the joints' position. In addition, the proposed approach utilizes gait phases information which combined with the length's preservation leads to a better localization of the internal skeleton and consequently, more accurate computation of joint kinematics.

This approach is not without limitations. Firstly, it is validated only on healthy subjects. Hence, future studies are required to validate such approaches on different pathologies and adapt them to diverse populations. Secondly, fitting shapes to point clouds involves computational complexities and may require substantial storage, especially for high-density point clouds. Hence, it is recommended that future studies consider the real-time applications of such geometric approaches for practical implementation. Lastly, due to the utilized conic shapes, the algorithm only provides frontal and
sagittal plane kinematics, and not the coronal plane kinematics. However, considering more complex
geometric shapes in future research that more accurately encompass the anatomical details of the bones
and condyles, or limbs' geometry, could resolve this limitation.

**5. Conclusion:**

In this research, an algorithm to improve the accuracy of lower limb kinematics in markerless gait
analysis using a single or dual-camera is presented. This algorithm refines the skeletons obtained by
AI-based pose estimation algorithms in a subject-specific geometric manner and utilizes gait phases
information. Results of the study reveal that utilizing a proper modelization of the body with
consideration of task-related constraints could improve the kinematics, resulting in the requirement of
fewer cameras that perform equal or even better in terms of analysis accuracy compared to multi-
camera approaches.

CRediT authorship contribution statement:

**Mehran Hatamzadeh:** Writing – review & editing, Visualization, Validation, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **Laurent Busé:** Writing – review & editing, Supervision, Resources, Project administration, Methodology, Funding acquisition, Conceptualization. **Katia Turcot:** Writing – review & editing, Supervision, Methodology, Conceptualization. **Raphael Zory:** Writing – review & editing, Supervision, Resources, Project administration, Methodology, Funding acquisition, Conceptualization.

**Declaration of Competing Interest:**

The authors declare that they have no known competing financial interests or personal relationships
that could have appeared to influence the work reported in this paper.
sagittal plane kinematics, and not the coronal plane kinematics. However, considering more complex
geometric shapes in future research that more accurately encompass the anatomical details of the bones
and condyles, or limbs' geometry, could resolve this limitation.

**5. Conclusion:**

In this research, an algorithm to improve the accuracy of lower limb kinematics in markerless gait
analysis using a single or dual-camera is presented. This algorithm refines the skeletons obtained by
AI-based pose estimation algorithms in a subject-specific geometric manner and utilizes gait phases
information. Results of the study reveal that utilizing a proper modelization of the body with
consideration of task-related constraints could improve the kinematics, resulting in the requirement of
fewer cameras that perform equal or even better in terms of analysis accuracy compared to multi-
camera approaches.

CRediT authorship contribution statement:

**Mehran Hatamzadeh:** Writing – review & editing, Visualization, Validation, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **Laurent Busé:** Writing – review & editing, Supervision, Resources, Project administration, Methodology, Funding acquisition, Conceptualization. **Katia Turcot:** Writing – review & editing, Supervision, Methodology, Conceptualization. **Raphael Zory:** Writing – review & editing, Supervision, Resources, Project administration, Methodology, Funding acquisition, Conceptualization.

**Declaration of Competing Interest:**

The authors declare that they have no known competing financial interests or personal relationships
that could have appeared to influence the work reported in this paper.
## Acknowledgment:

This work was supported by the COFUND BoostUrCAreer program of the Université Côte d'Azur, which has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Actions (MSCA, Grant Number: 847581), the Conseil Région SUD Provence-Alpes-Côte d'Azur, and the IDEX UCA JEDI.

## References:

[1] J.L. McGinley, R. Baker, R. Wolfe, M.E. Morris, The reliability of three-dimensional kinematic gait measurements: A systematic review, Journal of Gait & Posture, 29 (3) (2009) 360-369. https://doi.org/10.1016/j.gaitpost.2008.09.003

[2] R.M. Kanko, E.K. Laende, E.M. Davis, W.S. Selbie, K.J. Deluzio, Concurrent assessment of gait kinematics using marker-based and markerless motion capture, Journal of Biomechanics, 127 (2021) 110665. https://doi.org/10.1016/j.jbiomech.2021.110665

[3] S. Mehdizadeh, H. Nabavi, A. Sabo, T. Arora, A. Laboni, B. Taati, Concurrent validity of human pose tracking in video for measuring gait parameters in older adults: a preliminary analysis with multiple trackers, viewing angles, and walking directions. Journal of NeuroEngineering and Rehabilitation 18 (139) (2021). https://doi.org/10.1186/s12984-021-00933-0

[4] S. Chakraborty, A. Nandy, T. Yamaguchi, V. Bonnet, G. Venture, Accuracy of image data stream of a markerless motion capture system in determining the local dynamic stability and joint kinematics of human gait, Journal of Biomechanics, 104 (2020) 109718. https://doi.org/10.1016/j.jbiomech.2020.109718

[5] M. Dunn, A. Kennerley, Z. Murrell-Smith, K. Webster, K. Middleton, J. Wheat, Application of video frame interpolation to markerless, single-camera gait analysis, Journal of Sports Engineering, 26 (22) (2023). https://doi.org/10.1007/s12283-023-00419-3

[6] M. Hatamzadeh, L. Busé, F. Chorin, P. Alliez, J.D Favreau, R. Zory, A kinematic-geometric model based on ankles' depth trajectory in frontal plane for gait analysis using a single RGB-D camera, Journal of Biomechanics, 145 (2022) 111358. https://doi.org/10.1016/j.jbiomech.2022.111358

[7] B. Horsak, A. Eichmann, K. Lauer, K. Prock, P. Krondorfer, T. Siragy, B. Dumphart, Concurrent validity of smartphone-based markerless motion capturing to quantify lower-limb joint kinematics in healthy and pathological gait, Journal of Biomechanics, 159 (2023) 111801. https://doi.org/10.1016/j.jbiomech.2023.111801

[8] M.d.C. Vilas-Boas, A.P. Rocha, H.M.P. Choupina, M.N. Cardoso, J.M. Fernandes, T. Coelho, J.P.S. Cunha, Validation of a single RGB-D camera for gait assessment of polyneuropathy patients, Journal of Sensors, 19 (22) (2019) 4929. https://doi.org/10.3390/s19224929

[9] M.d.C. Vilas-Boas, H.M.P. Choupina, A.P. Rocha, J.M. Fernandes, J.P.S. Cunha, Full-body motion assessment: Concurrent validation of two body tracking depth sensors versus a gold standard system during gait, Journal of Biomechanics, 87 (2019) 189-196. https://doi.org/10.1016/j.jbiomech.2019.03.008

[10] B.F. Mentiplay, L.G. Perraton, K.J. Bower, Y.H. Pua, R. McGaw, S. Heywood, R.A. Clark, Gait assessment using the Microsoft Xbox One Kinect: Concurrent validity and inter-day reliability of spatiotemporal and kinematic variables, Journal of Biomechanics, 48 (10) (2015) 2166-2170. https://doi.org/10.1016/j.jbiomech.2015.05.021

[11] M. Moro, G. Marchesi, F. Hesse, F. Odone, M. Casadio, Markerless vs. marker-based gait analysis: A proof of concept study, Journal of Sensors. 22 (5) (2022) 2011. https://doi.org/10.3390/s22052011
## Acknowledgment:

This work was supported by the COFUND BoostUrCAreer program of the Université Côte d'Azur, which has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Actions (MSCA, Grant Number: 847581), the Conseil Région SUD Provence-Alpes-Côte d'Azur, and the IDEX UCA JEDI.

## References:

[1] J.L. McGinley, R. Baker, R. Wolfe, M.E. Morris, The reliability of three-dimensional kinematic gait measurements: A systematic review, Journal of Gait & Posture, 29 (3) (2009) 360-369. https://doi.org/10.1016/j.gaitpost.2008.09.003

[2] R.M. Kanko, E.K. Laende, E.M. Davis, W.S. Selbie, K.J. Deluzio, Concurrent assessment of gait kinematics using marker-based and markerless motion capture, Journal of Biomechanics, 127 (2021) 110665. https://doi.org/10.1016/j.jbiomech.2021.110665

[3] S. Mehdizadeh, H. Nabavi, A. Sabo, T. Arora, A. Laboni, B. Taati, Concurrent validity of human pose tracking in video for measuring gait parameters in older adults: a preliminary analysis with multiple trackers, viewing angles, and walking directions. Journal of NeuroEngineering and Rehabilitation 18 (139) (2021). https://doi.org/10.1186/s12984-021-00933-0

[4] S. Chakraborty, A. Nandy, T. Yamaguchi, V. Bonnet, G. Venture, Accuracy of image data stream of a markerless motion capture system in determining the local dynamic stability and joint kinematics of human gait, Journal of Biomechanics, 104 (2020) 109718. https://doi.org/10.1016/j.jbiomech.2020.109718

[5] M. Dunn, A. Kennerley, Z. Murrell-Smith, K. Webster, K. Middleton, J. Wheat, Application of video frame interpolation to markerless, single-camera gait analysis, Journal of Sports Engineering, 26 (22) (2023). https://doi.org/10.1007/s12283-023-00419-3

[6] M. Hatamzadeh, L. Busé, F. Chorin, P. Alliez, J.D Favreau, R. Zory, A kinematic-geometric model based on ankles' depth trajectory in frontal plane for gait analysis using a single RGB-D camera, Journal of Biomechanics, 145 (2022) 111358. https://doi.org/10.1016/j.jbiomech.2022.111358

[7] B. Horsak, A. Eichmann, K. Lauer, K. Prock, P. Krondorfer, T. Siragy, B. Dumphart, Concurrent validity of smartphone-based markerless motion capturing to quantify lower-limb joint kinematics in healthy and pathological gait, Journal of Biomechanics, 159 (2023) 111801. https://doi.org/10.1016/j.jbiomech.2023.111801

[8] M.d.C. Vilas-Boas, A.P. Rocha, H.M.P. Choupina, M.N. Cardoso, J.M. Fernandes, T. Coelho, J.P.S. Cunha, Validation of a single RGB-D camera for gait assessment of polyneuropathy patients, Journal of Sensors, 19 (22) (2019) 4929. https://doi.org/10.3390/s19224929

[9] M.d.C. Vilas-Boas, H.M.P. Choupina, A.P. Rocha, J.M. Fernandes, J.P.S. Cunha, Full-body motion assessment: Concurrent validation of two body tracking depth sensors versus a gold standard system during gait, Journal of Biomechanics, 87 (2019) 189-196. https://doi.org/10.1016/j.jbiomech.2019.03.008

[10] B.F. Mentiplay, L.G. Perraton, K.J. Bower, Y.H. Pua, R. McGaw, S. Heywood, R.A. Clark, Gait assessment using the Microsoft Xbox One Kinect: Concurrent validity and inter-day reliability of spatiotemporal and kinematic variables, Journal of Biomechanics, 48 (10) (2015) 2166-2170. https://doi.org/10.1016/j.jbiomech.2015.05.021

[11] M. Moro, G. Marchesi, F. Hesse, F. Odone, M. Casadio, Markerless vs. marker-based gait analysis: A proof of concept study, Journal of Sensors. 22 (5) (2022) 2011. https://doi.org/10.3390/s22052011
[12]. S.L. Colyer, M. Evans, D.P. Cosker, A.I.T. Salo, A review of the evolution of vision-based motion analysis and the integration of advanced computer vision methods towards developing a markerless system. Journal of Sports Medicine – Open, 4 (24) (2018). https://doi.org/10.1186/s40798-018-0139-y

[13]. T. Xu, D. An, Y. Jia, Y. Yue, A review: Point cloud-based 3D human joints estimation, Journal of Sensors, 21 (5) (2021) 1684. https://doi.org/10.3390/s21051684

[14]. M. Zago, M. Luzzago, T. Marangoni, M.D. Cecco, M. Tarabini, M. Galli, 3D Tracking of human motion using visual skeletonization and stereoscopic vision, Journal of Frontiers in Bioengineering and Biotechnology, 8 (2020) 2296-4185. https://doi.org/10.3389/fbioe.2020.00181

[15]. S. Yoon, H.W Jung, H. Jung, K. Kim, S.K. Hong, H. Roh, B.M. Oh, Development and validation of 2D-LiDAR-based gait analysis instrument and algorithm. Journal of Sensors 21 (2) (2021) 414. https://doi.org/10.3390/s21020414

[16]. B. Sheng, L. Chen, J. Cheng, Y. Zhang, Z. Hua, J. Tao, A markless 3D human motion data acquisition method based on the binocular stereo vision and lightweight Open Pose algorithm, Journal of Measurement, (2023) 113908, https://doi.org/10.1016/j.measurement.2023.113908

[17]. H. Sarbolandi, D. Lefloch, A. Kolb, Kinect range sensing: Structured-light versus Time-of-Flight Kinect, Journal of Computer Vision and Image Understanding, 139 (2015) 1-20. https://doi.org/10.1016/j.cviu.2015.05.006

[18]. X. Perez-Sala, S. Escalera, C. Angulo, J. González, A survey on model based approaches for 2D and 3D visual human pose recovery. Journal of Sensors, 14 (3) (2014) 4189-4210. https://doi.org/10.3390/s140304189

[19]. G. Ariyanto, M.S. Nixon, Model-based 3D gait biometrics, 2011 International Joint Conference on Biometrics (IJCB), Washington, DC, USA, (2011) 1-7. https://doi.org/10.1109/IJCB.2011.6117582

[20]. R. Poppe, Vision-based human motion analysis: An overview, Journal of Computer Vision and Image Understanding, 108 (1-2) (2007) 4-18. https://doi.org/10.1016/j.cviu.2006.10.016

[21]. Z.Q. Cheng, Y. Chen, R.R. Martin, T. Wu, Z. Song, Parametric modeling of 3D human body shape—A survey, Journal of Computers & Graphics, 71 (2018) 88-100. https://doi.org/10.1016/j.cag.2017.11.008

[22]. M.J. Foo, J.S. Chang, W.T. Ang, Real-time foot tracking and gait evaluation with geometric modeling, Journal of Sensors, 22 (4) (2022) 1661. https://doi.org/10.3390/s22041661

[23]. N. Sarafianos, B. Boteanu, B. Ionescu, I.A. Kakadiaris, 3D human pose estimation: A review of the literature and analysis of covariates, Journal of Computer Vision and Image Understanding, 152 (2016) 1-20. https://doi.org/10.1016/j.cviu.2016.09.002

[24]. Ł. Kidziński, B. Yang, J.L. Hicks, A. Rajagopal, S.L. Delp, M.H. Schwartz, Deep neural networks enable quantitative movement analysis using single-camera videos, Journal of Nature Communications, 11 (2020) 4054. https://doi.org/10.1038/s41467-020-17807-z

[25]. J. Wang, S. Tan, X Zhen, S. Xu, F. Zheng, Z. He, L. Shao, Deep 3D human pose estimation: A review, Journal of Computer Vision and Image Understanding, 210 (2021) 103225. https://doi.org/10.1016/j.cviu.2021.103225

[26]. E. Martini, M. Boldo, S. Aldegheri, N. Valè, M. Filippetti, N. Smania, M. Bertucco, A. Picelli, N. Bombieri, Enabling gait analysis in the telemedicine practice through portable and accurate 3D human pose estimation, Journal of Computer Methods and Programs in Biomedicine, 225 (2022) 107016. https://doi.org/10.1016/j.cmpb.2022.107016

[27]. N.J. Cronin, Using deep neural networks for kinematic analysis: Challenges and opportunities, Journal of Biomechanics, 123 (2021) 110460. https://doi.org/10.1016/j.jbiomech.2021.110460

[28]. Z. Ripic, M. Nienhuis, J.F. Signorile, T.M. Best, K.A. Jacobs, M. Eltoukhy, A comparison of three-dimensional kinematics between markerless and marker-based motion capture in overground gait, Journal of Biomechanics, 159 (2023) 111793. https://doi.org/10.1016/j.jbiomech.2023.111793

[29]. S.D. Uhlrich, A. Falisse, Ł. Kidziński, J. Muccini, M. Co, A.S. Chaudhari, J.L. Hicks, S.L. Delp, OpenCap: Human movement dynamics from smartphone videos, Journal of Plus Computational Biology, 19 (10) (2023) e1011462. https://doi.org/10.1371/journal.pcbi.1011462

[30]. D. Pagnon, M. Domalain, L. Reveret, Pose2Sim: An end-to-end workflow for 3D markerless sports kinematics—part 1: Robustness, Journal of Sensors, 21 (19) (2021) 6530, https://doi.org/10.3390/s21196530

[31]. D. Pagnon, M. Domalain, L. Reveret, Pose2Sim: An end-to-end workflow for 3D markerless sports kinematics—part 2: Accuracy, Journal of Sensors, 22 (7) (2022) 2712. https://doi.org/10.3390/s22072712

[32]. R. Li, W. Si, M. Weinmann, R. Klein, Constraint-based optimized human skeleton extraction from single-depth camera, Journal of Sensors, 19 (11) (2019) 2604. https://doi.org/10.3390/s19112604
[12]. S.L. Colyer, M. Evans, D.P. Cosker, A.I.T. Salo, A review of the evolution of vision-based motion analysis and the integration of advanced computer vision methods towards developing a markerless system. Journal of Sports Medicine - Open, 4 (24) (2018). https://doi.org/10.1186/s40798-018-0139-y

[13]. T. Xu, D. An, Y. Jia, Y. Yue, A review: Point cloud-based 3D human joints estimation, Journal of Sensors, 21 (5) (2021) 1684. https://doi.org/10.3390/s21051684

[14]. M. Zago, M. Luzzago, T. Marangoni, M.D. Cecco, M. Tarabini, M. Galli, 3D Tracking of human motion using visual skeletonization and stereoscopic vision, Journal of Frontiers in Bioengineering and Biotechnology, 8 (2020) 2296-4185. https://doi.org/10.3389/fbioe.2020.00181

[15]. S. Yoon, H.W Jung, H. Jung, K. Kim, S.K. Hong, H. Roh, B.M. Oh, Development and validation of 2D-LiDAR-based gait analysis instrument and algorithm. Journal of Sensors 21 (2) (2021) 414. https://doi.org/10.3390/s21020414

[16]. B. Sheng, L. Chen, J. Cheng, Y. Zhang, Z. Hua, J. Tao, A markless 3D human motion data acquisition method based on the binocular stereo vision and lightweight Open Pose algorithm, Journal of Measurement, (2023) 113908, https://doi.org/10.1016/j.measurement.2023.113908

[17]. H. Sarbolandi, D. Lefloch, A. Kolb, Kinect range sensing: Structured-light versus Time-of-Flight Kinect, Journal of Computer Vision and Image Understanding, 139 (2015) 1-20. https://doi.org/10.1016/j.cviu.2015.05.006

[18]. X. Perez-Sala, S. Escalera, C. Angulo, J. González, A survey on model based approaches for 2D and 3D visual human pose recovery. Journal of Sensors, 14 (3) (2014) 4189-4210. https://doi.org/10.3390/s140304189

[19]. G. Ariyanto, M.S. Nixon, Model-based 3D gait biometrics, 2011 International Joint Conference on Biometrics (IJCB), Washington, DC, USA, (2011) 1-7. https://doi.org/10.1109/IJCB.2011.6117582

[20]. R. Poppe, Vision-based human motion analysis: An overview, Journal of Computer Vision and Image Understanding, 108 (1-2) (2007) 4-18. https://doi.org/10.1016/j.cviu.2006.10.016

[21]. Z.Q. Cheng, Y. Chen, R.R. Martin, T. Wu, Z. Song, Parametric modeling of 3D human body shape—A survey, Journal of Computers & Graphics, 71 (2018) 88-100. https://doi.org/10.1016/j.cag.2017.11.008

[22]. M.J. Foo, J.S. Chang, W.T. Ang, Real-time foot tracking and gait evaluation with geometric modeling, Journal of Sensors, 22 (4) (2022) 1661. https://doi.org/10.3390/s22041661

[23]. N. Sarafianos, B. Boteanu, B. Ionescu, I.A. Kakadiaris, 3D human pose estimation: A review of the literature and analysis of covariates, Journal of Computer Vision and Image Understanding, 152 (2016) 1-20. https://doi.org/10.1016/j.cviu.2016.09.002

[24]. Ł. Kidziński, B. Yang, J.L. Hicks, A. Rajagopal, S.L. Delp, M.H. Schwartz, Deep neural networks enable quantitative movement analysis using single-camera videos, Journal of Nature Communications, 11 (2020) 4054. https://doi.org/10.1038/s41467-020-17807-z

[25]. J. Wang, S. Tan, X Zhen, S. Xu, F. Zheng, Z. He, L. Shao, Deep 3D human pose estimation: A review, Journal of Computer Vision and Image Understanding, 210 (2021) 103225. https://doi.org/10.1016/j.cviu.2021.103225

[26]. E. Martini, M. Boldo, S. Aldegheri, N. Valè, M. Filippetti, N. Smania, M. Bertucco, A. Picelli, N. Bombieri, Enabling gait analysis in the telemedicine practice through portable and accurate 3D human pose estimation, Journal of Computer Methods and Programs in Biomedicine, 225 (2022) 107016. https://doi.org/10.1016/j.cmpb.2022.107016

[27]. N.J. Cronin, Using deep neural networks for kinematic analysis: Challenges and opportunities, Journal of Biomechanics, 123 (2021) 110460. https://doi.org/10.1016/j.jbiomech.2021.110460

[28]. Z. Ripic, M. Nienhuis, J.F. Signorile, T.M. Best, K.A. Jacobs, M. Eltoukhy, A comparison of three-dimensional kinematics between markerless and marker-based motion capture in overground gait, Journal of Biomechanics, 159 (2023) 111793. https://doi.org/10.1016/j.jbiomech.2023.111793

[29]. S.D. Uhlrich, A. Falisse, Ł. Kidziński, J. Muccini, M. Co, A.S. Chaudhari, J.L. Hicks, S.L. Delp, OpenCap: Human movement dynamics from smartphone videos, Journal of Plus Computational Biology, 19 (10) (2023) e1011462. https://doi.org/10.1371/journal.pcbi.1011462

[30]. D. Pagnon, M. Domalain, L. Reveret, Pose2Sim: An end-to-end workflow for 3D markerless sports kinematics—part 1: Robustness, Journal of Sensors, 21 (19) (2021) 6530, https://doi.org/10.3390/s21196530

[31]. D. Pagnon, M. Domalain, L. Reveret, Pose2Sim: An end-to-end workflow for 3D markerless sports kinematics—part 2: Accuracy, Journal of Sensors, 22 (7) (2022) 2712. https://doi.org/10.3390/s22072712

[32]. R. Li, W. Si, M. Weinmann, R. Klein, Constraint-based optimized human skeleton extraction from single-depth camera, Journal of Sensors, 19 (11) (2019) 2604. https://doi.org/10.3390/s19112604
[33]. K. Jun, K. Lee, S. Lee, H. Lee, M.S. Kim, Hybrid Deep Neural Network Framework Combining Skeleton and Gait Features for Pathological Gait Recognition. Journal of Bioengineering 10 (10) (2023) 1133. https://doi.org/10.3390/bioengineering10101133

[34]. A.L. Bell, R.A. Brand, D.R. Pedersen, Prediction of hip joint centre location from external landmarks, Journal of Human Movement Science, 8 (1) (1989) 3-16. https://doi.org/10.1016/0167-9457(89)90020-1

[35]. A.L. Bell, D.R. Pedersen, R.A. Brand, A comparison of the accuracy of several hip center location prediction methods, Journal of Biomechanics, 23 (6) (1990) 617-621. https://doi.org/10.1016/0021-9290(90)90054-7

[36]. M.E. Harrington, A.B. Zavatsky, S.E.M. Lawson, Z. Yuan, T.N. Theologis, Prediction of the hip joint centre in adults, children, and patients with cerebral palsy based on magnetic resonance imaging, Journal of Biomechanics, 40 (3) (2007) 595-602. https://doi.org/10.1016/j.jbiomech.2006.02.003

[37]. Z. Cao, G. Hidalgo, T. Simon, S.E. Wei, Y. Sheikh, OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 (1) (2019) 172-186. https://doi.org/10.1109/TPAMI.2019.2929257

[38]. K.S. Arun, T.S. Huang, S.D Blostein, Least-Squares Fitting of Two 3-D Point Sets, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9 (5) (1987) 698-700. https://doi.org/10.1109/TPAMI.1987.4767965

[39]. M. Sandau, H. Koblauch, T.B. Moeslund, H. Aanæs, T. Alkjær, E.B. Simonsen, Markerless motion capture can provide reliable 3D gait kinematics in the sagittal and frontal plane, Journal of Medical Engineering & Physics, 36 (9) (2014) 1168-1175. https://doi.org/10.1016/j.medengphy.2014.07.007

[40]. J. Wang, Z. Yu, Quadratic curve and surface fitting via squared distance minimization, Journal of Computers & Graphics, 35 (6) (2011) 1035-1050. https://doi.org/10.1016/j.cag.2011.09.001

[41]. J. Nocedal, S.J. Wright, Sequential Quadratic Programming, Numerical optimization. Springer, New York, (2006) 529-562.

[42]. A. Ferrari, A.G. Cutti, A. Cappello, A new formulation of the coefficient of multiple correlation to assess the similarity of waveforms measured synchronously by different motion analysis protocols, Journal of Gait & Posture, 31 (4) (2010) 540-542. https://doi.org/10.1016/j.gaitpost.2010.02.009

[43]. T.C. Pataky, M.A. Robinson, J. Vanrenterghem, Vector field statistical analysis of kinematic and force trajectories, Journal of Biomechanics. 46 (14) (2013) 2394-2401. https://doi.org/10.1016/j.jbiomech.2013.07.031

[44]. T.C. Pataky, J. Vanrenterghem, M.A. Robinson, Zero-vs. one-dimensional, parametric vs. non-parametric, and confidence interval vs. hypothesis testing procedures in one-dimensional biomechanical trajectory analysis. Journal of Biomechanics. 48 (7) (2015) 1277-1285. https://doi.org/10.1016/j.jbiomech.2015.02.051

[45]. E.P. Washabaugh, T.A. Shanmugam, R. Ranganathan, C. Krishnan, Comparing the accuracy of open-source pose estimation methods for measuring gait kinematics, Journal of Gait & Posture, 97 (2022) 188-195. https://doi.org/10.1016/j.gaitpost.2022.08.008

[46]. Y. Ma, K. Mithraratne, N.C. Wilson, X. Wang, Y. Ma, Y. Zhang, The validity and reliability of a Kinect v2-based gait analysis system for children with cerebral palsy, Journal of Sensors, 19 (7) (2019) 1660. https://doi.org/10.3390/s19071660

[47]. Y. Ma, B. Sheng, R. Hart, Y. Zhang, The validity of a dual Azure Kinect-based motion capture system for gait analysis: a preliminary study, 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE (2020) 1201-1206.

[48]. X. Xu, R.W. McGorry, L.S. Chou, J.H. Lin, C.C. Chang, Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking, Journal of Gait & Posture, 42 (2) (2015) 145-151. https://doi.org/10.1016/j.gaitpost.2015.05.002

[49]. L.F. Yeung, Z. Yang, K.C.C. Cheng, D. Du, R.K.Y. Tong, Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2, Journal of Gait & Posture, 87 (2021) 19-26. https://doi.org/10.1016/j.gaitpost.2021.04.005
[33]. K. Jun, K. Lee, S. Lee, H. Lee, M.S. Kim, Hybrid Deep Neural Network Framework Combining Skeleton and Gait Features for Pathological Gait Recognition. Journal of Bioengineering 10 (10) (2023) 1133. https://doi.org/10.3390/bioengineering10101133

[34]. A.L. Bell, R.A. Brand, D.R. Pedersen, Prediction of hip joint centre location from external landmarks, Journal of Human Movement Science, 8 (1) (1989) 3-16. https://doi.org/10.1016/0167-9457(89)90020-1

[35]. A.L. Bell, D.R. Pedersen, R.A. Brand, A comparison of the accuracy of several hip center location prediction methods, Journal of Biomechanics, 23 (6) (1990) 617-621. https://doi.org/10.1016/0021-9290(90)90054-7

[36]. M.E. Harrington, A.B. Zavatsky, S.E.M. Lawson, Z. Yuan, T.N. Theologis, Prediction of the hip joint centre in adults, children, and patients with cerebral palsy based on magnetic resonance imaging, Journal of Biomechanics, 40 (3) (2007) 595-602. https://doi.org/10.1016/j.jbiomech.2006.02.003

[37]. Z. Cao, G. Hidalgo, T. Simon, S.E. Wei, Y. Sheikh, OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 (1) (2019) 172-186. https://doi.org/10.1109/TPAMI.2019.2929257

[38]. K.S. Arun, T.S. Huang, S.D Blostein, Least-Squares Fitting of Two 3-D Point Sets, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9 (5) (1987) 698-700. https://doi.org/10.1109/TPAMI.1987.4767965

[39]. M. Sandau, H. Koblauch, T.B. Moeslund, H. Aanæs, T. Alkjær, E.B. Simonsen, Markerless motion capture can provide reliable 3D gait kinematics in the sagittal and frontal plane, Journal of Medical Engineering & Physics, 36 (9) (2014) 1168-1175. https://doi.org/10.1016/j.medengphy.2014.07.007

[40]. J. Wang, Z. Yu, Quadratic curve and surface fitting via squared distance minimization, Journal of Computers & Graphics, 35 (6) (2011) 1035-1050. https://doi.org/10.1016/j.cag.2011.09.001

[41]. J. Nocedal, S.J. Wright, Sequential Quadratic Programming, Numerical optimization. Springer, New York, (2006) 529-562.

[42]. A. Ferrari, A.G. Cutti, A. Cappello, A new formulation of the coefficient of multiple correlation to assess the similarity of waveforms measured synchronously by different motion analysis protocols, Journal of Gait & Posture, 31 (4) (2010) 540-542. https://doi.org/10.1016/j.gaitpost.2010.02.009

[43]. T.C. Pataky, M.A. Robinson, J. Vanrenterghem, Vector field statistical analysis of kinematic and force trajectories, Journal of Biomechanics. 46 (14) (2013) 2394-2401. https://doi.org/10.1016/j.jbiomech.2013.07.031

[44]. T.C. Pataky, J. Vanrenterghem, M.A. Robinson, Zero-vs. one-dimensional, parametric vs. non-parametric, and confidence interval vs. hypothesis testing procedures in one-dimensional biomechanical trajectory analysis. Journal of Biomechanics. 48 (7) (2015) 1277-1285. https://doi.org/10.1016/j.jbiomech.2015.02.051

[45]. E.P. Washabaugh, T.A. Shanmugam, R. Ranganathan, C. Krishnan, Comparing the accuracy of open-source pose estimation methods for measuring gait kinematics, Journal of Gait & Posture, 97 (2022) 188-195. https://doi.org/10.1016/j.gaitpost.2022.08.008

[46]. Y. Ma, K. Mithraratne, N.C. Wilson, X. Wang, Y. Ma, Y. Zhang, The validity and reliability of a Kinect v2-based gait analysis system for children with cerebral palsy, Journal of Sensors, 19 (7) (2019) 1660. https://doi.org/10.3390/s19071660

[47]. Y. Ma, B. Sheng, R. Hart, Y. Zhang, The validity of a dual Azure Kinect-based motion capture system for gait analysis: a preliminary study, 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE (2020) 1201-1206.

[48]. X. Xu, R.W. McGorry, L.S. Chou, J.H. Lin, C.C. Chang, Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking, Journal of Gait & Posture, 42 (2) (2015) 145-151. https://doi.org/10.1016/j.gaitpost.2015.05.002

[49]. L.F. Yeung, Z. Yang, K.C.C. Cheng, D. Du, R.K.Y. Tong, Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2, Journal of Gait & Posture, 87 (2021) 19-26. https://doi.org/10.1016/j.gaitpost.2021.04.005