Velocity-based training assessment: Effect of Extended-Relative-Phase-based extracted features on classification

Authors: yeongju  woo
hang  thi
phuong  nguyen
Venue: KDD_org_2024_Workshop_AIDSH
Published: 2024-05-28 08:44:56

Abstract: Velocity-based training is one of the essential training methods that helps improve athletic performance by providing immediate feedback to athletes. However, there needs to be more ways to evaluate velocity-based training by analyzing the athlete's entire movement. Thus, this study aimed to verify the effectiveness of a newly proposed Extended Relative Phase (ERP) feature on the velocity-based training assessment by using the coordinates of most major joints using Human Pose Estimation (HPE). The difference between experts and novices was compared in the experiment using the proposed feature. The Relative Phase Angle ($RP_{Angle}$) exists to evaluate the combination of each joint's angular displacement and velocity. However, assessing the consistency of repeated movements and comparing angular displacement and velocity with experts takes work. For this reason, the Relative Phase Distance ($RP_{Distance}$) was proposed as a new feature. 
The dataset trained and predicted the performance verification, including each joint angle, $RP_{Angle}$, and $RP_{Distance}$. The 1D CNN-based deep learning model for training and prediction was used to compare each extracted feature. As a result, the newly proposed indicator had a good effect on the prediction performance of the velocity-based training evaluation.

Main paper: 
 introduction: 
 As strength is vital in measuring an individual's performance in sports, optimizing an athlete's strength capacity is often essential and beneficial [18, 31] . Therefore, coaches evaluate muscle strength before and after prescribing a training program to athletes and use it as a critical indicator to judge the performance of a given program. During previous years, the test of one repetition maximum (1-RM) was often regarded as the gold standard for assessing the strength capacity of individuals under practical environmental conditions. The 1-RM test is the maximum weight that can be lifted by an athlete simply once with the correct athletic form. The 1-RM test is the most commonly used test by strength and conditioning coaches to assess strength capacities and evaluate the effectiveness of training programs [6] .Although using a percentage of 1-RM is often referred to as either the traditional or percentage-based approach to calculating training intensity, Poliquin et al. [25] argued that this method based on fixed 1-RM becomes problematic because of the daily variation of 1-RM. Jovanovic et al. [9] and Zourdos et al. [32] have revealed that 1RM varies greatly depending on the condition of the athlete, showing a difference of ±18% and an overall range difference of 36% when 1RM is measured daily with a percentage-based approach. To solve complex problems, clinicians have started measuring body movement velocity during exercise as a marker of intensity rather than the percentage of 1-RM.Velocity-based training (VBT) is a method of assessing the strength of a given movement by calculating displacement and time through observation of the bar or body speed. The VBT may generally be considered a method of improving the dynamic strength at higher speeds, but, like the 1-RM test, the VBT is simply an objective way to assess the strength in a given movement. Since force and speed have a linear relationship, the strength of a given movement can be objectively quantified even using speed rather than a percentage of the 1-RM test. Through VBT, athletes can obtain information regarding their performance, and coaches can provide specific feedback.The most commonly employed sensors for measuring VBT are linear position transducers and accelerometers. O'Reilly et al. [22] have investigated whether a single lumbar-worn IMU could identify deviations of seven commonly observed squats, and Lee et al. [13] classified various squat postures through artificial intelligence using IMU sensors. Woo et al. [29] utilized the gait analysis data of the elderly individuals collected through the IMU sensor and found that the symmetries of the left and right feet were different in walking speeds.However, two things could be improved in the case of VBT measurements. Initially, most VBT-based studies used sensors, but since the number of sensors used in research is limited, the sensors are attached to parts of the body or exercise equipment for measurement. Thus, VBT-based research has the disadvantage that it is not accessible from the environment, and experimenting is possible only when the environment for the measurement is configured.Only limited studies have attempted to find a universal method for assessing support training; therefore, a comprehensive body movement analysis is warranted. The coupling of each body part during exercise can be expressed through a relative phase (RP) [7, 28] . Although RP is used to obtain information about the relationship (e.g., angle and angular velocity) between the trunk and lower limbs, there are some limitations to the demonstration of continuous exercise. 
related works 2.1 motion-capture-system-based training assessment: 
 Motion capture is the process of recording the movement of an object or person, which involves measuring the position and orientation of an object or person in physical space. In commercial motion capture, inertial sensors are standard; an example is the Xsens MVN [26] . Xsens utilizes 17 IMU sensors consisting of a combination of accelerometer, gyroscope, and magnetometer to track 6 DOF at the body's joints. Compared to vision-based motion capture, sensor-based motion capture can reduce space limitations but requires a lot of cost and time to install due to the large number of needed inertial sensors. Therefore, existing studies try to use a small number of sensors even if performance degradation occurs. In addition to using an inertial sensor, there is also an approach using motion capture using an optical sensor or video. The most common device for optical motion capture is OptiTrack [17] , which installs multiple infrared cameras that read information from capture sensors attached to the human body. When the sensors provide a 2D position, the motion capture software calculates it as 3D data. The advantage of this approach is that there are no restrictions on the movements performed, it can track many people, and it is also beneficial for fast movements. However, the sensors have disadvantages, such as needing more data due to markers being covered during operation or capturing in a limited space where the camera is installed.Single-camera-based Human pose estimation (HPE) has made great strides in overcoming the limitations of sensor or motion capture in recent years. For example, Openpose [4] extracted feature points in real-time regardless of the number of people using only videos or photos through deep learning in 2017. Markerless-based MCS(Motion Capture Systems) were less capable than marker-based MCS in studies requiring the tracking of detailed 3D kinematics or fine movements such as finger tracking. However, VideoPose3D [23] , announced in 2019, performed effective 3D reconstruction by applying a model based on dilated temporal convolution to 2D critical points of the image. BlazePose [2] , announced in 2020, enabled the model to infer the human pose in real-time, even on mobile devices. These models performed lightweight pose estimation using heatmap and regression. Although markerless MCS can offer great potential for extending the scope of movement analysis outside of laboratory settings in a practical way, there is still the problem of the need for more accuracy where detailed 3D kinematics are required for clinical decision-making.Human Pose Estimation (HPE) is computer vision technology that predicts a person's posture by specifying a person's joints or essential body parts as key points and is widely used in various fields such as autonomous driving, sports, medical, and metaverse [5, 11, 14, 16, 21] . In addition, as HPE technology advances, the fitness technology market is filled with AI-powered personal trainer apps. ALFA-AI [1] monitors the user's exercise execution with realtime AI analysis and provides real-time visual and auditory improvement feedback. The user's key joints are tracked through a two-dimensional coordinate system, and personal AI training is continuously adjusted according to the actual user's performance. infiGro [8] , created by Infivolve, is a fully automated, AI-powered, digital personal trainer app that guides, analyzes, corrects, and motivates in real time via your phone's camera. It shows an example video of an expert and counts the number of repetitions of the user through pose estimation.However, as we surveyed, these applications are programmed through simple algorithms such as estimating the angle through the body's coordinate system or whether or not a given threshold value is exceeded. Alternatively, there is a disadvantage that the program proceeds without considering the count of exercises and the condition or situation of the player performing the exercise. Alternatively, since only a two-dimensional coordinate system is used, it is challenging to apply in a wild environment. 
method 3.1 relative phase: 
 Continuous Relative Phase (CRP) indicates the positional change in coordination by describing phase relationships between the two joints [3, 12] . RP is often measured to quantify the relationship between the kinematics of two mechanically connected joints during a certain period to analyze specific movements, such as human gait. Figure 1 depicts the overall method for calculating the relative phase [20] . To calculate RP, we subtract the phase angle of the proximal segment (𝑃𝐴) from the distal segment for each 𝑖 𝑡ℎ data point calculated from the time-normalized phase portrait.However, since the Relative Phase analyzes the data using only the Phase Angle, it is suitable for temporally analyzing the coordination of two joints. Nevertheless, the Relative Phase's disadvantage is that it is inadequate to explain the strength or stability of motion.When multiple squat cycles are plotted on the same phase portrait, the amount of variability in the path of the trajectory can be used to qualitatively evaluate the stability of the neuromuscular system under the given exercise-intensity condition. Slight variations in the trajectory are because of the response of the neuromuscular system to global and local perturbations experienced during the squat cycle. Such flexibility enables the neuromuscular system to maintain a stable and proficient movement pattern. Therefore, excessive variability is associated with instabilities in the behavior of Therefore, an Extended Relative Phase is proposed that extends from the existing Relative Phase by adding the Phase Angle and the Phase Distance, which can describe the degree of variability and stability of motion.For calculating ERP, we first utilize Blaze Pose, which supports the ML (machine learning) kit pose detection API to detect the person's skeleton in the image. Blaze Pose is one kind of human pose estimation algorithm that infers 33 3D landmarks and background segmentation masks for whole bodies in RGB video frames. Table 1 shows the results of the pose estimation quality of the BlazePose GHUM model [30] used in MediaPipe Pose [19] . The result was evaluated through three different validation datasets: Yoga, Dance, and HIIT. The results confirmed that the used model in this study showed high performance with high accuracy.The Hampel identifier [24] was used to eliminate the outliers of the estimated 33 3D landmark coordinates. Subsequently, the Savitsky-Golay filter [27] was applied to smooth the coordinates. Only two coordinates of interest, such as the hip and knee, were used to evaluate the squats among the 33 adjusted 3D landmark coordinates and normalize the calculated Angle and angular velocity. For angle normalization, we applied the robust filter, which has strong characteristics in outliers, and the MinMax filter, which normalizes the data between -1.0 and 1.0. Through the preprocessing of these data, the value of the normalized Angle was obtained as follows [10] :EQUATIONEQUATIONwhere 𝜃 𝑖 ′ represents the normalized Angle, Q represents the Quantile range, 𝜃 𝑟𝑎𝑤 𝑖 represents the original Angle, 𝜃 𝑟𝑏 𝑖 represents the Angle to which the robust filter is applied, and 𝑖 indicates the point of the cycle. Compared with the Angle's normalization, the angular velocity normalized through the MaxAbs filter is as follows:EQUATIONwhere 𝜔 𝑖 ′ represents the normalized angular velocity, 𝜔 𝑖 represents the initial angular velocity (𝜔 = d𝜃 d𝑡 ), and 𝑖 indicates the point of the cycle.The Extended Relative Phase with the addition of phase distance, i.e., the radius of the phase, is composed of 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 and 𝑅𝑃 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒 and is calculated by the following expression:EQUATION)EQUATION)where 𝑃𝐴 describes the Phase Angle, 𝑃𝐷 describes the Phase Distance, 𝜔 represents the normalized angular velocity, and 𝜃 represents the normalized angular position.  
experimental environments and systems: 
 After capturing the user image, the 3D coordinates for each joint are estimated. Subsequently, the proposed ERP algorithm is applied for feature extraction in velocity-based training. Finally, the extracted features classify the difference between experts and users. The backsquat footage of seven non-professional participants was captured to collect the data related to velocity-based training for novices. The footage was captured using a resolution of 1920 × 1080 and a camera of 30 fps in the configured environment. The distance between the camera and the participants was set to 380(± 5) cm for the camera's focus. In addition, the camera's height was fixed to 130 cm, although the participants' heights were slightly different (mean ± standard deviation = 167.1 ± 8.7 cm) Table 2 summarizes the personal and physical information of the seven participants. All participants gave informed consent to include them before participating in the study. We also collected information regarding the athletic careers of the participants to provide reliability for the experiment. The experimental procedures were performed under the Declaration of Helsinki and approved by the Clinical Trial Center Ethics Committee, Department of Medical Innovation, Osaka University Hospital (no. 15408, 11 March 2016).The participants performed 20 squats in 3 sets with a 15 kg weight bar. The break time between each set was set to 60 seconds, and they tried to proceed with the squats at a constant speed and movement. 
extended-relative-phased-based extracted features on classification: 
 To verify the effectiveness of these indicators in actual case analysis, we utilized a 1D-CNN-based deep learning classification model. The labels were divided into quartiles according to the exercise abilities of the participants. We divided the models into several categories based on input differences, conducted individual training for each, and analyzed the results. Using 1D CNN for sequence classification has the advantage of directly learning from raw time series data without requiring domain-specific knowledge to engineer input features manually.We attempted data augmentation through sampling to address the limited amount of data. Considering that the information, such as trends and shapes in the time series data we want to use, is crucial, we judged that common techniques in data augmentation may not be suitable. Therefore, we utilized random sampling to augment angular position 𝜃 and angular velocity 𝜔 for each participant by 1,000 instances. Using the augmented 𝜃 and 𝜔, we calculated 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 and 𝑅𝑃 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒 . As the calculated ERP had significantly larger values than 𝜃 and 𝜔, we scaled the ERP between -1 and 1 using the MinMax filter to address data bias. The finalized dataset was split into train and test data in a 7-3 ratio, maintaining the class distribution ratio during the split.The model used for training has the structure shown in Figure 3 . In this study, we configured the network framework by adding Global Average Pooling (GAP) [15] to the 1D-CNN. To avoid overfitting, dropout is added after each layer. Finally, GAP is applied to reduce the number of output parameters before generating the output. Instead of adding fc-layers on top of the feature maps, we take the average of each feature map, and the resulting vector is fed directly into the softmax layer.We conducted four experiments by varying the input parameters of the model. In Experiment 1, we utilized only eight features, excluding ERP, using 𝜃 and 𝜔 of both hips and knees. Experiment 2 involved adding the existing 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 to the inputs, resulting in 10 features. For Experiment 3, we introduced a new feature, 𝑅𝑃 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒 , to the existing eight features, resulting in 10 features used in the experiment. In the final experiment, we included 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 and 𝑅𝑃 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒 , totaling 12 features in the input. We compared the results of these four experiments. Table 3 summarizes the features, accuracy, and loss values for each experiment. Furthermore, Figure 4 illustrates the confusion matrix for each experiment in the test dataset. In conclusion, the model incorporating the existing 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 (Experiment 2) performed better than Experiment 1, which used only 𝜃 and 𝜔. Furthermore, it was observed that the model trained with the addition of the proposed ERP to the existing features outperformed the models trained with only 𝑅𝑃 𝐴𝑛𝑔𝑙𝑒 or 𝑅𝑃 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒 added to the current features. 
conclusion: 
 This study proposes an Extended Relative Phase (ERP) indicator to assess velocity-based training. It utilizes this indicator extracted Our markerless-based research is a groundbreaking method with the advantage of being fast and free from spatial restrictions compared to conventional sensor or motion capture techniques. In specific exercises, coaches can simultaneously evaluate the relationship between the joint temporal coordination and the stability of the exercise intensity through ERP without specific technical knowledge. However, the method has certain limitations, such as being less accurate than sensor-based evaluation and having less internal data in the experiment.Given the growing interest in fitness worldwide, the indicators and evaluation methods presented in the paper are thought to be a new and objective way to evaluate a player's performance in the fitness field. We expect this method to expand to fitness and various fields, such as patient walking data for rehabilitation and posture data in multiple sports. 
 
 Appendix:  



Reviews: review: title: title: Good results achieved with a model based on relative phase (RP). Even though the novelty of this work seems not sufficient enough and results only have reported on private datasets.
main_review: main_review: This paper proposed a method based on relative phase distance to improve the prediction performance of the velocity-based
training evaluation.

# Quality:
On overall, the paper reads good, but there are many typos which the author should check very carefully.

# Clarity:
The entire presentation of the paper (considering only the main body without any supplementary file) falls between a border line paper and a weakly rejection.

# Originality and Significance:
It is not completely clear that it will be significant. Will it work on complex vision datasets having multiple objects ? That is the question.

# Limitations:
1.The generalization ability of the proposed method cannot be demonstrated. This paper only gives quantitative comparisons on private datasets. The authors should provide more quantitative and qualitative comparisons on real-world public datasets.
2.The main weakness of this paper is that the proposed approach is not technically sound.
Some key details are not explained clearly, and thus it is difficult to understand how it works.
3.Some commonly-used metrics for evaluating model performance, such as Precision, Recall
title: title: Doubtable experiment results due to very limited dataset size, potentially problematic data augmentation and uncomparable results.
main_review: main_review: This work proposes a new feature extraction technique for markerless velocity-based training assessment. This article suggests a new feature named extended relative phase (ERP) to enhance the evaluation for training in kinematics. I conclude the pros and cons as below:

Pros:

- The proposed feature is derived from the traditional phase indicators to capture a higher level of information with demonstrated examples to show its advantage compared to the traditional indicators.

- The experiment results validate the effectiveness of the proposed ERP feature by an ablation study.

Cons:

- The size of the used dataset is very small with only 7 subjects. Although certain data augmentation technique (random sampling) is used, I doubt if random sampling will help in data augmentation. If the trends and patterns in your time series data are crucial, simple random sampling might not be sufficient. It may not capture the underlying structure and could introduce noise instead of meaningful variation. Random sampling to augment angular position and angular velocity can be useful if these variables follow a predictable distribution, but there is no evidence of this assumption holding. I cannot find the clues to convince me random sampling is a good method in this case.

- I am also confused by the classification results shown in the experiments. What is your predicted target? In the confusion matrix, there are only encoded labels (0, 1, 2, 3), so what do these numbers mean? Really need more discussion and explanation here. And the reported accuracy is so high that can be suspicious (100% of accuracy!!). Just wonder if the authors tried the proposed method on other public datasets to compare with other work. 

- Some used symbols are not explained. For example, what is "Q" in Eq. 1? I can guess it should be a function, but what this function does is not explained.