# Analysis of Aerobics Auxiliary Training Based on Deep Learning

Authors: Can Li
Journal: Scientific Programming
 (2022)

---

## Abstract

With the in-depth integration of information technology and subject teaching, it is also an inevitable trend to apply modern information technology to aerobics teaching. In this paper, the N-best algorithm is used in the video and real-time camera in aerobics, so that the human posture parameters in a single-frame image can be estimated. By using the relative position and motion direction of each part of the human body to describe the characteristics of aerobics, the Laplace scoring method is used to reduce the dimension of the data, and the discriminant human motion feature vector with a strong local topological structure is obtained. Finally, the iterative self-organizing data analysis technology (ISODATA algorithm) is used to dynamically determine the keyframe. In the aerobics video keyframe extraction experiments, the ST-FMP model improves the recognition accuracy of nondeterministic body parts of the flexible hybrid articulated human model (FMP) by about 15 percentage points and achieves 81% keyframe extraction accuracy, which is better than the keyframe algorithms of KFE and motion block. The proposed algorithm is sensitive to human motion features and human pose and is suitable for motion video annotation review.


---

## Body

## 1. Introduction

In the twenty-first century, modern information technology has been widely used in the field of teaching, and multimedia and network technologies have played a role in achieving educational teaching reform, which has brought opportunities for today’s physical education teaching reform [1]. In the practice of physical education, giving full play to the advantages of digitalization, networking, intelligence, and multimedia of information technology has given life to physical education classroom teaching, stimulated students’ enthusiasm, and autonomy in learning physical education knowledge and skills and has had a multiplier effect on improving the quality of physical education classroom teaching. Modern information technology serves physical education classroom teaching, which plays a very important role in implementing the new curriculum standards for physical education and improving the quality of physical education classroom teaching [2].Aerobics is an important element in physical education. The teaching of aerobics is a very technical and contemporary practical class. The use of modern information technology can disseminate information to students through a variety of media such as images, sound, animation, and text, thus making the process of teaching aerobics simple, difficult, and easy, and showing students the teaching content that teachers cannot teach with words in a vivid way, and can also open up students’ horizons [3].Demonstration is a method often used in aerobics teaching, and it is the teacher rehearses the action for students to see, so that students have a visual impression of the action, to facilitate students to imitate and learn the common method [4].The use of multimedia means to improve the teaching of movement demonstration can make a fundamental change in the demonstration. The use of multimedia demonstration teaching not only has the characteristics of image, intuitive, vivid, and can be achieved through the media size, distance, fast, and slow changes for students to understand the teaching demonstration, but also has the teacher in the aerobics theory class teaching difficult to say clearly that students are not easy to grasp, difficult to understand the content of aerobics knowledge in front of the students, which helps to teach the key and difficult points to reach and solve [5, 6].It is not easy for students to capture the details of the action. Therefore, to understand the essence of these actions, multimedia technology needs to be used to assist students in understanding. The use of multimedia technology to assist teaching can visually show the decomposition and overall movements for students and synchronized with the teacher’s detailed language explanation, and students can easily understand the main points of the movement, laying the foundation for mastering the movement technology [7, 8]. Therefore, teachers inevitably encounter difficulties in the actual work of teaching perfunctory or errors. The application of modern information technology for teaching can avoid the above problems, reduce the teacher’s work intensity and pressure, make up for the teacher’s own lack of ability, present to students a modern, advanced, and accurate demonstration of technical movements, and give full play to the teacher’s leading role in teaching.The use of modern information technology in aerobics teaching can create a good teaching atmosphere, a high level of competitive technology, a beautiful demonstration of movement techniques, vivid images, and so on, making teaching vivid, lively, and fashionable, leaving a deep and unforgettable impression on students, stimulating their interest in learning and desire for knowledge, making the learning content intuitive and easy to understand, and reducing the difficulty of learning [9]. For the video and real-time camera of aerobics, this paper uses the N-best algorithm to estimate the human pose parameters in a single frame; the iterative self-organizing data analysis technique (ISODATA) algorithm is used to determine the keyframes dynamically. In the aerobics video keyframe extraction experiments, the ST-FMP model improves the recognition accuracy of nondeterministic body parts of the flexible hybrid articulated human model (FMP) by about 15 percentage points [10].


## 2. Combination of Aerobics Knowledge

### 2.1. Teaching Tasks of Aerobics

Task design and establishment are the first step in building the network teaching mode of college aerobics, using the teaching content to the network platform to design various teaching tasks. Here, taking the teaching of mass aerobics as an example, the teacher can introduce the collective competition mode in the teaching process, combining with the competition mode to first stimulate students’ enthusiasm for learning aerobics [11]. First of all, the teacher will make a microvideo according to the requirements of the designed group competition mode, show it to the students through the multimedia platform, so that the students can understand the aerobics learning tasks intuitively, and, at the same time, give the students time to discuss and ask questions to the teacher, increasing the interactive communication link before the class.The teacher will then design a teaching module to show all the decomposition teaching videos of each movement of the aerobics course, and let students watch the decomposition movements of aerobics through the operation of the computer learning platform, choose their favorite and suitable group of aerobics movements, and give feedback to the teacher by voting, while the teacher will choose which group of aerobics movements to teach in class according to the students’ voting results. After the movements are taught, competition protocols are created. Movement routines are choreographed for students, and competitions are organized to develop more in-depth teaching content [12].


### 2.2. Competition Process

After specifying the designated tasks, we should prepare to start the competition process, determine a reasonable competition plan for students, decompose aerobics movements around specific knowledge points and considerations, and develop independent aerobics competition protocols and competition rules for students. Finally, the most organized teaching program is chosen to start the competition protocol, and the subtask classification menu of aerobics teaching is set in combination with the Internet search process, and the process related to aerobics competition is clarified in combination with the menu content to ensure the independence of the teaching design program [13].


### 2.3. Optimized Movement Choreography

After completing the teaching preparation phase on the Internet platform, students should be grouped heterogeneously according to their actual learning status, and each group should form a team to participate in the teacher-designed competition. After the grouping, students also have to rehearse through aerobics microvideo demonstration teaching to form a tacit understanding between each other and to guide them in the aerobics training practice process [14]. After watching the demonstration video, the teacher will give a focused lecture for the students, combined with offline teaching, to form an intuitive impression of aerobics-related movements through hands-on demonstration and optimize the students’ aerobics-related skills ability by combining online and offline [15].


### 2.4. Completion of the Contest Content

In the final aerobics competition stage, the teacher will realize to explain the rules and precautions related to the competition for students and emphasize the importance of joint cooperation within the group by mobilizing the participation of group students. After the competition, the teacher will enter all students’ competition results into the online platform database for subsequent teaching assessment use [16].


## 2.1. Teaching Tasks of Aerobics

Task design and establishment are the first step in building the network teaching mode of college aerobics, using the teaching content to the network platform to design various teaching tasks. Here, taking the teaching of mass aerobics as an example, the teacher can introduce the collective competition mode in the teaching process, combining with the competition mode to first stimulate students’ enthusiasm for learning aerobics [11]. First of all, the teacher will make a microvideo according to the requirements of the designed group competition mode, show it to the students through the multimedia platform, so that the students can understand the aerobics learning tasks intuitively, and, at the same time, give the students time to discuss and ask questions to the teacher, increasing the interactive communication link before the class.The teacher will then design a teaching module to show all the decomposition teaching videos of each movement of the aerobics course, and let students watch the decomposition movements of aerobics through the operation of the computer learning platform, choose their favorite and suitable group of aerobics movements, and give feedback to the teacher by voting, while the teacher will choose which group of aerobics movements to teach in class according to the students’ voting results. After the movements are taught, competition protocols are created. Movement routines are choreographed for students, and competitions are organized to develop more in-depth teaching content [12].


## 2.2. Competition Process

After specifying the designated tasks, we should prepare to start the competition process, determine a reasonable competition plan for students, decompose aerobics movements around specific knowledge points and considerations, and develop independent aerobics competition protocols and competition rules for students. Finally, the most organized teaching program is chosen to start the competition protocol, and the subtask classification menu of aerobics teaching is set in combination with the Internet search process, and the process related to aerobics competition is clarified in combination with the menu content to ensure the independence of the teaching design program [13].


## 2.3. Optimized Movement Choreography

After completing the teaching preparation phase on the Internet platform, students should be grouped heterogeneously according to their actual learning status, and each group should form a team to participate in the teacher-designed competition. After the grouping, students also have to rehearse through aerobics microvideo demonstration teaching to form a tacit understanding between each other and to guide them in the aerobics training practice process [14]. After watching the demonstration video, the teacher will give a focused lecture for the students, combined with offline teaching, to form an intuitive impression of aerobics-related movements through hands-on demonstration and optimize the students’ aerobics-related skills ability by combining online and offline [15].


## 2.4. Completion of the Contest Content

In the final aerobics competition stage, the teacher will realize to explain the rules and precautions related to the competition for students and emphasize the importance of joint cooperation within the group by mobilizing the participation of group students. After the competition, the teacher will enter all students’ competition results into the online platform database for subsequent teaching assessment use [16].


## 3. Data Acquisition

Students’ formation and mastery of motor skills are a direct reflection of the effect of aerobics teaching. Therefore, it is the key and difficult point in teaching to let students master technical movements and develop motor skills as soon as possible. Traditional teaching uses demonstration, explanation, and practice, but it is difficult to break through the detailed part of each link, and the teaching effect is not good. If modern information technology is used to assist teaching, it can make up for the shortcomings of traditional teaching, taking the teaching experiment as an example: the following is the result of the comparison experiment between teaching with modern information technology (1 group) and traditional teaching (2 groups) as shown in Table1.Table 1
Comparison table of aerobics technical assessment results.
GroupTotal numberExcellentGoodSecondaryPassFailNumber of people%Number of people%Number of people%Number of people%Number of people%Group 1 technology4010241026123071812Group 2 traditional405145149191435718The above experimental results prove that, with no significant difference in the students’ own quality between the two groups, the performance of group 1 students’ assessment is significantly better than that of group 2. We can clearly see the advantages of using multimedia information technology to assist teaching.In the learning process of motor skills, the application of information technology discipline integration can increase the amount of teaching information, stimulate students’ senses in many aspects such as visual, auditory, and tactile senses, and deepen students’ understanding and mastery of the structure of technical movements and the inner connection between movements, thus promoting the formation and mastery of students’ motor skills [17]. In addition, students can also improve their ability to analyze and solve problems on the basis of understanding movement techniques and develop their ability to investigate and learn [18].The teaching of aerobics is highly flexible and requires high sensitivity, coordination, and appreciation of students. The teaching process should take students as the starting point, keep abreast of students’ need and acceptability, adjust the teaching plan at any time, and feedback any new problems found [19]. It is important to be relevant to different individuals, rather than relying too much on modern information technology and indoctrination in a uniform manner. The teaching of aerobics is mainly through the teacher’s body language explanation, demonstration, and students’ repeated physical exercises to develop the body and enhance physical fitness, the course is practical, and the teaching process includes the teacher’s emotional expression and the emotional communication between teachers and students, which is the reason why the traditional teaching mode cannot be completely replaced [20]. The use of modern information technology in classroom teaching is very significant, and the effect is quite obvious. At the same time, the use of modern information technology in the teaching of aerobics class must take into account its advantages and disadvantages [21]. Therefore, only by reasonably combining the two can we create a lively classroom teaching atmosphere, stimulate students’ interest in learning, breakthrough teaching difficulties, enable students to acquire knowledge through multiple senses, enhance understanding and memory, provide feedback to improve motor skills, and improve students’ self-improvement, so that modern information technology can better serve aerobics teaching and bring a qualitative leap to aerobics teaching [22].


### 3.1. Program of This Article

In this paper, we propose a motion video keyframe extraction process based on the ST-FMP model and dynamic clustering, referred to as the ST-FMP algorithm.First, the ST-FMP model is used to estimate the human pose parameters of the rth frame image to obtain the set of human partsp=pu\u∈V by Eq. and obtain the human motion vector by using Eq., and that is,(1)fr=fr1,fr2,…,frK,where K equals to 3d; fri denotes the number of motion features; Ir denotes the motion feature of the i-th body part in the motion video.Then, all motion vectors of the motion video are combined to obtain the combined vector of motion features; that is,(2)fcom=f1,f2,…,fKT,where T denotes the number of frames of the motion video. Then, the LS algorithm is used to calculate the Lr score of fcom to obtain the discriminative motion feature vector, denoted as fsub.Finally, the feature curve offsub is constructed by using the Lr component vector, and the local extreme points on the feature function are selected as the set of candidate keyframes by the principle of extreme value determination, which is denoted as f˜c.In practice, it has been found that when the time interval between some candidate keyframes inf˜c is short or the difference in pose parameters is small, it leads to a large amount of redundant data in f∼c; conversely, it reduces the motion representation capability of f˜c. Therefore, a keyframe selection process is needed to filter out keyframes. Considering the good adaptiveness of the dynamic clustering method ISODATA algorithm [23], this paper selects this algorithm for f˜c clustering selection.


## 3.1. Program of This Article

In this paper, we propose a motion video keyframe extraction process based on the ST-FMP model and dynamic clustering, referred to as the ST-FMP algorithm.First, the ST-FMP model is used to estimate the human pose parameters of the rth frame image to obtain the set of human partsp=pu\u∈V by Eq. and obtain the human motion vector by using Eq., and that is,(1)fr=fr1,fr2,…,frK,where K equals to 3d; fri denotes the number of motion features; Ir denotes the motion feature of the i-th body part in the motion video.Then, all motion vectors of the motion video are combined to obtain the combined vector of motion features; that is,(2)fcom=f1,f2,…,fKT,where T denotes the number of frames of the motion video. Then, the LS algorithm is used to calculate the Lr score of fcom to obtain the discriminative motion feature vector, denoted as fsub.Finally, the feature curve offsub is constructed by using the Lr component vector, and the local extreme points on the feature function are selected as the set of candidate keyframes by the principle of extreme value determination, which is denoted as f˜c.In practice, it has been found that when the time interval between some candidate keyframes inf˜c is short or the difference in pose parameters is small, it leads to a large amount of redundant data in f∼c; conversely, it reduces the motion representation capability of f˜c. Therefore, a keyframe selection process is needed to filter out keyframes. Considering the good adaptiveness of the dynamic clustering method ISODATA algorithm [23], this paper selects this algorithm for f˜c clustering selection.


## 4. Experimental Results and Analysis

In this paper, the keyframes of aerobics action video are used as an example for simulation experiments, and the experimental results are compared with the manual extraction and the extraction results of the latest motion video keyframe algorithm [24].


### 4.1. Experimental Data Samples and Feature Training

First, three students were invited to perform 120 s of popular aerobics movements twice, and the videos were recorded with a resolution of 640 × 480 using a common webcam at a sampling rate of 20 frames/s. Then, 300 frames from each of the six sets of videos were selected as experimental data starting from the 10th rhythmic beat; finally, each joint position was manually marked in 1,800 frames as shown in Figure1. Finally, 13 joint positions of each aerobic movement were marked manually in 1,800 frames as shown in Figure 1. In the simulation experiment, 900 frames of each of the first and second movements were selected as the training sample data set and the test sample data set.Figure 1
Aerobics posture.In the training process, in order to enrich the human action characteristics of the positive samples, first, image operations such as rotation (at four angles of −15°, −7°, 5°, 7°, 5°, and 15°) and mirror image are used to expand 900 training samples to 10 × 900 frame positive sample set, recorded asΦp∗. Then, by adding key points at the midpoint of two adjacent joint positions, the coordinates of 26 flexible human body parts of each image in Φp∗ are calculated. Finally, the ho g feature of each human body part is calculated with a ho g unit of 5 × 5 pixels, and the parameters such as φI,lu and ψlu−lv in equation (3) are determined by linear support vector machine.


### 4.2. Evaluation Criteria

All keyframes in the test sample set are manually extracted, and the commonly used keyframe extraction accuracy as the algorithm performance evaluation criterion, that is,(3)accuracy=∑i=1mδfi,rin,where n and m represent the number of manually and algorithmically extracted keyframes, respectively; fi and ri represent the algorithmically and manually extracted keyframes; δ(.) is the similarity function between fi and ri; and the value of δ(−) is 1 when fi and ri are the same; otherwise, it is 0.


### 4.3. Analysis of Experimental Results

#### 4.3.1. Comparison of the Effectiveness of Spatio-Temporal Feature Embedding in Nondeterministic Parts

In order to test the accuracy of the human body parts in video with the embedding of spatio-temporal features of nondeterministic parts, three different ST-FMP model implementations were used to conduct comparative experiments for two human body parts, elbow, and knee, according to different error pixel thresholds, and the experimental results are shown in Figure2.Figure 2
Comparison of body part recognition accuracy under different ST-FMP implementations.
(a)(b)Compared with the FMP model, the accuracy of nondeterministic parts within a certain pixel error range is significantly improved when the ST-FMP algorithm is used for human pose estimation in motion video. For example, with an error threshold of 20 pixels, the accuracy of the ST-FMP algorithm for the elbow and knee is about 15 and 19 percentage points higher than that of the FMP model, respectively. However, the difference in accuracy is not significant when the pixel error threshold is larger (e.g., greater than 40 pixels) or smaller (e.g., greater than 10 pixels). Figure2 also shows that the recognition accuracy of the elbow and wrist (knee and ankle) is higher than that of the direct FMP model but lower than that of the ST-FMP algorithm when only the temporal continuity of the nondeterministic parts of the upper (lower) limbs is maintained. The experimental results show that the ST-FMP algorithm significantly improves the recognition performance of nondeterministic parts in motion videos by optimizing the body part recognition results with the local time continuity constraint of body parts.This paper also compares the performance of the FMP model [25] and the ST-FMP [26] models and their different implementations for motion video keyframe extraction. The experimental results are shown in Figure 3. From Figure 3, we can find that (1) when the accuracy error is less than 30 pixels, the accuracy of the ST-FMP algorithm is higher and more stable, with an average improvement of about 11 percentage points than the keyframe algorithm using FMP, and the keyframe extraction accuracy of the FMP model is still improved by about 3 percentage points when only the temporal continuity constraint of the nondeterministic parts of the upper limb (lower limb) is added; (2) when the error accuracy is greater than 35 pixels, the accuracy of ST-FMP algorithm is more stable than that of the FMP algorithm; (3) when the error accuracy is greater than 35 pixels, the performance of ST-FMP algorithm still improves compared with the FMP model, but the accuracy is reduced by about 15 percentage points.Figure 3
Experimental results of keyframe extraction based on FMP and ST-FMP.Meanwhile, when the accuracy error is 30 pixels and the keyframes are extracted by the ST-FMP algorithm with a different number of motion features, the accuracy curve of the algorithm does not fluctuate drastically in the interval from 15 to 60 motion features, and the performance is stable, as shown in Figure4.Figure 4
Comparison of keyframe accuracy with a different number of motion features.The above experimental results show that the ST-FMP algorithm is sensitive to the human pose estimation results and the local topology of the human body, and the spatio-temporal constraint of the nondeterministic parts is maintained for the keyframe extraction performance.


#### 4.3.2. Keyframe Algorithm Performance Comparison

In order to compare the performance of the keyframe algorithm, the simulation experimental results of the ST-FMP algorithm are compared with the running results of the priority-based action video keyframe extraction algorithm (abbreviated as KFE algorithm) of the literature [26] and the motion block keyframe extraction algorithm (abbreviated as motion block algorithm) of the literature [27], as shown in Table 2. The experimental results in Table 2 show that the ST-FMP algorithm outperforms the other two algorithms in terms of accuracy and recall. First, the accuracy of the ST-FMP algorithm is about 18 and 26 percentage points higher than that of the KFE algorithm and the motion block algorithm, respectively; the KFE algorithm uses the motion direction of 16 predefined blocks to represent human motion features, while the ST-FMP algorithm uses the first 15 LS human motion pose feature values of each action video to represent human motion. Therefore, the ST-FMP algorithm uses motion feature vectors with less redundancy, less noise, and accurate local motion representation of human parts, which are beneficial for improving the accuracy of keyframe and action recognition [28].Table 2
Performance comparison of different keyframe extraction algorithms.
AlgorithmAccuracyRecallST-FMP algorithm0.810.82KFE algorithm0.630.59Motion block algorithm0.550.69Secondly, Table2 also shows that the recall rate of the ST-FMP algorithm is also significantly better than the other two algorithms, with an average of about 23 and 13 percentage points higher, respectively. Kph algorithm and motion block algorithm are both keyframe techniques based on the difference of underlying image features, and they filter keyframes by comparing motion changes in different regions of the image using specific thresholds. The ST-FMP algorithm describes the local motion characteristics of human body parts, which is essentially a semantic model that can analyze and understand human body movements in motion videos at a higher level, such as the participating parts of human body movements and their motion change trends, and use semantic rules such as human pose similarity to filter keyframes, which can obtain more accurate keyframe results in line with people’s cognitive process [29, 30].


#### 4.3.3. Aerobics Auxiliary Effect

As illustrated in Figure5 aerobics results, the ST-FMP algorithm is closer to the manual extraction results and more suitable for keyframe-based action video review because it not only has a strong ability to express the local topology of body part actions but also has the ability to support keyframe screening based on semantic rules. At the same time, ST-FMP splits the human body parts into different flexible components, recognizes the human pose through the local topology of the flexible components, and reduces the continuous error of human pose estimation with the help of temporal feature edge constraints, so it has strong robustness in complex scenes.Figure 5
Effect of aerobics joint recognition.


## 4.1. Experimental Data Samples and Feature Training

First, three students were invited to perform 120 s of popular aerobics movements twice, and the videos were recorded with a resolution of 640 × 480 using a common webcam at a sampling rate of 20 frames/s. Then, 300 frames from each of the six sets of videos were selected as experimental data starting from the 10th rhythmic beat; finally, each joint position was manually marked in 1,800 frames as shown in Figure1. Finally, 13 joint positions of each aerobic movement were marked manually in 1,800 frames as shown in Figure 1. In the simulation experiment, 900 frames of each of the first and second movements were selected as the training sample data set and the test sample data set.Figure 1
Aerobics posture.In the training process, in order to enrich the human action characteristics of the positive samples, first, image operations such as rotation (at four angles of −15°, −7°, 5°, 7°, 5°, and 15°) and mirror image are used to expand 900 training samples to 10 × 900 frame positive sample set, recorded asΦp∗. Then, by adding key points at the midpoint of two adjacent joint positions, the coordinates of 26 flexible human body parts of each image in Φp∗ are calculated. Finally, the ho g feature of each human body part is calculated with a ho g unit of 5 × 5 pixels, and the parameters such as φI,lu and ψlu−lv in equation (3) are determined by linear support vector machine.


## 4.2. Evaluation Criteria

All keyframes in the test sample set are manually extracted, and the commonly used keyframe extraction accuracy as the algorithm performance evaluation criterion, that is,(3)accuracy=∑i=1mδfi,rin,where n and m represent the number of manually and algorithmically extracted keyframes, respectively; fi and ri represent the algorithmically and manually extracted keyframes; δ(.) is the similarity function between fi and ri; and the value of δ(−) is 1 when fi and ri are the same; otherwise, it is 0.


## 4.3. Analysis of Experimental Results

### 4.3.1. Comparison of the Effectiveness of Spatio-Temporal Feature Embedding in Nondeterministic Parts

In order to test the accuracy of the human body parts in video with the embedding of spatio-temporal features of nondeterministic parts, three different ST-FMP model implementations were used to conduct comparative experiments for two human body parts, elbow, and knee, according to different error pixel thresholds, and the experimental results are shown in Figure2.Figure 2
Comparison of body part recognition accuracy under different ST-FMP implementations.
(a)(b)Compared with the FMP model, the accuracy of nondeterministic parts within a certain pixel error range is significantly improved when the ST-FMP algorithm is used for human pose estimation in motion video. For example, with an error threshold of 20 pixels, the accuracy of the ST-FMP algorithm for the elbow and knee is about 15 and 19 percentage points higher than that of the FMP model, respectively. However, the difference in accuracy is not significant when the pixel error threshold is larger (e.g., greater than 40 pixels) or smaller (e.g., greater than 10 pixels). Figure2 also shows that the recognition accuracy of the elbow and wrist (knee and ankle) is higher than that of the direct FMP model but lower than that of the ST-FMP algorithm when only the temporal continuity of the nondeterministic parts of the upper (lower) limbs is maintained. The experimental results show that the ST-FMP algorithm significantly improves the recognition performance of nondeterministic parts in motion videos by optimizing the body part recognition results with the local time continuity constraint of body parts.This paper also compares the performance of the FMP model [25] and the ST-FMP [26] models and their different implementations for motion video keyframe extraction. The experimental results are shown in Figure 3. From Figure 3, we can find that (1) when the accuracy error is less than 30 pixels, the accuracy of the ST-FMP algorithm is higher and more stable, with an average improvement of about 11 percentage points than the keyframe algorithm using FMP, and the keyframe extraction accuracy of the FMP model is still improved by about 3 percentage points when only the temporal continuity constraint of the nondeterministic parts of the upper limb (lower limb) is added; (2) when the error accuracy is greater than 35 pixels, the accuracy of ST-FMP algorithm is more stable than that of the FMP algorithm; (3) when the error accuracy is greater than 35 pixels, the performance of ST-FMP algorithm still improves compared with the FMP model, but the accuracy is reduced by about 15 percentage points.Figure 3
Experimental results of keyframe extraction based on FMP and ST-FMP.Meanwhile, when the accuracy error is 30 pixels and the keyframes are extracted by the ST-FMP algorithm with a different number of motion features, the accuracy curve of the algorithm does not fluctuate drastically in the interval from 15 to 60 motion features, and the performance is stable, as shown in Figure4.Figure 4
Comparison of keyframe accuracy with a different number of motion features.The above experimental results show that the ST-FMP algorithm is sensitive to the human pose estimation results and the local topology of the human body, and the spatio-temporal constraint of the nondeterministic parts is maintained for the keyframe extraction performance.


### 4.3.2. Keyframe Algorithm Performance Comparison

In order to compare the performance of the keyframe algorithm, the simulation experimental results of the ST-FMP algorithm are compared with the running results of the priority-based action video keyframe extraction algorithm (abbreviated as KFE algorithm) of the literature [26] and the motion block keyframe extraction algorithm (abbreviated as motion block algorithm) of the literature [27], as shown in Table 2. The experimental results in Table 2 show that the ST-FMP algorithm outperforms the other two algorithms in terms of accuracy and recall. First, the accuracy of the ST-FMP algorithm is about 18 and 26 percentage points higher than that of the KFE algorithm and the motion block algorithm, respectively; the KFE algorithm uses the motion direction of 16 predefined blocks to represent human motion features, while the ST-FMP algorithm uses the first 15 LS human motion pose feature values of each action video to represent human motion. Therefore, the ST-FMP algorithm uses motion feature vectors with less redundancy, less noise, and accurate local motion representation of human parts, which are beneficial for improving the accuracy of keyframe and action recognition [28].Table 2
Performance comparison of different keyframe extraction algorithms.
AlgorithmAccuracyRecallST-FMP algorithm0.810.82KFE algorithm0.630.59Motion block algorithm0.550.69Secondly, Table2 also shows that the recall rate of the ST-FMP algorithm is also significantly better than the other two algorithms, with an average of about 23 and 13 percentage points higher, respectively. Kph algorithm and motion block algorithm are both keyframe techniques based on the difference of underlying image features, and they filter keyframes by comparing motion changes in different regions of the image using specific thresholds. The ST-FMP algorithm describes the local motion characteristics of human body parts, which is essentially a semantic model that can analyze and understand human body movements in motion videos at a higher level, such as the participating parts of human body movements and their motion change trends, and use semantic rules such as human pose similarity to filter keyframes, which can obtain more accurate keyframe results in line with people’s cognitive process [29, 30].


### 4.3.3. Aerobics Auxiliary Effect

As illustrated in Figure5 aerobics results, the ST-FMP algorithm is closer to the manual extraction results and more suitable for keyframe-based action video review because it not only has a strong ability to express the local topology of body part actions but also has the ability to support keyframe screening based on semantic rules. At the same time, ST-FMP splits the human body parts into different flexible components, recognizes the human pose through the local topology of the flexible components, and reduces the continuous error of human pose estimation with the help of temporal feature edge constraints, so it has strong robustness in complex scenes.Figure 5
Effect of aerobics joint recognition.


## 4.3.1. Comparison of the Effectiveness of Spatio-Temporal Feature Embedding in Nondeterministic Parts

In order to test the accuracy of the human body parts in video with the embedding of spatio-temporal features of nondeterministic parts, three different ST-FMP model implementations were used to conduct comparative experiments for two human body parts, elbow, and knee, according to different error pixel thresholds, and the experimental results are shown in Figure2.Figure 2
Comparison of body part recognition accuracy under different ST-FMP implementations.
(a)(b)Compared with the FMP model, the accuracy of nondeterministic parts within a certain pixel error range is significantly improved when the ST-FMP algorithm is used for human pose estimation in motion video. For example, with an error threshold of 20 pixels, the accuracy of the ST-FMP algorithm for the elbow and knee is about 15 and 19 percentage points higher than that of the FMP model, respectively. However, the difference in accuracy is not significant when the pixel error threshold is larger (e.g., greater than 40 pixels) or smaller (e.g., greater than 10 pixels). Figure2 also shows that the recognition accuracy of the elbow and wrist (knee and ankle) is higher than that of the direct FMP model but lower than that of the ST-FMP algorithm when only the temporal continuity of the nondeterministic parts of the upper (lower) limbs is maintained. The experimental results show that the ST-FMP algorithm significantly improves the recognition performance of nondeterministic parts in motion videos by optimizing the body part recognition results with the local time continuity constraint of body parts.This paper also compares the performance of the FMP model [25] and the ST-FMP [26] models and their different implementations for motion video keyframe extraction. The experimental results are shown in Figure 3. From Figure 3, we can find that (1) when the accuracy error is less than 30 pixels, the accuracy of the ST-FMP algorithm is higher and more stable, with an average improvement of about 11 percentage points than the keyframe algorithm using FMP, and the keyframe extraction accuracy of the FMP model is still improved by about 3 percentage points when only the temporal continuity constraint of the nondeterministic parts of the upper limb (lower limb) is added; (2) when the error accuracy is greater than 35 pixels, the accuracy of ST-FMP algorithm is more stable than that of the FMP algorithm; (3) when the error accuracy is greater than 35 pixels, the performance of ST-FMP algorithm still improves compared with the FMP model, but the accuracy is reduced by about 15 percentage points.Figure 3
Experimental results of keyframe extraction based on FMP and ST-FMP.Meanwhile, when the accuracy error is 30 pixels and the keyframes are extracted by the ST-FMP algorithm with a different number of motion features, the accuracy curve of the algorithm does not fluctuate drastically in the interval from 15 to 60 motion features, and the performance is stable, as shown in Figure4.Figure 4
Comparison of keyframe accuracy with a different number of motion features.The above experimental results show that the ST-FMP algorithm is sensitive to the human pose estimation results and the local topology of the human body, and the spatio-temporal constraint of the nondeterministic parts is maintained for the keyframe extraction performance.


## 4.3.2. Keyframe Algorithm Performance Comparison

In order to compare the performance of the keyframe algorithm, the simulation experimental results of the ST-FMP algorithm are compared with the running results of the priority-based action video keyframe extraction algorithm (abbreviated as KFE algorithm) of the literature [26] and the motion block keyframe extraction algorithm (abbreviated as motion block algorithm) of the literature [27], as shown in Table 2. The experimental results in Table 2 show that the ST-FMP algorithm outperforms the other two algorithms in terms of accuracy and recall. First, the accuracy of the ST-FMP algorithm is about 18 and 26 percentage points higher than that of the KFE algorithm and the motion block algorithm, respectively; the KFE algorithm uses the motion direction of 16 predefined blocks to represent human motion features, while the ST-FMP algorithm uses the first 15 LS human motion pose feature values of each action video to represent human motion. Therefore, the ST-FMP algorithm uses motion feature vectors with less redundancy, less noise, and accurate local motion representation of human parts, which are beneficial for improving the accuracy of keyframe and action recognition [28].Table 2
Performance comparison of different keyframe extraction algorithms.
AlgorithmAccuracyRecallST-FMP algorithm0.810.82KFE algorithm0.630.59Motion block algorithm0.550.69Secondly, Table2 also shows that the recall rate of the ST-FMP algorithm is also significantly better than the other two algorithms, with an average of about 23 and 13 percentage points higher, respectively. Kph algorithm and motion block algorithm are both keyframe techniques based on the difference of underlying image features, and they filter keyframes by comparing motion changes in different regions of the image using specific thresholds. The ST-FMP algorithm describes the local motion characteristics of human body parts, which is essentially a semantic model that can analyze and understand human body movements in motion videos at a higher level, such as the participating parts of human body movements and their motion change trends, and use semantic rules such as human pose similarity to filter keyframes, which can obtain more accurate keyframe results in line with people’s cognitive process [29, 30].


## 4.3.3. Aerobics Auxiliary Effect

As illustrated in Figure5 aerobics results, the ST-FMP algorithm is closer to the manual extraction results and more suitable for keyframe-based action video review because it not only has a strong ability to express the local topology of body part actions but also has the ability to support keyframe screening based on semantic rules. At the same time, ST-FMP splits the human body parts into different flexible components, recognizes the human pose through the local topology of the flexible components, and reduces the continuous error of human pose estimation with the help of temporal feature edge constraints, so it has strong robustness in complex scenes.Figure 5
Effect of aerobics joint recognition.


## 5. Conclusions

For aerobics video and real-time camera, the N-best algorithm is used to estimate the human pose parameters in a single frame; in the aerobics action video keyframe extraction experiment, the ST-FMP model improves the recognition accuracy of nondeterministic body parts of the flexible hybrid articulated human model (FMP) by about 15 percentage points and achieves 81% keyframe extraction accuracy.



---

*Source: 9269988-2022-03-22.xml*