# Dance Movement Recognition Based on Modified GMM-Based Motion Target Detection Algorithm

Authors: Jing Tian; Xiaoqiang Yang
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Under the synergistic development of social economy and science and technology, the intelligent teaching of dance has become more and more popular. This teaching method can not only decompose dance movements more specifically, which is easy for students to understand and master, but also get rid of the time and space limitation in traditional dance teaching and provide more independent learning opportunities for students. The problem of low accuracy of dance movement recognition due to complex gesture changes in dance movements is addressed. To this end, this paper proposes a modified motion target detection algorithm based on GMM. The dance movement recognition algorithm first extracts the features of dance movements through a feature pyramid network, then uses a multi-feature fusion module to fuse multiple features to improve the algorithm’s estimation of complex postures, and finally completes the recognition of dance movements. Experiments show that our method can maintain a certain recognition rate in the case where the background and target are easily confused, and can effectively improve the dance action recognition accuracy, thus realizing the action correction function for dancers. This also verifies the effectiveness of the action recognition algorithm for dance movement recognition.


---

## Body

## 1. Introduction

Human pose estimation is a key technique in the field of human action recognition, which is based on the principle of recognizing human pose by extracting features in images [1]. This technique can be used in intelligent dance-assisted training to obtain a skeleton map of the dancer’s posture by extracting features from the dancer’s image. Thus, the dancer’s dance movements are recognized, and the dancer’s posture is evaluated and corrected [2].As an aid to human eye vision and an important component of automated systems, computer vision is widely used in medical and transportation fields [3]. Compared with the human eye, the advantage of computer vision is that it has much higher computational power than the human brain and higher analysis capability for complex images [4]. Action recognition in dance video images is an important application area of computer vision technology, which can be applied to many scenarios, such as competition arbitration, introductory learning for dancers, and movement correction for professional dancers [5, 6].Compared with low-level action recognition such as gesture recognition and simple limb action recognition, dance action recognition has penetrated into the level of motion recognition [7, 8]. Therefore, when simple limb localization algorithms are applied to dance movement recognition, it is usually difficult to obtain high recognition accuracy [9, 10]. The difficulties of dance movement recognition mainly include the following three points.Dance movements are complex and variable. From the most basic action elements such as “lifting,” “sinking,” “rushing,” and “leaning” to the coherent and complex actions such as “standing beat swallow,” “pouncing step,” “cloud step,” and “turning over,” there is a great degree of freedom [11, 12]. Therefore, it is more difficult to identify each movement accurately.Obscuration in dance is a serious problem. If there is only one dancer, some of the dancer’s limbs may be obscured by themselves, making it difficult to identify the position of certain limbs; if there are multiple dancers, the dancers will obscure each other [13, 14]. In particular, the dancers’ clothes are loose, such as long dresses with skirt support, so the obscured area will be larger. In addition, the angle of the photo or video can also cause some obstacles to the recognition of dance movements [15].The coherence of dance movements is strong. In simple body movements, the coherence of the movements is weak. Generally, everyday body movements change slowly and each body movement remains the same over a period of time. However, in dance, all movements are coherent and fluid, and fewer movements remain stationary. Therefore, it is more difficult to accurately detect the boundaries of each dance movement in time.Early human posture estimation mainly focused on human contour features or part models. For example, the literature [16, 17] designed a human pose estimation algorithm based on part detection by extracting edge force field features through boosting classifier. Literature [18, 19], on the other hand, proposed an appearance model combining histogram of oriented gradients (HOG) and color features for human pose estimation. However, due to the complex variation of human pose, the traditional methods are difficult to achieve effective pose estimation [20, 21]. Therefore, deep learning-based methods are gradually used for human pose estimation. In 2015, deep learning-based human pose estimation algorithms started to return to the human skeleton heat map [22]. In 2016, a research team from the University of Michigan [23] designed an hourglass-like neural network structure for extracting multi-scale features for human pose estimation. In 2017, the literature [24] proposed an approach using partial affinity domain to obtain human skeleton maps. In addition, numerous deep learning-based algorithms for human pose estimation have been proposed, all of which can be used for dance movement recognition to assist dancers’ training [25]. The rapid change of dancers’ movements and the variability of their postures pose a challenge for dancer-assisted training intelligence.To this end, a dance movement recognition algorithm based on multi-feature fusion is designed in the paper for learning complex and variable dancer movement recognition.


## 2. Motion Recognition System

The motion recognition system in this paper consists of a human detection module, a pose and feature detection module, and a motion recognition module. First, a modified GMM-based motion target detection algorithm is used to detect and segment the moving human body from the video. For the detected binarized human region, the pose, pose change rate, and human position change information are extracted, and the pose evaluation function is introduced to improve the accuracy of pose detection. In the process of action recognition, an action recognition algorithm based on multi-feature fusion is proposed in this paper. The algorithm not only analyzes the shape features of human appearance, but also fuses the motion features of human body, so the recognition results are more accurate. The algorithm is easy to understand and implement, with a small amount of operations and fast recognition speed. The flow of the whole algorithm is shown in Figure1.Figure 1
Action recognition system based on multi-features fusion.


### 2.1. Human Body Detection

The first step in human motion recognition is to detect and segment the human body in motion or at rest. Due to the various colors and textures of human clothing, the uncertainty of human posture, and the uncertainty of the visual background, there is still no feasible method to detect the human body from static images. Therefore, this paper uses motion detection to extract human targets in video images.Background subtraction (BS) is a general and widely used technique for generating foreground masks (i.e., binary images containing pixels belonging to moving objects in the scene) by using a static camera. BS is the most commonly used method for detecting motion targets, and it can detect motion targets in indoor environments very well. It is found that the GMM background model adopts a global uniform update strategy, which highlights its shortcomings in dealing with complex target motion forms. The main manifestation is that the suspended motion targets are absorbed as part of the background, resulting in incomplete extracted motion targets such as people. This absorption phenomenon is unavoidable due to the need for an adaptive background model to handle slow background changes (e.g., illumination).Therefore, the results of motion segmentation and the recognition of the target can be used to guide the background update. For example, if the motion targetOij is a human object, the corresponding background update confidence fBg takes the value of 0, and the pixels at that point do not participate in the background update; otherwise, the corresponding background update confidence fBg takes the value of 1, and the pixels at that point participate in the background update. The region-based background update strategy avoids pose false detection caused by local human motion and improves the accuracy of action recognition using pose changes.


### 2.2. Motion State Characterization

Different features reflect the characteristics of human motion states from different perspectives. When selecting features, we should consider not only their distinguishability, but also the difficulty of their extraction. The object of this paper is the whole human body, including the limbs, and the goal of the study is to identify the typical daily movements (standing up, lying down, etc.) and sudden abnormal movements (falling down) in a complex environment. Therefore, the key features related to the shape and movement of the human body as a whole are considered in the human motion state characterization. In this paper, we adopt the idea of feature fusion to characterize the human motion state by fusing multiple features, because there are shortcomings in using appearance-based shape features alone or motion features alone.


#### 2.2.1. Posture Features

Human motion in the home environment consists mainly of several key pose transformations, so this paper selects the overall human pose information to describe the human motion state based on appearance and shape. Model-based pose acquisition can describe complex poses, but the model is difficult to initialize, computationally intensive, and prone to local minima, making it difficult to find globally optimal and robust parameters. Therefore, this paper chooses the human body width and height ratio, which is invariant to the target size and distance and has little influence on the viewpoint, to describe the human body’s pose characteristics.


#### 2.2.2. Pose Change Rate Feature

The posture of the human body always changes smoothly in normal daily movements, but the rate of posture change can be dramatic when a sudden abnormal (fall) situation occurs. In this paper, the motion feature of the rate of change of posture is introduced to detect the fall of a person. In the two actions of falling and lying down, the critical posture change process is similar and the posture change rate is different.


#### 2.2.3. Position Change Feature

It is impossible to determine whether the human body is walking or standing by only relying on the selected posture features and their posture change rates. In these two kinds of movements, the human posture changes from the standing posture to the standing posture, and the introduction of the motion information of position change can solve this problem. Then, how to determine whether the position of the “human target” changes? After region segmentation, we can obtain the position of the moving human target in the image coordinate system (coordinates of the top left vertex of the smallest rectangle containing the foreground target) and compare the positions of the two frames before and after to determine whether the target position has changed.


### 2.3. Attitude and Feature Detection

#### 2.3.1. Pose Feature Detection

The basic postures of human daily actions are defined as stand, sit, and lay, and the set of posturesP.(1)P=stand,sit,lay.The currently detected posept∈P is considered to be detected only when the human body is in motion, so the human body is considered to remain in the same pose when there is no motion information.Since the motion body detection is a high-dimensional signal, it is not easy to recognize the pose in this high-dimensional space, and feature transformation is needed for later classification. The body posture ratiok is calculated for different postures in 900 daily actions, and the threshold value of k is set to distinguish between standing, sitting, and lying postures based on the minimum probability of misjudgment criterion, as described in Table 1.Table 1
Thresholds for different actions.
GestureStandSitLayThresholdkk≥1.81.8>k≥0.7k<0.7One of the problems found in the experiments that affects the accuracy of posture is the false detection of posture. For example, when the human arm is unfolded, the standing posture is mistakenly detected as a sitting posture, as shown in Figure2.Figure 2
Posture detection.By constructing a posture evaluation function to eliminate the misjudgment of posture due to “unusual” human movements, define the posture evaluation functionS as in equation.(2)S=∑FgST,where ∑Fg=∑x,y∈TIx,y. is the area of the foreground image of the moving human body and ST=W∗H. is the area of the smallest external rectangle of the foreground image. From the expression of the evaluation function, we can see that the value of the function varies in the range [0, 1]. The value of the evaluation function is highest when ∑Fg=ST, and becomes very low when the human arm is expanded. If the pose evaluation function is low, it is considered as an undefined pose and is not involved in action recognition.The pose recognition algorithm in Haritaoglu compares the pose recognition algorithm in this paper with the algorithm in Haritaoglu. The pose recognition algorithm in Haritaoglu projects the foreground image of a moving human body in thex-axis and y-axis directions, and matches the projected contour lines with the training contour line templates of different poses to obtain the human pose. The pose recognition rates of the two algorithms are comparable, but in terms of complexity, Haritaoglu’s algorithm is more complex than the one in this paper.


#### 2.3.2. Pose Change Rate Detection

Define the ratio of the body posture ratio of the previous frame to the body posture ratio of the current frame as the inter-frame change rate of human posture, denoted byQ, i.e.,(3)Q=kt−1kt.Q characterizes how quickly a person’s posture changes: when a person maintains the same posture, Q is close to 1; when a person sits or lies down normally, Q increases slowly; when a person falls, Q increases rapidly. Thus, the rate of change of the human posture ratio can be used to detect the falling action. The rate of change of human posture during normal movement Q < 1.5 is obtained statistically.


#### 2.3.3. Position Detection

The position of the human body in the image coordinate system is defined asPt,i, and the position of the human body changes using the Euclidean distance metric, i.e., kt,i=Pt+1,i−Pt,i; when kt,i is greater than the threshold value Ks, the human body position is in motion. The discriminant method is(4)ifkt,inKs,then Action=1;else Action=0;In order to accurately determine whether the position of the human body has changed and to improve the robustness of the algorithm, the concept of confidence is introduced. The confidence level is used to measure the degree to which the human target is in motion and ranges from 0 to 40. A confidence level of 0 means that the human body is definitely in motion and a confidence level of 40 means that the target is definitely not in motion. IfAction=0, then the confidence level of the target is increased by 1; otherwise, the confidence level is zero. Given a confidence threshold, the target is considered to be in a nonmotion state when the confidence level is greater than the threshold, which is set to 20 in the text. The confidence level is defined as CAction, the initial value of CAction is 0, and the confidence level is normalized to UAction. If UAction is greater than 0.5, the human position changes and vice versa, and there is no change. The specific method is(5)if Action=0,thenCAction++;elseCAction=0;ifCActionn40nthenCAction=40UAction=CAction/40.


### 2.4. Action Recognition

Define the daily actions of a person: walk, sit down, stand up, lay down, get up, fall down, and standstill, which form the action set A.(6)A=walk,sit down,stand up,lay down,get up,fall down,stand still.The current detected actionat∈A. According to the regularity that different human actions are composed of different postures, this paper detects human actions by using the posture change combined with the frame-to-frame change rate feature and the position change feature, as shown in Table 2. pt denotes the posture at moment t, and pt−1 denotes the posture at moment t−1.Table 2
Condition setting of different actions.
ActionConditionsatptpt−1QUActionWalkStandStand<1.5≤0.5Sit downStandSit<1.5—Stand upSitStand<1.5—Lay downSit, standLay<1.5—Get upLaySit, stand<1.5—Fall downStand, sitSit, lay≥1.5—Stand stillStandStand<1.5>0.5In order to filter out meaningless or undefined actions, this paper introduces a threshold model of the minimum number of frames of pose duration. The threshold model gives the bottom line for performing action judgments, and action judgments are performed only when the number of pose duration frames of the observed sequence is greater than the threshold; otherwise, the observed sequence is considered as a meaningless or undefined action. According to this criterion, the minimum number of frames that can be statistically obtained to describe the pose of each action is 10; i.e., the threshold value in the threshold model is 10 frames (the video image acquisition rate is 30 frames/s). This method is able to eliminate the false detection of motion due to noise.


### 2.5. Dance Video Image Motion Pose Extraction and Joint Modeling

With the development of human behavior recognition field and the depth of research tasks, from the initial recognition of simple single actions under restricted conditions to the complex group behavior recognition in real natural scenes nowadays, both the information acquisition equipment and algorithm capability have posed serious challenges. As an important part of the behavior recognition process, the result of feature extraction largely affects the real time and accuracy of the behavior recognition effect. As a classical problem in the field of computer vision and machine learning, feature extraction is different from feature extraction in image space, and the feature representation of human action in video not only describes the human form in image space, but also must extract the human appearance and posture changes, which extends the feature extraction problem from two-dimensional space to three-dimensional space-time, which greatly increases the complexity of behavior mode expression and subsequent recognition tasks. At the same time, it also broadens the thinking of vision researchers in terms of solution ideas and technical methods. Human features are the information that can be extracted from the underlying video sequence to characterize the target behavior, such as color, contour, texture, depth, or human motion direction, speed, trajectory, as well as spatiotemporal interest points and spatiotemporal context.


### 2.6. Identifying Action Features Using Pose Feature Extraction Method

#### 2.6.1. Dancer’s Action Recognition Feature Classification

There are great differences between the dance movements of dancers and the daily movements of ordinary people, and many movements require dancers to use their arms and legs to complete, so when selecting the target area for background recognition, it is necessary to grasp the whole body movement information of dancers to accurately identify their movements. Dancer’s movement recognition can be divided into several categories: static features, mainly in the form of dancer’s human target size, color, body contour, depth, etc., which can convey the overall information of dancer’s movement, such as the current basic shape that can be derived from the dancer’s contour features; dynamic features, mainly in the form of dancer’s movement speed, direction, and trajectory, which can reflect the dancer’s movement path. The identification of these features can calculate the movement direction characteristics of the dancer and create conditions for modeling; spatiotemporal features are mainly manifested as spatiotemporal shapes, points of interest, etc.; descriptive features include the scene the dancer is in, surrounding objects, posture, etc.


#### 2.6.2. Dancer Pose Feature Extraction

The pose feature extraction method can be used in conjunction with a pose estimation sensor, which is commonly used in the field of motion tracking and robot vision to determine the directional points of a dancer’s motion. It can use the optical flow value in the pose estimator to filter the background information in the image to obtain the dancer’s joint coordinate region, and can eliminate the occlusion and influence of factors such as the dancer’s clothes on the dancer’s motion as shown in Figure3.Figure 3
Dancer’s posture characteristics.


### 2.7. Dancer Joint Point Recognition Modeling Using Kinect

The Kinect method is used to consider the human body as an axis composed of 25 joint point coordinates, and the human skeletal structure of the dancer is built with these joint points to obtain the human skeletal model of the dancer as shown in Figure4.Figure 4
Dancer’s joint point recognition model by 2 Kinect method.From the model, it can be seen that the dancer’s joint points are mainly distributed in the extremities. There is one joint point in the center of the head, neck, spine, and shoulder, and the most concentrated joint points are located in the upper limbs. The left upper limb has joint points. The principle of using these joint points to build the model is to accurately record the movement of each joint point during the dancers doing various movements, accurately identify each movement of the dancer, and thus output the correct skeleton of the dancer’s movements. With the skeleton model of dancers’ movements, the recognition accuracy and efficiency of the computer vision system for dancers’ movements can be significantly improved, and the whole recognition process is shown in Figure5.Figure 5
Dancer’s joint point recognition process.


## 2.1. Human Body Detection

The first step in human motion recognition is to detect and segment the human body in motion or at rest. Due to the various colors and textures of human clothing, the uncertainty of human posture, and the uncertainty of the visual background, there is still no feasible method to detect the human body from static images. Therefore, this paper uses motion detection to extract human targets in video images.Background subtraction (BS) is a general and widely used technique for generating foreground masks (i.e., binary images containing pixels belonging to moving objects in the scene) by using a static camera. BS is the most commonly used method for detecting motion targets, and it can detect motion targets in indoor environments very well. It is found that the GMM background model adopts a global uniform update strategy, which highlights its shortcomings in dealing with complex target motion forms. The main manifestation is that the suspended motion targets are absorbed as part of the background, resulting in incomplete extracted motion targets such as people. This absorption phenomenon is unavoidable due to the need for an adaptive background model to handle slow background changes (e.g., illumination).Therefore, the results of motion segmentation and the recognition of the target can be used to guide the background update. For example, if the motion targetOij is a human object, the corresponding background update confidence fBg takes the value of 0, and the pixels at that point do not participate in the background update; otherwise, the corresponding background update confidence fBg takes the value of 1, and the pixels at that point participate in the background update. The region-based background update strategy avoids pose false detection caused by local human motion and improves the accuracy of action recognition using pose changes.


## 2.2. Motion State Characterization

Different features reflect the characteristics of human motion states from different perspectives. When selecting features, we should consider not only their distinguishability, but also the difficulty of their extraction. The object of this paper is the whole human body, including the limbs, and the goal of the study is to identify the typical daily movements (standing up, lying down, etc.) and sudden abnormal movements (falling down) in a complex environment. Therefore, the key features related to the shape and movement of the human body as a whole are considered in the human motion state characterization. In this paper, we adopt the idea of feature fusion to characterize the human motion state by fusing multiple features, because there are shortcomings in using appearance-based shape features alone or motion features alone.


### 2.2.1. Posture Features

Human motion in the home environment consists mainly of several key pose transformations, so this paper selects the overall human pose information to describe the human motion state based on appearance and shape. Model-based pose acquisition can describe complex poses, but the model is difficult to initialize, computationally intensive, and prone to local minima, making it difficult to find globally optimal and robust parameters. Therefore, this paper chooses the human body width and height ratio, which is invariant to the target size and distance and has little influence on the viewpoint, to describe the human body’s pose characteristics.


### 2.2.2. Pose Change Rate Feature

The posture of the human body always changes smoothly in normal daily movements, but the rate of posture change can be dramatic when a sudden abnormal (fall) situation occurs. In this paper, the motion feature of the rate of change of posture is introduced to detect the fall of a person. In the two actions of falling and lying down, the critical posture change process is similar and the posture change rate is different.


### 2.2.3. Position Change Feature

It is impossible to determine whether the human body is walking or standing by only relying on the selected posture features and their posture change rates. In these two kinds of movements, the human posture changes from the standing posture to the standing posture, and the introduction of the motion information of position change can solve this problem. Then, how to determine whether the position of the “human target” changes? After region segmentation, we can obtain the position of the moving human target in the image coordinate system (coordinates of the top left vertex of the smallest rectangle containing the foreground target) and compare the positions of the two frames before and after to determine whether the target position has changed.


## 2.2.1. Posture Features

Human motion in the home environment consists mainly of several key pose transformations, so this paper selects the overall human pose information to describe the human motion state based on appearance and shape. Model-based pose acquisition can describe complex poses, but the model is difficult to initialize, computationally intensive, and prone to local minima, making it difficult to find globally optimal and robust parameters. Therefore, this paper chooses the human body width and height ratio, which is invariant to the target size and distance and has little influence on the viewpoint, to describe the human body’s pose characteristics.


## 2.2.2. Pose Change Rate Feature

The posture of the human body always changes smoothly in normal daily movements, but the rate of posture change can be dramatic when a sudden abnormal (fall) situation occurs. In this paper, the motion feature of the rate of change of posture is introduced to detect the fall of a person. In the two actions of falling and lying down, the critical posture change process is similar and the posture change rate is different.


## 2.2.3. Position Change Feature

It is impossible to determine whether the human body is walking or standing by only relying on the selected posture features and their posture change rates. In these two kinds of movements, the human posture changes from the standing posture to the standing posture, and the introduction of the motion information of position change can solve this problem. Then, how to determine whether the position of the “human target” changes? After region segmentation, we can obtain the position of the moving human target in the image coordinate system (coordinates of the top left vertex of the smallest rectangle containing the foreground target) and compare the positions of the two frames before and after to determine whether the target position has changed.


## 2.3. Attitude and Feature Detection

### 2.3.1. Pose Feature Detection

The basic postures of human daily actions are defined as stand, sit, and lay, and the set of posturesP.(1)P=stand,sit,lay.The currently detected posept∈P is considered to be detected only when the human body is in motion, so the human body is considered to remain in the same pose when there is no motion information.Since the motion body detection is a high-dimensional signal, it is not easy to recognize the pose in this high-dimensional space, and feature transformation is needed for later classification. The body posture ratiok is calculated for different postures in 900 daily actions, and the threshold value of k is set to distinguish between standing, sitting, and lying postures based on the minimum probability of misjudgment criterion, as described in Table 1.Table 1
Thresholds for different actions.
GestureStandSitLayThresholdkk≥1.81.8>k≥0.7k<0.7One of the problems found in the experiments that affects the accuracy of posture is the false detection of posture. For example, when the human arm is unfolded, the standing posture is mistakenly detected as a sitting posture, as shown in Figure2.Figure 2
Posture detection.By constructing a posture evaluation function to eliminate the misjudgment of posture due to “unusual” human movements, define the posture evaluation functionS as in equation.(2)S=∑FgST,where ∑Fg=∑x,y∈TIx,y. is the area of the foreground image of the moving human body and ST=W∗H. is the area of the smallest external rectangle of the foreground image. From the expression of the evaluation function, we can see that the value of the function varies in the range [0, 1]. The value of the evaluation function is highest when ∑Fg=ST, and becomes very low when the human arm is expanded. If the pose evaluation function is low, it is considered as an undefined pose and is not involved in action recognition.The pose recognition algorithm in Haritaoglu compares the pose recognition algorithm in this paper with the algorithm in Haritaoglu. The pose recognition algorithm in Haritaoglu projects the foreground image of a moving human body in thex-axis and y-axis directions, and matches the projected contour lines with the training contour line templates of different poses to obtain the human pose. The pose recognition rates of the two algorithms are comparable, but in terms of complexity, Haritaoglu’s algorithm is more complex than the one in this paper.


### 2.3.2. Pose Change Rate Detection

Define the ratio of the body posture ratio of the previous frame to the body posture ratio of the current frame as the inter-frame change rate of human posture, denoted byQ, i.e.,(3)Q=kt−1kt.Q characterizes how quickly a person’s posture changes: when a person maintains the same posture, Q is close to 1; when a person sits or lies down normally, Q increases slowly; when a person falls, Q increases rapidly. Thus, the rate of change of the human posture ratio can be used to detect the falling action. The rate of change of human posture during normal movement Q < 1.5 is obtained statistically.


### 2.3.3. Position Detection

The position of the human body in the image coordinate system is defined asPt,i, and the position of the human body changes using the Euclidean distance metric, i.e., kt,i=Pt+1,i−Pt,i; when kt,i is greater than the threshold value Ks, the human body position is in motion. The discriminant method is(4)ifkt,inKs,then Action=1;else Action=0;In order to accurately determine whether the position of the human body has changed and to improve the robustness of the algorithm, the concept of confidence is introduced. The confidence level is used to measure the degree to which the human target is in motion and ranges from 0 to 40. A confidence level of 0 means that the human body is definitely in motion and a confidence level of 40 means that the target is definitely not in motion. IfAction=0, then the confidence level of the target is increased by 1; otherwise, the confidence level is zero. Given a confidence threshold, the target is considered to be in a nonmotion state when the confidence level is greater than the threshold, which is set to 20 in the text. The confidence level is defined as CAction, the initial value of CAction is 0, and the confidence level is normalized to UAction. If UAction is greater than 0.5, the human position changes and vice versa, and there is no change. The specific method is(5)if Action=0,thenCAction++;elseCAction=0;ifCActionn40nthenCAction=40UAction=CAction/40.


## 2.3.1. Pose Feature Detection

The basic postures of human daily actions are defined as stand, sit, and lay, and the set of posturesP.(1)P=stand,sit,lay.The currently detected posept∈P is considered to be detected only when the human body is in motion, so the human body is considered to remain in the same pose when there is no motion information.Since the motion body detection is a high-dimensional signal, it is not easy to recognize the pose in this high-dimensional space, and feature transformation is needed for later classification. The body posture ratiok is calculated for different postures in 900 daily actions, and the threshold value of k is set to distinguish between standing, sitting, and lying postures based on the minimum probability of misjudgment criterion, as described in Table 1.Table 1
Thresholds for different actions.
GestureStandSitLayThresholdkk≥1.81.8>k≥0.7k<0.7One of the problems found in the experiments that affects the accuracy of posture is the false detection of posture. For example, when the human arm is unfolded, the standing posture is mistakenly detected as a sitting posture, as shown in Figure2.Figure 2
Posture detection.By constructing a posture evaluation function to eliminate the misjudgment of posture due to “unusual” human movements, define the posture evaluation functionS as in equation.(2)S=∑FgST,where ∑Fg=∑x,y∈TIx,y. is the area of the foreground image of the moving human body and ST=W∗H. is the area of the smallest external rectangle of the foreground image. From the expression of the evaluation function, we can see that the value of the function varies in the range [0, 1]. The value of the evaluation function is highest when ∑Fg=ST, and becomes very low when the human arm is expanded. If the pose evaluation function is low, it is considered as an undefined pose and is not involved in action recognition.The pose recognition algorithm in Haritaoglu compares the pose recognition algorithm in this paper with the algorithm in Haritaoglu. The pose recognition algorithm in Haritaoglu projects the foreground image of a moving human body in thex-axis and y-axis directions, and matches the projected contour lines with the training contour line templates of different poses to obtain the human pose. The pose recognition rates of the two algorithms are comparable, but in terms of complexity, Haritaoglu’s algorithm is more complex than the one in this paper.


## 2.3.2. Pose Change Rate Detection

Define the ratio of the body posture ratio of the previous frame to the body posture ratio of the current frame as the inter-frame change rate of human posture, denoted byQ, i.e.,(3)Q=kt−1kt.Q characterizes how quickly a person’s posture changes: when a person maintains the same posture, Q is close to 1; when a person sits or lies down normally, Q increases slowly; when a person falls, Q increases rapidly. Thus, the rate of change of the human posture ratio can be used to detect the falling action. The rate of change of human posture during normal movement Q < 1.5 is obtained statistically.


## 2.3.3. Position Detection

The position of the human body in the image coordinate system is defined asPt,i, and the position of the human body changes using the Euclidean distance metric, i.e., kt,i=Pt+1,i−Pt,i; when kt,i is greater than the threshold value Ks, the human body position is in motion. The discriminant method is(4)ifkt,inKs,then Action=1;else Action=0;In order to accurately determine whether the position of the human body has changed and to improve the robustness of the algorithm, the concept of confidence is introduced. The confidence level is used to measure the degree to which the human target is in motion and ranges from 0 to 40. A confidence level of 0 means that the human body is definitely in motion and a confidence level of 40 means that the target is definitely not in motion. IfAction=0, then the confidence level of the target is increased by 1; otherwise, the confidence level is zero. Given a confidence threshold, the target is considered to be in a nonmotion state when the confidence level is greater than the threshold, which is set to 20 in the text. The confidence level is defined as CAction, the initial value of CAction is 0, and the confidence level is normalized to UAction. If UAction is greater than 0.5, the human position changes and vice versa, and there is no change. The specific method is(5)if Action=0,thenCAction++;elseCAction=0;ifCActionn40nthenCAction=40UAction=CAction/40.


## 2.4. Action Recognition

Define the daily actions of a person: walk, sit down, stand up, lay down, get up, fall down, and standstill, which form the action set A.(6)A=walk,sit down,stand up,lay down,get up,fall down,stand still.The current detected actionat∈A. According to the regularity that different human actions are composed of different postures, this paper detects human actions by using the posture change combined with the frame-to-frame change rate feature and the position change feature, as shown in Table 2. pt denotes the posture at moment t, and pt−1 denotes the posture at moment t−1.Table 2
Condition setting of different actions.
ActionConditionsatptpt−1QUActionWalkStandStand<1.5≤0.5Sit downStandSit<1.5—Stand upSitStand<1.5—Lay downSit, standLay<1.5—Get upLaySit, stand<1.5—Fall downStand, sitSit, lay≥1.5—Stand stillStandStand<1.5>0.5In order to filter out meaningless or undefined actions, this paper introduces a threshold model of the minimum number of frames of pose duration. The threshold model gives the bottom line for performing action judgments, and action judgments are performed only when the number of pose duration frames of the observed sequence is greater than the threshold; otherwise, the observed sequence is considered as a meaningless or undefined action. According to this criterion, the minimum number of frames that can be statistically obtained to describe the pose of each action is 10; i.e., the threshold value in the threshold model is 10 frames (the video image acquisition rate is 30 frames/s). This method is able to eliminate the false detection of motion due to noise.


## 2.5. Dance Video Image Motion Pose Extraction and Joint Modeling

With the development of human behavior recognition field and the depth of research tasks, from the initial recognition of simple single actions under restricted conditions to the complex group behavior recognition in real natural scenes nowadays, both the information acquisition equipment and algorithm capability have posed serious challenges. As an important part of the behavior recognition process, the result of feature extraction largely affects the real time and accuracy of the behavior recognition effect. As a classical problem in the field of computer vision and machine learning, feature extraction is different from feature extraction in image space, and the feature representation of human action in video not only describes the human form in image space, but also must extract the human appearance and posture changes, which extends the feature extraction problem from two-dimensional space to three-dimensional space-time, which greatly increases the complexity of behavior mode expression and subsequent recognition tasks. At the same time, it also broadens the thinking of vision researchers in terms of solution ideas and technical methods. Human features are the information that can be extracted from the underlying video sequence to characterize the target behavior, such as color, contour, texture, depth, or human motion direction, speed, trajectory, as well as spatiotemporal interest points and spatiotemporal context.


## 2.6. Identifying Action Features Using Pose Feature Extraction Method

### 2.6.1. Dancer’s Action Recognition Feature Classification

There are great differences between the dance movements of dancers and the daily movements of ordinary people, and many movements require dancers to use their arms and legs to complete, so when selecting the target area for background recognition, it is necessary to grasp the whole body movement information of dancers to accurately identify their movements. Dancer’s movement recognition can be divided into several categories: static features, mainly in the form of dancer’s human target size, color, body contour, depth, etc., which can convey the overall information of dancer’s movement, such as the current basic shape that can be derived from the dancer’s contour features; dynamic features, mainly in the form of dancer’s movement speed, direction, and trajectory, which can reflect the dancer’s movement path. The identification of these features can calculate the movement direction characteristics of the dancer and create conditions for modeling; spatiotemporal features are mainly manifested as spatiotemporal shapes, points of interest, etc.; descriptive features include the scene the dancer is in, surrounding objects, posture, etc.


### 2.6.2. Dancer Pose Feature Extraction

The pose feature extraction method can be used in conjunction with a pose estimation sensor, which is commonly used in the field of motion tracking and robot vision to determine the directional points of a dancer’s motion. It can use the optical flow value in the pose estimator to filter the background information in the image to obtain the dancer’s joint coordinate region, and can eliminate the occlusion and influence of factors such as the dancer’s clothes on the dancer’s motion as shown in Figure3.Figure 3
Dancer’s posture characteristics.


## 2.6.1. Dancer’s Action Recognition Feature Classification

There are great differences between the dance movements of dancers and the daily movements of ordinary people, and many movements require dancers to use their arms and legs to complete, so when selecting the target area for background recognition, it is necessary to grasp the whole body movement information of dancers to accurately identify their movements. Dancer’s movement recognition can be divided into several categories: static features, mainly in the form of dancer’s human target size, color, body contour, depth, etc., which can convey the overall information of dancer’s movement, such as the current basic shape that can be derived from the dancer’s contour features; dynamic features, mainly in the form of dancer’s movement speed, direction, and trajectory, which can reflect the dancer’s movement path. The identification of these features can calculate the movement direction characteristics of the dancer and create conditions for modeling; spatiotemporal features are mainly manifested as spatiotemporal shapes, points of interest, etc.; descriptive features include the scene the dancer is in, surrounding objects, posture, etc.


## 2.6.2. Dancer Pose Feature Extraction

The pose feature extraction method can be used in conjunction with a pose estimation sensor, which is commonly used in the field of motion tracking and robot vision to determine the directional points of a dancer’s motion. It can use the optical flow value in the pose estimator to filter the background information in the image to obtain the dancer’s joint coordinate region, and can eliminate the occlusion and influence of factors such as the dancer’s clothes on the dancer’s motion as shown in Figure3.Figure 3
Dancer’s posture characteristics.


## 2.7. Dancer Joint Point Recognition Modeling Using Kinect

The Kinect method is used to consider the human body as an axis composed of 25 joint point coordinates, and the human skeletal structure of the dancer is built with these joint points to obtain the human skeletal model of the dancer as shown in Figure4.Figure 4
Dancer’s joint point recognition model by 2 Kinect method.From the model, it can be seen that the dancer’s joint points are mainly distributed in the extremities. There is one joint point in the center of the head, neck, spine, and shoulder, and the most concentrated joint points are located in the upper limbs. The left upper limb has joint points. The principle of using these joint points to build the model is to accurately record the movement of each joint point during the dancers doing various movements, accurately identify each movement of the dancer, and thus output the correct skeleton of the dancer’s movements. With the skeleton model of dancers’ movements, the recognition accuracy and efficiency of the computer vision system for dancers’ movements can be significantly improved, and the whole recognition process is shown in Figure5.Figure 5
Dancer’s joint point recognition process.


## 3. Dance Movement Recognition

The new algorithm uses a feature pyramid network (FPN) for feature extraction, then deepens the extraction for different scale features, and finally upsamples each feature to the original image size for feature fusion, as shown in Figure6. The residual block in the figure indicates the residual module as shown in Figure 7.Figure 6
Dance movement recognition.Figure 7
Dance movement recognition residual module.


### 3.1. Feature Pyramid-Based Backbone Network

The shallow features inC1,C2,C3 have a high spatial resolution. However, the semantic information contained is not sufficient, while the opposite is true for the deeper features in C4,C5.Based on the FPN backbone network, as shown in Figure8, it is difficult to identify human pose key points in complex environments, such as occluded hidden key points. The localization of such complex key points usually requires richer feature information, for which a multi-feature fusion module is designed in the paper.Figure 8
Backbone network.


### 3.2. Multi-Feature Fusion Module

The FPN-based backbone network is used to identify the estimation of simple key points, and the multi-feature fusion module is used to handle the estimation of more complex key points, whose structure is shown in Figure9. To obtain better local features, this paper enhances the feature resolution at each stage by upsampling operation. Finally, the individual features from the FPN are fused into the CONCAT operation.Figure 9
Multi-feature fusion structure diagram.During the training process, the FPN extracts features and returns the human skeleton key points, and simple key points will be basically completed in the FPN stage. For complex key points, such as occlusion and hiding, the multi-feature fusion module will further deepen the learning of features from each layer of the FPN and fuse them, and finally return to the human skeleton heat map.


### 3.3. Loss Function

Human pose estimation is a regression problem, and the common loss functions in regression problems are L1 loss function and L2 loss function. The dancer movement recognition in this paper adopts the regression of the key points of the dancer’s skeleton, so the algorithm in this paper adopts the loss function of L2 parametric optimized Euclidean distance, as shown in (1).(7)Lθ=12N∑i=1NFXi;θ−Fi22,where θ denotes the dancer movement recognition network parameters to be optimized; N is the total number of dancer images involved in the learning training; Xi denotes the current learning dancer image sample i; Fi denotes the heat map of the i-th dancer image; and FXi;θ− denotes the key points of the dancer skeleton regressed by the model heat map of the key points of the dancer’s skeleton regressed by the model.


## 3.1. Feature Pyramid-Based Backbone Network

The shallow features inC1,C2,C3 have a high spatial resolution. However, the semantic information contained is not sufficient, while the opposite is true for the deeper features in C4,C5.Based on the FPN backbone network, as shown in Figure8, it is difficult to identify human pose key points in complex environments, such as occluded hidden key points. The localization of such complex key points usually requires richer feature information, for which a multi-feature fusion module is designed in the paper.Figure 8
Backbone network.


## 3.2. Multi-Feature Fusion Module

The FPN-based backbone network is used to identify the estimation of simple key points, and the multi-feature fusion module is used to handle the estimation of more complex key points, whose structure is shown in Figure9. To obtain better local features, this paper enhances the feature resolution at each stage by upsampling operation. Finally, the individual features from the FPN are fused into the CONCAT operation.Figure 9
Multi-feature fusion structure diagram.During the training process, the FPN extracts features and returns the human skeleton key points, and simple key points will be basically completed in the FPN stage. For complex key points, such as occlusion and hiding, the multi-feature fusion module will further deepen the learning of features from each layer of the FPN and fuse them, and finally return to the human skeleton heat map.


## 3.3. Loss Function

Human pose estimation is a regression problem, and the common loss functions in regression problems are L1 loss function and L2 loss function. The dancer movement recognition in this paper adopts the regression of the key points of the dancer’s skeleton, so the algorithm in this paper adopts the loss function of L2 parametric optimized Euclidean distance, as shown in (1).(7)Lθ=12N∑i=1NFXi;θ−Fi22,where θ denotes the dancer movement recognition network parameters to be optimized; N is the total number of dancer images involved in the learning training; Xi denotes the current learning dancer image sample i; Fi denotes the heat map of the i-th dancer image; and FXi;θ− denotes the key points of the dancer skeleton regressed by the model heat map of the key points of the dancer’s skeleton regressed by the model.


## 4. Long-Time Target Tracking Algorithm

The extraction of features directly affects the accuracy and efficiency of target tracking. Given a new image frame, two filtering templates, target position and scale prediction, are learned based on HOG and texture features, respectively. The filtering output is calculated using (2).(8)fz=γHOGfHOGz+γtexftexz.The contribution of the two feature responses isγHOG,γtex, and the calculation rule is shown in (3) and satisfies γHOG+γtex=1.(9)γHOG=fHOGzfHOGz+ftexz,γtex=ftexzfHOGz/fHOGz+ftexz.The filtered response values of the two features are linearly weighted and fused using (3), and the maximum response value after fusion is used to determine the target region.In this paper, a simple and effective region suggestion generation scheme, EdgeBox, is chosen to generate candidate regions for the whole image and calculate their confidence scores, and the candidate region with the highest confidence is the retracing result. For an image, the edge information is used to determine the bounding box of the object. Based on the number of contours within the bounding box and the number of contours overlapping with the edges of the bounding box, the bounding box is scored and the candidate region information is determined according to the order of the scores. The candidate regions generated by EdgeBox include two types, one near the predicted target (denoted byBs) and one for the whole image region (denoted by Bh). b is a candidate bounding box in Bs or Bh. Define gb as the maximum filter response, and detect the tracking result by checking whether gb is lower than the given threshold T1, which is less than the threshold value to indicate tracking failure and start the retracking procedure. In the normal tracking process, the filter template of the previous frame is generally used to find the target position of the current frame. However, during retracking, the tracking result of the previous frame is no longer reliable, so it is necessary to select a reference image from the label library as the retracking head (the first frame is used as the retracking head by default). The confidence level of all images in the label library is read, and the Euclidean distance between a candidate frame bti and these images bt−j,j=1⟶t is calculated for the current frame.(10)Dbti,bt−j=exp−12σ2xti,yti−xt−j,yt−j2,σ is the initial target size diagonal length. Based on the confidence, Euclidean distance, the best element is found as the retracking head and trained online to update the filtering model and regain the normal tracking pattern, so that the algorithm maintains high robustness and efficiency in long-time tracking.(11)argmini,jβgbti+1−βDbti,bt−j,gbti>T1.Theβ in (5) is a weight parameter that adaptively adjusts the confidence, the contribution of the Euclidean distance. If gb|b∈Bs is greater than gz (the confidence level of the current template), the new target size wt,ht is defined as(12)wt,ht=αwt∗,ht∗+1−αwt−1,ht−1,It can also be expressed as(13)wt,ht=wt−1,ht−1+αwt∗,ht∗−wt−1,ht−1,wt∗,ht∗ indicates the width and height of the maximum confidence candidate region, and wt−1,ht−1 is the width and height of the previous tracking target [26–28].


## 5. Results and Analysis

### 5.1. Algorithm Validation

In order to better verify the accuracy of the dance movement recognition method designed in this paper, two data sets, PASCAL VOC2011-val set and Stanford 40 actions, are more commonly used, and the collected dance images were used to conduct the experiments. All experiments were performed on a computer with Intel Core i7-4790 CPU and 16 GB RAM and Windows 10 operating system based on Visual Studio 2010 development platform and OpenCV2.4.3 programming environment [29, 30]. The complexity of each dance movement and the number of images in the training set are shown in Table 3.Table 3
Number and complexity of training set images for each movement.
Dance action nameNumber of training set imagesAction complexityLift1851Sink2101Charge1222Cloud step1081Turn over1502Stand up and shoot the swallow1883To verify the effectiveness of the algorithm, all heat maps are visualized as shown in Figure8. The left image is the input image, the middle image is the singer’s skeletal key point heat map, the right image is the computed singer heat map, and the right image is the maximum probability key point and key point limb region obtained from the computed singer heat map. The algorithm is trained and recognized, the accuracy on the training set and test set is shown in Figure 10, and the recognition accuracy on the specific test set is shown in Table 4.Figure 10
Confusion matrix of 6 dance movements.Table 4
Recognition results.
Movement typeAccuracy (%)Arms crossed98.9Arms raised85.3One-hand wave95.2One arm open85.1Both arms open96.9Walking95.8From Table4, it can be seen that the average recognition accuracy of this research method on the test set is more than 92%, and the overall recognition accuracy is high, but the recognition accuracy of arm raising and one-hand waving does not reach 85%. The reason is that the amplitude of the arm is larger for the arm raise and one-handed wave compared with the other four actions, and the status of the other hand is uncertain when waving with one hand, thus reducing the accuracy of the algorithm [31, 32]. In addition, the arm raise and one-hand wave movements may have certain deviations due to the distance between the human body and the camera and the different shooting angles, resulting in recognition errors. Therefore, the recognition accuracy of these two actions is low. In addition, the difference of data set is also the reason for the low recognition rate of arm raising and one-hand waving by this algorithm. One arm open and arm raise movements are less frequent than other movements in dance, so they have some influence on the recognition accuracy of the algorithm.The recognition accuracy of each dance movement is shown in Table5. To avoid the influence of chance on the experimental results, the number of test set images for all movements is 100.Table 5
Action recognition accuracy test results.
Dance action nameAccuracy (%)Lift88.62Sink92.35Charge86.35Cloud step72.22Turn over85.98Stand up and shoot the swallow73.15From the experimental results, it can be seen that the accuracy of the dance action recognition method proposed in this paper is above 70% for all kinds of dance actions, and the highest can be above 90%. Under the condition of similar motion complexity, the accuracy rate of dance motion recognition is higher; under the condition of the same number of images in the training set, the lower the complexity, the higher the accuracy rate of motion recognition.In order to solve the problem of low recognition accuracy caused by the differences in data sets, this study constructed confusion matrices for the above six dance movements in the data set processing, as shown in Figure10, to ensure that the number of each movement data set is basically the same, and then used this algorithm for recognition. According to the recognition results, the classification accuracy of all six dance movements reached over 90%, indicating that increasing the number of data sets of arm raising and one-handed waving movements through the confusion matrix can effectively reduce the influence of human differences on the recognition results and improve the recognition accuracy.


### 5.2. Comparison of Algorithms

As can be seen from Table6, the accuracy of our algorithm is higher, reaching more than 92%, which indicates that the present algorithm is more ideal for recognition of dance movements and has a higher accuracy rate. In addition, it is known by the experimental time that the present algorithm runs at 0.75 frames/s on the Tesla P4 graphics card and can recognize multi-person movements in a single picture.Table 6
Comparison of recognition accuracy of different algorithms.
MethodAccuracy (%)Residual network four-channel algorithm79.2CalculatingHu moment algorithm89.9Our method92.7To further verify the recognition efficiency of the algorithm, we tested it in scenes with 0 to multiple people, respectively, and found that the time spent by the algorithm gradually increases with the number of people in the image, but the magnitude is small. The running time of the Hoff orientation calculator (HOC) algorithm increases linearly with the number of people, as shown in Figure11. In contrast, the running time of the algorithm in this study essentially did not increase significantly. This indicates that the present algorithm is more efficient and the algorithm performs better.Figure 11
HOC algorithm efficiency.


## 5.1. Algorithm Validation

In order to better verify the accuracy of the dance movement recognition method designed in this paper, two data sets, PASCAL VOC2011-val set and Stanford 40 actions, are more commonly used, and the collected dance images were used to conduct the experiments. All experiments were performed on a computer with Intel Core i7-4790 CPU and 16 GB RAM and Windows 10 operating system based on Visual Studio 2010 development platform and OpenCV2.4.3 programming environment [29, 30]. The complexity of each dance movement and the number of images in the training set are shown in Table 3.Table 3
Number and complexity of training set images for each movement.
Dance action nameNumber of training set imagesAction complexityLift1851Sink2101Charge1222Cloud step1081Turn over1502Stand up and shoot the swallow1883To verify the effectiveness of the algorithm, all heat maps are visualized as shown in Figure8. The left image is the input image, the middle image is the singer’s skeletal key point heat map, the right image is the computed singer heat map, and the right image is the maximum probability key point and key point limb region obtained from the computed singer heat map. The algorithm is trained and recognized, the accuracy on the training set and test set is shown in Figure 10, and the recognition accuracy on the specific test set is shown in Table 4.Figure 10
Confusion matrix of 6 dance movements.Table 4
Recognition results.
Movement typeAccuracy (%)Arms crossed98.9Arms raised85.3One-hand wave95.2One arm open85.1Both arms open96.9Walking95.8From Table4, it can be seen that the average recognition accuracy of this research method on the test set is more than 92%, and the overall recognition accuracy is high, but the recognition accuracy of arm raising and one-hand waving does not reach 85%. The reason is that the amplitude of the arm is larger for the arm raise and one-handed wave compared with the other four actions, and the status of the other hand is uncertain when waving with one hand, thus reducing the accuracy of the algorithm [31, 32]. In addition, the arm raise and one-hand wave movements may have certain deviations due to the distance between the human body and the camera and the different shooting angles, resulting in recognition errors. Therefore, the recognition accuracy of these two actions is low. In addition, the difference of data set is also the reason for the low recognition rate of arm raising and one-hand waving by this algorithm. One arm open and arm raise movements are less frequent than other movements in dance, so they have some influence on the recognition accuracy of the algorithm.The recognition accuracy of each dance movement is shown in Table5. To avoid the influence of chance on the experimental results, the number of test set images for all movements is 100.Table 5
Action recognition accuracy test results.
Dance action nameAccuracy (%)Lift88.62Sink92.35Charge86.35Cloud step72.22Turn over85.98Stand up and shoot the swallow73.15From the experimental results, it can be seen that the accuracy of the dance action recognition method proposed in this paper is above 70% for all kinds of dance actions, and the highest can be above 90%. Under the condition of similar motion complexity, the accuracy rate of dance motion recognition is higher; under the condition of the same number of images in the training set, the lower the complexity, the higher the accuracy rate of motion recognition.In order to solve the problem of low recognition accuracy caused by the differences in data sets, this study constructed confusion matrices for the above six dance movements in the data set processing, as shown in Figure10, to ensure that the number of each movement data set is basically the same, and then used this algorithm for recognition. According to the recognition results, the classification accuracy of all six dance movements reached over 90%, indicating that increasing the number of data sets of arm raising and one-handed waving movements through the confusion matrix can effectively reduce the influence of human differences on the recognition results and improve the recognition accuracy.


## 5.2. Comparison of Algorithms

As can be seen from Table6, the accuracy of our algorithm is higher, reaching more than 92%, which indicates that the present algorithm is more ideal for recognition of dance movements and has a higher accuracy rate. In addition, it is known by the experimental time that the present algorithm runs at 0.75 frames/s on the Tesla P4 graphics card and can recognize multi-person movements in a single picture.Table 6
Comparison of recognition accuracy of different algorithms.
MethodAccuracy (%)Residual network four-channel algorithm79.2CalculatingHu moment algorithm89.9Our method92.7To further verify the recognition efficiency of the algorithm, we tested it in scenes with 0 to multiple people, respectively, and found that the time spent by the algorithm gradually increases with the number of people in the image, but the magnitude is small. The running time of the Hoff orientation calculator (HOC) algorithm increases linearly with the number of people, as shown in Figure11. In contrast, the running time of the algorithm in this study essentially did not increase significantly. This indicates that the present algorithm is more efficient and the algorithm performs better.Figure 11
HOC algorithm efficiency.


## 6. Conclusion

Dance video image recognition should take into account the influence of dance background, costume, etc., on action recognition, as well as the obscuration and self-obscuration in the dancer’s own movements, and an action recognition technique that can accurately and completely record and reflect the dancer’s action information should be used in order to obtain the dancer’s body static information and action information. The practical test verifies the feasibility and high efficiency of the method, and the design can be widely applied to the visual perception interaction between human and service robots in the future, so that the robots can understand human actions better and faster, and engage in general service work according to human behaviors.



---

*Source: 6023784-2022-08-22.xml*