# Intelligent Action Recognition and Dance Motion Optimization Based on Multi-Threshold Image Segmentation

Authors: Yang Han; Kaza Mojtahe
Journal: Mobile Information Systems
 (2022)

---

## Abstract

In order to improve the recognition and optimization effect of dance movements, this paper combines multi-threshold image segmentation technology to perform intelligent recognition of dance movements. Moreover, this paper analyzes the window focus calibration process in the cascade multi-threshold expansion method and focuses the window on the object body to expand from small to large. Then, due to the local optimum problem of single-layer expansion, this paper proposes a cascade expansion method, and comparative experiments analyze the effect of this method on window optimization. In addition, this paper combines the actual situation of dance movement training to construct an intelligent action recognition and dance movement optimization system based on multi-threshold image segmentation. The research shows that the intelligent action recognition and dance motion optimization method based on multi-threshold image segmentation proposed in this paper has a good effect.


---

## Body

## 1. Introduction

The training level of dance skills can intuitively reflect the professional level of dancers. It contains both kinematics and medical skills and can show the graceful and steady movements of the human body in the form of performance. At the same time, human action has always been a topic of discussion in the field of science, and dance includes not only human kinematics but also exercise physiology and psychology. At present, the scope of scientific application of dance teaching in our country includes human body warm-up methods, body therapy, related research on various functions of the body, and exercise methods. These dance teaching methods should be used more in practical training. Of course, it is not just for dance learners to learn more complex physical movement skills but through the explanation and study of the above dance teaching methods to understand the various movements that the body can reflect in dance teaching. At the same time, they need to feel the charm from dance skills, and then they can be enthusiastic about learning dance skills and show stable, graceful, unrestrained, and euphemistic dance charm.The training of dance skills is a method, and only by mastering its essence, we can perform better dance. There is a certain way of learning. First, you must have a comprehensive understanding of the content of the subject of dance and your own experience. The so-called “one minute onstage, ten years of work off stage,” the mastery of skills does not happen overnight. Only by mastering the dance training method can you train in dance. Therefore, the discussion of dance skills in class should be paid special attention and then the specific demonstration of dance will be followed. In dance teaching, we must pay attention to the explanation of dance movements, so that students are familiar with the connection and transformation skills between the front and rear movements. The strength and flexibility of the body are inseparable during training, which has high physical requirements for some movements. Therefore, the understanding of basic knowledge is what students must master. Second, dance teaching is to cultivate the artistic quality of dancers. Art has the ability to express, and dance can better exercise the expressive ability of dancers. In dance training, although the fluidity and coherence of body movements make the viewer feel comfortable, this is not the ultimate goal of dance teaching. Third, the time available for dance teaching classes is limited. Before the dance training starts, students should warm-up enough to keep their bodies active so as to avoid physical injuries during the training. In this case, the warm-up has taken up most of the classroom teaching time, leaving little time for students to actually learn dance skills.This paper combines multi-threshold image segmentation technology for intelligent recognition of dance movements and builds an intelligent system to improve the optimization effect of subsequent dance movements.


## 2. Related Work

Using gesture recognition to analyze human motion behavior has become an important research direction of relevant experts and scholars in recent years [1]. Dancing tossing skills are an important indicator for evaluating dancers' basic skills and comprehensive abilities. Identifying dancing tossing gestures are of great significance to art teaching and research. At present, human gesture recognition has achieved great results. The method of literature [2] first establishes a high-level description model of human body structure information through the spatio-temporal features of human posture and then uses image feature recognition to identify human posture. This method needs to recognize the pose through the human contour with high integrity, and most of the images in the video cannot extract the human contour with high integrity, so the recognition accuracy is low. The method of literature [3] first extracts human behavior features, uses human behavior features to establish a spatio-temporal and/or the graph model and then uses video sequence moving target detection and recognition methods to identify human poses in the spatio-temporal and/or graph model. This method can only recognize the human posture in the two-dimensional image and does not consider the change of the video image, resulting in poor recognition stability. The method in [4] uses emotion-oriented speech recognition to identify human pose features. This method selects a single feature to identify human poses, resulting in the inability to accurately distinguish the target area and the background area, and the accuracy of the recognition results is poor. Literature [5] proposes a dance tossing gesture recognition model based on the spatial frequency domain features of contour images. First, the frequency domain features and airspace contour features of the contour images are extracted, and then the hidden Markov model is used to identify the dancing tossing gesture features, which have a strong recognition ability performance, which is convenient for people to recognize the dancing and tossing gestures in the video.Dance training refers to a teaching activity that comprehensively develops the professional ability of the dance body through systematic reconstruction of the body structure and function of dancers and processing of movement techniques based on scientific principles. Different training courses in dance teaching have different content settings and course objectives, but the core is to train dancers' physical quality, function, psychology, technical skills, and artistic performance by applying training load stimulation, so that it meets the needs of dance majors [6]. The physical function of a dancer is largely determined by genetic factors, and some physiological indicators with high heritability should be used as an important basis for selecting dance materials. In addition, acquired growth and development, especially long-term professional training, will also affect it [7]. Therefore, dancers' physical functions are different from ordinary people, and they are also very different from athletes, with professional characteristics. After a long period of professional dance training, the dancer's motor system, circulatory system, respiratory system, and nervous system will all undergo adaptive changes, which is a manifestation of the improvement of the level of physical function [8]. For example, the impact of dance training on the motor system is manifested in changes in bone mineral density (BMD) and increases in muscle strength. Studies have shown that ballet training increases hip bone mineral density in adolescent females [9]. Long-term professional dance training can significantly improve the muscle strength of dancers' lower limbs, especially the relative strength is significantly higher than that of ordinary people. Long-term dance training can also enhance the elasticity of the lungs and the strength of the respiratory muscles and improve respiratory function [10].The physical function monitoring and evaluation of dancers is to apply the theories and methods of medicine, psychology, biochemistry, dance physiology, dance biomechanics, and other disciplines to the dance training process and use comprehensive methods and means to study the training process and the training effect. The purpose is to help the instructors adjust the training plan scientifically, so that the dancers can achieve the best physical, psychological, and technical state, so as to maximize the training effect and professional ability [11]. Physical function, as an internal factor that affects dancers' physical fitness, technical skills, and expressiveness, should be an important part of the monitoring and evaluation of dancers in dance training and competitions. The functional monitoring and evaluation of dance training is an important manifestation of the scientization of dance training [12]. A series of physiological changes of the body when dancers participate in dance training is the objective reflection of the body's training load, that is, the body's ability to stress to dance training. If the training load is too small, the improvement of professional ability will not be obvious; if the training load is too large, it will not only fail to improve the professional level, but also may damage the health and cause the occurrence of dance injury [13]. Through scientific monitoring of dancers' physical function indicators and athletic ability indicators during dance training, we can study the regular characteristics of dancers' physical function changes during training, select dance materials, implement dance training, improve dancers' professional level and prevent excessive fatigue and fatigue. The occurrence of dance injury provides a scientific guarantee [14].The selection of physical function monitoring indicators in dance training should be based on comprehensive consideration of training purposes, training methods, monitoring purposes, and dancer training levels [15]. At the same time, it should be noted that the selected indicators should be as simple as possible, strong pertinence, high accuracy, easy to operate, and fast feedback. Each index should have its own independence and can more sensitively reflect the changes of the training load or function from different aspects and can complement each other in order to obtain a more comprehensive evaluation [16]. In the process of operation, special attention should be paid to individual differences, professional characteristics, and systematic observation of dancers. Some indicators can reflect the situation of a training session, while other indicators need to be tracked for a long time and combined with other indicators for comprehensive comparative evaluation. It reflects the training effect and dancer's physical function level within a week or a period of time. The functional monitoring and evaluation indicators of professional dancers should include the physical functions that can reflect the dancers’ skeletal muscle system, cardiovascular system, respiratory system, nervous system, and other systems [17].


## 3. Multi-Threshold Image Segmentation Algorithm

The superpixel segmentation method based on graph theory has problems such as parameter setting, low computational efficiency, and poor segmentation effect. Aiming at the above problems, this paper applies the adaptive threshold SLICO method to general object generators for the first time. The experimental results verify that this method can generate adaptive thresholds for superpixel segmentation and has better segmentation results at faster processing speed, which solves the above problems well. The post-processing method proposed in this paper contributes in exploiting superpixel cues.Graph theory-based superpixel segmentation is a top-down global segmentation method. It regards the whole image as a weighted undirected graph, in which each pixel represents a node, the adjacent relationship between pixels represents a node, and the difference or similarity between pixel features corresponds to the weight on the edge. Then, the previous nodes are divided by various segmentation criteria on the established graph, so as to complete the superpixel segmentation process.The superpixel segmentation method based on graph theory maps an image to an undirected graph and defines the weight of each edgee as w(e). W(e) represents the degree of difference between the vertices vi and vj connected by edge e. The vertex set in the graph is V, and dividing V into different regions is a division s. A connected subgraph of the graph G′=(V, E′) corresponds to each small region C ∈ S, where the non-empty subset of E is denoted by E′.This method defines the internal difference of CeV as the maximum weight of the component’s minimum spanning tree MST(C, E), as shown in the following formula:(1)intC=maxe∈MSTC,Ewe.Then, it defines the difference between two partsC1,C2⊆V as the minimum weight edge connecting the two parts. The specific definition is shown in the following formula:(2)difC1,C2=minvi∈C1,vj∈C2,vi,vj∈Ewvi,vj.The threshold function is used to control that the degree of difference between the components must be greater than the minimum degree of internal difference, and the pairwise comparison judgment formulaDC1,C2 is specifically defined as follows:(3)DC1,C2=trueifdifC1,C2>MIntC1,C2falseotherwise..Its minimum internal difference degree MInt is defined as follows:(4)MIntC1,C2=minIntC1+τC1,intC2+τC2.Among them, for some regions, the function Int(C) does not perform well for data local feature evaluation. For example, in extreme cases, the region size |C| = 1 and Int(C) = 0. Therefore, a threshold functionτC=k/C based on the region size needs to be used, where k is a constant. That is, for small regions, stronger boundary evidence needs to be relied on.The graph-based method uses clustering to divide the nodes on the graph, and the minimum spanning tree of its pixel set is the superpixel. The segmentation effect diagram of the graph-based method is shown in Figure1.Figure 1
Graph-based segmentation rendering. (a) Original image. (b) Segmentation rendering.
(a)(b)It can be observed from Figure1 that although the graph-based method can segment images more accurately, the segmented superpixels are not very regular, there are many glitches, and multiple parameters need to be set. Usually, the parameter setting will have a greater impact on the final segmentation effect.The algorithm initializes the seed point. The seed point spacing isS=N/K, where N is the number of pixels in the image, and K is the number of superpixels to be segmented. The algorithm then centers the seed on the edge of the image and moves it to the lowest gradient position within the 3×3 neighborhood. The purpose of this is to avoid centering superpixels on edges and reduce the possibility of seeding superpixels with noisy pixels. In the assignment step, each pixel is associated with the nearest cluster center whose search area and its location overlap, as shown in Figure 2. Since the expected spatial extent of a superpixel is a region roughly S × S in size, SLIC performs a similar pixel search in a 2S × 2S region around the center of the superpixel.Figure 2
SLIC reduces the search range of pixels. (a) Search in the entire image. (b) SLIC searches within the restricted area.
(a)(b)Finally, the distance is measured, that is, the distance between the pixel point and the cluster center is calculated, and the most similar seed point label is attached to the pixel, and the process is iterated until convergence. The color of a pixel is represented in the CIELAB color space [l,a,b]T, the range of which is known. The position of the pixel is represented by [x,y]T. In order to combine the two distances into a single metric, it is necessary to normalize their respective color proximity max distance d and spatial proximity max distance d within the cluster. Then, the similarity D between pixels j and k is defined, and the specific formula is as follows:(5)dc=lj−lk2+aj−ak2+bj−bk2,dp=xj−xk2+yj−yk2,Ds=dc+mSdp.Making full use of the superpixel set generated after image segmentation plays a very important role in the post-processing process of this paper. Therefore, this part will focus on introducing the superpixel strategy, detailing the relationship between superpixel, objects and windows, and the application strategy. First, we will focus on the concept of superpixel straddling.Superpixel segmentation divides objects into small regions of uniform color or texture. Therefore, an important property of superpixels is to preserve the boundaries of objects, which can be considered as dividing objects into many superpixels. However, these superpixels do not cross the original boundary of the object.We assume that the initial generator generates a window set BB, and the initial window bb is some window in BB. A superpixel sp is considered to span the window bb if at least one pixel of the superpixel sp is outside bb and one pixel is inside bb. The superpixel spanning fraction SS is defined as follows:(6)SSbb,θss=1−∑minsp\bb,sp∩bbbb.Among them,SPθss is the set of superpixels with segmentation scale θss. The above formula also calculates the area sp∩bb of the superpixel inside the window bb and the area sp\bb of the outer area, and the minimum value of the above two items is divided by the window area value.If most of the spanning superpixel is inside or outside, its numerator value is small and contributes little to the overall summation formula. Therefore, the SS score will be highest when a window bb closely surrounds an object with a numerator value of 0, which is exactly the desired window.Figure3 is a superpixel spanning clue map. Figure 3(b) is the image obtained by superpixel segmentation of Figure 3(a). Observing the image, it is found that the object is composed of a subset of superpixels, and the boundary outside the superpixel is basically the boundary of the object itself. The superpixel spanning degree SSbb,θss is calculated according to the relationship between the window bb and the superpixel. In Figure 3(c), most of the superpixels are inside the boundary of the window bbs, whereas most of the superpixels are outside the window w2. Therefore, bb1 has a higher SS score than bb2, and bbs has a higher score because of its ability to closely fit the object.Figure 3
Superpixels spanning clues. (a) Original image. (b) Graph-based segmentation graph. (c) Superpixels spanning clues.
(a)(b)(c)In order to facilitate the application of the above concepts, a concept of superpixel straddling degree (SSD) is proposed. The specific definitions are shown in the following formulas:(7)SSDsp,bb=sp∩bbsp,(8)SPst=spstSSDspst,bb∈0,1.The definition of superpixel density ST is shown in the following formula:(9)STbb=∑sp∈SPsp⋅δsp−sp∩bbbb.Among them,δ denotes an impulse function that is non-zero if and only if sp−sp∩bb=0, where SP refers to the set of superpixels.The window focus correction procedure is used to adjust the initial window to focus on the main body of the object, which facilitates subsequent superpixel-based expansion across the threshold.Generally speaking, the initial window set generated by the initial generator can be directly input into the cascaded multi-threshold expansion model for correction. However, the window tightness of the initial window is actually very low, which may lead to the failure of the correction process. Taking BNG as an example, the windows have only 36 uniform scales, and such windows generally do not closely surround objects, so their window tightness is usually very low. However, cascading multi-threshold expansion directly on such a window usually cannot achieve very good results. The reason is that for such an already large window, expanding and optimizing it will usually only increase the positioning error of the window and frame more backgrounds.Therefore, using the window focus calibration method to generate a calibrated window is more beneficial for subsequent expansion. Since the true boundary of the object is not known in practical situations, the method adopts an approximation idea to calibrate the initial window. The window is aligned with the edge alignment of a superpixel set if it can just closely surround a subset of superpixels. If an initial window bb is given, and the entire superpixel set isSPall, the calculation formula of the superpixel set SPin inside the window and the superpixel set SPst spanned by the window is defined as follows:(10)SPin=sp∈SPSSDsp,bb=1,(11)SPst=sp∈SP0<SSDa,b<1.We assume that window bb(SP) is the smallest window that can just tightly enclose the superpixel set SP, andIOUbbm,bbn denotes the intersection ratio between windows bbm and bbn. Then, according to the intersection ratio between each superpixel and the window of the spanned superpixel set SPst, the superpixels spi,...,spj inside SPst are sorted in descending order, and they satisfy the following formula:(12)IOUbbSPin∪sm,bb≥IOUbbSPin∪sm,bb,∀m<n.Single-layer multi-threshold extension (MTSE) extends the focus-corrected window in multiple superpixel cross-threshold directions. Formula (13) represents the process of modifying the window ab according to the superpixel span threshold tr.(13)SPt1ab=sp∈SPSSD=1.IfSPinab=sp∈SPSSD=1 is satisfied, it means that the window ab tightly encloses the superpixel set that satisfies this condition. If there are other superpixels that satisfy the constraint SSD≥t1, they are combined into the previously tightly enclosed superpixel set SPinab, and the scale of the window ab will be adjusted accordingly to just tightly surround the new superpixel set SPt1ab. The threshold tl is the spanning criterion for single-layer superpixel expansion. The value of tl is smaller, the deformation of the corresponding window ab is larger, so that it is easier to approach the standard window of the manually calibrated object.


## 4. Intelligent Action Recognition and Dance Motion Optimization Based on Multi-Threshold Image Segmentation

After preprocessing the dance video image to be recognized through erosion, expansion, center normalization, etc., the energy map of dance tossing posture is extracted. In this paper, discrete cosine transform and transformation methods are used to extract the frequency domain features and multi-directional and multi-resolution spatial contour features in the energy map of dancing tossing poses. Moreover, this paper uses the extracted features to distinguish the high and low frequency components of the human body in the dance tossing posture and capture the detailed information of the dance tossing features. In addition, this paper adopts the feature-level fusion method to fuse the above features to obtain the spatial frequency domain feature vector set of the contour image for processing. Intelligent action recognition and dance motion optimization based on multi-threshold image segmentation are shown in Figure4.Figure 4
Intelligent action recognition and dance motion optimization based on multi-threshold image segmentation.In the process of visual motion tracking in basic dance training, the spatial limit range of the 3D motion tracking system should be selected first, and then the extreme points of the 3D motion tracking system should be located. In the extreme lattice, the lattice data of each key action and the parameters of each key action lattice vector are obtained, and then the key lattice feature vector is calculated. The three-dimensional tracking and decomposition of dance visual movements is completed by applying a multi-directional dual-camera space plane imaging system, as shown in Figure5.Figure 5
Flow chart of the dance visual action tracking decomposition method.As shown in Figure6, the system mainly includes three components, namely the optical motion capture system, the real-time motion recognition engine, and the animation drive engine. The motion capture system is responsible for capturing the user’s dance movements in real time and expressing them into effective feature forms such as joint angle feature forms. The action recognition engine is responsible for segmenting and identifying the captured dance movements in real time and determining whether they are legal movements so that the system can make corresponding decisions. If the current input action is a legal action, the animation driving engine needs to determine the frame block exactly corresponding to the current input segment from the interactive dance action of the legal action. At the same time, it displays the actions with animated characters, so that the user's dance and the displayed interactive dance are kept in sync with each other as much as possible.Figure 6
Overview of the structure of the dance interaction system.In this paper, the improved inverse kinematics method is used to decompose the video action tracking, and the multi-key points and key action parts in the dance movement are marked with the feature points and the marking parameters are set. The static displacement data of dance movements is calculated through the principle of visual action tracking and decomposition, and the dynamic capture is performed after the image segmentation and identification are clear. If the image is not segmented and marked clearly, the coefficient correction and recalculation will be performed until the dance action image is clearly segmented and marked. When a certain action is analyzed in detail, the system automatically tracks and decomposes without the need for teachers to make corresponding actions. The three-dimensional action tracking and decomposition imaging can be performed through the projection system or the teaching display. On this basis, the effect of the intelligent action recognition and dance motion optimization method based on multi-threshold image segmentation proposed in this paper is verified, and its dance motion segmentation effect and dance motion optimization effect are calculated, as shown in Tables1 and 2.Table 1
Segmentation effect of dance movements.
NumIdentify190.19291.69390.03486.65590.52684.81791.70885.87986.921087.201188.471291.591390.841491.261588.491685.971789.871886.011988.382087.812186.552286.802385.542490.502584.882684.712791.812884.174288.734390.704488.254589.894688.354790.784888.134991.395088.215184.635291.875390.375487.135588.715690.47Table 2
Optimization effect of the dance movement.
NumOptimization183.38286.19381.94482.32582.47682.11779.56887.95983.831080.671178.211282.571383.051481.571577.591682.941784.511878.031985.242086.992186.012285.822385.062477.112582.832684.862779.982884.314287.944381.554477.254580.484687.354785.834881.684977.335078.295180.925286.295384.075477.465580.345679.97It can be seen from the above research that the intelligent action recognition and dance motion optimization method based on multi-threshold image segmentation proposed in this paper has good results.


## 5. Conclusion

If we want to dance gracefully and naturally, and make our body movements full of power and beauty, then we need to match our body movements and breathing to the right musical frequency. When discussing a certain movement or a technical point in dance teaching, dance teachers have to make corresponding movements and explain them accordingly, which causes a lot of inconvenience. The traditional dance basic training adopts the uncalibrated global visual feedback method for training, that is, the traditional video decomposition training. When the visual motion is decomposed, the visual changes of adjacent key nodes and cannot be calculated, and there are problems of large 3D visual motion tracking error and unclear decomposition contours. This paper combines multi-threshold image segmentation technology to perform intelligent dance movement recognition, builds an intelligent system and improves the effect of subsequent dance movement optimization. The research shows that the intelligent action recognition and the dance movement optimization method based on multi-threshold image segmentation proposed in this paper has good results.



---

*Source: 5776642-2022-09-24.xml*