# Human Motion Representation and Motion Pattern Recognition Based on Complex Fuzzy Theory

Authors: Xiangkun Li; Guoqing Sun; Yifei Li
Journal: Complexity
 (2021)

---

## Abstract

With the development of science and technology, the introduction of virtual reality technology has pushed the development of human-computer interaction technology to a new height. The combination of virtual reality and human-computer interaction technology has been applied more and more in military simulation, medical rehabilitation, game creation, and other fields. Action is the basis of human behavior. Among them, human behavior and action analysis is an important research direction. In human behavior and action, recognition research based on behavior and action has the characteristics of convenience, intuition, strong interaction, rich expression information, and so on. It has become the first choice of many researchers for human behavior analysis. However, human motion and motion pictures are complex objects with many ambiguous factors, which are difficult to express and process. Traditional motion recognition is usually based on two-dimensional color images, while two-dimensional RGB images are vulnerable to background disturbance, light, environment, and other factors that interfere with human target detection. In recent years, more and more researchers have begun to use fuzzy mathematics theory to identify human behaviors. The plantar pressure data under different motion modes were collected through experiments, and the current gait information was analyzed. The key gait events including toe-off and heel touch were identified by dynamic baseline monitoring. For the error monitoring of key gait events, the screen window is used to filter the repeated recognition events in a certain period of time, which greatly improves the recognition accuracy and provides important gait information for motion pattern recognition. The similarity matching is performed on each template, the correct rate of motion feature extraction is 90.2%, and the correct rate of motion pattern recognition is 96.3%, which verifies the feasibility and effectiveness of human motion recognition based on fuzzy theory. It is hoped to provide processing techniques and application examples for artificial intelligence recognition applications.


---

## Body

## 1. Introduction

Because of the diversity of human movements, noisy scenes, and the changeable perspective of camera motion, it is more difficult to recognize human movements. Fuzzy recognition has become a pattern of human behavior recognition. Firstly, human behavior is not a single action, but a series of actions, such as picking up the mobile phone, standing up, and walking to answer the phone. At present, the mainstream action recognition algorithms such as dynamic time warping algorithm and hidden Markov model are generally used to analyze a single action, so if we want to identify a series of actions, we need to divide them into a single action for further research [1]. Secondly, the dimension selection of training samples also brings great difficulties to recognition. If the dimension of training samples is too small, the training samples may not include all human actions in the actual scene [2–4]. For example, Gaofeng et al. motion images are only the placement of the hands of the experimenter when walking. The experiment shows that the hands of the experimenter are only shaking slightly on both sides of the body when walking. The data training set of the joint points of the hands does not contain all the human movements. If the dimension chosen is too large, it will increase our calculations and complexity, resulting in extremely low efficiency. At the same time, the human joint point data will be mixed with irrelevant physical structural characteristics information. For example, when the person picks up the phone, the movement of the lower limbs of the body has little effect on the docking call, which causes interference in the motion recognition. Motion recognition can be fundamentally transformed into the problem of fuzzy control pattern recognition [5–8]. The development of fuzzy control technology can be divided into the following stages:The First Stage. Basic fuzzy controller: In many practical production processes, the precise model of the controlled object is usually difficult to obtain, so the same control method can not be used to complete the control of the controlled process. Its basic idea is to make some rules’ base through the experience of operators for those systems which are difficult to establish precise models so as to implement control, apply them to the actual controlled objects, adopt appropriate strategies, and then realize the control of the controlled objects [9].The Second Stage. Self-organizing and self-learning fuzzy controllers: Because of the complexity and variability of some controlled objects, it is difficult for people to acquire the practical experience of manipulators completely and accurately, thus obtaining very imperfect and rough control rules. At this time, if basic fuzzy controllers are used, the control effect will be affected. On the other hand, even if the obtained control rules are relatively perfect because the parameters of the controlled process are time-varying, if the controlled process is always controlled according to the fixed rules, the expected control effect can not be achieved [10].The Third Stage. Intelligent fuzzy control: Compared with basic fuzzy controllers, although self-learning and self-organizing fuzzy controllers can play a good role in some problems, self-organizing fuzzy control is still based on practical experience, according to people’s will, to complete a full division, so that it can be adjusted in the allowable interval, resulting in its limited control capacity. In this case, in order to effectively control the complex production process, the initial expert system is constructed through the fuzzy control rules while continuously understanding the mechanism process of the controlled object and combining it with the manipulation experience. Then, the actual production process is continuously supplemented, modified, and perfected by the production learning system, and the empirical specialty of mechanism operation is formed. In home system, the process of problem processing is decided by the software of production learning system, and the original system knowledge is revised by feedback. This is called intelligent fuzzy control [11, 12].According to the background of the Internet program and the above problems, medical wisdom and family has become an irresistible trend, and promoting home-based care and community care has become the best solution [13, 14]. Home care and community care can not only facilitate the elderly and reduce the burden of the family but also reduce the burden of medical staff [15]. This topic studies a set of intelligent telemedicine monitoring systems, which is used to monitor the elderly and patients with chronic diseases in the family and community [16], so that the guardians can monitor the basic situation of their own body at home and conduct remote video with the community medical staff [17], and at the same time, the medical staff of the community hospital can consult the health records of the guardians [18]. This system greatly reduces the burden of family care, facilitates the guardianship of the ward’s health, and can be used for some chronic diseases [19].With their unique advantages, wireless sensor networks are widely used in various fields. Target tracking is one of the important research hotspots in wireless sensor networks [20]. Due to the limitation of energy and data processing ability of sensor nodes, the research of target tracking technology based on wireless sensor networks mostly focuses on the effective tracking and monitoring of target nodes [21, 22], and at the same time, through the partial activation of sensor nodes in the network for collaborative tracking [23], so as to reduce the energy consumption of sensor nodes as much as possible and prolong the life of wireless sensor networks [24]. In the monitoring area, if there is only one target node or only for a specific target node, then the system belongs to a single target tracking system [25], and multitarget tracking system needs to deal with the situation of tracking and monitoring multiple mobile target nodes at the same time [26].Motion analysis allows people to learn the motion law of the target object and use it for analysis and modeling [27]. For example, in the field of medical rehabilitation, we can establish a remote monitoring network for patients to strengthen the behavior monitoring of patients [28], so as to timely feed back medical data; in the field of ergonomics, we can also provide enough accurate human posture data for research [29]; in the field of sports, motion analysis technology can be used to simulate training, record athletes’ action data, and compare with the standard template [30]. It can be used in the entertainment industry to generate vivid images for reference [31]. In the human motion recognition system based on sensors, the denoising and smoothing of data are usually used [32]. When collecting sensor data information, the jitter of the human body and equipment will bring the noise to the system and the measurement noise of the system is included in the collected sensor data information [33, 34]. Therefore, removing the interference noise will improve the effectiveness and reliability of the system [35].In addition to fuzzy recognition, effective feature system recognition is required for actions [36, 37]. Effective action features play a very important role in the process of motion recognition, which will have a significant impact on the accuracy of motion recognition. There are many different action characteristics in the action, and the ability of different types of features to describe the action is quite different [38]. Therefore, the establishment of effective human motion features has become the key to motion recognition. According to different application scenarios or different kinds of actions, different action features are needed for description. Generally speaking, motion features can be divided into two types, one is to mine the effective bottom features of the image, and the other is to get the three-dimensional structure model of human posture from the image and then extract the effective features to describe human motion [39]. The underlying features of the image can generally be divided into global features and local features. The global feature mainly represents the global region of interest of the image and encodes it as a whole representing the body motion information on a large scale. The global features mainly include spatiotemporal features, optical flow characteristics, motion energy maps, and silhouette features. In modern times, some people have proposed new methods. Lun and Zhao [40] proposed using a three-dimensional Harris detector to capture the geometric distribution of points of interest. This feature captures the geometric information of the points of interest and keeps the geometric transformation unchanged. Concerning strong robustness, Ferrer and Sanfeliu [41] proposed an optical flow analysis and random sample consistency method to represent the motion characteristics. By calculating the average and standard deviation of the horizontal and vertical directions of the feature points in the optical flow field, the motion of the target is detected. Concerning direction, Bütepage et al. [42] proposed combining the motion energy map and the motion history map as human motion features to represent the human body motion and then matching the time template with the stored template. The method automatically performs time division, which has little effect on the speed change. Guan et al. [43] proposed a probabilistic formula for joint contour extraction and 3D reconstruction and calculated the most likely three-dimensional shape by generating color information to construct a silhouette feature. Wang et al. [44] extracted the motion edges in polar coordinates and then calculated the optical flow of each motion region through the gradient operator in polar coordinates to complete the target tracking. The experimental results show that multitarget detection and tracking are effective.Based on the plantar pressure data collected by sensor networks in different movement modes, the characteristics of plantar pressure change at each measurement point are studied. On this basis, a set of algorithms for monitoring and identifying heel landing and toe-off gait events are designed. The algorithm can dynamically adjust its own parameters according to the current gait cycle, plantar pressure relative value amplitude, and other information, identify the key gait events in a variety of motion combinations, and provide important gait information for motion pattern recognition. Aiming at the quasi-periodic characteristics of acceleration changes at different points in the process of human motion, a method of human motion pattern recognition based on gait event information is designed. The method intercepts the three-axis acceleration data collected by accelerometers through key gait events and establishes a data set for human motion pattern recognition. For each data set, the LDA classifier is established to recognize the current motion pattern. At last, the voting decision is made on the classification results of all classifiers. The comprehensive recognition accuracy of the method for the four motion patterns can reach 95.84%.


## 2. Proposed Method

### 2.1. Structure of Fuzzy Neural Network

The fuzzy neural network is a kind of neural network with fuzzy weight coefficient or input signal is a fuzzy quantity. The modeling method of the fuzzy model is used to solve the nonlinear system. The input space is divided into several fuzzy regions with a linear model in each region. The output of the local model is weighted by the input membership function. Because there are a lot of ambiguities in the objective world, so “ambiguity” can describe the objective world more accurately than “clarity.” The fuzzy theory uses membership function to describe the degree of ambiguity of things. It is a precise method to solve the problem of uncertainty. Its expression is more in line with the logic of human thinking and can be applied to many disciplines and fields. In this paper, a four-layer fuzzy neural network model is used, which is a multi-input single-output system. It has nine input units, including the standard variance of eight actions and a centroid variation. Each input is a membership function. The first layer of the system is the input layer, which is expressed asX=x1,x2,…,xnT; n is the number of input variables. The second layer of the system is the membership function layer, whose function is to convert the input value into membership degree. The input value X′=x1′,x2′,…,xn′T in the video sequence is divided into r discrete sequence quantities using membership functions, with values ranging from [0, 1]. We use a Gaussian function as the conversion function for this layer.(1)uij=exp−xi−mij2σij2.i=1,2,…,n, j=1,2,…,r, r is the number of discrete partitions; mij is the average of the middle. σij is the width. We enter the value xijk=xik+uijk−1⋅θij at time k, where θij is the recursive unit weight. μij is the information at time k − 1. Each node in the third layer represents a rule. This rule comes from the learning algorithm. The current node and the preorder node depend on the rule relationship. Node functions rely on the application of rules:(2)Hk=u1k⋅u2k⋯unk=∏j−1zujk,1≤k≤z.Z is the number of third-level nodes, which is the number of rules. The fourth layer is the output layer. In a multi-input single-output system, the initial weight value Xωi is the membership of the rule. Then, using an iterative algorithm, the function of the output layer is(3)y=∑i=1zωiHi.


### 2.2. Feature Extraction

The precise extraction of the contours of the moving human body is an important preliminary work in behavior analysis. In practical engineering applications, the acquired images have more noise, and the existing edge detection algorithms usually use edge detection operators such as Canny, Robertscross, Prewitt, and Sobel to detect whether the gray value jumps. The edge of the human body is shown in the image. The contours detected by these methods are generally of different thicknesses, or the contours are composed of nonconnected parts. For this reason, after detecting the contour of the human body, it takes a lot of work to carry out the fusion and refinement of the contour. The flow of data preprocessing is shown in Figure1. It includes data cleaning, data integration, data transformation, and data specification. Data cleaning “cleans” data by filling in missing values, smoothing noise data, identifying or deleting outliers, and resolving inconsistencies. The data integration routine is to combine and store the data from multiple data sources to establish a data warehouse. Data transformation is to transform data into a form suitable for data mining by means of smooth aggregation, data generalization, and normalization. Data reduction technology can be used to get the reduced representation of the data set, which is much smaller but still close to maintaining the integrity of the original data, and the result is the same or almost the same as that before reduction. After a series of preprocessing of the originally collected data, feature extraction and selection are the most important steps before behavior recognition. The quality of feature extraction and selection will directly affect the accuracy of behavior recognition. After this step, you can achieve the following goals: reduce data storage and input data bandwidth, reduce redundancy, improve data classification, can find more meaningful latent variables, and help to have a deeper understanding of the data.Figure 1
The flow of data preprocessing.The purpose of feature extraction is to extract the most representative features from the original data, which can be used for logical interpretation of human behavior or signal comparison. The extracted eigenvalues have the following characteristics: the characteristics of different behaviors are distinct and easy to distinguish. At present, the main features used include time-domain features, frequency-domain features, and time-frequency features. Attitude angles (roll angle, pitch angle, and yaw angle) were originally used in aviation systems to represent the azimuth information of flying objects. When the human body wears inertial sensors to perform different behaviors, these devices bound to the human body may produce fluctuations up and down, left and right, and back, which are similar to the changes of aircraft operating in limited space. Inspired by this, researchers introduced the feature extracted from the attitude angle to classify human behavior. Some typical features are adopted: (1) absolute median deviation (MAD), (2) crest factor, (3) spectral energy (SpEy), and (4) pairwise correlation coefficient. At the same time, the complexity of the algorithm and the energy efficiency of the system need to be considered. Therefore, after extracting the features, the algorithm should be adopted. The selection of features is as follows.The purpose of feature selection is to select the most statistically significant feature values from the feature set and reduce the dimension of the original feature set. Algorithms for applying a wide range of feature selection are principal component analysis (PCA), linear discriminant analysis (LDA), and independent component analysis (ICA). In order to preserve the main motion feature information while reducing the fusion feature dimension, principal component analysis (PCA) is used to analyze the fusion characteristics of the angular feature, relative position feature, and velocity feature based on the joint 3D space coordinates. The postdimensional fusion feature is used as training and test data, and on this basis, the training of the classifier and the recognition of the human action type are realized. In feature fusion, only different single features can be merged when describing the same action. The three features described are the same type of action features and exhibit certain complementarity. Therefore, three single features can be merged into A feature to represent human motion. In order to effectively fuse the three features, the preprocessed multiple action features are combined, and the main components are extracted on this basis.The results of PCA human motion model were as follows:(4)D=Z1,Z2,Z3,Z4,Z5,…,Zn.The results of LDA human motion model were as follows:(5)F=f1′,f2′,f3′,…,fn′T.The results of ICA human motion model were as follows:(6)V=V1,V2,…,VnT.Among them,D is the feature of action angle, F is the feature of action position, and V is the feature of action speed. The multifeature combination model obtained by combining the three matrices is shown in formula (7), and then, a new low-dimensional matrix is obtained by reducing the dimension with the principal component fraction method as shown in formula (8).(7)X=D,F,V,(8)XPCA=D,F,VPCA.


### 2.3. Action Element

Action element has different meanings from “basic action” in action, action element, or MOD analysis. The purpose of establishing action elements is to reflect the mapping relationship between various design parameters of the device in the process of manipulation and various action attributes in the process of human motion. The basic action element in MOD analysis is similar to that of the action element in the sense of establishment. The time limit of coupling of multiaction elements is calculated. Bundles need to be used to calculate the time relationship in MOD analysis, but an action element can be a set of one or more basic actions, which contain the same human motion characteristics and physical attributes. Action element refers to the set of basic actions with the same action category and physical meaning in the course of the subject’s movement when people change the external object’s motion state. Combining various factors and their dynamic characteristics in comfort analysis of control devices, the conceptual model of action element is expressed as follows:(9)Mk=nk,λk,ωBiBi,ωSjSjk.Among them, when performing maneuvering work, the personnel are repetitive and periodic in many cases, and the human body feeling has a time accumulation effect. Even if the device satisfies the comfort requirement, after a period of maneuvering operation, It also feels discomfort due to the energy consumption of the body caused by the cumulative effect of time; therefore, when studying the comfort of the manipulating device, it is necessary to weaken or eliminate the repeated operation and the time accumulation effect, so that the comfort of the operating device can be objectively evaluated and analysis. Based on this consideration, the time required to use the device to accomplish a certain maneuvering purpose is the time zone to be studied, and the set of all the basic actions used by the operator in the time zone is used as the maneuvering process to be investigated when studying the comfort of the device. The action element is used to decompose the manipulation process; since the action element combination in the manipulation process is a dynamically changing process, various action elements are dynamically transferred between various manipulation states with a certain probability, and this probability is throughout the manipulation. The process is determined to be invariant, and the probability largely satisfies the distribution state of the classical probabilities. Therefore, the dimensionless probability is determined by calculating the time ability of the action element in the manipulation process, as each action element in the manipulation process. The time constraintsλ to achieve the purpose of approximating the cumulative effect of time are(10)λk=gkfm+α0.The human action element has the following properties. (1) The human action element has a hierarchical link structure and a motion level inheritance. The structure formed by the human action element according to the motion link relationship is a tree structure. Limb movement has a hierarchical inheritance; that is, parental limb movement can drive its children’s limbs to do the same exercise, while the child’s limb movement has no effect on their parents’ limbs. The hierarchical inheritance of the human action element can be used to decompose complex sports actions step by step, and the body shape and motion are represented by the positional parameters of the moving limb relative to the parent limb. (2) The movement of the human action element is subject to movement constraints. The human motion element has two movement modes, displacement and rotation. When the limb rotates around the joint point, its freedom of movement and rotation angle are subject to certain constraints. (3) The degree of influence of the movement of the human body on the overall movement is related to its position in the hierarchy.


### 2.4. Visualization of Human Movements

The purpose of visualization of human motion is to use the human motion data collected by the sensor to drive the human body model in the program, so that the human body motion can be reproduced into the virtual 3D environment. Compared with video recording human body motion, human motion storage and display systems based on sensor data and visualization methods have the advantages of small data volume, strong portability, and convenient observation and research. We use the Ogre rendering engine to visualize human motion. For system scalability, we encapsulate the driver method of the human body model in Ogre and provide a data format interface for the driver model. The data conversion module is responsible for converting human motion data of different formats into a data representation conforming to the interface, thereby achieving the purpose of control isolation and decoupling of the input data format and the Ogre model. Currently, our visualization system is compatible with two human motion input formats, including our defined pose-based quaternion-based human motion format and the BVH format common to human motion. In fact, after defining the data interface, the system can expand more human motion input formats and a variety of different rendering engines and virtual human models. The structure of the human motion visualization system is shown in Figure2.Figure 2
Structure diagram of human motion visualization system.According to the schematic system structure of Figure2, the driving mode of virtual human model, the visualization of human action based on quaternion representation, the visualization of human action in BVH format, and the control of camera in virtual scene are shown. Our system uses the human motion capture sensor produced by Beijing Noriton Technology Co., Ltd., to collect human motion data. Its equipment provides quaternion interface and BVH interface. Therefore, the original data format collected by this device is the standard for human motion visualization based on quaternion representation and BVH format. We also realized the visualization of human motion data in HDM05 human motion capture database.The complexity of different algorithms is shown in Table1. Among them, the stability of selection sort, Hill sort, quick sort, and heap sort is poor.Table 1
The complexity of different algorithms.
Sorting algorithmAverage time complexityBest caseWorst caseSpace complexityBubble sortO (n)O (n)O (n3)O (1)Select sortO (n3)O (n)O (n)O (1)Insertion sortO (n3)O (n)O (n3)O (1)Hill sortO (nlogn)O (nlog2n)O (nlog2n)O (1)Merge sortO (nlogn)O (nlogn)O (nlogn)O (n)Quick sortO (nlogn)O (nlogn)O (n)O (logn)Heap sortO (nlogn)O (nlogn)O (nlogn)O (1)Count sortO (n + k)O (n + k)O (n + k)O (k)Bucket sortO (n + k)O (n + k)O (n)O (n + k)Base sortO (n × k)O (n × k)O (n × k)O (n + k)


## 2.1. Structure of Fuzzy Neural Network

The fuzzy neural network is a kind of neural network with fuzzy weight coefficient or input signal is a fuzzy quantity. The modeling method of the fuzzy model is used to solve the nonlinear system. The input space is divided into several fuzzy regions with a linear model in each region. The output of the local model is weighted by the input membership function. Because there are a lot of ambiguities in the objective world, so “ambiguity” can describe the objective world more accurately than “clarity.” The fuzzy theory uses membership function to describe the degree of ambiguity of things. It is a precise method to solve the problem of uncertainty. Its expression is more in line with the logic of human thinking and can be applied to many disciplines and fields. In this paper, a four-layer fuzzy neural network model is used, which is a multi-input single-output system. It has nine input units, including the standard variance of eight actions and a centroid variation. Each input is a membership function. The first layer of the system is the input layer, which is expressed asX=x1,x2,…,xnT; n is the number of input variables. The second layer of the system is the membership function layer, whose function is to convert the input value into membership degree. The input value X′=x1′,x2′,…,xn′T in the video sequence is divided into r discrete sequence quantities using membership functions, with values ranging from [0, 1]. We use a Gaussian function as the conversion function for this layer.(1)uij=exp−xi−mij2σij2.i=1,2,…,n, j=1,2,…,r, r is the number of discrete partitions; mij is the average of the middle. σij is the width. We enter the value xijk=xik+uijk−1⋅θij at time k, where θij is the recursive unit weight. μij is the information at time k − 1. Each node in the third layer represents a rule. This rule comes from the learning algorithm. The current node and the preorder node depend on the rule relationship. Node functions rely on the application of rules:(2)Hk=u1k⋅u2k⋯unk=∏j−1zujk,1≤k≤z.Z is the number of third-level nodes, which is the number of rules. The fourth layer is the output layer. In a multi-input single-output system, the initial weight value Xωi is the membership of the rule. Then, using an iterative algorithm, the function of the output layer is(3)y=∑i=1zωiHi.


## 2.2. Feature Extraction

The precise extraction of the contours of the moving human body is an important preliminary work in behavior analysis. In practical engineering applications, the acquired images have more noise, and the existing edge detection algorithms usually use edge detection operators such as Canny, Robertscross, Prewitt, and Sobel to detect whether the gray value jumps. The edge of the human body is shown in the image. The contours detected by these methods are generally of different thicknesses, or the contours are composed of nonconnected parts. For this reason, after detecting the contour of the human body, it takes a lot of work to carry out the fusion and refinement of the contour. The flow of data preprocessing is shown in Figure1. It includes data cleaning, data integration, data transformation, and data specification. Data cleaning “cleans” data by filling in missing values, smoothing noise data, identifying or deleting outliers, and resolving inconsistencies. The data integration routine is to combine and store the data from multiple data sources to establish a data warehouse. Data transformation is to transform data into a form suitable for data mining by means of smooth aggregation, data generalization, and normalization. Data reduction technology can be used to get the reduced representation of the data set, which is much smaller but still close to maintaining the integrity of the original data, and the result is the same or almost the same as that before reduction. After a series of preprocessing of the originally collected data, feature extraction and selection are the most important steps before behavior recognition. The quality of feature extraction and selection will directly affect the accuracy of behavior recognition. After this step, you can achieve the following goals: reduce data storage and input data bandwidth, reduce redundancy, improve data classification, can find more meaningful latent variables, and help to have a deeper understanding of the data.Figure 1
The flow of data preprocessing.The purpose of feature extraction is to extract the most representative features from the original data, which can be used for logical interpretation of human behavior or signal comparison. The extracted eigenvalues have the following characteristics: the characteristics of different behaviors are distinct and easy to distinguish. At present, the main features used include time-domain features, frequency-domain features, and time-frequency features. Attitude angles (roll angle, pitch angle, and yaw angle) were originally used in aviation systems to represent the azimuth information of flying objects. When the human body wears inertial sensors to perform different behaviors, these devices bound to the human body may produce fluctuations up and down, left and right, and back, which are similar to the changes of aircraft operating in limited space. Inspired by this, researchers introduced the feature extracted from the attitude angle to classify human behavior. Some typical features are adopted: (1) absolute median deviation (MAD), (2) crest factor, (3) spectral energy (SpEy), and (4) pairwise correlation coefficient. At the same time, the complexity of the algorithm and the energy efficiency of the system need to be considered. Therefore, after extracting the features, the algorithm should be adopted. The selection of features is as follows.The purpose of feature selection is to select the most statistically significant feature values from the feature set and reduce the dimension of the original feature set. Algorithms for applying a wide range of feature selection are principal component analysis (PCA), linear discriminant analysis (LDA), and independent component analysis (ICA). In order to preserve the main motion feature information while reducing the fusion feature dimension, principal component analysis (PCA) is used to analyze the fusion characteristics of the angular feature, relative position feature, and velocity feature based on the joint 3D space coordinates. The postdimensional fusion feature is used as training and test data, and on this basis, the training of the classifier and the recognition of the human action type are realized. In feature fusion, only different single features can be merged when describing the same action. The three features described are the same type of action features and exhibit certain complementarity. Therefore, three single features can be merged into A feature to represent human motion. In order to effectively fuse the three features, the preprocessed multiple action features are combined, and the main components are extracted on this basis.The results of PCA human motion model were as follows:(4)D=Z1,Z2,Z3,Z4,Z5,…,Zn.The results of LDA human motion model were as follows:(5)F=f1′,f2′,f3′,…,fn′T.The results of ICA human motion model were as follows:(6)V=V1,V2,…,VnT.Among them,D is the feature of action angle, F is the feature of action position, and V is the feature of action speed. The multifeature combination model obtained by combining the three matrices is shown in formula (7), and then, a new low-dimensional matrix is obtained by reducing the dimension with the principal component fraction method as shown in formula (8).(7)X=D,F,V,(8)XPCA=D,F,VPCA.


## 2.3. Action Element

Action element has different meanings from “basic action” in action, action element, or MOD analysis. The purpose of establishing action elements is to reflect the mapping relationship between various design parameters of the device in the process of manipulation and various action attributes in the process of human motion. The basic action element in MOD analysis is similar to that of the action element in the sense of establishment. The time limit of coupling of multiaction elements is calculated. Bundles need to be used to calculate the time relationship in MOD analysis, but an action element can be a set of one or more basic actions, which contain the same human motion characteristics and physical attributes. Action element refers to the set of basic actions with the same action category and physical meaning in the course of the subject’s movement when people change the external object’s motion state. Combining various factors and their dynamic characteristics in comfort analysis of control devices, the conceptual model of action element is expressed as follows:(9)Mk=nk,λk,ωBiBi,ωSjSjk.Among them, when performing maneuvering work, the personnel are repetitive and periodic in many cases, and the human body feeling has a time accumulation effect. Even if the device satisfies the comfort requirement, after a period of maneuvering operation, It also feels discomfort due to the energy consumption of the body caused by the cumulative effect of time; therefore, when studying the comfort of the manipulating device, it is necessary to weaken or eliminate the repeated operation and the time accumulation effect, so that the comfort of the operating device can be objectively evaluated and analysis. Based on this consideration, the time required to use the device to accomplish a certain maneuvering purpose is the time zone to be studied, and the set of all the basic actions used by the operator in the time zone is used as the maneuvering process to be investigated when studying the comfort of the device. The action element is used to decompose the manipulation process; since the action element combination in the manipulation process is a dynamically changing process, various action elements are dynamically transferred between various manipulation states with a certain probability, and this probability is throughout the manipulation. The process is determined to be invariant, and the probability largely satisfies the distribution state of the classical probabilities. Therefore, the dimensionless probability is determined by calculating the time ability of the action element in the manipulation process, as each action element in the manipulation process. The time constraintsλ to achieve the purpose of approximating the cumulative effect of time are(10)λk=gkfm+α0.The human action element has the following properties. (1) The human action element has a hierarchical link structure and a motion level inheritance. The structure formed by the human action element according to the motion link relationship is a tree structure. Limb movement has a hierarchical inheritance; that is, parental limb movement can drive its children’s limbs to do the same exercise, while the child’s limb movement has no effect on their parents’ limbs. The hierarchical inheritance of the human action element can be used to decompose complex sports actions step by step, and the body shape and motion are represented by the positional parameters of the moving limb relative to the parent limb. (2) The movement of the human action element is subject to movement constraints. The human motion element has two movement modes, displacement and rotation. When the limb rotates around the joint point, its freedom of movement and rotation angle are subject to certain constraints. (3) The degree of influence of the movement of the human body on the overall movement is related to its position in the hierarchy.


## 2.4. Visualization of Human Movements

The purpose of visualization of human motion is to use the human motion data collected by the sensor to drive the human body model in the program, so that the human body motion can be reproduced into the virtual 3D environment. Compared with video recording human body motion, human motion storage and display systems based on sensor data and visualization methods have the advantages of small data volume, strong portability, and convenient observation and research. We use the Ogre rendering engine to visualize human motion. For system scalability, we encapsulate the driver method of the human body model in Ogre and provide a data format interface for the driver model. The data conversion module is responsible for converting human motion data of different formats into a data representation conforming to the interface, thereby achieving the purpose of control isolation and decoupling of the input data format and the Ogre model. Currently, our visualization system is compatible with two human motion input formats, including our defined pose-based quaternion-based human motion format and the BVH format common to human motion. In fact, after defining the data interface, the system can expand more human motion input formats and a variety of different rendering engines and virtual human models. The structure of the human motion visualization system is shown in Figure2.Figure 2
Structure diagram of human motion visualization system.According to the schematic system structure of Figure2, the driving mode of virtual human model, the visualization of human action based on quaternion representation, the visualization of human action in BVH format, and the control of camera in virtual scene are shown. Our system uses the human motion capture sensor produced by Beijing Noriton Technology Co., Ltd., to collect human motion data. Its equipment provides quaternion interface and BVH interface. Therefore, the original data format collected by this device is the standard for human motion visualization based on quaternion representation and BVH format. We also realized the visualization of human motion data in HDM05 human motion capture database.The complexity of different algorithms is shown in Table1. Among them, the stability of selection sort, Hill sort, quick sort, and heap sort is poor.Table 1
The complexity of different algorithms.
Sorting algorithmAverage time complexityBest caseWorst caseSpace complexityBubble sortO (n)O (n)O (n3)O (1)Select sortO (n3)O (n)O (n)O (1)Insertion sortO (n3)O (n)O (n3)O (1)Hill sortO (nlogn)O (nlog2n)O (nlog2n)O (1)Merge sortO (nlogn)O (nlogn)O (nlogn)O (n)Quick sortO (nlogn)O (nlogn)O (n)O (logn)Heap sortO (nlogn)O (nlogn)O (nlogn)O (1)Count sortO (n + k)O (n + k)O (n + k)O (k)Bucket sortO (n + k)O (n + k)O (n)O (n + k)Base sortO (n × k)O (n × k)O (n × k)O (n + k)


## 3. Experiments

### 3.1. Experimental Data

The experimental data set of this paper uses the human motion capture database. There are 2564 human motion data in the database. They are labeled as 130 human motion categories and 6244 motion examples. The iconic motion is consistent with its realistic reference motion, which can clearly see the actual meaning of this motion. Metaphorical movements express abstract content, which can be artificially endowed. Each motion data contains a sequence of motion data for 31 bones. Using the code provided on its website, it is easy to obtain data in the form of joint position and pose quaternion. After processing, we store each human motion in a matrix. Each row in the matrix represents a frame in the action, each human body has a different length of motion, and the longest motion contains 801 frames of data. The sampling rate of capturing the human body posture is 30 Hz, the accuracy of the joint position is 2 cm, and the movements of different lengths have different distribution histograms, as shown in Figure3.Figure 3
Histogram of human action length distribution.Each row of the matrix contains 124 floating-point numbers, of which each four represents the relative attitude quaternion on a bone in the human skeleton model, and each row has 31 quaternions. In each experiment, we will further process the data according to the need. Because the time length of each human action varies, the longest action length is 801 frames. TensorFlow requires that the shape of the input tensor be the same. We can extend the short human action frame to 801 frames. But it will waste a lot of computing time, and the length difference of the input sequence will make the model deviate greatly. So we scaled each human motion to 256 frames in a uniform length. For human actions of less than 256 frames, we use zero frames to complete them to 256 frames at the end of the action; for human actions of more than 256 frames, we randomly select 256 frames without repetition and make them form shortened sequences in the original order. This not only unifies the length of human action but also reduces the amount of calculation while retaining the action information as much as possible.The comparison of different algorithms is shown in Table2. The comparison of algorithms includes three aspects: time complexity, space complexity, and correctness. The fuzzy neural network in this paper has the lowest time complexity of 5%, the lowest space complexity of 10%, and the highest accuracy of 97.6%. Therefore, this study has a certain reference significance.Table 2
Comparison of different algorithms.
Time complexity (%)Spatial complexity (%)Correctness (%)Fuzzy neural network51097.6MVDR-based feature extraction [45]81892.3Pseudo-2D stick model and K-ary tree hashing [46]112194.1Genetic algorithm [47]141296.3Modified sampling consensus and kernel sliding perceptron [48]102593.5Dense depth maps [49]72591.1Multiple features and random forest [50]182690.4


### 3.2. Experimental Environment

The experimental hardware environment and operating system are shown in Table3.Table 3
Experimental hardware environment and operating system.
Environmental parametersEnvironmental parameter valueComputer modelLenovoCPU8750 HMemory capacity6 GBHard drive capacity512 GB + 2 TBCore/thread countSix cores/twelve threadsOperating systemWindows 10In the training phase of the action model, we first convert the sequence of actions in the subset of actions into a sequence of actions in the skeleton representation. A total of 15 divisions are performed, and 15 experimental sets are obtained. Each experimental set contains a training set and a test set. Then, for each experimental set, the action instance is segmented from the action sequence of the training set, the probability model dictionary is constructed on the action instance, then, the action sequence in the training set is rescanned, and the probability model of each frame data is extracted using the probability model dictionary. Finally, we use the probability feature vector to train our action model. In the online test phase, for each experimental set, we first use the probability model dictionary built on the training set to extract the probability feature vector of the test set action sequence. Then, the probability feature vector is input into the training action model, and the test results are obtained by comparing the database. Finally, the mean and variance of the test results on the 15 experimental sets are taken as the final result.


## 3.1. Experimental Data

The experimental data set of this paper uses the human motion capture database. There are 2564 human motion data in the database. They are labeled as 130 human motion categories and 6244 motion examples. The iconic motion is consistent with its realistic reference motion, which can clearly see the actual meaning of this motion. Metaphorical movements express abstract content, which can be artificially endowed. Each motion data contains a sequence of motion data for 31 bones. Using the code provided on its website, it is easy to obtain data in the form of joint position and pose quaternion. After processing, we store each human motion in a matrix. Each row in the matrix represents a frame in the action, each human body has a different length of motion, and the longest motion contains 801 frames of data. The sampling rate of capturing the human body posture is 30 Hz, the accuracy of the joint position is 2 cm, and the movements of different lengths have different distribution histograms, as shown in Figure3.Figure 3
Histogram of human action length distribution.Each row of the matrix contains 124 floating-point numbers, of which each four represents the relative attitude quaternion on a bone in the human skeleton model, and each row has 31 quaternions. In each experiment, we will further process the data according to the need. Because the time length of each human action varies, the longest action length is 801 frames. TensorFlow requires that the shape of the input tensor be the same. We can extend the short human action frame to 801 frames. But it will waste a lot of computing time, and the length difference of the input sequence will make the model deviate greatly. So we scaled each human motion to 256 frames in a uniform length. For human actions of less than 256 frames, we use zero frames to complete them to 256 frames at the end of the action; for human actions of more than 256 frames, we randomly select 256 frames without repetition and make them form shortened sequences in the original order. This not only unifies the length of human action but also reduces the amount of calculation while retaining the action information as much as possible.The comparison of different algorithms is shown in Table2. The comparison of algorithms includes three aspects: time complexity, space complexity, and correctness. The fuzzy neural network in this paper has the lowest time complexity of 5%, the lowest space complexity of 10%, and the highest accuracy of 97.6%. Therefore, this study has a certain reference significance.Table 2
Comparison of different algorithms.
Time complexity (%)Spatial complexity (%)Correctness (%)Fuzzy neural network51097.6MVDR-based feature extraction [45]81892.3Pseudo-2D stick model and K-ary tree hashing [46]112194.1Genetic algorithm [47]141296.3Modified sampling consensus and kernel sliding perceptron [48]102593.5Dense depth maps [49]72591.1Multiple features and random forest [50]182690.4


## 3.2. Experimental Environment

The experimental hardware environment and operating system are shown in Table3.Table 3
Experimental hardware environment and operating system.
Environmental parametersEnvironmental parameter valueComputer modelLenovoCPU8750 HMemory capacity6 GBHard drive capacity512 GB + 2 TBCore/thread countSix cores/twelve threadsOperating systemWindows 10In the training phase of the action model, we first convert the sequence of actions in the subset of actions into a sequence of actions in the skeleton representation. A total of 15 divisions are performed, and 15 experimental sets are obtained. Each experimental set contains a training set and a test set. Then, for each experimental set, the action instance is segmented from the action sequence of the training set, the probability model dictionary is constructed on the action instance, then, the action sequence in the training set is rescanned, and the probability model of each frame data is extracted using the probability model dictionary. Finally, we use the probability feature vector to train our action model. In the online test phase, for each experimental set, we first use the probability model dictionary built on the training set to extract the probability feature vector of the test set action sequence. Then, the probability feature vector is input into the training action model, and the test results are obtained by comparing the database. Finally, the mean and variance of the test results on the 15 experimental sets are taken as the final result.


## 4. Discussion

In the learning and training phase. In order to verify the results of the abnormal behavior of the fuzzy neural network, the unwrapping representation method is used in the image feature representation. The minimum moment distance is used to calculate the distance, and the minimum standard deviation is used as the input quantity. The classification algorithm of the network is as follows. The analysis of the joint angle feature vector angle and the skeletal point space position information is transmitted to the simulation system according to the predefined interactive gesture action, thereby completing the interaction with the virtual simulation combat system. In order to verify the effectiveness of the interactive system, it is difficult to obtain complete human body information and the diversity of human body shape due to the limitation of the moving human body detection and extraction technology. Therefore, it is difficult to completely extract the inscribed rectangle. In some cases, the extracted ideal inscribed rectangle may still fall into obvious local characteristics. So in practice, determining a rectangular region is usually not a problem of finding an optimal solution but looking for a suboptimal solution. The result obtained under normal circumstances is the trunk. Figure4 shows two different actions.Figure 4
Different behavioral actions.In practical applications, body motion recognition delay is very important. Usually, if the recognition results are not displayed after 1-2 seconds, it will seriously affect the user’s experience. Recognition actions are simple actions contained in the data set, and most of the action instances only last 35 frames. So, we take the end frame of the action instance and the 34 frames before it as an action.When moving object detection detects a moving object, because the shadow is distinct from the background and has the same motion attributes as the moving object, the shadow will be detected as the foreground together with the moving object, which implies two major shortcomings. The first is that the shape of the object is changed by the shadow and the shape-based features will be affected. The disadvantage is that the shadows produced by two or more objects may make different objects mistakenly considered to be connected, and they will be detected as an object, which will have a negative impact on subsequent applications such as target tracking and object counting, resulting in a higher error rate of the whole intelligent monitoring system and a decline in the overall performance of the system. Therefore, in order to correctly analyze moving objects, we must solve the influence of shadow. To determine the overlap degree of windows, we have to consider the influence of transitional behavior on the accuracy of recognition when recognizing human behavior. Transitional behavior refers to the transitional state which is different from arbitrary behavior in the process of changing from one behavior to another. It is necessary to reduce the interference of transition behavior on data segmentation, so the overlap degree of windows is set to 50%, and the image is shaded. The resulting image is shown in Figure5.Figure 5
Postprocessing of the image.It can be seen from the figure that the following motion models of each joint chain of human joints meet the requirements of motion, and the filtering method adopted also stabilizes the following motion and meets the requirements of virtual human skeleton driving interaction in a virtual simulation.Then, one human action is randomly selected from each of the 25 adjacent actions and put into the test set, and the other human actions are put into the training set. This ensures that the class distribution of the training set is consistent with that of the test set. The training model is iterated 100 times with all the actions of the training set. Each iteration will randomly rearrange the training set data. The results are shown in Figures6 and 7 .Figure 6
Curve of variation with iteration times.Figure 7
The curve of classification accuracy with iteration times.From Figures6 and 7, we can see that the value of loss function decreases with iteration, while the accuracy of classification increases with iteration, and eventually their changes tend to be stable, which conforms to the general training process of the neural network model. During the 100 iterations, the correct rate of feature extraction and pattern recognition is 90.2% and 96.3%, respectively. The loss function values of the model in training data and test data are reduced to 0.072 and 0.289, respectively.In order to obtain the results of the experimental action fuzzy control, the experimental results on the subset of the action guidance form obtained by the fuzzy control are shown in Figure8.Figure 8
Subset of action guidance for fuzzy control.As can be seen from the results in Figure8, the effect of the operation is not very significant. From the results alone, the action has a lot of freedom, which will make the collected action examples of the same kind of action show very rich intraclass action variability, the coverage of the action instance will be better, and the performer according to this is limited. The action made by the message will be inconsistent with the specified action, and the accuracy of the action will be poor. A comparison graph of tracking curve error is obtained by the fuzzy control method. In the whole process of human behavior, the tracking error interval obtained by fuzzy tracking control is relatively small, and the error variation interval is −1%∼+1%, which realizes the recognition of fuzzy behavior.


## 5. Conclusions

A new upsurge of research has emerged in the field of human motion recognition. Fuzzy motion recognition makes human motion recognition to a new height. It can be predicted that, with the deepening of research, human motion recognition is not only a research topic but also a new technology to facilitate our daily life. The main research content of this paper is human action recognition based on fuzzy theory. In view of the fact that most human action recognition systems use acceleration sensors as the main data source, the practicability of improving the accuracy of human action recognition is studied in depth. The main aspects of this study include the following:(1)
It introduced the human motion feature extraction methods and proposed different recognition methods for simple and complex movements. In this paper, the change characteristics of skeletal joint points based on fuzzy theory are regarded as the dynamic characteristics of human motion. Simple actions are identified by using the spatial coordinate information of joint points, while complex actions are identified by using a regularization algorithm. That is to say, according to the change of joint angle, the template matching method is used to find the action template with the highest similarity. At the same time, the shortcomings of the dynamic time warping algorithm are pointed out. Considering the slope and distortion threshold, the algorithm is optimized. The experimental results show that the accuracy of action recognition after optimization is higher than that of the traditional algorithm(2)
The dynamic manipulation process is divided based on the action element, and the time constraints of each action element in the manipulation process are analyzed according to the reservation time standard method. With the time constraints and weight constraints of the action element as coupling coefficients, the independent action elements in the manipulation process are coupled, and the coupled multiaction composite group is used as the description matrix of the manipulation process to realize the manipulation process. There is a digital description of Cheng. In this part, the concept of action element is proposed to describe the complex dynamic process with measurable action element which can be used for quantitative analysis of factors. A new solution is provided for the quantitative research of such complex nonlinear systems.(3)
A behavior-solving algorithm based on fuzzy theory is proposed. In order to test whether the feature group based on azimuth information can improve the performance of the human behavior recognition system, it is necessary to obtain accurate azimuth angles of different behaviors. The accuracy of the azimuth angle is improved successfully by the fuzzy algorithm, which provides a guarantee for obtaining accurate azimuth characteristics in the future.Considering that the physical implementation of the gait event information monitoring algorithm involved in this paper is relatively complex, the motion pattern recognition method has not been implemented in the actual sensor system in this paper, so at present, only the data collected by the sensor system is used to carry out the relevant simulation research on the recognition of human gait events and motion patterns. In the future, we can consider using classifiers including support vector machines and neural networks to study the recognition effect and implement these algorithms in the actual hardware system to evaluate the advantages and disadvantages of the algorithms. Due to the limitation of experimental conditions and time, this paper only analyzes and discusses the recognition of the four common motion patterns of a single person in normal walking, running, climbing stairs, and descending stairs. For the recognition of motion patterns such as uphill, downhill, turning and jumping, and the recognition of multiple experimenters’ motion patterns, more experimental data are needed.



---

*Source: 9923748-2021-10-14.xml*