Multi‐dimensional data modelling of video image action recognition and motion capture in deep learning framework

IET Image ProcessingVolume 14, Issue 7 p. 1257-1264 Special Section: Deep Learning for 3D Reconstruction Free Access Multi-dimensional data modelling of video image action recognition and motion capture in deep learning framework Peijun Gao, School of Physical Culture, Jiujiang University, Jiujiang, 332005 People's Republic of ChinaSearch for more papers by this authorDan Zhao, Jiujiang Maternal and Child Health Hospital, Jiujiang, 322000 People's Republic of ChinaSearch for more papers by this authorXuanang Chen, Corresponding Author chenxuanang002@163.com School of Physical Culture, Jiujiang University, Jiujiang, 332005 People's Republic of ChinaSearch for more papers by this author Peijun Gao, School of Physical Culture, Jiujiang University, Jiujiang, 332005 People's Republic of ChinaSearch for more papers by this authorDan Zhao, Jiujiang Maternal and Child Health Hospital, Jiujiang, 322000 People's Republic of ChinaSearch for more papers by this authorXuanang Chen, Corresponding Author chenxuanang002@163.com School of Physical Culture, Jiujiang University, Jiujiang, 332005 People's Republic of ChinaSearch for more papers by this author First published: 29 April 2020 https://doi.org/10.1049/iet-ipr.2019.0588Citations: 3 AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract In order to improve the accuracy of small-range human motion recognition in video and the computational efficiency of large-scale data sets, a multi-dimensional data model of motion recognition and motion capture in video image based on deep-learning framework was proposed. First, the moving foreground of the target is extracted by the Gauss mixture model, and the human body is recognised by the gradient histogram. At the second level, the dense trajectory feature and the deep learning feature are fused, according to the integration of global encoding algorithm and convolutional neural network. In the deep learning feature, the fusion of the deep video feature and the video RGB tricolour feature is taken as the feature of deep learning. Finally, the classification is based on the deep learning network model. The simulation experiments based on large-scale real data sets and small-scale gesture data sets show that the algorithm has high recognition accuracy for large-scale data sets and small-scale gesture actions. In addition, Imperial Computer Vision & Learning Lab human behaviour data set is used to classify the experimental data. The average classification accuracy is 85.79%. The algorithm can run at a speed of about 20 frames per second. 1 Introduction Computer recognition of human behaviour is widely used in many fields, including intelligent video surveillance, motion analysis, video retrieval and criminal investigation [1, 2]. According to the type of input video, motion recognition can be divided into surveillance video, sports video and movies. Different types of video have different characteristics: surveillance video usually uses static cameras recorded from side view or top view. Therefore, the background of surveillance video is relatively simple, and the object of surveillance research is people or vehicles [3]. However, due to the enormous changes in appearance, accurate motion recognition is a very challenging task [4]. For example, occlusion, non-rigid motion, scale change, viewpoint change and clothing colour with similar background colour are all important issues [5]. In addition, it is difficult to collect training samples manually, which requires a lot of manpower and time-consuming. Research on real-time processing of video and its application in the real environment has become one of the focuses of current research on human behaviour recognition [6]. At present, there are many models to identify human behaviour in the video. The methods can be divided into appearance-based model, motion-based model, space–time-based model and depth-based learning method [7]. Among them, in-depth learning is one of the focuses of this research. This document is a template, an electronic copy of which can be downloaded from the Research Journals Author Guide page on the IET's Digital Library. For questions on paper guidelines, please contact the relevant journal inbox as indicated on each journal's website [8]. Convolutional neural network (CNN) is one of the deep learning models [9]. It is a supervised machine learning algorithm [10]. It can learn the hierarchical structure of features by constructing high-level features from low-level features. CNN is an artificial neural network based on deep learning theory. It mainly uses the characteristic of weight sharing to reduce the problem of too many parameters in the ordinary neural network. In the forward calculation, the convolution kernel is used to convolute the input data. Generally, the output results are taken as the output of this layer through the Sigmoid and other non-linear functions. The convolution layer and convolution layer are separated. The sub-sampling layer can increase the local feature invariance and reduce the scale of the feature space, and then use the full connection layer for the final recognition. Some researchers have begun to use CNNs to identify human behaviour [11-13]. Different human behaviour recognition functions can be obtained by constructing different CNN structures [14]. Kim et al. [15] extend the traditional CNN to 3D CNN with time information, and calculate the features in the time and space dimensions of video data. In the convolution process, the feature map is connected with the data in several consecutive frames [16]. The literature uses single frame data and optical flow data to capture motion information [17, 18]. In this paper, a 3D CNN is used in a fixed-length video block. In order to improve the accuracy of small-range human motion recognition in video and the computational efficiency of large-scale data sets, a multi-dimensional data model of motion recognition and motion capture in video image based on deep-learning framework was proposed. First, the moving foreground of the target is extracted by the Gauss mixture model, and the human body is recognised by the gradient histogram (HOG). At the second level, the dense trajectory feature and the deep learning feature are fused. In the deep learning feature, the fusion of the deep video feature and the video RGB tricolour feature is taken as the feature of deep learning. Finally, the classification is based on the deep learning network model. The simulation experiments based on large-scale real data sets and small-scale gesture data sets show that the algorithm has high recognition accuracy for large-scale data sets and small-scale gesture actions. In addition, Imperial Computer Vision & Learning Lab (ICVL) human behaviour data set is used to classify the experimental data. The average classification accuracy is 85.79%. The algorithm can run at a speed of about 20 frames per second. 2 Deep-learning framework for action recognition and motion capture High-level semantic recognition, such as object recognition, scene recognition and human motion recognition, is a hot issue in the field of computer vision. The in-depth learning model provides a basic framework for solving this kind of problem. By mapping visual features into visual words, images or videos are represented as histogram features in a visual dictionary. This visual word-based modelling method provides new research ideas for low-dimensional structured description and semantic analysis of image and video. It not only deals with the 'semantic gap' between low-level visual features and high-level semantics, but also avoids the complicated manual identification in the recognition process. Based on the in-depth learning framework, a visual topic model (and the mapping relationship between visual words and visual topic features are obtained by analysing the distribution of visual words, and the classification based on behaviour recognition and action capture is finally completed. 2.1 Deep-learning framework for action recognition and motion capture The framework is designed as in Fig. 1: as shown in the figure, our model is roughly divided into three layers. Fig. 1Open in figure viewerPowerPoint Framework for action recognition and motion capture The first layer is the data set layer. In order to verify the validity of near-semantic visual dictionary in human motion description, this paper carries out experiments on KTH action database, ICVL action database, Weizmann action database and YouTube video set, respectively. The second layer is the image predictive power. Image pre-processing is to detect each character image and give it to the recognition module for recognition. This process is called image pre-processing. In image analysis, the input image is processed before feature extraction, segmentation and matching. The main purpose of image pre-processing is to eliminate irrelevant information in images, restore useful real information, enhance the detectability of relevant information and simplify data to the greatest extent, so as to improve the reliability of feature extraction, image segmentation, matching and recognition. Especially for video images, histogram-based algorithm is the most common method of scene segmentation. It is simple and convenient to process, and for most videos, it can achieve better results. The method based on histogram divides the grey level, brightness or colour of each pixel in adjacent frames into N levels, and then makes histogram comparison according to the number of pixels in each level. This method counts the total number of grey or colour distributions. It has good tolerance for the motion in the lens and the slow motion of the camera, but it may cause false detection or missed detection when the content of the lens changes rapidly and the lens changes gradually. The process of processing video into a frame image is shown in Fig. 2. The video image is processed into a frame sequence, 3D convolution is performed on the frame sequences, and through feature extraction, the optimal identification output can be obtained after the full connection layer. Fig. 2Open in figure viewerPowerPoint Processing of continuous video frames using 3D ConvNets The third layer is the training processing layer of the model: the cross-view human motion recognition algorithm involves at least two perspectives, and the camera captures the observed actions simultaneously or at the same time. This section only considers two perspectives, which are called source and destination perspectives, respectively. If only the camera from the source perspective captures the action or other factors, the corresponding samples cannot be found from the target perspective. These samples are called isolated action samples from the source perspective. The corresponding is the shared action samples, which are captured by the camera from different perspectives at the same time. Therefore, the action samples correspond to each other in two source and target perspectives. When isolated samples cover all action categories, these isolated samples can be identified to provide sufficient discriminant information for training action classifiers. On the one hand, these action classifiers depend on the perspective and cannot accurately identify the observed actions from other perspectives; on the other hand, the identification information of isolated samples cannot be shared from different perspectives, so there is a lack of sufficient identification samples to complete the classifier training from the purpose perspective. 2.2 Core algorithms of model training for action capture and identification In this section, we propose an algorithm to construct near-semantic visual dictionaries. We analyse the semantic properties of visual words in large-scale visual dictionaries and optimise the composition of visual dictionaries. Ideologically, similar to the visual topic model (probabilistic latent semantic analysis model and latent Dirichlet assignment model), the algorithm abstracts visual words, i.e. 'secondary abstraction' of underlying features, which avoids the problem of 'semantic gap' to a certain extent. The difference between the algorithm in this section and the topic model is that the two types of topic models are essentially probabilistic generation models, while the algorithm in this section is based on the spectral clustering algorithm. The difference between this section's algorithm and others lies in that, first, the definition of semantic similarity is different. The former defines the semantic similarity directly by the co-occurrence frequency of visual words, while the latter defines the semantic similarity based on point-by-point mutual information. Secondly, the former is based on spectral clustering algorithm, but the latter adopts the random walk graph method. The flow chart of human motion recognition algorithm based on the near-semantic visual dictionary is shown in Fig. 3. Fig. 3Open in figure viewerPowerPoint Human motion recognition algorithm based on near-semantic visual dictionary The underlying visual features cannot correspond to the high-level semantics. The clustering-based visual word mapping method only quantifies the underlying visual features according to Euclidean distance. Because the process of feature quantisation does not consider any semantic information, the generated visual words still face the problem of 'semantic gap'. The underlying visual features with the same semantic information are not necessarily mapped into the same visual word. The larger the size of the visual dictionary, the finer the features are quantified, and the greater the probability of this situation. At the same time, the underlying visual features with different semantic information may be mapped to the same visual word, especially when the size of the visual dictionary is smaller and the feature is quantified thicker, the probability of this situation will be greater. By studying the semantic context of visual words, we can easily understand the relationship between visual words and category information. 2.3 Integration of global encoding mode and CNN Based on the global pooling layer and encoding layer, this paper constructs the Global Encoding Module (GEModule), whose structure is shown in Fig. 4. Fig. 4Open in figure viewerPowerPoint Structure of GEModule Take size as an example : (i) parallel input to two branches. On the first branch, the input passes through the encoding layer, batch norm (BN) layer, rectified linear unit (ReLU) activation function and fully-connected (FC) layer, and finally generates the c length of the encoding vector ; on the other branch, the input passes through the global average pooling layer to generate the size of the feature map , and then through the Reshape layer, BN layer and FC layer, the final output of the c length of the feature vector is . (ii) The coding vectors , are connected with the feature vectors by concat operation, and the length of the feature vectors is generated. Finally, the global coding vectors are transformed into the global coding vectors by FC layer transformation. Considering that the input feature response contains M dictionary descriptors, there are M visual coded dictionaries . The output of Encoding layer is a residue coding vector with N lengths. The calculation is as follows: where , denotes the smoothing factor for each cluster centre, denotes the residual, denotes the weighted weight of the corresponding residual coding vector , and the output of encoding layer gets the final residual coding vector through BN layer and ReLU activation function. The final global coding vector is applied to the high-level feature response of the model, weighting each channel of the feature, and selecting more discriminative feature response. GEModule is used as shown in Fig. 5. Fig. 5Open in figure viewerPowerPoint Integrating GEModule into existed CNN architecture Specifically, GEModule is applied to high-level feature mapping to generate global coding vectors. The distinguishing feature graph is selected, and the final output is obtained by adding the original input features. The calculation formula is as follows: The parameters are the set of pixel coordinates of the input characteristic response, representing the sigmoid activation function. Based on the above formulas, the global coding module can be easily integrated into the existing semantics segmentation model and achieve end-to-end training. 3 Simulation and result analysis 3.1 Accuracy validation of human motion recognition algorithm based on near-semantic visual dictionary In order to prove the validity of near-semantic visual dictionary in video concept description, this section also carries out the experiment of video concept search algorithm in the video database. When the search optimisation problem is not considered, the video concept search problem can be simplified to the video classification problem. That is, learning concept detectors from all positive and negative samples in the training set. When searching for a concept in the test set, calculating the probability of the concept appearing in all videos in the test set according to the corresponding concept detectors, and then proceeding to the video according to the level of probability. Sort, and finally return the video with the highest probability. Similar to human motion recognition method, a video search method can use visual word bag description method to describe video content. The results show that the video description method based on near-semantic visual dictionary can effectively improve the search speed and accuracy. Mid-level descriptors represent a visual dictionary generated by K-means clustering, which represents the 'first abstraction' of low-level descriptors. Our near-semantic visual dictionary is called PPM based high-level descriptors because it is based on the Pearson correlation coefficient PPM (for the 'second time' of the initial visual words). At the same time, in order to compare the semantic similarity calculation method based on this section with the one based on the literature 8, the PMI calculation method used in this section will be used to generate a near-semantic visual dictionary, which is referred to as PMI-based high-level descriptors. The only difference between these two kinds of near-semantic visual dictionaries is that they adopt different methods of calculating semantic similarity. Fig. 6a shows the recognition rate of human motion with different dictionary sizes (mid-level descriptors). The results show that the recognition rate of PPM-based high-level descriptors is better than that of mid-level descriptors in the same dictionary size, especially when the dictionary size is 25, the improvement of recognition rate is the most obvious, the former achieves 84%, while the latter only achieves 72%. When the size of the dictionary increases gradually, the recognition rate of PMI high-level descriptors is less obvious than that of PMI high-level descriptors. The recognition rate of mid-level descriptors is very robust to the change of dictionary size. When the dictionary size is 400, the recognition rate of PMI high-level descriptors is as high as 90%, almost the same as the recognition rate of the initial visual dictionary with the dictionary size. Fig. 6b does not show the human motion recognition rate of different dictionary sizes (lower and other visual dictionaries). By comparing the recognition rates of high-level descriptors and PMI high-level descriptors under the same dictionary size, the conclusion is drawn that our proposed algorithm is superior to the semantic similarity calculation method based on, and the recognition rate is increased by 0.67% on average. In addition, by comparing the recognition rates of high-level descriptors and PMI high-level descriptors [19] under the same dictionary size, the same conclusion is drawn that the near-semantic visual dictionary generation algorithm proposed in this chapter is superior to the PPM-based high-level descriptors in recognition rate, which increases the average recognition rate by 2.09%. Fig. 6Open in figure viewerPowerPoint Comparison results under KTH data set (a) Comparison result between the mid-level descriptors and our algorithm, (b) Comparison result between the high-level descriptors, PMI high-level descriptors and our algorithm 3.2 Accuracy validation of human motion recognition algorithm based on near-semantic visual dictionary under different dataset In this section, the human motion recognition comparison experiment uses the literature 8. The proposed single-scale spatio-temporal interest point detection method and descriptor extract the underlying visual features from a motion video. The filtering scales of interest point detection in space and time are set to 0.25 and 2, respectively, and the number of spatio-temporal interest points in each video is set to 200. K-means clustering algorithm is used to cluster the underlying visual features collected from each perspective, and a 1000-scale view-dependent visual dictionary is constructed. Cross-view action recognition will be carried out from both source and target perspectives. First, a 100-scale cross-perspective visual dictionary is learned by sharing action videos from two perspectives. Because multi-view databases capture daily actions from a single perspective, 20 different perspectives will be considered, one as the source perspective and the other as the purpose perspective. Finally, considering the simplicity of calculation and the accuracy of recognition, the SVM classifier is used to classify actions, and its kernel function is the orthogonal kernel of the histogram. For the convenience of comparison, the following experiments are abbreviated as follows. First, this section assumes that the initial visual dictionary generated by K-means clustering depends on the perspective, and the motion recognition algorithm based on this kind of visual dictionary cannot deal with the situation of cross-perspective action, that is, the action classifier from the source perspective cannot accurately identify the unknown action captured from the target perspective. This section will validate this hypothesis by experiment. The motion recognition algorithm based on the initial visual dictionary will be abbreviated as Baseline VW. Furthermore, because the cross-view visual dictionary learning process proposed in this section involves multi-source constrained neighbourhood propagation, the cross-view motion recognition algorithm using a cross-view visual dictionary is referred to as our algorithm. Finally, in order to study the effect of multi-source semantic similarity constrained nearest-neighbour propagation technology on cross-view visual dictionary learning, this section will consider whether to use this technology in two cases, and compare the recognition rate of PMI high-level descriptors, our algorithm in two cases. The simulation results are as follows. Table 1 compares PMI high-level descriptors (Abbreviation: PMI) algorithm with our proposed algorithm in action recognition rate. It can be observed that PMI high-level descriptors algorithm achieves satisfactory recognition results under some combination of perspectives. For example, the PMI high-level descriptors algorithm achieves 75.06% recognition rate when the source view is view0 for View1 and 55.83% when the source view is view2. This is because the human motion appearances from these perspectives are not very different, so the visual dictionary's perspective dependence is not so obvious. When cross-view action recognition involves view4, the recognition results obtained by PMI high-level descriptors algorithm are not satisfactory, either as the source perspective or the purpose perspective. Because the human motion appearances collected below are quite different from other perspectives, the visual dictionary's perspective dependence seriously affects the recognition results. Compared with PMI high-level descriptors algorithm, cross-view visual dictionary greatly improves the recognition results of the algorithm. The average recognition rate of the algorithm is 15.8% higher than the average recognition rate of the algorithm. When the source view is chosen as the target view, the corresponding recognition rate reaches a maximum of 78.73%. Similarly, when cross-view motion recognition involves view4, the recognition rate is lower than that of other visual angle combinations. However, the recognition rate is still considerably improved compared with the PMI high-level descriptors algorithm. When view1 is chosen as the source view and view4 is chosen as the target view, the recognition rate is increased by up to 24%. Table 1. Comparison of recognition rates of human motion algorithms View0 View1 View2 View3 View4 PMI Our method PMI Our method PMI Our method PMI Our method PMI Our method View0 NA NA 64.38 77.02 49.31 62.10 48.08 68.70 23.47 42.05 View1 75.06 78.73 NA NA 43.68 65.53 50.20 63.57 20.86 35.96 View2 49.47 66.26 39.85 64.28 NA NA 55.83 68.95 38.38 53.79 View3 46.54 67.24 39.77 63.33 51.34 61.61 NA NA 24.45 36.92 View4 30.73 54.77 23.80 47.92 52.73 63.18 34.12 37.45 NA NA 3.3 Accuracy comparison under different action class In order to compare the effectiveness of our algorithm and PMI high-level descriptors algorithm, we will further compare the recognition rate of each action category from one source perspective and from another target perspective. Fig. 7 shows the rate of action recognition for different types of actions from other objective perspectives when choosing the source perspective. Our proposed algorithm is better than PMI high-level descriptors algorithm in recognising most action categories. Fig. 7Open in figure viewerPowerPoint Simulation results of our algorithm under KTH dataset (a) Test 1, (b) Test 2, (c) Test 3, (d) Test 4, (e) Test 5 As shown in Figs. 7 and 8, the action class includes Check-W, Cross-A, Scratch-II, Sit_down, Get_up, Turn_A, Walk, Wave, Punch, Kick, Pick-up. In order to study the impact of multi-source constrained near-neighbour communication technology on cross-perspective visual dictionary learning, this section will consider whether to use this technology in two cases, and compare the recognition rate of the sum of the two cases. The figure shows the comparison of the recognition rate of different actions between the algorithm and the algorithm when choosing the source angle of view. The recognition result of the algorithm is better than that of the algorithm, and its average recognition rate is higher than that of the algorithm. In addition, our method is superior to PMI high-level descriptors in recognising most action categories, especially when the target view is view1 to view2, the recognition rate is greatly improved. Fig. 8Open in figure viewerPowerPoint Simulation results of PMI high-level descriptors under KTH dataset (a) Test 1, (b) Test 2, (c) Test 3, (d) Test 4, (e) Test 5 3.4 Effectiveness validation of integration of global encoding mode and CNN In the paper, global encoding mode and CNN modules are integrated in the deep-learning framework, which are core components in the framework. Therefore, the effectiveness of the integration of global encoding mode and CNN is needed to validate. In order to verify the effectiveness of the integration of global encoding mode and CNN, we conduct a series of simulation in this section. The simulation results are shown in Fig. 9. Fig. 9Open in figure viewerPowerPoint Comparison results between with the integration and without integration under different datasets (a) Comparison between the integration and without integration under KTH dataset, (b) Comparison between the integration and without integration under Weizmann dataset, (c) Comparison between the integration and without integration under YouTube dataset As shown in Fig. 9, the model with integration and without integration are compared, the simulation shows that the model with integration yields the significant improvement for accuracy rate under different datasets. 3.5 Calculation speed validation of human motion recognition algorithm In order to verify the calculation speed of the proposed human motion recognition algorithm, we conduct a series of simulation in this section. The simulation results are shown in Fig. 10. Fig. 10Open in figure viewerPowerPoint Comparison results of calculation speed of human motion recognition algorithm (a) The calculation speed comparison between our algorithm and PMI algorithm under KTH dataset, (b) The calculation speed comparison between our model and PMI algorithm under Weizmann dataset, (c) The calculation speed comparison between our model and PMI algorithm under TouTube dataset As shown in Fig. 10, the calculation speed of the model is compared, the simulation shows that the model with integration yields the significant improvement for calculation under different datasets. 4 Conclusion In order to improve the accuracy of small-range human motion recognition in video and the computational efficiency of large-scale datasets, a multi-dimensional data model of motion recognition and motion capture in video image based on deep-learning framework was proposed. First, the moving foreground of the target is extracted by the Gauss mixture model, and the human body is recognised by the HOG. At the second level, the dense trajectory feature and the deep learning feature are fused, according to the integration of global encoding algorithm and CNN. In the deep learning feature, the fusion of the deep video feature and the video RGB tricolour feature is taken as the feature of deep learning. Finally, the classification is based on the deep learning network model. The simulation experiments based on large-scale real data sets and small-scale gesture data sets show that the algorithm has high recognition accuracy for large-scale data sets and small-scale gesture actions. In addition, ICVL human behaviour data set is used to classify the experimental data. The average classification accuracy is 85.79%. Searching optimisation will be considered in our future work. 5 Acknowledgments This work was supported by the 13th Five-Year Plan of Jiangxi Education Science, the subject of 'Research on Compensation Mechanism of Physical Health Education in Rural Primary Schools of Ganjiang New District' in 2019 (no. 19YB232). 6 References 1Laraba S., Brahimi M., and Tilmanne J. et al: ' 3D skeleton-based action recognition by representing motion capture sequences as 2D-RGB images', Comput. Animat. & Virtual Worlds, 2017, 28, (3–4), p. e1782 Wiley Online Library Web of Science®Google Scholar 2Panagiotakis C., Papoutsakis K., and Argyros A.: ' A graph-based approach for detecting common actions in motion capture data and videos', Pattern Recognit., 2018, 79, pp. 1– 11 Crossref Web of Science®Google Scholar 3Switonski A., Josinski H., and Wojciechowski K.: ' Dynamic time warping in classification and selection of motion capture data', Multidimens. Syst. Signal Process., 2018, 6, pp. 1– 32 Google Scholar 4Fanello S.R., Gori I., and Metta G. et al: ' Keep it simple and sparse: real-time action recognition', J. Mach. Learn. Res., 2017, 14.1, pp. 2617– 2640 Google Scholar 5Hachaj T., Piekarczyk M., and Ogiela M.R.: ' Human actions analysis: templates generation, matching and visualization applied to motion capture of highly-skilled karate athletes', Sensors, 2017, 17, (11), p. 2590 Crossref Web of Science®Google Scholar 6Hanley B., Tucker C.B., and Bissas A.: ' Differences between motion capture and video analysis systems in calculating knee angles in elite-standard race walking', J. Sports Sci., 2017, 36, (11), pp. 1– 6 PubMed Web of Science®Google Scholar 7Tu Z., Li H., and Zhang D. et al: ' Action-stage emphasized spatiotemporal VLAD for video action recognition', IEEE Trans Image Process, 2019, 28, (6), pp. 2799– 2812 Crossref Web of Science®Google Scholar 8Wei W., Zhou B., and Połap D. et al: ' A regional adaptive variational PDE model for computed tomography image reconstruction', Pattern Recognit., 2019, 92, pp. 64– 81 Crossref CAS Web of Science®Google Scholar 9Tits M., Tilmanne J., and Dutoit T.: ' Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging', Plos One, 2018, 13, (7), p. e0199744 PubMed Web of Science®Google Scholar 10Xin L., Xu M., and Peng S.J. et al: ' Efficient human motion capture data annotation via multi-view spatiotemporal feature fusion', IET Signal Process., 2018, 12, (3), pp. 269– 276 Wiley Online Library Web of Science®Google Scholar 11Li J., Mao X., and Wu X. et al: ' Human action recognition based on tensor shape descriptor', IET Comput. Vis., 2017, 10, (8), pp. 905– 911 Wiley Online Library CAS Web of Science®Google Scholar 12Hussain S., Keung J., and Khan A.A. et al: ' Implications of deep learning for the automation of design patterns organization', J. Parallel Distrib. Comput., 2018, 117, pp. 256– 266 Crossref Web of Science®Google Scholar 13Cui K., and Qin X.: ' Virtual reality research of the dynamic characteristics of soft soil under metro vibration loads based on BP neural networks', Neural Comput Appl, 2018, 29, (5), pp. 1233– 1242 Crossref Web of Science®Google Scholar 14Karatsidis A., Bellusci G., and Veltink P.H. et al: ' Estimation of ground reaction forces and moments during gait using only inertial motion capture', Sensors, 2017, 17, (1), p. 75 Web of Science®Google Scholar 15Kim Y., Baek S., and Bae B.C.: ' Motion capture of the human body using multiple depth sensors', ETRI J., 2017, 39, (2), pp. 181– 190 Wiley Online Library Web of Science®Google Scholar 16Aissaoui A., Ouafi A., and Pudlo P. et al: ' Designing a camera placement assistance system for human motion capture based on a guided genetic algorithm', Virtual. Real., 2017, 22, (sup1), pp. 1– 11 Google Scholar 17Guess T.M., Razu S., and Jahandar A. et al: ' Comparison of 3D joint angles measured with the Kinect 2.0 skeletal tracker versus a marker based motion capture system', J. Appl. Biomech., 2017, 33, (2), p. 1– Crossref PubMed Web of Science®Google Scholar 18Yang B., Dong H., and Saddik A.E.: ' Development of a self-calibrated motion capture system by nonlinear trilateration of multiple Kinects v2', IEEE Sens. J., 2017, 17, (8), pp. 2481– 2491 Crossref Web of Science®Google Scholar 19Sedmidubsky J., Elias P., and Zezula P.: ' Searching for variable-speed motions in long sequences of motion capture data', Inf. Syst., 2019, 80, pp. 148– 158 Crossref Web of Science®Google Scholar Citing Literature Volume14, Issue7May 2020Pages 1257-1264 FiguresReferencesRelatedInformation