# Research and Application of Multifeature Gesture Recognition in Human-Computer Interaction Based on Virtual Reality Technology

Authors: Zhao Feng; Jinlong Wu; Taile Ni
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Objective. To explore the research and application of multifeature gesture recognition in virtual reality human-computer interaction and to explore the gesture recognition technology scheme to achieve better human-computer interaction experience. Methods. Through the study of the technical difficulties of gesture recognition, comparative static gesture feature recognition and feature fusion algorithms are applied, in the process of research on gesture partition, and adjust the contrast of characteristic parameters, combined with the feature of space-time dynamic gesture tracking trajectory and dynamic gesture recognition and gesture recognition effect under different scheme. Results. The central region was divided into 0 regions, and the central region was divided into 1-4 regions in counterclockwise direction. Compared with the traditional gesture changes, the overlapping problem in the four partition modes was reduced, the gesture was better displayed, and the operation and use of gesture processing were realized more efficiently. Conclusion. Gesture recognition requires the combination of static gesture feature information recognition, gesture feature fusion, spatiotemporal trajectory feature, and dynamic gesture trajectory feature to achieve a better human-computer interaction experience.


---

## Body

## 1. Introduction

With the wide use of computer and Internet in people’s daily life, it is generally believed that the human-computer interaction technology, which is widely used at present with the mouse, keyboard, or touch screen as the interactive means, will become the bottleneck for the further effective utilization of information flow. With the continuous popularization of intelligent concepts and the rapid development of related technologies, people begin to explore more natural, more friendly, and more effective human-computer interaction methods [1, 2]. As a result, a variety of new human-computer interaction technologies continue to emerge, such as speech recognition, facial recognition, gesture recognition, and human movement recognition. Among these numerous technologies, gesture interaction technology has a wide range of application fields and broad application prospects because of its natural, intuitive, and easy to learn characteristics.Multifeature gesture recognition is carried out by using the virtual reality human-computer interaction software. Line of sight selection directly affects the final gesture recognition function [3]. Only by selecting the correct sight gesture recognition can it proceed smoothly, so as to meet the recognition and tracking requirements in the virtual reality human-computer interaction system [4]. Gesture recognition belongs to the dynamic information capture, tracking accuracy in information and update on time to achieve defined standards; implementation choice first needs to determine the control points, which is the line of sight in the process of exchange center, determined by the center for visual direction judgment, users in the process of watching interaction point, to achieve the transformation on the vision. And the multifeature gesture is recognized into the computer software [5]. In the tracking algorithm of multifeature gesture recognition, line of sight selection can also be understood as the control of gesture capture points. The basic anchor points of each gesture are captured to more accurately integrate multifeature dynamic gesture recognition into the human-computer interaction system.Virtual reality (VR for short) refers to the use of computer-related technology to simulate the real scene and experience, trying to create an immersive feeling. Among them, the word “virtual” means that it does not exist in the actual physical space but exists in the virtual/cyberspace of the image made by the computer. Virtual reality technology is an upgrade of display technology and a combination of highly developed computer technology in various fields. Virtual reality technology was mainly applied to the desktop in the early stage. Due to the large size and high price of immersive virtual reality display helmet, exists only in some research institutions and laboratories, but with the development of smart hardware technology, such as gyroscopes, small high-definition screens, and lightweight processors, it reduces the development cost of virtual reality equipment and improves related performance, which makes the Oculus Rift prototype appear in 2012 and immersive virtual reality technology enter the consumer level. Thus, virtual reality begins to transform from desktop to immersive [6].At present, the characteristics of virtual reality technology can be summarized into three points: immersion, interaction, and imagination, referred to as 3I characteristics [7]. In recent years, the development of virtual reality technology is mainly to solve the first attribute, namely, immersion. By improving the performance of related hardware, it tries to make users in the virtual environment reach the degree that it is difficult to distinguish the real from the fake. However, it should be noted that interaction is also a very important attribute, which is the core link for virtual reality to provide users with experience and move towards application.Real-time dynamic gesture recognition technology is based on computer vision, and using OpenTV computer vision library on the platform to design an application is based on this technology in multimedia teaching control. The main content is divided into four parts: gesture detection, gesture tracking, gesture recognition, and the final application implementation. In the part of gesture detection, first of all, the method is used to collect real-time video images, and then, the motion detection method combining background difference and color histogram information is used to complete gesture detection. In the experiment, the brightness values of the pixels in the moving skin region were specially adjusted to eliminate interference from the stationary face region.At present, gesture interaction generally uses computer vision algorithm to capture the user’s hand movement and convert it into human-computer interaction commands. Compared with traditional interaction methods, gesture interaction based on vision can enable the user to get rid of the constraints of input devices. It has the advantages of low attachment, weak invasion, and more natural interaction experience [8]. However, due to the excessive flexibility of gestures and the accuracy of the sensor itself, the existing gesture interaction accuracy still needs to be improved. In order to solve the above problems, there is an urgent need to develop new theories and methods of gesture interaction, so as to improve users’ interactive experience in the immersive virtual reality environment.


## 2. Research Methods

Gesture recognition research mainly includes two aspects: static gesture recognition and dynamic gesture recognition. The essence of static gesture recognition is to classify different static gestures, that is, to extract the features of the acquired gestures first, and then to use the classification algorithm to determine. Static gesture recognition is the foundation of dynamic gesture recognition, which is more complicated.


### 2.1. Static Gesture Recognition

The basic steps of static gesture recognition mainly include the acquisition of gesture image, gesture detection and segmentation, gesture feature analysis and feature extraction, gesture classification and recognition, and feedback of the final result. The basic steps of static gesture recognition are shown in Figure1.Figure 1
Basic steps for static gesture recognition.Gesture model construction is the key of gesture feature analysis. The essence of gesture model construction is to propose a practical and effective discriminator for gesture recognition through in-depth analysis of all valid features of gestures in practical application situations. The specific process includes two main steps: first, for the extracted gesture image, analysis and extraction of its main features. Second, the feature pattern library of different gestures is constructed according to the extracted gesture features [9]. Among them, the first step is the foundation, and the quality of the pattern library constructed in the second step will directly affect the recognition result. At present, static gesture modeling is divided into two categories: epigenument-based gesture modeling and 3D-based gesture modeling. The main difference lies in the perspective from which the model is modeled, 2D and 3D, respectively. In different application scenarios, different gesture modeling methods should be selected according to the conditions.Gesture modeling based on appearance analyzes and extracts the gesture feature information from the two-dimensional plane image of the gesture, and the commonly used information includes color information, contour information, and texture information. The color information of gesture is modeled by building skin color model. Gesture contour information is extracted after edge detection, which belongs to the whole geometric feature of gesture. Gesture texture information is the texture feature that describes the details of gesture image. In comparison, gesture modeling based on 3D model pays more attention to 3D characterization of the whole gesture, and the gesture information described is more accurate, and the model contains a lot of parameters. The construction of 3D model requires multiple cameras to collect gesture images from different perspectives, and the amount of data acquired is usually large, and the processing is complex. At the same time, in the real application environment, a single ordinary camera is relatively common. It can only obtain the user’s gesture image from a two-dimensional plane, extract the color, contour and texture information of the gesture, and construct the appearance model of the gesture [10].


### 2.2. Dynamic Gesture Tracking

Compared with static gesture recognition, the process of dynamic gesture recognition is more complicated and has more problems. Because dynamic gesture is composed of a series of continuous static gesture sequences, it not only contains all the characteristic information of a single static gesture, but also contains the shape change and position change of the gesture in space. Therefore, according to different recognition scenarios, different feature information should be selected to describe dynamic gestures.Gesture tracking is a key step in the process of visual dynamic gesture recognition. Gesture tracking refers to the real-time positioning and tracking of moving gestures in video sequence images by using certain characteristics of gestures, which is used to establish a time-domain model of dynamic gestures. The efficiency of gesture tracking will seriously affect the real-time performance of the whole interactive system, and the accuracy of tracking will directly affect the accuracy of subsequent gesture recognition results [11]. It is of great significance to build a robust gesture recognition interactive system through in-depth study of gesture tracking. Because gestures are nonrigid and highly flexible, and dynamic gestures may have random deformation, illumination intensity change, complex environment interference in real time (such as the interference of skin color objects), object blocking, and other situations during the movement, it is very challenging to solve these problems.In a complex environment, it is still a big challenge to be able to efficiently and accurately implement a realistic visual-based gesture tracking algorithm. Researchers often sacrifice the real-time requirement of the system by integrating the accuracy and robustness of multifeature reality tracking. There are various factors in the real environment, such as gesture deformation, motion speed, gesture scale, light intensity, and target occlusion, which affect the accuracy and real-time performance of visual gesture tracking.


### 2.3. Research and Application of Multifeature Gesture Recognition

#### 2.3.1. Gesture Recognition Feature Extraction

HOG feature extraction. In this feature extraction algorithm, image description is used to construct the scene. In the face of multifeature gestures, the first step in the extraction process is to determine the appearance, extract the features of the external contour, depict the virtual scene consistent with the gesture, and continue to enhance the accuracy of multifeature gestures through the construction of this virtual scene. This feature extraction algorithm is often used in security identification. The multifeature gesture changes a lot, and the final control efficiency will be significantly enhanced after the application of this tracking method. Using the HOG extraction method in multifeature gesture tracking, the gesture image will be subdivided into multiple units, and each unit has different features, so as to describe the edge image in the change of gesture [12]. And these subdivided cells are uniformly divided into the same area, so that the extracted gradient edge map is also connected together.Dynamic gesture image partition processing. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes. According to the gesture changes under different circumstances, it is divided into 5 regions, as shown in Figures 2–4.Figure 2
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 3
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 4
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.The central region of the palm was divided into 0 regions, and along the central region, 1-4 regions were divided in a counterclockwise direction. Compared with the traditional gesture change, this kind of gesture change reduces the overlapping problem in the four partition modes and presents the gesture better. In capturing gestures in different regions, error control should be kept consistent. After the overlap problem is solved, the details of gesture actions are also displayed, and the captured actions can be reflected in real time in the virtual reality human-computer interaction system, so as to realize more efficient operation and use of gesture processing. After the task of gesture capture, it can enter the following algorithm fusion. The field control is carried out through the cooperation between algorithms, and the specific changes of gestures are shown through the transformation between algorithms.HOG feature extraction and tracking algorithm. The Gaussian function calculation method was used to construct the field calculation and smooth processing of the grayscale image captured by the gesture edge. The calculation formula is as follows:
(1)δfδx=fx+1−fx,where it represents the position of captured dynamic points in coordinates, establishes a subset of captured dynamic recognition points of multifeature gestures, and improves the final capture effect in its own interactive processing. The gradient value of the edge part is also affected by the pixels in the process of gesture dynamic capture, and the influence of edge jitter is more serious in the process of gesture dynamic capture. In this case, the range of coordinate value subset can be designed in the process of pixel capture control. The system makes automatic error compensation according to the parameter results in the operation process. After adjusting the data, the captured edge of the gesture is more rounded.


#### 2.3.2. HOG Gesture Feature Fusion

Gesture feature fusion needs to be based on the weighted matrix, and interactive processing can be carried out through the default feature fusion within the system [13]. The mark points in the changes of different gestures are tracked and captured. In the process of feature fusion, it is only carried out for the changes of gesture movements. Therefore, in the primary processing stage, the grayscale processing is carried out on the image to avoid the influence of image color on feature fusion. The feature fusion adopts the mutual fusion between vector and matrix. The feature vector set of gesture image is FHOG=[FH1OG FH2OG... FHNOG], and the global feature is represented by the vector FHU=[FH1 FH2 FH3... FH8 FH9]. The influence degree of different features varies greatly. If the influence degree is directly applied to feature fusion, the interaction between different influences will result in the final gesture recognition error. Before feature fusion, one of the vectors will be normalized to determine that the subset can maintain the influence of the same vector in the mutual influence mode, so as to help improve the final extraction of virtual reality human-computer interaction algorithm and determine the required data subset range through this method.


#### 2.3.3. Dynamic Gesture Characteristics of Spatial-Temporal Trajectory

Gesture feature extraction based on spatiotemporal trajectory should first determine its position, observe the position of the gesture in the spatiotemporal trajectory, and then magnify the key part to facilitate the capture and recognition of the changes of the gesture. The description of position trajectory also needs to be carried out with the help of vector to determine the angle in the process of gesture change. Establish the following formula:θ=arctanyt+1−yt/xt+1−xt. Different angle coordinates in the process of gesture change can be brought into it, and more detailed results and values can be obtained through calculation, so as to capture dynamic gesture features under the spatiotemporal trajectory. At the same time, there may be interference factors in the picture entered into the virtual reality human-computer interaction mode. The purpose of determining the position through angle calculation is to screen out such interference data and realize more accurate multifeature gesture recognition and tracking.


#### 2.3.4. HMM Dynamic Gesture Track Recognition

Trajectory recognition is an important basis of gesture authentication. When the gesture to be recognized is in the dynamic mode, the trajectory of the whole movement process needs to be described in detail in the virtual reality human-computer interaction system. The trajectory description can accurately reflect the motion process of the gesture and refine the edge part [14]. HMM technology is to establish a statistical model to determine the trajectory through the various coordinate points and then through the establishment of a subset to restore the trajectory to the computer virtual system, showing the impact of the final grayscale processing. The specific program structure of trajectory recognition realized by HMM technology is shown in Figure 5. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect. The edge of dynamic gesture is more blurred than that of static gesture, so that the simulation effect can be kept similar.Figure 5
Composition of the HMM trajectory identification system. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect.


## 2.1. Static Gesture Recognition

The basic steps of static gesture recognition mainly include the acquisition of gesture image, gesture detection and segmentation, gesture feature analysis and feature extraction, gesture classification and recognition, and feedback of the final result. The basic steps of static gesture recognition are shown in Figure1.Figure 1
Basic steps for static gesture recognition.Gesture model construction is the key of gesture feature analysis. The essence of gesture model construction is to propose a practical and effective discriminator for gesture recognition through in-depth analysis of all valid features of gestures in practical application situations. The specific process includes two main steps: first, for the extracted gesture image, analysis and extraction of its main features. Second, the feature pattern library of different gestures is constructed according to the extracted gesture features [9]. Among them, the first step is the foundation, and the quality of the pattern library constructed in the second step will directly affect the recognition result. At present, static gesture modeling is divided into two categories: epigenument-based gesture modeling and 3D-based gesture modeling. The main difference lies in the perspective from which the model is modeled, 2D and 3D, respectively. In different application scenarios, different gesture modeling methods should be selected according to the conditions.Gesture modeling based on appearance analyzes and extracts the gesture feature information from the two-dimensional plane image of the gesture, and the commonly used information includes color information, contour information, and texture information. The color information of gesture is modeled by building skin color model. Gesture contour information is extracted after edge detection, which belongs to the whole geometric feature of gesture. Gesture texture information is the texture feature that describes the details of gesture image. In comparison, gesture modeling based on 3D model pays more attention to 3D characterization of the whole gesture, and the gesture information described is more accurate, and the model contains a lot of parameters. The construction of 3D model requires multiple cameras to collect gesture images from different perspectives, and the amount of data acquired is usually large, and the processing is complex. At the same time, in the real application environment, a single ordinary camera is relatively common. It can only obtain the user’s gesture image from a two-dimensional plane, extract the color, contour and texture information of the gesture, and construct the appearance model of the gesture [10].


## 2.2. Dynamic Gesture Tracking

Compared with static gesture recognition, the process of dynamic gesture recognition is more complicated and has more problems. Because dynamic gesture is composed of a series of continuous static gesture sequences, it not only contains all the characteristic information of a single static gesture, but also contains the shape change and position change of the gesture in space. Therefore, according to different recognition scenarios, different feature information should be selected to describe dynamic gestures.Gesture tracking is a key step in the process of visual dynamic gesture recognition. Gesture tracking refers to the real-time positioning and tracking of moving gestures in video sequence images by using certain characteristics of gestures, which is used to establish a time-domain model of dynamic gestures. The efficiency of gesture tracking will seriously affect the real-time performance of the whole interactive system, and the accuracy of tracking will directly affect the accuracy of subsequent gesture recognition results [11]. It is of great significance to build a robust gesture recognition interactive system through in-depth study of gesture tracking. Because gestures are nonrigid and highly flexible, and dynamic gestures may have random deformation, illumination intensity change, complex environment interference in real time (such as the interference of skin color objects), object blocking, and other situations during the movement, it is very challenging to solve these problems.In a complex environment, it is still a big challenge to be able to efficiently and accurately implement a realistic visual-based gesture tracking algorithm. Researchers often sacrifice the real-time requirement of the system by integrating the accuracy and robustness of multifeature reality tracking. There are various factors in the real environment, such as gesture deformation, motion speed, gesture scale, light intensity, and target occlusion, which affect the accuracy and real-time performance of visual gesture tracking.


## 2.3. Research and Application of Multifeature Gesture Recognition

### 2.3.1. Gesture Recognition Feature Extraction

HOG feature extraction. In this feature extraction algorithm, image description is used to construct the scene. In the face of multifeature gestures, the first step in the extraction process is to determine the appearance, extract the features of the external contour, depict the virtual scene consistent with the gesture, and continue to enhance the accuracy of multifeature gestures through the construction of this virtual scene. This feature extraction algorithm is often used in security identification. The multifeature gesture changes a lot, and the final control efficiency will be significantly enhanced after the application of this tracking method. Using the HOG extraction method in multifeature gesture tracking, the gesture image will be subdivided into multiple units, and each unit has different features, so as to describe the edge image in the change of gesture [12]. And these subdivided cells are uniformly divided into the same area, so that the extracted gradient edge map is also connected together.Dynamic gesture image partition processing. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes. According to the gesture changes under different circumstances, it is divided into 5 regions, as shown in Figures 2–4.Figure 2
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 3
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 4
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.The central region of the palm was divided into 0 regions, and along the central region, 1-4 regions were divided in a counterclockwise direction. Compared with the traditional gesture change, this kind of gesture change reduces the overlapping problem in the four partition modes and presents the gesture better. In capturing gestures in different regions, error control should be kept consistent. After the overlap problem is solved, the details of gesture actions are also displayed, and the captured actions can be reflected in real time in the virtual reality human-computer interaction system, so as to realize more efficient operation and use of gesture processing. After the task of gesture capture, it can enter the following algorithm fusion. The field control is carried out through the cooperation between algorithms, and the specific changes of gestures are shown through the transformation between algorithms.HOG feature extraction and tracking algorithm. The Gaussian function calculation method was used to construct the field calculation and smooth processing of the grayscale image captured by the gesture edge. The calculation formula is as follows:
(1)δfδx=fx+1−fx,where it represents the position of captured dynamic points in coordinates, establishes a subset of captured dynamic recognition points of multifeature gestures, and improves the final capture effect in its own interactive processing. The gradient value of the edge part is also affected by the pixels in the process of gesture dynamic capture, and the influence of edge jitter is more serious in the process of gesture dynamic capture. In this case, the range of coordinate value subset can be designed in the process of pixel capture control. The system makes automatic error compensation according to the parameter results in the operation process. After adjusting the data, the captured edge of the gesture is more rounded.


### 2.3.2. HOG Gesture Feature Fusion

Gesture feature fusion needs to be based on the weighted matrix, and interactive processing can be carried out through the default feature fusion within the system [13]. The mark points in the changes of different gestures are tracked and captured. In the process of feature fusion, it is only carried out for the changes of gesture movements. Therefore, in the primary processing stage, the grayscale processing is carried out on the image to avoid the influence of image color on feature fusion. The feature fusion adopts the mutual fusion between vector and matrix. The feature vector set of gesture image is FHOG=[FH1OG FH2OG... FHNOG], and the global feature is represented by the vector FHU=[FH1 FH2 FH3... FH8 FH9]. The influence degree of different features varies greatly. If the influence degree is directly applied to feature fusion, the interaction between different influences will result in the final gesture recognition error. Before feature fusion, one of the vectors will be normalized to determine that the subset can maintain the influence of the same vector in the mutual influence mode, so as to help improve the final extraction of virtual reality human-computer interaction algorithm and determine the required data subset range through this method.


### 2.3.3. Dynamic Gesture Characteristics of Spatial-Temporal Trajectory

Gesture feature extraction based on spatiotemporal trajectory should first determine its position, observe the position of the gesture in the spatiotemporal trajectory, and then magnify the key part to facilitate the capture and recognition of the changes of the gesture. The description of position trajectory also needs to be carried out with the help of vector to determine the angle in the process of gesture change. Establish the following formula:θ=arctanyt+1−yt/xt+1−xt. Different angle coordinates in the process of gesture change can be brought into it, and more detailed results and values can be obtained through calculation, so as to capture dynamic gesture features under the spatiotemporal trajectory. At the same time, there may be interference factors in the picture entered into the virtual reality human-computer interaction mode. The purpose of determining the position through angle calculation is to screen out such interference data and realize more accurate multifeature gesture recognition and tracking.


### 2.3.4. HMM Dynamic Gesture Track Recognition

Trajectory recognition is an important basis of gesture authentication. When the gesture to be recognized is in the dynamic mode, the trajectory of the whole movement process needs to be described in detail in the virtual reality human-computer interaction system. The trajectory description can accurately reflect the motion process of the gesture and refine the edge part [14]. HMM technology is to establish a statistical model to determine the trajectory through the various coordinate points and then through the establishment of a subset to restore the trajectory to the computer virtual system, showing the impact of the final grayscale processing. The specific program structure of trajectory recognition realized by HMM technology is shown in Figure 5. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect. The edge of dynamic gesture is more blurred than that of static gesture, so that the simulation effect can be kept similar.Figure 5
Composition of the HMM trajectory identification system. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect.


## 2.3.1. Gesture Recognition Feature Extraction

HOG feature extraction. In this feature extraction algorithm, image description is used to construct the scene. In the face of multifeature gestures, the first step in the extraction process is to determine the appearance, extract the features of the external contour, depict the virtual scene consistent with the gesture, and continue to enhance the accuracy of multifeature gestures through the construction of this virtual scene. This feature extraction algorithm is often used in security identification. The multifeature gesture changes a lot, and the final control efficiency will be significantly enhanced after the application of this tracking method. Using the HOG extraction method in multifeature gesture tracking, the gesture image will be subdivided into multiple units, and each unit has different features, so as to describe the edge image in the change of gesture [12]. And these subdivided cells are uniformly divided into the same area, so that the extracted gradient edge map is also connected together.Dynamic gesture image partition processing. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes. According to the gesture changes under different circumstances, it is divided into 5 regions, as shown in Figures 2–4.Figure 2
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 3
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.Figure 4
Area map of gesture. After applying the HOG feature region method, there is also a breakthrough innovation in the partitioning processing of gesture changes.The central region of the palm was divided into 0 regions, and along the central region, 1-4 regions were divided in a counterclockwise direction. Compared with the traditional gesture change, this kind of gesture change reduces the overlapping problem in the four partition modes and presents the gesture better. In capturing gestures in different regions, error control should be kept consistent. After the overlap problem is solved, the details of gesture actions are also displayed, and the captured actions can be reflected in real time in the virtual reality human-computer interaction system, so as to realize more efficient operation and use of gesture processing. After the task of gesture capture, it can enter the following algorithm fusion. The field control is carried out through the cooperation between algorithms, and the specific changes of gestures are shown through the transformation between algorithms.HOG feature extraction and tracking algorithm. The Gaussian function calculation method was used to construct the field calculation and smooth processing of the grayscale image captured by the gesture edge. The calculation formula is as follows:
(1)δfδx=fx+1−fx,where it represents the position of captured dynamic points in coordinates, establishes a subset of captured dynamic recognition points of multifeature gestures, and improves the final capture effect in its own interactive processing. The gradient value of the edge part is also affected by the pixels in the process of gesture dynamic capture, and the influence of edge jitter is more serious in the process of gesture dynamic capture. In this case, the range of coordinate value subset can be designed in the process of pixel capture control. The system makes automatic error compensation according to the parameter results in the operation process. After adjusting the data, the captured edge of the gesture is more rounded.


## 2.3.2. HOG Gesture Feature Fusion

Gesture feature fusion needs to be based on the weighted matrix, and interactive processing can be carried out through the default feature fusion within the system [13]. The mark points in the changes of different gestures are tracked and captured. In the process of feature fusion, it is only carried out for the changes of gesture movements. Therefore, in the primary processing stage, the grayscale processing is carried out on the image to avoid the influence of image color on feature fusion. The feature fusion adopts the mutual fusion between vector and matrix. The feature vector set of gesture image is FHOG=[FH1OG FH2OG... FHNOG], and the global feature is represented by the vector FHU=[FH1 FH2 FH3... FH8 FH9]. The influence degree of different features varies greatly. If the influence degree is directly applied to feature fusion, the interaction between different influences will result in the final gesture recognition error. Before feature fusion, one of the vectors will be normalized to determine that the subset can maintain the influence of the same vector in the mutual influence mode, so as to help improve the final extraction of virtual reality human-computer interaction algorithm and determine the required data subset range through this method.


## 2.3.3. Dynamic Gesture Characteristics of Spatial-Temporal Trajectory

Gesture feature extraction based on spatiotemporal trajectory should first determine its position, observe the position of the gesture in the spatiotemporal trajectory, and then magnify the key part to facilitate the capture and recognition of the changes of the gesture. The description of position trajectory also needs to be carried out with the help of vector to determine the angle in the process of gesture change. Establish the following formula:θ=arctanyt+1−yt/xt+1−xt. Different angle coordinates in the process of gesture change can be brought into it, and more detailed results and values can be obtained through calculation, so as to capture dynamic gesture features under the spatiotemporal trajectory. At the same time, there may be interference factors in the picture entered into the virtual reality human-computer interaction mode. The purpose of determining the position through angle calculation is to screen out such interference data and realize more accurate multifeature gesture recognition and tracking.


## 2.3.4. HMM Dynamic Gesture Track Recognition

Trajectory recognition is an important basis of gesture authentication. When the gesture to be recognized is in the dynamic mode, the trajectory of the whole movement process needs to be described in detail in the virtual reality human-computer interaction system. The trajectory description can accurately reflect the motion process of the gesture and refine the edge part [14]. HMM technology is to establish a statistical model to determine the trajectory through the various coordinate points and then through the establishment of a subset to restore the trajectory to the computer virtual system, showing the impact of the final grayscale processing. The specific program structure of trajectory recognition realized by HMM technology is shown in Figure 5. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect. The edge of dynamic gesture is more blurred than that of static gesture, so that the simulation effect can be kept similar.Figure 5
Composition of the HMM trajectory identification system. During recognition, relevant sequences will be established, and the edge area of gesture action will be formed through random generation inside the system to achieve the best simulation effect.


## 3. Conclusions

Virtual reality technology is a kind of technology that can create and experience the virtual world of computer technology, which uses computer to generate a simulation environment, and it is a kind of multisource information fusion of interactive 3D dynamic visual and physical behavior of the system simulation that can use sensing helmet, data glove, and other specialized equipment and allow the user to enter a virtual space, real-time perception, and operation of various objects in the virtual world. Thus, it is through the vision, touch, hearing, and so on to obtain the real experience of immersive. Gesture recognition requires the combination of static gesture feature information recognition and gesture feature fusion, as well as the combination of spatial-temporal trajectory features and dynamic gesture track features, to achieve a better human-computer interaction experience. Human-computer interaction multifeature gesture recognition is an important direction of virtual reality simulation technology. It is the integration of simulation technology and computer graphics, human-computer interface technology, multimedia technology, sensing technology, and network technology, and it is a challenging cross technology.



---

*Source: 3603693-2021-08-20.xml*