# A Real-Time Dynamic Gesture Variability Recognition Method Based on Convolutional Neural Networks
###### Abstract
Modern research results make it clear that machine learning methods [8, 9, 10, 11, 12, 13, 14, 15, 16] based on deep neural networks [8, 9, 10, 11, 12, 13, 14, 15], in comparison with traditional classical approaches [7, 16, 17], which are based on linear classifiers (e.g., the support vector method (SVM) or hidden Markov classification (HMC)), can demonstrate good results in segmentation, classification, and recognition as static and dynamic sign language (SL) elements. Thus, with the help of two-threaded convolutional neural networks [2,9, 10], it is possible to extract the spatiotemporal features of a gesture from full-color images (RGB format) and 3D frames (depth map) of video streams separately. In turn, a deep neural network (DNN) is used to perform segmentation and classification of the hand shape with multiple architectures and sizes of the input images [4, 11]. In addition, it was revealed that the architecture of the long short-term memory (LSTM) neural networks [1,3, 12, 13, 15, 18, 19, 20], with the help of long and short-term memory, can extract the spatiotemporal characteristics of a gesture from sequences of previously annotated 2D regions with a gesture. These methods considered extract spatial and temporal information at different stages or separately. Thus, if there is a complex dynamic background component on the scene, concurrently extracting both the spatial and temporal components of a gesture will be an effective solution. [PERSON] et al. [8] used a DNN to recognize hand gestures in real-time for seeking help, achieving an accuracy of 98.79%. [PERSON] and others [9] presented an American Sign Language translator based on convolutional neural networks (CNN) that consistently classify the letters a-e correctly and a model that correctly classifies the letters a-k in most cases. The network proposed by [PERSON] [4] and others eliminates the need for the preprocessing steps for the Massey University (MU) color image dataset by using a DNN for segmentation and classification of the hand shape. The system was tested in real-time on a community of 10 people, with an average accuracy of 81.74% and processing speed of 61.35 frames per second. Reference [10] proposes a comprehensive end-to-end system that uses hand gestures with sign language digits for contactless user authentication, achieving a 99.1% accuracy on the test dataset with a model output rate of 280 ms per image frame. Finally, study [11] presents a machine learning-based system that recognizes Turkish sign language in real-time with an accuracy of 98.97% using a cascade voting approach with five single classifiers. These studies demonstrate the potential of deep learning models for improving people's safety, simplifying processes, and creating contactless solutions for authentication and communication. Several research studies have proposed innovative systems for recognizing SL gestures in real time. One such system [12] uses electromyography (EMG) signals and a neural network with long-term and short-term memory to recognize hand movements in ASL. Another system [13] employs bidirectional long short-term memory (BLSTM) and multifunctional structures to recognize dynamic gestures in Hong Kong Sign Language (HKSL) using data collected from smartwatches. [PERSON], [PERSON] and et al. [14] developed a framework called SignBERT, which uses bidirectional encoder representations from transformers (BERT) based on a deep learning models and multimodal inputs to recognize dynamic gestures for continuous sign language recognition (CSLR). This system was tested on several datasets, including a new set of HKSL data, showing advanced results in recognition accuracy. Additionally, an experimental system was developed to support deaf people in the office when applying for an identity card using Polish Sign Language (PSL) expressions. The system [15] uses a Kinect sensor and a feature vector inspired by linguistic research. It was tested using three commonly used methods for dynamic gesture recognition.