# Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model
## 3 Material and Methods
### Proposed Method  
To understand the method which we are proposing, we need to look at the diagram presented in Figure 3 to understand better, generally, what and how our application is detecting objects. It is a kind of a general overview of the application that we have developed. The training process first requires collecting the dataset, and after that the next step is to label it, so we use YOLO annotation to label our data, which gives us some values that are later explained in the model process. After that, when the data is labeled, we then feed it to the DarkNet-53 model, which is trained according to our defined configuration. The image is captured through the camera that can be an integrated (primary) camera or it can be any external (secondary) camera. Other than that, the application can also detect gestures from a video input as well.  
Figure 3: The bounding box highlighting positions of the values for further classification.  
Figure 2: Heat map of the YOLO-labeled dataset.  
After capturing real-time objects with the help of the OpenCV [45] module, which is an open-source computer vision library, we can capture images and after that, we send them frame by frame from the real-time objects. Because of incorrect filtering, our dataset of collected images currently has a variable number of images. These images are labeled according to the classes we have stored for our YOLO algorithm, so we have successfully attained the coordinate and the class for our image set. After that, we can now set towards the training section. We then pass this set to our training algorithm, which is a deep neural network model YOLO.  
Further, we discuss the methodology how YOLO deals with the network desired output which is achieved by using a formula which takes different co-ordinates, such as \(pw\), \(ph\), \(tx\), \(ty\), \(tw\), \(th\), \(cx\), and \(cy\). These are the variables which we use for the bounding box dimensions. Obtaining the values of the boundary box (x-axis, y-axis, height, and width) is described by Equation (1).  
\[\begin{array}{l}b_{x}=\alpha\left(t_{x}\right)+Cx\\ b_{y}=\alpha\left(t_{y}\right)+Cy\\ b_{w}=p_{w}e^{t}w\\ b_{h}=phe^{t}h\end{array} \tag{1}\]  
where \(bx\), \(by\), \(bw\), and \(bh\) are the box prediction components, \(x\) and y refer to the center coordinates, and \(w\) and \(h\) refer to the height and width of the bounding box. Equation (1) used in the YOLOv3 algorithm shows how it extracts the values from the image in the bounding box, and below is the diagram of how these values are extracted from the bounding box.  
From Figure 3, we can understand how each value of the bounding box from the algorithm provides us with the co-ordinates of the center, \(x\) and \(y\). From the prediction here the next important thing comes, which is the sigmoid function, which we have already discussed above, which filters out data except for the main part which is going to be recognized.  
From Figure 4, we can understand the backend of the algorithm. When we pass an input, it first comes into the input layer, and after that it is further rendered and passes into the hidden layers, that are several in number and size, and are interconnected convolutions. These convolutional layers determine a special value, which is the value of confidence. In our case, if the value of the confidence threshold is greater than 0.5, then we assume that the application has successfully determined what object it has encountered.  
Figure 4: Architecture of the neural network for hand gesture recognition.