# Novel Hand Gesture Alert System
## 3 Proposed Model
### Detector and Classifier
We organized the training set, validation and test set by separating topics with a 3:1:1 ratio, which gave respectively: 1239 elements, 411 validation videos, and 431 test videos, with 14,416, 4768, and 4977 gesture samples. All models were first pre-trained on the Jester dataset [45]. For test set evaluations, we used both the training and the validation games for the training. To perform our experiments, we used a specific device as described in the table (Table 3). We were able to extract the ROI (Figures 7 and 8) and classify the gesture (Figure 7). In addition, Figure 9 shows the accuracy of our classifier after 350 epochs. Figure 6: (**A**) Set of predefined images that the counterGesture considers. (**B**) In case the classifier identifies one of these gestures, the counterGesture will increment its counter up to 3 and trigger the alert, if there are at least three occurrences of the predefined gestures. \begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{2}{c}{**Station Characteristics**} \\ \hline CPU & Intel(R) Core \(\mathrm{{}^{TM}}\) i7-3.20 GHz \\ RAM & 4096 MB \(\times\) 2 \\ \hline \multicolumn{2}{c}{**Device 0: “GeForce GTX 1070”**} \\ \hline Components & Specifications \\ CUDA Driver Version/ Runtime Version & 10.1/10.1 \\ CUDA Capability Major/Minor Version Number & 6.1 \\ Total Amount of Global Memory & 8118 MB \\ (15) Multiprocessors (MP), (128) CUDA Cores/MP & 1920 CUDA Cores \\ GPU Max Clock Rate: & 1721 MHz \\ Memory Bus Width & 256 bits \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Characteristics of the testing environment containing a graphic card “GeForce GTX 1070”. Figure 8: Considering the simplified chart, from left to right, we see how our model extracts the region of interest, processes the cropped images and classifies the gesture using the classifier. Nevertheless, the counterGesture integrated into the classifier block will trigger the alert if the specific gesture is identified three times. Figure 7: YouTube video containing a sexual assault attempt [46]. (**a**) We can see the victim being assault by an individual. (**b**) Our system performs an analysis of the scene, extracts the zone of interest and returns the response (triggering the alert or not). ## 5 Results and Discussion

During the experiment, we firstly studied the performance of various versions of the neural network VGG-16 and a custom 3D CNN architecture on the classification task. In addition to this, we paid attention to the performance result of the number of input frames submitted to our gesture classification. Figure 9 provides an overview of how well our classifier performs, and the results in Table 4 show that we achieve a better performance by increasing the size of entries for all modalities. This depends strongly on the characteristics of the datasets used, especially the average duration of the gestures. Figure 9: Performance of the classifier, where (**a**) represent the loss graph, where our classifier shows a decrease in the loss during the training phase; (**b**) represent the validation graph, which shows the reduction of the loss and the increase in the confidence of our classifier during the validation phase; and (**c**) show the accuracy graph, which shows an increase in accuracy of up to 91% after 350 epochs. Considering the video used in our experiment, let us have a close look at the portion of the video where the sexual assault is happening. Table 5 shows how Algorithm 1 (human detector) is applied on the video to detect a human, and it also gives an idea of when Algorithm 2 (hand gesture) detects the gesture. The hand gesture will activate the CounterGesture every time a specific gesture is identified. However, it is worth mentioning that in the case of a total occlusion of the hands of the victim, it is not possible for the system to detect the gesture and hence trigger the alert. As future work, we will explore the possibility of adding other factors to the decision making. Also, as shown in Table 5, the alarm is triggered after the counterGesture = 3.