# Novel Hand Gesture Alert System
## 3 Proposed Model
### Detector and Classifier  
Over the past few years, CNN-based models have shown impressive results when they are performing gesture and action recognition tasks. CNN 3D architectures are distinguished mainly by  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layers** & **Filters** & **Stride** & **Output** \\ \hline Conv & 32 & \(3\times 3\) & \(224\times 224\) \\ Maxpool & & \(\frac{1}{2}\times 2\times 2\) & \(112\times 112\) \\ Conv & 64 & \(3\times 3\) & \(112\times 112\) \\ Maxpool & & \(\frac{1}{2}\times 2\times 2  
video analysis because they use temporal relationships between images. A new frame is described in the following section, whose goal is the detection and recognition of a specific hand gesture that will trigger the alarm. Detector: Since we have no limitations regarding the size of the model, another architecture, with excellent classification performance, can be selected by a classifier. This leads us to use two recent 3D CNN architectures [43], the previously described YOLOv3 as the detector and a custom 3D CNN with a novelty, the introduction of \"counterGesture\", which is activated once a gesture is classified as belonging to our specific set. Given the context of this paper, the detector can be described as a tool responsible for processing sequential frames (video) and activating the classifier if there is a potential gesture in the video. It is worth mentioning that the human detector module mentioned above will have as its output a collection of hand images describing a gesture and associated to each subject. An algorithm (Algorithm 2) is given below. ``` Input:\(V_{c}\) Output: Specific Gesture
1 For j = 1 to m, do
2 For each \"frame window\" \(G_{j}\), do
3 Process a batch of hand images
4 classifierjActivated \(\gets True\)
5 alpha \(\gets Probability_{(j-1)}\times(i-1)\)
6 meanProbability = \(\big{(}alpha+G_{j}\times probability_{j}\big{)}/i\)
7 \((max_{1},max_{2})=



CounterGesture: the system comes with a novel module, which describes the counterGesture as a module responsible for counting the occurrences of gestures similar to the predefined set of gestures (Figure 5). As mentioned in the description of Figure 6, the counterGesture comes with two operators in its architecture: the first one sets the flag to 1, and the listener 1 starts the timer; the second operator is responsible for checking the time spent already and the value contained in the incrementor, and for triggering the alert if the condition below is respected. \[\text{getTime}<\text{n}\text{ and Counter }==3 \tag{5}\]

where n represents a predefined maximum duration allowed to collect the three gestures. Figure 5: A video is submitted to a module called the human detector, which extracts images containing a human. These frames are processed using scrolling sliding windows in which the detection queue is placed at the very beginning of the classifier assignment queue. If the detector recognizes an action/gesture, the classifier is activated via the Post Processing Service (PPS) [44] and, if it corresponds to one of the gestures contained in our specific set, the alert is triggered. ## 4 Experiments

The overall system can be divided into three parts: first, the extraction of the ROI as an output of the human detection module; second, the image classification, which categorizes all of the hand frames into one possible gesture; and third, a match that will trigger the alert. During the training part, we trained these three components separately. Furthermore, the counterGesture counts only the number of correspondences between the predicted gesture and an element of the predefined set of gestures. The latter is shown in Figure 6.

The EgoGesture dataset is a new multimodal dataset for the egocentric recognition of hand gestures [45], and it was created not only for the detection of gestures in continuous data but also for the classification of segmented gestures. This dataset contains eightty-three classes of dynamic and static gestures collected from six outdoor and indoor scenes.