# Content-Adaptive and Attention-Based Network for Hand Gesture Recognition
## 4 Experiment and Results
### Results and Discussion  
Learning Curves. Learning curves are widely used tools for deep learning to diagnose the learning and generalization behavior of models. The accuracy and loss curves of our model during the training and validation phases are illustrated in Figure 5, where the x-axis is the time axis, the primary y-axis(left) is the value of accuracy and the secondary y-axis(right) is the loss value. As can be seen from Figure 5, after 15 epochs, the plot of validation loss decreases to a stable point and has a small gap with the training loss, which indicates that our model acquires a good fit. We observed that the gap between the validation and training loss on the NVGesture dataset was different from that on the EgoGesture dataset. In other words, our model acquires a better fit on the EgoGesture dataset than on the NVGesture dataset. The learning curve of a good fit is usually influenced by the size of the training dataset and is also related to the complexity of the model. We analyzed that the gap between the training loss and validation loss in Figure 5a is caused by the small number of samples in the training set. Accuracy Comparison with State-of-the-Art Methods. We compared our results with the current state-of-the-art methods on NVGesture and EgoGesture datasets, and the results are summarized in Table 3. Both datasets provide corresponding color and depth videos, and in this work we evaluated the proposed model using only color videos. All these methods to be compared can be roughly divided into three categories. The first class is 2D CNN-based methods, such as TSN [14], which require pre-computation to obtain optical flow features. The second category is 3D CNN-based methods, such as R3D [42], ResNeXt [2] and C3D [16], which uses 3D convolution to learn the spatial-temporal structure of the video. The last class is the current popular 2D+1D CNN-based methods, including I3D [25] and TSM [17]. This method decouples the 3D convolution kernel to reduce the parameters of the kernel, but the spatial-temporal modeling capability of this method is limited. Unlike these three types of methods, our proposed approach is based on the attention mechanism and does not use any convolution or recurrent operations. Table 3 shows the comparison results of recognition accuracy on two datasets. The number of the sampling frames was set to 16 for the input of all comparison methods and the same modality (color) of the test set was used. Our method achieved recognition accuracies of 83.2% and 93.8% on NVGesture and EgoGesture datasets, respectively. From Table 3, it can be seen that the proposed approach achieves the best recognition performance  
Figure 5: Accuracy and loss curves of the model during training and validation phases: (**a**) NVGesture dataset; (**b**) EgoGesture dataset. compared to other methods. To measure the influence of our proposed adaptive sampler on the performance of the model, we conducted ablation experiments on two datasets. As shown in the last two rows of Table 3, our model achieves 80.7% and 92.6% accuracy on NVGesture and EgoGesture datasets, respectively, when using only the attention mechanism. After improving the sampling using the proposed sampler, the accuracy of the model was improved by 2.5% and 1.2%, respectively. This illustrates the effectiveness of our proposed sampler in improving the accuracy of model recognition. F1-score. The F1 score is a common metric for measuring the precision of gesture recognition models. The F1-score can be defined as the harmonic mean of the precision and recall. We counted the recall, precision and F1 score of our proposed model on both NVGesture and EgoGesture datasets, and the results are summarized in Table 4. As can be seen from Table 4, our proposed attention-based model achieves F1 score of 82.98% and 93.10% on the NVGesture and EgoGesture datasets, respectively. Model Complexity. When evaluating a deep model for gesture recognition, the complexity of the model should be considered in addition to performance metrics.