## 3 Proposed Method
### Hand Gesture Recognition  
We exploit 3 DCNN to recognize hand gestures by using the better capturing of the spatio-temporal information in videos. Compared with the two-dimensional convolutional neural networks (2 DCNNs), the 3 DCNN model has one advantage, that it captures motion information by applying the convolution operation in both time and space. The inputs are full frames of videos and do not rely on any pre-processing, so the model can easily be extended to large datasets. The details of the 3 DCNN architecture are shown in Figure 7.  
The network has six convolutional layers (Conv1, Conv2, Conv3, Conv4, Conv5, and Conv6), three max-pooling layers (pool1, pool2, and pool3), and two fully connected layers (fc1, fc2) before softmax output. The fully connected layer has 512 nodes with 50% dropout. Each input is a human action video. The input dimensions are \(d\times h\times w\), where \(d\) is the length of the normalized video inputs, and \(h\) and \(w\) are the height and width of the frame, respectively. Because the lengths of the frame numbers in the videos of the dataset are different, so as to normalize the temporal lengths of the videos, we down-sampled to d frames per video. In our architecture, \(d=20\) is the optimal number, and matches the capabilities of the computer. The 20 frames are chosen to spread from the beginning to end of videos using the following formula.  
\[d_{i}=\left[\frac{i*N}{20}\right],\;\;i=1,2,\ldots,20, \tag{2}\]  
where \(d_{i}\in d\) is the \(i\)th frame chosen from the \(N\) frames of original video, for real \(z\); \(\{z\}\) is a floor function [53], which returns the greatest integer less than or equal to \(z\); and \(N\) is the number of original input frames. We also re-sampled d frames to \(32\times 32\) (\(w\times h\)) per video.  
As shown in Figure 7, the proposed network has six layers of 3D convolutions, with each pair of convolutions followed by a max-pooling, and two fully connected layers. The filter numbers of Conv1, Conv2, Conv3, Conv4, Conv5, and Conv6 are 32, 32, 64, 64, 128, and 128, respectively. In each convolution layers, the size of the kernels is \(3\times 3\times 3\), where the first number 3 is in the temporal dimension and the remaining \(3\times 3\) is in the spatial dimension. The pool1, pool2, and pool3 are max-pooling following each pair of convolutions, and the size of fc1 and fc2 is 512 nodes. In particular, we add a dropout layer between the convolution layer and the max-pooling layer so as to eliminate overfitting. The output layer activation is implemented with softmax.  
In order to improve the accuracy of the model, we use an ensemble model of 3 DCNN that predicts the class labels based on the predicted probabilities for classifiers, as shown in Figure 8. 3 DCNN_1, 3 DCNN_2, and 3 DCNN_n are models 1, 2, and n in the n-model ensemble, respectively. Each model  
Figure 7: Block diagram of the three-dimensional convolutional neural network (3 DCNN).  
has a different weight volume. In order to aggregate the results of each individual 3 DCNN, we use the ensemble output to obtain a final probability. The optimal weighting of the final softmax, the maximum of averaging probability, is the output of the final prediction. The formula of the 3 DCNN Ensemble procedure is as follows:  
\[\hat{y}=\operatorname*{argmax}_{j}\left(\frac{1}{n}\sum_{i=1}^{n}p_{ij}\right). \tag{3}\]  
In Equation (3), \(p_{ij}\) is the predicted probabilities \(p\) of label \(j\)th of the \(i\)th classifier, \(n\) is the number of models used for the learning ensemble, and \(\hat{y}\) is the final prediction. The outputs of all single models in Figure 8 are combined to form the final prediction by optimal.