# Dynamic Gesture Recognition Based on MEMP Network
## 3 Proposed Method  
In this section, we will describe the internal structure of the MEMP Neural Networks in detail. Predicted results were obtained after the video frame sets operated by three consecutive 3 DCNN operations and ConvLSTM operations. CNN plays a significant role in image feature extraction [22]. Here F = \(\{f_{1},f_{2},.....f_{16}\}\) represents the frame set of video. The size of \(f_{i}\) is 64 \(\times\) 64. After a 3D CNN operation, the size of each feature map was not changed, and the number of channels increased to 8. This operation can extract spatial-temporal information of feature maps. Then, after a convLSTM operation, the set became F\({}^{\prime}=\left\{f^{\prime}_{1,1},f^{\prime}_{1,2},.....f^{\prime}_{16,16}\right\}\). The number of channels increased to 16, the number of each channel was 16 and the size of feature map is 64 \(\times\) 64. Then, through a 3D POOL operation, the number of feature maps in each channel and the length and width of each feature map were reduced by half. After similar operations of 3 DCNN, convLSTM and POOL, the size of feature set became F\({}^{\prime\prime}=\left\{f^{\prime\prime}_{1,1},f^{\prime\prime}_{1,2},.....f^{ \prime\prime}_{64,4}\right\}\), with 64 channels. The number of each channel was 4, and the size of feature map was 16 \(\times\) 16. After above operations, the spatial-temporal feature information of the frame set will be retained. At the end of the full connection layer, the results will be classified to obtain different types of gestures. The specific operation is as follows. 3D CNN is good at extracting features of video [23]. So in MEMP Neural Networks, 3D CNN is used to process spatial and temporal features of feature graph sets. As shown in Figure 2, there are three 3D convolution operations (C1, C2, C3), three convLSTM operations (CL1, CL2, CL3) and two pooling operations (P1, P2) in the network. The size of the three 3D convolution kernels is 2 \(\times\) 2 \(\times\) 2, and the step length is 1 \(\times\) 1 \(\times\) 1. Since the convolution mode of 'SAME' is adopted, the size of each feature map in the convolution operation will not change. The filter sizes corresponding to the three convolution operations are 8, 32 and 64 respectively. The pooled method is'max pooling' and the pooled size is 2 \(\times\) 2 \(\times\) 2. After each cisterization operation, a 'droupout' layer will be followed. 'droupout' will effectively reduce the occurrence of over-fitting, which can reach the effect of regularization to a certain extent [24]. The formula of 3D CNN is as follows:  
\[v_{ij}^{xyz}=ReLU(b_{ij}+\sum_{m}\sum_{p=0}^{P_{i}-1}\sum_{q=0}^{Q_{i}-1}\sum_{ r=0}^{R_{i}-1}w_{ijm}^{pqr}v_{(i-1)m}^{(x+p)(y+q)(z+r)}) \tag{1}\]  
'\(ReLU\)' is the activation function of the hidden layer. \(v_{ij}^{xyz}\) represents the current value of the coordinates (x, y, z) in the \(i\)-th and \(j\)-th feature graph sets. \(b_{ij}\) represents the bias of the \(i\)-th layer and the \(j\)-th feature graph set, \(w_{ijm}^{pqr}\) represents the weight of the \(m\)-th filter connected by the position (p, q, r) in the \(i\)-th layer and \(j\)-th feature graph set. \(P_{i}\), \(Q_{i}\) and \(R_{i}\) represent the height, width and depth of the convolution kernel respectively [23]. Figure 2: In Figure 2, the MEMP Neural Networks structure as well as the change of the feature graph in each layer of the network structure are shown. The whole process is divided into three parts, video processing, (3D CNN-convLSTM)*3 and FC. Initially, each video is split into 16 consecutive frames. Secondly, these feature sets need to be performed three convolution operations, three convLSTM operations and two pooling operations to obtain more spatial-temporal information. Finally, the network connects the full connection layer to classify these results. Compared with traditional LSTM, convLSTM can better extract spatial and temporal features of feature graph sets [10].