# Chinese Traffic Police Gesture Recognition Based on Graph Convolutional Network in Natural Scene
## 3 Methods
### Temporal Graph Convolution  
ST-GCN methods only connect the same joints in different frames in the temporal dimension, which can only extract the trace relationship of the same joints in neighbor frames. However, gestures used by traffic police consist of a set of different actions, so simply extracting the relative action features in neighbor frames is not enough to describe the semantics of whole gestures. Here, we connect the same joints in several frames to a joint in one frame to better extract rich traffic police gesture features, as shown in Figure 10. The left of Figure 10 is the temporal graph of the ST-GCN; the right of Figure 10 is the extended skeleton graph proposed by this paper. To better explain, the right of  
Figure 8: Channel attention module.  
Figure 9: Spatial attention module.  
Figure 10 only illustrates the connection situation of the adjacent five frames; all the joints are connected similarly in real operation.  
Here, we designed a TAS module; its structure is shown in Figure 11. It was used to extract traffic police gesture features in the temporal dimension. The TAS studies the dynamic features of each joint through every frame separately in the temporal dimension. Every joint is recognized as independent from one another; the TAS collects the relevance between frames by comparing the dynamic change of the same joint in the temporal dimension and mines the features ignored in the traditional ST-GCN model.  
This paper borrows the ideas used in the temporal convolution network (TCN) [35] and applies its mode of processing spatial features to the temporal domain. A TAS module can capture the dependency relationship in temporal sequence. It can extract partial features in temporal sequence by forming a sequence of feature vectors with feature vectors captured in each time frame. The convolution calculation is layerwise; it calculates the frames from every moment at the same time. From Equation (3), we can see that the selection of the convolution domain relies on the joints' distribution in space, and division is based on the distance of joints; convolution domain is determined by the distance between joints.  
Based on these considerations, we directionally transposed the input data and exchanged the feature data in the spatial dimension and temporal dimension, which transformed \((N,M,C,T,V)\) into \((N,M,C,V,T)\). \(N\) is batch_size, and \(C\) is the number of channels, taking \(C=3\). T is the figure for frames, and \(V\) is the number of skeleton joints. M is the amount of people that appeared in each frame of the video, which was \(M=1\) here because there was only one traffic police member in the video of traffic police on duty. In sequence data frames, we used the downscaling operation through convolution summation, which achieved temporal graph convolution, that is  
\[M_{T}(F)_{NCTW}=\sum_{K}\sum_{V}F_{NKCTV}\cdot A_{KVW} \tag{10}\]  
\(N,K,C,T,V,W\) represent Batch_Size, KernelSize, channels, the number of frames, the number of skeleton joints, and the number of people, respectively.  
Figure 11: Temporal attention module.  
Figure 10: The proposed skeleton graph on temporal dimension.  
Finally, the batch normal operation and dropout operation were performed on the features extracted, the amount of midfeature and data redundancy were reduced, the process of training was sped up, and the performance was improved. In the temporal dimension, the characteristics of the connection between skeleton joints are listed below:  
* Unlike the connection of skeleton joints in a frame, the connection of skeleton joints in the temporal sequence itself is used as a parameter to train. Furthermore, it can self-learn through a fully connected matrix and learnable weight hyperparameters.
* The size of the convolution kernel determines the connection effect between frames and enriches the contact between frames. It can be regarded as a dynamic recognition process from a certain point of view.