## 3 Methodology
### Spatial-Temporal Graph Convolution Network  
The Spatial-Temporal Graph Convolution Network (ST-GCN) consists of two key components: the graph convolutional network (GCN) and the temporal convolutional network (TCN). The GCN extracts feature information from each node and edge in each frame of the two-dimensional joint sequence, while the TCN captures temporal feature information, tracking the evolution of joint positions over time. ST-GCN introduces 2D convolution to image data and refines the graph convolution method, enabling effective extraction of spatial relationships between nodes in the skeletal structure. In 2D convolution, pixel adjacency is fixed, but in graph convolution, adjacency is determined by the adjacency matrix. Therefore, the design of the adjacency matrix and receptive field in ST-GCN's graph convolution are closely related. In 2D convolution, a \(3\times 3\) kernel covers a grid of nine pixels in an image, and by analogy, a kernel with a stride of 1 in image convolution corresponds to a joint and all its directly adjacent joints in graph convolution.  
Assuming the duration of the action is \(T\), the number of human joints is \(N\), and the dimension of each node's feature vector is \(C\), then \(H\) can be expressed as a \(C\times T\times N\) tensor, so we can find \(H\in R^{C\times T\times N}\). Using the ReLU activation function, the graph convolution response \(H_{s}^{l+1}\) is obtained. When considering a time span of \(\tau\) for each operation, since the feature graph \(H\) is represented as \(H\in R^{C\times T\times N}\), a \(\tau\times 1\) convolution kernel can be applied, similar to convolutional neural networks for images. This kernel weights the sum of corresponding node features across the adjacent \(\tau\) frames to obtain the node's response, taking into account temporal relationships.  
However, GCN utilizes predefined skeletal information as the adjacency matrix, which is prior knowledge based on human anatomy and the physiological structure of the human body. Although directly applying a fixed prior adjacency matrix can yield reasonable results in action recognition tasks, there are two main limitations: lack of flexibility and difficulty in capturing deep relationships. In addition, the TCN is applied separately from the GCN, resulting in insufficient consideration of the interdependencies between spatial and temporal features. And the size of the temporal convolution kernel corresponds to a relatively long time window, which may be effective for capturing long-term dependencies. Nevertheless, for complex actions or subtle movements, a multi-scale window works better.  
To address the aforementioned issues, we developed an approach that incorporates an adaptive adjacency matrix for the GCN. In addition to the prior matrix based on human skeletal structure, we introduced a fully parameterized adjacency matrix. This allows for a more flexible representation of the relationships between joints and enables the model to adjust the weights between different connections, thereby enhancing its sensitivity to the variations among different types of actions. Furthermore, the grouping is no longer limited to the three groups manually set in ST-GCN. By increasing the number of groups, the model can capture more complex spatial structures. For TCN, we introduced multi-scale temporal windows to fill the gap left by ST-GCN in capturing local temporal features. This enables the model to focus on the most critical local spatio-temporal features for each action category, while preserving global features to enhance the overall representational power of the model.