# GCN-Former: A Method for Action Recognition Using Graph Convolutional Networks and Transformer
## 2 Methods
### Contextual Temporal Attention  
The Contextual Temporal Attention (CTA) module is designed to overcome the limitations of traditional self-attention mechanisms when processing 2D feature maps. Specifically, the attention matrix is calculated based solely on isolated query-key pairs, which ignores the rich contextual information between neighboring keys. This limitation hinders the ability of self-attention to learn effectively and reduces the efficiency of visual representation learning. The Contextual Transformer (CoT) [29] integrates contextual information mining with self-attention learning into a unified architecture, enhancing the representational power of the output aggregated feature map. However, this approach only enhances the representation through local neighborhood relationships and the combination of static and dynamic information. To further enhance the representation of contextual information, we propose adding an independent temporal feature enhancement path to the existing CoT module. This path extracts the temporal dynamics of the input features by applying simple temporal convolutions and combines them with the original features. This approach improves the model's sensitivity to temporal information without significantly increasing computational overhead. The detailed structure of the proposed CTA module is shown in Figure 2. For the input feature map \(\text{X}\in\mathbb{R}^{\text{T}\times\text{H}\times\text{W}\times\mathbb{C}}\), we define the following: keys (K) are directly taken from the input feature map \(\text{X}\); queries (Q) are also directly taken from the input feature map \(\text{X}\); values (V) are obtained from the input feature map \(\text{X}\) via a linear transformation \(\text{W}_{\text{v}}\). Unlike traditional self-attention mechanisms that use \(1\times 1\times 1\) convolutions to encode each key, the CTA block utilizes \(1\times\text{k}\times\text{k}\) grouped convolutions to contextualize all local neighboring keys. This operation captures the local neighborhood relationships of each key within its \(\text{k}\times\text{k}\) grid, generating a contextualized key \(\text{K}^{1}\in\mathbb{R}^{\text{T}\times\text{H}\times\text{W}\times\mathbb{C}}\) that reflects the static contextual information between local neighboring keys. Next, the CTA block combines the contextualized key K1 and the query Q to generate the attention matrix A. Specifically, K1 and Q are concatenated and passed through two successive \(1\times 1\times 1\) convolution layers W\({}_{\theta}\) and W\({}_{\delta}\), where W\({}_{\theta}\) has a ReLU activation function, while W\({}_{\delta}\) does not. The formulation is as follows:  
\[A=\left[K^{1},Q\right]W_{\theta}W_{\delta}. \tag{1}\]  
Here, \(A\) represents the local attention matrix for each head, which is learned for each spatial position based on the query features and the contextualized key features, rather than relying on isolated query-key pairs. This approach enhances self-attention learning and introduces additional guidance from the mined static context \(K^{1}\). Based on the contextualized attention matrix \(A\), the CTA module computes the attention-refined feature map \(K^{2}\) by aggregating all the values \(V\). The formula is as follows:  
\[K^{2}=V\otimes A. \tag{2}\]  
Here, \(K^{2}\) captures the dynamic feature interactions between the inputs, and thus is referred to as the dynamic contextual representation of the input. Compared to the CoT module, the designed CTA module introduces a lightweight temporal convolution branch to extract temporal dynamic information. Specifically, this branch applies a \(k\times 1\times 1\) convolution operation along the temporal dimension of the input features to capture the dynamic variation patterns between different time steps. Finally, the output \(Y\) of the CTA module is obtained by adding the result of the temporal convolution to the fusion of the static context \(K^{1}\) and the dynamic context \(K^{2}\) via residual connections.