# GCN-Former: A Method for Action Recognition Using Graph Convolutional Networks and Transformer
## 2 Methods
### Transformer Block Module  
The Transformer Block consists of N Encoder modules, each containing Multi-Head Attention, Add & Norm, Feed Forward, and another Add & Norm (as shown in Figure 3). Notably, the Multi-Head Attention here contains multiple Convolutional Attention [30] layers. Specifically, the input X is passed through h different Convolutional Attention layers, resulting in h output matrices X\({}^{\prime}\). These matrices are then concatenated (Concat) and passed through a Linear layer to form the final output matrix Y of the Multi-Head Attention. Convolutional Attention replaces the matrix multiplication in traditional self-attention mechanisms with pixel-wise convolution operations, avoiding the need for feature flattening and reshaping. It not only simplifies the computational process but also reduces inference time, which is crucial for real-time applications such as video segmentation and action recognition. Unlike traditional self-attention mechanisms, Convolutional Attention uses convolution operations to compute similarity, preserving the spatial coherence of the input data. This is particularly important for data with inherent spatial layouts, such as images and videos. By extending learnable vectors into learnable kernels, Convolutional Attention can more precisely capture local context and retain more local spatial information. This approach helps better align the semantic information of the Transformer, thereby improving the model's expressiveness. Convolutional Attention is an innovative attention mechanism. Its core idea is to use convolution operations to simulate the similarity calculation process in self-attention mechanisms, while preserving the spatial structure of the input data. Unlike traditional methods that calculate the dot product between queries and keys, Convolutional Attention uses convolution kernels to compare the similarity between pixel blocks. This approach better captures local features and naturally incorporates positional information through the convolution operation. In addition, Convolutional Attention introduces a spe  
Figure 3: Structure of the Transformer Block (**a**) and details of the Convolutional Attention (**b**). GDN denotes Group Dual Normalization. \(\otimes\) represents the convolution operation, \(\oplus\) indicates addition, and k denotes the kernel size. cial normalization method--Grouped Dual Normalization (\(\theta\))--which ensure the effective allocation of attention weights. Specifically, given an input feature map XXX and a set of learnable queries (Q) and keys (K), the Convolutional Attention operation consists of two main steps: calculating the similarity matrix and aggregating the values. The formula for calculating the similarity matrix is as follows:  
\[\text{A}=\theta(\textbf{X}\otimes\textbf{K}). \tag{4}\]  
Here, A is the computed similarity matrix, which captures the local similarity between the input feature map X and the keys K. \(\text{X}\in\mathbb{R}^{\text{C}\times\text{H}\times\text{W}}\) is the input feature map, where C denotes the number of channels, and H and W represent the height and width, respectively. \(\text{K}\in\mathbb{R}^{\text{C}\times\text{N}\times\text{k}\times\text{k}}\) is the learnable query and key values, where N is the number of kernels, and k is the kernel size. \(\theta\) represents Grouped Dual Normalization (GDN), which includes Softmax normalization along the H \(\times\) W dimensions and group L2 norm normalization along the N dimension. \(\otimes\) denotes the convolution operation. First, the similarity between the input feature map X and the query/key values K is computed through the convolution operation. The resulting similarity matrix A reflects the correlation between different positions and is normalized using \(\theta\) to ensure the effective allocation of attention weights.