# Dynamic Graph Attention Network for Skeleton-Based Action Recognition
## 3 Method
### DGAN Module  
To address the limitations of GCN- and Transformer-based approaches, we propose DGAN, a Dynamic Graph Attention Network, which integrates local and global feature modeling while adapting to the dynamic spatiotemporal structures of skeleton data. The GCN captures local topology but struggles with non-adjacent interactions due to its reliance on a predefined graph. Transformer models long-range dependencies but are prone to noise from global self-attention. DGAN balances these strengths, dynamically adjusting to skeleton data's evolving structure. As shown in Algorithm 1, DGAN initializes its adjacency information using the basic skeletal topology, ensuring that connections between neighboring joints are prioritized, thereby preserving the continuity of local structures. Concurrently, DGAN employs attention mechanisms to dynamically determine the correlations between nodes within a specific action scenario, adaptively adding or suppressing node connections accordingly. Formally, for a given graph \(G=(V,E)\), where \(V\) represents nodes and \(E\) represents all edges, the adjacency matrix \(A\in\mathbb{R}^{|V|\times|V|}\) characterizes neighbor relations. The node attention scores are computed as \(Attention(G)\in\mathbb{R}^{|V|\times|V|}\), from which each node identifies its most correlated distant neighbors. For each node, the minimum attention score among its adjacent neighbors is defined as a threshold, forming a threshold matrix \(V_{min}\in\mathbb{R}^{|V|\times 1}\). For each node \(v\in V\), if the attention score with node \(v\) exceeds the threshold \(V_{min}\)  
Figure 2: Model architecture of DGAN. The DGAN model is composed of \(L\) stacked DGAN modules and \(L\) stacked MSTC modules, followed by a final classification module. the influence of node \(v\) is preserved; otherwise, it is masked, treating it as a negligible contextual connection for the current node. ```
0:\(A\): Initial adjacency matrix, shape \((25,25)\)

sim: Similarity score matrix, shape \((N,25,25)\)
0: Modified similarity matrix sim, shape \((N,25,25)\)
1:functionSoff_A(\(A\), sim)
2:\(A\gets A\).expand(sim.shape\([0],-1,-1\))\(\triangleright\) Expand to batch size \(N\)
3:\(\text{inf}\leftarrow\text{max\_value}(\text{sim.dtype})\)
4:\(\text{masked\_C}\leftarrow\text{where}(A==1,\text{sim},\text{inf})\)\(\triangleright\) Non-adjacent nodes \(\rightarrow\infty\)
5:\(\text{min\_values}\leftarrow\text{masked\_C}.\text{min}(\text{dim}=2)\)
6:\(\text{min\_expanded}\leftarrow\text{min\_values}.\text{unsqueeze}(-1)\)\(\triangleright\) Min neighbor similarity
7:\(\text{sim\_ge\_min}\leftarrow(\text{sim}\geq\text{min\_expanded})\)
8:\(\text{mask}\leftarrow\text{where}(\text{sim\_ge\_min}==0,-\infty,0)\)
9:\(\text{sim}\leftarrow\text{sim}+\text{mask}\)\(\triangleright\) Suppress weak connections
10:return sim
11:endfunction
```  
**Algorithm 1** Pytorch-like pseudocode of DGAN  
Based on this mechanism, the node attention values below the threshold are masked, preserving only the most discriminative interactions, thus dynamically forming adaptive connections. Specifically, nodes with attention scores greater than \(V_{min}\) are regarded as softly connected to the current node, thereby mitigating irrelevant noise and redundancy commonly encountered in the global attention mechanism of Transformer models. DGAN adjusts the attention mechanism matrix as follows:  
\[\text{MaskedAttention}(Q,K,V)=\text{soft}\max\left(M+\frac{QK^{\top}}{\sqrt{d_ {k}}}\right) \tag{1}\]  
\(M\) denotes the attention mask, whose value at each feature position \((x,y)\) is defined as:  
\[\mathcal{M}(x,y)= \begin{cases}-\infty&\text{if }QK^{\top}_{(x,y)}<V_{min(x,0)}, \\ 0&\text{otherwise}.\end{cases}  \tag{2}\]  
Here, \(Q\in\mathbb{R}^{|V|\times d_{k}}\), \(K\in\mathbb{R}^{|V|\times d_{k}}\), and \(V\in\mathbb{R}^{|V|\times d_{k}}\) are obtained by projecting the initial features \(X\in\mathbb{R}^{|V|\times d}\) through projection matrices \(W_{q},W_{k},W_{v}\), and \(d_{k}\) denotes the scaling factor.