# STGAT: Spatial-Temporal Graph Attention Neural Network for Stock Prediction
## 3 System Framework
### Spatial-Temporal Graph Attention Neural Network
#### 3.3.1 GAT Module Based on Residual Mechanism  
The graph attention module utilizes attention mechanisms [53; 54] to assign varying levels of importance to nodes within the same neighborhood, significantly enhancing model capacity while improving interpretability [49]. In this work, we adopted an enhanced graph attention network (GAT) to optimize information propagation beyond the capabilities of the original GAT framework. Specifically, we implemented a multi-layer GAT architecture augmented with residual connections, which effectively aggregates stock price feature information from strongly correlated nodes. As illustrated in Figure 3, the model stacks two GAT blocks to extract preliminary graph relationship features, while an additional GAT layer serves as a residual component. The residual connection preserves the initial stock features, preventing feature degradation caused by the stacking of multiple graph attention layers. This design ensures that both high-level relational patterns and low-level intrinsic features are retained, enhancing the model's ability to capture complex market dynamics.  
In each GAT layer, \(h=\{\vec{h_{1}},\vec{h_{2}},\dots,\vec{h_{N}}\},\vec{h_{i}}\in R^{F}\), represents a set of node features, \(N\) represents the number of nodes, and \(F\) is the number of features in each node. The layer's output is a set of new features \(h^{\prime}=\{\vec{h_{1}^{\prime}},\vec{h_{2}^{\prime}},\dots,\vec{h_{N}^{ \prime}}\},\vec{h_{i}^{\prime}}\in R^{F^{\prime}}\); thus the GAT layer needs to first convert \(h\) into features with dimension \(R^{F^{\prime}}\) by applying linear transformation with a weight matrix \(W\in R^{F\times F^{\prime}}\) to enhance its expressive capacity. In Equation (10), \(||\) represents concatenation, and \(\alpha\) denotes normalized attention coefficients, as shown in Figure 4. The equation represents the importance of node \(j\)'s features to node \(i\):  
\[e_{ij}=\alpha(W\vec{h_{i}}||W\vec{h_{j}}), \tag{10}\]  
\[\alpha_{ij}=\frac{\exp\Bigl{(}\text{LeakyReLU}\left(\vec{\mathbf{a}}^{T}[ \mathbf{W}\vec{h_{i}}|\mathbf{W}\vec{h_{j}}]\right)\Bigr{)}}{\sum_{k\in\mathcal{ N}_{i}}\exp\Bigl{(}\text{LeakyReLU}\left(\vec{\mathbf{a}}^{T}[\mathbf{W}\vec{h_{i}}| \mathbf{W}\vec{h_{k}}]\right)\Bigr{)}}. \tag{11}\]  
As shown in Figure 5, to better enhance the model's ability to capture diverse patterns in the data, the multi-head mechanism [55; 56] extends the standard attention framework by employing multiple independent attention heads, each with its own set of parameters. In doing so, the model can simultaneously focus on different aspects of the stock data, leveraging distinct subspaces within the multi-head matrix to represent information from various perspectives.  
Specifically, each of the K-independent attention heads performs a transformation as defined in Equation (12). These features generated in each head are then concatenated to form a comprehensive output, which integrates the diverse attention information captured by each head. Combined with the normalized attention coefficients, this mechanism  
Figure 4: Graph attention mechanism.  
Figure 3: GAT module’s framework.  
effectively avoids information redundancy, enabling the model to efficiently learn the relationships among critical nodes in the graph.  
\[\bar{h}_{i}^{\prime}=\Big{\|}_{k=1}^{K}\sigma\left(\sum_{j\in\mathcal{N}_{i}} \kappa_{ij}^{k}\mathbf{W}^{k}\bar{h}_{j}\right). \tag{12}\]  
Through the multi-head attention mechanism, node \(i\) can effectively aggregate the features of its neighboring nodes, enriching the local feature representation and capturing the overall graph structure more comprehensively. This approach not only enhances the model's ability to learn complex graph structures but also improves its robustness and generalization performance. The output feature generated by the GAT module includes both the high-level relationships between stocks and the subtle variations in stock correlations, providing a more refined understanding of the market's dynamics.