## 3 Experimental Results and Analysis
### Deep Learning Modeling Results
#### 3.4.2 Modeling Results of Deep Learning Models Based on Attention Mechanism  
To investigate the impact of different attention mechanisms on model performance, this study incorporated three attention mechanisms: self-attention (SA), squeeze-and-excitation (SE) attention, and local attention (LA). The benchmark models were based on three distinct types of deep learning architectures: convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and Transformers. (1) SA Attention Mechanism  
Self-attention (SA) is utilized to capture the dependencies among elements within a sequence [31]. Its core concept involves calculating the correlation weights between each element and all other elements in the sequence and dynamically assigning attention weights to each element, thereby integrating global information. The schematic diagram of SA is presented in Figure 7.  
Figure 6: Modeling results of deep learning model. (From left to right, they are R\({}^{2}\), RMSE, and MAE.) (a) Straight flow channel; (b) hexagonal flow channel. Figure 7: Schematic diagram of SA attention mechanism. First, each element of the input sequence is generated through three linear transformations:  
\[Q=XW^{Q} \tag{5}\]  
\[K=XW^{K} \tag{6}\]  
\[V=XW^{V} \tag{7}\]  
Among them, \(X\) is the input matrix, and \(Q\), \(K\), and \(V\) are the query vector, key vector, and value vector, respectively. \(W\emptyset\), \(W^{K}\), and \(W^{V}\) are the learnable parameter matrices. Then, calculate the dot product of \(Q\) and \(K\), and normalize it through Softmax after scaling:  
\[Attention(Q,K,V)=Softmax(\frac{QK}{\sqrt{d_{k}}})V \tag{8}\]  
Here, \(d_{k}\) represents the dimension of the key vector, which is used for scaling to prevent gradient explosion. Softmax normalizes the weights into a probability distribution. Then, the value vectors are summed up through weighted averaging:  
\[Z=Attention(Q,K,V) \tag{9}\]  
\(Z\) represents the output of the self-attention layer. (2) SE Attention Mechanism  
The squeeze-and-excitation (SE) attention mechanism is a module designed to enhance the feature representation capability of convolutional neural networks that comprises two key steps: squeeze and excitation [32]. Its core concept involves explicitly modeling inter-channel dependencies and dynamically recalibrating the weights of each channel, thereby amplifying important features while suppressing less relevant ones. A schematic illustration of the SE attention mechanism is presented in Figure 8.  
The squeeze operation performs global average pooling (GAP) on the input feature map \(X\) (with a shape of H \(\times\) W \(\times\) C), compressing the spatial information of each channel into a scalar:  
\[z_{c}=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}x_{c}(i,j) \tag{10}\]  
Here, \(x_{c}\)(_i,j_) is the feature value of the \(c\)th channel at position (i,j); \(z_{c}\) is the global feature descriptor of the \(c\)th channel; and \(z\) is the output vector with a shape of \(1\times 1\times C\). Then, the excitation operation learns the nonlinear relationships among channels through two fully connected layers (FC) to generate channel weights:  
Figure 6:0  
Here, \(W_{1}\) and \(W_{2}\) are the weight matrices of two fully connected layers, \(\delta\) is the ReLU activation function, \(\sigma\) is the Sigmoid activation function, the weights are normalized to \([0,1]\), and \(s\) is the output vector with a shape of \(1\times 1\times C\). Figure 6:1 Schematic diagram of SE attention mechanism. Finally, the feature recalibration operation multiplies the learned channel weights s with the original feature map \(X\), channel by channel, to obtain the enhanced feature map:  
Figure 6:2  
Here, \(x_{c}\) is the enhanced feature map of the c-th channel, and \(s_{c}\) is the weight of the c-th channel. (3) LA Attention Mechanism  
The LA mechanism is a mechanism that computes attention weights only for local regions in sequence data or image data [33].