## 3 Experimental Results and Analysis
### Deep Learning Modeling Results
#### 3.4.2 Modeling Results of Deep Learning Models Based on Attention Mechanism
Unlike global attention, self-attention mechanisms, and squeeze-and-excitation operations, local attention focuses only on a subset of the input sequence, thereby reducing computational complexity and improving efficiency. The network structure of the LA attention mechanism is shown in Figure 6:3  
Firstly, LA defines a local window. For each position \(i\) in the sequence, a local window [\(i-D\), \(i+D\)] is defined, where \(D\) is the window radius. The position \(j\) within the window satisfies the following:  
Figure 6:4  
Then, calculate the attention weights by computing the dot product of the query \(Qi\) and the key \(Kj\) within the local window and normalizing it through Softmax:  
Figure 6:5  
Here, \(Q_{i}\) is the query vector at position i, \(K_{j}\) is the key vector at position \(j\), and \(d_{k}\) is the dimension of the key vector, which is used for scaling. Finally, perform a weighted summation operation. Within the local window, conduct a weighted summation of the values \(V_{j}\) to obtain the output at position i:  
Figure 6:6  
The attention mechanism was used to enhance three typical deep learning models, namely, the CNN model, the LSTM model in RNN, and the Transformer model, which is more effective at processing sequence data. The modeling results are shown in Figure 6:7 The results indicate that the combination of different models and attention mechanisms has a significant impact on task performance. Among CNN, LSTM, and Transformer, the models combined with the SE attention mechanism all showed performance improvements. For instance, the R\({}^{2}\) value of LSTM-SE reaches 0.9387, and the RMSE and MAE were reduced to 1102.43 and 105.94, respectively, significantly outperforming other attention mechanisms. From the results of CNN-SE and Transformer-SE, it was also found that the performance of the models improved after adding SE, indicating that the SE mechanism can effectively enhance feature  
Figure 6:8 Schematic diagram of the LA attention mechanism. representation by explicitly modeling the inter-channel dependencies. Additionally, the SA mechanism brought performance improvements in both LSTM and Transformer, but it performed slightly worse in CNN. For example, the R\({}^{2}\) value of LSTM-SA was 0.9034, which is better than the basic LSTM model. However, the R\({}^{2}\) value of CNN-SA slightly decreased (0.8735), suggesting that the SA mechanism may have limited improvement on the feature extraction ability of CNN. Meanwhile, it was found that the local attention mechanism LA combined with each model performed poorly in the prediction of the heat transfer coefficient on the straight road. For instance, the R\({}^{2}\) value of LSTM-LA was only 0.8254, and the RMSE and MAE increased to 1608.54 and 211.49, respectively. This indicates that the local attention mechanism may not be able to fully capture the global dependencies, leading to a decline in performance. For the hexagonal flow channel, the same attention embedding operation was performed, and the results are summarized in Figure 6:9 The CNN model achieved an R\({}^{2}\) value of 0.996270, an RMSE of 55.751, and an MAE of 35.456. After incorporating the attention mechanism, the R\({}^{2}\) values of the CNN-SA, CNN-SE, and CNN-LA models slightly increased, while the RMSE and MAE values also rose, suggesting that the attention mechanism has a limited positive impact on the performance of the CNN model. Among these, the LA attention mechanism demonstrated a more noticeable improvement in the CNN model. Meanwhile, it was observed that for the LSTM and Transformer models, the introduction of the attention mechanism resulted in a decline in model performance. However, it was also noted that the LA attention mechanism contributed to enhancing the accuracy of both models, particularly for the Transformer model, where the accuracy significantly improved, with the R\({}^{2}\) increasing from 0.992415 to 0.999257. This indicates that the LA attention mechanism exhibits strong compatibility with the Transformer model.