## 2 Literature Review  
### Attention Mechanism  
The attention mechanism has emerged as a significant technique in the domain of deep learning in recent years [13; 20; 21]. Its primary objective is to allocate different attention weights to various portions of the input data, enabling models to focus more on information that is vital for specific tasks. The crux of the attention mechanism lies in computing a weight distribution, subsequently utilizing this distribution to conduct a weighted summation of the input data, culminating in a weighted representation. This representation captures pivotal information in the input data pertinent to the current task. Within the realm of computer vision, a quintessential application of the attention mechanism is the Squeeze-and-Excitation Network (SENet) [20], as shown in Figure 1.  
The SENet attains a global feature descriptor by globally pooling the feature map. Following this, a transformation is conducted via two fully connected layers, resulting in a weight distribution. This distribution is employed to modify the significance of each  
Figure 1: Illustration of the attention mechanism in SENet [20]. channel in the original feature map. Specifically, for a feature map \(X\in\mathbb{R}^{H\times W\times C}\), the SENet first calculates a channel descriptor:  
\[z=\text{GlobalPooling}(X) \tag{1}\]  
Subsequently, a weight distribution is derived through two fully connected layers:  
\[\alpha=\sigma(W_{2}\text{ReLU}(W_{1}z)) \tag{2}\]  
Here, \(\sigma\) represents the sigmoid activation function, and \(W_{1}\) and \(W_{2}\) are the weights of the fully connected layers. Lastly, the original feature map is weighted using the weight distribution \(\alpha\):  
\[X^{\prime}=\alpha\odot X \tag{3}\]  
In this context, \(\odot\) signifies element-wise multiplication. In tasks related to natural language processing, the attention mechanism was initially employed for sequence-to-sequence models in machine translation [13], as shown in Figure 2. In such models, the encoder first encodes the source language sentence into a fixed-length vector. The decoder then exploits this vector to generate the sentence in the target language. The attention mechanism facilitates the decoder in focusing on diverse portions of the source sentence during the generation of each word. Specifically, given the encoder's output \(H\in\mathbb{R}^{L\times d}\), where \(L\) denotes the length of the source sentence and \(d\) is the feature dimension, the attention weight \(\alpha_{t}\) for the hidden state \(s_{t}\) of the decoder at time \(t\) is computed as:  
\[e_{t}=\tanh(W_{h}H^{T}+W_{s}s_{t}) \tag{4}\]  
\[\alpha_{t}=\text{softmax}(e_{t}) \tag{5}\]  
Following this, the encoder's output is weightedly summed using the attention weight \(\alpha_{t}\), resulting in a context vector \(c_{t}\):  
\[c_{t}=\alpha_{t}\cdot H \tag{6}\]  
This context vector \(c_{t}\) is then used in tandem with the decoder's hidden state \(s_{t}\) for output generation. Figure 2: Illustration of the attention mechanism in Transformer [13]. For time series prediction tasks, such as stock price prediction, as discussed in this work, the attention mechanism aids the model in emphasizing historical data that are most pertinent to the current prediction timestamp. Specifically, given the features of the past \(n\) timestamps \(X\in\mathbb{R}^{n\times d}\), for the current hidden state \(s\), an attention weight distribution can be computed as:  
Figure 1:0  
Figure 1:1  
Subsequently, the past features are weightedly summed using the attention weight \(\alpha\), deriving a context vector:  
Figure 1:2  
This context vector \(c\) encapsulates information from past data most correlated with the current prediction timestamp, thus bolstering the model's predictive accuracy.