## 2 Materials and Methods
### iTransformer Model  
iTransformer is an improved Transformer model proposed by [PERSON] et al. in 2023 [35], which has achieved significant breakthroughs in complex multivariate time series forecasting tasks. iTransformer consists only of the encoder part and is composed of three main components: an embedding layer, multiple stackable Transformer modules, and a projection layer, as shown in Figure 3.  
Given a multivariate time series dataset \(X=\{\mathrm{x}_{1},\mathrm{x}_{2},\cdots,\mathrm{x}_{\mathrm{T}}\}\in R^{T \times N}\) with a historical time step length of \(T\) and \(N\) variables, iTransformer first maps each variable's sequence into a D-dimensional feature space through the embedding layer, obtaining a high-dimensional  
Figure 3: Structure of the iTransformer Module. Figure 2: Structure of the BiLSTM Module. feature representation \(h_{n}^{0}\). Each variable's feature vector is referred to as a \"Variate Token\", which contains all temporal variation information of that variable over the past \(T\) time steps, encoding the entire sequence into a single token. The feature vectors of all variables form a matrix \(H\in\{h_{1},h_{2},\cdots,h_{N}\}\in R^{N\times D}\), which represents the sequential characteristics of each variable in the multivariate time series. Compared to the Transformer model, iTransformer has an advantage in the embedding strategy, as it treats the entire time series segment as an independent variable (as shown in Figure 4), preserving the independence of each variable and avoiding the information mixing phenomenon during the embedding process in the traditional Transformer model. Meanwhile, iTransformer also reverses the roles of the multi-head self-attention mechanism and the position-wise feed-forward neural network in the Transformer: the traditional Transformer's multi-head self-attention mechanism mainly models the temporal dependencies of the same variable at different historical time steps, but its ability to model cross-variable interaction dependencies is limited; in contrast, iTransformer's multi-head attention mechanism is used to capture dependencies among different variables, while its position-wise feed-forward neural network is responsible for extracting the temporal characteristics of each variable, thus achieving the modeling and learning of the spatiotemporal dependencies of input variables. Within each Variate Token, layer normalization is applied to unify the measurement units and feature distributions among variables, accelerating model convergence and improving model stability. Moreover, since the decoder of the Transformer adopts an autoregressive prediction approach, which is prone to error accumulation, iTransformer directly projects each Variate Token through a projection layer implemented by a multi-layer perceptron (MLP), adopting a multi-task learning-like approach to directly output the final prediction result \(\hat{Y}_{:,n}\) of each variable, rather than using a decoder to generate predictions. The entire process can be expressed as the following computational workflow:  
\[h_{n}^{0}=Embedding(X:,n) \tag{3}\]  
\[H^{l+1}=TrmBlock(H^{i}),l=0,1,\cdots,L-1 \tag{4}\]  
\[\hat{Y}_{:,n}=\text{Projection}({h_{n}}^{l}) \tag{5}\]  
where \(X:,n\) represents the entire sequence of the \(n\)th variable, \(\hat{Y}_{:,n}\) denotes the feature representation at the \(i\)th layer, and \(\hat{Y}_{:,n}\) represents the model's predicted data for the \(n\)th variable. In the TrmBlock component, the multi-head attention mechanism computes attention weights for the multivariate time series variables after positional encoding. By assigning  
Figure 4: Differences in Variable Time Series Embedding Methods between Transformer and iTransformer. high weights to key features, it captures the nonlinear relationships between variables. The calculation formula is as follows:  
\[\left\{ \begin{array}{l}Q=W^{Q}X\\ K=W^{K}X\\ V=W^{V}X\end{array} \right.