## 3 Model Structure
### Two-Stage Transformer  
This study introduces an innovative framework for time-series forecasting algorithms, the Two-stage Transformer [10]. It optimizes the traditional Transformer forecasting method's structure by segmenting it into an Auto-Regression Stage and a Self-Regression Stage. This design addresses the discrepancies in statistical properties between the input and predicted sequences. Central to this framework, the Two-stage Transformer optimizes the traditional Transformer architecture through transformative and decoupling operations, significantly reducing time and space complexity.  
To further enhance model performance, this study substitutes the original self-attention mechanism with the CSPNet-Attention mechanism in the Encoder, augmenting the model's capacity to capture sequence features. Moreover, the Two-stage Transformer incorporates an innovative Error Score Modification (EMS) module, employing a learnable Gaussian distribution \(N(0,\sigma_{i}^{2})\), where \(i\in[1,l_{out}]\), to optimize the model's prediction accuracy and robustness. Within EMS, the distribution's center is set to zero, enabling the model to bias auto-attention score assignments towards earlier, more reliable predictive elements. Consequently, this approach amplifies the influence of earlier predictor elements on subsequent ones in the autoregressive process, thereby improving the model's overall performance in time-series prediction.  
Figure 3: Overall architecture of the cross-stage part of the network.  
Algorithm 1 illustrates the algorithmic flow of the error score correction module, with its algorithmic formulation provided as follows in Equation (6):  
\[ESM(Q,K)=QK^{\top}+\begin{bmatrix}N(0,\sigma_{1}^{2})\\ N(0,\sigma_{2}^{2})\\ \ldots\\ N(0,\sigma_{\mathit{lout}}^{2})\end{bmatrix} \tag{6}\]  
```
Input: Tensor \(x\in\mathbb{R}^{\mathrm{L}\times d}\) Layer params: Linear(x): Linear projection layer of x
1:\(\mathcal{Q},\mathcal{K},\mathrm{V}=\mathrm{Linear}_{1}(\mathrm{x})( \mathcal{Q},\mathcal{K},\mathrm{V}\in\mathbb{R}^{\mathrm{L}\times\mathrm{d}})\)
2:\(\sigma=\mathrm{Linear}_{2}(\mathrm{x})\big{(}\sigma\in\mathbb{R}^{\mathrm{ L}\times 1}\big{)}\)
3: Calculate self-attention scores \(\mathrm{A}=\mathrm{QK}^{\top}\big{(}\mathrm{A}\in\mathbb{R}^{\mathrm{L}\times \mathrm{L}}\big{)}\)
4:\(\sigma=\mathrm{Sigmoid}(5\sigma)+1\times 10^{-5}\)
5:\(\sigma=3^{\sigma}-1\)
6:\(\sigma=\mathrm{Repeat}\big{(}\sigma,\mathrm{dim}=1\big{)}\big{(}\sigma\in \mathbb{R}^{\mathrm{L}\times\mathrm{L}}\big{)}\)
7: Generate Gauss distribution matrix \(\mathrm{G}\sim\mathrm{N}\big{(}0,\sigma^{2}\big{)}\big{(}\mathrm{G}\in\mathbb{ R}^{\mathrm{L}\times\mathrm{L}}\big{)}\)
8: Add self-attention scores with ESM: \(\mathrm{A}=\mathrm{A}+\mathrm{G}\)
9: \(\mathrm{A}=\mathrm{mask}(\mathrm{A})\) Output: Masked self-attention scores A
```  
**Algorithm 1** Masked self-attention with ESM  
In the \(N(0,\sigma_{i}^{2})\) configuration, \(\sigma_{i}\) can undergo continuous adaptation throughout the model training process, rendering the error score correction module highly adaptable.  
We add CSPNet-Attention to replace the self-attention mechanism in the first stage, and the improved model architecture is shown in Figure 4 below.  
In this framework, we input the input variables into the Encoder in the first stage, followed by a convolution operation, and then the output is flattened into a one-dimensional tensor, and finally, a linear transformation is realized through a fully connected layer; in the second stage, the output of the first stage is first used as input  
Figure 4: Overall architecture of the Two-stage Transformer Model with integrated CSPNet-Attention.  
non-linear transformation and feature extraction are carried out through the feed-forward neural network (which consists of two FC layers and an activation function). Finally, the result is mapped back to the input dimension by linear projection.