# Long-Distance Shield Tunnelling Performance Prediction Based on Informer
## 2 Methodology  
### Basic Structure and Principles of Informer  
Informer was developed from Transformer, and its topological structure is shown in Figure 1, which consists of three parts: an input layer, an encoding-decoding layer, and an output layer. The input layer receives and preprocesses the original time series data. In the encoding-decoding layer, the encoder extracts long-term dependencies in the sequence through stacked self-attention layers and feedforward neural networks. It uses the ProbSparse self-attention mechanism to reduce computational complexity. The decoder combines the encoder's context information and input to generate predictions for future time steps. It uses a generative decoding strategy to generate multi-step predictions gradually. The output layer converts the output of the decoder into a final prediction, usually including a linear transformation and an inverse data preprocessing step to return to the original scale. This structure allows Informer to process large-scale time series data while remaining efficient and accurate [23].  
From the model structure, it can be seen that the ProbSparse self-attention mechanism in the encoder and the generative decoder play an important role in improving the performance of long-range prediction:  
1. ProbSparse self-attention is based on self-attention probability distributions with sparse sampling [25; 26]. Its distinctive feature is that it selectively computes the attention weights between a subset of key values, rather than computing the attention weights between all key values. In the case of extremely long time series, the conventional full attention mechanism requires the computation of attention weights with a complexity of \(O\big{(}N^{2}\big{)}\), where \(N\) represents the length of the sequence. In contrast, ProbSparse self-attention only requires the calculation of \(O(N\text{log}\,N)\) attention weights. This significantly reduces the computational complexity. The attention mechanism is embedded within the encoder and learns sparse attention matrices to reduce the cost of computations during the inference stage. This mechanism collaborates with other components within the encoder, such as feedforward neural networks and residual connections, to achieve efficient time series prediction.  
2. The generation-based encoder adopts the start token technique [27], which assists the model in accurately identifying the initial position of the predicted sequence and thereby reduces the cumulative error in long-range prediction. In long-range forecasting, the model's prediction of the future is always based on previous results. Suppose there are any errors in the prediction results of each step. In that case, these errors will accumulate with the increase in the prediction sequence length, ultimately leading to the deterioration of the overall prediction quality. Using the start token technique allows the model to reinitialize its position at the onset of each new prediction, thereby mitigating the impact of cumulative errors because the start token gives the model a novel reference point, enabling it to return to the correct starting point when generating each new sequence [28; 29].