## 3 Materials and Methods
### Backbone Models and Construction of Basic Natural Gas Purchase Forecasting Models
#### 3.2.1 Informer and Its Four Base Model Constructions  
The Informer aims to address the efficiency and accuracy issues present in the traditional Transformer, thereby enabling long-sequence time series forecasting. The core innovation of the Informer lies in its ability to process long sequences through self-attention and sparse attention mechanisms, significantly reducing computational complexity while maintaining high performance. The Informer employs a sparse self-attention mechanism to selectively focus on important parts of the input sequence, thereby reducing the computational overhead compared to the full attention mechanism used in standard Transformer [31]. This selective attention is achieved through two main strategies [32], namely the following:  
1. ProbSparse attention: By focusing on the most critical parts of the sequence, the quadratic complexity of traditional self-attention is reduced to logarithmic complexity. It is achieved by selecting the top \(U\) queries based on their impact, significantly speeding up the processing.
2. Distilling operation: The distilling operation reduces the sequence length by halving it at each layer while retaining key information, thereby decreasing the dimensionality of the sequence. This hierarchical structure enables the model to efficiently handle long sequences. In this study, four base models were constructed using the Informer to capture the impact of different feature combinations on the prediction of natural gas purchase volumes. The specific models are as follows:  
1. INF: This model utilizes only basic features for prediction. The model is formulated as follows: \[\hat{Y}_{INF}\ =\ \text{Informer}(X_{basic})\] (1) where \(X_{basic}\) represents the matrix of basic features, which includes time features (year, month) and temperature features (monthly average minimum and maximum temperatures). This can be expressed by the following formula: \[X_{basic}\ =\ X_{time}+X_{temperature}\] (2)
2. INF-RB: Based on the basic features, regional features (natural gas sales volumes in different regions) are added. The model is formulated as follows: \[\hat{Y}_{INF-RB}\ =\ \text{Informer}(X_{basic}+X_{region})\] (3) where \(X_{region}\) represents the matrix of regional features.
3. INF-UB: Based on the basic features, usage features (such as commercial, industrial, residential, etc.) are added. The model is formulated as follows: \[\hat{Y}_{INF-UB}\ =\ \text{Informer}(X_{basic}+X_{usage})\] (4) where \(X_{usage}\) represents the matrix of usage features.
4. INF-RB-UB: Combining basic features, regional features, and usage features, this model comprehensively considers all features for prediction. The model is formulated as follows: \[\hat{Y}_{INF-RB-UB}\ =\ \text{Informer}(X_{basic}+X_{region}+X_{usage})\] (5) The same feature combinations are applied to the MLR and SVR. Table 2 shows the parameters used for training and predicting with the Informer model.  
The Informer was trained using the Adam optimizer with a learning rate of 0.0001 and a batch size of 4. The training process spanned 5 epochs, with early stopping criteria based on validation loss.