# Composite Foundation Settlement Prediction Based on LSTM-Transformer Model for CFG
## 3 The Establishment of a CFG Compound Foundation Settlement Model Based on LSTM-Transformer
### The LSTM-Transformer Model Structure  
In time series tasks, there are typically two challenges: short-term dependencies and long-term dependencies. When future data is primarily influenced by historical data over a longer time span, this situation is referred to as long-term dependency. Conversely, when future data is mainly influenced by historical data over a shorter time span, this is referred to as short-term dependency. The LSTM processes data sequentially over time, relying on the memory of its cell state to propagate dependencies between time steps. Due to the potential loss of information within this process, LSTMs are adept at capturing short-term dependencies within sequences. However, their effectiveness in capturing long-term dependencies within se  
Figure 3: LSTM recurrent neural network diagram. quences is relatively limited, leading to challenges in addressing long-term dependency issues [34]. The Transformer model utilizes self-attention mechanisms to model relationships between different time steps. Due to its parallelized mechanism for extracting both short-term and long-term information, the Transformer does not exhibit bias towards recent time points, meaning it does not assign higher weights to the most recent data [35]. The parallel attention mechanism allows for the extraction of features from distant time steps. However, the parallel nature of the Transformer also introduces a challenge--it does not inherently capture sequential information in its input due to the lack of an explicit temporal order. Considering the two points mentioned above, a hybrid LSTM-Transformer model is proposed to effectively combine the ability of the LSTM to capture short-term sequential dependencies and the modeling capability of the Transformer for relationships between time steps, especially long-term dependencies. In this architecture, an LSTM encoder is used to encode the input sequence, while a Transformer decoder is employed to generate the output sequence. With this structure, the LSTM decoder can gradually produce the output sequence to retain temporal features, while the Transformer encoder can effectively capture relationships between different positions in the multivariate time series, thereby enhancing the overall model performance. The LSTM-Transformer architecture diagram is shown in Figure 4, and the specific process is as follows:  
1. Data processing  
The frequency of foundation monitoring changes over the operation time, with a higher frequency in the initial stages of construction and a lower frequency later on. The settlement observation data is a set of non-equidistant data, which does not allow for equidistant analysis. The use of cubic spline interpolation with segmented interpolation  
Figure 4: The LSTM\(-\)Transformer architecture diagram. significantly reduces the number of interpolations, facilitating the solution of the undetermined coefficients and resulting in a smoother curve. During the settlement monitoring process, accidental factors such as bad weather or instrument failure may cause some settlement observation data to deviate from normal. These anomalies can be determined based on the overall trend of the settlement curve. It is advisable to first clean and eliminate these data before proceeding with interpolation. Selection and Standardization of Input Features  
To predict settlement at time t, it is necessary to use settlement data before time t as inputs. The model's training effectiveness depends on these input variables. More input data extracted from the settlement-time curve helps the model's hidden layers learn complex, nonlinear relationships, improving predictions. The next moment's subgrade settlement trend is often closely related to the immediate past settlement. Considering this, the past 3, 6, 9 days' settlements amount to \(S_{t-3}\), \(S_{t-9}\), and \(S_{t-15}\); the settlement rate \(V_{t}\) and fill height \(H_{\text{t}}\) are selected as key factors.