# Composite Foundation Settlement Prediction Based on LSTM-Transformer Model for CFG
## 3 The Establishment of a CFG Compound Foundation Settlement Model Based on LSTM-Transformer
### The LSTM-Transformer Model Structure
\[x_{t}=\{S_{t-3},S_{t-15},S_{t-9},V_{t},H_{t}\}^{T} \tag{8}\]  
Considering that foundation settlement data is multivariate, with differences in dimensions and numerical values, directly applying it can affect model training and testing. Therefore, based on the formula, the original model is normalized to ensure that the numerical ranges of different variables are the same. This reduces biases and unequal weight distribution towards the data, thereby improving model accuracy and training speed. \[x^{\prime}{}_{t}=\frac{x_{t}-\min(x_{t})}{\max(x_{t})-\min(x_{t})} \tag{9}\]  
A sliding time window is used to segment normalized time series, with the calculation formula as follows:  
\[\left\{ \begin{array}{l}X_{1}=[x^{\prime}_{1},x^{\prime}_{2},x^{\prime}_{3} \cdots x^{\prime}_{k}]^{T}\\ X_{2}=[x^{\prime}_{2},x^{\prime}_{3},x^{\prime}_{4},\cdots x^{\prime}_{k+1}]^{T }\\ \vdots\\ X_{n}=[x^{\prime}_{n},x^{\prime}_{n+1},x^{\prime}_{n+2},\cdots x^{\prime}_{n+k -1}]^{T}\end{array} \right. \tag{10}\]  
In the equations, \(x^{\prime}_{t}\) represents the normalized cumulative settlement; \(X_{n}\) represents the segmented sliding sequence. The segmented time series \(X_{n}\) is input into the LSTM-Transformer model. The LSTM neural network layer is used to capture non-linear information hidden in cumulative settlement values at different times. This process yields new cell memory state information for each time step. At this point, the output hidden state vector is combined with the original input through a fusion layer, integrating different features learned from multiple layers and directions of the LSTM. This results in a more comprehensive and accurate representation, which is then used as input to the multi-head attention layer. During the calculation of the multi-head attention (MHA), the input from the previous module and the current input are jointly considered. The structure of the Transformer network layer is constructed, utilizing the self-attention mechanism of the Transformer to allocate weights for information exploration by the LSTM. Subsequently, in the feature layer, the input is reconstructed, learning the previous time step states and incorporating factors such as the current observed fill height to predict future foundation settlement. The calculation process is shown in Equations (11)-(13):  
\[X_{L}{}^{\text{t}},h_{L}{}^{\text{t}}=L(X_{n},g(h_{L}{}^{t-1})) \tag{11}\]  
\[X_{L}{}^{\text{t}}=F(X_{L}{}^{\text{t}},X_{n}) \tag{12}\]\[G^{t}=T(X_{L}{}^{t}) \tag{13}\]  
In the equations, \(X_{L}{}^{t}\) represents the time series group processed by the LSTM module; \(h_{L}{}^{t}\) is the memory output module representing each output from the LSTM module without gradient updates; \(L\) represents the output, denoted as \(X_{L}{}^{t}\) and \(h_{L}{}^{t}\), from the LSTM module without gradient updates, continuously fed into the multi-head attention (MHA) layer, constantly introducing the hidden states from the previous LSTM module into the current input. In Equation (12), \(X_{L}{}^{t}\) represents the output of the fusion layer; F denotes the combination of optional values \(X_{L}{}^{t}\) and \(X_{n}\) in the fusion layer; \(T\) signifies that the vector added in the multi-head attention (MHA) undergoes the same computational process [36]. In Equation (13), \(G^{t}\) represents the settlement prediction values output by the LSTM-Transformer model. The fusion layer combines the output vector with the corresponding input vector. The fused vector is then passed through an activation function, aiding the model in allocating attention weights based on various input features.