# Composite Foundation Settlement Prediction Based on LSTM-Transformer Model for CFG
## 1 Introduction  
High-speed railway trains are particularly sensitive to the settlement and deformation of the underlying track structure, especially when traveling at high speeds [1, 2]. Therefore, controlling the post-construction settlement of high-speed railway embankments has become a crucial aspect of high-speed railway embankment design. The use of cement fly ash gravel (CFG) piles to treat soft soil foundations at the location has advantages such as minimal settlement, high strength, and short construction period. It has become a frequently employed method in the construction of high-speed railways [3, 4, 5]. For composite foundations commonly encountered in high-speed railway construction, such as CFG pile composite foundations, accurately predicting their post-construction settlement is a key aspect of high-speed railway construction and settlement assessment work [6, 7]. Currently, there are mainly three categories of traditional foundation settlement prediction methods: The first category involves using theoretical methods, such as the layer sum method, to calculate the final settlement, then simplifying the consolidation formula to estimate the degree of consolidation, thereby predicting the development pattern of settlement; the second category is the numerical calculation method, which is based on consolidation theory and combined with various soil constitutive models to calculate the final amount of settlement and its development pattern. Both of these methods are theoretically reasonable and feasible, but the calculation parameters involved must be obtained through experiments. During the sampling process, the soil samples are inevitably disturbed, leading to discrepancies between experimental and actual parameters. Therefore, the predictionresults of these two methods usually have large errors. The third category includes methods such as the hyperbolic method [8], [PERSON]'s method [9], [PERSON]'s method [10], and the grey model method [11]. Since most of these prediction methods and models are singular and each has its characteristics and applicable scenarios, they cannot fully exploit the information of actual measurement data, and the accuracy of their predictions still needs to be improved. With the development of computer science technology, more scholars are utilizing the powerful nonlinear fitting capabilities of algorithms such as artificial neural networks and support vector machines [12; 13; 14] for the deformation prediction of foundations. [PERSON] [15] established a sparnow search algorithm optimized support vector machine regression (SSA-SVR) prediction model to forecast the settlement of coal gangue subgrade. [PERSON] et al. [16], based on the bidirectional long short-term memory network (Bi-LSTM), combined a rolling iteration method to update settlement predictions, which can effectively enhance the reliability of subgrade settlement forecasting. [PERSON] et al. [17], based on the GA-BP neural network model, predicted the post-construction settlement of loess high-fill sites. [PERSON] et al. [18] proposed a subway tunnel surface settlement prediction method using VMD-GRU for step-method construction, which can effectively predict short-term abrupt data changes. The aforementioned combined neural network models have achieved certain results in learning and predicting surface settlement data. However, these intelligent algorithms also have some shortcomings. For instance, support vector machines struggle with large sample data, and although neural networks have strong fitting capabilities, they face issues such as overfitting, local minima, or slow convergence during the training process. Therefore, there is a need to improve existing methods. Furthermore, they usually only use post-construction subgrade settlement data as sample information, ignoring the rich construction knowledge during the filling period. Subgrades often adopt the periodic characteristics of 'layered filling--filling intervals', and the post-construction settlement can be considered as a long filling interval.