## 5 Site Experiment Results
### Deformation Prediction Comparison  
Three deep learning models were trained and tested based on the adopted model structure settings in Table 3. The quantitative measured RMSE values in the validation and testing stages of the three deep learning models are shown in Table 4. The transformer model had the lowest RMSE value and outperformed the attention-enhanced LSTM model and the LSTM model. The two LSTM models have similar RMSE values, whereas the attention-enhanced LSTM gives a slightly smaller RMSE value. The attention module included in the attention-enhanced LSTM scored the hidden state better than the standard LSTM. The attention module can automatically increase the weights and significance of more relevant data inputs in the time series of the monitoring data, and the model prediction accuracy was slightly improved from the LSTM model (Table 4). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Case No.** & **Num\_Layers** & **Model\_Dim** & **Hidden\_Size** & **Max\_Len** &
\begin{tabular}{c} **Validation Data** \\ **RMSE/mm** \\ \end{tabular} \\ \hline
1 & 1 & 6 & 6 & 6 & 2.786 \\ \hline
2 & 1 & 6 & 6 & 8 & 0.909 \\ \hline
3 & 1 & 12 & 10 & 9 & 1.053 \\ \hline
4 & 1 & 12 & 12 & 9 & 0.868 \\ \hline
5 & 2 & 6 & 6 & 6 & 1.248 \\ \hline
6 & 2 & 6 & 6 & 9 & 0.943 \\ \hline
7 & 2 & 6 & 6 & 24 & 0.954 \\ \hline
8 & 2 & **18** & **10** & **30** & **0.784** \\ \hline
9 & 3 & 24 & 18 & 48 & 0.832 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Transformer parameter tuning and final model selection based on validation data. Figure 11: The training and validation loss curves of the transformer model. The prediction performance of the three models is shown in Figure 12. Both LSTM models showed poor predictions at both the training and testing stages. The LSTM model does not capture the trend of the tunnel face deformation. The results suggest that the LSTM model has a problem with gradient disappearance for the time-series prediction of long sequences. Compared to the LSTM model, the attention-enhanced LSTM model automatically evaluates the scores in the hidden layer units, highlighting the modelling effect of the most appropriate time steps and thus improving the prediction of the prediction stage. Although the correlation of long-distance time sequence data can be trained with the additional attentional module in the attention-enhanced LSTM model, it still suffers from the gradient vanishing problem. Compared to the two LSTM models, the transformer model showed good prediction performance at both the training and testing stages (Figure 12). The transformer model expands the feature dimension through embedding and extracts the correlation of time-series data in the higher-dimensional feature space through the multi-head attention mechanism. In addition, the transformer model fully considers the potential similarity of the deformation data sequence under different processes through the positional embedding. The encoder structure in the transformer fully extracts time sequence information in high-dimensional space, and the decoder structure has a better decoding and mapping function than the sliding window mechanism in the LSTM. The transformer can extract more information from the time sequence data than the attention-enhanced LSTM and LSTM. It should be noted that the teacher forcing used in the transformer model training phase improves the speed of model training. However, it is not suitable for use in the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Validation Data RMSE/mm** & **Testing Data RMSE/mm** \\ \hline LSTM & 2.035 & 8.120 \\ Attention-enhanced LSTM & 1.942 & 7.135 \\ \hline
**Transformer** & 0.784 & 3.810 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy comparison of LSTM, attention-enhanced LSTM, and transformer with a similar model structure. Figure 12: The training and testing results of the LSTM model, the attention-enhanced LSTM model, and the transformer model. testing phase because the positional embedding value needs to be input into the decoder.