## 3 Case Study
### Bayesian Hyperparameter Optimization for Temporal Attention-LSTM Model  
The dataset selected for this paper is from ring 4 to ring 453 of the south line of the shield tunneling project. The first 70% of the dataset is the training set, and the last 30% is the testing set. The last 20% of the training set is used as the validation set for hyperparameter tuning. Bayesian optimization is used to automatically select hyperparameters for the temporal attention-LSTM model. As shown in Table 4, the hyperparameters include \"LSTM unit\", \"dropout\", \"learning rate\", \"batch size\", and \"attention unit\". Specifically, the \"LSTM unit\" represents the hidden units in the LSTM network, which are the core components of the LSTM, and they are used to process, memorize, and transfer information. Dropout randomly turns off some units of the LSTM model with a certain probability \(p\) (usually between 0.1 and 0.5) to prevent the overfitting of the model. The learning rate controls the update rate of the model weights during the training process. If the learning rate is set too high, the model may hover around the optimal solution and fail to converge, while if the learning rate is set too low, the training process may be very slow and may fall into a local  
Figure 8: Denoising of advance rate. (**a**) MSE (mm/s)\({}^{2}\) of different db wavelet-based functions for advance rate and (**b**) comparison of raw data and denoised data.  
optimum. The \"batch size\" refers to the number of samples processed together during one training. A total of 500 epochs are used for each optimization search. One epoch means that the entire training dataset is trained once. The \"input sequence length\" refers to the number of historical data points utilized in predicting future values. The \"attention unit\" represents the number of neurons in the attention layer. These units are used for computing the attention weights, which help the model focus on the most relevant parts of the input data. The \"patience\" is set to 40, which means that if the model loss does not decrease for 40 epochs, then the training stops. The MSE (mean squared error) is selected as the Bayesian optimization evaluation metric, and the best score is 0.00416.  
The root mean square error (RMSE) and the coefficient of determination (R2) are used to evaluate the model prediction results. The RMSE measures the errors between the pre  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Hyperparameter**} & **LSTM Unit** & **Dropout** & **Learning Rate** & **Batch Size** & **Input Sequence Length** & **Attention Unit** \\ \hline Domain & [1, 256] & [0, 1] & [0.001, 0.1] & [1, 256] & [2, 10] & [1, 256] \\ Epoch & & & & 500 & & \\ Maximum iteration & & & & 100 & & \\ Patience & & & & 40 & & \\ Optimal value & 64 & 0.18 & 0.003 & 128 & 6 & 32 \\ Best score (MSE) & & & & 0.00416 & & \\ \hline \hline \end{tabular}
\end{table}
Table 4: Bayesian hyperparameter optimization results for temporal attention-LSTM.  
Figure 9: Correlation coefficient of shield tunneling construction parameters.  
dictions and measurements, which are on the same scale as the measurements. If there are several points with large errors, the value of RMSE tends to be higher. The R2 is mainly used to evaluate the degree of fit between the predicted values and measurements. The value of R2 usually ranges from 0 to 1. A value equal to 1 indicates that all predicted values exactly match the measurements, while a value equal to 0 indicates that the performance of the model is equivalent to simply using the mean of the measurements as the predicted values.  
\[\text{RMSE}=\sqrt{\frac{1}{m}\sum\limits_{i=1}^{m}\left(y_{i}-\hat{y}_{i} \right)^{2}} \tag{9}\]  
\[\text{R}^{2}=1-\frac{\sum\limits_{i=1}^{m}\left(y_{i}-\hat{y}_{i}\right)^{2}} {\sum\limits_{i=1}^{m}\left(y_{i}-\overline{y}\right)^{2}} \tag{10}\]  
where \(m\) is the total number of samples, \(y_{i}\) denotes the measurements, \(\hat{y}_{i}\) is the model prediction, and \(\overline{y}\) denotes the mean of the measurements.