## 1 Introduction
There is a lack of adequate data analytical methods to utilize the noisy monitoring data to predict the possible tunnel face deformation in the coming construction period. With recent advances in deep learning and data mining techniques, opportunities have opened up for developing a data-driven prediction model for the construction phase of the rock tunnel practice with dynamic construction processes. The main goal of the data-driven model is to provide a practical tool to predict the time sequence deformation responses in the construction process with the site monitoring input. In this context, recurrent neural networks (RNN) and long short-term memory (LSTM) networks are the go-to solutions for sequence data processing in geotechnical practice [27; 28], which originate from the problem of natural language processing (NLP). When provided with a data sequence, an RNN processes the first input data in the sequence and feeds the result into the layer that processes the following input. This enables the RNN to keep track of the entire data sequence instead of processing each piece of data separately. LSTM, the successor to RNN, can solve the vanishing gradients problem to some degree and handle more extensive data sequences. However, RNN and LSTM need to process data sequentially, which cannot take advantage of parallel computing hardware and graphics processing units (GPU) in training and prediction. In addition, RNN and LSTM cannot handle long sequences of data and can only capture the relations between inputs at nearby positions/time steps in the data sequence. Accurate prediction of tunnel face deformation requires consideration of the whole data sequence depending on the rock tunnel construction process. In this regard, the transformer model was introduced in 2017 by a team at Google Brain, a deep learning model that adopts the self-attention mechanism [29]. Compared to RNN or LSTM, the transformer model follows an encoder-decoder structure, which does not rely on recurrence and convolutions to generate an output. The attention mechanism provides context for any position in the input sequence and differentially weights the significance of each part of the input data. Hence, the transformer model makes it possible to process entire sequences in parallel and scale the speed and capacity of sequential deep learning models to unprecedented rates. Although the attention mechanism and the transformer model have been widely adopted in NLP tasks and other sequence data features, to the best of the authors' knowledge, there is limited study on adopting an attention-based deep learning model to perform tunnel deformation predictions in rock tunnelling practice. This study focuses on tunnel deformations in the construction process of rock tunnel excavation. Wireless tilt and distance sensor (WTDS) nodes were adopted in the WSNs to measure the deformations of excavated surrounding rocks. The sensor locations were carefully selected according to the engineering safety risks. A triangular truss protection cover was designed and deployed on site to ensure the durability of the WTDS in the harsh environment of rock tunnel excavation using the drill and blasting method. This study adopts three deep learning models for predicting the tunnel face deformation based on the monitoring data, namely LSTM, attention-enhanced LSTM, and the transformer model. These three models correlate the deformation of the nearby transverse tunnel section with the tunnel face deformation prediction to provide early warning of the risk of surrounding rock deformation. A tunnel site case study was conducted over three weeks to obtain high-granularity tunnel deformation data from the WTDS nodes. The obtained monitoring data were processed to create a dataset to train and assess the performance of the three deep learning models. Overall, the transformer model provides sound performance to predict the tunnel face deformation.