# Construction Project Cost Prediction Method Based on Improved BiLSTM
## 3 Predictive Modelling Design
### Capturing Inter-Temporal Interactions  
An LSTM network is a variant structure of a recurrent neural network (RNN) that solves the problem of gradient vanishing that occurs in RNNs when handling long sequential data. The structure of an LSTM network is illustrated in Figure 2.  
An LSTM unit contains three gate structures called the forget gate, input gate, and output gate, respectively. The forget gate determines that information \(f_{t}\) is overlooked based on the hidden state \(h_{t-1}\) in the previous moment and the input \(x_{t}\) in the current moment. The input gate takes the information update value \(i_{t}\) and candidate cell state \(\hat{c}_{t}\) through \(h_{t-1}\) and \(x_{t}\). We multiply \(f_{t}\) times the cell state \(c_{t-1}\) in the previous moment and add the data from the cell update to obtain the new cell state \(c_{t}\), which yields the initial output \(o_{t}\) through \(h_{t-1}\) and \(x_{t}\) and the current moment output \(h_{t}\) through \(c_{t}\) and \(o_{t}\). The corresponding formulas are defined as follows:  
\[f_{t}=\sigma\Big{(}w_{f}h_{t-1}+w_{f}x_{t}+b_{f}\Big{)} \tag{4}\]  
\[i_{t}=\sigma(w_{i}h_{t-1}+w_{i}x_{t}+b_{i}) \tag{5}\]  
Figure 1: Forecasting model structure.  
Figure 2: LSTM network structure.  
\[\hat{c}_{t} =\tanh(w_{c}h_{t-1}+w_{c}x_{t}+b_{c}) \tag{6}\] \[c_{t} =f_{t}c_{t-1}+i_{t}c_{t}\] (7) \[o_{t} =\tanh(w_{o}h_{t-1}+w_{o}x_{t}+b_{o})\] (8) \[h_{t} =o_{t}\text{tanh}(c_{t}) \tag{9}\]  
where \(w_{f},w_{i},w_{c}\), and \(w_{o}\) represent the weight matrices of the different gates; \(b_{f}\), \(b_{i}\), \(b_{c}\), and \(b_{o}\) represent the corresponding bias vectors; and \(\sigma\) and tanh represent the activation functions.  
In contrast to traditional LSTM, a BiLSTM network consists of two LSTM layers for the forward and backward directions and can fully account for both historical and future information. The structure of BiLSTM is illustrated in Figure 3.  
The BiLSTM layer performs inter-temporal interaction capture by conducting both forward and backward pass steps. During the forward pass, the data in each time step are processed stepwise, starting from the first time step of the time series data. For each time step, the BiLSTM calculates and updates the hidden state \(h_{t}^{f}\) of the current time step based on the input data \(x_{i}\) of the current time step and hidden state \(h_{i-1}^{f}\) from the previous time step. In contrast to the forward pass, in the backward pass, the BiLSTM begins processing forward from the last time step in the time series data. For each time step, BiLSTM calculates and updates the hidden state \(h_{i}^{b}\) of the current time step based on the input data \(x_{i}\) of the current time step and the hidden state \(h_{i+1}^{b}\) from the subsequent time step. Because the information transmitted in the past and the information transmitted in the future regarding the cost data have different degrees of importance, a more comprehensive information representation \(o_{t}\) is obtained by connecting \(h_{i}^{f}\) and \(h_{i}^{b}\) through the adaptive assignment of weights to capture the inter-temporal interactions at a deep level, which is calculated as follows:  
\[h_{t}^{f} =\text{LSTM}\Big{(}x_{t},h_{t-1}^{f}\Big{)} \tag{10}\] \[h_{t}^{b} =\text{LSTM}\Big{(}x_{t},h_{t-1}^{b}\Big{)}\] (11) \[o_{t} =w_{f}h_{t}^{f}+w_{b}h_{t}^{b}+b \tag{12}\]  
where \(w_{f}\) and \(w_{b}\) represent the weight matrices of the forward and backward LSTM layers, respectively, and \(b\) represents the bias vector.