# Construction Project Cost Prediction Method Based on Improved BiLSTM
## 3 Predictive Modelling Design
### Capturing Inter-Feature Interactions  
The attention mechanism is a powerful tool in deep learning, and its design is inspired by the information processing mechanism of the human brain [27; 28]. The attention mechanism makes the deep learning model pay more attention to the critical parts of the input data while ignoring the unimportant parts. The core idea is to assign different weights  
Figure 3: BiLSTM network structure.  
dynamically to different inputs so that the model can effectively learn and utilise important information related to the task and adapt more flexibly to the complex relationships in the data, thus improving the performance and generalisation of the model.  
Given the complex spatial-temporal interactions between features of cost data, this article proposes a hybrid attention mechanism (HAM) to capture the inter-feature interactions, aiming to focus on both the temporal and feature dimensions of the data and to improve the ability of the model to capture multivariate time-series information. HAM contains two modules: channel attention and spatial attention. The structure is shown in Figure 4.  
Here, the channel attention module aggregates the information about the feature dimensions through the tie pooling and maximum pooling operations to obtain the deep feature representation of the feature dimensions, which is then mapped into the attention weights on the feature dimensions by the multilayer perceptron. The _sigmoid_ activation function is used to activate the outputs of the summation to obtain the weights MC between zero and one. Unlike channel attention, the spatial attention module targets the temporal dimension by concatenating the pooled outputs and inputting them into a convolutional layer to extract features with high attentional weights in the temporal dimension, the convolution operation is utilised to emphasise the significance of the different time steps, and the _sigmoid_ activation function is used to activate the outputs of the summed outputs in order to obtain weights between zero and one MS. Connecting the channel-attention module and the spatial-attention module in a parallel way makes the modules consider the features of both the channel and spatial dimensions, avoiding the mutual influence of the channel and spatial dimensions and enabling the model to understand the multidimensional time series data better. This hybrid model helps the model capture key information about the data more comprehensively, which in turn improves the performance and generalisation of the model, as calculated in the following formula:  
\[Mc(F)=\sigma(MLP(AvePool(F))+MLP(MaxPool(F))) \tag{13}\]  
\[Ms(F)=\sigma(Conv(Conn[AvePool(F),MaxPool(F)])) \tag{14}\]  
\[F^{\prime}=Mc(F)\otimes F\otimes Ms(F) \tag{15}\]  
where \(\sigma\) represents the _sigmoid_ function, \(MLP\) is a multilayer perceptual machine, \(AvePool\) and \(MaxPool\) denote tie pooling and maximum pooling, \(Conv\) stands for convolution, \(Conn\) stands for connection, and \(Mc\) and \(Ms\) represent the outputs of channel attention and spatial attention, respectively.  
Figure 4: Structure of a HAM.