## 3 Neural Network-Based Bridge Maintenance Cost Prediction Framework
### CNN Model  
CNN is currently one of the most widely used deep learning algorithms. Compared to the fully-connected ANN, the most important features of CNN are local perception and parameter sharing [48]. The main parts of a CNN are the convolutional layer, pooling layer, and activation function [49]. CNN reduces the complexity of the network model through three strategies: a local receptive field, weight sharing, and downsampling. They are widely used in the fields of image classification, speech recognition, and intelligent prediction [50].  
CNN has proven to be a reliable technique for extracting hidden features because it can complete the automatic creation of filters [51]. By using the convolution filter to act on all the feature spaces of the input data, for each perception area, the data at the corresponding position are multiplied by the weight in the convolution filter, and then summed as the input of the next layer of neurons. In the same convolution layer of CNN, all neurons are set to have no connection and only the weight of the filter is shared. Therefore, CNN is more effective than multi-layer perceptron training with the same structure. Each convolution layer can be expressed as [52]:  
\[y_{i}=f\Big{(}\sum_{i=1}^{n}W_{i}x_{i}+b\Big{)} \tag{19}\]  
Figure 5: The calculation process chart for the fully-connected ANN model.  
where \(y_{i}\) indicates the results after convolution, f denotes the activation function, \(W_{i}\) indicates the weights of the convolution filter, b indicates the bias terms.  
The CNN model was built based on Python and TensorFlow in this study. On the basis of the aforementioned fully-connected ANN model, a convolution operation was added, and the other parts were similar to the fully-connected ANN model. The CNN consisted of an input layer, a convolutional layer, a flattening layer, a fully connected layer, and an output layer. The convolution kernel size was \(2\times 1\), the channels of the convolutional layer were 64. After convolution, the number of nodes changed from 9 to 576, realizing the deep extraction of features. In view of the few input parameters of this model, pooling operation was not performed.  
ReLU was used as the activation function of the convolutional layer and the fully connected layer. Tanh was used as the activation function of the output layer. Adam [53] was the optimization algorithm in this study.  
Adam can be defined as  
\[m_{t} =\mu\times m_{t-1}+(1-\mu)\times g_{t} \tag{20}\] \[n_{t} =\
u\times n_{t-1}+(1-\
u)\times g_{t}^{2}\] (21) \[\hat{m_{t}} =\frac{m_{t}}{1-\mu^{t}}\] (22) \[\hat{n_{t}} =\frac{n_{t}}{1-\
u^{t}}\] (23) \[\theta_{t} =-\frac{\hat{m_{t}}}{\sqrt{\hat{n_{t}}}+\epsilon}\times\eta \tag{24}\]  
where \(\mu\), \(\
u\) indicate the exponential decay rates for the moment estimates, \(g_{t}\) indicates the gradients of stochastic objectives at timestep \(t\), \(m_{t}\) indicates the biased first moment estimation of gradients, \(n_{t}\) indicates the second raw moment estimation of gradients, \(\hat{m_{t}}\) and \(\hat{n_{t}}\) respectively, indicate the unbiased estimate after correcting \(m_{t}\) and \(n_{t}\), \(\theta_{t}\) indicates the result parameter. After Bayesian optimization, the hyperparameters in the neural network are \(\eta=0.001\), \(\mu=0.9\), \(\
u=0.999\) and \(\epsilon=10^{-8}\). The basic flow of the above calculation process is shown in Figure 6:  
Figure 6: The calculation process chart for the CNN model.