# Identification of the Surface Cracks of Concrete Based on ResNet-18 Depth Residual Network
## 3 Deep Learning Model
### Selection of Hyperparameters  
The model was compiled using the Adam optimizer and the cross-entropy loss function in this study. In the training process, the input of training set data leads to prediction results through forward propagation. The obtained results are subsequently compared with the actual labels, and loss is computed to update model parameters through back propagation, as depicted in Figure 6. The model's performance is assessed by calculating the loss and accuracy on the validation set, which serves as an indicator of its generalization ability. The hyperparameters are fine-tuned and the model architecture is refined based on the performance of the model on the validation set and the weight file obtained from the initial round of training, with the aim of optimizing and enhancing its overall performance. Upon completion of model training, the performance of the model is assessed by evaluating it using the test dataset to obtain objective metrics such as recognition accuracy and loss rate. The incorporation of the rectified linear unit (ReLU) activation function in this study greatly amplified the expressive capacity, generalization capability, and training efficiency of neural networks through the introduction of nonlinearity, facilitation of sparse activation patterns, and mitigation of gradient vanishing problems, while ensuring stability and expedited computation. The mathematical expression of ReLU is as follows:  
\[\text{ReLU}=\left\{ \begin{array}{l}X,x>0;\\ 0,x\leq 0.\end{array} \right. \tag{2}\]  
The function expression demonstrates that the rectified linear unit (ReLU) effectively sets all values less than or equal to zero, while preserving values greater than zero. The model learning rate is a crucial hyperparameter employed for updating the model parameters during the training process of a neural network. It determines the degree to which the product of the gradient and learning rate influences parameter changes in each update. If the learning rate is set too low, it will significantly decelerate parameter updates,  
Figure 6: Training process of the model. thereby impeding the training process and potentially obstructing convergence towards the optimal solution. Excessively high learning rates can lead to a rapid and unstable pace of parameter updating, potentially impeding the training process or even causing failure to converge. The learning rate formula of the piecewise constant decay formula can be expressed as:  
\[lr=lr_{0}\cdot\gamma^{\left(\text{int}\left(T/T^{\prime}\right)\right)} \tag{3}\]  
where \(lr\) is the current learning rate, \(lr_{0}\) is the initial learning rate, \(\gamma\) is the attenuation factor, \(T\) is the current training cycle (epoch), and \(T^{\prime}\) is the number of cycles in each stage. The Adam optimization function was employed in this study to dynamically adjust the learning rate during training, enabling adaptation to gradient changes across different parameters. This approach accelerates convergence speed and facilitates quicker attainment of improved convergence effects. The Adam optimization function is a gradient-based optimization algorithm that utilizes estimations of first-order and second-order moments. The algorithm exhibits exceptional adaptability to handling large-scale datasets and sparse gradients, making it highly effective in tackling optimization challenges encountered in such scenarios. Furthermore, the algorithm can be easily implemented by solely storing the first- and second-moment estimations of each parameter, without requiring any additional memory for past gradient storage. Its mathematical expression is as follows:  
\[mt=\beta_{1}\cdot m+(1-\beta_{1})\cdot\delta \tag{4}\]  
\[vt=\beta_{2}\cdot v+(1-\beta_{2}\cdot\delta^{2}) \tag{5}\]  
In this context, \(mt\) represents the first-order moment and \(vt\) represents the second-order moment. \(\delta\) represents the instantaneous gradient.