# A Highway Pavement Crack Identification Method Based on an Improved U-Net Model
## 3 Methods
### Improved Loss Function  
So that the improved model does not suffer from loss non-convergence in training, the loss function was improved in this section. The cross-entropy loss function is often used as a loss function for segmentation networks, and it can be used in either binary classification tasks or multi-classification tasks. The output of this paper contained only two types of crack regions and other background regions, which belong to the binary classification task. The probability of predicting a pixel point as a crack is assumed to be \(q\), and the probability of predicting it as a background is \(1-q\). At this time, the cross-entropy loss L is defined as shown in Equation (9):  
\[L=\frac{1}{N}{\sum_{i}L_{i}}=\frac{1}{N}{\sum_{i}}-\left[z_{i}\cdot\log(q_{i} )+(1-q_{i})\cdot\log(1-q_{i})\right] \tag{9}\]  
In the formula, \(N\) represents the total number of samples, \(z\) represents the classification of sample \(i\), and \(q\) refers to the probability that the sample is predicted to be cracked. The cross-entropy loss function gives the same weight to the task of whether the sample is cracked or background. However, usually the crack-occupied area in the pavement crack image is small, resulting in a serious imbalance in the number of positive and negative samples. For this problem, this section used the weighted loss function Focal Loss as the loss function of the multiscale U-Net network. The weighted loss function is mainly improved on the basis of the cross-entropy loss function, and the improved loss function will improve the model's focus on hard-to-classify samples and positive samples. The weighted loss function adds a new factor \(\alpha\)\(\alpha\in[0,1]\). By adjusting the value of the factor \(\alpha\) to control the size of the weight of positive and negative samples for the overall loss, the smaller the value of \(\alpha\) that can reduce the weight of negative samples, the more improved the cross-entropy loss \(CE(q_{t})\), as shown in Equation (10):  
\[CE(q_{t})=-\alpha_{t}\log(q_{t}) \tag{10}\]  
\[\frac{\alpha}{1-\alpha}=\frac{n}{m} \tag{11}\]  
where \(m\) and \(n\) represent the number of positive and negative samples. The value of \(\alpha\) is determined by the ratio of the number of positive and negative samples. The weighted loss function incorporates a moderation factor, \((1-q_{t})^{\gamma}\), which serves to adjust the weights of the easy-to-classify samples and the hard-to-classify samples. When a sample is misclassified, it is considered a hard-to-classify sample, and the correct prediction probability \(q_{t}\) converges to 0. Then, the conditioning factor \((1-q_{t})^{\gamma}\) converges to 1. Conversely, when \(q_{t}\) converges to 1, the sample is considered as an easily classifiable sample and \((1-q_{t})^{\gamma}\) converges to 0. \[FL(q_{t})=-\alpha_{t}(1-q_{t})^{\gamma}\log(q_{t}) \tag{12}\]  
By doing so, the effect of the loss value of the easily classified samples on the overall loss value was attenuated. A Crack Segmentation Model for Road Pavements Based on Multi-Scale Structure and Attention Mechanism U-Net  
The proposed U-Net model for multi-scale feature prediction fusion was combined with the parallel attention module to construct a multi-scale attention mechanism U-Net pavement crack identification model, whose structure is shown in Figure 10. The improved structure of the bottleneck layer is shown in Figure 11  
The pavement crack image was input from this network model. Firstly, it passed through the encoder part, where the features were further extracted at the bottleneck layer; then, the image passed through the decoder part, the up-sampled output was fused together by multi-scale feature prediction, and the aggregated output was finally processed by the new parallel attention module to further optimize the crack features. Finally, the output of the new parallel attention module was processed by the sigmoid layer to classify each pixel in the image as crack or background to achieve the recognition of pavement cracks.