## 4 Experiments
### Highway Crack Detection Experiments
#### 4.1.2 Evaluation Metrics and Experimental Parameter Settings  
For highway crack detection, each image can be evaluated by comparing the crack detection map with the manually labeled ground-truth map to calculate the precision and recall rates. In addition, the F-Score can be used as an overall metric for performance evaluation. The F-Score is calculated as follows:  
\[\text{F-Score}\,=\,\left(1+\beta^{2}\right)\cdot\frac{\text{Precision}\,\cdot\, \text{Recall}}{\beta^{2}\,\cdot\,\text{Precision}\,+\,\text{Recall}} \tag{7}\]  
where Precision is the proportion of true positive samples among the predicted positive samples, which may contain negative samples. Recall is the proportion of true positive samples without any negative samples that are correctly predicted. \(\beta\) is the weight to balance Precision and Recall in the F-score calculation. Precision and recall are calculated using the following formulas:  
\[Precision=\frac{TP}{TP+FP} \tag{8}\]  
\[Recall=\frac{TP}{TP+FN} \tag{9}\]  
where \(TP\) denotes true positives (pixels classified as cracks and correctly identified), \(TN\) denotes true negatives (pixels classified as non-cracks and correctly identified), \(FP\) denotes false positives (pixels classified as cracks but actually not), and \(FN\) denotes false negatives (pixels classified as non-cracks but actually are).  
For subsequent evaluation of the results, probabilistic binarization of the crack detection result map is required to obtain a binary map. Therefore, a threshold value \(\eta\), which can also be called the confidence level, is set to 0 for positive cases and 1 for negative cases, if it is greater than \(\eta\).  
In this experiment, three different metrics based on F-Scores are used to evaluate the experimental results: optimal dataset scale (ODS), optimal image scale (OIS), and average precision (AP) where ODS is also known as global optimal, fixed contour threshold, and optimal on detection metric dataset scale. That is, the same threshold is set for all images such that the F-Score of the whole dataset is the maximum. OIS is also known as single-image best, best threshold on each image, and optimal on image scale, i.e., a different threshold, \(\eta\), is selected on each image that maximizes the F-Score of that image. AP, average accuracy, is the integral of the PR curve. Since it is difficult to integrate over the PR curve, AP is found by sampling the mean value over the PR curve.  
After preliminary experiments, the final detailed settings of each hyperparameter are listed in the following Table 2. In addition, this experiment uses AdamW Optimizer to dynamically adjust the learning rate of each parameter and update the network weights. To minimize unnecessary computational resource loss, an early stopping strategy is used during the training process to stop the training when the loss on the test set is no longer decreasing.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Original Images** & **Ground Truth** \\ \hline Training set & 4443 & 4443 \\ Validation set & 1480 & 1480 \\ Test set & 1480 & 1480 \\ Total & 7403 & 7403 \\ \hline \hline \end{tabular}
\end{table}
Table 1: UAVRoadCrack dataset.