# Automated Ground Truth Generation for Learning-Based Crack Detection on Concrete Surfaces
## 4 Quantitative Evaluation Using Different Datasets  
We generated an evaluation dataset in which the samples were collected from the concrete images of the DeepCrack [40] and edge-based labeled crack image (ELCI) [41] datasets. Because the GTs of the crack images in these two datasets were labeled by their provider, they are suitable for computing the performance metrics accurately. The intersection over union (IoU) metric was adopted as the main indicator for quantifying the performance of our proposed method. We measured the percent overlap between the annotated GT and the second-round GT yielded by our method, as expressed in (9).  
\[\text{IoU}=\frac{\left(\text{Annotated GT}\right)\cap\left(\text{prediction result}\right)}{\left(\text{Annotated GT}\right)\cup\left(\text{prediction result}\right)} \tag{9}\]  
For crack segmentation tasks, the IoU can be calculated on the basis of the true positive (TP), false positive (FP), and false negative (FN) values at the pixel level for the crack class.  
\[\text{IoU}=\frac{\text{TP}}{\text{TP}+\text{FP}+\text{FN}} \tag{10}\]  
The second-round GTs obtained by our proposed method are regarded as the prediction result in calculating the IoU values. In addition to the IoU value, the precision, recall, and F1-score are also computed as follows:  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{11}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{12}\]  
\[\text{F1}-\text{score}=\frac{2\times\text{Precision}\times\text{Recall}}{ \text{Precision}+\text{Recall}} \tag{13}\]  
In the quantitative evaluation experiments, we acquired 200 concrete images and their GTs to form the evaluation dataset from DeepCrack and ELCI. Each image and its crack GT were pre-processed and normalized into \(448\times 448\) pixels. Subsequently, the proposed method was applied to obtain the second-round GTs for the concrete images of the evaluation dataset; hence, the performance metrics for 200 images were computed and averaged. As shown in Figures 14 and 15, the vanilla version of our proposed method performed well and was the least affected by overfitting; hence, it was selected as the  
Figure 16: The input to output mapping surface of the proposed FIS to derive the degree to which a pixel belongs to the crack class.  
decisive model for computing the performance indicators. Table 7 lists the results in terms of the IoU, precision, recall, and F1-score. Because the cracks were extremely irregular and the GT was labeled manually, a small tolerance margin between the annotated GT and the prediction result can be used to measure the coincidence between the detected cracks and the GT [42]. In Table 7, the margin of \(n\) pixels (\(n=1,\ 2,\ 3\)) was used, i.e., TP pixels were included within an \(n\)-pixel vicinity of the GT. The notation 0-pixel denotes that the tolerance margin is not utilized, whereas 1-pixel and 2-pixel indicate that the tolerance margins with 1 and 2 pixels were employed, respectively. As shown in Table 7, our proposed method with a vanilla architecture can achieve 94.4% precision when the tolerance margin is 2-pixel in the vicinity. Figure 17 shows five samples from the evaluation dataset, in which the upper, middle, and bottom rows represent the concrete images, the GTs labeled by humans, and the prediction results (second-round GTs obtained using our method), respectively.