# Automated Ground Truth Generation for Learning-Based Crack Detection on Concrete Surfaces
## 3 Implementation and Experiments
### Crack Detection Models Based on U-Net  
In the present study, a U-Net-based model was implemented because it is superior to other conventional methods, such as CrackTree [37], CrackIt [38], and CrackForest [39]. In Section 2.2, a hybrid architecture of the U-Net and VGG16 was introduced to perform per-pixel crack segmentation. It is noteworthy that the U-Net encoder can be replaced by different backbones. Hence, we used the ResNet [21] for the encoder portion of the U-Net (the left half in Figure 6, including the blocks named Conv-1 to Conv-5). Table 5 summarizes the complete compositions of the encoder replaced by ResNet-18, 34, 50, and 101. Therefore, the vanilla version was compared with four U-Net-based models that involve different ResNets in this study. We named them Res-U-Net-18, Res-U-Net-34, Res-U-Net-50, and Res-U-Net-101. To evaluate the performance of these five models, we used the dataset introduced in Table 2 to train each model. Before implementing our proposed algorithm, all the images were normalized to a size of \(448\times 448\) pixels in advance because the width and height of the input images must be a multiple of 32 (the limitation of using the U-Net-based model). The main procedure of automated data labeling for obtaining the second-round GT is described below:  
1. Perform the algorithm of the first-round GT generation proposed in Section 2.1. 2. Pre-train the U-Net-based models, including the vanilla, Res-U-Net-18, Res-U-Net-34, Res-U-Net-50, and Res-U-Net-101 models, separately. The hyper-parameters used during this training stage are the same as those introduced in Section 2.2. 3. Use each learned model to obtain the crack prediction results of the training data. 4. On the basis of the prediction results, obtain the second-round GTs using the refinement scheme presented in Section 2.3. 5. Finally, use the second-round GTs to re-train the five pre-trained models separately. Hence, U-Net-based crack detection models with different types of encoders are obtained. For analyzing the time consumption of each model, we first investigated the number of model parameters and the computation time per image (sized of \(448\times 448\) pixels). As listed in Table 6, the Res-U-Net-18 model had the fastest execution time and the smallest number of parameters, whereas the vanilla architecture's execution time was at mid-level. Figure 14 shows the per-epoch trend of training and validation losses, trained by the second-round GTs. According to a comparison of the training loss, the Res-U-Net-101 model outperformed the others because it exhibited the most complicated architecture; however, it performed the worst in terms of validation loss. This is because the image features were memorized rather than learned in that model, i.e., overfitting occurred. As shown by the plots, the vanilla structure is suitable for crack detection in our study. Figure 15 shows several examples of the proposed models, in which the first and second columns show the original images and their first-round GTs, respectively, whereas those shown from the third to the last columns are the second-round GTs for the vanilla, Res-U-Net-18, Res-U-Net-34, Res-U-Net-50, and Res-U-Net-101 models, respectively. As shown by the results of the second-round GTs, the vanilla and Res-U-Net-34 yielded similar results. The vanilla U-Net-based model exhibited good performance in crack detection problems. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multirow{2}{*}{**Backbones**} & \multicolumn{3}{c}{**Time (Unit: ms)**} & \multicolumn{2}{c}{**Number of Model**} \\ \cline{3-6}  & & **Min.** & **Max.** & **Avg.** & **Parameters** \\ \hline Vanilla & VGG16 & 49.5 & 50.4 & 49.7 & 29,306,465 \\ Res-U-Net-18 & ResNet-18 & 41.9 & 42.8 & 42.2 & 25,009,737 \\ Res-U-Net-34 & ResNet-34 & 44.4 & 44.8 & 44.5 & 35,117,897 \\ Res-U-Net-50 & ResNet-50 & 56.1 & 56.7 & 56.4 & 57,677,897 \\ Res-U-Net-101 & ResNet-101 & 63.9 & 64.5 & 64.1 & 76,670,025 \\ \hline \hline \end{tabular}
\end{table}