## 2 Evaluation of Pavement Crack Damage by Deep Learning
### CNN Training for Accurate Detection of Cracks in Pravements
#### 2.2.2 Training of the CNN  
In this project, we are going to construct a CNN that analyzes small segmented images to judge if the image is a crack or not, as shown in Figure 5. It would be much easier and simpler if we could classify the output with the mere presence/absence of cracks. The result of our initial analysis, however, included false detections in images, where road markings or utility holes were detected as cracks. We, therefore, set six classes as shown in Table 1 to categorize the target images. These six classes of image are used to train the model. In general, the accuracy of deep learning models can be expected to improve as the amount of training data increases. However, simply increasing the amount of training data does not guarantee high accuracy, since the quality of the training data also affects the accuracy. In this study, we tried to develop a framework to improve the accuracy of the model efficiently by increasing the amount of data, selecting data that contributes to the improvement of accuracy. Specifically, we considered that adding images that are likely to be mistaken by the CNN model to the training data would be more efficient than randomly adding training data. The flowchart is shown in Figure 6. First, we perform training on a small randomly selected training data set. This is the initial model. We then let the model classify the new data not used for training (Verification A). Then, after the analysis in verification A is performed, images that could not be properly classified are added to the training dataset. The above flow is considered to be one cycle, and this cycle is repeated. In addition to the data to be added to the training data (the data used for verification A), we also prepare data for calculating the accuracy of the model (verification B). Then, when the accuracy converges, the repetition is stopped. In this study, we decided to stop when the accuracy did not improve for three consecutive cycles. Figure 5: The flow of training and analysis with the CNN in this study. In the case of the images used in this study, it took nine cycles for the accuracy to converge. Table 2 shows the amount of training data in each cycle, the accuracy in verification A and that in verification B. The accuracy was evaluated based on the existence or absence of cracks. For example, mistaking class 3 in Table 1 for class 1 does not count as a mistake. In this study, verification A used 30,000 pieces of data per cycle, and verification B used 20,000 pieces of data per cycle. The data in verification A was then added anew in each cycle. In the training, the ratio of training data to validation data was 4:1, and the number of epochs is 100. The final model is when the validation loss is minimized. In the final model, training loss and validation loss do not diverge, and there is no clear evidence \begin{table}
\begin{tabular}{c c c} \hline
**Class Number** & **Description** & **Sample Image** \\ \hline
1 & Cracked area & \\
2 & Non-cracked area & \\ \hline
3 & The area including road markings & \\  & like white lines (with cracks) & \\ \hline
4 & The area including road markings & \\  & like white lines (without cracks) & \\ \hline
5 & The area including road facilities & \\  & like utility holes or bridge joints & \\  & without white lines (with cracks) & \\ \hline
6 & The area including road facilities & \\  & like utility holes or bridge joints & \\  & without white lines (without cracks) & \\ \hline \end{tabular}
\end{table} 
Table 1: Image classification. of overfitting. Note that the images used here and in the following Section 3 were taken in the same city, but show different routes. The improvement of the analysis results is shown in Figure 7. For better understanding of the figure, we do not overlap the images when dividing them, but simply divide them into 256 \(\times\) 256 segments.