## 4 Advanced Identification of Expansion Gap Using Machine Learning
### Gap Identification Verification  
Based on the abovementioned results of AI-based gap identification, we randomly selected 10,526 places among 12,825 expansion joint device big-data images obtained previously to determine the discrimination of the expansion joint device gap. After dividing and refining 10,526 line-scan images into 19 image patches, 289,495 sets of training data and 45,950 tests of the classification model were constructed. A total of 21,604 sets of training data and 4174 of test data of the segmentation model for measuring the expansion joint gap were refined. The results are presented below for each expansion joint device type. The result of the position where the minimum spacing was measured is indicated by a red line. For rail-type joints in which several gaps appear at once, the starting and end gaps of the part with the smallest actual gap value are indicated by red lines (see Figure 15).  
We used Python 3 and TensorFlow 2 to implement and train a deep learning model using CNN, and development frameworks such as TensorFlow and PyTorch support libraries for implementing popular CNN layers and to support learning using GPUs.  
We used a single NVIDIA Tesla V100 graphics card and Tensorflow to accelerate the training of the model. The EfficientNet B0 model for classification of expansion joints completed training in less than 30 epochs and took up to 4 h. A total of 259,495 training images and 30,000 validation images were used. Overall, 45,950 images for testing did not participate in the training. The U-Net model for gap region extraction completed training in less than 20 epochs and took up to 4 h, and 19,304 training images and 2300 validation images were used, while 4174 images for testing did not participate in the training.  
In the environment for field application after training, one NVIDIA RTX 3070 graphics card and Tensorflow were used. In one line-scan image (1024 \(\times\) 10,000), it took an average of 0.6 s to find the expansion joint and measure the gap.  
After comparing the identification rate results from the existing traditional algorithm, we found that the finger type, which had the lowest discrimination accuracy, improved the most, by 41% to an accuracy of approximately 92%, and the overall average identification accuracy improved by 27.5% to 95%. Even though the identification accuracy was greatly improved, if a foreign substance existed inside the expansion gap, if a sealing agent was applied to the surface for preventing water leakage, if the body was damaged, and so on, these were included in the discriminant errors and manual correction would be required (see Table 8).  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Accuracy (\%)** & **Average** & \begin{tabular}{c} **Mono Cell** \\ **Type** \\ \end{tabular} & \
\begin{tabular}{c} **Finger** \\ **Type** \\ \end{tabular} &
\begin{tabular}{c} **Rail** \\ **Type** \\ \end{tabular} & **Others** \\ \hline Conventional algorithm & 67.5 & 71 & 51 & 87 & 61 \\ \hline AI algorithm (machine learning) & 95 & 98 & 92 & 99 & 91 \\ \hline Improvement rate & \(\uparrow\)27.5 & \(\uparrow\)27 & \(\uparrow\)41 & \(\uparrow\)12 & \(\uparrow\)30 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Accuracy of expansion gap identification using machine learning algorithm.  
Figure 14: Pseudocode to obtain minimum gap distance from U-Net output.  
Figure 15: Machine-learning-based verification results of expansion joint device gap identification: (**a**) original image; (**b**) extraction of expansion joint area image; (**c**) extraction of gap area image; (**d**) final measurement result image.