## 4 Advanced Identification of Expansion Gap Using Machine Learning
### Expansion Joint Device Recognition  
Line-scan images have a high resolution of more than 10,000,000 pixels and require considerable memory and computational resources when being analyzed using a deep neural network. Therefore, it is necessary to divide the image into small image patches for realizing effective computations. Through the window sliding method, images were extracted every 1000 pixels in the longitudinal direction to generate a total of 19 image patches with a size of 1000 \(\times\) 1024 (see Figure 10). Each image patch is again converted to a size of 224 \(\times\) 224 pixels and subjected to binary classification.  
Each image patch is again converted to a size of 224 \(\times\) 224 pixels for binary classification. By using a CNN, an effective neural network for image analysis, we labelled patches with the expansion joint device as y = 1 and patches without the expansion joint device as y = 0 and classified them by logistic regression. The logistic regression followed by CNN infers the probability P that there is an expansion joint in the image patch. The error of the model prediction for each image patch is calculated with binary cross entropy [64, 65, 66, 67, 68, 69], which compares each of the predictions to the actual class, which can be either 0 or 1. The errors of each set of training data are summed up. In stochastic gradient methods [70, 71, 72, 73, 74, 75, 76, 77], the cost and sum of errors is used to update current model parameters to reduce the distance from the optimal point in the parameter space. The equation of binary cross entropy is shown as follows:  
\[L\ =\ -ylogP-(1-y)log(1-P) \tag{1}\]  
Figure 10: Line-scan image segmentation to search for expansion joint devices.  
Figure 9: Spacing measurement process of expansion joint device.  
Because class imbalance exists in a ratio of 1:9, a weight is assigned to each class.  
\[L_{weighted}\ =\ -\epsilon\times ylogP-\mu\times(1-y)log(1-P) \tag{2}\]  
where \(\epsilon=9.0\), \(\mu\ =\ 1.0\).  
The cost function \(J\) is calculated by averaging the errors for \(N\) training data and adding the L2 regularization to reduce overfitting [78, 79, 80, 81, 82, 83, 84, 85] of the model.  
\[J\ =\ \frac{1}{N}\sum_{j=1}^{N}L_{weighted}^{j}+\lambda|w|^{2} \tag{3}\]  
where \(\lambda=10^{-4}\).  
The gradient descent method updates the model parameters in the direction of reducing the cost function \(J\) as follows:  
\[w\ \Leftarrow\ w\ -\ \alpha\cdot g \tag{4}\]  
where \(\alpha=10^{-4}\), \(g=\frac{\partial I}{\partial w}\).  
We tested ResNet, MobileNet, and EfficientNet, all of which are well-known CNN architectures [22, 23, 24]. Gradient vanishing is more likely to occur as the layers of the deep learning model deepen, and ResNet solved this problem by performing residual learning using skip connection. The structure obtained high accuracy compared to the scale of the model. MobileNet proposed a depth-wise separable convolution that reconstructed the existing convolution method to reduce the computational amount of the model. Compared to the popular models at the time of proposal, the amount of computation was significantly reduced, and the same accuracy was maintained. EfficientNet, which empirically reports a methodology to increase model complexity to improve performance, updated the state of the art for benchmark datasets.  
We trained the three models to a binary classification of expansion joints and non-expansion joints (see Figure 11). Through the test, the EfficientNet model showed the highest performance, with recognition accuracy of 97.57% for the expansion joint device. Further, it was better to start learning from randomized initial parameters than through transfer learning for ImageNet. This is because the analysis image has different characteristics from the general characteristics of ImageNet. We employed EfficientNet for expansion joint detection, as it has the highest accuracy.