# Pavement Distress Detection Using Three-Dimension Ground Penetrating Radar and Deep Learning
## 2 Proposed Approaches
### Learning Strategy  
(1) Overall  
The weights and bias of the location and classification models were updated by the backward propagation algorithm [14] as follows: \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Patch Pixel** & **Layer** & **Hidden Size** & **MLP Size** & **Heads** & **Params** & **FLOPs** \\ \hline SViT-3 & \(16\times 16\) & 3 & 256 & 1024 & 4 & 2.49 M & 1.09 G \\ SViT-6 & \(16\times 16\) & 6 & 256 & 1024 & 4 & 4.86 M & 2.15 G \\ SViT-9 & \(16\times 16\) & 9 & 256 & 1024 & 4 & 7.23 M & 3.21 G \\ SViT-12 & \(16\times 16\) & 12 & 256 & 1024 & 4 & 9.6 M & 4.27 G \\ SViT-15 & \(16\times 16\) & 15 & 256 & 1024 & 4 & 11.96 M & 5.33 G \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Detailed parameters of the 5 SViT models. Figure 14: Activation function of GELU. \[\text{W}_{\text{ij}}(\text{q}+1) =\text{W}_{\text{ij}}(\text{q})-\eta\frac{\partial\text{L}}{ \partial\text{W}_{\text{ij}}(\text{q})}, \tag{8}\] \[\text{b}_{\text{j}}(\text{q}+1) =\text{b}_{\text{j}}(\text{q})-\eta\frac{\partial\text{L}}{ \partial\text{b}_{\text{j}}(\text{q})}, \tag{9}\]  
where \(\frac{\partial\text{L}}{\partial\text{W}_{\text{ij}}(\text{q})}\) and \(\frac{\partial\text{L}}{\partial\text{b}_{\text{j}}(\text{q})}\) were the gradients of the loss function with respect to weight \(\text{W}_{\text{ij}}(\text{q})\) and bias \(\text{b}_{\text{j}}(\text{q})\), and \(\eta\) was the learning rate. The convergence of the model can be accelerated by using different learning rates during different training stages. In this study, an exponential descent function was used to represent the learning rate:  
\[lr=lr_{b}\times\gamma^{epoch}, \tag{10}\]  
where \(lr_{b}\) was the base learning rate; \(\gamma\) was the coefficient of learning rate decay. The location and classification models were trained in TensorFlow 2.5 framework. The training device was a cloud server with AMD EPYC 7302 CPU, 64 G RAM, and NVIDIA GeForce RTX 3090 GPU with 24 GB memory. The testing device was a laptop with Intel i7-9750H CPU, 16 G RAM, and NVIDIA GeForce GTX 1650 GPU with 4 GB memory. Training of CP-YOLOX  
In the learning strategy, three different loss functions were combined in the CP-YOLOX model to compute the gaps between predicted and target information, including the decision of anomalous waveform or not, confidence, and location of the prediction boxes. The loss of category and confidence was defined by the cross-entropy loss function, and the loss of box position was computed by the CIoU loss function [39] as follows:  
\[\text{L}_{\text{CIoU}} =1-\text{IoU}+\frac{\rho^{2}\big{(}b,\ b^{gt}\big{)}}{c^{2}}+ \alpha v, \tag{11}\] \[\alpha= \frac{v}{(1-\text{IoU})+v},\] (12) \[v= \frac{4}{\pi^{2}}\left(arctan\frac{w^{gt}}{h^{gt}}-arctan\frac{ w^{p}}{h^{p}}\right)^{2}, \tag{13}\]  
where IoU was the intersection over union between the prediction and truth boxes; \(b\) and \(b^{gt}\) represent the center points of the prediction and truth boxes, respectively; \(\rho\) was the Euclidean distance between the two center points; \(c\) represents the diagonal distance that can contain the minimum closed area of both the prediction and truth boxes; \(v\) represents the similarity of the aspect ratio; \(w^{gt}\), \(h^{gt}\), \(w^{p}\), and \(h^{p}\) were the length and width of the truth and prediction boxes; \(\alpha\) was the weight parameter. The proposed CP-YOLOX model was trained in three stages, and each stage had 100 epochs. The values of \(lr_{b}\) in the three stages were \(1\times 10^{-3}\), \(1\times 10^{-4}\), and \(5\times 10^{-5}\), while \(\gamma\) was 0.97, as shown in Figure 15a. The Adam optimizer [40] was used for training, and the parameters of the optimizer were 0.9 and 0.999, as shown in Table 7. The training of the model was terminated when the loss value of the validation set could no longer be reduced after 20 epochs.