## 4 The Application of CNN and Other Deep Learning Algorithms
The net input \(\mathbf{z}^{(l)}\) of layer \(l\) is the activation value \(\mathbf{a}^{(l-1)}\) of layer \(l-1\), and convolution with kernel \(\mathbf{w}^{(l)  
A typical convolutional network is formed by cross-stacking convolutional layers, pooling layers, and fully connected layers. A convolutional block is consecutive \(M\) convolutional layers and \(b\) pooling layers. In a convolutional network, \(N\) consecutive convolutional blocks can be stacked, followed by \(K\) fully connected layers. As an end-to-end algorithm, the concept of CNN has been proposed for a long time, developing from 1D structure to 2D and 3D structure. In addition, deep learning is a subset of machine learning. When the deep CNN trained by [PERSON] and his group achieved record-breaking performance in the ILSVRC-2012 [57], it attracted great attention from researchers in different fields, and CNN has gradually become one of the most popular models of deep learning since then. CNN was first reported to be applied in GPR classification in 2015; [PERSON] et al. utilized a CNN consisting of two convolutional layers and corresponding max-pooling layers in the classification of buried explosive hazards (BEH). They applied heuristics, including cross-validation, network weight regularization, and \"dropout\" to prevent over-training, to the CNN structure [58]. Almost at the same time, another study also presented the results of using a simplified CNN in the recognition of subsurface target [59]. This study tested different parameters of the network and provided some suggestions on parameter settings. Both of these studies showed that CNN had a promising performance in feature extraction and classification. In 2016, [PERSON] et al. compared the performance of CNN method and traditional algorithms including EHD and HOG for classification of BEH and false alarms [60]. It was shown in this paper that CNN outperformed the traditional methods in the probability of detection, and data augmentation could improve the performance of both methods significantly. In 2018, a nine-layer convolutional neural network developed by [PERSON], [PERSON] et al. can identify six materials with 80% accuracy in uneven underground media [61]. There are also some researchers who combine CNN with traditional machine learning methods. In 2020, [PERSON] et al. combined CNN with SVM to build the CSVM structure. CSVM has multiple convolutional and pooling layers and a single SVM classifier. Since the SVM layers are learned with an unsupervised approach, there is no need for backpropagation algorithm to update the weights. The proposed method improves the classification performance with low computational complexity [62]. The network structure of [62] is shown in Figure 12.  
Figure 11: Convolutional layer. Figure 10: Fully connected layer. The lack of real data for training, which can lead to an over-training problem, is always a big challenge of CNN application in GPR data. Therefore, cross-validation, weight regularization, dropout, pretraining, data augmentation, and other techniques to prevent over-training have been widely used in CNN structure for GPR. In 2017, [PERSON] et al. pretrained a CNN on the Cifar10 dataset and a dataset of high-resolution aerial imagery and showed that transferring weights from a pretrained CNN and fine-tuning the parameters could improve detection performance [63]. Another work in 2017 also adopted pretraining and data augmentation to CNN for BTD problems in GPR data [64]. [PERSON] et al. proposed an idea that CNN can be trained on synthetic data generated by gprMax [65]. In addition, they found that adding some background GPR acquisitions to the dataset can improve the rate of detection. In addition, a CNN-based system can perform better with traditional preprocessing methods. [PERSON] et al. proposed a system combining CNN with conventional image processing methods, including migration and thresholding; this system could reach more than 95% in detection accuracy of rebars in bridge decks [66].