# Deep Transfer Learning Approach for Identifying Slope Surface Cracks
## 1 Introduction
Commonly used identification models include LeNet5 [23], AlexNet [24], GoogLeNet [25; 26], VGG [27], ResNet (Deep Residual Network) [28], DenseNet (Dense Connected Convolutional Network) [29], and MobileNets [30], etc. Recently, the Vision Transformer (ViT) [31] model was proposed for image identification and widely used. Much research has been conducted to identify cracks using deep neural networks. For example, [PERSON] et al. [32] applied the deep learning method to road crack detection for the first time using a deep convolutional neural network to identify the cracks in 500 3264 \(\times\) 2448 images of a strong noise environment (vehicles, tree shadows, etc.) collected by smartphones, which broadens the application scope of deep learning crack recognition. [PERSON] et al. [33] proposed an automated road crack detection method for three-dimensional asphalt pavement, which is an efficient detection framework based on a convolutional neural network (CrackNet). [PERSON] et al. [34] proposed a damage assessment method based on deep learning, which is more robust and accurate in the detection of concrete cracks than traditional image detection methods. Moreover, [PERSON] et al. [35] proposed the NB-CNN network framework, which fused a convolutional neural network (CNN) and Naive Bayes data. The approach aggregated each frame of information in the video for data fusion to further perform crack detection. The overall performance and robustness of the detection system were improved. [PERSON] et al. [36] compared six traditional edge detection methods with a deep convolution neural network (DCNN) in terms of crack detection effects. The DCNN shows greater advantages for calculation time and accuracy, indicating the superiority of the DCNN in crack detection. [PERSON] et al. [37] used smartphones in cars to obtain pictures to establish a dataset and successfully detected eight road damage scenes by training a convolutional neural network damage detection model. In addition, we compared the accuracy and operation speed of GPU server detection and smartphone detection and developed a smartphone application that has been put into use. In addition, [PERSON] et al. [31] applied the Transformer in the NLP field to the computer vision field and proposed a model (Vision Transformer, ViT) to train the image classification models under the condition of reducing the modification of the Transformer as much as possible. When the dataset is large, the training result of the ViT model is almost the same as that of current optimal convolutional network structures, and the computing resources required for training are significantly reduced. The emergence of the ViT promotes the application of Transformer in the field of computer vision. For example, [PERSON] et al. [38] proposed the ViT-CNN ensemble model to classify cancer cells images and normal cells images to assist in the diagnosis of acute lymphoblastic leukemia. The ViT-CNN integrated model achieves 99.03% classification accuracy on the test set, which is better than other models by experimental comparison. [PERSON] et al. (2019) presented an approach for the multi-label classification of remote sensing images based on data-efficient transformers. The validity of the model was also demonstrated by an experimental validation on two datasets with a ground resolution of 2 cm collected in the cities of Trento and Civezzano. However, currently, there is little research work focusing on the use of convolutional neural network models or Transformer methods for the identification of slope surface cracks, especially for the surface cracks of high and steep slopes. This is because deep learning methods have a high dependence on the amount of data, and a large amount of training data is required for deep learning model training. However, it is difficult to obtain crack images of high and steep slopes, and the data sample size is small. This leads to difficulties in the construction of deep learning models.