# Transfer Learning Model for Crack Detection in Side SlopesBased on Crack-Net
## 3 Materials and Methods
### Transfer Learning Strategy  
In the transfer learning scheme, a three-stage training strategy is adopted to minimize overfitting and optimize model performance:  
The early layers of the encoder are frozen, and the model is trained for 30 epochs with a learning rate of \(1\times 10^{-4}\).  
The Down 2.nn2 module is unfrozen, and the learning rate is reduced to \(5\times 10^{-5}\), with training conducted for 50 epochs.  
All layers are unfrozen, and a learning rate of \(1\times 10^{-5}\) is used for 20 epochs of fine-tuning. This phase aims to further optimize the model's performance for the slope crack detection task.  
The entire training process utilizes the AdamW optimizer with the CosineAnnealingLR learning rate scheduler. This scheduler gradually reduces the learning rate to mitigate overfitting and enhance model convergence and stability. A suite of data augmentation techniques is also employed to improve the model's robustness and generalizability, including random rotation, scaling, and brightness adjustment.  
In network structure adjustment, this paper proposes the augmentation of the number of channels and heads within the high-level attention module, intending to enhance the model's ability to effectively capture and interpret complex slope textures.  
The rationale for this staged transfer learning strategy is grounded in established best practices in deep learning. Initially freezing the early encoder layers allows the model to preserve generic low-level features acquired from large-scale pre-training datasets, which are typically transferable and beneficial for downstream tasks. By restricting updates to the later layers, the model can efficiently adapt to the specific characteristics of the slope crack dataset without disrupting foundational representations. Gradually unfreezing additional layers and reducing the learning rate in subsequent stages facilitates a smooth adaptation process, thereby minimizing the risk of catastrophic forgetting and overfitting. Furthermore, the use of a decaying learning rate schedule ensures stable convergence and assists the model in achieving a better local optimum. Overall, this parameter configuration is designed to balance the retention of pre-trained knowledge with effective task-specific adaptation, thereby maximizing the benefits of transfer learning for slope crack detection.  
Figure 5 depicts the transfer learning workflow employed in this study. First, the CrackNet model is trained on the publicly available CrackVision12K dataset to obtain a source-domain model. Through training on a large-scale, generic crack dataset, the source-domain model develops robust crack feature extraction capabilities. Subsequently, the weights of the source-domain model are transferred (weight shift) to the target task of slope crack detection. Specifically, the Crack-Net model pre-trained on the CrackVision12K dataset serves as the initial model and is further fine-tuned on the slope crack dataset, resulting in a model optimized for slope crack detection. This approach effectively leverages the rich features of the public dataset, thereby improving the accuracy and robustness of crack detection in specific scenarios, such as slope surfaces.