# Transfer Learning Model for Crack Detection in Side SlopesBased on Crack-Net
## 3 Materials and Methods
### Detection Model with Crack-Net
#### 3.1.3 Bi-Attention Fusion Module  
At each stage, the feature vectors in the encoder and decoder are connected and fused through a bidirectional attention fusion module (BAFM) to generate a clear and distinct crack boundary map. The decoder module was engineered to improve segmentation performance through the combination of feature vectors. Specifically, an attention gate mechanism is implemented to generate an attention mask, which is normalized through a sigmoid activation function. Following this, element-wise multiplication is performed on features requiring refinement; this process is similar to a filter, activating the features of the region of interest while suppressing irrelevant features. Finally, the attention mask generated using the features in the encoder is used as the attention coefficient and multiplied by the corresponding features in the decoder to activate the crack-related features and suppress non-crack features. As illustrated in Figure 3, this process enables the refinement of crack-related features, enhancing the clarity and precision of the resulting crack boundary map. In the context of feature extraction in encoders and decoders, where there are often mismatches in the number of feature channels across layers, a Deep Feature Fusion Module (DFFM) is proposed. This module aims to enhance the fusion of deep and shallow features, focusing on optimizing the extraction and integration of features. The DFFM integrates a Channel Attention (CA) module, a Multi-Regional Fusion Module (MRF), and the relevance enhancement operation. This module has been shown to effectively fuse and streamline the intermediate features extracted by CNNs and Transformers, enhancing the overall efficiency of interaction between disparate feature domains and significantly improving the expressiveness of crack features. Figure 4 illustrates that the module introduces an adaptive weighting mechanism for feature importance. This mechanism learns the importance  
Figure 3: Structural diagram of the bi-attention fusion module. weights of different feature maps through global average pooling and a fully connected layer. This process has been demonstrated to help suppress invalid features and highlight valid features. The attention mechanism employs a bidirectional interaction structure, capturing complementary information by calculating the attention matrix between two feature maps. This mechanism involves the calculation of the attention matrix through the inner product of Q and K, with features calculated separately for Q, K, and V. The values are subsequently weighted and summed using these attention weights to generate an enhanced feature map:  
\[ \begin{split}\text{Attention}_{y}&=\text{softmax} \bigg{(}\frac{\mathbf{Q}_{y}\cdot\mathbf{K}_{x}^{\top}}{\sqrt{c}}\bigg{)} \cdot\mathbf{V}_{x}\\ \text{Attention}_{x}&=\text{softmax}\bigg{(}\frac{ \mathbf{Q}_{x}\cdot\mathbf{K}_{y}^{\top}}{\sqrt{c}}\bigg{)}\cdot\mathbf{V}_{y }\end{split}  \tag{7}\]  
In these equations, \(Q_{x},K_{x},V_{x}\) and \(Q_{y},K_{y},V_{y}\) denote the query, key, and value matrices derived from feature maps \(x\) and \(y\), respectively. For a feature map (\(x\in\mathbb{R}^{B\times C\times H\times W}\)), these matrices are generated by applying separate linear projections (such as \(1\times 1\) convolutions) to \(x\), resulting in \(Q_{x},K_{x},V_{x}\in\mathbb{R}^{B\times d\times N}\), where \(d\) is the embedding dimension and \(N=H\times W\) is the number of spatial locations. The same procedure is applied to \(y\). Attention weights are calculated by performing a scaled dot product between the query and key matrices, followed by a softmax operation along the spatial dimension. The resulting attention map is then used to weight the value matrix, yielding an enhanced feature representation that captures both local and global dependencies. This bidirectional attention mechanism allows the DFFM to effectively integrate complementary information from different feature domains, thereby enhancing the accuracy and robustness of crack detection.