# Adaptive Attention-Enhanced Yolo for Wall Crack Detection
## 3 Materials and Methods
### Swin Transformer-Based CSP Bottleneck with Three Convolutions  
In-depth analysis of image semantics and structure is an essential step in wall crack detection. In the backbone, we introduced the STC3 module, inspired by the structural principles of Swin Transformer, to improve feature extraction and better utilize input features. This module replaces the CSP Bottleneck, which originally used three convolutions.  
The STC3 structure mainly contains Layer Normalization (LN), MSA, and Multilayer Perceptron (MLP). This design allows STC3 to efficiently extract and fuse image features when dealing with tasks such as object detection [37].  
First, the normalized data \(I_{L1}\) are generated by LN of the input images for each channel. This process is designed to reduce fluctuations in the internal covariates, thereby speeding up the model's training and improving its generalization ability [38]. The normalized calculation formula is as follows.  
\[I_{L1}=\frac{I-\mu_{1}}{\sqrt{\sigma_{1}{}^{2}+\varepsilon_{1}}} \tag{1}\]  
In Formula (1), the input feature data are represented by \(I\), \(\mu_{1}\) represents the mean of every single layer, \(\sigma_{1}{}^{2}\) refers to the variance of each layer, and \(\varepsilon_{1}\) is a tiny constant used to avoid division by zero errors. The processed image data are then fed into the MSA, which is designed to capture long-range dependencies \(I_{MSA}\) in the image to help extract detailed image features and background information. In this process, the normalized data \(I_{L1}\) are converted into learnable parameter matrices, including query (Q), key (K), and value (V) [39].  
\[\left\{\begin{array}{l}Q=I_{L1}M^{Q}\\ K=I_{L1}M^{K}\\ V=I_{L1}M^{V}\end{array}\right. \tag{2}\]  
In Formula (2), \(M^{Q}\), \(M^{K}\), and \(M^{V}\) are three trainable parameter matrices. Then, the attention weight matrix is calculated from these matrices.  
\[A(Q,K,V)=f_{5}\left(\frac{QK^{T}}{\sqrt{\varepsilon_{k}}}\right)V \tag{3}\]  
Figure 1: The network structure of the improved Adaptive Attention-Enhanced Yolo model, including STC3, ARF, and HA components.  
In Formula (3), \(e_{k}\) refers to the dimension of the key vector, and \(f_{S}(*)\) represents a normalization function that makes the sum of the three arguments equal to 1. Generally, the value of \(i\) ranges from 1 to 8. The stitching process of the AM is repeated several times [40].  
\[m_{i}=A\Big{(}QM_{i}^{\text{\tiny{$\text{Q}$}}},KM_{i}^{K},VM_{i}^{V}\Big{)} \tag{4}\]  
\[MH(Q,K,V)=Contact(m_{1},\cdots,m_{h})M^{\text{\tiny{$\text{O}$}}} \tag{5}\]  
\[I_{MSA}=MH(Q,K,V) \tag{6}\]  
On this basis, the obtained feature map \(I_{MSA}\) is spliced with the original image to obtain \(I_{\text{\tiny{$\text{C1}$}}}\). In this way, the model is able to take full advantage of both types of information for deeper feature learning. The equation is shown below.  
\[I_{\text{\tiny{$\text{C1}$}}}=Concat(I_{MSA},I) \tag{7}\]  
The feature graph obtained in the previous step is also normalized by a layer, thus a more normalized feature \(I_{\text{\tiny{$\text{L2}$}}}\) is obtained, and its calculation formula is as follows.  
\[I_{\text{\tiny{$\text{L2}$}}}=\frac{I_{\text{\tiny{$\text{C1}$}}}-\mu_{2}}{ \sqrt{{\sigma_{2}}^{2}+\varepsilon_{2}}} \tag{8}\]  
In Formula (8), \(\mu_{2}\) represents the mean of each layer, and \({\sigma_{2}}^{2}\) represents the variance of each layer. After this, an MLP processes the feature map to extract crucial data relevant to the task, which are represented by \(I_{MLP}\). The calculation formula of this process is given below.  
\[I_{MLP}=f_{M}(I_{\text{\tiny{$\text{L2}$}}}) \tag{9}\]  
In Formula (9), \(f_{M}(*)\) represents the MLP. In this way, the STC3 network can fully integrate the advantages of the self-attention and the MLP to effectively extract and utilize the complex features in the wall crack image. This process improves the generalization ability and stability of the model, and makes the crack detection results more accurate.