# Adaptive Attention-Enhanced Yolo for Wall Crack Detection
## 3 Materials and Methods
### Multi-Head Attention  
Images of wall cracks have a variety of complex features and the model needs to have the ability to capture this information accurately. In the prediction part of Yolo, the addition of a Multi-Head Attention (MHA) structure can help to capture more image information, which to some extent solves the problem that the unified convolutional neural network cannot comprehensively extract effective feature information. This introduced structure contains attention mechanisms that can handle features of different dimensions, which are then combined with the original image to obtain feature information that is more favorable for prediction.  
Here, HA structure refers to a particular head of MHA. Looking at the HA module alone, it consists of three main branches. The SK attention mechanism [41] constitutes the upper branch of HA. First, the image information input to this branch is segmented by two different sizes of convolutional kernels to obtain two different feature sets \(G_{1}\) and \(G_{2}\).  
\[\left\{\begin{array}{l}G_{1}=C_{3}(splitsplit(l_{2})_{1})\\ G_{2}=C_{5}(splitsplit(l_{2})_{2})\end{array}\right. \tag{13}\]  
In Equation (13), \(I_{2}\) represents the input image's information. The resulting feature sets are then used to compute two feature weights \(v\) and \(\omega\)[42], a process that requires Global Average Pooling (GAP) and an FCL.  
\[X=f_{R}(f_{c}([f_{G}(G_{1}),f_{G}(G_{2})])) \tag{14}\]  
\[[v,\omega]=f_{S}(f_{c}(X)) \tag{15}\]  
In Equation (14), \(f_{G}(*)\) denotes the GAP, and \(f_{R}(*)\) is the activation function to be used for ReLU. Finally, the information of the two feature groups is processed and fused by combining the feature weight weighting, and \(F_{SK}\) is computed. This step can realize the adaptive adjustment of feature information to spatial dimension and channel dimension, and use the image information more rationally. The formula is as follows.  
\[F_{SK}=vG_{1}+\omega G_{2} \tag{16}\]  
The bottom branch consists of the parts of the Channel Attention Mechanism (CAM) [43]. Obviously, this branch focuses on the information in the channel dimension, paying attention to different channels according to their importance in real situations, and continuously capturing more effective information in order to finally obtain useful \(F_{CAM}\).  
\[F_{CAM}=f_{S}(f_{c}(f_{R}(f_{c}(f_{G}(I))))) \tag{17}\]  
The middle part of HA is mainly used to retain the original feature information \(F_{o}\) and pass it on to the next step as it is. This part aims to fully utilize the data information and prevent the other two branches from discarding important information during processing and making the detection results incomplete. After the three branches have obtained new feature information through their respective processing mechanisms, the module fusesthe outputs of the three parts to obtain the composite feature \(F_{fusion}\). The computational formula for the feature fusion process is as follows.  
\[F_{fusion}=F_{SK}\otimes F_{CAM}\otimes F_{o} \tag{18}\]  
The HA module effectively reduces the risk of information loss and noise interference by focusing on multiple information channels simultaneously. This multi-dimensional feature processing can improve the robustness of the model in recognizing cracks, making its performance more stable when dealing with complex backgrounds and variable environments. The introduction of the MHA structure can enhance the overall performance of the model, making it perform well in the wall crack detection task.