# Multi-Defect Detection Network for High-Voltage Insulators Based on Adaptive Multi-Attention Fusion
## 3 AC-YOLO Target Detection Network
### Adaptive Weight Distribution Multi-Head Self-Attention Module  
The complex diversity of the background of the high-voltage overhead transmission line and the distance of the UAV aerial photography result in the insulator defect target being detected with relatively little pixel information in the image. In this case, the original YOLOv5 model is susceptible to the interference of complex background information and other fixtures on the transmission line when extracting insulator breakage and flashover features, thus affecting the ability to extract and discriminate effective features of insulators and their defects. To completely utilize the computational resources and focus attention on the extraction of key information about insulators and their surface defects, [PERSON] et al. [20] proposed a multi-head self-attention mechanism in which each head learns a set of weights, which is used to weigh combinations of features for each head's attention. It allows more attention to be focused on the feature information of the insulators and their local defects in the labeled region, which leads to a more effective distinction between the complex background and the insulator region. However, the multi-head attention mechanism assigns relatively fixed weights to the insulator and defect process, and there is a problem of unbalanced weight distribution, which makes it difficult for the model to distinguish insulators and their defects, thus affecting the convergence effect of the model. To solve this problem, this paper introduces the AWDM (adaptive weight distribution mechanism) module based on the multi-head self-attention mechanism to design a new AWDMSM (adaptive weight distribution multi-head self-attention module), the structure of which is shown in Figure 4. The introduction of the AWDMSM makes the model more flexible in combining and weighing different features in the training process. Specifically, the model assigns smaller weights to large targets that are easy to detect, such as normal insulators, and larger weights to small targets that are not easy to detect, such as insulator breakage  
Figure 3: Structure of the AC-YOLO network model. and flashover. This significantly reduces the difficulty of the model in distinguishing insulators and their defects, thus accelerating the convergence of the model. The locally extracted feature map \(x\) is utilized as the input, and three \(1\times 1\) convolution operations are performed to yield three feature vectors: \(q\), \(k\), and \(v\). The objective of these convolution operations is to encode the positional information of each feature, thereby bolstering the model's perception of sequence structure capability. Subsequently, the correlations q(x), k(x), and v(x) between the corresponding local area features and global features are computed utilizing Equation (1):  
\[\left\{ \begin{array}{l}\text{q(x)}=\text{W}^{\text{q}}\cdot q\\ \text{k(x)}=\text{W}^{\text{k}}\cdot k\\ \text{v(x)}=\text{W}^{\text{v}}\cdot v\\ \end{array} \right., \tag{1}\]  
where W\({}^{\text{q}}\), W\({}^{\text{k}}\), and W\({}^{\text{v}}\) are weight matrices, which are learned during training. The imbalance between semantic and location information during the feature extraction process curtails the model's detection accuracy. To address this, we propose the AWDM, the structure of which is illustrated in Figure 5. In the AWDM, first, input the feature layer \(X_{n}\) (where \(n\in[0,1,2]\)) with positional encoding. Subsequently, each feature layer undergoes separate global average pooling to condense its spatial information. These pooled feature layers are then integrated to obtain the global balanced feature layer \(Z^{1\times 1\times 1}\), where \(\lambda=\lambda_{0}+\lambda_{1}+\lambda_{2}\). Following that, the globally balanced feature layer is deployed to equalize the location and semantic information of both shallow and deep layers through the application of Equation (2).