## 3 Materials and Methods
### Attention Module
#### 3.2.2 Neck: SPP-MHSA, FPN, and PANet  
The extraction of fusion features for target identification is often improved by adding layers between the backbone network and the output layer. YOLOv4's Neck module includes FPN, PANet, and SPP. The pooling layer, known as SPP, trees the network from the fixed-size constraint, resulting in the CNN no longer having a fixed-size input. The study, in particular, constructs an SPP layer above the final convolution layer. A top-down architecture with horizontal linkages is developed to generate a high-level map of semantic features on a global scale. The FPN design significantly outperforms other generic feature extractors in various applications. Further, PANet is quick, easy, and very efficient. It comprises components that improve information transmission across pipelines. It combines characteristics from all levels and even reduces the distance between the bottom and top layers. Moreover, each level's elements are enhanced by utilizing augmented routes. This study introduces the transformer's MHSA module for obtaining important and prominent features from diverse places in images following the SPP module and replacing the convolution block. Figure 7 shows the precise operational procedure. From global to local elements in the images, modeling may be performed. This procedure can be more simply integrated than before and extracts useful semantic and geographical information. Figure 7: Multihead Self-Attention (MHSA) layer in Auto-T-YOLO. Transformer's MHSA. The transformer model mainly introduces MHSA, composed of multiple self-attention mechanisms. For this self-attention model, the calculation process could be summarized as a function:  
\[A\text{tt}(Q,K,V)=\text{softmax}\bigg{(}\frac{QK^{T}}{\sqrt{d}}\bigg{)}V \tag{2}\]  
\[Q=W^{Q}\cdot x,K=W^{K}\cdot x,V=W^{V}\cdot x \tag{3}\]  
The input is \(x\), representing a matrix in image processing, and the elements in the matrix are called tokens. Linear operations are performed on \(x\) to obtain \(Q\), \(K\), \(V\), where \(Q\), \(K\) are calculated for similarity and obtain the correlation between each element in \(Q\) and \(K\). According to this correlation, the corresponding elements in \(K\) are used to construct the output. Similar to the parallel branch in CNN, self-attention can also be designed as a parallel structure called multihead self-attention (MHSA). In Auto-T-YOLO, this study used four heads:  
\[head_{i}=Attention\Big{(}QW_{i}^{Q},KW_{i}^{Q},VW_{i}^{V}\Big{)},i=1,2,3,4 \tag{4}\]  
\[MHAtt(Q,K,V)=Concat(head_{1},\cdots,head_{h})W^{Q},ie[1,h] \tag{5}\]  
The specific self-attention structure diagram is shown below. Relative Position Encodings: To perceive the position when performing attention operations, relative position coding is used in the transformer architecture [10]. However, recently, relative-distance-aware position encodings have been found. Relative Position Encodings are a sort of position embedding for Transformer-based models that aim to take advantage of pairwise, relative positional information. The model receives relative positioning information at two levels: values and keys. The coding approach proposed by [PERSON] et al. [44] is more suitable for visual processing tasks [45; 46]. This can be attributed to the attention mechanism considering the content information and the relative distance of the features in different positions. Based on these considerations, the perception of information and location can be linked more effectively. In SPP-MHSA, this study uses relative position-coding to perform self-attention operations [44; 47]. SPP-MHSA. According to the previous analysis, deep CNNs can easily lead to the optimal local solution. It is easy to ignore the salient global features of some targets. The MHSA mechanism aims to model global features. Based on the precedence of global features in visual perception, the fusion order of semantic features for each target is from global to local.