## 3 Materials and Methods
### YOLOv586 Yarn Hairiness Improvements
#### 3.4.3 Bot-Transformer Module and MHSA Block  
The C3_1_F block in the Neck was replaced with the Bot-Transformer module [53] (BoTNet) in the neural network. BoTNet, based on ResNet, replaces the standard 3 \(\times\) 3 convolution with a Multi-Head Self-Attention (MHSA) mechanism. This change significantly improves the detection and segmentation performance while reducing the number of model parameters [48; 49; 50]. The Bot-Transformer Module (Figure 5G(a,b) adds self-attention to various computer vision tasks, such as object detection and instance segmentation.  
This mechanism enables the network to learn important relationships between different parts of images, improving the performance on complex tasks [48; 49]. To apply BoTNet to YOLOv5, researchers adapted the C3 module of YOLOv586 by replacing the original Bottleneck with a Bottleneck Transformer, which incorporates the self-attention mechanism [48; 49; 50].  
The MHSA block is the central component of the Bottleneck Transformer used in the BoTNet. The MHSA is responsible for incorporating self-attention into various computer vision tasks, such as object detection and instance segmentation. Its structure is illustrated in Figure 6.  
Figure 6: Multi-Head Self-Attention (MHSA) Layer used in the Bot-transformer Block [48; 49; 50].  
Figure 6 illustrates the key components and operations of the MHSA block, shedding light on how this mechanism enhances the understanding of relationships within data sequences [48; 49; 50]:  
* **q, k, and v Matrices:** these matrices represent queries (q), keys (k), and values (v) and are integral to understanding input sequences.
* **qkT (Dot Product):** qkT represents the dot product between queries (q) and the transposed keys (k). This dot product quantifies the similarity between queries and keys, serving as the foundation for calculating attention weights.
* **qrT (Relative Position):** qrT symbolizes the dot product between queries (q) and transposed relative positions (r). The matrix r encodes positional or distance information between sequence elements.
* **qkT + qrT (Combined Attention):** This expression represents the summation of attentions derived from queries and keys and positional information. This cumulative measure captures both the overall attention and attention relative to the sequence's positions.
* **Softmax (qkT + qrT):** The Softmax function is applied to the result from the previous step. Softmax normalizes values, mapping them to a range between 0 and 1. Larger values receive higher importance, directing attention towards the most relevant elements in the sequence.
* **Weighted Multiplication:** the result of the Softmax operation is multiplied by the value matrices (v).  
The C2f, Bot-Transformer, and GeLU modules were incorporated into the YOLOv5s6 model for their abilities to enhance feature fusion, focus on relevant features, and handle nonlinearities.  
* **C2f (Cross-Stage Partial Networks Fusion):** Improves feature fusion at different network stages, improves the gradient flow, and extracts detailed features, crucial for detecting fine details like protruding and loop fibers.
* **Bot-Transformer:** uses multi-head self-attention to focus on relevant features, improving the detection and classification of hairiness defects and increasing the accuracy across various scales.
* **GeLu (Gaussian Error Linear Unit):** Handles nonlinearities, increases sensitivity to fiber variations, stabilizes training, and improves the convergence speed, enhancing the overall model performance and accuracy by providing smooth activation, improving convergence, and ensuring a better gradient flow.  
These modules were chosen over alternatives like deep convolution, ghost convolution, simple transformer blocks, CSP, and SPP bottleneck blocks for their superior ability to capture complex details. This capability is crucial for highlighting small details in high-resolution images, essential for accurately detecting loop and protruding fibers in yarn.