# Scale-Sensitive Attention for Multi-Scale Maritime Vessel Detection Using EO/IR Cameras
## 3 Proposed Method
### Scale-Sensitive and Self-Attention Mechanisms for Multi-Scale Detection
The attention score \(A_{s}\) for small object detection using SSCA and the attention score \(A_{l}\) for large object detection using SA can be found in Equation (3):  
\[A_{s}=\text{softmax}\bigg{(}\frac{Q_{s}K_{s}^{T}}{\sqrt{d_{k}}}\bigg{)},\quad A _{l}=\text{softmax}\bigg{(}\frac{Q_{l}K_{l}^{T}}{\sqrt{d_{k}}}\bigg{)} \tag{3}\]  
where \(Q_{s}\in\mathbb{R}^{(H_{s}W_{s})\times d_{k}}\) and \(K_{s}\in\mathbb{R}^{(H_{s}W_{s})\times d_{k}}\) denote the query and key matrices for small objects, respectively. \(d_{k}\) is the dimensionality of the key used for scaling. Similarly, \(Q_{l}\in\mathbb{R}^{(H_{l}W_{l})\times d_{k}}\) and \(K_{l}\in\mathbb{R}^{(H_{l}W_{l})\times d_{k}}\) represent the query and key matrices for large objects. The relationship between the query and key is computed using the dot product. Afterward, the attention scores are normalized across the feature dimension using the softmax function. After calculating the attention scores, the value matrix is weighted by these scores to generate the output. Equation (4) represents the process of the weighted sum for attention-based feature extraction. \[\text{Output}_{s}=A_{s}V_{s},\quad\text{Output}_{l}=A_{l}V_{l} \tag{4}\]  
where \(A_{s}\in\mathbb{R}^{(H_{s}W_{s})\times(H_{s}W_{s})}\) is the attention score matrix for small objects, and \(V_{s}\in\mathbb{R}^{(H_{s}W_{s})\times d_{p}}\) is the value matrix for small objects. In SSCA, the weighted sum of \(A_{s}\) and \(V_{s}\) produces the output feature for small objects. Similarly, \(A_{l}\in\mathbb{R}^{(H_{l}W_{l})\times(H_{l}W_{l})}\) is the attention score matrix for large objects, and \(V_{l}\in\mathbb{R}^{(H_{l}W_{l})\times d_{p}}\) is the value matrix for large objects. In SA, the weighted sum produces the output feature for large objects. This describes how the attention scores and value matrices are combined to generate output features for small and large objects in SSCA and SA, respectively. The multi-head attention mechanism performs the SSCA head for small object detection and the SA attention mechanism for large object detection in parallel, learning different relationships for each. This enables precise detection of objects of various sizes without missing any, ensuring robust detection across the entire scale range. The attention \(\text{mAP}_{s}\) produced by each head are ultimately integrated, combining the information from both SSCA for small objects and SA for large objects. This integration ensures that the model effectively captures and utilizes features from objects of varying sizes for more precise and comprehensive detection. The outputs of SSCA and SA are concatenated to form the final attention-enhanced feature map. This step ensures that both small and large objects are effectively represented in the final detection output. This process can be observed in Equation (5):  
\[\text{Multi-Head Attention}=\text{Concat}(\text{Output}_{s},\text{ Output}_{l}) \tag{5}\]  
where \(\text{Output}_{s}\) denotes the result of the SSCA module for small object detection, and \(\text{Output}_{l}\) represents the result of the SA module for large object detection. These multi-scale attention features are then fused into a single output tensor using concatenation. The bounding box results obtained from SSCA and SA are each displayed separately based on object size. Small objects are boxed using the results from SSCA, while large objects are boxed using the results from Self-Attention. Both sets of results are then upsampled to match the original image size and displayed in different colors to allow users to visually distinguish between them. This makes it easy to understand how objects of various sizes were detected within a single image.