# Scale-Sensitive Attention for Multi-Scale Maritime Vessel Detection Using EO/IR Cameras
## 3 Proposed Method
### Scale-Sensitive and Self-Attention Mechanisms for Multi-Scale Detection  
SSCA and SA are positioned between the neck and detection head of the YOLO architecture. These modules take the multi-resolution features output from the neck as input to compute query, key, and value. By applying a ML-MHA mechanism, different attention weights are assigned according to the size of the objects, optimizing detection for each object size. SSCA is an attention mechanism specialized for small object detection. It uses features extracted from the neck (such as 64 \(\times\) 64 and 128 \(\times\) 128 resolutions) and employs the features of small objects as the Query (Q) to capture fine details specific to small objects. For the Key (K), it uses mask information containing object location data, ensuring that attention is focused on the small objects' locations. By scoring the relationship between the query and key, Value (V) is generated, which helps refine detection performance for small objects. This cross-attention mechanism allows for more accurate detection of small objects, which are often difficult to detect. SA is an attention mechanism designed for large object detection, leveraging its strength in capturing global information from each location within the input feature map. It is applied to feature mAPs extracted from the neck at resolutions (such as 16 \(\times\) 16 and 32 \(\times\) 32). SA scores the relationship of each position with itself in the input feature map to generate an attention map, which effectively detects the locations of large objects. In this case, the Query, Key, and Value all have the same size as the input feature map, ensuring that the attention mechanism accurately identifies large object locations. Table 1 shows the feature map sizes extracted from the neck, categorized by the object sizes handled by the SSCA and SA modules. The attention mechanisms of the SSCA and SA modules can be expressed with the following formulas. Here, the subscripts \(s\) and \(l\) represent small objects and large objects, respectively. First, the formulas for each Query, Key, and Value can be seen in Equations (1) and (2):  
\[Q_{s}=F_{s}W_{q}^{s},\quad K_{s}=M_{s}W_{k}^{s},\quad V_{s}=F_{s}W_{v}^{s} \tag{1}\]  
\[Q_{l}=F_{l}W_{q}^{l},\quad K_{l}=F_{l}W_{k}^{l},\quad V_{l}=F_{l}W_{v}^{l} \tag{2}\] \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Object Sizes** & **Feature Map Sizes** \\ \hline SSCA & Very Small object & 128 \(\times\) 128 \\  & Small object & 64 \(\times\) 64 \\ SA & Medium object & 32 \(\times\) 32 \\  & Large object & 16 \(\times\) 16 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Object sizes based on the feature map sizes extracted from the neck. where \(F_{s}\) denotes the feature map for small objects output from the neck, and \(M_{s}\) is the mask containing object location information used as the key in SSCA. \(V_{s}\) denotes the score information based on the correlation between the query and the key. Additionally, \(W_{q}^{s}\), \(W_{k}^{s}\), and \(W_{p}^{s}\) are the learned weight matrices that denote the projections for the query, key, and value, respectively. Similarly, \(F_{l}\) denotes the feature map for large objects output from the neck, and \(W_{q}^{l}\), \(W_{k}^{l}\), and \(W_{p}^{l}\) denote the learned weight matrices used for the projections of the query, key, and value for large objects. Once the query, key, and value matrices are generated, the attention scores are calculated by taking the dot product of the query and key, followed by scaling and applying the softmax function.