# Multi-Resolution and Semantic-Aware Bidirectional Adapter for Multi-Scale Object Detection
## 3 The Proposed Method
### Semantic-Aware Refinement Transformer  
Based on earlier investigations [9; 55], it is evident that the semantic message contained in the high-level features significantly contributes to mitigating scale variations. However, in conventional approaches, there is a lack of distinction between different levels. Common methods merely employ high-level features to provide semantic information in their original states. Moreover, the transformer is designed to capture long-range semantic messages due to its self-attention mechanism. Nevertheless, directly applying the transformer to high-level features may disregard the variations in features across diverse representation situations. Thus, we propose the SRT transformer encoder to enhance the comprehensive semantic representation of high-level features across different feature states. This enhancement facilitates the acquisition of multi-scale semantic global information by high-level features.  
As illustrated in Figure 3, we employ SRT on \(C_{5}^{\prime}\) to augment the semantic information. The entire process of SRT can be elucidated as follows:  
\[\hat{M}_{5}^{\prime}=LN\{Attn_{\text{SRT}}(PE(C_{5}^{\prime}))+PE(C_{5}^{ \prime})\}, \tag{6}\]  
\[M_{5}^{\prime}=LN\{(FFN(\hat{M}_{5}^{\prime})+\hat{M}_{5}^{\prime})\}, \tag{7}\]  
where \(LN\) denotes the layer normailzation operation. \(PE\) introduces the position embedding for the feature and the \(FFN\) serves to enhance the non-linearity of these features. \(Attn_{\text{SRT}}\) signifies the novel SRT attention mechanism, enabling the query of the original feature to probe long-range semantic relationships across various feature states. Furthermore, the sufficient semantic information can be integrated through the SRT attention mechanism effectively. The process can be delineated as:  
\[Attn_{\text{SRT}}=Concat[\{Attn_{n}(q_{1},k_{i},v_{i})\}_{n=1}^{h}]\quad i=(1,2,3). \tag{8}\]  
The term \(q_{1}\) represents the query extracted from the original feature. The keys, namely \(k_{2}\), \(k_{3}\), along with the values \(v_{2}\), \(v_{3}\), signify the keys and values obtained through processing the corresponding features using average and max pooling operations. The processed features can achieve more expressive with tiny spatial size. The \(h\) denotes the number of attention heads. Following this, \(q_{1}\) engages in interactions with the other keys to amplify the semantic representation of the high-level feature under various representation states. The mechanism \(Attn\) is employed to calculate token-wise correlations among the features. Details can be formulated as follows:  
\[Attention(q,k,v)=Softmax(\frac{qk^{T}}{\sqrt{d_{k}}})v, \tag{9}\]  
where \(q\), \(k\), and \(v\) represent the query, key, and value, separately. \(d_{k}\) denotes the feature channels. Our proposed approach employs the initial query to compute correlations with other keys sourced from diverse sections of the feature. This process enables the sufficient extraction of semantic information from the high-level feature.  
In summary, our proposed SRT comprehensively investigates the semantic information across different states of the high-level feature. This facilitates the refinement and enhancement of multi-scale semantic details through long-range relationship interactions. Moreover, the computational cost remains minimal due to the small spatial size of the high-level feature.