# Multi-Resolution and Semantic-Aware Bidirectional Adapter for Multi-Scale Object Detection
###### Abstract  
Scale variation presents a significant challenge in object detection. To address this, multi-level feature fusion techniques have been proposed, exemplified by methods such as the feature pyramid network (FPN) and its extensions. Nonetheless, the input features provided to these methods and the interaction among features across different levels are limited and inflexible. In order to fully leverage the features of multi-scale objects and amplify feature interaction and representation, we introduce a novel and efficient framework known as a multi-resolution and semantic-aware bidirectional adapter (MSBA). Specifically, MSBA comprises three successive components: multi-resolution cascaded fusion (MCF), a semantic-aware refinement transformer (SRT), and bidirectional fine-grained interaction (BFI). MCF adaptively extracts multi-level features to enable cascaded fusion. Subsequently, SRT enriches the long-range semantic information within high-level features. Following this, BFI facilitates ample fine-grained interaction via bidirectional guidance. Benefiting from the coarse-to-fine process, we can acquire robust multi-scale representations for a variety of objects. Each component can be individually integrated into different backbone architectures. Experimental results substantiate the superiority of our approach and validate the efficacy of each proposed module. object detection; scale variation; transformer; multi-level fusion +
Footnote †: journal: Medical  
while low-level features preserve detailed information but grapple with capturing semantic context. The aforementioned issues have emerged as bottlenecks for contemporary detection algorithms. The implementation of multi-level feature integration serves as an effective strategy to mitigate these issues. For instance, FPN [8] employed a top-down feature integration method to combine features at different scales. Nonetheless, the input features of FPN are directly extracted from the backbone network. They may have already lost original information during the inference process. Additionally, some studies [9] have validated the significance of semantic information in the high-level features. However, there exists insufficient exploration and utilization of the high-level features in FPN. Additionally, the merging approach incorporates a rigid fusion of two features, neglecting the variability in features across different levels. Relying solely on high-level features to direct low-level features leads to an absence of low-level spatial information in the high-level features. Moreover, the direct fusion process from top to bottom may dilute the semantic information within the high-level features. This suggests an insufficient interaction among multi-level features. As depicted in Figure 1, the baseline network (FPN) in the left column struggles to effectively address multi-scale object challenges, resulting in numerous false positive cases. This is primarily due to the underutilization of features and the absence of precise object representations. Figure 1: Visual Comparison of Results. The top row displays the detection outcomes of Faster R-CNN using FPN (_left_) and MSBA (_right_). The MSBA-based results exhibit a significant reduction in false positives and a qualitative performance enhancement. \(AP_{S}\), \(AP_{M}\), and \(AP_{L}\) denote the AP of small, medium, and large objects. In the bottom row, a similar trend is observed for Mask R-CNN, where our approach (_right_) consistently outperforms the baseline (_left_). \(AP\)_bbox_ and \(AP_{S}\)_bbox_ pertain to detection performance and bbox AP for small objects. \(AP\)_mask_ and \(AP_{L}\)_mask_ correspond to instance segmentation performance and mask AP for large objects. To mitigate the constraints of current approaches, we introduce a novel and potent framework, termed a multi-resolution and semantic-aware bidirectional adapter for multi-scale object detection (MSBA).