SIA-OVD: Shape-Invariant Adapter for Bridging the Image-Region Gap in Open-Vocabulary Detection

Authors: zishuo  wang
wenhao  zhou
jinglin  xu
yuxin  peng
  sia-ovd
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-30 16:00:25

Abstract: Open-vocabulary detection (OVD) aims to detect novel objects without instance-level annotations to achieve open-world object detection at a lower cost. Existing OVD methods mainly rely on the powerful open-vocabulary image-text alignment capability of Vision-Language Pretrained Models (VLM) such as CLIP. However, CLIP is trained on image-text pairs and lacks the perceptual ability for local regions within an image, resulting in the gap between image and region representations. Directly using CLIP for OVD causes inaccurate region classification. We find the image-region gap is primarily caused by the deformation of region feature maps during region of interest (RoI) extraction. To mitigate the inaccurate region classification in OVD, we propose a new Shape-Invariant Adapter named SIA-OVD to bridge the image-region gap in the OVD task. SIA-OVD learns a set of feature adapters for regions with different shapes and designs a new adapter allocation mechanism to select the optimal adapter for each region. The adapted region representations can align better with text representations learned by CLIP. Extensive experiments demonstrate that SIA-OVD effectively improves the classification accuracy for regions by addressing the gap between images and regions caused by shape deformation. SIA-OVD achieves substantial improvements over representative methods on the COCO-OVD benchmark. The code is available at https://github.com/PKU-ICST-MIPL/SIA-OVD_ACMMM2024.

Main paper: 
 introduction: 
 Object detection is a fundamental visual task that requires a massive amount of annotated bounding boxes for training, which limits the ability of traditional object detectors to cope with the continual emergence of novel objects. Therefore, OVR-CNN [37] proposes an Open-Vocabulary Detection (OVD) setting, which trains object detectors without bounding box annotations for target objects. Subsequently, models are tested on object detection for target categories, thereby achieving the capability to detect newly introduced objects without the need for additional annotations. The OVD framework enables versatile real-world applications.Most OVD methods leverage the powerful visual-language model CLIP [24] pre-trained on large image-text datasets to achieve openvocabulary detection capabilities. CLIP embeds texts and images into a common feature space and shows remarkable zero-shot ability in open-vocabulary vision tasks. Mainstream OVD methods divide detection into localization and classification, and mainly focus on the classification stage. In this stage, CLIP is treated as the open-vocabulary classifier. The CLIP text encoder calculates textual embeddings of each category, and the CLIP image encoder calculates visual embeddings of each region proposal. Then, the final OVD results are gained from the cosine similarity of the category embeddings and region proposal embeddings.However, applying CLIP to OVD may raise a fundamental challenge: CLIP is pre-trained on image-text pairs, and the CLIP image encoder can effectively extract features from entire images, while OVD requires to capture visual features of regions containing objects of interest within an image. To this end, previous efforts [1, 2, 11, 20, 34] employed RoIAlign [10] to crop regions from entire images. However, the image-text alignment via CLIP was corrupted after using RoIAlign [40] , resulting in a gap between the image and region. Such a gap leads to poor classification accuracy for novel objects. Recent works have further attempted to eliminate this gap to improve the accuracy of region classification by regionlevel knowledge-distillation [6, 9, 21] , region-text pre-training/finetuning [16, 22, 32, 40] , and visual/textual prompting [5, 7, 33] . However, these methods fail to find the fundamental reason for this gap. In this work, we argue that object shape deformation brought by RoIAlign is the main cause of the image-region gap. In this work, we propose a new Shape-Invariant Adapter (SIA) to bridge the gap between the image and region caused by the object shape deformation. Specifically, SIA consists of a set of independent adapters that transform the object region's features into shapeinvariant region features to mitigate the image-region gap. Each SIA adapter is a lightweight bottleneck network to prevent the potential over-fitting problem on base categories since the shapes of object regions are a long-tailed distribution. In addition, we design a new adapter allocation mechanism in SIA to adaptively adjust the weights of different adapters according to objects' shapes. Thus, each adapter only needs to cope with regions with similar shapes, which reduces the image-region gap caused by shape variance. So far, SIA can better align the shape-invariant region features with text features to improve region classification accuracy in the OVD task. It is worth mentioning that SIA allows us to use the frozen CLIP image encoder as the backbone of our OVD model without fine-tuning its parameters, which brings the benefits of CLIP's open-vocabulary knowledge directly into the OVD task, especially in scenarios where traditional object detection models struggle with novel objects. This incorporation will lead to more robust and adaptable detection systems. Region-Text Pretraining based OVD methods tend to fine-tune visual encoders on region-text pairs to enhance the alignment between region and text features for more precise object classification.RegionCLIP [40] presents a two-stage training strategy, which involves pretraining by aligning region-text pairs, followed by transferring the acquired knowledge to an object detector. GLIP [16] unifies the formulation of object detection and phrase grounding, which learns the correlation between regions and sub-words. OWL-ViT [22] first conducts image-level contrastive pretraining and then transfers to object-level with a bipartite matching loss. RO-ViT [14] employs region positional embeddings that are randomly cropped and resized instead of using positional embeddings for the entire image to bridge the image-region gap. Edadet [26] proposes Early Dense Alignment to improve the generalization of local semantics and maintain the local fine-grained semantics. CLIM [32] aggregates multiple images and treats each image as a pseudo-region to form region-text pairs for fine-tuning CLIP. Moreover, it can seamlessly integrate with other OVD models as a versatile framework. YOLO-World [4] is pre-trained on large-scale datasets, which maintains powerful zero-shot recognition capabilities while still having efficient inference speed. Some of these methods suffer from the noisy pseudo-region-text pairs [12] . Additionally, fine-tuning the parameters of CLIP is likely to destroy its open-vocabulary ability to generalize to novel objects. In contrast, we keep the CLIP image encoder frozen to maintain CLIP's generalization ability.Prompt-Learning based OVD methods usually keep the CLIP image encoder frozen and introduce learnable prompts to adapt CLIP to the object detection task. DetPro [5] learns context tokens as input to the text encoder. It learns different prompts for proposals of different IoU with ground-truth boxes to address the variances among proposals. PromptDet [7] incorporates learnable prompt vectors into textual input, which are unrelated to categories, enabling generalization from base classes to target classes. Prompt-OVD [27] propose a prompt-based decoding module, in which both the image embedding and text embedding from CLIP are regarded as class prompts for each self-attention module of the decoder. CORA [33] utilizes ground truth bounding boxes from base classes to train position-embedding modules within the RN50 backbone, aiming to adapt CLIP to the OVD task better. According to the section 2.1, these prompt-learning-based OVD methods learn fixed prompts for all samples, which may limit their generalization ability for unseen categories. In contrast, we learn conditional adapters for different samples for better generalization performance. 
multi-modal prompt tuning: 
 Large-scale vision-language pre-trained model (VLM), e.g., CLIP, demonstrates impressive zero-shot performance across various downstream tasks [17, 18, 29, 35] . Adapting VLMs to a specific task commonly involves two approaches: fine-tuning or prompt tuning. Recently, the focus of research has gradually shifted towards prompt tuning, which keeps VLM encoders frozen and learns additional prompts as input to VLMs. CoOp [42] replaces the manual template of textual prompts with learnable context tokens, which are trained with all categories. However, such fixed context tokens result in poor zero-shot capabilities and generalization to unseen categories, so CoOp is not compatible with the OVD task. CoCoOp [41] integrates image features into the learning process by adding a lightweight meta-net to generate instance-conditional textual tokens to mitigate this issue. CoCoOp improves the open-vocabulary ability, but the memory consumption and computational cost are extremely high because the meta-net does not support batch processing. UPT [36] developed a modality-agnostic prompt tuning method, leveraging a lightweight transformer layer to concurrently fine-tune both the visual and textual branches. CLIP-Adapter [8] adapts the original clip model to downstream tasks by adding trainable linear layers after the vision encoder and text encoder. CLIP-Adapter achieves higher efficiency because the adapters are inserted after encoders, and the back-propagate paths are much shorter. FG-VPL [28] explores the potential for fine-grained image recognition of CLIP by introducing fine-grained visual prompts. In this paper, we follow these prompt tuning methods with frozen backbones to maintain the generalization ability of CLIP. Extra training datasets of image-text pairs (e.g., COCO Captions [3] and CC3M [25] ) are also available, but in this work, we do not utilize any extra datasets to train our model.Specifically, our open-vocabulary object detection process is divided into two branches: object localization and region classification. We mainly focus on improving the accuracy of region classification in this work. For object localization, we use DAB-DETR [19] as the localizer, which learns object queries with bounding boxes as input. DAB-DETR encodes the feature map output by the 3 𝑟𝑑 layer of the CLIP image encoder and decodes the outputs of the bounding boxes of detected objects in both base and novel categories. For region classification, given a region proposal of 4 dimensions (𝑥 𝑖 , 𝑦 𝑖 , ℎ 𝑖 , 𝑤 𝑖 ) with its confidence score 𝑐𝑜𝑛𝑓 𝑖 decoded from DAB-DETR, we utilize RoIAlign [10] to obtain the region feature from CLIP's image feature map. Then, we feed the region feature and the bounding box into the Shape-Invariant Adapter (SIA). SIA generates the weights for different adapters according to the shape of the bounding boxes and transforms the region features into shapeinvariant ones that are better aligned with text embeddings. We perform region classification by calculating the cosine similarities between region embeddings and text embeddings of all categories. The overall framework is illustrated in Figure 2 . 
shape-invariant adapter (sia): 
 Structure of SIA. In this section, we introduce our Shape-Invariant Adapter illustrated in Figure 3 . To mitigate the image-region gap in OVD, SIA aims to learn a mapping from the region feature space obtained by RoIAlign to the common space, which is aligned with CLIP visual and textual features. We are inspired by the fact that CoOp [42] learns fixed context tokens for all samples and performs poorly on unseen categories, while CoCoOp [41] learns conditional context tokens based on each sample and achieves better performance on unseen categories. Therefore, we try to learn conditional feature transformation based on the shapes of region proposals.Luckily, the shape information of a region is much simpler than its semantic information. We do not have to rely on the generative network of CoCoOp to generate conditional context, which results in a high computational cost. Specifically, SIA keeps a set of 𝑁 independent adapters {𝐴𝑑𝑎𝑝𝑡𝑒𝑟 𝑗 , 𝑗 = 1, ..., 𝑁 }. Each adapter consists of two fully connected layers W 𝑗 1 and W 𝑗 2 , a ReLU activation layer, and a residual factor 𝜆. When the 𝑖 𝑡ℎ region feature 𝑓 𝑖 𝑟 ∈ R 𝐷 ×1 is input to SIA, we obtain the adapted region embedding f 𝑖 𝑟 through a residual connection:EQUATIONEQUATIONFor the 𝑖 𝑡ℎ region proposal R 𝑖 , SIA outputs 𝑁 adapted region features:EQUATIONAdapter Allocation Mechanism. To mitigate the image-region gap caused by shape variance, we design an Adapter Allocation Mechanism, which generates weights for the 𝑁 adapter in SIA so that each adapter only needs to cope with regions with similar shapes. We quantize the region shapes as aspect ratios (height/width).Then we manually partition the real number axis of different aspect ratios into 𝑁 discrete intervals by 𝑁 + 1 points {𝑠 0 , 𝑠 1 , ..., 𝑠 𝑁 } (𝑠 0 = 0 and 𝑠 𝑁 → +∞). Given a region proposal R 𝑖 with a height of ℎ 𝑖 and a width of 𝑤 𝑖 from the object query of DETR, we compute the one-hot allocation weight according to its shape ℎ 𝑖 /𝑤 𝑖 :EQUATIONGiven the adapter allocation weights 𝑌 𝑖 for the 𝑖 𝑡ℎ region proposal and the region features adapted by all adapters in SIA 𝐹 𝑖 𝑟 , we multiply the region features by the weights to select the optimal adapter:EQUATIONRegion Features Finally, to perform classification, the adapted region feature 𝛽 𝑖 𝑟 is multiplied by the CLIP text embeddings for each category name.𝐖 𝟏 𝟏 𝐖 𝟏 𝟐 𝐖 𝟏 𝑵 ⋯ ×𝑁 ⋯ 𝐖 𝟐 𝟏 𝐖 𝟐 𝟐 𝐖 𝟐 𝐍 ×𝑁 ReLU ReLU ReLU ×𝑀 ⋯ ⋯ ×𝑀 Adapted Region Embeddings 𝑦 (") 𝑦 ($) 𝑦 (%) Adapter Weights ⋯ 𝛽 ! " 𝛽 ! # 𝛽 ! $ ℛ ! ℛ " ℛ # ℛ $ ⋯ Bounding Boxes Adapter Allocation Mechanism ℎ/𝑤 𝑠 & 𝑠 " 𝑠 $ 𝑠 %'" 𝑠 %'$ 𝑠 ( ×𝑀 𝑁 intervals ⋯ ⋯ ⋯ ℎ ! /𝑤 ! ℎ " /𝑤 " ℎ # /𝑤 # ℎ $ /𝑤 $The text embeddings are denoted as:EQUATION)where 𝐾 is the number of categories. The probability that region R 𝑖 belongs to category 𝑘 is denoted as:𝑝 𝑘 𝑖 = exp (𝛽 𝑖 𝑟 ) T (𝛼 𝑘 𝑡 )/𝜏 𝐾 𝑗=1 exp (𝛽 𝑖 𝑟 ) T (𝛼 𝑗 𝑡 )/𝜏 (7)Training. We adopt a two-stage training process. In the first stage, we freeze all parameters except the adapters in SIA. For a training sample R 𝑖 the ground-truth bounding boxes are input to the classification branch, and we optimize SIA with Cross-Entropy Loss:EQUATION)Then we keep the classification branch frozen and train the transformer encoder and decoder in the localization branch. Our localizer shares the same DAB-DETR structure as CORA [33] , so we utilize the same training object. Inference. During inference, for the 𝑖 𝑡ℎ object query, the bounding box is refined by the transformer decoder and then fed to SIA for classification. We obtain a localization score 𝑠𝑐𝑜𝑟𝑒 𝑙 and a classification score 𝑠𝑐𝑜𝑟𝑒 𝑐 respectively for the region proposal. The classification score denoted the probability that this proposal belongs to a certain category:EQUATIONThe localization score is from the anchor pre-matching strategy of CORA. We multiply 𝑠𝑐𝑜𝑟𝑒 𝑐 with 𝑠𝑐𝑜𝑟𝑒 𝑙 as the confidence of this region proposal: OV-LVIS Benchmark. Following [40] , we also conduct experiments on the OV-LVIS benchmark, which has 1023 categories in total. The model is trained on the common and frequent categories and tested on the rare categories. Evaluation Protocol. We mainly evaluate the model under the generalized setting, which demands the model to detect objects of both base and novel categories at the same time. We take AP50 (average precision with an IoU threshold of 0.5) as the evaluation metric and report the metric on base categories and novel categories separately. Additionally, to directly reflect the classification performance for objects, we also calculate AP50 and box classification accuracy (denoted as Acc.) with a ground-truth bounding box as input.EQUATION 
implementation details: 
 Model Architecture. The architecture of the object localizer is identical to CORA, which is a DAB-DETR [19] consisting of a 3-layer transformer encoder, a 6-layer transformer decoder, and 1,000 learnable object queries as input. We utilize the 80 prompt Comparison with State-of-the-Art. Table 1 presents our main results on the COCO-OVD benchmark. The compared OVD methods include knowledge-distillation based methods (F-VLM [15] , SIC-CADS [6] ), region-text pretraining based methods (RegionCLIP [40] , BARON [31] , CLIM [32] ), pseudo-labeling methods (VL-PLM [39] ) and prompt-learning methods (PromptDet [7] , DetPro [5] , CORA [33] ). We do not compare SIA with SIC-CADS and CLIM, which are attached to an existing OVD model.To conduct a fair comparison, we separately compare the aforementioned methods with different visual backbones: RN50 and RN50x4. For each method, we list its supervision from three aspects: Extra Dataset (data beyond the instance-level annotation for base categories), Pretrained Model (the CLIP image encoder and text encoder), and Require Novel Class (whether the names of novel categories is needed during training).Among these baselines, SIA outperforms CORA [33] by 0.4 AP50 on novel categories with RN50 backbone, and 0.3 AP50 with RN50x4 backbone. SIA only utilizes the pre-trained CLIP model and instancelevel annotations for base categories, without the need for any extra dataset. Moreover, during the training phase, novel class names are not required either, which makes it more convenient to train the model and easier to expand to unseen categories. Experiments on Ground-Truth Bounding Boxes. To verify the effectiveness of SIA on region classification, we take the groundtruth bounding boxes for novel categories as input and calculate AP50 and average classification accuracy. The results are shown in Table 2 . We compare SIA with CLIP [24] , CoOp [42] , CLIP-Adapter [8] , RegionCLIP [40] , and CORA [33] . Due to the noises in pseudolabels, pseudo-labels-based methods RegionCLIP and VL-PLM struggle to improve region classification to a higher level. CoOp and CLIP-Adapter suffer from the significant gap between Novel and Base. CoOp learns optimal context tokens for base categories, which are suboptimal for novel categories. Similarly, CLIP-Adapter learns optimal feature transformation for base categories, which are suboptimal for novel categories. Both the context tokens and the linear feature transformation are fixed, facing different objects, resulting in the overfitting of base categories. CORA learns region prompts from base categories and adds the prompts to the visual feature map, which has reduced the performance gap between novel and base categories, but this gap still exists.To further demonstrate that SIA effectively improves the classification accuracy for regions with extreme aspect ratios, we report the classification performance of regions with different shapes in Figure 4 . As shown in the figure, the original CLIP exhibits unsatisfactory classification accuracy for all shapes. CORA improves the overall performance of region classification, but there is still an obvious gap between regions with aspect ratios far smaller or larger than 1:1 and regions with aspect ratios close to 1:1. Our proposed approach, SIA, effectively enhances the classification performance of regions with extreme aspect ratios. We also report AP50 scores achieved by CLIP, CORA, and SIA (Ours) across each novel category, as illustrated in Table 3 , where 𝛿 denotes the difference between AP50 scores of SIA and CORA. SIA achieves improvements over CORA across most categories, particularly for tie (+7.51%), skateboard (+7.69%), and keyboard (+13.26%). For cat, umbrella, and cake categories, SIA exhibits slight decreases compared to CORA. Ablation Study. We conduct ablation studies of AP50 with groundtruth bounding boxes as input and RN50 as the backbone on the OV-LVIS benchmark. Figure 5 shows the effect of adapter numbers within SIA architecture. The numbers are selected as 1, 2, 4, 10, and 20. As illustrated in Figure 3 , the AP50 initially increases and then decreases as the adapter number increases, reaching its optimal value at 10 adapters. This phenomenon arises because, with fewer adapters, each adapter needs to handle regions with significantly variant shapes, resulting in a relatively severe misalignment between regions and texts. Conversely, with more adapters, each adapter only has a limited number of training samples, especially for regions with extreme aspect ratios.Classification Results for Region Proposals. Figure 6 shows the region classification results in the COCO-OVD validation set. SIA predicts more reasonable classification results than CLIP and CORA for region proposals, especially for regions containing severely deformed objects, e.g., surfboards, ties, and knives. For example, in the first column, SIA successfully identifies the surfboard, while CLIP and CORA misclassify it as an airplane. Distribution of Adapted Region Features Figure 7 shows the region features adapted by SIA for 17 novel categories in COCO-OVD validation set using t-SNE [23] . It is illustrated that SIA achieves a more obvious separation of embeddings belonging to different categories than CLIP and CORA. In (a) CLIP and (b) CORA, feature points of different categories near the center tend to intermingle with each other, making it hard to classify these regions. In (c) SIA, there are relatively more distinct boundaries among feature points of different categories near the center, which verifies that SIA achieves higher classification accuracy for novel categories. 
 
 Appendix:  


Decision: This paper analyzes and addresses the image-region gap in open-vocabulary detection (OVD) tasks. There exists an issue caused by object shape deformation induced by the RoIAlign operation. To tackle this problem, the paper introduces a set of learnable shape-invariant adapters (SIAs) designed to map region feature space to a common space. Additionally, an aspect-ratio-related adapter allocation mechanism is developed to handle various shapes effectively. Experimental results on the COCO-OVD benchmark demonstrate the efficiency of the proposed method.

The authors solved most reviewers' concerns in their rebuttal. After the rebuttal, the paper received 2 borderline accept and 1 weak accept.

Reviews: review: paper_summary: paper_summary: This paper analyzes and addresses the image-region gap in open-vocabulary detection (OVD) tasks, a fundamental issue caused by object shape deformation induced by the RoIAlign operation. To tackle this problem, the paper introduces a set of learnable shape-invariant adapters (SIAs) designed to map region feature space to a common space. Additionally, an aspect-ratio-related adapter allocation mechanism is developed to handle various shapes effectively. Experimental results on the COCO-OVD benchmark demonstrate the efficiency of the proposed method.
strength_weakness: strength_weakness: This paper analyzes the fundamental image-region gap issue in open-vocabulary detection tasks, underscoring its novelty. To address this issue, the paper introduces shape-invariant adapters and an adapter allocation mechanism, providing an efficient and feasible solution. Moreover, the paper is well-written and easy to follow, clearly elucidating its novelty, methodology, and experiments.
limitations: limitations: Although this paper is well-written, there is still room for improvement. 

- Mistake in Eq. (4): $y^{(K)}$
- Unuse notation in line 499: $f_{A}$

Some questions:
- In Sec. 3.2, the paper divides the real number axis of the aspect ratio into $N$ intervals. The reviewer would like to know the details of this division. For instance, is each interval $[s_i, s_{i+1}]$, (i=0,1,...,N-2) of equal length? The reviewer suggests that the division process might be improved by considering the distribution of aspect ratios, which could result in an uneven division.
- It would be better if the authors could make an explanation of the **bolded** and underlined metrics in tables. For example, in Table 1, with the RN50x4 backbone, there is no bolded metric, only an underlined metric.
- The paper conducts experiments only on the COCO-OVD benchmark. The reviewer thinks it would better substantiate the efficacy of the proposed method if the authors could provide additional results on other benchmarks, such as LVIS [1]. Since this paper addresses a fundamental issue in OVD tasks, providing sufficient experimental results would further strengthen the contribution of this work. LVIS benchmark is also considered in recent methods CORA [2] and BARON [3]. 

[1] Gupta, Agrim, Piotr Dollar, and Ross Girshick. "Lvis: A dataset for large vocabulary instance segmentation." In the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5356-5364. 2019. 

[2] Wu, Xiaoshi, Feng Zhu, Rui Zhao, and Hongsheng Li. "Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching." In the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7031-7040. 2023.

[3] Wu, Size, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. "Aligning bag of regions for open-vocabulary object detection." In the IEEE Conference on Computer Vision and Pattern Recognition, pp. 15254-15264. 2023.
paper_summary: paper_summary: This paper is about Open Vocabulary Detection (OVD), which aims to detect novel objects without instance-level annotations, enabling open-world object detection at low cost.
strength_weakness: strength_weakness: （1）This paper has some novelty, summarizes the problem of OAD nowadays, which is the existence of gaps between image regions, and proposes shape invariant adapters to choose different adapters according to the shape of the picture, and the experimental results prove the validity of the proposed module.
（2）The paper is well structured, the figures explain well how this thesis differs from conventional work, and the visualization is adequate and demonstrates well the validity of the proposed methodology of the thesis.
limitations: limitations: （1）In figure2, 'Overview of the OVD framework' or 'Overview of the SIA-OVD framework'?
（2）Insufficient experiments, no ablation experiments of the proposed module are given to carry out the validation of the module's effect.
（3）In the experimental section, comparative results such as the number of parameters and inference speed should be provided。
paper_summary: paper_summary: This paper proposes a new Shape-Invariant Adapter named SIA-OVD to bridge the image-region gap in the OVD task. SIA-OVD learns a set of feature adapters for regions with different shapes and designs a new adapter allocation mechanism to select the optimal adapter for each region. The adapted region representations can align better with text representations learned by CLIP. Extensive experiments demonstrate the effectiveness of SIA-OVD.
strength_weakness: strength_weakness: 1.	The paper is with clear structure and easy to understand.

2.	The viewpoint that object shape deformation brought by RoIAlign is the main cause of the image-region gap is meaningful and interesting.
limitations: limitations: 1.	What is the difference between RoIAlign for close-set object detection and open-vocabulary object detection? Why do gaps in image regions cause detection obstacles? The introduction did not provide a detailed explanation of Figure 1. Also, the method section is short. 

2.	The model is built based on CORA, but the improvement in novel categories in Table 1 is very limited  (RN50 + 0.4AP, RN50 * 4 + 0.2AP). Also, the performance advantage over the main counterpart (CORA) is insignificant.

3.	There seems to be no regularity in the impact of the number of adapters on the performance of base and novel categories. There will be performance fluctuations every time training without changing any structure.

4.	As a plug-and-play module, SIA should be tested on multiple baselines to demonstrate its effectiveness.

5.	Lack of results on the LVIS dataset and comparison with the latest SOTA methods, such as EdaDet [1] and OADP[2].
[1] EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment. ICCV 2023
[2] Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection. CVPR 2023

6.  This paper focuses on the shape-invariant feature presentation, however, from figure 1 and figure 6, this paper tends to find some tiny object.