InsVP: Efficient Instance Visual Prompting from Image Itself

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-11 10:28:32

Abstract: Visual prompting is an efficient methodology for finetuning pretrained visual models by introducing a small number of learnable parameters while keeping the backbone frozen. However, most existing visual prompting methods learn a shared prompt for all samples, making it challenging to grasp distinct characteristics among diverse samples, thereby limiting the model's performance. While other methods partially address this issue through sample clustering and learning multiple prompts, they still struggle to capture nuanced differences among instances and incur significant parameter overhead. Therefore, to comprehensively and efficiently leverage discriminative characteristics of individual instances, we propose an Instance Visual Prompting method, called InsVP. Initially, the instance image prompt is introduced to extract both crucial and nuanced discriminative information from the original image itself and is overlaid onto the input image. Furthermore, the instance feature prompt is designed to capture both commonalities and characteristics among individual instances, fed into the model's intermediate layers to facilitate feature extraction. Consequently, the instance image and feature prompts complement each other, enhancing the adaptation ability of pretrained models to extract discriminative features from individual instances. Extensive experiments on various large-scale benchmarks show that our InsVP achieves superior performance exceeding the state-of-the-art methods at a lower parameter cost. Our code will be released.

Main paper: 
 introduction: 
 Over the past years, the deep learning community has widely embraced the pretraining-finetuning paradigm, which has played a pivotal role in propelling the field of computer vision (CV) [14, 15, 21, 22] . However, with the explosive growth of model size and data scale, such a conventional paradigm suffers from unaffordable storage and computational overheads [23] . Therefore, the latest efforts [5, 16, 17, 53] have concentrated on how to adapt the pretrained models to a particular downstream task efficiently. Notably, the emergence of visual prompting technology [3, 20, 23] has taken the lead in addressing this challenge. By introducing a small number [3, 16, 20, 23] train a shared prompt for all samples or within clusters, struggling to capture the distinct characteristics of individual instances. In contrast, our InsVP employs an image-driven instance prompt, capturing distinctive areas of instances and guiding the pretrained model to focus on them. The visualization results by Grad-CAM [39] verify that InsVP focuses on discriminative regions of images and achieves excellent performance.of learnable parameters, visual prompting can efficiently adapt the pretrained models to downstream tasks while keeping the whole pretrained backbone frozen. Most existing visual prompting methods have opted for a shared prompt that is applied uniformly to all data [3, 16, 23, 48] , disregarding the potentially significant variations among different data, as shown in Figure 1 . Consequently, the learned prompts fail to capture the distinct characteristics of individual instances, considerably limiting the discriminative power of deep models [20] . To overcome this limitation, recent visual prompting methodologies [20] have endeavored to cluster samples and simultaneously learn a clusterspecific prompt for each cluster, thereby mitigating the issue to some extent. However, these approaches still struggle to capture the subtle nuances of individual instances, as even samples within the same cluster exhibit fine-grained distinctions. Moreover, these methods unavoidably introduce significant parameter overhead, substantially impeding the scalability of the pretrained model.To address these limitations, we propose a novel and efficient visual prompting method InsVP, namely Instance Visual Prompting. As illustrated in Figure 1 , InsVP extracts distinctive areas unique to individual instances from the image itself, guiding the pretrained model to focus on the exclusive discriminative characteristics of the instances. To achieve this, we first design the image-level instance visual prompting to highlight the discriminative areas of instances in the input image, which comprises two complementary components of a patch prompt and a global prompt. By forwarding the input image to the designed lightweight prompters, the obtained patch prompt captures fine-grained local information from individual patches, meanwhile the global prompt gathers overall information from the entire image. Together, they complement each other in extracting the distinguishing regions of the instances, resulting in an instance image prompt that is overlaid onto the original image.Furthermore, our InsVP also introduces the feature-level instance visual prompting to continuously incorporate instance information into the intermediate layers of the pretrained model. Considering both the commonalities and characteristics between different instances at the feature level, we design the learnable common prompt and the generated specific prompt respectively. Motivated by VPT [23] , several learnable tokens are introduced as the common prompt to distill overarching patterns and fundamental attributes, fostering a comprehensive exploration of commonalities across all images. Moreover, a lightweight specific prompter is proposed to enhance the distinctive features specific to the individual instance from the input image itself. The collaboration between the common and specific prompts improves the adaptation capacity of the pretrained model to different instance samples, leading to superior performance. The main contributions of this work are:• To address the limitations of existing visual prompting methods, we propose InsVP, an efficient approach aimed at comprehensively leveraging the discriminative instance-specific information of the input image itself to enhance the recognition capability of pretrained models.  
related work 2.1 parameter-efficient finetuning: 
 Vision Transformer (ViT) has made remarkable achievements in the field of computer vision [1, 7, 13, 30, 46] . However, with the rapid increase in model size, fully finetuning the pretrained ViT models for downstream tasks inevitably brings large storage and computing overhead. Therefore, recent works [16, 23, 53] started to focus on reducing the number of learnable parameters for efficient finetuning of pretrained models which can be broadly categorized into partial tuning-based, extra module-based, and prompt learningbased ones.Partial tuning-based approaches [17, 35, 49, 54] aim to freeze the majority of the pretrained backbone while finetuning a small portion of the model parameters. For instance, such methods might only adjust the Linear/MLP heads [17, 21] , or refine a part of layers within the backbone [35, 49, 54] . While these approaches are straightforward and simple to implement, they commonly exhibit a substantial performance gap when compared to fully finetuning [10, 32] . In contrast, extra module-based methods [5, 36, 38, 51, 53] design additional learnable plug-in architectures to finetune the pretrained model. [53] introduced an extra learnable side network while maintaining the original model frozen. Similarly, other studies [8, 36, 38] proposed to insert extra learnable residual units into the backbone. A limitation of these approaches lies in their customized nature for specific architectures, hindering generalizability to other models. Moreover, these modules obviously introduce more learnable parameters compared to partial tuning-based methods, making them difficult to apply in practice [16, 23] .Prompt learning techniques initially emerged in the field of natural language processing (NLP), involving the integration of a small set of learnable soft-prompt into input texts to tailor language models for specific downstream tasks [26] [27] [28] . Recent studies have extended prompt learning to visual tasks, termed visual prompt learning or visual prompting [3, 6, 16, 20, 23, 29, 48] . Compared with partial tuning-based and extra module-based methods, the visual prompting-based approaches introduce significantly fewer additional parameters and achieve better compatibility with models of different structures [16, 23] .Specifically, existing visual prompting methods usually follow two popular manners, task-level visual prompting [3, 16, 23, 41, 44, 48] and cluster-level visual prompting [20] . The former involves all downstream data samples learning a set of shared image prompts. VP [3] learned a single prompt for all samples, which is added around the image in the form of padding and then fed into the pretrained model together with the original image. VPT [23] introduced several learnable tokens as the prompt shared by all samples which are used in the multi-head self-attention (MSA) blocks of the pretrained ViT model. On the basis of VPT, E 2 VPT [16] pruned the learned prompts from both the token-wise and segment-wise perspectives, which reduces the impact of unfavorable prompts while reducing the number of additional parameters required. However, the above methods fail to capture the discriminative characteristics of instances, limiting the model performance [20] . The latest clusterlevel visual prompting method DAM-VP [20] involved clustering samples and learning a set of visual prompts for each cluster. This approach partly mitigates the above problem, yet still struggles to capture subtle differences among instances within the same cluster. In addition, it will bring large parameter overhead, resulting in poor scalability. In contrast, our proposed InsVP introduces lightweight prompters to generate an instance prompt for each sample. 
insvp: instance visual prompting: 
 In this section, we illustrate the proposed visual prompting method InsVP in detail. InsVP aims to generate instance-specific visual prompts for each individual image to adapt the pretrained model efficiently. The notations used in this paper are introduced in Section 3.1, followed by the introduction of the image-level instance prompting in Section 3.2 and feature-level instance prompting in Section 3.3. The overall pipeline of InsVP is depicted in Figure 2 .For a pretrained Vision Transformer (ViT) [13] backbone M, it contains 𝑁 MSA blocks {B 𝑗 } 𝑁 𝑗=1 where each B 𝑗 consists of multihead self-attention and feed-forward networks together with Lay-erNorm [2] and residual connections [18] . For an input image 𝒙 ∈ R 𝐻 ×𝑊 ×𝐶 , it is initially divided into several equally sized patches {𝒙 𝑖 } 𝑀 𝑖=1 ∈ R ℎ×𝑤×𝐶 , where (𝐻,𝑊 ) is the size of image 𝒙, 𝐶 is the number of channels of 𝒙, (ℎ, 𝑤) is the size of patch 𝒙 𝑖 , 𝑀 is the number of patches. Each patch 𝒙 𝑖 is then first embedded into a 𝑑-dimensional latent space as:EQUATIONwhere𝒉 1 𝑖 ∈ R 𝑑 , E (•)donates the embedding layer of the backbone M. Subsequently, all image tokens {𝒉 1 𝑖 } 𝑀 𝑖=1 along with an additional classification token 𝒄 1 ∈ R 𝑑 are fed into the 𝑁 MSA blocks {B 𝑗 } 𝑁 𝑗=1 to extract features as:EQUATIONwhere "[ ]" indicates stacking and concatenating on the sequence length dimension. Finally, the output 𝒄 𝑁 +1 from the last MSA block is passed through a classification head H to derive the predicted probability distribution 𝒚:EQUATIONTo capture distinctive information of the input instance image 𝒙, we initially propose image-level instance visual prompting, which involves generating the instance image prompt for each input image to improve the performance of the pretrained model M. Specifically, two lightweight networks, the global prompter G 𝑔 and the patch prompter G 𝑙 , are designed to readily explore the global and local discriminative characteristics of 𝒙. The global prompter G 𝑔 is composed of two layers of dilated convolution [50] , along with a ReLU activation layer and a Dropout layer. The utilization of dilated convolution increases the receptive field of the global prompter G 𝑔 without adding extra parameters, providing it with a broader global perspective across the entire image. Building on this design, the global promotor G 𝑔 is capable of extracting global discriminative information from the original image 𝒙 via the generated prompt 𝒑 𝑔 ∈ R 𝐻 ×𝑊 ×𝐶 , such as the object's position, shape, and contour details:EQUATIONAdditionally, to fully capture the locally fine-grained information of the individual image patch {𝒙 𝑖 } 𝑀 𝑖=1 , a patch image prompter G 𝑙 is leveraged. To generate the patch prompt efficiently, the patch prompter G 𝑙 consists of two fully connected layers, taking a single image patch 𝒙 𝑖 as input and producing the corresponding patch prompt 𝒑 𝑙 𝑖 . This patch-based design significantly reduces the input and output dimensions of the fully connected layers, thereby de EQUATIONwhere 𝒑 𝑙 𝑖 ∈ R ℎ×𝑤×𝐶 . Subsequently, all {𝒑 𝑙 𝑖 } 𝑀 𝑖=1 are concatenated in accordance with the relative order of the image patches {𝒙 𝑖 } 𝑀 𝑖=1 , forming a complete patch prompt 𝒑 𝑙 ∈ R 𝐻 ×𝑊 ×𝐶 for the input image 𝒙.Consequently, after generating the global image prompt 𝒑 𝑔 and the patch prompt 𝒑 𝑙 , the instance image prompt 𝒑 𝐼 ∈ R 𝐻 ×𝑊 ×𝐶 for the input image 𝒙 is obtained by directly merging both two prompts in a linear combination manner to amalgamate the global and local information from the image itself:EQUATIONwhere 𝛽 𝐼 is a pre-defined weight hyper-parameter. Finally, the image prompt 𝒑 𝐼 is superimposed onto the original image, serving as the input x of the pretrained model M:EQUATIONwhere x ∈ R 𝐻 ×𝑊 ×𝐶 is the prompted image.To further enhance the adaptation ability of the pretrained model M, we design feature-level instance visual prompting to continuously incorporate instance-specific information into the MSA blocks. Given that at the feature level, different instances may exhibit both similarities and differences in their characteristics, our feature-level instance visual prompting proposes to utilize a common prompt 𝒑 𝑐 and a specific prompt 𝒑 𝑠 to simultaneously capture the commonality information among different instances and the distinctive information of each individual instance respectively. Motivated by VPT [23] , we introduce a number of 𝐿 𝑝 learnable tokens as the common prompt. For the sake of simplicity, we collectively denote all introduced tokens as 𝒑 𝑐 without further distinction. The common prompt 𝒑 𝑐 consists of distinct prompts added to each MSA block which is commonly used for all instances:EQUATIONwhere 𝒑 𝑐 𝑗 ∈ R 𝑑 ×𝐿 𝑝 is the common prompt for 𝑗-th MSA block B 𝑗 . As for the specific prompt 𝒑 𝑠 ∈ R 𝑑 ×𝐿 𝑝 , it is directly obtained from the image 𝒙 itself. In detail, a three-layer convolutional network is designed as the specific prompter G 𝑠 to generate 𝒑 𝑠 for all MSA blocks:EQUATIONSpecifically, the specific prompter G 𝑠 gradually employs convolutional and pooling layers to encode the image 𝒙 into 𝒑 𝑠 ∈ R ℎ×𝑤× (𝐶 •𝐿 𝑝 ) . Finally, the output 𝒑 𝑠 is reshaped into the specific prompt 𝒑 𝑠 ∈ R 𝑑 ×𝐿 𝑝 . Subsequently, the complete instance feature prompt 𝒑 𝐹 𝑗 is formed by readily adding up the common prompt 𝒑 𝑐 𝑗 and specific prompt 𝒑 𝑠 :EQUATIONwhere 𝛽 𝐹 is a pre-defined weight hyper-parameter. The instance feature prompt 𝒑 𝐹 𝑗 is combined with the image patch tokens{𝒉 𝑗 𝑖 } 𝑀 𝑖=1and the classification token 𝒄 𝑗 , then collectively fed into the MSA block for feature extraction:EQUATIONwhere𝑯 𝑗 = 𝒉 𝑗 1 , 𝒉 𝑗 2 , • • • , 𝒉 𝑗 𝑀 .Notably p𝐹 𝑗+1 get from B 𝑗 is not utilized in the next block B 𝑗+1 . Extensively extracting distinctive characteristics from images, the instance feature prompt 𝒑 𝐹 and image prompt 𝒑 𝐼 facilitate the pretrained model in capturing discriminative features of individual instances. Ultimately, the 𝒄 𝑁 +1 obtained from the final MSA block B 𝑁 undergoes processing via a classification head H to get the predicted probability distribution 𝒚 via Equation 3.As mentioned above, our InsVP introduces only a few additional parameters:EQUATION)The extra parameters of InsVP are notably lightweight compared to the pretrained model and other visual prompting methods [16, 20, 23] as demonstrated in Section 4.4.5. Following previous works [16, 20, 23] , during training, we maintain the pretrained model's encoder frozen while allowing only the classification head to be trainable. The optimization objective is as follows:EQUATIONwhere L 𝑐𝑒 is cross-entropy loss, 𝑦 𝑔𝑡 is the label of image 𝒙. 
experiments 4.1 datasets: 
 Building upon previous works [16, 20, 23] , the experiments are conducted on four fine-grained datasets: CUB-200-2011 [43] , NABirds [19] ,Oxford Flowers [34] , and Stanford Dogs [24] . Additionally, following DAM-VP [20] , we also perform experiments on another six commonly used visual datasets, including DTD [11] , Food101 [4] , Cifar100 [25] , Cifar10 [25] , GTSRB [40] , and SVHN [33] . Following [23] , for datasets with only publicly available train and test sets, we randomly split the train set into a train set (90%) and a validation set (10%), and utilize the validation set to determine hyper-parameters. Additionally, we conduct experiments on the VTAB-1k benchmark [52] following [16, 23] . VTAB-1k is a benchmark that tests how well visual models perform across 19 different tasks. These tasks fall into three categories: Natural, for everyday image recognition; Specialized, for specific areas like medical images; and Structured, for understanding complex scenes, such as 3D object recognition.We compare our InsVP with both parameter-efficient finetuning methods and visual prompting methods. We also report the fullytuning results as a baseline. For parameter-efficient finetuning, we report the results of partial tuning-based methods including linear probing [21] , Partial [49] , MLP [17] , and the results of extra modulebased methods including Sidetune [38] , Bias [53] , Adapter [5] ,Table 1 : The comparison results against state-of-the-art methods on ten datasets. Partial, Extra, and Prompting represent partial tuning-based, extra module-based, and prompt learning-based parameter-efficient finetuning methods respectively. Following their paper, ILM-VP, Yoo et al and AutoVP utilize ResNeXt-101-32x8d [47] , MoCo v3 trained ViT-B/16 and CLIP [37] as the backbone respectively. The best results are bolded and the second-best results are underlined. AdaptFormer [8] . For visual prompting methods, we compare with the task-level visual prompting methods such as VP [3] , VPT [23] , ILM-VP [6] , Yoo et al [48] , E2VPT [16] , LION [44] , AutoVP [41] and the latest cluster-level visual prompting approach DAM-VP [20] and TransHP [45] .Our experiments involve three pretrained vision models including the ViT-B/16 [13] and Swin Transformer [30] which are supervised by ImageNet-21k [12] , and another ViT-B/16 that is learned via MoCo v3 [10] . Following DAM-VP [20] , we train for 100 epochs on all datasets. We utilize the AdamW [31] optimizer for optimization and implement cosine annealing. The hyper-parameters length of instance feature prompt 𝐿 𝑝 , fusion weight of image prompts 𝛽 𝐼 , and fusion weight of feature prompt 𝛽 𝐹 are set to 9, 0.7, and 0.5 respectively. The learning rate and weight decay on each dataset are detailed in the Supplementary Materials. instances which enhance the pretrained model's recognition capability, leading to more accurate prediction results.To verify this, we further present the obtained prompts and Grad-CAM visualization results of the samples in CUB. As shown in Figure 3 and Figure 4 , the visualization results reveal that DAM-VP employs the exact same prompt for the samples belonging to the same category but exhibiting significant differences. Moreover, it seems that the learned prompts of DAM-VP lack a direct connection with the image samples or categories, and explicit semantic information is not observed. The Grad-CAM visualization results also indicate that under their prompts, the pretrained model fails to focus on discriminative regions of the instances. In contrast, our InsVP precisely outlines the object and identifies discriminative regions, such as the location of the bird's head and neck. Consequently, the pretrained model can more precisely focus on the object itself rather than the background, leading to outstanding performance.Benchmark. To further validate the effectiveness of our InsVP, in addition to the ten datasets mentioned above, following [16, 23] , we also conduct experiments on another widely used VTAB-1k [52] benchmark. As shown in Table 2 , compared to the second-best method, E 2 VPT [16] , InsVP achieve improvements of 1.8%, 0.8%, and 1.0% in the three different tasks Natural, Specialized, and Structured, respectively. This further illustrates the robust adaptability of our instance-level visual prompting designed based on the original image across diverse tasks.To verify the generalization ability of our InsVP across different model architectures, we conduct experiments using the Swin Transformer [30] as the backbone. As shown in Table 3 , although the Swin Transformer For visual prompting methods, the number of introduced extra parameters is a crucial factor in determining their practical applicability. As illustrated in Figure 5 , we compare the additional parameter quantities introduced by InsVP and other visual prompting methods. In the case of task-level visual prompting methods, since they use a shared prompt for all samples, a significant number of learnable tokens are required to capture diverse features of different samples. On the other hand, for cluster-level visual prompting methods like DAM-VP, a set of prompts needs to be learned for each cluster, resulting in a substantial increase of extra parameter overhead, reaching several times or even hundreds of times [20] , thereby compromising the scalability.In contrast, our proposed InsVP adopts a more straightforward methodology by extracting crucial prompting information directly from raw images. This allows InsVP to efficiently capture discriminative characteristics by lightweight prompters. Therefore, as shown in Figure 5 , InsVP achieves optimal performance with minimal parameter costs. 5 . As demonstrated, when neither component is used, the InsVP is degraded to a frozen pretrained ViT model with a learnable classifier. Taking results on CUB as an example, when using only the patch prompt 𝒑 𝑙 , the model's performance improved by 1.8%. Furthermore, when both the patch prompt 𝒑 𝑙 and the global prompt 𝒑 𝑔 are used simultaneously, the model's performance further increases by 0.8%. This is because the local information captured by the patch prompt 𝒑 𝑙 and the global information in the global prompt 𝒑 𝑔 can complement each other, allowing for more accurate extraction of discriminative information of individual instances. Moreover, when adding the feature-level common prompt 𝒑 𝑐 , it captures the common characteristics of all instances, resulting in an additional 1.0% improvement in model performance. Finally, with the addition of the generated specific feature prompt 𝒑 𝑠 , the complete InsVP is obtained. The specific prompt 𝒑 𝑠 extracts unique discriminative characteristics from individual instances, enabling the model to more accurately extract features for each instance and achieve the best performance. The results on other datasets also demonstrate a consistent trend to that observed on CUB.Results of Specific Prompt 𝒑 𝑠 . To further explore the impact of the specific prompt, we perform t-SNE visualization for specific prompt 𝒑 𝑠 generated by specific prompter G 𝑠 on SVHN and GTSRB datasets. As illustrated in Figure 6 below shows a notable correlation between specific prompts' distribution and sample categories. Despite being added to the middle layer of the network, specific prompts effectively capture discriminative information unique to individual instances, showing variability across different categories.The length of the feature prompt 𝐿 𝑝 and the number of MSA layers applied are two crucial hyper-parameters in our InsVP. To investigate their impact, we have conducted extensive ablation experiments. As depicted in Figure 7 , the model's performance initially improves and then declines with the gradual increase of the feature prompt's length. This behavior arises due to the overfitting caused by an excessively large number of parameters with limited training data. Regarding the experiments on MSA layers, our feature prompt attains the best results when applied across all layers of ViT. This is because, at this point, the prompt can adapt the pretrained model across all network layers, enabling the pretrained model to better accommodate downstream tasks and consequently achieve superior performance.We also conduct ablation experiments to assess the impact of image prompt fusion weight 𝛽 𝐼 and feature prompt fusion weight 𝛽 𝐹 . As depicted in Figure 7 , when these two hyper-parameters are either too large or too small, the performance is degraded. When choosing intermediate values, patch prompt 𝒑 𝑙 , global prompt 𝒑 𝑔 , specific prompt 𝒑 𝑠 , and common prompt 𝒑 𝑐 can readily complement each other, fully unleashing the potential of our method. DAM-VP via t-SNE [42] . From the visualization results, it is evident that the features extracted by DAM-VP from samples of the same category are relatively scattered, and some are mixed with features from other categories. In contrast, the features extracted by our InsVP from the same category are tightly clustered together, and they exhibit good distinctiveness from features of other categories. This is attributed to our instance image prompt and instance feature prompt, which can recognize the most essential discriminative characteristics of different category samples. 
conclusion: 
 In this paper, we propose a novel and efficient instance visual prompting method, named InsVP. In comparison to the task-level or cluster-level visual prompting methods, our instance-level InsVP achieves outstanding performance by extracting discriminative characteristics of the individual instances using the proposed instance image prompt and instance feature prompt. We demonstrate that generating prompts directly from the original image itself is more efficient and the visualization results also illustrate our prompts are closely related to individual instances. In the future, it is interesting to further investigate how to explicitly explore hierarchical instance prompting to tackle instances with various recognition difficulties.  
 
 Appendix:  


Decision: This paper proposes a novel visual prompting method that uses each image (and its feature) as the corresponding prompt, therefore improving the adaptation to down-stream tasks. Initially the paper got 1 borderline accept, 1 weak accept and 2 borderline reject. Reviewers raised questions about the novelty, the additional training cost, the ablation study, etc. The authors provided responses to these questions point to point. Two reviewers are satisfied with the responses and maintained positive ratings. Reviewer pAV2 still had concern on the training cost and maintained BR. The AC thinks the idea instance prompting is novel  and the performance improvement is valuable, and thus recommends accepting this paper.

Reviews: review: paper_summary: paper_summary: InsVP (Instance Visual Prompting) utilizes the discriminative features of individual instances within input images to enhance the recognition capabilities of pre-trained models. The methodology in this paper includes two internal modules: Image-level Instance Visual Prompting and Feature-level Instance Visual Prompting. The former extracts crucial and subtle discriminative information directly from the raw images, imposing this information onto the input images to highlight the distinctions between different instances. The latter captures the commonalities and characteristics among individual instances, inserting this information into the model's intermediate layers, thereby bolstering the pre-trained model's ability to discern the unique features of each instance. Through these two modules, InsVP achieves more precise prediction on down-stream tasks.
strength_weakness: strength_weakness: 1. InsVP provides more refined visual cues, thereby improving model performance.
2. The writing is good, and the figures are well-drawn.
limitations: limitations: 1. The Image-level Instance Prompting module appears more as an image enhancement method rather than a visual prompting method for updating based on frozen networks, seeming to depart from the definition of visual prompt tuning. 
2. The ablation studies are insufficient. The effects of using the Feature-level Prompt method alone or comparing it to patch or global prompts have not been explored. It makes me confused whether the improvements are mainly due to the image enhancement module, i.e., the Image-level Instance Prompting module.
3. Table 1 includes comparisons among methods employing various backbones. However, it's unclear whether comparing across different backbones is fair. Additionally, it's not specified which backbone is used for the plug-and-play methods.
4. The proposed method introduces additional parameters due to the incorporation of several extra modules, such as the global prompter, local prompter, and specific prompter. However, Figure 5 demonstrates that the proposed method has fewer parameters compared to VPT. The authors should clarify why their method results in fewer parameters despite the added modules. Additionally, a comparison of the inference burden is necessary.
5. Figure 3/4 have shown some intermediate results (i.e., Grad-CAM / t-SNE figures) of the proposed method compared with DAM-VP method. Why you choose DAM_VP method for such kind of comparison? Can you choose more latest methods for further comparison and analysis?
paper_summary: paper_summary: This paper introduces the Instance Visual Prompting (InsVP) method, which significantly enhances the finetuning of pretrained visual models by capturing both commonalities and unique discriminative characteristics of individual instances. Unlike existing methods, InsVP employs instance-specific image and feature prompts, leading to superior performance and efficiency. Extensive experiments demonstrate that InsVP outperforms state-of-the-art methods with fewer parameters, and the authors plan to release the code for further research and application.
strength_weakness: strength_weakness: 1.The paper reads well and its easy to follow.
2.The conept of  Instance Visual Prompting (InsVP) is interesting.
limitations: limitations: 1.The customized prompts might require significant training resources compared to current methods. Additionally, there is a concern about the generalizability of this instance-specific approach in base-to-novel tasks.

2.The feature layer prompt appears similar to the method used in maple[1]. Please explain the differences between your approach and maple.
[1] MaPLe: Multi-modal Prompt Learning
3. It is recommended to include comparative experiments with maple to better illustrate the advantages of your method.
paper_summary: paper_summary: The paper proposes InsVP for efficiently fine-tuning the pre-trained vision models. The instance image prompt extracts discriminative information from the original image. Experimental results show that InsVP outperforms existing methods with fewer parameters.
strength_weakness: strength_weakness: - This paper is well-structured.
- Extensive experiments are conducted.
- The proposed method requires fewer parameters and outperforms other SOTA methods.
limitations: limitations: - Lack of novelty. The authors do not provide a comprehensive literature review to establish its novelty. They claim to propose a novel instance visual prompting method, but the concept of visual prompting from the image itself has already been explored in previous works[1, 2, 3]. They must be discussed. Quantitative analysis would be better.

- Lack of efficiency analysis. Despite certain improvements, a detailed efficiency analysis is recommended.


[1] Prompt Generation Networks for Input-based Adaptation of Frozen Vision Transformers.

[2] Explicit Visual Prompting for Low-level Structure Segmentations.

[3] LION: Implicit Vision Prompt Tuning.
paper_summary: paper_summary: The paper propose an Instance Visual Prompting method to capture nuanced differences among instances with a few parameters. This method leverages the discriminative instance-specific information of the input image itself to enhance the recognition capability of pretrained models.
strength_weakness: strength_weakness: This paper proves the effectiveness of each module through a large number of experiments and visualizations, and the experiments are sufficient and persuasive. Full text writing expression is smooth, clear and easy to understand.The analysis results are reasonable and compared with other methods in detail.The method presented in this paper achieves satisfactory result.
limitations: limitations: It would be better if the article could analyze whether it can be applied to other fields and practical application scenarios.