AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-05 10:01:18

Abstract: Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks even under a black-box setting where the adversary can only query the model. Particularly, query-based black-box adversarial attacks estimate adversarial gradients based on the returned probability vectors of the target model for a sequence of queries. During this process, the queries made to the target model are intermediate adversarial examples crafted at the previous attack step, which share high similarities in the pixel space. Motivated by this observation, stateful detection methods have been proposed to detect and reject query-based attacks. While demonstrating promising results, these methods either have been evaded by more advanced attacks or suffer from low efficiency in terms of the number of shots (queries) required to detect different attacks. Arguably, the key challenge here is to assign high similarity scores for any two intermediate adversarial examples perturbed from the same image. To address this challenge, we propose a novel Adversarial Contrastive Prompt Tuning (ACPT) method to robustly fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. With ACPT, we further introduce a detection framework AdvDet that can detect 7 state-of-the-art query-based attacks with >99% detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.

Main paper: 
 introduction: 
 In the past decade, deep neural networks (DNNs) have made remarkable achievements across a wide range of fields, such as computer vision [16, 22] , natural language processing [14, 50] , and multimodal learning [43, 44] . Despite these advancements, studies have shown that DNNs are extremely vulnerable to small adversarial perturbations at the inference stage [48] , which are input perturbations generated to maximize the prediction error of the model. The adversarially perturbed inputs are known as adversarial examples (attacks) and the weakness of DNNs to adversarial attacks is known as the adversarial vulnerability. This has raised serious security concerns on the development of DNNs in safety-critical scenarios, such as autonomous driving [4, 17] and medial diagnosis [20, 35] .MLaaS 𝑄 ! 𝑄 " 𝑄 # 𝑄 $ ⋯ AttackerUser Queries Queries Outputs Queries OutputsAn adversary could generate adversarial attacks in either a whitebox or a black-box setting according to the threat model. In the white-box setting, the adversary has full access to the model's parameters and thus can directly compute the adversarial gradients to generate adversarial examples [36, 48] . In the black-box setting, however, the adversary can only query the target model to estimate the adversarial gradients based on the model returns (probability vectors or hard labels) [3, 8, 57] . Black-box attacks can also be achieved by transfer-based attacks, i.e., generating the attacks based on a surrogate model that is similar to the target model and then applying the generated adversarial examples to attack the target model [7, 15, 51] . Compared to white-box attacks, black-box attacks pose a more practical threat as most commercial models are kept secret from the users except their APIs. In this work, we focus on query-based black-box adversarial attacks and study the detectability of the malicious queries made by these attacks to the target model.Existing defense approaches against adversarial attacks can be categorized into adversarial training methods [36, 55 ] and adversarial example detection methods [34, 52] . Although adversarial training has been demonstrated to be one of the most effective defense methods against white-box attacks, it relies on an expensive min-max training of the model. This reduces its utility on large models as even standard training could cost millions of dollars [46] . Adversarial example detection methods, on the other hand, were mostly developed for white-box attacks and thus cannot be applied to detect the intermediate queries made by a black-box adversary to the target model.One inherent weakness of query-based attacks is that they have to query the target model many times with similar (and partially adversarial) examples generated during the attack process. And those similar queries may be easily detected and rejected by the defender, ideally making the attack fail at the first few attempts. This is known as the stateful detection against black-box attacks [9, 13, 31] . As depicted in Fig. 1 , by maintaining a list of historical queries, stateful detection works to find the most similar historical query to the current query to determine whether the current query is an adversarial example. If the similarity exceeds a certain threshold, then the current query is detected as an adversarial example.Here, the length of the list introduces a tradeoff between defense effectiveness and storage cost, i.e., a longer list will make the defense more reliable and the attack more expensive but incurs more storage (for each user).As pixel space detection is sensitive to non-adversarial transformations (e.g., rotation and translation), Chen et al. [9] proposed to leverage a pre-trained CNN to extract features and compare the mean feature similarity between the current query and the last 50 queries from the same user to identify potential attacks. This method can be easily bypassed by Sybil attacks in which the adversary creates multiple fake accounts to evade detection. The Blacklight detection method [31] computes the feature similarity (according to the hamming distance) between the current query and each of the historical queries from all users, and detects if any similarity score is above a certain threshold. Blacklight is thus robust to Sybil attacks. However, it has been shown that existing stateful detection methods all suffer from a poor tradeoff between the detection rate and false positive rate [24] , i.e., their thresholds set for high detection rate tend to cause a high false positive rate. This will greatly harm the experience of benign users. Furthermore, the above detection methods have been bypassed by an adaptive attack that attempts to generate dissimilar queries using adaptive step sizes [19] .Arguably, the key to reliable detection of query-based attacks is training a robust feature extractor that always produces similar feature vectors for any two adversarial queries crafted from the same image, even for adaptive attacks. In light of this, we propose a simple yet effective framework, Adversarial Contrastive Prompt Tuning (ACPT), to train reliable feature extractors for accurate and robust detection of query-based attacks. Specifically, ACPT finetunes the CLIP image encoder on ImageNet via prompt tuning using two types of losses: 1) contrastive losses to pull together the representations of a clean image and all its adversarial counterparts under data augmentations, and 2) adversarial losses to make it robust to adaptive attacks. Although only finetuned on ImageNet, ACPT demonstrates superb zero-shot capability and achieves the best detection performance across a wide range of datasets.In summary, our main contributions are:• We propose a novel Adversarial Contrastive Prompt Tuning (ACPT) framework that can train robust feature extractors for stateful detection of query-based attacks. • We conduct extensive experiments on 5 benchmark datasets against 7 query-based attacks, and show that ACPT can achieve an average 97% and 99% detection rates under 3shot and 5-shot detection, surpassing the best baseline by > 48% and > 49%, respectively. • We also show that ACPT is robust to adaptive attacks created by either plugging in an adaptive strategy to existing attacks or a new adaptive strategy that exploits the CLIP image encoder backbone to evade the detection. 
related work: 
 Here, we briefly review related works on query-based attacks and stateful detection. We also review existing adversarial contractive learning techniques which are closely related to our adversarial contraction prompt tuning approach.Query-based Attacks. For decision-based attacks, the confidence scores are no longer accessible to the adversary, which can only use the label information as a substitute. The Boundary attack [3] and the label-only version of the NES attack [25] are pioneering works in this field. Cheng et al. [12] proposed a novel OPT approach to formulate decisionbased attacks as real-valued optimization problems. By using the gradients sign rather than the raw gradients, Cheng et al. [11] further introduced a query-efficient Sign-OPT method to overcome the query limitations faced by all query-based attacks. Another notable method HopSkipJumpAttack (HSJA) [6] employs unbiased gradient estimation at the decision boundary to make the attack more efficient. Following this, an array of decision-based attacks, such as QEBA [32] and SurFree [37] , have been developed to reduce the number of queries required to attack unseen DNNs, or decrease the maximum allowed perturbation strength [5] . The current query (e.g., an image) is compared in the embedding space of the CLIP image encoder (finetuned by our ACPT method) with all past queries to detect whether there exists a similar historical embedding. Once the query is detected as an attack (i.e., a similar historical embedding is found), a cashed output from its last queries can be directly returned to avoid returning new information to the adversary.feature and its 𝑘-nearest neighbors found in historical queries of the same user. SD is not robust to Sybil attacks where the adversary creates many fake accounts to distribute the queries and evade userwise detection. Unlike SD, the Blacklight [31] detection method replaces the feature extractor with the Pixel-SHA probabilistic hash function, which calculates the hash representation for the input image. By further creating a global hash-table to store the historical queries of all users, it establishes a lightweight detection module that can efficiently address the problem of Sybil attacks. Based on Blacklight, PIHA [13] adopts the perceptual image hashing scheme as its feature extractor. It has been shown that stateful detection is also effective against model extraction attacks, which also require a large number of queries to the target model. For example, the PRADA [28] method detects model extraction attacks by analyzing the distribution of consecutive API queries from a user and its deviation from a Gaussian distribution. The SEAT [56] method acquires a similarity encoder via adversarial training, which enables the identification of accounts conducting model extraction attacks. A summary of these methods can be found in Table 1 .Adversarial Contrastive Learning. Contrastive learning (CL) [10, 21, 40] is a self-supervised representation learning technique that leverages large-scale unlabeled datasets to train powerful feature extractors. Recently, the concept of adversarial contrastive learning (ACL) [18, 23, 26, 29, 33, 53, 54] , has been explored as a robust representation learning technique to combine adversarial training with contrastive learning. Inspired by SimCLR [10] , Jiang et al. [26] introduced an unsupervised robust pre-training framework that effectively combines adversarial learning with contrastive pre-training. To avoid implicit knowledge of invariance caused by static augmentation, Dynamic Adversarial Contrastive Learning (DYNACL) [33] employs a dynamic augmentation schedule to bridge the gap between training and test data distributions.Xu et al. [53] further incorporated causal reasoning and robustnessaware coreset selection (RCS) to help interpret ACL and improve its performance. 
proposed detection framework: 
 We first describe our threat model, formulate the detection problem, and then introduce the proposed ACPT framework that finetunes a robust feature extractor for embedding extraction and similarity calculation. Finally, we devise detection strategies based on the similarity scores and discuss possible defense actions against the detected attack queries.In this work, we assume a query-based black-box threat model where the adversary generates adversarial examples to attack a target model by making multiple queries to the model and using the model returns to optimize the adversarial examples in an iterative manner. Here, the defender is the owner of the target model who can deploy any defense strategies to defend against potential attacks. In this work, we focus on detection-based defense, which can be deployed in parallel with other defense strategies. However, the defender does not know which user is the attacker nor when the malicious query will arrive. Therefore, the defender may have to store a large number of historical queries of all users to allow a longrange detection of malicious queries. As such, there exists a tradeoff between query storage and detection range. The goal of the defender is to detect any query-based attacks within a minimum number of attempts by the attacker, which forms a few-shot detection setting. There may also exist adaptive attacks that exploit adaptive strategies to evade the detection. We denote 𝑓 𝜃 (x) → 𝑦 as a DNN parameterized by 𝜃 , where x ∈ X is a clean sample and 𝑦 ∈ Y is its ground-truth label. In image classification tasks, x represents a clean image and 𝑦 ∈ {𝑦 1 , 𝑦 2 , . . . , 𝑦 𝑘 } is its categorization label; whereas in image captioning tasks, x is a clean image and 𝑦 is its associated caption. Given a clean sample x ∈ [0, 1] 𝑑 and a target model 𝑓 𝜃 (•), a query-based adversarial attack aims to generate an adversarial example x ′ that maximizes the loss of the model as follows:EQUATIONwhere ℓ (•) is the loss function, x ′ is an intermediate-step adversarial example, and 𝜖 is the perturbation budget. An adversarial attack can either be untargeted as formulated above or targeted toward a target label 𝑦 ′ . Please note that our work does not differentiate between targeted and untargeted attacks.A query-based black-box attack solves the above adversarial optimization problem by estimating the adversarial gradients iteratively as follows:xEQUATIONwhere x ′ 𝑡 is the intermediate adversarial example obtained at the 𝑡-th iteration, 𝜂 is the perturbation step size, sign(•) is the sign function, and ĝ is the estimated gradient based on target model output 𝑓 (x ′ 𝑡 ) using a black-box optimization method such as finite difference [8] or NES [25] .For a current query x 𝑡 , the task of stateful detection is to determine whether there exists a historical query x 𝑘 such that their similarity exceeds a certain threshold 𝜇. Formally, it is:EQUATIONwhere 𝑑𝑒𝑡 (•) is the detection function, 𝑠𝑖𝑚(•, •) is the similarity function, 𝐸 (•) is an encoder (feature extractor) that extracts the embedding of x 𝑡 /x 𝑘 , 𝜇 is a threshold hyper-parameter, and 𝑄 is an embedding bank that stores the embeddings of historical queries from all users. Here, a 𝑑𝑒𝑡 (•) value of 1 indicates an attack. Note that 𝐸 (•) is a different model from the target model 𝑓 (•) and is an adversarially finetuned CLIP [43] image encoder by our ACPT method.3.3.1 Overview. As illustrated in Figure 2 , AdvQDet consists of 2 main components: 1) the ACPT finetuned image encoder and 2) a similarity calculation module. The detection procedure of AdvQDet is as follows. For a current query x 𝑡 , it first feeds the image into the ACPT finetuned image encoder to extract its embedding. The similarity calculation module then compares the embedding with 𝑁 -1 historical embeddings of the past queries (from all users) to compute the similarity scores. If any of the 𝑁 -1 similarity scores say x 𝑘 is above a pre-defined threshold 𝜇, query x 𝑡 will be determined as a potential attack. Instead of rejecting x 𝑡 , one plausible defense action is to just return the cached output for x 𝑘 . Note that existing detection methods employ two types of strategies for embedding bank 𝑄. The SD method [9] creates a local bank for each user, while later methods Blacklight [31] and PIHA [13] maintain a global bank for all users. Our AdvQDet also adopts the global bank strategy as it is robust to Sybil attacks. Next, we will introduce the two components in detail.Tuning. As depicted in Figure 3 , ACPT adopts a two-stream contrastive prompt tuning paradigm [26] : a clean stream and a adversarial stream. In the clean stream, the two augmented views (e.g., x𝑖 and x𝑗 ) of a clean image x form a Clean-to-Clean (C2C) pair. The purpose of the clean stream is to pull together the augmented versions of the same image, making it robust to different types of image transformations. In the adversarial stream, the adversarial examples of the two augmented images are generated using PGD [36] to form an Adversarial-to-Adversarial (A2A) pair. The purpose of the adversarial stream is to make it robust to adaptive attacks that exploit adversarial perturbation to bypass the detection. Together, the two streams robustify the image encoder against both regular transformations and adversarial perturbations. Note that the clean stream itself is the standard SimCLR [10] .To exploit the superb feature extraction capability of large-scale pre-trained models, we adopt the image encoder of CLIP [43] and apply ACPT to finetune the encoder on ImageNet. ACPT adopts visual prompt tuning with learnable prompt tokens concatenated to the original input tokens. The tuning loss of ACPT is defined as follows:EQUATIONEQUATIONEQUATIONwhere 𝑝 is the prompt token, 𝐸 (•) is the CLIP image encoder, 𝑠𝑖𝑚(•, •) is the cosine similarity function, 𝜏 is the temperature, and 𝛼 = 0.5 is a hyperparameter balancing the two loss terms.Comparing the definition of ℓ ACPT and Eq. ( 3), one might find that ℓ ACPT directly optimizes the feature similarity between the clean and adversarial image pairs. This effectively reduces the difference between variants of the same image in the latent space, making the detection of query attacks much easier.Following prior works [13, 31] , we extract and save the embedding of each query image into an embedding bank 𝑄. The embedding bank is maintained globally for all users so as to be robust to Sybil attacks. Two problems arise with the embedding bank: 1) the storage cost and 2) the computational cost. The two costs can be reduced by using the techniques introduced in [9] . Next, we will provide an analysis of the two costs and show that it is practically feasible to store a global embedding bank and perform similarity search efficiently.In terms of the storage cost, each query results in a vector embedding with dimension 𝑑 = 512, which takes 2048 bytes for float32 precision. Suppose there are 1 million users with each user querying 100 times, the storage it takes to store all these query embeddings is 190.73 GB. By switching to float16 precision, the storage can be reduced to 95.37 GB.In terms of computational cost, one can use the Automatic Mixed Precision (AMP) technique to reduce the memory cost and accelerate computations without sacrificing the detection performance. AMP automatically determines the appropriate precision-single or half-for each operation. When calculating the cosine similarity between an individual embedding vector and each embedding in the embedding bank, the computational complexity is 𝑂 (𝑛 × 𝑑), where 𝑛 is the number of embeddings in the bank and 𝑑 is the dimension of the embedding vector. There are established techniques we can use to speed up high-dimensional similarity search, such as product quantization (PQ), hierarchical navigable small worlds (HNSW), and locality sensitive hashing (LSH). Popular similarity search tools like clip retrieval [2] , Faiss [27] , and AutoFaiss all provide efficient solutions for searching over a large-scale vector database. Here, we conduct an efficiency test to compute the cosine similarity between two vectors of dimensions (1, 512) and (1𝑚, 512) using an NVIDIA RTX 3090 GPU, CUDA 11.3, and Pytorch v1.12.0. It takes 8.29 and 2.63 milliseconds for float32 and float16, respectively. These costs are manageable for an AI company to run a commercial product/service that supports up to 1 million users.Once a query is detected to be an attack, there are a few possible defense actions that can be taken by the defender: 1) rejecting the query, which is applicable when the false positive rate is low as otherwise may harm user experience; 2) limiting the query number and frequency of the user which will cause the attacker's attention; 3) returned intentionally perturbed outputs to the user which still has the risk to leak gradient (or other) information; 4) banning accounts or blocking IP addresses which is an aggressive action that should be taken only in extreme cases; and 5) simply returning the cashed output for the previous similar query which is a plausible action that does not expose new information to the user nor harm the user experience. 
experiments: 
 We evaluated our detection method against 7 state-of-the-art querybased attacks and 3 types of adaptive attacks. We first describe our experimental setting and then present the results of 1) defense effectiveness across different datasets, 2) robustness to adaptive attacks, and 3) ablation study.Datasets and Models. We experiment on 5 benchmark datasets: CIFAR-10 [30] , GTSRB [47] , ImageNet [45] , Flowers [39] , Pets [42] . We utilize ImageNet pre-trained models (such as ResNet20, ResNet101, and ViT-B/16) and then fine-tune them on the other four datasets. A summary of these datasets and the corresponding models can be found in the Appendix. Attack Configuration. We evaluate against 7 query-based attacks, including Boundary [3] , HSJA [6] , NESS [25] , QEBA [32] , Square [1] , SurFree [37] , and ZOO [8] , as described in Section §2. We also apply an adaptive strategy called Oracle-guided Adaptive Rejection Sampling (OARS) [19] to enhance the above query-based attacks and evaluate against these enhanced attacks. OARS utilizes an adapting distribution and resampling technique for gradient estimation, aiming to evade stateful defenses during the generation of adversarial examples. Throughout the experiment, we execute each attack until an adversarial example is successfully crafted or the maximum query limit is reached, whichever occurs first. The hyperparameters for these attacks are set following the Adversarial-Robustness-Toolbox(ART) library [38] . For the attacks, we set the perturbation budget to 𝜖 = 0.05 and limit the query budget to 100, 000. For CIFAR-10 and GTSRB datasets, we randomly choose 1,000 images from their respective test sets, uniformly across all categories. For ImageNet, Flowers, and Pets datasets, due to the high computational costs of query-based attacks, we select 100 images randomly from the validation/test sets. Defense Configuration. For existing stateful detection methods, we use their originally proposed configurations, as detailed in Table 1 . Specifically, for SD [9] defense, we set the number of neighbors to 𝑘 = 50 and the detection threshold to 𝜇 = 10. For Blacklight [31] , the quantization step is set to 50, with window sizes of 20 for CIFAR-10 and 50 for ImageNet. PIHA [13] adopts a block size of 7x7 and a detection threshold of 𝜇 = 0.05. There are 𝐾 = 20 learnable prompt tokens, optimized by SGD and adjusted by cosine annealing. For detection, we set a similarity threshold of 𝜇 = 0.95 for low-resolution datasets CIFAR-10 and GTSRB, and 𝜇 = 0.9 for high-resolution datasets ImageNet, FLowers, and Pets. Performance Metrics. We consider three performance metrics: 1) attack success rate (ASR), which is the percentage of successful adversarial examples under the attack budget; 2) 3/5-shots (queries) detection rate (DR) which is the successful detection rate when the defender sees 3/5 of the queries (i.e., a clean query followed by a sequence of adversarial queries), and 3) mean detection counts (mDC) which calculates the average number of queries required for the defender to detect each attack. We compare our AdvQDet method with 7 existing stateful detection methods. For a fair comparison, we adopt the same defense pipline for all methods. I.e., we detect each query based on the historical queries from all users, with only the similarity score is computed by different detection methods. The detection performance results are reported in Table 2 , where the 3-4 columns report the results of no defense. It is evident that, although most attacks can achieve a high ASR (nearly 100%) in the absence of detection, they often require a large number of queries to succeed. According to the results, the Square attack is the most efficient and effective attacks as it requires the minimum number of queries and achieves an ASR of 100% across all datasets.For the detection methods, our AdvQDet achieves the best average performance of 0% ASR, 97%/99% 3/5-shot detection rate, and an average of 2.66 query counts for successful detection, surpassing existing methods Blacklight and PIHA by a huge margin. Moreover, AdvQDet demonstrates the best performance and almost 100% 3/5shot detection rates in most scenarios. However, it is not always the best, for example, the Blacklight detection method works better against the NESS attack than AdvQDet in terms of 3/5-shot detection rates. This is because NESS attack uses a large Gaussian noise distribution to estimate the adversarial gradients which tend to cause large distortion to the query images and thus the features. However, Blacklight extracts the hashing of the image which is relatively robust to large perturbations. However, Blacklight fails badly against HSJA, QEBA, and SurFree attacks with almost 0% 3/5-shot detection rates. It is worth mentioning that AdvQDet is very close to Blacklight against NESS but can detect the attacks within fewer queries.Although query-based attacks generally require many queries while detention only needs a few queries, there are still attacks that can bypass existing detection methods Blacklight and PIHA. For example, the Square, NESS, and Boundary attacks on high resolution datasets ImageNet, Flowers, and Pets. By contrast, not a single existing query-based attack can evade our detection, leaving an ASR of 0% in all scenarios. Efficiency is another advantage of our AdvQDet method, i.e., it only takes 2.66 queries on average to detect all 7 attacks. Note that, the first query made by most attacks is a clean image, the second query is often an initialized image with Gaussian noise, and the third query is an adversarial query. This means that our method can detect most of the attacks based on the first two queries, for example, against HSJA and QEBA attacks.Here, we evaluate the robustness of our method to adaptive attacks where the attackers are aware of our detection pipeline. Particularly, we consider three adaptive attacks: 1) using OARS [19] adaptive strategy to boost existing attacks; 2) the attacker knows the backbone (CLIP image encoder) of our AdvQDet; and 3) white-box attacks where the attacker knows every detail of our detector (but the target model is still black-box).OARS Adaptive Attack. OARS employs step size adaptation and resampling mechanisms to evade stateful detection. We boost existing attacks including Boundary, HSJA, NESS, QEBA, Square, and SurFree using the OARS adaptive strategy. We did not consider the ZOO attack as its adaptive strategy is not compatible with OARS and it is also omitted from the OARS paper [19] . The robustness results on CIFAR-10 and ImageNet datasets are shown in Table 3 . It is clear that when there is no defense, all adaptive attacks achieve an ASR of ≥ 99 with the query number increase significantly on high resolution images (ImageNet).Our AdvQDet is robust to OARS adaptive attacks and can successfully detect all 6 adaptive attacks within an average of 3 shots while reducing the ASR to 0%. The SD detection method however fails on ImageNet as its feature extractor is dataset-dependent and thus is not applicable to ImageNet images. Since SD requires the last 50 queries to detect the current, the mean detection counts are all above 50. The Blacklight and PIHA have both been bypassed by all adaptive attacks, where the ASR jumps up to 37% -100%. Interestingly, Blacklight is more susceptible to adaptive attacks on low-resolution dataset CIFAR-10 while PIHA is more vulnerable on both low and high-resolution datasets CIFAR-10 and ImageNet.The Backbone is Compromised. Here, we test when the attacker knows the CLIP image encoder used in AdvQDet (but not the visual prompt token). In this case, the attacker can white-box attack the CLIP image encoder while query attacking the target model. Specifically, the attacker adopts an alternating optimize strategy to first perform one step (query) black-box attack and then 10 steps of white-box PGD attack. As shown in Figure 4 , AdvQDet is also robust to this adaptive attack, maintaining a high similarity score for the first 50 steps of queries. Moreover, AdvQDet becomes more robust when we increase the token length of ACPT.White-box Attack. In this case, we follow a similar adaptive pipeline as in the above backbone adaptive attack setting, but the attacker directly attacks our ACPT-tuned image encoder. The results are also presented in Figure 4 . The result indicates that AdvQDet is moderately robust to white-box attacks with a slightly reduced similarity score, and increasing the token length of ACPT can effectively increase the chance of the attack being detected. Note that in both experiments, the detection is deemed to be successful whenever the similarity score is above the threshold which occurs within the first 5 queries. We also observed that white-box attacks against our AdvQDet took roughly 100x more queries to converge. These results suggest that with ACPT, we can have a reliable query attack detector with good effectiveness, efficiency, and robustness. Effect of Prompt Token Length. Here, we analyze the impact of prompt token length of ACPT on the detection performance, with varying token lengths 𝐾 ∈ [0, 30]. Note that when 𝐾 = 0, the ACPT-tuned encoder degenerates to the vanilla CLIP image encoder. As depicted in Figure 5 , our AdvQDet can reliably distinguish between benign and adversarial queries, assigning high average similarity scores (close to 1 almost everywhere) to adversarial queries. The difference is more pronounced as the token length of ACPT increases.  
limitation: 
 As a stateful detection method, our AdvQDet also faces certain limitations that deserve further research. Notably, it cannot defend against transfer-based attacks as they do not need querying the target model. This limitation can potentially be addressed by incorporating white-box adversarial example detection methods into the pipeline of AdvQDet. The storage and computational costs mark another limitation of AdvQDet. More effective partitioning and acceleration techniques can be developed in future work to facilitate the industrial deployment of AdvQDet. On the other hand, besides its effectiveness, efficiency, and robustness, AdvQDet has the potential to be applied to detect multimodal query-based attacks against vision language models (VLMs) like GPT-4V [41] . Although there is still much room for improvement, we believe AdvQDet offers a reliable solution for detecting query-based adversarial attacks. 
conclusion: 
 In this paper, we proposed a novel stateful detection framework to detect query-based black-box adversarial attacks. Our work is motivated by the observation that query-based attacks launch multiple visually similar queries to the target model, which might be easily detected by a robust feature extractor (image encoder). To this end, we propose an efficient tuning-based method called Adversarial Contrastive Prompt Tuning (ACPT) to robustify the CLIP image encoder on ImageNet. The ACPT-tuned serves as a general-purpose encoder for the detection of query-based attacks, and demonstrates strong zero-shot generalization capability across different datasets.With ACPT, we introduce the AdvQDet framework that extracts and saves the embeddings of the query images and maintains a global embedding bank for all users. AdvQDet computes the embedding similarity between the current query and all historical queries to identify whether the query is malicious (similar to an existing one). We demonstrated the effectiveness, efficiency, and robustness of AdvQDet against existing query-based attacks, adaptive attacks, and even white-box attacks. Our work showcases the possibility of achieving strong and consistent defense against query-based adversarial attacks.  
 
 Appendix:  


Decision: The reviewers acknowledged the rebuttal letter, and have merged to similar ratiting for accepting this work. There are still some major concerns about providing more details and explanations about the failure of ACL and the application of the proposed method in zero-shot scenarios, providing more quantitative results to support the conclusion. Thus, I recommend the acceptance as a poster work.

Reviews: review: paper_summary: paper_summary: The manuscript introduces the Adversarial Contrastive Prompt Tuning (ACPT), a method designed to enhance the robustness of feature extraction against query-based black-box adversarial attacks. By fine-tuning the CLIP image encoder with contrastive and adversarial losses, ACPT generates similar embeddings for adversarial queries derived from the same image, demonstrating improved detection capabilities in stateful detection systems.
strength_weakness: strength_weakness: 1. The paper tackles a significant and timely challenge in adversarial attack detection by developing a feature extractor capable of reliably identifying similar adversarial queries.
2. The ACPT framework shows exceptional detection rates, significantly outperforming existing methods on multiple benchmarks, as evidenced by extensive experiments demonstrating up to 99% detection accuracy within 5 shots across diverse datasets.
3. The framework is robust against adaptive attacks, designed specifically to evade detection methods, highlighting its potential for practical security applications.
limitations: limitations: 1. The paper primarily focuses on query-based attacks, yet the current trend in the field is leaning towards both query- and transfer-based methods, which are not addressed in the experiments. This oversight limits the generalizability and applicability of the proposed method. Relevant studies in this area include:

   [1] Y. Feng, et al., "Boosting black-box attack with partially transferred conditional adversarial distribution," CVPR, 2022.

   [2] J. Liu, et al., "Difattack: Query-efficient blackbox adversarial attack via disentangled feature space," AAAI, 2024.

   [3] F. Yin, et al., "Generalizable black-box adversarial attack with meta learning," IEEE Trans. Pattern Anal. Mach. Intell., 2024.

2. There is a noticeable absence of an ablation study on the hyperparameter $\alpha$, which could provide deeper insights into the sensitivity and tuning of the model.
3. The manuscript lacks experimental results on the storage and computational costs associated with the AdvQDet, which are crucial for evaluating the practical deployment of the proposed framework.

Suggestions for Improvement:
- Expand the scope of the experiments to include both query- and transfer-based adversarial methods to enhance the relevance and applicability of the ACPT framework.
- Include an ablation study focusing on the hyperparameter $\alpha$ to detail its impact on the performance and optimization of the model.
- Provide a detailed analysis of the storage and computational requirements of the AdvQDet system to assess its feasibility for real-world applications.
paper_summary: paper_summary: This paper proposes a defense against query-based adversarial attacks that utilize similar queries to estimate local gradients. The method involves using a CLIP image encoder to embed each incoming query. By comparing these embeddings to historical ones, the system returns the cached output of the query whose embedding closely matches the incoming query's embedding, thereby preventing new information leakage to the attacker.
strength_weakness: strength_weakness: 1. This paper is well-written, and each concept is clearly explained.
2. The proposed ACPT method effectively detects various query-based attacks across different benchmark datasets, surpassing the state-of-the-art (SOTA) baseline.
limitations: limitations: 1. **Limited Novelty:** The proposed Adversarial Contrastive Prompt Tuning (ACPT), which aims to train robust feature extractors, is highly similar to existing adversarial contrastive learning methods[18, 23, 26, 29, 33, 53, 54]. Both approaches share the core idea of combining adversarial training with contrastive learning.
   
2. **Underexplained Key Insights:** The role of contrastive vision prompt tuning and its advantages and disadvantages compared to other contrastive learning methods are not sufficiently explained. For instance, the integration of the CLIP encoder may limit the embedding power of the image encoder due to the necessity to align with the text encoder. Additionally, if the adversarially trained image encoder is already robust to adversarial inputs, why not use it directly for classification? Maintaining an embedding encoder and historical embeddings incurs additional costs, but it is unclear how much this enhances robustness.

3. **Computational Cost:** The training cost of adversarial contrastive learning is not provided. Does this method share the same computational drawbacks as adversarial training, as mentioned in lines 101-105?

4. **Missing Details of the Adaptive Attack:** There is a lack of information about the adaptive step size and the reasons for the adaptive attack's failure.
paper_summary: paper_summary: The paper employs the CLIP model as an image encoder and integrates adversarial contrastive prompt tuning to enhance the discriminative performance of adversary embeddings. The results indicate superior performance compared to other stateful detection methods using simpler encoders.
strength_weakness: strength_weakness: * The paper introduces a novel stateful detection scheme that leverages the CLIP encoder combined with contrast learning, showcasing a robust approach to defense against query-based adversarial attacks.

* The evaluation of existing attacks and defensive baselines demonstrate the effectiveness of the proposed method.
limitations: limitations: * The paper does not explore alternative adversarial sample optimization strategies beyond PGD in the context of Adversarial Contrastive Prompt Tuning, raising concerns about the generality of the proposed method.

* It is not clear whether the CLIP model requires ACPT for each dataset as well as whether the benefits observed on high-resolution datasets (e.g., ImageNet) could effectively transform to lower-resolution datasets.

* The explanation of "white-box attacks" in Section 4.3 is vague, particularly in how an attacker uses knowledge of the CLIP image encoder.

* The benefits of contrast training on adversarial samples appear trivial as depicted in Figure 5. Given the high threshold (e.g., 0.9 on ImageNet), the original CLIP encoder might already sufficiently classify adversarial samples without additional training.