TARP-VP: Towards Evaluation of Transferred  Adversarial Robustness and Privacy on Label  Mapping Visual Prompting Models

Authors: zhen  chen
yi  zhang
fu  wang
xingyu  zhao
xiaowei  huang
wenjie  ruan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 17:09:09

Abstract: Adversarial robustness and privacy of deep learning (DL) models are two widely studied topics in AI security. Adversarial training (AT) is 
an effective approach to improve the robustness of DL models against adversarial attacks. However, while models with AT demonstrate enhanced robustness, they become more susceptible to membership inference attacks (MIAs), thus increasing the risk of privacy leakage. This indicates a negative trade-off between adversarial robustness and privacy in general deep learning models. Visual prompting is a novel model reprogramming (MR) technique used for fine-tuning pre-trained models, achieving good performance in vision tasks, especially when combined with the label mapping technique. However, the performance of label-mapping-based visual prompting (LM-VP) under adversarial attacks and MIAs lacks evaluation. In this work, we regard the MR of LM-VP as a unified entity, referred to as the LM-VP model, and take a step toward jointly evaluating the adversarial robustness and privacy of LM-VP models. Experimental results show that 
the choice of pre-trained models significantly affects the white-box adversarial robustness of LM-VP, and standard AT even substantially degrades its performance. In contrast, transfer AT-trained LM-VP achieves a good trade-off between transferred adversarial robustness and privacy, a finding that has been consistently validated across various pre-trained models. Code is available at https://github.com/TrustAI/TARP-VP.

Main paper: 
 introduction: 
 Deep learning models have gained great success, yet concerns regarding their security continue to grow, as they are susceptible to various attacks [30, 13, 23, 29, 35] . In addition to the deep learning models, training samples are also the key to this success. Consequently, attacks targeting the relationship between training samples and models have emerged, such as adversarial attacks [23, 14, 43, 37] and membership inference attacks (MIAs) [25, 28, 30, 32] . Adversarial attacks are gradient-based methods that introduce imperceptible perturbations on inputs, generating adversarial examples (AEs) that cause the target models to give incorrect predictions. Adversarial training (AT) [23] , proposed by Szegedy et al. has been recognized as one of the most effective defenses against such attacks. The basic idea of AT is to incorporate AEs into the training process, resulting in significantly improved performance under adversarial attacks compared to standard training (ST).MIAs, on the other hand, aim to determine whether a specific sample was part of the model's training data. Various ideas exist for performing MIAs, e.g., shadow model-based attacks [30] and prediction confidence-based attacks [39, 28, 32] . The success of these attacks largely depends on the model's generalization error during training and testing [31] . Therefore, models with lower generalization errors are inherently more resistant to such attacks.AT-trained models exhibit more severe privacy risks compared to ST: (1) larger generalization error, manifested in both natural and adversarial examples [32] ; (2) higher sensitivity on training data compared with ST [32] ; (3) robust overfitting [27] , where the model's adversarial robustness declines despite the natural accuracy continuing to increase at a certain training stage. These issues result in a negative trade-off between adversarial robustness and privacy. As illustrated in Fig. 1 , while AT significantly enhances adversarial robustness, it is more susceptible to MIAs, especially after robust overfitting, i.e., between 100-150 epochs in Fig. 1d , where the MIA success rate increases significantly. Notably, the above conclusions are only for general deep-learning models. Visual prompting (VP) [3] is a model reprogramming (MR) [10, 7] technique for pre-trained models, used for downstream image classification tasks. Initially, VP involves adding a single, input-agnostic prompt to input images to enhance a pre-trained model's generalization ability. Label mapping (LP) further improves VP's performance by mapping source labels to target labels, denoted as LM-VP, which exhibits strong performance in downstream tasks [3] . In this paper, we regard a general pre-trained model after LM-VP as a new model, and its security remains under-explored, including its susceptibility against AEs and MIAs, and compatibility with AT. We consider two forms of AT: the standard AT for white-box adversarial robustness and transfer AT for black-box transferred adversarial robustness, i.e., generating adversarial examples through another threat model. We empirically demonstrate that the intuitions and relationships between adversarial robustness and privacy observed in general models do not always hold for the LM-VP model. In summary, our contributions lie in:• From a novel perspective of considering LM-VP as a distinct model, we conduct the first evaluation of its security, i.e., (transferred) adversarial robustness and privacy; • Based on the concept of transfer attacks, we implement transferred adversarial training for the LM-VP model to enhance its transferred adversarial robustness; • We empirically demonstrate that intuitions regarding privacy in general models do not necessarily apply to LM-VP models. Furthermore, we show that standard AT is invalid for LM-VP, while transfer AT on LM-VP exhibits a superior trade-off between transferred adversarial robustness and privacy across various pre-trained models.2 Related Work and Background 2.1 Visual Prompting.The prompt technique [18, 20, 15] was originally employed in NLP tasks. In essence, it modifies the original input text to enhance specific task performance without altering the parameters of the pre-trained model. For instance, prompts indicating sentiment can be added for text classification [24] , while prompts indicating the target language can be used for translation tasks [42] . Bahang et al. [3] first transfers this idea to computer vision tasks, i.e., VP. Compared to traditional transfer learning methods like fine-tuning [33] and linear probes [1] , VP does not modify the parameters of the pre-trained model. In [19] , they conclude the impact of the rescale ratio on the training of LM-VP models, i.e., a larger rescale ratio yields better performance. However, an excessively large rescale ratio can also lead to overfitting to the target domain. In this section, we compare two rescale ratios, corresponding to the two cases shown in Fig. 2 . In terms of performance, there is only a minimal gap between the two ways.Natural Acc Adversarial Acc 0  
adversarial robustness evaluation: 
 Adversarial robustness is an important metric for evaluating model robustness, referring to a model's performance under adversarial attacks. Adversarial attacks target image gradients, iteratively introducing imperceptible perturbations to images to generate AEs. These AEs often cause standard-trained deep learning models to misclassify them with high confidence. Commonly used adversarial attacks include PGD [23] , FGSM [14] , and CW [4] attacks. AT is an approach to improve the adversarial robustness of deep learning models. It can be formulated as a min-max problem, where the inner maximization searches for perturbations that maximize the loss, while the outer minimization optimizes the model, i.e.,EQUATIONVarious strategies exist to solve the inner maximization in AT, including PGD-AT [23] , TRADES [43] ,and MART [37] , etc. LOAT [40] boosts AT via a Fisher-Rao norm-based regularization, SEAT [36] extends AT to medical segmentation and FAAL [44] ensure both robustness and fairness during AT, Chen et al. [8] proposes NRAT to enhance adversarial robustness under noisy labels. While robustness evaluation mainly focuses on discriminative models, Zhang et al. [45] introduces a robustness notion of text-to-image (T2I) generative models and proposes the ProTIP framework for evaluation.MIAs refer to determining whether a given data point was part of the training data for a trained model, raising significant data privacy concerns. Shokri et al. [30] introduce the first MIA on classification models, i.e., shadow training attack, which involves creating several shadow models that simulate the target model, with these shadow models trained on data records similar to those used for training the target model. An attack model is then trained to recognize the relationship between the members of the shadow models' training data and the shadow models' outputs, which turns out to be a binary classification. This attack model can subsequently infer the membership of the target model's training dataset. There are two findings in this work [30] : (1) the higher the degree of the model's overfitting, the higher the attack's success rate, and (2) the more complex the training dataset, the higher the MIA success rate. Intuitively, increasing the number of shadow models improves attack performance by providing more samples for training the attack model, but this also requires more computational resources. Yeom et al. [39] propose threshold-based MIAs, which compare the target model's prediction confidence for the true label against a certain threshold. This method achieves performance similar to the shadow model training method with significantly reduced computational resource consumption. Song et al. [31] further proposes the class-dependent thresholds for a more powerful attack and implements the MIA based on prediction entropy. 
robustness and privacy evaluation of visual prompting models: 
 In this section, we first provide an overall design of LM-VP models, including the prompt designs, label mapping techniques, LM-VP model training, as well as the tricks we adopt. Then, we analyze the white-box adversarial robustness of LM-VP, demonstrating that pre-trained models largely influence its white-box adversarial robustness. We further propose transfer AT to enhance black-box transferred adversarial robustness. Finally, we analyze the intuition between LM-VP models and privacy.LM-VP model aims to keep the parameters of the pre-trained models fixed while performing input transformation and output transformations, i.e., label mapping. Therefore, we divide the LM-VP model into three parts: (1) prompt generation; (2) label mapping; and (3) model training.For the prompt generation, we first rescale the images from the target domain under a certain rescale ratio to a size smaller than that of the source domain. Then, trainable noise w 1 , i.e., prompt is added around the image to ensure the final image size matches that of the source domain. Therefore, the length (height and width) of each pixel patch p isEQUATIONwhere H 1 and W 1 represent the height and width of the source domain, H 2 and W 2 represent the height and width of the rescaled target domain, then the final shape of prompts P isEQUATIONwhere C is the image channels, [H 1 /p + W 1 /p -4] represents the amount of pixel patches in each channel. In vanilla VP [3] , they rescale the target images to the size of the source domain and replace the edges with random noise, i.e., prompts. In comparison, although the final prompts P are the same, we preserve the edge information of the target domain images, which is more intuitively reasonable as we retain all the information of target images. Preserving the edge information might help increase the correlation between prompts and images during training. Fig. 2 shows the difference between these two ways of prompt generation.For output transformation, vanilla VP uses random mapping, randomly selecting some labels from the source domain to match those of the target domains and discarding the remaining unused labels. However, this random mapping often leads to a performance drop in VP as it may ignore some important information in the unused labels [19] . Therefore, we consider label mapping as a trainable component: using a fully connected layer to train the mapping from source labels to target labels, which is similar to [19] and [2] . Consequently, in LM-VP, we introduce two trainable parameters: the prompt noise parameters w 1 and the parameters w 2 in the label mapping layer f ℓ (w 2 ; θ 2 ).For LM-VP model training, our objective is to modify the prompts by updating the noise parameters w 1 , and the parameters w 2 of FC layers of the label mapping. During the testing phase, the same optimized prompts are applied to all test data, fed into the frozen pre-trained model, and finally output the label mapping results. LM-VP relies on the pre-trained model and label mapping, indicating that it does not require a large target dataset. Therefore, we can train LM-VP models using a subset of the target dataset without a significant performance drop compared with training on the entire target dataset, which significantly improves training efficiency. In contrast, general models tend to underfitting when trained on small subsets, leading to poor generalization performance. Although the LM-VP model does not heavily rely on training data, insufficient data can still affect its generalization. (2) Bottom: rescale a target image to a size smaller than the source domain and add prompts to make it the same size as source domain. Therefore, we use the SAM [12] version of SGD optimizer to update the weights of w 1 to improve the LM-VP model's generalization, i.e.,EQUATIONwhere η is the learning rate, ϵ is the parameter to maximize the loss function L, and λ is the weight decay. Given the prediction on source domain ŷS , the final prediction label on target domain ŷT is obtained by ŷT = softmax (f ℓ (ω 2 ; ŷS )) . The LM-VP model also exhibits the property of rapid convergence, i.e., it can quickly achieve a near-optimal performance and then remain steady with continued training, for both the natural samples and adversarial samples, as shown in Fig. 4 , it takes about 10 epochs to convergence for both training and testing. Therefore we can set a small number of epochs to improve training efficiency on LM-VP models. We also notice that the standard AT is invalid in LM-VP, with really poor results in both natural and adversarial performance, see Table . Regarding how pre-trained models affect downstream adversarial robustness, [38] and [34] provide more insights, e.g., Yamada et al. [38] conclude that network architecture is a strong source of robustness in transfer learning. In this sense, different pre-trained models may lead to different boundary relationships between adversarial robustness and privacy, evaluating LM-VP using white-box adversarial attacks may make it difficult to reach consistent conclusions.For the evaluation of the transferred adversarial robustness of the LM-VP model, we use another general model as the threat model to produce adversarial examples and train the LM-VP model to defend against them. In this scenario, the intensity of the transfer attack remains constant once the threat model is selected. This consistency holds true regardless of the chosen pre-trained model. This inherent consistency is thus helpful for exploring and establishing a sensible relationship between transferred adversarial robustness and privacy within LM-VP models. Compared to white-box adversarial robustness which is heavily influenced by the pre-trained models, utilizing transferred adversarial robustness serves as a more reliable and insightful evaluation method for LM-VP models.Within the framework of transfer AT, the LM-VP model which comprises VP, a pre-trained model, and LM, is treated as a unified black-box system, A fixed-parameter threat model, excluded from the training process, is employed to generate AEs x ′ , and then train LM-VP models using adversarial loss, this transfer AT consistently optimizes in the same direction since the attack remains constant.In our work, we use the resistance of a model against MIAs to reflect its privacy. According to [32] and [31] , AT-trained general models exhibit larger generalization errors and higher sensitivity to training data, wherein generalization errors and sensitivity are two key factors that influence the success of MIAs on general models, thus indicating a contradiction between adversarial robustness and privacy for general models. However, in the LM-VP model, the architecture of pre-trained models appears to be the more significant factor affecting its (white-box) standard adversarial robustness (Table. 1). As a result, we are not able to establish a reliable boundary relationship between its standard adversarial robustness and privacy, which has prompted us to focus primarily on the relationship between its transferred adversarial robustness and privacy.In the training of LM-VP models, the parameters (weights and biases) of the pre-trained models are fixed, and the trainable parameters are noise parameters and label mapping parameters. This significantly reduces the influence of the input (both natural and adversarial samples) on the LM-VP model compared to general models, which also enables effective training of LM-VP models with a small subset of data, indicating lower sensitivity of LM-VP models to training data, the results in Fig. 3 also support this statement. Additionally, the generalization ability of LM-VP models mainly relies on the pre-trained models, which have been trained on large-scale datasets and learned rich universal features. With fixed model parameters, the risk of overfitting is reduced.As shown in Fig. 4 However, apart from generalization errors and sensitivity, there might be other factors influencing the LM-VP model's resistance to MIAs, such as the prior knowledge embedded in different pre-trained models. In our experiments, we empirically demonstrate that MIA analyses applied to general models do not always hold for LM-VP models, and transferred adversarial robustness and privacy can be improved simultaneously using transfer AT. 
experiments: 
 In this section, we conduct comprehensive experiments to evaluate the performance of LM-VP models under transferred adversarial attacks and a threshold-based MIA. Regarding the trade-off among standard accuracy, transferred adversarial robustness, and MIA success rate, we comprehensively compare different pre-trained models in LM-VP models. We conduct main experiments on CIFAR-10 and additional experiments on Tiny-ImageNet to show the good generalization performance of transfer AT. We implement all experiments on a server with an RTX3090 GPU.For adversarial attacks, all experiments follow the standard settings: ℓ ∞ threat models for all methods, the perturbation limit ϵ = 8/255, and step size 2/255, we mainly choose ResNet18 as the threat model. For LM-VP models, we use the source models pre-trained on 224x224 ImageNet. For training, we follow the settings in [19] , i.e., SGD with SAM technique and a momentum of 0.9 to optimize the LM-VP defense models, the total training epoch is 20. We choose ResNet50 [17] , ResNet152 [17] , WideResNet-50-2 [41] , VIT [9] , Swin Transformer [21] , ConvNext [22] , and EVA [11] To evaluate the performance of LM-VP models using standard training, i.e., its training loss is given by Based on the parameter capacity of the pre-trained models, we regard them as small, medium, and large models. Specifically, ResNet50, ResNet152, and WRN-50-2 are small models, ViT and Swin Transformer are medium models, and ConvNext and EVA are large models. We report only the natural accuracy during training, while different attacks are employed during testing. The results in Table . 2 show that: (1) There is a clear hierarchy in natural accuracy and transferred adversarial robustness based on the size of the pre-trained models, i.e., small models have a natural accuracy below 90%, medium models around 92%, and large models around 98%; for transferred adversarial robustness, small models are below 45%, medium models range from 50% to 60%, and large models range from 85% to 90%; (2) In LM-VP models, the transferred adversarial robustness is not significantly affected by the size of the threat model, i.e., a larger threat model (WRN-34-10) may not be more challenge compared with ResNet18. For instance, small and medium models sometimes have higher transferred adversarial robustness achieved under WRN-34-10; (3) In LM-VP models, even when more attack steps are used, the transferred adversarial robustness of the test data often remains higher than that of the training data set, although more attack steps being considered more powerful (comparing PGD-10 tr and PGD-20).EQUATIONWe implement the transfer AT of LM-VP models proposed in Section 3.3 and report the performance in Table . 3. Specifically, during training, we use PGD attack with 10 steps to generate AEs:EQUATIONx t+1 = Π x+S (x t + αsign(∇ x L(θ, x t , y)), (8) x 0 is obtained by perturbing x with random noise σ sampled from the normal distribution N (0, 1), t denotes the current attack step, α is the step size, Π denotes the projection function, S ⊆ R d denotes the perturbation set of AEs, we train LM-VP models with the following training loss:EQUATIONwhere x ′ i denotes the AE after PGD. To ensure consistency with the ST in Table . 2, we report the results under the same metrics. Consistent with our findings in Section 4.2, AEs generated by different The results in Table . 3 indicate that: (1) Transferred AT significantly enhances the transferred adversarial robustness at the cost of reduced natural accuracy. Specifically, the transferred adversarial robustness improvement is around 20%-35% for small models, 20%-25% for medium models, and 3%-6% for large models; (2) Compared to the standard-trained LM-VP models, the transferred adversarial robustness of the training set (PGD-10 tr ) is usually higher than that of the test set (PGD-20) in transferred AT-trained LM-VP models.In this section, we evaluate the privacy performance of LM-VP models under the threshold-based MIA. Our attack implementation is based on [16] , the MIA success rate with a threshold η is given by:EQUATION)where the η optim is obtained by computing all possible η that maximizes the MIA success rate, i.e.,EQUATION)This attack is mainly based on the model generalization error, i.e., models with higher generalization error are more susceptible to the attack. Conversely, the success rate should approach 50% for a model with little generalization error. However, this principle does not always consistently apply to LM-VP models. From Table . 4, for standard-trained LM-VP models, only the VIT and Swin Transformer can effectively resist the attack with an MIA success rate near 50%. Another observation is the MIA success rate on AEs is lower for the standard-trained LM-VP models.For transferred-AT trained LM-VP models, the MIA success rate for most cases is around 50%, markedly reducing the risk of training data privacy leakage. This indicates that transferred adversarial robustness and privacy can be simultaneously achieved in LM-VP models. One plausible explanation is that during transfer AT, the original training examples are perturbed before feeding into the model, thus these data are not exposed to the trained model, this may help mitigate the MIA issue since LM-VP models also do not suffer from large generalization error (Table . 2 and Table . 3) and increased training data sensitivity (Fig. 3 ) and transfer AT do not train the original training examples.To demonstrate the efficiency of transferred AT on LM-VP models, we provide the results on Tiny-ImageNet which has a resolution of 64x64 and contains 200 classes, results shown in Table . 5, LM-VP models with transfer AT improve transfer adversarial robustness by 3%-24% and mitigate the MIA success rate by 3%-12% compared to LM-VP models with standard training. 
conclusion: 
 In this paper, we regard the models trained using the LM-VP technique as a novel model type and analyze its adversarial robustness and privacy. We find the choice of pre-trained models significantly influences the white-box adversarial robustness of LM-VP, making it hard to draw consistent conclusions. Therefore we focus more on its transferred adversarial robustness and its interaction with MIA-based privacy. To address both concerns, we propose the transfer AT method for LM-VP models to enhance performance on both fronts. Experiments across various pre-trained models demonstrate that: (1) Both standard-trained and transfer AT-trained LM-VP models show a positive correlation between transferred adversarial robustness and pre-trained model size, and (2) Transfer AT significantly boosts the transferred adversarial robustness of LM-VP models while also enhancing its training data privacy. These findings indicate the advantage of LM-VP models trained with transfer AT in AI security. However, the analysis of the LM-VP model currently lacks theoretical support and interpretability. In our future work, we intend to integrate relevant theories from related domains to delve deeper into LM-VP models' security implications.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: Our experiments are based on public datasets, we have not yet released our code but we will create a GitHub repo for the code soon.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We have a setup section to show the above information.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: The experiments can support the main claims in this paper.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We report the computer resources in the experiment section.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. Justification: Please refer to our conclusion. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 
 
 Appendix: A.We set ϵ = 4/255 and evaluate the LM-VP model based on the ResNet50 and ConvNext pre-trained models. The result is consistent with our main experiment where ϵ = 8/255, i.e., transfer AT achieves better transferred adversarial robustness and privacy trade-offs at the cost of natural accuracy. • If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Justification: There is no such issues in this paper. Guidelines:• The answer NA means that the paper poses no such risks.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 


Decision: In this manuscript, the authors aim to introduce a new method to address both the transfer adversarial robustness and privacy of LM-VP models simultaneously.
The motivation and contribution are clear.
As for the reviewers' concerns, including novelty, white-box adversarial robustness, more experiments, unclear wordings, etc, they have been properly clarified.
Overall, this manuscript is suggested to accepted. 
However, in responding to the concerns of Reviewer J3te and other reviewers, the rebuttal contents must be included in the revised version.

Reviews: review: paper_summary: paper_summary: Adversarial robustness and privacy are important considerations in AI security, particularly in deep learning models. Adversarial training (AT) is effective in enhancing robustness against attacks, but it increases vulnerability to membership inference attacks (MIAs), compromising privacy. This trade-off between robustness and privacy highlights the need for evaluation. Visual prompting, a model reprogramming technique, shows promise in vision tasks, but its performance under attacks and MIAs requires further assessment. This study evaluates the joint adversarial robustness and privacy of label-mapping-based visual prompting (LM-VP) models, combined with transferred AT, demonstrating a favorable trade-off between the two.
strength_weakness: strength_weakness: 1. The article lacks theoretical support and interpretability for the analysis of LM-VP models, which may limit the depth of understanding of the security implications of these models.
2. The evaluation primarily relies on empirical findings, which may not fully capture the theoretical underpinnings of the observed relationships between adversarial robustness and privacy in LM-VP models.
questions: questions: Can the authors provide more detailed insights into the theoretical foundations and assumptions underpinning the empirical findings on the trade-off between adversarial robustness and privacy in Label Mapping Visual Prompting (LM-VP) models?
limitations: limitations: The article does not extensively discuss the theoretical assumptions and proofs underlying the empirical findings, potentially limiting the generalizability and robustness of the conclusions drawn. While the paper addresses the trade-off between adversarial robustness and privacy in LM-VP models, it may not fully explore the broader societal impacts and ethical considerations of deploying such models in real-world applications.
paper_summary: paper_summary: The works shows that LM-VP models can achieve the great adversarial robustness and privacy at the same time, different from full model adversarial training. Across different pre-trained models, the proposed transferred adversarial training achieves good classification accuracy and low MIA success rates.
strength_weakness: strength_weakness: 1.  Considering the efficiency of LM-VP adaptation, why not try different datasets other than CIFAR10? The current results are restricted to CIFAR10, which is a 10-class classification problem and low resolution (which can perform OK in LM-VP setting). But what about a higher resolution dataset with more classes? 
2. The main concerns come from the effectiveness of adversarial attacks against LM-VP, which also raises the concern about the high adversarial robustness mentioned in the paper. The high robust accuracy for ConvNext and EVA might be due to the low transfer attack success rates from ResNet. Can you also show me the success rates of transfer attacks generated on ResNet-18 to attack ConvNext and EVA? If this is very low, it is not surprising that so-called robust accuracy is similar to standard accuracy.
3. It is still not clear to me why standard white-box adversarial attacks can not be applied to LM-VP models. From my perspective, both LM and VP should be treated as parameters and the adversarial attacks can be applied accordingly. 
4. In Section 3.3, it is stated that the trainable parameters are noise parameters. However, the LM as a fc layer has also parameters.
questions: questions: See Weakness.
limitations: limitations: Theoretical work is considered limitation, which is claimed to be solved in the future.
paper_summary: paper_summary: This paper explores the trade-offs between adversarial robustness and privacy in deep learning models, highlighting that while AT improves robustness but it increases vulnerability to MIA. The authors introduce an ANF-based graph structure and CryptoANFNet, a neural network model for cryptographic problem-solving, demonstrating that their approach achieves a good balance between robustness and privacy.
strength_weakness: strength_weakness: NA
questions: questions: No
limitations: limitations: NA
paper_summary: paper_summary: This paper investigates the adversarial robustness and privacy aspects of models trained using the Language Model Visual Prompting (LM-VP) technique, which has not been done before. The results suggest that LM-VP models trained with transfer AT have advantages in AI security.
strength_weakness: strength_weakness: - Ln267-268: "MIA success rate near 50%"  is the statement in the discussion section. 50% means for me random. It is missing for me, if this  is an improvement or not.  I am asking myself if there a baseline exists which could be even lower.  It should have been shortly mentioned how to order this in general in MIA defenses.

Comments on writing:
 - Ln83: No AutoAttack [1] mentioned, but later in Ln219. 
 - Eq1: A dot instead of a comma at the end of line.
 - Eq3: The multiplications signs look like a convolution.

[1] https://robustbench.github.io/#div_imagenet_Linf_heading
[2] https://ml.cs.tsinghua.edu.cn/ares-bench/#/leaderboard
questions: questions: - In the experimental setup, it is not clear to the reviewer, which MIA attack did you use? In Ln102 the authors state two threshold-based MIAs. Is it one of these? Or both?
limitations: limitations: - Adversarial attack setup and limitations: An ablation study on different epsilon sizes would give more insights on the limitations. Ln212: The attack epsilon size 8/255 (for CIFAR-10) is larger than the standard size for adversarial training is 4/255 [1,2] for ImageNet. 

In light of the current results, these limitations only could be interesting for the appendix as additional insights and does not have to appear in the  main sections.

Comments: title: Thank you for your clarification
Good clarification
About the novelity
comment: Thank you for your reply and suggestions. We will add more epsilon size options in our updated version to explore the ablation effects of this parameter.
Thank you for raising the score and for your comments. 

We will incorporate the analysis of white-box adversarial robustness during the rebuttal into our manuscript to enhance our contribution. This will include the impact of different pretrained models on white-box adversarial robustness and the analysis of the failure of standard adversarial training on the LM-VP model, which we believe will bring more insights into the security assessment of the LM-VP model. We will also revise the title to ensure clarity and avoid ambiguity.
Thank you for your suggestion and for raising the score. We will add the analysis of white-box adversarial robustness during the rebuttal into our revised manuscript.
Dear authors and AC,

Thank you for your detailed response on transferred adversarial robustness and the specialty of the pretrained model effects on downstream tasks. I would like to raise my score to 4 pts. 

I still hope that in the future works, the discussion around white-box adversarial robustness can be included because the current version seems to mix up the transferred adversarial robustness and adversarial robustness. Like the title for this work, "Towards Evaluation of **Adversarial Robustness** and Privacy on Label Mapping Visual Prompting Models", the first impression for me is definitely that you use LM-VP to improve both adversarial robustness and privacy at the same time.

I will see the opinion by the AC in the next discussion phase.
Authors provide a clear clarification for my concerns in this round, I hope you can add these rebuttal texts into the revised version if it is accepted.  Blackbox evaluation is indeed a bigger challenge than whitebox.
Thank you for your thorough response. I appreciate the time you took to address each of the points I raised.
Well, you could have taken 1/255 or 16/255 as well as epsilon size for the proposed ablation study.
We once again thank the reviewer for the valuable feedback. 

In this response, we aim to further clarify the experimental results we provided in our previous response regarding PGD-20 attacks. Additionally, as we clarified in our response, this paper does not intend to claim that transfer adversarial attacks should replace white-box attacks for evaluating adversarial robustness. Rather, the main purpose of this paper is to systematically evaluate both standard training and transfer AT across various pre-trained models on LM-VPs, **focusing on the trade-off between transferred adversarial robustness and privacy**. Below are our responses:

1. Yes, we study transfer adversarial robustness and its relationship with MIA-based privacy for LM-VP models.

2. We did not study the white-box adversarial robustness in this work. During the rebuttal, based on one comment, we provided additional results on white-box PGD-20 adversarial robustness. The results in Table 2 represent the **best (highest) adversarial robustness**, which was only observed in the early stages of training. Considering the characteristics of LM-VP, in the early stages of training, PGD-20 has not fully utilized the information of the victim model, and those values reflect more about the inherent robustness of different fixed pretrained models, thus showing no consistent pattern. However, **as training progresses, for all pretrained models, the adversarial robustness continues to decline until it reaches a stabilized status**. To support this, we provide the changing trend of adversarial robustness for 10 epochs, as supplemental results to Table 2 (see the table below).

We can also refer to [1] and [2] for how pretrained models impact downstream adversarial robustness. However, their research is limited to ResNet50 or Swin, and Yamada et.al conclude that **“network architecture is a strong source of robustness when we consider transfer learning”** [2]. If different pretrained models yield different results, it becomes challenging to draw a consistent conclusion on the adversarial robustness of LM-VP, making it even more difficult to study its trade-off with privacy.

**Response to the final points of the reviewer**:

1. I understand and agree with this statement. Our rebuttal Table 2 is based on PGD-20 attacks but only includes the best adversarial robustness. From the table below, after certain iterations, adversarial robustness tends to be stable and potentially reflects the worst-case performance for LM-VP models. In contrast, transfer adversarial robustness remains stable during training, the worst-case adversarial robustness is definitely lower than transferred adversarial robustness.

2. "If you find ... in your experimental setup." In our previous rebuttal Table 2, the best adversarial robustness result leads to the misunderstanding. Our additional results about **epoch-wise changes in adversarial robustness** of that experiment prove that white-box attacks are stronger than transfer attacks (see the table below).

We agree with the statement that "transferred adversarial robustness cannot reflect the overall adversarial robustness." However, the focus of this paper is to study the trade-off between transfer adversarial robustness and privacy. We believe that transferred adversarial robustness is a very important attribute to study, especially **when different pretrained models are used**, where transferred adversarial attacks can provide more consistent evaluation results.

In the paper, we state "a better trade-off between adversarial robustness and privacy." Here, adversarial robustness specifically refers to transfer adversarial robustness. Please refer to our response to Question 1,  we only consider transfer adversarial robustness in our manuscript. As mentioned, in the next version of the manuscript, we will **add "transferable" or "transferred" in places where it was omitted** in the current version.

3. As we clarified in our rebuttal, we did not consider white-box adversarial robustness in this work, since the main purpose of this paper is to explore the trade-off between transfer adversarial robustness and privacy for LM-VP models, especially where different pre-trained models are employed, and we demonstrate that transferred AT can improve both simultaneously.

|Epoch|1|2|3|4|5|6|7|8|9|10|
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
|ResNet50|7.37|**8.33**|5.28|3.13|1.80|0.78|0.33|0.29|0.16|0.25|
|ResNet152|**57.09**|56.12|50.49|42.13|37.09|30.12|22.83|20.97|19.81|20.77|
|WideResNet|**40.29**|39.26|30.28|26.65|21.34|17.88|15.08|12.24|12.33|10.09|
|ViT|**19.28**|17.09|13.09|9.88|6.70|5.27|5.00|3.28|1.90|2.01|
|Swin|0|0|0|0|0|0|0|0|0|0|

[1] Vaishnavi P, et.al. A Study of the Effects of Transfer Learning on Adversarial Robustness[J]. TMLR.

[2] Yamada Y, Otani M. Does robustness on imagenet transfer to downstream tasks?[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.
We sincerely thank the reviewer for the valuable feedback.

The idea of this paper, i.e., studying the relation of the adversarial robustness and privacy within the LM-VP model stems from similar concerns observed in general trained models. In the context of general trained models, a key consideration is the boundary relationship between standard adversarial robustness and privacy. However, existing research [1] highlights a conflict between these boundaries when employing white-box adversarial attacks. As a result, in this paper, we use the transferred adversarial attacks to study the robustness instead of using white-box attacks, which from our point of view, is more sensible; the specific reasons are listed below:

**1**. A crucial distinction between the LM-VP model and a general model lies in the presence of a pre-trained model that does not participate in training [2]. Evaluating the LM-VP model using white-box adversarial robustness metrics can be heavily influenced by the choice of pre-trained model, as shown in Table 2 of the PDF. There is no clear pattern in their best adversarial robustness, potentially leading to different boundary relationships between adversarial robustness and privacy. Moreover, since we train the LM-VP model on the target dataset, but the generation of adversarial examples relies on the fixed pre-trained model from the source dataset domain, as a result, different pre-trained models may result in varying boundary performances, i.e., in this sense, using white-box adversarial attacks for the evaluation would make it difficult to draw a consistent conclusion.

**2**. In contrast, when considering the transferred adversarial robustness of LM-VP models, the intensity of the transfer attack remains constant once the attack model is selected. This consistency holds regardless of the chosen pre-trained model, ensuring that the transfer adversarial training process consistently optimizes in the same direction. This inherent consistency thus is more helpful for exploring and establishing a sensible boundary relationship between transferred adversarial robustness and privacy within LM-VP models. Therefore, utilizing transferred adversarial attacks serves as a more reliable and insightful evaluation method in this context.

**Modeling Approach**:
Within the framework of transfer AT, the LM-VP model, comprising VP (Visual Prompt), a pre-trained model, and LM (label mapping), is treated as a unified black-box system. A fixed-parameter attack model, excluded from the training process, is employed to conduct transferred adversarial attacks. Under this setup, this paper systematically evaluates both standard training and transfer AT across LM-VP models that are based on various pre-trained models, with a focus on their impact on the trade-off between transfer adversarial robustness and privacy.

**Quantification and Analysis**:
To quantify the boundary relationship between robustness and privacy, we leverage numerical metrics of transfer adversarial robustness and the success rate of membership inference attacks. By systematically comparing these metrics under both standard training and transfer AT across a range of pre-trained models, we aim to arrive at a consistent and generalizable conclusion. This analysis seeks to demonstrate that transfer AT effectively achieves a superior balance between transfer adversarial robustness and privacy boundaries, ultimately establishing it as a more secure training approach for LM-VP models.

[1] Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 241–257, 2019.

[2] Bahng H, Jahanian A, Sankaranarayanan S, et al. Exploring visual prompts for adapting large-scale models[J]. arXiv preprint arXiv:2203.17274, 2022.
Dear Reviewers,

The authors have responded to your valuable comments.
Please take a look at them!
Thanks!

Best,
AC
First, thank you for your patient rebuttal and response. If I understand correctly:
1. You are studying transferred adversarial robustness for LM-VP model trained with transferred AT;
2. You did not study adversarial robustness by the white-box attacks because it is not consistent when evaluating LM-VP with different pretrained models due to Table 2 first two columns;

Then, I want to finalize my opinion to both you and **AC**:
1. the adversarial robustness should be the worst case (the strongest) attack when evaluating a model, which means, at least the stronger one between the transferred attack you have tried by PGD against ResNet18 and the PGD you tried during the rebuttal.
2. If you find that the PGD-20 against the victim model is even weaker than PGD-20 against another substitute model (ResNet-18 or WRN-34-10), PGD is not a good way to evaluate adversarial robustness in your experimental setup. Therefore, your so-called transferred adversarial robustness can not reflect the adversarial robustness. Then your so-called the better trade-off between adversarial robustness and privacy might be not convincing. 
3. The main assumption by the paper is wrong: "Notably, the LM-VP models are incapable of generating standard forms of AEs like general models due to the input transformation"; "LM-VP models do not inherently have the traditional adversarial robustness property". This has been admitted by the authors in the rebuttal as well by their PGD-20 experiments on Swin. You can argue with **AC** about this.
Thanks for considering my concerns. 

However，the author doesn't seem to understand what I'm trying to say. I know, in this article, the main contribution is to evaluate the adversarial robustness and privacy of the LM-VP models, using the method of Transferable AT. I want to ask why Transferable AT can be used for evaluation the boundary of adversarial robustness and privacy, and how is it modeled and quantified? In my opinion, this paper only uses an existing method to evaluate a complex problem. Maybe, further discussions about the motivation and original contributions rather than more experiments should be highlighted.
We agree that transferred adversarial robustness is not exactly the same as adversarial robustness; to clarify this, we will use the term "transferred adversarial robustness" to make our work as precise as possible in the updated version, we appreciate the reviewer pointing this out. We also agree that the white-box standard adversarial robustness of LM-VP models is a topic worth studying. However, we want to provide some concerns about why evaluating the standard adversarial robustness of LM-VP models can be challenging. In contrast, the black-box transfer adversarial robustness of LM-VP models does not suffer from the same issues described below.

- **The standard adversarial robustness of LM-VP models may largely reflect more about the inherent properties of a fixed pretrained model rather than an effective evaluation of the LM and VP components**. In Table 2 of our rebuttal, there are significant differences in the best adversarial robustness of LM-VP models with different pretrained models. Despite using the same VP and LM components, their best adversarial robustness varies greatly and lacks a clear pattern, e.g., larger size pretrained models may not have better (or worse) adversarial robustness. This indicates the standard adversarial robustness may significantly be influenced by inherent features of the fixed pretrained model rather than the whole LM-VP model, whereas our goal is to evaluate the whole LM-VP model rather than just the pretrained component.
-  **Whether the standard adversarial robustness makes sense for LM-VP models or not**: Adversarial examples generated by the LM-VP model are significantly influenced by the fixed pretrained model trained on the **source dataset**. The validity of using these samples to assess the LM-VP model's adversarial robustness on the **target dataset** requires further consideration.

**Q1**. Since our manuscript focuses on the transferred adversarial robustness of LM-VP models, we frequently use the terms “transfer” or “transferred” throughout the paper, e.g., we mention we enhanced the transferred adversarial robustness in the contribution in Ln 57. The original manuscript does not consider the standard adversarial robustness, thus in some parts, we use adversarial robustness as transferred adversarial robustness. During rebuttal, since we rethink the standard adversarial robustness in LM-VP, we will check and correct the relevant statements in the revised manuscript and add the analysis of the standard adversarial robustness.

**Q2**. The two columns on the left of Table 2 in the rebuttal are the results after standard training on LM-VP models. **“Best performance” refers to the performance under the epoch of the best (standard or transferred) adversarial robustness**, i.e., in Table 2 of rebuttal, when the standard PGD-20 adversarial robustness is best at 8.33%, the corresponding natural accuracy is 80.52% under that epoch; In contrast, in the manuscript, when the ResNet18 transfer PGD-20 adversarial robustness is best at 35.61%, it has a natural accuracy of 86.3%. We will add the description of the term “best performance” in the revised manuscript to clarify this, as it can indeed be easily misunderstood.

**Q3**. Table 2 of the rebuttal reflects the standard adversarial robustness of LM-VP models. Based on these two values alone (57.09% and 35.99%), we can only infer that, at a specific stage of LM-VP model training, the adversarial examples generated by ResNet18 are more aggressive than those generated by the LM-VP model based on pretrained ResNet152. For the general trained model, we generally believe that white-box attacks are stronger than black-box transfer attacks. However, the LM-VP model is influenced by pretrained models, which are trained on source datasets. It is conceivable that the adversarial examples under some pretrained models are strong and some are weak. For example, from Table 2 of the rebuttal, pretrained Swin shows an extremely strong white-box adversarial attack while pretrained ResNet152 is relatively weak. However, the strength of ResNet18 transfer adversarial attack remains consistent. 

We would also like to mention that 57.09% of the standard adversarial robustness occurs only in the first epoch of training. This early-stage standard adversarial robustness likely reflects the inherent properties of the pretrained ResNet152, but its standard adversarial robustness decreases along with training, e.g., only 19.81% remains at epoch 9. Conversely, the ResNet18 transfer adversarial robustness maintains a relatively stable level, fluctuating between 32% and 36% for pretrained ResNet152.

**Q4**. Regarding the ResNet18, WRN-34-10 transfer adversarial attack, and the standard adversarial attack, our settings are consistent: PGD with epsilon=8/255, num_steps=10, step_size=2/255 for training, and only change the num_steps=20 for testing, the above settings are the same in the manuscript and rebuttal.
I still think that the so-called transferred adversarial robustness is far away from the adversarial robustness. 

1. In the paper, "adversarial robustness" is used multiple times, Line 156, Line 254, Line 272. However, it is "transferred adversarial robustness" according to the author's response.
2. In the second table of your rebuttal, ResNet50 standard test accuracy is 80.52% and in Table 1 of your submission, ResNet50 standard test accuracy is 86.3%. I am confused about your setup.
3. In the second table of your rebuttal, ResNet152 standard PGD accuracy is 57.09% and is even higher than 35.99% in your submission, as transfer attack PGD20 accuracy. I am confused why the white-box attacks are even worse.
4. I need authors to double check the adversarial attack setting in both the original submission and the rebuttal.