CNCA: Toward Customizable and Natural Generation of Adversarial Camouflage for Vehicle Detectors

Authors: linye  lyu
jiawei  zhou
daojing  he
yu  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 14:48:59

Abstract: Prior works on physical adversarial camouflage against vehicle detectors mainly focus on the effectiveness and robustness of the attack. The current most successful methods optimize 3D vehicle texture at a pixel level. However, this results in conspicuous and attention-grabbing patterns in the generated camouflage, which humans can easily identify. To address this issue, we propose a Customizable and Natural Camouflage Attack (CNCA) method by leveraging an off-the-shelf pre-trained diffusion model. By sampling the optimal texture image from the diffusion model with a user-specific text prompt, our method can generate natural and customizable adversarial camouflage while maintaining high attack performance. With extensive experiments on the digital and physical worlds and user studies, the results demonstrate that our proposed method can generate significantly more natural-looking camouflage than the state-of-the-art baselines while achieving competitive attack performance.

Main paper: 
 introduction: 
 Prior advanced physical attacks against vehicle detectors use adversarial camouflage technique [22; 21; 18; 19] . This technique fully covers the whole vehicle's surface with adversarial texture, which leads to better attack performance regardless of the viewing positions. Leveraging a differentiable renderer can effectively optimize 3D vehicle texture to deceive vehicle detectors via gradient backpropagation. However, all these methods suffer certain issues. Firstly, there is no prior knowledge of naturalness to guide the camouflage generation, resulting in conspicuous and attention-grabbing camouflage patterns as shown in Table 4 . Secondly, all the current methods optimize the adversarial camouflage at a pixel level, making it challenging to resemble natural-looking patterns. Last but not least, none of the methods can customize the appearance of camouflage, making it hard to adapt to specific environments like forests and deserts.To address the above issues, we propose CNCA, a novel framework to generate customizable and natural adversarial camouflage against vehicle detectors as shown in Figure 1 . Our insight is that: to gain naturalness and customizability, we need to leverage models that are equipped with prior knowledge of naturalness and allow conditional input signals. Motivated by this insight, we leverage an off-the-shelf pre-trained diffusion model to generate adversarial texture images with user-specific text prompts. The challenge of this approach is how to guide the adversarial gradient from the detection model to the image generation process. We introduce an adversarial feature to combine with the original text prompt feature. The combined feature forms the conditional input of the diffusion model. Thus, the resulting image is both natural and adversarial. Furthermore, we apply a clip strategy to the adversarial features to balance the trade-off between naturalness and attack performance. The combination of diffusion models, adversarial features, and clipping strategies facilitates the generation of customizable and natural camouflage.The main contributions of our work are summarized as follows:• To the best of our knowledge, our work is the first to investigate natural physical adversarial camouflage generation with diffusion models. It is also the first that can generate various styles of adversarial camouflage against vehicle detectors. • We introduce an adversarial feature that can be combined with the conditional input of the diffusion models, enabling gradient-based adversarial camouflage generation. • We propose to apply a clipping strategy for the adversarial feature to balance the trade-off between naturalness and attack performance.We conduct a comprehensive evaluation with popular vehicle detectors and datasets in both digital and physical settings, and the results show that our method is effective in generating natural and customized adversarial camouflage. 
related work: 
 Adversarial Camouflage. Accurate detection of nearby vehicles is a crucial safety requirement of self-driving cars. Therefore, there has been a growing interest in crafting adversarial camouflage to attack vehicle detection systems. Most current research uses a 3D simulation environment [3] to generate 2D rendered vehicle images with various transformations to develop robust adversarial camouflage. Early works of adversarial camouflage against vehicle detection are mostly black-box because the rendering process of the traditional rendering method is non-differentiable. The first work of vehicle adversarial camouflage [25] CAMOU trains a neural network to mimic the behavior of both the rendering and detection of the camouflage vehicles. Then, they can use this network to optimize the adversarial texture. [23] propose to use a genetic algorithm to search the optimal parameters for the synthesis of the adversarial texture pattern. Then, they enlarge and repeat the pattern to cover the whole surface of the vehicle.Recent advanced methods introduce neural rendering to enable direct optimization of adversarial texture via gradient back-propagation algorithms. Dual Attention Suppression attack (DAS) [22] suppresses model and human attention on the camouflaged vehicle. However, it suffers a limited attack success rate because the adversarial pattern only covers part of the vehicle surface. Then, Full-Coverage Attack (FCA) [21] optimizes the entire surface of the vehicle in multi-view settings. Furthermore, Differentiable Transformer Attack (DTA) [18] proposes a differentiable renderer that can express complex environment characteristics like shadow and fog on the vehicle surface. ACTIVE [19] introduces a new texture mapping that incorporates depth images and tries to improve the naturalness of the camouflage by using larger texture resolutions and applying a smooth loss. Despite prior works achieving impressive attack performance, these methods optimize the camouflage patterns at the pixel level without prior knowledge of naturalness. Consequently, the generated camouflage is conspicuous and attention-grabbing for human observers.Diffusion Models. Diffusion Models (DMs) [6] are widely used to generate natural images of higher quality and diversity. Since billions of image-text dataset pairs [16] are used to train these models, DMs provide a strong prior knowledge of natural and realistic images and their corresponding text captions. As a result, DMs can produce highly realistic and varied images across different user-specific prompts.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss the limitation of our work in the section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methods: 
 In this section, we present an overview of our framework for generating customizable and natural adversarial camouflage while maintaining comparable attack performance. Subsequently, we provide a detailed explanation of the essential components of our framework. B = I in • (1 -M ) (1) X ref = I in • M (2)Then, we use a neural renderer N R to obtain rendered vehicle images X nr with 3D mesh M sh, UV texture image obtained from UV map mask and adversarial texture image T adv , and camera parameters Φ cam of the vehicle. Next, X ref and X nr forward into a neural network called Environment Feature Renderer (EFR), which extracts the environmental features from X ref and render these features into X nr to obtain X ren . We then add X ren with background B to obtain the realistic camouflaged vehicle images I out . Then, we input I out into the target object detector to obtain the detection results R.Since we introduce a pre-trained text-to-image (T2I) diffusion model to generate the UV-map texture images, we can provide text prompt P txt to customize the texture image. The text encoder processes the text prompt to obtain the text feature F txt . Then, the text feature is combined with the adversarial feature F adv , which will be optimized during the camouflage generation. We apply a clip function κ(•) to F adv during optimization to balance naturalness and attack performance. Then, the combined features are fed into the pre-trained T2I model, which outputs the adversarial texture images T adv .EQUATIONEQUATIONIn the end, we can obtain the final adversarial camouflage by minimizing the below adversarial loss function from the target detector:EQUATIONwhere x is the input image for the target detector, D b (x) is the detection bounding box, gt is the ground-truth bounding box. We calculate the Intersection over Union (IoU) between D b (x) and gt. This IoU score allows the optimization to focus on the bounding box with larger intersections with ground truth. D o (x) and D c (x) are the objectiveness score and the class confidence score for the bounding box, respectively. We obtain our detection score D s (x) by multiplying the IoU score, objectiveness score, and class confidence score. We select the highest D s (x) to compute L adv (x) using a log loss. By minimizing L adv (x), we encourage the camouflaged vehicle to be undetected or misclassified by the detector.The prior works from DTA and ACTIVE [18; 19] prove that realistic rendering of the vehicle camouflage is one of the keys to successful physical adversarial attack. To achieve this, we use two rendering components: the first render component is a differentiable neural renderer, which takes the 3D properties of the vehicle to output the vehicle's foreground images. However, it struggles to render complex environmental characteristics on the vehicle surface. To alleviate this, we use an environmental feature renderer that can combine the environmental characteristics and neural renderer output to produce realistic and accurate camouflaged vehicle images.We select Pytorch3D as differential renderer [12] to generate camouflaged vehicle images because it supports differentiable path to the UV-map texture image. Following the method proposed in DTA [18] and ACTIVE [19] , we use a U-Net network for EFR to extract environmental characteristics from X ref and combine them with X nr . EFR outputs the camouflaged vehicle with environmental characteristics X ren .Before camouflage generation, we need to train EFR for its optimal performance. The training of EFR needs masked vehicle images X ref , 3D mesh M sh, camera positions Φ cam , and various preset color texture T as input. Meanwhile, we obtain images of different preset colors from the Carla simulation environment. Then, we mask out the vehicle parts as the network ground truth GT . To optimize the parameters of EFR, we use the following loss function: EQUATIONPrior works optimize the adversarial camouflage in the pixel space, which leads to unnatural and uncontrollable adversarial texture patterns. To alleviate this, we leverage an off-the-shelf stable diffusion model [15] to generate a vehicle UV-map texture image. The diffusion model is trained on a subset of LAION-5B [16] , a dataset of billions of image-text pairs. Since the diffusion model learns the manifold of natural images and corresponding text captions, it can generate adversarial texture images that look more natural and relevant to the given text prompt.To make the generated UV-map images adversarial, we introduce an adversarial feature vector F adv that can be optimized during the camouflage generation with diffusion models. The adversarial feature vector has the same hidden dimension as the text feature vector F txt . Therefore, it can concatenate with F txt to form the conditional input to the T2I model. As a result, the generated image reflects the control signal from the text feature while being adversarial against the target detector.We also reordered the UV map of the vehicle texture so that the vehicle surface could be connected as much as possible. Figure 3 shows the UV mapping before and after reordering. The reordering makes the generated camouflage can keep more natural patterns generated by the diffusion model. Hence, it improves the naturalness of the vehicle camouflage.Without any constraints, the framework can move the combined feature from F txt and F adv out of the feature space of the text prompt. Consequently, we can no longer expect the generated UV-map images to look natural and consistent with the input text prompt. Since the diffusion model is trained to generate natural and controllable images with original text features, there is a higher chance of generating natural images if the concatenated feature is closer to the original text feature.To preserve naturalness and controllability, we assure that the adversarial feature F adv will not exceed a norm greater than a threshold τ . Tuning the norm threshold τ enables us to trade off naturalness and controllability for attack performance.We follow PGD and choose ℓ p norm to constrain F adv . We update F adv using the formula below:EQUATIONEQUATION)where t is the time step, η is the step size, ∇L adv is the gradient of the adversarial loss, κ is the clipping function defined as in Eq. 8, where F i is the i-th element of F .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We have provided the related data and code link in the abstract for reproducing our results. Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details. • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental settings: 
 Datasets: We utilize the Carla [3] simulator to generate datasets for our experiments. To have a comparative analysis with prior studies [22; 21; 18; 19] , we select the Audi E-Tron as the target vehicle model. We create datasets using various simulation settings, resulting in 69,120 and 59,152 photo-realistic images for EFR training and testing. These images cover 16 distinct weather conditions, combining four sun altitudes and four fog densities. Additionally, we generate a dataset of 40,960 images for camouflage generation and a test dataset of 8192 images for adversarial camouflage evaluation. The weather conditions included in these two datasets are the same as those used during the training and testing of EFR. We print out five types of adversarial camouflages for physical world evaluation and apply them to 1:12 Audi E-Tron car models. For each model, we capture 96 pictures under different elevations, azimuths, and distance parameters.Baselines: We compare our method with four advanced adversarial camouflage methods: DAS [22] , FCA [21] , DTA [18] , and ACTIVE [19] . DAS and FCA optimize the 3D texture by minimizing the attention-map scores and detection scores of the detector, respectively. DTA and ACTIVE both optimize a square texture pattern with a neural network and cover it on the vehicle's surface repeatedly. We compare our results using the official textures generated by these methods. We apply these textures to the same car model for a fair comparsion.Evaluation metrics: For training the EFR component, we follow the setting from [18] to use the Mean Absolute Error (MAE) as a loss to measure the difference between the output of ERP and the ground truth. To evaluate the effectiveness of the adversarial camouflage, we utilize the AP@0.5 benchmark [4] , as it provides a comprehensive assessment of recall and precision values at a detection IOU threshold of 0.5.Target detection models: Aligning with previous work, we adopt YOLOv3 [13] as the white-box target detection model for adversarial camouflage generation. To evaluate the effectiveness of the optimized camouflage, we utilize a collection of widely used object detection models treated as blackbox models, including YOLOF [2] , Deformable DETR (DDTR) [27] , Dynamic R-CNN (DRCN) [24] , Sparse R-CNN (SRCN) [17] , and Faster R-CNN (FrRCNN) [14] . They are trained on the COCO dataset and implemented in MMDetection [1] .Training details: Following [18] , we utilize the Adam optimizer with a learning rate 0.01 for EFR training and camouflage generation. We train the EFR for 20 epochs and choose the model with the best performance on the test dataset. We use stable diffusion v1.5 to generate UV-map texture image with DDIM sampler. We set the sampling step to 20. The optimization of the adversarial camouflage takes a duration of five epochs. We conduct experiments on a cluster with eight NVIDIA RTX A800 80GB GPUs.In this section, we compare our method to current advanced adversarial camouflage methods, including DAS [22] , FCA [21] , DTA [18] , and ACTIVE [19] . We run an extensive attack comparison using diverse detection models. Since the target model is YOLOv3, we use various detection models to evaluate the transferability of the camouflage in a black-box setting. We use 'colorful camouflage' as the text prompt in this experiment.The results are shown in Table 1 , showing that our method has competitive performance with the current state-of-the-art baselines. DAS performs only better than normal car painting, primarily due to the limitations of partially painted camouflage. Meanwhile, FCA exhibits sub-optimal performance, only slightly better than random camouflage, because it cannot render sophisticated environment characteristics. DTA and ACTIVE have comparable attack performance to CNCA, but our method achieves the best attack performance in total. Figure 4 shows the summarized performance of each camera pose and weather parameter; values are car AP@0.5 averaged from the detectors used in Table 1 . We can see that the camouflage produced by our method shows competitive performance compared to DTA and ACTIVE. 4 : Attack comparison on different camera poses and weather parameters. "ele" denotes elevation, "azi" denotes azimuth, "dis" denotes distance, "fog" denotes fog density, and "sun" denotes sun altitude angle. Values are car AP@0.5 (%) averaged from all models.We conduct the physical world evaluation using 1:12 Audi E-Tron car models with camouflages generated by different methods. Table 2 shows the attack performance of these camouflages against real-time object detectors in the real world: YOLOv3, YOLOX [5] , SSD [10] , CenterNet [26] , RetinaNet [9] . Our method achieves the best attack performance against four out of five detectors and the best overall performance. Furthermore, Figure 5 shows that our method can successfully attack the detectors in both indoor and outdoor environments compared to other methods. In summary, the physical evaluation results demonstrate that our method is transferable to the real world.Our method enables customizable camouflage generation with user-specific input. Table 3 shows the various styles of adversarial camouflages with their corresponding text prompts and AP@05 values (target YOLOv3). Our method can directly customize the color choices and patterns of the camouflage with the input texture prompt. We notice that there are some differences in attack performance regarding different prompts. During our experiments, we found the prompts that describe natural objects (columns 4,5,6 in Table 3 ) are likely to have lower attack performance(average around 0.03 in AP@0.5) than the more abstract ones (columns 1,2,3 in Table 3 ). Despite this, the average AP@0.5 of all these camouflages generated by CNCA is 0.506, comparable to the reported result in Table 1 . Our proposed approach aims to improve the naturalness of the generated adversarial camouflage to humans. Therefore, following [7; 8], we conducted a subjective evaluation to estimate the naturalness score of the adversarial camouflages. For a fair comparison, we generate a series of vehicle images using the same set of camera positions for each type of camouflage. Then, we require each participant to give a naturalness score for each type of camouflage from a scale of 1 (not natural at all) to 5 (very natural). Besides the advanced adversarial camouflages from prior work, we also include the normal car texture as the control group. To further demonstrate the significance of the differences in mean scores, we also conduct t-tests and report the t and p values. As shown in Table 4 , the p values of t-tests are lower than 0.05. Therefore, our method's naturalness score is significantly higher than those of the four adversarial camouflages.The impact of each CNCA pipeline component. Table 5 shows the results of the ablation studies for each component of the pipeline. We gradually add each component during the ablation study to see their contribution to attack performance and naturalness. All the test pipelines with the diffusion model use the same input text prompt: "yellow black graffiti." The discussion for each test pipeline is the following: No Diff. directly optimizes the texture image of the vehicle at a pixel level, resulting in an unnatural texture; Diff. introduces the diffusion model to generate the texture image compared to No Diff., which improves the naturalness score; Diff.+Adv. introduces the adversarial feature compared to Diff., which enables the texture image generation guided by the adversarial gradient from the detector. The adversarial feature improves the attack performance but compromises the naturalness; Diff.+Adv.+Clip introduces the clipping strategy compared to Diff.+Adv., which improves the naturalness; Diff.+Adv.+Clip+Reorder is our final pipeline, which uses a reordered texture map compared to Diff.+Adv.+Clip, which further improves the attack performance and naturalness. To summarize, the ablation studies demonstrate that all the components of our pipeline contribute to improving the camouflage's attack performance and naturalness.There is inevitably a trade-off between naturalness and attack performance. The optimization space for adversarial attacks decreases when increasing naturalness of the attack. Therefore, increasing naturalness will typically decrease attack performance. Our method allows users to balance this trade-off based on their preference by adjusting the norm threshold τ . To illustrate this trade-off, we generated adversarial UV texture images with the same text prompt but different norm threshold settings for the adversarial features. In addition, we conduct a subjective survey to rank the naturalness of the images and their relevance regarding the text prompt. Table 6 shows the average rank of each image, its corresponding AP@0.5, and the norm threshold settings. It can be seen that when the norm threshold starts to increase, the texture images become less natural-looking, and the attack performance increases. Therefore, it requires the user to decide the acceptable naturalness level. Limitations. Our current method has the following limitations: firstly, the back-propagation is more expensive than the previous gradient-based methods, but we believe the strong prior knowledge of the diffusion model can further advance adversarial attacks and defenses; secondly, our method needs the 3D mesh of the target object to generate camouflage. If the 3D mesh of the object is not available, the user needs additional efforts such as photogrammetry or 3D scanning to obtain it; lastly, for each text prompt, a manual hyperparameter tuning for norm threshold τ is needed. One future work is to automatically adjust τ based on the relevance score between the image and text prompt, for instance, CLIP score [11] .Societal Impact. This paper presents work whose goal is to advance the safety of AI systems. While the proposed adversarial attack method could be potentially used by malicious users, it can also support future efforts to enhance the robustness of AI system via adversarial training, adversarial testing and adversarial example detection, thereby safeguarding the security of AI systems. 
conclusion: 
 We propose a novel physical adversarial camouflage attack framework with diffusion models. With different user-specific text prompts, our method can generate adversarial camouflage with diverse colors and patterns. In particular, we apply a clipping strategy to an adversarial feature to balance the adversarial camouflage's naturalness and attack performance. With extensive experiments on the digital and physical world and user studies, the results demonstrate that our methods improve the naturalness and enable the customizability of camouflage generation while maintaining competitive attack performance. We ask the participants to rate the naturalness of the camouflage on a scale of 1 to 5.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: We have described our experiment setting and details in the section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: For our subjective evaluation of naturalness, we conduct t-tests to demonstrate the significant differences of the mean scores between our method and previous methods . Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. We have discussed both the potential positive and negative societal impact in the section 5. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: We have discussed safeguards to enhance the AI systems against our proposed attack in the section 5.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We confirmed that all the assets that we used in our worked are properly credited. The related license and terms of use explicitly are mentioned and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. Justification: We have included the full text of instructions and screenshot to participants of subjective evaluation in the appendix section 6. The participants were compensated with small gifts such as (bottled water or online discount coupons) since our evaluation did not take a significant amount of time to complete (an average of five minutes). Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [No] Justification: There is no obvious potential risk for our participants. We have confirmed that the researchers in our research institution can make an initial determination of minimal risk for studies involving surveys with limited evaluation time and negligible psychological or physical impact. However, we recognize that consulting an IRB or equivalent body would provide a more formal risk assessment, and we will take this step in future studies Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: To better evaluate the naturalness of CNCA compared with another adversarial camouflage attack, we conducted a survey among humans with the assistance of an online form. The user interface of the survey is shown in Figure 6 , where the participants are asked to give a 1(very unnatural) to 5(natural) rating regarding the naturalness of the vehicle's appearance. We collect surveys from 45 participants up to the completion of the writing and most of them are not familiar with adversarial attacks. Table 4 illustrates the results, indicating that CNCA achieves higher naturalness among human participants compared with previous advanced adversarial camouflage.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: We clearly state the contributions and scope of our paper in the abstract and introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: This paper introduces a natural way to generate adversarial camouflage using diffusion models, claiming that the proposed method is more natural than the closest method (ACTIVE [19]). Naturalness is achieved by using diffusion models and is evaluated via human evaluation. 

This paper is on the borderline, but I all reviewers agreed on the novelty of this paper on the natural and customized adversarial camouflage via diffusion model. I think this paper could introduce a novel view in generating adversarial camouflage and this overweights the technical limitations. So, I vote for acceptance. 

In your final manuscript, please address discussed reviewers’ concerns. Additionally, 
* To mitigate the complexity of the algorithm, we recommend releasing code for the research community. 
* Please discuss limitations related to focusing on vehicle detection. We agree that vehicle detection is an important task, but to my understanding, having UV maps and 3D mesh could introduce difficulties in using this attack. Please discuss this limitation in Section 5 if you agree.

Reviews: review: paper_summary: paper_summary: The paper introduces a novel framework, CNCA, for generating customizable and natural adversarial camouflage for vehicle detectors using a diffusion model. This work addresses critical limitations in current adversarial camouflage techniques by focusing on naturalness and customizability, which are often neglected in favor of attack performance. While the paper presents a significant advancement in adversarial camouflage, several areas require improvement to enhance rigor and presentation. The proposed CNCA framework holds substantial promise, but further validation and detailed comparison are essential to establish its superiority and practical relevance.
strength_weakness: strength_weakness: 1.	The explanation of the adversarial feature generation and its integration with the diffusion model is somewhat convoluted. Quantitatively define the evaluation indicators of naturalness and attack performance, or provide relevant references.
2.	The evaluation in the physical world is limited to small-scale models and specific conditions. Extend the evaluation to a broader range of vehicle detection models and datasets, including those used in autonomous driving (e.g., KITTI, Waymo Open Dataset). Assess the scalability of CNCA by testing on larger, more complex scenes and different environmental conditions to validate its general applicability.
3.	The paper lacks ablation studies to isolate the impact of different components of the proposed framework. Conduct ablation studies to demonstrate the contribution of each component (e.g., the diffusion model, adversarial feature clipping) to the overall performance.
questions: questions: Experimental Statistical Significance: The authors recruited 45 participants to subjectively evaluate the naturalness of different camouflages, reporting the mean scores and standard deviations (SD) for naturalness of each type of camouflage. While this is good, merely reporting the mean scores and SD does not statistically demonstrate whether the differences in mean scores are significant. It would be more convincing to conduct t-tests or ANOVA (preferably repeated measures ANOVA with post hoc tests, based on the current experimental design) and report the relevant statistics (e.g., t and F values, as well as p values). 

Physical World Evaluation: In the physical world evaluation, the paper only compared two models, one for a normal and another for the generated camouflage. Have the authors considered including models with other adversarial camouflage methods for comparisons, as the authors did in the digital world?
limitations: limitations: Perform a thorough comparison with state-of-the-art methods like AdvCam and UAPs that are known for their effectiveness. Discuss the differences in performance metrics such as attack success rate, naturalness, and computational efficiency. Highlight the advantages and limitations of CNCA relative to these methods.
paper_summary: paper_summary: The manuscript presents a novel approach to generating physical adversarial camouflage against vehicle detectors, leveraging a pre-trained diffusion model. The proposed method, called Customizable and Natural Camouflage Attack (CNCA), aims to produce adversarial camouflage that is both natural-looking and customizable via user-specific text prompts. This approach addresses the limitations of previous methods that produced conspicuous and unnatural camouflage, maintaining effectiveness in adversarial attacks while enhancing the camouflage's appearance to blend seamlessly into its surroundings.
strength_weakness: strength_weakness: The approach involves complex integration of diffusion models with adversarial attack frameworks, which may increase the computational overhead and complexity compared to more straightforward adversarial techniques.

Although the manuscript includes extensive testing, the evaluations focus primarily on vehicle detection in controlled settings. The performance and practicality of CNCA in more varied or less controlled environments remain to be fully explored.
questions: questions: See strength and weakness above.
limitations: limitations: No. While the paper discusses potential positive impacts, such as improving AI robustness, the technique could also be used maliciously to evade surveillance, posing ethical and security concerns.
paper_summary: paper_summary: The paper introduces a interesting idea and also a novel method called Customizable and Natural Camouflage Attack (CNCA) to generate adversarial camouflage against vehicle detectors, leveraging a pre-trained diffusion model. This approach allows the generation of natural-looking and user-customizable adversarial patterns that maintain robust attack performance across various digital and physical settings. The paper's contributions include a unique application of diffusion models to adversarial camouflage, introduction of adversarial features for gradient-based generation, and a clipping strategy to balance naturalness with attack performance. Extensive experiments and user studies demonstrate the effectiveness of CNCA in producing more natural-looking camouflage with competitive attack performance.
strength_weakness: strength_weakness: - Some expressions are not clear, making it difficult for those unfamiliar with the field to understand. For example, lines 32 to 35. It would be better and easier to understand if some visual evidence were provided regarding these limits.
- Figure 1 is not correctly referenced.
- The experimental section would be more convincing if the effectiveness of the proposed components and methods were demonstrated through ablation experiments.
- Concerning anonymity: Some comments in the provided code reveal personal information. Please be aware of this!
questions: questions: - I am curious about its complexity. The method involves multiple components and parameters (e.g., adversarial features, clipping strategy), which might complicate its deployment in practical applications without substantial customization and tuning.
- It would be more helpful if more ablation experiments were added to individually demonstrate the functions of each component.
limitations: limitations: Refer to the “Questions” section.
paper_summary: paper_summary: This paper presents a novel method CNCA for generating customizable and natural adversarial camouflage of fooling vehicle detectors. This work is an interesting contribution in the field of adversarial attacks, especially improving the naturalness of the camouflage while maintaining high attack performance.
strength_weakness: strength_weakness: The clipping strategy lacks innovation; it has been used in PGD for a long time, and it is not worth spending too much space on it.
questions: questions: 1. Why is there no comparison with baselines in the physical world?
2. Whether other detection models can be used as white-box to generate adversarial textures to verify the transferability of CNCA.
3. Only use the subjective evaluation may not be convincing for experimental verification, and visual observation does not seem to be more natural than previous methods. Is there a more convincing scoring rule to evaluate naturalness?
limitations: limitations: The experimental results are insignificant.

Comments: title: Reply to Reviewer oknY
Response to Reviewer TBRf
Request for a higher Rating  from Reviewer guSy
Friendly Reminder: Follow-Up on Rebuttal for Submission 17110
Reply to the Ethics Review Questions from Reviewer pBgC
comment: The author addressed most of my concerns, I will rise the score.
Dear Reviewer oknY,

Thank you for your time and effort in reviewing our work and rebuttal. Your feedback has helped us improve. We are grateful that you raised the rating for our work after we provided the clarifications in the rebuttal!

Best Regards，

Authors of Submission 17110
Dear Reviewer TBRf, 

We sincerely appreciate your time and effort in reviewing our work and rebuttal. We kindly ask you to **consider whether our improvements might justify a higher rating**. We believe the **extended ablation studies for each component in our pipeline and both indoor and outdoor physical evaluations with previous methods** have strengthened the quality of our work. We would be happy to provide further clarifications if you have further questions. 

Thanks again for your time and effort in reviewing our paper!

Best Regards，

Authors of Submission 17110
Dear Reviewer guSy, 

Thanks for taking the time to review and reply to our rebuttal. We are grateful for your feedback, which has helped us to improve our work. We understand and respect your decision to maintain your current rating. However, we kindly ask you to consider whether our clarifications justify a higher rating. We believe the enhancements made during the rebuttal, specifically the extended ablation studies for each component in our pipeline and both indoor and outdoor physical evaluations with previous methods,  have strengthened the quality and clarity of our work. We appreciate your understanding and consideration of this request. We would like to provide further clarification if there are any issues you would like us to address. 

Thanks again for your time and effort in reviewing our paper!

Best Regards，

Authors of Submission 17110
Dear Reviewer MuNz, 

We are writing to follow up on the rebuttal we submitted regarding your review comments for our paper. We appreciate your time and effort in reviewing our work and providing valuable feedback. We have made a sincere effort to address each of your comments and questions in the rebuttal. We believe the clarifications and improvements we made in response to your suggestions have strengthened the paper significantly.  

We kindly request that you review our rebuttal as soon as possible ( today is the final day for discussion ) and consider increasing your rating for our paper with the provided changes and clarifications. Thank you again for your dedication to the review process; we look forward to hearing from you!

Best Regards，

Authors of Submission 17110
The authors provided thorough explanations and additional experiments to address concerns. The integration of adversarial features has been clarified with the diffusion model and provided relevant metrics for naturalness and attack performance. Additional studies were conducted, isolating the impact of each component in the framework. The authors extended their evaluation to more complex scenarios, though they acknowledge limitations due to resource constraints.

The authors have mentioned that this paper need larger-scale evaluations in future work, with more immediate tests in varied environments.
We appreciate your time and effort in providing insightful reviews. Here are the answers to your questions:

**Q1: Is the determination of "no potential risk" made by the authors themselves?**
    
Yes, the determination of "no potential risk" is made by the authors after careful consideration of the limited evaluation time (an average of five minutes per person) and the expected minimal impact on participants.


**Q2: If so, is making this determination without an IRB or equivalent permissible in their research institution or workplace?**

We have confirmed that the researchers in our research institution can make an initial determination of minimal risk for studies involving surveys with limited evaluation time and negligible psychological or physical impact. However, we recognize that consulting an IRB or equivalent body would provide a more formal risk assessment, and we will take this step in future studies.

**Q3: Were human subjects compensated for participating in the study?**

Yes, our participants were compensated with small gifts such as (bottled water or online discount coupons) since our evaluation did not take a significant amount of time to complete (an average of five minutes).

**Q4: Were human subjects screened for colorblindness or other vision impairments?**

In our current survey, human subjects were not screened for colorblindness or other vision impairment. Given the evidence that colorblind individuals may have different abilities in detecting camouflage, we will include screening for color vision impairments as part of the participant selection process in future research.
The author's response addressed some of my questions and I decided to keep my rating.
Dear Reviewers, 

The discussion period ends within 3 days. Authors made tremendous efforts on providing responses to your concerns. So, if you haven’t yet, please check responses and express your opinions whether you want to initiate discussion or accept the authors’ responses. 


Best, 
Your AC