CNCA: Toward Customizable and Natural Generation of Adversarial Camouflage for Vehicle Detectors
Authors: Linye Lyu
Jiawei Zhou
Daojing He
YU LI
Venue: NeurIPS 2024 poster (2024)
Keywords: Physical Adversarial Attack; Vehicle Detection; Customiable and Natural Adversarial Camouflage;
Primary area: safety_in_machine_learning

Abstract: Prior works on physical adversarial camouflage against vehicle detectors mainly focus on the effectiveness and robustness of the attack. The current most successful methods optimize 3D vehicle texture at a pixel level. However, this results in conspicuous and attention-grabbing patterns in the generated camouflage, which humans can easily identify. To address this issue, we propose a Customizable and Natural Camouflage Attack (CNCA) method by leveraging an off-the-shelf pre-trained diffusion model. By sampling the optimal texture image from the diffusion model with a user-specific text prompt, our method can generate natural and customizable adversarial camouflage while maintaining high attack performance. With extensive experiments on the digital and physical worlds and user studies, the results demonstrate that our proposed method can generate significantly more natural-looking camouflage than the state-of-the-art baselines while achieving competitive attack performance.

CNCA: Toward Customizable and Natural Generation of Adversarial Camouflage for Vehicle Detectors
Linye Lyu, Jiawei Zhou, Daojing He, Yu Li
Harbin Institute of Technology, Shenzhen
(lyulinye, zhoujiawei6666, yu. yu.li salllllee@gmaillcoom com com hedaoj ing.intnn3........ com
Abstract
Prior works on physical adversarial camouflage against vehicle detectors mainly focus on the effectiveness and robustness of the attack. The current most successful methods optimize 3D vehicle texture at pixel level. However, this results in conspicuous and irrention-grabbing patterns in the generated camouflage, which humans can easily identify. To address this issue, we propose a Customizable and Natural Camouflage Attack (CNCA) method by leveraging an off-the-sheil pre- trained diffusion model. By sampling the optimal texture image from the diffusion model with a user-specific text prompt, our method can generate natural and cus- tomizable adversarial camouflage while maintaining high attack performance. With extensive experiments on the digital and physical worlds and user studies, the results demonstrate that our proposed method can generate significantly more natural- looking camouflage than the state-of-the- ofettherrt baselines while achieving competitive attack performance. Our code available at https://iitubb.oo//RAAaabba
Introduction
Over the past years, Deep Neural Networks (DNNs) have revolutionized a wide range of research domains, especially in computer vision tasks, such as image classification, object detection, and semantic segmentation. DNNs are widely used in real-world systems, such as face recognition and autonomous driving. Despite their impressive success, DNNs are found vulnerable to adversarial examples 200] which are carefully crafted to deceive DNNs.
Generally, adversarial attacks can be classified into two categories: digital attacks, which primarily add small pixel-level perturbations the input images; physical attacks, which manipulate the object's 's physical properties, such as its shape, surface, or surroundings, to deceive the target model in the real world. Physical attacks are more challenging than digital attacks, as they must remain effective under various complex physical conditions, including different viewing angles, distances, and lighting conditions. This paper focuses on physical attacks against vehicle detection models since they play critical roles in real-world applications like surveillance and autonomous driving systems.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

umprelg
um g
umbre
UORIS
skateboard 0.69 skateboord
skateboard skateboord O 69
skateboard
car 0.77
(a) (b) Colorfal Graffii
(+) Zebra Stripes
(c) Colorfal
(e) Yellow Black GrafTit
Figure 1: Customized and natural adversarial camouflage with various styles. (a) A car with normal texture; (e) are the different styles of camouflage generated by our method CNCA. Their captions are user-specified input prompts.
Prior advanced physical attacks against vehicle detectors use adversarial camouflage technique [22] 18: [9]. This technique fully covers the whole vehicle's surface with adversarial texture, which leads to better attack performance regardless of the viewing positions. Leveraging a differentiable renderer can effectively optimize 3D vehicle texture to deceive vehicle detectors via gradient back- propagation. However, all these methods suffer certain issues, Firstly, there is no prior knowledge of naturalness guide the camouflage generation, resulting in conspicuous and attention grabbiing camouflage patterns as shown in Table Secondly, all the current methods optimize the adversarial camouflage at a pixel level, making challenging to resemble natural-looking patterns. Last but not least, none of the methods can customize the appearance of camouflage, making it hard to adapt to specific environments like forests and deserts.
To address the above issues, we propose CNCA, novel framework to generate customizable and natural adversarial camouflage against vehicle detectors as shown in Figure [ Our insight is that: to gain naturalness and customizability. we need to leverage models that are equipped with prior knowledge of naturalness and allow conditional input signals. Motivated by this insight, we leverage an off- sif-the-sheff pre-trained diffusion model to generate adversarial texture images with user-specific text prompts. The challenge of this approach how to guide the adversarial gradient from the detection model to the image generation process. We introduce an adversarial feature to combine with the original text prompt feature. The combined feature forms the conditional input of the diffusion model. Thus, the resulting image is both natural and adversarial. Furthermore, we apply a clip strategy to the adversarial features to balance the trade-off between naturalness and attack performance. The combination of diffusion models, adversarial features, and clipping strategies facilitates the generation of customizable and natural camouflage.
The main contributions of our work are summarized as follows:
To the best of our knowledge, work is the first investigate natural physical adversarial camouflage generation with diffusion models. It is also the first that can generate various styles of adversarial camouflage against vehicle detectors.
We introduce an adversarial feature that can be combined with the conditional input of the diffusion models, enabling gradient-based adversarial camouflage generation.
We propose apply a clipping strategy for the adversarial feature balance the trade-off between naturalness and attack performance.
We conduct a comprehensive evaluation with popular vehicle detectors and datasets in both digital and physical settings, and the results show that our method is effective generating natural and customized adversarial camouflage.
2 Related Work
Adversarial Camouflage. Accurate detection of nearby vehicles is crucial safety requirement of self-driving cars. Therefore, there has been growing interest in crafting adversarial camouflage to attack vehicle detection systems. Most current research uses a 3D simulation environment to generate 2D rendered vehicle images with various transformations to develop robust adversarial camouflage. Early works of adversarial camouflage against vehicle detection are mostly black-box because the rendering process of the traditional rendering method is non non-differentiable. The first

---

Realistic Naural Rendering
lnn
x
Background B
Object Dtoctar
Result R
Canera
Msh
X
Xyen
Ldv
Adversarial Texture Generation
Backpropagation
Clipped ddv Features
x())
Adv Features
UV mag
Todo
Pre- trained T2I Model
Text Encoder
Text Features
Text Prompt
Figure 2: CNCA framework for generating customizable and natural adversarial camouflage.
work of vehicle adversarial camouflage 25 CAMOU trains a neural network mimic the behavior of both the rendering and detection of the camouflage vehicles. Then, they can use this network optimize the adversarial texture. [23] propose use a genetic algorithm to search the optimal parameters for the synthesis of the adversarial texture pattern. Then, they enlarge and repeat the pattern cover the whole surface of the vehicle.
Recent advanced methods introduce neural rendering to enable direct optimization of adversarial texture via gradient proppaatont algorithms. Dual Attention Suppression attack (DAS) [22] suppresses model and human attention on the camouflaged vehicle. However, it suffers limited attack success because the adversarial pattern only covers part of the vehicle surface. Then, Full-Coverage Attack (FCA) [2] optimizes the entire surface of the vehicle in multi-view settings. Furthermore, Differentiable Transformer Attack (DTA) [8] proposes differentiable renderer that can express complex environment characteristics like shadow and fog on the vehicle surface. ACTIVE 19 introduces new texture mapping that incorporates depth images and tries to improve the naturalness of the camouflage by using larger texture resolutions and applying smooth loss. Despite prior works achieving impressive attack performance, these methods optimize the camouflage patterns at the pixel level without prior knowledge of naturalness. Consequently, the generated camouflage is conspicuous and attention- attention-grabbing for human observers.
Diffusion Models. Diffusion Models (DMs) 66 are widely used to generate natural images of higher quality and diversity. Since billions of image-text dataset pairs [66] are used to train these models, DMs provide a strong prior knowledge of natural and realistic images and their corresponding text captions. As result, DMs can produce highly realistic and varied images across different user-specific prompts.
3 Methods
In this section, we present an overview of our framework for generating customizable and natural adversarial camouflage while maintaining comparable attack performance. Subsequently, we provide a detailed explanation of the essential components of our framework.
3.1 Overview
Figure 2 illustrates our whole framework for adversarial camouflage generation. First, we obtain vehicle image dataset from the Carla simulation environment. The dataset includes the original input

---

images Ims ground truth labels Y, camera pose parameters Peam (position and angle), and vehicle mask M. With in and M, we can obtain the background images B and foreground vehicle reference images Xref:
B= Inn (1 M)) Xres = Is" M
(1)
(2)
Then, we use a neural renderer N R to obtain rendered vehicle images Xnr with 3D mesh M sh, UV texture image obtained from UV map mask and adversarial texture image Tade, and camera parameters Peam of the vehicle. Next, Xref and Xur forward into a neural network called Environment Feature Renderer (EFR), which extracts the environmental features from Xref and render these features into Arr to obtain X, ren-
We then add Xren with background B obtain the realistic camouflaged vehicle images Lout- Then, we input Lout into the target object detector to obtain the detection results R.
Since we introduce a pre-trained text-to-image (T2I) diffusion model to generate the UV-map texture images, we can provide text prompt Pizt customize the texture image. The text encoder processes the text prompt to obtain the text feature Fiut- Then, the text feature is combined with the adversarial feature Fades which will be optimized during the camouflage generation. We apply clip function x(-) to Fado during optimization to balance naturalness and attack performance. Then, the combined features are fed into the pre-trained T2I model, which outputs the adversarial texture images T Tado-
Fizt = Pnc(P
T Tade T21 (2I Feas))
In the end, we can obtain the final adversarial camouflage by minimizing the below adversarial loss function from the target detector:
Da(x)= = ToU (D& (z).gt) De (x) Do (x)
Lado (x) - log (1 max (D)()))),
where x is the input image for the target detector, Dx(x)) is the detection bounding box, gt is the ground-truth bounding box. We calculate the Intersection over Union (loU) between Da and gt. This loU score allows the optimization to focus on the bounding box with larger intersections with ground truth. Dz(x) and Da(x:) are the objectiveness score and the class confidence score for the bounding box, respectively. We obtain our detection score (x) by multiplying the IoU score, objectiveness score, and class confidence score. We select the highest (2 compute Ladu (ar) using a log loss. By minimizing Lado we encourage the camouflaged vehicle to be undetected or misclassified by the detector.
3.2 Realistic Neural Rendering
The prior works from DTA and ACTIVE 19
prove that realistic rendering of the vehicle cam- ouflage is one of the keys to successful physical adversarial attack. To achieve this, rendering components: the first render component a differentiable neural renderer, which takes the 3D properties of the vehicle output the vehicle's foreground images. However, it struggles to render complex environmental characteristics on the vehicle surface. To alleviate this, we use an environmental feature renderer that combine the environmental characteristics and neural renderer output to produce realistic and accurate camouflaged vehicle images.
two
We select Pytorch3D differential renderer [12] to generate camouflaged vehicle images because it supports differentiable path to the UV-map texture image. Following the method proposed in DTA [1]] and ACTIVE [19]. we use a U-Net network for EFR to extract environmental characteristics from Xref and combine them with X,r- EFR outputs the camouflaged vehicle with environmental characteristics Xren-
Before camouflage generation, we need to train EFR for its optimal performance. The training of EFR needs masked vehicle images X Xref, 3D mesh Ms,, camera positions &eam. and various preset color texture T as input. Meanwhile, we obtain images of different preset colors from the Carla simulation environment. Then, we mask out the vehicle parts as the network ground truth GT. To optimize the parameters EFR, we use the following loss function:
LEFR(X rof) W (Xret) BCE (X GT),
(6)

---

(A) Hefore rearder
(b) After reordrr
Figure 3: Reordered texture UV map to improve camouflage naturalness.
H.W is weight function. H and where BCE is the binary cross-entropy loss, and W (X s W are the heights and widths of Xref, and S is the number of pixel points in the vehicle part of Xref- W (Xreff) can balance EFR rendering optimization across various camera angles, especially for views where the vehicle occupies small area of the image.
3.3 Adversarial Texture Generation with Diffusion Model
Prior works optimize the adversarial camouflage in the pixel space, which leads to unnatural and uncontrollable adversarial texture patterns. To alleviate this, we leverage an off-the-shelf stable diffusion model [15] generate vehiele UV-map texture image. The diffusion model is trained on a subset of LAION-5B [16]. a dataset of billions of image-text pairs. Since the diffusion model learns the manifold of natural images and corresponding text captions, it can generate adversarial texture images that look more natural and relevant to the given text prompt.
To make the generated UV-mmap images adversarial, introduce adversarial feature vector Fadu that can be optimized during the camouflage generation with diffusion models. The adversarial feature vector has the same hidden dimension the text feature vector Fext- Therefore, it can concatenate with Fta: to form the conditional input to the T2I model. As a result, the generated image reflects the control signal from the text feature while being adversarial against the target detector.
also reordered the UV map the vehicle texture so that vehicle surface could be connected as much possible. Figure 3 shows the UV mapping before and after reordering. The reordering makes the generated camouflage can keep more natural patterns generated by the diffusion model. Hence, it improves the naturalness of the vehicle camouflage.
3.4 Naturalness vs Attack Performance Trade-off
Without any constraints, the framework can move the combined feature from Fert and Fadv out of the feature space of the text prompt. Consequently, we can no longer expect the generated UV-map images to look natural and consistent with the input text prompt. Since the diffusion model is trained to generate natural and controllable images with original text features, there is higher chance of generating natural images the concatenated feature closer to the original text feature.
To preserve naturalness and controllability, we assure that the adversarial feature Fado will not exceed norm greater than threshold 5. Tuning the norm threshold enables us trade off naturalness and controllability for attack performance.
We follow PGD and choose ep norm constrain Fadv- We update Fads using the formula below:
Ft. adde =(FF-1 Fadv V (addv):
<(F) (F. F. min (max (Fi, -T) (F)
(8)
where / the time step, 7 is the step size, Viada the gradient of the adversarial loss, i the clipping function defined as in Eq. where F; is the i-th element of F.
5

---

Experiments
4.1 Experimental Settings
Datasets: We utilize the Carla 3 simulator generate datasets for our experiments. To have a comparative analysis with prior studies [22] 21
18 19. we select the Audi E- E-Troo as the target vehicle model. We create datasets using various simulation settings, resulting in 69,120 and 59,152 photo-realistic images for EFR training and testing. These images cover 16 distinct weather conditions, combining four sun altitudes and four fog densities. Additionally, we generate a dataset of 40,960 images for camouflage generation and test dataset of 8192 images for adversarial camouflage evaluation. The weather conditions included in these two datasets are the same as those used during the training and testing of EFR. We print out five types of adversarial camouflages for physical world evaluation and apply them to 1:12 Audi T-on car models. For each model, we capture 96 pictures under different elevations, azimuths, and distance parameters.
Baselines: We compare our method with four advanced adversarial camouflage methods: DAS 22 FCA (21). DTA [18], and ACTIVE [19]. DAS and FCA optimize the 3D texture by minimizing the attention-map scores and detection scores of the detector, respectively. DTA and ACTIVE both optimize square texture pattern with a neural network and cover it on the vehicle's ' surface repeatedly. We compare our results using the official textures generated by these methods. We apply these textures to the same car model for a fair comparsion.
Evaluation metries:
For training the EFR component, we follow the setting from 18 to use the Mean Absolute Error (MAE) as a loss to measure the difference between the output of ERP and the ground truth. To evaluate the effectiveness of the adversarial camouflage, we utilize the AP@0.5 benchmark [4) as it provides comprehensive assessment of recall and precision values at detection IOU threshold of 0.5.
Target detection models: Aligning with previous work, we adopt YOLOv3 [3] as the white-box target detection model for adversarial camouflage generation. To evaluate the effectiveness of the optimized camouflage, we utilize a collection of widely used object detection models treated as black- box models, including YOLOF 2 Deformable DETR (DDTR) (27), Dynamic R-CNN (DRCN) [24]. Sparse R-CNN (SRCN) 17 and Faster R-CNN (FTRCNN) 14. They are trained on the COCO dataset and implemented in MMDetection [I].
Following [8], we utilize the Adam optimizer with a learning rate 0.01 for EFR training and camouflage generation. We train the EFR for 20 epochs and choose the model with the best performance on the test dataset. We use stable diffusion v1.5 to generate UV-map texture image with DDIM sampler. We set the sampling step 20. The optimization of the adversarial camouflage takes duration of five epochs. We conduct experiments on a cluster with eight NVIDIA RTX A800 80GB GPUs.
Training details:
4.2 Attack Performance Evaluation
4.2.1 Attack in the Digital World
In this section, we compare our method to current advanced adversarial camouflage methods, includ- ing DAS (22), FCA (21). DTA [18), and ACTIVE [19]. We run an extensive attack comparison using diverse detection models. Since the target model YOLOv3, we use various detection models to evaluate the transferability of the camouflage in black-box setting. We use 'colorful camouflage' as the text prompt in this experiment.
The results are shown in Table D showing that our method has competitive performance with the current -tate-of-the-ar baselines. DAS performs only better than normal car painting, primarily due to the limitations of partially painted camouflage. Meanwhile, FCA exhibits sub-optimal performance, only slightly better than random camouflage, because it cannot render sophisticated environment characteristics. DTA and ACTIVE have comparable attack performance to CNCA, but our method achieves the best attack performance in total. Figure 4 shows the summarized performance of each camera pose and weather parameter; values are car AP@ averaged from the detectors used in Table T We can see that the camouflage produced by our method shows competitive performance compared to DTA and ACTIVE.

---

Table 1: Comparison of the effectiveness of camouflages across various object detection models. Values are A.@0.5 of the car.
SINGLE-STAGE
TWO-STAGE
METHODS
TOTAL
YOLOV33
YOLOF
DDTR
DRCN
SRCN
FRRCN
NORMAL
0.712
0.824
0.803
0.778
0.786
0.771
0.779
RANDOM
0.642
0.753
0.625
0.694
0.681
0.672
0.678
DAS
0,671
0.769
0.738
0.715
0.724
0.719
0.723
FCA
0,581
0.725
0.603
0.678
0.642
0.668
0.650
DTA
0.521
0.657
0.402
0.614
0.488
0.562
0.541
ACTIVE
0.577
0.436
0.534
0.484
0.520
0.504
CNCA
0.485
0.538
0,436
0.536
0.470
0.504
0.495
Normal
Random
DAS
FCA
DTA
ACTIVE
CNCA
5
AP@0.
0.2
0.0
0.0
22.5
45.0
67.5
45
50
l3s
180
ais 5 16 15 20:90 dis(m)
30 sun
50
30 6o
50
ele(")
azi(")
fog
Figure 4: Attack comparison on different camera poses and weather parameters. "ele" denotes elevation, "azi" denotes azimuth, "dis" denotes distance. "fog" denotes fog density, and "sun" denotes sun altitude angle. Values are car AP@0.5 (%) averaged from all models.
4.2.2 Attack the Physical World
We conduct the physical world evaluation using 1:12 Audi E-Tron car models with camouflages generated by different methods. Table shows the attack performance of these camouflages against real-time object detectors in the real world: YOLOv3, YOLOX [5], SSD [10], CenterNet [26]. RetinaNet [9]. Our method achieves the best attack performance against four out of five detectors and the best overall performance. Furthermore, Figure Shohows that our method can successfully attack the detectors in both indoor and outdoor environments compared other methods. In summary, the physical evaluation results demonstrate that our method is transferable to the real world.
4.3
Customzablility and Naturalness Evaluation
4.3.1
Customizable Camouflage Generation
Our method enables customizable camouflage generation with user-specific input. Table 3oble3sshe shows the various styles of adversarial camouflages with their corresponding text prompts and AP@05 values (target YOLOv3). Our method can directly customize the color choices and patterns of the camouflage with the input texture prompt. We notice that there are some differences attack performance regarding different prompts. During our experiments, we found the prompts that describe natural objects (columns 4,5,6 Table are likely to have lower attack performance(averga, around 0.03 in AP@0.5) than the more abstract ones (columns 1,2,3 in Table 3 Despite this, the average AP@0.5 of all these camouflages generated by CNCA is 0.506, comparable the reported result in Table 1

---

Table 2: A.@ of the different methods in the physical world evaluation.
METHODS
YOL.OV3
YOLOX
SSD
CENTERNET
RETINANET
TOTAL
NORMAL
0.778
0.936
0.890
0.916
0.981
0.900
DAS
0.734
0.878
0.847
0.873
0.955
0.857
FCA
0.560
0.770
0.767
0.798
0.921
0.763
DTA
0.566
0.689
0.786
0.854
0.886
0.756
ACTIVE
0.518
0.563
0.574
0.743
0.735
0.627
CNCA
0.439
0.464
0.557
0.698
0.780
0.588
Normal
DAS
FCA
DTA
ACTIVE
CNCA
Indoor
Outdoor
Figure 5: Examples of real-world evaluation for different methods. The evaluation includes both indoor and outdoor environments.
Table Customizable camouflages with different text prompts. The AP@0. of each camouflage is shown below. The normal car texture baseline is 0.712.
CUSTOMIZABLE CAMOUELAGE GENERATION WITH USER-SPECIFIC TEXT PROMPT.
colorful graffiti
yellow black graffiti colorful camouflage
colorful balls
snake texture
zebra strips
0.508
0.479
0.485
0.516
0.486
0.561
Table 4: Subjective tests for the naturalness evaluation of our adversarial camouflage with other baselines. The naturalness reporting mean and standard deviations of each type of camouflage, we also conduct t-tests and report the and p values verify the differences means scores are significant (p value less than 0.05). As shown in the results, our method's 's score is significantly higher than the previous four methods.
is scaled from (not natural all) 5(very natural). Besides
IMAGES
SCORE
4.68 + 0.67
2.05 + 1.03
2.11 1.17
1.86 0.98
1.84 0.99
2.84 + 1.09
VALUE
8.72
-3.18
-2.77
-4.04
-4.13
NA
VALUE
<0.00001
0.00108
0.00358
0.00007
0.00005
NA
SOURCE
NORMAL
DAS
FCA
DTA
ACTIVE
CNCA(OURS)

---

Table Ablation study of CNCA pipeline components. The input text prompt is "yellow, black graf- fiti." We evaluate each test pipeline in terms of attack performance and naturalness. The description of each test pipeline the following: No Diff: we directly optimize the UV map texture of the vehicle without the integration of the diffusion model; Diff.: we use the diffusion model to generate UV-map texture but without introducing adversarial feature; Diff.+ Adv. Introduces the adversarial feature but without the clipping strategy; Adv. introduces clipping strategy but without reordering of texture map; Div..Adv.+ Cld.++CCiiReeodeer is our final pipeline with reordering the texture mask.
PIPELINE
No DIFF.
DIFF.
DIFF ADV.
DIFF ADV. CLIP
DIFF.
ADV. CLIP REORDER
TEXTURE
AP
0,619
0,553
0,520
0.494
0.479
SCORE
1
3.37
1.71
2.75
3.33
4.3.2 Naturalness Score by Subjective Evaluation
Our proposed approach aims to improve the naturalness of the generated adversarial camouflage to humans. Therefore, following [7. we conducted subjective evaluation to estimate the naturalness score of the adversarial camouflages. For fair comparison, we generate a series of vehicle images using the same set camera positions for each type of camouflage. Then, we require each participant give naturalness score for each type of camouflage from scale of 1 (not natural all) 5 (very natural). Besides the advanced adversarial camouflages from prior work, we also include the normal car texture as the control group. To further demonstrate the significance of the differences in mean scores, we also conduct t-tests and report the and values. As shown in Table the p values of I-tests are lower than 0.05, Therefore, our method's naturalness score is significantly higher than those of the four adversarial camouflages.
4.4 Ablation Studies
The impact of each CNCA pipeline component. Table 5 shows the results of the ablation studies for each component of the pipeline. We gradually add each component during the ablation study to see their contribution to attack performance and naturalness. All the test pipelines with the diffusion model use the same input text prompt: "yellow black graffiti." " 'he discussion for each test pipeline is the following: No Diff. directly optimizes the texture image of the vehicle at a pixel level, resulting in an unnatural texture; Diff. introduces the diffusion model to generate the texture image compared to No Diff., which improves the naturalness score; Diff.+Adv. introduces the adversarial feature compared to Diff., which enables the texture image generation guided by the adversarial gradient from the detector. The adversarial feature improves the attack performance but compromises the naturalness: AAd. +Clip introduces the clipping strategy compared to Diff.+Adv. which improves the naturalness; Diff. Diff.+Adv.+Ciipppp +ll.v++Ciiip+RRordee is our final pipeline, which uses a reordered texture map compared to Diff.+Adv.+Clip. which further improves the attack performance and naturalness. To summarize, the ablation studies demonstrate that all the components of our pipeline contribute to improving the camouflage's attack performance and naturalness.
Trade-off between Naturalness and Attack Performance. There is inevitably a trade-off between naturalness and attack performance. The optimization space for adversarial attacks decreases when increasing naturalness of the attack. Therefore, increasing naturalness will typically decrease attack performance. Our method allows users balance this trade-off based on their preference by adjusting the norm threshold т. To illustrate this trade-off, we generated adversarial UV texture images with the same text prompt but different norm threshold settings for the adversarial features. In addition, we conduct subjective survey to rank the naturalness of the images and their relevance regarding the text prompt. Table shows the average rank of each image, its corresponding AP@0.5, and the norm threshold settings. It can be seen that when the norm threshold starts to increase, the texture images become less natural-looking. and the attack performance increases. Therefore, requires the user to decide the acceptable naturalness level.

---

Table Naturalness test average score against attack performance using YOLOv3. The input text prompt is "yellow black graffiti..
THRESHOLD
0.1
0.5
1.5
TEXTURE
AP@0.5
0.606
0.449
0.479
0.481
0.520
SCORE
4.79
3.54
3.33
1.46
1.71
5 Limitations & Societal Impact
Limitations. Our current method has the following limitations: firstly, the back pack-propagation is more expensive than the previous gradient-based methods, but we believe the strong prior knowledge of the diffusion model can further advance adversarial attacks and defenses; secondly, our method needs the 3D mesh of the target object to generate camouflage. If the 3D mesh of the object is not available, the user needs additional efforts such as photogrammetry or 3D scanning obtain it; lastly, for each text prompt, manual hyperparameter tuning for norm threshold is needed. One future work is to automatically adjust T based on the relevance score between the image and text prompt, for instance, CLIP score [TI.
Societal Impact. This paper presents work whose goal is to advance the safety of Al systems. While the proposed adversarial attack method could be potentially used by malicious users, it can also support future efforts enhance the robustness of AI system via adversarial training, adversarial testing and adversarial example detection, thereby safeguarding the security of Al systems.
6 Conclusion
We propose a novel physical adversarial camouflage attack framework with diffusion models. With different user-specific text prompts, our method can generate adversarial camouflage with diverse colors and patterns. particular, we apply clipping strategy to an adversarial feature to balance the adversarial camouflage's naturalness and attack performance. With extensive experiments on the digital and physical world and user studics, the results demonstrate that our methods improve the naturalness and enable the customizability of camouflage generation while maintaining competitive attack performance.
Acknowledgment
This research is supported by the National Natural Science Foundation of China (Grant: 62306093, 62376074), the Guangdong Provincial Key Laboratory of Novel Security In- telligence Technologies (Grant No. 2022B 002201212010055,, the Shenzhen Science and Tech- nology Program (Grans/SGGKKODD22221 101 115655007. SGDX202301 (160912440044 the Fundamental Research Funds for the Central Universities (Grant No. HIT.OCEF 20244024477) and the Fok Ying Tung Education Foundation of China (Grant 171058). Daojing He and Yu Li are the corresponding author of this article.
BKX202311100908500
References
[1] Kai Chen, Jiaqi Wang. Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. 2019. MMDetection: Open MMLab Detection Toolbox and Benchmark CoRR, abs/1906.07155.
10

---

[2] Qiang Chen, Yingming Wang, Tong Yang, Xiangyu Zhang, Jian Cheng, and Jian Sun. 2021. You only look one-level feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13039-13048.
[3] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Conference on robot learning, pages 1-16.
PMI.R
[4] Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2015. The pascal visual object classes challenge: A retrospective. International journal of computer vision, 111:98-136.
[5] Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. 2021. YOLOX: Exceeding YOLO Series in 2021 CoRR, abs/2107.08438
[6] Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851.
[7] Yu-Chih-Tuss Hu, Bo-Han Kung, Daniel Stanley Tan, Jun- Jun-Cheng Chen, Kai-Lung Hua, and Wen-Huang Cheng. 2021. Naturalistic physical adversarial patch for object detectors. In Proceedings of the IKEF/CVF International Conference Computer Vision, pages 7848-7857.
[8] Zhanhao Hu, Wenda Chu, Xiaopei Zhu, Hui Zhang, Bo Zhang, and Xiaolin Hu. 2023. Physi- cally realizable natural-looking clothing textures evade person detectors via 3d modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16975-16984.
[9] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming Hc, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pages 2980-2992
[10] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In Computer Vision- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part 14, pages 21-37. Springer.
[11] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. 2021. Learning transferable visual models from natural language supervision In Pro- ceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748-8763. PMLR.
[12] Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan- Yen Lo, Justin Johnson, and Georgia Gkioxari. 2020. Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501
[13] Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint drXiv:1804.02767.
[14] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2017. Faster R- R-CNN: Towards Real-Time Object Detection with Region Proposal Networks) IEEE Trans. Pattern Anal. Mach. Intell., 136)-1177119199
[15] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 10684-10695.
[16] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis. Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaezmarezyk, and Jenia Jitsev. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models Advances in Neural Information Processing Systems, volume 35, pages Curran Associates, Inc.

---

[17] Peize Sun, Rufeng Zhang, Yi Jiang. Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. 2021. Sparse r-cnn: End-to-end object detection with learnable proposals. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14454-14463.
[18] Naufal Suryanto, Yongsu Kim, Hyoeun Kang, Harashta Tatimma Larasati, Youngyeo Yun, Thi- Thu-Huong Le, Hummin Yang, Se- Yoon Oh, and Howon Kim. 2022. DTA: Physical Camouflage Attacks Using Differentiable Transformation Network. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 15305-15314.
[19] Naufal Suryanto, Yongsu Kim, Harashta Tatimma Larasati, Hyoeun Kang, Thi-Thu-Huong Le, Yoonyoung Hong, Hunmin Yang, Yoon Oh, and Howon Kim. 2023. ACTIVE: Towards Highly Transferable 3D Physical Camouflage for Universal and Robust Vehicle Evasion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4305-4314.
[20] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Good- fellow, and Rob Fergus. 2014. IInriiguing properties of neural networks) In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.
[21] Donghua Wang, Tingsong Jiang, Jialiang Sun, Weien Zhou, Zhiqiang Gong, Xiaoya Zhang, Wen Yao, and Xiaoqian Chen. 2022. Fea: Learning a 3d full-coverage vehicle camouflage for multi-view physical adversarial attack. In Proceedings the AAAI conference artificial intelligence, volume 36, pages 2414-2422
[22] Jiakai Wang, Aishan Liu, Zixin Yin, Shunchang Liu, Shiyu Tang, and Xianglong Liu. 2021. Dual attention suppression attack: Generate adversarial camouflage physical world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8565-8574
[23] Tong Wu, Xuefei Ning. Wenshuo Li, Ranran Huang, Huazhong Yang, and Yu Wang. 2020. Physi- cal adversarial attack on vehicle detector in the carla simulator. arXiv preprint arXiv:2007.1611 16118
[24] Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, and Xilin Chen. 2020. Dynamic R-CNN: Towards high quality object detection via dynamic training. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XV 16, pages 260-275. Springer.
[25] Yang Zhang, PD Hassan Foroosh, and Boqing Gong. 2019, camouflage for physical adversarial attack on object detections in the wild. ICLR.
Camou: Learning a vehicle
[26] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. 2019. Objects as points. arXiv preprint 1904v0190.007870
[27] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. 2021. Deformable DETR: Deformable Transformers for End-to-End Object Detection) In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OvenReview.net.
12

---

A Subjective Human Evaluation
Vehicle Camouflage Naturalness Survey
andinany very compploosss.
00 Rebe anturanness the vehicle
003 Rate the naturainess the vehiccle
"0s Rate the naturalness the vetrctte
*S naturalness the velicle
Figure 6: The interface of our Survey for several pictures of the camouflaged vehicle for each method. The order of the methods is random. We ask the participants to rate the naturalness of the camouflage on scale of to 5.
We present the participants with
To better evaluate the naturalness of CNCA compared with another adversarial camouflage attack, we conducted survey among humans with the assistance an online form. The user interface of the survey is shown in Figure/o where the participants are asked to give (vvery unnatural) to 5(natural) rating regarding the naturalness of the vehicle's 's appearance. We collect surveys from 45 participants up to the completion of the writing and most of them are not familiar with adversarial attacks. Table Iilluttrates the results, indicating that CNCA achieves higher naturalness among human participants compared with previous advanced adversarial camouflage.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clearly state the contributions and scope of paper in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of our work in the section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Our work result are mostly experimental and does not involve obtaining theoretical result.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have described the detailed information to reproduce our experiment experiment setting section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We have provided the related data and code link the abstract for reproducing our results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/ccpppllliggudeeeeeeeeeeeee Cooee///ddeeebmmmcii ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have described experiment setting and details in the section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For our subjective evaluation of naturalness, we conduct t-tests to demonstrate the significant differences of the mean scores between method and previous methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have described the required hardware computer resources information to reproduce our experiment in the section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpps://eurris..ccppbllc (tt.aiiieeehiisuueellle.ee..
Answer: [Yes]
Justification: We have reviewd the NeurIPS code of Ethics and confirm our search are conducted aligned with the NeurIPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed both the potential positive and negative societal impact in the section 5
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: We have discussed safeguards to enhance the Al systems against our proposed attack the section
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We confirmed that all the assets that we used in our worked are properly cred- ited. The related license and terms use explicitly are mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, patasets.papersstt paperswi thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pace se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The only new asset introduced in the paper is the Environment Feature Renderer (EFR). The information about this asset such as model structure, training and testing are documented in the Methods section
Guidelines:
The answer NA means that the paper does release new assets. Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We have included the full text of instructions and screenshot to participants of subjective evaluation in the appendix section 6 The participants were compensated with small gifts such as (bottled water or online discount coupons) since our evaluation did not take a significant amount of time to complete (an average of five minutes).
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [No]
Justification: There is no obvious potential risk for our participants. We have confirmed that the researchers in our research institution can make an initial determination of minimal risk for studies involving surveys with limited evaluation time and negligible psychological or physical impact. However, we recognize that consulting an IRB or equivalent body would provide more formal risk assessment, and will take this step in future studies
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
20


Reviews:
[{"review_id": "esO86Cwh7H", "reviewer": "Reviewer_TBRf", "summary": "The paper introduces a novel framework, CNCA, for generating customizable and natural adversarial camouflage for vehicle detectors using a diffusion model. This work addresses critical limitations in current adversarial camouflage techniques by focusing on naturalness and customizability, which are often neglected in favor of attack performance. While the paper presents a significant advancement in adversarial camouflage, several areas require improvement to enhance rigor and presentation. The proposed CNCA framework holds substantial promise, but further validation and detailed comparison are essential to establish its superiority and practical relevance.", "questions": "Experimental Statistical Significance: The authors recruited 45 participants to subjectively evaluate the naturalness of different camouflages, reporting the mean scores and standard deviations (SD) for naturalness of each type of camouflage. While this is good, merely reporting the mean scores and SD does not statistically demonstrate whether the differences in mean scores are significant. It would be more convincing to conduct t-tests or ANOVA (preferably repeated measures ANOVA with post hoc tests, based on the current experimental design) and report the relevant statistics (e.g., t and F values, as well as p values). \n\nPhysical World Evaluation: In the physical world evaluation, the paper only compared two models, one for a normal and another for the generated camouflage. Have the authors considered including models with other adversarial camouflage methods for comparisons, as the authors did in the digital world?", "limitations": "Perform a thorough comparison with state-of-the-art methods like AdvCam and UAPs that are known for their effectiveness. Discuss the differences in performance metrics such as attack success rate, naturalness, and computational efficiency. Highlight the advantages and limitations of CNCA relative to these methods.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.\tThe use of a diffusion model for generating natural and customizable adversarial camouflage is novel.\n2.\tThe extensive experiments, including both digital and physical settings, provide strong evidence of the method's effectiveness.", "weaknesses": "1.\tThe explanation of the adversarial feature generation and its integration with the diffusion model is somewhat convoluted. Quantitatively define the evaluation indicators of naturalness and attack performance, or provide relevant references.\n2.\tThe evaluation in the physical world is limited to small-scale models and specific conditions. Extend the evaluation to a broader range of vehicle detection models and datasets, including those used in autonomous driving (e.g., KITTI, Waymo Open Dataset). Assess the scalability of CNCA by testing on larger, more complex scenes and different environmental conditions to validate its general applicability.\n3.\tThe paper lacks ablation studies to isolate the impact of different components of the proposed framework. Conduct ablation studies to demonstrate the contribution of each component (e.g., the diffusion model, adversarial feature clipping) to the overall performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: The explanation of the adversarial feature generation and its integration with the diffusion model is convoluted. Quantitatively define the evaluation indicators of naturalness and attack performance or provide relevant references.**\n\nDuring the normal T2I diffusion model inference process, the text prompt is encoded to a text feature vector with the shape of (N, W) to guide the denoising process. N is the number of tokens, and W is the feature dimension of the token embedding. Our method introduces an adversarial feature vector with the shape of  (M, W). M is a hyperparameter that defines the size of adversarial information. During the adversarial camouflage generation, the clipping strategy regulates the adversarial feature vector. Then, it is concatenated with the text embedding feature to form the combined feature vector with the shape of (M+N, W) as input to the diffusion model. The diffusion generates vehicle texture, and the detector processes the camouflaged vehicle image. The adversarial feature is optimized by the detector's adversarial loss.\n\nThe quantitative measurement of attack effectiveness used in our paper is car AP@0.5. This metric is computed based on the precision-and-recall curve for the car category. It's a popular performance metric for object detection, whose definition can be found in[1]. Previous state-of-the-art methods like DTA and ACTIVE have used this metric to evaluate their attack performance. The most relevant previous work on quantitative measuring of naturalness is S. Li et al. [2]. They collected vehicle images and human rating data to train a model to assess the naturalness automatically. We try to use their trained model in our case. However, we found a bias of low scoring towards vehicles with full-covered paintings, even if the painting is human-designed and looks natural. All the baselines except DAS receive low scores. We suspect it is because the dataset lacks full-cover painting vehicle images. As a result, we follow the previous work of Hu et al. [3] to conduct a subjective survey to evaluate the naturalness directly. To maintain the fairness of the survey, we invited 45 participants from different ages, backgrounds, and genders, which is twice the number (24) in [3].\n\n[1] Everingham, Mark, et al. \"The Pascal visual object classes (VOC) challenge.\" International journal of computer vision 88 (2010): 303-338.\n\n[2] S. Li et al., “Towards Benchmarking and Assessing Visual Naturalness of Physical World Adversarial Attacks,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, pp. 12324–12333, May 2023.\n\n[3] Hu, Yu-Chih-Tuan, et al. \"Naturalistic physical adversarial patch for object detectors.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n**W2: Assess the scalability of CNCA by testing on larger, more complex scenes and different environmental conditions to validate its general applicability.**\n\nThe camouflage generated by our work needs to be deployed on a 3D model in the simulator or on a real vehicle model to get test data. Our generated camouflage cannot extend to the KITTI and Waymo datasets, which only contain 2D videos and images of vehicles. We try to extend our physical evaluation in indoor and outdoor scenarios and compare our method with previous baselines, as shown in Table 2 and Figure 1 in the attached PDF. The results show that our methods can achieve competitive attack performance among previous baselines. We acknowledge that our current evaluation is limited in scale and conditions due to our limited resources and budget. We agree that the road test evaluation for our method is essential, especially for autonomous driving. But due to its high time and labor cost, we plan to explore this in our future work.\n\n**W3: The paper lacks ablation studies to isolate the impact of different components of the proposed framework.**\n\nWe have added extra ablation studies to demonstrate the contribution of each component, including the diffusion model, adversarial feature, clipping strategy, and mask reordering. Due to the word limits, please find the details of the ablation study in G1 section the global Author Rebuttal.\n\n**Q1: Experimental Statistical Significance: conduct t-test and ANOVA test to prove the significance of the naturalness survey data.**\n\nThanks for your suggestion. We conduct a t-test and ANOVA with a post-hoc Tukey HSD test on our data. The results of the t-test are:\n\n| Compare Method | t Value | p Value |\n|----------------|---------|-----------|\n| Normal         | 8.72    | < .00001  |\n| DAS            | -3.18   | 0.00108   |\n| FCA            | -2.77   | 0.00358   |\n| DTA            | -4.04   | 0.00007   |\n| ACTIVE         | -4.13   | 0.00005   |\n\nThe results of the ANOVA with post-hoc test are:\n\n| Compare Method | Q Value | p Value |\n|----------------|---------|---------|\n| Normal         | 11.18   | 0.001   |\n| DAS            | 4.77    | 0.011   |\n| FCA            | 4.44    | 0.023   |\n| DTA            | 5.91    | 0.001   |\n| ACTIVE         | 6.08    | 0.001   |\n\nThe p values of the t-test and ANOVA test are lower than 0.05. Hence, we can conclude that the differences between the baselines and CNCA in the naturalness evaluation are significant. \n\n\n**Q2: Include models with other adversarial camouflage methods for comparisons in the physical world evaluation.**\n\nPlease refer to the reply to W2. \n\n**L1: Compare CNCA with methods like AdvCam and UAPs.**\n\nMethods like AdvCam and UAPs can generate natural and effective physical adversarial examples based on one 2D image of the object. However, these physical adversarial examples are not robust against diverse viewing angles because they are typically optimized for a fixed viewing angle. As a result, these methods are not competent for our task because the attack needs to be effective at various viewing angles and distances. Therefore, we did not compare these methods with CNCA.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zs3OlM73EX", "reviewer": "Reviewer_guSy", "summary": "The manuscript presents a novel approach to generating physical adversarial camouflage against vehicle detectors, leveraging a pre-trained diffusion model. The proposed method, called Customizable and Natural Camouflage Attack (CNCA), aims to produce adversarial camouflage that is both natural-looking and customizable via user-specific text prompts. This approach addresses the limitations of previous methods that produced conspicuous and unnatural camouflage, maintaining effectiveness in adversarial attacks while enhancing the camouflage's appearance to blend seamlessly into its surroundings.", "questions": "See strength and weakness above.", "limitations": "No. While the paper discusses potential positive impacts, such as improving AI robustness, the technique could also be used maliciously to evade surveillance, posing ethical and security concerns.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "CNCA introduces a novel application of diffusion models for generating physical adversarial camouflage, a significant shift from the traditional pixel-level optimization methods.\n\nThe method allows for the generation of camouflage that is not only effective in evading detection but also customizable and more natural-looking, meeting specific user requirements.\n\nThe manuscript provides a comprehensive evaluation of the CNCA approach, including both digital and physical world tests and user studies, demonstrating its effectiveness and practical applicability.", "weaknesses": "The approach involves complex integration of diffusion models with adversarial attack frameworks, which may increase the computational overhead and complexity compared to more straightforward adversarial techniques.\n\nAlthough the manuscript includes extensive testing, the evaluations focus primarily on vehicle detection in controlled settings. The performance and practicality of CNCA in more varied or less controlled environments remain to be fully explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: Integration of Diffusion models with adversarial attack frameworks may increase computational overhead and complexity.**\n\nWe have discussed this weakness in the section on Limitations & Societal Impact. We would like to clarify that our work's novelty enables the naturalness and customizability of the physical adversarial attack. Integration with diffusion models is the key to achieving this. Despite its higher computational cost, camouflage can be generated offline at a one-time cost. The generated camouflage can be used to attack a wide range of detectors. In summary, introducing diffusion models is still valuable, although it increases the computation cost. \n\n**W2: The evaluations focus primarily on vehicle detection in controlled settings. The performance and practicality of CNCA remain to be fully explored in more varied or less controlled environments .**\n\nVehicle detection is crucial in autonomous driving and traffic monitoring. Therefore, many previous methods chose to research the physical attack on this task setting. Our work follows this research direction. We agree that we need to validate our method's performance and practicability in more varied environments. Therefore, we extend our physical evaluation to both indoor and outdoor environments, as shown in Figure 1 in the attached PDF. The results demonstrate that our method is transferable in varied real-world environments. \n\n**L1: The technique could also be used maliciously to evade surveillance, posing ethical and security concerns.**\n\nWe have discussed this in the section of Limitations & Societal Impact. We acknowledge the potential malicious usage of our technique. However, although our method generates physical attack examples, these examples can be used for the research of defense methods, such as adversarial training, adversarial testing, and adversarial example detection. The research of defense can ultimately safeguard AI systems.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jrLPSu1VSr", "reviewer": "Reviewer_MuNz", "summary": "The paper introduces a interesting idea and also a novel method called Customizable and Natural Camouflage Attack (CNCA) to generate adversarial camouflage against vehicle detectors, leveraging a pre-trained diffusion model. This approach allows the generation of natural-looking and user-customizable adversarial patterns that maintain robust attack performance across various digital and physical settings. The paper's contributions include a unique application of diffusion models to adversarial camouflage, introduction of adversarial features for gradient-based generation, and a clipping strategy to balance naturalness with attack performance. Extensive experiments and user studies demonstrate the effectiveness of CNCA in producing more natural-looking camouflage with competitive attack performance.", "questions": "- I am curious about its complexity. The method involves multiple components and parameters (e.g., adversarial features, clipping strategy), which might complicate its deployment in practical applications without substantial customization and tuning.\n- It would be more helpful if more ablation experiments were added to individually demonstrate the functions of each component.", "limitations": "Refer to the “Questions” section.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper proposes an interesting and useful application direction, namely natural and customized adversarial camouflage. The research motivation has substantial practical significance, and the proposed method appears intuitively reasonable.\n- This study is the first to apply diffusion models for natural adversarial camouflage generation. It is also the first to generate various 52 styles of adversarial camouflage against vehicle detectors.\n- The experiments are thorough, and the results are statistically significant, indicating high-quality research.\n- The code is provided.", "weaknesses": "- Some expressions are not clear, making it difficult for those unfamiliar with the field to understand. For example, lines 32 to 35. It would be better and easier to understand if some visual evidence were provided regarding these limits.\n- Figure 1 is not correctly referenced.\n- The experimental section would be more convincing if the effectiveness of the proposed components and methods were demonstrated through ablation experiments.\n- Concerning anonymity: Some comments in the provided code reveal personal information. Please be aware of this!", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: lines 32-35 are not clear to understand. Provide visual evidence to illustrate.**\n\nWe would like to clarify that lines 32-35 explain the two reasons why the previous camouflage methods lack naturalness. Firstly, these methods lack prior knowledge of naturalness to guide the camouflage generation. Secondly, these methods optimize the adversarial texture at a pixel level, making it difficult to form a natural-looking texture pattern. For instance, the previous state-of-the-art methods like FCA, DTA, and ACTIVE generate suspicious and attention-grabbing patterns, resulting in a low-score performance in naturalness, as shown in Table 4 in our paper. \n\n**W2: Figure 1 is not correctly referenced.**\n\nWe will correct it in future versions of our paper.\n\n\n**W3: The experimental section would be more convincing if the effectiveness of the proposed components and methods were demonstrated through ablation experiments.**\n\n| Pipeline                    | No Diff. | Diff. | Diff. + Adv. | Diff. + Adv. + Clip | Diff. + Adv. + Clip + Reorder |\n|-----------------------------|----------|-------|--------------|---------------------|------------------------------|\n| AP@0.5                 | 0.619    | 0.553 | 0.520        | 0.494               | 0.479                        |\n| Natural Score                   | 1.00        | 3.37  | 1.71         | 2.75                | 3.33                         |\n\n\nThe above table shows the results of the ablation studies for each component of the pipeline. During the ablation study, we gradually add the components individually to see their contribution to attack performance and naturalness. All the test pipelines with the diffusion model use the same input text prompt: \"yellow black graffiti.\" The description for each test pipeline is the following: \n\n- **No Diff.** directly optimizes the texture image of the vehicle at a pixel level, resulting in an unnatural texture;\n\n- **Diff.** introduces the diffusion model to generate texture image compared to **No Diff.**, which improves the naturalness score; \n\n- **Diff.+Adv.** introduces the adversarial feature compared to **Diff.**. The attack performance improves but compromises the naturalness; \n\n- **Diff.+Adv.+Clip** introduces the clipping strategy compared. to **Diff.+Adv.**, which improves the naturalness; \n\n- **Diff.+Adv.+Clip+Reorder** is our final pipeline, which uses a reordered texture map compared to **Diff.+Adv.+Clip**, which further improves the attack performance and naturalness. \n\n**No Diff.** optimizes the texture image of the vehicle with no prior knowledge of naturalness at a pixel level, which results in an unnatural texture image. To improve naturalness, **Diff.** leverages the prior knowledge from the diffusion model. **Diff.+Adv.** introduces the adversarial feature, which enables the texture image generation guided by the adversarial gradient from the detector. It improves the attack performance but compromises the naturalness. **Diff.+Adv.+Clip** introduces the clipping strategy to regulate the perturbation level of diffusion, which improves the naturalness. Our final pipeline **Diff.+Adv.+Clip+Reorder** uses a reorder texture map to keep more natural patterns generated by the diffusion model, further improving the attack performance and naturalness. \n\nTo summarize, the ablation studies demonstrate that all the components of our pipeline contribute to improving the camouflage's attack performance and naturalness. Please find the above table and corresponding generated texture images in Table 1 from the attached PDF.\n\n\n**W4: Concerning anonymity: Some comments in the provided code reveal personal information.**\n\nWe have made the necessary modifications to remove the personal information in the code. \n\n**Q1: The complexity during its deployment in practical applications without substantial customization and tuning.**\n\nIn the early stage of our experiment, we indeed spent some effort manually tuning the clipping threshold to generate the adversarial camouflage for a certain prompt. To amend this, we propose an automatic tunning method to adjust the clipping threshold dynamically. We calculate the relevance score between the input prompt and the generated image. If the relevance score is high above the pre-defined threshold, the clip threshold will use a larger value, allowing more exploration of the adversarial feature. If the relevance score is lower than the threshold, which means the generated images cannot match the input prompt, the clipping threshold will use a smaller value to constrain the adversarial feature. To calculate the relevance score, we leverage the CLIP[1] model,  an effective pre-trained model that learns the relevance between visual images and text captions. \n\nTo validate this idea, we conducted an ablation study on manual and automatic tunning under the same input prompt (\"yellow black graffiti\") and detection model (YOLOv3) settings, showing that they achieve comparable attack and naturalness performance measured by AP@0.5 and naturalness score, as shown in the table below. \n\n|     Tuning Method               | Manual Tuning | Automatic Tuning |\n|--------------------|---------------|------------------|\n| AP@0.5| 0.479         | 0.509            |\n| Natural Score        | 3.33          | 3.28             |\n\n[1] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" International conference on machine learning. PMLR, 2021.\n\n**Q2: Add more ablation experiments to demonstrate the functions of each component individually.**\n\nPlease refer to the reply to W3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xRaFKHVZcH", "reviewer": "Reviewer_oknY", "summary": "This paper presents a novel method CNCA for generating customizable and natural adversarial camouflage of fooling vehicle detectors. This work is an interesting contribution in the field of adversarial attacks, especially improving the naturalness of the camouflage while maintaining high attack performance.", "questions": "1. Why is there no comparison with baselines in the physical world?\n2. Whether other detection models can be used as white-box to generate adversarial textures to verify the transferability of CNCA.\n3. Only use the subjective evaluation may not be convincing for experimental verification, and visual observation does not seem to be more natural than previous methods. Is there a more convincing scoring rule to evaluate naturalness?", "limitations": "The experimental results are insignificant.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "It is interesting to apply diffusion models to physical adversarial attacks to generate natural camouflage for the first time, and it is also of practical value to generate natural adversarial camouflage with customizable styles based on text prompts.", "weaknesses": "The clipping strategy lacks innovation; it has been used in PGD for a long time, and it is not worth spending too much space on it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: The clipping strategy lacks innovation; it has been used in PGD for a long time, and it is not worth spending too much space on it.**\n\nThe core contribution of our work is introducing the diffusion model to enable the customizable and natural generation of physical adversarial camouflage. The clipping strategy from PGD itself is not novel, but our contribution is adapting it to regulate the perturbation level of the diffusion model. We describe the details of clipping ( in only 3 lines) because we want to make our method clear to readers who are not familiar it. \n\n**Q1: Why is there no comparison with baselines in the physical world?**\n\nFor our physical world evaluation, we follow the setting from the previous state-of-the-art works of DTA and ACTIVE, which only compares the performance between the normal and the generated camouflage. For the completeness of the evaluation, we extend the physical evaluation with previous baselines and both indoor and outdoor environments. The results are as shown in the table below:\n\n| **Methods** | YOLOv3 | YOLOX | SSD  | CenterNet | RetinaNet | Total |\n|-------------|--------|-------|------|-----------|-----------|-------|\n| **Normal**  | 0.778  | 0.936 | 0.890| 0.916     | 0.981     | 0.900 |\n| **DAS**     | 0.734  | 0.878 | 0.847| 0.873     | 0.955     | 0.857 |\n| **FCA**     | 0.560  | 0.770 | 0.767| 0.798     | 0.921     | 0.763 |\n| **DTA**     | 0.566  | 0.689 | 0.786| 0.854     | 0.886     | 0.756 |\n| **ACTIVE**  | 0.518  | 0.563 | 0.574| 0.743     | **0.735** | 0.627 |\n| **CNCA**    | **0.439** | **0.464** | **0.557** | **0.698** | 0.780 | **0.588** |\n\n\nThe results demonstrate that our method achieves comparable performance with previous baselines in the real world, which matches the results from the digital world evaluation. The details of the extended physical evaluation can be found in the G2 section of the global rebuttal. The physical examples can be found in Figure 1 in the attached PDF.\n\n**Q2: Whether other detection models can be used as white-box to generate adversarial textures to verify the transferability of CNCA.**\n\nYes, they can. We currently use YOLOv3 to generate adversarial camouflage for a fair comparison with previous methods (FCA, DTA, and ACTIVE), which also use this detector to generate camouflage. To validate the transferability, we use YOLOv5 as the attacked white-box detector to generate camouflage. We keep the other experiment settings the same, such as input prompt (yellow black graffiti) and clipping threshold (value is 1). We report the attack and naturalness performance using the average of car AP@0.5 over 5 different detectors and human evaluation. The results are as follows:\n\n|  Attack Detector             | YOLOv3 | YOLOv5 |\n|----------------|--------|--------|\n| Averaged AP@0.5         | 0.522  | 0.518  |\n| Natural Score  | 3.33   | 4.00      |\n\nThe results demonstrate that CNCA has similar attack performance regardless of the white-box models in the CNCA framework, which verifies its transferability.\n\n**Q3: Subjective evaluation may not be convincing for experimental verification, and visual observation does not seem to be more natural than previous methods. Is there a more convincing scoring rule to evaluate naturalness?**\n\nEvaluating the naturalness of physical adversarial attacks is a challenging task. We have surveyed the existing work related to this task. Among these works, S. Li et al. [1] is the most relevant one because they evaluate the naturalness of the physical attack for vehicle detection. They trained an evalution model with vehicle images and the corresponding human ratings. However, we found the trained model shows a bias of low scoring towards the full-covered painting vehicle image, even if the vehicle painting is designed by humans and looks very natural. As a result, all the full-cover baselines receive low scores. We believe it is caused by the training dataset's lack of full-cover painting vehicle images. To avoid this bias, we decide to follow the work of [2] and conduct a subjective survey to directly evaluate the naturalness by considering full-cover painting vehicles (for instance, racing cars with banners and logos) still natural. To maintain the fairness of the survey, we invited 45 participants from different ages, backgrounds, and genders, which is twice the number in [3] ( which is 24).\n\n[1] S. Li et al., “Towards Benchmarking and Assessing Visual Naturalness of Physical World Adversarial Attacks,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, pp. 12324–12333, May 2023.\n\n[2] Hu, Yu-Chih-Tuan, et al. \"Naturalistic physical adversarial patch for object detectors.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n**L1: The experimental results are insignificant.**\n\nOur primary goal is to improve naturalness and enable customization of the physical adversarial camouflage. It is challenging to achieve this goal while maintaining competitive attack performance. We leverage the pre-trained diffusion model to generate adversarial texture images with input prompts to tackle this. Besides, we introduce adversarial features so that the adversarial gradient from the detector can guide the camouflage generation. Finally, we adapt the clipping strategy to balance the trade-off between attack performance and naturalness. **Experimental results show that our attack performance is close to the state-of-the-art method (ACTIVE), but our method's naturalness score is 54% higher, which is a significant improvement**. While our work isn't perfect, it represents a significant initial effort towards customizable and natural camouflage generation. We believe it will draw increased research attention to this field.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We want to thank all the reviewers for their insightful comments on our work. Most reviewers mention the ablation studies of CNCA pipeline components and comparisons with the previous baselines in the physical world. Hence, we have extended the ablation study and physical evaluation as suggested, which are discussed in the following sections. 

**G1. Ablation studies for each CNCA pipeline component.**

| Pipeline                    | No Diff. | Diff. | Diff. + Adv. | Diff. + Adv. + Clip | Diff. + Adv. + Clip + Reorder |
|-----------------------------|----------|-------|--------------|---------------------|------------------------------|
| AP@0.5                 | 0.619    | 0.553 | 0.520        | 0.494               | 0.479                        |
| Natural Score                   | 1.00        | 3.37  | 1.71         | 2.75                | 3.33                         |


The above table shows the results of the ablation studies for each component of the pipeline. During the ablation study, we gradually add each component to see their contribution to attack performance and naturalness. All the test pipelines with the diffusion model use the same input text prompt: "yellow black graffiti." The description for each test pipeline is the following: 

- **No Diff.** directly optimizes the texture image of the vehicle at a pixel level, resulting in an unnatural texture;

- **Diff.** introduces the diffusion model to generate the texture image compared to **No Diff.**, which improves the naturalness score; 

- **Diff.+Adv.** introduces the adversarial feature compared to **Diff.**. This enables the texture image generation guided by the adversarial gradient from the detector. With this component, the attack performance is improved, but the naturalness is compromised; 

- **Diff.+Adv.+Clip** introduces the clipping strategy compared. to **Diff.+Adv.**, which improves the naturalness; 

- **Diff.+Adv.+Clip+Reorder** is our final pipeline, which uses a reordered texture map compared to **Diff.+Adv.+Clip**, which further improves the attack performance and naturalness. 

To summarize, the ablation studies demonstrate that all the components of our pipeline contribute to improving the camouflage's attack performance and naturalness. Please refer to Table 1 in the attached PDF for the above table and the corresponding generated texture images.

**G2. Comparison with the previous baselines in the physical world.**

We implement four previous baseline camouflages in the physical world for comparison and add more physical world testing scenarios. To be specific, our latest physical experiments are conducted in both indoor and outdoor scenarios. In each scenario, we choose two distances and two elevation angles. We also choose 24 azimuth angles for the outdoor scenario and 27 azimuth angles for the indoor scenario. The physical test set contains 204 images for each method. In the attached PDF,  the comparison of the methods is shown in Table 2; examples of indoor and outdoor environments are shown in Figure 1. The results show that our method still achieves comparable attack performance with previous baselines in the physical world.

Meta-review: This paper introduces a natural way to generate adversarial camouflage using diffusion models, claiming that the proposed method is more natural than the closest method (ACTIVE [19]). Naturalness is achieved by using diffusion models and is evaluated via human evaluation. 

This paper is on the borderline, but I all reviewers agreed on the novelty of this paper on the natural and customized adversarial camouflage via diffusion model. I think this paper could introduce a novel view in generating adversarial camouflage and this overweights the technical limitations. So, I vote for acceptance. 

In your final manuscript, please address discussed reviewers’ concerns. Additionally, 
* To mitigate the complexity of the algorithm, we recommend releasing code for the research community. 
* Please discuss limitations related to focusing on vehicle detection. We agree that vehicle detection is an important task, but to my understanding, having UV maps and 3D mesh could introduce difficulties in using this attack. Please discuss this limitation in Section 5 if you agree.
Decision: Accept (poster)