Fusion Is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection
Authors: Zhiyuan Cheng
Hongjun Choi
Shiwei Feng
James Chenhao Liang
Guanhong Tao
Dongfang Liu
Michael Zuzak
Xiangyu Zhang
Venue: ICLR 2024 poster (2024)
Keywords: Adversarial Attacks
3D Object Detection
Autonomous Driving
Primary area: general machine learning (i.e., none of the above)
TL;DR: We develop an adversarial attack framework to compromise camera-LiDAR fusion-based 3D object detection models from only the camera modality.

Abstract: Multi-sensor fusion (MSF) is widely used in autonomous vehicles (AVs) for perception, particularly for 3D object detection with camera and LiDAR sensors. The purpose of fusion is to capitalize on the advantages of each modality while minimizing its weaknesses. Advanced deep neural network (DNN)-based fusion techniques have demonstrated the exceptional and industry-leading performance. Due to the redundant information in multiple modalities, MSF is also recognized as a general defence strategy against adversarial attacks. 
In this paper, we attack fusion models from the camera modality that is considered to be of lesser importance in fusion but is more affordable for attackers. We argue that the weakest link of fusion models depends on their most vulnerable modality and propose an attack framework that targets advanced camera-LiDAR fusion-based 3D object detection models through camera-only adversarial attacks. 
Our approach employs a two-stage optimization-based strategy that first thoroughly evaluates vulnerable image areas under adversarial attacks, and then applies dedicated attack strategies for different fusion models to generate deployable patches. The evaluations with six advanced camera-LiDAR fusion models and one camera-only model indicate that our attacks successfully compromise all of them. Our approach can either decrease the mean average precision (mAP) of detection performance from 0.824 to 0.353 or degrade the detection score of a target object from 0.728 to 0.156, demonstrating the efficacy of our proposed attack framework. Code is available.

Published as a conference paper at ICLR 2024
FUSION Is NOT ENOUGH: SINGLE MODAL ATTACKS ON FUSION MODELS FOR 3D OBJECT DETECTION
Zhiyuan Cheng'
Hongjun Choi Michael Zuzak3
Shiwei Feng' James Liang?
Guanhong Tao
Dongfang Liu
Xiangyu Zhang' feng292, taog, xyzhang) @purdue.
Purdue University (cheng443,
DGIST hong junedgist .ww.sst.cc..... kr 3 Roochestr Institute of Technology (jc13689, dongfang.liu, mjzeec] (rieec)@ritteee
ABSTRACT
Multi-sensor fusion (MSF) is widely used autonomous vehicles (AVs) for per- ception, particularly for 3D object detection with camera and LiDAR sensors. The purpose of fusion is to capitalize on the advantages of each modality while minimizing its weaknesses. Advanced deep neural network (DNN)- fusion techniques have demonstrated the exceptional and industry-leading performance. Due to the redundant information in multiple modalities, MSF is also recognized as a general defence strategy against adversarial attacks. In this paper, we attack fusion models from the camera modality that is considered be of lesser impor- tance in fusion but is more affordable for attackers. We argue that the weakest link of fusion models depends on their most vulnerable modality, and propose an attack framework that targets advanced namera-LiDAR fusion-based 3D object detection models through camera-only adversarial attacks. Our approach employs a two-stage optimization- based strategy that first thoroughly evaluates vulnerable image areas under adversarial attacks, and then applies dedicated attack strategies for different fusion models to generate deployable patches. The evaluations with six advanced camera-LiDAR fusion models and one camera-only model indicate that our attacks successfully compromise all of them. Our approach can either decrease the mean average precision (mAP) of detection performance from 0.824 to 0.353, degrade the detection score of target object from 0.728 to 0.156, demonstrating the efficacy of our proposed attack framework. Code is available.
INTRODUCTION
3D object detection is a critical task in the perception of autonomous vehicles (AVs). In this task, AVs employ camera and/or LiDAR sensors input to predict the location, size, and categories of surrounding objects. Camera-LiDAR fusion models, which combine the high-resolution 2D texture information from camera images with the rich 3D distance information from LiDAR point clouds, have outperformed the detection accuracy of models that rely solely cameras or LiDAR. (Yang ct al. 2022 Liu et al. 20236b Li et al. 2022b). Additionally, multi-sensor fusion (MSF) techniques generally recognized as defensive measure against attacks (Cao et al. 2021 Liang al.. 2022), as the extra modality provides supplementary information to validate detection results. Viewed in this light, a counter-intuitive yet innovative question arises: Can we attack fusion models through a single modality, even the less significant one, thereby directly challenging the security assumption of MSF? Yet, this fundamental question has not been sufficiently answered in the literature.
Previous research has demonstrated successful attacks against Lamera-LiDAR fusion models by targeting either multiple modalities (Cao al. [022]] Tu et al.] [021] or the LiDAR modality alone (Hallyburton et al., 2022). However, these approaches are not easy to implement and require additional equipment such as photodiodes, laser diodes (Hallyburton et al. (022), and industrial- grade 3D printers (Cao et al. (021]: Tu et al., (021) to manipulate LiDAR data, thus increasing the deployment cost for attackers. Consequently, we explore the possibility of attacking fusion models via the camera modality, attackers can more easily perturb captured images using affordable ad- versarial patches. Nevertheless, this attack design presents additional challenges. For example, the camera modality is considered less significant in fusion models for 3D object detection since LiDAR provides abundant 3D information. The performance of both state-of-the tof-the-art LiDAR-based models and ablations of fusion models using only LiDAR surpasses their solely camera-based counterparts

---

Published as a conference paper at ICLR 2024
Fusion Model Input
I: Sensitivity Recognition
I1: Single Modal Modal Attacks
Undotocted Pedustriane
Multi-view Images
Global Sensitivity
Scene-oriented Attacks
AND
OR
Veltide
Lidar Point Cloud
Object Sensitivity
Object-oriented Atlacks
Figure 1: Single-modal attacks against camera- LiDAR fusion model using samera-modalle
significantly (Liang et al., 2022 Liu et al. 2023b Motional 2023) (see more experimental results in Appendix[A] A). The less significance of camera modality in fusion limit its impact on detection re- sults. Moreover, different fusion models can exhibit distinet vulnerabilities in the camera modality, necessitating varying attack strategies. The cutting-edge adversarial patch optimization technique against camera-only models (Cheng et al.) (002)) has limitations in generating deployable patches viewing the entire scene, as they fail to consider the semantics of the input. Hence, a problem remains open: 2 How to design single-modal attack to effectively subvert fusion models?
In response to and we propose novel attack framework against camera-LiDAR fusion models through the less significant camera modality. We utilize adversarial patches as the attack vector, aiming to cause false negative detection results, and our main focus lies on the early-fusion scheme, including data-level and feature-level fusion strategies. As shown in Figure [] our attack employs two-stage approach to generate an optimal adversarial patch for the target fusion model. In the first stage (2nd column), identify vulnerable regions in the image input using our novel sensitivity distribution recognition algorithm. The algorithm employs an optimizable mask to identify the sensitivity of different image areas under adversarial attacks. Based the identified vulnerable regions, we then classify the fusion model as either object-sensitive or globally sensitive, enabling tailored attack strategies for each type of model. In the second stage (3rd column), we design two attack strategies for different types of models to maximize attack performance. For globally sensitive models, we devise scene-oriented attacks, wherein adversarial patches can be placed on static background structures (c.g., roads or walls) to compromise the detection of arbitrary nearby objects (see undetected pedestrians the red circle of Figure [ For object-sensitive models, we implement object-oriented attacks that can compromise the detection of a target object by attaching the patch to (see the undetected vehicle in the red circle of Figure Compared to Cheng et al. 2022), the patches generated by our proposed framework offer significant advantage by being both physically deployable and effective (see comparison in Appendix(]] Our contributions are:
We present single-modal attacks against advanced camera-LiDAR fusion models leveraging only the camera modality. thereby further exposing the security issues of MSF-based AV perception.
We develop an algorithm for identifying the distribution vulnerable regions images, offering
comprehensive assessment of areas susceptible adversarial attacks.
We introduce framework for attacking fusion models with adversarial patches, which is a two- stage approach and involves different attack strategies based on the recognized sensitivity type of the target model. The threat model is detailed in Appendix
We evaluate our attack using six state-of-the-arr musion-based and one camera-only models on Nuscenes (Caesar et al. 2020), real-world dataset collected from industrial AV sensor arrays. Results show that our attack framework successfully compromises all models. Object- oriented attacks are effective all models, reducing the detection score of target object from 0.728 to 0.156 on average. Scene-oriented attacks are effective for two globally sensitive models, decreasing the mean average precision (mAP) of detection performance from 0.824 to 0.353. Experiments in simulation and physical-world also validate the practicality of our attacks in the real world. Demo video is available pntps:s:s /tt:::///ouuuu/m be/ xhXt zDezeaM
2 RELATED WORK
Camera-LiDAR Fusion. AVs are typically equipped with multiple surrounding cameras, provid- ing comprehensive view, and LiDAR sensors usually mounted centrally top of the vehicle, enabling a 360-degree scan of the surrounding environment, resulting in a 3D point cloud. Images and point clouds represent distinct modalities, and numerous prior works have investigated methods
2

---

Published as a conference paper at ICLR 2024
to effectively fuse them for improved object detection performance. Specifically, the fusion strate- gies can be categorized into three types based on the stage of fusion: 1) data-level fusion, which leverages the extracted features from modality to augment the input of the other modality (Yin) et al. 2021] Vora et al., 2020] Wang et al. 202T); : 2) decision-level fusion, which conducts inde- pendent perception for each modality and subsequently fuses the semantic outputs (BaiduApollo)
and 3) feature-level fusion, which combines low-level machine-learned features from each modal-
ity yield unified detection results (Liu et al. 2023b; Liang al.] 2023; Yang al.) 2022
et al. [022b; Bai et al., 2022 Chen et al. (002b). Feature-level fusion can be further divided into
alignment- and non- con-alignment-baatt
fusion. Alignment-bas based fusion entails aligning camera
and LiDAR features through dimension projection at the point level (Li et al.
2020; Vora et al.) 2020
Chen et al. 2022a), the voxel level (Li et al. 2022b)Jiaa et al. 2022), the proposal level (Chen et al.,
Ku Ku et al. 2018), or the bird's eye view (Liu et al. [023b] Liang et al.) (022) before concate-
nation. For non non-alignment-bed based fusion, cross- cooss-ateention mechanisms in the transformer architecture
are employed for combining different modalities (Yang et al., 2022 Bai et al. 2022). Contempo-
rary fusion models primarily use feature-level fusion for its superior feature extraction capability and performance. Hence, we focus on introducing and analyzing this type of fusion strategy in method design. It is worth noting that our approach can also be directly applied to data-level fu- sion, as demonstrated in our evaluation (see Section 55. More discussion of fusion strategies is in Appendix B Appendix C introduces the general architecture of camera-LiDAN fusion.
3D Object Detection Attacks. 3D object detection models (Cheng et al. 2022 Liu al. 2021a
Cui et al. 2021) can be classified into three categories: camera-based, based models. Attacks targeting each category have been proposed in the context of AV systems. 1) For camera-based models, adversaries typically employ adversarial textures to manipulate the pixels captured by AV cameras (Zhang et al., 202T; [Boloor al.) 2020). This approach is cost- effective and can be easily implemented through printing and pasting an adversarial patch. Recent studies have concentrated on enhancing the stealthiness of the adversarial patterns (Cheng et al.) 2022 Duan et al. 2020). 2) In the case of --DeRddased models, some attackers utilize auxiliary equipment, such as photodiodes and laser diodes, to intercept and relay the laser beams emitted by AV LiDAR systems, thereby generating malicious points in the acquired point cloud to launch the attack (Cao et al. 2019 2023) Sun et I 2020). Alternatively, others employ malicious physical ob- jects with engineered shapes introduce adversarial points in attacks (Tu et al.) 2020 Abdelfattah
and fusion-
ctt al. a Cao et al.) 2019). 3) Regarding camera-LiDAR fusion models, multi-modal attacks have been developed that perturb both camera and LiDAR input either separately (Tu et al.) 202T Abdelfattah et al., 2021b) or concurrently (Cao et al. 202I), using the previously mentioned at- tack vectors. Additionally,
angle-modal attacks on solely LiDAR input have been conducted in a black-box manner (Hallyburton et al. (022) to fool fusion models. For camera-oriented single modal attacks, there are prior works investigating the robustness of fusion models when subjected to noisy camera input (Park al. [021] Kim & Ghosh| 2019). However, Kim & Ghosh| (2019) mainly considered random noise, specifically Gaussian noise, instead of physical-wond adversarial attacks. Park et al. (2021) mainly focused on digital-space attacks and exclusively targeted an early model using single-view images. Differently, our study considers physically practical attacks and investigates fusion models utilizing multi-view images and transformer-batt based detection head.
3 MOTIVATION
Despite the challenges mentioned in Section D it is still theoretically possible to conduct camera-only attack against fusion models. The intuition behind is that adversarial effects from the camera modal- ity can propagate through model layers, contaminate the fused features, and ultimately impact the model output (See Appendix for detailed feasibility anal- ysis). To examine the actual performance of camera- only adversarial attacks on SOTA fusion models, we illustrate an example in Figure 2 A frame is derived from the Nuscenes dataset containing both camera and LiDAR data (the first row). It represents see- nario where the eeo-vehicle is navigating a road pop- ulated with multiple cars and pedestrians. In be-
Ground-truth
Front-camera Input
iddrr Input
Deeppintercction
Patch
BEVFusion PKU
petections
Figure 2: Motivating example of adversarial patch attack on images against fusion models.
3

---

Published as a conference paper at ICLR 2024
Stage Sensitivity Distribution Recognition
Stage Attack Strategies
@
Camora-Lida Fusson Model (
Global Sensitivtt::
Projction Image
Multiniew Images
Advenariial Images (r)
Pertorbation (p)
Scene- Attacks
Benign Lidar Input
Desection Results
Object Global Sonstriaty:
Perturbation Masks (M)
Project Taegan Object
Global Sensitmin::
Object Semoliivity
Masks
OR
Perturbation Masks (M)
Pecturbation Masks (14)
Object-Driente Amacks
Figure 3: Framework single-modal attacks against camera-LiDAA fusion model with adversarial patches.
nign cases, two cutting-edge fusion models, DeepInteraction (Yang et al., 2022) and BEVFusion- PKU (Liang et al., 2022), can accurately detect objects in the given scene. We then implement a conventional patch attack (Brown et al. 2017) by generating patch on the road induce false negative detections. The performance of DeepInteraction is undisturbed by the attack, illustrated is successfully disrupted, evidenced by its inability to detect objects proximal to the patch, highlighted by red circles in the third row. This discrepancy in the models' responses confirms that exploiting the camera modality can impact fusion models while highlights that uniform attack strategies may not be universally effective due to the inherent unique vulnerabilities in different models, such as varying susceptible regions. Despite the SOTA patch attack can be adapted to optimize the patch region over the scene, the generated patch is not deployable (see Appendix D. limiting its application.
in the second row of Figure 2 In contrast, BEVFusion-PKe
To characterize the susceptible regions, we introduce the concept of "sensitivity" as property of areas in input images. It measures the degree to which specific area of an image impacts adversarial goals. An area with high sensitivity means perturbations there have large influence and can achieve good attack performance. Hence, sensitive regions are more vulnerable to ad- versarial attacks than other regions. Formally, the sensitivity SA of area A is defined as SA x maxp Lodd (x, [) - Lodu (z), where x' = x o A) + p O A. Here, x is the in- put image, is the LiDAR point cloud and x' is the adversarial image with perturbations p in region A. Lade denotes the adversarial loss defined by adversarial goals. Examining the sensitivity of each area on the image through individual patch optimization is very time consuming and it be- comes unaffordable as the granularity of the considered unit area increases. Despite the availability of various interpretation methods for model decisions (e.g., GradCAM (Selvaraju et al., (017) and ScoreCAM (Wang et al.) (2020), which can generate heatmaps to highlight areas of attention in im- ages, it essential to distinguish between interpreting model decisions and recognizing sensitivity. For instance, our motivating example presents that the road is susceptible region for adversarial at- tacks on HEVFusion-PKUL However, the main focus of object detection should be directed towards the objects themselves rather than the road, 2022]. Therefore, to recognize the sensitivity distribution on input images efficiently, we propose novel optimization-based method in the first stage, and design different attack strategies in the second stage to maximize attack performance.
interpretation method would show (Gildenblat)
METHOD
Overview. Figure presents the framework of our single-modal adversarial attack on fusion models using an adversarial patch, employing two-stage approach. Initially, we identify the sensitivity distribution of the subject network, and subsequently, we launch an attack based on the identified sensitivity type. During the first stage, to recognize the sensitivity distribution, we define pertur- bations and perturbation masks with dimensions identical to the multi-view image input. We then compose the adversarial input by applying the patch and mask to images of a scene sampled from the dataset (step D). After feeding the adversarial input images and corresponding benign LiDAR data to the subject fusion model, we obtain object detection results (step 2 We calculate the adversarial loss based on the detection scores of objects in the input scene (step 3 and utilize back- propagation and gradient descent to update masks and perturbations, aiming to minimize adversarial loss and mask loss (step 4). We repeat this process for thousands of iterations until convergence is
4

---

Published as a conference paper at ICLR 2024
achieved, and then visualize the final mask as heatmap to determine the sensitivity type (step @). regions sigmify areas more susceptible to adversarial attacks. Based
The heatmap's 's on the distribution of sensitive areas, classify the fusion model into two types: global sensitivity and object sensitivity. Global sensitivity refers to the distribution of sensitive areas covering the entire scene, including objects and non-object background. Object sensitivity, on the other hand, indicates that only object areas are sensitive to attacks.
In the second stage, we adopt different attack strategies based on the identified sensitivity heatmap type. For global sensitivity, we implement scene-oriented attacks. By placing a patch on the static background (e.g., the road), we deceive the fusion model and compromise the detection of arbitrary objects surrounding the patch. For both object sensitivity and global sensitivity, we can employ object-oriented attacks. In this approach, we attach a patch to a target object, causing failure in de- tecting it while leaving the detection of other objects unaltered. Since adversarial patches, optimized as 2D images, would deployed physically during attacks, we employ projections (Cheng et al.) 2023) to simulate how the patch would look on the scene image once it is physically deployed (refer to Figure 4. which enhances the physical-wond world robustness. The two attack strategies differ mainly in their projection functions and the scope of affected objects. Details are discussed later.
Sensitivity Distribution Recognition. We leverage the gradients of images with respect to the adversarial loss as overall indicator to understand the sensitivity of different image areas, since they are closely related to the relative weights assigned to the camera modality. (See detailed analysis in Appendix [ In formal setting, the proposed sensitivity distribution recognition algorithm can be articulated as an optimization problem. The primary objective is to concurrently minimize an adversarial loss and mask loss, which can be mathematically represented follows:
arg min Lado + Nemask, s.t. E [0, 1j3xhx ,m E [=x[M/=]x(u/a]]
(1)
p.m
where Teado MSF (Jscores (d²,1),0; 0); I Tamask MSF(M,,))
(2)
= x M) + M: M[i.j] 2 x tanh( mll 1, 2
1
(3)
Here, x is the image input, which is normalized and characterized by dimensions (height) and w (width). The symbols I, p, m, and À represent the LiDAR input, the perturbations on image with dimensions equal to x, the initial mask parameters, and the mask loss weight hyperparameter, re- spectively. The desired sensitivity heatmap corresponds to the perturbation mask M. Visualization of variables can be found in Figure 3 Initially, the mask parameters m E R [h/=]x[w//s are trans- formed into the perturbation mask M E [0, 1] 1]1xhx= using Equation We use tanh () function to map values in into the [0, 1] range, and its long-tail effect encourages the mask M values to gravitate towards either 0 or 1. The hyperparameters y and s modulate the convergence speed and heatmap granularity, respectively. Subsequently, the perturbation mask M is utilized to apply the perturbation p to the input image x, resulting in the adversarial image x'. C denotes element-wise multiplication. Adversarial image x' and benign LiDAR data / are then feed to the fusion model facores- Since our attack goals are inducing false negative detection results, one objective of our optimization is to minimize the detected object scores. Hence, we use the mean square error (MSE) between the scores and zero as the adversarial loss Tadde- In this context, the output of Sscores consists of the detected object scores (confidence). The optimization's secondary objective is to minimize the perturbation mask values, achieved by incorporating a mask loss Lmaak.
The optimization of these two losses is a dual process. Minimizing the adversarial loss (i.e., max- imizing attack performance) necessitates a higher magnitude of perturbations on the input. Con- versely, minimizing the mask loss indicates a lower magnitude of perturbations. As result, the dual optimization process converges on applying higher magnitude perturbations on sensitive areas (to improve attack performance) and lower magnitudes for insensitive parts (to minimize mask loss). Hence, the mask M serves as good representation of the sensitivity distribution, and visualizing M allows for the attainment of the sensitivity heatmap. Then we can further classify the fusion model into object sensitivity global sensitivity by comparing the expectation of the average intensity of object areas with non-object background in each scene as follows:
Object, E,r [E oA")
> SE "EM""((AAS))
T(
Σ Ag
(1-
Global,
otherwise
Here, T(f) represents the sensitivity type of fusion model f. and A is a mask with values 1 for ob- ject areas and 0 for non-object areas in scene image x. The mask A5 is generated by considering the
5

---

Published as a conference paper at ICLR 2024
W
ZD Parch
Patch World Patch
2D Parch
Paich on Captaned Image
3D World
Captured
Image
(a) Scene-oriented Attacks.
(b) Object-oriented Attacks.
Figure 4: Projections in different attack strategics.
pixels covered by bounding boxes of detected objects benign cases. M refers to the recognized sensitivity heatmap of x. B3 is the classification threshold and set to 3 in experiments.
Attack Strategies. Two attack strategies. namely scene-oriented attacks and object-oriented attacks, introduced based on the fusion model's sensitivity type. Both strategies employ optimization- based patch generation methods. Back-propagation and gradient descent are utilized iteratively to solve the optimization problem. Formally, the problem is defined as:
arg min E(g,1)-D [SE((.(2)) .)], s.t. p E [0, 1] 3x3xxx ,M E (0, 1xxxx,
(5)
where ='=O M₂) + +2 Mc; M, pr)j P. proj.(p).
(6)
Here, scene images and LiDAR data / are randomly sampled from the training set D. The mask M represents patch area for cropping the patch image, with values equal to inside a predefined patch area and 0 elsewhere. Unlike Equation [ M contains discrete values and is not optimizable. projz () sigmifies the projection of the original patch image (see the "2D patch image" in Figure 4ah onto specific area of the scene image x (see the "captured image" in Figure 4a simulate how the patch would look once it's physically deployed, which minimizes the disparity between digital space and the physical world. The target region contingent upon the attack strategy. Similarly, the output of the fusion model fa consists of detected object scores, which vary in scope depending on spe- cific attack strategies. We minimize the MSE between detected object score(s) and zero achieve false negative detection results, and we leverage the Expectation of Transformation (EoT) (Athalye al., 2018) across all training samples and color jitters (i.e., brightness, contrast and saturation changes) in the patch to enhance the physical robustness and generality of our attack. The adversar- ial pattern can be concealed within natural textures (e-g., dirt or rust), utilizing existing camouflage techniques DDan et al. 2020) to remain stealthy and persistent, avoiding removal.
Specifically, for scene-oriented attacks, the goal is to compromise the detection of arbitrary objects near adversarial patch attached to static structures (e.g., the road) of target scene. In this sce- nario, the training set D is composed of the target scene in which the ego-vehicle stationary (e.g., stop at an intersection vehicle in the scene can change dynamically. The output of the fusion model fa during optimization the detection score of all detected objects in the target scene. To simulate the appearance of the patch on the scene image once deployed, projæ first projects pixels of the patch image and mask into 3D space on the road (step 1 in Figure 4a). Then the function maps them back onto the scene image (step (2). The patch's 's 3D position is predefined with distance d and viewing angle a by the attacker. The victim vehicle's camera height g above ground can be known from the dataset, which ensures by definition that the patch is on the road. This process can be expressed formally with Equation 2 where (w", ) is a pixel's coordinates on the patch image p. "e".,""""" the 3D coordinates of the pixel on the ground in the camera's 's coordinate system, (u", v*) the corresponding pixel on scene image, K the camera intrinsic parameters. Other variables are defined in Figure 4
parking lot). The categories and locations of objects surrounding the ego-
coo 0
a
W/2
T
y
0
0
0
uP
(7)
zP
sin a 0
cOs a 0
H/h
H/2
0
1
0
0
TP
C08
sin a
[W/w
0
-
-W/27
T
y
0
H/h
-H/2
yP
(8)
2PP
sin
COS a
d
0
0
0
0
0
Ju" "
1/2" P. K [xP
zP
(9)

---

Published as a conference paper at ICLR 2024
Socne
Sceme
Scone
Scene
Augmaning
Scene
Insage
BENFamerr BiyF BEVFusion Prine
Tran-
Fussn
-PKL MMIT
Deep Insernction
UVTR
Figure 5: Sensitivity heatmaps six camera-L.ib LiDAR fusion models and a camera-only model on two scenes.
For object-oriented attacks, the goal is to compromise the detection of the target object with an attached adversarial patch while keeping other objects unaffected. In this case, the training set D is composed of frames in which the target object appears. For example, the ego-vehicle may follow target vehiele with the background changes dynamically in the scene. The output of the fusion model fa during optimization is the detection score of the target object exclusively. The function proze projects the patch image and mask onto the target object in the scene image using Equation and corresponding to step ① and 2 in which the location of the patch defined by attackers using longitudinal distance d, lateral distances and viewing angle a, in cejjcct-rieeted attacks, these projection parameters change dynamically depending on the position of the target object in training data. Hence, we innovatively extract them from the estimated 3D bounding box of the target object before projecting the patch.
respectivell. Unlike the acene-oriented attack, in
EVALUATION
six state-of-the-an camera-LiDAR fusion-based 3D ob- ject detection models that are published recently. These models include Transfusion (Bai et al. DeepInteraction (Yang al.) 2022). UVTR eti al. 2022b), PointAugmenting (Wang al.
Model selection In evaluation,
2022
1 BEVFusion-MITT (Liu et al. 20235 and BEVFusion-P PKU (Liang et al. (002)). These models cover data-level and feature-level fusion strategies and contain a diverse range feature-level fusion approaches, including alignment-basee head designs. Additionally, we use camera-only model called BEVFormer (Li al. 2022c) as
fusion, con-alignment-beee based fusion, and various detection
comparisor Detailed selection criteria can be found in Appendix
Scene selection. Our evaluation scenes are selected from the Nuscenes dataset (Caesar al. 2020
This dataset contains real-world multi-view images and point cloud data collected from industrial- grade sensor array, and they are derived from hundreds driving clips. The selected scenes for test- ing in our evaluation contains 375 data frames, encompass diverse road types, surrounding objects and of-e-of-day situations. Additionally, we conduct experiments in simulation and in the physical world. By leveraging this rich dataset along with simulation and thysical-world experiments, evaluation framework benefits from an accurate representation of real-world driving scenarios.
5.1 SENSITIVITY DISTRIBUTION RECOGNITION
This section reports on the evaluation of our sensitivity distribution recognition method. We present the qualitative results of the sensitivity heatmap generated by our method and validate the property of the heatmap Appendix G We utilize Equation [ to generate the sensitivity heatmap for the six fusion models, using two different data frames, each with varying proportions of vehicles and pedestrians. Detailed experimental setups can be found in Appendix HH] [igureB]depice the gener- ated sensitivity heatmaps. The first two images are the scene images captured by the front camera of the ego vehicle while the subsequent rows exhibit the sensitivity distributions of the corresponding scene image using different models. The brightness warmth of colors in the heatmap corresponds to the sensitivity of a particular region to adversarial attacks. Higher brightness areas signify higher susceptibility attacks, while lower brightness denotes more robustness. Observe that the sensitive regions for the initial four models, namely Transfusion (Bai et al. 2022). DeepInteraction (Yang
al. 2022). UVTR (Li et al. 2022b) and PointAugmenting (Wang al. 2021 primarily lie on
'The code is available https://githuu: hwtp://ittuuuccooooooeoeooe wttp:////oooobcceee/c/oo-ii---e. onAttack

---

Published as a conference paper at ICLR 2024
Table 1: Attack performance of the scene-oriented Table 2: Attack performance of the object-oriented adversarial patch attack against 3D object detection. adversarial patch attack.
Modils
MAP
CR
TK
BS
TR
PI
Bi
Targeted object
Other objects
Bes. 0.824
0.453
0,445
1.000
0.991
0,895
0.990
0.989
Models
Ben. Scure
Adv. Score
Diff.
mAP
Ben.
mAP
Adv.
Diff
RF-PKU
Adv. 0.353
0.524
0.611
0.242
0.604
Duft 57.2%
70.0%
74.1%
47.6%
75.9%
32.0%
75.6%
38.9%
TF
0.655
0,070
89.24%
0.921
0.923
0.30%
Res. 0,586
0,858
0.895
0,989
0.900
DI
0.658
0.110
83.32%
0.964
0.965
0.13%
DF MIT
Adv. 0.553
0,623
0.337
0.772
UVTR
0.894
0.189
78.83%
0.963
0.963
0.00%
57.6%
48.1%
30.6%
16.1%
50.8%
65.9%
22.0%
PointAug
0.734
0.177
75.89%
0.954
0.955
0.10%
Ben. 0.759
0,700
0.991
0.692
0.989
0.990
BF-MIT
0.714
0.219
69.37%
0.965
0.968
0.34%
TF
Adv.
0,404
0,452
0.706
0.989
0.589
BF.PKU
0.712
0.168
76.38%
0.956
0.958
Average
0.728
0.156
78.63%
0.951
0.955
0.17%
DI
Bes. 0.807
Adv.
0.459
0.522
0.947
01550
0.990
0.751
0.750
0.989
0.989
0.989
0.989
BFM
0.955
0.095
90.02%
0.578
0.571
1.08%
TE.
TransFusion (Bra at al. 2022 DI: Depplttencioon Yang 2s al,
0.1%
X02
UVTR (La el ai. 20zz0 FoinAug: PoimAnggeennng Wang
Den.
0.754
0,590
0.736
0.982
0.989
aL.
3021
PI-P AgVFasson PKU (Liang ot al 2002
LVIR
Adv.
0.989
0.786
0.590
0.741
0.982
0.989
MIL
REV Pisioo-MIT [Lin al. 202th
BFM BEVForer La al.
1.4%
Table 3: Physical-world attack performance.
Bes. 0.724
0.406
0.683
01592
0.714
0,984
0.981
PoinAng
Adv. 0.716
00,667
0.468
0.679
0.988
0.705
0,984
0.981
Podestrian ID Original
Benign
Adverariil
Difference
Din 1.1%
0.8%
01.4%
1.38
0.0%
0.685
0.194
72.01%
Ros. 0.519
01417
0.811
0.280
01.44
0.712
0.650
0.518
2
0.674
0.642
0.219
65.89%
RFM
Adv. 0.514
0,432
0,799
0.284
0.247
0.711
0.605
0.518
3
0,659
0,681
0,237
65.20%
Ddft 1.1%
3,6%
1.5%
1.4%
0.0%
0,115
6.9%
0.0%
Average
0.672
0.217
67.76%
CR: Car, TK: Truck, BS: Bus, TR: Traler, BR: Barner PD: Pedestrian. Bl: Bicycle.
areas of objects like vehicles and pedestrians. This suggests that attacks on objects could prove to be more effective, whereas non-object areas such as the road and walls are more resistant. The following two models (i.e., BEVFusion-MIT" (Liu et al. 2023b) and BEVFusion-PKUU (Liang al.) )022l) demonstrate high sensitivities throughout the entire scene, irrespective of objects or back- ground regions. This indicates their vulnerability at global level. Our technique also works on camera-only models. As shown in the last row, the camera-only model (i.e., BEVFormer (Li et al.) 2022ch) demonstrates higher sensitivity in the object area and is also classified as object sensitivity according to Equation[4] Since different sensitivity types demonstrate distinct vulnerability patterns, diseuss the reason behind in our defense discussion (Appendix N
5.2 SCENE- ATTACKS
Scene-oriented attacks are primarily aimed at fusion models with global sensitivity. Such models are vulnerable to adversarial patches placed on non-object background structures (c.g., the road). Our attack is universal, which can affect the detection of arbitrary dynamic objects in a given scene, even those that were not initially present during patch generation (training). Therefore, our attack is more practical in real-world scenarios attackers can effortlessly paste generated patches onto the ground, rendering victim vehicles in close proximity blind. This could pose a great risk to pedestrians and surrounding vehicles. Detailed experimental setups can be found in Appendix H
Table D presents the quantitative results of our evaluation and qualitative examples can be found in Appendix [ In Table [] the first column shows various models, the third column presents the mAP of object detection results in the test set, and the subsequent columns denote the average precision (AP) of different objects categories. We report the subject model's 's benign performance (no patch), adversarial performance (patch applied) and their difference in percentage (attack performance) for each model. Our findings indicate that the detection accuracy of the two globally sensitive models (i.e., BEVFusion-PKU and BEVFusion-Min
has considerably decreased, for all object categories. The mAP decreased more than 35% However, the other five models with object sensitivity remain unaffected. These results align with our conclusion in Section and further reveal the vulnerability of globally sensitive models to more influential scene-orienter attacks. Additionally, our experiment confirms the robustness of abject-sensitive models under attacks in non-object background areas. In comparison, the camera-based model (i.e., BEVFormer) demonstrates worse benign performance than all fusion-based models, but is also robust to scene-oriented attacks due its object-sensitive nature. Demo video is available at httt:::///ouuu.b////ztttc/// be
5.3 OBJECT-ORIENTED ATTACKS
Object-oriented attacks target -bject-sensitive models that are more robust to attacks in non-object background areas. The influence of this attack is more localized, opposed the scene-oriented
8

---

Published as a conference paper at ICLR 2024
Tape Measure
Comera Tripod
Patch
(a) Equipment
(b) Dataset image
(c) Benign physical-world image (d) Adversarial physical world image
Figure 6: Attacks in the physical world.
attacks. It concentrates the impact on specific target object, leaving the detection of other objects unaltered. This approach offers a higher degree of customization for attackers, enabling them to manipulate the impact at the object level rather than the entire scene. Detailed experimental setups can be found in Appendix[H] Our evaluation results are presented in Table qualitative examples are Appendix [] As shown, the first column represents various fusion models and a camera-only model for comparison, the second to fourth columns display the average detection score of the target object, and the fifth to seventh columns indicate the mAP of other objects (including car, bus, pedestrian and motoreycle). The results demonstrate substantial decrease in the target object's detection scores for fusion models, from 0.728 to 0.6 on average, thus validating the efficacy of our object-oriented adversarial attacks across all models regardless of the fusion strategies. Furthermore, the detection results of other objects in the scene remain virtually unaffected, as evidenced by the negligible change in mAP. This phenomenon also holds for the camera-only model, and shows worse benign mAP and more performance degradation under attack. Videos of the attack can be found at
5.4 PRACTICALITY
To assess the practicality of our single-modal attacks on fusion models, we conducted experiments in both simulated and physical-world environments. Attacks in simulation can be found in Ap- pendix K We assess the feasibility of our attack in a real-world setting by replacing the front-view images of 30 data frames in the dataset with our custom images taken in the physical world, leaving other views and LiDAR data unchanged. To ensure the compatibility of the dataset's LiDAR data with our custom images, we maintain the 3D geometry of our physical scenario consistent with the original dataset images. Figure oop and Figure[i: illustrate an original image and a custom scenario in our experiment respectively. Note that both images maintain similar 3D geometry, with pedestri- ans crossing the road located at similar positions in both cases. Detailed setups are in Appendix H Figure (6]a exhibits the experimental equipment, while Table 3 details the attack performance. The pedestrian ID,, corresponding to the pedestrians in Figure is denoted in the first column of Table with the subsequent columns reporting the average detection scores in original dataset images (Fig- ure (p). our benign physical-world images (Figure (b), and the adversarial physical-worde images (Figure (t.). The last column denotes the difference between benign and adversarial physical-world performance. The comparative detection scores in our benign physical-worid images and the origi- nal dataset images validate the consistency between the original LiDAR data and our custom images, thereby substantiating the efficacy of our image replacement method. Furthermore, the deployment of the adversarial patch results in a significant reduction in the pedestrian detection scores, empha- sizing the practicality and effectiveness of our attack strategy in the physical world. We discuss the implications on AV security in Appendix
Ablation Studies and Defense Discussions. We conducted ablation studies about the attack per- formance of various distance and viewing angles of the adversarial patch (Appendix D). and various granularity of the sensitivity heatmap (Appendix M.. We discussed both architectural-leve defense and DNN-level defense strategies in Appendix and the limitations are discussed in Appendix(O)
CONCLUSION
We leverage the affordable adversarial patch to attack the less significant camera modality in 3D ob- ject detection. The proposed optimization-bant based two-stage attack framework can provide a compre- hensive assessment of image areas susceptible to adversarial attacks through a sensitivity heatmap, and can successfully attack six state-of-the-ate models real- world dataset with customized attack strategies. Results show that the adversarial patch generated by our attack can effectively decrease the mAP of detection performance from 0.824 to 0.353 or reduce the detection score of a target object from 0.728 0. 0.156 on average.
Lamera-LiDAN
iusion-based and one camera-only

---

Published as a conference paper at ICLR 2024
7 ETHICS STATEMENT
Most of our experiments are conducted in the digital space or a simulated environment. Our physical-world study involving human subjects underwent thorough scrutiny and approval by institutional IRB. Notably, we conducted physical experiments in a controlled environment on a closed road, utilizing a camera and tripod capture scenes instead of employing real cars, as eluci- dated in Appendix H This deliberate choice minimizes potential threats to the safety of participants. Stringent protocols were implemented, including participants not facing the camera, wearing masks during experiments, and blurring their faces in the photos. No identifiable information from the volunteers retained by the researchers.
ACKNOWLEDGEMENTS
This research supported, in part by TARPA TrojAl W911NF-19-S-0012, NSF 2242243, 1901242 and 1910300, ONR N000141712045, N00014-1410468 and N000141712947, National Research Foundation of Korea(NRF) grant funded by the Korean government (MSIT) RS-2023- 00209836.
REFERENCES
Mazen Abdelfattah, Kaiwen Yuan, Z Jane Wang, and Rabab Ward. Adversarial attacks on camera- lidar models for 3d car detection. In IROS, 2021a.
Mazen Abdelfattah, Kaiwen Yuan, 7. Jane Wang, and Rabab Ward. Towards universal physical attacks on cascaded camera-lidar 3d object detection models. In ICIP, 2021b.
AlDay. Tesla Autopilot Uses Transformer, 2022. 621
Rhett Allain. What Is the Angular Field of View for iPhone 13?, 2022. https://rjall:
in. www.mmdium...mm com/ what is the angular- field- of-view- for an--an--- iphone
1-199999882331
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In ICML, 2018.
Autoware. Autoware. htttp:://wwe wwwp://wwwwwwwww..rooooo
Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. Transfusion: Robust lidar-camera fusion for 3d object detection with transformers. In CVPR, 2022.
BaiduApollo. Baidu /nte htmml
https::/apollo..ee
Adith Boloor, Karthik Garimella, Xin He, Christopher Gill, Yevgeniy Vorobeychik, and Xuan Zhang. Attacking vision-based perception in end-to-end autonomous driving models. Journal of Systems Architecture, 2020.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint arXiv: 1711v:171..09665. 2017.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In CVPR, 2020.
Yulong Cao, Chaowei Xiao, Benjamin Cyr. Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z. Morley Mao. Adversarial sensor attack on lidar-based perception in autonomous driving. In CCS, 2019.
Yulong Cao, Ningfei Wang. Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li. Invisible for both camera and lidar: Security of multi-senson fusion based perception in autonomous driving under physical-wond world attacks. In S&P. 2021.
10

---

Published as a conference paper at ICLR 2024
Yulong Cao, S. Hrushikesh Bhupathiraju, Pirouz Naghavi, Takeshi Sugawara, Z. Morley Mao, and Sara Rampazzi. You can't see me: Physical removal attacks on lidar-based autonomous vehicles driving frameworks. In USENIX Security, 2023.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, In ECCV, 2020.
Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In CVPR, 2017.
Xuanyao Chen, Tianyuan Zhang, Yue Wang, Yilun Wang, and Hang Zhao. Futr3d: unified sensor fusion framework for 3d detection. arXiv preprint (rXi:2203.10642 2022a.
Zehui Chen, Zhenyu Li. Shiquan Zhang, Liangji Fang, Qinhong Jiang, and Feng Zhao. Autoalignv2: Deformable feature aggregation for dynamic multi-modal 3d object detection. arXiv preprint arXiv:2207. 103 16, 2022b.
Zhiyuan Cheng, James Liang Hongjun Choi, Guanhong Tao, Zhiwen Cao, Dongfang Liu, and Xiangyu Zhang. Physical attack on monocular depth estimation with optimal adversarial patches. In ECCV, 2022.
Zhiyuan Cheng, James Chenhao Liang, Guanhong Tao, Dongfang Liu, and Xiangyu Zhang. Adver- sarial training of sule-supe monocular depth estimation against physical-wond attacks. In ICLR, 2023.
Yiming Cui, Liqi Yan, Zhiwen Cao, and Dongfang Liu. Ti-blender: Temporal feature blender for video object detection. ICCV, 2021.
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, 2017.
Ranjie Duan, Xingjun Ma, Yisen Wang, James Bailey, A Kai Qin, and Yun Yang. Adversarial camouflage: Hiding physical- world attacks with natural styles. In CVPR, 2020.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
Jacob Gildenblat. FigenCAM for YOLO5, 2022. https://jacobgi wtw.:/oobblg.......b.o.... io/pttrrrr///orrrroo html
ch Ttt.ga/ccaa.bboo///eeeeee0200000
R Spencer Hallyburton, Yupei Liu, Yulong Cao, Z Morley Mao, and Miroslav Pajic. Security analy-
sis of [Camera-LiDAR) fusion against [Black-Box] attacks on autonomous vehicles. In USENIX Security, 2022.
Zhongyuan Hau, Kenneth T Co, Soteris Demetriou, and Emil C Lupu. Object removal attacks lidar-based 3d object detectors. arXiv preprint arXiv:2/02.03722 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Lifeng Huang, Chengying Gao, Yuyin Zhou, Cihang Xie, Alan Yuille, Changqing Zou, and Ning Liu. Universal physical camoutlage attacks on object detectors. In CVPR, 2020.
Yang Jiao, Zequn Jie, Shaoxiang Chen, Jingjing Chen, Xiaolin Wei, Lin Ma, and Yu-Gang Jiang. Msmdfusion: Fusing lidar and camera at multiple scales with multi-depth seeds for 3d object detection. arXiv preprint arXiv:2209.03 000020 2022.
Zizhi Jin, Xiaoyu Ji, Yushi Cheng, Bo Yang, Chen Yan, and Wenyuan Xu. Pla-lidar: Physical laser attacks against lidar-based 3d object detection in autonomous vehicle. In S&P, pp. 1822-1839. IEEE, 2023.
Taewan Kim and Joydeep Ghosh. On single source robustness in deep fusion models. NeurIPS, 32, 2019.
11

---

Published as a conference paper at ICLR 2024
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, and Steven L Waslander. Joint 3d proposal generation and object detection from view aggregation. In IROS, 2018.
Fred Lambert. Hacker shows what Tesla Full Self- vision depth perception neural net can full -slff-ri
see, 2021. . htte:///eeeooooo/aar..//////////
Hongyang Li, Chonghao Sima, Jifeng Dai, Wenhai Wang, Lewei Lu, Huijie Wang, Enze Xie, Zhiqi Li, Hanming Deng, Hao Tian, Xizhou Zhu. Li Chen, Yulu Gao. Xiangwei Geng, Jia Zeng. Yang Li, Jiazhi Yang. Xiaosong Jia, Bohan Yu, Yu Qiao, Dahua Lin, Si Liu, Junchi Yan, Jianping Shi, and Ping Luo. Delving into the devils of s-eye-vieww perception: A review, evaluation and recipe. arXiv preprint arXiv:2209.05324 2022a.
Ruihui Li, Xianzhi Li, Pheng-Ann Heng, and Chi-Wing Fu. Pointaugment: an auto-augmentation framework for point cloud classification. In CVPR, 2020.
Yanwei Li, Yilun Chen, Xiaojuan Qi. Zeming Li, Jian Sun, and Jiaya Jia. Unifying voxel-based representation with transformer for 3d object detection. In NeurIPS, 2022b.
Yingyan Li, Lue Fan, Yang Liu, Zehao Huang, Yuntao Chen, Naiyan Wang, Zhaoxiang Zhang, and Tieniu Tan. Fully sparse fusion for 3d object detection. arXiv preprint arXiv:2304. 12310, 2023.
Zhiqi Li. Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's see's-eee-view representation from multi-camera images via spa- tiotemporal transformers. ECCV, 2022c.
James Chenhao Liang, Yiming Cui, Qifan Wang, Tong Geng, Wenguan Wang, and Dongfang Liu. Clusterfomer: Clustering as a universal visual learner. In NeurlPS, 2023.
Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang. Tao Tang, Bing Wang, and Zhi Tang. BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frame- work. NeurIPS, 2022.
Dongfang Liu, Yiming Cui, Wenbo Tan, and Yingjie Chen. Sg-net: Spatial granularity network for one-stage video instance segmentation. CVPR, 2021a.
Han Liu, Yuhao Wu, Zhiyuan Yu, Yevgeniy Vorobeychik, and Ning Zhang. Slowlidar: Increasing the latency of based detection using adversarial examples. In CVPR, 2023a.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021b.
Zhijian Liu, Haotian Tang, Alexander Amini, Xingyu Yang, Huizi Mao, Daniela Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird's s-eye view representation. In ICRA, 2023b.
Momenta. Momenta Autodrive. http:://wwww.mmmet.aaa www.:mmmmetaaacceee....
Motional. Nuscenes Object Detection Leaderboard, 2023. hutpp://www.uusccessoorrg &maaDataaaaaaa..mm
- deettddteccioon innettectioneeetemreeee ottpr//eeeerrrtaaaaaeaaaaaaamaaa htt.apaataaaallllmlmllllleeeaaaa
Won Park, Nan Liu, Qi Alfred Chen, and Morley Mao. Sensor adversarial traits: Analyzing
robustness of 3d object detection sensor fusion models. In ICIP, 2021.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.
Charles Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas Guibas. Frustum pointnets for 3d object detection from rgb-d data. In CVPR, 2018
12

---

Published as a conference paper at ICLR 2024
Raul Quinonez, Sleiman Safaoui, Tyler Summers, Bhavani Thuraisingham, and Alvaro A Cardenas. Shared reality: detecting stealthy attacks against autonomous vehicles. In Proceedings of the 2th Workshop on CPS&IoT Security and Privacy, 2021.
Takami Sato, Junjie Shen, Ningfei Wang. Yunhan Jia, Xue Lin, and Qi Alfred Chen. Dirty road can attack: Security of deep learning based automated lane centering under physical-worke attack. In USENIX Security 21, 2021.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In ICCV, 2017.
Jiachen Sun Sun, Yulong Cao Cao, Qi Alfred Chen, and Z Morley Mao. Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermea- sures. In USENIX Security, 2020.
James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. Physically realizable adversarial examples for lidar object detection. In CVPR, 2020.
James Tu, Huichen Li, Xinchen Yan, Mengye Ren, Yun Chen, Ming Liang, Eilyan Bitar, Ersin Yumer, and Raquel Urtasun. Exploring adversarial robustness of multi-sensor perception systems in self driving. arXiv preprint
2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurlPS, 2017.
Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. Pointpainting: Sequential fusion for 3d object detection. In CVPR, 2020.
Chunwei Wang, Chao Ma, Ming Zhu, and Xiaokang Yang. Pointaugmenting: Cross-modal aug- mentation for 3d object detection. In CVPR, 2021.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Seirh-we visual explanations for convolutional neural networks. In CVPR Workshop, 2020.
Waymo. Waymo Tech. hctps://waymo.. com.tcc.ccccccoooeo
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In ICCV, 2017.
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 2018.
Zeyu Yang, Jiaqi Chen, Zhenwei Miao, Wei Li. Xiatian Zhu, and Li Zhang. Deepinteraction: 3d object detection via modality interaction. In NeurlPS, 2022.
Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Multimodal virtual point 3d detection. NeurIPS, 2021.
Junko Yoshida. Teardown: Tesla's Hardware Retrofits for Model 3, 2020. https::/www. 5 sia.
rew.dwawr-eettttffeeeee for-
Jindi Zhang, Yang Lou, Jianping Wang, Kui Wu, Kejie Lu, and Xiaohua Jia. Evaluating adversarial attacks on driving safety in dision-basee autonomous vehicles. IEEE Internet of Things Journal, 2021.
Qingzhao Zhang, Shengtuo Hu, Jiachen Sun, Qi Alfred Chen, and Z Morley Mao. On adversarial robustness of trajectory prediction for autonomous vehicles. In CVPR, 2022.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR, 2018
Zijian Zhu, Yichi Zhang, Hai Chen, Yinpeng Dong, Shu Zhao. Wenbo Ding, Jiachen Zhong, and Shibao Zheng. Understanding the robustness of 3d object detection with bird's view repre- sentations in autonomous driving. In CVPR, 2023.
13

---

Published as a conference paper at ICLR 2024
Appendix
Fusion is not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection
This document provides more details about our work and additional experimental settings and result. We organize the content of our appendix as follows:
Section
Evaluation varying significance of different modalities in fusion.
Section
Discussion of other fusion strategies.
Section
General architecture of Camera-LiDAR Fusion.
Section D
Feasibility analysis of single-modal attacks against fusion models.
Section
Gradient-based analysis of sensitivity distribution recognition.
Section
Model selection criteria.
Section
Property validation of sensitivity heatmap.
Section
Detailed experimental setups.
Section 1
Qualitative results of attacks.
Section
Comparison with patch attack against camera-only models.
Section
Additional practicality validation.
Section
Varying distance and viewing angles.
Section
M
Varying granularity of sensitivity heatmap.
Section
Defense discussion.
Section
Limitations.
Section
Threat model for our attacks.
Section
Discussion on autonomous vehicle security.
A VARYING SIGNIFICANCE OF DIFFERENT MODALITIES IN FUSION
LiDAR and camera sensors contribute uniquely to 3D object detection, offering depth and texture information, respectively. These modalities are integrated into fusion models, where their relative importance may depend on their individual contributions to the final output. LiDAR is often con- sidered more criticul for 3D object defection due to ils rich 3D information. Evidence supporting this can be found in Table 4 which presents the reported performance of five state-of-the-as fu- sion networks, each trained with different modalities. The -DDeR-aased models have performance close-to their fusion-based counterparts, outperforming the camera-based models significantly. Fur- ther supporting this point, Figure 7 demonstrates that the performance of fusion-based models is considerably more affected by reduction of information in LiDAR input than in camera input, sup- porting the greater contribution of the LiDAR modality in detection performance. In this controlled experiment, we reduce the input information by introducing random noise. For the camera, we ran- domize values of uniformly sampled pixels input images, and for the LiDAR, we randomize the
0.6
Camera
LIAAR
0.4
0.2
0.0
100
60
20
10
0
100
80
20
10
5
100
60
20
10
5
100
60
20
10
100
60
20
10
D
Ramaining Ratio (%)
Remaining Ratio (96)
Ramaining Ratio (%)
Remainina Ratio (%)
Remaning Rañio (%)
(a) BEVFusion-MMI
(b) BEVFusion-H PKU
(c) Transfusion
(d) DeepInteraction
(c) UVTR
Figure 7: Performance degradation of fusion-based models when camera and LiDAR information reduced
respectively.
14

---

Published as a conference paper at ICLR 2024
positions of uniformly sampled points in the input point cloud. The detection performance is eval- uated on the Nuscenes dataset (mini) (Caesar et al., 2020). Only one modality is altered at a time. The results of these investigations clearly demonstrate that the camera modality's 's contribution to the fusion process is of lesser significance, which implies that mounting an attack via a less influential modality, such the camera, may prove to be challenging endeavor.
Table 4: Performance comparison of single modal models and fusion models. Metries; mAP.
BEVFusion-MII MIT
BEVFusion-FF PKU
Transnusion
DeepInteraction Yang al. 2022
UVTR
LLuu et al. 2023b
(Liang et al. 2022
(Bai et al. 2022
LLi et al.
2022b
C
0.333
0.227
0.362
L
0.576
0.649
0.655
0.692
0.609
CL
0.664
0.679
0.689
0.699
0.654
* C: Camera, L: LiDAR, CL: Camera-LiDAR Fusion,
B DISCUSSION OF OTHER FUSION STRATEGIES
For the data-level fusion strategy, some work (in al., 2021) projects the pixels images into 3D space to create virtual points and integrate with LiDAR points, then uses point cloud-based models on the integrated points for 3D object detection. Perturbations the camera modality would directly affect the projected virtual points (Cheng et al. (002). Since point cloud-based models are
vulnerable to point- manipulation attacks (Cheng et al. 2022 Cao al. 2019 Sun al. 2020 Jin
eet al. 2023 Liu et al. 2023a; Hau et al. 2021)) it is easy to compromise this kind of fusion models through the camera modality as the virtual points projected from images are the weakest link for such models. Other works (Wang et al. [022;; Vora et al.) 2020; Qi et al. 2018: Li et al. 2223) use the extracted image features to augment the point data. For example, Wang et al. (2021) utilizes image features extracted by CNN to enhance LiDAR points. In Qi et al. (2018) and Li et al. (2023), authors employ CNN-based 2D object detection or semantic segmentation to identify object regions in the image input, then use the extruded 3D viewing frustum to extract corresponding points from the LiDAR point cloud for bounding box regression. Adversarial attacks on images can deceive the object detection or semantic segmentation result in Huang al. (2020); Xie et al. (2017). leading to the failure of the extruded 3D frustum in capturing the corresponding LiDAR points of
each object. This would subsequently compromise the 3D bounding box regression. Our evaluation
with Wang et al. (2021) has shown that data-level fusion is also vulnerable to adversarial attacks on the camera modality. Hence, our attack techniques effective for both data-level fusion and feature-level fusion strategies.
The decision-level fusion is less adopted in advanced fusion models. fuses the prediction re- sults from two separate single-modal models in a later stage. Our camera-oriented attacks will only affect the result of the camera- based model, and the fusion stage can be completed in track- ing (BaiduApollo). The attack performance heavily depends on the relative weights assigned to the two modalities. This dependency LiDAR-only attacks (Hallyburton et al. (022) have succeeded on Baidu Apollo (BaiduApollo), it can be attributed to the current higher weighting of LiDAR results in Apollo. Some anomaly detec- tion algorithms like Quinonez et al. (2021) may leverage the inconsistency between two modalities' output to alert attacks, but which one to trust and operate with remains uncertain. Developing robust fusion models to defend all kinds of single-mon modal attacks is an open research question and we leave it as our future work.
common limitation for single-modal attacks. Although prior
C GENERAL ARCHITECTURE OF CAMERA-LIDAR FUSION
Figure S illustrates the general architecture of cutting-edge Lamera-LiDAR fusion models using feature-level fusion. From left to right, multi-view images obtained from cameras and the point cloud data from LiDAR sensors are initially processed independently by neural networks to extract image and LiDAR features. The networks responsible for feature extraction are commonly referred
to as "backbones". which encompass ResNet50 (He et al. 2016). ResNet101 (He et al., 2016). SwinTransforrer (Liu et al. 2021b) for images, and SECOND Yan et al. 2018), PointNet (Qi
15

---

Published as a conference paper at ICLR 2024
Extract Image
Image
FFN
Features
Features
Detection Results
Transformer Decoder
Fused Features
Cross Attention Layer
as Q
Extract
Lidar
Lidar
Queries Generation
Features
Features
Figure 8: general architecture of concurrent camera- camera-LiDAR fusion models for 3D object detection. FFN denotes Feed-Forward Network.
et al. 2017p. VoxelNet (Zhou & Tuzel 2018) for point cloud data, among others. Subsequently. the extracted features from each modality fused together employing either aligement-based or non-alignment-basee designs, which vary from model to model. After fusion, a detection head employed generate the final predictions. The 3D object detection head generally adopts trans- former decoder-based architecture in cutting-edge fusion models since the efficacy of transformers in object detection has been substantiated by DETR (Carion al. 2020). In the transformer- detection head, the input consists of three sequences of feature vectors named queries (Q), keys (K) and values (V). Each input query vector corresponds an output vector, representing detection results for an object, including bounding box, object category, and detection score. Input keys and values, derived from fused features, provide scene-specific semantic information.
The initial queries generation in various models exhibits distinct design characteristics. For instance, UVTR (Li et al. 2022b) uses learnable parameters as queries, DeepInteraction (Yang et al., 2022) and TransFusion (Bai al. 2022) employ LiDAR features sampled using fused features, while BEVFusion-MIT [Lu et al. 2023b) and BEVFusion-PKI (Liang et al. (2022) utilize bird siyee--
view fused features. The decoder output is subsequently processed by a Feed-Forward Network (FFN) for final regression and classification results. Our attack general to both alignment-bor and non-alignment-baasee based fusion approaches regardless of the design of detection head.
D FEASIBILITY ANALYSIS
We analyze the feasibility of single-modal attacks on fusion models in this section. Since modern camera-LiDAR fusion models are all based on DNNs, we start with a simplified DNN-based fusion model. As shown in Figure 9 vector and use T = [bi, b2, by] to denote the LiDAR feature vector. Suppose these features are concatenated a unified vector during fusion and used calculate the next layer of features C [c1, (2,,c2,00 ca] with weight parameters W E R3x5 Hence we have:
AT [d1, a2) represent the extracted image feature
C = W vstack(A, B),
(10)
where vstack () is a concatenation operation. Specifically, the elements of are calculated as follows:
Cj
Wi aj Σ W,2+3 by (i 11=1,2,3)
(11)
Now, suppose adversarial perturbations are applied to an area the input image and some image features (i.c., aj) are affected while LiDAR features remain benign. Let A [ai A ag] be the adversarial image features. Then the fused features are C*T =G ch. cg] calculated as follows:
2
3
df
Wis
+ aj
Σ Wi2++ by + 1i,1 A
(12)
1=1
1=1
+ W
(6=1,3,3)
As we show, every fused feature is tainted by the adversarial features and the effect will finally propagate to the 3D object detection results, making single-modal attacks prediction results
16

---

Published as a conference paper at ICLR 2024
A
Extract Image
W1.J
Features
W1,2
a2
Fused Features
3D Detector
W1
B
b
11a
Extract
Lidar
b
11,
Features
b
Figure 9: Simplified illustration of singlc-modal attacks against fusion models for 3D object detection. denotes image features, b; Lidar features and o; fused features. Nodes marked in red denote affected features by perturbations image.
possible. The degree of effect the result depends the weights of the image features (e.g., Wi, in the model. Larger weights could have severe consequence. In addition, the fusion approach could also affect the result. For example, if we only concatenate the second element (a2) of with LiDAR features to calculate C' in the model, the previous adversarial attack on image cannot work anymore.
E GRADIENT-BASED ANALYSIS OF SENSITIVITY DISTRIBUTION RECOGNITION
Models employing different fusion approaches may exhibit varying distributions of vulnerable re- gions since the network design influences the training process and weights assignment. As fusion model may have millions of weight parameters, identifying sensitive areas by weight analysis is infeasible. Hence, we proposed an automatic method recognize the sensitivity distribution of fusion model on single-modal input.
The main idea is to leverage the gradients of input data with respect to the adversarial loss. Larger gradients in an input region indicate that small perturbations have higher impact on the adversarial goal, making it more "sensitive" area. Taking Figure as an example, through back- pack-propagation, the calculation of input pixel 2; gradient with respect to adversarial loss Ladv is shown in Equation 13
Σ abady Ozz
2 @oaato oaj) Ou, Dz;
V
3
Ladd dck Ocy
dag
(13)
daj
Oxi
/=1
Laad
Oaj
Σ
wk.i
Ock
Oz.
As demonstrated, the weights of the single modality in fusion (i.e., We.(( 1,2) are involved the gradient calculation, and higher weights can lead to larger gradients. Therefore, leverage the gradients as overall indicator to understand the significance or vulnerability of different areas within the single-modal input.
Upon analyzing the optimization problem from gradient perspective, it becomes evident that the gradients of M regarding the Lmmak steer towards the direction of minimizing mask values. In contrast, the gradients of M with respect Ladv exhibit an opposite direction. As indicated in Equation [3] areas with higher sensitivity and greater weights fusion possess larger gradients regarding Tuado: resulting in areas with higher intensity the mask following optimization.
17

---

Published as a conference paper at ICLR 2024
Deepiieemtion
BEV FusionPPUU
Despinwmntion
Degiimeention
BEVTnske-PRU
1,00
Patch
Roal
0.75
0.50
0.50
Olijoss
Paich
0.00
0.000
Beniga
outchon
Patch on objects
Denign
Patch objects
Heammp
read
(a) All objects.
(b) Patched objects.
Figure 10: Property validation the sensitivity Figure 11: Performance of image-specific adversarial
heatmap using image-specific adversarial patches.
patch in validating properties of sensitivity heatmap.
F MODEL SELECTION CRITERIA
1) Representativenes:: The selected models are the latest and most advanced fusion-based or camera-only models for 3D object detection. Published in 2022, each model's performance ranked top in the Nuscenes 3D object detection leaderboard (Motional 2023). Additionally, each model employs the Transformer architecture (Vaswani et al., (017) the detection head, which is widely recognized as a cutting-edge design in object detection models and has been adopted Tesla Au- topilot (AIDay 2022).
2) Practicality: The inputs to these models are multi-view images captured by six cameras sur- rounding vehicle and the corresponding LiDAR point cloud collected by 360-degree LiDAR sensor positioned on the vehicle's roof. This configuration of sensors is representative of practical autonomous driving systems and provides more comprehensive sensing capability when compared to models that rely solely on front cameras (e.g., KITTI dataset (Geiger et al. 2012
3) Accessibility: All models are publicly available, ensuring that they can be easily accessed by researchers. The best-performing version of each model that utilizes camera-LiDAR fusion (or camera-only for BEVFormer) was selected and utilized in our experiments and can be found on their respective project repositories on GitHub.
Overall, these six models were selected as they provide a comprehensive and representative as- sessment of the latest advancements in camera-LiDAR fusion-based and camera-based 3D object detection and were deemed practical, accessible and relevant the context of autonomous driving applications. The confidence (detection score) threshold for valid bounding box detection is set to 0.3 our evaluation to balance the false positive and false negative results.
G PROPERTY VALIDATION OF SENSITIVITY HEATMAP
To validate the utility of sensitivity heatmaps in identifying vulnerable regions adversarial tacks, we employ traditional image-specific adversarial patch on regions with distinct levels of sen- sitivity and evaluate the adversarial performance on two fusion models (i.e., DeepInteraction and BEVFusion-PKU) with distinct sensitivity types. We define a patch on both object and non-object areas to compare models' vulnerabilities since the two areas demonstrate different brightness on the sensitivity heatmap of DeepInteraction. More specifically, we define patch area of size 50 x 50 on objects or roads within Scene 2. and generate the patch to minimize all object scores in the scene. We utilize the Adam optimizer with learning rate of 0.001 and execute the optimization for 5000 iterations. Our experimental results are shown in Figure TO and Figure TT
In Figure 10, the first and second rows illustrate the patch road and objects, respectively. The third row shows the sensitivity heatmaps with selected patch areas designated in red for reference purposes. Each column represents a distinct model. Our findings demonstrate that the effect of adversarial patches on the detection capability road; however, the patch on the object leads to compromised detection of the patched objects. These results align with the sensitivity heatmap, where the object area shows greater intensity compared to the road area, meaning the object area more vulnerable. Contrarily, BEVFusion- PKU is globally
DeepInteraction is negligible for the patch on the
18

---

Published as a conference paper at ICLR 2024
Benign Detection
Adversarial Detection
Benign Detection
Adversarial Detection
Figure 13: Examples of object-oriented attacks.
Benphh actionn
Cntinaae Parsh
Lidor
BEViioo PXU
Comparison between the benign and adver- Figure 14: Regional optimized adversarial patch with sarial cases of scene-oriented attacks.
Figure 12:
one-stage optimization technique.
sensitive, as shown by the sensitivity heatmap. is observed that patches at both locations can cause false negative detection of surrounding objects, which is consistent with the heatmap and thus validates its accuracy.
Quantitative results are presented in Figure 11 We show in Figure 11a the average precision (AP) of the detection results of all pedestrians the current scene, and we show Figure TTb the average detection score of the two pedestrians covered by the patch in the patch-on-objects case. Our findings model, and has no significant impact indicated by the unchanged AP. However, the patch object is demonstrated to be effective for both models, as the detection scores for the patched objects decrease substantially in Figure [Tb Notably, the impact of the adversarial patch on DeepInteraction is confined mainly to the patched object, with only minor effect on the AP of the scene (see the minor decrease in the is more comprehensive,
reveal that the patch on road only affects the BEVFusion-PEE on DeepInteraction,
third blue bar of Figure Tap. In contrast, the impact on BEVFusion- affecting not only the patched object but also surrounding objects, leading to greater decrease in AP than in DeepInteraction (see the third red bar of Figure 11a).
In summary, our findings substantiate the reliability of sensitivity heatmaps as an effective metric to identify susceptible regions and determine optimal targets for adversarial patch attacks. Our study has additionally revealed that models with global sensitivity are more susceptible to such attacks since both object and non-object areas can be targeted, while only object areas can exploited in models with object sensitivity. Consequently, we have devised distinct attack strategies for models with different sensitivity distributions, which are described and assessed Section 5.2 and Sec- tion 5.3
H DETAILED EXPERIMENTAL SETUPS
Training Devices and Computational Overhead. Adversarial patches are generated utilizing a single GPU (Nvidia RTX A6000) equipped with memory capacity of 48G, in conjunction with an Intel Xeon Silver 4214R CPU. On average, the generation of a sensitivity heatmap for scene input for a specific fusion model requires approximately 1.2 hours, and average across 10 unique scenes's 's sensitivity heatmap for the sensitivity type classification with Equation 4 hence the computational overhead for the first stage of attack framework is around 12 hours. For
19

---

Published as a conference paper at ICLR 2024
the second stage, around 3.1 hours are required to generate an effective adversarial patch for either object-oriented or scene-oriented attacks.
Sensitivity Distribution Recognition. During the optimization, set the hyperparameters, in Equation and Equation | À to 1, s to 2, and y to 1. We adopt an Adam (Kingma & Bal 2014) op- timizer with learning rate of 0.001 and conduct 2000 iterations of optimization. Due to the unique size of the input images z for each model, we scale and crop the generated sensitivity heatmap of different models 256 704 for better visualization.
Scene-oriented Attacks. We select one scene from the Nuscenes (Caesar et al.. 2020) dataset in which the ego-vehicle is stationary at traffic light (see Figure 12). Note that a Nuscenes scene represents a driving clip that lasts about one minute. This scene includes 490 frames of multi-modal data (multi-view images, 360-degree LiDAR point clouds, and object annotations), and the object types and locations vary across different frames. We split the scene into two subsets: the first 245 frames are used as the "training set" to generate an adversarial patch on the ground in front of the victim vehicle, using Equation S The remaining 245 frames are used as the "test set" to measure the attack performance. We report mean Average Precision (mAP) seven categories of objects as the overall metric and average precision (AP) for each category as the specific metrics. The dimensions of the designated patch are mx2.5 m. situated at a distance of 7 m from the ego vehicle on the ground. We project the patch region in the physical world onto the front-camera image with the LiDAR-to-image projection matrix obtained from the dataset. The optimization process utilizes the Adam algorithm, incorporating a batch size of 5 and a learning rate of 0.01, executed over 1000 iterations.
Object-oriented Attacks. We select an additional scene from the Nuscenes dataset, featuring target vehicle driving in close proximity ahead of the ego-vehicle (see Figure This scene com- prises 260 sample frames, exhibiting dynamic variations in object types, positions, and background scenarios. Similar to the scene-oriented attacks, we utilize the first half of the frames as the "training set" and the latter half as the "testing set." " We utilize the object-oriented projection (see Figure 4b) to map the patch image onto an area of the target object in the scene image, with physical location parameters d, q and a extracted from the ground-truth 3D bounding boxes of the object. This area covers approximately one-ninth of the target vehicle's rear area with a size of I m x 1 m. Using Equation [5] the optimization process employs the Adam optimizer with a batch size of 5 and a learning rate of 0.01, executed for 1000 iterations. During testing, we measure and report the target object's 's average detection score and the mAP of detection results for other objects in the scene, including car, bus, pedestrian and motorcycle.
Physical-world Experiments. The image is captured using an iPhone 13, mounted on a camera tripod at height of 1.5m, with 70 degrees horizontal field-of-view (Allain) 2022) and height aligned with the front camera configuration used in the Nuscenes dataset. Note that using a smart- phone camera our experimental equipment is not compromised choice. Considering the overall cost and marginal performance improvement, cameras used on autonomous vehicles are usually not high-end ones and cameras on Tesla cost about 65$ each (Yoshida) 2020). In addition, the key sensor specifications (c.g., resolution and angular field-of-view) of our smartphone camera are kept consistent with the dataset sensors to achieve a faithful reproducement. Same as in the simulation, we begin by recording the intersection in a benign scenario, generate the scene-oriented adversarial patch targeting BEVFusion-PPie people to stand at locations similar to those of pedestrians in the original dataset image, capturing benign and adversarial data with and without the deployed patch for validation.
and print it. The printed patch measures 2.52.mm. We ask three
I QUALITATIVE RESULTS
Scene-oriented Attacks. Figure 12 demonstrates the benign and adversarial scenarios in the test set and the corresponding object detection outcomes of BEVFusion-PKL (Liang et al. (022). It is evident from the right column that the majority of objects in proximity to the adversarial patch are undetected, encompassing cars, buses, pedestrians, and other entities, which highlights a consider- able safety concern for both the veo-vehicle and individuals at the intersection.
Object-oriented Attacks. Figure 13 demonstrates the patched target vehicle in multiple frames and the failure of detecting it with DeepInteraction the subject fusion model.
20

---

Published as a conference paper at ICLR 2024
(a) Benign Scenario
(b) Patch on the Ground
(c) Adversarial Scenario
Figure 13: Attacks in CARLA.
Table 5: Attack performance at different times of the day with various lighting conditions.
Time of the day
Benign mAP
Adversarial mAP
Difference f
9:00 AM
0.428
0.191
55.37%
12:00 PM
0.457
0.184
59.74%
3:00 PM
0.481
0.165
65.70%
6:00 PM
0,43
0.204
52.56%
J COMPARISON WITH PATCH ATTACK AGAINST CAMERA-ONLY MODELS
We compare our two-stage attack framework with the state-of-the-ar one-stage patch optimization technique that targets camera-only models (Cheng et al. 2022). This method, originally designed to attack monocular depth estimation models with regionally optimized patch the target object, employs a differentiable representation of a rectangular patch mask, parameterized by four border parameters, to simultancously optimize both patch region and content. Consequently, it can pinpoint the most sensitive area for maximizing the attack impact. In our scenario, we optimize the patch region and content over the entire scene with the objective of false negative detection of objects. The results of incorporating this technique in our attack against fusion models are illustrated in Figure [4] As shown, the patches in Figure [4 using the same number of perturbed pixels as in Figure 2 could undermine detection results for DeepInteraction and BEVFusion-PKU on a single- image. These findings corroborate the efficacy of the patch optimization technique (Cheng et al.] 2022) on individual images. Nevertheless, the resulting patch presents significant limitations due to impracticality in real-world contexts. Its broad reach, which includes both dynamic entities and static backgrounds, renders it unsuitable for physical deployment and thereby undermines its overall applicability. Moreover, the variability of background scenes and objects across different image frames imposes significant fluctuations technique, thereby undermining its generality. In [Ceng et al. (202)). the patch region is narrowly constrained to specific target object, as opposed to optimizing across the entire scene in context. This vircumvents the aforementioned limitations. In contrast, our two-stage approach decouples the location and content optimization, and it is guaranteed to generate both deployable and effective patches for any scene (see Figure [2 and Figure 13). Consequently, our proposed two-stage method emerges as a more universal framework, with the potential to integrate prior methods into its second stage for refinement of the patch region following the determination of the vulnerable regions and attack strategies.
the optimal patch region as determined by this
K ADDITIONAL PRACTICALITY VALIDATION
To gather data a format similar to the Nuscenes dataset, we utilize the CARLA (Dosovitskiy et al, 2017) simulator and calibrate our sensors to match the Nuscenes configuration. The collected data is organized to match the format of the Nuscenes dataset, allowing the fusion models to process the simulator data without retraining. As shown in Figure 15p. we first collect benign data, in which the eeo-vehicle stops at an intersection and the surrounding pedestrians can be detected correctly. Sub- sequently, the scene-oriented adversarial patch for the intersection is generated using the gathered data, as outlined in Section 5.2 We use BEVFusion- of dimensions 2m x 2.5m and situated 7m from the ego-vehicle, is then integrated into the simu-
as the subject fusion model. This patch,
21

---

Published as a conference paper at ICLR 2024
Table Physical attack performance under various lighting conditions.
Time
Benign Score
Adversarial Score
Difference
9:00 AM
0.663
0.253
61.84%
6:00 PM
0.672
0.217
67.71%
Table 7: Attack performance with various patch dis- Table 8: Attack performance with various patch an- tances. Metric: The % of AP degradation under attack. gles. Metric: The % of AP degradation under attack.
Automotive
Pedestrain
Bicycle
Anples
mAP
Automotivv
Barrier
Pedlesmain
Bicycle
64.77%
70.04%
100.00%
-15
49.92'
45.45%
47.88
76.87%
27.98%
7.5m
62.55 53%
54.01%
45.43%
61.52%
85.05%
68.13%
6.77%
61.01%
100.00%
8.0m
70.49%
83.92%
49.33%
75.76%
74.95%
-5
49.31
44.73%
19.78%
76.46%
24.95%
8.5m
71.25%
3,26%
98.44%
65.76%
100.00%
0
70.49%
83.92%
49.33%
75.76%
74.95%
59.17%
68.49%
51.72%
100.00%
65.65%
100.00%
9.5m
8.56%
1.97%
16.82%
17.07%
0.00%
10
48.39%
13.62%
46.88%
76.46%
25.05%
10.0m
0.27%
0.04%
0.00%
1,018
0.00%
15
69. 00.200%
2.11%
82.52%
73,641%
100.00%
Average
53%
29.17%
58.09%
53.32%
76.67%
Average
60.16%
32.49%
64.92%
70.79%
64,70%
lation environment, at the intersection, as depicted in Figure 15p. As the ego vehicle approaches the patch in the adversarial scenario (See Figure [55p), the fusion models fail to detect surrounding pedestrians. The AP of the model's detection performance for pedestrians has dropped by 59%, from 0.457 the benign case to 0.184 in the adversarial case. Videos of the simulation can be found at https:: / /youtu..u.. be xhXt zDezeaa Note that the benign data used for patch gener- ation incorporates different pedestrians from the adversarial scenario, thus underscoring the patch's generality.
To further validate the robustness of our attacks in different lightning conditions, we repeated the above experiment at different times throughout the day to account for changes in light direction and intensity. The results are shown in Table 5 The first column denotes the time we conduct the experiment in simulation. The second and third columns indicate the benign and adversarial perfor- mance of BEV SEVFusionnPPKC degradation by comparing the benign and adversarial performance. As shown, our patch remains robust attack performance at different lighting conditions. To evaluate the impact of varying light- experiments at different
respectively. The fourth column presents the percentage of performance
ing conditions in the physical world, we also conduct the physical-workd times, results can be found in Table As shown, at different times of the day, the average detection scores of the pedestrians are significantly decreased after the patch is deployed, which validates the physical robustness of our patch under different lighting conditions.
L VARYING DISTANCE AND VIEWING ANGLES
In order to conduct comprehensive evaluation of our adversarial attacks, we modify the distance and viewing angles of the adversarial patch and assess the scene-oriented attack performance on the (YVFFusionPPK((LLA
et al. 2022) model. During the distance evaluation, with dimensions of 3m x 5m on the ground in front of the ego-vehicle, varying the distance from 7 meters to 10 meters. In the viewing angle assessment, we place the patch 8 meters from the ego-vehicle and rotate it around the z-axis (i.e., the vertical axis perpendicular to the ground) from -15 degrees 15 degrees. All other experimental settings remain consistent with Section [5.2] The performance degradation caused by our attack is reported in Table and Table The first column represents various distances and viewing angles, while the second column exhibits the percentage of mAP degradation of detection results. Subsequent columns display the specific average preci- sion degradation for individual object categories. The automotive group encompasses cars, trucks, buses, and trailers. As illustrated in Table 7 the adversarial patch demonstrates strong attack per- formance within the range of meters to meters, with diminishing effectiveness beyond meters. This decrease in efficacy may be attributed to the patch appearing smaller in the camera's 's field of view as the distance increases, resulting in fewer perturbed pixels and a consequent decline in at- tack performance. In real-world scenarios, distances less than meters are practical for initiating
position patch
22

---

Published as a conference paper at ICLR 2024
Table Detection score of the target object under different distances and patch sizes.
Imx Im
1mx0.5m
0.5mx
Distance
Ben.
Adv.
Diff.
Adv.
Diff.
Adv.
Diff.
5m
0.711
0,074
89.59%
0.081
88.61%
0.146
79.47%
6m
0.726
0.112
84.57%
0.108
85.12%
0.144
80.17%
8m
0.682
0.105
84.60%
0.137
79.91%
0.191
71.99%
10m
0.643
0.127
80.25%
0.242
62.36%
0.402
37.48%
15m
0.655
0.146
77.71%
0.48
26.72%
0.626
4.43%
Deepintenction
HEVPusioe-PKC
Scene Image
Trunsformer Backbone
ResNet50 Backbone
Soune
Imago
a-4
Seasitivity Hentmap
858
6-16
Figure 17: Sensitivity heatmap of BEVFusion-Min
ct al. 2023b) using different image backbones.
Figure 16: Sensitivity heatmap with different granularity.
scene-oriented attacks. For example, street paint at intersections is typically less than meters to the leading vehicle. Observations from Table indicate that the patch maintains robust attack perfor- mance across varying angles, with optimal performance occurring at no rotation (09). Both distance and angle results reveal that attack performance is better for foreground objects (e.g.. pedestrians and bicycles) situated closer to the adversarial patch and ego-vehicle.
We also conduct object-oriented attacks with different patch sizes and shapes various distances. We use the CARLA simulator to collect data with the target object locating at a range of distances from the ego-vehicle. Then generate object-oriented adversarial patch against the target object following Equation 5 The subject fusion model is DecpInteraction and we test the attack perfor- mance on another scene collected from the simulator. Experimental results are presented in Table The first column denotes the distance with the target vehicle. The second column reports the be- nign detection score of the target object without patch being attached. The following columns show the adversarial detection scores and the difference with benign case in percentage after the patch is applied. Larger difference indicates higher performance degradation and better attack performance. Patches of three difference sizes evaluated and results show that the minimum patch size neces- sary for a successful attack is contingent upon the distance between the target object and the victim vehicle. The critical factor is the pixel count influenced by the patch in the input images, and smaller patch can exhibit efficacy when in close proximity to the victim vehicle. The 0.5m x 0.5m patch still performs well at closer distance (e.g. less than 8m). Moreover, the patch does not need to be in square shape. Attackers have the flexibility to define patches with rectangular or arbitrary shapes to circumvent covering the target vehicle's license plate.
M VARYING GRANULARITY OF SENSITIVITY HEATMAP
Our sensitivity recognition algorithm employs the hyper-parameter in Equation 1 to regulate the granularity of the mask M applied to the perturbation, which denotes the size unit area. In order to assess whether the distribution of sensitivity would be influenced by variations in mask granularity, we perform experiments with diverse settings of s. By default, we establish as 2, and the corresponding outcomes are illustrated in Figure Outcomes derived from different granularity settings are depicted in Figure I6, We assign values of 4, and 16 to s, and generate the sensitivity heatmap for DeepInteraction and BEVFusion-PKUJ The first row in Figure 16 exhibits the origi-
23

---

Published as a conference paper at ICLR 2024
nal scene image and ground-truth objects. As the results indicate, the sensitivity distribution for given scene remains generally consistent across differing granularity of masks, and the two types of sensitivity continue to exhibit unique attributes. DeepInteraction retains its object-sensitive nature, whereas BEVFusion-PKU persistently demonstrates global sensitivity.
N DEFENSE DISCUSSION
Architecture-level defense. Our analysis reveals that camera-LiDAR fusion models exhibit two types of sensitivity to adversarial attacks: global and object sensitivity. Globally sensitive models are more vulnerable as they are susceptible to both scene-oriented and object-oriented attacks. In contrast, object-sensitive models are more robust due to their smaller sensitive regions and resistance to non-object area attacks. Both model types, however, perform similarly in benign object detec- tion. We investigate the architectural designs to understand the cause of different sensitivity types. We find that object-sensitive models (Deeplnteraction ( (Yang et al. 2022), UVTR (Li et al. 2022b),
Transfusion (Bai al. 2022) and BEVFormer (Li et al.) 2022c employ CNN-based Res (He
it al.
2016) as their image backbone. while globally sensitive models (BEVFusion-PK)
(Liang
al.
2022 and BEVFusion-MIT (Liu al. 2023b)) use wansformer-basee SwinTransformer (Liu
b). To further investigate, we retrain BEVFusion-MIT (Liu et al. 2023b) with ResNet50 et al.
and compare the sensitivity heatmap to the original SwinTransformer model, as shown in Figure 17 The results indicate that sensitive regions are more focused on objects when using ResNet, sug- gesting that the image backbone significantly impacts model vulnerability. An explanation that the CNN-based ResNet focuses more on local features due to its small convolutional kernels (usu- ally 3x3 pixels), while the mansffomer-bbased SwinTransformer captures more global information through self-attention mechanism. Consequently, adversarial patches distant from objects still affect detection in transformer-han hased backbones. To enhance the model's security against such at- tacks, incorporating ResNet50 the image backbone a preferable architectural choice.
LIMITATIONS
While our results demonstrate successful attacks against state-of-the-ane camera-LiDAR fusion mod- els using the camera modality, we have not conducted an end-to-end evaluation on an actual AV to illustrate the catastrophic attack outcomes (e.g., collisions sudden stops). This limitation stems from cost and safety concerns and is shared by other studies in AV security research (Cao et al,, 2021 2019 Sato et al. (02T). It should be noted that the most advanced fusion models examined in this work have not yet been implemented in open-sourced production-grade autonomous driving systems. Publicly available systems, such as Baidu Apollo (BaiduApollo) and Autoware.ai (Auto- ware), employ decision-leved fusion rather than the advanced feature-level or data-level fusion here. As result, we did not demonstrate an end-to-end attack in simulation. However, feature-level fusion is gaining attraction in both academia (Liu et al. 20236 Liang et al.] 2022 Yang et al.) 2022) and industry (Li et al., 2022a), driven by advancements in network designs and enhanced performance. Our technique is applicable to all feature- and data-level fusion models. Furthermore, our eval- uations on real-world dataset, in simulation, and the physical world, using industrial-grade AV sensor array underscore the practicality of our attack.
P THREAT MODEL
Our attack assumes that the attacker has complete knowledge of the camera-LiDAR fusion model used by the target autonomous driving vehicle. Therefore, our attack model is considered to be in a white-bos box setting. This assumption is congruent with analogous endeavors in the literature dedicated to adversarial attacks on autonomous driving systems (Cao et al. 2021 Sato et al. 2021 Zhang et al., 2022; Zhu et al.] 2023] Din et al. 2023). To achieve this level of knowledge, the attacker may employ methodologies such as reverse engineering the perception system of the victim vehicle (Lambert) 2021). employing open-sourced systems, exploiting information leaks from insiders. In addition, the attacker is clear about the target object and driving scenes he wants to attack. For example, he can record both video and LiDAR data using his vehicle and devices while following a target object (for object-oriented attacks) or stopping at a target scene (for scene- oriented attacks). Leveraging this pre-recorded data, the attacker undertakes one-time effort to
24

---

Published as a conference paper at ICLR 2024
generate adversarial patch using our proposed approach. The attacker then print and deploy the generated patch on the target object or in the target scene for real-time attacks against victim vehicles in the physical world.
DISCUSSION ON AUTONOMOUS VEHICLE SECURITY
While some production-gan grade Autonomous Vehicles (AVs) employ entirely vision-based percep- tion algorithms, such as Tesla, the majority utilize camera-L.iDAR fusion techniques (e.g.. Baidu Apollo (BaiduApollo), Google Waymo Waymo and Momenta (Momenta)). The focus of this search is on 3D object detection, a crucial task in AV perception that allows the AV to understand surrounding obstacles and navigate safely within the physical environment. The attacks proposed in this study against advanced fusion models challenge the security assumption of multi-sensor fusion- based perception algorithms, particularly the widely used early-fusion scheme (i.e., data-level and feature-level fusion strategies). Given that the adversarial patch we generated impacts object detec- tion in each data frame, subsequent tasks such as motion tracking and trajectory prediction within the victim vehicle will also be compromised. This increases the likelihood of the victim vehicle colliding with undetected targets, potentially resulting life- affethhrattnnng situations. is noteworthy that the attack performance intensifies the distance between the affected vehicle and the adver- sarial patch decreases, implying that the driver may not have adequate reaction time, leading to a braking distance that exceeds the actual distance.
The conduct of the planning module on the vietim vehicle depends on the speed of the victim vehicle and its relative speed in relation the target vehicle, as well surrounding environments. When the two speeds are relatively low, the victim vehicle may not necessarily change lanes or brake in advance before the target vehicle approaches within certain distance. For instance, in scenarios of traffic congestion, the victim vehicle may follow the target vehicle in front at close distance (e.g., less than 8 meters) due their shared low speed. Moreover, in intersections or parking lots, the victim vehicle may halt behind the target vehicle, thus failure in detection could result in collision when the victim vehicle starts.
Given that our attacks are executed in a white-box setting, it is possible that the AD industry could enhance its security mechanisms prevent model leakage. We anticipate that our investigation into the single-modal robustness of fusion models, along with the proposed attack methods, will attract the attention of developers and engineers within the AD industry. This could potentially stimulate the creation of more robust perception modules and autonomous driving systems.
25


Reviews:
[{"review_id": "rcMCkdVz9Y", "reviewer": "Reviewer_qqbx", "summary": "This paper studies the vulnerability of multi-sensor fusion to adversarial attacks in autonomous driving. The authors propose to leverage the adversarial patch to attack the camera modality in 3D object detection. Specifically, they propose an attack framework employing a two-stage optimization-based strategy that first evaluates vulnerable image areas under adversarial attacks, and then applies dedicated attack strategies for different fusion models to generate deployable patches.", "questions": "- What is the threat model. What information is available to the attacker during the attack? How feasible is it for the attacker to access this information in a real-world setting? What are the attacker's capabilities?\n\n- What is the major advantage of this work compared to existing attacks against camera-LiDAR fusion models?\n\n- How to place the adversarial patch on a pedestrian in the real world (as shown in Figure 10)? How to place the patch on the back of a vehicle? Does the patch hide the license plate of the vehicle?\n\n- Does the proposed attack maintain its effectiveness under varying light conditions in the real world?\n\n- The impact of the proposed attack on the vehicle's motion remains unclear. Is the perception system of the vehicle consistently deceived by the attack? Is the vehicle's trajectory affected by the attack?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This paper studies an important concern in autonomous driving, i.e., the vulnerability of multi-sensor fusion to adversarial attacks.\n\n- Multiple feature-level fusion models are considered in this paper.\n\n- The performance of the proposed framework is evaluated through both simulated and real-world experiments.", "weaknesses": "- This paper does not provide the threat model. What information is available to the attacker during the attack? How feasible is it for the attacker to access this information in a real-world setting? What are the attacker's capabilities?\n\n- The practicality and generalizability of the proposed attack are limited due to its ineffectiveness on decision-level fusion models, which are widely used in many autonomous driving systems, such as Baidu Apollo. Although the proposed attack can alter camera inputs, it fails to affect the outputs of decision-level fusion models. Moreover, these models tend to depend more on LiDAR detection results, further diminishing the practicality of the proposed attack.\n\n- I found it hard to understand the positioning of this paper. There are many existing works studying the attacks against camera-LiDAR fusion models [1,2]. These methods can be used to attack all three types of sensor fusion models including data-level fusion, feature-level fusion, and decision-level fusion. However, the method proposed in this paper can only be used to attack the first two types of fusion models. So, what is the major advantage of this work compared to those existing works? The authors should compare their method with existing attacks to demonstrate superiority of the proposed attack. \n\n[1] Exploring adversarial robustness of multi-sensor perception systems in self driving.\n[2] Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks.\n\n- The practicability of the adopted adversarial patch is questionable. Table 7 shows that the minimum dimensions of the patch are 1 meter by 1 meter, and the patch is too large to be practical. How to place such a large patch on a pedestrian in the real world? In addition, it's impractical to place such a large patch on the back of a vehicle. The patch may hide the license plate of the vehicle, which is prohibited by the traffic law. \n\n- The real-world evaluation is weak. The authors propose to use a patch with a special color pattern to conduct the attack. Such a color pattern can be affected by many factors in the physical world such as light condition, the distance between the camera and the patch, as well as the view angle of the camera. However, the authors do not evaluate the impact of these factors in the real-world setting. \n\n- The impact of the proposed attack on the vehicle's motion remains unclear. Is the perception system of the vehicle consistently deceived by the attack? Is the vehicle's trajectory affected by the attack?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hMaycWLJmJ", "reviewer": "Reviewer_QA2X", "summary": "This paper proposes a new approach to attacking fusion models from the camera modality, which successfully compromises advanced camera-LiDAR fusion models and demonstrates the weaknesses of relying solely on fusion for defence against adversarial attacks. The proposed attack framework is based on a novel PointAug method that perturbs the point cloud data and generates realistic-looking adversarial examples. The experiments conducted in both simulated and physical-world environments show the practicality and effectiveness of the proposed approach.", "questions": "1.Comparison with Pre-existing Methodologies: Given the emergence and evolution of methodologies targeting fusion models, how does the proposed approach position itself relative to these existing strategies? An analytical juxtaposition against established techniques would elucidate its uniqueness, advantages, and potential shortcomings.\n2.Defensive Countermeasures against the Approach: While the paper sheds light on an innovative adversarial approach, it begs the question: What are the viable defensive strategies that can be deployed to counteract its effects? Unveiling potential countermeasures not only underscores the resilience of the approach but also aids in the development of more robust fusion models.\n3.Extension and Scalability Concerns: Fusion models are diverse and multifaceted. How malleable is the proposed approach in its application to other fusion model variants? A deeper dive into its adaptability would provide insights into its scalability and flexibility across various fusion paradigms.\n4.Implications for Autonomous Vehicle Security: Given the pivotal role of fusion models in autonomous vehicle systems, the proposed adversarial approach inevitably raises safety and security concerns. How might these attacks compromise the integrity and reliability of autonomous driving systems? A comprehensive discussion on this would be crucial for stakeholders in the autonomous vehicle domain.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.Sophisticated Camera-LiDAR Fusion Model: The proposed methodology exemplifies a commendable synthesis of advanced camera-LiDAR fusion models. This amalgamation not only taps into the inherent strengths of individual modalities but also crafts a synergistic fusion, ensuring that the combined system is more robust and efficient than its constituent parts in isolation.\n2.Efficacy of PointAug in Generating Adversarial Samples: An intrinsic highlight of the paper is the PointAug method, which adeptly fabricates realistic-looking adversarial examples. Such capability is pivotal, particularly in the realm of robust machine learning, as it enables researchers to thoroughly evaluate the resilience of models against potential adversarial threats.\n3.Rigorous Experimental Validation in Diverse Environments: The rigorousness and diversity of experiments set this work apart. By conducting evaluations in both simulated and real-world environments, the paper fortifies the assertion of the proposed approach's practicality and efficacy. This dual-pronged validation underscores the method's adaptability and reliability in a wide range of scenarios.", "weaknesses": "1.Limited Fusion Model Efficacy: The methodology, while promising, seems to be narrowly tailored for a specific set of fusion models. This raises concerns about its universality. An in-depth exploration into its effectiveness against a broader spectrum of fusion models would have provided a more comprehensive perspective, allowing for a holistic understanding of its potential and pitfalls.\n2.Data Dependency and Generalizability Concerns: The experiments, predominantly based on a circumscribed dataset, cast doubts on the model's capacity to generalize across diverse scenarios. The exclusive reliance on a limited dataset can inadvertently introduce biases, thereby undermining the robustness of the approach when deployed in novel, real-world situations.\n3.Inadequate Security Analysis and Potential Resource Constraints: While the paper delves into several aspects of the proposed approach, it seems to sidestep a comprehensive analysis of its security implications. Given the pivotal role of security in such contexts, a detailed discourse would have been invaluable. Furthermore, the potentially substantial computational overhead required to generate adversarial samples may render the approach untenable for resource-constrained environments. The paper's omission of a thorough dissection of its inherent limitations further obscures the potential challenges one might encounter in its adoption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ji7vbx0VqJ", "reviewer": "Reviewer_DXNR", "summary": "This paper proposes an adversarial attack method, which could attack multi-modal 3D object detection methods from the image input, making the attack easy to implement.\nIt first recognizes the proper attack type and proposes different methods for each type.\nExperiments are conducted with multiple popular multi-modal detectors, showing promising performance.", "questions": "- In attack strategies, is the noise patch shared by the whole dataset? I saw the patch keeping changing in object-level attacks but remaining unchanged in scene-level attacks.\n- Is the mask in Sensitivity Distribution Recognition shared by the whole dataset?\n- I do not understand the form of the proj_x in Eq. 6 and why it is necessary.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The motivation that attacks from image input is reasonable, which is practical real world.\n- The proposed method overall makes sense.\n- Authors provide a demo, which makes the application of the proposed method more clear.\n- Authors conducted extensive experiments with many SOTA the art detectors, making the results convincing.", "weaknesses": "- Attack from only the image side has limited application. In particular, some methods do not conduct feature-level fusion between image and LiDAR like [1][2].  The two modalities are decoupled in these methods. Even if the image modality is totally failed, they can output reasonable results.\n- The writing is not clear, especially in page 4 and page 5. There are very long paragraphs and many notations without clear organization, making it hard to follow the detailed method. I list some detailed questions in the following question box. \n- The method is not well-motivated. The proposed method seems to be a general attacking method. Are there any special designs to solve problems in image-only attacking or autonomous driving scenes? What is the difficulty of image-only attacks?\n\n[1] Frustum PointNets for 3D Object Detection from RGB-D Data \\\n[2] Fully Sparse Fusion for 3D Object Detection", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NdDu3iyiHy", "reviewer": "Reviewer_nx8r", "summary": "The author proposes a two-stage optimization strategy for camera-LiDAR fusion models via using adversarial patches in camera images. The paper explores single-modal attack on fusion models, an interesting yet under-explored problem.", "questions": "* What is the sensitivity of single-modal methods vs multi-modal (LiDAR/camera) methods?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-written and explores an interesting topic. \n* The proposed method can decrease the model performance by a large margin, showing the possibility of dramatically deteriorating the system performance by only attacking single-modality.", "weaknesses": "* The effectiveness of the proposed two-stage optimization approach needs further justifications. Only showing the performance drop on fusion models is not enough. Comparisons with other single-stage attacks are also needed to demonstrate the effectiveness. Without proper benchmarks and comparisons with other SOTA algorithms, it is hard to justify the effectiveness of the technical contributions. \n* How to ensure the feasibility of the adversarial patches? Since the gradient optimization may find patches in the undeployable areas e.g., sky, can the proposed approach ensure the attack is feasible in the real physical world? Also in the paper, the author assumes the lidar data would not be changed. Since the patch may influence the lidar intensity or introduce extra points, please provide justifications for this assumption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)