Title: Adv3D: Generating 3D Adversarial Examples in Driving Scenarios with NeRF
Venue: NeurIPS 2023
Accepted: no
Keywords: Adversarial Examples

Autonomous Driving

3D Object Detection

NeRF
Abstract: Deep neural networks (DNNs) have been proven extremely susceptible to adversarial examples, which raises special safety-critical concerns for DNN-based autonomous driving stacks (i.e., 3D object detection). Although there are extensive works on image-level attacks, most are restricted to 2D pixel spaces, and such attacks are not always physically realistic in our 3D world. Here we present Adv3D, the first exploration of modeling adversarial examples as Neural Radiance Fields (NeRFs). Advances in NeRF provide photorealistic appearances and 3D accurate generation, yielding a more realistic and realizable adversarial example. We train our adversarial NeRF by minimizing the surrounding objects' confidence predicted by 3D detectors on the training set. Then we evaluate Adv3D on the unseen validation set and show that it can cause a large performance reduction when rendering NeRF in any sampled pose. To generate physically realizable adversarial examples, we propose primitive-aware sampling and semantic-guided regularization that enable 3D patch attacks with camouflage adversarial texture. Experimental results demonstrate that the trained adversarial NeRF generalizes well to different poses, scenes, and 3D detectors. Finally, we provide a defense method to our attacks that involves adversarial training through data augmentation.


Abstract

Deep neural networks (DNNs) have been proven extremely susceptible to adversarial examples, which raises special safety-critical concerns for DNN-based autonomous driving stacks ( i.e. , 3D object detection). Although there are extensive works on image-level attacks, most are restricted to 2D pixel spaces, and such attacks are not always physically realistic in our 3D world. Here we present Adv3D, the first exploration of modeling adversarial examples as Neural Radiance Fields (NeRFs). Advances in NeRF provide photorealistic appearances and 3D accurate generation, yielding a more realistic and realizable adversarial example. We train our adversarial NeRF by minimizing the surrounding objects' confidence predicted by 3D detectors on the training set. Then we evaluate Adv3D on the unseen validation set and show that it can cause a large performance reduction when rendering NeRF in any sampled pose. To generate physically realizable adversarial examples, we propose primitive-aware sampling and semantic-guided regularization that enable 3D patch attacks with camouflage adversarial texture. Experimental results demonstrate that the trained adversarial NeRF generalizes well to different poses, scenes, and 3D detectors. Finally, we provide a defense method to our attacks that involves adversarial training through data augmentation.

1 Introduction

The perception system of self-driving cars heavily rely on DNNs to process input data and comprehend the environment. Although DNNs have exhibited great improvements in performance, they have been found vulnerable to adversarial examples [2,15,24,41]. These adversarial examples crafted by adding imperceptible perturbations to input data, can lead DNNs to make wrong predictions. Motivated by the safety-critical nature of self-driving cars, we aim to explore the possibility of generating physically realizable adversarial examples to disrupt 3D detectors in driving scenarios, and further improve the robustness of 3D detectors through adversarial training.

The 2D pixel perturbations (digital attacks) [15, 41] have been proven effective in attacking DNNs in various computer vision tasks [13,53,56]. However, these 2D pixel attacks are restricted to digital space and are difficult to realize in our 3D world. To address this challenge, several works have proposed physical attacks. For example, Athalye et al. [2] propose the framework of Expectation Over Transformation (EOT) to improve the attack robustness over 3D transformation. Other researchers generate adversarial examples beyond image space through differentiable rendering, as seen in [54,59]. These methods show great promise for advancing the field of 3D adversarial attacks and defense but

Given the safety-critical demand for self-driving cars, several works have proposed physically 34

realizable attacks and defense methods in driving scenarios. For example, Cao et al. [5, 6] propose to 35

learn 3D-aware adversarial attacks capable of generating adversarial mesh to attack 3D detectors. 36

However, their works only consider learning a 3D adversarial example for a few specific frames. 37 Thus, the learned example is not universal and may not transfer to other scenes. To mitigate this 38 problem, Tu et al. [43, 44] propose to learn a transferable adversary that is placed on top of a vehicle. 39 Such an adversary can be used in any scene to hide the attacked object from 3D detectors. However, 40 reproducing their attack in our physical world can be challenging since their adversary must have 41 direct contact with the attacked object. We list detailed comparisons of prior works in Tab. 1. 42

43

44

45

46

47

48

49

50

51

52

53

54

To address the above challenges and generate 3D adversarial examples in driving scenarios, we build Adv3D upon recent advances in NeRF [35] that provide both differentiable rendering and realistic synthesis. In order to generate physically realizable attacks, we model Adv3D in a patch-attack [40] manner and use an optimization-based approach that starts with a realistic NeRF object [26] to learn its 3D adversarial texture. We optimize the adversarial texture to minimize the predicted confidence of all objects in the scenes, while keeping shape unchanged. During the evaluation, we render the input agnostic NeRF in randomly sampled poses, then we paste the rendered patch onto the unseen validation set to evaluate the attack performance. Owing to the transferability to poses and scenes, our adversarial examples can be executed without prior knowledge of the scene and do not need direct contact with the attacked objects, thus making for more feasible attacks compared with [43,44,57,62]. Finally, we provide thorough evaluations of Adv3D on camera-based 3D object detection with the nuScenes [4] dataset. Our contributions are summarized as follows:

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

We introduce Adv3D, the first exploration of formulating adversarial examples as NeRF to attack 3D detectors in autonomous driving. Adv3D provides 3D-aware and photorealistic synthesis that was previously unavailable.

By incorporating the proposed primitive-aware sampling and semantic-guided regularization, Adv3D generates adversarial examples with enhanced physical realism and realizability.

We conduct extensive real-world experiments and demonstrate the transferability of our adversarial examples across unseen environments and detectors.

2 Related Work

2.1 Adversarial Attack

DNNs are known to be vulnerable to adversarial attacks, where a small perturbation in the input data can cause drastic changes in the output predictions. Szegedy et al. [41] first discovered that adversarial examples, generated by adding visually imperceptible perturbations to the original images, make DNNs predict a wrong category with high confidence. These vulnerabilities were also discovered in object detection and semantic segmentation [30,56]. Moreover, DPatch [30] proposes transferable patch-based attacks by compositing a small patch to the input image. However, perturbing image pixels alone does not guarantee that adversarial examples can be created in the physical world. To address this issue, several works have performed physical attacks [3, 8, 18, 23, 46, 52, 58, 61] and exposed real-world threats. For example, Athalye et al. [2] generated robust 3D adversarial objects by introducing the Expectation Over Transformation (EOT) method. Cheng et al. [11] developed an adversarial patch with physical-oriented transformations to attack a depth estimation network. In our work, we mainly aim to generate 3D adversarial examples for 3D object detection in driving scenarios.

2.2 Robustness in Autonomous Driving

With the safety-critical nature, it is necessary to pay special attention to robustness in autonomous 77 driving systems [47]. LiDAR-Adv [6] proposes to learn input-specific adversarial point clouds to 78 fool LiDAR detectors. Tu et al. [44] produces generalizable point clouds that can be placed on a 79

80

81

82

83

84

85

86

87

88

89

vehicle roof to hide it. Furthermore, several work [1, 5, 43] try to attack a multi-sensor fusion system by optimizing 3D mesh through differentiable rendering. We compare our method with prior works in Tab. 1. Our method demonstrates stronger transferability and fewer requirements than prior works.

2.3 Image Synthesis using NeRF

NeRF [35] enables photorealistic synthesis in a 3D-aware manner. Recent advances [45,60] in NeRF allow for control over materials, illumination, and 6D pose of objects. Additionally, NeRF's rendering comes directly from real-world reconstruction, providing more physically accurate and photorealistic synthesis than previous mesh-based methods that relied on human handicrafts. Moreover, volumetric rendering [19] enables NeRF to perform accurate and efficient gradient computation compared with dedicated renderers in mesh-based differentiable rendering [9,21,29].

Recently, there has been tremendous progress in driving scene simulation using NeRF. Block90 NeRF [42] achieves city-scale reconstruction by modeling the blocks of cities with several isolated 91 NeRFs to increase capacity. FEGR [51] learns to intrinsically decompose the driving scene for 92 applications such as relighting. Lift3D [26] use NeRF to generate new objects and augment them to 93 driving datasets, demonstrating the capability of NeRF to improve downstream task performance. The 94 driving scene simulation provides a perfect test bed to evaluate the effectiveness of self-driving cars. 95 Our method is related to Lift3D, but aims to understand and improve the robustness of 3D detectors. 96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

3 Preliminary

3.1 Camera-based 3D Object Detection in Autonomous Driving

Camera-based 3D object detection is the fundamental task in autonomous driving. Without loss of generality, we focus on evaluating the robustness of camera-based 3D detectors.

The 3D detectors process image data and aim to predict 3D bounding boxes of all surrounding objects. The parameterization of a 3D bounding box can be written as b = { R , t , s , c } , where R ∈ SO (3) is the rotation of the box, t = ( x, y, z ) indicate translation of the box center, s = ( l, w, h ) represent the size (length, width, and height) of the box, and c is the confidence of the predicted box.

The network structure of camera-based 3D object detectors can be roughly categorized into FoVbased (front of view) and BEV-based (bird's eye view). FoV-based methods [48,49,50] can be easily built by adding 3D attribute branches to 2D detectors. BEV-based methods [38,39] typically convert 2D image feature to BEV feature using camera parameters, then directly detect objects on BEV planes. We refer readers to recent surveys [25,31] for more detail.

3.2 Differentiable Rendering using NeRF

Our method leverages the differentiable rendering scheme proposed by NeRF [35]. NeRF parameterizes the volumetric density and color as a function of input coordinates. NeRF uses multi-layer perceptron (MLP) or hybrid neural representations [7, 14, 36] to represent this function. For each pixel on an image, a ray r ( t ) = r o + r d · t is cast from the camera's origin r o and passes through the direction of the pixel r d at distance t . In a ray, we uniformly sample K points from the near plane t near to the far plane t far , the k th distance is thus calculated as t k = t near +( t far -t near ) · k/K . For any queried point r ( t k ) on the ray, the network takes its position r ( t k ) and predicts the per-point color c k and density τ k with:

Note that we omit the direction term as suggested by [16]. The final predicted color of each pixel C ( r ) is computed by approximating the volume rendering integral using numerical quadrature [34]:

We build our NeRF upon Lift3D [26]. Lift3D is a 3D generation framework that generates photoreal121 istic objects by fitting multi-view images synthesized by 2D generative modes [20] using NeRF. The 122 network of Lift3D is a conditional NeRF with additional latent code input, which controls the shape 123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

Adversarial Loss

𝑥𝑥

𝑦𝑦

𝑧𝑧

Shape

Texture

NeRF network

Sampled

Original Image

Pretrained

3D Detectors

Primitive-aware Sampling

Attacked Image

Optimize

Step1: Adversary Generation

Step2: Gradient Propagation

Image

Composition

Objective:

minimize all the

confidences

1

and texture of the rendered object. The conditional NeRF in Lift3D is a tri-plane parameterized [7] generator. With its realistic generation and 3D controllability, Lift3D has demonstrated that the training data generated by NeRF can help to improve downstream task performance. To further explore and exploit the satisfactory property of NeRF, we present a valuable and important application in this work: we leverage the NeRF-generated data to investigate and improve the robustness of the perception system in self-driving cars.

4 Method

We illustrate the pipeline of Adv3D in Fig. 1. We aim to learn a transferable adversarial example in 3D detection that, when rendered in any pose ( i.e. , location and rotation), can effectively hide surrounding objects from 3D detectors in any scenes by lowering their confidence. In Sec. 4.1, to improve the physical realizability of adversarial examples, we propose (1) Primitive-aware sampling to enable 3D patch attacks. (2) Disentangle NeRF that provides feasible geometry, and (3) Semantic-guided regularization that enables camouflage adversarial texture. To enhance the transferability across poses and scenes, we formulate the learning paradigm of Adv3D within the EOT framework [2] in Sec. 4.3.

4.1 3D Adversarial Example Generation

We use a gradient-based method to train our adversarial examples. The training pipeline involves 4 steps: (i) randomly sampling the pose of an adversarial example, (ii) rendering the example in the sampled pose, (iii) pasting the rendered patch into the original image of the training set, and finally, (iv) computing the loss and optimizing the latent codes. During inference, we discard the (iv) step.

4.1.1 Pose Sampling

To achieve adversarial attack in arbitrary object poses, we apply Expectation of Transformation (EOT) [2] by randomly sampling object poses. The poses of adversarial examples are parameterized as 3D boxes b that are restricted to a predefined ground plane in front of the camera. We model the ground plane as a uniform distribution B in a specific range that is detailed in the supplement. During training, we independently sample the rendering poses of adversarial examples, and approximate the expectation by taking the average loss over the whole batch.

4.1.2 Primitive-aware Sampling

Wemodel the primitive of adversarial examples as NeRF tightly bound by 3D boxes, in order to enable non-contact and physically realizable attacks. During volume rendering, we compute the intersection of rays r ( t ) with the sampled pose b = { R , t , s } ∈ B , finding the first hit point and the last hit point of box ( t near , t far ) by the AABB-ray intersection algorithm [33]. We then sample our points inside

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

the range ( t near , t far ) to reduce large unnecessary samples and avoid contact with the environment.

where ˜ r ( t ) is the sampled points with additional global to local transformation. Specifically, we use a 3D affine transformation to map original sampled points r ( t ) = r o + r d · t into a canonical space ˜ r = { x, y, z } ∈ [ -1 , 1] . This ensures that all the sampled points regardless of their distance from the origin, are transformed to the range [ -1 , 1] , thus providing a compact input representation for NeRF network. The transformation is given by:

where b = { R , t , s } , R ∈ SO (3) is rotation matrix of the box, t , s ∈ R 3 indicate translation and scale vector that move and scale the unit cube to desired location and size. The parameters of b are sampled from a pre-defined distribution B detailed in the supplement.

Then, the points lied in [ -1 , 1] are projected to exactly cover the tri-plane features z for interpolation. Finally, a small MLP takes the interpolated features as input and predicts RGB and density:

The primitive-aware sampling enables patch attacks [40] in a 3D-aware manner by lifting the 2D patch to a 3D box, enhancing the physical realizability by ensuring that the adversarial example only has a small modification to the original 3D environment.

4.1.3 Disentangled NeRF Parameterization

The original parameterization of NeRF combines the shape and texture into a single MLP, resulting in an entangled shape and texture generation. Since shape variation is challenging to reproduce in the real world, we disentangle shape and texture generation and only set the texture as adversarial examples.

We obtain texture latents z tex . and shape latents z shape from the Lift3D. During volume rendering, we disentangle shape and texture generation by separately predicting RGB and density:

where z shape is fixed and z texture is being optimized. Our disentangled parametrization can also be seen as a geometry regularization in [43, 44] but keeps geometry unchanged as a usual vehicle, leading to a more realizable adversarial example.

4.1.4 Semantic-guided Regularization

Setting the full part of the vehicle as adversarial textures is straightforward, but not always feasible in the real world. To improve the physical realizability, we propose to optimize individual semantic parts, such as doors and windows of a vehicle. Specifically, as shown in Fig. 2 (d, e)), we only set a specific part of the vehicle as adversarial texture while maintaining others unchanged. This semantic-guided regularization leads to a camouflage adversarial texture that is less likely spotted in the real world.

To achieve this, we add a semantic branch to Lift3d [26] to predict semantic part labels of the sampled points. We re-train Lift3d by fitting multi-view images and semantic labels generated by editGAN [28]. Using semantic-guided regularization, we maintain the original texture and adversarial part texture at the same time but only optimize the adversarial part texture while leaving the original texture unchanged. This approach allows us to preserve a large majority of parts as usual, but to alter only the specific parts that are adversarial (see Fig. 2 (b, c)). Potential attackers can easily print the adversarial sticker and stick it on the semantic part of vehicles to hide surrounding objects.

In our implementation, we query the NeRF network twice, one for the adversarial texture and the other for the original texture. Then, we replace the part of original texture with the adversarial texture indexed by semantic labels in the point space.

4.2 Gradient Propagation

After rendering the adversarial examples, we paste the adversarial patch into the original image through image composition. The attacked image can be expressed as I 1 × M + I 2 × (1 -M ) where I 1 and I 2 are the patch and original image, M is foreground mask predicted by NeRF. Next, the attacked images are fed to pretrained and fixed 3D detectors to compute the objective and backpropagate the gradients. Since both the rendering and detection pipelines are differentiable, Adv3D allows gradients from the objective to flow into the texture latent codes during optimization.

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

(a)

(b)

(c)

(d)

(e)

4.3 Learning Paradigm

We formulate our learning paradigm as EOT [2] that finds adversarial texture latent codes by minimizing the expectation of a binary cross-entropy loss over sampled poses and original images:

where b is the rendering pose sampled from the predefined distribution of ground plane B , x is the original image sampled from the training set X , I ( x , b , z tex . ) is the attacked image that composited by the original image x and the adversarial patch rendered using pose b and texture latent code z tex . , and P ( I ( · )) represents the confidence of all proposals predicted by detectors. We approximate the expectation by averaging the objective of the independently sampled batch. The objective is a binary cross-entropy loss that minimizes the confidence of all predicted bounding boxes, including adversarial objects and normal objects.

Built within the framework of EOT, Adv3D helps to improve the transferability and robustness of adversarial examples over the sampling parameters (poses and scenes here). This means that the attack can be performed without prior knowledge of the scene and are able to disrupt models across different poses and times in a non-contact manner.

4.4 Adversarial Defense by Data Augmentation

Toward defenses against our adversarial attack, we also study adversarial training to improve the robustness of 3D detectors. Adversarial training is typically performed by adding image perturbations using a few PGD steps [32,55] during the training of networks. However, our adversarial example is too expensive to generate for the bi-level loop of the min-max optimization objective. Thus, instead of generating adversarial examples from scratch at every iteration, we directly leverage the transferable adversarial examples to augment the training set. We use the trained adversarial example to locally store a large number of rendered images to avoid repeated computation. During adversarial training, we randomly paste the rendered adversarial patch into the training images with a probability of 30% , while remaining others unchanged. We provide experimental results in Sec. 5.4.

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

Pred.

GT

Ego

Clean Images

Attacked Images

Clean/Attacked BEV Visualization

(a)

(b)

(c)

5 Experiments

In this section, we first describe the training detail of our adversarial attacks. Then we present the experiments of semantic-guided regularization in Sec. 5.1, the analysis of 3D attack in Sec. 5.2, the transferability across detectors in Sec. 5.3, and our adversarial defense method in Sec. 5.4.

Dataset We conduct our experiments on the widely used nuScenes dataset [4]. This dataset is collected using 6 surrounded-view cameras that cover the full 360° field of view around the egovehicle. The dataset contains 700 scenes for training and 150 scenes for validation. In our work, we train our adversarial examples on the training set and evaluate performance drop on the validation set.

Training We implement our methods using PyTorch [37] and MMDetection3D [12]. All detectors are resumed from checkpoints available on their open-source repositories to match the original performance exactly. We only select one instance from Lift3D [26] as the initialization of examples. We conduct our experiments using 8 NVIDIA A100 80G GPUs. We use the Adam optimizer [22] with a learning rate of 1e-3 for texture latents. In practice, we optimize texture latents on the training set for five epochs with the same batch size as used during training detectors. We do not use any regularization except for semantic-guided regularization. In all experiments without specified, we render two adversarial examples per image. We ablate the number of rendered adversaries in the supplement.

Target Detectors and Metrics Weevaluate the robustness of six representative detectors. Three are FoV-based, and three are BEV-based. The FoV-based methods are FCOS3D [48], PGD-Det [49] and DETR3D [50]. The BEV-based methods are BEVDet [17], BEVFormer-Tiny [27] and BEVFormerBase. Following prior work [57], we evaluate the performance drop on the validation set after the attack. Specifically, we use the Mean Average Precision (mAP) and nuScenes Detection Score (NDS) [4] to evaluate the performance of 3D detectors.

Quantitative Results We provide the experimental results of adversarial attacks in Tab. 2. The attacks are conducted in a full-part manner without semantic-guided regularization to investigate the upper limit of attack performance. We found that, in spite of FoV-based or BEV-based, they display similar robustness. Meanwhile, we see a huge improvement of robustness by utilizing a stronger backbone (ResNet101 versus ResNet50) when comparing BEVFormer-Base with BEVFormer-Tiny. We hope these results will inspire researchers to develop 3D detectors with enhanced robustness.

Visualization Results We visualize our attack results with semantic-guided regularization in Fig. 3 (a,b), and without regularization in Fig. 3 (c). The disappearance of detected objects is caused by their lower confidence scores. For example, the confidence predicted by detectors in Fig. 3 (a) have declined

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

from 0 . 6 to 0 . 4 , and are therefore filtered out by the visualization threshold of 0 . 6 . In Fig. 3 (a), we find that our adversarial NeRF is realistic enough to be detected by a 3D detector if it doesn't display much of the adversarial texture. However, once the vehicle shows a larger area of the adversarial texture as seen in Fig. 3 (b), it will hide all objects including itself due to our untargeted objective.

5.1 Semantic Parts Analysis

In Tab. 3, we provide experiments on the impact of different semantic parts on attack performance. Specifically, we focused on three salient parts of the vehicle: the front, side, and rear. Our results show that compared with adversarial attacks using full parts, the semantic-guided regularization leads to a slightly lower performance drop, but remains a realistic appearance and less likely spotted adversarial texture as illustrated in Fig. 2 (b).

Since we do not have access to annotation during training, we additionally conduct "No Part" experiment that no part of the texture is adversarial, to evaluate the impact of the collision and occlusion. We acknowledge that part of performance degradation can be attributed to the occlusion to original objects and the false positive prediction of adversarial objects (see Fig. 3 (a)), since we do not update the ground truth of adversarial objects to the validation set.

5.2 Effectiveness of 3D-aware attack

To validate the effectiveness of our 3D attacks, we ablate the impact of different poses on the attack performance. In Fig. 4 (a), we divide the BEV plane into 10 × 10 bins ranging from x ∈ [ -5 m, 5 m ] and z ∈ [10 m, 15 m ] . We then evaluate the relative mAP drop (percentage) of BEVDet [17] by sampling one adversarial example inside the bin per image, while keeping rotation randomly sampled. Similarly, we conduct experiments of 30 uniform rotation bins ranging from [0 , 2 π ] in Fig. 4 (b). The experimental results demonstrate that all aspects of location and rotation achieve a valid attack (performance drop > 30% ), thereby proving the transferability of poses in our 3D-aware attack.

A finding that contrasts with prior work [44] is the impact of near and far locations in z axis. Our adversarial example is more effective in the near region compared with the far region, while Tu et al. [44] display a roughly uniform distribution in all regions. We hypothesize that the attack performance is proportional to the area of the rendered patch, which is highly related to the location

30.0

32.5

35.0

37.5

40.0

42.5

45.0

47.5

𝑥𝑥

𝑧𝑧

-5𝑚𝑚

10𝑚𝑚

15𝑚𝑚

%

5𝑚𝑚

0

1

2

𝜋𝜋

𝑥𝑥

𝑧𝑧

34.0

36.0

38.0

40.0

42.0

44.0

46.0

%

(a) Location

(b) Rotation

𝜃𝜃

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

of objects. Similar findings are also displayed in rotation. The vehicle that poses vertically to the ego vehicle results in a larger rendered area, thus better attack performance.

5.3 Transferability Across Different Detectors

In Tab. 5, we evaluate the transferability of adversarial examples across different detectors. To this end, we train a single adversarial example of each detector separately, then use the example to evaluate the performance drop of other detectors. We show that there is a high degree of transferability between different models. Among them, we observe that DETR3D [50] appears to be more resilient to adversarial attacks than other detectors. We hypothesize this can be attributed to the sparsity of the query-based method. During the projection of 3D query to the 2D image plane, only a single point of the feature is indexed by interpolation, thus fewer areas of adversarial features will be sampled. This finding may have insightful implications for the development of more robust 3D detectors in the future.

5.4 Adversarial Defense by Data Augmentation

We present the results of adversarial training in Tab. 4. The symbol † indicates attacks using the same adversarial example used in adversarial training, while § indicates a different example. We observe that incorporating adversarial training improves not only the robustness against the seen adversarial examples, but also the clean performance. However, we also note that our adversarial training is not capable of transferring to unseen adversarial examples trained in the same way, mainly due to the fixed adversarial example during adversarial training. Furthermore, we hope that future work can conduct in-depth investigations and consider handling the bi-level loop of adversarial training in order to better defend against adversarial attacks.

6 Limitation and Future Work

Learning to Sample and Attack As we do not have access to the dataset annotations, we can not model the explicit relationship between adversarial and normal objects to avoid collision, and the collision itself can cause a performance drop ("No Parts" in Tab. 3). Future work can apply geometryaware composition [10] to mitigate this problem. Additionally, future research can explore learning to predict optimal poses of adversarial objects to maximize the effectiveness of attacks.

Potential Harmful Consequences The trained adversarial examples have the potential to induce serious traffic accidents in driving scenarios. However, our work is not intended to cause disruptions in autonomous driving systems. Instead, our goal is to use the examples to gain a deeper understanding of the systems and improve their robustness. We hope our work will draw more attention of the community to further verify and enhance the robustness of autonomous driving systems.

7 Conclusion

In this paper, we propose Adv3D , the first attempt to model adversarial examples as NeRF. Adv3D enhances the physical realizability of attacks through our proposed primitive-aware sampling and semantic-guided regularization. Compared with prior works of adversarial examples in autonomous driving, our examples are more threatening in practice as we carry non-contact attacks, have feasible 3D shapes as usual vehicles, and display camouflage adversarial texture. Extensive experimental results also demonstrate that Adv3d transfers well to different poses, scenes, and detectors. We hope our work provides valuable insights for creating more realistic evaluations to investigate and improve the robustness of autonomous driving systems.

References 323

Mazen Abdelfattah, Kaiwen Yuan, Z Jane Wang, and Rabab Ward. Adversarial attacks on camera-lidar 324 models for 3d car detection. In IROS , 2021. 3 325

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. 326 2018. 1, 2, 4, 6 327

Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv 328 preprint arXiv:1712.09665 , 2017. 2 329

Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, 330 Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. 331 In CVPR , 2020. 2, 7 332

Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan 333 Liu, and Bo Li. Invisible for both camera and lidar: Security of multi-sensor fusion based perception in 334 autonomous driving under physical-world attacks. In IEEE Symposium on Security and Privacy (SP) , 2021. 335 1, 2, 3 336

Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. Adversarial 337 objects against lidar-based autonomous driving systems. arXiv preprint arXiv:1907.05418 , 2019. 1, 2 338

Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio 339 Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, and Gordon Wetzstein. Efficient 340 geometry-aware 3D generative adversarial networks. In CVPR , 2022. 3, 4 341

Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Chau. Shapeshifter: Robust physical 342 adversarial attack on faster r-cnn object detector. In ECML PKDD , 2019. 2 343

Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. 344 Learning to predict 3d objects with an interpolation-based differentiable renderer. NeurIPS , 32, 2019. 3 345

Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Sivabalan Manivasagam, Shangjie 346 Xue, Ersin Yumer, and Raquel Urtasun. Geosim: Realistic video simulation via geometry-aware composi347 tion for self-driving. In CVPR , 2021. 9 348

Zhiyuan Cheng, James Liang, Hongjun Choi, Guanhong Tao, Zhiwen Cao, Dongfang Liu, and Xiangyu 349 Zhang. Physical attack on monocular depth estimation with optimal adversarial patches. In ECCV , 2022. 2 350

MMDetection3D Contributors. MMDetection3D: OpenMMLab next-generation platform for general 3D 351 object detection. https://github.com/open-mmlab/mmdetection3d , 2020. 7 352

Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu. Benchmarking 353 adversarial robustness on image classification. In CVPR , 2020. 1 354

Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 355 Plenoxels: Radiance fields without neural networks. In CVPR , 2022. 3 356

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. 357 ICLR , 2015. 1 358

Jiatao Gu, Lingjie Liu, Peng Wang, and Christian Theobalt. Stylenerf: A style-based 3d aware generator 359 for high-resolution image synthesis. In ICLR , 2022. 3 360

Junjie Huang, Guan Huang, Zheng Zhu, Ye Yun, and Dalong Du. Bevdet: High-performance multi-camera 361 3d object detection in bird-eye-view. arXiv preprint arXiv:2112.11790 , 2021. 6, 7, 8, 9 362

Lifeng Huang, Chengying Gao, Yuyin Zhou, Cihang Xie, Alan L. Yuille, Changqing Zou, and Ning Liu. 363 Universal physical camouflage attacks on object detectors. In CVPR , 2020. 2 364

James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. ACM SIGGRAPH computer 365 graphics , 18(3), 1984. 3 366

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and 367 improving the image quality of StyleGAN. In CVPR , 2020. 3 368

Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. Neural 3d mesh renderer. In CVPR , 2018. 3 369

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 370 arXiv:1412.6980 , 2014. 7 371

Stepan Komkov and Aleksandr Petiushko. Advhat: Real-world adversarial attack on arcface face id system. 372 In ICPR . IEEE, 2021. 2 373

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In 374 ICLR Workshop , 2017. 1 375

Hongyang Li, Chonghao Sima, Jifeng Dai, Wenhai Wang, Lewei Lu, Huijie Wang, Enze Xie, Zhiqi Li, 376 Hanming Deng, Hao Tian, Xizhou Zhu, Li Chen, Yulu Gao, Xiangwei Geng, Jia Zeng, Yang Li, Jiazhi 377 Yang, Xiaosong Jia, Bohan Yu, Yu Qiao, Dahua Lin, Si Liu, Junchi Yan, Jianping Shi, and Ping Luo. 378 Delving into the devils of bird's-eye-view perception: A review, evaluation and recipe. arXiv preprint 379 arXiv:2209.05324 , 2022. 3 380

381

382

Leheng Li, Qing Lian, Luozhou Wang, Ningning Ma, and Ying-Cong Chen. Lift3d: Synthesize 3d training data by lifting 2d gan to 3d generative radiance field. In CVPR , 2023. 2, 3, 4, 5, 7

383

384

385

Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's-eye-view representation from multi-camera images via spatiotemporal transformers. ECCV , 2022. 6, 7, 9

Huan Ling, Karsten Kreis, Daiqing Li, Seung Wook Kim, Antonio Torralba, and Sanja Fidler. Editgan: 386 High-precision semantic image editing. In NeurIPS , 2021. 5 387

Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. Soft rasterizer: A differentiable renderer for image388 based 3d reasoning. In ICCV , 2019. 3 389

Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Hai Li, and Yiran Chen. Dpatch: An adversarial patch 390 attack on object detectors. arXiv preprint arXiv:1806.02299 , 2018. 2 391

Yuexin Ma, Tai Wang, Xuyang Bai, Huitong Yang, Yuenan Hou, Yaming Wang, Y. Qiao, Ruigang Yang, 392 Dinesh Manocha, and Xinge Zhu. Vision-centric bev perception: A survey. 2022. 3 393

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards 394 deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017. 6 395

Alexander Majercik, Cyril Crassin, Peter Shirley, and Morgan McGuire. A ray-box intersection algorithm 396 and efficient dynamic voxel rendering. Journal of Computer Graphics Techniques Vol , 7(3), 2018. 4 397

Nelson Max. Optical models for direct volume rendering. IEEE TVCG , 1995. 3 398

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren 399 Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV , 2020. 2, 3 400

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives 401 with a multiresolution hash encoding. ACM ToG , 2022. 3 402

403

404

405

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NeurIPS , 32, 2019. 7

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In ECCV , 2020. 3

Cody Reading, Ali Harakeh, Julia Chae, and Steven L. Waslander. Categorical depth distributionnetwork for monocular 3d object detection. CVPR , 2021. 3

Abhijith Sharma, Yijun Bian, Phil Munz, and Apurva Narayan. Adversarial patch attacks and defences in vision-based tasks: A survey. arXiv preprint arXiv:2206.08304 , 2022. 2, 5

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. ICLR , 2014. 1, 2

Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-nerf: Scalable large scene neural view synthesis. In CVPR , 2022. 3

James Tu, Huichen Li, Xinchen Yan, Mengye Ren, Yun Chen, Ming Liang, Eilyan Bitar, Ersin Yumer, and Raquel Urtasun. Exploring adversarial robustness of multi-sensor perception systems in self driving. arXiv preprint arXiv:2101.06784 , 2021. 2, 3, 5

James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. Physically realizable adversarial examples for lidar object detection. In CVPR , 2020. 2, 5, 8

Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-nerf: Structured view-dependent appearance for neural radiance fields. In CVPR , 2022. 3

Jiakai Wang, Aishan Liu, Zixin Yin, Shunchang Liu, Shiyu Tang, and Xianglong Liu. Dual attention 424 suppression attack: Generate adversarial camouflage in physical world. In CVPR , 2021. 2 425

Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas, Mengye Ren, 426 and Raquel Urtasun. Advsim: Generating safety-critical scenarios for self-driving vehicles. CVPR , 2021. 2 427

Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. FCOS3D: Fully convolutional one-stage monocular 428 3d object detection. In ICCV Workshop , 2021. 3, 6, 7, 9 429

Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Probabilistic and Geometric Depth: Detecting 430 objects in perspective. In CoRL , 2021. 3, 6, 7, 9 431

Yue Wang, Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, , and Justin M. Solomon. Detr3d: 432 3d object detection from multi-view images via 3d-to-2d queries. In CoRL , 2021. 3, 6, 7, 9 433

Zian Wang, Tianchang Shen, Jun Gao, Shengyu Huang, Jacob Munkberg, Jon Hasselgren, Zan Gojcic, 434 Wenzheng Chen, and Sanja Fidler. Neural fields meet explicit geometric representations for inverse 435 rendering of urban scenes. In CVPR , June 2023. 3 436

Tong Wu, Xuefei Ning, Wenshuo Li, Ranran Huang, Huazhong Yang, and Yu Wang. Physical adversarial 437 attack on vehicle detector in the carla simulator. arXiv preprint arXiv:2007.16118 , 2020. 2 438

Chong Xiang, Charles R Qi, and Bo Li. Generating 3d adversarial point clouds. In CVPR , 2019. 1 439

Chaowei Xiao, Dawei Yang, Bo Li, Jia Deng, and Mingyan Liu. Meshadv: Adversarial meshes for visual 440 recognition. In CVPR , 2019. 1 441

Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc V Le. Adversarial 442 examples improve image recognition. In CVPR , 2020. 6 443

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples 444 for semantic segmentation and object detection. In CVPR , 2017. 1, 2 445

Shaoyuan Xie, Zichao Li, Zeyu Wang, and Cihang Xie. On the adversarial robustness of camera-based 3d 446 object detection. arXiv preprint arXiv:2301.10766 , 2023. 2, 7 447

Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi 448 Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in a physical world. In ECCV , 2020. 2 449

Xiaohui Zeng, Chenxi Liu, Yu-Siang Wang, Weichao Qiu, Lingxi Xie, Yu-Wing Tai, Chi-Keung Tang, and 450 Alan L Yuille. Adversarial attacks beyond the image space. In CVPR , 2019. 1 451

Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul Debevec, William T Freeman, and Jonathan T 452 Barron. Nerfactor: Neural factorization of shape and reflectance under an unknown illumination. ToG , 453 2021. 3 454

Yang Zhang, Hassan Foroosh, Philip David, and Boqing Gong. Camou: Learning physical vehicle 455 camouflages to adversarially attack detectors in the wild. In International Conference on Learning 456 Representations , 2019. 2 457

Zijian Zhu, Yichi Zhang, Hai Chen, Yinpeng Dong, Shu Zhao, Wenbo Ding, Jiachen Zhong, and Shibao 458 Zheng. Understanding the robustness of 3d object detection with bird's-eye-view representations in 459 autonomous driving. arXiv preprint arXiv:2303.17297 , 2023. 2 460

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a new attack on monocular 3D object detectors utilising NERF representation to create multi-view consistent attacks utilising Lift3D [26]. The authors show successful attacks on nuScenes dataset and the effect of their design choices on the attack success. Furthermore, a mitigating strategy is shown how to make monocular 3D object detection robust to these types of attacks.\n\nSTRENGTHS:\nThis manuscript includes a comprehensive set of ablation studies and detailed analyses, which effectively highlight the influence of different components in the proposed pipeline. Figure 4 is especially illustrative and insightful. Notably, the successful tackling of numerous molecular 3D detectors, each with varying detection strategies, is commendable.\n\nThe authors present an effective strategy to counteract the proposed adversarial attacks on molecular detection systems. \n\nThe overall presentation of the paper is lucid and the figures contribute significantly to accurately conveying the intended ideas.\n\nWEAKNESSES:\n1- A comparison against baselines is missing, most notably the mesh-based baseline [43]. This comparison is very important since the proposed setup in Lift3D [26] is very similar when constraining the Nerf to shape and texture latents , resembling the mesh generation and texturing scheme in [43]. \n\n2- A crucial evaluation protocol has been overlooked. In the context of adversarial attacks, the imperceptibility of the attack is a significant factor; how discernible is the attack in contrast to the clean sample? This critical information is absent in this work. It leaves us questioning the perceptibility of image corruption in terms of pixel alteration. If the corruption is so pronounced that even a human observer fails to detect the cars, can it still be considered a successful attack? Previous studies typically provide this information [43]. Moreover, the inclusion of tests on KITTI should be considered vital to the evaluation process.\n\nQUESTIONS:\nDespite some concerns, the work presents an innovative methodology for crafting adversarial attacks on the crucial application of monocular 3D object detection in autonomous vehicles. The manuscript is well-structured and successfully demonstrates attacks, supported by a wealth of insightful analysis. However, the absence of comparisons with similar baselines and the lack of detailed evaluation metrics assessing imperceptibility prevent me from fully endorsing the paper at this stage. I would appreciate further clarification from the authors on these points before finalising my decision.\n\nMinor Comments:\n\nThe authors could consider including additional related references [a,b] that were overlooked. An animated presentation of a KITI video demonstrating successful attacks would be an informative addition.\n\n[a] AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds, ECCV 2020\n\n[b] SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications, AAAI 2020\n\n---------------------------------------------\npost rebuttal thoughts :\n\nThe authors have addressed my concerns regarding the baselines and the imperceptibility of the attack in the rebuttal. I dont think real-life applications are necessary here ( as other reviewers think) and hence I increase my score to accept\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
This work develops adversarial attacks against 3D object detectors by utilizing instance-level NeRFs.  
They start with a representation of a vehicle, parameterized by a NeRF that predicts both geometry and texture and render the vehicle into a image, which they compose into the original image by copy-pasting. They use the composited image to adversarially attack 3D object detectors, which provides a gradient signal used to optimize the NeRF (texture only).  
Experiments show their adversarial examples are effective against a variety of different 3D object detectors, and they show that training on these samples improves robustness (and even overall performance).

STRENGTHS:
* Novel application of NeRFs, utilizing the fully differentiability to optimize for adversarial texture.
* Work is well written and overall clear to follow.  
* Analysis provides good insights (referring to Sec 5.3 analysis of 3D detector architecture robustness and Sec 5.4 adversarial training actually boosts performance).
* Multiple architectures used in experiments.

WEAKNESSES:
* Section 4.4 could use more elaboration - this is a key section for the overall work and in the current revision is quite vague.
* Some of the attacks (Fig. 3) do not look photorealistic - how can the reader be convinced these adverarial samples would actually work in the real world?

QUESTIONS:
* Section 4.4: How many rendered images are used? Are there any heuristics used to make sure the pasted patches are physically realizable?
* What are the computational costs of generating a single attack?

LIMITATIONS:
Authors provide discussion of limitations (real world safety).
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDeep neural networks (DNNs) have shown susceptibility to adversarial examples, which raises significant safety concerns, particularly in safety-critical applications like DNN-based autonomous driving systems and 3D object detection. While there is a wealth of research on image-level attacks, most of them focus on the 2D pixel space, which may not always translate into physically realistic attacks in our 3D world. In this paper, the authors present Adv3D, the first exploration of modeling adversarial examples as Neural Radiance Fields (NeRFs).\n\nThe utilization of NeRFs allows for the generation of adversarial examples that possess photorealistic appearances and accurate 3D generation, thereby enabling more realistic and realizable adversarial attacks in the 3D domain. The authors train their adversarial NeRF by minimizing the confidence of surrounding objects predicted by 3D detectors on the training set. They evaluate Adv3D on an unseen validation set and demonstrate its ability to significantly degrade performance when rendering the NeRF in various sampled poses.\n\nTo ensure the practicality of the adversarial examples, the authors propose primitive-aware sampling and semantic-guided regularization techniques, which facilitate 3D patch attacks with camouflage adversarial textures. The experimental results showcase the generalizability of the trained adversarial NeRF across different poses, scenes, and 3D detectors. Additionally, the authors provide a defense mechanism against these attacks through adversarial training via data augmentation.\n\nIn summary, the authors introduce Adv3D as a novel approach that models adversarial examples using NeRFs, resulting in more realistic and realizable attacks in the 3D domain. They demonstrate the effectiveness of their method through extensive evaluations and propose a defense strategy to mitigate the impact of these attacks.\n\nSTRENGTHS:\n1. The writing and presentation of this paper are good and clear.\n2. The idea of leveraging NeRF in generating adversarial examples is interesting\n3. The authors conducted sufficient evaluation of the proposed method\n\nWEAKNESSES:\n1. The practicality of the proposed attack is questionable\n2. There is a lack of real-world experiments\n3. The study seems to lack technical insights as the adversarial attacks are well established. There is no surprise that using some fancy new idea can lead to adversarial examples, but the essence is the same.\n\nQUESTIONS:\nPlease see the weakness section.\n\nLIMITATIONS:\nThe authors has discussed the potential negative societal impact of this study, so it should be fine.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors proposed new generative adversarial examples in the form of NeRFs, in the context of driving scenarios. The training objective is minimizing the 3D detection confidence from a variety of views. The parameters to optimize are the latent input to the NeRF, that encodes shape and texture info. Rendering is naturally differentiable due to the usage of NeRF. To improve the physical realizability, they propose three methods: primitive-aware sampling, NeRF disentanglement, and semantic-guided regularization. The authors conducted experiments on the widely used nuScenes dataset to evaluate the performance drop. The results show that their method is able to reduce the detection performance of various detectors, whether they are FOV-based detectors or birdview-based ones. They also evaluated the transferability of their method, and the adversarial training defense method.\n\nSTRENGTHS:\nUsing NeRF as 3D adversarial example representation seems novel and interesting. The NeRF representation naturally is differentiable in terms of rendering, so it makes the adversarial attack problem easier. Also, with more uses of NeRF in 3D vision, it is important to explore the vulnerability in NeRF itself. Such adversarial attacks may highlight the potential security issues in NeRF.\n\nThe attacking framework (expectation over transformation), the NeRF rendering framework they use (Lift3D) are standard. The method is mostly built upon existing works; it seems not hard to implement their method.\n\nThe writing is clear.\n\nWEAKNESSES:\nMy major concern is that whether the formulation of NeRF is necessarily, from the motivation perspective. In line 175, they fixed the shape and only optimized the texture latent code. The optimization is essentially finding the color, density of the volume. However, I believe most vehicle objectives are not translucent; the optimized 3D object is very hard to realize. This is evident as authors need to improve the physical realizability (line 180).\n\nThis leads to the Occam's razor principle: do we really need NeRFs to reach the effectiveness/realizability of the 3D attack? So we are missing a baseline here: optimizing the surface texture as a 3D mesh, using existing differentiable mesh renderers (such as Neural Mesh Renderer). The latter is easier to optimize (2D texture space), and more physically realizable (because it is a texture map rather than a volume). In line 166, the authors said \"enables patch attacks in a 3D-aware manner by lifting the 2D patch to a 3D box\", so we really need a baseline to showcase such lifting is necessary. Also it is not clear how rendering the NeRF into 3D scenes is done. In Fig. 3, the lighting of the NeRF object is not consistent with the environment, and we can see typical blurriness of NeRF.\n\nAnother weakness is that the setting is not sophisticated enough to be \"Driving Scenarios\". At first glance, it looks like attacking self-driving algorithms, but the point clouds are not used (correct me if I am wrong). The detection methods (FCOS3D, PGD-Det etc) are based on monocular/multi-view 2D images instead of multi-sensor. In Fig. 3, the inserted adversarial example does not seem to block the LiDAR rays. The experiment is not done through a full driving simulation software, but by rendering 3D objects into existing 3D data. Whether such mixed environment can represent the real-world driving scenario is not clear. It'll be better to claim general 3D detection scenario and do more experiments with other objects, instead of only claiming driving-specific scenarios.\n\nIn general, my decision largely depends on the first point: the NeRF representation may not be necessary under the current settings. Optimizing the texture image should just work; such volume formulation will make it harder to physically realize and does not bring much benefit other than differentiable rendering.\n\nQUESTIONS:\n1. How does the method to a naive baseline, such as direct optimization of the texture of the car? Is it possible to do any simple ablation experiments?\n2. Is it possible to reconstruct the NeRF, 3D-print it, then test in the real world?\n3. How is the NeRF rendered into nuScenes? There are shadows under the NeRF renderings, but somehow the lighting is not consistent. Is it because the NeRF formulation omits shading (line 119)? Can you clarify?\n\nLIMITATIONS:\nThe authors addressed the limitations about dataset annotations and potential harmful consequences.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes to generate 3D adversarial examples for attacking 3D object detectors in driving scenarios using NeRF. In particular, it integrates a series of techniques, including primitive-aware sampling and semantic-guided regularization, to ensure the physical realism and realizability of the generated adversarial examples. Extensive experiments have validated the effectiveness of the proposed method in reducing detection performance and serving as data augmentation.\n\nSTRENGTHS:\n1. As an early attempt of generating 3D adversarial examples using NeRF, this work could offer a new perspective for the community in understanding and tackling real-world 3D adversarial attacks. \n\n2. The extensive experiments validate the superiority of NeRF as a 3D adversarial attack generator. In particular, it is interesting to see that the generated adversarial examples can serve as a data augmentation to improve clean performance, which aligns with the previous observations in classification.\n\nWEAKNESSES:\n1. My major concern is the assumed attacking setting of this work, i.e., how to leverage the proposed method in real-world driving scenarios. If only a static adversarial example is attached to the scene, generating other static objects on the road may be more practical than generating a vehicle; Otherwise, the authors are expected to show a video under an egocentric view to demonstrate the attack effectiveness, i.e., whether the dynamically moving adversarial vehicles can consistently mislead the 3D detectors from different view directions.\n\n2. The claim \"the first exploration of modeling adversarial examples as Neural Radiance Fields (NeRFs)\" in the abstract may not be accurate. ViewFool [1] also models adversarial examples using NeRF although only the view direction is adversarially optimized. It will be more accurate if the authors highlight this work as the first 3D adversarial example generator using NeRF.\n\n3. Missing references regarding the early attempts of marrying NeRF and adversarial attacks (which are mostly orthogonal with this work):\n\n[1] \"ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints\", Y. Dong et al., NeurIPS'22.\n\n[2] \"NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations\", Y. Fu et al., ICML'23.\n\n[3] \"Aug-NeRF: Training Stronger Neural Radiance Fields With Triple-Level Physically-Grounded Augmentations\", T. Chen et al., CVPR'22.\n\n4. Minor issue: There exists some inconsistency in terms of tense and punctuation, which could be improved in the final version.\n\nQUESTIONS:\n1. In my understanding, the primitive-aware sampling essentially performs a coordinate transformation between the world coordinate of the driving scenario and the canonical space defined by Lift3D. Will this transformation stretch the shape of the generated adversarial vehicles?\n\n2. How could you generate adversarial vehicles in different styles if the mapping between the optimized texture code and the vehicle texture is deterministic?\n\nLIMITATIONS:\nAlthough the developed adversarial attacks may cause security concerns, this work intended to gain a deeper understanding of 3D adversarial examples and improve the achievable robustness on them, thus not suffering from negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-13"}