Original: BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework
Revised: BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Fusing the camera and LiDAR information has become a de-facto standard for 3Dobject detection tasks.
Revised: Fusing the camera and LiDAR information has become a de-facto standard for3D object detection tasks.

Original: Current methods rely on point clouds from the LiDAR sensoras queries to leverage the feature from the image space.
Revised: Current methods rely on point clouds from the LiDARsensor as queries to leverage the feature from the image space.

Original: However, people discoverthat this underlying assumption makes the current fusion framework infeasible toproduce any prediction when there is a LiDAR malfunction, regardless of minor ormajor.
Revised: However, peoplediscovered that this underlying assumption makes the current fusion frameworkinfeasible to produce any prediction when there is a LiDAR malfunction, regardlessof minor or major.

Original: This fundamentally limits the deployment capability to realistic autonomousdriving scenarios.
Revised: This fundamentally limits the deployment capability to realisticautonomous driving scenarios.

Original: In contrast, we propose a surprisingly simple yet novel fusionframework, dubbed BEVFusion, whose camera stream does not depend on theinput of LiDAR data, thus addressing the downside of previous methods.
Revised: In contrast, we propose a surprisingly simpleyet novel fusion framework, dubbed BEVFusion, whose camera stream does notdepend on the input of LiDAR data, thus addressing the downside of previousmethods.

Original: Weempirically show that our framework surpasses the state-of-the-art methods underthe normal training settings.
Revised: We empirically show that our framework surpasses the state-of-the-artmethods under the normal training settings.

Original: Under the robustness training settings that simulatevarious LiDAR malfunctions, our framework signiﬁcantly surpasses the state-ofthe-art methods by 15.7% to 28.9% mAP.
Revised: Under the robustness training settingsthat simulate various LiDAR malfunctions, our framework signiﬁcantly surpassesthe state-of-the-art methods by 15.7% to 28.9% mAP.

Original: To the best of our knowledge, we arethe ﬁrst to handle realistic LiDAR malfunction and can be deployed to realisticscenarios without any post-processing procedure.
Revised: To the best of our knowledge,we are the ﬁrst to handle realistic LiDAR malfunction and can be deployed torealistic scenarios without any post-processing procedure.

Original:  Introduction
Revised:  Introduction

Original: Vision-based perception tasks, like detecting bounding boxes in 3D space, have been a criticalaspect of fully autonomous driving tasks [55, 40, 56, 39].
Revised: Vision-based perception tasks, like detecting bounding boxes in 3D space, have been a criticalaspect of fully autonomous driving tasks [55, 41, 56, 40].

Original: Among all the sensors of a traditionalvision on-vehicle perception system, LiDAR and camera are usually the two most critical sensorsthat provide accurate point cloud and image features of a surrounding world.
Revised: Among all the sensors of a traditionalvision on-vehicle perception system, LiDAR and camera are usually the two most critical sensorsthat provide accurate point cloud and image features of a surrounding world.

Original: In the early stage ofperception system, people design separate deep models for each sensor [35, 36, 57, 15, 52] andfusing the information via post-processing approaches [30].
Revised: In the early stage ofperception system, people design separate deep models for each sensor [36, 37, 57, 16, 52] andfusing the information via post-processing approaches [31].

Original: Note that, people discover that bird’s eyeview (BEV) has been an de-facto standard for autonomous driving scenarios as, generally speaking,car cannot ﬂy [19, 22, 37, 15, 53, 31].
Revised: Note that, people discover that bird’s eyeview (BEV) has been an de-facto standard for autonomous driving scenarios as, generally speaking,car cannot ﬂy [20, 23, 38, 16, 53, 32].

Original: However, it is often difﬁcult to regress 3D bounding boxes onpure image inputs due to the lack of depth information, and similarly, it is difﬁcult to classify objectson point clouds when LiDAR does not receive enough points.
Revised: However, it is often difﬁcult to regress 3D bounding boxes onpure image inputs due to the lack of depth information, and similarly, it is difﬁcult to classify objectson point clouds when LiDAR does not receive enough points.

Original: Recently, people have designed LiDAR-camera fusion deep networks to better leverage informationfrom both modalities.
Revised: Recently, people have designed LiDAR-camera fusion deep networks to better leverage informationfrom both modalities.

Original: Speciﬁcally, the majority of works can be summarized as follow: i) given oneor a few points of the LiDAR point cloud, LiDAR to world transformation matrix and the essentialmatrix (camera to world); ii) people transform the LiDAR points [41, 44, 46, 45, 16, 59] or proposals[5, 60, 2, 20] into camera world and use them as queries, to select corresponding image features.
Revised: Speciﬁcally, the majority of works can be summarized as follow: i) given oneor a few points of the LiDAR point cloud, LiDAR to world transformation matrix and the essentialmatrix (camera to world); ii) people transform the LiDAR points [42, 45, 46, 45, 17, 58] or proposals[5, 59, 2, 21] into camera world and use them as queries, to select corresponding image features.

Original: Thisline of work constitutes the state-of-the-art methods of 3D BEV perception.
Revised: Thisline of work constitutes the state-of-the-art methods of 3D BEV perception.

Original: However, one underlying assumption that people overlooked is, that as one needs to generate imagequeries from LiDAR points, the current LiDAR-camera fusion methods intrinsically depend on theraw point cloud of the LiDAR sensor, as shown in Fig.
Revised: However, one underlying assumption that people overlooked is, that as one needs to generate imagequeries from LiDAR points, the current LiDAR-camera fusion methods intrinsically depend on theraw point cloud of the LiDAR sensor, as shown in Fig.

Original: In the realistic world, people discover thatif the LiDAR sensor input is missing, for example, LiDAR points reﬂection rate is low due to object
Revised: In the realistic world, people discover thatif the LiDAR sensor input is missing, for example, LiDAR points reﬂection rate is low due to object

Original: [Figure]
Revised: [Figure]

Original: texture, a system glitch of internal data transfer, or even the ﬁeld of view of the LiDAR sensor cannotreach 360 degrees due to hardware limitations [1], current fusion methods fail to produce meaningfulresults 1 .
Revised: texture, a system glitch of internal data transfer, or even the ﬁeld of view of the LiDAR sensor cannotreach 360 degrees due to hardware limitations [1], current fusion methods fail to produce meaningfulresults 1 .

Original: This fundamentally hinders the applicability of this line of work in the realistic autonomousdriving system.
Revised: This fundamentally hinders the applicability of this line of work in the realistic autonomousdriving system.

Original: We argue the ideal framework for LiDAR-camera fusion should be, that each model for a singlemodality should not fail regardless of the existence of the other modality, yet having both modalitieswill further boost the perception accuracy.
Revised: We argue the ideal framework for LiDAR-camera fusion should be, that each model for a singlemodality should not fail regardless of the existence of the other modality, yet having both modalitieswill further boost the perception accuracy.

Original: To this end, we propose a surprisingly simple yet effectiveframework that disentangles the LiDAR-camera fusion dependency of the current methods, dubbed
Revised: To this end, we propose a surprisingly simple yet effectiveframework that disentangles the LiDAR-camera fusion dependency of the current methods, dubbed

Original: BEVFusion.
Revised: BEVFusion.

Original: Speciﬁcally, as in Fig.
Revised: Speciﬁcally, as in Fig.

Original: 1 (c), our framework has two independent streams that encodethe raw inputs from the camera and LiDAR sensors into features within the same BEV space.
Revised: 1 (c), our framework has two independent streams that encodethe raw inputs from the camera and LiDAR sensors into features within the same BEV space.

Original: Wethen design a simple module to fuse these BEV-level features after these two streams, so that the ﬁnalfeature can be passed into modern task prediction head architecture [19, 58, 2].
Revised: Wethen design a simple module to fuse these BEV-level features after these two streams, so that the ﬁnalfeature can be passed into modern task prediction head architecture [20, 57, 2].

Original: As our framework is a general approach, we can incorporate current single modality BEV models forcamera and LiDAR into our framework.
Revised: As our framework is a general approach, we can incorporate current single modality BEV models forcamera and LiDAR into our framework.

Original: We moderately adopt Lift-Splat-Shoot [32] as our camerastream, which projects multi-view image features to the 3D ego-car coordinate features to generatethe camera BEV feature.
Revised: We moderately adopt Lift-Splat-Shoot [33] as our camerastream, which projects multi-view image features to the 3D ego-car coordinate features to generatethe camera BEV feature.

Original: Similarly, for the LiDAR stream, we select three popular models, twovoxel-based ones and a pillar-based one [57, 2, 19] to encode the LiDAR feature into the BEV space.
Revised: Similarly, for the LiDAR stream, we select three popular models, twovoxel-based ones and a pillar-based one [57, 2, 20] to encode the LiDAR feature into the BEV space.

Original: On the nuScenes dataset, our simple framework shows great generalization ability.
Revised: On the nuScenes dataset, our simple framework shows great generalization ability.

Original: Following thesame training settings [19, 58, 2], BEVFusion improves PointPillars and CenterPoint by 18.4% and7.1% in mean average precision (mAP) respectively, and achieves a superior performance of 69.2%mAP comparing to 68.9% mAP of TransFusion [2], which is considered as state-of-the-art.
Revised: Following thesame training settings [20, 57, 2], BEVFusion improves PointPillars and CenterPoint by 18.4% and7.1% in mean average precision (mAP) respectively, and achieves a superior performance of 69.2%mAP comparing to 68.9% mAP of TransFusion [2], which is considered as state-of-the-art.

Original: Underthe robust setting by randomly dropping the LiDAR points inside object bounding boxes with aprobability of 0.5, we propose a novel augmentation technique and show that our framework surpassesall baselines signiﬁcantly by a margin of 15.7% ⇠ 28.9% mAP and demonstrate the robustness of ourapproach.
Revised: Underthe robust setting by randomly dropping the LiDAR points inside object bounding boxes with aprobability of 0.5, we propose a novel augmentation technique and show that our framework surpassesall baselines signiﬁcantly by a margin of 15.7% ⇠ 28.9% mAP and demonstrate the robustness of ourapproach.

Original: Our contribution can be summarized as follow: i) we identify an overlooked limitation of current
Revised: Our contribution can be summarized as follow: i) we identify an overlooked limitation of current

Original: LiDAR-camera fusion methods, which is the dependency of LiDAR inputs; ii) we propose a simpleyet novel framework to disentangle LiDAR camera modality into two independent streams that cangeneralize to multiple modern architectures; iii) we surpass the state-of-the-art fusion methods underboth normal and robust settings.
Revised: LiDAR-camera fusion methods, which is the dependency of LiDAR inputs; ii) we propose a simpleyet novel framework to disentangle LiDAR camera modality into two independent streams that cangeneralize to multiple modern architectures; iii) we surpass the state-of-the-art fusion methods underboth normal and robust settings.

Original: Code will be released upon acceptance of this paper.
Revised: Code will be released upon acceptance of this paper.

Original: Here, we categorize the 3D detection methods broadly based on their input modality.
Revised: Here, we categorize the 3D detection methods broadly based on their input modality.

Original: Camera-only.
Revised: Camera-only.

Original: In the autonomous driving domain, detecting 3D objects with only camera-inputhas been heavily investigated in recent years thanks to the KITTI benchmark [10].
Revised: In the autonomous driving domain, detecting 3D objects with only camera-inputhas been heavily investigated in recent years thanks to the KITTI benchmark [11].

Original: Since there isonly one front camera in KITTI, most of the methods have been developed to address monocular3D detection [27, 38, 26, 18, 61, 63, 37, 48, 47].
Revised: Since there isonly one front camera in KITTI, most of the methods have been developed to address monocular3D detection [28, 39, 27, 19, 60, 63, 38, 48, 47].

Original: With the development of autonomous drivingdatasets that have more sensors, like nuScenes[3] and Waymo [42], there exists a trend of developingmethods [49, 50, 52] that take multi-view images as input and found to be signiﬁcantly superior tomonocular methods.
Revised: With the development of autonomous drivingdatasets that have more sensors, like nuScenes[3] and Waymo [43], there exists a trend of developingmethods [49, 50, 52] that take multi-view images as input and found to be signiﬁcantly superior tomonocular methods.

Original: However, voxel processing is often accompanied by high computation.
Revised: However, voxel processing is often accompanied by high computation.

Original: As in common autonomous driving datasets, the objects in general move on ﬂat ground, PointPillars[19] proposes to map the 3D features onto a bird’s eye view 2D space to reduce the computationaloverhead.

Original: It soon becomes a de-facto standard in this domain [37, 19, 15, 53, 31, 22].

Original: Lift-SplatShoot (LSS) [32] uses depth estimation network to extract the implied depth information of multiperspective images and transform camera feature maps into 3D Ego-car coordinate.
Revised: As in common autonomous driving datasets, Lift-Splat-Shoot (LSS) [33] uses depth estimationnetwork to extract the implied depth information of multi-perspective images and transform camerafeature maps into 3D Ego-car coordinate.

Original: Methods[37, 15, 53] are also inspired by LSS [32] and refer to the LiDAR for the supervision on depthprediction.
Revised: Methods [38, 16, 53] are also inspired by LSS [33] andrefer to the LiDAR for the supervision on depth prediction.

Original: A similar idea can also be found in BEVDet [15, 14], the state-of-the-art method inmulti-view 3D object detection.
Revised: A similar idea can also be found in BEVDet [16, 15], the state-of-the-art method in multi-view 3D object detection.

Original: MonoDistill [7] and LiGA Stereo [11] improve performance byunify LiDAR information to a camera branch.
Revised: MonoDistill [7] and LiGA Stereo [12] improve performance by unify LiDAR information to a camera branch.

Original: LiDAR-only.
Revised: LiDAR-only.

Original: LiDAR methods initially lie in two categories based on their feature modality: i)point-based methods that directly operate on the raw LiDAR point clouds [36, 35, 34, 40, 56, 21];and ii) transforming the original point clouds into a Euclidean feature space, such as 3D voxels [62],feature pillar [19, 51, 58], and range images [9, 43].
Revised: LiDAR methods initially lie in two categories based on their feature modality: i)point-based methods that directly operate on the raw LiDAR point clouds [37, 36, 35, 41, 56, 22];and ii) transforming the original point clouds into a Euclidean feature space, such as 3D voxels [62]and feature pillar [20, 51, 57].

Original: Recently, people start to exploit these two featuremodalities in a single model to increase the representation power [39].
Revised: Recently, people started to exploit these two feature modalities in asingle model to increase the representation power [5, 61, 40].

Original: Another line of work is toexploit the beneﬁt of the bird’s eye view plane similar to the camera perception [19, 9, 43].
Revised: Another line of work is to exploit thebeneﬁt of the bird’s eye view plane [20, 10, 44].

Original: LiDAR-camera fusion.
Revised: LiDAR-camera fusion.

Original: As the features produced by LiDAR and camera contain complementaryinformation in general, people start to develop methods that can be jointly optimized on bothmodalities and soon become a de-facto standard in 3D detection.
Revised: As the features produced by LiDAR and camera contain complementaryinformation in general, people started to develop methods that can be jointly optimized on bothmodalities and soon become a de-facto standard in 3D detection.

Original: As in Fig.
Revised: As in Fig.

Original: 1, these methods can bedivided into two categories depending on their fusion mechanism, (a) point-level fusion where onequeries the image features via the raw LiDAR points and then concatenates them back as additionalpoint features [16, 41, 46, 59]; (b) feature-level fusion where one ﬁrstly projects the LiDAR points into a feature space [60] or generates proposals [2], queries the associated camera features then concatenates back to the feature space [23].
Revised: 1, these methods can bedivided into two categories depending on their fusion mechanism, (a) point-level fusion where onequeries the image features via the raw LiDAR points and then concatenates them back as additionalpoint features [17, 42, 46, 58]; (b) feature-level fusion where one ﬁrstly projects the LiDAR pointsinto a feature space [59] or generates proposals [2], queries the associated camera features thenconcatenates back to the feature space [24].

Original: The latter constitutes the state-of-the-art methods in3D detection, speciﬁcally, TransFusion [2] uses the bounding box prediction of LiDAR featuresas a proposal to query the image feature, then adapts a Transformer-like architecture to fuse theinformation back to LiDAR features.
Revised: The latter constitutes the state-of-the-art methods in3D detection, speciﬁcally, TransFusion [2] uses the bounding box prediction of LiDAR featuresas a proposal to query the image feature, then adapts a Transformer-like architecture to fuse theinformation back to LiDAR features.

Original: Similarly, DeepFusion [20] projects LiDAR features on eachview image as queries and then leverages cross-attention for two modalities.
Revised: Similarly, DeepFusion [21] projects LiDAR features on eachview image as queries and then leverages cross-attention for two modalities.

Original: An overlooked assumption of the current fusion mechanism is they heavily rely on the LiDAR pointclouds, in fact, if the LiDAR input is missing, these methods will inevitably fail.
Revised: An overlooked assumption of the current fusion mechanism is they heavily rely on the LiDAR pointclouds, in fact, if the LiDAR input is missing, these methods will inevitably fail.

Original: This will hinderthe deployment of such algorithms in realistic settings.
Revised: This will hinderthe deployment of such algorithms in realistic settings.

Original: In contrast, our BEVFusion is a surprisinglysimple yet effective fusion framework that fundamentally overcomes this issue by disentangling thecamera branch from the LiDAR point clouds as shown in Fig.
Revised: In contrast, our BEVFusion is a surprisinglysimple yet effective fusion framework that fundamentally overcomes this issue by disentangling thecamera branch from the LiDAR point clouds as shown in Fig.

Original: 1(c).
Revised: 1(c).

Original: Other modalities.
Revised: Other modalities.

Original: There exist other works to leverage other modalities, such as fusing camera-radarby feature map concatenation [4, 17, 29, 28].
Revised: There exist other works to leverage other modalities, such as fusing camera-radarby feature map concatenation [4, 18, 30, 29].

Original: While interesting, these methods are beyond thescope of our work.
Revised: While interesting, these methods are beyond thescope of our work.

Original: Despite a concurrent work [6] aims to fuse multi-modalities information in asingle network, its design is limited to one speciﬁc detection head [52] while our framework can begeneralized to arbitrary architectures.
Revised: Despite a concurrent work [6] aims to fuse multi-modalities information in asingle network, its design is limited to one speciﬁc detection head [52] while our framework can begeneralized to arbitrary architectures.

Original:  BEVFusion: A General Framework for LiDAR-Camera Fusion
Revised:  BEVFusion: A General Framework for LiDAR-Camera Fusion

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, we present our proposed framework, BEVFusion, for the 3D object detectionin detail.
Revised: 2, we present our proposed framework, BEVFusion, for the 3D object detectionin detail.

Original: As our fundamental contribution is disentangling the camera network from the LiDARfeatures, we ﬁrst introduce the detailed architecture of the camera and LiDAR stream, then present adynamic fusion module to incorporate features from these modalities.
Revised: As our fundamental contribution is disentangling the camera network from the LiDARfeatures, we ﬁrst introduce the detailed architecture of the camera and LiDAR stream, then present adynamic fusion module to incorporate features from these modalities.

Original: 2 D B ac kbon e 3 D B ac kbon e 3 D
Revised: 2 D B ac kbon e 3 D B ac kbon e 3 D

Original: O
Revised: O

Original: b
Revised: b

Original: j ec t
Revised: j ec t

Original: D e t ec ti on
Revised: D e t ec ti on

Original: H ea d
Revised: H ea d

Original: Multi view Images
Revised: Multi-view Images

Original: Point Clouds 2 D → 3 D
Revised: Point Clouds 2 D → 3 D

Original: P r o j ec t o r
Revised: P r o j ec t o r

Original: F u s i on M odu l e LiDAR Detection Result
Revised: F u s i on M odu l e LiDAR Detection Result

Original: B E V
Revised: B E V

Original: E n c od e r
Revised: E n c od e r

Original: Camera Stream
Revised: Camera Stream

Original: LiDAR Stream
Revised: LiDAR Stream

Original: FP N w / AD
Revised: FP N w / AD

Original: P
Revised: P

Original: Multi-view Features
Revised: Multi-view Features

Original: Final Detection Result
Revised: Final Detection Result

Original: Camera Detection Result
Revised: Camera Detection Result

Original: BEVFusioin Prediction
Revised: BEVFusioin Prediction

Original: [Figure]
Revised: [Figure]

Original: 3.1 Camera stream architecture: From multi-view images to BEV space
Revised: 3.1 Camera stream architecture: From multi-view images to BEV space

Original: As our framework has the capability to incorporate any camera streams, we begin with a popularapproach, Lift-Splat-Shoot (LSS) [33].
Revised: As our framework has the capability to incorporate any camera streams, we begin with a popularapproach, Lift-Splat-Shoot (LSS) [34].

Original: As the LSS is originally proposed for BEV semanticsegmentation instead of 3D detection, we ﬁnd out that directly using the LSS architecture has inferiorperformance, hence we moderately adapt the LSS to improve the performance (see Sec.
Revised: As the LSS is originally proposed for BEV semanticsegmentation instead of 3D detection, we ﬁnd out that directly using the LSS architecture has inferiorperformance, hence we moderately adapt the LSS to improve the performance (see Sec.

Original: 4.5 forablation study).
Revised: 4.5 forablation study).

Original: In Fig.
Revised: In Fig.

Original: 2 (top), we detail the design of our camera stream in the aspect of an imageview encoder that encodes raw images into deep features, a view projector module that transformsthese features into 3D ego-car coordinate, and an encoder that ﬁnally encodes the features into thebird’s eye view (BEV) space.
Revised: 2 (top), we detail the design of our camera stream in the aspect of an imageview encoder that encodes raw images into deep features, a view projector module that transformsthese features into 3D ego-car coordinate, and an encoder that ﬁnally encodes the features into thebird’s eye view (BEV) space.

Original: Image-view Encoder aims to encode the input images into semantic information-rich deep features.
Revised: Image-view Encoder aims to encode the input images into semantic information-rich deep features.

Original: It consists of a 2D backbone for basic feature extraction and a neck module for scale variate objectrepresentation.
Revised: It consists of a 2D backbone for basic feature extraction and a neck module for scale variate objectrepresentation.

Original: Different from LSS [32] which uses the convolutional neural network ResNet [12] asthe backbone network, we use the more representative one, Dual-Swin-Tiny [24] as the backbone.
Revised: Different from LSS [33] which uses the convolutional neural network ResNet [13] asthe backbone network, we use the more representative one, Dual-Swin-Tiny [25] as the backbone.

Original: Following [32], we use a standard Feature Pyramid Network (FPN)
Revised: Following [33], we use a standard Feature Pyramid Network (FPN)

Original: [25] on top of the backbone toexploit the features from multi-scale resolutions.
Revised: [26] on top of the backbone toexploit the features from multi-scale resolutions.

Original: To better align these features, we ﬁrst propose asimple feature Adaptive Module (ADP) to reﬁne the upsampled features.
Revised: To better align these features, we ﬁrst propose asimple feature Adaptive Module (ADP) to reﬁne the upsampled features.

Original: Speciﬁcally, we apply anadaptive average pooling and a 1 ⇥ 1 convolution for each upsampled feature before concatenating.
Revised: Speciﬁcally, we apply anadaptive average pooling and a 1 ⇥ 1 convolution for each upsampled feature before concatenating.

Original: See Appendix Sec.
Revised: See Appendix Sec.

Original: A.1 for the detailed module architecture.
Revised: A for the detailed module architecture.

Original: View Projector Module.
Revised: View Projector Module.

Original: As the image features are still in 2D image coordinate, we design a viewprojector module to transform them into 3D ego-car coordinate.
Revised: As the image features are still in 2D image coordinate, we design a viewprojector module to transform them into 3D ego-car coordinate.

Original: We apply 2D !
Revised: We apply 2D !

Original: 3D view projectionproposed in [32] to construct the Camera BEV feature.
Revised: 3D view projectionproposed in [33] to construct the Camera BEV feature.

Original: The adopted view projector takes the imageview feature as input and densely predicts the depth through a classiﬁcation manner.
Revised: The adopted view projector takes the imageview feature as input and densely predicts the depth through a classiﬁcation manner.

Original: Then, accordingto camera extrinsic parameters and the predicted image depth, we can derive the image-view featuresto render in the predeﬁned point cloud and obtain a pseudo voxel V 2 R X ⇥ Y ⇥ Z ⇥ C .
Revised: Then, accordingto camera extrinsic parameters and the predicted image depth, we can derive the image-view featuresto render in the predeﬁned point cloud and obtain a pseudo voxel V 2 R X ⇥ Y ⇥ Z ⇥ C .

Original: BEV Encoder Module.
Revised: BEV Encoder Module.

Original: To further encode the voxel feature V 2 R X ⇥ Y ⇥ Z ⇥ C into the BEV spacefeature ( F Camera 2 R X ⇥ Y ⇥ C Camera ), we design a simple encoder module.
Revised: To further encode the voxel feature V 2 R X ⇥ Y ⇥ Z ⇥ C into the BEV spacefeature ( F Camera 2 R X ⇥ Y ⇥ C Camera ), we design a simple encoder module.

Original: Instead of applying poolingoperation or stacking 3D convolutions with stride 2 to compress z dimension, we adopt the Spatial to
Revised: Instead of applying poolingoperation or stacking 3D convolutions with stride 2 to compress z dimension, we adopt the Spatial to

Original: Channel (S2C) operation [53] to transform V from 4D tensor to 3D tensor V 2 R X ⇥ Y ⇥ ( ZC ) via
Revised: Channel (S2C) operation [53] to transform V from 4D tensor to 3D tensor V 2 R X ⇥ Y ⇥ ( ZC ) via

Original: [Figure]
Revised: [Figure]

Original: Adaptive FeatureSelection
Revised: Adaptive Feature

Original: [Figure]
Revised: [Figure]

Original: reshaping to preserve semantic information and reduce cost.
Revised: reshaping to preserve semantic information and reduce cost.

Original: We then use four 3 ⇥ 3 convolution layersto gradually reduce the channel dimension into C Camera and extract high-level semantic information.
Revised: We then use four 3 ⇥ 3 convolution layersto gradually reduce the channel dimension into C Camera and extract high-level semantic information.

Original: Different from LSS [32] which extracts high-level features based on downsampled low-resolutionfeatures, our encoder directly processes full-resolution Camera BEV features to preserve the spatialinformation.
Revised: Different from LSS [33] which extracts high-level features based on downsampled low-resolutionfeatures, our encoder directly processes full-resolution Camera BEV features to preserve the spatialinformation.

Original: 3.2 LiDAR stream architecture: From point clouds to BEV space
Revised: 3.2 LiDAR stream architecture: From point clouds to BEV space

Original: Similarly, our framework can incorporate any network that transforms LiDAR points into BEVfeatures, F LiDAR 2 R X ⇥ Y ⇥ C LiDAR , as our LiDAR streams.
Revised: Similarly, our framework can incorporate any network that transforms LiDAR points into BEVfeatures, F LiDAR 2 R X ⇥ Y ⇥ C LiDAR , as our LiDAR streams.

Original: A common approach is to learn a parameterized voxelization [62] of the raw points to reduce the Z-dimension and then leverage networksconsisting of sparse 3D convolution [55] to efﬁciently produce the feature in the BEV space.
Revised: A common approach is to learn a parameterized voxelization [62] of the raw points to reduce the Z-dimension and then leverage networksconsisting of sparse 3D convolution [55] to efﬁciently produce the feature in the BEV space.

Original: Inpractice, we adopt three popular methods, PointPillars [19], CenterPoint [58] and TransFusion [2] asour LiDAR stream to showcase the generalization ability of our framework.
Revised: Inpractice, we adopt three popular methods, PointPillars [20], CenterPoint [57] and TransFusion [2] asour LiDAR stream to showcase the generalization ability of our framework.

Original: 3.3 Dynamic fusion module
Revised: 3.3 Dynamic fusion module

Original: To effectively fuse the BEV features from both camera ( F Camera 2 R X ⇥ Y ⇥ C Camera ) and LiDAR( F LiDAR 2 R X ⇥ Y ⇥ C LiDAR ) sensors, we propose a dynamic fusion module in Fig.
Revised: To effectively fuse the BEV features from both camera ( F Camera 2 R X ⇥ Y ⇥ C Camera ) and LiDAR( F LiDAR 2 R X ⇥ Y ⇥ C LiDAR ) sensors, we propose a dynamic fusion module in Fig.

Original: Given two featuresunder the same space dimension, an intuitive idea is to concatenate them and fuse them with learnablestatic weights.
Revised: Given two featuresunder the same space dimension, an intuitive idea is to concatenate them and fuse them with learnablestatic weights.

Original: Inspired by Squeeze-and-Excitation mechanism [13], we apply a simple channelattention module to select important fused features.
Revised: Inspired by Squeeze-and-Excitation mechanism [14], we apply a simple channelattention module to select important fused features.

Original: Our fusion module can be formulated as:
Revised: Our fusion module can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where [ · , · ] denotes the concatenation operation along the channel dimension.
Revised: where [ · , · ] denotes the concatenation operation along the channel dimension.

Original: f static is a static channeland spatial fusion function implemented by a 3 ⇥ 3 convolution layer to reduce the channel dimensionof concatenated feature into C LiDAR .
Revised: f static is a static channeland spatial fusion function implemented by a 3 ⇥ 3 convolution layer to reduce the channel dimensionof concatenated feature into C LiDAR .

Original: With input feature F 2 R X ⇥ Y ⇥ C LiDAR , f adaptive is formulated as:
Revised: With input feature F 2 R X ⇥ Y ⇥ C LiDAR , f adaptive is formulated as:

Original: [Equation]
Revised: [Equation]

Original: where W denotes linear transform matrix (e.g., 1x1 convolution), f avg denotes the global averagepooling and  denotes sigmoid function.
Revised: where W denotes linear transform matrix (e.g., 1x1 convolution), f avg denotes the global averagepooling and  denotes sigmoid function.

Original: 3.4 Detection head
Revised: 3.4 Detection head

Original: As the ﬁnal feature of our framework is in BEV space, we can leverage the popular detection headmodules from earlier works.
Revised: As the ﬁnal feature of our framework is in BEV space, we can leverage the popular detection headmodules from earlier works.

Original: This is further evidence of the generalization ability of our framework.
Revised: This is further evidence of the generalization ability of our framework.

Original: Inessence, we compare our framework on top of three popular detection head categories, anchor-based[19], anchor-free-based [58], and transform-based [2].
Revised: Inessence, we compare our framework on top of three popular detection head categories, anchor-based[20], anchor-free-based [57], and transform-based [2].

Original:  Experiments
Revised:  Experiments

Original: In this section, we present our experimental settings and the performance of BEVFusion to demonstrate the effectiveness, strong generalization ability, and robustness of the proposed framework.
Revised: In this section, we present our experimental settings and the performance of BEVFusion to demonstrate the effectiveness, strong generalization ability, and robustness of the proposed framework.

Original: 4.1 Experimental settings
Revised: 4.1 Experimental settings

Original: Dataset.
Revised: Dataset.

Original: We conduct comprehensive experiments on a large-scale autonomous-driving dataset for3D detection, nuScenes [3].
Revised: We conduct comprehensive experiments on a large-scale autonomous-driving dataset for3D detection, nuScenes [3].

Original: Each frame contains six cameras with surrounding views and one pointcloud from LiDAR.
Revised: Each frame contains six cameras with surrounding views and one pointcloud from LiDAR.

Original: There are up to 1.4 million annotated 3D bounding boxes for 10 classes.
Revised: There are up to 1.4 million annotated 3D bounding boxes for 10 classes.

Original: Weuse nuScenes detection score (NDS) and mean average precision (mAP) as evaluation metrics.
Revised: Weuse nuScenes detection score (NDS) and mean average precision (mAP) as evaluation metrics.

Original: See
Revised: See

Original: Appendix A.2 for more details.
Revised: Appendix B.1 for more details.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Implementation details.
Revised: Implementation details.

Original: We implement our network in PyTorch using the open-sourced MMDe-
Revised: We implement our network in PyTorch using the open-sourced MMDe-

Original: [Table]
Revised: [Table]

Original: tection3D [8].
Revised: tection3D [8].

Original: We conduct BEVFusion with Dual-Swin-Tiny [24] as 2D bakbone for image-viewencoder.
Revised: We conduct BEVFusion with Dual-Swin-Tiny [25] as 2D bakbone for image-viewencoder.

Original: PointPillars [19], CenterPoint [57]
Revised: PointPillars [20], CenterPoint [57]

Original: [Table]
Revised: [Table]

Original: and TransFusion-L [2] are chosen as our LiDAR streamand 3D detection head.
Revised: and TransFusion-L [2] are chosen as our LiDAR streamand 3D detection head.

Original: We set the image size to 448 ⇥ 800 and the voxel size following the ofﬁcialsettings of the LiDAR stream [19, 57, 2].
Revised: We set the image size to 448 ⇥ 800 and the voxel size following the ofﬁcialsettings of the LiDAR stream [20, 57, 2].

Original: Our training consists of two stages: i)
Revised: Our training consists of two stages: i)

Original: We ﬁrst train the
Revised: We ﬁrst train the

Original: LiDAR stream and camera stream with multi-view image input and LiDAR point clouds input, respectively.
Revised: LiDAR stream and camera stream with multi-view image input and LiDAR point clouds input, respectively.

Original: Speciﬁcally, we train both streams following their LiDAR ofﬁcial settings in MMDetection3D[8]; ii) We then train BEVFusion for another 9 epochs that inherit weights from two trained streams.
Revised: Speciﬁcally, we train both streams following their LiDAR ofﬁcial settings in MMDetection3D[8]; ii) We then train BEVFusion for another 9 epochs that inherit weights from two trained streams.

Original: Note that no data augmentation (i.e., ﬂipping, rotation, or CBGS [64]) is applied when multi-viewimage input is involved.
Revised: Note that no data augmentation (i.e., ﬂipping, rotation, or CBGS [64]) is applied when multi-viewimage input is involved.

Original: During testing, we follow the settings of LiDAR-only detectors [19, 58, 2]in MMDetection3D [8] without any extra post-processing.
Revised: During testing, we follow the settings of LiDAR-only detectors [20, 57, 2]in MMDetection3D [8] without any extra post-processing.

Original: See Appendix Sec.
Revised: See Appendix Sec.

Original: A.2 for the detailedhyper-parameters and settings.
Revised: B.1 for the detailedhyper-parameters and settings.

Original: 4.2 Generalization ability
Revised: 4.2 Generalization ability

Original: To demonstrate the generalization ability of our framework, we adapt three popular LiDAR-onlydetector as our LiDAR stream and detection head, PointPillars [19], CenterPoint [58] and TransFusionL [2], as described in Sec. 3.
Revised: To demonstrate the generalization ability of our framework, we adapt three popular LiDAR-onlydetector as our LiDAR stream and detection head, PointPillars [20], CenterPoint [57] and TransFusionL [2], as described in Sec. 3.

Original: If not speciﬁed, all experimental settings follow their original papers.
Revised: If not speciﬁed, all experimental settings follow their original papers.

Original: In
Revised: In

Original: Table 1, we present the results of training two single modality streams, followed by jointly optimized.
Revised: Table 1, we present the results of training two single modality streams, followed by jointly optimized.

Original: Empirical results show that our BEVFusion framework can signiﬁcantly boost the performanceof these LiDAR-only methods.
Revised: Empirical results show that our BEVFusion framework can signiﬁcantly boost the performanceof these LiDAR-only methods.

Original: Despite the limited performance of the camera stream, our fusionscheme improves PointPillars by 18.4% mAP and 10.6% NDS, and CenterPoint and TransFusion-Lby a margin of 3.0% ⇠ 7.1% mAP.
Revised: Despite the limited performance of the camera stream, our fusionscheme improves PointPillars by 18.4% mAP and 10.6% NDS, and CenterPoint and TransFusion-Lby a margin of 3.0% ⇠ 7.1% mAP.

Original: This evidences that our framework can generalize to multiple
Revised: This evidences that our framework can generalize to multiple

Original: LiDAR backbone networks.
Revised: LiDAR backbone networks.

Original: As our method relies on a two-stage training scheme, we nonetheless report the performance of asingle stream in the bottom part of Table 1.
Revised: As our method relies on a two-stage training scheme, we nonetheless report the performance of asingle stream in the bottom part of Table 1.

Original: We observe that the LiDAR stream constantly surpassesthe camera stream by a signiﬁcant margin.
Revised: We observe that the LiDAR stream constantly surpassesthe camera stream by a signiﬁcant margin.

Original: We ascribe this to the LiDAR point clouds providingrobust local features about object boundaries and surface normal directions, which are essential foraccurate bounding box prediction.
Revised: We ascribe this to the LiDAR point clouds providingrobust local features about object boundaries and surface normal directions, which are essential foraccurate bounding box prediction.

Original: FOV ( −"/2 ， "/2 )
Revised: FOV ( −"/2 ， "/2 )

Original: Objects failure
Revised: Objects failure

Original: B E V F u s i on
Revised: B E V F u s i on

Original: T r a n s
Revised: T r a n s

Original: F u s i on
Revised: F u s i on

Original: G r ound t r u t h
Revised: G r ound t r u t h

Original: Limited FOV Objects failure ( − ! " , ! " )
Revised: Limited FOV Objects failure ( − ! " , ! " )

Original: (a) Two Failure Scenarios (b) Visualization of Model Predictions
Revised: (a) Two Failure Scenarios (b) Visualization of Model Predictions

Original: FOV (
Revised: FOV (

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: 4.3 Comparing with the state-of-the-art methods
Revised: 4.3 Comparing with the state-of-the-art methods

Original: Here, we use TransFusion-L as our LiDAR stream and present the results on the test set of nuScenesin Table 2.
Revised: Here, we use TransFusion-L as our LiDAR stream and present the results on the test set of nuScenesin Table 2.

Original: Without any test time augmentation or model ensemble, our BEVFusion surpasses allprevious LiDAR-camera fusion methods and achieves the state-of-the-art performance of 69.2% mAPcomparing to the 68.9% mAP of TransFusion[2].
Revised: Without any test time augmentation or model ensemble, our BEVFusion surpasses allprevious LiDAR-camera fusion methods and achieves the state-of-the-art performance of 69.2% mAPcomparing to the 68.9% mAP of TransFusion[2].

Original: Note that we do not conduct data augmentationwhen multi-view image input is involved, while data augmentation plays a critical part in other cuttingedge methods.
Revised: Note that we do not conduct data augmentationwhen multi-view image input is involved, while data augmentation plays a critical part in other cuttingedge methods.

Original: It is worth noticing that the original TransFusion[2] is a two-stage detector, whosemodel consists of two independent detection heads.
Revised: It is worth noticing that the original TransFusion[2] is a two-stage detector, whosemodel consists of two independent detection heads.

Original: By contrast, our BEVFusion with TransFusion-Las LiDAR backbone only contains one detection head, yet still outperforms the two-stage baseline by0.3% mAP.
Revised: By contrast, our BEVFusion with TransFusion-Las LiDAR backbone only contains one detection head, yet still outperforms the two-stage baseline by0.3% mAP.

Original: As the only difference between our framework and TransFusion is the fusion mechanism,we ascribe this performance gain to a comprehensive exploration of the multi-modality modelingpower of BEVFusion.
Revised: As the only difference between our framework and TransFusion is the fusion mechanism,we ascribe this performance gain to a comprehensive exploration of the multi-modality modelingpower of BEVFusion.

Original: 4.4 Robustness experiments
Revised: 4.4 Robustness experiments

Original: Here, we demonstrate the robustness of our method against all previous baseline methods on twosettings, LiDAR and camera malfunctioning.
Revised: Here, we demonstrate the robustness of our method against all previous baseline methods on twosettings, LiDAR and camera malfunctioning.

Original: See [1] for more details.
Revised: See [1] for more details.

Original: 4.4.1 Robustness experiments against LiDAR Malfunctions
Revised: 4.4.1 Robustness experiments against LiDAR Malfunctions

Original: To validate the robustness of our framework, we evaluate detectors under two LiDAR malfunctions:i) when the LiDAR sensor is damaged or the LiDAR scan range is restricted, i.e., semi-solid lidars;ii) when objects cannot reﬂect LiDAR points.
Revised: To validate the robustness of our framework, we evaluate detectors under two LiDAR malfunctions:i) when the LiDAR sensor is damaged or the LiDAR scan range is restricted, i.e., semi-solid lidars;ii) when objects cannot reﬂect LiDAR points.

Original: We provide visualization of these two failure scenariosin Fig.
Revised: We provide visualization of these two failure scenariosin Fig.

Original: 4 (a) and evaluate detectors on nuScenes validation set.
Revised: 4 (a) and evaluate detectors on nuScenes validation set.

Original: Data augmentation for robustness.
Revised: Data augmentation for robustness.

Original: We propose two data augmentation strategies for above twoscenarios: i) we simulate the LiDAR sensor failure situation by setting the points with limited
Revised: We propose two data augmentation strategies for above twoscenarios: i) we simulate the LiDAR sensor failure situation by setting the points with limited

Original: Field-of-View (FOV) in range (  ⇡ / 3 , ⇡ / 3 ), (  ⇡ / 2 , ⇡ / 2 ).
Revised: Field-of-View (FOV) in range (  ⇡ / 3 , ⇡ / 3 ), (  ⇡ / 2 , ⇡ / 2 ).

Original: ii)
Revised: ii)

Original: To simulate the object failure, we use adropping strategy where each frame has a 0.5 chance of dropping objects, and each object has a 0.5chance of dropping the LiDAR points inside it.
Revised: To simulate the object failure, we use adropping strategy where each frame has a 0.5 chance of dropping objects, and each object has a 0.5chance of dropping the LiDAR points inside it.

Original: Below, we ﬁnetune detectors with these two dataaugmentation strategies, respectively.
Revised: Below, we ﬁnetune detectors with these two dataaugmentation strategies, respectively.

Original: LiDAR sensor failure.
Revised: LiDAR sensor failure.

Original: The nuScenes dataset provides a Field-of-View (FOV) range of (  ⇡ ,⇡ ) for LiDAR point clouds.
Revised: The nuScenes dataset provides a Field-of-View (FOV) range of (  ⇡ ,⇡ ) for LiDAR point clouds.

Original: To simulate the LiDAR sensor failure situation, we adopt the ﬁrstaforementioned robust augmentation strategy in Table 3.
Revised: To simulate the LiDAR sensor failure situation, we adopt the ﬁrstaforementioned robust augmentation strategy in Table 3.

Original: Obviously, the detector performancedegrades as the LiDAR FOV decreases.
Revised: Obviously, the detector performancedegrades as the LiDAR FOV decreases.

Original: However, when we fuse camera stream, with the presenceof corruptions, the BEVFusion models, in general, are much more robust than their LiDAR-only
Revised: However, when we fuse camera stream, with the presenceof corruptions, the BEVFusion models, in general, are much more robust than their LiDAR-only

Original: [Table]
Revised: [Table]

Original: Here, we report the results of baseline and our method that trained on the nuScenes dataset with and without the proposed robustness augmentation (Aug.).
Revised: Here, we report the results of baseline and our method that trained on the nuScenes dataset with and without the proposed robustness augmentation (Aug.).

Original: All settings are the same as in Table
Revised: All settings are the same as in Table

Original: [Table]
Revised: [Table]

Original: counterparts, as shown in Fig.
Revised: counterparts, as shown in Fig.

Original: 4 (b).
Revised: 4 (b).

Original: Notably, for PointPillars, the mAP increases by 24.4% and 25.1%when LiDAR FOV in (  ⇡ / 2 , ⇡ / 2 ), (  ⇡ / 3 , ⇡ / 3 ), respectively.
Revised: Notably, for PointPillars, the mAP increases by 24.4% and 25.1%when LiDAR FOV in (  ⇡ / 2 , ⇡ / 2 ), (  ⇡ / 3 , ⇡ / 3 ), respectively.

Original: As for TransFusion-L, BEVFusionimproves its LiDAR stream by a large margin of over 18.6% mAP and 5.3% NDS.
Revised: As for TransFusion-L, BEVFusionimproves its LiDAR stream by a large margin of over 18.6% mAP and 5.3% NDS.

Original: The vanilla
Revised: The vanilla

Original: LiDAR-camera fusion approach [2] proposed in TransFusion (denoted as LC in Table 3 and Table 4)heavily relies on LiDAR data, and the gain is limited to less than 3.3% mAP while NDS is decreased.
Revised: LiDAR-camera fusion approach [2] proposed in TransFusion (denoted as LC in Table 3 and Table 4)heavily relies on LiDAR data, and the gain is limited to less than 3.3% mAP while NDS is decreased.

Original: The results reveal that fusing our camera stream during training and inference compensates for thelack of LiDAR sensors to a substantial extent.
Revised: The results reveal that fusing our camera stream during training and inference compensates for thelack of LiDAR sensors to a substantial extent.

Original: LiDAR fails to receive object reﬂection points.
Revised: LiDAR fails to receive object reﬂection points.

Original: Here exist common scenarios when LiDARfails to receive points from the object.
Revised: Here exist common scenarios when LiDARfails to receive points from the object.

Original: For example, on rainy days, the reﬂection rate of somecommon objects is below the threshold of LiDAR hence causing the issue of object failure [1].
Revised: For example, on rainy days, the reﬂection rate of somecommon objects is below the threshold of LiDAR hence causing the issue of object failure [1].

Original: To simulate such a scenario, we adopt the second aforementioned robust augmentation strategyon the validation set.
Revised: To simulate such a scenario, we adopt the second aforementioned robust augmentation strategyon the validation set.

Original: As shown in Table 4, when we directly evaluate detectors trained withoutrobustness augmentation, BEVFusion shows higher accuracy than the LiDAR-only stream and vanilla
Revised: As shown in Table 4, when we directly evaluate detectors trained withoutrobustness augmentation, BEVFusion shows higher accuracy than the LiDAR-only stream and vanilla

Original: LiDAR-camera fusion approach in TransFusion.
Revised: LiDAR-camera fusion approach in TransFusion.

Original: When we ﬁnetune detectors on the robust augmentedtraining set, BEVFusion largely improves PointPillars, CenterPoint, and TransFusion-L by 28.9%,22.7%, and 15.7% mAP.
Revised: When we ﬁnetune detectors on the robust augmentedtraining set, BEVFusion largely improves PointPillars, CenterPoint, and TransFusion-L by 28.9%,22.7%, and 15.7% mAP.

Original: Speciﬁcally, the vanilla LiDAR-camera fusion method in TransFusion has again of only 2.6% mAP, which is smaller than the performance before ﬁnetuning, we hypothesizethe reason is that the lack of foreground LiDAR points brings wrong supervision during trainingon the augmented dataset.
Revised: Speciﬁcally, the vanilla LiDAR-camera fusion method in TransFusion has again of only 2.6% mAP, which is smaller than the performance before ﬁnetuning, we hypothesizethe reason is that the lack of foreground LiDAR points brings wrong supervision during trainingon the augmented dataset.

Original: The results reveal that fusing our camera stream during training andinference largely compensates for the lack of object LiDAR points to a substantial extent.
Revised: The results reveal that fusing our camera stream during training andinference largely compensates for the lack of object LiDAR points to a substantial extent.

Original: We providevisualization in Fig.
Revised: We providevisualization in Fig.

Original: 4 (b).
Revised: 4 (b).

Original: 4.4.2 Robustness against camera malfunctions
Revised: 4.4.2 Robustness against camera malfunctions

Original: We further validate the robustness of our framework against camera malfunctions under three scenariosin [1]: i) front camera is missing while others are preserved; ii) all cameras are missing except forthe front camera; iii) 50% of the camera frames are stuck.
Revised: We further validate the robustness of our framework against camera malfunctions under three scenariosin [1]: i) front camera is missing while others are preserved; ii) all cameras are missing except forthe front camera; iii) 50% of the camera frames are stuck.

Original: As shown in Table 5, BEVFusion stilloutperforms camera-only [52] and other LiDAR-camera fusion methods [46, 41, 2] under abovescenarios.
Revised: As shown in Table 5, BEVFusion stilloutperforms camera-only [52] and other LiDAR-camera fusion methods [46, 42, 2] under abovescenarios.

Original: The results demonstrate the robustness of BEVFusion against camera malfunctions.
Revised: The results demonstrate the robustness of BEVFusion against camera malfunctions.

Original: 4.5 Ablation
Revised: 4.5 Ablation

Original: Here, we ablate our design choice of the camera stream and the dynamic fusion module.[Table] [Table]
Revised: Here, we ablate our design choice of the camera stream and the dynamic fusion module.

Original: [Table]

Original: Components for camera stream.
Revised: Components for camera stream.

Original: We conduct ablation experiments to validate the contribution ofeach component of our camera stream using different components in Table 6.
Revised: We conduct ablation experiments to validate the contribution ofeach component of our camera stream using different components in Table 6.

Revised: The naive baseline

Original: The naive baselinewith ResNet50 and feature pyramid network as multi-view image encoder, ResNet18 as BEV encoderfollowing LSS [32], PointPillars [19] as detection head only obtains 13.9% mAP and 24.5% NDS.
Revised: [Table] [Table] [Table] with ResNet50 and feature pyramid network as multi-view image encoder, ResNet18 as BEV encoderfollowing LSS [33], PointPillars [20] as detection head only obtains 13.9% mAP and 24.5% NDS.

Original: As shown in Table 6, there are several observations: (i) when we replace the ResNet18 BEV encoderwith our simple BEV encoder module, the mAP and NDS are improved by 4.0% and 2.5%.
Revised: As shown in Table 6, there are several observations: (i) when we replace the ResNet18 BEV encoderwith our simple BEV encoder module, the mAP and NDS are improved by 4.0% and 2.5%.

Original: Adding the adaptive feature alignment module in FPN helps improve the detection results by 0.1%.(3)
Revised: Adding the adaptive feature alignment module in FPN helps improve the detection results by 0.1%.(3)

Original: Concerning a larger 2D backbone, i.e., Dual-Swin-Tiny, the gains are 4.9% mAP and 4.0% NDS.
Revised: Concerning a larger 2D backbone, i.e., Dual-Swin-Tiny, the gains are 4.9% mAP and 4.0% NDS.

Original: The camera stream equipped with PointPillars ﬁnally achieves 22.9% mAP and 31.1% NDS, showingthe effectiveness of our design for the camera stream.
Revised: The camera stream equipped with PointPillars ﬁnally achieves 22.9% mAP and 31.1% NDS, showingthe effectiveness of our design for the camera stream.

Original: Dynamic fusion module.
Revised: Dynamic fusion module.

Original: To illustrate the performance of our fusing strategy for two modalities,we conduct ablation experiments on three different 3D detectors, PointPillars, CenterPoint, and
Revised: To illustrate the performance of our fusing strategy for two modalities,we conduct ablation experiments on three different 3D detectors, PointPillars, CenterPoint, and

Original: TransFusion.
Revised: TransFusion and the LiDAR stream as baseline.

Original: As shown in 7, with a simple channel& spatial fusion (left part in Fig. 3), BEVFusiongreately improves its LiDAR stream by 16.5% ( 35 . 1% !
Revised: As shown in 7, with a simple channel& spatial fusion(left part in Fig. 3), BEVFusion greately improves its LiDAR stream by 16.5% ( 35 . 1% !

Original: 6% ) mAP for PointPillars, 5.9%( 57 .
Revised: 6% )mAP for PointPillars, 5.9% ( 57 .

Original: 63 . 0% ) mAP for CenterPoint, and 2.4% ( 64 . 9% !
Revised: 63 . 0% ) mAP for CenterPoint, and 2.4% ( 64 . 9% !

Original: 3% ) mAP for TransFusion.
Revised: 3% )mAP for TransFusion.

Original: Whenadaptive feature selection (right part in Fig. 3) is adopted, the mAP can be futher improved by1.9%, 1.2%, and 0.6% mAP for PointPillars, CenterPoint, and TransFusion, respectively.
Revised: When adaptive feature selection (right part in Fig. 3) is adopted, the mAP canbe futher improved by 1.9%, 1.2%, and 0.6% mAP for PointPillars, CenterPoint, and TransFusion,respectively.

Original: The resultsdemonstrate the necessity of fusing camera and LiDAR BEV features and effectiveness of our Dynamic Fusion Module in selecting important fused features.
Revised: The results demonstrate the necessity of fusing camera and LiDAR BEV features andeffectiveness of our Dynamic Fusion Module in selecting important fused features.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we introduce BEVFusion, a surprisingly simple yet unique LiDAR-camera fusionframework that disentangles the LiDAR-camera fusion dependency of previous methods.
Revised: In this paper, we introduce BEVFusion, a surprisingly simple yet unique LiDAR-camera fusionframework that disentangles the LiDAR-camera fusion dependency of previous methods.

Original: Ourframework comprises two separate streams that encode raw camera and LiDAR sensor inputs intofeatures in the same BEV space, followed by a simple module to fuse these features such that they canbe passed into modern task prediction head architectures.
Revised: Ourframework comprises two separate streams that encode raw camera and LiDAR sensor inputs intofeatures in the same BEV space, followed by a simple module to fuse these features such that they canbe passed into modern task prediction head architectures.

Original: The extensive experiments demonstrate thestrong robustness and generalization ability of our framework against the various camera and LiDARmalfunctions.
Revised: The extensive experiments demonstrate thestrong robustness and generalization ability of our framework against the various camera and LiDARmalfunctions.

Original: We hope that our work will inspire further investigation of robust multi-modality fusionfor the autonomous driving task.
Revised: We hope that our work will inspire further investigation of robust multi-modality fusionfor the autonomous driving task.

Original: Broader Impacts Statement and Limitations.
Revised: Broader Impacts Statement and Limitations.

Original: This paper studies robust LiDAR-camera fusion for3D object detection.
Revised: This paper studies robust LiDAR-camera fusion for3D object detection.

Original: Since the detection explored in this paper is for generic objects and does notpertain to speciﬁc human recognition, so we do not see potential privacy-related issues.
Revised: Since the detection explored in this paper is for generic objects and does notpertain to speciﬁc human recognition, so we do not see potential privacy-related issues.

Original: However,the biased-to-training-data model may pose safety threats when applied in practice.
Revised: However,the biased-to-training-data model may pose safety threats when applied in practice.

Original: The researchmay inspire follow-up studies or extensions, with potential applications in autonomous driving tasks.
Revised: The researchmay inspire follow-up studies or extensions, with potential applications in autonomous driving tasks.

Original: While our study adopts a simple camera stream as the baseline, we also encourage the community toexpand the architecture, e.g, with temporal multi-view camera input.
Revised: While our study adopts a simple camera stream as the baseline, we also encourage the communityto expand the architecture, e.g, with temporal multi-view camera input and alignment betweenintermediate LiDAR and camera features.

Original: We leave the extension of ourmethod towards building such systems for future work.
Revised: We leave the extension of our method towards buildingsuch systems for future work.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to 462 answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to 461 answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: You arestrongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of 464 your paper or providing a brief inline description.
Revised: You arestrongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of 463 your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes]
Revised: [Yes]

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.•
Revised: [No] The code and the data are proprietary.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the Checklist 469 section does not count towards the page limit.
Revised: Note that the Checklist 468 section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keepthe Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keepthe Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] See Sec. 1.(b)
Revised: [Yes] See Sec. 1.(b)

Original: Did you describe the limitations of your work?
Revised: Did you describe the limitations of your work?

Original: [Yes] See Sec.
Revised: [Yes] See Sec.

Original: 4.2 and Broader Impacts Statementand Limitations.(c)
Revised: 4.2 and Broader Impacts Statementand Limitations.(c)

Original: Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Broader ImpactsStatement and Limitations.(d)
Revised: [Yes] See Broader ImpactsStatement and Limitations.(d)

Original: Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]Personally identiﬁable information is blurred for privacy protection.2.
Revised: [Yes]Personally identiﬁable information is blurred for privacy protection.2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b)
Revised: [N/A](b)

Original: Did you include complete proofs of all theoretical results?
Revised: Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?

Original: [Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Sec.
Revised: [Yes] See Sec.

Original: 4.1 and Appendix Sec.
Revised: 4.1 and Appendix Sec.

Original: A.2.
Revised: B.1.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experimentsmultiple times)?

Original: [N/A] Our experiments are stable with multiple runs.
Revised: [N/A] Our experiments are stable with multiple runs.

Original: All results are observedvia a ﬁxed seed.
Revised: All results are observedvia a ﬁxed seed.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,

Original: internal cluster, or cloud provider)?
Revised: internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Sec. 4.1.
Revised: [Yes] See Sec. 4.1.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No] The data used in our paper are publicly released.
Revised: [No] The data used in our paper are publicly released.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] See
Revised: [Yes] See

Original: Appendix.
Revised: Appendix.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re us-
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re us-

Original: ing/curating?
Revised: ing/curating?

Original: [Yes] The data used in our paper are publicly released.
Revised: [Yes] The data used in our paper are publicly released.

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable in-
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable in-

Original: formation or offensive content?
Revised: formation or offensive content?

Original: [Yes] The data used in our work does not contain personally
Revised: [Yes] The data used in our work does not contain personally

Original: identiﬁable information or offensive content.
Revised: identiﬁable information or offensive content.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB)
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB)

Original: approvals, if applicable?
Revised: approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on

Original: participant compensation?
Revised: participant compensation?

Original: [N/A]
Revised: [N/A]