Is Your LiDAR Placement Optimized for 3D Scene Understanding?
Authors: Ye Li
Lingdong Kong
Hanjiang Hu
Xiaohao Xu
Xiaonan Huang
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Autonomous Driving
LiDAR Semantic Segmentation
Sensor Placement
Primary area: machine_vision
TL;DR: Place3D is a full-cycle pipeline that encompasses LiDAR placement optimization, data generation, and downstream evaluations.

Abstract: The reliability of driving perception systems under unprecedented conditions is crucial for practical usage. Latest advancements have prompted increasing interest in multi-LiDAR perception. However, prevailing driving datasets predominantly utilize single-LiDAR systems and collect data devoid of adverse conditions, failing to capture the complexities of real-world environments accurately. Addressing these gaps, we proposed Place3D, a full-cycle pipeline that encompasses LiDAR placement optimization, data generation, and downstream evaluations. Our framework makes three appealing contributions. 1) To identify the most effective configurations for multi-LiDAR systems, we introduce the Surrogate Metric of the Semantic Occupancy Grids (M-SOG) to evaluate LiDAR placement quality. 2) Leveraging the M-SOG metric, we propose a novel optimization strategy to refine multi-LiDAR placements. 3) Centered around the theme of multi-condition multi-LiDAR perception, we collect a 280,000-frame dataset from both clean and adverse conditions. Extensive experiments demonstrate that LiDAR placements optimized using our approach outperform various baselines. We showcase exceptional results in both LiDAR semantic segmentation and 3D object detection tasks, under diverse weather and sensor failure conditions.

Is Your LiDAR Placement Optimized for 3D Scene Understanding?
Ye Li!
Lingdong Kong²
Hanjiang Hu
Xiaohao Xu
Xiaonan Huang'
"University of Michigan, Ann Arbor
Nattoonal University of Singapore
*Carnegie Mellon University
Abstract
The reliability of driving perception systems under unprecedented conditions is crucial for practical usage. Latest advancements have prompted increasing interest in multi-LiDAR perception. However, prevailing driving datasets predominantly utilize single-LiDAR systems and collect data devoid of adverse conditions, f'illing to capture the complexities of real-world environments accurately. Addressing these gaps, we proposed Place3D, a full-cycle pipeline that encompasses Li- DAR placement optimization, data generation, and downstream evaluations. Our framework makes three appealing contributions. 1) To identify the most effective configurations for multi-LiDAR systems, we introduce the Surrogate Metric of the Semantic Occupancy Grids (M-SOG) to evaluate LiDAR placement quality. 2) Leveraging the M-SOG metric, we propose novel optimization strategy to refine multi-L.iDAR placements. 3) Centered around the theme of multi-condition multi-LiDAR perception, we collect a 280,000-frame dataset from both clean and adverse conditions. Extensive experiments demonstrate that LiDAR placements optimized using our approach outperform various baselines. We showcase excep- tional results both LiDAR semantic segmentation and 3D object detection tasks, under diverse weather and sensor failure conditions.
Introduction
Accurate 3D perception plays crucial role in autonomous driving, involving detecting the objects around the vehicle and segmenting the scene into meaningful semantic categories. LiDARs are be- coming crucial for driving perception due to their capability to capture detailed geometric information about the surroundings [3, 109, 53, 54, 118]. While the latest models achieved promising accuracy on standard datasets, e.g., nuScenes [7] and SemanticKITTI [6], improving the resilience of perception under corruptions and sensor failures is still critical yet under-explored task [46, 94, 95].
Recent studies have primarily focused on refining the sensing systems by designing new algorithms with novel model architectures [15] or representations [28, 99, 11]. The selection of LiDAR configurations, however, often relies on industry experience and design aesthetics. Therefore, existing literature has potentially overlooked optimal LiDAR placements for maximum sensing efficacy.
An intuitive approach for optimizing LiDAR configurations involves a comprehensive cycle of data collection, model training, and validation across various LiDAR setups enhance the autonomous driving system's 's perception accuracy [24, 52]. However, this approach faces significant challenges due to the substantial computational resources and extensive time required for data collection and processing [57, 17]. Although recent works [34, 65] made preliminary attempts to explore the impact
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

of LiDAR placements on perception accuracy, they have neither proposed an optimization method nor evaluated the performance adverse conditions.
In this work, we delve into the optimization of sensor configurations for autonomous vehicles by tackling two critical sub-problems: 1) the performance evaluation of sensor-specific configurations, and 2) the optimization of these configurations for enhanced 3D perception tasks, encompassing 3D object detection and LiDAR semantic segmentation. To achieve this goal, we propose a systematic LiDAR placements evaluation and optimization framework, dubbed Place3D, The overall pipeline endowed with the capability to synthesize point cloud data with customizable configurations and diverse conditions, including common corruptions, external disturbances, and sensor failures.
masy-to-compure Surrogate Metrie of Semantic Occupancy Grids (M-SOG) to evaluate the equality of LiDAR placements. Next, we propose a novel optimization approach utilizing surrogate metric based on Covariance Matrix Adaptation Evolution Strategy (CMA-ES) find the near-optimal LiDAR placements. To verify the correlation between our surrogate metric and assess the effectiveness of our optimization approach on both clean and adverse conditions, we design an automated multi-condition multi-LiDAR data simulation platform and establish a comprehensive benchmark consisting of a total of seven LiDAR placement baselines inspired by existing self-driving configuration from the autonomous vehicle companies.
We first introduce
Our benchmark, along with large-scale multi-condiine multi-LiDAR perception dataset, encom- passes state-of-the-art learning- based perception models for 3D object detection [51, 107, 63, 109] and LiDAR semantic segmentation [18, 11,, 86, 117,
well as six distinct adverse conditions coped with weather and sensor failures [46, 95]. Utilizing the proposed framework, we explored how
various perturbations and downstream 3D perception tasks affect optimization outcomes.
To summarize, this work makes the following key contributions:
To the best of our knowledge, Place3D serves as the first attempt at investigating the impact of multi-LiDAR placements for 3D semantic scene understanding in diverse conditions.
We introduce M-SOG, innovative surrogate metric to effectively evaluate the quality of LiDAR placements for both detection (sparse 3D) and segmentation (dense 3D) tasks.
We propose a novel optimization approach utilizing our surrogate metric refine LiDAR placements, which exhibit excellent LiDAR semantic segmentation and 3D object detection accuracy and robustness, outperforming baselines for relatively 9% in metrics.
We contribute lish a comprchensive benchmark for LiDAR- 3D scene understanding evaluations. We hope this work can lay solid foundation for future research in this relevant field.
multi-condition multi-LiARR point cloud dataset and estab-
2 Related Work
LiDAR Sensing. LiDAR sensing is critical in autonomous vehicles, providing essential structural
information for their operation [59, 3, 73]. Utilizing 3D data, LiDAR supports tasks such as semantic
scene understanding [7, 26, 29, 6, 85, 48, 10), generation [75, 120, 74, 97], decision making [25, 8, 19, 87], and simulation [23, 68, 67, 93]. This work specifically targets LiDAR-based perception and simulation, which are at the forefront of current research in autonomous vehicle technology.
LiDAR-Based 3D Scene Understanding, LiDAR segmentation and 3D object detection are key tasks for 3D scene understanding. Segmentation models are categorized into point-base, [88, 35, 36, 110, 77], range view [70, 90, 21, 98, 113, 16, 47, 2, 45, 99], bird's eye view [111, 115, 12], voxel-based [18, 86, 117, 33]. and view fusion [58, 38, 78, 60, 62, 14, 13, 61, 100] methods. While they benchmarks, their performance across different LiDAR configurations [82, 106, 105, 116] or voxel-based [114, 51, 69, 92, 83, 55, 66, 104] representations, with recent works favoring fusion for improved detection [64, 80, 81, 63, 56]. Our study complements these approaches by optimizing LiDAR placements enhance performance under various real-world conditions.
show promising results
is not well explored. For 3D object detection, models typically use point-
LiDAR Perception Robustness. The reliability of HIDAR-based 3D scene understanding models in real-world scenarios is crucial [84, 39]. Recent studies have explored the robustness of 3D perception models against adversarial attacks [112, 96], common corruptions [46, 49, 94, 103, 43, 4], adverse
weather [31, 30, 79, 37], sensor failures [108, 15, 28], and combined motion and sensor perturbations

---

Surrogate Metric of Semantic Occupancy Grids (M-SOG)
Optimizing LiDAR Placements
Laser Ray
Map PSDG
M-SDG
H
W
Covered Voxels
[a] 3D Driing
(b) Probabilistic Semantic
[c] Compute Entropy Obtain M-50G
[d] Evolution Optimization
Scenes
Occupancy Grids
(e) Mult--Conditinn
3D Detaction & Segmentation Model Training
Perception
Baseline
Muli-LIDAR
Performance
LIDAR Placement
CARLA
Dataset
Eveluation
Figure 1:
Place3D pipeline for multi-LiDAR placement optimization. We first utilize the semantic point cloud synthesized in CARLA (a) to generate Prohabilistic SOG (b) and obtain voxels covered by LiDAR rays to compute M-SOG (c). We propose a CMA-ES- optimization strategy to maximize M-SOG, finding optimal LiDAR placement (d). To verify the effectiveness of our LiDAR placement optimization strategy, we contribute a multi-condition multi-LiDAR dataset (e) and evaluate the performance of baselines and optimized placements on both clean and corruption data (1).
[102, 95]. To our knowledge, no prior work has focused on optimizing LiDAR placement to improve semantic 3D scene understanding robustness. Our Place3D addresses this gap by studying various placement strategies to enhance robustness in both in-domain and out-of-domain scenarios.
Sensor Placement Optimization. Optimizing sensor placements can enhance performance and address the challenges of heuristic design [42, 101]. In autonomous driving, optimizing LiDAR place- ments is relatively new [65]. Hu et al. [34] studied multi-LiDAR placements for 3D object detection, while Li et al. [57] examined LiDAR-camera configurations for multi-modal detection. Other works [41, 44, 9, 40] explored roadside LiDAR placements for V2X applications. Distinguishing from these efforts, our work is the first to investigate LiDAR placements for robust 3D scene understanding in challenging conditions. We establish benchmarks for both LiDAR semantic segmentation and 3D object detection, providing an in-depth analysis of optimal placements for robust perception.
Place3D: A Full-Cycle LiDAR Placement Pipeline
In this section, we introduce the Surrogate Metric of Semantic Occupancy Grids (M-SOG) assess the 3D perception performance of sensor configurations (Section 3.2). Leveraging M-SOG, we propose novel optimization strategy for multi-LiDAR placement (Section 3.3). Our approach is theoretically grounded; we provide an optimality certification to ensure that optimized placements are close to the global optimum (Section 3.4). Figure depicts overview of Place3I) framework.
3.1 Preliminary Formulation of Surrogate Metric
Region of Interest. Following the literature of sensor placement [34, 57J, we define the Region of Interest (ROI) for perception as finite 3D cuboid space [1, w, with ego-vehicle in the center, and divide the ROI space into voxels with resolution [5), 8 dr. as S = {vi, U2, UN }, where
N p on denotes the total number of voxels.
Probabilistic Occupancy Grids (POG). We define POG as the joint probability of all non-zero voxels in the ROI as prog = p(v,, o2, vN) I p(n))= P(v:), where each voxel is assumed to be independently and identically distributed. The voxel 8; is occupied it is within any 3D bounding box be of object class at frame t, which is denoted as v; = (t) Therefore, the probability of being occupied for voxel u; among all T frames can be estimated p(v.) Σt [(: )
Probabilistic Surrogate Metric. Now given some LiDAR placement I., the conditional POG can be found the conditional joint distribution PPOGIL p(U1, U2, ..., ( in the literature [34].
3

---

To show how much uncertainty the placement L can reduce in the ROI, the naive surrogate metric S-MIG [34] is introduced the reduction of the entropy rorlLL E'= ( L) with con- stant HroG = - ((ss), where (v),, H (m)]..) can be found through the entropy of Bernoulli distribution p(v:), p(zs) [L), respectively.
3.2 M-SOG: Surrogate Metric of Semantic Occupancy Grids
The naive surrogate metric S-MIG [34] employs 3D bounding boxes as priors understand 3D scene distribution. However, this approach exhibits substantial deviations from the actual physical boundaries of the objects and overlooks the occlusion relationships between objects and the environ- ment in LiDAR applications. Furthermore, it is limited to detection tasks only. To overcome these limitations, we propose the Surrogate Metric of Semantic Occupancy Grids to assess and optimize LiDAR placement for 3D scene understanding tasks. Leveraging the Semantic Occupancy Grids derived from diverse environments, our approach can be expanded to tackling adverse conditions.
Semantic Occupancy Grids (SOG). Given a 3D driving scene with a set of semantic classes Y {y/, 1/2: ... War where M represents the total number of semantic classes in the scene and assuming that each voxel can only be occupied by one semantic tag for each frame, we denote y (v;) as the semantic class of voxel e E {U1, U2,..., .... UN) J at time frame t E {1, 2, T). Notably, empty voxels are also considered a semantic class. Accordingly, the set of voxels occupied by semantic
class ye at frame I is defined as 5.. = (u:ly(9) (vx) = ye), c 1, 2, .... M, L = 1,2, T. Then, we introduce SOG describe the total semantic voxel distribution:
SOG = ( S S ya ). t = I......,
Probabilistic Semantic Occupancy Grids. In contrast the Bernoulli distribution POG [34], we propose more accurate Multinomial distribution: Probabilistic Semantic Occupancy Grids (P-SOG), denoted as psog, represent the probability distribution of voxels belonging certain semantic classes. Before estimating PSOG, we first traverse all frames from given scenes to obtain the probability و for each voxel v2 occupied by each semantic class ye:
p(u: = yc) H[t E (1, E s(U) H
c 1,2 M.
(2)
T
Notably, we denote voxel iy being occupied by semantic class ye as v/ ye and indicates estimated distributions from observed samples, whereas p refers to the statistical parameters be estimated, which are unknown and non-random. We compute the joint probability of all non-zero voxels in the ROI to estimate the psog. Following the literature [34], the joint distribution over the voxel set S is:
PSOG =p(v1, 02 U2,...UNN
II
p(v;)
f=1, p(v.)
The uncertainty of Probabilistic Semantic Occupancy Grids reflects the 3D scene understanding capability of sensors within a given scene. To quantify this uncertainty, we calculate the entropy for the
probability distribution of each voxel v; in ÅsoG as: H (v:) = EM p(v: = n.)) log p(v; (1 = uc)-
Based on the independent and identically distributed assumption of SOG, the entropy of P-SOG over the voxel set S is given by: HooG Eup-ps Σ (w;),, where ps denotes the PsoG over the voxel set S. From the perspective of density estimation, the true P-50G and its corresponding entropy can be estimated as psoG = psoG and HSOG = HsoG. respectively.
Surrogate Metric of Semantic Occupancy Grids. To evaluate LiDAR placements, we further analyze the joint probability distribution of voxels covered by LiDAR rays with varied LiDAR placements. Leveraging the Amanatides and Woo's 's Fast Voxel Traversal Algorithm [1], we obtain the voxels covered by LiDAR rays as S L) FastVT((S L,) {0'²" .o2 by wk)) given LiDAR configuration L Lj. j 1, 2, ..... J, where j indexes the LiDAR placements, J is the number of total configurations, and N, is the number of voxels covered by rays of LiDAR configuration L = Lj- Then, the semantic entropy distribution of P-SOG over the voxel set S|L, can be estimated as:
N
H H. = E 0y -PsLL
Σ
(4)

---

Algorithm Multi-LiDAR Placement Optimization for Semantic 3D Scene Understanding
Initialize: k <- 0, (n(0))(()) C NR. Ms. vk E 20,1, K)
2: for k == 0, 0... K do
3: for irri= N₂ do
4: Sample (()) ~ N(( (4), (o(1)))(()) from d-density gird-level candidates
5: Calculate G(u!.))
6: end for
ù, 8: Update ((++1) and (k++1 via Equation (7), (8), (9), and (10)
7: Update m (k+1) based the top Mr best solutions (k) via Equation (6)
9:
end for
(a) Center
(b) Line
(c) Pyramid
(d) Square
(c) Trapezoid
(f) Line-roll
(g) Pyramid-roll
Ours (Det)
Ours (Seg)
(j) Ours (2D)
Figure 2: Visualized LiDAR Placements. We compare the LiDAR placements optimized from our proposed M-SOG metric (for LiDAR semantic segmentation and 3D object detection) and heuristic LiDAR placements utilized by major autonomous vehicle companies (see Appendix Section B.
We further define the information gain of scene understanding as AH = SoG H Soo to describe the perception capability of LiDAR configuration Ly. Since HSOG constant given certain 3D semantic scenes with the fixed P-SOG distribution, we propose the normalized Surrogate Metric of Semantic Occupancy Grids (M-SOG) as follows to evaluate the perception capability:
MSoG (L2) H
1
E of PSIL
Σ H(
N,
Nj
N, Σ
pl ye) logg(v bo =ye)
(5)
N
3.3 Sensor Configuration Optimization
We adopt the heuristic optimization based on the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [32] to find an optimized LiDAR configuration.
We define the objective F(u,) Msoo(L,), where u, l R represents the LiDAR configuration Lj and is subject to the physical constraint P(u,) = 0, and P(u) > if it is violated, e.g., mutual distance between LiDARs and distance from 2D plane. In our optimization, uj includes the 3D coordinates and rolling angles LiDARs, 24 is the finite cuboid space above the vehicle roof. Without ambiguity, we omit subscript j in the following text. We then transform the constrained optimization into the unconstrained Lagrangian form as G(u)) F()) + AP()), where À is the Lagrange multiplier. We optimize G(u) through an iterative process that adapts the distribution of candidate solutions as Algorithm 1.
Objective Function.
Optimization Approach. We first define a multivariate normal distribution N ((m (k), ( ((()), where (*), (8), and (k) are the mean vector, step size, and covariance matrix of the distribution of iteration k. respectively. We discretize the configuration space 2/ with density d and sample Nu
(k),
5

---

(k) (mm (o ((()))))))) in each iteration k; where i indexes the candidates. We candidates ui
update mean vector m +++ for the next iteration k + 1 as the updated center of the search distribution for LiDAR configuration. The overall process can be depicted as follows:
(k+1)
Σ w.u (k) G(ul G(ü (8) S .... S..
(6)
where Mg is the number best solutions adopt to generate m (k+1), and wy are the weights based on solution fitness. We obtain the evolution path Pc direction of successful steps follows:
(k+1) that accumulates information about the
(b++) (R)
(k+1) PC = -1 -1-cc) PC + cc)2
(k)
My
(kk
where cc is the learning rate for the covariance matrix update. The covariance matrix C controls the shape and orientation of the search distribution for LiDAR configurations. can be updated at each iteration & in the following format:
(k++1) (1- cc) + ccpC
(++1)
Pc
(K+1)
(8)
Similarly, we update Pa as the evolution path for step size adaptation. Then, the global step size can be found below for the scale of search to balance exploration and exploitation:
(k+1)
(())
(++1) (1 Dd
Co )o (k)
+ V
(1
c²)
m
M Gi=l
((k)
(k+1) - exp
Co
(10)
do
(0.
where ca is the learning rate for updating the evolution path Pa. d, is normalization factor to calibrate the pace at which the global step size is adjusted.
3.4 Theoretical Analysis
Once the evolution optimization empirically converges, it holds that the optimized solution is the local optima of the 5-density Grids space of LiDAR configuration space. In this section, we provide a stronger optimality certification to theoretically ensure that the optimized placement is close to the global optimum under the assumption of bounded and smooth objective function. Due to space limits, the full proof has been attached to the Appendix Section E.
Theorem (Optimality Certification). Given the continuous objective function RR" - R. with Lipschitz constant kG w.r.t input l C R under lz norm, suppose over ô-density Grids subset S C u, the distance between the maximal and minimal of function G over S is upper-bounded by Cm, and the local optima is u arg millues G(x), the following optimality certification regarding
x E U holds that:
G("") G( & Cas + kgō,
(11)
where u² is the global optima over
The global optimality certification Theorem is applicable in practice because the Lipschitz constant kG and the distance between the maximal and minimal of objective function G over S can be approximated easily through calculating (u (2) of Algorithm for each sampled u over the density Grids subset. Besides, we have more general corollary below to further relax the assumption of bounded objective function.
When U is a hyper-rectangle with the bounded E norm of domain U, at each dimension i, with i = 1, 1,2,....n, Theorem I can hold in a more general way by only assuming that the Lipsehitz constant ke of the objective function is given, where Equation (11) becomes:
Corollary 1.
[[G(a") G(u's)l22 S kG
U + k
(12)

---

[b] Dense
[c] Semantic
Point Cloud
Dcc Grids (SDG)
H
Merge
Voxelize
Compute
Probability
W
[a] Rew Point Cluud
One Subset. of All Frames
[d] Probabilistic SDG (P-SDG
Figure 3: Pipeline of Probabilistic SOG generation. We first merge multiple frames of raw point
clouds (a) into dense point clouds (b). Then, we voxelize dense point clouds into SOG, i.e., semantic occupancy grids (c), and traverse all frames of dense point clouds synthesize probabilistic SOG (d).
Table 1: Optimized M-SOG Metrics. We report the Msog scores for both detection and segmen- tation tasks in our pipeline. The scores are calculated based on the car class for detection and all semantic classes for segmentation. Lile-roll and Pyramid-rold are abbreviated as L-roll and P-roll.
Metries [Msoc]
Center
Line
Pyramid
Square
Trapezoid
I.roll
P-ool
Ours
Detection (x10 3D Segmentation (x 10-9)
-1.26
-165
1.54
-1.52
-1.41
-1.35
-1.88
-158
2.62
2.56
2.35
2.89
3.13
-.63
-122
Experiments
4.1 Benchmark Setups
Data Generation. We generate LiDAR point clouds and ground truth using CARLA [23]. We use the maps of Towns 1, 4, and 6 and set 6 ego-vehicle routes for each map. We incorporate 23 semantic classes for LiDAR semantic segmentation and 3 instance classes for 3D object detection. Data collection is performed for 10 LiDAR placements, resulting in total of 280,000 frames: 1) For each placement, we gather 340 clean scenes and 360 corrupted scenes, with each scene consisting of 40 frames. 2) The clean set comprises 13,600 frames, including 11 1,200 samples (280 scenes) for training and 2,400 samples (60 scenes) for validation. following the split ratio used in nuScenes [7]. 3) The corruption is used for robustness evaluation and contains 6 different conditions, each with 2,400 samples (60 scenes). More details are in the Appendix Section A.
LiDAR Configurations. We adopt five commonly employed heuristic LiDAR placements, which have been adopted by leading autonomous driving companies, as our baseline. These placements are represented in Figure 2a-e. Following KITTI [29], configured the LiDAR sensor with vertical field of view of [-24.8, 2.0] degrees. To achieve the Line-roll and Pyramid-roll configurations, we adjusted LiDAR roll angles for the Line and Pyramid setups, as depicted in Figure 2. We present detailed LiDAR configurations in the Appendix Section B.1.
Corruption Types. To replicate adverse conditions, synthesized six types of corrupted point clouds on the validation set of each sub-set, following the settings of the Robo3D benchmark [46]. These corruptions
be categorized into 1) severe weather conditions, including "snow". "fog", and "wet ground", 2) external disturbances, including "motion blur", and 3) internal sensor failures, including "crosstalk" and "incomplete echo". We include the features and implementation details of corruptions in the Appendix Section B.2.
P-SOG Synthesis. We follow the steps as depicted in Figure 3 to generate semantic occupancy grids for each LiDAR scene. We first collect point clouds and semantic labels using high-resolution LiDARs and sequentially divide all samples into multiple subsets. Through the transformation of world coordinates of the ego-vehicle, frames of point clouds each subset are aggregated into one frame of dense point cloud. Then, we utilize the voting strategy to determine the semantic label for each voxel in ROI and generate the SOG. This process executed across all subsets to produce P-SOG. Notably, the POS is only generated the scenes of the training set.
Surrogate Metric SOG. We compute the scores of M-SOG separately for 3D detection and segmentation, as shown in Table 1. To evaluate M-SOG for segmentation, we utilize all semantic classes to generate semantic occupancy grids. For detection, we specifically focus on the car semantic type, while merging the remaining semantic classes into single category for M-SOG analysis.
7

---

Table 2: Performance evaluations of LiDAR semantic segmentation under clean and adverse conditions. For each LiDAR placement strategy, we report the mloU scores (1), represented in percentage (%). The average scores only include adverse scenarios.
Center SPV
Line SPV
Pyramid SPV
Square SPV
Method
Mink
Cy3D
Mink
Cy3D
Mink
CyAD
Mink
Cy3D
Clean
65.7
67.1
72.7
39.7
59.3
68.9
62.7
67.6
68.1
60.7
63.4
69.9
Fog
55.9
39.3
55.6
51.7
12.8
55.5
52.9
18.6
51.0
55.6
10.7
52.0
Wet Ground
63.8
66.6
64.4
60.2
57.9
66.4
66.6
52.2
61.9
64.3
55.6
Snow
25.1
35.6
16.7
35.5
31.3
4.7
25.2
30.2
5.0
33.5
38.3
2.7
Motion Blur
35.8
35.6
37.6
52.0
46.1
39.4
50.7
55.1
42.5
51.5
53.9
44.2
Crosttak
24.7
19.5
36.9
27.1
13.6
34.3
17.3
14.8
26.6
26.5
18.6
37.1
Incomplete Echo
64.5
66.8
71.5
59.2
57.1
68.3
65.9
60.9
61.2
63.7
68.7
Average
45.0
43.9
47.1
47.6
41.5
44.8
46.9
39.7
48.4
46.6
43.4
Trapezoid SPV
Line-Roll SPV
Pyramid-Roll
Method
Mink
Cy3D
Mink
Cy3D
Mink
SPV
Cy3D
Mink
SPV
Cy3D
Clean
59.0
61.0
68.5
58.5
60.6
69.8
62.2
67.9
69.3
66.5
68.3
73.0
Fog c
49.7
40.9
52.1
48.6
42.2
19.7
62.2
47.2
G0.7
59.5
59.1
57.6
Wet Ground
61.
39.2
67.1
66.6
Snow
27.6
33.6
3.1
26.9
27.0
2.6
26.6
31.6
2.1
17.6
24.0
5.9
Motion Blur
19.1
36.7
30.1
52.5
56.5
56.7
56.0
Crosstalk
18.4
16.9
30.0
21.2
16.5
27.3
19.3
13.7
31.9
24.5
18.7
41.0
Incomplete Fcho
59.3
60.7
65.6
58.0
61.0
67.8
60.8
60.7
70.0
66.9
06.9
63.3
Average
44.5
43.8
42.0
44.1
43.1
41.7
45.4
47.1
44.5
48.6
48.6
47.3
Table 3: Performance evaluations of 3D object detection under clean and adverse conditions. For each LiDAR placement strategy, we report the mAP scores (t) for the car class, represented in percentage (%). The average scores only include adverse scenarios.
Center
Line
Pyramid Center
Square Center
Method
Center
BEV
Pillar
Center
BEV
Pillar
Clean
46.5
55.8
52.5
43.4
54.0
40.3
46.1
55.9
51.0
43.8
54.0
49.2
Fug
15.1
20.2
26.0
20.8
17.2
Wet Ground
36.3
17.3
36.8
39.6
19.2
38.0
38.7
19.6
38.1
40.0
541.2
39.1
27.0
33.6
36.1
21.1
15.0
32.5
Motion Blur
27.1
27.3
12.8
27.1
9.7
23.6
29.2
28.6
29.1
26.1
13.0
23.6
Crosstalk
24.9
23.7
22.3
Incomplete Echo
26.2
22.3
20.9
25.2
21.5
21.6
25.8
25.9
21.6
25.7
2-66
22.
Average
28.4
20.8
26.3
22.6
28.9
21.7
27.3
22.3
Trapezoid
Line-Roll
Pyramid-Roll Center
Ours Center
Method
BEV
Pillar
BEV
Pillar
Clean
43.5
56.4
50.2
44.6
55.2
50.8
46.1
56.2
50.7
46.8
67.1
53.0
Fog
22.1
19.6
22.8
18.3
Wet Ground
40.0
51.7
39.22
40.3
49.6
38.3
39..1
49.2
40.3
40.1
51.1
41.3
33.7
5.2
36.4
Motion Blur
25.9
15.3
26.9
26.9
15.5
19.5
27.3
29.0
30.3
26.5
27.7
32.0
Crosstalk
27.2
20.0
25.1
26.4
22.9
Incomplete Echo
24.9
23.9
22.6
25.3
23.2
21.9
25.2
25.5
21.8
25.8
23.6
22.7
Average
26.9
28.1
Detector and Segmentor: For the benchmark, we conduct experiments using four LiDAR semantic
segmentation models, i.e., MinkUNet [18], SPVCNN [86], PolarNet [1 [111], and Cylinder3D [117], and four 3D object detection models, iee., PointPillars [51], CenterPoint [107], BEVFusion-L. [63],
and FSTR [109]. The detailed training configurations are included in the Appendix Section B.3,
4.2 Comparative Study
We conduct benchmark studies to evaluate the performance varied LiDAR placements in both clean and adverse conditions. We extensively examine the correlation between the proposed surrogate metric, known as M-SOG, and the final performance results. Through our analysis, we are able to demonstrate the effectiveness and robustness of the entire Place3D framework.
Effectiveness of M-SOG Surrogate Metric in Place3D. In Table 1, we report the scores of M-SOG surrogate metric for various LiDAR placements. In Tables and 3, we benchmark the 3D object detection and LiDAR semantic segmentation performance of varied LiDAR placements with state-of-tl oott----rtttttt-- algorithms. Figure 4 illustrates the correlation between the proposed M-SOG

---

72
SPOONN
78
68
86.5
67.8
68.3
70.0
65.
67.8
70.9
62.7
67.1
09.
72.7
872
87.7
69.3
71.0
62.2
000
6100
68
68
60.0
68.8
67.7
M
68.0
1
58.3
1503(10
(000
25
20
-15
-10
20
-15
-.0
(a) PolarNet
(b) Min
(c) SPVCNN
(d) Cylinder3D
Figure 4: The correlation between M-SOG and LiDAR semantic segmentation [18, 86, 111, 117] models performance in the clean condition.
MAP
68.0
MAP
MAP [P]]
46.5
48.0
46.1
52.5
D2
46
50.8
48.1
46.1
49.3
50.2
50.7
50.2
50.0
44.8
44.6
49.2
48.3
43.5
43.8
(10
[MG
MS03109
SMV5
(10*)
47
48
42
.15
-15
-14
.55
50
1.7
-88
.15
5.5
-30
45
BEVFusion (M-SOG)
(b) BEVFusion (S-MIG)
(c) PointPillars (M-SOG)
(d) PointPillars (S-MIG)
Figure 5: Comparisons of M-SOG with S-MIG [34] using BEVFusion-L [63] and PointPillars [51].
surrogate metric and perception capacity. The results demonstrate a clear correlation, where the performance generally improves the M-SOG increases. While there might be fluctuations in some placements with specific algorithms, the overall relationship follows a linear correlation, highlighting the effectiveness our M-SOG for computationally efficient sensor optimization purposes. We show more plots in the Appendix Sections C.I and C.2.
Comparisons to Existing Sensor Placement Methods. The effectiveness of LiDAR sensor place- ment metrics lies in the linear correlation between metrics and actual performance. We set the same ROI size and conduct guantitative comparison with S-MIG [34] on 3D detection (as S-MIG [34] only works for 3D detection task). As shown in Figure 5, our metric exhibits better linear correlation. Since introduce semantic occupancy information more accurately characterizes the boundaries in the 3D semantic scene and addresses the issue of objects and environment occlusions when using 3D bounding boxes priors. Moreover, our LiDAR placement method makes the first attempt to include both detection and segmentation tasks. We present additional comparisons the Appendix Section C.3.
prior for the evaluation metric, method
Superiority of Optimization via Place3D. As shown in Tables 2 and 3, our optimized configurations achieved the best performance in both segmentation and detection tasks under both clean and adverse conditions among all models. We report per-class quantitative results in the Appendix Sections C. and C.2. The improvement from optimization remains significant even when comparing our optimized configurations against the best rest-performing baseline in each task. For segmentation, optimized placements outperform the persormrforinn baseline by up to 0.8% on clean datasets and by as much as 1. 1.5% on corruption. For detection, optimized configurations exceed the best pest-rerfonmnn baseline by up to 0.7% clean datasets and by up to 0.3% corruption.
Robustness of Optimization via Place3D. The M-SOG utilizes semantic occupancy as prior knowl- edge, which is invariant to changing conditions, thereby enabling robustness of optimized placement in adverse weather. Under the corrupted setting, although the correlation between M-SOG and perception performance is not obvious that in the clean setting and shows some fluctuation, our optimized LiDAR configuration consistently maintained its performance in adverse conditions, mirroring its effectivvnees in the clean condition. While the top-performing baseline LiDAR config- urations in clean datasets might be notably worse compared to others when faced with corruption, the optimized configuration via Place3D consistently shows the best performance under adverse conditions. We showcase several qualitative results on this aspect in the Appendix Section D.
Intuitive Interpretation. We observe several intuitive reasons why refined sensor placement is beneficial our experiments. 1) LiDAR heights. Increasing the average height of the LiDARS improves performance, likely due to expanded field of view. 2) Variation heights. A greater

---

aDui
WOis (Chrmq Cyanoud
78
TTappzzid
20
Trepeoid
BSquarg
47.4
48.B
48
48.
73
63
88
880
71
924
59
05 .
60
703
41
33
40.8
00
07.6
41.55
64.8
415.
58
35
Polerfeet MilUNN SPVONN Cylnder3D
PdeeNet
MirktNie
SPVONN
Dyinder3D
PolerNit
MrkUNNt
SPVONN
Oyinde33
Against Corrupt
(b) 2D Placements (Clean)
(c) 2D Placements (Comuup
Figure 6: Ablation results (mloU) of placement strategies on segmentation models [111, 18, 86, 117].
difference in LiDAR heights enhances perception, as varied height distributions capture richer features from the sides of objects. 3) Uniform distribution. The pyramid placement performs better in segmentation, as a more spread-owl and uniform distribution of LiDARs captures a detailed surface.
4.3 Ablation Study
In this section, we further analyze the interplay between our proposed optimization strategy and perception performance to address two questions: 1) How does our optimization strategy inform LiDAR placements improve robusiness against various forms of corruption? 2) How optimization strategy enhance perception performance in scenarios with limited placement options?
Optimizing Against Corruptions. In the previous subsection, we deploy the placement optimized on the clean point clouds directly on corrupted ones assess the robustness. To further showcase the capability of our method against corruptions, we compute the M-SOG scores utilizing the P- SOG derived from the corrupted semantic point clouds under adverse conditions and perform the optimization process, as shown in Figure 6a. Quantitatively compared with Ours in Table 2, the configurations derived from optimization on adverse data outperform the configurations generated solely on clean data. These results indicate that customizable optimization tailored for adverse settings is an effective approach to enable robust 3D perception.
Optimizing 2D Placements. Placing LiDARs at different heights on the roof might affect automotive aesthetics and aerodynamics. Therefore, we investigate the effect of our optimization algorithm in the presence of constraints, by limiting the LiDARs to the same height and considering only 2D placements. We fix the height of all LiDARs find the optimal placement on the horizontal plane of the vehicle roof. We use the Line, Square, and Trapezoid placements in Figure 2 for comparison. Experimental results Figures 6b and 6C show that our optimized LiDAR placements outperform baseline placements at the same LiDAR height on both clean and corruption settings.
Influence of LiDAR Roll Angles. Fine-tuning the orientation angles of the LiDARs on autonomous vehicles is an effective strategy to minimize blind spots and broaden the perception range. For 3D segmentation, Pyramid-roll slightly outperforms Pyramid, whereas Line-roll falls short of Line, which is highly aligned with scores of M-SOG in Table 1. For 3D detection tasks, the situation is reversed, but still consistent with the M-SOG results. This suggests that adjusting the LiDAR's ' angle can have varying impacts on the performance of 3D object detection and LiDAR semantic segmentation.
5 Conclusion
In this work, we presented Place3I), a comprehensive and systematic LiDAR placement evaluation and optimization framework. We introduced the Surrogate Metric of Semantic Occupancy Grids (M-SOG) as novel metric to assess the impact of various LiDAR placements. Building on this metric, we culminated an optimization approach for LiDAR configuration that significantly enhances LiDAR semantic segmentation and 3D object detection performance. We validate the effectiveness of optimization approach through a comprehensive benchmark for evaluating both baseline and our optimized LiDAR placements on detection and segmentation in diverse conditions. Our optimized placements demonstrate superior robustness and perception capabilities, outperforming all baseline configurations. By shedding light on refining the robustness of LiDAR placements for both tasks under diverse driving conditions, anticipate that our work will pave the way for further advancements in this field of research.
condition muulii-LiDAY point cloud dataset and establish
10

---

Acknowledgments
This work was supported by the Robotics Department at the University of Michigan, Ann Arbor.
References
[1] John Amanatides and Andrew Woo. fast voxel traversal algorithm for ray tracing. Eurographics, 87(3);3-10, 1987.
[2] Angelika Ando, Spyros Gidaris, Andrei Bursuc, Gilles Puy, Alexandre Boulch, and Renaud Marlet. Rangevit: Towards vision transformers for 3d semantic segmentation in autonomous driving. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5240-5250, 2023.
[3] Claudine Badue, Rånik Guidolini, Raphael Vivacqua Carneiro, Pedro Azevedo, Vinicius B. Cardoso, Avelino Forechi, Luan Jesus, Rodrigo Berriel, Thiago M. Paixão, Filipe Mutz, Lucas de Paula Veronese, Thiago Saveira-Santos, and Alberto Souza, Self-driving cars: survey. Expert Systems with Applications, 168:113816, 2021.
[4] Till Beemelmanns, Quan Zhang, and Lutz Eckstein. Multicorrupt: A multi-modal robustness dataset and benchmark of lidar-camera fusion for 3d object detection. In IEEE Intelligent Vehicles Symposium, pages 9255-3261. 2024.
[5] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Jürgen Gall, and Cyrill Stachniss. Towards 3d lidar-based semantic scene understanding of 3d point cloud sequences: The semantickitti dataset. International Journal of Robotics Research, 40:959-96, 96, 2021.
[6] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Juergen Gall. Semantickiti: A dataset for semantic scene understanding of lidar sequences. In /EEE/CVF International Conference on Computer Vision, pages 9297-9307, 2019.
[7] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora. Venice Erin Liong, Qiang Xu, Anush Krishnan. Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11621-11631, 2020.
[8] Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang. Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml aal-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106. 11810. 2021.
[9] Xinyu Cai, Wentao Jiang, Runsheng Xu, Wenquan Zhao, Jiaqi Ma, Si Liu, and Yikang Li. Analyzing infrastructure lidar placement with realistic lidar simulation library. In International Conference on Robotics and Automation, pages 15877, 2023.
[10] Anh-Quan Cao and Raoul De Charette. Monoscene: Monocular 3d semantic scene completion. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3991-4001, 2022.
[11] Anh- nah-Quan Cao, Angela Dai, and Raoul de Charette. Pasco: Urban 3d panoptic scene completion with uncertainty awareness. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14554-14564, 2024.
[12] Qi Chen, Sourabh Vora, and Oscar Beijbom. Polarstream: Streaming lidar object detection and seg- mentation with polar pillars. In Advances in Neural Information Processing Systems, volume 34, pages 26871-26883, 2021.
[13] Runnan Chen, Youquan Liu, Lingdong Kong, Nenglun Chen, Xinge 7hu, Yuexin Ma, Tongliang Liu, and Wenping Wang. Towards label-free scene understanding by vision foundation models. In Advances in Neural Information Processing Systems, volume 36, pages 75896-75910, 75910, 2023.
[14] Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. Clip2scene: Towards label-efficient 3d scene understanding by clip. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7020-7030, 2023.
[15] Siran Chen, Yue Ma, Qiao, and Yali Wang. M-bev: Masked bev perception for robust autonomous driving. In AAAI Conference on Artificial Intelligence, pages 1183-1191, 2023.
[16] Huixian Cheng, Xianfeng Han, and Guoqiang Xiao. Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving. In IEEE International Conference on Multimedia and Expo, pages 1-6, 2022.

---

[17] Huixian Cheng, Xianfeng Han, and Guoqiang Xiao. Transrvnet: Lidar semantic segmentation with transformer. IEEE Transactions Intelligent Transportation Systems, 2466):5855-5977, 2023.
[18] Christopher Choy, Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075-3084, 2019.
[19] Laurene Claussmann, Marc Revilloud, Dominique Gruyer, and Sébastien Glaser. A review of motion planning for highway autonomous driving. IEEE Trunsactions on Intelligent Transportation Systems. 21(5):1826-1848, 2019.
[20] MMDetection33 Contributors. MMDetection3D: OpenMMLab next-generation platform for general 3D object detection, https://github. com/ comm/open--ae papeen-mlaane wtw.aaaammeettttiiiooo.o on3d, 2020.
[21] Tiago Cortinhal, George Tzelepis, and Eren Erdal Aksoy. Salsanext: Fast, uncertainty-awan semantic segmentation of lidar point clouds. In International Symposium on Visual Computing, pages 207-222, 2020.
[22] Cruise. San francisco seeks stop sign driverless cars. httpt//eeeee222/////vvveeeeee -ar.aaaaocaaa... Accessed: 2024-03-06, 2022.
[23] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. Conference on Robot Learning, pages 1-16, 2017.
[24] Bertrand Douillard, James Underwood, Noah Kuntz, Vsevolod Vlaskine, Alastair Quadros, Peter Morton, and Alon Frenkel. On the segmentation of 3d lidar point clouds. In IEEE International Conference on Robotics and Automation, pages 2798-2805, 2011.
[25] Scott Ettinger, Shuyang Cheng. Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp. Charles R. Qi, Yin Zhou, Zoey Yang, Aurélien Chouard, Pei Sun, Jiquan Ngiam, Vijay Vasudevan, Alexander McCauley, Jonathon Shlens, and Dragomir Anguelov. Large scale interactive motion forecasting for autonomous driving: The waymo open motion datasct. In JEEE/CVF International Conference on Computer Vision, pages 9710-9719, 2021.
[26] Whye Kit Fong, Rohit Mohan, Juana Valeria Hurtado, Lubing Zhou, Holger Caesar, Oscar Beijhom, and Abhinav Valada. Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking. IEFE Robotics and Automation Letters, 7:3795-3802, 2022.
[27] Ford. How ford's next-gen test vehicle lays the foundation for our self-driving business. https:
/ //n/dium...mmmmm// com/ self wwwpiiieewoooooood-eeeeeeeeeeeee nexrrdd--eet-eee gen- test-veit cl -- the lahe-foundatti foundation- or- our www-oursslf--iiweeeee drivin---nn------ www.ddiinnngnniiisseeaaaeeeeeeee bww.bbss-nnsubbb aad.ae270a.......e.66....0. Accessed: 2024-03-06, 2020.
[28] Chongjian Ge, Junsong Chen, Enze Xie, Zhongdao Wang, Lanqing Hong, Huchuan Lu, Zhenguo Li, and Ping Luo. Metabev: Solving sensor failures for 3d detection and map segmentation. In IEFF/CVF International Conference on Computer Vision, pages 8721-8731, 2023.
[29] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3354-3361, 2012.
[30] Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc Van Gool. Lidar snowfall simulation for robust 3d object detection. In IEEE/CVF Conference Computer Vision and Patten Recognition, pages 16364-16374, 2022.
[31] Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Fog simulation on real lidar point clouds for 3d object detection in adverse weather. In IEEE/CVF International Conference on Computer Vision, pages 15283-15292, 2021.
[32] Nikolaus Hansen. The ema evolution strategy: A tutorial, arXiv preprint 16Xiv: 1604.00772, 2016.
[33] Fangzhou Hong, Lingdong Kong, Hui Zhou, Xinge Zhu, Hongsheng Li, and Ziwei Liu. Unified 3d and 4d panoptic segmentation via dynamic shifting networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3480-3444 2024.
[34] Hanjiang Hu, Zuxin Liu, Sharad Chitlangia. Akhil Agnihotri, and Ding Zhao. Investigating the impact of multi-lida placement on object detection for autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2022.
12

---

[35] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang. Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11108-11117, 2020.
[36] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang. Niki Trigoni, and Andrew Markham. Learning semantic segmentation large-scale point clouds with random sampling. IEFE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8338 8354, 2022.
[37] Yihao Huang, Kaiyuan Yu, Qing Guo, Felix Juefei-Xu Xiaojun Jia, Tianlin Li, Geguang Pu, and Yang Liu. Improving robustness of lidar-camera fusion model against weather corruption from fusion strategy perspective. arXiv preprint urXiv:2402.027.66 2024.
[38] Maximilian Jaritz, Tuan-Hung Vu, Raoul de Charette, Emilie Wirbel, and Patrick Pérez. xmuda: Cross- modal unsupervised domain adaptation for 3d semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12605-12614, 2020.
[39] Peng Jiang. Philip Osteen, Maggie Wigness, and Srikanth Saripallig. Rellis-3d dataset: Data, benchmarks and analysis. IEEE International Conference on Robotics and Automation, pages 1110-1116, 2021.
[40] Wentao Jiang, Hao Xiang, Xinyu Cai, Runsheng Xu, Jiaqi Ma, Yikang Li, Gim Hee Lec, and Si Liu. Optimizing the placement of roadside lidars for autonomous driving. /EEE/CVF International Conference on Computer Vision, pages 18381-18390, 2023.
[41] Shaojie Jin, Ying Gao, Fei Hui, Xiangmo Zhao, Cheng Wei, Tao Ma, and Weihao Gan. A novel information theory-hased metric for evaluating roadside lidar placement. IEFE Sensors Journal, 22(21):21009-2100 2022.
[42] Siddharth Joshi and Stephen Boyd, Sensor selection via convex optimization. IEEE Transactions on Signal Processing, 57(2):451 462, 2008.
[43] Oguzhan Fatih Kar, Teresa Yeo, Andrei Atanov, and Amir Zamir. 3d common corruptions and data augmentation. TEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8963-18974, 2022.
pagee
[44] Tae- Tae-Hyeon Hyeong Kim. Gi-Hwan Jo, Hyeong- Seok Yun, Kyung-Su Su Yun, Tae-Hyoung Park. Plaeement method of multiple lidars for roadside infrastructure in urhan environments. Sensors, 2382))8880,, 2023.
[45] Lingdong Kong. Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou. Yu Qiao. and Ziwei Liu. Rethinking range view representation for lidar segmentation. In JEEE/CVF International Conference on Computer Vision, pages 228-240-- 2023.
[46] Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Roho3d: Towards robust and reliable 3d perception against corruptions. In IERF/CVF International Conference on Computer Vision, pages 19994- 2023.
[47] Lingdong Kong, Niamul Quader, and Veniee Erin Liong. Conda: Unsupervised domain adaptation for lidar segmentation via regularized domain concatenation. In IEEE International Conference on Robotics and Automation, pages 9338-9345, 2023.
[48] Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu. Lasermix for semi-supervised lidar semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21705- 21715, 2023.
[49] Lingdong Kong, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng. Benoit R. Cottereau, and Wei Tsang Ooi. Robodepth: Robust duitt--iitti depth estimation under corruptions. In Advances in Neural Information Processing Systems, volume 36, pages 11299-213444 2023.
[50] Lingdong Kong, Xiang Xu, Jiawei Ren, Wenwei Zhang, Liang Pan, Kai Chen, Wei Tsang Ooi, and Ziwei Liu. Multi-modal data-efficient 3d scene understanding for autonomous driving. arXiv preprint arXiv:2405.05255. 2024.
[51] Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12697-12705, 2019.
[52] Li Li, Hubert P. H. Shum, and Toby P. Breckon. Less is more: Reducing task and model complexity for 3d point cloud semantic segmentation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3661-9371, 2023.
13

---

[53] Rong Li, Raoul de Charette, and Anh-Quan Cao. Coarse3d: Class-proiotypes for contrastive learning in weakly-supervised 3d point cloud segmentation. In British Machine Vision Conference, 2022.
[54] Rong Li, Shijie Li, Xieyuanli Chen, Teli Ma, Wang Hao, Juergen Gall, and Junwei Liang. Tfnet: Exploiting temporal cues for fast and accurate lidar semantic segmentation. In IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pages 4547-4556, 2024.
[55] Xin Li, Tao Ma, Yuenan Hou, Botian Shi, Yucheng Yang, Youquan Liu, Xingjiao Wu, Qin Chen, Yikang Li, Yu Qiao, and Liang He. Logonet: Towards accurate 3d object detection with local-to-global cross-modal fusion. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 17524-17534, 2023.
[56] Xin Li, Botian Shi, Yuenan Hou, Xingjiao Wu, Tianlong Ma, Yikang Li, and Liang Hc. Homogeneous multi-modal feature fusion and interaction for 3d object detection. In European Conference on Computer Vision, pages 691-707, 2022.
[57] Ye Li. Hanjiang Hu, Zuxin Liu, Xiaohao Xu, Xiaonan Huang, and Ding Zhao. Influence of camera-lidar configuration 3d object detection for autonomous driving. In IEEE International Conference on Robotics and Automation, pages 9018-9025, 2024.
[58] Veniee Erin Liong, Thi Ngoc Tho Nguyen, Sergi Widjaja, Dhananjai Sharma, and Zhuang Jie Chong. Amvnet: Assertion- based multi-view fusion network for lidar semantic segmentation. arXiv preprint arXiv:2012.04934, 2020.
[59] Mingyu Liu, Ekim Yurtsever. Xingcheng Zhou, Jonathan Fossaert, Yuning Cui, Bare Luka Zagar, and Alois C. Knoll. A survey on autonomous driving datasets: Data statistic, annotation, and outlook. arXiv preprint arXiv:2401. 240.24001.0144, 2024.
[60] Youquan Liu, Lingdong Kong, Jun Cen, Runnan Chen, Weawei Zhang, Liang Pan, Kai Chen, and Ziwei Liu. Segment any point cloud sequences by distilling vision foundation models. In Advances in Neural Information Processing Systems, volume 36, pages 37193-37229 2023.
[61] Youquan Liu, Lingdong Kong, Xiaoyang Wu, Runnan Chen, Xin Li, Liang Pan, Ziwei Liu, and Yuexin Ma. Mulci-space alignments towards universal lidar segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14648 14661, 2024.
[62] Youquan Liu, Xin Li Runnan Chen, Lingdong Kong. Yuchen Yang, Zhaoyang Xia, Yeqi Bai, Xinge Zhu, Yuexin Ma, Yikang Li, Yu Qiao, and Yuenan Hou. Uniseg: A unified multi-modal lidar segmentation network and the openpeseg codebase. In IEEE/CVF International Conference on Computer Vision, pages 21662-21673 2023.
[63] Zhijian Liu, Haotian Tang. Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L. Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird's-eye view representation. In IEEE International Conference on Robotics and Automation, pages 2771-2781. 2023.
[64] Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han. Point-voxel enn for efficient 3d deep learning. In Advances in Neural Information Processing Systems, volume 32, pages 965-975, 2019.
[65] Zuxin Liu, Mansur Arief, and Ding Zhao. Where should we place lidars on the autonomous vehicle?- an optimal design approach. In IEEE International Conference on Robotics and Automation, pages 2793-2799, 2019.
[66] Tao Ma, Xuemeng Yang. Hongbin Zhou, Xin Li, Botian Shi, Junjie Liu, Yuchen Yang, Zhizheng Liu, Liang He, Yu Qiao, et al. Detzero: Rethinking offboard 3d object detection with long-term sequential point clouds. In IEEF/CVF International Conference on Computer Vision, pages 6736-6747, 2023.
[67] Sivabalan Manivasagam, Ioan Andrei Bårsan, Jingkang Wang, Ze Yang, and Raquel Urtasun. Towards zero domain gap: A comprehensive study of realistic lidar simulation for autonomy testing, In IEEE/CVF International Conference on Computer Vision, pages 8272- 0282, 2023.
[68] Sivabalan Manivasagam. Shenlong Wang, Kelvin Wong. Wenyuan Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang. Wei-Chio Ma. and Raquel Urtasun. Lidarsim: Realistic lidar simulation by leveraging the real world. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11167-11176, 2020.
[69] Jiageng Mao, Yujing Xue, Minzhe Niu, Haoyue Bai, Jiashi Feng, Xiaodan Liang, Hang Xu, and Chunjing Xu. Voxel transformer for 3d object detection. In IEEE/CVF International Conference on Computer Vision, pages 3164-3 3173, 2021.
14

---

[70] Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. Rangenet++: Fast and accurate lidar semantic segmentation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4213-4220, 2019.
[71] Momenta. Several key members from oppo's zeku team joins momenta. attps://pandale cow/ sev eral- eral-key- mee-mrrs--r---e- froo--roooo------ oppos--pp------ serpos--eku--- team- joins-ons---- momenta, Accessed: 2024-03-06, 2023.
[72] Motional. Uber launches hailing of motional robotaxis in las vegas. https::/www.
m//new///ee 2aeer/22222 //wp///////b/reeeeeeeeeeeeer lawnchls---ee-e hailinng wwwiiiinmmooiinn---ooiioo www.ooooa---oooooi-----------ee- Accessed: 2024-03-06, 2022.
[73] Khan Muhammad, Amin Ullah, Jaime Lloret, Javier Del Ser, and Victor Hugo C. de Albuquerque. Deep learning for safe autonomous driving: Current challenges and future directions. IEEE Transactions on Intelligent Transportation Systems, 22(7):4316-433 2020.
[74] Kazuto Nakashima, Yumi Iwashita, and Ryo Kurazume. Generative range imaging for learning scene priors of 3d lidar data. In JEEE/CVF Winter Conference on Applications of Computer Vision, pages 1256-1266, 2023.
[75] Kazuto Nakashima and Ryo Kurazume. Learning drop points for lidar synthesis. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 222-229, 2021.
[76] Pony.ai. Pony.ai begins fully driverless testing in california. buttp:://wwn wnwouousse clentt in- califor
ernational. com/ news/ / /tttttingggggeeeeeeeeeee haagg//ooyaaaaeeee a1- ai-begins- www.bbenggggn--eeeeeeeeeeeeeeeee fully-drrivereess-eee nia. html, Accessed: 2024-03-06, 2022.
[77] Gilles Puy, Alexandre Boulch, and Renaud Marlet. Using waffle iron for automotive point cloud semantic segmentation. In /EEF/CVF International Conference on Computer Vision, pages 3379-3389, 2023.
[78] Corentin Sautier, Gilles Puy, Spyros Gidaris, Alexandre Boulch, Andrei Bursuc, and Renaud Marlet. Image-to-lidar self-supervised distillation for autonomous driving data. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9891-9901, 2022.
[79] Alvari Seppänen, Risto Ojala, and Kari Tammi. 4denoisenet: Adverse weather denoising from adjacent point clouds. IEEE Robotics and Automation Letters, 8(1):456 463,,4433363 2022.
[80] Shaoshuai Shi, Chaoxu Guo, Li Jiang. Zhe Wang, Jianping Shi. Xiaogang Wang, and Hongsheng Li. Pv- renn: Point-voxel feature abstraction for 3d object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10529-10538, 2020.
[81] Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang. and Hongsheng Li. Pv-ren+++: Poxnt-voxel feature set abstraction with local vector representation for 3d object detection. International Journal of Computer Vision, 131(2):531-551 2023.
[82] Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrenn: 3d object proposal generation and detection from point cloud. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-779, 2019.
[83] Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. From points parts: 3d object detection from point cloud with part- and pagr-aggreg network. IEEE Transactions on Pattern Analysis and Machine Intelligenee 43(8):2647 26644, 2020.
[84] Ziying Song, Lin Liu, Feiyang Jia, Yadan Luo, Guoxin Zhang, Lei Yang, Li Wang. and Caiyan Jia. Robustness-aware 3d object detection in autonomous driving: A review and outlook. arXiv preprint arXiv:2401.065: 065442, 2024.
[85] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Pamaikk Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens. Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open datasct. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 2446 2454, 2020.
[86] Haotian Tang. Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han. Searching efficient 3d architectures with sparse voxel convolution. In European Conference on Computer Vision, pages 685-702. 2020.
15

---

[87] Siyu Teng, Xuemin Hu, Peng Deng, Bai Li. Yuchen Li, Yunfeng Ai, Dongsheng Yang, Lingxi Li, Zhe Xuanyuan, Fenghua Zhu, and Long Chen. Motion planning for autonomous driving: The state of the art and future perspectives. IEEE Transactions on Intelligent Vehicles, 866):3992-3711, 2023.
[88] Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J. Guibas. Kpconv: Flexible and deformable convolution for point clouds. In /EEE/CVF International Conference on Computer Vision, pages
2019.
[89] Toyota, Ad/adas technology. https:// http:///wwwnoototttt/ooooooo toyota/eaaaaaaa hataaa/e///ddddeeeeee ad- adas- tectas-ecchhlogy, Accessed: 2024-03-06, 2024.
[90] Larissa T Triess, David Peter, Christoph B Rist, and J Marius Zöllner. Scan-based semantic segmentation lidar point clouds: An experimental study. In IEEE Intelligent Vehicles Symposium, pages 1116-1121, 2020.
[91] Waymo. Waymo to provide fully driverless rides in san francisco, califoriia. https://www theverge 20mm/2222/1111/2344889 2ttp/////4477//////// waymo -ayyo--rovidee prrovide-full alifornia, Accessed: 2024-03-06, 2022.
wwwlly-ddiverless-rddeee pr..iia--llll--..eeeeeeeee---ee-
san- -rana--cc francisco-ccc
[92] Yi Wei, Zibu Wei, Yongming Rao, Jiaxin Li, Jie Zhou, and Jiwen Lu. Lidar distillation: bridging the beam-induced domain gap for object detection. In European Conference on Computer Vision, pages 179-195, 2022.
[93] Aoran Xiao, Jiaxing Huang, Dayan Guan, Fangneng Zhan, and Shijian Lu. Transfer learning from synthetic to real lidar point cloud for semantic segmentation. In AAAI Conference on Artificial Intelligence, pages 2795-2803, 2022.
[94] Shaoyuan Xie, Lingdong Kong, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Robobev: Towards robust bird's eye view perception under coruuptions arXiv preprint arXiv:2304.0677 2023.
[95] Shaoyuan Xie, Lingdong Kong, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Benchmarking and improving bird's eye view perception robustness in autonomous driving. arXiv preprint 2405:24005.17220 2024.
[96] Shaoyuan Xie, Zichao Li, Zeyu Wang, and Cihang Xie. On the adversarial robustness of camera-hased 3d object detection. Transactions on Machine Learning Research, 2024.
197] Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1074-1083, 2023.
[98] Chenfeng Xu, Bichen Wu, Zining Wang. Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Squeezesegv3: apatially-adaptive convolution for efficient point-cloud segmentation. In European Conference on Computer Vision, pages 1-19, 2020.
[99] Xiang Xu, Lingdong Kong, Hui Shuai, and Qingshan Liu. Frnet: Frustum-range networks for scalable lidar segmentation. arXiv preprint arXiv:23/2.04484, 2023.
[100] Xiang Xu, Lingdong Kong, Hui Shuai, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu, and Qingshan Liu. 4d contrastive superflows are dense 3d representation learners. In European Conference on Computer Vision, pages 58-80, 2024.
[101] Xiaohao Xu, Zihao Du, Huaxin Zhang, Ruichao Zhang, Zihan Hong, Qin Huang, and Bin Han. Opti- mization of forcemyography sensor placement for arm movement recognition. IEEF/RSI International Conference on Intelligent Robots and Systems, pages 9845 9950,, 2022.
[102] Xiaohao Xu, Tianyi Zhang, Sibo Wang, Xiang Li, Yongqi Chen, Ye Li, Bhiksha Raj, Matthew Johnson- Roberson, and Xiaonan Huang. Customizable perturhation synthesis for robust slam benchmarking. arXiv preprint arXiv:2402.088 2002000/225..... 2024.
[103] Xu Yan, Chaoda Zheng, Ying Xue, Zhen Li, Shuguang Cui, and Dengxin Dai. Benchmarking the robust- ness of lidar semantic segmentation models. International Journal of Computer Vision, 132:2674-2697, 2024.
[104] Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018.
16

---

[105] Zetong Yang. Yanan Sun, Shu Liu, and Jiaya Jia. 3dssd: Point-based 3d single stage object detector. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11040-11048, 2020.
[106] Zetong Yang. Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Std: Sparse-to-dense 3d ohject detector for point cloud. In IEEE/CVF International Conference on Computer Vision, pages 1951-1960, 2019.
[107] Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1184-11 11793, 2021.
[108] Kaicheng Yu, Tang Tao, Hongwei Xie, Zhiwei Lin, Zhongwei Wu, Zhongyu Xia, Tingting Liang, Haiyang Sun, Jiong Deng. Dayang Hao. Yongtao Wang. Xiaodan Liang, and Bing Wang. Benchmarking the robustness lidar-camera fusion for 3d object detection. arXiv preprint arXiv:2205.14955: 2022.
[109] Diankun Zhang, Zhijie Zheng, Haoyu Niu, Xueqing Wang, and Xiaojun Liu. Fully sparse transformer 3d detector for lidar point cloud. IEEÉ Transactions on Geoscience and Remote Sensing, 61:1- 12, 2023.
[110] Tunhou Zhang, Mingyuan Ma. Feng Yan, Hai Li, and Yiran Chen. Pids: Joint point interaction--dimesion search for 3d point cloud. IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1298-1307, 2023.
[111] Yang Zhang. Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi. Boqing Gong, and Hassan Foroosh. Polarnet: An improved grid representation for online lidar point clouds semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9601-9610, 2020.
[112] Yifan Zhang, Junhui Hou, and Yixuan Yuan. comprehensive study of the robustness for lidar-based 3d object detectors against adversarial attacks. International Journal of Computer Vision, 132(5):1592 1524- 2024.
[113] Yiming Zhao, Lin Bai, and Xinming Huang, Fidnet: Lidar point cloud semantic segmentation with fully interpolation decoding. In IEEE/RSJ International Conference Intelligent Robots and Systems, pages 4453 4458, 2021.
[114] Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages
2018
[115] Zixiang Zhou, Yang Zhang, and Hassan Foroosh. Panoptic-polameet Proposal-fice lidar point cloud panoptic segmentation. IEEE/CV Conference on Computer Vision and Pattern Recognition, pages 13194-1 13203, 2021.
[116] Zixiang Zhou, Xiangchen Zhao. Yu Wang, Panqu Wang, and Hassan Foroosh. Centerformer: Center-based transformer for 3d object detection. European Conference on Computer Vision, pages 496-513. 2022.
[117] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong. Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9939-9948, 2021.
[118] Zhuangwei Zhuang, Rong Li, Kui Jia, Qicheng Wang, Yuanqing Li, and Mingkui Tan. Perception- multi-senso fusion for 3d lidar semantic segmentation. In JEEE/CVF International Conference on Computer Vision, pages (6280-1629)) 2021.
[119] Zoox. Zoox autonomous e-shuttle begins operating on public roads in california. https: om/ h///autttoiieeeoooooooooooo http://ooxxeeeetrreiceo 2023.
cettttaxxipubiiiee. roads call-caliiii california, Accessed: 2024-03-06.
[120] Vlas Zyrianov, Xiyue Zhu, and Shenlong Wang. Learning to generate realistic lidar point clouds. In European Conference Computer Vision, pages 17-35, 2022.
17

---

Appendix
A. The Place3D Dataset
18
A. Statistics
A.2 Data Collection
18
A.3 Label Mappings
19
A.4 Adverse Conditions
20
A.5 License
20
B. Additional Implementation Detail
20
B.1 LiDAR Placement
20
B.2 Corruption Generation
22
B.3 Hyperparameters
22
C. Additional Quantitative Result
23
C.I LiDAR Semantic Segmentation
23
C.2 3D Object Detection
24
C.3 Compare to Existing Approaches
25
Additional Qualitative Result
D.I Clean Condition
25
D.2 Adverse Conditions
25
E. Proofs
E. Full Proof of Theorem
33
E.2 Proof of Corollary 1
F. Discussions
35
F.1 Limitations
35
F22 Potential Societal Impact
G. Public Resources Used
A The Place3D Dataset
In this section, present additional information the statistics, features, and implementation details of the proposed Place3D dataset.
A.1 Statistics
Our dataset consists of total of eleven LiDAR placements, in which seven baselines inspired by existing self-driving configurations from autonomous vehicle companies, and four LiDAR placements obtained by optimization. Each LiDAR placement contains four LiDAR sensors. For each LiDAR configuration, the sub-dataset consists of 13,600 frames of samples, comprising 11,200 samples for training and 2,400 samples for validation, following the split ratio used in nuScenes [7]. We combined every 40 frames of samples into one scene, with time interval of 0.5 seconds between each frame sample.
A.2 Data Collection
In this section, we elaborate on more details of the simulator setup and LiDAR setup procedures used in collecting the Place3D dataset.
A.2.1 Simulator Setup
We choose four maps (Towns 3, 4, and 6, cf. Figure 7) in CARLA 10.9.10 to collect point cloud data and generate ground truth information. For each map, we manually set six ego-vehicle routes to cover all roads with no roads overlapped. The frequency of the simulation is set to 20 Hz.
18

---

(a) Town (b) Town
(c) Town
(d) Town 6
Figure 7: The maps used to collect the Place3D data in CARLA v0.9.10.
Table 4: Comparisons between the established benchmark and popular driving perception datasets with support for both the LiDAR semantic segmentation and 3D object detection tasks.
nuScenes
Waymo
SemKITTI
Place3D
Detection Classes Segmentation Classes
10
2
3
3
23
19
# of 3D Boxes Points Per Frame
12M
1.4M
80K
71K
34K
177K
120K
16K
# of LiDAR Channels Vertical FOV
1 x 32
64
64
x
10.00, 10.0)
17,,, 2.4]
[-24.8, 2.0]
2-248, 2.0]
Placement Strategy
Single No
Single No
Single No
Multiple
Adverse Conditions
Yes
Table 5: The attributes of the semantic LiDAR sensors used for acquiring the data.
Attribute
Value
Attribute
Value
Channels Range
16
Upper FOV
2.0 degrees
100.0 meters
Lower FOV
-24.8 degrees
Points Per Second
5,000 channels
Horizontal FOV
360.0 degrees
Rotation Frequency
20.0 Hz
Sensor Tick
0.5 second
A.2.2 LiDAR Setup
LiDAR point cloud data is collected every 0.5 simulator seconds utilizing the Semantic LIDAR sensor in CARLA. The attributes of Semantic L/DAR sensor listed in Table 5. Notably, we utilize LiDARs with relatively low resolution to increase the challenge in object detection and semantic segmentation, leading to lower scores than those on the nuScenes [7] leaderboard. This is based on the observation that LiDAR point clouds become sparse when detecting distant objects and the use of multiple LiDARs is specifically to enhance the perception of sparse point clouds at long distances in practical applications. Our experiments with low-resolution LiDARs allow for more evident assessment of the impact of LiDAR placements on the perception of sparse point clouds.
A.3 Label Mappings
In this section, we provide more details for the class definitions of the LiDAR semantic segmentation and 3D object detection tasks.
A.3.1 LiDAR Semantic Segmentation
There are a total of 21 semantic categories in our LiDAR semantic segmentation dataset, which are 1 Buildingg Fence, OOther, 'Pedestrian, Polee, Road Line, TRoad, "Sdewalk,, Vegetation, V0hccie 11 Wall, 12rraffic Sign, Groundd Bridge, 15 Rail Track, 16 Guard Rail, 17Traffic Light, 18ttatic, 19 Dynamic, 20 Water, and 21 Terrain. Table presents the detailed definition of each semantic class in Place3D. Additionally, we include Unlabeled tag to denote elements that have not been categorized.
19

---

(a) Waymo
Motional
Cruise
Pony.ai
(c) 7oox
(f) Toyota
(g) Momenta
(h) Ford
Figure 8: A diverse spectrum of existing multi-LiDAR placements employed by major autonomous
vehicle companies [91, 72, 22, 76, 119, 89, 71, 27]. Images adopted from original websites.
A.3.2 3D Object Detection
We set 3 types of common objects the traffic scene for the 3D object detection tasks, i.e., Car,, Bicyclist, and 3 Pedestrian, following the same configuration KITTI [29].
A.4 Adverse Conditions
As shown in Figure and Figure 10, the adverse conditions in the Place3D dataset can be categorized into three distinct groups:
Severe weather conditions, including "fog", which causes back- seak-scattering and attenuation of LiDAR points due to water particles in the air; "snow", where snow particles intersect with laser beams, affecting the beam't 's reflection and causing potential occlusions; and "wet ground", where laser pulses lose energy upon hitting wet surfaces, leading to attenuated echoes. These are commonly occurring corruptions in real driving conditions.
External disturbances like "motion blur", which results from vehicle movement that causes blurring in the data, especially on bumpy surfaces or during rapid turns.
Internal sensor failure, such as "crosstalk", where the light impulses from multiple sensors interfere with each other, creating noisy points; well as "incomplete echo", where dark- colored objects result incomplete LiDAR readings.
A.5 License
The Place3D dataset is released under the CC BY-NC-SA 4.0 license'.
B
Additional Implementation Detail
In this section, we present additional implementation details to facilitate the reproduction of the data generation, placement optimization, and performance evaluations this work.
B.1 LiDAR Placement
We adopt five commonly employed heuristic LiDAR placements, which have been adopted by several leading autonomous driving companies (see Figure 8), our baseline. We create another two baseline placements by adjusting the rolling angles of LiDAR sensors. We obtain two placements in optimization and two placements in the ablation study. All configurations are presented in Table 7.
lattp:://creattivecomons http://mmmmmmmrii/meesssseeeseee
20

---

Table 6: Summary of the semantic categories and their semantic coverage in the Place3D dataset.
Class
ID
Description
Building
1
Houses, skyscrapers, etc., and the manmade elements attached to them, such seaffolding, awning, and ladders.
Fence
Barriers, railing, and other upright structures that are made by wood or wire assemblies and enclose an area of ground.
Other
3
Everything that does not belong to any other category.
Pedestrian
4
Humans that walk, ride, or drive any kind of vehicle or mobility system, such as bicycles, scooters, skatehoards, wheelchairs, etc..
Pole
Small mainly vertically oriented poles, such sign poles and traffic light poles.
Road Line
6
Markings on the road.
Road
7
Part of ground on which vehicles usually drive.
Sidewalk
8
Part of ground designated for pedestrians or cyclists, which is delimited from the road by some obstacle, such as curbs or poles.
Vegetation
Trees, hedges, and all other types of vertical vegetation.
Vehicle
10
Cars, vans, trucks, motoreycles, bikes, buses, and trains.
Wall
Individual standing walls that are not part of a building.
Traffic Sign
12
Signs installed by the city authority and are usually for traffic regulation.
Ground
13
Horizontal ground-level structures that do not match any other category.
Bridge
14
The structure of the bridge.
Rail Track
15
All types rail tracks that are not driveable by cars.
Guard Rail
16
All types of guard rails and crash barriers.
Traffic Light
17
Traffic light boxes without their poles.
Static
18
Elements in the scene and props that are immovable, such fire hydrants, fixed benches, fountains, bus stops, etc.
Dynamic
19
Elements whose position is susceptible change over time, such movable trash bins, buggies, bags, wheelchairs, animals, etc.
Water
20
Horizontal water surfaces, such as lakes, sea, and rivers.
Terrain
21
Grass, ground-level vegetation, soil, and sand.
Table 7: The configuration coordinates of different LiDAR placement strategies in this work with respect to their ego- vehicle coordinate frames from the four LiDAR sensors.
LIDAR #1
LIDAR
LIDAR
LIDAR
Placement
roll
g
roll
roll
roll
Center
0.0
0.0
2.2
0.0
0.0
0.0
2.4
0.0
0.0
0.0
2.6
0.0
0.0
0.0
2.8
0.0
Line
0.0
-0.6
2.2
0.0
0.0
-0.4
2.2
0.0
0.0
0.4
2.2
0.0
0.0
0.6
2.2
0.0
Pyramid
-0.2
-0.6
2.2
0.0
0.4
0.0
2.4
0.0
-0.2
0.0
2.6
0.0
-0.2
0.6
2.2
0.0
Square
-0.5
0.5
2.2
0.0
-0.5
-0.5
2.2
0.0
0.5
0.5
2.2
0.0
0.5
-0.5
2.2
0.0
Trapezoid
-0.4
0.2
2.2
0.0
-0.4
-0.2
2.2
0.0
0.2
0.5
2.2
0.0
0.2
-0.5
2.2
0.0
1 Line-roll
0.0
-0.6
2.2
-0.3
0.0
-0.4
2.2
0.0
0.0
0.4
2.2
0,0
0.0
0.6
2.2
-0.3
Pyramid-roll
-0.2
-0.6
2.2
-0.3
0.4
0.0
2.4
0.0
-0.2
0.0
2.6
0.0
-0,2
0.6
2.2
-0.3
Ours-det
0.5
0.5
2.5
-0.3
-0.4
0.1
2.6
-0.2
0.0
0.0
2.8
0.0
-0.1
-0.5
2.7
0.0
Ours
0.0
0.5
2.6
-0.3
0.6
0.3
2.8
0.0
0.4
0.0
2.5
0.0
0.1
006
2.8
0.2
Commttion
0.4
0.5
2.6
-0.3
0.5
-0.4
2.7
0.0
-0.4
-0.3
2.7
0.1
.0.3
0.5
2.7
0.0
2D-plane
0.6
0.6
2.2
0.0
0.5
-0.4
2.2
0.0
-0.5
-0.6
2.2
0.0
0.3
2.2
21

---

(a) fog
(b) wet ground
(c) snow
(d) motion blur
incomplete echo Figure 9; Visual examples of LiDAR point clouds under adverse conditions in Place3D.
(c) crosstalk
B.2 Corruption Generation
In this work, we adopt the public implementations from Robo3D [46] to generate corrupted point clouds. We follow the default configuration of generating nuScenes-C to construct the adverse condition sets in Place3D. Specifically, for "fog" generation, the attenuation coefficient is randomly sampled from [0, 0.005, 0.01, 0.02, 0.03, 0.06] and the back-scattering coefficient is set as 0.05. For "wet ground" generation, the water height is set as 1.0 millimeter. For "snow" generation, the value of snowfall rate parameter is set to 1.0. For "motion blur" generation, the jittering noise level is set as 0.30. For "crosstalk" generation, the disturb noise parameter is 0.07. For "incomplete echo" generation, the attenuation parameter is set to 0.85.
B.3 Hy Hyperparameters
In this work, we build the LiDAR placement benchmark using the MMDetection3D codebase [20]. Unless otherwise specified, we follow the conventional setups in MMDetection3D when training and evaluating the LiDAR semantic segmentation and 3D object detection models.
The detailed training configurations of the four LiDAR semantic segmentation models, i.e., Mink [18), SPVCNN [86], PolarNet [], and Cylinder3D [117], are presented in Table 8. The detailed training configurations of the four 3DD object detection models, i.e., PointPillars [51], CenterPoint [107]. BEVFusion-L [63], and FSTR [109], presented in Table 9,
22

---

(a) fog
(b) wet ground
(c) snow
(d) motion blur
(c) crosstalk
incomplete echo
Figure 10: Visual examples of LiDAR point clouds under adverse conditions in Place3D.
All LiDAR semantic segmentation models are trained and tested on eight NVIDIA A100 SXM4 80GB GPUs. All 3D object detection models are trained and tested on four NVIDIA RTX 6000 Ada 48GB GPUs. The models are evaluated on the validation sets. We do not include any type of test time augmentation or model ensembling when evaluating the models.
C Additional Quantitative Result
In this section, we provide additional quantitative results to better support the findings and conclusions drawn in the main body of this paper.
C.1 LIDAR Semantic Segmentation
We provide the complete (i.e., the class-wise IoU scores) results for LiDAR semantic segmentation under different LiDAR placement strategies.
We showcase the per-class LiDAR semantic segmentation results of MinkUNet [18), SPVCNN [86]. PolarNet [111], I]. and Cylinder3D [117] in Table 11. The performance of these methods is evaluated under different LiDAR placement strategies. Different LiDAR placement strategies demonstrated varying propensities towards particular classes. For instance, Pyramid performs well building, guard ruil, and vegetation compared with other placements, possibly due to an increased vertical field of view that captures these taller or layered structures more effectively. Ours provides the generally best balance of performance across categories, with high scores in building, road, and
23

---

Table 8: The training and optimization configurations of the four LiDAR semantic segmentation models [18, 86, 111, 117] used in our Place3D benchmark.
Hyperparameter
MinkUNet
SPVCNN
PolarNet
Cylinder3D
Batch Size
8 x b2
8 b2
8 b2
b2
Epochs
50
50
50
Optimizer
Adam W
A
Adam W 8.0e-3
AdamW
Learning Rate
8.0c-3
8.0e-3
8.0e-3
Weight Decay
0.01
0.01
0.01 1.0e-6
0.01
Epsilon 1.0e-6
1.0e-6
1.0e-6
Table 9: The training and optimization configurations of the four 3D object detection models [51, 107, 63, 109] used in our Place3D benchmark.
Hyperparameter
PointPillars
CenterPoint
BEVFusion-L.
FSTR
Batch Size Epochs
4 x b4
4 b4
4 x b4
x b4
24
20
20
20
Optimizer
Adam
AdamW
Adam
AdamW
Learning Rate
1.0c-3
1.0e-4 4 0.01
1.0c-4
1.0e
Weight Decay
0.01
0.01
0.01
Epsilon
1.0e-6
1.0e-6
1.0e-6
1.0e-6
vehicle suggesting effective all-around coverage for various object types. The complete benchmark results of LiDAR semantic segmentation under clean and adverse conditions are presented Table 12. We showcase the correlation between M-SOG and LiDAR semantic segmentation performance under clean and adverse conditions Figure and 12.
We showcase the complete results of the ablation study for semantic segmentation in Table 14. we first explore the performance of our optimization algorithm under placement constraints, specifically, standardize the elevation of LiDARs to analyze their optimal positioning on the vehicle's 's roof's horizontal plane. Our optimized placement 2D plane achieved better performance compared with line, square, and trapezoid configurations. Further, we optimize the placement for adverse conditions. While the observation suggests trade-off in clean data compared with the Ours, the configuration optimized on corruption data (corruption optimized) achieves significantly higher performance in adverse conditions than both baselines in Table 12 and Ours, indicating that custom optimization strategies, tailored for challenging conditions, represent an effective methodology to enhance robust perception capabilities adverse environments.
C.2 3D Object Detection
In this section, we present more experimental results for the 3D robustness evaluation of 3D ob- ject detection under adverse conditions. The performance of PointPillars [51], CenterPoint [107], BEVFusion-L. [63], and FSTR [109] is evaluated under different LiDAR placement strategies in Table 13. Across all placements and conditions, there is a trend where all methods suffer drop in performance in adverse conditions compared clean conditions, highlighting the challenge that adverse conditions pose to 3D object detection. While some placements, like Pyramid and Square, appear to maintain relatively high performance under both clean and adverse conditions, the Ours placement has the highest average mAP under adverse conditions. In addition, certain placements exhibit relative strengths against specific types of corruption, which can inform the development of more resilient object detection systems for autonomous vehicles. For instance, Pyramid and Pyramid- Roll, appear to handle fog better than others, possibly due to a configuration that captures a more diverse set of angles which could mitigate the scattering effect of fog on LiDAR beams. We showease the correlation between M-SOG and 3D Object Detection performance under clean and adverse conditions in Figure 13 and 14.
24

---

Table 10: Comparisons the key differences among existing sensor placement approaches.
Method
Venue
Deployment
Sensor
Prior Info
Task
Optimizing
S-MIG [34]
CVPR 2022
Vehicle Road Side
LiDAR
3D bbox
Det
X
Jiang's s [40]
ICCV 2023
LÍDAR
3D bhox
Det
S-MS [57]
ICRA 2024
Vehicle
LIDAR + Camera
3D bbox
Det
X
Place3D
Ours
Vehicle
LIDAR
Semantic Oce
Seg +
Det
C.3 Compare to Existing Approaches
The effectiveness of sensor placement metrics is determined by the linear correlation between these metrics and actual performance. As shown in Table 10, Jiang's method [40] is applied to roadside placements, while comparable on-vehicle methods, S-MIG [34] and S-MS [57]. follow the same process for LiDARs. All these methods use 3D bounding boxes priors to understand the 3D scene distribution. However, this approach leads to significant deviations from the actual physical boundaries of objects and fails to account for occlusion relationships between objects and the environment in LiDAR applications. Consequently, they cannot accurately describe the scene or effectively optimize LiDAR placements. Moreover, these methods are limited to evaluating LiDAR placements for detection tasks since they solely rely on 3D object distribution information.
Our method addresses these limitations by introducing semantic occupancy information prior for the evaluation metric. This allows for more accurate characterization of boundaries in the scene and effectively addresses occlusion issues between objects and the environment. Additionally, our method leverages semantic distribution information under diverse conditions, thereby enhancing the reliability of perception in adverse weather conditions and during sensor failures.
D Additional Qualitative Result
In this section, we provide additional qualitative examples to help visually compare different condi- tions presented in this work.
D.1 Clean Condition
We showcase the qualitative results of the MinkUNet [18] model using our optimized LiDAR place- ment. As shown in Figure 15, the depiction across the first four rows demonstrates commendable performance of our LiDAR placement strategy during clean conditions. This is evidenced by the predominance of gray in the error maps, which indicates high rate of correct predictions. Also, the fifth row presents a notable exception, illustrating a failure case. This particular scene exhibits heightened complexity with a diverse array of objects and potential occlusions that challenge predictive accuracy. The qualitative results underscore the importance of continuous refinement of LiDAR placement to achieve better perception across comprehensive spectrum of scenarios.
D.2 Adverse Conditions
In Figure 16, we showcase the qualitative results of MinkUNet [18] under different adverse conditions utilizing our optimized LiDAR placement. The corruptions from top to bottom rows are "fog". "wet ground", "motion blur", "crosstalk", and "incomplete echo". As can be seen, the LiDAR semantic segmentation model encounters extra difficulties when predicting under such scenarios. Erroneous predictions tend to appear in regions contaminated by different types of noises, such as airborne particles, disturbed laser reflections, and jittering scatters. It becomes apparent that enhancing the model's robustness under these adverse conditions is crucial for the practical usage of driving perception systems. To achieve better robustness, we design suitable LiDAR placements that can mitigate the degradation caused by corruptions. As discussed in Table 12, Table 13, and Table 14, our placements can largely enhance the robustness of various models.
25

---

72
MERTIN
78
SPOONN
75
Cytnderdo
76
78.75
68
89
87.9
73.0
67.8
72
70.8
64
62.7
8G
69.
68.8
72.7
67.
88.8
O
67.7
1
71.0
60
58.5
88.0
$9.7
10
61
60
61.0
89.3
63.
M903(00
68
68.5684
68.8
M-00
88.8
56
5.0
3D
20
-16
-10
3B
300
.5
20
-15
-10
35
30
25
20
-1.
-10
-sn
3.0
20
.20
.15
.10
(a) PolarNet
(b) MinkUNet
(c) SPVCNN
(d) Cylinder3D
Figure 11: The correlation between M-SOG and LiDAR semantic segmentation [18, 86, 111, 117] models performance in the clean condition.
50
(lb
PolerNes
58
[m6 [18
MinkUNes:
[66
SPVONN
52
Dytider3Di
47.4
50
48.4
4B.6
40.6
48
44.1
44.4
4G
43.8
48
41.7
44.5
44
44.5
47.8
42
40
42.00
43.4
37.2
1-5
M503(10))
39.7
40
39
38
3.5
30
255
20
-155
-10
35
3.
25
2.0
-15
-1.0
35
3.0
225
20
-15
-10
3.5
3.0
25
.20
-15
1.0
(a) PolarNet
(b) MinkUNet
(c) SPVCNN
(d) Cylinder3D
Figure 12: The correlation between M-SOG and LiDAR semantic segmentation [18, 86, 111, 117] models performance in the adverse condition.
(MA
PointPiltins
BEVRUSION
(MA
69
MMAP
48.8
54
07:1
48.1
52
$1.0
55.4
58.2
85.8
65
55
55.3
44
43.4
43.9
50
48.3
30.2
54
54.0
05.2
53
53.5
M50910
49.
1-5
M803100
82.8
M5006(1
-.7
-16
4.5
-14
..3
-12
1.7
-15
-4
-13
.12
1.7
.16
.15
-1
-.3
-12
1.7
-16
.1.5
.14
.13
.12
(3) PointPillars
(b) BEVFusion-L.
(c) CenterPoint
(d) FSTR
Figure 13: The correlation between M-SOG and 3D object detection [51, 107, 63, 109] models
performance in the clean condition.
(75)
PointPilaars
28 [10
BEVFusionL
32
34
26
25.8
24.7
28.1
30
28.4
28.6
32
30.9
32.1
32.1-0
28.3
27.3
28.0
28.3
84
21.9
22.3
23.9
24
22.8
28.1
28.4
30
28
28.7
31.1
26.3
80
88
28.8
100
18.5
1S00
232
MS03100)
MSS001199)
10
-1.7
.15
455
-14
..3
.12
-177
.18
.15
.14
.13
.22
-17
.15
.15
-14
-13
-12
.1.7
-15
.1.5
-14
.13 .18
(a) PointPillars
(b) BEVFusion-L.
(c) CenterPoint
(d) FSTR
Figure 14: The correlation between M-SOG and 3D object detection [51, 107, 63, 109] models performance in the adverse condition.
26

---

Ground Truth
Prediction
Error Map
Qualitative assessments of the MinkUNet [18] model using our LiDAR placement strategy. The model is tested under the clean condition. The error maps show the correct and incorrect predictions in gray and red, respectively. Kindly refer to Table for color maps. Best viewed in colors and zoomed-in for details.
Figure 15:
27

---

Ground Truth
Prediction
Error Map
Qualitative assessments of the MinkUNet [18] model using our LiDAR placement strategy. The model is tested under adverse conditions. The error maps show the correct and incorrect predictions in gray and red, respectively. Kindly refer to Table for color maps. Best viewed in colors and zoomed-in in for details.
Figure 16:
28

---

Table 11: The per-class LiDAR semantic segmentation results of MinkUNet [18], SPVCNN [86],
PolarNet [111], and Cylinder3D [117], under different LiDAR placement strategies. All mloU (t) and class wiss IoU (1) scores are given in percentage (%).
Method
huilding
other
pedesitann
pob
vegetation
vebbcle
vall
Tratlee sioo
bolgo
ral mmck
goand
siate
Placeeeent:
Center
Mink
65.7
87.3
60.7
61.3
75.8
67.5
1.8
95.1
70.9
86.7
95.1
86.4
55.5
65.8
56.0
0.0
82.5
72.4
64.0
70.9
60.5
61.9
SPV
67.1
87.8
63.3
63.9
74.1
67.5
4.9
05.5
69.0
86.6
94.6
87.6
S4.3
50.1
83.6
0.2
84.4
74.0
65.8
73.1
65.6
63.8
Polor
71.0
89.2
63.9
56.9
77.0
83.1
10.5
95.3
$3.0
84.3
94.8
85.3
60.5
75.6
60.2
35.7
$4.6
73.9
62.3
75.8
60.8
75.6
C,3D
22.7
90.7
67.6
71.5
79.4
90.8
11.4
96.1
74.6
86.8
8
90.4
56.0
63.
85.4
30.2
92.9
$4.4
56.0
74.3
66.3
64.0
Avg
61.1
88.7
65.1
63.9
76.6
74.7
7.2
95.5
74.4
86.1
94.8
88.2
56.7
63.7
66.8
16.5
86.1
77.4
64.5
73.5
63.3
66.3
Placement:
Line
Mink
36.7
84.1
48.2
61.9
73.1
69.4
2.01
95.0
72.6
80.5
93.7
X1.7
42.5
60.0
40.1
0.0
79.4
37.7
53.5
59.2
57.7
61.8
SPV
50.5
84.7
90.6
64.9
14.7
69.1
4.5
94.9
62.1
81.0
94.0
85.7
48.6
40.5
76.4
0.0
80.1
40.0
56.8
58.6
33.0
47.2
Polar
07.7
80.6
65.1
59.6
79.6
83.9
3.2
95.1
81.1
82.8
94.4
87.7
49.1
72.6
69.4
16.5
81.1
54.7
59.6
72.8
53.7
71.0
Cy30
68.0
87.3
63.3
74.1
70.5
$0.4
70.9
84.1
52.1
57.4
80.9
87.6
71.9
64.0
68.9
51.7
60.2
63.0
86.4
56.8
74.5
75.7
95.3
71.7
82.1
94.6
85.0
46.1
57.6
66.7
82.1
58.5
64.9
49.0
60.1
Placement:
Pyramid
Mink
60.7
87.4
57.6
64.9
76.5
73.5
2.4
94.5
65.1
82.3
93.7
85.2
49.6
47.7
57.1
0.0
83.5
65.3
99.1
60.9
54.2
55.6
SPV
88.2
70.2
72.6
87.4
55.5
90.3
55.1
$4.9
95.2
85.3
84.7
87.4
57.8
74.8
52.2
28.5
85.8
58.9
60.9
74.4
70.5
85.1
85.0
Cy30
Avg
08.5
84.1
60.1
Trapezoid
84.7
Cy3D
87.9
64.8
Avg
63.8
87.2
56.0
67.1
76.9
76.1
7.5
95.4
71.0
82.8
95.1
85.7
51.1
57.7
$3.9
$1.9
59.1
58.6
64.0
46.2
58.5
Placement:
Line-Roll
Mink
38.5
85.3
57.9
73.8
69.6
95.1
71.8
79.2
91.4
81.6
47.7
56.7
22.2
0.0
75.3
45.5
54.2
56.1
53.6
58.2
SPV
611.6
X5.5
53.6
62.5
74.3
70.5
3.4
95.3
67.1
81.0
91.6
84.5
53.5
49.9
82.6
0.2
75.3
44.8
54.2
61.7
27.4
51.8
Polar
07.2
90.6
63.5
54.1
79.4
$4.5
2.8
94.8
81.5
83.0
94.4
MN.2
50L1
68.9
39.3
21.2
87.4
63.8
56.4
71.6
64.3
72.9
Cy3n
60.8
87.1
53.9
74.5
50.7
61.8
13.4
06.5
76.0
84.1
96.6
86.0
55.5
63.7
62.4
23.8
90.5
75.0
66.6
74.9
55.0
62.0
Avg
64.0
87.1
56.3
62.3
77.1
76.6
3.5
05.4
74.1
$1.8
93.5
85.1
51.7
59.8
51.6
11.1
83.6
57.5
$7.9
66.1
49.3
61.2
Placement:
Partamidd-Be Roll
Mink
62.2
87.5
52.2
63.5
76.1
73.7
2.1
94.3
63.3
81.3
94.9
$1.1
55.0
43.5
50.4
0.0
$1.0
64.6
61.5
59.3
62.0
35.5
SPV
67.0
83.5
61.2
89.5
73.4
73.8
7.6
95.1
72.2
83.9
94.4
87.6
56.2
55.7
87.9
0.0
83.3
65.0
66.2
68.7
64.7
61.6
Polar
71..
9033
67.11
61.0
79.1
$4.7
5.1
95.1
32.1
85.0
95.5
88.4
64.0
73.3
50.1
24.7
89.3
73.6
61.7
75.4
68.4
73.8
Cy3D
00.3
87.8
60.2
69.0
$1.0
90.9
12.0
98.1
65.1
85.3
96.3
88.3
53.5
57.6
92.8
32.7
88.5
77.0
65.9
70.2
57.0
48.1
Avg
67.9
88.5
61.2
69.5
78.4
73.6
7.4
95.1
72.2
83.0
94.4
87.6
58.2
55.7
87.9
0.0
83.3
65.0
66.2
68.7
64.7
64.6
Placement:
Durs
Mink
66.5
89.6
56.7
65.1
73.3
76.4
4.2
94.6
70.7
36.0
94.0
83.8
62.8
55.9
49.3
2.6
85.4
74.6
61.3
66.5
78.6
60.4
SPV
68.3
89.8
59.5
66.8
75.2
75.8
6.11
04.6
74.2
85.9
93.4
85.3
611
58.7
75.5
0.0
85.5
76.
64.9
65.2
77.3
64.7
Polar
70.7
93.5
70.6
60.7
79.6
86.1
8.6
96.0
$6.1
87.2
94.9
88.5
68.0
78.8
56.6
76
89.0
86.
63.6
79
82.2
78.2
C330
73.0
91.7
66.5
12.7
82.2
86.9
95.3
54.0
58.4
28.0
90.8
83.9
73.8
79.3
74.7
69.3
95.3
87.7
78.2
29

---

Table 12:
Benchmark results LiDAR semantic segmentation under clean and adverse condi- tions. For each placement, report the mloU (f), mAec (1), and ECE (L) scores for models under the clean condition, and mloU (f) scores for models under adverse conditions. The mloU and MAcc scores are given in percentage (%).
Method
Clean mAce
Adverse Blur
mloU
BCE
Fog
Wet
Snow
Cross
Echo
Avg
Placement: Center
MinkUNet [18]
T2.4
25.1
35.8
64.5
SPVCNN [86]
67.1
74.4
0.034
39.3
66.6
35.6
35.6
19.5
66.8
43.9
PolarNet [111]
71.0
76.0
0.033
43.1
59.2
11.4
36.1
23.1
69.4
40.4
Cylindor3D [117]
72.7
79.2
0.041
55.6
64.4
16.7
37.6
36.9
71.5
47.1
Placement: Line
MinkUNet [18]
59.7
67.7
0.037
51.7
60.2
35.5
52.0
27.1
59.2
47.6
SPVCNN
66.7
42.8
31.3
46.1
13.6
57.1
PolarNet [111]
67.7
74.1
0.034
43.1
61.6
4.4
48.4
18.9
67.3
40.6
Cylinder3D [117]
68.9
76.3
0.045
55.5
66.4
4.7
39.4
34,3
68.3
44.8
Placement: Pyramid
MinkUNel [18]
62.7
70.6
0.072
52.9
60.3
25.2
50.7
17.3
60.2
44.4
SPVCNN [86]
74.0
48.6
30.2
65.9
PolarNet [111]
67.7
73.0
0.032
34.3
61.0
2.3
66.9
37.2
Cylinder3D [117]
68.4
76.0
0.093
51.0
52.2
5.0
42.5
26.6
60.9
39.7
Placement: Square
MinkUNet 18]
60.7
68.4
0.043
55.6
61.9
33.5
51.5
26.5
61.2
48.4
SPVCNN [86]
63.4
70.2
0.031
40.7
64.3
38.3
53.9
63.7
PolarNet [1]1]
74.7
50.0
6.1
68.8
Cylinder3D [117]
69.9
76.7
0.044
52.0
55.6
2.7
44.2
37.1
68.7
43.4
Placement: Trapezoid
MinkUNet [18]
59.0
66.2
0.040
49.7
60.4
27.6
51.7
18.4
$9.3
44.5
SPVCNN [86]
61.0
68.8
0.044
40.9
61.3
33.6
49.1
16.9
60.7
43.8
PolarNet [111]
72.3
37.5
65.3
2.8
46.7
67.8
Cylinder3D [117]
68.5
75.4
0.057
52.1
64.6
3.1
36.7
30.0
65.6
42.0
Placement: Line-Roll
MinkUNet [18]
58.5
66.4
0.047
48.6
59.2
26.9
50.4
21.2
58.0
44.1
SPVCNN [86]
60.6
68.0
0.034
42.2
62.0
27.0
49.9
16.5
61.0
43.1
PolarNet [111]
72.8
0.037
38.2
62.9
2.2
65.4
Cylinder3D [117]
69.8
77.0
0.048
49.7
65.4
2.6
37.4
27.3
67.8
Placement: Pyramid-Roll
MinkUNet [18]
62.2
69.6
0.051
52.2
60.9
26.6
52.5
19.3
60.8
45.4
SPVCNN [86]
67.9
74.2
0.033
47.2
67.1
31.6
56.5
13.7
66.7
47.1
PolarNet [111]
70.9
75.9
0.035
36.3
2.3
51.4
68.6
36.8
Cylinder3D [117]
77.0
70.0
Placement: Ours
MinkUNet
66.5
73.2
0.031
59.5
66.6
56.7
24.5
66.9
48.6
SPVCNN [86]
68.3
74.6
0.034
59.1
66.7
24.0
56.0
18.7
66.9
48.6
PolarNet [1]1]
76.7
81.5
0.033
57.3
65.8
2.8
55.0
27.3
76.1
47.4
Cylinder3D [117]
73.0
78.9
0.037
57.6
67.2
5.9
48.7
63.3
47.3
30

---

Table 13: Benchmark results of 3D object detection under clean and adverse conditions. For each placement, we report the mAP (t) scores of three classes (car, pedestrian, and bicyclist) for models under the clean condition, and mAP (1) scores of car for models under adverse conditions. The mAP scores are given in percentage (%).
Clean
Adverse Blur
Method
Car
Ped
Bicy
Fog
Wet
Snow
Cross
Echo
Avg
Placement: Center
PointPillars [51]
46.5
19.4
27.1
17.1
36.3
37.4
27.1
25.7
26.2
28.3
CenterPoint [107]
55.8
28.7
28.8
23.2
47.3
18.9
27.3
31.6
22.3
28.4
BEVFusion-L [63]
32.2
19.2
36.8
27.0
20.9
20.8
ESTR [109]
55.3
27.7
29.3
23.8
44.1
30.7
25.4
25.0
25.9
29.2
Placement: Line
PointPillars [51]
43.4
22.0
27.7
15.1
39.6
33.6
27.1
16.9
25.2
26.3
CenterPoint [107]
54.0
34.2
37.7
20.2
49.2
22.8
9.7
12.0
21.5
22.6
BEVFusion-L
18.6
38.0
12.2
23.6
17.2
21.6
FSTR [109]
51.9
30.2
33.0
22.7
45.4
27.0
23.9
20.1
27.0
27.7
Placement: Pyramid
PointPillars [51]
46.1
24.4
29.0
17.2
38.7
36.1
29.2
26.3
25.8
28.9
CenterPoint [107]
55.9
37.4
35.6
26.0
49.6
21.1
28.6
24.9
25.9
BEVFusion-L. [63]
27.9
38.1
15.0
21.6
FSTR [109]
55.7
29.4
33.8
25.8
44.8
24.0
33.4
28.5
28.7
30,9
Placement: Square
PointPillars [S1]
43.8
20.8
27.1
17.2
40.0
32,5
26.1
22.3
25.7
27.3
CenterPoint [107]
54.0
35.5
34.1
23.4
50.2
19.2
13.0
14.0
24.6
24.1
BRVFusion-L.
27.0
20.7
39.4
23.6
22.7
FSTR [109]
52.8
30.3
31.3
23.7
47.2
23.4
25.1
23.1
29.0
28.6
Placement: Trapezoid
PointPillars [51]
43.5
21.5
27.3
16.0
40.0
31.3
25.9
18.6
24.9
26.1
CenterPoint [107]
55.4
35.6
37.5
22.1
51.7
14.6
15.3
11.6
23.9
BEVFusion-L [63]
50.2
30.0
31.7
19.2
39.2
19.8
26.9
27.2
22.6
25.8
FSTR [109]
54.6
30.0
33.3
22.9
46.9
26.0
26.5
23.4
26.5
28.7
Placement: Line-Roll
PointPillars [51]
44.6
21.3
27.0
15.2
40.3
33.7
26.9
20.0
25.3
26.9
CenterPoint [107]
55.2
32.7
37.2
19.6
49.6
16.0
15.5
23.2
BEVFusion-L [63]
50.8
29.4
29.5
15.2
38.3
11.2
19.5
10.7
21.9
19.5
FSTR [109]
53.5
29.8
32.4
21.0
46.2
24.2
24.1
16.2
27.8
26.6
Placement: Pyramid-Rol
PointPillars [51]
46.1
23.6
27.9
17.5
39.1
33.7
27.3
25.1
25.2
28.0
CenterPoint [107]
36.5
35.9
24.8
49.2
15.9
29.0
24.0
25.5
28.1
BEVFusion-L [63]
50.7
22.7
28.2
22.8
40.3
5.2
30.3
22.9
21.8
DSTR [109]
55.5
29.9
32.0
26.3
47.2
22.9
33.5
28.1
28.8
31.1
Placement: Ours
PointPillars [51]
46.8
24.9
27.2
18.3
40.1
36.4
26.5
26.4
25.8
28.9
CenterPoint [107]
34.4
37.3
22.3
51.1
27.7
23.6
BEVFusion-L [63]
53.0
28.7
29.5
21.8
41.3
15.9
32.0
22.9
22.7
26.1
FSTR [109]
56.6
31.9
34.1
25.1
47.5
28.8
32.1
30.6
28.2
32.1
31

---

Table 14: Ablation study of LiDAR semantic segmentation under clean and adverse conditions. For each placement, we report the mloU (f), mAce (t), and ECE (L) scores for models under the clean condition, and mloU (f) scores for models under adverse conditions. The mloU and mAcc scores are given in percentage (%).
Method
Clean mAce
Adverse Blur
mIoU
Fog
Wet
Snow
Cross
Echo
Avg
Placement: Line
MinkUNet [18]
59.7
67.7
0.037
51.7
60.2
35.5
52.0
27.1
59.2
47.6
SPVCNN [86]
59.3
66.7
0.068
42.8
$7.9
31.3
46.1
13.6
57.1
41.5
PolarNet [111]
67.7
74.1
0.034
43.1
61.6
4.4
48.4
67.3
Cylinder3D [117]
76.3
55.5
66.4
4.7
39.4
34.3
68.3
Placement: Square
MinkUNet [18]
60.7
68.4
0.043
55.6
61.9
33.5
51.5
26.5
61.2
48.4
SPVCNN [86]
63.4
70.2
0.031
40.7
64.3
38.3
53.9
18.6
63.7
46.6
PolarNet [111]
69.3
74.7
0.033
39.9
50.0
6.1
49.1
15.1
68.8
38.2
Cylinder3D [117]
76.7
52.0
55.6
2.7
44.2
37.1
68.7
43.4
Placement: Trapezoid
MinkUNet
59.0
66.2
0.040
49.7
60.4
27.6
51.7
18.4
59.3
44.5
SPVCNN [86]
61.0
68.8
0.044
40.9
61.3
33.6
49.1
16.9
60.7
43.8
PolarNet [1]1]
66.8
72.3
0.034
37.5
65.3
2.8
46.7
15.4
67.8
39.3
Cylinder3D [117]
75.4
0.057
52.1
64.6
3.1
36.7
65.6
42.0
Placement: Line-Roll
MinkUNet
58.5
66.4
48.6
26.9
50.4
58.0
SPVCNN [86]
60.6
68.0
0.034
42.2
62.0
27.0
49.9
16.5
61.0
43.1
PolarNet [111]
67.2
72.8
0.037
38.2
62,9
2.2
46.3
14.2
65.4
38.2
Cylinder3D [117]
69.8
77.0
0.048
49.7
65.4
2.6
37.4
27.3
67.8
41.7
Placement: Ours 2D) Plane
MinkUNet [18]
71.0
56.1
58.7
32.8
25.6
66.1
SPVCNN
65.0
71.8
0.050
56.9
66.1
37.9
56.4
26.0
65.7
51.5
PolarNet [111]
71.2
76.4
0.046
52.8
64.4
5.0
51.7
23.0
69.8
44.5
Cylinder3D [117]
70.3
75.7
0.047
52.6
67.7
5.0
45.9
36.8
71.1
46.5
Placement: Corruption Optimized
MinkUNet [18]
62.9
70.6
0.062
57.0
59.9
31.9
52.4
25.0
67.6
49.0
SPVCNN
72.0
0.051
56.4
36.8
57.0
64.8
PolarNet [1]1]
73.9
79.3
0.037
60.7
66.4
5.6
54.4
34.8
76.8
49.8
Cylinder3D [117]
72.5
79.1
0.040
54.0
69.9
5.9
47.3
37.8
72.7
47.9
Placement: Ours
MinkUNel [18]
66.5
73.2
0.031
59.5
66.6
17.6
56.7
24.5
66.9
48.6
SPVCNN [86]
74.6
0.034
59.1
66.7
24.0
56.0
66.9
PolarNet [1]1]
76.7
81.5
0.033
57.3
65.8
2.8
55.0
27.3
76.1
47.4
Cylinder3D [117]
73.0
78.9
0.037
57.6
67.2
5.9
48.7
41.0
63.3
47.3
32

---

E
Proofs
In this section, we present the full proof Theorem I certify the global optimally regarding the solved local optima. Additionally, we present the proof of Corollary as special case with bounded hyper-rectangle search space only given the Lipschitz constant of the objective function.
E.1 Full Proof of Theorem
Theorem 2 (Optimality Certificatiinn. Given the continuous objective function G :RR - RR with Lipschitz constant kG w.r.t input U Rd under & norm, suppose over ô-density Grids subset C u, the distance between the maximal and minimal of function G over is upper- upper-bounded by Cm, and the local optima is u arg minues G()), the following optimality certification regarding
I E U holds that:
(G(a")-G(u3))) S CM + kigō
(13)
where u² is the global optima over U.
Proof. Based on the sampling over subset l with & no-m density 6, we have:
Vu E U, min ||u usll2 S 6.
(14)
usES
From the continuous objective function G R - R with Lipschitz constant ke w.r.t. input E U IR" under & norm, holds that:
[[G(u)) G(u2)]la < kallu: u2l12, Vu,, E U.
(15)
Since the distance between the maximal and minimal of function G over S is upper-bounded by Cy- we then have:
[G((u) - Guu2)ll
max G(u) min G(u)lla Cm, Vuj, U2 S.
(16)
wES
weS
Then by absolute value inequality with us S c U stt. |lu us 8 and combining all inequalities above, we have:
[[G(u*) G(ug)lla s |[G(u") - G(us)lla + [[G((")) G(us)]l2
(17)
kallu* uslla CM
(18)
< kgō Car
(19)
which concludes the proof.
n
E.2 Proof of Corollary 1
When U is a hyper-rectangle with the bounded f2 norm of domain U2 R at each dimension i, with i = 1,2,...,1,, Thm. can hold in more general way by only assuming that the Lipschitz constant kg of the objective function is given, where the following optimality certification regarding E U holds that:
Corollary 2.
|[G(n") G(u)) < kG > U. + kgo.
(20)
Proof. Similar to the proof of Thm. 2, by the sampling over subset C 24 with mn-mr density 8, we have:
Yu E U, min
| us 12 I 8.
(21)
uSES
From the continuous objective function G Rd - R with Lipschitz constant ky w.r.t. input l T under & norm, holds that:
[[G(u,) G(uz)lla < kallu u2122. Vu, un E 4.
(22)
33

---

Now since the input space L/ is a hyper-rectangle with the bounded & norm of domain U3 E RR at each dimension i, with i 1,2, , n., for any i-th dimension u u46)) C IR it holds that:
2 ll S Us Yuj u 2 1.2 n, u
(23)
By summing up all dimensions with absolute value inequality at Umazy Unxins have:
max
G(u))
min G(u)ll2 = [G((maa)) wES
G(mmmilll
(24)
WES
G
G
min ll2
(25)
max
kg ||
(+)
min lle
(26)
max
i=1
1=1
kg
mar
min
(6)
ll2
(27)
i=1
kg
U
(28)
Since the distance between the maximal and minimal of function G over S is upper-bounded by Cm. we then have:
max
G(u) min G(u)lle S kG Σ Up Vu,, U2 E S.
(29)
wES
wES
Then by absolute value inequality with us S C U s.t. |uu uslle ö and combining all inequalities above, we have:
[[G(u)) G(u)))l2 < |[G((")) - G(us)lla + ||G((u) G(us)lla
(30)
S Skallu uslla KG Σ U
(31)
s kgō + KG Σ Uz.
(32)
1=1
which concludes the proof.
34

---

Discussions
In this section, we discuss the limitations and potential negative societal impact of this work.
E. Limitations
E111 Possible Limitation in Data Collection
In this work, we utilize CARLA to generate LiDAR point clouds instead of collecting data in the real world. Although the delicacy and realism of simulation technologies now remarkably close to the real world, there still exists gap. For example, LiDAR cannot detect transparent objects, and LiDAR generates noise when encountering walls; these characteristics simulator. In addition, for each placement, we collected 13,600 frames of data; however, real-world applications would require significantly larger dataset.
well-represented by the
F11.2 Possible Limitation in LiDAR Configurations
Although we selected seven simplified placements as baselines for extensive experimentation. our baseline placements cannot reflect and cover all possible LiDAR arrangements for various car manufacturers. Furthermore, the optimized placements we obtained are only near-optimal for our dataset. Identifying the globally optimal placements requires further analysis. addition, our dataset employs spinning LiDAR technology. Yet, the latest advancements in autonomous driving have indicated a trend toward adopting semi-solid-state and solid-state LiDAR systems, necessitating further research. To demonstrate the impact of LiDAR placement, we selected LiDAR sensors with relatively low resolution. The outcomes might differ when the total number of LiDAR sensors not four or when using LiDAR sensors with different channels and scanning frequencies.
F113 Possible Limitation in Surrogate Metric
When generating the SOG (semantic occupancy grids) for computing the M-SOG metric, we use voting algorithm to determine the semantic label for each voxel. This approach introduce potential errors in cases where multiple semantic labels within a voxel have a similar number of points. Additionally, to accurately describe the semantic distribution of scene, smaller voxel sizes are often required.
F.2 Potential Societal Impact
LiDAR systems, by their very nature, are designed to capture detailed information about the environ- ment. This can include not just the shape and location of objects, but potentially also capturing point cloud data of individuals, vehicles, and private property in high resolution. The data captured might be misused if not properly regulated and secured. Further, With autonomous vehicles navigating spaces using LiDAR and other sensors, there could be shift in how spaces are designed, potentially prioritizing efficiency for autonomous vehicles over numan-centered design principles. In addition, over-reliance LiDAR and perception systems lead to questions about the trustworthiness and reliability of these systems.
35

---

G
Public Resources Used
We acknowledge the use of the following public resources, during the course this work:
CARLA
MIT License
Scenario Runner
MIT License
MMCV
Apache License 2.0
MMDDetection
Apache License 2.0
MMDDetection3D
Apache License
MMEnginee
Apache License 2.0
nuScenes-devkit"
Apache License 2.0
SemanicKITTI-API"
MIT License
MinkowskiEngine 10
MIT License
TorchSparse"
MIT License
SPVNAS
MIT License
PolarSeg
BSD 3-Clause License
Cylinder3D 14
Apache License 2.0
PointPillars'
MIT License
CenterPoint
MIT License
BEVFusion
Apache License 2.0
FSTR 18
Apache License 2.0
Roooo3
CC BY-NC-SA 4.0
phttpp://ggthabb.cc/////////////
36

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction sections have been phrased to reflect our contributions and scope.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The potential limitations and societal impact are discussed in Section F.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Full of assumptions and a complete proof included. Please refer to Section E.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have fully disclosed all the information needed to reproduce the experi- mental results of this paper.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We attached the code for our implementations and for generating the training and evaluation data in the attachment.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper has specified all the training and test details. Please refer to Section 4 and Section B.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Not applicable.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
37

---

Sufficient information on the computing resources used in this work has been clearly discussed. Please refer to Section 4 and Section B.
Justification:
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //t::://neuupppeeee hccppullic/tthicsuuiieee
Answer: [Yes]
Justification: The conducted research sticks with the NeurIPS Code of Ethics.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discussed these societal impacts in the Appendix. Please refer to Sec- tion F.2.
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: We foresee our paper poses no such risks.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited and listed all existing assets in the Appendix. Please refer to Section G.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We uploaded the code as supplementary material with clear readme file. 14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This study does not involve crowdsourcing or research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This study does not involve crowdsourcing or research with human subjects.
38


Reviews:
[{"review_id": "hCbrfce0v0", "reviewer": "Reviewer_8eRR", "summary": "This paper proposes a framework called Place3D to investigate the placement of multiple LiDAR sensors for semantic segmentation and object detection task under various weather conditions.  Place3D mainly introduce a Surrogate Metric of Semantic Occupancy Grids (M-SOG) to evaluate the perception performance with different sensors placements.  Besides, an optimization approach is applied to refine the LiDAR placement. The method is evaluated on the simulated dataset with CARLA, showing the advantages of the refined placements.", "questions": "- In the dataset comparison part, it is written that no adverse weather for Nuscenes and Waymo.  They may include rainy weather and even more (e.g., Snow).  Although in the appendix part, please also have a check on that.\n- How is the solution spaces for the LiDAR placement? Are the displacements along z-axis or even rotation are considered during the optimization?\n- Is there any intuitive interpretation about why refined sensor placement are beneficial to different tasks?", "limitations": "The authors discussed the possible limitation in data collection.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation is interesting and the paper is well-written.\n- A new metric and a dataset are proposed for the evaluation of the placement. \n- The experiments are extensive, three typical models are evaluated on 7 weather scenarios. \n- Comparison with 7 popular placement methods show the advantages of the refined placement with the proposed methods.", "weaknesses": "- Although the proposed method  investigates LiDAR placement for more tasks under more weathers, the core formulation is similar to that in [34] as cited in the paper.\n- Only one type of 16-beam LiDAR sensor is investigated. Actually, sensors with more laser beams (64 or even 128) are becoming low-cost and popular. It is important to evaluate the effect of the placement as the point cloud data get dense.\n- During the companions in Tab.2,3, different refined placements are applied for different tasks. How is the performance if applied a Seg-optimized placement to the Det task? It is impractical to change the placement for different task. Is that possible to find an optimal placement for both tasks?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer `8eRR`,**\n\nThanks for devoting time to this review and providing valuable comments.\n\n---\n\n>Q: Although the proposed method investigates LiDAR placement for more tasks under more weather, the core formulation is similar to that in [34] as cited in the paper.\n\nA: Thank you for the insightful comment. We acknowledge the reviewer's observation regarding the similarities between our core formulation and [34]. However, our method M-SOG addresses critical limitations present in S-MIG [34] and makes significant improvements as follows:\n- While S-MIG primarily focuses on object binomial occupancy, our M-SOG incorporates semantic differentiation to form a multinomial occupancy distribution, providing a richer and more detailed understanding of the scene. This allows our metric to better capture the true nature of the environment. \n- S-MIG constructs occupancy grids with 3D bounding boxes, overlooking the occlusion relationships between objects and the environment. Our M-SOG addresses this critical limitation by leveraging semantic distribution, which only incorporates the reachable surface of objects and environments for LiDAR rays.\n- We compare S-MIG and our M-SOG in terms of the linear consistency between metrics and perception performance (Lines 244 to 251) and illustrate our superiority in Fig 4. Greater linear consistency enables the surrogate metric to better predict perception performance. Thus M-SOG enhances applicability to downstream optimization tasks.\n\n> Q: Only one type of 16-beam LiDAR sensor is investigated. Sensors with more laser beams (64 or even 128) are becoming low-cost and popular. It is important to evaluate the effect of placement as the point cloud data gets dense.\n\nA: Thanks for your insightful comment. We appreciate the reviewer's comment on the using denser beam LiDARs. In our work, we specifically chose to utilize 16-beam LiDARs to perform experiments, totaling 64 beams with 4 LiDARs, based on the following goals:\n- Using low-resolution LiDARs increases the challenge in both object detection and semantic segmentation, making it easier to observe the impact of LiDAR placements on perception.\n- LiDAR point clouds become sparse at long distances and the use of multiple LiDARs is specifically to enhance the perception of sparse point clouds. To this end, we use low-resolution LiDARs to simulate the conditions of sparse point clouds from high-resolution LiDARs at long distances. This highlights the role of multiple LiDARs in improving the perception of sparse point clouds, especially for distant objects.\n\nWe added experiments on 64-beam LiDARs and found that placement remains important with higher-resolution LiDARs. The results are as follows.\n|Beam|Method|Center|Line|Pyramid|Trapezoid|\n|-|-|:-:|:-:|:-:|:-:|\n|16|BEVFusion-L|52.5|49.3|51.0|50.2\n|64|BEVFusion-L|77.1|80.4|79.3|77.5\n\n> Q: During the companions in Tabs 2 and 3, different refined placements are applied for different tasks. How is the performance if applied a Seg-optimized placement to the Det task? It is impractical to change the placement for different tasks. Is it possible to find an optimal placement for both tasks?\n\nA: Thanks for asking. We show the performance of applying Seg-optimized placement to Det tasks and list the performance of Det-optimized layouts as a comparison. While the performance is close, the Seg-optimized placement performance still lags slightly behind the Det-optimized placement performance. \n\nTo find optimal placement for both tasks, a possible solution is to define a weighted objective function for optimizing, such as $F(L_j) = M_{SOG|Seg}(L_j) + \\lambda M_{SOG|Det}(L_j)$, where $\\lambda$ is the weight which should be designed carefully.\n\n|Method|Center|Line|Pyramid|Square|Trapezoid|Line-roll|Pyramid-roll|Seg-optimized|Det-optimized|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|BEVFusion-L|52.5|49.3|51.0|49.2|50.2|50.8|50.7|51.8|53.0\n|CenterPoint|55.8|54.0|55.9|54.0|56.4|55.2|56.2|56.4|57.1\n\n>Q: In the dataset comparison part, it is written that there is no adverse weather for nuScenes and Waymo. They may include rainy weather and even more (e.g., Snow). Although in the appendix part, please also have a check on that.\n\nA: Sorry for the oversight and thanks for your comment. We will address this in the revision. \n\nWhile nuScenes and Waymo include adverse weather, our dataset provides categorized weather data under identical scenarios. This improvement allows for systematic study and fair evaluation of the degradation of perception models in adverse conditions, compared to the naturally distributed weather data in nuScenes and Waymo.\n>Q: How are the solution spaces for the LiDAR placement? Are the displacements along the Z-axis or even rotation considered during the optimization?\n\nA: Thanks for asking. In our work, Z-axis and rotation are exactly considered in the optimization. The solution space for LiDAR placement in our optimization framework is defined as a 3D cuboid space above the vehicle roof. This space includes both the location coordinates (X, Y, Z) and roll angles for each LiDAR unit. Specific placements and a detailed explanation are discussed in Appendix B.1. \n>Q: Is there any intuitive interpretation of why refined sensor placement is beneficial to different tasks?\n\nA: Thanks for your question. There are several intuitive reasons why refined sensor placement is beneficial.\n- LiDAR heights: Increasing the average height of the 4 LiDARs improves performance, likely due to an expanded field of view.\n- Variation in heights: A greater difference in LiDAR heights enhances perception, as varied height distributions capture richer features from the sides of objects.\n- Uniform distribution: The pyramid placement performs better in segmentation, as a more spread-out and uniform distribution of 4 LiDARs captures richer surface features.\n\nAs discussed, our optimized placements are anticipated to align with these findings in the experiments, leading to better performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VoJIOTT7B8", "reviewer": "Reviewer_DpmT", "summary": "The authors present a very interesting study of optimal lidar placement on autonomous vehicles. They introduce a novel measure of the quality of lidar placement called M-SOG, and also discuss an optimization approach to find the best placement. They measure the impact of the placement on the important vision tasks in autonomous vehicles, such as object detection and semantic segmentation, showing that the placement metric is aligned with the improved quality. Lastly, in addition to clear weather they also analyze the impact on data when the conditions are degraded (such as fog or faulty sensors). Moreover, they also provide the data and the code that supports their work.", "questions": "- Line 107, \"voxelized grips\" -> \"voxels\"?\n- Line 110, \"where each voxel is assumed to be independently and identically distributed\", what does this actually mean? The voxels are clearly not independent of each other.\n- The notation can be improved, as it is confusing at times and not properly introduced. E.g., line 112, what is T, 1.\n- Along similar lines, H is not properly introduced a few lines below.\n- Line 143, should P_SOG also have k-index in its name? As it is defined for a particular k, as per the given equation.\n- In the paragraph starting with line 150, the authors discuss the metric. However, given that the metric is computed using only a subset of voxels, is there an invalid setup that the metric would not capture properly, such as putting lidars below the car or something similar?\n- Fig 2, would be good to zoom in on the roof of the car to make it more visible, as that is the relevant part in this figure.\n- The optimization in Section 3.3 is not well explained, and the notation is also not properly introduced.\n- E.g., the authors do not properly explain what is the lidar configuration space, or what are the constraints. Some details can be gleaned here and there, but given how critical this part is for their work the authors should do a much better job explaining it.\n- Along these lines, line 173 mentions cov-mat of distribution, distribution of what? This is very poorly explained.\n- Also, in this paragraph the authors use k-index for samples, but k was already used for classes?\n- How significant and how practical is the theoretical analysis? The authors introduce C_M as a gap, but this constant is not discuss anywhere, or how large it is. This makes the theoretical discussion quite weak. More discussion and more clarifications are needed.\n- Line 238, Tables 2 and 3 don't actually show this.\n- The detectors used in these experiments are not clearly discussed.\n- In general, it seems that the authors tried to stuff too many things at the same time in the work and in the experiments. This made them relegate many results and discussions to the appendix, even those that are critical to the work. It seems that it would be much better to focus on one aspect first (e.g., metric + impact on vision results in clear weather) and only then expand from there.\n- What are the points in Fig 4, not clear and not discussed.\n- The adverse aspect is very important, yet the authors do not spend a lot of time/space on it, and the results are just given. This is unfortunate, and weakens their argument around this part of their work.\n- Line 253, this was already mentioned in line 238?\n- Line 263, why is the placement good for adverse weather? Any deeper insights? How generalizable are these placements? All these important questions are not discussed at all, and the authors just skip them. This is another symptom of stuffing too many things in one paper and losing focus and depth.\n- Line 270, the authors just move all important ablation results to the appendix, which is not acceptable. At this point the authors are just abusing the appendix. Please note that the appendix is not meant to increase the page limit, and critical experiments/results/discussions should NOT be moved there.\n- More discussions and insights about the found placement should be added, such as around 2D or roll solution. Any advice for the designers, any interesting findings? This is not provided at all.\n\n###################\nFEEDBACK AFTER THE REBUTTAL\n###################\n\nI would like to thank the authors for responding to my comments, and also to the other reviewers who helped me further understand the paper. However, after reading through all the other responses/discussion it seems that my main concerns were not addressed. E.g.,while I said that the paper is too stuffed and poorly explained, the authors just said that they will add more explanations and that's that. First, I'm not sure where would this be added since there is no space and the paper is overstretched as is, and second, the required changes are significant and just saying that it will be addressed is not enough, the paper would need to be re-reviewed. Moreover, the authors marked my comments under \"Other Minor Modifications\", which just shows that they did not appreciate the severity of my comments.\nI do like the idea and I do think that the work has future, but with the current execution I just can not see how can I increase my recommendation. The paper needs quite a lot of attention and rewriting, so much so that I am not confident that the authors would not be able to do that with quick updates, especially given their responses that made light of some of my more important comments.", "limitations": "The authors do not discuss the limitations in a separate section.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- Very interesting and timely work.\n- The new metric and the results could be of interest to the self-driving community, and the methods should not be difficult to implement.\n- The presented results are quite promising.", "weaknesses": "- The paper writing is can be improved significantly, as the method is not actually clearly explained.\n- The authors tried to stuff too many things in a single paper (metric and degraded sensors), which degraded focus and impact of the work.\n- The experiments can be improved significantly.\n(detailed comments on the above weaknesses can be found in the section below)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer `DpmT`,**\n\nThanks for devoting time to this review and providing valuable comments.\n\n---\n>Q: Writing can be improved.\n\nA: Thanks for your comment. Due to limited space for response, for minor modifications (e.g. notations), please refer to the above Author Rebuttal section. For specific questions, please refer to the following.\n>Q: The voxels are not independent of each other.\n\nA: Sorry for the confusion. The voxels in a 3D space are not independent of each other due to spatial correlations in a single frame. We intended to convey that the probability distribution of occupancy for each voxel can be modeled independently across a large number of frames.\n>Q: Is there an invalid setup where the metric would not capture properly?\n\nA: Thanks for asking. In our optimization, we consider LiDAR placement within a limited hyper-rectangle space on the vehicle roof. This ensures practical and relevant placements, avoiding physically or operationally infeasible configurations.\n>Q: The optimization in Sec 3.3 is not well explained.\n\nA: In Sec 3.3, we introduce an optimization approach using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to find optimal LiDAR placements. This method iteratively searches and refines LiDAR configurations in hyper-rectangle solution space to maximize the M-SOG based objective function. We will include more in the revision.\n>Q: How significant and practical is theoretical analysis?\n\nA: Thanks for the question. The theoretical analysis gives a guarantee for the optimality of the local optima from heuristic CMA-ES under the assumption of the Lipschitz constant $k_G$ and the distance $C_M$ between the maximal and minimal of objective function G. As we have shown on Line 201-204,  Lipschitz constant $k_G$ and the difference between maximum and minimum of G $C_M$  over S can be approximated through calculating $G(u^k_i )$ of Algorithm 1 for each sampled $u^k_i$  over the δ density Grids subset. More specifically, in each iteration $k$, $k_G \\sim = \\max_i (G(u^k_{i+1}) - G(u^k_i))/(u^k_{i+1} - u^k_i)$, $C_M \\sim= \\max_i G(u^k_i) - \\min_i G(u^k_i)$. As a special case, if input space $U$ is bounded as hyper-rectangle $U$ (as in our experiments), the calculation $C_M$ can be omitted as shown in Corollary 1. We humbly argue that the theoretical analysis is significant in giving theoretical insight into the optimization of NN performance from semantic transformation input (e.g. LiDAR placement in Euclidean space), which may inspire robustness analysis from other semantic transformation perturbations, e.g., rotation, translation, brightness change, etc.\n>Q: Line 238, Tabs 2 and 3 don't show the correlation between M-SOG and performance.\n\nA: Thanks for your comment. Tabs 2 & 3 show the performance of different LiDAR placements, and Tab 1 shows the M-SOG of LiDAR placements. We add a plot of M-SOG and performance in the rebuttal PDF to illustrate it better.\n>Q: The detectors used in these experiments are not clearly discussed.\n\nA: Thanks for your comment. We used 4 detectors including PointPillars, BEV method CenterPoint, and BEVFusion-L, spatio-temporal representation method FSTR, as discussed in Appendix B.3. We will include more discussion in the revision.\n>Q: The authors tried to stuff too many things in one paper.\n\nA: Thanks for your comment. We intend to provide a comprehensive view of our contributions, including the introduction of M-SOG metric, the optimization of LiDAR placements, and the evaluation of placements under both clean and adverse weather conditions. Covering all these aspects is essential to fully demonstrate the value and impact of our method.\n>Q: What are the points in Fig 4?\n\nA: Thanks for asking. Fig 4 illustrates comparisons to existing sensor placement methods, as discussed in Sec 4.2 (Lines 244 to 251). We compare S-MIG [34] and our M-SOG in terms of the linear consistency between metrics and performance, with M-SOG doing better. Greater linear consistency indicates that metrics can effectively predict perception performance, enhancing downstream optimization tasks.\n>Q: The authors do not spend a lot of time/space on the adverse aspect.\n\nA: Thanks for the comment. In Sec 4.1, we detailed the types of adverse conditions considered, including severe weather, external disturbances, and sensor failures. In Sec 4.2, we validated the performance of optimized LiDAR placements under adverse conditions, showing consistent outperformance compared to baselines. In Sec 4.3, we investigated how our method informs LiDAR placements to improve robustness against corruptions. As suggested, we will further explain and analyze the adverse aspects during revision.\n>Q: Why is the placement good for adverse weather?\n\nA: Thanks for asking. The M-SOG utilizes semantic occupancy as prior knowledge, which is invariant to changing conditions, thereby enabling robustness of optimized placement in adverse weather. We also design an ablation study to explore the interplay between the optimization method and perception performance under corruptions in Sec 4.3.\n>Q: How generalizable are these placements?\n\nA: To clarify, our method is data-driven, and the optimized LiDAR placement aligns with the deployment scenarios. This is common industry practice that deploy specific vehicle configurations for specific country or region. However, our method itself is generalizable for any scenario.\n>Q: More discussions and insights about the found placement. Any interesting findings?\n\nA: Our experiments revealed several findings:\n- LiDAR heights: Increasing the average height of the 4 LiDARs improves performance, likely due to an expanded field of view. \n- Variation in heights: A greater difference in LiDAR heights enhances perception, as varied height distributions capture richer features from the sides of objects.\n- Uniform distribution: For Seg, the Pyramid placement performs better, as a more spread-out and uniform distribution of 4 LiDARs captures continuous surface.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rHJ1pJjTKh", "reviewer": "Reviewer_PfQe", "summary": "The paper targets an overlooked but important aspect of 3D scene understanding -- the influence of LiDAR sensor placements for the success of 3D perception tasks, such as 3D detection and 3D segmentation.\n\nThe authors introduce Place3D, a comprehensive pipeline for optimizing LiDAR placement in autonomous driving scenarios to enhance 3D scene understanding, especially under some adverse conditions. \n\nIndeed, current single-LiDAR systems and datasets tend to fail to capture the complexities of real-world environments. This work proposes to address this by using the Surrogate Metric of the Semantic Occupancy Grids (M-SOG) to evaluate LiDAR placement quality. M-SOG measures the information gain and scene understanding capability by calculating the entropy of semantic voxel distributions.\n\nThe optimization strategy employs the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to iteratively improve LiDAR configurations. The objective function is the M-SOG score, ensuring that the optimized placements are close to the global optimum. Compared to the existing S-MIG, this work makes a significant contribution by extending the binomial occupancy distribution to multinomial occupancy distribution through multiple semantic labels for better scene understanding.\n\nPlace3D also introduces a dataset of 364,000 frames collected under various conditions, including severe weather and sensor failures. The optimized placements show superior performance in LiDAR semantic segmentation and 3D object detection tasks compared to existing baselines.", "questions": "Please refer to the above section for detailed questions.", "limitations": "The authors have included some discussions of the potential limitations and societal impact in the appendix.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. This work focuses on a less-explored but rather important perspective of 3D scene understanding. The established LiDAR placement evaluation benchmark and the proposed placement optimization method could open up some new and interesting research directions in related research areas.\n\n2. An innovative metric,  M-SOG, for evaluating LiDAR placement quality by incorporating semantic information, which improves accuracy over previous methods. Additionally, this work contributes a full-cycle approach from LiDAR placement optimization to data generation and downstream evaluation.\n\n3. The proposed Place3D dataset includes several adverse conditions for realistic and practical evaluation, demonstrating significant improvements in robustness. Additionally, the use of CMA-ES ensures that LiDAR placements are close to the global optimum, with strong theoretical optimality certification.", "weaknesses": "1. In the experiments, although the baseline LiDAR placement methods are based on existing configuration of current autonomous vehicle systems (Fig. 7), it is better to compare the proposed method with some randomly sampled LiDAR placements. This can further justify the effectiveness of CMA-ES compared to a wider range of scenarios.\n\n2. It is not clear how the density of the occupancy grid (Sec. 3.3) will influence the surrogate metric, as the estimation of semantic distribution depends on the number of samples.\n\n3. The experiments are conducted in the simulation. Although CARLA is a rather realistic simulator and has been widely adopted in academia, it is unknown how well the proposed method works in the real-world setting. Perhaps it is not feasible to conduct on-site experiments, but more justifications regarding this aspect are needed for further discussion. \n\n4. The optimization process and data generation might require substantial computational resources. Either the main body or the appendix omitted such details. It is highly recommended to supplement such details as they are critical for follow-up works.\n\n5. Can the proposed benchmarks and optimization strategies be extended to other 3D scene understanding tasks, such as 3D object tracking, semantic occupancy prediction, etc?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer `PfQe`,**\n\nThanks for devoting time to this review and providing valuable comments.\n\n---\n\n>Q: In the experiments, although the baseline LiDAR placement methods are based on the existing configuration of current autonomous vehicle systems (Fig 7), it is better to compare the proposed method with some randomly sampled LiDAR placements. This can further justify the effectiveness of CMA-ES compared to a wider range of scenarios.\n\nA: Thanks for the insightful comments. We acknowledge that the theoretical upper bound for the performance of the placements optimized by our method cannot be exceeded by random LiDAR placements. \n\nAlthough enumerating all possible LiDAR placements for random selection is computationally infeasible, we add experiments on evaluating the performance of three random placement strategies as follows:\n|Method|Center|Line|Pyramid|Square|Trapezoid|Line-roll|Pyramid-roll|Random 1|Random 2|Random 3|Ours|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|PolarNet|71.0|67.7|67.7|69.3|66.8|67.2|70.9| 65.4 | 64.7 | 65.1 | 76.7   \n|MinkUNet|65.7|59.7|62.7|60.7|59.0|58.5|62.2| 60.1 | 59.6 | 59.9 | 66.5   \n|SPVCNN|67.1|59.3|67.6|63.4|61.0|60.6|67.9| 59.4 | 59.1 | 59.6 | 68.3  \n|Cylinder3D|72.7|68.9|68.4|69.9|68.5|69.8|69.3| 68.0 | 66.7 | 67.9 | 73.0       \n\nAs can be seen from the above study, randomly sampled placements tend to be sub-par compared to the existing and our optimized strategies. This verifies the effectiveness of the proposed CMA-ES under more general scenarios.\n\n>Q: It is not clear how the density of the occupancy grid (Sec 3.3) will influence the surrogate metric, as the estimation of semantic distribution depends on the number of samples.\n\nA: Thanks for your question. \n\n1) Density of samples: As the number of samples used to merge and synthesize dense semantic occupancy grids increases, the accuracy of M-SOG metrics is enhanced. This improvement is attributed to the smoother and more continuous semantic representation of the scene resulting from the increased density of semantic LiDAR points. A higher sample density provides a more detailed and granular depiction of the environment, reducing gaps and ambiguities.\n\n2) Density of voxels: As the grids become denser, the M-SOG scores for all placements will slightly increase, and the linear consistency between our metric and perception performance improves. However, with increasing density, linear consistency of M-SOG may converge as the voxel size decreases to very small dimensions. In the experiments, we used voxel sizes matching those in the detection or segmentation training parameters to compute M-SOG, aiming for better linear consistency. For example, for segmentation, we used voxel sizes of [0.10, 0.10, 0.10]. We computed the M-SOG of other voxel sizes for comparison as follows:\n|Voxel Grid Size|Line-roll|Trapezoid|Line|Pyramid|Square|Pyramid-roll|Center|Ours|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|[0.20, 0.20, 0.20]|-5.64|-6.72|-6.13|-5.88|-4.30|-4.24|-4.52|-4.70|\n|[0.10, 0.15, 0.20]|-3.80|-3.65|-3.22|-3.29|-2.67|-2.39|-2.47|-2.28|\n|[0.10, 0.10, 0.10]|-3.13|-2.89|-2.62|-2.56|-2.35|-1.63|-1.58|-1.29|\n|[0.05, 0.10, 0.10]|-2.97|-2.75|-2.40|-2.33|-2.12|-1.37|-1.24|-1.13|\n\n>Q: The experiments are conducted in the simulation. Although CARLA is a rather realistic simulator and has been widely adopted in academia, it is unknown how well the proposed method works in a real-world setting. Perhaps it is not feasible to conduct on-site experiments, but more justifications regarding this aspect are needed for further discussion.\n\nA: Thanks for your insightful comments. We appreciate the reviewer's concern regarding the applicability of our proposed method in real-world settings. \n\nThe main challenge of conducting real-world experiments is the inability to ensure identical scenarios for testing different placements. Given multiple LiDAR placements, we need to collect data multiple times in the same area. Even if we ensure that the ego vehicle follows the exact same route and the data is collected at the same time each day, we cannot guarantee the activities of traffic participants within the scene will be identical. This variability in scenarios can lead to difficulties in making fair comparisons. \nReal-world experiments are also time-consuming and resource-intensive, which can limit the extent and frequency of testing. We will include more discussion of extensions to real-world settings in the revised version.\n\n>Q: The optimization process and data generation might require substantial computational resources. Either the main body or the appendix omitted such details. It is highly recommended to supplement such details as they are critical for follow-up work.\n\nA: Thanks for your suggestion. For data generation in CARLA, we recommend using at least 8GB GPUs. For the optimization process, the hardware requirements primarily depend on the resolution of the voxel representation used for the scene. \n\nA CPU with a substantial number of cores and at least 64GB of RAM is necessary for efficient optimization through parallel processing.  The amount of RAM needed scales with the resolution of the voxel representation. We will include details in the revision.\n\n>Q: Can the proposed benchmarks and optimization strategies be extended to other 3D scene understanding tasks, such as 3D object tracking, semantic occupancy prediction, etc?\n\nA: Thanks for your question. Yes, the proposed benchmarks and optimization strategies can indeed be extended to other 3D scene understanding tasks, such as 3D object tracking and semantic occupancy prediction, provided we can obtain the semantic prior knowledge from the training dataset. \n\nIn fact, the M-SOG is task-agnostic as it only evaluates the overall semantic coverage quality provided by the LiDAR placements. The differences in optimization results only arise from the varying semantic settings defined in different tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bsxXqVTZqv", "reviewer": "Reviewer_iXBb", "summary": "This paper proposes Place3D, a full-cycle pipeline that encompasses LiDAR placement optimization, data generation, and downstream evaluations. The framework makes three appealing contributions. 1) To identify the most effective configurations for multi-LiDAR systems, this paper introduces a Surrogate Metric of the Semantic Occupancy Grids (M-SOG) to evaluate LiDAR placement quality. 2) Leveraging the M-SOG metric, this paper propose a novel optimization strategy to refine multi-LiDAR placements. 3) Centered around the theme of multi-condition multi LiDAR perception, the authors collect a 364,000-frame dataset from both clean and adverse conditions. Extensive experiments demonstrate that LiDAR placements optimized using palce3D outperform various baselines.", "questions": "- I'm curious about the influence of the LiDAR numbers. It's better to add a experiment to explore the most cost-effective LiDAR number.\n- M-SOG seems to be a global metric. Can It also reflect the scene coverage of a specific local area?", "limitations": "Nan", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper first attempt at investigating the impact of multi-LiDAR placements for 3D semantic scene, which is seldom explored in LiDAR-based detection and segmenation areas.\n- This paper proposes a more comprehensive metric M-SOG. Experiments show M-SOG is more relevant to the model performance than S-MIG, demonstrating its effectiveness to represent the scene coverage.\n- Guided by M-SOG, a LiDAR placement optimization methods is proposed in this work which can provide a solid baseline for LiDAR placement optimization.", "weaknesses": "- The representation may be a little confused. The M-SOG and the optimization method, under my understand, is task-independent. However, there are two optimized LiDAR placement related to Det and Seg tasks, which is a little confused.\n- To demonstrate the effectiveness of M-SOG, it's recommended to draw a scatter plot of M-SOG and model performance in Tab2 and Tab3, which can be more intuitive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer `iXBb`,**\n\nThanks for devoting time to this review and providing valuable comments.\n\n---\n>Q: The representation may be a little confusing. The M-SOG and the optimization method, under my understanding, are task-independent. However, there are two optimized LiDAR placements related to Det and Seg tasks, which is a little confusing.\n\nA: Sorry for the confusion and thanks for the comment. To clarify, the M-SOG and placement optimization strategy are task-independent, but the optimization result depends on the semantic classes involved. \n\nFor object detection (Det) and semantic segmentation (Seg), the semantic classes differ. Det considers three object categories: Car, Pedestrian, and Bicycle, while Seg includes 21 object and environment classes, such as Building and Fence. Consequently, the placement optimization result varies between Det and Seg due to the different semantic class definitions.\n\n>Q: To demonstrate the effectiveness of M-SOG, it's recommended to draw a scatter plot of M-SOG and model performance in Tab 2 and Tab 3, which can be more intuitive.\n\nA: Thanks for your suggestion. We include the plot in the attached single-page rebuttal PDF. \n\nAs illustrated, the results demonstrate a clear correlation, where the performance generally improves for both tasks as the M-SOG increases. While there might be fluctuations in some placements with specific algorithms, the overall relationship follows a linear correlation, highlighting the effectiveness of our M-SOG for computationally efficient sensor optimization purposes.\n\n>Q: I am curious about the influence of LiDAR numbers. It is better to add an experiment to explore the most cost-effective LiDAR number.\n\nA: Thanks for your suggestion. We appreciate the reviewer's comment highlighting the importance of determining the most cost-effective number of LiDARs. In our study, we investigate the strategic placement of 4 LiDARs, as preliminary experiments indicated that a perception system utilizing 4 LiDARs is more cost-effective compared to other configurations. We test the detection and segmentation performance using 1 to 5 LiDARs, respectively. The results of experiments are as follows:\n\n|Method|1x LiDAR|2x LiDAR|3x LiDAR|4x LiDAR|5x LiDAR|\n|-|:-:|:-:|:-:|:-:|:-:|\n|BEVFusion-L|35.1|39.4|46.2|52.5|55.3|\n|Improvement|-|+4.3|+6.8|+6.3|+2.8|\n\nWe observe that the detection performance of BEVFusion-L improved as the number of LiDARs increased. The most cost-effective LiDAR number can be 4 LiDARs as 5 LiDARs only have slight improvement. We will include more experiments on this point using other detectors in the revision.\n\nAdditionally, we also supplement the segmentation result of SPVCNN using 1 to 5 LiDARs, respectively.\n\n|Method|1x LiDAR|2x LiDAR|3x LiDAR|4x LiDAR|5x LiDAR|\n|-|:-:|:-:|:-:|:-:|:-:|\n|SPVCNN|40.3|56.6|62.7|68.6|69.1|\n|Improvement|-|+16.3|+6.1|+5.9|+0.5|\n\nSimilar to detection, we observe improved segmentation performance when the use of more LiDARs. Again, using 4 LiDARs tends to achieve the optimal trade-off between the segmentation accuracy and computational cost. To further consolidate this finding, more results on other LiDAR segmentation methods will be supplemented during this revision.\n\n>Q: M-SOG seems to be a global metric. Can it also reflect the scene coverage of a specific local area?\n\nA: Thanks for your question. Yes, the M-SOG can be adapted to reflect the scene coverage of specific local areas by segmenting the scene into smaller regions.\n\nThe score of M-SOG is connected to the voxels traversed by LiDAR rays in the scene. In our study, the LiDAR is set to sweep [-24.6, 2.0] vertically and [-180.0, 180.0] horizontally in degree. To compute the M-SOG of local areas, we can reset the sweep range of LiDAR rays according to the specific local areas we are interested in, and then get the local traversed voxels. By performing the same process of global M-SOG to local traversed voxels, we can obtain the M-SOG scores which reflect the scene coverage of specific areas.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Dear Reviewers, Area Chairs, and Program Chairs,**

We sincerely thank the reviewers, ACs, and PCs for the time and efforts devoted during this review. 

We especially appreciate our reviewers for offering valuable comments, providing positive feedback, and drawing insightful suggestions. 

---

We are encouraged that our reviewers recognize this work:
- **Reviewer `8eRR`:**
  - *"has interesting motivation and extensive experiments"*, *"proposes a new metric and a dataset"*, and *"the paper is well-written"*.
- **Reviewer `DpmT`:** 
  - *"is very interesting and timely"*, *"could be of interest to the self-driving community"*, and *"results are quite promising"*.
- **Reviewer `PfQe`:**
  - *"focuses on a less-explored but rather important task"*, "*could open up new and interesting research directions in related research areas"*, *"contributes a full-cycle approach from LiDAR placement optimization to data generation and downstream evaluation"*, and "*has strong theoretical optimality certification"*.
- **Reviewer `iXBb`:**
  - *"presents the first attempt at investigating the impact of multi-LiDAR placements for the 3D semantic scene"*, *"demonstrates effectiveness to represent the scene coverage"*, and "*provides a solid baseline for LiDAR placement optimization"*.

---

As suggested by our reviewers, we have revised the manuscript accordingly. We present a **summary of changes** as follows:

- **Methods & Technical Details:**
  - As suggested by **Reviewer `8eRR`**, we have clarified the core formulation and highlighted the improvements over the work cited in [34], specifically addressing critical limitations in S-MIG.
  - As suggested by **Reviewer `DpmT`**, we have improved the explanation of the optimization approach using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and added more details to the theoretical analysis.
  - As suggested by **Reviewer `PfQe`**, we have included additional experiments comparing the proposed method with randomly sampled LiDAR placements to justify the effectiveness of our method.
  - As suggested by **Reviewer `iXBb`**, we have clarified the task-independent nature of M-SOG and the optimization method and explained the variations in optimized placements for different tasks.

- **Experiments:**
  - As suggested by **Reviewer `8eRR`**, we have expanded our experiments to include higher-resolution LiDARs, demonstrating that placement remains important even with denser point clouds.
  - As suggested by **Reviewer `DpmT`**, we have clarified our experiments for comparison between our M-SOG and existing placement method S-MIG in Fig 4 and provided several findings and insight of LiDAR placement in our experiments..
  - As suggested by **Reviewer `PfQe`**, we have included details on the computational resources required for the optimization process and data generation.
  - As suggested by **Reviewer `iXBb`**, we have added experiments to explore the influence of the number of LiDARs on performance, showing the most cost-effective LiDAR numbers.

- **Elaboration:**
  - As suggested by **Reviewer `8eRR`**, we have intuitive interpretations of why refined sensor placements are beneficial to different tasks, such as increasing the field of view and capturing richer features.
  - As suggested by **Reviewer `DpmT`**, we have improved the clarity of our figures, such as zooming in on the relevant parts in Fig 2. To better explain Tabs 2 and 3, we have added a scatter plot of M-SOG and model performance, as shown in Rebuttal PDF.
  - As suggested by **Reviewer `PfQe`**, we have discussed the feasibility of extending our benchmarks and optimization strategies to other 3D scene understanding tasks.
  - As suggested by **Reviewer `iXBb`**, we have included discussions on the local area coverage using M-SOG, adapting the metric to reflect specific regions.

- **Other Minor Modifications:**
  - As suggested by **Reviewer `8eRR`**, we have checked and verified the presence of adverse weather conditions in Waymo and nuScenes, and included the major difference between our dataset and other datasets in the aspect of adverse weather in our revision.
  - As suggested by **Reviewer `DpmT`**, we have clarified notations and improved the overall writing to make the method more understandable. For typos, We correct "voxelized grid" to "voxels". For notations, we properly introduce notation $T$ and $H$, include the $k$ index in $P_{SOG}$, and revise the $k$ index in optimization to avoid overlap. We also do a thorough check to ensure all notations are properly introduced and clearly explained.
  - As suggested by **Reviewer `PfQe`**, we have supplemented details on the influence of the density of the occupancy grid on the surrogate metric.
  - As suggested by **Reviewer `iXBb`**, we have added scatter plots of the M-SOG and model performance, which can be found in the attached single-page Rebuttal PDF file.

For detailed responses regarding each of the above aspects, please kindly refer to the response **rebuttal windows** in the review section.

---

Last but not least, we sincerely thank our reviewers, ACs, and PCs again for the valuable time and efforts devoted and the constructive suggestions provided during this review.

---

Yours sincerely,

The Authors of Submission 1401

Meta-review: The paper presents a metric and optimization strategy to obtain LiDAR placements for autonomous driving that achieve improvements for perception tasks like 3D object detection and semantic segmentation, including under adverse conditions of sensor function and weather. It also develops a dataset and benchmark in an attempt to standardize multi-LiDAR configurations. Reviewers 8eRR, PfQe and iXBb recommend accepting the paper, while DpmT argues for rejection. All the reviewers appreciate the value of the contributions. The main issues raised by DpmT are on improved notations and density of information in the main paper. This was discussed extensively, especially by PfQe and DpmT, where the former champions the paper and finds that the presentation does not occlude the contributions. The AC concurs that the issues raised by DpmT are important, but have been sufficiently addressed by the author rebuttal. The authors are encouraged to include the notational improvements requested by DpmT in the final version of the paper. The paper does solve an under-studied but important problem for autonomous driving, where the proposed dataset and methods will help alleviate significant costs associations with trial-and-error involved in setting up expensive multi-LiDAR systems. In balance, the paper is recommended for acceptance at NeurIPS.
Decision: Accept (spotlight)