GS-LiDAR: Generating Realistic LiDAR Point Clouds with Panoramic Gaussian Splatting
Authors: Junzhe Jiang
Chun Gu
Yurui Chen
Li Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: Gaussian Splatting
LiDAR Simulation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: LiDAR novel view synthesis (NVS) has emerged as a novel task within LiDAR simulation, offering valuable simulated point cloud data from novel viewpoints to aid in autonomous driving systems. However, existing LiDAR NVS methods typically rely on neural radiance fields (NeRF) as their 3D representation, which incurs significant computational costs in both training and rendering. Moreover, NeRF and its variants are designed for symmetrical scenes, making them ill-suited for driving scenarios. To address these challenges, we propose GS-LiDAR, a novel framework for generating realistic LiDAR point clouds with panoramic Gaussian splatting. Our approach employs 2D Gaussian primitives with periodic vibration properties, allowing for precise geometric reconstruction of both static and dynamic elements in driving scenarios. We further introduce a novel panoramic rendering technique with explicit ray-splat intersection, guided by panoramic LiDAR supervision. By incorporating intensity and ray-drop spherical harmonic (SH) coefficients into the Gaussian primitives, we enhance the realism of the rendered point clouds. Extensive experiments on KITTI-360 and nuScenes demonstrate the superiority of our method in terms of quantitative metrics, visual quality, as well as training and rendering efficiency.

Published as a conference paper at ICLR 2025
GS-LIDAR: GENERATING REALISTIC LIDAR POINT CLOUDS WITH PANORAMIC GAUSSIAN SPLATTING
Junzhe Jiang, Chun Gu, Yurui Chen, Li Zhang* School Data Science, Fudan University
https: / //gtttbtt..//ggg///g/// com/ fudan 2vg/GS- LiDAR G
GS-LiDAR (1) (o9)
Ground Truth
3DGS Train 1 CD. 6.209
LiDAR4D Trann 2.5h, CD. 0.0847
Figure 1: GS-LiDAR achieves superior LiDAR simulation quality for novel view synthesis while maintaining fast training and rendering speed.
ABSTRACT
LiDAR novel view synthesis (NVS) has emerged as novel task within LiDAR simulation, offering valuable simulated point cloud data from novel viewpoints aid in autonomous driving systems. However, existing LiDAR NVS methods typically rely on neural radiance fields (NeRF) as their 3D representation, which incurs significant computational costs in both training and rendering. Moreover, NeRF and its variants are designed for symmetrical scenes, making them ill-suited for driving scenarios. To address these challenges, we propose GS-L.DAR, a novel framework for generating realistic LiDAR point clouds with panoramic Gaussian splatting. Our approach employs 2D Gaussian primitives with periodic vibration properties, allowing for precise geometric reconstruction of both static and dy- namic elements in driving scenarios. We further introduce a novel panoramic ren- dering technique with explicit ray-splat intersection, guided by panoramic LiDAR supervision. By incorporating intensity and ray-drop spherical harmonic (SH) co- efficients into the Gaussian primitives, we enhance the realism of the rendered point clouds. Extensive experiments on KITTI-360 and nuScenes demonstrate the superiority of our method terms of quantitative metries, visual quality, as well as training and rendering efficiency.
INTRODUCTION
Captured data in driving scenarios is critically important, as it serves to train and simulate au- tonomous driving systems. However, the collection of driving data is both costly and inefficient.
"Corresponding author 1 fuddagff@Fuuannn.. cwd..ee..cccac..cdccmm

---

Published as a conference paper at ICLR 2025
This underscores the necessity for LiDAR simulation algorithm capable of generating realistic Li- DAR data more efficiently. A common approach in the field is to reconstruct 3D street scenes from sparse data, which can then be used to generate novel view data. Most previous works (Xie et al., 2023b; Yang et al., 2024a; Chen et al., 2023a; Yan et al., 2024) have concentrated on novel view synthesis for cameras. These approaches use RGB images captured by vehicle-mounted cameras as input to reconstruct 3D scenes and render images from novel perspectives. Although significant progress has been made in camera-based novel view synthesis, the simulation of LiDAR remains underexplored. Due to the inherent sparsity of LiDAR point clouds. the challenge lies in accurately reconstructing 3D scenes using only LiDAR data. Furthermore, LiDAR sensors do not capture all emitted beams, as factors such as the reflective properties of objects affect beam reception, leading to point cloud dropout, which further increases the difficulty of 3D scene reconstruction.
Traditional LiDAR simulation methods can be broadly classified into two categories: virtual en- vironment modeling (Dosovitskiy et al., 2017; Shah et al., 2018; Koenig & Howard, 2004) and reconstruction-basse approaches (Manivasagam et al., 2020; Li et al., 2023; Guillard et al., 2022). The former involves creating 3D virtual worlds using physics engines and handcrafted 3D assets, though these simulators are constrained by the limited diversity and high cost of generating 3D sets. Additionally, there remains a significant domain gap between simulations and the real world. In contrast, iiconsttuctiion-asee methods aim address these limitations by reconstructing 3D assets from real-world captured data. However, the complexity of multi-step algorithms in this category limits their practical application in industry.
In recent years, neural radiance field (NeRF) (Mildenhall et al., 2020) has emerged as a foundational technique in the field of 3D reconstruction. With its effective implicit representation and high-quality volumetric rendering, NeRF has been proposed as a novel approach for LiDAR simulation. NeRF- LiDAR (Zhang et al., 2024) utilizes real images and LiDAR data to learn a Neural Radiance Field, generating point clouds and rendering semantic labels. On the other hand, LiDAR-NeRF (Tao et al., 2023) introduces novel LiDAR view synthesis task, which uses only LiDAR data as input to recon- struct 3D scene. However, LiDAR-NeRR is limited to modeling static scenes, whereas dynamic vehicles and pedestrians are common in driving scenarios. To address this, LiDAR4D (Zheng et al., 2024) introduces hybrid 4D representation for novel space-time LiDAR view synthesis. Although these NeRe-based methods mark significant progress compared to traditional techniques, they suf- fer from slow training and rendering speeds, limitation inherent to NeRF. Furthermore, efficient NeRF- architectures like HashGrid (Müller et al., 2022), designed for symmetrical scenes, are not well-suited for driving scenarios.
In this paper, we propose GS-LiDAR, a novel framework for generating realistic LiDAR point clouds using panoramic Gaussian splatting. While 3D Gaussian splatting struggles with geometric model- ing and tends to overfit sparse views, we leverage view-consistent 2D Gaussian primitives Huang et al. (2024) for more accurate geometric representation. Moreover, considering the dynamic na- ture of driving scenarios, we introduce periodic vibration properties Chen et al. (2023a) into the Gaussian primitives, enabling the uniform representation of various objects and elements in dy- namic environments. Focusing on the task of novel LiDAR view synthesis, we introduce novel panoramic rendering process to facilitate fast and efficient rendering of panoramic depth maps using 2D Gaussian primitives. Through carefully designed ray-splat intersections, the resulting panoramic depth maps are geometrically accurate and view-consistent. Each Gaussian is assigned additional LiDAR-specific attributes, such as view-dependent intensity and ray-drop probability, which are ag- gregated into intensity maps and ray-drop maps through alpha-blending in the splatting process. By using ground-truth LiDAR range maps and intensity maps for supervision, GS-LiDAR can effec- tively simulate LiDAR point clouds. As illustrated in Figure I, GS-LiDAR achieves superior LiDAR simulation quality in novel LiDAR view synthesis and significantly outperforms the previous state- of-the-an art method, LiDAR4D (Zheng et al., 2024), in both training and rendering speed.
We conduct extensive experiments to evaluate the effectiveness of our method on two major bench- marks: KITTI-360 (Liao et al., 2022) and nuScenes (Caesar et al., 2020). For static scenes, we test our approach on the KITTI-360 dataset, achieving a substantial 10.7% reduction in the RMSE metric compared to the leading competitor, LiDAR4D (Zheng et al., 2024). For dynamic scenes, our method outperforms LIDAR4D, with RMSE reductions of 11.5% on KITTI-360 and 13.1% on nuScenes. Moreover, our approach demonstrated significantly faster training and rendering times than previous NeRe-based methods, with a speedup of 1.67 times in training and a notable increase of 31 times in rendering LiDAR novel views compared to LiDAR4D.
2

---

Published as a conference paper at ICLR 2025
Our contributions are summarized as follows: (1) We propose GS-LiDAR, novel differentiable framework for generating realistic LiDAR point clouds. (2) We employ 2D Gaussian primitives with periodic vibration properties, enabling precise geometric reconstruction of various objects and (3) We introduce a novel panoramic rendering technique based on 2D Gaussian primitives, with geometrically accurate ray-splat intersection, where the rendered panoramic maps are supervised by the ground-truth data. (4) Extensive experiments demonstrate the superiority of our method across quantitative metrics, visual quality, as well as training and rendering speeds, when compared to previous approaches.
elements in dynamic scenarios.
2 RELATED WORK
Novel view synthesis Novel view synthesis (NVS) is a critical and challenging aspect of 3D re- construction. Since the advent of neural radiance fields (NeRF) (Mildenhall et al., 2020), there have been significant advancements in 3D reconstruction and NVS. NeRF utilizes a multi-layer perceptron (MLP) to model geometric shapes and view-dependent appearances, rendering through volume rendering. NeRF has demonstrated that implicit radiance fields can effectively learn scene representations and synthesize high-quality novel views. Despite its profound impact, NeRF faces notable challenges, including slow rendering speeds and aliasing. To address these issues, various research efforts (Barron et al., 2021; 2022: 2023; Hu et al., 2023; Reiser et al., 2021; Yu et al., 2021: Reiser et al., 2023; Hedman et al., 2021; Yariv et al., 2023; Chen et al., 2023c; 2022; Müller et al., 2022; Liu et al., 2020; Sun et al., 2022; Chen et al., 2023b; Fridovich- Keil et al., 2022) have developed variants that focus on enhancing rendering quality as well accelerating training and rendering speeds. Recently, 3D Gaussian splatting (3DGS) (Kerbl et al., 2023) has introduced point-based 3D scene representation that innovatively combines high-quality alpha-blending with rapid rasterization. 3DGS has been swiftly extended to various domains to enhance its rendering capabilities (Xie et al., 2023a; Huang et al., 2024; Gao et al., 2024) and its potential to represent dy- namic scenes (Yang et al., 2024b; Yan et al., 2024; Zielonka et al., 2023; Chen et al., 2023a). In this paper, we adopt 2D Gaussian primitives with periodic vibration properties as scene representation to characterize the accurate geometry of both static and dynamic elements.
LiDAR simulation Traditional simulators can be classified into two categories. The first type of method (Dosovitskiy et al., 2017; Shah et al., 2018; Koenig & Howard, 2004) utilizes physics en- gines to generate LiDAR point clouds through ray casting within handcrafted virtual environments. However, these simulators are limited by their diversity and the high cost of 3D assets, and they exhibit a significant domain gap when compared to real-world data. In contrast, the second type of methods (Manivasagam et al., 2020; Li et al., 2023; Guillard et al., 2022) have sought to address these limitations by reconstructing scenes from real data for simulation. For instance, LiDARsim (Manivasagam et al., 2020) and PCGen (Li et al., 2023) utilize a multi-step, data-driven approach to simulate point clouds from real data. However, the complexity of these multiple steps impacts their applicability and scalability. More recent works (Tao et al., 2023; Zhang et al., 2024; Zheng et al., 2024; Xue et al., 2024; Tao et al., 2024; Wu et al., 2024) have utilized NeRF for scene reconstruc- tion and LiDAR simulation, achieving higher quality and more reliable results. However, NeRF is consuminn due to its implicit representation and exhaustive ray marching. Moreover, NeRF and its variants are mostly designed for symmetrical scenes, making them ill-suited to the driving scenarios. To this end, GS-LiDAR employs the explicit representation of Gaussian splatting, which enables efficient and flexible LiDAR simulation.
METHOD
In this section, we propose GS-LiDAR, novel framework for generating realistic LiDAR point clouds with Gaussian splatting. We begin by introducing the necessary background on 3D Gaus- sian splatting in Section 3.1. For geometrically accurate reconstruction and the modeling of both static and dynamic elements, we employ 2D Gaussian primitives with periodic vibration properties as scene representation, pose an innovative panoramic rendering technique with explicit splat intersection, described in Section 3.3. Next, detail the LiDAR modeling approach, including the rendering of depth maps,
outlined in Section 3.2. To integrate LiDAR supervision, we pro-
3

---

Published as a conference paper at ICLR 2025
Panoramic Gaussian splatting
Panoramic Ray-drop refinement
Synthesis LIDAR
PVy
Intensity
Pibrating posittio
Ray-drap
dynamic
XAIP
Depth
deptt
GT Depth, intensity, Ray-drop
fu
Interscciio culculation
SUPERVISION
SUPERVISION
Time conditioned Gousstan
Splatting detail
Overview of the GS-LiDAR framework: GS-LiDAR is based 2D Gaussian primitives with periodic vibration properties, allowing for dynamic modeling of position and opacity along with accurate geometry. At given timestamp, Gaussians query their states and utilize the proposed panoramic Gaussian splatting technique to render panoramic maps of depth, ray-drop, and intensity. For each ray and Gaussian primitive, calculate their intersection to obtain the depth and o, ensuring more geometrically consistent renderings. The results are subsequently refined by a well- trained U-Net further enhance the quality of the point clouds.
Figure 2:
intensity maps, and ray-drop probability maps, in Section 3.4. Finally, in Section 3.5, we discuss the various functions used optimize the scenes. An overview of our pipeline is provided in Figure 2.
3.1 PRELIMINARY: 3D GAUSSIAN SPLATTING
3D Gaussian splatting (3DGS) (Kerbl et al., 2023) employs set of anisotropic Gaussian primitives to represent a static 3D scene, which is subsequently rendered vis differentiable splatting. By utiliz- ing a tile-based rasterizer, this approach facilitates the real-time rendering of novel views with high visual fidelity. Each Gaussian primitive characterized by position vector μ IR a covariance matrix Σ E R3x3, an opacity parameter E (0, 1), and color E R modeled by spherical harmon- ics (SH). The influence ((x) of a given Gaussian primitive on a spatial position æ is defined by an unnormalized Gaussian function:
((x) exp(- - n)).
To render an image, the Gaussian primitives are initially transformed into the camera coordinate system via the view transformation matrix W. Following this transformation, each Gaussian prim- itive is projected onto the image plane through local affine transformation J, which maps the 3D) structure into 2D image space. The 2D covariance matrix Σ' of the projected Gaussian primitive in camera coordinates is computed as: E' = JW EW JJ The final rendering process employs alpha-blena blending, wherein the color e of target pixel is obtained by aggregating the contribution G' of each relevant projected 2D Gaussian, proceeding from front to back:
k-1 C ob 9% II - o 9y),,
where & denotes the index a Gaussian primitive, with K representing the total number of Gaussian primitives in the scene.
3.2 PERIODIC VIBRATION 2D GAUSSIAN
Given the constant presence of moving vehicles and pedestrians in driving scenarios, we aim to utilize unified representation to model the various objects and elements within the scene. We employ 2D Gaussian primitives (Huang et al., 2024) with periodic vibration properties (Chen al., 2023a) to accurately capture surface behavior across both space and time. For 2D Gaussian defined by its central point μ E R3 opacity parameter E [0. 1), two principal tangential vectors t. R and t, E R3 and a scaling vector = (sn, s,) E R², we introduce additional learnable attributes that govern the variation of its central point and opacity. These include the vibrating direction
4

---

Published as a conference paper at ICLR 2025
LIDAR coordinate system
Depth rendering
mean depth
ray
ray
weight transparency
W
0
Z
Fay
median depth
Figure 3: Our LiDAR coordinate system and two ways of depth rendering. The mean depth refers to the weighted average each depth using the rendering weights, while the median depth is defined as the maximum depth with transparency, i.e., I (1 Oj Gj), no greater than 0.5.
E R3, life peak E R. and time decay rate B E R:
sin( 2T(1 T) TT)
((() = +
(3)
2T
o(t) . exp( 2/(1 (t- 7
where ji(t) represents the vibrating motion around μe and r, and o(t) represents the decayed opacity. The hyper-parameter denotes the cycle length, which serves with a simple motion, the Gaussian primitives can join to represent any complex motion of dynamic elements relay manner. The influence G of each 2D Gaussian disk is defined within its local tangent plane in world space:
prior for the scene. Each equipped
u + 2
g(u, v) exp
(5)
where (u, v) are the coordinates within the local tangent plane space (UV space). The transformation from UV space to screen space is parameterized as:
x(u, v) (i((t) + s,t, + s, v) = W (w,,,,,))
(6)
where H suty s.ty
ji(t)
E Rdx
Here, W F4x4 represents the view transformation matrix, and x (x, N, 2, denotes the homogeneous coordinates in sensor space.
3.3 PANORAMIC GAUSSIAN SPLATTING
LiDAR emits laser pulses and measures the time-of-flight (ToF) to determine object distances, as well as the intensity of reflected light. Spinning LiDAR provides 360-degree horizontal field of view (-T, π) and limited vertical field of view (VFOV min: VFOV max). allowing it to perceive the environment with specific angular resolution. The angular configuration within the LiDAR coordinate system is depicted in Figure 3. Given the pixel coordinates of a point on the range image (E, 1),, the corresponding radian angles can be computed using the following equation:
(2C ">
W) VFOV min) VFOV min
i
(8)
where (W, H) represent the width and height of the range image, respectively. homogeneous point in the LiDAR coordinate system then be determined from (Ø, 0) follows: (I,, y, 1)
(r sin 9 sin Φ - cos 0, sin 9 cos p, 1)T, where /2² y2 22 denotes the distance from the point to the center of the LiDAR.
Ray-splat intersection We define the ray as the intersection of two orthogonal homogeneous planes, h (cos é, 0, sin o, 0) and hy (cos sin á sin 0, cos cos Φ, T,T, characterized by their normal vectors. These satisfy the conditions h 0 and h 0. Consequently, given
5

---

Published as a conference paper at ICLR 2025
(a)
(b)
tile
transform
(5.)
X
tu
tile
($.0)
360
ray
(u,
sort by distance
pixel
depth
Figure 4: Panoramic Gaussian rasterization details. (a) Our method employs tile-based sorting and rendering. For panoramic ray maps, we first transform the epipolar coordinate system into the pixel coordinate system. The pixel map is then divided into small tiles, and within each tile, Gaussian primitives are sorted based on their distance to the LiDAR origin. (b) During pixel rendering, the a and depth are computed by calculating the intersection between the ray and the Gaussian primitíve.
the ray angles (#, 0), the ray-splat intersection (u, v) is determined by solving the following linear equations:
[hz hyl (u, 1)" 0.
Let [R, h)] [1e, hy] W H, where R R2x2 and h IR,, From this, we can solve for and as follows:
R h.
(10)
3.4 LIDAR RENDERING
In the novel LiDAR view synthesis task, LiDAR point clouds, which include intensity values, can be projected to panoramic range maps and intensity maps. To simulate LiDAR point clouds, we assign each Gaussian a view-dependent intensity value À and view-dependent ray-drop probability p, both which are modeled using spherical harmonics.
Depth map Considering the ray-splat intersection within the LiDAR coordinate system, the depth value corresponds to the distance from the intersection point to the center of the LiDAR. Based on Equation and the conversion between (s, y/ =) and (ø) 0). we have:
(r sin 0 sin Φ, -r cos 0, r sin 0 cos o 1) W
(11)
By multiplying (sin sin d. - cos sin cos o, 0) on both sides, the distance r can be computed as:
(sin 0 sin db cos 0, sin 0 cos db 0) W H (u, v. T)
(12)
Although we calculate the intersection of each ray with the Gaussian primitives, we still adopt the tile-based rendering method proposed in 3D Gaussian splatting Kerbl et al. (2023) achieve efficient rendering. As illustrated in Figure 4, we first transform the epipolar coordinates into pixel coordinates and sort the Gaussian primitives within each tile. For pixel rendering, the a and depth are determined based on the intersection results.
During the training process, we utilize both the mean depth and the median depth, and supervise them using the projected ground truth range map as follows:
Rmeaa
Σ A=1
rh Ok Gr
II 0z 9j),
(13)
h-1
Rmedian = max
Tb
(1 0, ,, 0.5
(14)
La Rmean Ratll
Rmedian
Rge
(15)

---

Published as a conference paper at ICLR 2025
GT
GS-LiDAR
LiDAR4D
LiDAR LiDAR-NeRF
Figure 5: Comparison of 3D LiDAR point cloud. GS-LiDAR produces more cohesive LiDAR
point cloud compared to LiDAR-NeRI (Tao al., 2023) and LiDAR4D (Zheng et al., 2024).
Table 1: State-of-the-art comparison on KITTI-360 Static Scene Sequence. We color the top
results as best and second best.
Methad
CD)
Pain Cloud
F-ecoret
RMSE
MrdAF,
Depas
LPPPS
SSIM
PSKR1
RMSE.
MadAF.
Idiasitt
1PPPS
SSIMI
PSNR1
LiDARsim (Manirasagam 20200
2.2244
0.366)
0,0739
0.2289
0.7157
21.7746
0.1532
0.0505
0.2522
0,6479
16,3045
NKSR (Uloang 2027)
0,5780
0.3685
4,6647
0.0598
0.2295
0.7052
22,5350
0,1565
0.0536
0.2429
0.4200
16,1159
PCGmn et atal,, 2023)
0,3090
0.8597
4.8838
0.1735
0.5216
0.5033
24.3050
0.2005
0.0818
06100
0.1248
13,9506
LIDAR NcRF
0.0925
0.9226
0.8667
0.6043
26.7663
0.1557
0,0549
16.1685
LIDARAD (2heng 2024)
0.0894
0.0507
0.7218
27.8840
0.1343
17.4529
GS-LIDAR (Ours)
0.0847
0.923
2,8895
0,0411
0,0897
0,8454
28,8907
0,1211
0,0359
0.8630
0.5756
18.3506
Intensity map LiDAR intensity refers to the strength of the laser pulse upon its return to the re- ceiver, which is influenced by material properties, surface characteristics, distance, and the incident angle. Intensity data contributes to terrain analysis, facilitates the differentiation of various features, and enhances the classification and precision of point cloud data. Similar to the rendering of mean depth, we aggregate the intensity map / using alpha-blending: T= E ÀR Og O& [1;=1 9z), which is subsequently supervised by the ground truth intensity:
Lint || lgtll,-
(16)
Ray-drop probability map LiDAR ray-drop refers to the phenomenon where some laser pulses do not return to the sensor after striking surfaces, often due to obstructions, absorption by vegetation, or unfavorable angles of incidence. Through
we get a ray-drop probability map from Gaussians: Pæs = Ek= o& G& 11 - 0g Gj). Additionally, since the LiDAR ray-drop is also related to the characteristics of the LiDAR itself, we introduce learnable prior P Prior for the ray- drop. The final ray-drop probability map is expressed as: P Pprior + P Pprior) ggs which is supervised by the ground truth mask using a binary cross-entropy loss:
Pgst
Larop BCE (P, Pgt).
(17)
However, this modeling of ray-drop does account for factors such distance to the sensor, potentially leading to unreliable results. To mitigate this issue, following NeRF-LiDAR (Zhang al., 2024), we utilize a U-Net (Ronneberger et al., 2015) with residual connections globally refine the ray-drop mask, thereby preserving consistent patterns across regions. Specifically, the U- Net takes the rendered ray-drop probability map P. depth map Rmeans and intensity map I inputs, and outputs the refined ray-drop mask Punct- After training the Gaussians, we continue optimizing the U-Net by supervising the refined ray-drop mask using the same loss function as Equation 17.
3.5
Loss FUNCTION
To achieve accurate geometry and align the 2D splats with surfaces, we integrate the depth distortion loss and normal consistency loss from 2DGS Huang et al. (2024). Additionally, we employ the chamfer distance loss (Fan et al., 2017) to minimize the disparity between our simulated LiDAR point clouds and the ground truth data.
Depth distortion
The depth distortion loss aims reduce the distance between ray-splat intersec- tions, encouraging the 2D Gaussian primitives to concentrate on the surface:
Ldist
Σ wrooy(r: rz)²,
(18)
1.
where W; represents the blending weight of the i-th intersection, and r; denotes the depth of the intersection points.
7

---

Published as a conference paper at ICLR 2025
Table 2: State-of-the-art comparison on KITTI-360 dataset. We color the top results as best and
second best
Method
CD]
Point Clond
F-ecoreT
RMSE
McdAF.
L
Depei
SSIM
PSNRT
RMSE.
ModAF]
Iniessity LPIPS
SSIMT
PSNRT
LiDARsim Maniraggaam clal., 2020)
3.2228
0.7157
6.915
0.1279
0.2925
0.6342
21.4508
0,1665
0:0569
0.3276
0.3502
15.5853
2023)
1.8982
016855
0.99%
0.6499
23.0368
0.1742
0.0290
15,2081
PCGes 2023)
0.3023
0.3440
0.5391
0.983
23.1675
0.3970
0.076
0.5926
14.1181
LIDAR NeRF al. 2023)
0.1438
0.509]
4,1755
0.056
0.2797
04568
25 9378
0,1404
0.313
0,3831
17.1549
D NeRl (Pamarola al,, 2021)
0.1442
0.9125
4,0198
0,008
0.3061
0.634
26,2344
0.1369
0.0480
0.3488
0,5748
17,3554
TiNes
all,
0.1748
0,0502
0.1363
0.4365
K-Planes (Pradriic
0.1302
0.0539
260236
0.1415
0.4151
17.0067
LIDARAD (2heng 2024)
0.1845
18.5561
GS-LIDAR
0.1085
Table State-of-the-art comparison on nuScenes dataset. The notations are consistent with the KITTI-360 Table 2 above.
Method
CD)
RMSE
McdAFL
Lxppti
L.PIPS]
SSIMT
PSNR1
RMSF]
MalAFI
Iniesaas LPIPS
PSNR1
LiDMKam Manivangaam at (202))
12.3333
0.0612
10,559
0.357
0.187]
0.3553
17.758]
01635
0.015
0.1100
0,3170
23.7791
NKSR Nunng etal. (2003)
11.4910
0.8178
9.3731
0.763
0.2111
0.5337
18:7774
0.0600
0.119
0.1290
0.501
23,4905
P0Gm at (2023)
2.1998
0.834]
88864
0.4011
0.1793
0.5400
19.7799
00758
00147
0.1308
27.6478
HDDARN.RF
(2023)
7.1566
0.338
0.070
0.788
21:2129
00667
0,076
0.483
D-NeRF Pursmile (2021)
7.1099
0.368
0,070
0,130
21.
00667
0.0800
0.0493
0.7180
Vox Pong et (2022)
0.3920
0.8627
7,2083
0.0200
0.1589
0.6873
21 1930
00662
0.0080
0.1294
0,2107
K-PPnnes Fridoicc-
(2023)
0.2922
0.6887
6
0.0399
0.1218
0.7258
21 8003
0.:.38 38
0.0076
0.1127
0.7364
27,4227
LIDARAD (2emp el al, 207-)
6.881
0.058
0.000
0.7988
21
0.007
010890
0.2488
27.290
GS-LDAAR (Oun))
5.8925
0.4708
22.3482
0.0627
Normal consistency Since approach is based on 2D Gaussian primitives, it is crucial to ensure that all 2D splats are locally aligned with the actual surfaces. To achieve this, we align the rendered normal map N with the pseudo-normal map Ñ, which is computed from the gradients of the depth maps:
Ln = N.
(19)
Chamfer distance loss We also incorporate chamfer distance to introduce explicit geometric con- straints from the input LiDAR point clouds. By baok-projecting both the rendered and ground truth range images into 3D point clouds, Srender and Sg,, respectively, minimize the distance between the two point clouds using the chamfer distance (Fan et al., 2017):
Lch = CD(Seeuder: Sgt)
(20)
The overall loss function is defined as:
L Aa + Aint Cint + Adrop Cdrop + Adist Caiss + XuLu + Ach
(21)
4 EXPERIMENT
4.1
EXPERIMENT SETUP
Datasets We conduct extensive experiments on both dynamic and static scenes using the KITTI- 360 (Liao et al., 2022) and nuScenes (Caesar et al., 2020) datasets, with the dynamic scenes featuring a significant number of moving vehicles. The KITTI-360 dataset employs 64-beam LiDAR with vertical field of view (FOV) of 26.4 degrees and an acquisition frequency of 10Hz. Following LiDAR4D (Zheng et al., 2024), we select 51 consecutive frames as a single scene and hold out samples at 10-frame intervals for novel view synthesis (NVS) evaluation. For the nuScenes dataset, the LiDAR system uses 32 beams with a 40-degree vertical FOV and a 20Hz acquisition frequency. To ensure consistency with KITTI-360, we maintain sampling frequency of 10Hz. We also provide the results on the Waymo (Sun et al., 2020) dataset in Appendix A.2.
Competitors We evaluate our method alongside recent data-driven approaches LiDARsim (Mani- vasagam et al., 2020) and PCGen (Li et al., 2023). Additionally, we compare results with the per- scene optimized reconstruction method NKSR (Huang et al., 2023). LiDAR-NeRF (Tao et al., 2023) and the ofate-of-the-ar art method, LiDAR4D (Zheng et al., 2024). We also include competitive dy- namic neural radiance field methods, such as D-NeRF (Pumarola et al., 2021), K- K-Planes (Fridovich- Keil et al., 2023), and TiNeu Vox (Fang al., 2022), for comparison.
8

---

Published as a conference paper at ICLR 2025
GT
Scene 1908
GS-LiDAR (Ours)
LIDAR4D
LIDAR-NeRF
Scene 10750
GS-LiDAR (Ours]
LIDARAD
LiDAR-NeRF
Figure 6: Comparison of the rendered depth map with competitors.
GT
Scene 1908
GS LiDAR (Ours)
LIDAR4D
LIDAR-NeRF
GT
Scene 10750
GS-LIDAR (Ours)
LiDAR4D
LiDAR-NeRF
Figure 7: Comparison of the rendered intensity map with competitors.
Metrics We employ a comprehensive set of evaluation metrics for assessing point cloud, depth, and intensity measurements. Chamfer distance (Fan et al., 2017) is used to quantify the 3D geomet- ric discrepancy between the generated and ground-truth point clouds based on nearest neighbors. Additionally, report the F-score with 5cm error threshold. For pixel-level error analysis of projected LiDAR range maps, we introduce Root Mean Square Error (RMSE) and Median Absolute Error (MedAE). Furthermore, utilize LPIPS (Zhang al., 2018), SSIM (Wang et al., 2004), and PSNR to assess image-level error on depth and intensity.
Implementation details We randomly sample x 10th LiDAR points for point initialization. For the loss function and regularization terms, we use the following coefficients: Xa- 10, Aint 0.05, Adrop = 0.05, Adist = 0.1, A, 0.1, and Ach 0.1. All experiments are conducted on a single NVIDIA RTX A6000 GPU, with total of 30,000 iterations, taking approximately 1.5 hours to produce the final results. The rendering speed reaches up to 11 frames per second (FPS).
4.2 EVALUATION ON STATIC SCENES
Table provides the quantitative results for static scenes in KITTI-360 dataset across all methods. GS-LiDAR outperforms the competitors on most metrics. Notably, there a 5.3% reduction in the Chamfer distance of the simulated point cloud, 10.7% reduction in the RMSE of simulated

---

Published as a conference paper at ICLR 2025
Table 4: Ablation studies on various components of GS-LiDAR.
Mothod
CD)
Point Cloud
F-scoret
RMSEL
MedAEl
Depth LPIPS]
SSIMC
PSNKT
RMSE,
MedAE]
Insensity LPIPS]
SSIM?
PSNK-
GS-LIDAR
0.1085
3.1212
25.1381
0.5914
18,7452
wil ray splat intersection
3,1605
0.3091
6,6871
0.2367
0.4233
03754
21.3918
0.2320
0.1170
0.3165
0.1976
12,7099
wio periodic vibcation
0.1333
0.9100
3.1738
0.0359
0.8445
03577
28.2816
0.1162
0.6342
0.2166
0.5849
18.725
wo median depth loss
0.1254
0.9125
3.1414
0.0347
0.0941
08574
28.3567
0.1163
0.0313
0,1831
0.5902
18.7262
wio depth distection loss
0.1237
0.9229
3,2016
0.0341
0,0388
08567
28. 2854
0.1176
0,0814
0.1842
0.583
18. 7077
consistency
0.1158
0.9250
0.0544
0.1850
18,7085
chamfer weo my-diop refinement
0.8559
18.7333
0.1121
0.9233
4,0791
0.0424
0.1952
0.7433
36,
0.1346
0.0384
0.3477
0.450)
17,441
depth, and 9.8% reduction in the RMSE of simulated intensity. As shown in Figure 5, GS-LiDAR produces a more cohesive LiDAR point cloud, which can be attributed to the accurate range maps generated by the proposed panoramic Gaussian splatring technique.
GT
w/ splat intersection
w/o splat intersection
Figure 8: Comparison of w/ and w/o ray-splat intersection. A more detailed discussion is provided in Appendix A.I.
4.3 EVALUATION ON DYNAMIC SCENES
To further validate the effectiveness of Li-LiDAR, we conduct LiDAR synthesis evaluations on dy- namic scenes from the KITTI-360 and nuScenes datasets. For the KITTI-360 dataset, as shown in Table 2, our method demonstrates superior performance, achieving a 0.3% reduction in chamfer distance for simulated point cloud, 1 11.4% reduction in RMSE for simulated depth, and 2.8% re- duction in RMSE for simulated intensity. As illustrated in Figure 6 and Figure 7, GS-LiDAR achieves significantly better visual quality in simulated depth and intensity maps compared to competitors. This improvement is primarily due to the use of 2D Gaussian primitives with periodic vibration properties, enabling precise modeling of both static and dynamic geometries. For the nuScenes dataset, as shown in Table 3, GS- also showcases notable performance, with a 2.5% reduction in chamfer distance for simulated point cloud, a 13.15 1% reduction in RMSE for simulated depth, and a 2.6% reduction in RMSE for simulated intensity.
4.4 ABLATION STUDY
We provide quantitative ablation studies on various components of GS-LiDAR Table As shown in Figure 8, we find that the use of ray-splat intersection enhances the capability of GS-LiDAR in modeling the surface of the scene, and a more detailed discussion provided in Appendix A. For the "w/o ray-splat intersection" case, we implement basic 3D Gaussian splatting approach by adapting the projection calculation method specifically for panoramic maps. The integration of periodic vibration properties further improves GS-LiDAR capability in handling dynamic elements. Regularization terms, including median depth loss, depth distortion loss, and chamfer distance loss, contribute the improved quality of the simulated LiDAR point clouds. Additionally, the ray- refinement technique improves the accuracy of the ray-drop mask, resulting in substantial gains in the metrics for simulated depth and intensity.
5 CONCLUSION
We present GS-LiDAR, a novel framework designed to generate realistic LiDAR point clouds. To uniformly model the accurate surface of various elements in driving scenarios, we employ 2D Gaus- sian primitives with periodic vibration properties. Furthermore, we propose a novel panoramic Gaussian splatting technique with explicit ray-splat intersection for fast and efficient rendering of panoramic depth maps. By incorporating intensity and ray-drop SH coefficients into the Gaussian primitives, we enhance the realism of the rendered point clouds, making them more closely resemble actual LiDAR data. Our method significantly surpasses previous NeRF-based approaches in both computational speed and simulation quality on the KITTI-360 and nuScenes datasets.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work was supported in part by National Natural Science Foundation of China (Grant No. 62376060).
REFERENCES
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In ICCV, 2021.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022.
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. ICCV, 2023.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom, nuscenes: multimodal dataset for autonomous driving. In CVPR, 2020.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In ECCV, 2022.
Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian:
Dynamic urban scene reconstruction and real-time rendering. arXiv preprint, 2023a.
Zhang Chen, Zhong Li, Liangchen Song, Lele Chen, Jingyi Yu, Junsong Yuan, and Yi Xu. Neurbf: A neural fields representation with adaptive radial basis functions. In ICCV, 2023b.
Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In CVPR, 2023c.
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladien Koltun. Carla: open urban driving simulator. In CoRL, 2017.
Haoqiang Fan, Hao Su, and Leonidas J Guibas. point set generation network for 3d object recon- struction from single image. In CVPR, 2017.
Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia, 2022.
Sara Fridovich-Keil. Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In CVPR, 2022.
Sara Tridovich-Keiln Giacomo Meanti, Frederik Rahbak Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In CVPR, 2023.
Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing. In ECCV, 2024.
Benoît Guillard, Sai Vemprala, Jayesh K Gupta, Ondrej Miksik, Vibhav Vineet, Pascal Fua, and Ashish Kapoor. Learning to simulate realistic lidars. In IROS, 2022.
Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. In ICCV, 2021.
Wenbo Hu, Yuling Wang, Lin Ma. Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen Ma. Tri-miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields. In ICCV, 2023.

---

Published as a conference paper at ICLR 2025
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH. 2024.
Jiahui Huang, Zan Gojeic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In CVPR, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM TOG, 2023.
Nathan Koenig and Andrew Howard. Design and use paradigms for gazebo, an open-source multi- robot simulator. In IROS, 2004.
Chenqi Li, Yuan Ren, and Bingbing Liu. Pegen: Point cloud generator for lidar simulation. In ICRA, 2023.
Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE TPAMI, 2022.
Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. NeurIPS, 2020.
Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong. Wenyuan Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang, Wei-Chiu Ma, and Raquel Urtasun. Lidarsim: Realistic lidar simula- tion by leveraging the real world. In CVPR, 2020.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In ECCV, 2020.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with multiresolution hash encoding. ACM TOG, 2022.
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In CVPR, 2021.
Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps. In ICCV, 2021.
Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, and Peter Hedman. Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM TOG, 2023.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In MICCAI, 2015.
Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics: Results of the 11th International Conference, 2018
Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast conver- gence for radiance fields reconstruction. In CVPR, 2022.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofcev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Tang Tao. Longfei Gao, Guangrun Wang. Peng Chen, Dayang Hao, Xiaodan Liang, Mathieu Salz- mann, and Kaicheng Yu. Lidar-nerf: Novel lidar view synthesis via neural radiance fields. arXiv preprint, 2023.
Tang Tao, Guangrun Wang, Yixing Lao, Peng Chen, Jie Liu, Liang Lin, Kaicheng Yu, and Xiaodan Liang. Aligumif: Geometry-aligned multimodal implicit field for lidar-camera joint synthesis. In CVPR, 2024.
12

---

Published as a conference paper at ICLR 2025
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 2004.
Hanfeng Wu, Xingxing Zuo, Stefan Leutenegger, Or Litany, Konrad Schindler, and Shengyu Huang. Dynamic lidar re-simulation using compositional neural fields. In CVPR, 2024.
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Phys- gaussian: Physics-integrate 3d gaussians for generative dynamics. arXiv preprint, 2023a.
Ziyang Xie, Junge Zhang, Wenye Li, Feihu Zhang, and Li Zhang. S-nerf: Neural radiance fields for street views. In ICLR, 2023b.
Weiyi Xue, Zehan Zheng, Fan Lu, Haiyun Wei, Guang Chen, and Changjun Jiang. Geonlf: Geome- try guided pose-free neural lidar fields. arXiv preprint, 2024.
Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic urban scenes. In ECCV, 2024.
Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, et al. Emernerf: Emergent apatial-emppora scene de- composition via supervuuervi...n In ICLR, 2024a.
Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In ICLR, 2024b.
Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P. Srinivasan, Richard Szeliski, Jonathan T. Barron, and Ben Mildenhall. Bakedsdf:: Meshing neural sdfs for real-time view synthesis. arXiv preprint, 2023.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng. and Angjoo Kanazawa. PlenOctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
Junge Zhang, Feihu Zhang, Shaochen Kuang, and Li Zhang. Nerf-lidar: Generating realistic lidar point clouds with neural radiance fields. In AAAI, 2024.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, 2018.
Zehan Zheng, Fan Lu, Weiyi Xue, Guang Chen, and Changjun Jiang. Lidar4d: Dynamic neural fields for novel space-time view lidar synthesis. In CVPR, 2024.
Wojeiech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollhöfer, Justus Thies, and Javier Romero. Drivable 3d gaussian avatars. arXiv preprint, 2023.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 RAY-SPLAT INTERSECTION
Although 3DGS performs well in camera image rendering, it not suitable for panoramic LiDAR rendering. This limitation arises from the complexity of transforming 3D spatial points into LiDAR range map, which makes challenging for the Jacobian matrix of the transformation to accurately approximate the effects of such transformation. Given point (z, v) in the camera space, its corresponding radian angles ( 0) can calculated as:
( = attan((v2 ( atam2(s, 2) +2, )
(22)
By combining this with Equation from the paper, we can derive the projection of this point onto the LiDAR range map, resulting in its pixel coordinates (5, 7)) as:
W 2 atan2(xx, 2) H otan VFOV (VFOV may- VFOV
{ 11
(23)
Thus, the Jacobian matrix can be computed as follows:
W デ++z
(24)
H
92
Vz
where W = W. and 11 VFOV VFOV min
H
We tile the plane with 2D and 3D Gaussian primitives of different colors and render their color and depth. As illustrated Figure 9, we can observe that for 3D Gaussian primitives, due to the poor ap- proximation of the Jacobian for the projection transformation. the originally tightly arranged points become scattered, which also leads to the discontinuity in depth shown Figure of the paper. In contrast, the 2D Gaussian primitives, being based ray intersection without approximation, exhibit the desired effect.
2DGS
3DGS
RGB
Depth
Figure 9; Ray-splat intersection v.s. 3D Gaussian splatting.
A.2 EXPERIMENTS WAYMO DATASET
We additionally conducted experiments sequences selected by PVG (Chen et al., 2023a) from the Waymo (Sun et al., 2020) dataset. Since methods like LiDAR4D (Zheng ct al., 2024) have not been implemented on the Waymo dataset, we only report own results in Table 5. The experimental results are consistent with those KITTI-360 and nuScenes across the evaluation metries, showing the scalability and generalization ability of our method. We have also additionally visualized LiDAR point clouds from the Waymo dataset Figure 10 and Figure 11, showcasing the generalization ability of our approach again.
14

---

Published as a conference paper at ICLR 2025
Table 5: Metrics Waymo dataset.
Method
CDL 0.2382
Point Cloud
F-scoref
RMSE] 5.8925
ModAEJ
Depth LPIPSL 0.0708
SSIM
PSNR| 22.2482
RMSE,
MedAE, 0.0067
Incensity
LPIPS
SSIM
PSNRI 27.7420
IDAR (Ours)
0.7291
66-1 (Dun))
6C LOAR IOurs
Figure 10:
Comparison of the rendered depth and intensity map with ground truth. The first two rows depict the depth maps, while the subsequent two rows illustrate the intensity maps.
Figure 11: The visualization of LiDAR point clouds in three-dimensional space.
15


Reviews:
[{"review_id": "AtJJLkxgy8", "reviewer": "Reviewer_aQ3K", "summary": "This paper proposes GS-LiDAR, which introduces a novel framework for LiDAR novel view synthesis (NVS). Compared to previous NeRF-based works designed with fixed resolution and symmetrical structure, this paper employs 2D Gaussian primitives with periodic vibration properties to reconstruct the dynamic driving scenarios more effectively. Furthermore, based on the panoramic LiDAR rendering technique, the authors calculate the ray-splat intersection to achieve explicit and accurate results. And the spherical harmonic (SH) of Gaussian primitives can also support intensity and ray-drop reconstruction. Experiments on KITTI-360 and nuScenes demonstrate the superiority of GS-LiDAR in reconstruction quality and rendering efficiency.", "questions": "Besides the concerns mentioned in the weaknesses, there are additional questions as follows: \n\n- It introduces normal consistency term in the loss function and uses depth gradient as pseudo ground-truth. However, will it produce incorrect results due to LiDAR projection, ray-drop or at object edges? Is it more plausible to use 3D point clouds for normal estimation? In addition, a comparison of this loss term is missing in the ablation study.\n\n- How to initialize Gaussian points for a dynamic scene? If all point clouds are aggregated and sampled, this may lead to unreasonable initialization of dynamic objects (especially with long trajectories). Furthermore, are different scenes with different sequence lengths and ranges initialized using the same number of points? And how about the final number of Gaussian points? \n\n- As can be seen in Figure 1, the results of 3D-GS are very poor, but the paper lacks details of its implementation and the analysis of its failure. Since 3D-GS could not be directly migrated to LiDAR-NVS, relevant explanations are necessary.\n\n- Is it reasonable to model long trajectory motion with periodic vibration properties? It would be better to visualize the motion trajectories of Gaussian primitives to prove their temporal consistency.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Previous NeRF-based works with Hashgrid have a symmetrical structure and fixed resolution for each axis, which is not suitable and effective enough for driving scenarios. This paper first adopts GS representation for LiDAR NVS, making the reconstruction and rendering more efficient.\n\n- 2D Gaussian primitives have higher geometric accuracy. Moreover, it incorporates the periodic vibration properties, allowing for both static and dynamic scene reconstruction.\n\n- Based on the projection nature of LiDAR point clouds, this paper utilizes the panoramic rendering technique to accommodate Gaussian Splatting, and further proposes ray-splat intersection to obtain more accurate rendering results.", "weaknesses": "- In terms of the core methodology, this paper is more of a combination of existing methods (2D-GS and PVG) and does not substantially innovate them, which results in a limited contribution. It should include more insights and analysis of the methods and why they are well suited for LiDAR-NVS. \n\n- The concept of panoramic Gaussian splatting is a little bit confusing in Section 3.3. And according to the proposed ray-splat intersection, it's more like the \"ray tracing\" manner than the \"splatting\" one. If so, it needs to compute the intersection of every ray and all Gaussian primitives, which is still not efficient enough.\n\n- Despite the efficiency gains due to the Gaussian itself, the final rendering results don't seem to be a huge improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7dEaNH2pTI", "reviewer": "Reviewer_YP8o", "summary": "This paper addresses the task of LIDAR novel view synthesis and introduces a framework called GS-LIDAR, which generates realistic LiDAR point clouds using panoramic Gaussian splatting. To accurately reconstruct both static and dynamic elements in driving scenarios, the authors utilize 2D Gaussian primitives with periodic vibration properties. Additionally, the paper presents a novel panoramic Gaussian splatting technique that incorporates explicit ray-splat intersection for rapid and efficient rendering of panoramic depth maps. Experimental results demonstrate the framework's state-of-the-art performance in terms of quantitative metrics, visual quality, and efficiency.", "questions": "1. The novelty of the proposed method, especially in comparison to LIDAR4D, requires clearer articulation, which should focus on  how the paper's contributions go beyond simply replacing NeRF with Gaussian Splatting.\n2. The Method section includes extensive background information on existing techniques, such as ray-drop refinement and various loss functions, which do not directly contribute to the core innovations of the paper. Consider condensing these parts or moving them to the supplementary materials. What's more, a more detailed explanation and accompanying diagram (i.e. a geometric illustration) of the Ray-splat intersection would improve clarity and focus.\n3. Please explain in detail how the addition of the Ray-splat intersection mechanism significantly enhances the performance of the proposed method.\n4. Additional experiments and analyses, such as testing on the Waymo dataset  with different sensor configurations or environmental conditions, would further validate the effectiveness and generalization ability of the approach.\n5. Typo: In Line 323, the ray-drop probability signal $\\rho_{k}$ is wrongly spelled by $\\lambda_{k}$.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is generally well-presented, with a system architecture that is clearly outlined and easy to follow. Additionally, the experimental results—encompassing both quantitative and qualitative evaluations—are impressive, demonstrating performance that surpasses the current state-of-the-art in this specific task. Furthermore, the ablation study is thorough and effectively substantiates the proposed approach.", "weaknesses": "1) This article appears to present an incremental improvement rather than a significant innovation. The primary contribution seems to be the replacement of NeRF in LIDAR4D with Gaussian Splatting. While this method offers advantages in training and rendering speed compared to LIDAR4D, these benefits are inherently tied to the properties of Gaussian Splatting itself.\n2) Regarding the methodology and experiments, the most notable innovation appears to be the Ray-splat intersection in panoramic Gaussian Splatting. However, this aspect is not clearly articulated in the methods section. For instance, in Line 249, the physical significance of the variables u and v is not intuitively conveyed, nor is the derivation of the transformation from UV space to screen space adequately explained. The authors should provide a more detailed explanation of this section or include a schematic diagram  (i.e. a geometric illustration) to enhance reader comprehension.\n3) Furthermore, the authors need to conduct a thorough analysis of how the Ray-splat intersection, as demonstrated in the ablation experiment, enhances the capability of GS-LiDAR in accurately modeling surface features of the scene and leads to significant performance improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xmAGh8IieU", "reviewer": "Reviewer_t1nR", "summary": "This paper introduces a novel method for LiDAR novel view synthesis called GS-LiDAR. GS-LiDAR utilizes 2D Gaussian Splatting (2DGS) as a foundational representation for LiDAR reconstruction and incorporates periodic vibration for enhanced results. The reconstruction process is supervised by panoramic LiDAR data. The method is thoughtfully designed, incorporating various strategies to adapt 2DGS to LiDAR NVS scenarios, and is backed by extensive experimentation to demonstrate its performance. However, several concerns are noted below.", "questions": "1. Can the proposed method handle solid-state LiDAR?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-structured and easy to read.\n2. The application of Gaussian Splatting to achieve LiDAR novel view synthesis is an innovative approach.\n3. The method is well-conceived, integrating the unique properties of LiDAR point clouds into 2DGS.", "weaknesses": "1. The title of the paper suggests that the method is intended for “Generating” LiDAR point clouds. However, GS-LiDAR is designed for “reconstructing” LiDAR point cloud scenes. While the method does achieve novel view synthesis, the concept of reconstruction differs from generation. Generation [1][2] implies the creation of LiDAR point clouds from other user inputs, not just from an existing sequence of point clouds.\n2. The rationale for using 2DGS instead of 3DGS as the primary representation is unclear. Although 2DGS has demonstrated superiority in geometric reconstruction, a comparative analysis between 2DGS and 3DGS (with the proposed strategies, rather than a standard 3DGS) would strengthen the paper.\n3. The performance improvement over LiDAR4D, as shown in Tables 1 to 3, appears marginal. Additionally, the visualization results in Figure 4 do not clearly demonstrate significant performance gains. Including comparisons in video format would help substantiate the results.\n\n[1] Ran, Haoxi, Vitor Guizilini, and Yue Wang. \"Towards Realistic Scene Generation with LiDAR Diffusion Models.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2024.\n\n[2] Zyrianov, Vlas, et al. \"LidarDM: Generative LiDAR Simulation in a Generated World.\" *arXiv preprint arXiv:2404.02903* (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hZwJDgGhce", "reviewer": "Reviewer_K7sa", "summary": "In this paper, the authors present GS-LiDAR, a novel framework for generating realistic LiDAR point clouds. They employ 2D Gaussian primitives with periodic vibration properties to precisely model both static and dynamic elements in driving scenarios. Additionally, they introduce a panoramic rendering technique to obtain depth, intensity and ray-drop probability maps.", "questions": "1）\tThe presentation of lidar rendering is not clear enough to make it difficult to follow. The authors highlight that they introduce a novel panoramic rendering technique with explicit ray-splat intersection, guided by panoramic LiDAR supervision. However, it is even unclear whether the image is rendered by ray marching, rasterizing or a hybrid approach.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1)\tThe authors successfully take advantage of periodic vibration 2D Gaussian primitives to reconstruct dynamic point cloud from lidar measurements, and achieve state-of-the-art performance in terms of rendering speed and accuracy.", "weaknesses": "1）\tInsufficient reference. The authors emphasized they employed 2D Gaussian primitives with periodic vibration properties and counted it as one of the contributions. However, periodic vibration and 2D Gaussian primitives are proposed by [1] and [2] respectively, which haven’t been credited in the introduction section.\n\n[1] Huang, Binbin, et al. \"2d gaussian splatting for geometrically accurate radiance fields.\" ACM SIGGRAPH 2024 Conference Papers. 2024. \n[2] Yurui Chen, et al. \"Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering.” arXiv preprint, 2023a.\n\n2）\tAmbiguous symbol. (u, v) in Line 204 and equation 2 denotes the target pixel of the rendering image, but later it becomes the coordinates within the local tangent plane space of 2D Gaussian in Line 245-255. \n\n3）\tIncorrect definition. The formulations of intensity map I in Line 316 and ray-drop probability map P_gs in Line 32 are identical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)