LiT: Unifying LiDAR "Languages" with LiDAR Translator
Authors: Yixing Lao
Tao Tang
Xiaoyang Wu
Peng Chen
Kaicheng Yu
Hengshuang Zhao
Venue: NeurIPS 2024 poster (2024)
Keywords: LiDAR Translation
Domain Unification
3D Detection
Primary area: machine_vision
TL;DR: LiT, the LiDAR Translator, directly translates point clouds across domains, unifying diverse LiDAR datasets to support scalable cross-domain and multi-domain learning for autonomous driving.

Abstract: LiDAR data exhibits significant domain gaps due to variations in sensors, vehicles, and driving environments, creating “language barriers” that limit the effective use of data across domains and the scalability of LiDAR perception models. To address these challenges, we introduce the LiDAR Translator (LiT), a framework that directly translates LiDAR data across domains, enabling both cross-domain adaptation and multi-domain joint learning. LiT integrates three key components: a scene modeling module for precise foreground and background reconstruction, a LiDAR modeling module that models LiDAR rays statistically and simulates ray-drop, and a fast, hardware-accelerated ray casting engine. LiT enables state-of-the-art zero-shot and unified domain detection across diverse LiDAR datasets, marking a step toward data-driven domain unification for autonomous driving systems. Source code and demos are available at: https://yxlao.github.io/lit.

LiT: Unifying LiDAR "Languages" with LiDAR Translator
Yixing Lao
Tao Tang Sun Yat-sen University ttang@mail, sysu. edu. en
Xiaoyang Wu
The University of Hong Kong yxlaoêcs hku.hi hk
The University of Hong Kong xywuêcs hku hk
Peng Chen
Kaicheng Yu
Hengshuang Zhao
Cainiao Group
Westlake University yulaaichengdwesttlakee edu.cn
The University of Hong Kong hszhaouca hku.
yuanshang www.:aaiancpcciiii.c..oo com
Abstract
LiDAR data exhibits significant domain gaps due to variations in sensors, vehicles, and driving environments, creating "language barriers" that limit the effective use of data across domains and the scalability of LiDAR perception models. To address these challenges, introduce the LiDAR Translator (LiT), framework that directly translates LiDAR data across domains, enabling both cross- adaptation and multi-domain joint learning. LiT integrates three key components: a scene modeling module for precise foreground and background reconstruction, a LiDAR modeling module that models LiDAR rays statistically and simulates ray- drop, and fast, sandware-aacclerattee ray casting engine. LiT enables state-of-the- zero-shot and unified domain detection across diverse LiDAR datasets, marking a step toward data-driven domain unification for autonomous driving systems. Source code and demos available at: https: http:://xxlao.../// github. io/lit.
Frame #000
Groand-rull
Geoursl-tuuch
pathern
pattem
Frame #020
Frame #040
Frame #060
Source LiDAR (Waymo, 64-beam)
Source scene modeling
Translated LiDAR (nuScenes, 32-beam)
Translated LiDAR (KITTI, 64-beam)
Figure 1: LiT translates LiDAR scenes across domains, capturing target domains' characteristics.
By unifying the LiDAR "languages" LiT enables effective zero-shot and multi multi-dataset joint learning.
"Corresponding authors.
38th Conference Neural Information Processing Systems (NeurIPS 2024).

---

Introduction
LiDAR provides accurate geometry measurements of the surrounding environment, making it one of the most popular sensor choices for various autonomous systems [1, 3, 4, 5]. Despite its importance, perception models trained on a specifie LiDAR setup often struggle to generalize across different sensors [6, 7, 8], primarily due to the domain gaps attributable to variations in sensor intrinsic characteristics and driving environments. These gaps, akin to "language barriers," " significantly degrade model performance when transferring between different LiDAR sensors and hinder effective joint training across multiple datasets. In a real-world scenario. it takes tremendous time and resources to collect, annotate, and train just one model for specific LiDAR setup [9, 10, 11J. Consequently, this barrier acts as a bottleneck in training scale-up, which has been a driving force behind recent rapid advancements in other areas of representation learning, such as 2D vision [12, 13, 14, 15] and natural language processing (NLP) [16, 17, 18, 19]. Addressing the "language barriers" in the context of LiDAR data therefore erucial for unlocking similar advancements within autonomous driving.
Recent efforts [6, 20] have already acknowledged the significance of "language barriers" within LiDAR data meaningful challenge. However, solutions to date primarily focus on model-driven approaches, attempting to bridge these gaps through adaptations within the perception model itself. While these methods have shown promise in bridging domain gaps in certain contexts, they introduce a notable drawback: the cost of customizing model structure and training data for new, specific domains. This requirement demands significant resources and limits the scalability and flexibility of deploying autonomous systems across varied environments.
To this end, we present the LiDAR Translator (LiT), novel data-driven approach that functions similarly to a language translator, converting disparate LiDAR "dialects" into a common "language" (Fig. 1), and further unleashing the zero-shot capacity and joint training potential of LiDAR-based models. The overall pipeline is available in Fig. 4. Specifically, LiT reconstructs the source do- main scene with neural implicit representations, capturing both the static background and dynamic foreground elements (Sec. 4. Then, LiT simulates the target domain LiDAR sensor model using custom GPU-accelerated ray casting engine, generating translated LiDAR scans that faithfully replicate the target domain's 's characteristics (Sec. 4.2). By doing so, LiT effectively bridges the gap between different LiDAR "languages." " enabling seamless domain adaptation and unification of disparate LiDAR datasets. Through this well-designed framework, LiT effectively bridges the gap between different LIDAR "languages," facilitating seamless domain unification of LiDAR datasets.
LiT advances domain adaptation through data-side unification and multi-domain joint learning [6]. We show in experiments that, in single-source adaptation scenarios, LiT demonstrates superior zero-shot detection performance compared to existing model-driven and data-driven techniques. Addi- tionally. by translating multiple data sources into unified target domain, LiT facilitates multi-dataset joint training, where combining data from various sources not only improves model performance beyond single-source training but also establishes a new paradigm for data-centric learning strategies.
2 Related work
3D representation learning. In the realm of autonomous driving, achieving the unification of perception models for LiDAR data across different domains presents a distinct challenge, one that diverges from the significant progress observed in 2D domains where large-scale pre-training has dramatically enhanced downstream task performance [21]. Specifically in the domain of 3D representation learning, the field is still in an exploratory phase. Many existing studies have focused on training models from scratch, tailored to specific datasets [22]. Initially, this research largely targeted the recognition of individual objects [23, 24, 25, 26, 27]. Over time, it has evolved to address the more complex task of interpreting real-world, scene-centric point clouds [22, 28, 29, 30], marking notable advance in our understanding of 3D data. Inspired by the achievements of large-scale representation learning and the effectiveness of multi-dataset synergistic approaches, such as Point Prompt Training (PPT) [6], LiT aims to overcome the constraints of model-side adaptations by utilizing a data-driven approach to bridging domain gaps.
LiDAR domain adaptation. LiDAR domain adaptation is critical for enabling models trained on one domain to effectively operate in another, accommodating variations such as sensor types and environmental conditions [31, 32, 33, 34, 35]. Techniques have varied widely: SN [36] tackles

---

Table 1: Vanilla zero-shot capacity. Without LiDAR translation, the performance of the 3D object detection model drops dramatically when applied to different domains. LiT is capable of translating LiDAR data across domains and can significantly improve the performance of the 3D object detection
model. BPBEY and 3D of the car category at loU = 0.7 of the SECOND--IOU [50] model are shown.
Waymo + KITTI
Waymo - nuScenes
nuScenes -> KITTI
Method
APBEV |
APso
APBEV 1
APso
APBEV f
APıD
Without translation With LiT translation
67.64 82.55
27.48
32.91
17.24
51.84 80.54
17.92
69,94
37.00
22.19
60.13
object size discrepancies through normalization based on object statistics; ST3D [7] and ST3D++ [8] utilize a self-training pipeline with pseudo-labels for fine-tuning; LiDAR-Distillation [37] mitigates beam-induced domain shifts with a progressive framework; SPG [38] aims to fill in missing points in foreground areas; and 3D-CoCo [39] employs
encoders and contrastive learning for transferable representation. LiT aligns with these efforts by employing zero-shot object detection as primary benchmark measure its effectiveness in domain adaptation. However, LiT's ambition extends beyond mere adaptation: it aims to unify the "languages" of multiple LiDAR domains, exploring the synergistic potential of such unified data.
Autonomous driving simulator. Simulators play an important role in autonomous driving re- search by making data collection and annotation much easier and cheaper. Initial efforts leveraged ground-truth information extracted from graphic engines, such as CARLA [40], by utilizing various simulators [41, 42, 43, 44, 45]. Despite their utility, these sim-to-real approaches often involve complex dependencies on simulator engines, which can be cumbersome and costly to establish, and still result in a notable sim-to-real gap. In response to these challenges, subsequent studies like LiDARsim [46] and PCGen [47] have attempted to bridge this gap by simulating point clouds from real data through multi-step, data-driven pipelines. More recent advances, including UniSim [48] and L.DAR-NeRE [49], have explored the use of neural implicit fields to simulate new sensor views. However, these methods primarily generate data reflective of the source domain and do not adequately address variations across datasets. Different from these works, LiT enables direct translation of real-world source data into target domain LiDAR characteristics, effectively bridging the domain gap through simulation.
3 Pilot study
3.1 Exploring language barriers among LiDARs
The progression of autonomous driving technologies is intimately tied to the diversity and quality of LiDAR data, which provides detailed three-dimensional representation of environments. Operating in varied settings, autonomous vehicles utilize diverse LiDAR sensors, each characterized by unique specifications like field of view, beam count, and ray-drop behavior. This results in notable variations across datasets such as Waymo [10], nuScenes [11], and KITTI [9], creating significant domain gaps akin to language barriers between LiDAR "languages".
These gaps manifest in various forms, including alterations in point cloud density due to beam count differences and changes in spatial coverage from FoV variations. Additionally, scene content differences (such as vehicle sizes, orientations, and types) mirror the geographical and operational diversity of each dataset, adding layers of complexity the comain--peciffi challenges (as visualized in Fig. 2). Such disparities obstruct the direct application of models trained on one dataset to another (see Table 1) and further pose significant obstacles to exploring synergistic training across these datasets. By tackling these barriers head-on, LiT aims to unlock the potential of data scale.
We wish to break the language barriers among LiDARs with our LiDAR Translator.
3.2 Embracing data-driven domain unification
Previous attempts to bridge domain gaps [7, 8), particularly through unsupervised domain adaptation techniques, have achieved limited success. These model-based approaches focus on adapting the model to fit the target domain, often neglecting the underlying variations in data characteristics. As
3

---

LiDAR
Frequency
Frequency
Prequency
vertical
angle
-00
10
90 25
10
20
Vertical
Vertical anglo
Vertical angle
Dataset vehicle size
Widh
Widhh
200
20-
20-
4.0
55
60
3.5
3.5
5.0 Length
5.5
Longth
Length
Ray cast at 20 meters
Waymo
nuSeenes
KITTI
Figure 2: Domain gaps for LiDARs. Top Row: LiDAR ray angles have significantly different distributions. We model their statistical distributions detailed in Sec. 4.2. Middle Row: Foreground vehicle sizes can differ across datasets. Bottom Row: We show the ideal ray casting results from LiDARs mounted 1.6 meters height reconstructed vehicle placed 20 meters in front.
No adaptation
(c) LiT: Data-based adaptation with LiDAR translation
Train
Eval
Translate
Train
Eval
LiDAR
Model
LiDART
LiDARS
LiDAR T(S)
Model
LiDAR T
Translate
Model-based adaptation
LIDAR S
LIDAR T(S,)
Joint
Adapt
Translate
train
Eval
LiDAR S,
LiDAR T(S,)
Model
LiDAR T
Train
LiDAR S
Model
LiDART
Translate
Eval
LIDAR S
LIDAR T(S
Figure 3: Comparing LiT with model-based adaptation. (a) Training a model on source domain data and directly applying to the target domain typically results in poor performance due to the domain gap. (b) Model-based adaptation techniques [7] adapt the model to the target domain but do not explicitly model the target domain data LiDAR characteristics and data distribution. (c) LiT directly translates LiDAR data from the multiple source domains to a unified target domain, effectively bridging the domain gaps and enabling joint training across multiple datasets.
result, while they may narrow the domain gap to a degree, they do not fully exploit the potential of diverse datasets bolster model robustness and generalization.
Contrary to model-side adaptation, data-driven adaptation presents a more practical and efficient avenue for LiDAR domain unification without the need for additional adjustment the base model. By translating LiDAR data from various source domains into a standardized target domain "language", can unify discrepancies across datasets. This harmonization creates cohesive dataset that reflects the diversity of multiple sources, addressing the issue of negative transfer and enriching the training data to boost model performance and generalizability in different settings.
Specifically, we can translate data from an available source domain to an unseen target domain, can further use the translated data for model training and fulfill downstream tasks on the target domain, effectively achieving zero-shot cross-domain learning. Moreover, we can unify multiple source domains into target domain, model training can further take advantage of larger scale of data training to push the model's generalization capabilities to a higher level, as illustrated Fig. In word, embracing data-driven domain unification has great potential in both zero-shot adaptations from source to target and large-scale multi-domain joint training.
We treat data-drivg driven domain unification as the original point of our LiDAR Translator.

---

Source LiDAR scans
Foreground & background modeling
Ray casting with target LiDAR
Translated LiDAR scans
Figure 4: LiT pipeline overview. LiT translates LiDAR data across domains, integrating scene modeling (Sec. 4.1) and LiDAR modcling (Sec. 4.2) with GPU-accelerated ray casting. LiT highly efficient, as it can translate multi-frame LiDAR scene in typically less than one minute (Table 6).
4 LiDAR translator
4.1
Scene modeling
Foreground modeling. Previous work ReSimAD [51] relies on hand-crafted 3D vehicle assets from simulator engines [40], which limits the diversity of the foreground objects and the realism of the simulated data. In contrast, LiT reconstructs the foreground objects from multi-frame LiDAR point clouds, which are more representative real-world data with diverse vehicle geometries. To achieve this, we first track the vehicles across multiple LiDAR frames, where the tracking information typically provided by LiDAR datasets 19, 10, 11]. Inspired by the recent success of implicit neural representations [52, 53] in reconstructing shapes from LiDAR point clouds, we train an SDF-based reconstruction model on ShapeNetV2 [54] vehicle category and apply reconstruct the foreground objects from mulli--ramee LiDAR point clouds. With this approach, we can accurately reconstruct the detailed geometry of diverse vehicle shapes. As shown our experiments in Table the increase in foreground diversity leads to significant performance increase in zero-shot detection tasks. We refer the reader Sec. A.1 for details and visualizations on the foreground modeling.
Background modeling. The primary goal of background modeling is to faithfully reconstruct the background environment. However, the efficiency and scalability of the background modeling method are equally important, especially when dealing with joint training on multiple large-scale datasets. ReSimAD [51] uses NeuS [55] for background reconstruction, which requires custom training for each scene with time measured in hours. In contrast, LiT models the scene as the zero-level set of 3D implicit field defined by a hierarchical neural kernel field [56, 57J. This approach is generalizable, eliminating the need for retraining on every new LiDAR sequence and significantly reducing LiT's full-scene translation time less than minute for 200-frame LiDAR scene, detailed in Table This efficiency plays a crucial role in enabling the scalability of LiT across various datasets. We refer readers Sec. A.2 in the supplementary material for further details and visualizations on background modeling.
Module adaptability. Both the foreground and background modeling crucial modules for achieving high fidelity in scene representation, enabling accurate domain adaptation. LiT, these modules are designed be swappable, allowing for the integration of more advanced scene representation methods as they are developed. While the specific modeling techniques are important, primary contribution lies in how these components work together within domain translation framework to enable effective domain unification.
4.2 LiDAR modeling
Modeling the different LiDAR characteristics is a critical step in domain adaptation, it directly affects the simulated data distribution and characteristics of the target domain. Unlike existing work [51] that relies on full-seale simulators, we develop custom LiDAR modeling and ray casting pipeline that is more flexible and efficient, and it does not require complex dependencies when using simulator engines. We model the LiDAR characteristics of beam angles. All of these are made possible by our hardware-acceleratee LiDAR ray casting engine.
5

---

Statistical modeling of LiDAR ray angles. The naive implementation for LiDAR ray gen- eration is to evenly distribute the rays in the hor- izontal axis, based on the LiDAR's s horizontal field of view (FOV) and beam counts. How- ever, this approach does not accurately reflect the real-world LiDAR characteristics, which are often non non-homonn homogeneous in the vertical direction, shown in the top of Fig. 2. To address this issue, we conduct a statistical analysis of LiDAR rays, focusing on the ray distribution in the vertical axis. First, for each LiDAR ray in the dataset, compute its vertical angle based on the ray's xy-plane distance and z-axis dis- tance. Then, we calculate the distribution of the vertical angles and identify the peak angles where the majority of the LiDAR rays are con- centrated. Specifically, this is done by finding H statistical modes in the distribution of vertical angles, where H is the height of the range image or the number of beams the LiDAR sensor. Notably, we do not require manual labeling of target domain data, only small amount target domain LiDAR data to perform this statistical analysis. After the modeling, our LiDAR ray generation module then generates rays based on these identified peak angles, ensuring that our simulated LiDAR data representative of real- world LiDAR characteristics. Fig. 5 shows the average Chamfer distance between the real-world LiDAR data and the self-to-self translated LiDAR data on nuScenes. The chamfer distance is reduced when statistical modeling is applied (blue curve) compared to the vanilla case (red curve), indicating that the simulated LiDAR data more closely resemble the real-world LiDAR data.
Vanilla
With statistical modeling
With statistical and ray-drop modeling
2.0
Probability density
1.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Chamfer distance (clipped at 1.0)
Figure 5: LiDAR modeling with statistical ray angles and ray-drop. We compare the chamfer distance between the real-world LiDAR data and the simulated LiDAR data on nuScenes. The cham- fer distance is smallest when both statistical model- ing and ray-drop modeling are applied. The cham- fer distances are clipped 1.0m for visualization.
LiDAR ray-drop modeling. Ray-drop of LiDAR rays is frequent in real-world scenarios due to various factors, including object reflectivity, absorption, or occlusion. We model this phenomenon by training a ray-drop predictor model that predicts the likelihood of a ray being dropped. To collect the ray-drop dataset, we reconstruct background and foreground scenes and perform ray casting using the dataset's 's own LiDAR parameters. For each ray. we record whether it hits a mesh or not in the simulated scene. Inspired by PCGen [47], we collect the ray direction r, the point distance and the ray incident angle / for each ray. Then, we train an MlP-b ray-drop model with positional encoding [58], to predict the likelihood of ray being dropped. The MLP network takes (r, d, f) with positional encoding as input, and outputs the probability of the ray being dropped.
For quantitative evaluation, perform self-to-self translation task the nuScenes dataset to assess the quality of the ray-drop modeling. Fig. 5 shows the chamfer distance decreases when ray-drop modeling is applied, and it shows the combined effects of applying both statistical modeling and ray- ray-drop modeling together. For qualitative evaluation, we present the visualization of the ray-drop model trained on the nuScenes dataset in Fig. 6. The simulated LiDAR points, especially those proximal to vehicles, effectively resemble the nuScenes dataset's 's characteristic patterns.
Translated target LiDAR without ray-drop
Translated target LiDAR with ray-drop
Ground-truth target pattern from a different scene
Figure 6: Rry-drop modeling visualization. The left image shows the translated nuScenes frame without ray-drop modeling, where it has dense circular LiDAR pattern near the vehicle. The middle image, with ray-drop modeling applied, displays sparser LiDAR points near the vehicle, closely resem- bling nuScenes' sean patterns. The right image is real nuScenes LiDAR sean from another scene.

---

Figure 7: Deploying LiT- LiDAR to new scene. The LiT pipeline allows flexible compo- sition of LiDARs and scenes, even when the scene is not modeled by LiT. We show visualizations of the synthetic Mai City scene [59] (column 1), and the LiT-simulated LiDAR scans in nuScenes patterns following moving vehicle trajectory (column 2-4).
Hardware accelerated LiDAR ray casting. While existing work [51] relies on integrating with full-scale simulators like CARLA [40], which can be cumbersome to recompile and less efficient to run, we have developed a bardware-accellratee LiDAR ray casting engine for LiT that offers greater flcxibility and efficiency. Specifically, modeling, computes the location of the ray-mesh hit point, the surface normal at that point, and the incident angle of the ray, providing the necessary data to simulate target domain points. Given the surface normal n of the triangle hit by the ray and the ray direction r, the incident angle is computed as 0 = arccos ((r n)/ /([[rr] [m]))) /(])) LiT's ray casting engine is accelerated
LiDAR ray casting engine emits rays based on the LiDAR
both CPU with Intel Embree and GPU with Nvidia OptiX, enabling real-time ray casting (22Hz to 31Hz) when running on single GPU. The efficiency of this engine allows LiT to translate multi-frame LiDAR scene typically in less than one minute, making large-scale domain unification tasks feasible. For more detailed runtime statistics, refer to Table 6.
Flexible composition of LiDARs and scenes. LiT's flexible pipeline allows seamless deployment of LiT-modeled LiDARs to new environments that are not necessarily modeled by LiT. As demonstrated Fig. 7, we can deploy LiT-modeled LiDAR nuScenes's style in a given new environment, such as the Mai City [59] scene. This flexibility allows LiT to work with various sources of reconstructed 3D maps, while being able to simulate LiDAR scans with target LiDAR sensor characteristics, which particularly useful for deploying new LiDARs or simulating out-of distrritittiittinnnn scenarios.
Experiment
5.1
Experimental settings
Datasets. Our experiments are conducted on three widely utilized datasets for autonomous driving research: KITTI [9]. Waymo [10], and nuScenes [11]. These datasets present diverse range of LiDAR characteristics, including varying beam counts and sensor configurations, making them ideal for evaluating LiDAR translation approach, LiT. Adaptation scenarios covered include cross-beam-count adaptations such as Waymo -> nuScenes (high-beam to low-beam) and nuScenes KITTI (low-beam high-beam), as well adaptations within the same beam count but different LiDAR characteristics like Waymo- - KITTI. Table 6 provides an overview of the dataset statistics, including details on LiDAR beams and the distribution of foreground objects.
Evaluation metrics. To evaluate how well our translated point clouds match the statistical distribu- tion of the target domain, we follow LiDARGen [60] and Ultral [61] to compute Maximum- Mean Discrepancy (MMD) and Jensen- Shannon divergence (JSD) in BEV voxel occupancy between the source domain point clouds translated to the target domain style and the actual target domain point clouds. As the source and target domains come from entirely different scenes, we measure the overall distributional similarity rather than per-scene alignment, providing a quantitative measure of how well LiT captures the general characteristics of the target domain.
The downstream detection task evaluation follows the KITTI benchmark standards, focusing on the car category. We report Average Precision (AP) across recall positions at an Intersection over Union (IoU) threshold of 0.7 for both bird's eye view (BEV) and 3D detection. For KITTI, metrics are reported for front-view annotations only, while for Waymo and nuScenes, we extend our evaluation to include full ring-vie view point clouds generated from tou-moddted LiDAR sensors, while other LiDARs
7

---

Table 2: Statistical alignment with target domain. We report the distributional differences between translated and ground ground-truth target domains with Maximum- Mean Discrepancy (MMD) and Jensen- Shannon divergence (JSD). Baseline represents LiDAR data from the source domain.
MMDBEV
JSD
Task
Beam count
Bascline
LiT
Bascline
LiT
Waymo KITTI
64-bcam + 64-beam
8.817c-04
3.268e-04
0.273
0.180
Waymo nuScenes
64-bcam + 32-bcam
2.310c-03
6.583e-04
0.380
0.205
nuSecnes + KITTI
32-bcam + 64-bcam
8.725c-04
2.107e-04
0.220
0.164
Table 3: Single source domain unification. We compare the APBEV and AP3D of the car category
at IoU = 0.7 as well as the domain gap closed by different methods. Source only denotes that the pre- mreitranned detector is directly evaluated on the target domain, and Oracle represents the detection results trained on the fully annotated target domain. We highlight the best results in bold.
SECOND-IoU (50)
PV-RCNN [62]
Task
Method
APRrV 1/ AP
Closed gap
APpry A 1
Closed gap
Source only
67.64 / 27.48
61.18/2.00
[36]
78.96 / 59.20
+72.33% / +69.00%
79.78 63.60
+66.91%
+68. 76%
Waymo-
KITTI
ST3D [7]
82.19/ 61.83
+92.97% / +74.72%
84.10/ 64.78
+82.45%
+70.71%
ReSimAD [51]
81.01 / 58.42
+71.33%
+60.199 19%
LiT
82.55/ / 69.94
+95.27%
+92.36%
84.35 65.68
+83,35%
/ 172.199
Oracle
83.29 73.45
88.98 82.50
Source only
32.91 / 17.24
34.50 21.47
SN [36]
33.23 / 18.57
+01.69% / 54%
34.22 / 22.29
.01.50%/ +0%.80%
Waymo + nuSeenes
ST3D [7]
35.92 20.19
+15.87% +16.73%
36.42/ 22.99
+10.32%
+08 89%
ReSimAD [SI]
37.85 21.33
+18.00%
-10.81%
LiT
37.00/ / 22.19
+21.56% / +28.08%
38.77 23.48
+22.94% +11.76%
Oracle
51.88 / 34.87
53.11 38.56
Source only
51.84 17.92
68.15 37.17
[36]
40.03 21.23
-37.55% +05, +05.96% +76.63% / +59.50%
60.48 49,44 .47
-36.82%
+27.13%
nuSeenes KITTI
ST3D [7]
75.94 54.13
78.36 70.85
+49.02% +74.30%
ReSimAD [51]
LiT
80.54 / 60.13
+91.26% / +76.01 %
85.82/ / 74.87
+84.83% / +83.17%
Oracle
83.29 73.45
88.98 82.50
not used in our experiment. The evaluations conducted using the SECOND-IOU [50] and PV-RCNN [62] models, following the evaluation protocol of previous works [36, 7, 51].
Baselines and comparison. For the domain unification tasks, LiT is evaluated against key baselines in autonomous driving domain adaptation: (i) SN [36], which normalizes object sizes using target domain statistics; (ii) ST3D [7], employing iterative pseudo label generation and curriculum-bar training; (iii) ReSinAD [51], a method combining reconstruction and simulation; and (iv) Oracle, the theoretical performance upper bound via full target domain supervision.
5.2 Experimental results
Statistical alignment with target domain. To validate the effectiveness our LiDAR translation approach directly, we compare the distributional differences between translated and ground-truth target domains, as shown in Table 2. Across all adaptation scenarios, LiT significantly reduces both MMD and JSD metrics compared to the baseline, which the LiDAR data from the source domain. More substantial improvements are observed in cross-beam-count scenarios (64-beam + 32-beam and 32-beam 64-beam), where LiT achieves 3 ~ 4x reduction in MMDBEV.
Single souree domain unification. In the single source domain adaptation setting as shown in Table 3, LiT consistently outperforms all baselines, including the model-based adaptation approach ST3D and data-based adaptation approach ReSimAD, in both the APBEY and AP metrics across different settings. Notably, in the challenging nuScenes - KITTI tasks, where the low-beam LiDAR
8

---

Table 4: Multi source domain unification. We compare models trained with one, two, and three source domains. In the table, W Waymo, N nuScenes, and K = KITTI. We report the APBEV and AD of the car category at loU = 0.7. The last row represents the "oracle" K K model trained with full target-domain supervision. Remarkably, in the W + N K task, LiT surpasses the oracle performance in SECOND-IOU's BEV even when the target domain is never seen during training. When all three source domains are used, LiT surpasses the oracle performance in all metrics.
SECOND-IOU
PV-RONN [62]
Task
Method
Zero-shot
APBEv APso 1
Closed gap
APsEv 1 APso 1
Closed gap 1
Source Only
Yes
67.64/27.45 82.55 / 69.94
61.18 22.01
W +
K
LiT
Yes
+95.27% +92.36%
84.35 65.68
+83.35% +72. 19%
Source Only
Yes
51.84 17.92
68.15 37.17
N
K
LiT
Yes
80.54 / 60.13
+91.26% / +76.01%
85.82 74.87
+84.83% +83.17%
Source Only
Yes
67.26 / 22.05
77.82/ 34,05
W K
LIT
Yos
84,45 71.58
+107.24% +96.36%
84.15 77.50
+56.72% / +8..55%
W+N+K
LiT
87.52 75.76
90.67 /2.67
K
Oracle
No
83.29 / 73.45
88.98 82.50
Table 5: Ablation studies. We report the performance of LiT with nuScenes + KITTI translation tasks with SECOND-IOU [50] model to study the effects of different configurations in LIT.
Group
Setting
APREv
AP3o t
Source only
51.84
17.92
Shared 1 foreground mesh
76.51
55.30
Foreground diversity
Shared 50 foreground mesh
77.23
57.27
Foreground simulation only
80.14
48.45
Noise std 0.01m
77.14
57.84
Foreground inaccuracies
Noise std 0.02m
77.25
56.78
Noise stdd 0.055
69.57
35.37
Noise sti 0.01m
78.02
61.43
Background inaccuracies
Noise stdd 0.02m
78.70
57.99
Noise std
0.05m
76.76
58.21
Noise std 0.01m
77.54
59.60
Foreground and background inaccuracies
Noise std 0.02m
76.90
56.09
Noise std 0.05m
72.03
36.18
LiT full model
80.54
60.13
data from nuScenes is adapted to the high-beam LiDAR data from KITTI, LiT achieves significant improvements over ST3D,, closing the gap of APBEV from 76.63% to 91.26% and 49.02% to 84.83% for SECOND-IOU and PV-RCNN, respectively.
Multi source domain unification. As shown in Table 4, the performance of LiT improves further when trained with multiple source domains, highlighting the benefits of being able to leverage diverse data sources via LiDAR translation. In zero-shot setup, combining Waymo and nuScenes training sets naively (source only) does not improve the performance much or may harm the performance in some scenarios. After LiT translation, the combined Waymo nuScenes training set achieves much better performance compared to the naive combination and single source domain training. Remark- ably, LiT surpasses the oracle performance (achieving AP of 84.45 over 83.29) in SECOND-IOU under the Waymo nuScenes > KITTI adaptation task, demonstrating its effectiveness zero-shot scenario when the target domain is never during training. When all three source domains used, LiT achieves the best performance, surpassing the oracle performance in all metrics.
Foreground diversity. The diversity of foreground objects plays an important role in the domain adaptation's 's performance. Specifically, when transitioning from a single shared foreground mesh to 50 shared meshes sees an improvement in both APBEV and APıp. Furthermore, with only foreground simulation, LiT still manages to achieve reasonable performance, showcasing LiT's robustness. The results are shown in Table 5.

---

Table Full-scene LiDAR translation in under minute. We present averaged runtime and key
statistics for the LiT LiDAR translation pipeline. Here, a "scene" is a LiDAR sequence containing
multiple LiDAR point cloud "frames". Runtime is measured on a single NVIDIA RTX 4090 GPU.
Pipeline
Item
Waymo KITTI
Waymo nuScenes nuSeenes KITTI
Frames per scene
198.07
198.07
40.26
Points per frame
180,580.51
146,580.51 29.033,201.66
23,988.50
Points per scene
22.033,201.66
965,776.97
Background modeling
Vertices of recon mesh
1.843,176.99
1,843,176.99
1,200,037.31
Recon. time per frame Recon. time per scene
0.11 Sec.
0.11 Sec.
0.16 Sec.
21.97 Sec.
21.97 Sec.
6,51 Sec.
Vehicles per scene
45.20
45.20
49.15
Foreground
LiDAR frames per vehicle Multi-Frame points per vehicle
79.03
79.03
11.42
modeling
40,506.01
40,506.01
1,089.57
Recon, time per scene
24.74 Sec.
24.74 Sec.
29.04 Sec.
Emitted LiDAR rays per frame Ray hit per frame
119,232.00
34,880.00
119,232.00
115,937.99
29,834.54
116,058.03
Ray casting
Ray casting time per frame
0.04 Sec.
0.03 Sec.
0.03 Sec.
Ray casting time per scene
8.89 Sec.
6.09 Sec.
1.27 Sec.
LiT translation time per scene (all frames)
55.60 Sec.
52.80 Sec.
36.82 Sec.
Scene modeling inaccuracies. To understand how modeling accuracy affects performance, simulate inaccuracies by adding Gaussian noise with std = 0.01m, m, 0.02m, and 0.05m to the recon- structed mesh vertices of foreground-only. background-only and both foreground and background. As shown Table 5, the detection model is more sensitive to inaccuracies in the foreground compared background, which is expected as foreground objects the main target for object detection. Even with scene modeling inaccuracies, LiT still substantially outperforms the source-only (no translation) baseline, demonstrating the robustness of approach.
Runtime performance. Table shows the runtime performance and key statistics of LiT's LiDAR translation process. We report the average runtime and data statistics of 50 scenes from each dataset. The runtime measured on desktop PC with a single NVIDIA RTX 4090 GPU. Notably, LiT capable of translating full After the reconstruction steps are done, the LiDAR ray casting process is highly efficient and can run in real-time. The efficiency of LiT demonstrates its scalability in real-world applications, where the ability to quickly adapt sensor data to different domains can enhance perception systems.
LiDAR scene from one domain to another in under minute.
6 Conclusion
In conclusion, present the LiDAR Translator (LiT), pioneering framework that unifies LiDAR data into a common "language" in a comprehensive system consisting scene modeling, LiDAR modeling, and domain adaptation. LiT overcomes domain discrepancies in LiDAR data from diverse sensor setups and environments, enabling seamless integration of multiple LiDAR datasets, enhancing zero-shot detection capabilities, and improving the representation quality of pre-traine pre-trained models across a variety of scenarios. Additionally, LiT's efficiency streamlines data preprocessing, reducing both time and computational demands, and facilitating quicker development cycles in autonomous systems. This acceleration promotes rapid advancements and the deployment of safer, more effective vehicles.
Social impact and limitations. The LiT framework significantly advances autonomous driving by facilitating domain adaptation for LiDAR data, which could enhance transportation safety and efficiency. However, it has limitations, such as its exclusive reliance on LiDAR data, its dependency on annotated source datasets for foreground reconstruction, and its current focus on vehicle objects solely. Future enhancements should aim to incorporate other object types and sensor modalities, as well as reduce dependency on extensive labeling, to broaden LiT's applicability and impact.
This work is supported by the National Natural Science Foundation of China (No. 62201 Alibaba Innovative Research Fund, HKU Startup Fund, and HKU Seed Fund for Basic Research.
Acknowledgement.
10

---

References
[1] Alex H Lang. Sourabh Vora, Holger Caesar. Lubing Zhou, Jiong Yang. and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In CVPR, 2019. 2
[2] Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based 3d object detection and tracking. In CVPR, 2021.
[3] Lue Fan, Ziqi Pang. Tianyuan Zhang, Yu-Xiong Wang, Hang Zhao, Feng Wang, Naiyan Wang. and Zhaoxiang Zhang. Embracing single stride 3d object detector with sparse transformer. In CVPR, 2022.
Chao Ma Guangsheng Shi, Ruifeng Li. Pillamet: Real-time and high-performance pillar-hased 3d object detection, ECCV, 2022.
[5] Pointcept Contributors. Pointeept: codebase for point cloud perception research. https://githhr Com//Polntceet/PPottcoopo 2023.
[6] Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, and Hengshuang Zhao. Towards large-scale 3d representation learning with multi-dataset point prompt training. In CVPR, 2024.
[7] Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. ST3D: self-training for unsuper- vised domain adaptation on 3d object detection. In CVPR, 2021. 2, 3, 8,8
[8] Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. St3d++; Denoised self-training for unsupervised domain adaptation on 3d object detection. PAMI, 2022. 2,
[9] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research, 2013. 2, 5,7, 14
[10] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksci Timofcev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, 2020. 2, 3, 5, 7, 14
[11] Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020. 2, 3,5, 5, 7, 14
[12] Yonglong Tian, Olivier J Henaff' and Aäron van den Oord. Divide and contrast: Self-supervised learning from uncurated data. In CVPR, 2021. 2
[13] Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, et al. Sule-vupe pretraining of visual fcatures in the wild. arXiv:2103.01988 2021.
[14] Xinlong Wang, Wen Wang. Yue Cao. Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In CVPR, 2023. 2
[15] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yee Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. In ICCV, 2023.
[16] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws neural language models, arXiv:2001.0830 2020. 2
[17] Vamsi Aribandi, Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler. Ex15: Towards extreme multi-task scaling for transfer learning. In ICLR, 2022. 2CLR.2022.2 2
[18] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. XrXiv 13971, 2023. 2
[19] OpenAL. Gpt-4 technical report. ar arXiv:2303.08774 2023. 2
[20] Bo Zhang. Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3d: A unified baseline for multi-dataser 3d object detection. In CVPR, 2023,

---

[21] Mathilde Caron. Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In CVPR, 2021. 2
[22] Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi. Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for point cloud understanding. In ECCV, 2020. 2
[23] Yue Wang and Justin M Solomon. Deep closest point: Learning representations for point cloud registration. In ICCV, 2019.
[24] Jonathan Sauder and Bjarne Sievers. Self-supervised deep learning on point clouds by reconstructing space. In NeurlPS, 2019. 2
[25] Aditya Sanghi. Info3d: Representation learning on 3d objects using mutual information maximization and contrastive learning. In ECCV, 2020. 2
[26] Yatian Pang, Wenxiao Wang, Francis EH Tay. Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud supersud learning. In ECCV, 2022.
[27] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang. Jie Zhou, and Jiwen Lu. Point-BERT: Pre-training 3D point cloud transformers with masked point modeling. In CVPR, 2022. 2
[28] Ji Hou, Benjamin Graham, Matthias Nießner, and Saining Xie. Exploring data-efficient 3d scene under- standing with contrastive scene contexts. CVPR, 2021.
[29] Xiaoyang Wu, Xin Wen, Xihui Liu, and Hengshuang Zhao. Masked scene contrast: scalable framework for unsupervised 3d representation learning. In CVPR, 2023. 2
[30] Haoyi Zhu, Honghui Yang. Xiaoyang Wu, Di Huang. Sha Zhang, Xianglong He, Tong He, Hengshuang Zhao, Chunhua Shen, Yu Qiao, et al. Ponderv2: Pave the way for 3d foundataion model with universal pre-training paradigm. arXiv: 23.::2300,08886, 2023. 2
[31] Zhuoxiao Chen, Yadan Luo, Zi Huang, Zheng Wang, and Mahsa Baktashmotlagh,, Revisiting domain- adaptive 3d object detection by reliable, diverse and class-balanced pseudo-labeling. In ICCV, 2023.
[32] Jiakang Yuan. Bo Zhang, Xiangchao Yan, Tao Chen, Botian Shi, Yikang Li, and Yu Qiao. Bi3d: Bi-domain active learning for cross-domain 3d object detection. In CVPR, 2023. 2
[33] George Eskandar, Chongzhe Zhang, Abhishek Kaushik, Karim Guirguis, Mohamed Sayed, and Bin Yang. An empirical study of the generalization ability of lidar 3d object detectors to unseen domains. arXiv:2402. 17562, 2024. 2
[34] Yurong You. Cheng Perng Phoo, Katie Luo, Travis Zhang, Wei-Lun Chao, Bharath Hariharan. Mark Campbell, and Kilian Q Weinberger. Unsupervised adaptation from repeated traversals for autonomous driving. NeurlPS, 2022.
[35] Ziyu Li, Jingming Guo, Tongtong Cao, Liu Bingbing, and Wankou Yang. Gpa-3d: Geometry aware prototype alignment for unsupervised domain adaptive 3d object detection from point clouds. In ICCV, 2023. 2
[36] Yan Wang, Xiangyu Chen, Yurong You, Li Erran Li, Bharath Hariharan, Mark E. Campbell, Kilian Q Weinberger. and Wei-Lun Chao. Train in germany, test in the USA: making 3d object detectors generalize. In CVPR, 2020. 2,8
[37] Yi Wei, Zibu Wei, Yongming Rao. Jiaxin Li, Jie Zhou, and Jiwen Lu. Lidar distillation: Bridging the beam-induced domain gap for 3d object detection. ECCV, volume 13699, 2022.
[38] Qiangeng Xu, Yin Zhou, Weiyue Wang. Charles R Qi. and Dragomir Anguelov. Spg: Unsupervised domain adaptation for 3d object detection via semantic point generation. In ICCV, 2021. 3
[39] Zeng Yihan, Chunwei Wang, Yunbo Wang, Hang Xu, Chanqiang Ye, Zhen Yang, and Chao Ma. Learning transferable features for point cloud detection via 3d contrastive co-training. NeurlPS, 2021. 3
[40] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In CoRL, 2017. 5,7
[41] Bernhard Wymann, Eric Espié, Christophe Guionneau, Christos Dimitrakakis, Rémi Coulom, and Andrew Sumner. Torcs, the open racing car simulator. Software available at http://www.torrs.orr.. 2000.
12

---

[42] Shital Shah, Debadeepta Dey. Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Roboticx, 2018. 3
[43] Jean-Emmanuel Deschaud. Kitti-carla: a kitti-like dataset generated by carla simulator. arXiv:2109.00892,
2021. 3
[44] Nathan Koenig and Andrew Howard. Design and use paradigms for gazebo, an open-source multi-robot simulator. In IROS, 2004.
[45] Xiangyu Yue, Bichen Wu, Sanjit A Seshia, Kurt Keutzer, and Alberto I. -anggioaani-Vincentellli A lidar point cloud generator: from virtual world autonomous driving. In ICMR, 2018. 3
[46] Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang. Wei-Chiu Ma, and Raquel Urtasun. Lidarsim: Realistic lidar simulation by leveraging the real world. CVPR, 2020.
[47] Chenqi Li, Yuan Ren, and Bingbing Liu. Pegen: Point cloud generator for lidar simulation. In ICRA, 2023.
3,
[48] Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Manivasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Urtasun. Unisim: A neural closed-loop sensor simulator. In CVPR, 2023. 3
[49] Tang Tao, Longfei Gao, Guangrun Wang, Peng Chen, Dayang Hao, Xiaodan Liang. Mathieu Salzmann, and Kaicheng Yu, Lidar-nerf: Novel lidar view synthesis via neural radiance fields. arXiv:2304.. 10406, 2023. 3
[50] Yan Yan, Yuxing Mao, and Li. SECOND: sparsely embedded convolutional detection. Sensors, 2018.
3, 8, 9
[51] Bo Zhang, Xinyu Cai, Jiakang Yuan, Donglin Yang. Jianfei Guo, Xiangchao Yan, Renqiu Xia, Botian Shi, Min Dou, Tao Chen, Si Liu, Junchi Yan, and Yu Qiao. Resimad: Zero-shot 3d domain transfer for autonomous driving with source reconstruction and target simulation. In ICLR, 2024. 5, 8, 14
[52] Jeong Joon Park, Peter Florence, Julian Straub, Richard A. Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, 2019. 5, 14
[53] Yibo Liu, Kelly Zhu, Guile Wu, Yuan Ren, Binghing Liu, Yang Liu, and Jinjun Shan. My-dcepsdf: Implicit modeling with multi-sweep point clouds for 3d vehicle reconstruction in autonomous driving. In ICCV, 2023. 5
[54] Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang. Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-Fe 3d model repository. arXiv:1512. 03012, 2015. 5,
[55] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In NeurlPS, 2021. 15
[56] Francis Williams, Zan Gojeic, Sameh Khamis, Denis Zorin, Joan Bruna. Sanja Fidler, and Or Litany. Neural fields as learnable kernels for reconstruction. In CVPR, 2022. 5
[57] Jiahui Huang. Zan Gojeic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. CVPR, 2023. 5, 15
[58] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren
Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. ECCV, 2020.
[59] I. Vizzo, X. Chen, N. Chehrolu, Behley, and C. Stachniss. Poisson Surface Reconstruction for LiDAR Odometry and Mapping. In ICRA, 2021.7 7
[60] Vlas Zyrianov, Xiyue Zhu, and Shenlong Wang. Learning generate realistic lidar point clouds. In ECCV, 2022.7 7
[61] Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In CVPR, 2023.
[62] Shaoshuai Shi, Chaoxu Guo, Li Jiang. Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. PV-RCNN: poxet-voxel feature set abstraction for object detection. In CVPR, 2020. 8,9 9
[63] Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In CVPR, 2019. 14
13

---

LiT: Unifying LiDAR "Languages" with LiDAR Translator Supplementary Materials
Contents
A.1 Foreground modeling details
14
A.2 Background modeling details
15
A.3 LiDAR statistical modeling visualization
15
A.4 Evaluation of scene and LiDAR modeling
17
A.5 Training details
17
A.1 Foreground modeling details
Tracking and fusion. Reconstructing foreground objects directly from each LiDAR frame is suboptimal as it does not take into account the multiple viewing angles of the object as the LiDAR sensor or the object moves. To address this issue, we fuse the point clouds from multiple LiDAR frames to reconstruct the foreground objects. Typically, tracking information is provided as unique object identifiers in the LiDAR datasets [9, 10, 11]. where vehicles are tracked across multiple frames using these identifiers. For each identified vehicle, its point clouds from different frames are aligned based on their bounding box information and then fused.
Mesh reconstruction. With multiple LiDAR frames, the fused point clouds are incomplete and noisy due to the sparsity and scanning gaps inherent in LiDAR data. To faithfully reconstruct the foreground objects while addressing the noise and sparsity issues, need introduce prior knowledge of vehicle geometries. Inspired by the success of neural implicit surface reconstruction for object shape modeling [63, 52], first train SDF-based model with the ShapeNetV2 [54] vehicle category and then apply this model to reconstruct the foreground object meshes. In essence, given latent code, the model maps 3D points to signed distance field (SDF) representing the mesh surface, where the key here is to solve for the latent code given multi-view point cloud observations.
Given a set of points from LiDAR scans, {æ;} ( our goal is to find a latent code, z, that best represents the underlying surface that these points belong to. This optimization process is guided by minimizing the Signed Distance Function (SDF) loss, defined as:
1 N
LSDF
Σ
(llfe (2, - O11,) + 2|
where fo (z, denotes the predicted SDF value for point x; given the latent code z, and the target SDF value for points on the object surface is set to zero. The term >|22| adds an L2 regularization the latent code to encourage generalization by penalizing its magnitude, with À being the regularization coefficient.
Notably, as our mesh is reconstructed from point clouds, it is not limited to using human-created vehicle 3D assets, which a key difference from ReSimAD [51]. This optimization process effectively adapts the generic vehicle geometry learned from ShapeNet to fit the specific geometry of the vehicle represented in the sparse and noisy LiDAR point cloud. By iteratively refining the latent code, we can generate mesh that accurately captures the detailed geometry of the vehicle, even in the presence of data sparsity and noise inherent to LiDAR scans. show some example visualizations foreground modeling in Fig. 8.
The hyperparameters for scene modeling are shown in Table For background modeling, while most parameters are consistent across both Waymo and nuScenes datasets, key difference is their frame rates. Specifically, every other frame skipped the Waymo dataset, which is attributed to its denser point clouds and higher LiDAR frame rate of 10Hz, in contrast to the 2Hz frame rate of nuScenes. Consequently, we sample the Waymo dataset at SHz to ensure comparable point cloud density and maintain reconstruction quality across datasets.
14

---

kiew LiDAR scans
Reconstructed mesh
Figure 8: Foreground modeling samples. We show some examples of foreground modeling with LIT. The left columns show the original LiDAR point clouds collected from multiple LiDAR frames. The rightmost column shows the reconstructed mesh from the multi-view LiDAR inputs. The reconstructed mesh will then be used by LiT to perform target-domain LiDAR ray casting.
A.2 Background modeling details
Next, we reconstruct the background scene mesh with point clouds from multiple LiDAR frames. For each LiDAR frame point cloud, foreground objects are removed based on the bounding box information. Then, point clouds from multiple frames are transformed and fused in world coordinates by the ego vehicle's ' pose and LiDAR-to-ege pose.
To reconstruct the background scene mesh from the fused point cloud, follow neural kernel field [57], where the reconstructed shape is modeled as the zero level set of 3D implicit field
fo R I defined a hierarchical neural kernel field. This field combines positive definite kernels, weighted and conditioned on the inputs, and centered at the midpoints of voxels within the predicted hierarchy:
fel = Σ a (K (x, x! (ens
where oz are scalar coefficients at the ith voxel at level in the hierarchy. Then, the kernel for the 1th level, K() is defined as:
KS x (0) =x/ o6 (x'(Xi, K,
(xxxxx,)) (x,x').
Given the voxel hierarchy and predicted normals, the implicit surface is computed by minimizing a specific loss function to find optimal coefficients. This process aims to align the neural kernel all input points. We
field gradient with the normals voxel centers and approximate employ the pre-trained kitchen-and-sint model which is able to generalize to diverse datasets. For fair comparison, the model has not been trained on the LiDAR datasets that use in this work. Compared with ReSimAD with their NeuS [55]-based background modeling, our approach is more efficient as it does not require custom training for each scene.
The hyperparameters for scene modeling are shown in Table Additional visualizations of the background modeling are provided in Fig. 9. For foreground modeling, utilize fixed number of steps and predefined learning rate to reconstruct the foreground objects. We use the same parameters for both Waymo and nuScenes datasets.
A.3 LiDAR statistical modeling visualization
As discussed in Sec. 4.2, we conduct statistical modeling of LiDAR rays to model the distribution of vertical angles. We identify the peak angles where the majority of the LiDAR rays are concentrated and generate rays based on these identified peak angles. This ensures that our simulated LiDAR data representative of real-word weal-world LiDAR characteristics. We illustrate this effect visually in Fig. 10, while the LiDAR statistics of each dataset are presented in Fig. 2.
15

---

Table 7: Parameters for scene modeling. The left table shows the parameters for background modeling. The right table shows the parameters for foreground modeling.
Background parameters
Waymo
nuScencs
Foreground parameters
Valuc
Skip every N frames
2
Latent code optimizer iterations
500
LiDAR frame rate
5Hz
2Hz
Optimizer learning rate
5c-5
Enabled LiDARs
[TOP]
[TOP]
Number of samples per iteration
8000
Voxel size
0.25
0.25
Meshing voxel resolution
128
Normal estimation kNN
64
64
Maximum batch size
32
Normal estimation drop angle
85
85c
Latent code clamp distance
0.1
Solver max iteration
2,000
2,000
Statistical outlier kNN
20
Solver convergence tolerance
le-5
le-5
Statistical outlier std ratio
1.0
Background modeling
Background modeling (nuScenes)
(Waymo)
Figure 9: Background modeling samples. We provide additional visualization samples of back- ground modeling for Waymo and nuScenes.
Range image sampled without statistical modeling
Range image sampled with statistical modeling
Figure 10: Effects of LiDAR statistical modeling visualized with range image. We illustrate the effect of statistical modeling of LiDAR ray angles with scene from nuScenes. The top row shows 2D range image sampled without statistical modeling, and the bottom row shows 2D range image sampled using statistical modeling. The bottom row shows fewer artifacts (e.g. the horizontal gap) as the sampled rays are more concentrated around the peak angles.
16

---

A.4 Evaluation of scene and LiDAR modeling
We perform evaluation on the quality of the scene and LiDAR modeling To do this, we perform self-to-self translation task on the source domain, where we use LiT to reconstruct the scene and perform LiDAR ray casting using the same dataset's LiDAR parameters. We then calculate the Chamfer distance between the original point cloud Pai and the ray casted point cloud Psim
integrated system.
CD(Psim Pg)
min x yEP&
y
min ly x122. XEPin yE
Σ
P
sim
PAt
X Pum
The distributions of the Chamfer distances for the Waymo and nuScenes datasets are shown in Fig. 11. Waymo
nuScenes
Probability density
Probubility density
2.0
1.0
1.0
0.5
0.0
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
Chamfer distance
Chamfer distance
Figure 11: Chamfer distance distributions. We evaluate the quality of the scene and LiDAR modeling. For a given dataset, we use LiT to reconstruct the scene and perform a self-to-self translation of the scene. We then measure the Chamfer distance between the original and the translated point cloud. The distributions of the Chamfer distances for the Waymo and nuScenes datasets are shown. In general, Waymo has lower Chamfer distances compared to nuScenes, which attributed to the denser point cloud from the higher LiDAR resolutions and sampling rate in Waymo.
A.5 Training details
We provide hyperparameters used for training in Table
Table 8: Training hy typeraarameters for domain adaptation tasks. This summary presents the hyperparameters for training detection models under single-source domain unification settings. The table includes configurations for Second-IOU and PV-RCNN models.
Waymo KITTI
Waymo - nuScenes
nuScenes KITTI
Second-IOU
PV-RCNN
Second-IOU
PV-RCNN Second-IOU PV-RCNN
Optimizer Scheduler
Adam
Adam
Adam
Adam
Adam
Adam
One-Cycle
One-cycle
One-csycle
One-Cycle
One-Cycle
One-Cycle
Learning rate
le-4
le-4
le-4
le-4
le-4
le-4
Momentum
0.9
0.9
0,9
0.9
0,9
0.9
Weight decay
le-2
le-3
le-2
le-3
le-2
le-3
Batch size
32
16
32
16
32
16
Epochs (ft.)
10
10
10
10
10
10
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We confirm that the main claims made in the abstract and introduction accu- rately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in the conclusion section of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
We do not include theoretical results in our paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe the data processing, experimental setup, and evaluation metrics in detail in the main paper and supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification: We provide access to the code, with instructions on data preparation, training and evaluation.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips. cripsscc/public//uu guwdee/dss///// CCodeSubeesiaceecceee simiisssinn ssssonoliccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed information on the dataset, hyperparameters, and evalua- tion metrics in the main paper and supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Our evaluation and comparison are based on standard metrics as used in prior works in the domain. We have not included statistical significance tests the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
20

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide hardware details and runtime statistics for our pipeline.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics attps://neurips.. httppp///////tccccccccieeeeeeee dees?
Answer: [Yes]
Justification: We reviewed the NeurlPS Code of Ethics and confirm that our research conforms the guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the societal impacts of our work in the conclusion section of the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite and respect the licenses of all assets used in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new assets in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "jUByKun8dg", "reviewer": "Reviewer_tU6b", "summary": "The paper presents a novel framework designed to unify LiDAR data into a single target “language” and unified domain detection capabilities across diverse LiDAR datasets, marking a step toward domain unification for LiDAR-based autonomous driving systems. Experiments on dataset KITTI, Waymo, and nuScenes demonstrate the superiority of the proposed method in the task of single-source and multi-sources domain adaptation.", "questions": "1.\tHow can we ensure the accuracy of modeling the target data? Will the differences between simulated data and real data have negative impacts?\n2.\tModeling the target LiDAR data and target scenes requires a lot of prior information. When this information is unknown, how can we use the method proposed in this paper to solve the domain adaptation problem? From my understanding, the objective of domain adaptation is to address the challenge of having limited data or labels in the target domain.", "limitations": "The authors discussed potential limitations about the data, annotation and the category of object.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper is novel in introducing LiDAR Translator (LiT) to joint training across multiple datasets. LiT enables efficient state-of-the-art zero-shot and unified domain detection capabilities across diverse LiDAR datasets. \n2.\tThe paper is well-written and easy to follow, especially the part explaining the background. \n3.\tIt presents good experimental results and intuitive visualizations, convincingly demonstrating its effectiveness.", "weaknesses": "1.\tThe motivation of this paper is not clear. If it is possible to accurately model the target domain data, why is there a need to translate the source domain data into the target domain data?\n2.\tAs the core component of this work, the translator requires more direct experimental validation, such as measuring the distributional differences between the translated data and the target data, rather than solely relying on verification through downstream domain adaptation tasks.\n3.\tIt  lacks of comparative experiments with the latest state-of-the-art methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W1. Motivation of the work\n\nWe thank the reviewer for highlighting the importance of clarifying the motivation for our work. \n\n- **Background:** Imagine an autonomous driving company that has collected a substantial amount of LiDAR data from different sensors (LiDAR-A and LiDAR-B). The company has also annotated the data for object detection for LiDAR-A and LiDAR-B. However, they want to deploy the object detection model on a new LiDAR sensor (LiDAR-T, T for target) mounted on their production vehicles.\n- **Classic approach:** Traditionally, this involves sending out cars equipped with LiDAR-T, collecting a new large dataset, annotating this data for object detection, and retraining the model. This process is expensive, time-consuming, and does not scale well with the deployment of newer LiDAR models, nor does it effectively leverage the existing annotated data from LiDAR-A and LiDAR-B.\n- **Our approach:** We aim to utilize the existing annotated data from LiDAR-A and LiDAR-B to train a model that can be directly used on LiDAR-T. To achieve this, we collect a small dataset from LiDAR-T (without needing to label it for object detection), model the characteristics of LiDAR-T, and then use the LiDAR Translator to translate the data from LiDAR-A and LiDAR-B to match the sensor pattern of LiDAR-T. This approach offers three main advantages:\n  - **Utilization of mixed data:** We leverage historical annotated data from LiDAR-A and LiDAR-B, ensuring that past investments in data collection are not wasted.\n  - **Scalability:** Our model demonstrates excellent scalability. We show our method is better than the baseline in the (train: LiDAR-A, test: LiDAR-T) setting, our model performs better when more training data is involved (train: LiDAR-A + LiDAR-B, test: LiDAR-T), and if the target data is also known (train: LiDAR-A + LiDAR-B + LiDAR-T, test: LiDAR-T), our model outperforms training purely on LiDAR-T data. This demonstrates the scalability of our method as it shows improved performance when more data collected from different sensors is utilized.\n\nWe hope this clarifies the motivation behind our work and the reasons for modeling the target domain LiDAR pattern with a small amount of unannotated data and translating source domains to the target domain. We thank the reviewer for acknowledging the importance of this clarification, and we will include the above explanation in the revised manuscript to more clearly articulate the real-world problem we are trying to solve.\n\n### W2. Direct verification for translation\n\nWe have added a new experiment to directly compare the translated target domain with ground-truth target domain. We follow LiDARDM and UltraLiDAR to evaluate the distribution differences with Maximum-Mean Discrepancy (MMD) and Jensen–Shannon divergence (JSD). The results are shown below:\n\n- Waymo -> KITTI \n  | Input style | GT style | MMD (↓) | JSD (↓) |\n  | ------------------------- | -------- | ------------- | --------- |\n  | Waymo (baseline) | KITTI | 8.817e-04 | 0.273 |\n  | Waymo translated to KITTI (ours) | KITTI | **3.268e-04** | **0.180** |\n- Waymo -> nuScenes \n  | Input style | GT style | MMD (↓) | JSD (↓) |\n  | ------------------------- | -------- | ------------- | --------- |\n  | Waymo (baseline) | nuScenes | 2.310e-03 | 0.380 |\n  | Waymo translated to nuScenes (ours) | nuScenes | **6.583e-04** | **0.205** |\n- nuScenes -> KITTI \n  | Input style | GT style | MMD (↓) | JSD (↓) |\n  | ------------------------- | -------- | ------------- | --------- |\n  | nuScenes (baseline) | KITTI | 8.725e-04 | 0.220 |\n  | nuScenes translated to KITTI (ours) | KITTI | **2.107e-04** | **0.164** |\n\nThe MMD and JSD metrics are significantly reduced after translation, indicating that the translated data is closer to the target domain data. \n\n### W3. Comparison with other method\n\nIn the paper, we have compared the state-of-the-art ReSimAD (ICLR2024) method, which is a recent work that is closely related to ours. We have shown that our method outperforms ReSimAD in domain translation tasks, as shown in Table 2 and summarized below:\n\n| Domains | Method | AP_BEV (↑) | AP_3D (↑) |\n| ----------------- | -------------- | ---------- | --------- |\n| W -> K    | ReSimAD        | 81.01      | 58.42     |\n| W -> K    | **LiT (ours)** | **84.35**  | **65.68** |\n| W -> N | ReSimAD        | 37.85      | 21.33     |\n| W -> N | **LiT (ours)** | **38.77**  | **23.48** |\n\n### Q1. Data-driven vs model-driven\n\n**Orthogonality of data-driven and model-driven approaches**: We would like to point out that our data-driven approach is actually orthogonal to the previous model-driven domain adaptation methods as it attacks the domain adaptation problem from a different angle. We hope this could inspire future work in this domain, including the potential combination of data-driven and model-driven approaches.\n\n### Q2. Cost of LiDAR modeling\n\n- We agree with the reviewer that domain adaptation aims to tackle limited data or labels in the target domain.\n- We added an experiment showing that only a small amount of unannotated target domain data is needed. Using 1, 2, and 4 **unannotated** scenes for LiDAR modeling, we measured the run time for each LiDAR modeling step, as well as the MMD/JSD metrics.\n\n| Num scenes used for LiDAR modeling | 0| 1| 2| 4|\n| ---------------------------------- | --------- | ------------- | ------------- | ------------- |\n| Data preprocessing time (sec)        | N/A       | 27.04      | 55.53      | 101.87     |\n| Ray drop MLP training time (sec)     | N/A       | 366.68     | 381.68     | 369.46     |\n| Statistical modeling time (sec)      | N/A       | 4.08       | 4.09       | 4.43 sec      |\n| Total LiDAR modeling time          | N/A       | 6.63 min      | 7.35 min      | 7.93 min      |\n| MMD (↓)| 3.303e-04 | **9.535e-05** | **8.131e-05** | **7.884e-05** |\n| JSD (↓)| 0.108     | **0.065**     | **0.062**     | **0.061**     |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k6lvGU4CfN", "reviewer": "Reviewer_Kzeh", "summary": "This paper proposed a unifying LiDAR Translator named LiT to achieve LiDAR domain adaptation. Differing from current model-driven approaches, LiT adopts a novel data-driven approach, embedding disparate LiDAR attributes into a common representation. LiT\nproposes a generalizable scene modeling and LiDAR statistical modeling. Besides, an efficient ray-casting engine is proposed to accelerate the above models. LiT also achieves efficient SoTA performance on several LiDAR datasets.", "questions": "Q1. The authors argue that model-driven approaches need extra training for new domains, while the proposed LiT also needs extra LiDAR statistical modeling. Can the authors provide a detailed comparison to prove that data-driven approaches are significantly better than traditional model-driven ones?\n\nQ2. Since datasets will be unified into a common representation, why LiT needs different training hyperparameters for distinct domain adaptation tasks, as shown in Table 7? It seemed to contradict the original motivation of this paper, i.e., unifying different types of LiDAR sensors.", "limitations": "Although the proposed data-driven approach seemed to be a promising research direction, LiT lacks sufficient comparisons with the model-driven approaches. Besides, LiT seemed to show \"ununified\" for the training process of different adaptation tasks.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. LiT adopts a novel data-driven approach instead of the classical model-driven approach, embedding disparate LiDAR attributes into a common representation. This research direction provides much value for real-world applications in autonomous driving industries. \n\nS2. An effective ray-casting engine is proposed to accelerate LiT on GPUs.\n\nS3. Experiments on widely used datasets demonstrate the SOTA performance of LiT.", "weaknesses": "W1. This work looks like a data normalization operation, only modifying different datasets into a unified representation.\n\nW2. The authors argue that model-driven approaches will introduce considerable costs associated with customizing model structure and training data for new, specific domains. However, this work has an extra LiDAR statistical modeling, this operation also causes additional costs.\n\nW3. Table 7 shows that LiT may not avoid the problem of model-driven approaches, that is, requiring different configurations for distinct datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W1. About dataset normalization\n\n- **Dataset normalization is non-trivial.** It is actually non-trivial to normalize different datasets into a unified representation. The LiDAR sensors have different specifications, such as the number of beams, vertical and horizontal resolution, and field of view. These differences make it challenging to directly combine data from different sensors. Our method, LiT, addresses this challenge by modeling the target domain LiDAR pattern with only limited unannotated target domain scenes. We show that LiT outperforms the SOTA data-based domain adaptation method ReSimAD.\n- **Data-based and model-based domain adaptation are orthogonal approaches.** We would like to point out that our data-driven approach is actually orthogonal to the previous model-driven domain adaptation methods as it attacks the domain adaptation problem from a different angle. We hope this could inspire future work in this domain, including the potential combination of data-driven and model-driven approaches.\n\n### W2. Cost of LiDAR modeling\n\nWe appreciate the reviewer's question about LiDAR modeling costs. We demonstrate that only a small amount of unannotated target domain data is needed. Our experiment compares MMD and JSD metrics on a nuScenes-to-nuScenes translation task using 0, 1, 2, and 4 scenes for LiDAR modeling. The \"0\" scene baseline relies solely on LiDAR specifications. We evaluate MMD and JSD performance with 50 unseen scenes.\n\n| Num scenes used for LiDAR modeling | 0         | 1             | 2             | 4             |\n| ---------------------------------- | --------- | ------------- | ------------- | ------------- |\n| Data preprocessing time (s)        | N/A       | 27.04 sec     | 55.53 sec     | 101.87 sec    |\n| Ray drop MLP training time (s)     | N/A       | 366.68 sec    | 381.68 sec    | 369.46 sec    |\n| Statistical modeling time (s)      | N/A       | 4.08 sec      | 4.09 sec      | 4.43 sec      |\n| Total LiDAR modeling time          | N/A       | 6.63 min      | 7.35 min      | 7.93 min      |\n| MMD (↓)                            | 3.303e-04 | **9.535e-05** | **8.131e-05** | **7.884e-05** |\n| JSD (↓)                            | 0.108     | **0.065**     | **0.062**     | **0.061**     |\n\n- Our results show that only a small amount of unannotated target domain data is needed for effective LiDAR modeling. Even with just 1 scene, there are significant improvements compared to no modeling, with further gains from additional scenes being marginal.\n- Total run time (including data preprocessing, MLP training, and statistical modeling) is minimal, typically under 10 minutes.\n\n### W3. Hyperparameter consistency\n\nThank you for the comment regarding the hyperparameter settings. Actually, the parameters across different datasets are **the same** in Table 7. We only use different parameters for different backbone models as they require different amounts of memory. This appears to be a misunderstanding that needs clarifying.\n\n- **Consistency across datasets**: For each model, the hyperparameters are the **same** across all datasets in Table 7. \n- **Model-dependent settings**: The variations in hyperparameters are solely due to the differences between the detection models (Second-IOU vs. PV-RCNN), not the datasets. PV-RCNN, due to its higher memory demands, requires smaller batch sizes compared to Second-IOU.\n\n### Q1. Data-driven vs model-driven\n\n- **Comparison with model-driven approaches**: We compare LiT with the traditional ST3D method, a model-driven domain adaptation approach based on pseudo-labels. Our method outperforms ST3D in all translation scenarios.\n- **Cost of LiDAR modeling**: An additional experiment (in W2) demonstrates that modeling the target domain LiDAR is minimal in cost. We require only a small amount of **unannotated** target domain data to model the LiDAR pattern. Details of this experiment are provided in the response to W2.\n- **Orthogonality of data-driven and model-driven approaches**: Our data-driven approach is orthogonal to existing model-driven methods, addressing domain adaptation from a different perspective. We hope this perspective will inspire future work and encourage exploring potential synergies between data-driven and model-driven approaches.\n\n### Q2. Hyperparameter consistency \n\nThank you for the comment regarding the hyperparameter settings. Actually, the parameters across different datasets are **the same** in Table 7. We only use different parameters for different backbone models as they require different amounts of memory. Please kindly see the response to W3 for more details.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aV9bCiR4sF", "reviewer": "Reviewer_Q1U8", "summary": "To address the significant gap between different LiDAR datasets (related to sensors, environments, etc.), this paper proposes a solution that differs from the existing model-based adaptation approach. By employing a scene-reconstruction-data-simulation approach, it achieves consistent representation of different LiDAR datasets. This data-driven method partially resolves issues such as domain shift in autonomous-driving-related 3D  point cloud learning.", "questions": "- How does the modeling accuracy of different foreground/background components affect the results of this paper?\n\n- Since the background is static, can it be replaced by other data sources? For example, historical high-precision drone point clouds or three-dimensional maps from scene reconstruction?", "limitations": "The analysis and discussion regarding scene reconstruction need improvement, as suggested by the previous  comments.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Innovatively analogizing the domain gap between different LiDAR data to that between languages, this paper proposes a data-driven cross-sensor training method from a \"translation\" perspective.\n\n- The proposed method shows good performance across different datasets, especially in terms of the AP3D metric.\n\n- The paper is well-written with clear logic and comprehensive experiments.", "weaknesses": "- Does \"foreground\" only refer to vehicles? Do pedestrians, bicycles, and similar entities fall into this category?\n\n- Similarly, in background reconstruction, is consideration limited to rigid bodies like the ground? In autonomous driving scenarios, is there no need to consider non-rigid objects such as vegetation?\n\n- In the current version, it seems that scene variations are not significant. Does this mean it's difficult to address zero-shot scenarios? For instance, if the source data are all from residential areas, is it challenging to accurately simulate point clouds from downtown areas?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> W1. Does \"foreground\" only refer to vehicles? Do pedestrians, bicycles, and similar entities fall into this category?\n\nCurrently, we only focus on the vehicle category in the foreground modeling. In this paper, the main motivation is to demonstrate the effectiveness of model-based domain adaptation by directly translating the LiDAR patterns. However, we do plan to extend our method to other object categories such as pedestrians, bicycles, and similar entities in the revised version. We thank the reviewer for pointing out this.\n\n> W2. Similarly, in background reconstruction, is consideration limited to rigid bodies like the ground? In autonomous driving scenarios, is there no need to consider non-rigid objects such as vegetation?\n\n- **Rigid vs non-rigid background objects.** For background modeling, we consider a point as background point if it is not in one of the foreground annotated bounding boxes. Therefore, the background modeling step includes all rigid objects in the background and potentially include non-rigid objects such as vegetation.\n- **Effects of background inaccuracy.** We have conducted an experiment to study the effects of inaccuracies in the background modeling on the performance of the translated model. From the experiment, we see that the model is less sensitive to inaccuracies in the background modeling. However, foreground modeling is more critical for accurate object detection.\n  | Condition | AP_BEV (↑) | AP_3D (↑) |\n  | --- | --- | --- |\n  | Noise in background (std 0.01m) | 78.02 | **61.43** |\n  | Noise in background (std 0.02m) | 78.70 | 57.99 |\n  | Noise in background (std 0.05m) | 76.76 | 58.21 |\n  | Baseline LiT model without noise | **80.54** | 60.13 |\n\n> W3. In the current version, it seems that scene variations are not significant. Does this mean it's difficult to address zero-shot scenarios? For instance, if the source data are all from residential areas, is it challenging to accurately simulate point clouds from downtown areas?\n\nWe can indeed model the \"LiDAR pattern\" in residential areas, and simulate the point cloud in unseen \"downtown areas\". There is a difference between \"scene variations\" and \"LiDAR pattern variations\". We could model the LiDAR pattern regradless of the scene variations.\n\nTo illustrate this, we first model the LiDAR pattern with nuScenes dataset. Then, we simulate the LiDAR points with a scene taken from the Mai City dataset (cite: Poisson Surface Reconstruction for LiDAR Odometry and Mapping), which is unrelated to the nuScenes. We show that the simulated LiDAR points closely match the pattern of the nuScenes LiDAR, this is as if you are driving a \"nuScenes car\" in a \"Mai City\" environment. We provide visualizations in the rebuttal PDF (Figure x). Please refer to **Figure R1** in the attached rebuttal PDF.\n\n### Questions\n\n> Q1. How does the modeling accuracy of different foreground/background components affect the results of this paper?\n\nTo explore how the modeling accuracy of different foreground/background components affects the results, we simulate inaccuracies by adding noise to the foreground and background reconstructed meshes. Specifically, we study the nuScenes->KITTI task, where we train a SECOND-IOU model with LiT-translated nuScenes and evaluate on the original KITTI dataset. The noise is added to the reconstructed mesh vertices, and the translated LiDAR point cloud is generated by ray casting from the noisy mesh. The results are shown in the table below:\n\n- Inaccuracies in foreground modeling\n  | Condition | AP_BEV (↑) | AP_3D (↑) |\n  | --- | --- | --- |\n  | Noise in foreground (std 0.01m) | 77.14 | 57.84 |\n  | Noise in foreground (std 0.02m) | 77.25 | 56.78 |\n  | Noise in foreground (std 0.05m) | 69.57 | 35.37 |\n  | Baseline LiT model without noise | **80.54** | **60.13** |\n\n- Inaccuracies in background modeling\n  | Condition | AP_BEV (↑) | AP_3D (↑) |\n  | --- | --- | --- |\n  | Noise in background (std 0.01m) | 78.02 | **61.43** |\n  | Noise in background (std 0.02m) | 78.70 | 57.99 |\n  | Noise in background (std 0.05m) | 76.76 | 58.21 |\n  | Baseline LiT model without noise | **80.54** | 60.13 |\n\n- Inaccuracies in both foreground and background modeling\n  | Condition | AP_BEV (↑) | AP_3D (↑) |\n  | --- | --- | --- |\n  | Noise in both foreground and background (std 0.01m) | 77.54 | 59.60 |\n  | Noise in both foreground and background (std 0.02m) | 76.90 | 56.09 |\n  | Noise in both foreground and background (std 0.05m) | 72.03 | 36.18 |\n  | Baseline LiT model without noise | **80.54** | **60.13** |\n\nFrom the table above, we see that the model is more sensitive to inaccuracies in the foreground compared to background. This is expected as the foreground objects are more critical for object detection, demonstrating the importance of accurate foreground modeling for the performance of the translated model.\n\n> Q2. Since the background is static, can it be replaced by other data sources? For example, historical high-precision drone point clouds or three-dimensional maps from scene reconstruction?\n\nYes. We can indeed replace the background with other data sources. We added a new visualization where we put a nuScenes modeled LiDAR in an unseen Mai City scene. Please refer to **Figure R1** in the attached rebuttal PDF.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TT7NGOSCFH", "reviewer": "Reviewer_omtq", "summary": "In this paper, the authors propose a method to help alleviate the domain gaps among different datasets with different LiDAR sensors, which can enable zero-shot detection on a new dataset. The proposed method including Scene Modeling for foreground and background reconstruction and LiDAR Modeling with statistical and ray-drop modeling. Another contribution is that the authors also accelerate the ray casting algorithm using GPU. The authors conducted single-domain and Multi-domain unification experiments on Waymo, nuScenes, and KITTI datasets, which achieves SOTA performance compared to previous works. The authors also provide ablation studies on foreground diversity and LiDAR noise injection. In addition, the authors show the run time performance after the GPU acceleration.", "questions": "In the multi-domain unified training experiments, have you considered including comparisons with other non-reconstruction semi-supervised learning methods?  (like pseudo-labels and so on)", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The foreground and background reconstruction and LiDAR Modeling and the statistical and ray-drop modeling in LiDAR Modeling make the paper differ from previous works.\nQuality: The code is provided. The performance is evaluated on multiple datasets, and achieves SOTA performance compared to previous works, and ablation studies are good. The GPU acceleration is also good.\nClarity: The images in the paper are clear and easy to understand.\nSignificance: The paper demonstrates the potential of zero-shot detection on a new dataset by 3D reconstruction from multiple different dataset and LiDAR settings and LiDAR simulation.", "weaknesses": "In the title of the paper, the use of terms such as \"Language,\" \"Translator,\" and \"LiT\" appears to be capitalizing on the popularity of the trending terms \"LLM\", \"ViT\", and \"DiT\", potentially misleading readers.\nSECOND and PV-RCNN are relatively old detection models, it's better to have experiments on more recent models such as CenterPoint, and other SOTA models to further demonstrate the effect of domain unification on SOTA models and even achieve new SOTA results. This would significantly enhance the paper's persuasiveness and impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W1. Usage of terms and naming of the method\n\nWe thank the reviewer for bringing up the point on the naming and terms used in the paper. Regarding the terms \"language\" and \"translator\", \"language\" refers to a particular LiDAR pattern of a specific LiDAR sensor, and the \"translator\" refers to the LiT model that translates the LiDAR pattern from one sensor to another. We will revise it to make it more clear and avoid any potential confusion. We will think of some new names to better reflect the core idea of the paper in the revised manuscript, for instance:\n\n- LidarTranslator\n- LidarUnifier\n- LidarAdapter\n- LidarTransformer\n\nWe will consider using these new terms in the updated version of the paper.\n\n### W2. Inclusion of newer detection models\n\nWe thank the author for the suggestion. We do believe this is very valuable. We will include experiments with more recent models such as CenterPoint in the revised manuscript. We will also compare the performance of LiT with CenterPoint and other SOTA models to further demonstrate the effectiveness of domain unification.\n\n### Q1. Comparison to non-reconstruction methods in multi-domain learning\n\nWe thank the reviewer for the question. We add an additional experiment to compare LiT with ST3D, which is a model-driven pseudo-label approach. Specifically, we evaluate the \"W + N -> K\" tasks where the mix of Waymo (W) and nuScenes (N) are used for training and KITTI (K) is used for testing. We show AP_BEV and AP_3D for both SECOND-IOU and PV-RCNN models. Specifically, we compare:\n\n- **Source only**: naive mix of Waymo and nuScenes samples\n- **ST3D**: pseudo-label with a mix of Waymo and nuScenes samples\n- **LiT (ours)**: LiT translates Waymo and nuScenes samples to KITTI style\n- **Oracle**: direct train on KITTI, with full target domain information available\n\n| Training set                     | SECOND-IOU AP_BEV (↑) | SECOND-IOU AP_3D (↑) | PV-RCNN AP_BEV (↑) | PV-RCNN AP_3D (↑) |\n| -------------------------------- | --------------------- | -------------------- | ------------------ | ----------------- |\n| Source only (naive mix of W + N) | 67.26                 | 22.05                | 77.82              | 34.05             |\n| ST3D (pseudo-label on W + N)     | 75.91                 | 39.08                | 69.69              | 31.08             |\n| **LiT (ours, translated W + N)** | **84.45**             | **71.58**            | **84.15**          | **75.50**         |\n| Oracle (direct train on K)       | 83.29                 | 73.45                | 88.98              | 82.50             |\n\nThe results show that LiT outperforms ST3D in multi-domain unified training tasks, where we jointly train with a mix of Waymo and nuScenes samples. This demonstrates the effectiveness of our data-driven domain unification method in such tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Please refer to individual rebuttal comments. The rebuttal PDF is attached.

Meta-review: All reviewers recommended acceptance, with two borderline accept, one weak accept, and one accept, This AC sees no reason to override the collective recommendations of the reviewers.
Decision: Accept (poster)