NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation
Authors: Chaokang Jiang
Dalong Du
Jiuming Liu
Siting Zhu
Zhenqiang Liu
Zhuang Ma
Zhujin Liang
Jie Zhou
Venue: NeurIPS 2024 poster (2024)
Keywords: Neural Fields
Point Cloud Interpolation
Gaussian Deformation Fields
Primary area: machine_vision
TL;DR: NeuroGauss4D-PCI is a novel spatiotemporal point cloud interpolation framework that integrates neural fields, dynamic soft clustering, 4D Gaussian deformation fields, and transformers to achieve high fidelity point cloud interpolation across time.

Abstract: Point Cloud Interpolation confronts challenges from point sparsity, complex spatiotemporal dynamics, and the difficulty of deriving complete 3D point clouds from sparse temporal information. This paper presents NeuroGauss4D-PCI, which excels at modeling complex non-rigid deformations across varied dynamic scenes. The method begins with an iterative Gaussian cloud soft clustering module, offering structured temporal point cloud representations. The proposed temporal radial basis function Gaussian residual utilizes Gaussian parameter interpolation over time, enabling smooth parameter transitions and capturing temporal residuals of Gaussian distributions. Additionally, a 4D Gaussian deformation field tracks the evolution of these parameters, creating continuous spatiotemporal deformation fields. A 4D neural field transforms low-dimensional spatiotemporal coordinates ($x,y,z,t$) into a high-dimensional latent space. Finally, we adaptively and efficiently fuse the latent features from neural fields and the geometric features from Gaussian deformation fields.
NeuroGauss4D-PCI outperforms existing methods in point cloud frame interpolation, delivering leading performance on both object-level (DHB) and large-scale autonomous driving datasets (NL-Drive), with scalability to auto-labeling and point cloud densification tasks.

NeuroGaus<4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation
Chaokang Jiang' Dalong Du Jiumingg Liu", Siting Zhu? Zhenqiang Liu' Zhuang Ma', Zhujin Liang', Jie Zhou
PhiGent Robotics, *Shanghai Jiaotong University, TTsinghua University
Abstract
Point Cloud Interpolation confronts challenges from point sparsity, complex spa- tiotemporal dynamics, and the difficulty of deriving complete 3D point clouds from sparse temporal information. This paper presents NeuroGauss4D-CC) which excels at modeling complex non-rigid deformations across varied dynamic scenes. The method begins with an iterative Gaussian cloud soft clustering module, offering structured temporal point cloud representations. The proposed temporal radial basis function Gaussian residual utilizes Gaussian parameter interpolation over time, en- abling smooth parameter transitions and capturing temporal residuals of Gaussian distributions. Additionally, 4D Gaussian deformation field tracks the evolution these parameters, creating continuous spatiotemporal deformation fields. A 4D neural field transforms dineeneniional spatiotemporal coordinates (x. y, 2, t) into a high-dimensions the latent features from neural fields and the geometric features from Gaussian deformation fields. NeuroGauss4D cloud frame interpolation, delivering leading performance both object-level (DHB) and large-scale autonomous driving datasets (NL--rrive), with scalability to auto-labeling and point cloud densification tasks. The source code is released giiboboorringghhckkngeeonooeeorr
latent space. Finally, we adaptively and efficiently fuse
outperforms existing methods in point
Introduction
Point cloud frame interpolation (PCI) 2 aims to estimate intermediate frames given more point cloud frames. This task enables the generation temporally smooth and continuous point cloud sequences at arbitrary timestamps, which is crucial for applications such as autonomous driving and virtual reality [5] 6 77). PCI can be expressed with the following formula:
3 4
Fo E N.X3 0.4..4..... Ti
Fe (P, Ti
{PPred ER (I)
Training Data
Inference Inpot
PCI faces several challenges due to the unique characteristics of point cloud data and the complexity of modeling spatiotemporal dynamics: Point clouds are inherently sparse and unordered, lacking the regular structure of images. For instance, NeuralPCI 3 simply concatenates spatial and temporal coordinates as inputs to an MLP, struggling to adequately represent the motion and correlation of multiple unordered point clouds over time. 2) PCI involves modeling the spatiotemporal dynamics of point clouds, requiring the interpolation model Fe to capture the spatial structure and temporal evolution of the scene from 4D training data ((P E N X Jt=0,4,8... T..0,4,8...) and model the non-rigid deformations and non- non-linear trajectories of discrete 3D points. This leads to the linear motion assumption of PointlNet (8), which uses bidirectional scene flow to warp input frames for
Equal contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Interpolation Frror DHR
Interpelation Frror NI-Déve
NeuralPCI
NSFP
Clurtie Errot (M (M100)
24
PalrdNcc
Churaler 0
PHAAFT
NeurtiFCI
NeurelPCI
27sFLabelion
5
3DSF Labelabeling
3D8 Paahlllgg
(FLOT)
1024
NeanoKinns4D (Ouns)
Interval InsercalFamee Frames
Point Number
Small
Large
Figure 1: NeuroGauss4D-PP robustly outperforms existing methods B 9
across multiple
datasets, frame intervals, and point cloud densities, consistently achieving lower interpolation errors. NeuroGaass4DDPPG robustly handles minute non-rigid deformations, large-scale unstructured scenes, dynamic environments with non-uniform data, and extensive motions. The proposed method consis- lently achieves precise local and global point cloud predictions.
estimating intermediate frames, failing to capture complex non-linear motion. Moreover, 3D scene flow estimation methods, such as 3DSFLabelling (9. PV-RAFT [O] and NSFP []]. can only express the motion field between two frames but cannot robustly represent higher-order dynamic scenes over longer time spans. 3) The inference process in PCI faces the challenge of generalizing from sparse temporal samples. The model Fe generates an accurate point cloud P/red E R R.Nxx for time f= based on Pt= and time frame at Tı= and the time frame at Tz=j- This demands strong interpretabiliiy and 4D modeling capabilities from the model to accurately predict the interpolated point cloud from minimal information. IDEA-Net 2 addresses the correlation between two input frames by learning one-to-one alignment matrix and refining the linear interpolation results using trajectory compensation module. However, for large-scale, occluded autonomous driving point cloud scenes, IDEA-Net's one-to-one correspondence assumption struggles to accurately predict the point cloud at interpolated timestamps from sparse temporal samples.
For the challenges of point cloud frame interpolation, we propose a novel 4D apatio-temporty modeling method based on Gaussian representations, NeuroGaus<4D-PCI. geometric structure of point clouds through iterative Gaussian soft clustering, providing structured representations for unordered data. Then, the temporal radial basis function Gaussian residual module interpolates over discrete time steps, learning the continuous dynamics of Gaussian parameters to capture the spatio-temporal evolution of point clouds. The 4D Gaussian Deformation Field further models the spatio-temporal variation trends of Gaussian parameters, effectively overcoming the limitations of linear interpolation. Finally, the attention fusion module adaptive fusion of latent neural representations and explicit geometric representations, enhancing the modeling capability of spatio-temporal correlations. As shown in Fig. I this compact and efficient method demonstrates superior point cloud sequence generation performance multiple datasets.
first captures the
We propose NeuroGaus44D-PCI, novel 4D spatio-tempon modeling method for point cloud frame interpolation. It captures geometric structures through iterative Gaussian soft clustering, providing structured representations for unordered point clouds, while adaptively fusing the latent neural field features. A Temporal Radial Basis Function Gaussian Residual module is designed to interpolate over discrete time steps, learning continuous Gaussian parameter dynamics to effectively model the spatio-temporal evolution of point clouds. An innovative 4D Gaussian deformation field introduced to model the spatio-temporal variations of Gaussian parameters, achieving smooth and realistic point cloud deformations through 4D Gaussian graph convolutions and Gaussian representation pooling. Our method outperforms others on multiple datasets, generating accurate point cloud sequences with minimal errors. It can be easily extended to tasks like lidar- camera temporal synchronization and point cloud densification.
Contributions:
2 Related Work
Neural Field In computational geometry, latent neural fields 12 II 13 encode complex 3D scenes with the principle (e(())) = o. x denotes the input parameters, such as coordinates in a 3D space. The function is a feature transformation applied to x, which enhances the neural network's ability to model complex patterns by providing a high-dimensional representation of the input space. The neural network fe. parameterized by O then maps these transformed features to the output

---

This output can represent various aspects of the 3D scene, such color [4], density [[5]. or surface normals [16], depending on the task. GPNF [77 presents method using neural fields for efficient and topology-flexibl -leexille geometry processing, outperforming traditional mesh methods in shape editing tasks. NSFP [ introduces novel implicit regularizer based on neural scene flow priors, utilizing coordinate-basse MLPs for robust scene flow optimization. Implicit neural representations [T8] ensure differentiable interpolation for 3D point clouds, facilitating end-to-end learning [19], yet their performance is limited by dependency on high-quality training data [9] and computational demands with inherent trade-off's in interpretability.
20
3D Gaussian Splatting Recent studies have demonstrated that 3D Gaussian Splatting (3D GS)
21 22 23 leverages an explicit radiance field for efficient:
T(p) Σ wy G.EG E2)
Σ wy exp((=
E.) 14)),
(2)
where (p) represents the rendered intensity or color at pixel location NN is the total number of 3D Gaussian models in the scene, w; is the weight of the i-th Gaussian model affecting the final image, G(p; pa; Ex) is the 3D Gaussian function defined as exp (-1 (p- p4:) ",:)"E;" ' (p
contribution to
u4)).
34 is the center of the Gaussian representing the position in 3D space, and Σ; is the covariance matrix that determines the shape and scale of the Gaussian distribution. 3DGS [21] innovatively employs 3D Gaussians to represent scenes, optimizing volumetric radiance fields with minimal computational waste, enabling high-quality real-time novel-view synthesis. Recent works (24 25 26 focused
on creating 4D scene representations using Gaussian splatting from video to model geometry and appearance changes across frames. Few explored modeling temporal 3D motion changes from raw, sparse 3D point clouds, requiring tracking spatial coordinates over time and dynamically adjusting Gaussian parameters to accurately reflect temporal deformations.
Point Cloud Interpolation (PCI) Existing PCI models are categorized into three types: based on 3D scene flow [10 20: 27: 8 28 neural fields 3] and feature alignment [29 trajectory regression [2]. 3D scene flow methods estimate correspondences between frames and use linear interpolation for intermediate frames. PointlNet [ calculates bidirectional 3D scene flow to warp point clouds and insert new frames. IDEA-Net 2 trajectory nonlinearities. enhancing interpolation. NeuralPCI (3] implicitly learns the nonlinear temporal motion characteristics of muulti-rrame point clouds through neural field. However, these methods struggle with modeling 4D geometric deformations point cloud sequences, impacting robustness. Our method combines 4D neural fields and Gaussian deformation fields to effectively model and predict dynamic point cloud changes, capturing nonlinear motion and deformation accurately.
applies aligned feature embeddings to manage
3
The Algorithm
3.1 Overview of 4D Neural Field And Gaussian Deformation Field
As depicted in Figure 2 temporal point cloud coordinates are encoded using Fourier basis functions encapsulate periodic information as follows:
Posenc(P((x.,y z, t)) (x), (y)) (z), (t)):; (x) [x, sin(x), cos(x)
(3)
To enhance the modeling capability for periodic features, the original coordinate features are aug- mented by incorporating their sine and cosine values. Features from Posenc(P) (x, y. z, t)) are further encoded using 4D neural field. The iterative Gaussian cloud soft clustering module employs Gaussian statistics to extract characteristics such as mean, covariance and geometric features from the point cloud, offering a statistical description of internal dynamic changes. Furthermore, the proposed temporal radial basis function Gaussian residual (RBF-GR) module employs radial basis functions to interpolate these Gaussian parameters across different time steps. The designed 4D Gaussian deformation field module learns the temporal evolution of point cloud Gaussian parameters, generating a continuous spatiotemporal deformation field. Finally, an fast latent- geoomeri fusion module adaptively fusion features, enabling the generation of point clouds through prediction head.
3

---

Input
Neural Field
Latent
Latent Features
Basis Enceding
Features
Gaussjan Deformation
Residual Gaussian
Time
Sec. 3.4
Sec. 3.2
Prediction
Head
Leer
CLEZT)
Iterable Gaussian
SoftClusterig Sec. 3.2
Lyne (Er)
RBF-GR Module
Temporal
Fass-LG-Fusion
Sec. Q
Fourier Basis Encoding
See. 3.3
Fent
Predicted time
[ Encoded
Gaussian mean
Erow
Gaussian
@ rent feutures
Gaussian
Gaussian
A
Residual identihcatton
>MM:Nuber of Gaussian spheres
covartance
sphere
Figure Three key steps of NeuroGaus4DDPCC. 1) Latent feature learning and point cloud Gaussian
representation: Fourier feature mappings and 4D neural fields map low-dimensional temporal coordinates (z, y/ 2, t) to high-dimensional latent features, while representing the original temporal point clouds as robust multi-Gaussian ellipsoids (j, Σ, L). 2) The temporal radial basis function Gaussian residual (RBF-GR) module captures residuals among temporal Gaussian distributions, fusing smooth temporal Gaussian distributions with latent features to construct 4D Gaussian deformation field that learns and smoothens point cloud deformations. 3) An efficient transformer architecture aggregates features from the 4D deformation field and latent features, enabling point cloud interpolations at any given timestamp through a point cloud prediction head.
3.2 Iterative Gaussian Soft Clustering and 4D Neural Field
Algorithm 1 Iterable Gaussian Soft Clustering
The iterative Gaussian cloud soft-clustering representation, given input point coordinates P. number of Gaussians M, consists of three steps: 1) Soft-clustrring of point clouds to Gaussian distributions achieved by intro- ducing soft assignment weights S.j, where S.y represents the probability of the i-th point belonging the j-th Gaussian, calculated based on the Euclidean distances Dis be- tween points and Gaussian centers C, over- coming the limitations of traditional hard- clustering methods. 2) The Gaussian means μ. and covariance matrices Σ are iteratively optimized over iterations, with μ updated by minimizing the distances Dij and Σ calcu- lated based on the distribution of points P m relative to the Gaussian means, adaptively fit- ting the spatial distribution of point clouds. 3) Multi-scale Gaussian features 47 are extracted by applying DGCNN 30) and self- attention mechanisms to points assigned to each Gaussian, learning rich local and global features. Finally, the point cloud is assigned to the corresponding Gaussians through the index matrix A.
Require: P E R Nx3, point coordinates, M: num- ber of Gaussians, K: number of iterations Ensure: u Mxx Gaussian Means, Σ R Mx3x3; Gaussian Covariances Matrix, feati
Gaussian features, A E R:: Pouus-to
Index Matrix
1: Initial centers matrix C R M3 by randomly selecting M points from P
2: for 7 1 to K do
3: Di; = P C, 2.
D
E NXM
4: S.y ΣE= exp(
exp(
S E NXM
5: Update C = (S P) 1
1
6: end for
7: u
8: A, arg max, S.,, v, E NI,
9: for = = M do
10: P.. = P m| pm
11: Σ. = Pm Epep... pp + cIs
end for
13: for
to M do
14:
Dgcun( P(A == i) SelfAttentio Φ Jeat aF
FG
Eq.
FC
Eq.
16:
17:
end for
The raw point cloud coordinates lack topological structure and contain noise and redundancy. We propose an iterative Gaussian soft-clustering point cloud representation module (Algorithm) T that converts point clouds into structured Gaussian representations. The module introduces DGCNN 30 (Eq. to extract local geometric features, where x, is the i-th point, X, is its k-nearest neighbor, I groph the graph convolution operation. Og is the convolution parameter. I represents tensor concatenation; Self-Attentinn (Eq. 55) to capture global contextual information, where W Q Wx. Wv are the weight parameters of the query, key, and value matrices, respectively, and d2 is the dimension of the key matrix.
F
x
mAX JEKNN(L.)) (1.))
Retu (I graph (((x, - xx) Ux,)((0)))) V, MEP]
(4)

---

Radial Bosis Function Grussinn Residual
2P Weyr" expl
KIN Actrotte:
expl
-- XBF Soaks
PC
c Kall Canter
Gussiia lneepoloor.
Apil vecl 1 Σ (ta) - vext
Updare Gausian Hpooo Infecmation
Fesiue
Ei ARe tt Do, AR rndy
4D Castvisa Ckaph
Uptredii oral Deformable
Gaussian Fealure
Upsbbed Ell Repuii
Conssann Flow Foeld
(PC,
Exurenion & Aggrogation
Parag
DOC SelAAtttiio
Disoomed FIl Fllipoci
Couseaa Fraturos
Figure 3: Temporal radial basis function Gaussian residual (RBF-GR) and 4D Gaussian Deformation Field. The normalized RBF weights & (12) are used to compute the residuals of Gaussian means Ape rotations A - and features / li+tz between time 4 and l2. The covariance matrix Mi is then updated using the learned rotation residual. Wobf fotmaa (M ((b,
(), where Wrey denotes the attention weights for RBF activations, adaptively adjusted based on Gaussian features. The Temporal Seapporal-RBF-GEE RBF-GR employs radial basis funetions (RBFs) with learnable centers c; and scales s, to capture the temporal evolution of Gaussian parameters.
(FoWe) ")
FC (FCWv) Softtaa
dd
Except for Gaussian cloud representation, we introduce 4D neural fields, parameterizing a continuous, latent opatio-temporal field function using multi-layer perceptron (MLP) networks, as shown in Fig. 2. This maps low-dimesiional spatio-temporal coordinates (z. 2, t) to a high-dimensional latent Teature space, representing the motion and changes of point clouds in the spatio-temporal domain. Compared to Gaussian point cloud representations, deep networks can fit more complex non-linear spatio-temporal correlations.
Point cloud data from LiDAR includes noise, which, while not affecting subsequent perception tasks, impedes the learning of temporal consistency. We apply statistical outlier removal [31] during preprocessing to eliminate noise by calculating each point's average distance its neighbors and identifying outliers using global mean and standard deviation. Experiments show (Table 2 this method markedly decreases interference during network training.
3.3 Temporal Radial Basis Function Gaussian Residual (RBF-GR) Module
This module accomplishes the interpolation and updating of Gaussian distribution parameters in the continuous time domain, which can be primarily divided into three parts (Fig. 3): RBF Activator: The RBF Activator converts discrete time steps into continuous time representation by mapping scalar time to radial basis function activation values grb (t). RBF networks encode time locally, with each kernel function centered at a specific time e; and exponentially decaying activations. The RBF activation values 2.5((1 (1) reflect the similarity between time and center times c;, enabling smooth interpolation of Gaussian parameters. Gaussian Interpolator: The Gaussian Interpolator interpolates the mean 14 and rotation matrix R2 of the Gaussian distribution using RBF activation values. Gaussian (0)
parameters estimated at discrete time steps are interpolated to obtain parameter residuals Aut -tt
and AR( - from time 1 to 12. The interpolation process exploits the linearity and closure property of Gaussian distributions. Feature Interpolator: The Feature Interpolator interpolates Gaussian features using RBF activation values to obtain a continuous feature representation time. The module enables smooth updating of Gaussian distribution parameters in continuous time, allowing effective fitting and generation of tary-ng point cloud sequences.
3.4 4D Gaussian Deformation Field
The temporal Gaussian graph convolutional (TG-GCN) deformation field plays a key role capturing the spatiotemporal features from Gaussian point cloud representations (Fig. 4). It takes input
5

---

Pe En feat 1
feat
t
feat
2
4D Gaussian Graph Convolutional Deformable
Pc La a
Q
ReLU
Gaussian Deformation Field
Motion
Pa-
Implicit
Feature Field Head
Agregationn& Upsampling of 4D Motion and Features
Flow Hcad
Features
Pooling
FlowField Gaussian
MLP
Gaussian Representation Pooling
Features
Figure 4: 4D Gaussian Deformation Field. Utilizing Gaussian means μ4, covariances Σ, and features &feat and time (f1, f2, ...) as inputs, the spatio-temporal graph convolutional network captures spatio-temporal patterns learn continuous 4D deformation fields. The Gaussian Representation pooling module projects the point cloud onto the Gaussian ellipsoids and upsamples, using max- pooling extract salient features while capturing complex point dynamics and temporal evolution.
the Gaussian means 14, covariances Σ, and features ofeatt extracted from the time-series radial basis function Gaussian residual module, extracts the geometric topological features of local and global Gaussian clouds through the TG-GCN structure, effectively capturing geometric and temporal patterns. The information on Gaussian means μ aids in encoding the average positions (x, y, of points at different time steps, modeling overall motion and deformation trends; The information on Gaussian covariances Σ describe the deformation degree and direction of each point over time, facilitating the modeling of complex dynamics; The Gaussian features ffeat extract the geometric and temporal patterns in the point cloud, enhancing the model's expressive power and providing geometric features. Furthermore, the incorporation of temporal information aids in establishing the temporal consistency relationships within the 4D spatiotemporal point cloud.
Based on the features extracted from the deformation field, the motion field head and deformation field feature head respectively predict the motion flow and feature field (Fig. 4 with the principle being the learning a continuous deformation field, utilizing the time information in the Gaussian point cloud representation and the spatial dependencies captured by graph convolution, to achieve smooth and realistic deformation of the point cloud, enabling accurate modeling of 4D point cloud sequences and interpolation between time steps.
The Gaussian representation pooling module first projects the original point cloud coordinates and latent features onto Gaussian ellipsoids centered at each point, leveraging the previously established index mapping between the points and the Gaussian ellipsoids (Sec. 3.2). It then applies max- pooling to extract the most prominent features within each Gaussian ellipsoid, enhancing the model's perception of deformation flow and features. However, this Gaussian ellipsoid projection and max- pooling operation inevitably leads certain degree of information loss. To compensate for the potential information loss in the feature extraction process of the Gaussian representation pooling module, further introduce neural field module to learn and refine the point cloud deformation flow and feature expressions.
3.5 Fast Latent-Geometric Feature Fusion (Fast-LG-FFusion) Module
The proposed module fuses the complementary advantages of the latent features Fc from point cloud latent representations and the geometric features Fg from 4D Gaussian deformation fields via an attention- based mechanism. Fc serves as the query, while Fg serves as the key and value. Through linear projections (W, q. WA. W.), the features are mapped into a shared latent space. An attention score is computed between the projected query and key, which then normalized via softmax to obtain attention weights. These weights adaptively assign importance to different feature components, enabling effective fusion. The weighted sum of the projected value features is then computed, integrating multi-modas information. Finally, residual connection with the original query

---

Fe yields the fused representation Fe&g. preserving temporal information:
WaFe (W F (W.Fo).
Fikg = FC + Softmaa
(6)
vdh
The module's efficiency stems from its simple linear operations and avoidance of iterative or recurrent mechanisms, enabling fast computation. Furthermore, the global attention mechanism provides a comprehensive receptive field, enhancing representational capacity compared to naive concate- nation. By adaptively fusing geometric and temporal features, the module improves the model's expressiveness for capturing complex spatio-temporal patterns in large-scale dynamic LiDAR scenes.
3.6 3D Point Cloud Prediction Head
The 3D Point Cloud Prediction head accurately predicts the point cloud at the target time step by fusing multi-source spatio-temporal information, including features Fe&g from the 4D neural field and Gaussian deformation field, time encoding Povenc(T) representing the target time step, and the initial prediction flow + P. This fusion provides rich spatio-tempora priors, enabling the model to reason about the target point cloud's is structure and motion patterns. Through a lightweight MLP, the fused features directly mapped to the residual flow Af*, avoiding prediction from scratch and leveraging the initial interpolated flow field based on the Gaussian deformation field:
f ML Posenc @ (flow P)).
Adding f to the current frame accurately predicts the point cloud at the target time step.
3.7 Objective Function
The proposed NeuroGauss4D-Po optimizes the weights of 4D neural fields and Gaussian deforma- tion fields using three self supervised losses, climinating the need for labeled data. We employ cumulative temporal loss to refine network parameters, facilitating efficient spatiotemporal modeling of 4D point clouds while ensuring smoothness and distribution consistency.
Chamfer Distance Loss: Measures the bidirectional distance between the predicted point cloud Pf and the ground truth Pz. promoting proximity between the predicted and ground truth points.
(cD
Σ PIEP
min ||Ii p2ll2 +
Σ P2E
min llp pill2
(8)
PoE
P.PP
Smoothness Constraint Loss: Enhances the smoothness of the predicted inter-frame flow field AS", where N(A);) denotes the neighbors of Aʃ. - It encourages similarity in flow vectors among neighboring points, ensuring smoother transformations in large-scale scenes. This loss is not required when modeling Dynamic Human Bodies.
l Smooth (A)))
Σ [N(ST) EFEAT
(())(11))- - - 61JEN(61))
Σ
Earth Mover's Distance: Calculates the minimum cost to move points from PT to match the distribution of Pz. where Ty,-P- is the optimal transport map. It promotes a distribution in the predicted point cloud that resembles the ground truth.
LEMD (P
min Tp.
1
Σ PTEPT
llpi - Tpp-1s (pi) 12-
(10)
N
The overall loss integrates these functions across all time steps 4; tú and temporal frames b;, by. weighted by λ1, 22.22.s, It jointly optimizes the geometric precision, smoothness, and distribution similarity of the predicted point clouds across multiple time steps and temporal frames.
C= Σ Σ 4. bE
(Ai p!!)) )+++ + (Af + AaLEMD(PH),PP) p()). (11)
7

---

Table 1: Quantitative comparison with open-source methods DHB-Dataset 2 Errors scaled by x10 to emphasize small- differences in human body metrics. " means lower is better.
" means higher is better. Red and blue denote the first and second best metrics, respectively.
Longdress
Loot
Red&
Soldier
Squat
Swing
Overall
Methods
Param. V
CD EMD
CD
EMD
CD
EMD
CD EMD
CD
EMD
CD
EMD
CI)
J EMD
IDEA-Net 2
0.89 6.01 0.86 8.62 0.94 10.34 1.63 30.07 0.62
6,68 1.24
6.93
1,02
12.03
PointlNet 8
0.98
10.87 0.85 12.10 0.87 10.68
0.97 12.39 0.90
13.99 1.45
14.81
0.96
12.25
1.3M
NSFP [
1.04
7.45
0.81
7.13
0.97
8.14
0.68
5.25
1.14
7.97
3.09
11.39
1.22
7.81
0.12M
PV-RAFT 10
1,03
6,88
0.82
5.99
0.94
7.03
0.91
5.31
0.57
2.81
1.42
10.54
0,92
6.14
0.11 M
NeuralPCI [3]
0.70
4.36
0.61
4.76
0.67
4.79
0.59
4.63
0.03
0.02
0.53
2.22
0.54
3.68
1.85 M
Ours
0.68
3.69
0.59
4.12
0,65
4.20
0.57
4.14
0.00
0,00
0.00
0.00
0.42
2.69
0.10 M
Table 2: Quantitative comparison with other advanced methods the NL Drive dataset. Frame-1, 1,
Frame-2, 2, and Frame-3 denote three interpolated frames evenly spaced between two middle input frames. The symbol i signifies outlier removal during preprocessing (Sec. 3.2)
Frame-1
Frame-2
Frame-3
Average
Methods
Type
CD
EMD
EMD
EMD
CD L EMD $
Forward Flow
0.94
95.18
1.75
132.30
2.55
168.91
1.75
132.13
NSFP III
Backward Flow
2.53
168.75
1.74
132.19
0.95
95.23
1.74
132.05
Forward Flow
1.36
104.57
1.92
146.87
1.63
169.82
1.64
140.42
PV-RAFT 10
Backward Flow
1.58
173.18
1.85
145.48
1,30
102.71
1.58
140.46
PointINet g
Bi-directional Flow
0.93
97.48
1.24
110.22
1.01
95.65
1.06
101.12
NcuralPC1
Neural Field
0.72
89.03
0.94
113.45
0.74
88.61
0.80
97.03
Ours
4D Gaussian Deformation
0.70
86,90
0.93
112.1
0.72
88.85
0.78
95.95
Ours!
4D Gaussian Deformation
0.64
71.92
0.88
91.9
0.65
72.16
0.72
78.66
Experiments
4.1
Implementation Details
Datasets:
Dynamic Human Bodies (DHB) dataset [2 contains 14 sequences of non-rigid human
motions. The NL-Drive 3
dataset captures large-scale motion scenes from KITTI (32, Argoverse
2 [33]. and Nuscenes
3
for autonomous driving. KITTIS (35 32] and KITTIo (36 32 are two
versions of the KITTI Scene Flow dataset with and without occlusion masks, respectively.
Like existing methods 2 8, we use Chamfer Distance (CD) and Earth Mover's Distance (EMD) to measure the consistency between predicted and ground truth point clouds. For the 3D scene flow task, we employ two error metrics EED and Outliers and two accuracy metries, ACC's and ACCR. to quantify performance.
Metries:
Experimental Setup:
We sample the input points to 1024 for object-level scenes and 8192 for autonomous driving scenes. NeuroGaus44-PCG consists of 5 components, as shown in Table
4 Refer to the supplementary materials for the parameter settings of each component and more experimental details.
4.2 Result Comparison on Point Cloud Interpolation
daaooet]deoosttrttes the superior performance of the proposed method, consistently outperforming others on CD and EMD metrics, achieving near-zero errors for sequences like Squat and Swing. The dynamic soft Gaussian representation effectively models non-rigid deformations in human motions by transforming raw point clouds into multiple Gaussian distributions, with radial basis functions learning their temporal residuals. Fusing 4D neural field features preserves per-point detail. The extremely low errors on the DHB dataset validate the model's excellence in non-rigid motion modeling and point cloud interpolation. Notably, the proposed method achieves the best performance with significantly fewer parameters (0.09M) compared to methods like PointlNet [8] (1.30M) and NeuralPCI [3] (1.85M). This is attributed to the efficient Gaussian representation and deformation field design, enabling superior performance through more compact model.
On large-scale autonomous driving LiDAR datasets 3 NeuroGauss4D-PC. demonstrates exceptional temporal point cloud prediction/inneepolaaiio perfoaammannnnonoanaaeaaaaaan performance, significantly outperforming uni-directional
8

---

Table 3: Comparison of method with the best-performing methods multiple datasets and metrics. 'Self" and 'Full' represent self- self-suuer, and supervised methods, respectively. Unlike other methods imputs that use adjacent point clouds (P1, P2), NeuroGaus<4D-PC utilizes (P1, t2).
Inference Input
KITTIs
BS
32
KITTle
364
32
Method
Sup.
EPEsn
ACCs
ACCC
Oulliers
EPEsp 4
ACCss 1 ACCa
Oulhers
FlowNet3D [3
Full
PP..))
0.1767
37.38
66.77
52.71
0.183
9.8
39.4
79.9
FointPWC [3]
Full (P.P8)
0.0694
72.81
88.84
26.48
0.118
40.3
75.7
49.6
Bi- Pow 38
Full.
(P. Pd)
0.0300
92.00
96.00
14.10
0,065
76.9
90.6
26.4
PI-FlowNet 39
Pull
(P, Pd)
0.0224
95.31
98.38
11.86
GMSF HD
Full.
(P.
0.0215
96.22
98.50
9.84
0.033
91.6
95.9
13.7
SCOOP [42
Self.
(P,, Pb)
0.0390
93.60
96.50
15.20
0.047
91.3
95.0
SPFlowNet
41
Self.
(Pt.Pg Pg)
0,0362
87.24
95.79
17.71
0.086
61.1
82.4
39.1
Ours
Self.
(P1.f2)
0.0190
97.72
99.48
9.43
0.035
94.8
97.5
11.1
Input Point Cloud Fmme for The Model:
Frame 5
Frame 9
Frumme 6
Fame
Frame 8
NenroGause 1DSFLabllling NeuraiCC (Oun)
Ground Truth Point Cloud Frame for Interpolation:
Framo 6
Frame 7
Framo
Predicted Point Clouds for All Methods And Time Steps on The Right Are Shown in Orangee:
Predicted Freme
Frame Interpolation in Lagge-scale LIDAR Secnes
Better Alignment Derween Green&Orange Reflects Lighher Accuracy
Figure 5: Compared to advanced point cloud interpolation algorithms 3 9. our method aligns better with the ground truth in predicting point cloud positions and geometry on the NL Drive autonomous driving dataset (33 34 32
3D flow 11, 10 and neural field method 3 as shown in Table It effectively handles challenges in temporal LiDAR scenes, such as large-scale non-linear motions, occlusions, and non-uniform data distributions (Fig. and Fig. (5). NeuroGauss4D- models the 4D temporal point cloud as Gaussian deformation helld with continuous and differentiable Gaussian representations, ensuring smooth interpolation and robust predictions. Temporal Gaussian graph convolutions capture local and global apatio-tempore correlations, enabling fine-graine motion prediction. Notably, Gaussian representations offer parameter efficiency, expressing complex geometric shapes and non- non-linear motions smoothly with fewer parameters compared to neural field methods.
4.3 Result Comparison on Point Cloud Scene Flow
To demonstrate the effectiveness of the proposed point cloud interpolation model in capturing complex deformations and transient motion patterns between two frames, we evaluated its performance on 3D scene flow estimation using the Scene Flow KITTI dataset (both non-occluded 335 32 and
occluded [36] (2)) ) using error and accuracy metrics. As shown in Table 3, our method outperformed
approaches based on feature pyramids (PointWWC (37], Bi-PointFlow (38) ). complex 3D point cloud transformers (PT-FlowNet [39]. GMSF (40),, and self-supervised learning (SPFlowNet (41),
42) across multiple metries. Unlike other methods that require learning inter-frame correspondences, NeuroGauss4D-PP D only needs single point cloud and target timestamp to predict the corresponding point cloud during inference. In the learning phase, NeuroGauss4D-PCI accurately models complex non-rigid deformations and details in temporal point clouds by integrating latent neural fields, iterative Gaussian representations, and 4D deformation fields. Its compact iterative Gaussian representations significantly reduce parameters while effectively capturing continuous point cloud changes by learning temporal residuals Gaussian distribution parameters through radial basis functions, exhibiting superior capability in fraer-frame point cloud matching.
SCOOP*

---

Table 4: Ablation study of the proposed components DHB 2 and NL-Drive [] The components
include: Neural Field for learning spatio-temporal features and iterative Gaussian representation of
3D point plouds ( Gauss. PC )Secc 3.2 T-RBF-GR represents Temporary Radial Basis Function
Gaussian Residual Module (Sec 3 4D Deformation for modeling 4D deformation fields (Sec 3.4
L.-Catt and Fast-I.G-Fusion are different latent geometric feature fusion methods (Sec 3.5
Components
DIIB
10 3)
NL-Drive
Neural Field Gauss.PC' T-RBF-GR 4D Deformation LG-Cat Fast-LG-Fusion CD + EMD + CD + EMD
Param.
0.58
3.70
1.06
105.43
0.027M
0.57
3.62
1,03
103.58
0.028M
/
0.59
3.81
1.04
107.86
0.029M
0.50
3.04
0.80
98.57
0.092M
/
0.49
2.99
0.80
98.03
0.093M
/
0.42
2.69
0.79
97.36
0.099M
/
0.44
2.48
0.78
95.95
0.103M
4.4 Ablation Study
Table shows the effect of each component on model performance. Using just the neural field, the Chamfer Distance (CD) measures 0.58x10 on the DHB dataset and 1.06 on the NL-Drive dataset, showcasing its capacity to capture spatio-temporal features. Adding the Gaussian point cloud representation slightly lowers the CD by 0.01 and 0.03 on DHB and NL-Drive, respectively, indicating limited benefits from this integration. Removing the neural field and relying solely on the Gaussian point cloud and T-RBF-CU module significantly worsens performance, emphasizing the neural field's 's's critical role in modeling 4D spatio-temporal point clouds. However, integrating the 4D Gaussian Deformation Field with the Gaussian representation markedly enhances performance, with 14% and 19% decrease in CD and a 17% and 6.5% reduction in Earth Mover's Distance (EMD) on the two datasets, respectively. This highlights the deformation field's effectiveness in accurately representing dynamic variations. Adding the T-RBF-GE module further improves performance, showcasing its utility in addressing dynamics and temporal correlations. By fusing the latent features and the explicit 4D deformation field features, effectively reduce errors in point cloud prediction. The Fast-LG-Fusion module, in particular, achieves the best performance, attaining the lowest CD and EMD across datasets, except for slight 0.02 increase in CD the DHB dataset compared to direct concatenation.
Conclusion
NeuroGauss4D-PCT is proposed for accurate 4D temporal point cloud modeling and interpolation. 4D neural fields encode latent spatio-temporal features, and an iterative Gaussian cloud representation structures point clouds. A temporal RBF Gaussian residual module smoothly updates Gaussian distribution parameters. 4D Gaussian deformation fields leverage temporal graph convolutions for robust dynamic point cloud modeling. Efficient fusion of latent seatio-temporal and robust Gaussian deformation features enables 3D point cloud prediction. Evaluated on multiple datasets, the method demonstrates significant superiority over existing approaches, particularly in complex dynamics scenarios. Ablation studies validate the contribution each module to overall performance.
Broad Implications: Our work is the first to successfully combine radial basis functions for predicting 3D Gaussian distribution residuals and introduce 4D Gaussian deformation fields into the temporal point cloud prediction task. As the supplementary materials indicate, this is widely applied yet underexplored issue. Finally, we hope that the proposed NeuroGaus<4D-PC) can provide valuable assistance the community in exploring temporal point cloud understanding and modeling.
Limitations: 1) Interpretability: The integration of various features and the opacity of deep neu- ral networks pose challenges in understanding the decision-making process and the fundamental principles behind model predictions. 2) Efficiency: Similar NeRF [43, nearly 90% of the time consumed by runtime scene optimization, with inference accounting for only about 10% of the time.
10

---

References
[1] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 3d-coded: 3d correspondences by deep deformation. In Proceedings of the european conference on computer vision (ECCV), pages 230-246, 2018.
[2] Yiming Zeng, Yue Qian, Qijian Zhang. Junhui Hou, Yixuan Yuan, and Ying He. Idea-net: Dynamic 3d point cloud interpolation via deep embedding alignment. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6338-6347, June 2022.
[3] Zehan Zheng, Danni Wu, Ruisi Lu, Fan Lu, Guang Chen, and Changjun Jiang. Neuralpei: Spatio-temporal neural field for 3d point cloud multi-frame non-linear interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 909-918, 2023.
[4] Charles R Qi. Yin Zhou, Mahyar Najibi, Pei Sun, Khoa Vo, Boyang Deng, and Dragomir Anguelov. Offboard 3d object detection from point cloud sequences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 6134-6144, 2021.
[5] André Zenner, Akhmajon Makhsadov, Sören Klingner. David Liebemann, and Antonio Krüger. Immersive process model exploration in virtual reality. IEEE transactions on visualization and computer graphics. 26(5):2104- 14, 2020.
[6] Haiyan Wang. Liang Yang. Xuejian Rong, Jinglun Feng. and Yingli Tian. supersipe 4d spatio- temporal feature learning via order prediction sequential point cloud clips. In Proceedings of the IEFF/CVF Winter Conference on Applications of Computer Vision, pages 3762-3771, 2021.
Zihao Zhang, Lei Hu, Xiaoming Deng, and Shihong Xia. Sequential 3d human pose estimation using adaptive point cloud sampling strategy. In IJCAI, pages 1330-1337, 2021.
[8] Fan Lu, Guang Chen, Sanqing Qu, Zhijun Li, Yinlong Liu, and Alois Knoll. Pointinet: Point cloud frame interpolation network. Proceedings of the AAAI Conference on Artificial Intelligece, volume 35, pages 2251-2259, 2021.
[9] Chaokang Jiang, Guangming Wang. Jiuming Liu, Hesheng Wang. Zhuang Ma. Zhenqiang Liu, Zhujin Liang. Yi Shan, and Dalong Du 3dsflahelling: Boosting 3d scene flow estimation by pseudo auto-lahelling. arXiv preprint arXiv:2402.18146 2024.
[10] Yi Wei, Ziyi Wang. Yongming Rao. Jiwen Lu, and Jie Zhou. Pv-raft: Point-voxel correlation fields for scene flow estimation of point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6954 0144-6663,44 2021.
[11] Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey. Neural scene flow prior. Advances in Neural Information Processing Systems, 34:7838-78511 2021.
[12] Xianghui Yang, Guosheng Lin, Zhenghao Chen, and Luping Zhou. Neural vector fields: Implicit represen- tation by explicit learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 16727-16738, June 2023.
[13] Amirali Molaei, Amirhossein Aminimehr, Armin Tavakoli, Amirhossein Kazcrouni, Bobby Azad, Reza Azad, and Dorit Merhof. Implicit neural representation medical imaging: A comparative survey. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2381-2391, 2023.
[14] Tero Karras, Miia Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in neural information processing systems, 34:852-863, 2021.
[15] Xin Huang, Qi Zhang, Ying Feng, Xianyu Li, Xuan Wang, and Qing Wang. Local implicit ray function for generalizable radiance field representation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 97-107, 2023.
[16] Tiago Novello, Vinicius Da Silva, Guilherme Schardong, Luiz. Schirmer, Helio Lopes, and Luiz Velho. Neural implicit surface evolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14279 14289, 2023.
[17] Guandao Yang, Serge Belongie. Bharath Hariharan, and Vladlen Koltun. Geometry processing with neural fields. Advances in Neural Information Processing Systems, 34:22483 -22497, 2021.
[18] Daniele Grattarola and Pierre Vandergheynst. Generalised implicit neural representations. Advances in Neural Information Processing Systems, 11130046304555 2022.

---

[19] Kashyap Chitta, Aditya Prakash, and Andreas Geiger. Neat: Neural attention fields for end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 15793-15803, 2021.
[20] Xueqian Li. Jianqiao Zheng, Francesco Ferroni. Jhony Kaesemodel Pontes, and Simon Lucey. Fast neural scene flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9878-9890, 2023.
[21] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkütherr and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4):1- 14, 2023.
[22] Byeonghyeon Lee, Howoong Lee, Xiangyu Sun, Usman Ali, and Eunbyung Park. Deblurring 3d gaussian splatting. arXiv preprint arXiv:2401.00N 2024.
[23] Guikun Chen and Wenguan Wang. survey on 3d gaussian splatting. arXiv preprint arXiv:2401.080 2024.
[24] Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation. arXiv preprint 12X1v:2403.13365. 2024.
[25] Yifei Zeng, Yanqin Jiang, Siyu Zhu, Yuanxun Lai, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, and Yao Yao. Stag4d: Spatial- Spatial-temporal anchored generative 4d gaussians. arXiv preprint arXiv:2403.149) 14939, 2024.
[26] Guanjun Wu, Taoran Yi, Jiemin Fang. Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. arXiv preprint arXiv:2310.88526 2023.
[27] Guangming Wang, Chaokang Jiang, Zehang Shen, Yanzi Miao, and Hesheng Wang. Sigan: Unsupervised generative adversarial learning of 3d scene flow from the 3d scene self. Advanced Intelligent Systems, 4(4):2100197, 2022.
[28] Juuming Liu, Guangming Wang. Weicai Ye, Chaokang Jiang, Jinru Han, Zhe Liu, Guofeng Zhang, Dalong Du, and Hesheng Wang. Difflow3d: Toward robust uncertainty-aware scene flow estimation with diffusson model. arXiv preprint arXiv:2311.17456. 2023.
[29] Fan Lu, Guang Chen, Zhijun Li, Lijun Zhang, Yinlong Liu, Sanqing Qu and Alois Knoll. Monet: Motion-based point cloud prediction network. IEEE Transactions on Intelligent Transportation Systems, 23(8):13794-1380, 2022.
[30] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein. and Justin M Solomon. Dynamic graph for learning on point clouds. ACM Transactions on Graphics (log), 38(5):1-12, 2019.
[31] Qian-Yi Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3d: A modern library for 3d data processing. arXiv preprint arXiv: 1 09847, 2018.
[32] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. The kitti vision benchmark suite. URL http://www. cvlibs. net/ddatasts//kiti, 1-5):1113, 2015.
[33] Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagject Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, al. Argoverse: tracking and forecasting with rich maps. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 8748-8757 2019.
[34] Holger Caesar, Varun Bankiti, Alex H Lang. Sourabh Vora, Venice Erin Liong. Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IELE/CVF conference on computer vision and puttern recognition, pages 11621-11631, 2020.
[35] Xiuye Gu, Yijie Wang, Chongruo Wu, Yong Jae Lee. and Panqu Wang. Hplflownet: Hierarchical permutohedral lattice flownet for scene flow estimation on large-scale point clouds. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 3254-3263, 2019.
[36] Xingyu Liu, Charles R Qi. and Leonidas Guibas. Flownet3d: Learning scene flow in 3d point clouds. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 529-537. 2019.
[37] Wenxuan Wu, Zhi Yuan Wang, Zhuwen Li, Wei Liu, and Li Fuxin. Pointpwe- net: Cost volume on point clouds for (self-) supervised scene flow estimation. In European Conference on Computer Vision, pages 88 107, 2020.
12

---

[38] Wencan Cheng and Jong Hwan Ko. Bi-pointllowee: Bidirectional learning for point cloud based scene flow estimation. In European Conference on Computer Vision, pages 108-124. Springer, 2022.
[39] Jingyun Fu, Zhiyu Xiang, Chengyu Qiao, and Tingming Bai. Pt-flownet: Scene flow estimation on point clouds with point transformer. IEEE Robotics and Automation Letters, 8(5):2566-255 2573, 2023.
[40] Yushan Zhang, Johan Edstedt, Bastian Wandt, Per-Erik Forssén, Maria Magnusson, and Michael Felsberg. Gmsf: Global matching scene flow. arXiv preprint arXiv:2305. 174322 2023.
[41] Yagi Shen, Le Hui, Jin Xie, and Jian Yang. Self-supervised 3d scene flow estimation guided by superpoints. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5271- 5280, 2023.
[42] Itai Lang, Dror Aiger, Forrester Cole, Shai Avidan, and Michael Rubinstein. Scoop: Self-supervised correspondence and optimization based scene flow. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5281-5290, 2023.
[43] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99- ([06))9--06,,) 2021.
[44] Jiuming Liu, Dong Zhuo, Zhiheng Feng. Siting Zhu, Chensheng Peng, Zhe Liu, and Hesheng Wang. Dvlo: Deep visual-lida odometry with local-to-global feature fusion and bi-directional structure alignment. arXiv preprint X4x:33403. 18274, 2024.
[45] Chaokang Jiang, Guangming Wang, Jinxing Wu, Yanzi Miao, and Hesheng Wang. Ffpa- net: Efficient feature fusion with projection awareness for 3d object detection. arXiv preprint wxXiv2209.004119 2022.
[46] Jiuming Liu, Guangming Wang. Zhe Liu, Chaokang Jiang, Mare Pollefeys, and Hesheng Wang. Regformer: efficient projection-aware transformer network for large-scale point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8451 141600 2023.
[47] Jiuming Liu, Guangming Wang, Chaokang Jiang. Zhe Liu, and Hesheng Wang. Translo: A window-based masked point transformer framework for large-scale lidar odometry. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1683-1691, 2023.
[48] Yanzi Miao, Huiying Deng, Chaokang Jiang, Zhiheng Feng. Xinrui Wu, Guangming Wang, and Hesheng Wang. Pseudo-lidas lidar for visual odometry. IEEE Transactions on Instrumentation and Measurement, 2023.
[49] Guangming Wang. Zhiheng Feng, Chaokang Jiang, and Hesheng Wang. Unsupervised learning of 3d scene flow with 3d odometry assistance. arXiv preprint (XXiv:2209.04945 2022.
[50] Yuwen Xiong, Wei-Chiu Ma. Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1074-1083, 2023.
[51] Chaokang Jiang. Guangming Wang, Yanzi Miao, and Hesheng Wang. 3d scene flow estimation on pseudo-lidar: Bridging the gap on estimating point motion. IEEE Transactions on Industrial Informatics, 2022.
[52] Ran Cheng, Christopher Agia, Yuan Ren, Xinhai Li, and Liu Bingbing, S3cnet: A sparse semantic scene completion network for lidar point clouds. In Conference on Robot Learning, pages 2148-2161. PMLR, 2021.
[53] Yudi Dai, Yitai Lin, Chenglu Wen, Siqi Shen, Lan Xu, Jingyi Yu, Yuexin Ma. and Cheng Wang. Hsc4d: Human-centered 4d scene capture in large-scalc indoor-outdoor space using wearable imus and lidar. In Proceedings of the TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6792-6802, June 2022.
[54] Siting Zhu, Renjie Qin, Guangming Wang, Juuming Liu, and Hesheng Wang. Semgauss-slam: Dense semantic gaussian splatting slam. arXiv preprint arXiv:2403.07494. 2024.
[55] Siting Zhu, Guangming Wang, Hermann Blum, Jiuming Liu, Liang Song, Mare Pollefeys, and Hesheng Wang. Sni-slam: Semantic neural implicit slam. arXiv preprint arXiv:2311.11016 2023.
13

---

Supplemental Material
A Datasets
Dynamic Human Bodies (DHB) dataset consists of point cloud sequences of non-rigid deformable human motions. Six sequences--Lngddres Loot, Redandblack, Soldier, Squat 2, Swing are designated for testing, and the remaining eight for training. The NL-Drive dataset [3, integrating
data from KITTI odometry [32], Argoverse 2 sensors [33]. and Nuscenes [34]. focuses on capturing
large-scale motion scenes and divided into training. validation, and test sets in 14:3:3 ratio.
specifically targets hard samples with significant inter-frame motion and large self-motion, employing detailed 6-DOF pose transformations and filtering mechanism to support the training and testing of advanced autonomous driving algorithms. KITTIs [35 32 and KITTIo [36] (32, two versions of the KITTI Scene Flow dataset processed for 3D point cloud scene flow evaluation, exclude occluded points in the former, containing 142 training scenes, while the latter includes all points with occlusion masks, featuring 150 training scenes.
B
Method Details
Table 5: Model Input and Hyperparameter Settings
Component
Setting
Model Input
4 frames of point clouds sampled regular 3-frame intervals:
DHB dataset: 1024 points per frame
NL-Drive dataset: 8192 points per frame
Time step 0 to 1, spatial and temporal dimensions: and
Iterative Gaussian Cloud
Number of Gaussian components M: 16 (LiDAR), 8 (DHB)
Soft Clustering
Number iterations x for fitting Gaussian ellipsoids: 200
DGGNNN for local geometric feature learning:
convolutional blocks with kernel size 1, batch normalization,
ReLU activation, and dropout (0.3 probability)
Input channels: 16 x 2, 16 x 2, 16 x 2, 16 x 2, 16 x 4
Output channels: 16, 16, 16, 16, 64
Dynamic k-NN with k ranging from to 32
4D Neural Field
Input MLP: width 1280, depth
Hidden MLPs: with width 32, depth
Decoder MLP: width 32, depth
Leaky ReLU activation
Temporal RBF
RBF centers (for time steps)
Gaussian Residual
Input feature dimension: 32
RBF centers evenly spaced between 0 and
Learnable standard deviations for RBFs
Learnable translation, rotation, scale parameters for Gaussian ellipsoids
Learnable transformation parameters for input features
4D Gaussian Deformation Field
Temporal encoding dimension:
Intermediate and output dimensions for TGCN: 32
Optimization
AdamW optimizer, learning rate 0.001, weight decay 0 5000 iterations, early stopping on optimal solution chamfer distance (1.0), earth mover's distance (50.0). smoothness regularization (1.0) Poly learning rate scheduler
Weighted loss
Input Setting: For dynamic point cloud sequence evaluation, the input comprises 4 point cloud frames sampled at regular 3-frame intervals. Each frame contains 1024 points for the DHB dataset and 8192 points for the NL-Drive dataset. The temporal resolution is 1, with the initial frame at
14

---

time step 0 and the final frame at time step 1. The spatial and temporal dimensions are 3 and 1, respectively.
Iterative Gaussian Cloud Soft Clustering: A key hyperparameter is the number of Gaussian components M, set to 16 for LiDAR and 8 for DHB. The iteration count K for Gaussian ellipsoid parameter fitting is 200. A 5-block DGCNN learns local geometrie features from the input Gaussian cloud data. Each block employs 2D convolution (kernel size 1), batch normalization, ReLU activation, and dropout (0.3 probability). The input channels for blocks 1-4 are 16 * 2, with 16 output channels each. The final block has 16 * input channels and 32 output channels. For graph representation, DGCNN uses k-NN with l ranging from 8 to 32 based on input size.
4D Neural Field: The comprises an input MLP (width 1280, depth 1), two hidden MLPs (width 32, depth 5), and a decoder MI.P (width 32, depth 1), utilizing Leaky ReIUU activation.
Hyperparameters include 4 RBF centers (corresponding to time steps) and 32-dimensional input features. RBF centers are evenly distributed between 0 and I, with learnable standard deviations. Translation, rotation, scale parameters of Gaussian ellipsoid distributions, and input feature transformation parameters are learnable.
Temporal Radial Basis Function Gaussian Residual:
4D Gaussian Deformation Field: Temporal information is encoded as an 8-dimensional vector. Intermediate and output dimensions for temporal Gaussian graph convolutional network are 32.
A optimizer (learning rate 0.001, weight decay 0), 5000 iterations, minimizing a weighted loss (chamfer distance loss weight 1.0, earth mover's 's distance loss weight 50.0, smoothness regularization loss weight 1.0), with early stopping. Poly learning rate scheduler is employed.
Optimization:
Computational Performance: The average training time is 1.31 seconds per iteration, and the average evaluation time is ~ 0.23 seconds per frame. The GPU memory consumption 7436 MiB. The model is evaluated implementation based on the PyTorch deep learning framework, utilizing CUDA acceleration for efficient parallel computation the GPU.
NVIDIA GeForce RTX 3090 GPU with 24GB of VRAM. The
C Applications
Heterogeneous sensors like LiDAR and cameras often suffer from asynchronous data acquisition
44 45), posing challenges fusing their complementary data streams. As illustrated in Fig. 6 NeuroGaus44D-PC offers a powerful solution by enabling the interpolation of arbitrary point cloud frames from continuous point clouds, facilitating precise multi-sesse time synchronization. This capability unlocks seamless integration of data from multiple sensors, enhancing the robustness and accuracy of perception systems in dynamic environments.
Moreover, NeuroGauss4D-PCI can be effectively applied to 4D automatic annotation tasks, assigning point-wise labels unlabeled intermediate frames based on sparse 3D labels. This automated labeling process significantly reduces the time and effort required for manual annotation, a critical bottleneck in many computer vision applications. Consequently, it accelerates the development and deployment of systems relying on labeled point cloud data.
Furthermore, NeuroGauss4D-PCC demonstrates remarkable accuracy in LiDAR point cloud densi- fication, generating high-resolution, dense point clouds from sparse inputs. Accurate dense point clouds serve invaluable data sources for wide range of autonomous driving perception tasks 46
48 49,, including depth estimation [50, 51, occupancy prediction [52,, and 4D scene
reconstruction [33 (4 55). The ability to obtain high-fidelity representations of the environment from limited sensor data highlights the immense potential of NeuroGauss4IDPCT in enabling robust and reliable perception systems.
D Supplementary Visualization
In the qualitative comparison (Fig. 7 on the DHB dataset (2). NeuroGauss4D-PP demonstrates superior performance compared to state-of-the-art open-source models, including PointlNet (8], IDEAA-Ntt 2 and NeuralPCI 3. in the task of temporal point cloud interpolation. The proposed method excels capturing and modeling non-rigid human motions with high fidelity, as evident from the visual results. NeuroGaaus44-PCI represents the input point cloud as set of Gaussian ellipsoids
15

---

Tempona Synchnomizationn Auto-labeling
Densilying Point Clouds with NeunoGauss-DDPCC
Interpolated Frames (F F2
Denstiomion
Figure 6: Visualization of NeuroGauss4D-PCI for multi-sensor time synchronization and point cloud densification applications. In point cloud densification, sparse ground truth points are shown in green, while the predicted dense point cloud shown in red, exhibiting good overlap with the sparse ground truth.
GT
PointINet
IDEA-Net
NeuralPCI
Ours
Figure 7: Qualitative visualizations on the DHB dataset demonstrate the evident superiority of the proposed method in reconstructing fine details when compared to existing state-of-the-at open-source models 37 2 3]
through the Iterative Gaussian Cloud Soft Clustering module, effectively encoding local geometric structures and establishing meaningful correspondences across temporal frames. Furthermore, the Temporal Radial Basis Function Gaussian Residual module enables effective learning of temporal dynamics and residual deformations by incorporating radial basis functions (RBFs) centered at different time steps, allowing for accurate interpolations, especially in scenarios involving complex non-rigid motions. The 4D Neural Field module, the core of NeuroGaus<4D-PCe leverages the expressive power of neural fields to model the continuous 4D space of point cloud sequences, encoding both spatial and temporal information in unified neural field to generate high-quality interpolated frames that maintain spatial and temporal coherence. Additionally, the Feature Interpolator module employs an attention-based mechanism to aggregate and interpolate features from neighboring time steps, enabling cross-temporal propagation and integration of relevant information, further enhancing the capability to handle non-rigid deformations. Overall, NeuroGauss4D-PGn PCC'ss innovative
16

---

Ground Truth
NeuralPCI
NeuroGauss4D-PG
CD Emer 0
CD Emor 25cm
Small
Figure 8: Quantitative comparison with pure neural field method B3 the NL-Drvie dataset.
FARE
Hlepoting Aarg dee paoreeet
Figure 9: Point cloud interpolation challenging autonomous driving scenarios. Blue/yellow: input clouds at // Green: ground truth at T6,7. Pink: predictions To,7. The scenes feature local occlusions, sparse point clouds, repeating structures, and large displacements simulated by extended frame intervals. The overlap between green and pink points demonstrates our algorithm's accuracy in these complex scenarios, showcasing its robustness to data sparsity, occlusions, ambiguous temporal features from repeating structures, and large environmental displacements.
architecture, combining Gaussian ellipsoid representations, temporal RBF residuals, 4D neural fields, and attention-based feature interpolation, endows it with unique ability to accurately model and interpolate non-rigid point cloud sequences. The qualitative results on the DHB dataset demonstrate the method's superiority in handling complex human motions.
The qualitative visualization (Fig. on the NL-Drive dataset showcases NeuroGaus44D-PPe exceptional performance in point cloud interpolation and prediction for autonomous driving scenarios. Compared NeuralPCI, our method exhibits superior temporal consistency and local coherence, benefiting from the robust mathematical constraints imposed by the Gaussian parameterization. For unstructured and sparse point clouds, the 4D modeling approach enables more accurate predictions in sparse and unstructured regions. Additionally, NeuroGauss4D-PC) demonstrates robustness in handling occluded areas, as evident in the second row of the visualization.
17

---

PC PC.
CC
PC.
Nosee Rata:: U1
Noise Rator Mari PC
Naise STOV
Noise STEP
Nouse
Imput Point Clouds
Point Clouds and
Point Clouds and
Inpur Point Cluuds
GT and Predicted 4
GT and Predicted Point Better Alignment Between Green And Pink Points Indicates Higher Prediction Accuracy
GT and Predicted Point GT and Predicted Point at
Figure 10: Robustness analysis of our point cloud interpolation method under varying noise conditions.
Top row: Input point clouds PCr=o (blue) and PCr=s (yellow) with different noise ratios (0.4, 0.8) and standard deviations (0.4, 0.8). Bottom row: Comparison between ground truth PC (green) and predicted PC (pink) point clouds at = 6. Noise is added to input point clouds using Gaussian distribution, where noise ratio determines the proportion of points affected, and noise STD defines the standard deviation of the noise. The alignment between green and pink points indicates prediction accuracy. Results demonstrate our method's resilience increasing noise levels, maintaining reasonable performance even under severe noise conditions (noise ratio 0.8, STD 0.8).
18

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The abstract and introduction accurately reflect the main contributions and scope of the paper. The paper proposes novel point cloud interpolation method called NeuroGauss4D-PC) which effectively represents point cloud data using iterative Gaussian clouds. The experimental section demonstrates and discusses the effectiveness of the proposed method.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed in detail the limitations of this work in the conclusion
section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
19

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The code has been open-sourced in the supplementary material, providing the necessary information to reproduce our experiments. Furthermore, all experimental details, including datasets, model architectures, training and evaluation procedures, have been meticulously described in the paper, ensuring the reproducibility of the reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
20

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Please refer to the supplementary materials, we have provided detailed open- source code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccm htpppp.ccpubllcgguddeeeee. wttp://giiieeeeeeeeee /oweeedee/ooeeeemmeeeeeeee splicclicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detail the experimental details of our method in the main experimental section and in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification; The paper does not report error bars or provide information about the statistical significance of the experiments. However, we provide statistical error results on the test set, such as Chamfer Distance error, Earth Mover's Distance error, and average endpoint error, which are widely accepted evaluation metrics in the community. Notably, many of the compared methods the paper also do not report such information. The lack of error bars statistical significance measures is common practice in this field.
21

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We illustrate type of compute workers, memory, time of execution of our model in detail in the experimental details in the main text and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't 'tttt make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics http:///wrrrpp.ccooo/a//ccccccea
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification: We have carefully reviewed the requirements of the NeurolPS Code of Ethics and confirmed that our paper complies with the NeurolPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
22

---

Answer: [Yes]
Justification:
We discussed the broad positive impact of our work at the end of the paper.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use some the code from NeuralPCI 3 and all the code this repository uses Apache License 2.0. And we have correctly cited this work in the main paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
23

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide detailed documentation on the model architecture, training proce- dures, and evaluation protocols in the main paper and supplementary material. In addition, we also release the code and pre-trained models in the supplementary material, enabling reproducibility and promoting future research in this area. We carefully anonymize sets during the submission process and will provide publicly accessible zip file upon publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
25


Reviews:
[{"review_id": "PC1XnobyGf", "reviewer": "Reviewer_xq5Q", "summary": "This paper proposes a method, NeuroGauss4D, to tackle the problem of 4d point cloud interpolation. NeuroGauss4D consists of the following 5 components:\n1. \"Iterative Gaussian Soft Clustering\": a module to encode the scene to Gaussian representation with DGCNN features (map: X, Y, Z, T -> Gaussians(Mu, Cov, Feat)).\n2. \"4D Neural Field\": a neural field with Fourier features and MLP to capture spatio-temporal features in latent space (map: X, Y, Z, T -> latent).\n3. \"Temporal RGB-GR\": a module with temporal radial basis functions to interpolate Gaussian parameters in continuous time (map: Mu, Cov, Feat, T -> ΔMu, ΔCov, ΔFeat, ΔT).\n4. \"4D Gaussian Deformation Field\": a module that take Gaussian parameters (Mu, Cov, Feat) and the latent features as input to predict the motion and feature deformation field (map: Mu, Cov, Feat, T -> deformation field).\n5. \"Fast-LG-Fusion\": an attention mechanism that fuses the latent features (FL) and the geometric features (FG, from the deformation field) to predict the point cloud at the target time step (map: FL, FG, T -> point cloud).", "questions": "1. It would be good to clarify in paper the exact meaning of \"Frame-1, Frame-2, and Frame-3\" in Table 2. Does it mean that we fit the scene with frame {0, 4, 8, 12, ...}, and then we evaluate on frame {1, 2, 3, 5, 6, 7, 9, 10, 11, ...}, and we call frame {1, 5, 9, ...} as Frame-1, frame {2, 6, 10, ...} as Frame-2, and frame {3, 7, 11, ...} as Frame-3?\n2. In Table 5, when we say \"4 frames of point clouds sampled at regular 3-frame intervals\", does it mean that for each scene, we only train on 4 frames {0, 4, 8, 12} and we evaluate on 9 frames {1, 2, 3, 5, 6, 7, 9, 10, 11}? If this is the case, why do we only train on these 4 frames but not fit the full point cloud sequence of the scene in the raw dataset?\n3. In line 456, when we say \"initial frame at time step 0 and the final frame at time step 1\", what do the \"0\" and \"1\" refer to? For example, in equation 1, it is clear that there can be time steps larger than 1, e.g. time step 4, 8, etc. Is the \"0\" and \"1\" in line 456 referring to some form of normalized time step? So, what exactly is the time step gap (in seconds) between the training frame and evaluation frames?\n4. From equation 1, it seems that the method only fits one model for one whole scene, which is good. Can you confirm this?\n5. In line 476 we report the \"average training time is ∼ 1.31 seconds per iteration\". Per my understanding, this \"iteration time\" means the training iteration. It is not clear if the time includes \"Iterative Gaussian Cloud Soft Clustering\" (which has to be done for each time step), and other feature extraction steps. It would be more clear to give a high-level timing number, for example, could you report \"how long does it take to fit a full of ~X training frames each containing ~Y number of points\"?", "limitations": "Yes, the authors have discussed the above.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This method shows the application of dynamic Gaussian representation to the problem of 4D point cloud interpolation.\n2. This method outperforms previous methods on the set of evaluations provide by the author.", "weaknesses": "1. This method shows the application of dynamic Gaussian representation to the problem of 4D point cloud interpolation, which is a good idea. However, the execution of the idea seems to make the method overly complex. NeuroGauss4D-PCI's pipeline contains 5 different components, where a mixture of representations are used. Besides more hand-tuning for hyperparameters and longer training time, this complexity could make it hard to interpret the fundamental principles and the key components behind the full model.\n2. It seems that the method is not very efficient. For example, the \"Iterable Gaussian Soft Clustering\" procedure shown in Algorithm 1 is only capable of taking one timestamp of point cloud (N, 3) as input, which means that this has to be done for each timestamp of the point cloud sequence.\n3. Similar to the point above, the method might be limited on the size of the point cloud sequence that it can handle. For example, line 218 says \"we sample the input points to 1024 for object-level scenes and 8192 for autonomous driving scenes\". A point cloud of 1024 points in autonomous driving scenes is very small, which could limit the method's applicability to real-world scenarios. The authors can provide more information on how the method will perform when we have a larger point cloud size or a longer sequence of point clouds.\n4. The author provides rich information about the method, which is good, but the clarity of the writing can be improved. Some of these are mentioned in the \"Questions\" section, and it would be nice to include them in future versions of the paper. There are other inconsistencies in the paper, for example:\n    - Equation 1 uses 0-based indexing, but line 29 uses 1-based indexing.\n    - Line 109 has an extra double quote.\n    - Line 117 says \"Iterative Gaussian Cloud Soft Clustering\" but the Figure 2 uses \"**Iterable** Gaussian ...\".\n    - Equations shall be written as part of a sentence, not as separate sentences (Eq. 5, 8, 9, 10, 11; and also Eq. 4).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough review and valuable feedback. Here are our responses to your questions:\n## Weaknesses\n### Q1: Is the method overly complex with its 5 components?\n**A1:** Our method is designed for efficiency and effectiveness:\n1. **Parameter Efficiency:** NeuroGauss4D-PCI uses only about 0.1 million parameters, significantly fewer than similar methods requiring millions.\n2. **Logical Pipeline:** Our components form a clear sequence:\n\n   a) Structure point clouds\n\n   b) Model temporal evolution\n\n   c) Capture fine details\n\n   d) Fuse information\n3. **Component Breakdown:**\n   - Gaussian Soft Clustering: Organizes unstructured point clouds\n   - Temporal RBF-GR Module: Enables smooth temporal interpolation\n   - 4D Gaussian Deformation Field: Simulates long-term motion trends\n   - 4D Neural Field: Captures high-frequency details\n   - Fusion Module: Combines geometric and latent features\n4. **Benefits:**\n   - Improved Interpretability\n   - Reduced Parameter Count\nEach component addresses specific challenges in point cloud interpolation.\n### Q2: Is it inefficient that the algorithm must perform operations on each timestamp of the point cloud sequence?\n**A2:** NeuroGauss4D-PCI is designed for per-scene optimization, not real-time processing.\n- Processes single-frame point clouds sequentially\n- Accumulates loss across the entire sequence\n- Performs backpropagation and optimization after processing all frames\n\nSingle-frame processing takes about 10ms. The majority of computation time is spent on backpropagation and parameter updates after processing the full sequence. Nevertheless, optimizing a large-scale LiDAR scene requires only 2-3 minutes.\n### Q3: Can the method handle large point cloud sizes or sequences?\n**A3:** We use different point cloud sizes for distinct scenarios:\n- Object-level scenes: 1024 points. (Aligns with Idea-net)\n- Autonomous driving scenes: 8192 points. (Consistent with NeuralPCI)\n\nWe also provide quantitative comparisons of different point cloud densities:\n\n**Table 1: Quantitative comparison results on NLDrive dataset with different input point cloud densities.**\n| Point Cloud Size | NSFP | PV-RAFT | PointINet | NeuralPCI | 3DSFLabelling | Ours† |\n|:----------------:|:----:|:-------:|:---------:|:---------:|:------------:|:----:|\n|       1024       | 6.81 |  6.92   |   5.64    |   5.04    |     6.19     | **4.01** |\n|       2048       | 4.01 |  3.98   |   3.10    |   2.59    |     3.41     | **2.00** |\n|       4096       | 2.60 |  2.51   |   1.70    |   1.38    |     1.80     | **0.99** |\n|       8192       | 1.80 |  1.65   |   1.10    |   0.80    |     1.19     | **0.72** |\n|      16384       | 1.30 |  1.21   |   0.72    |   0.48    |     0.92     | **0.38** |\n\n### Q4: How will you improve writing clarity?\n**A4:** We will:\n- Unify to 0-based indexing\n- Remove punctuation errors\n- Use consistent terminology\n- Integrate equations into sentences\n\n## Questions\n### Q5: Clarify \"Frame-1, Frame-2, and Frame-3\" in Table 2.\n**A5:** We use frames {0, 4, 8, 12} as input and evaluate on frames {5, 6, 7}.\n### Q6: Explain the training and evaluation process in Table 5.\n**A6:** We optimize on frames {0, 4, 8, 12} and evaluate on {5, 6, 7}. We also present experimental results for both shorter and longer frame intervals:\n\n**Table 2: Comparison of CD errors (×10^-2^) for different methods across various time intervals**\n| Time Interval (frames) | NSFP | PV-RAFT | PointINet | IDEA-Net | NeuralPCI | 3DSFLabelling | Ours |\n|:----------------------:|:----:|:-------:|:---------:|:--------:|:---------:|:-------------:|:----:|\n| 1 | 0.69 | 0.78 | 0.91 | 1.01 | 0.49 | 0.69 | **0.39** |\n| 2 | 0.98 | 0.86 | 0.913 | 1.02 | 0.52 | 0.73 | **0.40** |\n| 3 | 1.22 | 0.90 | 0.92 | 1.03 | 0.53 | 0.89 | **0.42** |\n| 4 | 1.48 | 1.03 | 0.99 | 1.09 | 0.63 | 1.03 | **0.43** |\n| 5 | 1.87 | 1.30 | 1.132 | 1.192 | 0.72 | 1.12 | **0.46** |\n| 6 | 2.01 | 1.38 | 1.23 | 1.32 | 0.90 | 1.30 | **0.48** |\n| 7 | 2.20 | 1.502 | 1.41 | 1.48 | 1.06 | 1.48 | **0.51** |\n| 8 | 2.42 | 1.83 | 1.77 | 1.50 | 1.31 | 1.80 | **0.54** |\n| 9 | 2.83 | 2.02 | 1.81 | 1.76 | 1.48 | 1.99 | **0.57** |\n\nOur method shows the lowest CD error across all time intervals.\n### Q7: Explain the time steps \"0\" and \"1\" in line 456.\n**A7:** We normalize all input times to 0-1 to handle different sequence lengths, simplify training, and enhance generalization.\n### Q8: Does the method fit one model per scene?\n**A8:** Yes, NeuroGauss4D-PCI uses per-scene fitting, allowing high-quality interpolation at any time point after a 3-minute optimization.\n### Q9: Can you provide detailed timing information?\n**A9:** Here's our time consumption breakdown:\n**Table 3: Algorithm Time Consumption Statistics**\n| Processing Step                            | 1024 points | 8192 points |\n|:-------------------------------------------|------------:|------------:|\n| **Single Frame**                           |             |             |\n| Time Encoding                              |      0.0003 |      0.0003 |\n| 4D Neural Field                            |      0.0007 |      0.0008 |\n| RBF-GR+4DGD                                |      0.0041 |      0.0034 |\n| LG-fusion                                  |      0.0004 |      0.0028 |\n| Prediction Head                            |      0.0005 |      0.0023 |\n| Loss Calculation                           |      0.0048 |      0.0022 |\n| **Total (Single Frame)**                   |  **0.0108** |  **0.0118** |\n| **Sequence (4 Frames)**                    |             |             |\n| Loss Backpropagation + Optimizer Update    |      0.0567 |      0.0590 |\n| **Total (One Sequence Iteration)**         |  **0.0753** |  **0.1572** |\n\nNote: The Iterative Gaussian Cloud Soft Clustering module runs only once in the preprocessing stage, taking about 0.2248 seconds, and is not included in the table. The total time for one sequence iteration includes the processing time for 4 single frames plus additional sequence-level operations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yi9CVKMDgK", "reviewer": "Reviewer_ZfCH", "summary": "This paper proposes a novel method for point cloud interpolation, which aims to address complex non-rigid scenarios. It turns point clouds into 3D Gaussians via iterative soft clustering, and then utilizes several 4D spatio-temporal modules to fuse latent features with neural fields. Specifically, this paper employs the temporal radial basis function for Gaussian interpolation, the graph convolutional network for feature extraction and the attention module for feature fusion. Comprehensive experiments demonstrate the leading performance across indoor and outdoor datasets, along with a thorough ablation study for the proposed components.", "questions": "- This paper may not be related to the work of 3DGS series. It just uses 3D Gaussians but has nothing to do with \"splatting\". Also, the proposed method is more similar to the super-point concept or the Gaussian Mixture Model in clustering and should be discussed.\n- Does DGCNN use pre-trained models or is it trained from scratch? Retraining DGCNN for each optimization can be time consuming and may not be meaningful for fitting individual samples.\n- In the Temporal RBF-GR Module, the 3D Gaussian needs to be interpolated, so how to ensure the correspondence? Also, is it feasible to visualize the Residual Gaussian to prove its reasonableness and validity?\n- Quantitative comparisons of efficiency with other methods. It would be better to analyze the training/inference time for each module. In addition, can the proposed method be extended to more frames or more points of input, and what is the corresponding efficiency?\n\n\n\ntypo: \n\n- L136 - \"Table4.2\"?\n\n- missing arrows of ACC_S and ACC_R in Table 3", "limitations": "- The entire framework can be further simplified and improve its efficiency.\n- Additional supervision or constraints can be introduced for 3D Gaussians rather than only points.\n- Use more challenging datasets beyond DHB. In Table 1, the results for both Squat and Swing sequences are 0.00, which may need to retain higher precision or change to other datasets.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- It is the first to introduce 3D Gaussians for point cloud interpolation, which shows great performance in modeling non-rigid deformations and non-linear trajectories.\n\n- The proposed 4D Gaussian deformation fields leverage temporal graph convolutions for spatio-temporal feature aggregation.\n\n- The experiments are conducted on multiple datasets, including indoors and outdoors. It is also compared on the scene flow benchmark to demonstrate its effectiveness. And the ablation study is comprehensive for each proposed module.", "weaknesses": "- The proposed method is overly complex and redundant, which shows a trivial combination of existing modules and limited contribution. Moreover, the proposed modules lack strong motivation and necessity, and the interpretability of the intermediate features is difficult to clarify.\n- According to the Supplementary Material, nearly two hours of optimization time is required for just four frames of input, which is much more than the previous method. The problem may lie in the design of proposed method, which is not suitable for an optimization-based approach, but rather a learning-based approach. For example, some feature extraction or fusion may not be meaningful or necessary for a single sample fitting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful critique.\n## Q1: Algorithm complexity and limited contributions\nA1:\n**Due to space constraints, we kindly refer you to Reviewer xq5Q's Q1 for the explanation on model complexity. We apologize for any inconvenience.**\n\n**Design Motivation:**\n- Gaussian Soft Clustering: Structures multiple unordered point clouds, improving upon methods like NeuralPCI\n- Temporal RBF-GR Module: Overcomes linear motion assumptions of methods such as PointINet\n- 4D Gaussian Deformation Fields: Captures complex motion patterns over extended time spans, surpassing scene flow-based approaches\n\n**Key Innovations**\n - 4D Gaussian representation for spatiotemporal features\n - Temporal RBF-GR for smooth parameter interpolation\n - 4D Gaussian Graph Convolutional Deformable module for spatiotemporal learning\n\n**Improved Interpretability**\n   - Uses explicit geometric principles: Gaussian means (μ), covariance matrices (Σ), deformation fields\n## Q2: Long optimization time and suitability for optimization-based approach\nA2:\nFollowing your Q4 suggestion, we've significantly reduced the algorithm's runtime.\n\nOur core components suit optimization methods because:\n\n**Temporal RBF-GR, 4D Gaussian Deformation**: Optimize meaningful mathematical parameters.\n\n**4D Neural Field**: Expresses highly complex non-linear functions through optimization.\n\n## Q3: Relationship to 3D Gaussians, splatting, and clustering\nA3:\nRelationship to 3DGS:\n - Uses 3D Gaussians, inspired by 3DGS concepts\n - Doesn't employ \"splatting\" techniques\n - Will reduce 3DGS discussion in the paper\n\nOur method uses EM-like clustering and GMM concepts, enhanced by DGCNN and self-attention for Gaussian deformation, focusing on point cloud interpolation. Detailed relationships will be clarified in the main text.\n\n## Q4: DGCNN training approach\nA4:\n - Moved Iterative Gaussian Cloud Soft Clustering to preprocessing stage\n - Froze weights of DGCNN and SelfAttention networks\n - Networks now act as fixed feature extractors\n\n**Performance Impact**\n   - Model with frozen weights performs comparably to original model\n\n| Methods               | Longdress |      | Loot   |      | Red&Black |      | Soldier |      | \n|-----------------------|-----------|------|--------|------|-----------|------|---------|------|\n|                       | CD        | EMD  | CD     | EMD  | CD        | EMD  | CD      | EMD  |\n| NeuralPCI             | 0.70      | 4.36 | 0.61   | 4.76 | 0.67      | 4.79 | 0.59    | 4.63 | \n| Ours                  | **0.68**  | **3.69** | **0.59** | 4.12 | **0.65** | **4.20** | **0.57** | 4.14 |\n| Ours (Freeze weight)  | 0.69      | 3.92 | **0.59** | **4.03** | 0.66    | 4.51 | **0.57** | **4.13** |\n\n**Efficiency Gains**\n   - Processing time for 4-frame sequence (8192 points) reduced to 0.1572 seconds.\n   - Iterative Gaussian Cloud Soft Clustering now takes about 0.2248 seconds in preprocessing\nFor detailed timing of individual model components, please refer to our response to Reviewer xq5Q's Q9.\n\n## Q5: 3D Gaussian correspondence during interpolation, and residual Gaussian visualization.\nA5:\nReason:\n1. Each Gaussian is uniquely identified and tracked across all timesteps.\n2. Gaussian parameters evolve smoothly over time via RBF interpolation.\n3. Self-attention mechanism ensures coherent global deformation.\n\nResidual Gaussian visualization provided in Figure 1 of response PDF.\n## Q6: Model efficiency analysis for more frames/points, and writing corrections\nA6:\n1. Detailed timing for each module provided in reviewer xq5Q's A9 for 1024 and 8192 points.\n2. For optimization time comparisons across models with varying frame counts and point cloud sizes:\n\n| Method        | 4 Frames |         |         | 6 Frames |         |         | 8 Frames |         |         |\n|:--------------|:--------:|:-------:|:-------:|:--------:|:-------:|:-------:|:--------:|:-------:|:-------:|\n|               | 1024 pts | 8192 pts| 16384 pts| 1024 pts | 8192 pts| 16384 pts| 1024 pts | 8192 pts| 16384 pts|\n| 3DSFLabelling | 0.1324   | 0.3620  | 0.7500  | 0.1986   | 0.5430  | 1.1250  | 0.2648   | 0.7240  | 1.5000  |\n| NeuralPC      | 0.0447   | 0.0861  | 0.1924  | 0.0671   | 0.1292  | 0.2886  | 0.0894   | 0.1722  | 0.3848  |\n| Ours          | 0.0753   | 0.1572  | 0.3948  | 0.1130   | 0.2358  | 0.5922  | 0.1506   | 0.3144  | 0.7896  |\n\nThe time consumption of our algorithm is within a reasonable range.\n\n3. Writing corrections: \"Table 4.2\" reference on line 136 and missing arrows for ACC_S and ACC_R in Table 3.\n## Q7: Framework simplification for improved efficiency\nA7: Framework simplified as described in A4, significantly improving efficiency.\n## Q8: Additional supervision for 3D Gaussian distributions\nA9:\nExplored two additional constraints:\n\n**Gaussian Distribution Consistency Constraint**: Similar to the Smoothness Loss in the paper, we enforced continuity and consistency between Gaussian distributions of adjacent time steps.\n\n**Local Anisotropy Constraint**: We quantified the similarity of local directional and structural features by comparing covariance matrices of corresponding local regions in target and predicted point clouds.\n\nResults:\n| Constraint | Dataset | CD | EMD |\n|------------|---------|----:|----:|\n| Baseline | DHB | 0.44 | 2.48 |\n| | NL-Drive | 0.78 | 95.95 |\n| Gaussian Consistency | DHB | 0.43 | 2.53 |\n| | NL-Drive | 0.79 | 95.06 |\n| Local Anisotropy | DHB | 0.44 | 2.39 |\n| | NL-Drive | 0.79 | 93.22 |\n\nPrecision remains nearly constant, but at the cost of increased computation time.\n\n## Q9: Evaluation on more challenging datasets\n\nA10:\n1. Evaluated on NL Drive dataset (KITTI, Argoverse, Nuscenes) in Table 2 of paper.\n2. Additional testing on Waymo dataset:\n\n| Method         |   CD |   EMD |\n|----------------|-----:|------:|\n| 3DSFLabelling  | 0.72 | 83.07 |\n| NeuralPC       | 0.51 | 60.94 |\n| Ours           | **0.48** | **50.71** |\n\nThese results further validate our model's effectiveness in complex, real-world autonomous driving scenarios.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6HNlqYrVmw", "reviewer": "Reviewer_Z54i", "summary": "The paper introduces NeuroGauss4D-PCI, a model designed to address the challenges of point cloud interpolation (PCI) by using Gaussian soft clustering and a 4D neural field to model complex non-rigid deformations in dynamic scenes. The model excels in capturing spatial and temporal dynamics from sparse data, showing superior performance on object-level and large-scale datasets.", "questions": "1. How does NeuroGauss4D-PCI ensure robustness against varied data sparsity in real-world scenarios?\n\n2. Can the model handle rapidly changing dynamic scenes, such as explosions or quick animal movements?\n\n3. What are the computational costs associated with training and inference, and how do they compare to simpler models?\n\n4. How does the model perform under conditions of significant occlusion or sensor noise?", "limitations": "1. The complex interactions between Gaussian fields and neural networks may hinder understanding and tuning of the model.\n\n2. The high computational demand may limit deployment in time-sensitive applications.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "It is interesting to utilize Gaussian clustering and 4D neural fields for solving PCI problems.", "weaknesses": "1. The color scheme of the figures in the article is very poor, making many details difficult to discern.\n\n2. Point cloud interpolation is a relatively niche direction in point cloud analysis. The author should more clearly emphasize the significance of studying this issue.\n\n3. The most related work reported in this work might be the CVPR 23 paper Neuralpci.  Does that mean no more recent related works?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their insightful questions and provide detailed responses below\n## Q1: Poor color scheme.\nA1:\nWe commit to redesigning the color scheme in our main text to improve visibility and readability. For your reference, we've implemented this new color scheme in our PDF response.\n## Q2: Emphasize the significance of research direction.\nA2: We will reemphasize the research significance of this task in our paper:\n1. Bridges the gap between current sensor limitations and high-frequency data demands in autonomous driving and AR.\n2. Improves object tracking and motion prediction in dynamic environments.\n3. Facilitates world model construction and AI-generated 3D content.\n4. Supports multi-sensor synchronization, auto-labeling, and point cloud densification.\n\nOur supplementary materials demonstrate these applications, highlighting PCI's versatility and potential across various 3D vision tasks.\n## Q3: Latest relevant works.\nA3:\nNeuralPCI (CVPR 2023) is the most recent specialized study on point cloud interpolation in our citations. While 3DSFLabelling (CVPR 2024) addresses related tasks, it focuses on 3D scene flow rather than sequence interpolation. Recent applications of point cloud interpolation include skeletal animation generation (PC-MRL, arXiv 2024) and optimal transport evaluation (CL+SW2, WACV 2024).\n## Q4: Ensuring robustness to different data sparsities.\nA4:\nNeuroGauss4D-PCI achieves robust performance across data sparsity levels due to:\n\n1. Adaptive Representation: Gaussian soft clustering extracts meaningful features even from sparse data.\n\n2. Flexible Temporal Modeling: RBF-GR module interpolates smoothly between frames, regardless of point density.\n\n3. Multi-scale Capture: 4D Gaussian fields model both global structure and local details, adapting to varying sparsity.\n\nThese components synergistically handle different data densities. Our algorithm's robustness across diverse point cloud densities is demonstrated by these results:\n\n| Point Cloud Size | NSFP | PV-RAFT | PointINet | NeuralPCI | 3DSFLabelling | Ours† |\n|:----------------:|:----:|:-------:|:---------:|:---------:|:------------:|:----:|\n|       1024       | 6.81 |  6.92   |   5.64    |   5.04    |     6.19     | **4.01** |\n|       2048       | 4.01 |  3.98   |   3.10    |   2.59    |     3.41     | **2.00** |\n|       4096       | 2.60 |  2.51   |   1.70    |   1.38    |     1.80     | **0.99** |\n|       8192       | 1.80 |  1.65   |   1.10    |   0.80    |     1.19     | **0.72** |\n|      16384       | 1.30 |  1.21   |   0.72    |   0.48    |     0.92     | **0.38** |\n\n## Q5: Handling rapidly changing dynamic scenes？\nA5:\nIn our response PDF (Fig. 2), we present visualizations of frame interpolation predictions for scenarios involving long intervals and rapid motion patterns. Our predicted results demonstrate strong alignment with ground truth.\n\n## Q6: Training and inference costs comparison.\nA6:\nOur model's computation costs for different point cloud sizes are detailed below:\n\n| Processing Step                            | 1024 points | 8192 points |\n|:-------------------------------------------|------------:|------------:|\n| **Single Frame**                           |             |             |\n| Time Encoding                              |      0.0003 |      0.0003 |\n| 4D Neural Field                            |      0.0007 |      0.0008 |\n| RBF-GR+4DGD                                |      0.0041 |      0.0034 |\n| Fast LG-fusion                             |      0.0004 |      0.0028 |\n| Prediction Head                            |      0.0005 |      0.0023 |\n| Loss Calculation                           |      0.0048 |      0.0022 |\n| **Total (Single Frame)**                   |  **0.0108** |  **0.0118** |\n| **Sequence (4 Frames)**                    |             |             |\n| Loss Backpropagation + Optimizer Update    |      0.0567 |      0.0590 |\n| **Total (One Sequence Iteration)**         |  **0.0753** |  **0.1572** |\n\nNote: The Iterative Gaussian Cloud Soft Clustering module runs only once in the preprocessing stage, taking about 0.2248 seconds, and is not included in the table.\n\nFor 8192 points, our model takes 0.1572 seconds per iteration, or about 157.2 seconds for 1000 iterations. Simpler models like NeuralPCI take about 60 seconds for 1000 iterations, while more complex models like 3DLSFLabelling take over 10 minutes for a single point cloud pair. Our algorithm's time consumption is within an acceptable range for optimization-based methods designed for accuracy.\n\n## Q7: Performance under occlusion or sensor noise?\nA7:\nFigures 2 and 3 in our response PDF demonstrate our model's visual results under significant occlusion and various noise conditions, respectively. The results show that our model maintains acceptable accuracy even in these challenging scenarios.\n## Q8: Complex interactions between modules.\nA8:\nOur experiments demonstrate that integrating Gaussian fields with neural network features consistently yields performance improvements, regardless of the specific fusion method employed:\n\n| Method        | DHB (×10⁻³)           | NL-Drive               |\n|:--------------|:----------------------|:-----------------------|\n|               | CD ↓    | EMD ↓      | CD ↓    | EMD ↓        |\n| Baseline      | 0.49    | 2.99       | 0.80    | 98.03        |\n| LG-Cat        | 0.42    | 2.69       | 0.79    | 97.36        |\n| Fast-LG-Fusion| 0.44    | 2.48       | 0.78    | 95.95        |\n\nThe combination provides:\nGaussian fields capture spatial structure and global information, while neural networks learn complex non-linear mappings and local features.\n\n## Q9: High computational cost limiting real-time applications.\nA9:\nAs shown in the runtime analysis in Answer 6, our algorithm's computational cost falls within an acceptable range. Our approach, similar to NeRF, is a coordinate-based, per-scene fitting method primarily designed for offline optimization of individual scenes.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uQrA3MR8db", "reviewer": "Reviewer_ddrR", "summary": "The paper presents a novel model, “NeuroGauss4D-PCI,” for point cloud frame interpolation, a popular but challenging 3D computer vision task in real-world scenarios such as Lidar point cloud densification. The model outperforms existing PCI methods on the most popular benchmark datasets, DHB and NL-Drive, showcasing scalability to tasks like auto-labeling and point cloud densification.", "questions": "I don’t have additional detailed questions for this paper, as I may not be good enough to understand all the details of the proposed algorithm.", "limitations": "I wonder if it is convincing that all are testing their work on only two open datasets used in this paper.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "-\tThe paper is well written and well-organized, a good review of related works supports the motivation, the experiment is well designed, and the results and additional information are satisfied. \n-\tAlthough I’m not an expert in developing such algorithms in detail, as a user from the application side, I can understand that using 4D Gaussian deformation fields and temporal radial basis function Gaussian residual modules to capture complex spatiotemporal dynamics is a novel approach.", "weaknesses": "-\tThere are no details of the experiment platform or discussion of the computational efficiency of the proposed method.  \n-\tThe potential of NeuroGauss4D-PCI seems significant. But as we all know, NeuralPCI was said to be the best model for PCI until now. A more detailed comparison with this model could be crucial in providing valuable insights and strengthening the paper's contribution to the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your positive feedback on our paper's organization, literature review, experimental design, and results. Your recognition of our novel approach using 4D Gaussian deformation fields and temporal radial basis function Gaussian residual modules to capture complex spatiotemporal dynamics is particularly encouraging. We are grateful for your detailed comments and will address each of your concerns in the following responses.\n\n## Q1: Discussion on computational efficiency\nA1: \n - Experimental Platform:\nOur algorithm was tested on a platform equipped with an NVIDIA RTX 3090 GPU.\n\n - Computational Efficiency:\nWe recorded detailed computational costs for different point cloud sizes:\n\nTable 1: Time consumption statistics (in seconds)\n| Processing Step                            | 1024 points | 8192 points |\n|:-------------------------------------------|------------:|------------:|\n| **Single Frame**                           |             |             |\n| Time Encoding                              |      0.0003 |      0.0003 |\n| 4D Neural Field                            |      0.0007 |      0.0008 |\n| RBF-GR+4DGD                                |      0.0041 |      0.0034 |\n| LG-fusion                                  |      0.0004 |      0.0028 |\n| Prediction Head                            |      0.0005 |      0.0023 |\n| Loss Calculation                           |      0.0048 |      0.0022 |\n| **Total (Single Frame)**                   |  **0.0108** |  **0.0118** |\n| **Sequence (4 Frames)**                    |             |             |\n| Loss Backpropagation + Optimizer Update    |      0.0567 |      0.0590 |\n| **Total (One Sequence Iteration)**         |  **0.0753** |  **0.1572** |\n\n - Efficiency Analysis\n1. Single frame processing: Only 0.0118 seconds for 8192 points, demonstrating high efficiency.\n2. Sequence processing: 0.1572 seconds for a 4-frame sequence (8192 points/frame), or about 157.2 seconds for 1000 iterations.\n3. Preprocessing optimization: Iterative Gaussian Cloud Soft Clustering module moved to preprocessing stage, running once per sequence (~1 second) instead of every iteration.\n4. Weight freezing: DGCNN and SelfAttention network weights frozen to reduce updatable parameters.\n\n## Q2: Detailed comparison with NeuralPCI.\nA2:\nWe appreciate the reviewer's valuable feedback and recognition of NeuroGauss4D-PCI's potential. As requested, we have expanded our comparison with NeuralPCI:\n\n1. Visual comparison in Figures 1, 6 and 7 in the original paper.\n2. Quantitative comparisons across various point cloud densities, frame intervals, datasets, and metrics are presented in Tables 1 and 2, and Figure 1 in the original paper.\n\nKey advantages of NeuroGauss4D-PCI over NeuralPCI include:\n\n1. Structured Representation: Our iterative Gaussian soft clustering module provides a structured temporal point cloud representation, better capturing geometric features and spatio-temporal dynamics. NeuralPCI, in contrast, directly inputs spatial and temporal coordinates to an MLP.\n\n2. Spatio-temporal Modeling: Our Temporal RBF-GR module and 4D Gaussian deformation fields enable more precise modeling of complex non-rigid deformations and non-linear trajectories, particularly beneficial for long time spans and complex dynamic scenes.\n\n3. Feature Fusion: Our fast latent-geometry fusion module adaptively combines implicit features from neural fields with explicit geometric features from Gaussian deformation fields, enhancing spatio-temporal correlation modeling.\n\nThese improvements collectively contribute to NeuroGauss4D-PCI's superior performance in various challenging scenarios.\n\n## Q3: Convincingness of testing on only two open datasets.\nA3:\nThank you for raising this important question. I understand your concerns about the potential limitations of testing only on two open datasets, as well as the reliability of the experimental design and results of all the work.\n\n1. Dataset Representativeness:\n   - DHB (object-level) and NL-Drive (large-scale autonomous driving) represent two major application scenarios in point cloud interpolation.\n   - NL-Drive combines KITTI, Argoverse, and Nuscenes datasets, offering diverse, real-world autonomous driving environments.\n   - These datasets cover a wide range from fine-grained object deformations to complex, large-scale scenes.\n\n2. Fair Comparison:\n   - We benchmark against state-of-the-art methods including IDEA-Net, PointINet, NSFP, PV-RAFT, NeuralPCI, and 3DSFLabelling.\n   - We use results provided by Zheng et al. in the NeuralPCI paper, validating their experimental outcomes for consistency.\n\n3. Open-source Verification:\n   - All compared algorithms are open-sourced, enhancing result credibility and reproducibility.\n   - We conducted independent experiments using NSFP and 3DSFLabelling's open-source code on the test set, further ensuring fair and accurate comparisons.\n\n4. Extensibility Validation:\n   - We demonstrate NeuroGauss4D-PCI's potential in related tasks such as auto-labeling and point cloud upsampling, showcasing its versatility and adaptability.\n\nThis comprehensive approach aims to establish the reliability and broad applicability of our method across diverse scenarios in point cloud interpolation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iEi5VnGzgI", "reviewer": "Reviewer_2Dkp", "summary": "Point Cloud Interpolation (PCI) is the task of predicting intermediate point cloud features from a sparser representation, often to construct point cloud representations for intermediate time frames. Contrary to interpolation in other fields, point clouds are largely unstructured and do not preserve consistent information across different sequence frames, and thus cannot be addressed adequately with classical interpolation functions such as linear interpolation. To properly represent point clouds in a latent space for learned approaches, a method that respects the volumetric nature of point distributions is crucial for minimising noisy artifacts caused by the unordered nature of point clouds.\n\nThis paper proposes three new components of PCI learning architectures that address this necessity. First, a Gaussian clustering function is used to group point sets into distinct regions, which are less prone to misrepresenting unintended variations in point cloud data, and facilitates a necessary structure for PCI. To my understanding, with the sparse latent representations, interpolation is achieved by first using Radial Basis Functions (RBF) to provide an initial smooth estimate of the interpolated Gaussian cluster sequence, followed by a learned graph-based network to determine a final interpolation based on the composition of RBF sequences from every cluster.", "questions": "1. Figure 2: Is it correct to call the output of the soft-clustering process \"spheres\"? The figure appears to describe \"ellipsoids\" rather than consistent radius spheres.\n2. Outlier removal in Section 4.2/Table 2 appears to improve the results of the method quite significantly. How does it affect other state-of-the-art methods?", "limitations": "Yes, limitations of the method are discussed briefly in the manuscript regarding its applicability in practical scenarios.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper is technically sound with strong results against state-of-the-art methods in PCI of dynamic environmental LiDAR scenes. The methodology presentation is largely clear and well structured. A detailed ablation study is provided to demonstrate the importance of each component for effective PCI based on standard Chamfer Distance and Earth Mover Distance metrics.", "weaknesses": "Section 1: Introduction\n- The introduction writing structure is quite unconventional, and repeats itself word for word in the contributions statement. I suggest the introduction of the proposed method be explained in more detail with regards to its overall architectural structure, to incorporate a consistent flow with the presentation of the method's components.\n- The mentioned challenges of PCI are sporadic and inconsistent with the way PCI is introduced as a research problem. The use of a mathematical formula to introduce PCI takes away from the intended motivation of PCI-centric research, and is therefore difficult to link to each challenge. e.g. If solving Eq.(1) is the goal, what does the representation of \"multiple unordered point clouds\" have to do with it? This is unclear with the current presentation.\n- Figure 1 is a mess of graphs and qualitative examples that belong as separate (sub)figures in the Experiments section. The introduction of Figure 1 in the beginning of the work is inappropriate. What is the expected ground truth for the right side figures? It is unclear what the right side figure is trying to demonstrate.\n\nSection 2: Related works\n- Like the introduction, the presented explanation is centred around explaining mathematical formulae, rather than intuitively presenting technical problems and expressing it as formulae if necessary. It is generally unnecessary for the formula to be understood in order to analyse the current state of research for each subsection. For this reason, by linking the motives of past research to solving an otherwise arbitrary formula, the flow of the related works section is unclear.\n- L81: \"for efficient:\" appears to be in error.\n\nSection 3: Methodology\n- 1), 2), 3) are not labelled in the diagram itself, and thus can be difficult to associate.\n- Figure 3 fails to visualise the purpose of each component, i.e. RBF Activator, Gaussian/Feature Interpolators, in the proposed method. It is generally expected that a flowchart is used to both illustrate the information flow of the system, and provide labels for the purpose, i.e. I/O, of each component. Mathematical equations are highly ineffective for this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your meaningful suggestions and questions, and we will address each of them in our response.\n## Q1.1: Non-traditional writing structure of the introduction.\nA1.1: We will rigorously revise our introduction structure according to your recommendations. Due to space constraints, please refer to the end of our response to A1 for the detailed revision plan.\n## Q1.2: Sporadic PCI challenges and unclear logical relationship to Formula 1.\nA2:\nWe appreciate your feedback and will clarify that \"multiple unordered point clouds\" refers to the inherently unordered nature of input point cloud signals. To enhance clarity and logical flow, we will separate the introduction of PCI concepts from the discussion of current challenges in the main text. Additionally, we will provide a detailed explanation for each term in Equation 1.\n## Q1.3: Inappropriate introduction of Figure 1 at the beginning.\nA3:\nWe greatly appreciate your valuable feedback on Figure 1. We agree that introducing detailed experimental results at the beginning of the paper may not be ideal.\n\nOur original intent was to quickly showcase our method's advantages, including:\n1. Interpolation accuracy across various frame intervals\n2. Performance with different data sparsity levels\n3. Effectiveness on both human body and autonomous driving datasets\n\nTo address your concern, we will:\n1. Split the existing Figure 1\n2. Move qualitative and quantitative results to the experimental section\n3. Ensure readers fully understand our method before encountering these results\n\nRegarding the right side of Figure 1, we will provide further explanations in our PDF response (Fig. 4).\n\nThe following is the revised Introduction outline, and we will re-revise the text strictly according to your suggestions:\n1. Point Cloud Interpolation (PCI)\n   - Concept and application scenarios, incorporating Equation 1\n   - Clear explanation of each term in Equation 1\n\n2. PCI Challenges and Research Significance\n   - Point cloud data characteristics and resulting challenges\n   - Complexity of spatio-temporal dynamic modeling\n   - Difficulty in generalizing from sparse temporal samples\n\n3. NeuroGauss4D-PCI: Key Components and Their Roles\n\n   a. Iterative Gaussian Cloud Soft Clustering\n      - Structured representation for unordered point clouds\n      - Geometric feature capture addressing data irregularity\n\n   b. Temporal RBF-GR Module\n      - Complex non-linear temporal dynamics modeling\n      - Smooth interpolation between sparse temporal samples\n\n   c. 4D Gaussian Deformation Field\n      - Long-term motion trend and non-rigid deformation capture\n      - Cross-frame geometric consistency maintenance\n\n   d. 4D Neural Field\n      - Fine-grained spatiotemporal detail representation\n      - Multi-scale dynamic modeling complement to Gaussian representation\n\n   e. Fast Latent-Geometric Feature Fusion Module\n      - Adaptive combination of Gaussian and neural features\n      - Handling of varying point densities and enhanced spatiotemporal modeling\n\n4. Innovations and Contributions\n   - Novel 4D representation combining Gaussian soft clustering and neural field features\n   - Continuous time modeling via Temporal RBF-GR module\n   - Advanced 4D Gaussian deformation field using graph convolutions\n\n## Q2: Unclear flow in the related works section\nA2: We will improve the related works section by:\n1. Reorganizing the structure of each subsection (neural fields, 3D Gaussian splatting, and point cloud interpolation)\n2. Reducing reliance on mathematical formulas\n3. Correcting grammatical errors\n## Q3.1: Unlabeled components in Figure 2\nA3.1: We will revise Figure 2's caption to match the module names in the diagram and avoid using numbering:\n\nFigure 2: NeuroGauss4D-PCI architecture. The system processes input point clouds (X, Y, Z, T) and a predicted time T_pred. Latent Feature Learning occurs via Fourier Basis Encoding and Neural Field. Gaussian Representation is achieved using Iterable Gaussian SoftClustering. Temporal Modeling employs the RBF-GR Module. The 4D Deformation Field integrates latent features and temporal information. Feature Fusion is performed by the Fast-LG-Fusion module. Finally, Point Cloud Generation is accomplished using a Prediction Head. This pipeline enables effective spatiotemporal modeling and interpolation of point clouds.\n## Q3.2: Figure 3 fails to visualize component purposes\nA3.2: We've enhanced Figure 1 in the PDF reply as follows:\n\n1. Gaussian Ellipsoid Visualization:\n   Added pre- and post-interpolation Gaussian ellipsoids, showing model inputs and outputs.\n\n2. Intermediate Process Visualization:\n   Included visualizations of intermediate products to illustrate data transformation.\n\n3. Component Functionality:\n   Clarified inputs, outputs, and functions of RBF-GR module components.\n\n4. Information Flow Diagram:\n   Will add a flow diagram in the final revision, annotating each component's role and data flow.\n## Q4: Incorrect terminology in Figure 2.\nA4: Using \"sphere\" to describe soft clustering output is indeed inaccurate.. We will:\n\n- Change \"Gaussian spheres\" to \"Gaussian ellipsoids\" in Figure 2's caption and explanation\n- Review and correct all related descriptions in the main text\n\n## Q5: Impact of outlier removal on other methods\n\nA5: Outlier removal improved results for all tested methods. Here's a comparison:\n\n| Methods   | CD ↓   | EMD ↓   |\n|-----------|--------|---------|\n| NSFP      | 1.75   | 132.13  |\n| NSFP†     | 1.72   | 129.05  |\n| NeuralPCI | 0.80   | 97.03   |\n| NeuralPCI† | 0.75  | 83.56   |\n| Ours      | 0.78   | 95.95   |\n| Ours†     | **0.72**   | **78.66**   |\n\nNote: † indicates results with outlier removal.\n\nNeuroGauss4D-PCI benefits most from outlier removal due to:\n1. Our Gaussian representation amplifies the benefits of outlier removal.\n2. Clean data significantly enhances our 4D deformation field's accuracy.\n3. Outlier removal maximizes our Temporal RBF-GR module's effectiveness.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the Area Chair for their time and effort in handling our paper, and all reviewers for their detailed and valuable suggestions, which are crucial for improving our work.

The reviewers have positively acknowledged our method's novelty, impact, accuracy, and potential, as highlighted below:

Our proposed methodology presents a novel approach to addressing the Point Cloud Interpolation (PCI) challenge, introducing 3D Gaussian representation for PCI (Reviewer **ZfCH**) and innovatively combining Gaussian clustering with 4D neural fields (Reviewer **Z54i**). The use of 4D Gaussian deformation fields and temporal radial basis function Gaussian residual modules is noted as an innovative method for capturing complex spatiotemporal dynamics (Reviewer **ddrR**). The method demonstrates excellent performance in non-rigid deformation and non-linear trajectory modeling (Reviewer **ZfCH**). The experimental design is considered comprehensive and reasonable, including evaluation on multiple indoor and outdoor datasets (Reviewers **ZfCH**, **2Dkp**) and detailed ablation studies (Reviewers ZfCH, 2Dkp). Results show our method outperforming existing approaches on standard evaluation metrics (Reviewers **xq5Q**, **2Dkp**). Additionally, the paper's organization, writing quality (Reviewers **ddrR**, **2Dkp**), and review of related work supporting the research motivation (Reviewer **ddrR**) are commended.

After careful analysis of all comments, the main issues can be summarized as:

1. Model structure complexity
2. Algorithmic efficiency of components
3. Model's adaptability to varying frame numbers and point cloud densities
4. Latest PCI research and its significance
5. Performance in complex scenarios (noise, rapid motion, occlusion)
6. Writing errors, structural clarity, and figure readability
7. Performance on challenging datasets and impact of outliers

We have meticulously revised the manuscript based on these valuable suggestions. Due to space constraints, we've provided concise responses here, with detailed illustrations and experimental results in the reviewer-specific replies.

Brief responses to the main issues:

1. While our model comprises multiple components, each addresses specific PCI challenges. We've clarified component relationships and demonstrated their effectiveness through ablation studies.
2. We provide detailed timing statistics for each component across various point cloud sizes.
3. Our model shows robust performance across different point cloud densities (1024 to 16384 points) and frame intervals, with comparative results demonstrating superior performance in various scenarios.
4. We've re-explained the latest PCI research and its specific significance.
5. Visualizations in our response demonstrate effective interpolation under occlusion, noise, and rapid motion conditions, maintaining good alignment with ground truth.
6. We will address all mentioned and potential writing and presentation issues, improve figure clarity, and resolve terminology inconsistencies to enhance overall readability and comprehension.
7. We've tested on challenging datasets like Waymo and provided comparative results. Outlier removal benefits all methods, with our approach showing the most significant improvements due to its Gaussian representation and temporal modeling capabilities.

We look forward to engaging in further discussions with you over the next few days. We are eager to address any remaining concerns you may have about our paper and are committed to providing comprehensive clarifications to ensure the highest quality of our research. 

Once again, we extend our sincere gratitude to the Area Chair and all reviewers for their invaluable input and dedication!

Meta-review: This paper proposes a novel method for point cloud interpolation using Gaussian soft clustering and a 4D neural field. The experimental results on multiple datasets, including both indoor and outdoor environments, demonstrate the effectiveness of the proposed method.

Four reviewers (2x BA, 1x WA, 1x SA) acknowledged the strengths of this paper, and the authors' rebuttal successfully addressed most of the concerns raised by these reviewers.

Reviewer Z54i, however, provided a negative rating of Borderline Reject, concerning issues with "the poor color scheme of the figures," "the significance of the research direction," and "robustness to more challenging scenes." The authors responded to these concerns, but no further feedback was received from the reviewer.

After reviewing the paper, the reviews, and the rebuttal discussion, the AC has decided to recommend acceptance. For the camera-ready version, please improve the color scheme in the figures and revise the writing to enhance visibility and readability. Additionally, include the new experiments and discussions to further strengthen the manuscript.
Decision: Accept (poster)