Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction
Authors: Jiacong Chen
Qingyu Mao
Youneng Bao
Xiandong MENG
Fanyang Meng
Ronggang Wang
Yongsheng Liang
Venue: NeurIPS 2025 poster ()
Keywords: Free-viewpoint video
3D Gaussian Splatting
Real-time transmission
4D reconstruction
Compression
Streaming
Primary area: applications
TL;DR: A storage-efficient solution for online Free-viewpoint video recosntruction using 3D Gaussian Splatting.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a high-fidelity and efficient paradigm for online free-viewpoint video (FVV) reconstruction, offering viewers rapid responsiveness and immersive experiences. However, existing online methods face challenge in prohibitive storage requirements primarily due to point-wise modeling that fails to exploit the motion properties. To address this limitation, we propose a novel Compact Gaussian Streaming (ComGS) framework, leveraging the locality and consistency of motion in dynamic scene, that models object-consistent Gaussian point motion through keypoint-driven motion representation. By transmitting only the keypoint attributes, this framework provides a more storage-efficient solution. Specifically, we first identify a sparse set of motion-sensitive keypoints localized within motion regions using a viewspace gradient difference strategy. Equipped with these keypoints, we propose an adaptive motion-driven mechanism that predicts a spatial influence field for propagating keypoint motion to neighboring Gaussian points with similar motion. Moreover, ComGS adopts an error-aware correction strategy for key frame reconstruction that selectively refines erroneous regions and mitigates error accumulation without unnecessary overhead. Overall, ComGS achieves a remarkable storage reduction of over 159 × compared to 3DGStream and 14 × compared to the SOTA method QUEEN, while maintaining competitive visual fidelity and rendering speed. Project page: https://chenjiacong-1005.github.io/ComGS/.

Motion Matters: Compact Gaussian Streaming for Free- Viewpoint Video Reconstruction
Jiacong Chen 1.2
Qingyu Mao Youneng Bao* Xiandong Meng
Fanyang Meng
Ronggang Wang Yongsheng Liang!
'College of Applied Technology, Shenzhen University, Shenzhen, China Coollege of Big Data and Internet, Shenzhen Technology University, Shenzhen, China
College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China *Department of Computer Science, City University of Hong Kong, Hong Kong, China Pengghhng Laboratory, Shenzhen, China "chooo of Electronic and Computer Engineering, Peking University, Shenzhen, China
Ours
QUEENA
PSNR:333.010
PSNR: 33.61
Cankil-
Starge: 7, F.0MB//mmme
Storag::
0.049M M.4MB.2rmme
QUEINa
IDGS
PONK (OLD)
K.Hax
elbpotas
Bater
159K smaller,
Hyperheat
improved
Starkk
fidelity
Contiti
per
Figure 1: Left: Experimental results N3DV dataset []] showcase the effectiveness of method, which reduces the storage requirement of 3DGStream [2] by 159 x, with enhanced visual quality. Right: Comparison with existing methods in storage and reconstruction fidelity. Hollow circles denote offline methods, while solid circles represent online methods.
Abstract
3D Gaussian Splatting (3DGS) has emerged as a high-fidelity and efficient paradigm for online free-viewpoint video (FVV) reconstruction, offering viewers rapid responsiveness and immersive experiences. However, existing online meth- ods face challenge in prohibitive storage requirements primarily due to point-wise modeling that fails to exploit the motion properties. To address this limitation, we propose novel Compact Gaussian Streaming (ComGS) framework, leveraging the locality and consistency motion dynamic scene, that models object-consistent Gaussian point motion through keypoint-driven motion representation. By transmit- ting only the keypoint attributes, this framework provides a more storage-efficient solution. Specifically, we first identify sparse set of motion-sensitio keypoints localized within motion regions using a viewspace gradient difference strategy. Equipped with these keypoints, nism that predicts a spatial influence field for propagating kcypoint motion to neighboring Gaussian points with similar motion. Moreover, ComGS adopts error-aware correction strategy for key frame reconstruction that selectively re- fines erroneous regions and mitigates overhead. Overall, ComGS achieves a remarkable storage reduction of over 159 compared 3DGStream and 14 compared the SOTA method QUEEN, while maintaining competitive visual fidelity and rendering speed. Project page: https://chenjjaccong-- 1005. hwt.05.iituu.io//cooo iomoGGS//
propose an adaptive motion-driven mecha-
accumulation without unnecessary
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Reconstructing free-viewpoint video (FVV) from multi-view videos captured by cameras with known poses has attracted growing interest in the field of computer vision and graphics. FVV exhibits great potential as generatiertn visual medium that enables immersive and interactive experiences, with broad application in virtual and augmented reality (VR/AR) applications [3.
Recently, 3D Gaussian Splatting (3DGS) has become promising method for FVV reconstruction, due to its significant advancements in real-time rendering and high-fidelity view synthesis. These approaches typically fall into two categories: 1) incorporating temporal function into Gaussian primitives and optimizing directly [4H6], and 2) applying deformation field to capture the spatio- temporal transformations of canonical Gaussians [7H1]. While these FVV reconstructions accurately represent dynamic scenes, they are trained in an offline manner and require transmitting the full set of reconstructed parameters prior to rendering.
In contrast, by enabling per-frame training and progressive transmission, online FVV reconstruction allows immediate playback without the overhead of full-scene preloading. As a pioneer work, 3DGStream extends 3DGS to online FVV reconstruction using InstantNGP 12] to model the geometric transformation trame-by-frame While achieving impressive rendering speed, its structural constraint hinders the volumetric representation performance and degrades the visual quality. Building on this paradigm, subsequent works [13 44 enhance model expressiveness through explicitly optimizing Gaussian attribute residuals, achieving competitive synthesis quality and higher robustness. However, the storage demands of these methods remain prohibitively high for real-time transmission, with reconstructed data typically exceeding 20 MB per second.
In this paper, we aim to design a storage- -ffiggeefficient solution for FVV streaming that minimizes bandwidth requirements and enables real-time transmission. In online FVV reconstruction, since dynamic scenes contain a large proportion of static regions, the key to efficient reconstruction lies in motion modeling. Our first insight, therefore, is to only model the Gaussian attribute residuals in the motion regions, which eliminates the unnecessary updates in static regions. Building on motion modeling, we note that scene motion tends to be consistent, where Gaussian points associated with the same object typically exhibit the same similar motion in dynamic scene representation. Our second insight, based on this observation, is to use a shared motion representation to model the attribute residuals with similar motion. This contrasts with existing online methods [2] [3] that utilize point-wise strategy to update the attribute residuals in motion regions, and the result motion redundancy climination and more compact storage. Lastly, we exploit a key frame fine-tune strategy to handle the error accumulation brought by non-rigid motion and novel objects emergence.
Specifically, to accomplish this, we propose Compact Gaussian Streaming (ComGS) framework that leverages set of keypoints (= 200), significantly fewer than the full set of Gaussian points (< 200K), to holistically model motion regions at each timestep. ComGS begins with a motion- sensitive keypoints selection through a viewspace gradient difference strategy. This ensures that the selected keypoints are accurately positioned within motion regions and prevents redundant or incorrect modeling of static areas. Subsequently, we design an adaptive motion-driven mechanism that defines a keypoint-specifis spatial influence field, with which neighboring Gaussian points can share the motion of the kcypoint. Unlike conventional k-nearest neighbor (KNN) methods [15) (6)) the spatial influence field can accommodate the complexity and variability of motion structure in dynamic scenes, so that keypoints can more accurately drive the motion of the surrounding region. Finally, to mitigate error accumulation in compact and effective manner, propose error-aware correction strategy for key frame reconstruction that selectively updates only those Gaussians with reconstruction errors.
Our major contributions can be summarized as follow:
We introduce a motion-sensitiin keypoint selection to accurately identify keypoints within motion regions, and an adaptive motion-driven mechanism that effectively propagates motion to neighboring points. These leverage the locality and consistency of more storage-efficient solution for online FVV reconstruction.
motion and achieve a
We propose error-aware correction strategy for key frame reconstruction that mitigates error accumulation over time by selectively updating Gaussian points with reconstruction errors, which ensures long-term consistency and minimizes redundant correction.

---

Experiments on two benchmark datasets show that the effectiveness of our method and its individual components. Our method achieves a compression ratio of 159x over the 3DGStream and 14x over state-of-the-arr model QUEEN, enabling real-time transmission while preserving competitive reconstruction quality and rendering speed.
2 Related work
2.1 Dynamic Gaussian Splatting
Recently, 3D Gaussian Splatting (3DGS) T7 22 has attracted great attention in Free- viewpontt video (FVV) reconstruction for its high photorealistic performance and real-time rendering speed. expand temporal variation as function and optimize directly for modeling Gaussian attributes across frames. For instance, 4D Gaussian Splatting [4] incorporates time-conditioned 3D Gaussians and auxiliary components into 4D Gaussians, while ST-GS [6] models the transformation of structural attributes and opacity as a temporal function to represent scene motions. To support long FVVs representation, TGH [23) introduces a multi-level hierarchy of 4D Gaussian primitives that exploits various degrees of temporal redundancy in dynamic scenes. While these time variant-based methods achieve superior rendering efficiency, they often suffer from prohibitive storage requirements. Other works [8] 25 27 employ vanilla 3D Gaussians as a canonical space and a deformation field to represent the dynamic scene. In this category, 4D- GS g utilizes hexplanes 2 six orthogonal planes, as latent embeddings and deliver them into small MI.P to deform temporal transformation of Gaussian points, achieving efficient computational complexity and lightweight storage requirement. Building upon this, GD-GS [11] further improves scene modeling accuracy by incorporating geometric priors, which provides a more structured and precise representation of dynamic scene. Among them, both SC-GS (15] and SP-GS (16) adopt sparse control points to control scene motion using a k-nearest neighbor (KNN) 29 strategy for motion modeling. While these methods achieve notable improvements in computational efficiency and rendering speed, they are designed for offline FVV reconstruction and do not support frame-by-frame delivering. Additionally, motion-insensinn modeling lead to redundant representation of static regions and reduced deformation accuracy in dynamic scenes. Our online method addresses these limitations by selecting keypoints from motion regions at each timestep and modeling motion with awareness of local motion scales, which enables more accurate and efficient modeling of online FVV.
Several works 6 23 24
control point selection and agale-agnostic KNN motion
2.2 Online Free-Vies Pre--Viepoinnt Video Reconstruction
Compared to the offline methods, online reconstruction enables FVV to be incrementally trained and transmitted per-frame manner, which allows users to preview or interact immediately with the video content. Leveraging the high-fidelity view synthesis capabilities of Neural Radiance Field (NeRF) 122 36) a set of studies have explored NeRF-based methods (37H41) for online FVV reconstruction, such as StreamRF (37). VideoRF [38 and TeTriRF [40]. Despite advanced visual quality, NeRF- based methods are hindered by their limited rendering speeding of implicit structure, which limits their practical applications.
With the utilization of 3DGS [77, 3DGStream introduces a hash-based MLP encode the position and rotation transformation of Gaussian points at each frame, and designs an adaptive Gaussians addition strategy for novel objects across frames. Based this paradigm, QUEEN [13 proposes a Gaussian residual-based framework for model expressiveness enhancement and learned quantization- -pparssty framework for residuals compression. HiCoM [14] designs a hierarchical coherent motion mechanism to effectively capture and represent scene motion for fast and accurate training. To deploy into mobile device, V 42) presents a novel approach that compresses Gaussian attributes as a 2D video to facilitate hardware video codecs. IGS (43 proposes generalized anchor- driven Gaussian motion network that learns residuals with a singe step, achieving a significant improvement of training speed. Nevertheless, these methods face challenge in real-time transmission, due to their substantial storage requirements. This overhead mainly stems from redundant updates of static Gaussian points across frames, as well as repeated modeling of Gaussian points with similar motion. Our study exploits the locality and consistency of motion by leveraging notion--enniiive keypoints to adaptively drive motion regions, and this avoids redundant storage and transmission.
3

---

Non- key frame reconstruction
Key fimme
Dynamic scores
Mgg
Vg Mpp
Aqp
Keypoiat
Top values
No Neighboring point
Spotial intluenno nell
Vissspaca Grsdiat Difference
Non-key frame
b1 Gasssiess
Hlanet
Keypoints
Scene
Motion-Sensitive Keypoint Selection
(c) Adaplive Molion- Driven Mechanism
Non-key frame
Key frame reconstruction
Vi +11 exror G aussi
Gaussians
Kay Frome
Error- awkame gate
Gaussians
EGaussiars
(a) Group of Frames
(d) Error-A ware Correction
Figure 2: The overall pipeline of ComGS framework. (a) The reconstruction process starts from the first frame initialized using vanilla 3DGS [T7] Subsequent frames are organized into groups of frames (GoFs). For non-key frames, (b) we begins with a motion-sensitive keypoint selection using a viewspace gradient difference strategy, (c) and utilizes an adaptive motion-driven mechanism to control neighboring points motion. For key frames, (d) an error-aware correction strategy is introduced to mitigate the error accumulation across frames.
2.3 3D Gaussian Representation Compression
Despite 3DGS-based methods achieve impressive performance in novel view synthesis 17 the massive size of Gaussian points hinder them for efficient storage and transmission. Several studies propose variety of compression techniques for reducing the required storage, which can be categorized into either post-processing- based 119 21
45 B
or neural contextual coding- based
Post-processing-bale based approaches include removing unimportant Gaussian pruning spherical harmonic coefficients [9 21, and applying vector quantization [I3 to compress the parameter representation. The latter methods utilize sophisticated entropy modeling to accurately predict probability distributions that exploit global context for compressing 3D Gaussian representation more effectively. In this paper, focus leveraging the locality and consistency motion in dynamic scene and mitigating the redundancy reconstruction on static and similar motion regions, introducing novel and more compact method for online FVV reconstruction.
methods 10 46 49]
points
45
3
Methods
Our goal is to reconstruct and transmit FVV in storage-efficient and streaming manner. To achieve it, we propose a Compact Gaussian Streaming (ComGS) framework for online FVV reconstruction, illustrated in Fig. First, ComGS begins with motion-sensitivity keypoint selection using viewspace gradient difference, ensuring subsequent motion control learning (Sec. (.2). Second, develop an adaptive motion-driven mechanism that applies spatial influence field control neighboring point motion (Sec. 3.3). Third, we devise an error-aware correction strategy for key frame reconstruction mitigate error accumulation brought by non-rigid motion and novel objects emergence in online reconstruction (Sec. 3.4). Finally, we introduce our compression techniques and optimization process in Sec. 3.5
3.1 Preliminary
3DGS 17 models 3D scene as a large amount of anisotropic 3D Gaussian points in world space as an explicit representation. The central position and geometric shape of each Gaussian point in world space defined by mean vector μ4, and covariance matrix Σ, mathematically represented
Gx = exp( )(x (x- p.) -.) p4))
(1)

---

For differentiable optimization, the covariance matrix Σ; is decoupled into scaling matrix S and rotation matrix R. Each Gaussian point characterized by its color e; and opacity og, For novel view synthesis, the covariance matrix Σ in camera coordinate given as:
Σ JWE, W"
where J is the Jacobian of the affine approximation of the perspective projection and W represents the view transformation matrix mapping world coordinates.
During rendering, the Gaussian points are initially projected into viewing plane, and the final color C can be obtained by a-blending of the N ordered 3D Gaussian points overlapping the pixel as:
C -= c
(3)
3=1
where represents the blending weight of the jth Gaussian point.
3.2 Motion-Sensitive Keypoint Selection
Establishing effective keypoint-driven motion representation necessitates to select appropriate keypoints. Considering motion locality, keypoints should be located in motion regions, which avoids redundant modeling in static areas and enables accurate modeling of complex motions
Thus, inspired by 13], we propose a motion-sensitive keypoint selection based on viewspace gradient difference (Fig. (b)). The core idea is to identify the dynamic Gaussian by the gradient change of rendering loss in inter-frames, and based on the gradient values, the & Gaussian points with the largest gradient are selected keypoints. Specifically, following the gradient computation in 3DGS [17),
compute gradients using the previous Gaussian positions Pt-1, the rendered images f, the reconstruction loss Crecon, and the ground-truth images 4--1 and 1;:
9--1 Úp:-1
Cl = Crecon (fo- I(-1)
(4)
9: = 0pi-1
Creco =
Lrecoo (1-1: I()
(5)
Dynamic significance scores A9, E RN (N is the number of Gaussians) were calculated by means of absolute values of gradient differences:
AG. Σ 1 -((1)
where V the number of the training viewpoints. Finally, we select the top k high dynamic significance scores from all Gaussian points as keypoints K; at timestamp t. Selecting the top-k Gaussian points with the highest dynamic scores not only identifies those located in motion regions, but also naturally allocates more keypoints to the areas with complex motion, facilitating more accurate modeling of such regions.
In this paper, for a balance of training efficiency and reconstructed quality, we set k = 200.
3.3 Adaptive Motion-Driven Mechanism
Equipped with the selected keypoints Kr, at current timestep, the next step is to determine which neighboring points are controlled by these keypoints, and apply their transformations to drive the motion of the controlled neighboring points. Previous works [15 [6] employ k-nearest neighbor (KNN) [29 search to predict the motion of each Gaussian points, showing advanced results in monocular synthetic video reconstruction, but they do not fully consider unnecessary modeling in static region and motion scale difference, which leads to computational redundancy and inaccurate representation.
In contrast, we propose an adaptive motion-driven mechanism that enables each keypoint to drive neighboring points through spatial influence field, illustrated Fig. (c). Specifically, motivated
5

---

by [17], for each keypoints K2 at t timestep, we initialize a quaternion Padop E R and scaling vector s Fadop E R to compute the spatial influence field Σ Fadap R For neighboring Gaussian
point Gj with position μj, its distance to keypoint KC; is given by dı = Hj - Mkç. The influence weight is then computed as:
1 Wij exp 1 ddds))
If way exceeds predefined threshold Tadap, the Gaussian G3 is considered to be controlled by keypoint K²::
C = Waj 2 Tadap)
(8)
where C₁ denotes the set of Gaussian points controlled by keypoint XCj.
To model motion, each keypoint Kj is further assigned a learnable translation offset Apk; E R
and rotation represented by quaternion Aqc; E R... For Gaussian G5 controlled by multiple keypoints (Ki),ezz' its overall motion is computed by aggregating the motions of its associated keypoints, weighted by their influence scores Wiy'
Wij
Agd-
Σ iEZ?
Wij Agc;
BEZ
where And and Agl indicate the transformation of Gaussian j at timestep, and Z represents the set of keypoints that control the motion of Gaussian j.
By leveraging compact set of keypoints with spatial influence fields, method enables accurate and efficient control of Gaussian motions at each frame. Since Gaussians share motion attributes through keypoints, only 14 parameters per keypoint are required, significantly reducing storage demands and mitigating data redundancy.
3.4 Error-Aware Corrector
By using keypoints to drive scene motion, we model the transformation of Gaussian points from the previous frame to the current frame with an extremely compact parameters. Nevertheless, keypoint- based motion controlling only supports to represent rigid motion effectively and faces challenge to handle non-rigid motion and novel objects emergence, which results error accumulation across frames.
straightforward solution mitigate error accumulation and ensure accurate long-term FVV representations is to separate the video into frame groups and update the attributes of all Gaussian points key frames. However, this strategy would lead to substantial of unnecessary parameters updating, since most of parameters are already correctly representing the scene and do not require modification. To mitigate error accumulation aware corrector strategy that only finetunes the Gaussian points with detected errors, significantly decreasing storage demands and promoting more accurate scene reconstruction, as illustrated in Fig. (d).
compact and efficient manner, we propose an error-
Specifically, given a video sequence, we select a key frame every s frames, forming the key frame sequence {fe, fe....., }. as shown in Fig. (a). Note that in this paper, key frames are used for error correction, and only the first frame of The video sequence is independently reconstructed. The remaining frames are reconstructed by keypoints driven. During key frame reconstruction, given the attributes of a Gaussian point at previous timestep 0: (μ; --1 f , -f-)), we introduce of learnable parameters A0, to model the attribute residuals. To identify which Gaussian points require correction, we predict learnable mask m, for each point. A sigmoid function used map m, the range (0, 1), which refers as soft mask:
m zoft Sigmoid(mm). m; moft E (0, 1)
(10)
Similar to 20 (21) the soft mask is binarized into hard mask using a predefined threshold @shress where the non-differentiable binarization is handled with the straight-through estimator (STE) to enable gradient flow, represented
dehres) - mofft) + mi ., hard sg( (m, soft m
soft hard E {0,1) (1I)

---

Table 1: Quantitative comparisons Neural 3D Video (N3DV)
and MeetRoom datasets (37
PSNR
Storage (MB)
Training
Rendering
Dataset
Category
Method
SSIM
LPIPS
(dB) 1
(sec)
(FPS) 1
NeRFPlayer
41
30.69
0,932
0.209
17.10
72
0.05
RyperReel
52
31.10
0,928
1.20
2.00
Offline
4D-GS g
31.15
0,964
0.149
0.13
34
SpaceTime
32.05
0.948
0.67
StreamRF 337
30.68
31.4
15
8.3
N3DV
TeTriRF
40
30.43
0.906
0.248
0.06
39
4
3DGStream
31.67
0.941
0.140
7.80
8.5
261
Online
QUEEN-s 13
31.89
0.945
0.139
0.68
4.65
345
QUEEN-I 13
32.19
0.946
0.136
0.75
7.9
248
ComGS-s (ours)
31.87
0.943
0.132
0.049
37
91
ComGS-I (ours)
32.12
0.945
0.129
0.106
43
147
I-NGP 559
28.10
48.2
66
4.1
Static
3DG-S 17
31.31
21.1
156
571
MeetRoom
StreamRF 37
26.72
9.0
10.2
10
Online
3DGStream
30.79
0.950
0.188
4.1
4.9
350
QUEEN-S 13
31.14
0.954
0.173
0.45
3.8
421
ComGS-s (ours)
31.49
0.955
0.171
0.028
28.3
98
where 1 is the indicator function and sy indicates the stop gradient operation. Then, the maard is applied to the attribute residuals before rendering. followed as:
8 = 0 + m; hard Ad
(12)
Meanwhile, we define a optimized function regulate the perceptual error while encouraging sparse residual updates:
Lerror
1 N
Σ
soft m
(13)
where N is the number of all Gaussian points. After optimization for the current key frame, only the tribute residuals A@t = [1f maaadd = 1] and the hard mask set M hard {mfarddj 1, 2,2,,.., ...; N) need to be stored and transmitted, minimizing the required data redundancy and transmission over- head.
3.5 Optimization and Compression
For the first frame optimization, we employ COLMAP [50] to generate the initial point cloud and follow the pipeline of 3DGStream (2]. The optimization for both the first frame and non-key frames
supervised by the reconstruction loss Lrecon, which is composed by an Tn-norm loss Ln and
D-SSIM loss LD- SSIM 551:
Gracm =
+ AD-SSSMMD--T SSIM
(14)
For key frame optimization, we minimize combined loss consisting of Lyacon and Lerron:
Lratat Lreem Carror
(15)
where Aerror controls the degree of error awareness, thereby balancing reconstruction quality and memory efficiency. We set AD-SSTM = 0.2 and Xerror 0.001 in this paper.
After optimization, the initialized Gaussians 8 and the residuals A for key frame error correction further compressed through quantization and entropy coding, enabling compact storage without performance degradation. More details are provided in the Appendix.
Experiments
4.1 Experimental Setup
We evaluate our method on two widely-used public benchmark datasets. (1) Neural 3D Video
(N3DV) [ consists of six indoor video sequences captured by 18 to 21 viewpoints. (2) Meet
7

---

Table 2:
Quantitative comparisons the long video sequence flame salmon from the N3DV
dataset 1
Method
PSNR (dB)
SSIM
LPIPS
Storage (MB)
Training (sec) |
Rendering (FPS)
E-NeRF
23.48
0.692
13.8
4DGS
28.89
0.952
0.197
31.2
90
TGH
29.44
0.214
0.075
6.3
550
ComGS-s (ours)
29.56
0.053
45.4
StreamRF
3DGStream
Ours
GT
3DGStram
Ours
GT
Figure 3: Quantitative comparison. We visualize our method and other online FVV methods on N3DV and MeetRoom 37 dataset.
Room 337 comprises four indoor scenes recorded with cameras multi-view system. In both of two datasets, we employ the first view for testing. Our method is implemented on an NVIDIA A100 GPU. We train 150 epochs for non-key frames reconstruction and 1000 epochs for key frames fine-tuning, We measure the visual quality of rendered images by average PSNR, required storage, rendering FPS and training time. More implement details are provided in the Appendix.
4.2
Quantitative Comparisons
conduct quantitative comparisons on existing online methods including StreamRF TeTriRF [40]. 3DGStream !2] and QUEEN [33, as well as the SOTA offline FVV approaches
37 6
41 [2) on N3DV and Mcetroom (Tab. D. Our method is evaluated in two variants: ComGS-s (small)
and ComGS-1 (large), using key frame intervals of s=10 and s=2, respectively.
Tab. shows that our ComGS achieves competitive results among existing online FVV methods on N3DV dataset. Notably, ComGS-s achieves substantial reduction in storage by 159x compared to 3DGStream and 14x compared QUEEN. This advantage enables real-time transmission in limited bandwidth and enhances the overall user viewing experience. On MeetRoom dataset, our method outperforms 3DGStream, obtaining +0.7dB PSNR and 146 x smaller size. Our advantages are mainly due to two factor: 1) using keypoint as shared representation requires transmitting only a small number of keypoint attributes; and 2) the error-aware correction module effectively rectifies regions with scene inaccuracies using minimal additional parameters. In the Appendix, the quantitative results are provided for each scene to offer a more detailed comparison.
We further evaluate the effectiveness of ComGS on handling long videos. We compare our method with the TGH (23 (which is the most recently proposed for handling long video sequences) on the Flame Salmon sequence (1200 frames) from the N3DV dataset [I] Tab. showw that our method achieves competitive results on rendering quality and required storage.
4.3 Qualitative Comparisons
As shown Fig. we compare our reconstructed results to other online FVV methods on N3DV and MeetRoom. ComGS effectively reconstructs both motion and static regions and provides more closer results to the ground truth. Fig- sssows that 3DGStream introduces noticeable artifacts due to its global update of Gaussian points across the entire scene, which often leads to incorrect updates

---

Table 3: Ablation study proposed components. Flame Steak and Flame Salmon scenes from the N3DV dataset.
Flame Steak
Flame Salmon
Experiments
Selection
Adaptive
Correction
PSNR (dB)t
Storage (KB))
PSNR (dBy)
Storage (KB)L
33.27
46.7
29.22
56.7
X
32.82
36.4
28,96
45.7
31.26
37.9
27.75
46.4
V
V
31.67
26.9
28.74
26.9
5
V
33.49
46.5
29.32
53.4
Figure 4: Visualization of our keypoint-driven motion representation. Top: selected keypoints are concentrated in motion regions. Bottom: adaptive control of neighboring points also focuses areas, enabling accurate and efficient motion modeling.
in static regions. In contrast, our method restricts modeling motion regions and applies targeted corrections in error- prror-prone areas, resulting in more accurate and robust scene reconstruction. More qualitative results are offered in Appendix.
4.4 Ablation Study
To validate the effectiveness of our proposed methods, we ablate three components of ComGS framework in Tab. B
In the Experiment 1, we ablate the motion- keypoint selection and instead select keypoints randomly. Since the random selection is not guided by motion regions, it may result in ineffective modeling of static areas and inadequate representation on motion regions (Fig. (b)), which leads to slight degradation in PSNR. Experiment 2 removes the adaptive motion-driven mechanism and models scene motion only using the selected keypoints, without incorporating any neighboring points. The resulting drop in reconstruction quality demonstrates that effective motion modeling relies not only on accurately keypoint selection, but also on the selection of their neighboring points. In the Experiment 3, we reconstruct FVV only relying frame correction, without modeling non-key frames by keypoint reconstruction, which results in significant performance drop. We emphasize that although the parameters of keypoints few, the seypoint-based modeling plays a crucial role in FVV reconstruction. Experiment 4 ablates the error-aware correction in key frame reconstruction. The performance degradation demonstrates that the error-aware correction in key frames would solve the error accumulation across frames.
the first frame reconstruction and key
Table 4: Ablation study on comparing control strategies for neighboring points.
Table Ablation study of the error-aware correc-
tion strategy.
Control tech
PSNR (dB)
Storage (KB)
Configuration
PSNR (dB)
Storage (KB)
KNN Adaptive
31.39
44.1
w/o error-aware
31.65
373
31.87
49.0
with arror-aaare
31.87
49.0
To further investigate the role of keypoint-driven motion representation, we visualize the selection and driven process in Fig. 4 The top shows that keypoints are predominantly selected in motion

---

(a) Farthest keypoint selection
(b) Random keypoint selection
(c) Ous
Figure 5: Visualization of different selection methods and corresponding updated regions.
12
19801
(8)
(d)
Figure 6: (a) PSNR comparison over time. Visualizations of (b) w/o key frames correction. (c)
ComGS-s. (d) ComGS-L.
regions, such the human body and moving objects. The bottom row highlights the adaptively controlled areas for neighboring points, which similarly focus on regions with significant motion (e.g., the person and the dog). Fig. [ visualizes Gaussians updated region using farthest keypoint selection [16,, random keypoint selection and our method, respectively, which demonstrates that our method accurately captures motion-intensive areas. These results indicate that ComGS can effectively leverage the locality and consistency of scene motion.
We also evaluate a KNN-based method [29] for selecting neighboring points around keypoints (Tab. 4. This approach shows inferior performance, as it does not distinguish between static and motion regions, leading to redundant modeling and poor adaptation to varying motion scales.
Fig. Bevaluates the effect of key frame correction. The visual results Fig. 6 (b-d) further highlight that key frame correction significantly reduces artifacts in motion regions such as flames, helping to maintain finer temporal consistency throughout the sequence. Tab. JJsoss that correction without error-aware leads to significantly higher storage due to redundant Gaussians updating. Moreover, without focusing on high-error regions, updates may affect error-free areas and result in suboptimal performance. Therefore, enabling error-awareness improves both accuracy and efficiency.
5 Conclusion
In this paper, we proposed ComGS, a storage-efficient framework for online FVV real-time transmis- sion. We utilized a keypoint-driven motion representation to models scene motion by leveraging the locality and consistency of motion. This approach significantly reduces storage requirements through keypoint selection and an adaptive motion driven mechanism. To address error accumulation over time, we further introduce an error-aware correction strategy that mitigates these error in an efficient manner. Experiments demonstrate the surpassing storage efficiency, competitive visual fidelity and rendering speed of our method.
Notably, our method still remains few limitations. First, as the first frame serves as the foundation for subsequent frame updates, poor initialization would lead to error propagation and degraded performance. Developing robust and efficient initialized strategy for first frame could further improve the visual quality and storage efficiency of online FVV. Second, our method relies on the dense view videos as inputs, which is expensive for practical applications. Future work will explore extending the framework to sparse-view or monocular inputs for real-worke scenarios. Additionally, this method does not fully consider the training time in the encoding stage, leaving room for further improvements in training efficiency. In future work, we aim to design practical solution on novel applications, such as 3D video conference and volumetric live streaming, providing viewers with immersive and interactive experiences.
Limitations:
10

---

Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant No. 60.620310,3), the Guangdong Province Key Construction Discipline Scientific Research Capacity Improvement Project (Grant No. 10.2002ZDIS117), Engineering Technology R&D Center of Guangdong Provincial Universities (Grant No. 20244GZ00000 and the Pengcheng Laboratory.
References
[1] T. Li M. Slavcheva, M. Zollhoefer, S. Green, C. Lassner, C. Kim, T. Schmidt, S. Lovegrove, M. Goesele, R. Newcombe et al., "Neural 3d video synthesis from multi-view video," in Proceedings of the IEER/CVF Conference on Computer Vision and Pottern Recognition, 2022, pp. 5521-5531.
[2] J. Sun, H. Jiao, G. Li, Z. Zhang. L. Zhao, and W. Xing, "3dgstream: On-the-fly training of 3d gaussians for efficient streaming of photo-realistic free-viewpoint videos." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 20.75220688
[3] Y. Chen, Q. Wang. H. Chen, X. Song, H. Tang. and M. Tian, "An overview of augmented reality technology." Journal of Physies: Conference Series, vol. 1237, no. 2. IOP Publishing. 2019, p. 022082.
[4] Z. Yang. Yang, Z. Pan, and Zhang, "Real-time photorealistic dynamic scene representation and rendering with gaussian splatting," arXiv preprint arXiv:2310. 10642, 2023.
[5] J. Luiten, G. Kopanas, B. Leibe, and D. Ramanan, "Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis." in 2024 International Conference on 3D Vision (3DV). IEEE, 2024, pp. 500-809.
[6] Z. Li, Z. Chen, Z. Li, and Y. Xu, "Spacctime gaussian feature splatting for real-time dynamic view synthesis," in Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024, pp. 8508-8520.
[7] 7. Yang, Gao, W. Zhou, S. Jiao, Y. Zhang, and X. Jin, "Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction," Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 331-20 341..
[8] G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang. "4d gaussian splatting for real-time dynamic scene rendering," in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024, pp. 20 10310-20320 320.
[9] Z. Lu, X. Guo. L. Hui, T. Chen, M. Yang, X. Tang, F. Zhu, and Y. Dai, "3d geometry -aware aware deformable gaussian splatting for dynamic view synthesis," in Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024, pp. 8900-8910.
[10] M. Liu, Q. Yang. H. Huang, W. Huang, Z. Yuan, Z. Li, and Y. Xu, "Light4gs: Lightweight compact 4d gaussian splatting generation via context model," arXiv preprint arXiv:2503. 133488 2025.
[11] J. Bae, S. Kim. Y. Yun, H. Lee, G. Bang, and Y. Uh, "Per-gaussian embedding based deformation for deformable 3d gaussian splatting," in European Conference Computer Vision. Springer, 2024, pp. 321-335.
[12] T. Müller, A. Evans, C. Schied, and A. Keller, "Instant neural graphics primitives with a multiresolution hash encoding," " ACM transactions on graphics (TOG), vol. 41, no. 4, pp. 1-15, 2022.
[13] S. Girish, Li, A. Mazumdar, A. Shrivastava, De Mello er al., "Queen: Quantized efficient encoding of dynamic gaussians for streaming free-viewpoint vidcos," Advances in Neural Information Processing Systems, vol. 37, pp. 43 435-43 467, 2025.
[14] Q. Gao, J. Meng, C. Wen, J. Chen, and J. Zhang, "Hicom: Hierarchical coherent motion for dynamic streamable scenes with 3d gaussian splatting," in The Thirty-eighth Annual Conference Neural Information Processing Systems.
[15] Y.H.L Huang, Y.-T. Sun, Z. Yang. X. Lyu. Y.-. Cao, and X. Qi, "Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes." " in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 4220-4230.
[16] D. Wan, R. Lu, and G. Zeng, "Superpoint gaussian splatting for real real-time high-fidelity dynamic scene reconstruction," in Proceedings of the 41st International Conference on Machine Learning, 2024, pp. 49 957 972

---

[17] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. "3d gaussian splatting for real-time radiance field rendering." ACM Trans. Graph. vol. 42, no. 4, pp. 139-1, 2023.
[18] K. Navaneet, K. P. Mcihodi, S. A. Koohpayegani, and H. Pirsiavash, "Compact3d: Compressing gaussian splat radiance field models with vector quantization." arXiv preprint arXiv:331 18159, 2023.
[19] Z. Fan, K. Wang, K. Wen, Z. Zhu, D. Xu, and Z. Wang, "Lightgaussiam: Unbounded 3d gaussian compression with 15x reduction and 200+ fps." arXiv preprint arXiv:2311. 2023.
[20] J. C. Loc, D. Rho, X. Sun, J. H. Ko, and E. Park, "Compact 3d gaussian representation for radiance field." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21 719-212277 728.
[21] H. Wang, H. Zhu, T. He, R. Feng, J. Deng. J. Bian, and Z. Chen, "End-to-end rate-distortion optimized 3d gaussian representation," in European Conference on Computer Vision. Springer, 2025, pp. 76-92.
[22] P. Papantonakis, G. Kopanas, B. Kerbl, A, Lanvin, G. Drettakis, "Reducing the memory footprint of 3d gaussian splatting." Proceedings of the ACM on Computer Graphics and Inteructive Techniques, vol. 7, no. 1, pp. 1-17, 2024.
[23] Z. Xu, Y. Xu, Z. Yu, S, Peng. J. Sun, H. Bao, and X. Zhou, "Representing long volumetric video with temporal gaussian hierarchy," ACM Transactions on Graphics (TOG), vol. 43, no. 6, pp. 1-18, 2024.
[24] Y. Wang, Yang. Z. Xu, Sun, Zhang, Y. Chen, H. Bao, S. Peng, and X. Zhou, "Freetimegs: Free gaussian primitives at anytime anywhere for dynamic seene reconstruction," in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 21 750-21 760.
[25] W. O. Cho, I. Cho, S. Kim, J. Bac, Y. Uh, and S. J. Kim, "4d scaffold gaussian splatting for memory efficient dynamic scene reconstruction," arXiv preprint arXiv:2411.17044 2024.
[26] D. Sun, H. Guan, K. Zhang, X. Xie, and K. Zhou, "Sdd-4dgs: Static-dynamic aware decoupling in gaussian splatting for scene reconstruction, arXiv preprint (rXiv:2503.093:3 2025.
[27] J. Yan, R. Peng, L. Tang, and R. Wang, "4d gaussian splatting with scale-aware residual field and adaptive optimization for real-time rendering of temporally complex dynamic scenes," in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 7871-7880.
[28] A. Cao and Johnson, "Hexplane: A fast representation for dynamic scenes," in Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2023, pp. 130-141.
[29] L. E. Peterson, "K "K-nearest neighbor," Scholarpedia, vol. 4, no. 2. 1883, 2009.
[30] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T.. Barron, R. Ramamoorthi, and R. Ng. "Nerf: Representing scenes as neural radiance fields for view synthesis." Communications of the ACM, vol. 65, no. 1, pp. 99-106, 2021.
[31] C. Sun, M. Sun, and H.-T. Chen, "Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction." in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2022, pp. 5459-5469
[32] A. Chen, Z. Xu, A. Geiger. J. Yu, and H. Su, "Tensorf: Tensorial radiance fields." in European conference on computer vision. Springer, 2022, pp. 333-350.
[33] A. Chen, 7. Xu, X. Wei, S. Tang, H. Su, and A. Geiger, "Dictionary fields: Learning a neural hasis decomposition," ACM Transactions on Graphics (TOG), vol. 42, no. 4, pp. 1-12, 2023.
[34] S. Fridovich- Keil, A. Yu, M. Tancik, Q Chen, B. Recht, and A. Kanazawa, "Plenoxels: Radiance fields without neural networks." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5501-5510.
[35] J. T. Barro, B. Mildenhall, M. Tancik, P. Hedman, R. Martin- Brruala,, and P. P. Srinivasan, "Mip-nerl: A multiscale representation for anti-aliasing neural radiance fields," in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 5855-5864.
[36] D. Verhin, P. Hedman, B. Mildenhall, T. Zickler, T. Barron, and P. P. Srinivasan, "Ref-nerf: Structured view-dependent appearance for neural radiance fields." in 2022 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022. pp. 5481-5490.
[37] L. Li, Z. Shen, Z. Wang, L. Shen, and P. Tan, "Streaming radiance fields for 3d video synthesis," Advances in Neural Information Processing Systems, vol. 35, pp. 13 498 2022.
12

---

[38] L. Wang, K. Yao. C. Guo. Z. Zhang, Q. Hu. J. Yo. L. Xu, and M. Wu, "Videorf: Rendering dynamic radiance fields as 2d feature video streams," in Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2024. pp. 470-481.
[39] L. Wang, Q. Hu, Q. He, Z. Wang, J. Yu, T. Tuytelaars, L. Xu, and M. Wu, "Neural residual radiance fields for streamably free-viewpoint videos," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 76-87.
[40] M. Wu, Z. Wang, G. Kouros, and T. Tuytelaars, "Tetrirf: Temporal tri-plane radiance fields for efficient free-viewpoint video," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 6487-6496.
[41] L. Song, A. Chen, Z. Li, Z. Chen, Chen, J. Yuan, Y. Xu, and A. Geiger, "Nerfplayer: streamable dynamic scene representation with decomposed neural radiance fields," IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 5, pp. 2732 2742, 2023.
[42] P. Wang. Z. Zhang. L. Wang. K. Yao, S. Xie, J. Yu, M. Wu, and L. Xu, "V" 3: Viewing volumetric videos on mobiles via streamable 2d dynamic gaussians," ACM Transactions on Graphics (TOG), vol. 43, no. 6, pp. 1-13, 2024.
[43] J. Yan, R. Peng, 7. Wang, L. Tang, J. Yang, J. Liang, J. Wu, and R. Wang, "Instant gaussian stream: Fast and generalizable streaming of dynamic scene reconstruction via gaussian splatting," arXiv preprint arXiv:2503. 16979, 2025.
[44] T. Lu, M. Yu, L. Xu, Y. Xiangli, L. Wang, D. Lin, and B. Dai, "Scaffold-gs: Structured 3d gaussians for view-adaptive rendering," in Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2024, pp. 20 0654-00664 604
[45] S. Niedermayr, J. Stumpfegger, and R. Westermann, "Compressed 3d gaussian splatting for accelerated novel view synthesis," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 10349-10 358.
[46] Y. Chen, Q. Wu, W. Lin, M. Harandi, and Cai, "Hac: Hash-grid assisted context for 3d gaussian splatting compression," " in European Conference on Computer Vision Springer, 2025, pp. 422-438.
[47] Y.TT. Zhan, C.-Y. Ho, H. Yang. Y.H.. Chen, J. C. Chiang, Liu, and W.-H. Peng. "Cat-3dgs: A context- adaptive triplane approach to rate-distortion-opiiiiee 3dgs compression," arXiv preprint arXiv:2503.00333 2025.
[48] Z. Chen, Z. Chen, W. Jiang, W. Wang, L. Liu, and D. Xu, "4dgs-ce: A contextual coding framework for 4d gaussian splatting data compression," arXiv preprint #rXiv22504.88225 2025.
[49] L. Tang, J. Yang. R. Peng. Zhai, S. Shen, and Wang. "Compressing streamable free-viewpoint videos to 0.1 mb per frame," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 7, 2025, pp- 7257-7265.
[50] J. L. Schonberger and J.-M. Frahm, conference on computer vision and pattern recognition, 2016, pp. 4104-4113.
revisited," in Proceedings of the IEEE
[S1] Z. Wang, A. C. Bovik, R. Sheikh, and P. Simoncelli, 'Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004.
[52] B. Attal, J.-B. Huang, C. Richardt. M. Zollhoefer, J. Kopf. M. O'Tolle, and C. Kim, "Hyperreel: High- fidelity 6-dof video with ray-conditioned sampling." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 16610-16620,
[53] Y. Jiang, K. Yao, Z. Su, Z. Shen, H. Luo, and L. Xu, "Instant-nvr: Instant neural volumetric rendering for human-object interactions from monocular rgbd stream," in Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 595-605.
[54] H. Lin, S. Peng, Z. Xu, Y. Yan, Q. Shuai, H. Bao, and X. Zhou, "Efficient neural radiance fields for interactive free-viewpoint video," in SIGGRAPH Asia 2022 Conference Pupers, 2022, pp. 1-9.
[55] D. A. Huffman, "A method for the construction of minimum-redundancy codes." Proceedings of the IRE, vol. 40, no. 9, pp. 1098-1101, 1952.
[56] S. Fridovich- Keil, G. Meanti, F. R. Warburg, B. Recht, and A. Kanazawa, "K-planes: Explicit radiance fields in space, time, and appearance," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 12 22479-12 488.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We show comprehensive experiments and ablation studies on the datasets that are broadly used in this area. Our claims accurately reflect the contribution and scopt of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in the conclusions section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification: Our study is not theory work. Sec. elxplains all components and equations in detail, and cite relevant theoretical works in the related work and method sections. Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed explanations of each component of our method in Sec. and additional implementation details presented in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We would release the code after acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.com cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/nips
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We follow benchmark and evaluation metric which are widely used by ex- isting works in this area. More experimaent details and hyperparameters are provided in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We follow benchmark and evaluation metires that are widely used by existing work in the area. To our knowledge, most of the existing work in this area do not provide statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide experimental details comprise of computational hardware, training times, storage requirements and dataset specification in the implementation details in the main paper and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, our research conducted in the paper conform, in every respect, with the
NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential positive societal impacts in Appendix.
Guidelines:
17

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Our method is evaluated several public datasets and we followed their license, as well as credited and cited their work and dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy httpee///earrrpooccc///oeeeerere for what should or should not be described.
20

---

Appendix
We provide material to supplement main paper. This appendix first introduces more implementation details Sec. A Then, we provide additional experimental results in Sec. and
broader impact in Sec. C
A More Implementation details
Our code is based on the open-source code of 3DGStream 22] MeetRoom dataset, utilize COLMAP [50] generate the initial point cloud and vanilla 3DGS [7 to initialize the Gaussians for 3000 epochs at first frame. Subsequently, our ComGS reconstructs the non key frames for 150 epochs and key frames for 1000 epochs. For the balance of visual quality and storage requirements, we set spherical harmonics (SH) degree to 1. During training, the learning rate for Gaussian attributes is set to 0.002, for the attributes of the adaptive influence region 0.02, and for the learnable mask my to 0.01. Other learning rates follow the setting of 3DGStream 2
Training:
On both N3DV and
Compression: For the reconstruction process, the uncompressed Gaussian attributes and their residuals have substantial memory requirements. We employ quantization and entropy coding to further compress them. Specifically, for the first frame reconstruction, apply 16-bit quantization to the position attributes due to their higher sensitivity, while the other attributes are quantized to bits. For the correction in key frame reconstruction, Notably, the attributes of a keypoint play a crucial role in guiding the motion of nearby non-keypoints. As result, minor quantization eтгоrs in keypoints may be amplified throughout the scene. To preserve modeling accuracy, we thus refrain from quantizing keypoint attributes. Finally, we deliver these quantized values entropy coding (55]
quantize all attribute residuals using 8 bits.
Datasets: (1) Neural 3D Video (N3DV) dataset 1 comprises six indoor scenes captured by a multi-view system of 18 to 21 cameras at resolution of x704 2028 and 30 FPS. Following the previous works (2 8). we downsample the vidcos by a factor of for training and testing and employ the central view for testing view. (2) MeetRoom dataset (37 is captured by a 13-camera multi-view system, including four dynamic scenes at 1280 720 resolution and 30 FPS. The center reference camera is also used for testing. As the aforementioned two datasets contain 300 frames, also conduct long video reconstruction evaluation on the Flame Salmon scene with 1200 frames from the N3DV dataset. We perform distortion for this dataset following the settings of the 3DGS [17 to improve the reconstruction quality.
Table 6: Quantitative results the random access version on N3DV dataset 1
Metric
Coffee Martini
Cook Spinach
Cut Beef
Flame Salmon
Flame Steak
Sear Steak
PSNR (dB 1)
28.52
32.31
32.97 119.3
29.19 168.6
33.01 114.7
33.51
Storage (KB
177.4
115.3
105.3
Table 7: Ablation study on Number of keypoints.
#Keypoints
100
200
300
500
PSNR (dB 1)
31.77 44.4
31.85
31.87 50.1
31.84
31.86
31.80
Storage (KB 4)
46.2
50.2
54.4
57.3
B
Additional Experimental Results
B.1
Random Access
Random access is crucial for video streaming and interactive experiments. However, existing online FVV reconstruction methods [3] [4] rely on the Gaussian points the previous frame during each current frame reconstruction, thus only supporting forward playback from the first frame.
In contrast, our method enables random access by simply modifying a small part of the system configuration. Specifically, compared to the original setting, instead reconstruct nen frames
21

---

(8) Error-aware correction
(b) Error-aware aware region
(c) Rendered image
(d) GT
Figure 7: (a) Visualization of error-aware Gaussians. (b) Visualization of error regions between key
frame and previous frame. (c)(d) Comparison on rendered images and original images.
using the keypoints from their nearest preceding key frame. Key frames are reconstructed based on the previous key frame using error-aware correction. Additionally, to further decouple key frames from earlier ones, we introduce periodic I-frames (e.g., every 60 frames), in which all Gaussian primitives are either saved or re-optimized independently. With this adaptation, accessing a specific frame only requires access to its nearest preceding key frame and the associated keypoints, making random access feasible. Tab. 6 presents the quantitative results of the random access version.
Table Ablation study on Group of Frames.
#Frames
2
5
10
15
20
PSNR (dB 1)
32.12 108.3
32.01
31.87
31.78
31.66
Storage (KB
66.6
50.1
43.2
40.0
B.2 More Ablation Study
In this section, we further investigate the hyperparameters and analyze the impact of the proposed components N3DV [I] dataset, achieve balance between performance and efficiency.
Effect of the keypoint numbers: struction quality and compression efficiency. we conduct an ablation study by varying the number of keypoints from 50 to 500. As shown in Tab. 7 the reconstruction performance peaks when using 200 keypoints. This observation aligns with the nature of dynamic scenes, where motion typically occurs a limited spatial region. Using 200 keypoints is sufficient to capture these areas for effective reconstruction. Increasing the number of keypoints beyond this leads to redundant incorrect representation in static regions. Therefore, using 200 keypoints strikes a good balance between performance and storage, and is adopted the default configuration in our method.
To investigate the impact of the number of keypoints on recon-
evaluate how the size of the Group of Frames (GoF) affects reconstruction quality and storage, as shown in Tab. [ These results indicate that shorter GoFs can better handle non-rigid motions and novel objects. which are difficult to be reconstructed by keypoint-driven motion. Larger GoFs exploit temporal redundancy for better compression, but may accumulate errors in the presence motion and scene changes. our setting, we use GoF as our large model for high-fidelity reconstruction, and GoF = 10 small model for compact representation.
Effect of group of frames:
Effect of error-aware correction: We explore the effect of the parameter Aeror on reconstruction quality and storage, as shown in Tab. / While a larger Aumor improves compression by focusing only on perceptually salient errors, it may overlook subtle regions, which leads to degraded reconstruction. In contrast, smaller values retain more points, which helps suppress error accumulation across frames, albeit with higher storage costs.
22

---

Table Effect of Aerror on reconstructed quality and storage.
Aerror
0
0.0001
0.001
0.01
PSNR (dB 1)
31.91
31.91
31.87
31.79
Storage (KB І)
183.0
96.3
50.1
29.2
Table 10: Per-scene quantitative results on the N3DV dataset. Offline and online methods are separated for clarity.
Coffee Martini
Cook Spinach
Cut Beef
Method
PSNR
Storage (MB J)
PSNR
Storage (MB })
PSNR
Storage
(dB 7)
(dB f)
(dB f)
(MB J)
KPlanes
56
29.99
1.0
32.60
1.0
31.82
1.0
NeRFPlayer 41
31.53
18.4
30.56
18.4
29.35
18.4
HyperReel
52
28.37
1.2
32.30
1.2
32.92
1.2
4DGS
28.33
29.0
32.93
29.0
33.85
29.0
4D-GS
27.34
0.3
32.46
0.3
32.49
0.3
Spacetime- GS 6
28.61
0.7
33.18
0.7
33.52
0.7
E-D3DGS 11
29.33
0.5
33.19
0.5
33.25
0.5
StreamRF 37
27.84
31.84
31.59
31.84
31.81
31.84
3DGStream 2
27.75
7.80
33.31
7.80
33.21
7.80
QUEEN-I
28.38
1.17
33.40
0.59
34.01
0.57
ComGS-s (ours)
28.63
0.058
32.94
0.047
33.30
0.051
ComGS-I (ours)
28.76
0.154
33.26
0.094
33.53
0.104
Flame Salmon
Flame Steak
Sear Steak
PSNR
Storage (MB L)
PSNR
Storage (MB }}
PSNR
Storage
(dB
(dB f)
(dB
(MB
KPlanes
50
30.44
1.0
32.38
1.0
32.52
1.0
NeRFPlayer
41
31.65
18.4
31.93
18.4
29.12
18.4
HyperReel
52
28.26
1.2
32.20
1.2
32.57
1.2
4DGS
29.38
29.0
34.03
29.0
33.51
29.0
4D-GS 29.20
0.3
32.51
0.3
32.49
0.3
Spacetime-Gr
29.48
0.7
33.40
0.7
33.46
0.7
E-D3DGS
29.72
0.5
33.55
0.5
33.55
StreamRF 37
28.26
31.84
32.24
31.84
32.36
31.84
3DGStream
28.42
7.80
34.30
7,80
33.01
7,80
QUEEN-I 13
29.25
1.00
34.17
0.59
33.93
0,56
ComGS-s (ours)
29.31
0.052
33.42
0.045
33.59
0.040
ComGS-I (ours)
29.58
0.129
33.84
0.083
33.74
0.0704
Fig. 7 (a) visualizes the error- aware Gaussian points identified by error-aware correction, while (b) shows heatmap of differences between the key frame and the previous frame, which highlights the error regions. We observe that the error-aware points in (a) align well with the high-error regions in (b). which indicates that method effectively captures areas likely suffer from accumulation. Fig. (c) and (d) compare our rendered images with the ground truth. The results show that our method significantly reduces artifacts in dynamic regions, confirming the effectiveness of our error-aware correction.
B.3 More Results
To offer a more comprehensive comparison, the per-scene quantitative results are presented on
N3DV I and MeetRoom 37 in Tab. 10 and Tab. [1] respectively. Moreover, we also provide
the experimental results of existing offline and online methods Tab. To as reference. Further qualitative results with StreamRF [37] and 3DGStream 2 are indicated in Fig. g and Fig.
23

---

C Broader Impact
Our work is a positive technology. This method reconstructs free-viewpoint videos from multi-view 2D videos in a streaming manner, which can improve the immersive and interactive experience of viewers. As discussed in the introduction, this technology has potential to benefit various aspects of daily life, including applications in remote diagnosis and 3D video conferencing.
Table 11: Per-scene quantitative results on the MeetRoom dataset.
Metries
Discussion
Stepin
Trimming
VrHeadsct
PSNR (dB f)
31.72 37.5
30.17 24.2
32.12 27.0
31.95
Storage (KB ()
24.5
Stream RF
3DGStream
Ours
GT
(n) Flame Steak
(b) Cook Spinach
(c) Cut Roasted Beef
(d) Flame Salmon
Figure 8: Comparison on N3DV
dataset.
24

---

3D G Stream
Ours
GT
(a) Discussion
(b) Stepin
(c) Trimming
Figure 9: Comparison on MeetRoom 37 dataset.
25


Reviews:
[{"review_id": "vlMOfxjYie", "reviewer": "Reviewer_JVLS", "summary": "This paper introduces **Compact Gaussian Streaming (ComGS)**, a keypoint-driven method for novel view synthesis of dynamic scenes, applied to free-viewpoint video reconstruction.\nIn the first frame, a static model is trained, and the proposed **Motion-Sensitive Keypoint Selection** identifies which Gaussians should serve as keypoints using the next frame's image. For non-keyframes, scene motion is driven by these keypoints, significantly reducing data storage since only the motion of **N keypoints** (much fewer than all Gaussians) with 14 parameters each is used.\nTo address **accumulated errors**, a new keyframe is introduced every **s frames**, using **error-aware correction**, which updates Gaussians with significant errors. The residual of updated Gaussians will be transmitted.\nExperiments show that ComGS achieves **0.05-0.1 MB/frame storage**, **competitive rendering quality**, and a **7-14x compression ratio** compared to existing SOTA work, QUEEN.", "questions": "1. **Training time and rendering speed**: It would be helpful to understand why the training time and rendering speed are impacted. While we understand that the increased complexity may contribute to the performance tradeoff, could you clarify why this is acceptable in the context of the method's goals? \n2. **Support for random access**: Does the proposed method support random access, similar to conventional video playback, where users can start viewing from any arbitrary time point? Based on the description, it seems that decoding subsequent keyframes depends on previous non-keyframes, which might limit random access. If this is the case, could you share your thoughts on addressing this challenge?\n3. **Control points in the background (Figure 4)**: In Figure 4, some control points appear in the background. Could this suggest that the method for selecting control points might need further refinement to improve robustness?\n4. **Ablation study (Lines 261-272)**: The description of the ablation experiment is a bit difficult to follow, and some details seem to be missing. Could you kindly provide a more detailed explanation of the setup and results?\n5. **Rate-Distortion curve**: Since this is a compression-focused work, presenting a Rate-Distortion (RD) curve would be very insightful for evaluating the tradeoff between compression efficiency and reconstruction quality.\n\nIf the authors provide clear feedback for my concerns, I will consider upgrading my evaluation from borderline reject to borderline accept.", "limitations": "From a compression perspective, the method significantly improves compression efficiency but at the cost of substantially increased training time (which can also be viewed as encoding time). From a practical application standpoint, this cannot be considered an online reconstruction method (even though 3DGStream-like methods tend to make such claims). The reviewer believes that unless subsequent frames (after the first frame) can be reconstructed within 30ms, it cannot be classified as online reconstruction.\n    \nOther limitations are related to the writing and analysis, as detailed in the question section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. **Strong performance**: The proposed method significantly enhances the compactness of 3DGStream-like approaches, requiring only **0.05-0.1 MB/frame** for storage. This is a remarkable result with valuable implications for the transmission and distribution of free-viewpoint video (FVV).\n    \n2. **Reasonable and effective design**: The reviewer finds the proposed **Motion-Sensitive Keypoint Selection** and **Adaptive Motion-Driven Mechanism** both novel and effective, making them well-justified design choices.\n\n## Weakness\n1. **Lack of videos for dynamic scene free-viewpoint rendering**: For a paper on dynamic scene novel view synthesis, providing rendering videos is crucial to demonstrate subjective quality. Without them in the supplementary materials, the reviewer cannot assess whether the method is temporally stable.\n    \n2. **Comparison with baselines**: The reviewer noticed that in quantitative comparisons, the MeetRoom dataset does not include QUEEN's rendering results. Similarly, in qualitative comparisons, neither Neu3DV nor MeetRoom includes QUEEN's results. Since QUEEN is now open-sourced and serves as a strong baseline, the reviewer believes it is necessary to update the comparisons to include it.\n    \n3. **Compromised training complexity and rendering speed**: This method increases training time significantly—taking 40 seconds per frame—which suggests it cannot be considered an online reconstruction method. Additionally, rendering speed is noticeably slower, making it a major drawback. On low-end GPUs (e.g., RTX 2080), this method might not support real-time dynamic rendering.", "was_revised": true, "final_justification": "I appreciate the authors' detailed response which has addressed most of my concerns. Regarding the training time and rendering time, although the authors provided potential solutions in the rebuttal, they didn't show the experimental results of the implementation. I look forward to their future work on this aspect.\n\nOther concerns have been largely addressed, and I appreciate the authors' work. Given the impressive compression performance of this work, despite significant compromises in training time and rendering FPS, it may still be worth accepting. Therefore, I have decided to change my decision from borderline reject to borderline accept.", "rebuttal": "Thank you for your valuable feedback. We appreciate the time and effort you have dedicated to reviewing our work. We will address the weaknesses and questions you raised below.\n\n## Q1: Training time and rendering speed.\n**Definition on online reconstruction:** In prior published works such as 3DGStream, QUEEN, and HiCom, online reconstruction is defined as the ability to process and transmit frames sequentially, without requiring access to the full video sequence. Our method follows this established definition and adopts the same frame-by-frame processing paradigm. This is different from traditional video coding, where \"online\" typically implies strict real-time constraints (e.g., 30 ms per frame). In online FVV domain, such latency is not a formal requirement. For example, even the fastest existing method, QUEEN, requires over 4000 ms per frame.\n\n**Our primary focus:** In this work, our primary focus is on reducing storage requirements for online FVV reconstruction, which is crucial for practical applications involving data storage and transmission, yet remains a challenging problem. Our method models the motion regions using only 200 keypoints per frame that offers a storage-efficient solution.\n\n**Why the training time and rendering speed are impacted?** Our current implementation does not specifically optimize for temporal efficiency in keypoint computation. Specifically, in Eq. (6), we compute the influence weight matrix $w$ (size ≈ [200000, 200]) **using all Gaussian primitives (~200,000)**. In Eq. (8), this matrix is directly used to apply translation offsets (size = [200, 3]) and rotation to the keypoints. These two steps together account for **approximately 78% of the training time **(0.1248s out of 0.16s per iteration). \n\n**Solution to the above issue:** These resulting computational complexity can be efficiently and readily resolved, and therefore do not pose a practical limitation. We provide several feasible solutions: (1) **Pruning:** Introduce a classifier to coarsely separate foreground and background Gaussian primitives, and compute $w$ only for the foreground Gaussian primitives, significantly reducing its size and may yield a 2-3× speed up. (2) **Sparse:** Note that $w$ is extremely sparse (about 99.85% of values are zero); leveraging sparse matrix techniques could greatly improve both training and rendering speed by 3-4×. (3) **Parallel acceleration:** Our adaptive motion-driven mechanism is inspired by 3DGS, which benefits from distributed optimization and GPU-accelerated algorithms—these techniques are also applicable to keypoints training. \n\nBy integrating the above solutions along with engineering optimization, we estimate that the computational cost associated with Eq.6 and Eq.8 can be accelerated by at least 10×. This optimization would lead to an overall training time of under 10 seconds, and a rendering speed comparable to QUEEN.\n\nTo summarize, our method provides a new perspective on addressing a fundamental issue in online reconstruction—storage consumption—enabling efficient transmission and storage for FVV. As for the computational complexity introduced by keypoint processing, our aforementioned method can address it with relative ease, therefore, we believe the trade-off is acceptable in this context.\n\n## Q2: Support for random access.\nWe appreciate the reviewer’s insightful question. Our method does support random access with minor modifications to the current setup.\n\n**(Random access v1)** In the original setting, each frame is reconstructed based on the previous frame. To enable random access, we can instead reconstruct non-key frames based on their nearest preceding key frame using the keypoints. The error-aware correction module can also be modified to rely on this preceding key frame. With this adaptation, accessing a specific frame only requires access to its nearest preceding key frame and the associated keypoints, making random access feasible.\n \n**(Random access v2)** To further decouple key frame from earlier key frames, we can introduce periodic I-frames (e.g., every 60 frames), in which all Gaussian primitives are either saved or re-optimized independently. These I-frames serve as standalone reference primitives, enabling more flexible and efficient random access across the sequence.\n \nTo demonstrate their effectiveness, we provide the following experiments:\n \n**Tab. R1:** Performance of Random Access Version on sear steak.\n \n| Methods                           | PSNR  | Size (MB) |\n|--|---|----|\n| Original     | 33.59 | 0.040      |\n| Random access v1     | 33.51 | 0.048      |\n| Random access v2     | 33.51 | 0.101      |\n \nThese results will also be included in the revised version.\n \nInterestingly, if the reconstruction of non-key frames is based solely on key frames, it enables parallel training across frames. This directly addresses the concern raised in **Q1**，effectively accelerating the training process.\n\n## Q3: Keypoints in the background (Figure 4).\nWe appreciate the reviewer's insightful comments. As the reviewer correctly pointed out, Fig.4 in the manuscript demonstrates that the motion regions are well covered by the distributed keypoints across all images, indicating that these motion areas can be effectively modeled by the selected keypoints. While some redundant keypoints do appear in the background, they do not negatively affect the quality of scene reconstruction.\n \nHowever, this observation motivates us to explore a more robust keypoint selection strategy that avoids redundancy and improves overall efficiency. For example, we can apply the classifier-based filtering method (as discussed in Q1) to exclude background keypoints and focus computation on motion regions.\n\n## Q4: Confusion on ablation study.\nWe sincerely apologize for the confusion caused in the ablation study section. Below, we provide a detailed clarification of our experimental settings and results:\n \n**Experiment 1:** We ablate the motion-sensitive keypoint selection and instead select keypoints randomly. This strategy fails to accurately localize motion-sensitive regions (as shown in Fig. 5(b)), resulting in a noticeable performance drop.\n \n**Experiment 2:** We remove the adaptive motion-driven mechanism and model scene motion using only the selected keypoints, without incorporating any neighboring points. This simplification leads to a drop in performance, underscoring that effective motion modeling relies not only on accurately selecting keypoints in motion regions, but also on the selection of their neighboring points.\n \n**Experiment 3:** We model FVV solely based on the reconstruction of the first frame and correction within keyframes, without modeling non-key frames by keypoint reconstruction. This results in a significant performance drop. We emphasize that although the parameters of the keypoints are few, the keypoint-based modeling plays a crucial role in reconstructing non-key frames.\n \n**Experiment 4:** We ablate the correction mechanism in keyframes. This leads to a severe performance drop due to the presence of non-rigid motion in the scene. Relying only on keypoint-based modeling is insufficient, and error accumulation over time severely degrades subsequent reconstructions.\n \nWe hope the above explanations help clarify the experimental design and alleviate your concerns.\n \n## Q5: Lack of experiments.\n**Videos for dynamic scene free-viewpoint rendering:** We completely agree with the reviewer that rendered videos are essential for evaluating the temporal consistency of the method. However, due to conference policy restrictions, we are unable to include any visual results during the rebuttal phase. We will provide rendered videos in the revised version via our GitHub project page.\n \nTo allow the reviewer to quantitatively assess the temporal consistency of our results even in the absence of visualizations, we follow the video smoothness evaluation metric in [A] and compare our method with 3DGStream.\n \n**Tab. R2:** Temporal smoothness on N3DV (higher is better).\n \n| Method    | Smoothness |\n|---------|-------|\n| 3DGStream |  1.049 |\n| Ours      |  1.065 |\n \n> **Note:** A score of 1 indicates temporal consistency equal to the ground truth.\n \n[A] Eilertsen G, Mantiuk R K, Unger J. Single-frame regularization for temporally stable cnns[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 11176-11185.\n\n**Comparison with baselines:** At the time of submission, the implementation of QUEEN was not publicly available, and thus we were unable to include its qualitative results or quantitative performance on the MeetRoom scene in our work. Now that the code has been released, we provide the quantitative results of QUEEN on the MeetRoom scene below. The remaining qualitative results will be included in the revised version.\n \n**Tab. R3:** Quantitative results on the MeetRoom dataset.\n \n| Method    | PSNR | Size (MB) |\n|-----------|-------|-------|\n| QUEEN |  31.14 |   0.45    |\n| Ours      |  31.49 |  0.028     |\n \n**Rate-Distortion curve:** We fully agree with the reviewer that rate-distortion (R-D) curves are an effective way to evaluate the trade-off between compression efficiency and reconstruction quality. In the right part of Fig.1, we present a scatter plot comparing our method with other approaches in terms of storage vs. PSNR, which clearly demonstrates the superiority of our method in both compression efficiency and performance.\n \nRegarding the reviewer’s suggestion to provide a more detailed trade-off analysis, we have already included experiments in the supplementary materials that analyze PSNR and storage size under varying keypoint counts and keyframe intervals. These results correspond directly to specific bitrate points on the R-D curve. Furthermore, we will include additional R-D curves for the ablated versions of our method in the revised submission to better illustrate the contribution of each component."}, {"review_id": "YNOJz0hUd5", "reviewer": "Reviewer_ffQB", "summary": "This paper proposes a compact and motion-aware Gaussian Splatting framework for dynamic scene modeling. Existing methods suffer from excessive storage requirements due to redundant motion modeling. ComGS addresses this by (1) identifying sparse keypoints (≈200) in motion regions using viewspace gradient differences, avoiding redundant motion modeling; (2) propagating keypoint motion to neighboring Gaussian points via a spatial influence field, enabling shared motion representation and reducing storage; (3) periodically refines only error-prone Gaussian points to mitigate error accumulation without redundant updates. Experiments on N3DV and MeetRoom datasets show ComGS achieves a 159× storage reduction over 3DGStream and 14× over SOTA method QUEEN, while maintaining competitive PSNR and rendering speed.", "questions": "1. The motion-sensitive keypoint selection uses a gradient difference threshold. How is this threshold determined? Does it adapt to scene complexity?\n\n2. The paper uses 200 keypoints. What is the impact of varying this number (e.g., 100 vs. 300) on storage and fidelity? Is there an optimal range?\n\n3. The words ''spatial influence field'' is confusing. Is it just the covariance?\n\nCurrently my suggestion is boarderline, my concerns and questions are listed above. The comparison with Representing Long Volumetric Video with Temporal Gaussian Hierarchy is necessary, which also focuses on the efficiency of reconstructing dynamic multi-view videos.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1. By focusing on motion regions and shared keypoint-driven motion, ComGS drastically reduces per-frame storage (0.049 MB/frame vs. 7.8 MB/frame for 3DGStream), enabling real-time transmission.\n2. ComGS maintains high visual quality (PSNR: 33.61 dB) and fast rendering (up to 147 FPS), outperforming baselines in both metrics.\n3. The ablation studies are adequate. \n\nWeakness\n1. The paper lacks the discussion and comparison with highly related works, i.e. Representing Long Volumetric Video with Temporal Gaussian Hierarchy[1], FreeTimeGS[2].\n\n[1] Xu, Zhen, et al. \"Representing long volumetric video with temporal gaussian hierarchy.\" ACM Transactions on Graphics (TOG) 43.6 (2024): 1-18.\n[2] Wang, Yifan, et al. \"FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n2. Keypoint-driven motion works seems similar to KNN-based method like SC-GS, the ablation study for KNN and Adaptive control shows that the performance is comparable.\n\n3. Performance depends on key frame spacing (e.g., ComGS-s uses s=10, ComGS-l uses s=2). Optimal intervals may vary across scenes, requiring manual tuning.\n\n4. If keypoints fail to accurately identify motion regions (e.g., in low-contrast scenes), motion propagation could be inaccurate, degrading quality.\n\n5. The rendering speed drops a lot compared to 3DGStream, QUEEN.", "was_revised": true, "final_justification": "I have read the responses and my concerns have been addressed. The discussions and comparisons metioned above should be included in the final version.", "rebuttal": "We appreciate the reviewer for the careful and thoughtful review of our work. In the following we address the concerns in the order as pointed out by the reviewer.\n\n## Q1: Lack of comparison and discussion on highly related work.\nWe sincerely appreciate the reviewer for pointing out the lack of discussion and comparison with highly relevant works, and for recommending two high-quality papers that are closely related to our topic.\n \nAlthough both methods are designed for offline reconstruction, we have greatly benefited from studying these works. Specifically, [B] proposes a multi-level 4D Gaussian representation that captures scene content at different levels of detail, which enables efficient reconstruction of long FVV sequences. [C], on the other hand, introduces a motion function for each Gaussian primitive, allowing it to handle complex dynamic scenes and achieve impressive reconstruction quality.\n \nWe will incorporate a detailed discussion of these two works in the Related Work section of our revised version, and will also include quantitative comparisons in the Experiments section. As a preliminary step, we provide the following quantitative results for comparison:\n \n**Tab. R4:** Quantitative results on the flame salmon (1200 frames).\n \n| Method     | PSNR  | Size (MB) |\n|------------|--------|--------|\n| [B]      | 29.44  | 0.075  | \n| Ours       | 29.58  | 0.054  | \n \n**Tab. R5:** Quantitative results on the N3DV dataset.\n \n| Method     | PSNR  | Size (MB) |\n|------------|--------|--------|\n| [C]      | 32.97  | 0.137  |\n| Ours-L       | 32.12  | 0.106  |\n \nWhile our method does not yet match the reconstruction quality of [C], our focus is on online reconstruction with high storage efficiency, which addresses a different and complementary aspect of the problem. We believe that the insights from [B] and [C] will inspire future improvements in our method's representation capacity, and we plan to explore these directions in subsequent work.\n \n[B] Xu, Zhen, et al. \"Representing long volumetric video with temporal gaussian hierarchy.\" ACM Transactions on Graphics (TOG) 43.6 (2024): 1-18. \n[C] Wang, Yifan, et al. \"FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n \n## Q2: Confusion on adaptive motion-driven mechanism.\n**Similar to KNN-based works:** We sincerely apologize for the confusion regarding the distinction between our method and KNN-based approaches. We provide a detailed explanation below.\n \nKNN-based methods typically compute neighbor weights solely based on Euclidean distances, without adapting to the motion scale in the scene. In contrast, our method is designed for online reconstruction and leverages a spatial influence field that is learned in a motion-adaptive manner. This enables our model to dynamically adjust the spatial extent of each keypoint’s influence according to the underlying motion scale, leading to more accurate motion modeling.\n \nAs the reviewer correctly pointed out, the results in Table 5 show that our method performs comparably to the KNN-based baseline. This is because both methods in this ablation study include our third proposed component—error-aware correction—which effectively mitigates the error accumulation typically observed in KNN-based approaches. However, even under this setting, our method still outperforms the KNN-based approach by 0.48 dB in PSNR, demonstrating the effectiveness of our design.\n \n**Confusion on spatial influence field:** Our spatial influence field is parameterized by a learnable covariance matrix. By learning this matrix, our method is able to control both the selection of sptially neighboring points and the influence strength of each keypoint on its neighbors. This formulation allows the model to adaptively adjust the influence region to suit different motion scales in the scene. Therefore, we refer to this learned control region as a spatial influence field.\n \n## Q3: Key frame spacing.\nWe thank the reviewer for the insightful comment. We have indeed considered this issue. Currently, the interval between keyframes in our method is adjusted solely based on the target compression rate, and is therefore fixed across all datasets.\n \nAs the reviewer rightly pointed out, the optimal keyframe interval is likely scene-dependent. We agree that a fixed interval based only on compression rate is a simple heuristic, and a more adaptive strategy could achieve a better balance between reconstruction quality and storage efficiency. In future work, we plan to explore dynamically adjusting the interval based on factors such as scene dynamics or reconstruction error.\n \n## Q4: Confusion on keypoint selection.\n**Indentifying motion region:** Currently, publicly available datasets lack of the low-contrast scenes. In our work, we conduct extensive evaluations on several widely used benchmark datasets, demonstrating strong motion region identification performance. Notably, the N3DV dataset contains relatively subtle and fine-grained motions and our method is still able to effectively detect and localize these motion regions. In addition, we also evaluate our method on the Google Immersive dataset, which contains scenes with larger and more complex motion. Our method successfully extracts meaningful motion keypoints in these challenging scenarios as well, further demonstrating its effectiveness across a wide range of motion magnitudes.\n \nOur keypoint selection is based on the gradient of Gaussian primitives in dynamic regions across frames. This mechanism is driven by inter-frame changes rather than absolute contrast, and therefore, we believe it would remain effective even in low-contrast scenes.\n \n**Keypoint selection based on gradient difference:** We apologize for the confusion caused. In our method, a dynamic significance score is computed for each Gaussian point based on **the gradient difference** across frames (as defined in Eq. 5). This score reflects the temporal variation of each point, allowing us to identify regions with significant motion. We then rank all points based on these scores and select **the top-k** points (where k is the number of keypoints) as the keypoints for the current frame.\n \nOur experiments and ablation studies demonstrate that selecting 200 keypoints is sufficient to cover most motion regions in the scene and to capture the underlying dynamics effectively.\n \n**Numbers of keypoints:** We thank the reviewer for raising this important question. We have also considered how to set the number of keypoints, and we conducted an ablation study on this in the supplementary material. For clarity, we include the results in our rebuttal:\n \n**Tab. R6:** Ablation study on numbers of keypoints.\n \n| #Keypoints     | 50 | 100 | 200 | 300 | 400 | 500 |\n|------------|--------|--------|---------|--------------|--------------|--------------|\n| PSNR      | 31.77  | 31.85  | 31.87 | 31.84 | 31.86 | 31.80 |\n| Size (KB)  | 44.4  | 46.2  | 50.1 | 50.2 | 54.4 | 57.3 |\n \nThe results show that selecting around 200 keypoints provides the best trade-off between reconstruction quality and storage cost. Interestingly, we observed that increasing the number of keypoints beyond this range does not improve performance, and in some cases, even slightly degrades it. We hypothesize that assigning too many keypoints may overfit simple motions, making them appear more complex than they actually are, which ultimately reduces the accuracy of motion modeling.\n \n## Q5: Rendering speed.\nIn this work, our primary goal is to minimize storage consumption for online FVV, thereby reducing storage and transmission requirements—an important and challenging issue in this field. Although reducing storage introduces some compromises in rendering speed, we do not consider this a fundamental limitation. Below, we explain the reasons behind the slowdown and provide corresponding solutions to address it.\n \nIn particular, our implementation of Eq.6 and Eq.8 involves computations over a large number of background Gaussian primitives in influence weight matrix (size $\\approx$ [200000, 200], which introduces significant redundancy and slows down rendering. However, we believe this can be addressed with the following strategies: (1) **Pruning:** Introduce a classifier to coarsely separate foreground and background Gaussian primitives, and compute $w$ only for the foreground Gaussian primitives, significantly reducing its size and may yield a 2-3× speed up. (2) **Sparse:** Note that $w$ is extremely sparse (about 99.85% of values are zero); leveraging sparse matrix techniques could greatly improve both training and rendering speed by 3-4×. (3) **Parallel acceleration:** Our adaptive motion-driven mechanism is inspired by 3DGS, which benefits from distributed optimization and GPU-accelerated algorithms—these techniques are also applicable to keypoints training. \n\nBy integrating the proposed solutions along with engineering optimization, we estimate that the computational cost associated with Eq.6 and Eq.8 can be accelerated by at least 10×. This optimization would lead to a rendering speed comparable to QUEEN.\n\nWe plan to incorporate these improvements in future work to enhance the rendering efficiency."}, {"review_id": "xKIwWgBazJ", "reviewer": "Reviewer_w6Sx", "summary": "The paper proposes the ComGS framework, which optimizes online free-viewpoint video (FVV) reconstruction and transmission through keypoint-driven motion modeling. The core innovations include:\n1. Motion-sensitive keypoint selection, which locating dynamic regions based on viewpoint spatial gradient differences;\n2. Adaptive motion-driven mechanism through propagating keypoint motion to neighboring Gaussian points via a spatial influence field;\n3. Error-aware correction strategy which selectively optimizes error-prone regions in keyframes.\nExperiments show that ComGS achieves significant performance improvements and real-time rendering speed.", "questions": "1. While Section 3.4 examines selective residual updates, it lacks in-depth discussion regarding the handling of newly emerging objects. Can you further explain how this article addresses this issue?\n\n2. The current methods for FVV are largely confined to existing benchmark datasets and lack validation on challenging scenarios involving non-rigid objects, varying illumination conditions, and rapid motions. Can the method proposed in this article be applied to such scenarios?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThis paper proposes many effective methods to improve performance of FVV task, which includes:\n1. The keypoint selection strategy effectively focuses on dynamic regions; \n2. The spatial influence field replaces traditional KNN, adaptively accommodating varying motion scales;\n3. Error-aware correction effectively identifies the Gaussians need to be corrected.\n\nComprehensive comparisons with online/offline methods on mainstream datasets and ablation studies validate the performance of the proposed method and the necessity of each module.\n\n**Weakness**\n\n1. Relies on the rigid motion assumption, which may fail for non-rigid deformations. Besides, fixed number of keypoints is not adapted to varying motion complexity across scenes.\n2. For quality results, only PSNR is reported.  Including more evaluation (SSIM, LPIPS, etc.)metrics can better reflect the performance of the proposed method in the paper.", "was_revised": true, "final_justification": "While minor points remain, the authors' revisions have sufficiently improved the manuscript to warrant acceptance.", "rebuttal": "We sincerely appreciate the reviewer for the careful and thoughtful review of our work. Below, we address the concerns and limitations you pointed out, and provide our corresponding responses and solutions.\n \n## Q1: Failure for non-rigid motion\nWe sincerely appreciate your valuable feedback. We have indeed considered this issue during the development of our method. To effectively model complex motion, we propose the Adaptive Motion-Driven Mechanism and the Error-Aware Corrector, which work together to address this challenge.\n \nSpecifically, in non-keyframes, our method models motion by leveraging the spatial influence fields of multiple keypoints. We allocate a sufficient number of keypoints to ensure that even complex or non-rigid motions can be accurately captured through their combined influence. However, this approximation can introduce errors. To address this, during keyframe stages, we apply the Error-Aware Corrector to identify and correct Gaussian primitives with accumulated errors caused by non-rigid or more complex motion patterns.\n \nSince our method aims to minimize the storage required for FVV, there is an inherent trade-off in reconstruction accuracy. Nevertheless, this can be mitigated by reducing the interval between keyframes, allowing the method to adaptively retain higher fidelity when needed.\n\n## Q2: Fixed numbers of keypoints.\nIn our experimental setup, we used a fixed number of keypoints (200) across all datasets and videos. Our experiments and visualizations show that this number is sufficient to effectively cover motion regions and model dynamic changes in various scenes.\n \nWe also greatly appreciate the reviewer’s insightful suggestion. We believe that exploring a strategy to adaptively determine the number of keypoints, based on the scale of motion in the scene, could lead to more efficient keypoint allocation and better adaptability to different motion complexities. This is a promising direction we plan to investigate in future work.\n \n## Q3: Lack of more evaluation metric.\nWe greatly appreciate the reviewer’s insightful suggestion, as we fully agree that incorporating more evaluation metrics is essential for providing a more comprehensive and balanced assessment of the performance and effectiveness. We include additional evaluation results below and will further incorporate these metrics into the revised version to ensure a more thorough and rigorous evaluation of our approach.\n \n**Tab. R7:** Quantitative results on the N3DV dataset.\n \n| Method     | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|------------|--------|--------|---------|\n| 3DGStream      | 31.67  | 0.9410  | 0.1400   |\n| Ours       | 31.87  |  0.9434 |  0.1320  |\n \n**Tab. R8:** Quantitative results on the MeetRoom dataset.\n \n| Method     | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|------------|--------|--------|---------|\n| 3DGStream      | 30.79  | 0.9498  | 0.1882   |\n| Ours       | 31.49  |  0.9548 |  0.1709  |\n \n## Q4: Discussion on newly emerging objects.\nSimilar to the discussion in Q1, the modeling of newly emerging objects is handled through the error-aware corrector applied at key frames. Although our method does not introduce new Gaussian primitives specifically for the newly appeared objects, the large number of redundant Gaussians[D] retained in the scene provides sufficient representation capacity to model such objects effectively.\n \nAs discussed in Q1, minimizing the storage may lead to a drop in reconstruction quality. However, this issue can be alleviated by reducing the keyframe interval, which allows for more frequent correction and better adaptation to scene changes. Despite the lack of additional Gaussian primitives for new objects, our method still demonstrates robust performance in handling them.\n \nFor example, our experiments on the stepin (MeetRoom dataset) where a new person enters the meeting room. Our method achieves superior reconstruction quality, both in quantitative metrics and visual results. The quantitative results are as follows:\n \n**Tab. R9:** Quantitative results on the stepin.\n \n| Method     | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|------------|--------|--------|---------|\n| 3DGStream      | 28.35  | 0.9351  | 0.2084   |\n| Ours       | 30.04  |  0.9451 |  0.1862  |\n \n[D] Wang H, Zhu H, He T, et al. End-to-end rate-distortion optimized 3d gaussian representation[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 76-92.\n \n## Q5: Lack of validation on challenging scenarios.\nDue to the capability of our method to model complex motion and to correct modeling errors, it is well-suited to handle challenging scenarios such as non-rigid objects, varying illumination conditions, and rapid motions.\n \nTo further validate the effectiveness of our approach in such settings, we conducted additional experiments on the Google Immersive dataset, which features highly dynamic scenes with extensive motion and newly appearing elements such as flames. Our method achieves comparable reconstruction quality on this dataset. The experimental results are presented below and will be included in the revised version:\n \n**Tab. R10:** Quantitative results on the Google immersive daatset.\n \n| Method     | PSNR| Size(MB) |\n|------------|--------|--------|\n| 3DGStream      | 25.18  | 8.83  |\n| Ours       | 27.24  |  0.16 |"}, {"review_id": "hLytfIEZlN", "reviewer": "Reviewer_HKLW", "summary": "Paper addresses the task of Streaming for Free-Viewpoint Video Reconstruction:\n\nFor this the frames are first grouped (GpF) with fixed size(?).\nThe first frame is reconstructed with 3DGS and the remaining frames of the group are modeled with deforming the Gaussians of the first frame.\nKeypoints with Gaussian influence fields are used to model motion in compact efficient manner.\nError aware masking is used to only make necessary update of parameters between key frames.", "questions": "I am not familiar with the prior works like QUEEN and 3DGSteam. I think parts of the paper are hard to understand since the proposed methods heavily builds on these works. \n\nNevertheless, most of the paper is well written and I was able to understand almost all of it. \nMy only concern is the section \"Error-Aware Corrector\". \nTo my understanding each keyframe is reconstructed individually (or just the very first one?). How are the Gaussian's matched between keyframes? \nIf the reconstruction is only done once for the first keyframe, then how does the densification work? Or this there none?\nImproving this part of the paper would definitively help to make the whole paper easier to understand. \n\nIf the authors address my concerns regarding that section I will consider raising the score.", "limitations": "Limitations are addressed but only in appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The proposed method achieves SOTA compression with very good visual quality\n- An extensive ablation study is performed to show the effect of each component \n- good visualization of components like key point selection which helps to understand the method and gives an intuition on how it works\n- easy to follow, compact mathematical notation\n\nWeaknesses:\n\n- Limitations are in appendix. IMO need to be moved (or short version of it) to main paper\n- No 3DGS compression methods in related work\n- Preliminary is missing the EWA splatting part\n- Evaluation datasets are limited. What about orbital camera datasets?", "was_revised": true, "final_justification": "The authors addressed my concerns in their rebuttal. Nevertheless, I believe that the section on the  error-aware corrector should be revised beyond a minor change. Therefore, I am leaving my rating at “borderline accept\"", "rebuttal": "We sincerely appreciate the reviewer’s positive feedback and the careful review of our work. Below, we address each of the concerns in detail.\n \n## Q1: Confusion on error-aware corrector.\nWe sincerely apologize for the confusion caused in this section.\n \nTo clarify, the keyframes in our method are not reconstructed independently. Performing independent reconstruction for every keyframe would result in significant storage overhead. In fact, only the first frame of the entire video is reconstructed and stored with its full set of Gaussian primitives—this frame serves as the **I-frame**. **The subsequent keyframes do not involve independent reconstruction**. Instead, their primary purpose is to correct the accumulated errors introduced by keypoint motion modeling. These keyframes provide periodic corrections to ensure fidelity in the online reconstruction process. To eliminate any ambiguity, we will explicitly clarify this point in the revised version.\n \nFor **subsequent frame reconstruction** (i.e., non-I-frame reconstruction), our method reuses the Gaussian primitives from the previous frame as a reference. Specifically:\n \n- **For non-keyframes**, the modeling relies on motion estimation between the current and previous frame to select keypoints, and the scene is modeled based on the Gaussian primitives from the previous frame.\n- **For keyframes**, we also use the Gaussian primitives from the previous frame, but apply error-aware correction to update those primitives that contain modeling errors.\n \nTherefore, only the first frame is reconstructed independently and includes a Gaussian densification process. Subsequent frames do not involve independent reconstruction or densification.\n \nWe hope this explanation resolves the confusion. If there are any remaining concerns, we would be happy to discuss further. We will also revise the corresponding section in revised version to make this process more clearly stated.\n \n## Q2: Limitations in appendix.\nThank you for your valuable suggestion. We agree with the importance of presenting the limitations in the main paper. In the revised version, we will include a condensed version of the limitations discussion in the conclusion section. We also present the summarized version below in this rebuttal:\n\n**Limitations.** While our method achieves notable improvements, several limitations remain. First, the quality of the initial frame significantly impacts subsequent performance, and poor initialization may lead to error accumulation. Second, although our approach focuses on compact representation, the training efficiency of the encoding stage can be further optimized. Lastly, our method currently requires dense-view video input, which may limit its practicality; future work will explore extensions to sparse-view or monocular settings.\n \n## Q3: 3DGS compression methods in related work.\nWe appreciate the reviewer’s valuable suggestion. Discussing 3DGS compression methods in the related work section will indeed enhance the completeness of our paper.\n \nExisting 3DGS compression approaches can be broadly categorized into two types:\n- Reducing the number or precision of Gaussian parameters。\n- Exploiting spatial structural correlations to reduce the overall size of 3DGS.\n \nThe first category includes techniques such as removing unimportant Gaussians, pruning spherical harmonic (SH) coefficients, and applying vector quantization to compress the parameter representation. The second category focuses on structured predictions of unstructured 3D points, typically through multi-resolution grids or triplane-based representations, which effectively reduce the storage requirements of 3DGS.\n \nWe will incorporate a discussion of these 3DGS compression techniques in the revised version.\n \n## Q4: EWA splatting.\nWe appreciate the reviewer for the insightful suggestion. We agree that EWA splatting[E] is indeed relevant, as it provides the theoretical foundation for projecting 3D Gaussians into 2D image space using Jacobian of the affine approximation of the projective transformation. We will incorporate a discussion of EWA splatting and cite the relevant literature in the revised version to clarify the connection. \n\n[E] Zwicker M, Pfister H, Van Baar J, et al. Ewa volume splatting[C]//Proceedings Visualization, 2001. VIS'01. IEEE, 2001: 29-538.\n\n## Q5: Limited in evaluation dataset.\nWe appreciate the reviewer’s concern regarding dataset selection.\n \nIn our experiments, we use N3DV and MeetRoom, both of which are widely adopted in the online Free-Viewpoint Video (FVV) reconstruction community and consist of multi-view video sequences. In contrast, datasets captured with orbital camera setups are less commonly used in online FVV research, as they typically do not provide the temporally synchronized multi-view video inputs required for frame-by-frame training.\n \nTo further demonstrate the generalizability and effectiveness of our method, we have additionally conducted experiments on the Google Immersive dataset, which is another widely used benchmark for free-viewpoint rendering. We provide the quantitative results below, and will include them in the revised version of the paper.\n \n**Tab. R11:** Quantitative results on the Google immersive dataset.\n \n| Method     | PSNR| Size(MB) |\n|------------|--------|--------|\n| 3DGStream      | 25.18  | 8.83  |\n| Ours       | 27.24  |  0.16 |\n \nWe hope this response clarifies your concerns. Thank you again for your valuable comments and encouraging review."}]

Author rebuttal:

Meta-review: Following the author–reviewer discussion, this paper received positive reviews from all reviewers. The method demonstrates clear improvements over the previous online compression approach, QUEEN, achieving significantly better performance. I believe this constitutes a valuable contribution to the community.

However, the paper would benefit from substantial improvements in writing quality, particularly with respect to notational clarity and consistency, several examples of which are listed below. In a journal setting, such issues would likely require a comprehensive major revision. 

1. Eq (1): x -> \mathbf{x}, \mu -> \mu_i, G(x) -> G_i(x)

2. L139: \alpha_i

3. Eq (2): ‘n’ is a scalar number; i \in n can not be used here

4. L152: I would not use GT as a symbol; it may represent G \times T

5. L174: \mu_{ij}, use something other than \mu since it was used differently before

6. Eq (7): Rewriting is necessary, e.g., by using a set notation

7. L179: \Delta \mu_\Kappa^i. You have used \Kappa to denote whole indices of keypoints across all timestamps before

8. L181: What is I_j? It has not been defined until this point

9. L204: ‘sh’ may look like s \times h

10. Eq (11): ‘*’ usually means ‘convolution’

12. L213: What is an ‘attribute-specific’ update?

13. Eq (12): What is N? N is the number of all Gaussians?

14. And more...

Nonetheless, I trust that the authors will address these notational issues and improve the presentation in the final version. With these corrections in place, I recommend acceptance of this paper.
Decision: Accept (poster)