Title: ReCon-GS: Continuum-Preserved Gaussian Streaming for Fast and Compact Reconstruction of Dynamic Scenes
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Gaussian Splatting

Streaming Free-viewpoint Video Reconstruction

Novel View Synthesis
Abstract: To address these challenges, we propose the Reconfigurable Continuum Gaussian Stream, dubbed ReCon-GS, a novel storage-aware framework that enables high-fidelity online dynamic scene reconstruction and real-time rendering. Specifically, we dynamically allocate multi-level Anchor Gaussians in a density-adaptive fashion to capture inter-frame geometric deformations, thereby decomposing scene motion into compact coarse-to-fine representations. Then, we design a dynamic hierarchy reconfiguration strategy that preserves localized motion expressiveness through on-demand anchor re-hierarchization,  while ensuring temporal consistency through intra-hierarchical deformation inheritance that confines transformation priors to their respective hierarchy levels. Furthermore, we introduce a storage-aware optimization mechanism that flexibly adjusts the density of Anchor Gaussians at different hierarchy levels, enabling a controllable trade-off between reconstruction fidelity and memory usage. Extensive experiments on three widely used datasets demonstrate that, compared to state‐of‐the‐art methods, ReCon-GS improves training efficiency by approximately 15% and achieves superior FVV synthesis quality with enhanced robustness and stability. Moreover, at equivalent rendering quality, ReCon-GS slashes memory requirements by over 50% compared to leading state‑of‑the‑art methods. Code is avaliable at: https://github.com/jyfu-vcl/ReCon-GS/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ReCon-GS, a method that dynamically assigns multi-level anchor Gaussians to capture inter-frame geometric deformation. This approach reduces storage requirements while maintaining high reconstruction quality. Experiments on multiple datasets shows this method achieves better reconstruction accuracy and storage efficiency than current state-of-the-art online methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n - The idea of coarse-to-fine multi-level anchors is well-motivated and aligns with real-world motion patterns, leading to reduced storage overhead.\n\n - The experiments are thorough, and the proposed method outperforms existing state-of-the-art online approaches.\n\nWeaknesses\n\n - The final multi-level anchor modeling lacks visualization, which would help justify its effectiveness and spatial structure.\n\n - The comparisons in the offline setting omit some recent baselines.\n\nQUESTIONS:\n- Could the authors visualize the spatial extent and assignment of anchors at each level? This would provide intuitive evidence that the multi-level scheme effectively captures motion in a coarse-to-fine manner.\n\n - How does the proposed method compare with recent offline baselines such as STG[1] and SplineGS[2]? If ReCon-GS not outperforms these, please consider adding these to Tables 1 and 7. If the proposed online method outperforms these offline methods, a more detailed explanation of why online is better would strengthen the paper. Notably, the results reported for STG and 4DGS on N3DV seem weaker than those in the SplineGS paper, clarification would be helpful.\n\n - Some tables lack proper units for storage and training time, which may hinder reproducibility and clarity.\n\n - Would replacing the MLP with an explicit trajectory representation (e.g., as in SplineGS) further improve performance or interpretability?\n\n[1] Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis. CVPR 2024\n\n[2] SplineGS: Learning Smooth Trajectories in Gaussian Splatting for Dynamic Scene Reconstruction. ICLR 2025\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents ReCon-GS, a novel framework for real-time free-viewpoint video (FVV) reconstruction that balances high rendering fidelity with memory efficiency. It introduces an Adaptive Hierarchical Motion Representation using Anchor Gaussians to model scene dynamics in a coarse-to-fine manner. A Dynamic Hierarchy Reconfiguration mechanism addresses anchor drift while preserving motion coherence via intra-level deformation inheritance. Additionally, a storage-aware optimization strategy enables flexible trade-offs between quality and memory. Experiments show that ReCon-GS outperforms prior streaming methods with faster training and lower memory usage.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The proposed multi-level anchor-driven deformation model captures scene dynamics in a coarse-to-fine manner, providing an efficient and compact motion representation.\n2. The reconfiguration strategy helps mitigate anchor drift and improves temporal coherence.\n3. The ablation study and per-scene result is detailed.\n\nWeakness:\n1. While the method shows improvement in streaming settings, it underperforms compared to offline methods. In some cases, it requires higher storage, longer training times, and achieves lower reconstruction quality.\n2. Some details are not clear to me: how often the dynamic hierarchy reconfiguration is triggered during training and whether the framework is sensitive to this parameter, A clearer formulation explain how Anchor Gaussians quantitatively govern or parameterize general Gaussian deformation. In Table 3: It is surprising that removing the Hierarchical Motion Representation leads to lower storage, which I think is proposed to improve efficiency and compactness.\n3. The method relies on freezing appearance attributes initialized from the first frame during the first training phase. This introduces sensitivity to initial conditions when the first frame poorly captures the scene. Do you have examples show that.\nI will consider raise rating if my above weakness point are answered.\n\nQUESTIONS:\nMost of my questions have been raised in the Weaknesses section, and also is there a explicit motion relation between anchor gaussians across different level?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

comments: SUMMARY:
This paper proposes a multi-level anchor Gaussian design for real-time free-viewpoint video reconstruction. To support the hierarchical structure, the method also decomposes rigid transformations in a hierarchical manner. Additionally, the Gaussian hierarchy is reconfigured during training to better capture motion. The proposed approach achieves improved performance while offering faster training and reduced storage requirements.

STRENGTHS AND WEAKNESSES:
### Strengths:
- The writing is clear and easy to follow. The methodology figure is well-designed and effectively conveys the main idea, allowing readers to grasp it quickly.
- The method uses less training cost to get better performance.

### Weaknesses:
- The hierarchical idea in this paper is similar with ScaffoldGS [https://arxiv.org/pdf/2312.00109] as well as the level-of-detail concept in OctreeGS [https://arxiv.org/pdf/2403.17898], with the addition of rigid transformations. As it currently stands, the contribution appears to be more of a technical extension. Further clarification is needed to better highlight the novelty of the proposed approach.
- Only Table 1 reports LPIPS metrics. It would be beneficial to include perceptual metrics in other experiments as well to enable a more comprehensive comparison.
- This is a minor question: does the reported training time include the first-frame reconstruction? According to the paper, this step requires 10,000 to 15,000 iterations, so it seems unlikely to be completed within 10 seconds. Additionally, how many Gaussian primitives are generated in total?

QUESTIONS:
Please kindly refer to the "Weaknesses".

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author proposed a hierarchy-reconfigurable variant of HiCoM. To accomplish such capability of reconfiguration, they repropose a new set of anchor point Gaussians per fixed frame span. To further make sure a smooth transition after hierarchy reconfiguration, they derive the new anchor's deformation field with an inheritance from the previous hierarchy setup.\n\nSTRENGTHS AND WEAKNESSES:\nGood:\n- it works great with best performance compared with other sota works. Also baseline works are appropriately selected.\n- I do like Figure 6 in appendix which demonstrate the effectiveness of the reconfiguration and would encourage the author to put it in the front sections. Also it will be more convincing to eval the proposed method on longer dataset than those 300s datasets.\n\nWeakness:\n- grid based anchor point sampling is BORING.\n- **the research innovation of the work is very limited.** The main difference compared with previous HiCom is merely this reconfiguration setup, which is implemented in a naive method: repropose a new set of anchors. The inheritance seems to be unnecessary. \n- still it is a deformation-based method. Just like any other deformation-based dynamic modeling, it cannot efficiently encode highly dynamic scenes. \n\nothers:\n- overall its a well-written paper with abundant visualization. yet readers with little background for HiCom might get lost.\n\nQUESTIONS:\n- not quite sure the meaning of \"Continuum\" in the title\n- since you always have a stage 1 for training the deformation field after a new frame comes, why bother when a reconfiguration occurs? or does such \"inheritance\" based deformation field initialization effort help the stage 1 training of the deformation field? Honestly it seems to be marginally helpful. \n- Densification is per-frame or per-reconfiguration?\n- try eval on Technicolor and longer sequence.\n- there actually should be some extra overhead for repetitive anchor reconfiguration: record the index of those anchor points at different timestamp requires extra storage. So how your method achieves a more compact representation than HiCoM?\n\nLIMITATIONS:\nas mentioned above, it is still a deformation-based method, simply a better way (easier-to-train actually) to represent the gaussian deformation across frames. Just like any other deformation-based dynamic modeling, it cannot efficiently encode highly dynamic scenes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}