StreamSplat: Towards Online Dynamic 3D Reconstruction from Uncalibrated Video Streams
Authors: Zike Wu
Qi Yan
Xuanyu Yi
Lele Wang
Renjie Liao
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Gaussian Splatting; Monocular Video Reconstruction
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We proposed an efficient, scalable, and feed-forward framework for online dynamic 3D reconstruction.

Abstract: Real-time reconstruction of dynamic 3D scenes from uncalibrated video streams demands robust online methods that recover scene dynamics from sparse observations under strict latency and memory constraints. Yet most dynamic reconstruction methods rely on hours of per-scene optimization under full-sequence access, limiting practical deployment.
In this work, we introduce **StreamSplat**, a fully feed-forward framework that instantly transforms uncalibrated video streams of arbitrary length into dynamic 3D Gaussian Splatting (3DGS) representations in an online manner. 
It is achieved via three key technical innovations: 1) a probabilistic sampling mechanism that robustly predicts 3D Gaussians from uncalibrated inputs; 2) a bidirectional deformation field that yields reliable associations across frames and mitigates long-term error accumulation; 3) an adaptive Gaussian fusion operation that propagates persistent Gaussians while handling emerging and vanishing ones.
Extensive experiments on standard dynamic and static benchmarks demonstrate that StreamSplat achieves state-of-the-art reconstruction quality and dynamic scene modeling. Uniquely, our method supports the online reconstruction of arbitrarily long video streams with a $1200\times$ speedup over optimization-based methods. 
Our code and models are available at https://streamsplat3d.github.io/.


Reviews:
[{"review_id": "g7FabXDBxi", "reviewer": "Reviewer_YF1b", "summary": "StreamSplat introduced a feed-forward framework for reconstructing dynamic 3D scenes from uncalibrated video streams. \n\nStreamsplat work simliarly as current Dust3R style model and predict the 3D gaussian parameters with a vision transformer.\n\nStreamSplat also introduce three key technical innovation: 1. a probabilistic sampling mechanism 2. a bidirectional deformation field 3. an adaptive Gaussian fusion operation", "questions": "Please see the major weakness above. \n\n1. In the section on the Bidirectional Deformation Field, the authors describe how forward and backward deformations enhance temporal consistency. However, given that StreamSplat is evaluated on 5-frame and 8-frame interval tasks, it’s unclear why the model does not increase the number of active Gaussians to better represent accumulated motion over longer intervals.\n\n2. Since the input frames are two, why not use the pretrain model from Croco (e.g. Dust3R, Cut3R)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Strength:\n1. The paper is well-written and easy to follow.\n2. StreamSplat is feedforward and increase the speed of reconstruction.\n3. In the figure 4, StreamSpalt shows persistent gaussians across frames, which shows the potential of long-term modeling.", "weaknesses": "Major Weakness:\n\n1. Lack of Rigorous Evaluation Protocols: The evaluation does not follow established dynamic reconstruction benchmarks such as DyCheck or NVIDIA Dynamic Scene Dataset. The chosen datasets (DAVIS and YouTube-VOS) are more typical for video segmentation or interpolation, not 4D reconstruction.\n\n2. Limited Training Dataset: The paper uses a mix of static (CO3Dv2, RealEstate10K) and limited dynamic (DAVIS, YouTube-VOS) datasets for training. However, DAVIS contains only a few short clips, this raises concerns about the model’s generalization to complex  motion and real-world application.\n\n3. Dynamic reconstruction typically involves multi-view or multi-camera setups to test geometric consistency (e.g., DyCheck). StreamSplat, in contrast, evaluates on DAVIS using middle-frame interpolation and trivializes the problem.\n\nMinor Weakness:\n\n1.In Figure 1, “curent” should be corrected to “current.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g5FawxD3TC", "reviewer": "Reviewer_Soee", "summary": "The paper first points out the challenge of achieving real-time dynamic 3D reconstruction from uncalibrated video streams while maintaining the quality and functionality of offline methods. To address this, the paper proposes an online dynamic 3D reconstruction framework called StreamSplat, which integrates probabilistic 3D Gaussian encoding, a bidirectional deformation field, and adaptive Gaussian fusion for temporally coherent and efficient scene modeling. The method aims to enable scalable, feed-forward, and fully online reconstruction of arbitrarily long video streams with state-of-the-art quality and significant speed improvement over optimization-based approaches.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper tackles an underexplored yet practically important problem: real-time dynamic 3D reconstruction from uncalibrated video streams, which existing 3DGS and NeRF-based methods generally overlook due to their offline and per-scene optimization nature.\n\n* Unlike prior optimization-based dynamic 3DGS methods, StreamSplat introduces a fully feed-forward pipeline that supports online inference without requiring camera calibration or pre-computed poses, making it highly suitable for real-world deployment scenarios.\n\n* The experiments cover static and dynamic benchmarks, interpolation tasks, and zero-shot evaluation, providing convincing evidence of the model’s robustness, scalability, and temporal coherence.", "weaknesses": "* It would be helpful if the authors could clarify whether their framework is capable of predicting or estimating camera poses, given that StreamSplat operates under uncalibrated input conditions. If not, discussing potential extensions in this direction would strengthen the paper's completeness.\n\n* The paper would benefit from additional discussion or experiments on highly dynamic scenes with significant topological changes (e.g., frequent object entries and exits from the field of view). It remains unclear how robust the proposed method is under such conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "05iISwoJxm", "reviewer": "Reviewer_a3WQ", "summary": "This paper proposes **StreamSplat**, a **feed-forward framework for online dynamic 3D reconstruction from uncalibrated video streams**.  \nIt aims to replace conventional optimization-based dynamic 3DGS pipelines with a real-time, fully feed-forward alternative.\n\n**Key contributions include:**\n1. **First feed-forward dynamic 3D reconstruction framework** under an *uncalibrated* camera setting.  \n2. **Effective technical components:**\n   - *Probabilistic position sampling* for robust Gaussian position inference under geometric uncertainty.  \n   - *Bidirectional deformation fields* that jointly model forward and backward motion to ensure temporal coherence.  \n3. **High-quality results:** Both quantitative tables and qualitative visualizations show consistent improvements and strong temporal coherence across datasets.", "questions": "**Q1.** Could you provide a visualization of *multi-point tracking*? Figures 3 and the supplementary videos only illustrate a single tracked point. From my understanding, a per-pixel feed-forward network may struggle with precise point correspondence. Showing multiple tracked points (e.g., ~30) in a local neighborhood would give a fairer sense of temporal coherence.  \n\n**Q2.** Please include a **runtime breakdown**—how much time is spent in the encoder, deformation module, and rasterization, respectively? This would help assess real-time feasibility.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Impressive empirical results:** The method achieves substantial gains over prior 3DGS and NeRF-based approaches, setting a new benchmark for *online dynamic reconstruction from uncalibrated video streams*.  \n- **Reasonable and clear pipeline:** The proposed two-stage static/dynamic training scheme is well-motivated and reproducible. The combination of a strong image encoder and a dynamic decoder makes architectural sense.  \n- **Excellent efficiency:** StreamSplat attains orders-of-magnitude speedup (seconds vs. hours) compared to optimization-based methods.  \n- **Comprehensive evaluation:** The paper compares against strong baselines on multiple datasets, including both static (RE10K, CO3Dv2) and dynamic (DAVIS, YouTube-VOS) settings.", "weaknesses": "Major Weaknesses\n\n**W1.** The formulation in *line 157* appears problematic: $(u,v)$ represents pixel coordinates while the offset $o_i$ is in unit space. Their direct addition may be incorrect if the coordinate system is rectilinear. A clarification of this coordinate transformation is needed.  \n\n**W2.** Algorithm 2’s *aggregation and fusion* step is ambiguous. The operation `UPDATE` in line 228 is not clearly defined—does it simply replace $\\tilde{\\mathcal{G}}$ with$ \\mathcal{G}_{k-1}^+ $, or else?  \nAdditionally:\n- The notation $Î_{t_{k−1}→t_k}$ is unclear—what is the meaning of the arrow?  \n- The definition of $π$ (projection parameters) is missing.  \n- The term **“cached”** in line 219 feels implementation-specific and should be formalized or rephrased.  \nFinally, “rendered frames” in line 221 conflicts conceptually with “dynamic scenes” in line 292—please clarify the intended distinction.\n\nMinor Weaknesses\n\n**W3.** The frame indices in *Figure 2* appear inconsistent. Section 3.2 (line 209) describes deformation between $t=0$ and $t=1$; thus, the figure should use either $(t_n, t_{n-1})$ or $(t_0, t_1)$ for consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)