Title: Temporal Smoothness-Aware Rate-Distortion Optimized 4D Gaussian Splatting
Venue: NeurIPS 2025
Accepted: yes
Keywords: compression

video

4D

gaussian splatting
Abstract: Dynamic 4D Gaussian Splatting (4DGS) effectively extends the high-speed rendering capabilities of 3D Gaussian Splatting (3DGS) to represent volumetric videos. However, the large number of Gaussians, substantial temporal redundancies, and especially the absence of an entropy-aware compression framework result in large storage requirements. Consequently, this poses significant challenges for practical deployment, efficient edge-device processing, and data transmission.
In this paper, we introduce a novel end-to-end RD-optimized compression framework tailored for 4DGS, aiming to enable flexible, high-fidelity rendering across varied computational platforms.
Leveraging Fully Explicit Dynamic Gaussian Splatting (Ex4DGS), one of the state-of-the-art 4DGS methods, as our baseline, we start from the existing 3DGS compression methods for compatibility while effectively addressing additional challenges introduced by the temporal axis. In particular, instead of storing motion trajectories independently per point, we employ a wavelet transform to reflect the real-world smoothness prior, significantly enhancing storage efficiency.
This approach yields significantly improved compression ratios and provides a user-controlled balance between compression efficiency and rendering quality. Extensive experiments demonstrate the effectiveness of our method, achieving up to 91$\times$ compression compared to the original Ex4DGS model while maintaining high visual fidelity. These results highlight the applicability of our framework for real-time dynamic scene rendering in diverse scenarios, from resource-constrained edge devices to high-performance environments. The source code is available at https://github.com/HyeongminLEE/RD4DGS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a rate-distortion optimized compression framework for 4D Gaussian Splatting. Built on Ex4DGS, the method introduces wavelet-based trajectory compression and adaptive quantization to significantly reduce model size while preserving rendering quality.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Strong empirical performance: The proposed method achieves up to 98.9% reduction in storage at Level 1 compression with competitive PSNR and SSIM values across both N3V and Technicolor datasets.\n2. Effective trajectory modeling using wavelet transform: The paper leverages temporal smoothness priors via a Haar wavelet transform, which improves compression efficiency while preserving fidelity.\n3. Clarity and completeness: The paper is well written, structured, and covers a comprehensive set of ablations and comparisons, including against NeRF-based and 4DGS-based baselines.\n\nWeaknesses:\n1. Prior art on RD optimization for splatting may be under-acknowledged: While the paper claims novelty in RD-optimized 4DGS, similar ideas (e.g., RD cost applied to SH pruning in Compact3DGS, RD3DGS, or Light4GS) have appeared in recent papers. The authors should clarify the distinction from prior RD-based compression methods, particularly for dynamic scenes.\n2. Ablation study (Table 1) suggests limited contribution from some proposed modules. This raises the question of whether much of the gain is from the baseline design rather than the proposed enhancements.\n3. Dynamic points remain a bottleneck: Dynamic Gaussians still dominate the storage cost at higher quality levels. Although some improvement is shown using wavelets, more sophisticated modeling (e.g., motion basis or predictive coding) could further help.\n\nQUESTIONS:\n1. Clarification on RD novelty: How does your RD framework differ in principle and implementation from Compact3DGS, RD3DGS, and recent 4DGS works like Light4GS or QUEEN that also use entropy-aware coding and SH pruning? A table or paragraph contrasting them would help.\n2. Ablation on baseline vs. proposed methods: To what extent does the wavelet transform or adaptive quantization contribute to gains over just Ex4DGS + pruning + VQ? Could you show RD curves or numbers with each module ablated?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method for reducing model size in 4D Gaussian Splatting and evaluates its performance. To achieve model compression, the authors apply aggressive pruning to the 3D Gaussians and Spherical Harmonics (SH) coefficients, use vector quantization (VQ), and represent positional changes of the 3D Gaussians using wavelet transforms. In particular, for representing 3D Gaussian positional changes with wavelet transforms, only the low-frequency components of the wavelet coefficients are retained, leveraging the inherent smoothness of spatio-temporal motion in videos.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**\n1. The paper lacks sufficient variation in the choice of comparison methods. While the experiments include comparisons with standard 4D Gaussian Splatting approaches such as 4D Gaussians[1] and Ex4DGS[2], performance and model size should also be evaluated against other lightweight methods[3][4]. For instance, LGS[5] could serve as a relevant baseline, as its implementation is publicly available.\n2. The Light4GS[3] paper also evaluates its method using the same Neural 3D Video (N3V) dataset as this paper (excluding the “Coffee Martini” sequence from their evaluation). Light4GS reports a PSNR of 31.48 dB with a model size of 3.77 MB. In contrast, according to Table 2 of this paper, the proposed method (L1) achieves a PSNR of 27.04 dB with a model size of 1.26 MB. Although the proposed method achieves a more compact model, the rendering performance degrades significantly. This performance drop appears disproportionately large relative to the reduction in model size.\n\n**Clarity**\n1. The paper is well written, and the proposed methodology is clearly presented, making it easy for readers to understand.\n2. The authors clearly articulate the rationale behind the design choices of the proposed model structure and justify the necessity of its architecture.\n\n**Significance**\n1. Model size reduction in 3D Gaussian Splatting is a critical issue in the field, and this paper specifically addresses this challenge in the context of 4D Gaussian Splatting. The proposed ideas are straightforward and practical, making them easy to implement.\n\n**Originality**\n1. The wavelet transform-based compression proposed in this paper is novel and has not been explored in existing methods.\n\n[1] Yang, Z., Yang, H., Pan, Z., & Zhang, L. (2023). Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642.\n\n[2] Lee, J., Won, C., Jung, H., Bae, I., & Jeon, H. G. (2024). Fully explicit dynamic gaussian splatting. Advances in Neural Information Processing Systems, 37, 5384-5409.\n\n[3] Liu, M., Yang, Q., Huang, H., Huang, W., Yuan, Z., Li, Z., & Xu, Y. (2025). Light4gs: Lightweight compact 4d gaussian splatting generation via context model. arXiv preprint arXiv:2503.13948.\n\n[4] Yuan, Y., Shen, Q., Yang, X., & Wang, X. (2025). 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering. arXiv preprint arXiv:2503.16422.\n\n[5] Liu, H., Liu, Y., Li, C., Li, W., & Yuan, Y. (2024, October). Lgs: A light-weight 4d gaussian splatting for efficient surgical scene reconstruction. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 660-670). Cham: Springer Nature Switzerland.\n\nQUESTIONS:\n1. As noted in the “Strengths and Weaknesses” section, the superiority of the proposed method over Light4GS remains unclear. The authors should discuss the advantages of their method in comparison to Light4GS, referencing its experimental results.\n2. Why were other lightweight methods[3][4][5] not included as comparison baselines in the experiments?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a 4D Gaussians compression method with flexible control of quality-cost tradeoff (rate-distortion) and a temporally smoothness formulation of dynamic positions.\n\nThe method is an offline compression system given the input of already trained 4DGS representations (Ex4DGS in this case). Then the proposed method will prune the attributes and apply existing quantization technique ECVQ to compress them. For the dynamic positions, the authors propose to use a compact truncated wavelet representation to represent the dynamic trajectory. There are some detailed choices on not compressing certain attributes (rotations for example) for fidelity. The final optimization scheme considers the RD tradeoff with a rate term along with a fidelity (image reconstruction) term with some regularization from Ex4DGS.\n\nThe paper presents some good compression rates and quantitative/qualitative results and comparisons to support their design choices.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- The proposed method overall makes sense: using a low-frequency wavelet representation to enforce some spatial smoothness for dynamic positions, incorporating the RD optimized formulation from work such as ECVQ into 4DGS compression.\n- Good compression rates based on the results. \n- The paper is overall well-written.\n\n\nWeakness:\n- Some parts of the method are not very clear (see item 1 to 4 in the question section).\n- There are still some questions on evaluation metrics and results (see item 5 to 9 in the question section).\n- The novelty can be limited as the paper seems to be applying ECVQ techniques on Ex4DGS instead of 3DGS. The wavelet formulation of position trajectory is also not surprising (and not very well validated — please add discussion and experiments). It would also be great to highlight the difference to ECVQ in the method section.\n\nQUESTIONS:\nSome parts of the method are not very clear.\n\n\n1. The loss function of L_{rate} is not clearly defined. I do see Eq. 4 and 5. But It would be great to clearly define the loss function. Furthermore, on this rate loss term, what is the difference compared to ECVQ? From the current writing, I feel the two are the same or highly similar. It would be great to highlight the difference (and clearly define the loss function). Furthermore, please also clarify in the method section (3.2, 3.3), the difference to the ECVQ paper.\n\n\n2. Line 162: I assume the sg() function is the stop gradient operation. Please confirm and clearly define every term/function in the method section.\n\n\n3. The regularization term (In Eq. 8), is it exactly the same as the one in Ex4DGS? Please confirm. If not, please highlight the difference as well.\n\n\n4. The design choice of truncated Wavelet for smooth dynamic positions, though sounding reasonable, is not very well validated.\n- (a) Firstly, let me confirm the actual computation process, with the cubic Hermite interpolation of the keyframe positions from Ex4DGS, the proposed system would firstly evaluate those positions (via interpolation) to obtain the \\mu_{d} in Line 190 for every frame, and then apply the wavelet conversion. Is that right?\n- (b) Could we use DCT or Fourier transform as well? What is the actual advantage of Wavelet transformation? It would be great to explain and provide some empirical analysis.\n- (c) In Eq. 7, the method would remove the high-frequent components of the wavelet and enforce some smooth output on F_{masked}. It would be great to see the actual effects by varying this frequency cut hyperparameter as an ablation study.\n- (d) With this low-frequency “projection”, we can get a smooth output, but the method faces another risk of losing capacity to handle fast motion. Will the method have this limitation? Any results on fast motion?\n\n\n\n\nSome questions on evaluation metrics and results:\n\n\n5. All the model sizes reported in this paper: I assume they are the model size of the full sequence, not per-frame, is that correct? Please confirm. It would be great to write it clearly. For sequences of different lengths, it would be great to report per-frame model size.\n\n\n6. The ablation in Fig. 3 for wavelet formulation of dynamic positions: what is the case of “without transform”? Directly storing \\mu_{d} for all frames, for keyframes only, or applied ECVQ? Currently it’s not clear how to interpret the results.\n\n\n7. Fig. 1 and 2 shows the RD tradeoffs. Even the highest quality case (L6) still presents some distance to the uncompressed case (upper bound): sometimes >2 in PSNR difference. I do see the high compression rate (L1-6 are mostly at the left side of the graphs). Can we further improve quality (L > 6 for example) and trade in more model size? What is the limit of reconstruction fidelity? The current results appear to have some inevitable quality loss.\n\n\n8. What is the runtime of the proposed system?\n\n\n9. Comparisons:\n- (a) Visual comparisons. There are only comparisons to the upper bound case Ex4DGS and L1/L6 of the proposed method. Can we see visual comparisons among the existing methods?\n- (b) It seems that Tab 2 could add recent work on dynamic 3D scene compression such as TeTriRF (Wu, Minye, et al. \"Tetrirf: Temporal tri-plane radiance fields for efficient free-viewpoint video.\" CVPR 2024) and QUEEN (Girish, Sharath, et al. \"Queen: Quantized efficient encoding of dynamic gaussians for streaming free-viewpoint videos.\" NeurIPS 24). It seems that TeTriRF might have a similar level of performance on compression (but please validate this with results) and QUEEN with higher visual quality. It would be great to add them and provide some analysis/discussion.\n- (c) It would be great to provide video results, as the method works on 4DGS / 3D video compression. Static evaluation of per-frame image quality does not fully represent the visual fidelity (for videos).\n\n\n10. Novelty\n - (a) A concern is that this proposed method is very close to the case of applying ECVQ techniques on Ex4DGS with an updated wavelet-formulation of positions. I think it is a good effort nevertheless. But it is important to highlight the actual difference and improvement\n - (b) Furthermore, the system is built to compress Ex4DGS, which is one of the current 4DGS representations. Will it work on other 4DGS representations (polynomial based, or the one in “FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstruction”)?\n\nLIMITATIONS:\nI would hope the authors to address the potential limitations in the following aspects:\n- Fidelity loss (in item 7 in “Questions” section above)\n- The difference and novelty based on ECVQ and Ex4DGS seem limited (in item 10 in “Questions” section above)\n- Runtime (no numbers provided yet)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}