Title: ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS
Venue: NeurIPS 2025
Accepted: yes
Keywords: novel view synthesis

feed-forward 3DGS

gaussian splatting
Abstract: Feed-forward 3D Gaussian Splatting (3DGS) models have recently emerged as a promising solution for novel view synthesis, enabling one-pass inference without the need for per-scene 3DGS optimization. However, their scalability is fundamentally constrained by the limited capacity of their encoders, leading to degraded performance or excessive memory consumption as the number of input views increases. In this work, we analyze feed-forward 3DGS frameworks through the lens of the Information Bottleneck principle and introduce ZPressor, a lightweight architecture-agnostic module that enables efficient compression of multi-view inputs into a compact latent state $Z$ that retains essential scene information while discarding redundancy. Concretely, ZPressor enables existing feed-forward 3DGS models to scale to over 100 input views at 480P resolution on an 80GB GPU, by partitioning the views into anchor and support sets and using cross attention to compress the information from the support views into anchor views, forming the compressed latent state $Z$. We show that integrating ZPressor into several state-of-the-art feed-forward 3DGS models consistently improves performance under moderate input views and enhances robustness under dense view settings on two large-scale benchmarks DL3DV-10K and RealEstate10K.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper revisits feed-forward 3DGS prediction from an information bottleneck perspective and introduces ZPressor, a plug-and-play module that can improve the model performance and reduce inference/training memory. The method includes group-based compression with selected anchor views. The authors verified the proposed ZPressor on multiple base models and datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper revisits feed-forward 3DGS reconstruction from an information bottleneck perspective, which recalls me of the discussion why do we need inductive bias for 3D. And ZPressor is a kind of inductive bias of compression. This makes a lot of sense to me, as the essence of 3D reconstruction is just compression (of the repetitive and redundant information).\n- Based the results shown in the paper, the proposed ZPressor is effective and efficient, which improves performance and reduces computation at the same time.\n- The authors verified the effectiveness of ZPressor on multiple base model and datasets.\n\n\nWeaknesses:\n- However, at the same time, I do have some concerns about the claims and performance patterns of the baseline models of the paper. And I would like to get more feedback from the authors. Please see details below.\n- First, the performance in Table 1 and Table 2 is weird to me. The results show that **with more input views, the performance drop**. The authors have also mentioned it multiple times in the writing, e.g., Line 4 and Line 31. However, this is counter-intuitive. Personally, I have never observed any performance pattern like this in my own experiments. And this is not the performance pattern of every paper that I read. For example, in Long-LRM [1], MegaSynth [2], the performance improved with more input views. I encourage the authors to explain a bit more on this.\n- Second, my conjecture is that, the weird performance pattern is because of a biased base model choice. Concretely, the base models, pixelSplat, DepthSplat and MVSplat, are based on pairwise computation, e.g., cost volume calculation; while the two paper I mentioned above adopts global attentions on all views, which has less inductive bias in model architecture design. It would be beneficial to explore whether this assumption is true by adapting ZPressor on models with global attention.\n- Third, in Table 2 and 3, after adding the ZPressor module, the performance still drop with more input images. Thus, this experimental result cannot support the claim of the author on solving the ``degraded performance'' problem.\n- Forth, the authors mentioned *limited capacity of the encoder* multiple times. However, I am not able to understand this claim. What exact ``capacity'' the authors were referring to here?\n- Finally, the visualization examples in the supplementary videos is in very low quality to me. In general, it is weird that the performance is so bad and the renderings are so blurry.\n\n[1] Chen, Ziwen et al. “Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splats.” ArXiv 2024. \n[2] Jiang, Hanwen et al. “MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data.” CVPR 2025.\n\nQUESTIONS:\nIn general, I believe this is an interesting paper, which improving the feed-forward 3DGS reconstruction by revisiting the information bottleneck theory. However, based on the current performance patterns, biased baseline choice, and the unclear claims, I will give a boarderline reject. I encourage the authors to include more analysis regarding my questions listed in the weakness section. And I would be happy to increase my score if it is reasonable to me.\n\nLIMITATIONS:\nPlease see the sections above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ZPressor, a lightweight, architecture-agnostic module designed to enhance the scalability of feed-forward 3D Gaussian Splatting (3DGS) models, which are prominent for real-time novel view synthesis. Traditional 3DGS approaches struggle to efficiently process a large number of input views due to their limited encoder capacity, resulting in degraded performance and high memory usage as the number of views increases. The authors analyze these limitations through the lens of the Information Bottleneck (IB) principle, which emphasizes the preservation of task-relevant information while discarding redundancy. Inspired by this, ZPressor employs a strategy where input views are divided into anchor views and support views. Cross-attention mechanisms fuse the support view information into anchor views, compressing the dense multi-view data into a compact latent representation Z that retains essential scene details. By integrating ZPressor into existing feed-forward 3DGS models, they demonstrate significant improvements: models can handle over 100 dense input views at high resolution (480P) on standard GPUs without performance drops or excessive memory consumption. Their experimental validation on large-scale benchmarks (like DL3DV-10K and RealEstate-10K) shows consistent enhancements in quality and efficiency, making 3D scene reconstruction more practical for real-world applications.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- This paper is well-written, motivations and significance of the task are clearly described in the paper.\n- The use of the Information Bottleneck principle provides a solid theoretical foundation for addressing the limitations of existing 3DGS models, offering insights into why they struggle with dense views and how to mitigate these issues.\n- ZPressor is designed to be architecture-agnostic, allowing it to be seamlessly integrated into various existing feed-forward 3DGS models, which broadens its applicability.\n- Extensive experiments on large-scale benchmarks like DL3DV-10K and RealEstate-10K demonstrate consistent performance improvements, showing that the approach is effective both qualitatively and quantitatively.\n\n**Weaknesses:**\n- The authors acknowledge that ZPressor may be less effective when dealing with extremely dense views (e.g., thousands of views), where even the compressed representations may become computationally challenging to handle.\n- Incorporating cross-attention-based compression modules introduces extra complexity into the pipeline, which may affect training stability, implementation, or inference speed, although it is designed to be lightweight.\n\nQUESTIONS:\n- In the experiments, how do you split input views and target novel views?\n- Do you select the same target novel views for different numbers of input views?\n- For anchor view selection, did you try other strategies? e.g., k-means clustering.\n\nLIMITATIONS:\nLimitations are mentioned in weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author tackles the view scalability problem of the modern feedforward based 3DGS method. They find that with increasing number of views,  methods like MVSplat, pixelSplat, DepthSplat struggle with reconstruction quality. The author attributes such degradation to the \"limited capacity of the encoder\". Hence, motivated by information bottleneck theory, they propose a plug-and-play model ZPressor that dramatically reduce the input features from image encoder and generate enhanced visual quality.\n\nSTRENGTHS AND WEAKNESSES:\nGood part:\n- the paper is well written and easy to follow with abundant visual results.\n- the identified problem is a true pain for sparse view feedforward 3DGS prediction and leveraging cross attention for information fusion is generally a plausible solution\n- the module as a feature digester does reduce the hardware overhead of previous methods and achieves remarkable enhancement.\n\nWeaknesses:\n- the key claim of the paper is that \"limited capacity of encoder\" as the main drawback of previous methods. Yet for all three methods they have modified are SINGLE VIEW image encoder which means that no information has been exchanged between views during the encoder stage. **That's to say, the key reason of bad performance of previous methods is not the encoder**\n- the key reason of poor performance of those methods are very simple actually: they are all trained on very sparse views typically 2-4, with depthSplat at most 6 views. Yet the evaluated views of ZPressor is 8~36, which falls far outside the distribution of training data of those methods. **That's to say: the baselines are simply too weak, and are not expected to handle the dense view scenario**.\n- The author use cross attention to fuse information from support views to anchor views, yet the ablation study of tab 3 shows very limited enhancement from anchor-fusion to support fusion. Considering the proposed method is trained with 6+6 views, **such enhancement could simply be attributed to the model's getting familiar with more views, instead of learning extra information from support views.**\n- It is not supposed to claim a method as `plug-and-play` if you integrate your module and still need to train the integrated new model from scratch.\n- IB theory is a little bit out-of-place here.\n\nQUESTIONS:\n- confusion definition of \"variational rendering loss\" at the end of section  3.\n- what do you mean by \"information overload\" in section 4.2? intuitively, those information from different views should be redundant, what does overload mean here and why it leads to performance degradation then?\n- since the current baseline is too weak, plz **demonstrate your module on a feedforward 3DGS methods that natively support input views more than 6**. Recent works, for example anysplat, FLARE, InstantSplat, GPSGaussian++, and any other method are acceptable. Or you can simply choose to fine-tune MVSplat/PixelSplat/DepthSplat with multi views as the baseline. Don't have to stick with single 80GB GPU constraints.\n\nLIMITATIONS:\nIt relies on known cam pose for selecting anchor views. Yet today's sota work could also works on poes-free inputs (noposplat, anysplat, PF3plat). How your method could help in this scenario ?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGaussian Splatting reconstruction has recently become a popular approach for feed-forawrd reconstruction. However, many approaches reconstruct the scene by predicting Gaussian Splats from every image, which becomes memory intensive when the number of input frames is high. This paper addresses this issue with an idea inspired by the information bottleneck principle: effectively compressing a long sequence of frames into a latent representation that operates on a smaller set of keyframes.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- Compression for Gaussian Splatting representations is an important topic that is currently undexplored. \n- The paper is well written and good to follow. The notation is clear and precise. \n- The qualitative and quantitative results show that the method is effective at compression and thus can eliminate some floater artefacts. This, in turn, results in better reconstruction scores. \n\n## Weaknesses\nThere is a disconnect between the theoretical motivation through the information bottleneck and the actual implementation. Line 202 reads “we optimize the feed-forward 3DGS with ZPressor by the variational rendering loss, e.g., MSE and LPIPS.”. In my understanding, MSE is not a variational loss. It can be part of a variational loss if used as a reconstruction loss in a VAE-like formulation, but this would require a KL term. Similarly, while the framework is motivated via Eq. (3), but since there is no explicit term measuring the compression score, the hyperparameter $\\beta$ is unused, and no balance between compression and reconstruction is learned. It is, in a sense, pre-defined by the architecture choice.\nThe IB connection does not really help in designing, training, or improving the model .\n\nThe choice of anchor frames is purely based on the camera position. Typically, in SLAM-like applications, the view angle is taken into account when identifying new keyframes. This could be done in many ways, for example by computing the overlap in view frustrums. This clustering is chosen ad-hoc, and likely needs to be manually tuned. Can the IB formulation be used to learn the anchor frame selection strategy?\n\nMV Splat associates a confidence with every Gaussian. A simple baseline would use the confidence to prune Gaussians for compression. Would that reduce the floater artefacts present in the original MVSplat reconstructions?\n\nQUESTIONS:\nEq 6: I think the function operating on $Q, K, V$ is attention and not Cross-attn. What makes this cross attention is the source of $K$ and $V$.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the scalability problem of feed-forward 3DGS, which is limited by the encoder's capacity, leading to performance degradation or high memory use as the number of input views grows. The authors introduce ZPressor, a lightweight, architecture-agnostic module that efficiently compresses multi-view inputs into a compact latent state. This compression retains essential scene details while removing redundancies.\n\nZPressor splits the views into anchor and support sets. Anchor views serve as compressed states, and support views’ information is compressed into these anchors using cross-attention. Farthest point sampling is used to select anchor views, maximizing coverage with fewer views. Remaining views are assigned to the nearest anchor based on camera distance, and their features are fused into the anchors via cross-attention blocks.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n## Effectiveness:\nZPressor enables feed-forward 3DGS models to scale to over 100 input views at 480P resolution on an 80GB GPU. The results demonstrate that integrating ZPressor consistently improves the performance of baseline models, especially with a moderate number of input views (e.g., 12 views). It helps maintain reasonable accuracy and computational cost even with dense inputs (e.g., 36 views), where original models would typically degrade or run out of memory.\n\n## Versatility:\nZPressor is architecture-agnostic, meaning it can be easily integrated into various feed-forward 3DGS models. The module improves performance under moderate input views and enhances robustness with dense view settings. It was successfully integrated into several state-of-the-art models (like PixelSplat, MVSplat, and DepthSplat) and tested on large-scale benchmarks (DL3DV-10K and RealEstate10K), consistently showing improvements.\n\n## Insightful Experiments:\nThe paper provides valuable insight, particularly in Tables 1 and 2, where the performance of the state-of-the-art models drop as input views become denser. This highlights the importance of the proposed method in improving scalability.\n\n# Weaknesses\n\n## Lack of Pose-Free Settings:\nThe paper does not explore pose-free settings such as NoPoSplat, FLARE, or AnySplat, which could provide additional context to the proposed method’s versatility.\n\n## Unexplained Performance Drop with Too Many Anchor Views:\nFigure 6, which analyzes the bottleneck constraint, introduces an interesting idea. However, it doesn’t fully explain why performance drops when there are too many anchor views. While it's clear that too few anchor views lead to performance degradation due to over-compression, the impact of having too many anchor views is less clear. A more detailed explanation of this phenomenon would be beneficial.\n\n##  Adaptive Selection of Anchor Views:\nThe method could benefit from a more flexible approach, allowing for adaptive selection of the number of anchor views based on input conditions. This could help optimize the performance further in different scenarios.\n\nQUESTIONS:\nPose-Free Settings:\nDemonstrating the effectiveness of ZPressor in such configurations could strengthen the case for its versatility and robustness.\n\nClarification of Figure 6:\nA deeper analysis of this phenomenon would help clarify the trade-off involved.\n\nAdaptive Selection of Anchor Views:\nIf you could demonstrate how this adaptation might work, it could further enhance the practical utility of ZPressor.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}