Title: TRIM: Scalable 3D Gaussian Diffusion Inference with Temporal and Spatial Trimming
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Generation

Gaussian Splatting

Stable Diffusion

Inference-Time Scaling
Abstract: Recent advances in 3D Gaussian diffusion models suffer from time-intensive denoising and post-denoising processing due to the massive number of Gaussian primitives, resulting in slow generation and limited scalability along sampling trajectories.
To improve the efficiency of 3D diffusion models, 
we propose $\textbf{TRIM}$ ($\textbf{T}$rajectory $\textbf{R}$eduction and $\textbf{I}$nstance $\textbf{M}$ask denoising), a post-training approach that incorporates both temporal and spatial trimming strategies, to accelerate inference without compromising output quality while supporting the inference-time scaling for Gaussian diffusion models.
Instead of scaling denoising trajectories in a costly end-to-end manner, we develop a lightweight selector model to evaluate latent Gaussian primitives derived from multiple sampled noises, enabling early trajectory reduction by selecting candidates with high-quality potential.
Furthermore, we introduce instance mask denoising to prune learnable Gaussian primitives by filtering out redundant background regions, reducing inference computation at each denoising step.
Extensive experiments and analysis demonstrate that TRIM significantly improves both the efficiency and quality of 3D generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a training-free, model-agnostic framework to enhance the efficiency and quality of 3D diffusion models, through two key components: Trajectory Reduction and Instance Mask Denoising.\n\nTrajectory Reduction involves sampling multiple inference trajectories but fully denoising only those that are likely to yield high-quality results. To enable this, the authors train a latent selector on a synthesized dataset to predict latent quality. This selector approximates the relationship between intermediate latent representations (latent splats) and final rendered image scores.\n\nInstance Mask Denoising focuses on reducing computation by identifying background regions and progressively masking them out during the denoising process, thereby minimizing redundant token processing in the transformer.\n\nThe proposed method significantly improves the efficiency of text-to-3D and image-to-3D generation while maintaining or enhancing output quality.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n* **S1:** The motivation to discard low-quality samples early in the diffusion process is well-founded and aligns effectively with the goal of improving computational efficiency.  \n* **S2:** The Spatial Trimming Scheme offers a practical way to reduce redundant computation by concentrating on foreground regions, which is a sensible strategy.  \n* **S3:** The method demonstrates consistent improvements in both efficiency and generation quality across several benchmarks, supporting its practical value.\n\n## Weaknesses and Suggestions for Improvement\n* **W1:** The paper frequently describes the method as training-free (e.g., lines 5, 31, 51), but this characterization is somewhat misleading since a latent selector is trained to guide trajectory pruning. It would strengthen the paper to acknowledge this training component more transparently and clarify its role.  \n* **W2:** The explanation of the latent selector’s training process is occasionally unclear. For instance, lines 115–120 describe a training approach that the authors later state is not used, without sufficient explanation for its inclusion. Providing clarity here would help reduce ambiguity for readers.  \n* **W3:** Equation (1) is introduced but does not appear to be used in the final method, which makes its inclusion somewhat confusing. Meanwhile, Equation (2) is employed for training the latent selector, but key details—such as how the timestep \\( t \\) is sampled—are missing. Clarifying whether the selector is trained across all timesteps or at a fixed one would improve both the interpretability and reproducibility of the method.  \n* **W4:** Pruning all but one trajectory during inference may limit output diversity, as it commits early to a single generation path. Addressing how this approach impacts diversity, particularly in multi-modal generation tasks, would be valuable.  \n* **W5:** Some baseline results reported differ noticeably from those in the original papers, without any explanation or discussion. For example, in Table 2, the DiffSplat baseline shows PSNR, SSIM, and LPIPS scores of 16.20, 0.79, and 0.19, respectively, while the original DiffSplat paper reports 22.91, 0.892, and 0.107 for these metrics. Providing clarification on how baseline results were obtained and discussing these discrepancies would enhance the credibility of the comparisons.  \n* **W6:** The ablation study focuses primarily on variations in the latent selector architecture. It would be beneficial to also analyze the impact of individual trimming components—such as the Temporal Trimming Scheme, Progressive Mask Expansion Scheduler, Token Merging and Padding, and Post-denoising Correction—to better understand their contributions to overall performance.\n\nQUESTIONS:\n* Q1: Some baseline results (e.g., DiffSplat) differ notably from those in the original papers, without explanation. Could the authors clarify how these baselines were evaluated or re-implemented?\n\n* Q2: The description of the latent selector’s training is confusing—why is the initially described approach not used, and how is timestep \n$t$ sampled during training?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

comments: SUMMARY:
The authors propose TRIM (Trajectory Reduction and Instance Mask denoising), a training-free approach that incorporates both temporal and spatial trimming strategies. A lightweight selector model is introduced to evaluate latent Gaussian primitives derived from multiple sampled noises, enabling early trajectory reduction by selecting candidates with high-quality potential.

STRENGTHS AND WEAKNESSES:
strength

1. the authors demonstrate inference-time scaling in 3D diffusion models by trajectory pruning, showing that increasing the number of sampled trajectories during inference significantly improves the chance of producing high-quality results.
2. the authors conduct comprehensive ablation studies on computation, inference scaling, and selector.

weakness
1. the latent selector seems to rely heavily on the data synthesis and metrics score. What's the off-the-shelf image metric evaluator? Does it lead to overfitting and having to re-train the selector when changing the dataset?
2. The authors mention one of the significant improvements is the inference speed due to the Instance Masking (IM). Can the authors include the inference time comparison with other works?

QUESTIONS:
1. how many trajectories do you have for TPIM? Does the Inference Step Scaling mean increasing denoising steps in inference?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to accelerate text-to-3D generation using 3D Gaussian diffusion by reducing inference time. Unlike existing state-of-the-art approaches that rely on lengthy denoising and post-processing, the authors introduce two key improvements. First, they generate synthetic data to train a selector model—comprising a CNN for feature extraction and an MLP for binary classification—that identifies the optimal latent trajectory early during inference. Second, they leverage the fact that image corners typically represent background, using these regions as reference to classify latent patches as foreground or background. This classification enables selective denoising, starting from outer regions and moving inward, thereby avoiding unnecessary computation. Their approach enhances both speed and quality in text-to-3D tasks using Gaussian diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper is clearly written and easy to follow. It addresses a practically important and timely problem in 3D generation, inference efficiency and introduces a training-free, model-agnostic solution. The proposed method is both conceptually simple and effective. The authors support their claims with comprehensive experiments and ablation studies, demonstrating consistent improvements across multiple benchmarks.\n\nWeaknesses:\n\nThe masking strategy relies on a corner-based attention heuristic, which assumes that objects are centered in the scene. This design limits generalizability to more diverse or complex layouts where foreground and background regions are not spatially separable.\n\nAlthough the selector is described as lightweight, the paper does not provide a quantitative analysis of its memory or computational overhead.\n\nThe method, as explained in the paper, is not as effective on image-to-3D to 3D scenarios.\n\nQUESTIONS:\nTo better evaluate the efficiency gains of the proposed method, please provide timing breakdowns for each added component—specifically the trajectory selection process and instance mask denoising steps.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents TRIM, a training-free method to improve the efficiency and scalability of 3D Gaussian diffusion models. TRIM reduces computation by selecting high-quality trajectories early and masking redundant background regions. It can be integrated with existing models without retraining, boosting both efficiency and quality in 3D generation tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.It proposes the TRIM framework, which effectively enhances the inference efficiency of 3D Gaussian diffusion models through temporal trajectory reduction and spatial instance mask denoising strategies.\n2.The TRIM framework is plug-and-play, compatible with various Transformer-based 3D diffusion models without requiring retraining, demonstrating strong adaptability.\n3.Extensive experiments on the DiffSplat backbone show that the model achieves approximately 45% reduction in computational resources (FLOPs) while improving generation quality, such as more detailed features in text-to-3D and image-to-3D tasks.\n4.Ablation experiments thoroughly analyze the impact of inference step scaling and trajectory scaling on TRIM, verifying that sampling diverse trajectories significantly boosts generation quality compared to simply increasing denoising steps.\n\nWeaknesses:\n1.The writing lacks clarity in some areas. For example, the definitions of Trajectory Scaling and Inference Step Scaling should be provided earlier in the paper. Additionally, Figure 6 is difficult to understand. There is also a lack of implementation details and specific parameters used.\n2.The experimental metrics are not sufficiently comprehensive. The paper uses CLIP similarity, CLIP R-Precision, ImageReward, PSNR, SSIM, and LPIPS to demonstrate the alignment between 3D objects and text/images, as well as human aesthetic preferences. However, these metrics are not exhaustive. A more comprehensive evaluation could be conducted by referring to the metrics used in 3DGen-Bench or by conducting a user study, which would provide a more thorough and nuanced assessment.\n3.The paper claims to be compatible with various Transformer-based 3D diffusion models without requiring retraining, but it lacks sufficient comparisons to show how the performance changes when TRIM is applied to different backbones.\n4.There is a lack of adequate qualitative comparisons; the paper only provides a direct comparison with DiffSplat, which is not enough for a thorough evaluation.\n\nQUESTIONS:\n1.Clarification of Definitions: Can you clarify Trajectory Scaling and Inference Step Scaling in the beginning of Ablation and Analysis? \n2.Could you redraw Figure 6 to make it clearer, or consider splitting it into multiple figures for better clarity? This would help readers better understand the presented information.\n3.More Details: Could you provide more detailed information on the experimental parameters, such as the optimal settings and the network architecture parameters? \n4.Comprehensive Metrics: The experimental metrics used (CLIP, PSNR, SSIM, etc.) seem limited. Could you consider incorporating additional evaluation metrics, such as those in 3DGen-Bench or conducting a user study for more comprehensive results?\n5.Comparison with Different Backbones: The paper mentions compatibility with various Transformer-based models, but lacks sufficient comparison across different backbones. Could you include experiments showing the impact of TRIM on various model architectures?\n6.Qualitative Comparisons: The paper mainly compares with DiffSplat. Could you provide more qualitative comparisons with other state-of-the-art methods to better demonstrate TRIM’s performance and benefits?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}