LP-3DGS: Learning to Prune 3D Gaussian Splatting

Authors: zhaoliang  zhang
tianchen  song
yongjae  lee
li  yang
cheng  peng
rama  chellappa
deliang  fan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 17:49:37

Abstract: Recently, 3D Gaussian Splatting (3DGS) has become one of the mainstream methodologies for novel view synthesis (NVS) due to its high quality and fast rendering speed. However, as a point-based scene representation, 3DGS potentially generates a large number of Gaussians to fit the scene, leading to high memory usage. Improvements that have been proposed require either an empirical pre-set pruning ratio or importance score threshold to prune the point cloud. Such hyperparameters require multiple rounds of training to optimize and achieve the maximum pruning ratio while maintaining the rendering quality for each scene. In this work, we propose learning-to-prune 3DGS (LP-3DGS), where a trainable binary mask is applied to the importance score to automatically find a favorable pruning ratio. Instead of using the traditional straight-through estimator (STE) method to approximate the binary mask gradient, we redesign the masking function to leverage the Gumbel-Sigmoid method, making it differentiable and compatible with the existing training process of 3DGS. Extensive experiments have shown that LP-3DGS consistently achieves a good balance between efficiency and high quality.

Main paper: 
 introduction: 
 Novel view synthesis (NVS) takes images and their corresponding camera poses as input and seeks to render new images from different camera poses after 3D scene reconstruction. Neural Radiance Fields (NeRF) (Mildenhall et al. [2021] ) uses multi-layer perceptron (MLP) to implicitly represent the scene, fetching the transparency and color of a point from the MLPs. NeRF has gained considerable attention in the NVS community due to its simple implementation and excellent performance. However, in order to obtain a point in the space, NeRF needs to perform an MLP inference. Each pixel rendered requires processing a ray and there are many sample points along each ray. Consequently, rendering an image requires a large amount of MLP inference operations. Thus, rendering speed becomes a major drawback of the NeRF method.Besides NeRF, explicit 3D representations are also widely used. Compared to NeRF, the advantage of point-based scene representation is that it is well-supported by modern GPU rendering, enabling fast render speed. 3D Gaussian Splatting (3DGS) (Kerbl et al. [2023] ) achieves both good quality and high rendering speed, making it a hot topic in the community. 3DGS uses 3D Gaussian models with color parameters to fit the scene and develops a training framework to optimize the model parameters. However, the number of points required to reconstruct the scene is enormous, usually in the millions. In practice, each point has dozens of floating-point parameters, which makes 3DGS a memory-intensive method. Some recent works have attempted to mitigate this problem by pruning the points, such as Light-Gaussian (Fan et al. [2023] ), RadSplat (Niemeyer et al. [2024] ), and Mini-Splatting (Fang and Wang [2024] ). These methods follow a similar pruning approach through defining an importance score for each Gaussian point and prune the points with such importance score below a preset empirical threshold. However, a major drawback of these methods is that the preset threshold must be manually tuned through multiple rounds of training process to identify the favorable pruning ratio to minimize the number of Gaussian points while keeping the rendering quality. To make matters worse, such number of points may vary depending on different scenes, which requires manual pruning ratio searching for each scene. For example, the blue and red lines in Figure 1 show the rendering quality of Kitchen and Room scenes, respectively, in MipNeRF360 scenes (Barron et al. [2022] ), with sweeping 12 different pruning ratios (i.e., 12 rounds of training) following the prior RadSplat (Niemeyer et al. [2024] ) and Mini-Splatting (Fang and Wang [2024] ) method. It is clearly seen that a smaller pruning ratio will not hamper the rendering quality, and the rendering quality will start to decrease with much more aggressive pruning ratios. Both scenes exhibit an optimal pruning ratio region that could maximize the pruning ratio and maintain the rendering quality. It could also be seen that the ideal pruning ratio differs for these two scenes. In this paper, we propose a learning-to-prune 3DGS (LP-3DGS) methodology where a trainable mask is applied to the importance score. Notably, it is compatible with different types of importance scores defined in prior works. Instead of a preset threshold to determine the 3DGS model pruning ratio, as illustrated by the red triangle symbol in Figure 1 , our method aims to integrate with existing 3DGS model training process to learn the favorable pruning ratio for minimizing the model size while maintaining the rendering quality. Since the traditional hard-threshold-based binary masking function is not differentiable, a recent prior work, Compact3D (Lee et al. [2023] ), leverages the popular straight through estimator (STE) (Bengio et al. [2013] ) to bypass the mask gradient for adaption to the backpropagation process. However, this approach often results in suboptimal pruning ratios. In contrast, in our work, we propose to redesign the masking function leveraging the Gumbel-Sigmoid activation function to make the whole masking function differentiable and integrate with the existing training process of 3DGS. Consequently, LP-3DGS automatically minimizes the number of Gaussian points for each scene with just a single round of training.In summary, the technical contributions of our work are:• To address the effortful tuning of 3DGS pruning ratios, we propose a learning-to-prune 3DGS (LP-3DGS) methodology that leverages the differentiable Gumbel-Sigmoid activation function to embed a trainable mask with different types of existing importance scores designed for pruning redundant Gaussian points. As a result, instead of fixed model size, LP-3DGS could learn an favorable Gaussian point size for individual scene with only one-time training. • We conducted comprehensive experiments on state-of-the-art (SoTA) 3D scene datasets, including MipNeRF360 (Barron et al. [2022] ), NeRF-Synthetic (Mildenhall et al. [2021] ), and Tanks & Temples (Knapitsch et al. [2017] ). We compared our method with SoTA pruning methods such as RadSplat (Niemeyer et al. [2024] ), Mini-Splatting (Fang and Wang [2024] ), and Compact3D (Lee et al. [2023] ). The experimental results demonstrate that our method can enable the model to learn the favorable pruning ratio and that our trainable mask method performs better than the STE mask. 
related work: 
 Neural radiance fields (NeRFs) NeRFs (Mildenhall et al. [2021] ) targets to represent the scene in multilayer perceptrons (MLPs) based on multi-view image inputs, enabling high-quality novel view synthesis. Due to its advancement, numerous follow-up works improved it in either rendering quality (Barron et al. [2021 (Barron et al. [ , 2022]] ) or efficiency (Müller et al. [2022] , Chen et al. [2022] , Fridovich-Keil et al.[2022]).Although NeRF models demonstrate impressive rendering capabilities across numerous benchmarks, and considerable efforts have been made to enhance training and inference efficiency, they typically still face challenges in achieving fast training and real-time rendering.Radiance Field Based On Points. In addition to implicit representations, several works have focused on volumetric point-based methods for 3D presentation (Gross and Pfister [2011] ). Inspired by neural network concepts, (Aliev et al. [2020] ) introduced a neural point-based approach to streamline the construction process. Point-NeRF (Ding et al. [2024] ) further applied points for volumetric representation, enhancing the effectiveness of point-based methods in radiance field modeling.Gaussian Splatting 3D Gaussian Splatting (3DGS) (Kerbl et al. [2023] ) represents a significant advancement in novel view synthesis, utilizing 3D Gaussians as primitives to explicitly represent scenes. This approach achieves state-of-the-art rendering quality and speed while maintaining relatively short training time. A series of methods have been introduced to improve the rendering quality through using regularization for better optimization, including depth map (Chung et al. [2023] , Li et al. [2024a] ), surface alignment (Guédon and Lepetit [2023] , Li et al. [2024b] ) and rendered image frequency (Zhang et al. [2024] ). However, the extensive number of Gaussians required for scene representation often results in a model that is too large for efficient storage. Recent research has focused on compression methods to enhance the efficiency of this representation. Notably, several studies (Fan et al. [2023] , Fang and Wang [2024] , Niemeyer et al. [2024] ) have proposed using predefined scores as pruning criteria to keep Gaussians that significantly contribute to rendering quality. Compact3D (Lee et al. [2023] ) introduces a method that applies a trainable mask on scale and opacity to each Gaussian and utilizes a straight-through estimator (Bengio et al. [2013] ) for gradient updates. LightGaussian (Fan et al. [2023] ) employs knowledge distillation to reduce the dimension of spherical harmonics. Additionally, (Fan et al. [2023] , Lee et al. [2023] ) also explored quantization techniques to further compress model storage. The previously proposed pruning methods primarily rely on predefined scores to determine the importance of each Gaussian. These approaches present two main challenges: first, whether the criteria accurately reflect the importance of the Gaussians, and second, the need for a manually selected pruning threshold to decide the level of pruning. In this work, we address these issues by introducing a trainable mask activated by a Gumbel-sigmoid function, applied to the scores derived from prior methods or directly to the scale and opacity of each Gaussian for more flexibility. Our approach automatically identifies the balance between the pruning ratio and rendering quality, eliminating the need to test on various pruning ratios.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations in Section 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 The conventional pruning methods leveraging predefined importance score require pruning ratio as a manually tuned parameter to reduce the size of Gaussian points in 3DGS. To seek for a favorable pruning ratio, these methods may need to perform multiple rounds of training for each individual scene, which is inefficient. Motivated by this, we propose a learning-to-prune 3DGS (LP-3DGS) algorithm which learns a binary mask to determine the favorable pruning ratio for each scene automatically. Importantly, the proposed LP-3DGS is compatible with various types of pruning importance scores. In this section, we will: 1) introduce the preliminary of the original 3DGS and recap different importance metrics for pruning that are proposed by prior works, and 2) present the proposed learning-to-prune 3DGS algorithm.3.1 3DGS Background 3DGS Parameters 3DGS is an explicit point-based 3D representation that uses Gaussian points to model the scene. Each point has the following attributes: position p ∈ R 3 , opacity σ ∈ [0, 1], scale in 3D s ∈ R 3 , rotation represented by 4D quaternions q ∈ R 4 and fourth-degree spherical harmonics (SH) coefficients k ∈ R 48 . In summary, one gaussian point has 59 parameters. The center point X of a Gaussian model is represented by p and covariance matrix Σ is denoted by s and q. The SH coefficients model the color as viewed from different directions. The parameters of the Gaussians are optimized through gradient backpropagation of the loss between the rendered images and the ground truth.Rendering on 3DGS In order to render an image, the first step is projecting the Gaussians to 2D camera plane by world to camera transform matrix W and Jacobian J of affine approximation of the projective transform. The covariance matrix of projected 2D Gaussian isEQUATIONThe projected Gaussians would be rendered as splat (Botsch et al. [2005] ), the color of one pixel could be rendered asEQUATION)Where i is the pixel index, j is the Gaussian index and N is the number of the Gaussians in the ray. c j is the color of the Gaussian calculated by SH coefficients, α j = (1 -exp -σj δj ), σ j is the opacity of the point and δ j is the interval between points. T j = j-1 k=1 (1 -α k ) is the transmittance from the start of rendering to this point. G 2D j is the 2D Gaussian distribution.Adaptive Density Control of 3DGS At the start of training, the Gaussians are initialized using Structure-from-Motion (SfM) sparse points. To make the Gaussians fit the scene better, 3DGS applies an adaptive density control strategy to adjust the number of Gaussians. Periodically, 3DGS will grow Gaussians in areas that are not well reconstructed, a process called "densification." Simultaneously, Gaussians with low opacity will be pruned.A straightforward way to prune the Gaussians is by sorting them based on a defined importance score and then removing the less important ones. Consequently, one of the main objective of prior 3DGS pruning works is to define an effective importance metric.RadSplat (Niemeyer et al. [2024] ) defines the importance score as the maximum contribution along all rays of the training images, written asS i = max I f ∈I f ,r∈I f α r i • T r i (3)Where α r i • T r i is the contribution of Gaussian G i along ray r. RadSplat performs pruning by applying a binary mask according to the importance score, where the mask value for Gaussian G i isEQUATIONWhere t prune ∈ [0, 1] is the threshold of score magnitude for pruning, 1[•] is the indicator function. Another recent work, Mini-Splatting (Fang and Wang [2024] ), uses the cumulative weight of the Gaussian as the importance score, which can be formulated as:EQUATIONWhere K is the total number of rays intersected with Gaussian G i , ω ij is the color weight of Gaussian G i on the j-th ray.The overall LP-3DGS learning process is shown in the Figure 2 . In general, it mainly can generally be divided into two stages:1) densification stage, and 2) learning-to-prune stage. Following the original 3DGS, densification stage applies an adaptive density control strategy to gradually increase the number of Gaussians. As revealed by prior pruning works (Lee et al. [2023] , Niemeyer et al. [2024] ), 3DGS contains a significant number of redundant Gaussians. Subsequently, in the learning-to-prune stage, the proposed LP-3DGS learns a trainable mask upon a previously defined importance metric to compress the number of Gaussians with an favorable pruning ratio automatically. Specifically, to learn a binary mask, we first initialize a real-value mask m i for each point i, and then adopt the Gumbel-sigmoid technique to binarize the mask value differentially.The binarization operation for real-value mask in pruning typically involves a hard threshold function, determining the binary mask should be 0 or 1. However, such hard threshold function is not differentiable during backpropagation. To address this issue, popular straight through estimator (STE) method (Bengio et al. [2013] ) is widely used, as it skips the gradient of the threshold function during backpropagation. Such process may create to a gap between trainable real-value mask and binary mask. As shown in Figure 3 (a), the trainable mask values exhibit certain ratios across the entire range from 0 to 1 after Sigmoid function, which can be inaccurate when further converting to binary mask via a hard threshold function. To better optimize the trainable mask towards binary values, we propose to apply Gumbel-Sigmoid function to learn the binary mask.The Gumbel distribution is used to model the extreme value distribution and generate samples from the categorical distribution (Gumbel [1954] ). This property is then utilized to create the Gumbel-Softmax (Jang et al. [2016] ), a differentiable categorical distribution sampling function. The sample of one category is given by:y i = exp((log(π i ) + g i )/τ ) k j=1 exp((log(π j ) + g j )/τ ) (6)Where τ is the input adjustment parameter, g i is sample from Gumbel distribution. Inspired by the Gumbel-Softmax, we treat learning the binary mask of each point as a two-class category problem. Thus, we replace the Softmax function to Sigmoid function, referring to Gumbel-Sigmoid: By using such Gumbel-Sigmoid function, the output value is either close to 0 or 1, as shown in Figure 3 , allowing it to be integrated as an approximation of a binary masking function. More importantly, this function remains differentiable, thus can be integrated during backpropagation.EQUATIONMoreover, to prune the selected Guassians practically according to the learned binary mask, we further apply the mask value on opacity, which can be mathematically formulated aso im = o i * gs(m i * S i ) (8)where S i is the defined importance score of each Gaussian point. The closer the mask value is to 0, the less corresponding Gaussian point contributes to the rendering. In practice, after learning the trainable mask, a one-time pruning is applied to the corresponding Gaussian points with mask value of 0.In order to compress the model as much as possible, we apply a L1 regularization term (Lee et al. [2023] ) to encourage the trainable mask to be sparse, which can be formulated as:R mask = 1 N N i=1 |m i | (9)Upon that, the final loss function is defined as:EQUATION)L L1 is the L1 loss between rendered image and ground truth. L ssim is the ssim loss. λ ssim and λ m are two coefficients.Moreover, we find that the trainable mask can be effectively learned in just a few hundred iterations, compared to the thousands required for the overall training process. In practice, the mask learning function is activated for only 500 iterations. Once the mask values are learned, we follow the 3DGS training setup to further fine-tune the pruned model while maintaining the same total number of training iterations. The detailed hyper parameters are described in the later experiment section.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The code is listed in https://github.com/dexgfsdfdsg/LP-3DGS.git and the data used are all open benchmarks. A readme file is in the code for instructions.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental settings: 
 Dataset and Baseline We test our method on two of the most popular real-world datasets: Mip-NeRF360 dataset (Barron et al. [2022] ), which contains 9 scenes, and the Train and Truck scenes from the Tanks & Temples dataset (Knapitsch et al. [2017] ). We also evaluate our method on the NeRF-Synthetic dataset (Mildenhall et al. [2021] ), which includes 8 synthetic scenes. In this section, we only present the results on MipNeRF360 dataset, rest of them are listed in appendix A. In this paper, we use the SoTA RadSplat (Niemeyer et al. [2024] ) and Mini-Splatting (Fang and Wang [2024] ) as the baselines, both of which propose different pruning importance scores. First, we evaluate the performance of these two methods under various pruning ratios. Since neither method is open-sourced at the time of writing, we reproduced them based on the provided equations. For each pruning ratio, we calculate the corresponding threshold based on the magnitude of the importance scores and prune the Gaussians with scores below this threshold. Note that each pruning ratio requires one round of training. We use peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) (Zhang et al. [2018] ) as rendering evaluation metrics.Implement Details The machine running the experiments is equipped with an AMD 5955WX processor and two Nvidia A6000 GPUs. It should be noted that our method does not support multi-GPU training. We ran different experiments simultaneously on two GPUs. We train each scene under every setting for 30,000 iterations and with the mask training occurring from iteration 19,500 to 20,000, updating the importance score every 20 iterations. The value of τ in Equation 7 is 0.5 and the coefficient λ m of mask loss is 5e-4.The blue lines in Figure 4 show the results of sweeping pruning ratios using 2 shows the training cost of LP-3DGS on MipNeRF360 dataset. In our setup, after 20000th iteration, the model is pruned based on the learned mask values. The number of Gaussian points will be significantly reduced, resulting in the later stages of training taking much less time than in the non-pruned version. Even with the embedding of the mask learning function, the overall training cost is comparable to that of the vanilla 3DGS. In most cases, the peak training memory usage is slightly higher because training the mask requires more GPU memory. However, after pruning, the 3DGS model size becomes much smaller, leading to a significant improvement in rendering speed, measured in terms of FPS. A recent prior work, Compact3D (Lee et al. [2023] ) proposes to leveraging the straight-through estimator (STE) to train a binary mask based on the opacity and scale of Gaussian parameters. To conduct a fair comparison between STE based mask and our LP-3DGS, we perform two ablation studies, one involves replacing the STE mask in Compact3D with our method, and the other applies the STE mask to the importance score of RadSplat. The formula of STE mask isEQUATION∇ / means stop gradients, 1[•] is the indicator function and f (•) is sigmoid function, ϵ is masking threshold.We first apply the Gumbel-sidmoid activated mask, instead of STE mask, on the opacity and scale of gaussians in the same manner as proposed in Compact3D. The threshold ϵ in Equation 11 and mask loss coefficient follows the default settings in Compact3D. In most cases, our LP-3DGS learns a higher pruning ratio, except for Stump, Garden and Flowers scene. In terms of rendering quality, our LP-3DGS outperforms Compact3D using the STE based mask achieving even smaller model sizes in most scenes.We also apply STE mask on the pruning importance score to compare with our method. The Equation 8 would be rewriten asEQUATION)where M is shown in Equation 11. The same as mentioned before, the parameters for STE mask are default values in Compact3D.  
discussion and conclusion: 
 Broader Impact and Limitation LP-3DGS compresses the 3DGS model to an ideal size in a single run, saving storage and computational resources by eliminating the need for parameter sweeping to find the favorable pruning ratio. However, the limitation of this work is that the rendering quality after pruning varies depending on the the definition of importance scores. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The detailed experiment settings are listed in Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: This paper follows the existing work about 3DGS and lists the essential detailed quantitative results in Section 4.2, Section 4.3 and Appendix A. But it does not list error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: This paper lists the hardware used in Section 4.1 and shows the graphic memory required in Section 4.2.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research conducted in this paper conform the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: This paper discusses the potential societal impacts in Section 5.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper does not pose such risk.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: This paper follows the applicable licenses and terms of usage.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: Yes, the codes introduced in the paper are well-documented and detailed instructions are provided in https://github.com/dexgfsdfdsg/LP-3DGS.git.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this paper, we present a novel framework, LP-3DGS, which guides the 3DGS model learn the best model size. The framework applies a trainable mask on the importance score of the gaussian points. The mask is trained for a specific period and used to prune the model once. Our method compressed the model as much as possible without significantly sacrificing performance and is capable to achieve the favorable compression rate for different test scenes. Compared with the STE mask method, ours achieves better performance. Justification: LP-3DGS is able to enable the model to learn the favorable size, achieving a good balance between efficiency and quality, as demonstrated in Figure 1 , Figure 4 and Figure 6 .Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: This paper received mixed scores (two Weak Accepts, Borderline Reject, and Borderline Accept) from 4 reviewers. Three reviewers (o7SR, fvLt, 1UP2) are mainly satisfied with the responses from the authors during the rebuttal phrase. Reviewer CWFJ initially suggests Borderline Reject and has concerns in technical contribution, optimal Gaussian point size, and discussion with existing works. During the rebuttal phase, the reviewer did not response. These concerns are shared by other reviewers and are believed that they are clarified. The AC agrees with the majority of reviewers that the work can be accepted. The authors are encouraged to include these important clarifications into the final paper when preparing the camera ready version.

Reviews: review: paper_summary: paper_summary: Whereas other works set a fixed pruning ratio or threshold, this paper introduces a method to learn how to prune Gaussians from a scene while retaining high image fidelity. Different scenes have a drop in fidelity at different pruning percentages and the goal of this work is to remove the need to train the model multiple times while searching for a pruning percentage near, but before, that drop. Specifically this work uses a Gumbel-Sigmoid — which is a 2-class Gumbel softmax followed by a Sigmoid function —  to enable a differentiable mask that can learn to prune Gaussians during training. This mask is then applied to set the opacity of each Gaussian. Gaussians where the mask goes to 0 are pruned. Evaluations are performed on 2 real world datasets — MipNerf 360 and Tanks & Temples — and one synthetic dataset — NeRF-Sythetic.
strength_weakness: strength_weakness: **Main Weaknesses:**

W1: This paper uses the terms “optimal” and “best” to describe the pruning ratio found by learning. Specifically a soft definition of the optimal pruning ratio is given in line 41: “minimize the number of Gaussian points while keeping the rendering quality.” But this method does not “keep” the rendering quality — image quality metrics decrease after pruning. Generally the terms “optimal” and “best” are used to describe maximal or minimal points in some meaningful measure. In the domain of pruning 3D-GS models, a more appropriate definition of optimal pruning would be the highest possible compression that can be applied to the model while retaining rendering quality to some percent of the original rendering quality.

W2: It would be helpful to also provide analysis on pruning threshold vs. image quality metrics. It’s possible that there exists a pruning threshold for RadSplat and/or Mini-Splatting that prunes a large percent of Gaussians while retaining image quality. 

W3: Table 2: Minutes isn’t a great metric here. FLOPs or MACs over training would be a lot better. Other works that want to compare to this method will need to use exactly this manuscript’s architecture to replicate its results.



**Other minor considerations:**

MC1: There are still several hyper parameters that haven’t been ablated. This is mostly fine, but it would be good to know how big of an impact these parameters have:
- Gumbel $\tau$
- $\lambda_m$
- Applying the pruning earlier/later than 20000 epochs

MC2: It would be interesting to ablate equation 8 by removing $S_i$. Can you simply just learn $m_i$ parameter without computing an importance score?


**Cosmetic issues:**

C1: line 46, line 50, line 62: could -> can

C2: $T$ in eq. 7 should be a $\tau$

C3: Tables 1 and 2, move the metric to the first column for clarity (like tables 4 and 5)
questions: questions: My biggest concern for this manuscript is that the use of terms “optimal” and “best” is misleading. This method can find a good pruning ratio in one-shot, but it’s not clear that this ratio is optimal. I’m rating this manuscript a borderline reject for now because I strongly think this wording needs to change. I will raise my score if the authors can provide a detailed description of how they will update this wording in their rebuttal.

I would also like to see W2 and W3 addressed, but these are likely computationally expensive and aren’t as important to me as W1.
limitations: limitations: Once the wording in this manuscript is changed, I think the main limitations will be addressed.
paper_summary: paper_summary: This paper proposes a method to optimally prune Gaussians that do not participate in the rendering or optimizing process of the 3D Gaussian Splatting 3D reconstruction algorithm. The key idea in this paper is to use a Gumbel-sigmoid function instead of the standard Sigmoid function to optimize a binary mask based on transmittance of each splatted Gaussian. Experimental results show that by doing so the number of Gaussians can be reduced without losing reconstruction quality, keeping a near optimal balance between quality and computational speed. As a result, higher frame rate can be obtained for rendering.
strength_weakness: strength_weakness: -	Except for replacing the sigmoid function with the Gumbel-Sigmoid function it seems that there is no other novel technical point in the paper. From the results it seems that a threshold value of 0.6 would be better than the naïve 0.5. I am wondering how the ablation study changes if using different threshold.
-	In the experiments, no comparisons to other methods that focus on optimizing number compressing the model. For example, comparison with LightGaussian would be interesting. It seems from the Figure 4 that the proposed method performs slightly worse than mini splatting.
-	What is the meaning of the “pruning ratio” for the proposed method in Figure 4? It seems to be different for the different datasets. However, as far as I understood, the proposed method does not use a pruning ratio: l. 194 => pruning […] points with mask values of 0. This is confusing.
-	There is no significant gain in accuracy, training time or memory consumption. The only significant gain is in rendering speed. This is interesting but because original 3DGS is already quite fast, in practice I am not sure the impact is high. For example if using a simple threshold of 0.5 in Compact3D how does frame rate compares?
-	Writing can be brushed up.
o	L.176-178 is not correct.
o	In figure 4, it seems legends are wrong in the lines for Mini-splatting => blue is described as RadSplat.
o	Tables are not easy to read. Add some special fonts like bold or underline to emphasize best and second-best results.
questions: questions: -	Clarify Figure 4 and explain what the meaning of the pruning ratio for the proposed method is.
-	Add more comparative experiments with other compact 3DGS methods.
limitations: limitations: -	Limitations are properly discussed in the conclusion section.
paper_summary: paper_summary: This work introduces LP-3DGS, aiming to compress 3DGS by replacing the previously manually set threshold with a learning-to-prune scheme.  In particular, LP-3DGS learns a binary mask to automatically prune 3DGS, where such a mask is regularized with the Gumbel-Sigmoid technique. Experiments on three benchmarks showcase the soundness of the introduced methods.
strength_weakness: strength_weakness: * Limited technical contributions. This work seems to be a direct application of Gumbel-Sigmoid, which is a general engineering trick (https://github.com/AngelosNal/PyTorch-Gumbel-Sigmoid) to approximate discrete sampling in a differentiable manner. It needs to be clarified what the main technical insights this work aims to bring to the 3DGS compression community. To me, replacing the threshold setting with the Gumbel-Sigmoid is too intuitive to hit the standard of a top-tier conference.

* It is difficult to know whether the learned binary mask achieves the “optimal Gaussian point size”. This work demonstrates that it learns the “optimal” solutions by plotting several line charts without any theoretical analysis. In some cases, LP-3DGS did not find the best balance, e.g., Fig. 4 Mini-Splatting on Kitchen. Besides, LP-3DGS seems to lose the flexibility of finding better solutions compared to its baseline, e.g., RadSplat can set a smaller ratio to compress further while maintaining reasonable visual quality.


* Lack of discussions with some recent related works. Several more recent approaches in the 3DGS compression community need discussion, and it would be better to add some comparisons with them to understand the advantages of LP-3DGS better.
  * Girish, Sharath, Kamal Gupta, and Abhinav Shrivastava. "Eagles: Efficient accelerated 3d gaussians with lightweight encodings." arXiv:2312.04564.
  * Lu, Tao, et al. "Scaffold-gs: Structured 3d gaussians for view-adaptive rendering." CVPR2024.
  * Chen, Yihang, et al. "HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression." arXiv:2403.14530.
questions: questions: Please kindly refer to the [Weaknesses].
limitations: limitations: Please kindly refer to the [Weaknesses].
paper_summary: paper_summary: The paper proposed a pruning method for Gaussian Splatting training, which employ a learnable mask to find optimal pruning rate.
strength_weakness: strength_weakness: * The proposed method use a regularization loss to encourage the model to prune, so that the weight parameter is an important balancing factor. The discussion on how to set such parameter is missing.

* Figure 2 is not clear enough for me to well understand the proposed idea. In Figure 3, why pattern like (b) is better than (a)?

* Experiments section use existing works for comparison, but they are all self-implemented. The performance in their original paper should be included for reference.

* The author claim the proposed method can find optimal pruning ratio. Based on my observation, the conclusion is not solid. First, what is definition of optimal? Second, not all the subfigures in Fig. 4 & 6 support this conclusion.
questions: questions: Please see the "weaknesses".
limitations: limitations: No potential negative social impact.

Comments: title: last day of rebuttal response
comment: Dear Reviewer 1UP2,

Thanks for the suggestion and we will update the manuscript as suggested. Appreciate the discussion and consideration for our work
Dear reviewer CWFJ,

today is the last day of reviewer-author discussion period. We hope you have checked our rebuttal and found it could address your previous concerns. We would be happy to provide clarification of our work if you have any further comment.

thanks for your review.
"Almost no rendering quality drop" is vague. I recommend you specify the value range and the compression rate in your Tables. The most controversial point we have discussed origins from the task itself. I will raise my score.
Dear reviewer 1UP2

thanks for the comments.

We did not only use metric of 'falling into the band' as the evaluation, this is what we want to show the motivation of this work. we actually reported quantitive results of pruning performance in table 1 of our original manuscript. Please refer to that. 
We did not even set 1db drop as the bar, but targeting to achieve almost no rendering quality drop and learn the corresponding pruning ratio. it is proven in the quantitive results in table 1. 

For the MipNeRF360 dataset, the average PSNR is 27.48 for unpruned 3DGS model (baseline in table 1). our learned pruned model based on Radsplat score has an average PSNR of 27.47 with average pruning ratio of 0.63. the learned pruned model based on mini-splatting score has an average PSNR of 27.12 with average pruning ratio of 0.6. This difference comes with different definitions of pruning importance scores in both methods, but showing similar performance. 
Also, we want to note that we only use the pruning importance scores defined in Radsplat and Mini-Splatting as the pruning metric from baseline 3DGS model, without any other tricks in those works to further improve the quality, for fair comparison. Therefore, the theoretical performance ceiling is the unpruned model. 

For comparing with random pruning ratio selection, the pruning sensitivity analysis in figure 4 and 6 already shows the rendering quality remains almost the same when it is smaller than our learned pruning ratio for each scene. The rendering quality starts to drop when it is larger than our learned pruning ratio.  

Hope this is what the reviewer requested for proving the 'effectiveness' of our method.

thanks





prune the vanilla
I understand that you are talking about the trade-off: better compression ratio means lower rendering quality.  However, to demonstrate the proposed method is better than random selection in empirical interval is necessary. The metric that falling into the "band" is so loose.  How about defining a metric for such trade-off task?  Take PSNR as example. In the case of maximum loss of 1dB(example value), the higher the compression rate, the better.
Dear reviewer 1UP2

thanks for the comments.

1. If you agree there is no 'better' pruning ratio, but how to guarantee our proposed method could learn the pruning ratio located within the cutoff band in the pruning sensitivity analysis plot shown in figure 4 and 6, what does that mean "At least a number larger than 50% can prove that the proposed method is effective." ? We believe all our learned pruning ratios for every scene is already located within the cutoff band of pruning sensitivity analysis plot. the table 1 also reports, with our learned pruning ratio, almost every scene has the same rendering image quality with the vanilla 3DGS. The average rendering image quality is also the same as the unpruned version with an average of 63% pruning ratio. We do not quite understand what are the requested extra  experiments to prove we are at least 50% better. if you mean rendering quality, the average rendering quality of the pruned model is already the same as unpruned version, but with 2.7X smaller model size (i.e. average 63% pruning ratio)

2. We already setup the experiments with different vanilla 3DGS parameters, which will lead to different initial model redundancy. Hope we could get the results before rebuttal deadline. We will post the results once we get it
Dear authors,

I do agree that "our learned pruning ratio is always 'better' than random chosen pruning ratio, since there is no 'better' pruning ratio in model pruning problem". However, in my previous comment, I only expect that your method is better in over 80% scenes.  I do not require an absolute better or optimal. At least a number larger than 50% can prove that the proposed method is effective.  I do not think a successful rate of 80% is a high standard for a publishable paper. Learnable and automatic method is not preferred if just a random manual number setting can solve the problem comparably. 
This is the most important point in my opinion. If the proposed method can not demonstrate enough effectiveness, I can only regard it as an engineering trick in GS reconstruction.

As for the comment "such 0.5 to 0.8 pruning ratio interval "found in this work" is based on the vanilla and default 3DGS training setup, ... Different 3DGS training hyperparameter settings...". I want to clarify that:
If you want to prove your method is effective on other variants of GS, please just show evidence.
If you want to prove that finding empirical prune ratio of other GS is more difficult compared with adopting your method, please show evidence.

Finally, as for your comment "We kindly recommend the reviewer to check our discussion with the reviewers o7SR and fvLt above. Both agreed the community will benefit from our proposed automatic pruning",  I do have seen all the discussions above and **I will keep my independent thinking**.  I've spent lots of time on this paper and I am not an outsider of GS. I kindly recommend the authors repect every review equally no matter they are positive or negative.
Dear reviewer,

1. We did not misunderstand your first comment. In the previous response, we just used one sentence to clarify the word of "optimal". While, the majority rest is about the band interval discussion and clarification of our method. please re-evaluate.

2. In our reported experiments and our several rounds of discussion above, we already stated very clearly that we never argued our learned pruning ratio is always 'better' than random chosen pruning ratio, since there is no 'better' pruning ratio in model pruning problem. It is always a tradeoff between pruning/compression ratio and quality, which is the nature of this problem. But the key point is our method is learnable and automatic that could efficiently and effectively achieve a pruning ratio that is always within the "band" mentioned by the reviewer, ''with only one round of training and embedding with the 3DGS model learning phase, rather than costly manual tuning''. From this point of view, we strongly believe all the experiment results and extra results provided during this rebuttal period to all reviewers are sufficient and strong enough to prove the motivation and effectiveness of our proposed method. 
We kindly recommend the reviewer to check our discussion with the reviewers o7SR and fvLt above. Both agreed the community will benefit from our proposed automatic pruning and it is more efficient than running entire pruning sensitivity analysis for every scene when compressing the large size of 3DGS model.

3. As for the suggestions of adding more pruning results during 0.5 to 0.8 interval.  we will start running more experiments as suggested. However, we do not see the point why it is helping for our method, since our method already learned a large pruning ratio that has almost no quality drop for every unique scene, with only one round of training. More points within the 0.5 to 0.8 interval means more training cost for manual pruning ratio tuning, it will only make the benchmarking baseline worse. 

4. Again, we want to highlight that such 0.5 to 0.8 pruning ratio interval "found in this work" is based on the vanilla and default 3DGS training setup. It reveals certain redundancy during the Gaussian densification process. Different 3DGS training hyperparameter settings, for example, the gradient threshold of densification process, will significantly change the number of Gaussian of the final trained model. Then, it is obvious that such "empirical 0.5 to 0.8 interval" will change based on the model redundancy. In this case, even with prior general pruning experience, the cutoff band would change accordingly. To optimizing the tradeoff between compression ratio and rendering quality, new costly pruning sensitivity analysis will be required to get such new cutoff band. 

we already open-sourced all our codes and results in the github for the community to peer-review and test on their different pruning importance score. 

We appreciate the reviewer's comments and discussion. But for scientific research, we want to make sure the major contribution of this work is highlighted and understood well.
Maybe the authors misunderstand my first point. I'm not criticizing the wording. In the comment the authors claimed that "the "band" is in general within around 0.5 to 0.8, rather than 0.7 to 0.8 or 0.6 to 0.8".  I do not think there is a substantial difference between them.  

However, I would like to provide some advice to evaluate your method more scientifically.

To prove that your method is more effective than randomly selecting the prune ratio in the empirical interval e.g. 0.5-0.8, you can randomly select many points from the interval to get the expectation of these metrics including PSNR/LPIPS/SSIM. Compared with LP-3DGS, if there is a significant improvement in each indicator and scenario, it indicates that your method is effective.  For example, I would expect that LP-3DGS outperforms the random strategy in 80%-90% scenes.  As for now, the evaluation results are still vague.

As for the presentation, you need to demonstrate the strong correlation between the selected prune ratio and these metrics, in the empirical interval of 0.5-0.8 rather than the 0.0-1.0 shown in your figures. The big interval demonstrated(0.0-1.0) in Fig4&6 contains too many impossible value while overwhelming the effectiveness of the proposed methods.
Dear reviewer, we hope our rebuttal has effectively addressed your comments and concerns. As the deadline for the author-reviewer discussion is tomorrow, we kindly request your prompt feedback on our response and ask you to reconsider the review score.
Thank you very much for your time and consideration.
Thanks for your comments

### For "optimal" wording
As for the term of "optimal pruning ratio", in our response to the common question, we already claimed that we will not use it and change to "learned pruning ratio" with more detailed definition. Please refer to above for our detailed clarification. The reviewer o7SR suggested us to change to "learned effective pruning ratio" and we plan to use such in our updated manuscript.
 For the sensitivity analysis of rendering image quality against pruning ratios, as we show in our experiment results (figure 4 and 6), each scene has different sensitivities against pruning and the "band" is in general within around 0.5 to 0.8, rather than 0.7 to 0.8 or 0.6 to 0.8, where the rendering image quality starts to drop with increasing pruning ratio. In order to find such "band" for different scenes, a manual tuning of pruning ratios with multiple rounds of training is still needed in prior works, which is costly and not an "easy approach". We kindly disagree that just setting the pruning ratio to 0.75 will be just fine. 
To justify such, we conducted extra experiments based on the pruning ratio of 0.75 based on the Radsplat score for kitchen and room scenes, due to the limited rebuttal time left. Note that, table 1 in our manuscript reported the complete set of our method learned pruning ratio and its corresponding image quality of each scene. From table 1, it can be seen that, in average, our model with learned pruning ratio has almost the same rendering image quality as the unpruned version, which is what we defined in the loss function and the experiment results support that well. In particular, the learned pruning ratio is 0.58 for kitchen scene and 0.74 for room scene. It can be seen from the extra experiments in below table that the kitchen scene has a significant quality drop with a random 0.75 pruning ratio than that of learned value of 0.58. While, since the randomly chosen 0.75 pruning ratio is close to the learned ratio value of 0.74 for Room scene, the rendering image quality is very similar. 
Furthermore, such 0.75 pruning ratio is actually not "random", but from the thorough complete pruning sensitivity analysis reported in our submitted manuscript. We believe these ablation study could further prove the motivation of this work and effectiveness of our proposed method. 
To summarize, for scientific research and contributions, we believe our work discovered the pattern of pruning sensitivity in 3DGS and further propose a general learning method that could be applied to different types of pruning importance scores with a target of automatic pruning, which is a solid contribution to the community and will inspire more research in the future.

|       | Pruning Raio | Kitchen |  Room  |
|:-----:|:------------:|:-------:|:------:|
|       |       0      |  31.581 | 31.500 |
|  PSRN |    Learned   |  31.515 | 31.490 |
|       |     0.75     |  31.187 | 31.420 |
||
|       |       0      |  0.9230 | 0.9249 |
|  SSIM |    Learned   |  0.9311 | 0.9243 |
|       |     0.75     |  0.9270 | 0.9235 |
||
|       |       0      |  0.1164 | 0.1978 |
| LPIPS |    Learned   |  0.1194 | 0.2032 |
|       |     0.75     |  0.1293 | 0.2045 |

### We are not sure whether the reviewers are criticising the effectiveness or efficiency. We discussed about both in below to clarify.

Method effectiveness: For the Bicycle-Radsplat subfigure of Figure 6, all the quantitative metrics, i.e., PSNR/SSIM/LPIPS, that are very popularly used in this community to evaluate the rendering image quality are starting to become worse beyond 0.6 pruning ratio. It actually shows the effectiveness of our method that could learn such pruning ratio with only one time training. The pruning is always a tradeoff process between compression ratio and rendering quality, which is the nature of this problem. We never claimed our learned pruning ratio is always better than other manual pruning ratio settings, but the key point is our method is learnable and automatic that could efficiently and effectively achieve a pruning ratio that is always within the "band" mentioned by the reviewer, rather than costly manual tuning. 

Method efficiency: We conducted comprehensive training cost analysis in table 2. The actual training time compared with vanilla 3DGS reduces even for single round of training. Compared with other manual pruning method, as we discussed before, our method could achieve a learned effective pruning ratio, embedding with the 3DGS model construction learning process, without the costly pruning sensitivity analysis shown in the figure 4 and 6 that require multiple rounds of training with preset pruning ratio. We believe our method could improve both the 3DGS learning efficiency and pruning efficiency significantly compared with prior works.

We kindly request the reviewer to reconsider our technical contributions to the community, as well as the final review score of our work.
While the authors addressed the most of my minor concerns, I still insist on my opinion about the key point about the "optimal prune ratio". 

First, according to the experiments, most of the "optimal" prune ratios are similar, around 0.7-0.8 without specific pattern.  It is a very narrow band. In such cases, the "learning" task for the prune ratio may be a trivial task since the prune ratio is easy to approach. Just set a 0.75 value may be good enough for most of the experiments in Fig.4 & 6.

Second, even though we assume the task is non-trivial, the experiment cannot prove the proposed strategy is efficient enough. For example, in the Bicycle-Redsplat subfigures of  Fig.6, we cannot tell the prune ratio ~0.6(LP-3DGS) is better than 0.7 or 0.8. Most of the experiments are with the same problem: no convincing metric to examine the efficiency of the proposed method. Actually, I think it is the problem of this task as I said in the first paragraph.

In other words, if all the "answers" fall in around 0.6-0.8, and we cannot find the concrete value which is significantly better than others, the results are too vague to support the claims.

Therefore for now I will keep my score.
I have read the authors' response and the other reviews.

Thanks for the detailed answers. 
I had similar question as reviewer o7SR about W2. Figure 1-4 kind of gave the impression that the pruning ratio can be decided. It is now clarified that the parameter is the importance score, which results in some pruning ratio. And the proposed method allows to find the score to achieve desired pruning ratio.
Other comments have also been addressed the rebuttal. I raise my rating to 6.
Interesting, yes this addresses my concern regarding W2. I'll update my score to a 6.
We appreciate all the reviewer’s valuable comments and insights, which have genuinely helped improve our work. We would be grateful if the reviewer could consider raising the final score.
# W1
We appreciate your comment. we will change to learned effective pruning ratio.

# W2
We thank you for your further clarification. Please see our detailed response below:

***"It’s possible that for one scene a score may be at the 40% pruning ratio whereas that same score may be at the 60% pruning ratio in a different scene."***

We totally agree with the reviewer. To justify such, we conducted extra experiments to get the pruning ratio while fixing the pruning threshold of RadSplat defined importance score. The results are listed in below table. It can be clearly seen that the same threshold score for different scenes lead to different pruning ratios for different scenes. This actually further justifies the importance of our work, as it implies that different scenes indeed need the "learned effective pruning ratio'' to optimize tradeoff between compression ratio and rendering quality.  
Due to the fact that it is a most common setting in prior works to measure the rendering image quality with target pruning ratios, for fair comparison, we design our framework to learn an effective pruning ratio for each scene, without multiple rounds of manual tuning of pruning ratios as used in prior works

| Threhold | Bicycle | Bonsai | Counter | Kitchen |  Room  |  Stump | Garden | Flowers | Treehill |
|:-----:|:-------:|:------:|:-------:|:-------:|:------:|:------:|:------:|:-------:|:--------:|
| 0.025 |  50.88% | 43.79% |  47.86% |  38.46% | 52.73% | 51.07% | 48.05% |  50.65% |  41.26%  |

***"It’s entirely possible that a specific score may provide a better threshold across scenes because it likely will not corespondent to a specific pruning ratio, but will be a measure of the confidence in a particular Gaussian."***

This is a very interesting point. We conducted such experiments on different scenes using different threshold values, based on RadSplat pruning importance score and the results are shown in figure below. It can be seen that, in the kitchen scene, the PSNR starts to drop significantly when the threshold is larger than 0.2. However, in the room scene, such cutoff point is around 0.15. It shows different scenes have different sensitivity against pruning threshold. 

[Kitchen](https://drive.google.com/uc?export=view&id=1dTP7mA_MIw_ROhxstItbThJyI6m-Sr3L)
[Room](https://drive.google.com/uc?export=view&id=1qU1gwcNFo6hwcDOC7qch71YZcTcET0rU)

More interestingly, since our proposed method learns the effective pruning ratio for different scenes during 3DGS model construction, we discovered that it naturally captures such unique cutoff threshold score for different scenes.

# MC2
We appreciate you for sharing this valuable insight.
In equation 8, if we only use $m_i$, then we could regard opacity ($o$) as the importance score. It is also true that the opacity itself could be one type of importance score. Our objective is to propose a method that is general, learnable, and orthogonal to other prior or potentially future works exploring different definitions of effective 3DGS pruning importance scores.
Dear reviewer, 
We hope our rebuttal can address your comments and concern. As the deadline for the author-reviewer discussion is approaching, would you please share your thoughts on our rebuttal and reconsider the review score? thanks a lot
Dear reviewer, 
We hope our rebuttal can address your comments and concern. As the deadline for the author-reviewer discussion is approaching, would you please share your thoughts on our rebuttal and reconsider the review score? thanks a lot
Dear reviewer, 
We hope our rebuttal can address your comments and concern. As the deadline for the author-reviewer discussion is approaching, would you please share your thoughts on our rebuttal and reconsider the review score? thanks a lot
Apologies to the authors, I meant to raise my score to a 5 earlier based on the rebuttal. I'm happy to raise my score to a 6 if my concern regarding W2 can be addressed.
W1:
Thanks for the detailed reply. I think it’s fine if you include a different adjective to sell this method, just not “optimal”. For instance I think it’s totally fine to claim this is a “learned *effective* pruning ratio”.

W2: 
I think there may be some confusion here. I’m referring to the actual score computed by these methods, $s \in \mathbb{R}$. A score is computed per Gaussian, then these scores are ordered to prescribe, for a given pruning ratio $x$%, which of them should be pruned. It’s not possible, given only the Gaussian’s ordering (i.e., the pruning ratio) to obtain the original scores.

This criticism was meant to call out that all plots are relative to this percentage, but not the actual scores. It’s possible that for one scene a score $s$ may be at the $40$% pruning ratio whereas that *same* score may be at the $60$% pruning ratio in a different scene. It’s entirely possible that a specific score may provide a better threshold across scenes because it likely will not corespondent to a specific pruning ratio, but will be a measure of the confidence in a particular Gaussian.

W3:
I agree other methods report minutes. I am not overly concerned by this and don’t require the authors to run this.
Note: by FLOPs above I mean “floating point operations”, not FLOPS “floating point operations per second”.

MC2: 
This comment was that it would be interesting to ablate how well this method performs *without* the use of an auxiliary pruning score. It’s possible that $m_i$ may prove to be a good pruning score on its own. I suspect that $S_i$ may be only providing a good “initialization”.

Overall:
I still find this method interesting and my primary concern, the wording of the manuscript, has been addressed.