Diff-PCC: Diffusion-based Neural Compression for 3D Point Clouds

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 08:05:56

Abstract: Stable diffusion networks have emerged as a groundbreaking development for their ability to produce realistic and detailed visual content. This characteristic renders them ideal decoders, capable of producing high-quality and aesthetically pleasing reconstructions. In this paper, we introduce the first diffusion-based point cloud compression method, dubbed Diff-PCC, to leverage the expressive power of the diffusion model for generative and aesthetically superior decoding. Different from the conventional autoencoder fashion, a dual-space latent representation is devised in this paper, in which a compressor composed of two independent encoding backbones is considered to extract expressive shape latents from distinct latent spaces. At the decoding side, a diffusion-based generator is devised to produce high-quality reconstructions by considering the shape latents as guidance to stochastically denoise the noisy point clouds. Experiments demonstrate that the proposed Diff-PCC achieves state-of-the-art compression performance (e.g., 7.711 dB BD-PSNR gains against the latest G-PCC standard at ultra-low bitrate) while attaining superior subjective quality. Source code will be made publicly available.

Main paper: 
 introduction: 
 Point clouds, composed of numerous discrete points with coordinates (x, y, z) and optional attributes, offer a flexible representation of diverse 3D shapes and are extensively applied in various fields such as autonomous driving [8] , game rendering [35] , robotics [7] , and others. With the rapid advancement of point cloud acquisition technologies and 3D applications, effective point cloud compression techniques have become indispensable to reduce transmission and storage costs.Prior to the widespread adoption of deep learning techniques, the most prominent traditional point cloud compression methods were the G-PCC [39] and V-PCC [40] proposed by the Moving Picture Experts Group(MPEG). G-PCC compresses point clouds by converting them into a compact tree structure, whereas V-PCC projects point clouds onto a 2D plane for compression. In recent years, numerous deep learning-based methods have been proposed [50, 45, 11, 12, 7, 30, 46, 14, 42] , which primarily employ the Variational Autoencoder (VAE) [1, 2] architecture. By learning a prior distribution of the data, the VAE projects the original input into a higher-dimensional latent space, and reconstructs the latent representation effectively using a posterior distribution. However, previous VAE-based point cloud compression architectures still face recognized limitations: 1) Assuming a single Gaussian distribution N (µ, σ 2 ) in the latent space may prove inadequate to capture the intricate diversity of point cloud shapes, yielding blurry and detail-deficient reconstructions [56, 10] ; 2) The Multilayer Perceptron (MLP) based decoders [50, 45, 11, 12, 46] suffer from feature homogenization, which leads to point clustering and detail degradations in the decoded point cloud surfaces, lacking the Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.Figure 1 : Diff-PCC pipeline. X t and Xt represents the tth original point cloud and noisy point cloud, respectively; p refers to the forward process and q refers to the reverse process; N (0, I) means the pure noise. Entropy model and arithmetic coding is omitted for a concise explanation. ability to produce high-quality reconstructions. Recently, Diffusion models (DMs) [5] have attracted considerable attention in the field of generative modeling [34, 48, 41, 19] due to their outstanding performance in generating high-quality samples and adapting to intricate data distributions, thus presenting a novel and exciting opportunity within the domain of neural compression [33, 44, 25] .By generating a more refined and realistic 3D point cloud shape, DMs offer a distinctive approach to reduce the heavy dependence of reconstruction quality on the information loss of bottleneck layers.Building on the preceding discussion, we introduce Diff-PCC, a novel lossy point cloud compression framework that leverages diffusion models to achieve superior rate-distortion performance with exceptional reconstruction quality. Specifically, to enhance the representation ability of simplistic Gaussian priors in VAEs, this paper devises a dual-space latent representation that employs two independent encoding backbones to extract complementary shape latents from distinct latent spaces.At the decoding side, a diffusion-based generator is devised to produce high-quality reconstructions by considering the shape latents as guidance to stochastically denoise the noisy point clouds. Experiments demonstrate that the proposed Diff-PCC achieves state-of-the-art compression performance (e.g., 7 .711 dB BD-PSNR gains against the latest G-PCC standard at ultra-low bitrate) while attaining superior subjective quality.Main contributions of this paper are summarized as follows:• We propose Diff-PCC, a novel diffusion-based lossy point cloud compression framework.To the best of our knowledge, this study presents the first exploration of diffusion-based neural compression for 3D point clouds.• We introduce a dual-space latent representation to enhance the representation ability of the conventional Gaussian priors in VAEs, enabling the Diff-PCC to extract expressive shape latents and facilitate the following diffusion-based decoding process.• We devise an effective diffusion-based generator to produce high-quality noises by considering the shape latents as guidance to stochastically denoise the noisy point clouds. 
point cloud compression: 
 Classic point cloud compression standards, such as G-PCC, employ octree [29] to compress point cloud geometric information. In recent years, inspired by deep learning methods in point cloud analysis [26, 27] and image compression [1, 2, 22] , researchers have turned their attention to learningbased point cloud compression. Currently, point cloud compression methods can be primarily divided into two branches: voxel-based and point-based approaches. Voxel-based methods further branch into sparse convolution [36, 37, 38, 49, 51, 52] and octree [9, 24, 31] . Among them, sparse convolution derives from 2D-pixel representations but optimizes for voxel sparsity. On the other hand, octree-based methods, utilize tree structures to eliminate redundant voxels, representing only the occupied ones.Point-based methods [11, 50, 45, 46] are draw inspiration from PointNet [26] , utilizing symmetric operators (max pooling, average pooling, attention pooling) to handle permutation-invariant point clouds and capture geometric shapes. For compression, different quantization operations categorize point cloud compression into lossy and lossless types. In this paper, we focus on lossy compression to achieve higher compression ratios by sacrificing some precision in the original data.Recently, diffusion models have ignited the image generation field [58, 17, 32] , inspiring researchers to explore their potential in point cloud applications. DPM [20] pioneered the introduction of diffusion models in this domain. Starting from DPM, PVD [57] combines the strengths of point cloud and voxel representations, establishing a baseline based on PVCNN. LION [47] employs two diffusion models to separately learn shape representations in latent space and point representations in 3D space. Dit-3D [23] innovates by integrating transformers into DDPM, directly operating on voxelized point clouds during the denoising process. PDR [21] employs diffusion model twice during the process of generating coarse point clouds and refined point clouds. Point•E[] utilizes three diffusion models for the following processes: text-to-image generation, image-to-point cloud generation, and point cloud upsampling. PointInfinity [13] utilizes cross-attention mechanism to decouple fixed-size shape latent and variable-size position latent, enabling the model to train on low-resolution point clouds while generating high-resolution point clouds during inference. DiffComplete [4] enhances control over the denoising process by incorporating ControlNet [53] , achieving new state-of-the-art performances. These advancements demonstrate the promise of DMs in point cloud generation tasks, which motivates our exploring its applicability in point cloud compression. Our research objective is to explore the effective utilization of diffusion models for point cloud compression while preserving its critical structural features. Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Limitations are discussed in Sec. 5.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 Denoising Diffusion Probabilistic Models (DDPMs) comprise two Markov chains of length T: diffusion process and denoising process. Diffusion process adds noise to clean data x 0 , resulting in a series of noisy samples {x 1 , x 2 ...x T }. When T is large enough, x T ∼ N (0, I). The denoising process is the reverse process, gradually removing the noise added during the diffusion process. We formulate them as follows:EQUATIONEQUATIONwhere β is a hyperparameter representing noise level. t ∼ Unif{1, . . . , T } represents time step. Via reparameterization trick, we can sample from q(x t |x t-1 ) and p θ (x t-1 |x t ) as following: where α t = 1 -β t , ᾱt = t i=1 α i , ϵ denotes random noise sampled from N (0, I). Note that ϵ θ (x t , t) is a neural network used to predict noise during the denoising process, and x t can be directly sampled viaEQUATIONEQUATIONx t = √ ᾱt x 0 + √ 1 -ᾱt ϵ.DDPMs train the reverse process by optimizing the model parameters θ through noise distortion. The loss function L(θ, x 0 ) is defined as the expected squared difference between the predicted noise and the actual noise, with the mathematical expression as follows:EQUATION3.2 DIFF-PCCAs shown in Fig. 2 , two key components, i.e., compressor and generator, are respectively utilized in the diffusion process and denoising process. In Diff-PCC, the diffusion process is identified as the encoding, in which a compressor extracts latents from the point cloud and compresses latents into bitstreams; at the decoding side, the generator accepts the latents as a condition and gradually restoring point cloud shape from noisy samples.Several research have demonstrated that a simplistic Gaussian distribution in the latent space may prove inadequate to capture the complex visual signals [56, 3, 6, 10] . Although previous works have proposed to solve these problems using different technologies such as non-gaussian prior [15] or coupling between the prior and the data distribution [10] , these techniques may not be able to directly employed on neural compression tasks.In this paper, a simple yet effective compressor is introduced, which composed of two independent encoding backbones to extract expressive shape latents from distinct latent spaces. Motivated by PointPN [55] , which excels in capturing high-frequency 3D point cloud structures characterized by sharp variations, we design a dual-space latent encoding approach that utilizes PointNet to extract low-frequency shape latent and leverages PointPN to characterize complementary latent from high frequency domain. Let x be the original input point cloud, we formulate the above process as:EQUATIONwhere y l ∈ R 1×C and y h ∈ R S×C represent the low-frequency and high-frequency latent features,respectively; E l and E h refer to the PointNet and PointPN backbones, respectively. Next, the quantization process Q is applied on the obtained features ȳl and ȳh , i.e.,EQUATIONwhere function Q refers to the operation of adding uniform noise during training [1] and the rounding operation during test.Then, fully factorized density model [1] and the hyperprior density model [2] are employed to fit the distribution of quantized features ȳl and ȳh , respectively. Particularly, the hyperprior density model p φ (ȳ h ) can be described as:EQUATIONwhere U -1 2 , 1 2 refers to the uniform noise ranging from -1 2 to 1 2 ; N (µ, σ 2 ) refers to the normal distribution with expectation µ and standard deviation σ, which can be further estimated by ahyperprior encoder E hyper and decoder D hyper :EQUATIONIn this way, a triplet containing quantized low-frequency feature ȳl , quantized high-frequency feature ȳh , and quantized hyperprior z will be compressed into three separate streams. Let p(•) and p (... ) (•)respectively represents the actual distribution and estimated distribution of latent features, then the bitrate R can be estimated as follows:EQUATIONThe generator takes noisy point cloud x t at time t and necessary conditional information C as input.We hope generator to learn positional distribution F of x t and fully integrate F with C to predict noise ϵ t at time t. In this paper, we consider all information that could potentially guide the generator as conditional information, including time t, class label l, noise coefficient β t , and decoded latent features (ȳ l and ȳh ).DiffComplete [4] uses ControlNet [54] to achieve refined noise generation. However, the denoiser of DiffComplete is a 3D-Unet, adapted from its 2D version [16] . This structure is not suitable for our method, because we directly deal with points, instead of voxels. We embraced this idea and specially designed a hierarchical feature fusion mechanism to adapt to our method. Note that 3D-Unet can directly downsample features F through 3D convolution with a stride greater than one. It is very complex for point-based methods to achieve equivalent processing. Therefore, we did not replicate the same structure as DiffComplete does, but directly used AdaLN to inject conditional information, formulated as:EQUATIONwhere F in denotes the original features in the Generator and C denotes the condition information.Now we detail the structure: First, we need to exact the shape latent of noise point cloud x t and we choose PointNet for structural consistency. However, in the early stages of the denoising process,x t lacks a regular surface shape for the generator to learn. Therefore, we adopt the suggestion from PDR [23] , adding positional encoding to each noise point so that the generator can understand the absolute position of each point in 3D space. Then we inject shape latent ȳl from the compressor via ADaLN. We formulate the above process as:EQUATIONEQUATIONNext, we need to fuse high-frequency features. We extract the local high-frequency features of x t using PointPN and add them to F from the previous step, Then we inject the high-frequency features from the compressor via AdaLN. We use K-Nearest Neighbor (KNN) operation to partition locally and set the number of neighbor points to 8, which allows the generator to learn local details. We formulate the above process as:EQUATIONEQUATIONAfter that, we use the self-attention mechanism to interact with information from different local areas.And through a feature up-sampling module, we generate features for n points. Finally, we output noise through a linear layer. We formulate the above process as:EQUATIONEQUATION)EQUATIONWe follow the conventional rate-distortion trade-off as our loss function as follows:EQUATIONwhere D refers to the evaluated distortion; R represents bitrate as shown in Eq. 10; λ serves as the balance the distortion and bitrate. Specifically, a combined form of distortion D is used in this paper, which considers both intermediate noises (ϵ, ε) and global shapes (x 0 , x0 ):EQUATIONwhere D M SE denotes the Mean Squared Error (MSE) distance; D CD refers to the Chamfer Distance;γ means the weighting factor. Here, the overall point cloud shape is additively supervised under the Chamfer Distance D CD (x 0 , x0 ) to provide a global optimization. The following function is utilized to predict the reconstructed point cloud x0 in practice:EQUATIONwhere ᾱt means the noise level; x t refers to the noisy point cloud at time step t; ϵ θ denotes the predicted noise from the generator; c represent the conditional information we inject into the generator. Baselines & Metric We compare our method with the state-of-the-art non-learning-based method:G-PCC, and the latest learning-based methods from the past two years: IPDAE, PCT-PCC, Following [45, 46] , we use point-to-point PSNR to measure the geometric accuracy and the number of bits per point to measure the compression ratio.Implementation Our model is implemented using PyTorch [27] and CompressAI [4] , trained on the NVIDIA 4090X GPU (24GB Memory) for 80,000 steps with a batch size of 48. We utilize the Adam optimizer [21] with an initial learning rate of 1e-4 and a decay factor of 0.5 every 30,000 steps, with β 1 set to 0.9 and β 2 set to 0.999. Since the positional encoding method requires the dimension (dim) to be a multiple of 6, we designed the bottleneck layer size to be 288. For diffusion, we employ a cosine preset noise parameter, setting the denoising steps T to 200, which is used for both training and testing. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: Source code will be made publicly available once paper is accepted.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
baseline comparisons: 
 Objective Quality Comparison Table 1 shows the quantitative indicators using BD-Rate and BD-PSNR, and Fig. 3 demonstrates the rate-distortion curves of different methods. It can be seen that, under identical reconstruction quality conditions, our method achieves superior rate-distortion performance, conserving between 56% to 99% of the bitstream compared to G-PCC. At the most minimal bit rates, point ot point PSNR of our proposed method surpasses that of G-PCC by 7.711 dB.different methods. We choose three point cloud:airplane, chair ,and mug. to be tested across a comparable bits per pixel (bpp) range. The comparative analysis reveals that at the lowest code rate, our method preserves the ground truth's shape information to the greatest extent while simultaneously achieving the highest Peak Signal-to-Noise Ratio (PSNR).We conduct ablation studies to examine the impact of key components in the model. Specifically, we investigate the effectiveness of low-frequency features, high-frequency features, and the loss function designed in Sec. 3.2.4. As shown in Table 2 , utilizing solely low-frequency features to guide the reconstruction of the diffusion model results in a 20% reduction in the code rate, along with a decrease in the reconstruction quality by 0.397dB. This indicates that high-frequency features play an effective role in guiding the model during the reconstruction process. Conversely, discarding the low-frequency features, which represent the shape of the point cloud, leads to a reduction in the code rate and significantly diminishes the reconstruction quality. Therefore, we argue that the loss of the shape variable is not worth it. Lastly, we ascertain the impact of D CD (x 0 , x0 ), and the results indicate that this loss marginally increases the bits per point (bpp) while diminishing the reconstruction quality.  
limitations: 
 Although our method has achieved advanced rate distortion performance and excellent visual reconstruction results, there are several limitations that warrant discussion. Firstly, the encoding and decoding time are relatively long, which could potentially be improved by the acceleration techniques employed in several explorations [18, 19] . Secondly, the model is currently limited to compressing small-scale point clouds, and further research is required to enhance its capability to handle large-scale instances. 
conclusion: 
 We propose a diffusion-based point cloud compression method, dubbed Diff-PCC, to leverage the expressive power of the diffusion model for generative and aesthetically superior decoding. We Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Experimental setting and details are fully disclosed in Sec. 4.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Error bars are not reported due to the specificity of the compression task. The rate-distortion curve (Fig. 3 ) and Bjontegaard metric (Tab. 1) could be convincing enough.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: Sufficient information on the computer resources is disclosed in the experiment setting (Sec. 4.1).Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: This research does not involve human subjects or participants. This paper conform with the Code of Ethics in every respect.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no safeguard risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: This paper follows the license of the datasets used. Original papers are properly cited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: This paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.• According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. 
 
 Appendix: Justification: There is no societal impact of the work. The proposed method is limited to compression and reconstruction and cannot be used to generate deepfakes or disinformation.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: The reviewers agreed that this paper is not ready for publication yet.
In particular, there was a concern about the evaluation of the method and its stochasticity - how the sample diversity affects reconstruction fidelity and what benefits can it bring. Concerns about comparison with more recent works on more common datasets, were not adequately addressed.

Reviews: review: paper_summary: paper_summary: In this paper, they introduce the diffusion-based point  cloud compression method, dubbed Diff-PCC, to leverage the expressive power of the diffusion model for generative and aesthetically superior decoding. They get better performance than G-PCC and two deep learning methods.
strength_weakness: strength_weakness: Firstly, how do we obtain a point cloud with added noise in the decoder? We have no knowledge of any other information about the original point cloud, except for the information in the bitstream. This will result in the inability to decode.
This manuscript claims to achieve state-of-the-art compression performance, but it only compares with two deep learning methods from the past two years. It does not compare with the most advanced methods such as CNet, SparsePCGC, and so on.
questions: questions: How do we obtain a point cloud with added noise in the decoder? We have no knowledge of any other information about the original point cloud, except for the information in the bitstream.
How does your method's performance compare to CNet and SparsePCGC?
How does your method perform on datasets such as MPEG 8i and MVUB?
limitations: limitations: The decorder will not work
paper_summary: paper_summary: In this work, the authors propose a diffusion-based point cloud compression framework. Low frequency and high frequency features are extracted via PointNet and PointPN from input point clouds, which are quantized and encoded for compression. During decompression, the quantized features would be decoded to condition a diffusion model to construct the decompressed results. The experiments on 15K points data from ShapeNet, ModelNet10, and ModelNet40 show superiority over compared methods.
strength_weakness: strength_weakness: 1. The comparison is not convincing enough. Some commonly used compression methods are not compared, while the evaluation is limited to sparse point clouds with relatively simple structures from ShapeNet, ModelNet;

2. The motivation of using diffusion model for compression is questionable. As a sampling-based framework, diffusion models may construct different results during decompression from variant sampled noises each time. I am not so sure if the diffusion model is more appropriate than existing AE or VAE-based frameworks for the compression task, which may need decompression as accurate as possible;
Please check the questions for more details, thanks.
questions: questions: 1. Some popular methods are not compared, including PCGC[1], PCGCv2[2], and 3QNet[3];

[1] Lossy point cloud geometry compression via end-to-end learning

[2] Multiscale point cloud geometry compression

[3] 3qnet: 3d point cloud geometry quantization compression network

2. Only sparse objects on ShapeNet and ModelNet are used for comparison. How is the compression performances on more complex and dense shapes, including 8iVFB[4] or  RWTT[5]? Besides, point clouds with 15K points are too sparse for evaluation as dense points are main targets for compression. Methods mentioned in Question 1 can deal with dense points.

[4] 8i voxelized full bodies-a voxelized point cloud dataset

[5] Real-world textured things: A repository of textured models generated with modern photo-reconstruction tools

3. How do you deal with the uncertainty of diffusion models? The sampling-based generation process may produce different decompressed results between multiple inferences;

4. Could you compare the computational cost, e.g., compression efficiency between different methods?

Some minor problems: 

5. In Eq.20, how do you calculate $\bar{x}_0$? As the whole denoising process may be not affordable during training.

6. In Eq.14, is the $F_{in}$ actually $F'_{xt}$ in Eq.13?
limitations: limitations: Yes.
paper_summary: paper_summary: The paper proposes the first diffusion-based point cloud compression method called Diff-PCC.  
A dual-space latent representation is devised in this paper, where a compressor composed of two independent encoding backbones is used to extract expressive shape latents from different latent spaces.    
At the decoding side, a diffusion-based generator is devised to produce high-quality reconstructions by considering the shape latents as guidance to stochastically denoise the noisy point clouds.    
Experiments demonstrate that the proposed Diff-PCC achieves state-of-the-art compression performance (e.g., 7.711 14 dB BD-PSNR gains against the latest G-PCC standard at ultra-low bitrate) while attaining superior subjective quality.
strength_weakness: strength_weakness: More work on diffusion model for data compression could be discussed, like ‘Idempotence and Perceptual Image Compression, ICLR 2024’. In addition, although this paper focuses on point cloud compression, the way of applying diffusion model should be compared with those learned image compression works in the related work part. From my impression, the method in this paper is still novel compared with those learned image compression paper using diffusion model.    

More recent learned point cloud compression method [30][14] should be compared in Table 1, Figure 3 and Figure 4, regarding rate distortion and encoding/decoding speed. Besides, only object point cloud is considered currently, large scale point cloud like SemanticKITTI could be compared [30][14].  
 
It is not clear how the speed is measured in Table 1. The hardware and commend line shoud be provided in the supplementary material.  

Minor:  
L86, Point·E[] is a typo.  
[30] and [31] are the same.   
L202, the reference should be fixed.   
What is the FPS in eq 14? farthest point sampling?
questions: questions: see the weakness part
limitations: limitations: The limitation is addressed in the manuscript.

Comments: title: Reply to rebuttal
Response
comment: -Q1:  The importance of conducting sparse point clouds compression is not so strong as dense point cloud compression.

-A1:  Thank you very much for your reply. We agree with your viewpoint.
    Undoubtedly, dense point clouds are more common in practical applications.
    The dense point cloud compression offers a wider range of potential applications.
    However, the original purpose of Diff-PCC was to combine Diffusion with Point Cloud Compression(PCC) to explore the feasibility of this novel technological approach.
    Although it seems more suitable for sparse point clouds at present, it does not imply that there is no room for improvement in future work.
    To extend its application to dense point clouds, we are experimenting with the following two methods:
    
(1) DDPM-based Upsampling: By utilizing the skeleton points and corresponding features of dense point clouds as conditions, we employ diffusion model to upsample large-scale of point, thereby reconstructing dense point clouds.
    
(2) LDM-based PCC: We map dense point clouds into a latent space with lower dimensions and then diffuse them to reduce computational demands.
    
Finally, we would like to respectfully point out:
LiDAR point clouds, as a type of large-scale sparse point cloud, are widely used in the field of autonomous driving and are becoming increasingly important. Voxel models are not adept at handling this kind of data, while point-based models like ours naturally have great potential for processing these point clouds.
  
-Q2: how do you compress the centers and scales of different patches?

-A2: Yes. We calculate the mean and variance of the point cloud patches and normalize the patches, resulting in arrays with shapes (1, 3) and (1, 1), respectively. The data type is float32, which is expected to occupy 24 bytes for storage.
In fact, we do not compress the center and scale, but transmit them directly to the decoding end for inverse normalization during this rebuttal period. We will consider to use octree coding to compress the centers.
Thank you very much for raising the question!

-Q3: About the uncertainty.

-A3:
  Thank you for your reply.
  To address your concerns about the uncertainty of the sampling results,
  In our experiments, we have chosen to fix the random seed, and the code is as follows:
  

    torch.manual_seed(2024)
    np.random.seed(2024)
    random.seed(2024)


By using this setting, we can ensure that the same random noise is taken during multiple sampling processes, thereby producing stable and consistent decompression results.
    Despite this, randomness can still lead to some additional issues, such as outliers and rough edges, which are problems we aim to address in the future.

 The above are our views on your question. Thank you very much for your reply. We welcome any insightful suggestions to improve our work.
We apologize for not thoroughly cleaning up your confusion.Now we will try our best to explain it below:

 -Q1:is this reverse process still effective.

 -A1:In fact, the reverse process contains the information of X_0 because C contains the features extracted from X_0.
We know that the reverse process starts from X_T (a completely random Gaussian distribution), continuously predicting and removing noise.The generator combines the conditional information C, the time step t, and the noise point cloud X_t to predict the noise.If we do not use C to guide the generator to predict specific noise, the reconstructed point cloud is likely to be very different from the original point cloud. For example, if we input an armchair, without C guidance, it is likely to reconstruct a chair without armrests.However, by following the guidance of C, we can limit the denoising direction and ultimately reconstruct an armchair corresponding to the original point cloud.This process demonstrates the strong generative capabitliy of diffusion model in point cloud generation.We will release our codes and pre-trained model in the near future to demonstrate the reverse process is effective and our decoder can work effectively as illustrated in the paper.

-Q2:Compare with state-of-the-art methods on more common dafasets such as MPEG 8i, Ford, KlTTl. etc.

-A2:
Thank you for agreeing that “Compressing point clouds using diffusion models is a good idea”,
    and we  agree that “this work still needs further improvement”.
    We will  try our best to solve your doubts and the following is our response to this question:
    By dividing the point cloud into patches, we compared four human point clouds (longdress, loot, redblack, soldier) from MPEG-8i with G-PCC, as shown in the attached PDF document.
    The configuration file of G-PCC refers to PCGV2.
We set some parameters as follows:

 
     --positionQuantizationScale=1
     --trisoupNodeSizeLog2=0
     --neighbourAvailBoundaryLog2=8
     --intra_pred_max_node_size_log2=6
     --inferredDirectCodingMode=0
     --maxNumQtBtBeforeOt=4
 
For different point clouds, we select different positionQuantizationScale to control BPP to meet our needs.
In addition, since SparsePCGC, CNet, and other methods are not fully open-sourced, it is difficult for us to reproduce and we are unable to compare with them currently.

Finally, we welcome any insightful suggestions to improve our work. Thank you very much.
Thank you for addressing the concerns raised in our initial feedback. We acknowledge the effort put into revising the manuscript. However, upon further review, we find that some of the issues highlighted in the weaknesses section are still not adequately resolved.
1. As writing in the manuscript. The generator takes noisy point cloud x_t at time t and necessary conditional infommation C as input. lf x_t is from x_T (a completely random Gaussian distribution), the reverse process has no information about the original point cloud x_0. In other words, for any original point cloud x_0, the diference in their x_t is only due to randomness, rather than their own characteristics. So is this reverse process still effective?
2. G-PCC has no relevant configuration fles, which will lead to a decrease in performance. Therefore, l think the experiment on ShapeNet and ModelNet is unfair. You said your model is more generic, which support the compression of point clouds of any types, spare or dense, voxelized or non-structured. So you should compare with state-of-the-art methods on more common dafasets such as MPEG 8i, Ford, KlTTl. etc.
Compressing point clouds using diffusion models is a good idea, but this work still needs further improvement. After thorough consideration, we have decided to maintain our original evaluation and rating of the manuscript.
1. As writing in the manuscript 'The generator takes noisy point cloud x_t at time t and necessary conditional information C as input.' If x_t is from x_T (a completely random Gaussian distribution), the reverse process has no information about the original point cloud  x_0. In other words, for any original point cloud  x_0, the difference in their x_t is only due to randomness, rather than their own characteristics. So is this reverse process still effective? 
2. G-PCC has no relevant configuration files, which will lead to a decrease in performance. Therefore, I think the experiment on ShapeNet and ModelNet is unfair. You said your model is more generic, which support the compression of point clouds of any types, spare or dense, voxelized or non-structured. So please compare with state-of-the-art methods on more common datasets such as MPEG 8i, Ford, KITTI, etc.
Compressing point clouds using diffusion models is a good idea, but this work still needs further improvement.  Therefore, we maintain our previous conclusion
Thanks for the rebuttal of the authors. It has addressed some of my concerns. However, I still have some problems for now:
(1) I agree that PCGC and PCGCv2 may be unsuitable for sparse point cloud compression. But I hold the opinion that the importance of conducting sparse point clouds compression is not so strong as dense point cloud compression.
(2) For your comparisons on dense point clouds, do you normalize the points in different patches? If you do that, then how do you compress the centers and scales of different patches?
(3) For the claim about the uncertainty, I think the authors should provide some experiments about the variance of the decompressed results between multiple inferences. Otherwise, it may be difficult to judge if the diffusion framework is appropriate for the task of point cloud compression, due to its instability;
Thanks for the reply