ViewPCGC: View-Guided Learned Point Cloud Geometry Compression

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 09:47:15

Abstract: With the rise of immersive media applications such as digital museums, virtual reality, and interactive exhibitions, point clouds, as a three-dimensional data storage format, have gained increasingly widespread attention. The massive data volume of point clouds imposes extremely high requirements on transmission bandwidth in the above applications, gradually becoming a bottleneck for immersive media applications. Although existing learning-based point cloud compression methods have achieved specific successes in compression efficiency by mining the spatial redundancy of their local structural features, these methods often overlook the intrinsic connections between point cloud data and other modality data (such as image modality), thereby limiting further improvements in compression efficiency. To address the limitation, we innovatively propose a view-guided learned point cloud geometry compression scheme, namely ViewPCGC. We adopt a novel self-attention mechanism and cross-modality attention mechanism based on sparse convolution to align the modality features of the point cloud and the view image, removing view redundancy through Modality Redundancy Removal Module (MRRM). Simultaneously, side information of the view image is introduced into the Conditional Checkboard Entropy Model (CCEM), significantly enhancing the accuracy of the probability density function estimation for point cloud geometry. In addition, we design a View-Guided Quality Enhancement Module (VG-QEM) in the decoder, utilizing the contour information of the point cloud in the view image to supplement reconstruction details. The superior experimental performance demonstrates the effectiveness of our method. Compared to the state-of-the-art point cloud geometry compression methods, ViewPCGC exhibits an average performance gain exceeding 10% on D1-PSNR metric.

Main paper: 
 related work 2.1 learning-based point cloud compression: 
 Over recent years, the field of deep learning has seen significant growth, leading to an increased number of learning-based methods in point cloud geometry compression. According to the data structure, the leading point cloud geometry compression frameworks can be classified into three main types: point-based methods, octree-based methods, and voxel-based methods. Point-based methods draw inspiration from works like PointNet [33] and PointNet++ [34] , utilizing 3D convolutional networks to primarily extract features from point clouds and restore them at the decoding end. Yan et al. [48] introduce the first learning-based point cloud compression framework, defining a series of optimization paradigms for this field. Gao et al. [14] employ graph convolutional methods to construct local graphs for the neighbors of each point and associate attributes, performing well on the ShapeNetCorev2 dataset [5] . Multimodal compression aims to enhance data compression efficiency by fusing diverse data types. The core advantage of the method lies in its ability to leverage the complementary information among different modalities, thus achieving a higher compression ratio than single-modality compression. To date, image compression has witnessed the emergence of an extensive array of multimodal compression schemes across its various subfields. In stereo image compression, Liu et al. [24] are the first to propose a learning-based stereo image compression network that uses a parameter skip function to remove modality redundancy between two views. Although the performance improvement is not significant, it lays the foundation for subsequent research. Deng et al. [10] utilize a homography matrix to apply a transformation from the left view to the right view and obtained performance gains based on residual encoding. Zhang et al. [51] propose CAMSIC, introducing a novel content-aware masking image modeling technique and adopting a transformerbased entropy model to capture spatial and disparity dependencies, achieving a performance breakthrough.In RGB-X image compression, Chen et al. [8] use a prior fusion method to extract multiscale information from the encoder and decoder, fusing cross-modality features to assist in the generation [44] ), and a sparse self-attention (SSA) module. Conversely, the "UIS" module parallels "DIS" in structure but diverges by substituting downsampling with upsampling processes. "MRRM" is an acronym for Modality Redundancy Removal Module. The abbreviations "Conv" and "SConv" refer to 2D convolution for image data and 3D sparse convolution for point cloud data, respectively. "CCEM" represents Conditional Checkerboard Entropy Model. "RFD" and "RFU" are acronyms for residual feature downsampling and upsampling. "VG-QEM" denotes View-Guided Quality Enhancement Module. "Q" denotes Quantization. "AE" and "AD" are arithmetic encoder and decoder, while "OE" and "OD" are octree encoder and decoder. of depth maps. Lu et al. [25] design a multimodal compression framework that uses learnable parameters to perform affine transformations, converting the latent features of infrared features into RGB features and aggregating the features to achieve efficient multimodal compression. Zheng et al. [52] introduce intra-modality attention and cross-modality attention, effectively eliminating redundancy between modalities and significantly enhancing compression performance. In text-guided image compression, Jiang et al. [21] address issues such as blurriness and loss of detail at low bitrates by using text as semantic prior information to guide image compression, significantly enhancing compression quality at low bitrates. Qin et al. [35] leverage the CLIP text encoder, integrating text-level semantic details into image decoding to enhance the compression process.In the field of point cloud geometry compression, attention is increasingly attracted towards the utilization of view information. Lin et al. [23] introduce a pioneering multimodal compression framework specifically designed for lidar point clouds. This innovative approach leverages depth estimation techniques to reconstruct 3D scene information from 2D images. The reconstructed scene information is then systematically aligned with different levels within an octree structure, aiding in the accurate estimation of symbol probability densities.From the analysis above, we can infer that current multimodal compression works are primarily concentrated in the image compression domain, with multimodal compression targeting point clouds being exceedingly rare. This scarcity is attributed to the highly sparse and irregular characteristics of point cloud data, which make it challenging to effectively extract and utilize highlevel abstract semantic features, thereby limiting the ability to align information with other modalities. The work proposed by Lin et al. [23] presents one of the few multimodal compression endeavors involving point clouds and images. However, this work fails to adequately address modal redundancy, which in turn limits the potential for notable performance improvements. Therefore, in the current field of point cloud geometric compression, particularly for dense object geometries, there is an urgent need for targeted multimodal compression works. 
methodology 3.1 overview: 
 The overall architecture of ViewPCGC is depicted in Fig. 1 . The network is innovatively designed with a dual-branch structure. It integrates a framework based on sparse convolution to compress point cloud data, while utilizing a local attention module to extract features from images. The original point cloud 𝑥 𝑝 and the corresponding view image 𝑥 𝑣 are fed into the encoder. Here, 𝑥 𝑝 is subjected to downsampling and feature aggregation through DIS, while 𝑥 𝑣 undergoes downsampling and residual feature extraction via RFD. Subsequently, the latent representations of both modalities are fed into the MRRM for modality interaction, effectively reducing the redundancy in the point cloud modality. Following the encoding process, we obtain the latent representations for the point cloud, 𝑦 𝑝 , and the view image, 𝑦 𝑣 . The 𝑦 𝑝 is then divided into the coordinate component, 𝑦 𝑝𝑐 , and the feature component, 𝑦 𝑝 𝑓 , to facilitate further compression. Both 𝑦 𝑝 𝑓 and 𝑦 𝑣 are directed to the quantizers, while 𝑦 𝑝𝑐 undergoes octree-based compression through OE and OD. The quantized latent representations for the point cloud features, ŷ𝑝𝑓 , and the image, ŷ𝑣 , are then input into CCEM for precise symbol probability estimation. On the decoder side, ŷ𝑝𝑓 and 𝑦 𝑝𝑐 are concatenated to obtain ŷ𝑝 , which is used for point cloud feature reconstruction. Both ŷ𝑝 and ŷ𝑣 are then processed through UIS and RFU, respectively, for upsampling and feature restoration. Finally, the VG-QEM is utilized to enhance the geometry structure and finer details of the point cloud further, thereby yielding the reconstructed point cloud x𝑒 𝑝 . We designate the encoder, split operation, quantizer, concat operation, and decoder succinctly represented as follows:EQUATIONTo clearly describe the details of each module, we illustrate the detailed structures of certain modules (including DIS, UIS, RFD, RFU, SSA) in the Supplementary Materials. It should be emphasized that due to the massive volume of point cloud data, we adopt linear residual mechanism in place of non-local attention mechanism [46] , which significantly improves the inference speed while essentially maintaining consistent performance.In the encoder and decoder stages, we utilize the MRRM to eliminate modal redundancy. The framework of the MRRM is shown in Fig. 2 . MRRM contains two branches, each receiving the input latent representations of the point cloud 𝑓 𝑝 and the image 𝑓 𝑣 , respectively. For point cloud processing, SRB is used to obtain aggregated point cloud features 𝑓 * 𝑝 , while RB is employed on the image side to attain extracted image features 𝑓 * 𝑣 . Subsequently, the features extracted from both are input into sparse cross-attention (SCA) to get the output point cloud latent representation 𝑓 ′ 𝑝 . Similar to sparse selfattention (SSA), we adopt the residual mechanism instead of scale dot-product attention to accelerate the inference process. Within SCA, the latent representation of the image is reshaped into a twodimensional tensor 𝑓 𝑠 𝑣 that resembles the sparse tensor feature structure. Through continuous linear layer transformations, channel dimension alignment is achieved. The Sigmoid function maps image features to a mask 𝑓 𝑚 𝑣 with values ranging from [0, 1], which is then used to weight the extracted point cloud latent representation 𝑓 𝑒𝑥 𝑝 . 𝑓 𝑒𝑥 𝑝 is obtained from 𝑓 * 𝑝 through consecutive SRBs and SConv. The final output is the point cloud latent representation 𝑓 ′ 𝑝 , which have undergone modality interaction. The processing of SCA can be summarized as:EQUATIONwhere 𝜎 (•) represents the Sigmoid function, and 𝐹 𝐿 (•) encompass the continuous linear layers. After the application of SCA, we concatenate 𝑓 ′ 𝑝 with 𝑓 * 𝑝 along the channel dimension. The concatenation facilitates the feature fusion, resulting in a refined latent representation denoted as 𝑓 𝑟 𝑝 . Subsequently, the latent mask 𝑚 𝑝 is derived by mapping 𝑓 𝑟 𝑝 to the weight domain, which serves to selectively emphasize certain region within the point cloud latent representation. Finally, we remove the redundant information from the input point cloud feature 𝑓 𝑝 . The above processing procedure of MRRM can be formulated as:EQUATIONwhere 𝐹 𝑝 (•) is composed of two consecutive sparse convolution layers. After a thorough examination of the various components of MRRM and their interactions, it is evident that MRRM enhances the expressive capability of point cloud features through effective feature fusion and the elimination of redundant modality information, which is particularly crucial in complex point cloud geometry compression tasks.Previous point cloud geometry compression methods typically model the latent representation directly [43, 44] or employ basic side information in the hyperprior network to aid probability prediction [36] . Such methods, however, encounter limitations in acquiring sufficient prior information, which in turn impacts the capacity to to predict symbol probabilities accurately. VoxelDNN [29] aims to address these limitations by employing an autoregressive context prediction method similar to those utilized in image compression [27] , effectively harnessing spatial information from the adjacent neighborhood. Despite this advancement, the method suffers from exceedingly prolonged decoding time, rendering it impractical for direct model deployment. In response to these challenges, we introduce the Conditional Checkboard Entropy Model (CCEM), which is inspired by [16] . The architecture of CCEM is depicted in Fig. 3 .After the encoding phase, ŷ𝑝𝑓 and ŷ𝑣 are directed towards the hyper encoder, to derive hyperprior latent representations, culminating in ẑ𝑝𝑓 and ẑ𝑣 . The hyperprior latent representations are fed into the hyper decoder, leading to the extraction of spatial distribution information 𝜑 𝑝 and 𝜑 𝑣 , which are divided into anchor parts 𝜑 𝑎 𝑝 , 𝜑 𝑎 𝑣 and non-anchor parts 𝜑 𝑛 𝑝 , 𝜑 𝑛 𝑣 , respectively. It should be emphasized that CCEM aims to refine the accuracy of probability density estimations for point cloud symbols. For images, due to their limited transmission overhead, after obtaining 𝜑 𝑣 , we directly feed 𝜑 𝑣 into the Entropy Prediction Module (EPM) to acquire entropy parameters of Gaussian distribution 𝜇 𝑣 and 𝜎 2 𝑣 . The quantized view image latent representation ŷ𝑣 is characterized as follows:EQUATIONwhere N ( where 𝑔 𝑛 𝑒𝑝 (•) refers to EPM n . In contrast to VoxelDNN, which operates on voxel-level dependencies, our method embraces slice-level dependency, significantly enhancing the decoding speed. Moreover, for the first time, we incorporate side information from different modalities into the entropy model, enhancing its capability to predict symbolic probabilities more accurately. The detailed structures of additional sub-modules within CCEM will be presented in the Supplementary Materials. In the training stage, we define the loss function 𝐿 in the following manner:EQUATIONwhere 𝑅 𝑝 and 𝐷 𝑝 denote the rate loss and Binary CrossEntropy (BCE) loss of the point cloud, respectively. Similarly, 𝑅 𝑣 and 𝐷 𝑣 stand for the rate loss and Mean Squared Error (MSE) loss of the view image. 𝛼, 𝛽 are the hyper-parameters of the optimization paradigm. Furthermore, 𝑅 𝑝 , 𝑅 𝑣 , 𝐷 𝑝 , 𝐷 𝑣 can be expressed as:EQUATIONwhere 𝑁 𝑝 and 𝑁 𝑣 represent the symbol length of ŷ𝑝 and ŷ𝑣 , respectively. 𝑀 signifies the number of scales utilized within the decoder.𝑗 𝑖 stands for the status of a voxel, indicating its occupation or vacancy, and 𝑝 𝑗 𝑖 reflects the likelihood of the voxel being occupied. It is crucial to recognize that although our primary concern in this research is the visual quality of the point cloud, the significance of the view image in reconstructing the point cloud has been previously highlighted in modules like MRRM, CCEM, VG-QEM. Consequently, we have included MSE loss for the view image in the loss function, to some extent preserving the reconstruction quality of the image. Moreover, in order to reduce the bit overhead associated with transmitting images, we aim to minimize the bit rate allocated to images during training. The dependency relationship between image reconstruction and point cloud reconstruction will be elaborated upon in more detail during our ablation study. 
experiments 4.1 datasets: 
 4.1.1 Training Dataset. ShapeNet [5] is a large-scale dataset of 3D objects and stands as one of the most comprehensive datasets with rich annotations of 3D objects to date. The dataset covers several categories such as furniture, vehicles, and buildings. In our study, we randomly select 25,000 3D models from ShapeNet and divide the data into training and validation sets in a 9:1 ratio. These 3D models are processed through sampling and quantization to obtain object point clouds. To generate the view image corresponding to each point cloud, we employ the virtual camera to capture the point clouds from the fixed perspective, thus obtaining training pairs. 4.1.2 Test Dataset. The 8iVFB dataset [11] specializes in highly realistic human three-dimensional scanning data. The MVUB dataset [6] , provided by Microsoft, concentrates on the three-dimensional voxel representation of the upper body. Owlii, a company renowned for its advanced 3D scanning and reconstruction technology, offers a dataset [49] that primarily encompasses high-accuracy 3D human models and movements. These datasets are well-suited to meet the point cloud geometry compression requirements of MPEG standard [37] and JPEG standard [32] . Consequently, for our evaluation, we meticulously select a total of twelve point clouds from 8iVFB, MVUB, and Owlii datasets.We train the entire network jointly. View-PCGC is implemented using PyTorch with CUDA support. During the training process, we utilize the Adam optimizer [22] . The learning rate is initialized at 1𝑒 -4 and gradually decreases as the model updates, eventually reaching 1𝑒 -5 . The batch size is set to 4. Training is conducted on an NVIDIA RTX 3090, with each model undergoing approximately 40 training epochs. The point clouds in the input training pairs are quantized to 7 bits, and view images are cropped to 256 × 256 to facilitate model inference. The hyperparameter 𝛽 is set to 200. Meanwhile, 𝛼 is adjusted over a range from 0.5 to 10. The selection details of the hyper-parameter 𝛽 will be discussed in the Supplementary Materials.We adopt point-to-point PSNR (D1 PSNR) and point-to-plane PSNR (D2 PSNR) [20] as evaluation metrics, which reflect the fidelity of the spatial position of points in the point cloud and the fidelity of the geometric structure of the point cloud, respectively. Furthermore, Bjontegaard delta rate (BD-Rate) [4] is utilized to obtain a quantitative rate-distortion performance.Baseline. We compare our method with several superior deep learning-based point cloud geometry compression methods, including GeoCNNv2 [36] , PCGCv1 [45] , PCGCv2 [44] , and SparsePCGC [43] . Additionally, the classical point cloud encoding tool, G-PCC [28] , is also incorporated into the comparison. The reference implementation for G-PCC is TMC13v23. 1 presents the RD performance comparison of ViewPCGC on 8iVFB, MVUB, and Owlii datasets against other competitive methods. Overall, our proposed method achieves an average improvement over the existing state-of-theart model by more than 10% and 8% on D1-PSNR and D2-PSNR metrics, respectively. Specifically, our method performs best on the Owlii dataset, attributed to its vast number of points, which further reduces the image bitstream expense. The aspect will be further discussed in the ablation study. Moreover, we plot the RD curves for Soldier and Dancer to further visualize the performance gap between various methods, as shown in Fig. 5 . It illustrates that ViewPCGC is capable of better restoring the point cloud structure and improving the signal fidelity of the point cloud as the bitrate increases. 3 . "✓" signifies that the proposed module is utilized within the baseline model. BD-Rate reflects the performance gap between the ablation model and the model proposed in this paper. Notably, when the CCEM is omitted, we employ a factorized entropy model [2] solely on the point cloud as an alternative. Table 3 illustrates the performance gap among models with various configurations, using ViewPCGC as the baseline model.The experimental results underscore the necessity of our proposed modules, among which CCEM contributes most significantly to performance enhancement, which can be attributed to the efficient utilization of modality contexts and accurate symbol probability estimation. To determine the appropriate resolution for the view image, we perform the ablation study with the view image at several resolutions. Using the "256×256" resolution from our scheme as the baseline, Table 4 shows the bit overhead for utilizing the image at other resolutions as auxiliary information (left) and the average performance gap compared to the baseline (right) under D1-PSNR metric across three datasets. The bit overhead 𝑅 𝑜 for the point cloud can be defined as:EQUATIONwhere 𝑅 𝑣 denotes bpp for compressing the view image. 𝐻 and 𝑊 stand for height and weight of the view image, respectively. 𝑁 is the point number of the point cloud. It is clearly observed that with the increase in image resolution, the rapid growth in image bitrate overhead leads to a sharp decline in overall performance. Conversely, when the image resolution is too low, the inability to extract sufficient modality information becomes the bottleneck, limiting performance improvement. To validate the effectiveness of the joint optimization paradigm, we establish control groups for comparative analysis: A1 focuses solely on optimizing the bitrate and distortion of the point cloud; A2 extends this optimization to include the image's bitrate alongside the point cloud's bitrate and distortion; and A3, our adopted method, optimizes both the bitrate and distortion for the point cloud and the image. Results across three datasets using D1-PSNR metric reveal that A3 surpassed A1 by 5.72% and A2 by 8.14%. The results imply that while our goal is to reduce the image's bitrate overhead as much as possible, the quality of the image can not be completely overlooked, as it would affect the utilization of modality information from the image by the point cloud. 
conclusion: 
 In this paper, we propose a novel view-guided learned point cloud geometry compression framework, namely ViewPCGC. To the best of our knowledge, it is also the first work to harness modality information from the projected view in learned object point cloud geometry compression. We employ MRRM to eliminate modality redundancy, CCEM to capture inter-modality context dependencies to assist in symbol probability estimation, and VG-QEM to restore local details of point cloud geometry and maintain spatial structure. The outstanding experiment results unveil that a new perspective of object point cloud geometry compression has been established by emphasizing the critical role of view image information.  
: 
 in various fields such as digital museums [3, 41] , virtual reality [1, 12] , 3D reconstruction [7, 26, 50] , cultural heritage preservation [31, 39] , and medical imaging [38, 42] . In contrast to 2D data (such as images), point clouds have become indispensable due to their ability to describe objects in the physical world more accurately. However, high-quality point cloud data often comes with a huge volume of data, posing significant challenges to storage and transmission. Therefore, efficient point cloud geometry compression technology is particularly important.In fact, point cloud geometry compression methods have developed rapidly in the past decade. Before 2020, research in the field primarily focused on optimizing the MPEG standardized point cloud compression method G-PCC [15] . G-PCC is well known for its excellent compatibility and standardization across different application scenarios and platforms. Nonetheless, the advent of learning-based point cloud compression techniques has amplified the limitations of G-PCC concerning the performance of point cloud geometry compression. Researchers' attention has shifted towards deep learning-based approaches. In the early stages, numerous point-based methods [14, 17, 19, 48] are developed for object point clouds. These methods perform excellently when dealing with simple structures and fewer points, such as ShapeNet [5] , but their performance degrades with dense point clouds due to the massive number of points and poor compression effects. Subsequently, researchers design voxel-based methods [29, 30, 45] targeting the spatial structure of dense point clouds. Point clouds are discretely depicted as three-dimensional grids, enabling more efficient storage and processing of point cloud data. However, the extensive spatial scale of dense point clouds results in many voxels being unoccupied, leading to spatial wastage and significantly affecting computational efficiency. In order to address the shortcoming, some studies [43, 44, 47] reduce the computational complexity by employing sparse convolution [9] , which significantly improved the compression efficiency of learning-based models in dense point cloud geometry.However, the above learning-based models usually extract features from the intrinsic geometric structure of the point cloud, reducing spatial redundancy through successive downsampling. Although they effectively utilize the intrinsic attributes of point clouds, they often overlook the potential insights gained from external views. Compared to point clouds, the bit cost of the view image during transmission is almost negligible. Such view image can provide additional contextual information, which is not easily visible from the point clouds alone, leading to more efficient compression. In light of this, we propose View-Guided Learned Point Cloud Geometry Compression (ViewPCGC) in this paper. The core of this framework is the use of view information to assist in point cloud compression. By leveraging self-attention and crossattention mechanisms based on sparse convolution, we designed Modal Redundancy Reduction Module (MRRM) to eliminate the redundancy between modalities. In the entropy model, we adopt a conditional entropy model, introducing side information of the view hyperprior as context for point cloud encoding, making the probability estimation of entropy parameters more accurate. At the end of the decoder, to fully utilize the view information, we design a quality enhancement network that further uses the structural information of the view to restore local details of the point cloud.The contributions of our proposed method can be summarized as follows:  
 
 Appendix:  


Decision: This paper initially got mixed reviews. The authors' rebuttal addressed all reviewers' concerns, and all reviews reached a consensus on the acceptance of the paper.

Reviews: review: paper_summary: paper_summary: This document introduces ViewPCGC, a novel view-guided learned point cloud geometry compression framework, which leverages self-attention and cross-modal attention mechanisms based on sparse convolution to efficiently compress point cloud data. By utilizing the contour information from view images, ViewPCGC effectively reduces redundancy between modalities, enhances the accuracy of entropy estimation, and improves detail reconstruction in point clouds. Demonstrated through extensive experimentation, ViewPCGC achieves superior performance compared to existing methods, offering significant improvements in terms of compression efficiency and quality of reconstructed point clouds.
strength_weakness: strength_weakness: **StrengthsStrengths**
1. **Innovative Fusion Framework:** ViewPCGC utilizes view-guided learning for point cloud geometry compression, leveraging view information and cross-modal interactions to enhance compression efficiency. This method integrates self-attention and cross-attention mechanisms based on sparse convolution, effectively handling modal redundancies.
2. **Enhanced Compression Efficiency:** The method uses the Modality Redundancy Removal Module (MRRM) and the Conditional Checkboard Entropy Model (CCEM), which significantly enhance the accuracy of probability density function estimation for point cloud geometry. These innovations lead to notable performance improvements over existing methods.
3. **Quality Enhancement:** The View-Guided Quality Enhancement Module (VG-QEM) in the decoder uses the contour information of the point cloud from the view image to reconstruct details more accurately, preserving the geometric integrity and detail of the point cloud.
4. **Robust Experimental Validation:** Extensive experiments demonstrate superior performance compared to state-of-the-art point cloud geometry compression methods, with significant gains in standard metrics like D1-PSNR.
limitations: limitations: **Limitations:**
1. **Complexity and Resource Intensity:** The architecture, while effective, is complex, potentially requiring significant computational resources for processing, especially in real-time applications. This may limit its practical deployment in resource-constrained environments.
2. **Dependency on High-Quality View Images:** The effectiveness of the method is partly dependent on the quality of the view images used for guiding the compression. Poor quality or inadequately aligned view images could potentially degrade the performance of the system.
3. **Limited Research on Modality Discrepancies:** While the method effectively handles modality redundancies through sophisticated modules, the inherent discrepancies and alignment issues between different modalities (e.g., discrepancies between point cloud data and 2D view images) are not thoroughly addressed, which could affect the overall compression quality under varying conditions.
paper_summary: paper_summary: In this paper, the authors propose ViewPCGC, a learning-based compression model for point clouds. ViewPCGC suggests to incorporate a 2D view image into the point cloud encoding and decoding process. Specifically, a model redundancy reduction module (MRRM) is designed to eliminate redundancy between modalities. In the entropy model, the conditional checkboard entropy model (CCEM) is proposed to refine the probability estimation for point cloud compression. In the decoder, a view-guided quality enhancement module (VG-QEM) is used to improve the output point cloud quality with the reconstructed view image. The experimental results show that ViewPCGC outperforms other PCGC methods on various datasets.
strength_weakness: strength_weakness: The draft is well-organized and is easy to follow. Employing the view image to improve the compression performance of point clouds is an interesting idea. The experimental results justify the designs, where the proposed method outperforms other learning-based compression methods on various data.
limitations: limitations: As the major contribution of this paper is that a view image is used in the learning-based compression model, the authors do not illustrate how they generate the view image. I only find a sentence 'To generate the view image corresponding to each point cloud, we employ the virtual camera to capture the point clouds from the fixed perspective, ...'. What is the fixed perspective to capture the view image and why use such a fixed perspective is not discussed. I am also wondering if the view image has colors or not. Since the latent features of the view image are generated and compressed, the authors are also encouraged to also report its size in the compression domain rather than only saying '... the bit cost of the view image during transmission is almost negligible.'

Since the details of generating view images are missing, my major concern is that this paper fails to illustrate why incorporating a view image could improve the performance of a learning-based compression model that uses sparse convolution. Does the view image provide additional information? If yes, what is the additional information and why don't the authors use multiple view images from different angles? If no, what is the reason for the superior performance by using the view image?
paper_summary: paper_summary: Point clouds typically have large volumes, making them inconvenient to transmit over networks. Therefore, compression encoding and decoding are very important. The ViewPCGC point cloud geometry compression framework proposed in this paper introduces image information to optimize the compression and detail reconstruction of point clouds, improving compression efficiency and enhancing compression quality. In addition, the authors conducted extensive ablation experiments on each module, and the overall structure of the paper is complete.
strength_weakness: strength_weakness: see the summary part
limitations: limitations: 1.	The reviewer suggests that the authors provide a detailed explanation of how the sparse self-attention (SSA) module and sparse convolution are implemented, as the current description is not specific enough to allow for reproducibility.
2.	Point cloud compression technology is a hot topic in academic research; the authors should compare their work with the state-of-the-art papers.
Rui Song, Chunyang Fu, Shan Liu, Ge Li; “Efficient Hierarchical Entropy Model for Learned Point Cloud Compression”  CVPR, 2023, pp. 14368-14377
Junteng Zhang et al. “YOGA: Yet Another Geometry-based Point Cloud Compressor”, ACM MM 2023
3.	The author mentioned that “Compared to point clouds, the bit cost of the view image during transmission is almost negligible”. The authors should provide detailed data to substantiate this viewpoint, such as discussions in Section 4.4.2 on the impact of the file size of images at different resolutions on transmission.
4.	As the authors reviewed in Section 4.3.1, the model proposed in this paper performs best on the Owlii dataset, with the author's explanation being that the larger number of points reduces the image bitstream expense. The reviewer believes that a specific analysis should be conducted on the impact of the density and sparsity of point clouds on the algorithm's performance.
5.	If the surface of a point cloud is uneven and bumpy, it can be difficult for the image to cover the surface texture comprehensively. The question remains whether the model can still be effective under such circumstances. The author should conduct more experiments to test the model's scalability.
paper_summary: paper_summary: The core concept of the ViewPCGC framework is to utilize view information to assist in point cloud compression. By leveraging self-attention and cross-attention mechanisms based on sparse convolution, the framework aims to eliminate redundancy between modalities and enhance compression efficiency by incorporating side information from the view image into the encoding process. Additionally, the framework includes a View-Guided Quality Enhancement Module (VG-QEM) in the decoder to leverage the structural information of the view image for restoring local details in the point cloud, thereby improving the reconstruction quality.
strength_weakness: strength_weakness: - The paper introduces a novel view-guided learned point cloud geometry compression framework, which leverages modality information from view images to enhance compression efficiency. This innovative approach sets it apart from traditional compression methods.
- The framework effectively incorporates self-attention and cross-attention mechanisms based on sparse convolution to eliminate redundancy between modalities, leading to improved compression ratios and performance gains.
limitations: limitations: I apologize for my limited understanding of this field, but I think some additional clarifications are required in this paper.

- Table 1 presents several point cloud models for comparison, aligning with previous research practices. However, the lack of explanations within the table content might confuse readers, particularly those new to the field. Is the score of viewPCGC in D1-PSNR considered 100%? How is the average calculated? Providing such details would enhance reader comprehension.
- Why is PCGCv1 not included in Table 2 for comparison?
- To improve clarity, arrows could be added to Table 3 to indicate that a lower BD-rate is preferable. Additionally, in the ablation study case 1, explaining how the gap is calculated and which dataset is used would be beneficial.

Miscellaneous:

- The text in Figure 5 should be enlarged as it is currently difficult to read.