ROI-Guided Point Cloud Geometry Compression Towards Human and Machine Vision

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 13:35:33

Abstract: Point cloud data is pivotal in applications like autonomous driving, virtual reality, and robotics. However, its substantial volume poses significant challenges in storage and transmission. In order to obtain a high compression ratio, crucial semantic details usually confront severe damage, leading to difficulties in guaranteeing the accuracy of downstream tasks. To tackle this problem, we are the first to introduce a novel Region of Interest (ROI)-guided Point Cloud Geometry Compression (RPCGC) method for human and machine vision. Our framework employs a dual-branch parallel structure, where the base layer encodes and decodes a simplified version of the point cloud, and the enhancement layer refines this by focusing on geometry details. Furthermore, the residual information of the enhancement layer undergoes refinement through an ROI prediction network. This network generates mask information, which is then incorporated into the residuals, serving as a strong supervision signal. Additionally, we intricately apply these mask details in the Rate-Distortion (RD) optimization process, with each point weighted in the distortion calculation. Our loss function includes RD loss and detection loss to better guide point cloud encoding for the machine. Experiment results demonstrate that RPCGC achieves exceptional compression performance and better detection accuracy (10\% gain) than some learning-based compression methods at high bitrates in ScanNet and SUN RGB-D datasets.

Main paper: 
 introduction: 
 With the advancement of multimedia technologies, immersive multimedia experiences increasingly capture public attention. Point clouds, epitomizing three-dimensional (3D) representation, offer exact depictions of objects and scenes. Their extensive application, like virtual reality, augmented reality, and autonomous driving, underscores their significance. Unlike two-dimensional (2D) images, point clouds offer a more visceral and comprehensive sensory experience. A point cloud, quite literally, is a "cloud" formed by an aggregation of points. In computational terms, these are typically stored as a series of Cartesian coordinates, constituting point clouds' fundamental geometry data. However, relying solely on this information sometimes fails to fulfill users' visual requirements. Therefore, additional attributes are often appended to point clouds. These include but are not limited to, the color, normal vectors, etc.As 3D scanning sensors evolve, generating vast amounts of point cloud data, the compression of point clouds emerges as a pivotal process. This compression significantly reduces the size of point cloud data and minimizes storage and transmission pressure. The Moving Picture Experts Group (MPEG) is a frontrunner in point cloud compression. They pioneered the establishment of compressing point clouds' geometry and attributes. Meanwhile, the MPEG unveils two point cloud compression standards. The first, the Videobased Point Cloud Compression (V-PCC) [50] employs established video encoding techniques to compress the geometry and attribute of point cloud sequences. The second is the Geometry-based Point Cloud Compression (G-PCC) [25] , which encodes geometry using an octree or a trisoup model. Based on the compressed geometry, attribute information can also be encoded either lossy or losslessly.As neural network-based solutions achieve significant success in image and video compression, learning-based technology gradually finds its application in point cloud compression. Numerous end-to-end point cloud compression methods are proposed. Such as point-based compression methods [9, 14, 44, 47, 48] . Most of these methods take the point cloud as input, use networks like PointNet [31] or PointNet++ [32] for feature extraction, and then construct a variational autoencoder for point cloud compression. However, these methods are mostly limited to handling a fixed number of points, and their compression performance is constrained. Lately, there are many point cloud compression methods based on octrees, such as [8, 13, 35, 37] . In these methods, each octree node is represented by an eight-bit occupancy code. The occupancy codes are then probabilistically estimated using neural network architectures such as Multilayer Perceptron (MLP) or Transformer [6] . Meanwhile, numerous point cloud compression strategies is surrounded by sparse tensors, as in [16, 26, 27, 41, 42] . These methods convert the point cloud into a unified format that intertwines coordinates with attribute features, subsequently applying multi-scale sparse convolution and Arithmetic Encoder to code the point cloud.Despite numerous research efforts in point cloud compression, they primarily focus on fidelity optimization. However, in practical applications, the purpose of compression is to facilitate machine analysis. For instance, in autonomous driving, lossy compression significantly loses detailed information on the outdoor scenes. In 3D reconstruction, the compression process can also result in the loss of semantic information (Contour) in scene data. Therefore, we propose a novel compression paradigm for scene point clouds, drawing inspiration from [2, 24, 28] . This paradigm can enhance point cloud detection performance during compression. A critical characteristic of object detection is the accurate localization of the bounding box in the point cloud, which is why our approach strongly emphasizes the coding of the ROI region. We establish a geometry compression framework for point clouds, including base and enhancement layers. The base layer (BL) is responsible for encoding geometry information. The enhancement layer (EL) adopts weighted supervision, guided by masks from the ROI prediction network (RPN) and ROI Searching Network (RSN). Additionally, we weight the distortion function (RD) calculation in a per-point manner according to the generated mask values. To further enhance the effectiveness of the coding process, we incorporate detection loss into the compression task, enabling joint optimization of compression and detection. In summary, the main contributions include:• To address the issue of semantic information loss in point cloud compression, we propose an ROI-guided point cloud compression paradigm, which optimizes both machine perception performance and visual fidelity simultaneously. • We divide the point cloud encoding into base and enhancement layers, with the former encoding coordinates and the latter encoding residual information. The mask information from our RPN and RSN supervises these residuals. We also incorporate mask information into the distance calculation of RD optimization in a point-wise manner, aiding in focusing on critical areas during encoding. Additionally, our loss function integrates detection loss to more effectively guide the point cloud encoding process tailored for detection tasks. The methods [17, 24, 51 ] also concern the ROI areas coding. 
point cloud compression for human: 
 Existing point cloud compression algorithms focus on fidelity optimization, which encompass data structure representation, encoding/decoding transformations, and entropy estimation. For instance, Huang et al. [14] introduce a learning-based point cloud compression method using an autoencoder and sparse coding structure, achieving high compression with minimal loss. Xie et al. [46] introduce a coding network utilizing sparse convolution and design to extract semantic information for classification tasks concurrently. Ulhaq et al. [40] present a PointNet based codec specialized for classification, offering a superior rate-accuracy tradeoff with significant BD-Rate reduction and propose two lightweight configurations for low-resource devices. Ma et al. [23] propose the human-machine balanced compression method, which utilizes a pre-trained lightweight backbone and a semantic mining module for multi-task feature aggregation, balancing signal and semantic distortion with multi-task learning. Liu et al. [18] propose a learning-based point cloud compression framework optimized for object detection, featuring a gradient bridge function for seamless codec-detector integration and a progressive training strategy, achieving significant compression with maintained detection accuracy. Liu et al. [20] introduce a new point cloud compression framework for human and machine vision, featuring a two-branch structure with a shared octree-based module and a point cloud selection module for sparse point optimization. Although the above methods involve machine perception, most of them are simplistic and do not achieve joint optimization or supervision. 
methodology 3.1 overall framework: 
 We propose a point cloud geometry compression scheme supervised by the ROI region, designed to simultaneously optimize the fidelity and the detection performance of point clouds. As illustrated in Fig. 1 , our approach comprises the following steps: Encoding and Residual Generation. Given the input point cloud (𝑥 0 , 𝑦 0 , 𝑧 0 ), the process begins with quantization (𝑄). Subsequently, the quantized point cloud follows two paths. The first path involves G-PCC encoding to generate a coarse geometry bitstream, forming the BL bitstream. The second path involves de-quantizing the quantized point cloud using 𝑄 -1 to retrieve the geometry coordinates (𝑥 1 , 𝑦 1 , 𝑧 1 ). The residual information 𝑅 is calculated by subtracting these obtained coordinates from the original input. Meanwhile, the RAM extracts finer-grained features 𝐹 from 𝑅.G-PCC Encoder Q Q -1 G-PCC Decoder Q -1ROI Prediction and Mask Generation. Concurrently, the original input point cloud undergoes processing via our specially designed RPN to create masks. This RPN employs a sparse convolutional U-Net architecture [36] , accepting color or instance labels for each point and yielding probability outputs.Mask Processing and Feature Enhancement. The instance information predicted by the RPN is converted into a 3D mask through the RSN. This mask is weighted into feature 𝐹 , forming a new semantic-information-bearing 𝐹 ′ . Then, the 𝐹 ′ is encoded into an EL bitstream by an Entropy Encoder. Moreover, the mask influences the RD optimization process, which guides the bitrate allocation towards the target regions within the point cloud.Decoding and Analysis. On the decoding side, the coarse point cloud from the BL is decoded using a G-PCC decoder, and dequantization (𝑄 -1 ) yields(𝑥 ′ 1 , 𝑦 ′ 1 , 𝑧 ′ 1). An Entropy Decoder decodes the EL bitstream to obtain 𝐹 ′′ , which, through an RSM, reconstructs the residual features 𝑅 ′ . The restored point cloud [21] for detection analysis. Furthermore, the loss function from the detection is integrated into the RD optimization.(𝑥 ′ 0 , 𝑦 ′ 0 , 𝑧 ′ 0 ) is ob- tained by summing 𝑅 ′ with (𝑥 ′ 1 , 𝑦 ′ 1 , 𝑧 ′ 1 ). Then, (𝑥 ′ 0 , 𝑦 ′ 0 , 𝑧 ′ 0 ) is put into the Group-FreeResidual Weighting. Our research introduces a hierarchical mask mechanism that significantly boosts performance. We achieve this enhancement by applying weights to both the residual feature and the loss function. Assuming the input point cloud is 𝑥, we process it through a RPN. Inspired by the U-Net [36] architecture, our RPN adopts a similar approach to U-Net's encoding and decoding stages, enabling more effective feature reuse. The network's encoding path of RPN captures essential low-level features, such as coordinates and local properties, while the decoding path reconstructs higherlevel semantic features. Subsequently, the output (𝑥 𝑟𝑜𝑖 ) of RPN is fed into RSN. Adopting the region searching algorithm, we divide the original point cloud into foreground color map 𝑥 𝑓 𝑔 and background color map 𝑥 𝑏𝑔 . We calculate the nearest neighbor distance of the residual feature map 𝑥 𝑒𝑛ℎ and search the corresponding mask value for each element in the residual feature map (Nearest neighbor matching process). We obtain a new mask 𝑥 𝑚 , and the residual feature map is weighted by 𝑥 𝑚 through the above process. We describe the process as below:EQUATIONwhere 𝑅𝑃𝑁 ( Loss Weighting. We utilize Information Bottleneck (IB) theory [39] to optimize the loss function. For the human vision task 𝑣 and machine vision task 𝑡, we assume 𝐼 (•) denotes mutual information function. 𝑝 (𝑣, 𝑡 | 𝑥) represents the mapping process from the input point cloud to the latent representation, and then to the machine tasks and the reconstructed point cloud. 𝜇 and 𝜏 denote the Lagrange multiplier of human and machine vision. Since 𝑥 𝑏𝑎𝑠𝑒 and 𝑥 ′ 𝑒𝑛ℎ fully conditioned on 𝑥 are completely dependent, then: minEQUATIONEQUATIONwhere 𝐻 (•) and 𝐻 (• | •) denote the entropy and the conditional entropy, respectively. Therefore, we can obtain:EQUATIONwhere 𝑔 𝑎 , 𝑔 𝑠 , 𝑔 𝑝 , and 𝑔 𝑚 denote the transformation of RAM, RSM, RPN, and RSN, respectively. 𝜆 ℎ and 𝜆 𝑚 represent the weight of human and machine vision distortion. We support 𝜆 ℎ 𝐷 ℎ as the proxy for 𝜇𝐼 𝑥 base , 𝑥 ′ enh ; x , and 𝜆 𝑚 𝐷 𝑚 as a surrogate for 𝜏𝐼 𝑥 base , 𝑥 ′ enh ; 𝑡 . Moreover, we apply the weight mask 𝑥 𝑚 to the Chamfer Distance (CD) [7] , the process can be formulate as:EQUATIONwhere 𝑃 1 and 𝑃 2 represent the two point clouds to be compared. 𝑎 and 𝑏 represent the point in 𝑃 1 and 𝑃 2 , respectively. The RSN generates mask information (probability) for each point, represented by 𝑀 1 (•). The first term of Eq. ( 5) incorporates this mask information into the distance calculation between points (𝑎, 𝑏). 𝑀 2 (•) denotes using the nearest neighbor search algorithm to calculate the distance between points (𝑎, 𝑏), recording the distance's index. This index allows the retrieval of the closest mask value from 𝑀 1 (•) to assign to the corresponding point. The second term of Eq. ( 5) applies weighting the distance calculation between points (𝑏, 𝑎) in this manner. Finally, we obtain the total loss function as:EQUATIONIn Eq. ( 6), L 𝑟 , L 𝑣 , and L 𝑡 represent rate, distortion, and machine task loss, respectively. 𝛿 and 𝛽 are hyper-parameters of base and enhancement rate loss. While 𝛼 and 𝛾 adjust the ROI Weight-CD (RW-CD) and machine task loss. The calculation of 𝐷 𝑠𝑝 and 𝐷 𝑑𝑒𝑡 is described in Group-Free [21] .Our ROI region process is divided into two procedure: the RPN and the RSN structure, as shown in Fig. 1 (2) and (3).The ROI Prediction Network employs a U-Net architecture built by the Minkowski Engine [4] . Its foundational block in the RPN, integrates a Sparse Convolution (SConv) with two Residual Block A (ResBlockA) layers [12] to create a feature extraction module. This module conducts down-sampling through a combination of SConv and ResBlockA, and up-sampling by pairing Sparse Transposed Convolution (SConvTr) with ResBlockA, incorporating a skip connection operation. These skip connections effectively link layers of matching stride sizes, ensuring smooth feature transitions between the down-sampling and up-sampling phases. The structure of ResBlockA, shown in Fig. 2 (c using Softmax. During the Region Selection, background and foreground are identified, "furniture", "doors", "floors", and "walls" are selected as the background, with the remaining categories comprising the foreground. The Mask Searching process transforms the foreground and background into a mask tensor. For the mask information of the foreground area, we apply it to the residual features with double the weight. For the background area, we maintain the original mask values unchanged. Then, the Feature Alignment Module (FAM), which includes an FC layer and a convolution layer, aligns the generated mask to match the dimensions of the residual feature. The aligned result is then multiplied by the residual feature map, as described in Eq. ( 1). This process is referred to as strong supervision. Meanwhile, we apply weak supervision to the RD loss optimization process through per-point weighting as in Eq. ( 5).Encoder. We adopt the residual coding method commonly used in multimedia coding [28] . As shown in Fig. 1 (1), this process following a coarse-to-fine coding strategy. Suppose the input is denoted as (𝑥 0 , 𝑦 0 , 𝑧 0 ), and it first undergoes coordinate quantization, followed by a rounding operation for integer conversion. Then, we employ the G-PCC codec to compress the quantized coordinates, thus generating the BL bitstream. Lately, we utilize the de-quantization operation 𝑄 -1 to obtain finer geometry coordinates (𝑥 1 , 𝑦 1 , 𝑧 1 ). These reconstructed coordinates are differenced from the original coordinates (𝑥 0 , 𝑦 0 , 𝑧 0 ) to extract residual information 𝑅. The next step involves feature extraction and residual encoding using the RAM and Entropy Encoder, as shown in Fig. 2 (a) . The RAM is divided into two sub-modules: the first is feature extraction, combining KNN [11] , Centering, and MSFEM (Fig. 2 (g)) to extract latent features of the residual. Compared to PointNet, MSFEM is more adept at extracting multi-scale and local details information from the point cloud. The second module comprises two downsampling stages, which utilize sparse convolution, three Inception-Residual Networks (IRN: Fig. 2 (f )), and one SAM (Fig. 2 (e)), to further refine the features of the residual. Constructed from Residual Blocks B (ResBlockB: Fig. 2 (d) ), the SAM expands the receptive field and enhances performance in downstream tasks. Our experimental results highlight the significant role of the SAM in improving the detection performance. Let 𝑥 be the input tensor, 𝐶𝑜𝑛𝑣 𝑘,𝑠,𝑑 (•) a convolution with kernel size 𝑘, stride 𝑠, and dilation 𝑑, 𝐵𝑁 (•) batch normalization, 𝑅𝑒𝐿𝑈 (•) the ReLU activation function, ⊕ tensor element-wise addition, and 𝐷𝑆 (•) down-sampling of 𝑥 if needed. Then, the SAM is defined as:EQUATIONwhere 𝑜𝑢𝑡 1 , 𝑜𝑢𝑡 2 , and 𝑜𝑢𝑡 3 represent the outputs of the consecutive layers in the SAM. The final output is the ReLU activation of the element-wise sum of 𝑜𝑢𝑡 3 and the residual (either 𝑥 or the downsampled 𝑥). After extracting features from the residuals twice, we employ an entropy model for encoding, obtaining the BL bitstream.Decoder. As shown in Fig. 1 (4), the decoder involves two main steps. Initially, the G-PCC Decoder decodes the BL bitstream, and then de-quantization follows to produce (𝑥 ′ 1 , 𝑦 ′ 1 , 𝑧 ′ 1 ). Subsequently, an Entropy Decoder decodes the EL bitstream to generate 𝐹 ′′ , which is then input into the RSM for feature up-sampling. The RSM includes two fundamental components: the first is a feature up-sampling layer that uses two SConvTr blocks and "Prune Voxel" plus SAM block, as shown in Fig. 2 (b) . Meanwhile, the "Prune Voxel" step removes empty voxels for better reconstruction. The second component, a feature restoration module, involves two linear layers that produce the residual 𝑅 ′ . Finally, the process adds𝑅 ′ to (𝑥 ′ 1 , 𝑦 ′ 1 , 𝑧 ′ 1 ) to reconstruct the point cloud (𝑥 ′ 0 , 𝑦 ′ 0 , 𝑧 ′ 0 ), which is then used for object detection through the Group-Free. Additionally, the detection loss function is added to the RD optimization process.We implement a two-stage optimization process. The first stage encompasses the ROI prediction network, optimized using a Cross Entropy loss function. The second stage focuses on RD optimization, which splits the loss function into two parts. Initially, we address RD optimization for compression tasks by calculating distortion using our unique weighted distortion strategy. Subsequently, we feed the reconstructed point cloud into the detection network to compute the detection loss. This loss becomes an integral part of the overall RD optimization process. Our versatile framework enables joint optimization for various downstream tasks, such as classification and segmentation. The loss function is described as:EQUATIONEQUATIONEQUATIONIn Eq. ( 8), 𝑀 is the total number of categories, 𝑐 specifies a particular category, and 𝑁 denotes the overall class count. 𝑦 𝑖𝑐 indicates each point cloud's assigned category, and 𝑝 𝑖𝑐 represents the predicted probability. In Eq. ( 9), the first item of L 𝑑𝑒𝑡𝑒𝑐𝑡𝑖𝑜𝑛 represents the average loss across all decoding stages of the decoding head in Group-Free, where 𝐿 signifies the stage of decoding. The loss L 𝑑𝑒𝑐𝑜𝑑𝑒𝑟 aggregates five distinct types of losses: objectness prediction, box classification, center offset prediction, size classification, and size offset prediction losses. The second item of L 𝑑𝑒𝑡𝑒𝑐𝑡𝑖𝑜𝑛 represents the loss function of the Group-Free sampling head L 𝑠𝑎𝑚𝑝𝑙𝑒𝑟 . Eq. ( 10) describes the RD optimization integrated with the detection task, utilizing parameters consistent with those in Eq. ( 6). 
experiments 4.1 experimental conditions: 
 Training Dataset. We employ two distinct datasets for compression and detection tasks. The first dataset, ScanNetv2 [5] , is an indoor scene dataset that includes bounding box labels, as well as semantic and instance segmentation labels for objects. ScanNetv2 dataset contains 312. We initially sample each point cloud to 50,000 points and then reduce it to approximately 30,000 points through quantization. The second dataset is SUN RGB-D [38] , a single-view indoor dataset comprising 50,000 RGB and depth images. Each associated point cloud includes semantic labels and bounding box information. While the original annotations cover 37 object categories, our detection efforts concentrate on the ten most prevalent categories. We sample the original point clouds of SUN RGB-D to approximately 20,000 points, which serve as inputs for our compression network. The SUN RGB-D test dataset includes 500 point clouds. For brevity in subsequent experiments, we will refer to ScanNetv2 as ScanNet.Training Setting. We utilize the ScanNet and SUN RGB-D datasets to train RPN architecture, adhering to the standard data partitioning strategy outlined in [21, 30] . We employ a Momentum SGD optimizer with an initial learning rate at 1𝑒 -1 throughout the training process of RPN. We incorporate various data augmentation techniques, such as random scaling, rotation, and translation, to enhance the model's generalization capabilities. Following previous research practices, we select the mean Intersection over Union (mIoU) as evaluation metrics for the RPN model to ensure the reliability and consistency of the results.In the training phase of the compression task, we apply the RD plus detection loss defined in Eq. ( 10) as our optimization function, and the training parameters are consistent with Group-Free [21] . We set the rate loss parameter 𝛽 to 1, the distortion parameter 𝛼 to {1, 2, ..., 5}, the detection parameter 𝛾 to 0.01, and adjust the quantization value range from 0.15 to 0.45. For measuring distortion in the compression, we utilize D1 PSNR and D2 PSNR [15] , with bits per point (bpp) as the rate metric and calculating the gain using DB-PSNR. We employ the following methods as the benchmark: (1) G-PCC [25] TMC13 v22 octree codec and AVS PCRM [10] V12 codec.(2) PCGCv2 [42] and SparsePCGC [41] (abbreviated as SPCGC in the table) lossy mode without offset prediction. We employ the mean Average Precision (mAP) under different IoU thresholds as the evaluation criterion for the detection task, including mAP@0.25 and mAP@0.5 thresholds. We also consider factors such as the model size and coding runtime. For the detection, we employ the Group-Free as the detection framework for joint training and testing.It is important to emphasize that due to the relatively low performance of the ROI prediction task (with mIoU only 60%), when calculating the mask values for the SUN RGB-D dataset, we use the original segmentation labels instead of the predictions from the RPN model. This approach is primarily adopted to ensure our weighted method operates on a more accurate prediction base. To evaluate the generalizability of our method on the MPEG dataset, we extend our experiments to the MVUB [22] dataset, which includes the Phil, Sarah, and Queen data samples. These samples are quantized to 10 bits. For a comparative analysis, we incorporate learning-based methods such as pcc_geo_cnn_v1 [33] , pcc_geo_cnn_v2 [34] , and PCGCv1 [43] . Additionally, we include G-PCC Octree/Trisoup and AVS PCRM codec for a comprehensive evaluation.Human Fidelity Evaluation. As shown in Fig. 3 (a) and (e), RPCGC demonstrates superior compression performance at a higher bitrate on the ScanNet dataset compared to other learning-based methods. Nonetheless, it still lags slightly behind G-PCC. This discrepancy is primarily attributed to the significant variability in the dataset's distribution, which presents considerable compression challenges for RPCGC. The sampling of the ScanNet dataset consists of 50,000 points, with the quantized data approximately amounting to 20,000 points, and a portion of this distribution is somewhat uneven, placing these data between dense and sparse distributions. Therefore, traditional methods like G-PCC are more effective in restoring the original coordinates, while learning-based methods suffer from a degree of randomness in reconstruction due Meanwhile, its performance is slightly less than PCGCv2 when assessed using the D2 PSNR. The quantitative analysis shows in Tab. 1 using BD-PSNR metric, reveals that on the ScanNet dataset, RPCGC surpasses G-PCC, AVS, PCGCv2, and SparsePCGC in D1 PSNR by 0.184, 12.109, 9.129, and 1.211, respectively. In D2 PSNR, RPCGC outperforms these models by -0.084, 11.984, 4.822, and 2.769, respectively. On the SUN RGB-D dataset, RPCGC's D1 PSNR gains over the same models are 0.838, 0.252, 2.834, and -0.126, respectively, with D2 PSNR improvements of 0.576, -0.185, -1.414, and 0.308. In summary, RPCGC shows superior compression on ScanNet in high bitrate, indicating enhanced performance over current learningbased methods. However, its efficacy on SUN RGB-D is limited, potentially due to sparser distribution within the dataset.Machine Vision Evaluation. Fig. 3 (c ) and (g) display the detection performance of various algorithms on the ScanNet dataset. Unlike other algorithms that put compressed data into a Group-Free detector, RPCGC utilizes a joint optimization approach. This strategy, along with the unique module we designed, enables RPCGC to significantly outperform traditional and learning-based methods at higher bitrate ranges (0.5-3 bpp). However, at lower bitrates, G-PCC and AVS demonstrate superior performance. This is because the reconstructed point clouds at low bitrates lose a significant amount of contour information on the object, leading to a lower detection accuracy of RPCGC. This challenge is consistent with that observed in algorithms like PCGCv2 and SparsePCGC. We Table 3 : The comparison of coding times tested on NVIDIA T4 GPU. "Enc" and "Dec" denote the encoding and decoding time with both units in seconds (lower is better). The coding time of RPCGC is within an acceptable range. attempt various strategies, including up-sampling, to improve detection performance at low bitrates, yet the improvements remain modest. Furthermore, in practical applications, it is evident that detection performance deteriorates at lower bitrates, reducing its utility, as machine analysis tasks typically operate at higher bitrates. Therefore, our method is likely better suited for applications that require high bitrates specifications. Fig. 3 (d ) and (h) show the rate-detection (R-mAP) curves for the SUN RGB-D dataset. RPCGC exhibits a competitive advantage at higher bitrate ranges. However, it is important to note that the detection performance significantly deteriorates at lower bitrates, which has limited value in practical application. Tab. 2 presents the average detection precision at mAP@0.25 and mAP@0.5 for the ScanNet and SUN RGB-D datasets. Each algorithm computes detection accuracy at a similar bpp. The analysis yields two main conclusions: (1) The detection performance of the point clouds compressed by G-PCC and AVS is consistent. (2) RPCGC surpasses the state-of-the-art detection results of learning-based methods by an average of roughly 10%, highlighting the effectiveness of our designed approach. Further, we evaluated the encoding and decoding times of RPCGC on T4 GPU during inference testing. According to Tab. 3, both the encoding and decoding durations are within an acceptable range. Additionally, our compression model is memory-efficient with a size of 4.5M. Fig. 4 illustrates that our method captures more detailed information than other algorithms.To evaluate the RPCGC's performance, we conducted ablation studies from three perspectives: (1) Variations in the detection network, (2) The effects of our designed modules on compression and detection tasks, and (3) The generalizability and robustness of RPCGC across different datasets.Detection Models. We evaluate the effectiveness of our approach through joint testing with different detection models. Specifically, we employ four sets of trained models from Group-Free, which vary in layer level and Transformer depth. As shown in Fig. 5 (a) and (b), the "w2×" signifies that we double the width of the PointNet++ [32] backbone in Group-Free. "L" represents the depth of the decoder, and "O" denotes the number of object candidates. For instance, "(L6, O256)" refers to one with a 6-layer decoder (i.e., six attention modules) and 256 object candidates. Our experiments reveal that, despite using models of varying depths for detection, the differences in detection results were not significant, indicating that the performance of detection tasks is greatly affected by compression distortion, especially at lower bitrates where the detection performance is generally poorer.Different Strategies. In the second experiment, we examine each designed module and strategy to assess their effects on compression and detection tasks. As illustrated in Fig. 5 Generalization Test. In the third experiment, we isolate the designed loss function and ROI prediction network from RPCGC to evaluate their compression performance on MPEG datasets. We trained on the ModelNet [45] dataset and then evaluate the impact of these configurations on the MPEG standard compression dataset. As shown in Fig. 5 (e-g), we plot RD curves based on D2 PSNR indicators on the Phil, Queen, and Sarah, demonstrating that RPCGC surpasses traditional algorithms like AVS PCRM and G-PCC (Octree/Trisoup) codec on specific 10-bit datasets. Additionally, our approach exhibits advantages over learning-based methods, such as pcc_geo_cn_v1, pcc_geo_cn_v2, and PCGCv1. 
conclusion: 
 This paper introduces an innovative point cloud compression method comprising two key components: a BL and an EL. The BL is responsible for processing the geometry coordinates, while the EL focuses on encoding additional residual information of the point cloud. This information is effectively processed and optimized through the ROI prediction network and mask weighting process. We further develop a new RD optimization strategy that incorporates mask information, enhancing the encoding quality of critical areas and integrating detection loss into the total loss function to better guide detection tasks. To extract point cloud contour features more accurately, we design a MSFEM and a SAM in the residual encoding process for finer semantic information extraction. Experiment results demonstrate that our method improves compression performance on the ScanNet and SUN RGB-D datasets and achieves more accuracy of point cloud detection at high bitrates compared to traditional and learning-based point cloud compression methods. In future work, we aim to investigate multimodal point cloud compression alongside analysis tasks in the LiDAR dataset.  
 
 Appendix:  


Decision: All reviewers like to vote for acceptance of this submission. Authors need to carefully consider reviewers comments like method description, relatively poor writting, insufficient expeirments and to include them.

Reviews: review: paper_summary: paper_summary: The huge data volume of point clouds poses challenges to point cloud storage and transmission, necessitating the design of point cloud compression algorithms. Current point cloud compression methods mainly focus on fidelity optimization but the practical applications require the facilitation for machine analysis. To solve this problem, this paper proposes a compression paradigm for scene point clouds like autonomous driving scenarios. The proposed framework is in dual-branch architecture, including base and enhancement layers. The base layer encodes and decodes a simplified version of point clouds, and the enhancement layer refines it by focusing on geometry details. The enhancement layer achieves this by refining the residual information through an ROI prediction network, which generates maks information and provides weighted supervision.
strength_weakness: strength_weakness: 1) The proposed method seems novel, and the motivation of using ROI to guide the compression of scene point clouds is reasonable.
2) The experimental results are abundant to make it convincing, including the RD, R-mAP comparison, visualization, and adequate ablations. And the experimental settings are in details.
limitations: limitations: 1) The paper writing is not clear and needs improvement. For example, the working mechanism of the proposed modules are described in contributions part in Introduction. I recommend the last paragraph in Introduction describes the working mechanism to make this part more readable. And the MS-FEM and SAM modules are not mentioned in the last paragraph.
2) Too many modules are mentioned in Methodology part, making it somewhat hard to read. For example, the FAM module can be confused with RAM, SAM modules, I recommend to call it Alignment module to make it clearer.
paper_summary: paper_summary: This paper presents a model for joint point cloud compression and detection, aimed at managing point cloud compression while ensuring that the reconstructed point cloud can effectively perform downstream detection tasks. Specifically, the model has two branches: one handles the encoding and decoding of coarse point clouds, and the other supplements geometric details on this basis. Experiments show that the model not only performs well in the compression task but also achieves more accurate point cloud detection results at high bitrates than traditional and learning-based compression methods.
strength_weakness: strength_weakness: a) The motivation is clear and important. Most point cloud compression methods focus on reconstruction fidelity, often neglecting the relationship between the reconstructed point clouds and downstream tasks. This paper proposes a multi-task learning approach to achieve simultaneous optimization for both compression and detection.

b) The proposed method is reasonable. The authors choose a joint compression and detection task, and focus on the detection part since it's more difficult.To obtain detailed results, the paper introduce a dual-branch structure, similar to global-to-local or coarse-to-fine based methods. In the enhancement layer (EL), the authors design a delicate network to capture geometry details. Also, the EL contains the ROI Prediction Network and the ROI Searching Network, making the model better suited for detection task.

c) Experiment results show that the proposed method can achieve comparable compression performance, while maintaining better detection results at high bitrates.
limitations: limitations: a) Question about contributions. In Section 2.3, the authors mentioned some similar methods in the field of joint learning of semantic information and point cloud compression, so is this task not a newly proposed one? If it is not a newly proposed task, then why are there no comparisons with this methods in the experiments? Also, the author mentioned "Although the above methods involve machine perception, most of them are simplistic and do not achieve joint optimization or supervision", I believe that if the authors could provide a more detailed explanation of the differences and contributions of this method, it would enhance readability.

b) The experiments might appear somewhat insufficient. The authors have designed a very intricate neural network, yet it seems that a considerable amount of computation is dedicated to the EL branch, which is also reasonable given that the detection task is more challenging than the compression task. However, this also leaves concern about the model's computational cost, such as the number of parameters and FLOPs. If the authors could provide a comparison of the computational costs of all methods, as well as a detailed analysis of the computational costs for each branch of the RPCGC, the paper would be more understandable. Additionally, I am interested in the results of ablation studies on some of the main modules, such as the direct removal of the EL, which the authors do not seem to have demonstrated.

c) Some equations are confusing. What is $x_{rm}$ in Equation 1, and how is $x_{enh}$ obtained? The paper mentions two loss functions, Equations 6 and 10, which are confusing regarding their relationship and differences. Also, what is the mathematical expression for $L_{decoder}$ in Equation 9?

d) Some improvements could be made to the figures. In Figure 1, it is not very clear where the model is divided into two branches. In Figure 2, it is difficult to identify which part each sub-network belongs to. In Figure 4, it would be better to include the ground truth results.

I am willing to raise my score if the author addresses the aforementioned issues.
paper_summary: paper_summary: The paper proposes a novel Region of Interest (ROI)-guided point cloud geometric compression method (RPCGC), aimed at improving the efficiency of point cloud data compression and the accuracy of object detection in machine vision tasks. Its advantage lies in achieving a high compression ratio while significantly improving the detection accuracy of point clouds, especially at higher bit rates.
strength_weakness: strength_weakness: see the summary part.
limitations: limitations: 1.The reviewer believes that the authors' description of the model is not detailed enough. For instance, what are the specific algorithms used for quantization and de-quantization? Additionally, the authors should conduct more detailed ablation studies on each module to explore its intrinsic role. For example, how does the precision of quantization and the strategy of de-quantization affect the final compression performance and detection accuracy? How does the prediction accuracy of the ROI Prediction Network (RPN) impact compression and detection performance? The reviewer suggests that the authors add experiments to elucidate the internal workings of the model.
2. The reviewer suggests that the author's model shows limited improvement in performance compared to existing models, and the experiments conducted are insufficient. It is recommended that the authors include comparisons on more universally recognized datasets for point cloud compression, such as 8iVFB and Owlii, and also compare against more recent and state-of-the-art models.
Rui Song, Chunyang Fu, Shan Liu, Ge Li; “Efficient Hierarchical Entropy Model for Learned Point Cloud Compression”  CVPR, 2023, pp. 14368-14377
Junteng Zhang et al. “YOGA: Yet Another Geometry-based Point Cloud Compressor”, ACM MM 2023
3. The loss function of this paper includes three parts, and the reviewer believes that the authors should provide a detailed explanation of how to balance these three components. Furthermore, the reviewer is interested in the intrinsic impact of the selection of hyperparameters, such as α β γ δ, on the model. It would be beneficial to elucidate how these hyperparameters influence the model's behavior and its performance. 
4. The model presented in the paper involves numerous neural network components, which suggests a potentially significant computational demand. However, the authors have not elaborated on the specifics of the deployment environment, neither in terms of hardware nor software. Additionally, there is a lack of information regarding the training overhead and the computational expenses involved in the encoding and decoding processes. The reviewer recommends that the authors provide a comprehensive account of the deployment infrastructure, including the type of hardware used, the software framework, and any optimizations that were applied. Furthermore, the authors should detail the training costs in terms of time and resources, as well as the runtime complexity for both encoding and decoding operations.
paper_summary: paper_summary: The main concept of this paper is the introduction of a novel Region of Interest (ROI)-guided Point Cloud Geometry Compression method for human and machine vision. The paper addresses the challenges of point cloud data compression, focusing on optimizing both machine perception performance and visual fidelity simultaneously. The proposed method involves a dual-branch parallel structure with a base layer encoding geometry information and an enhancement layer refining this by focusing on geometry details. Additionally, the residual information of the enhancement layer undergoes refinement through an ROI prediction network, generating mask information that is incorporated into the residuals. The paper also introduces a Multi-Scale Feature Extraction Module (MSFEM) and a Semantic-aware Attention Module (SAM) to enhance the extraction of semantic information during the encoding and decoding transformation. The method aims to achieve exceptional compression performance and better detection accuracy compared to traditional and learning-based compression methods at high bitrates.
strength_weakness: strength_weakness: - The paper introduces a novel ROI-guided Point Cloud Geometry Compression method. By integrating detection loss into the compression task, the method enables joint optimization of compression and detection, leading to improved detection accuracy.
- The paper introduces modules like the MSFEM and SAM to enhance the extraction of semantic information during the encoding and decoding transformation, contributing to better compression and detection results.
limitations: limitations: I apologize for my limited understanding of this field, but I believe the essay could benefit from additional clarification.

- The research primarily concentrates on two datasets, ScanNet and SUN RGB-D, potentially restricting the applicability of the findings to a broader spectrum of point cloud data. Including additional datasets like OwLii would strengthen the validity of the conclusions.
- The experimental results indicate a significant impact of compression distortion on detection tasks, particularly evident at lower bitrates. Further exploration into how different detection models interact with compression could yield valuable insights for optimizing detection performance.
- In line 734, the statement "*RPCGC outperforms these models by -0.084*" appears perplexing, as it actually implies that RPCGC performs worse than certain methods. Additionally, enhancing the explanation of metrics would enhance clarity.