LDCNet: Long-Distance Context Modeling for Large-Scale 3D Point Cloud Scene Semantic Segmentation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 05:10:08

Abstract: Large-scale point cloud semantic segmentation is a challenging task in 3D computer vision. A key challenge is how to resolve ambiguities arising from locally high inter-class similarity. In this study, we introduce a solution by modeling long-distance contextual information to understand the scene's overall layout. The context sensitivity of previous methods is typically constrained to small blocks(e.g. $2m \times 2m$) and cannot be directly extended to the entire scene. For this reason, we propose \textbf{L}ong-\textbf{D}istance \textbf{C}ontext Modeling Network(LDCNet). Our key insight is that keypoints are enough for inferring the layout of a scene. Therefore, we represent the entire scene using keypoints along with local descriptors and model long-distance context on these keypoints. Finally, we propagate the long-distance context information from keypoints back to non-keypoints. This allows our method to model long-distance context effectively. We conducted experiments on six datasets, demonstrating that our approach can effectively mitigate ambiguities. Our method performs well on large, irregular objects and exhibits good generalization for typical scenarios.

Main paper: 
 related work: 
 Point cloud semantic segmentation. Unlike grid-like 2D image data, the disordered and unstructured nature of point clouds poses challenges and difficulties in the design of point cloud deep networks. PointNet [29] was the first to explore the design paradigm of point cloud deep networks and to address the problem of permutation invariance. Many subsequent approaches have followed this idea [9, 12, 18, 20, 30, 35, 41, 42, 51, 52] . Though these methods have achieved appealing results on object-level point cloud semantic segmentation, the large-scale scene semantic segmentation is still under-explored.Context modeling for point clouds. The methods can be broadly categorized into three groups. One group restricts the input point cloud to small blocks, allowing the standard self-attention mechanism to directly extract global context due to the small spatial extent and point count within each block [23, 50] . However, this approach is limited to small chunks, rendering it ineffective for large blocks (i.e. the entire scene) with a high number of points.In response, another group of methods attempts to enhance the local receptive field at each layer, implicitly extending the range of contexts modeled as the network deepens [17, 18, 24, 39, 52] . Our experiments reveal that achieving an effective context for the entire scene requires considerable network depth using this approach. The final group of methods aims to convert a 3D representation into a 2D representation; for instance, RangeFormer [16] transforms a point cloud into a range image, extracting long-distance context in 2D. However, this method is best suited for sparse point clouds, as dense point clouds risk losing significant information. Compared to these methods, in terms of context sensitivity, our method can cover long-distance ranges or even the entire scene. For representation, we utilize key points and local descriptors, effectively preserving 3D information.Large-scale point cloud scene semantic segmentation. Few methods have been specifically tailored for large-scale scenes. SP-Graph [19] preprocesses extensive point clouds into super-point graphs to learn per-super-point semantics. They employ an RNN to capture contextual information between superpoints. However, the limited long-term modeling capability of RNNs hinders the coverage of context over great distances. RandLA-Net [14] proposes using random sampling instead of farthest sampling to expedite network inference. They employ the concept of 'dilation' [47] to locally expand the receptive field, aiming to continuously broaden the context during network feedforward. Nonetheless, achieving context extension to the entire scene requires a very deep network. Vision Transformer. In recent years, the Transformer model has achieved remarkable success in modeling powerful long-distance context in NLP and 2D image domains [10, 25, 27, 37, 38, 48] . While Transformers are widely popular in 2D and NLP, their application to point clouds has been relatively unexplored until recently. Several transformer-based point cloud networks have emerged [18, 28, 52] .For instance, PointTr [52] applies the Transformer in a local kneighborhood, while StratifiedTr [18] applies the transformer in a window neighborhood. FastPointTr [28] first voxelizes the point cloud and then applies the transformer in a local region. Given the substantial number of points in a scene's point cloud, applying the Transformer globally may seem impractical. Our key insight is that it's unnecessary to apply the Transformer to all points to capture long-distance context; applying the Transformer to key points is sufficient. 
method: 
 Our goal is to explicitly model long-distance context across the entire scene with Transformer, rather than within local blocks. Our approach comprises two main parts, as illustrated in Figure 2 (b): a point cloud backbone network for generating dense semantic predictions and a separate new branch that models long-distance contextual information and aggregates this information into the point cloud backbone. We will first review existing point cloud networks and then delve into the details of our design.For the convenience of the discussion that follows, we will only use 𝑥 to denote the feature of a point without using a variable to denote 𝑥𝑦𝑧 unless otherwise stated in the text. But the 𝑥𝑦𝑧 of a point will be used in a 𝑘NN query process(including distance calculation) by default. The existing point cloud networks are summarized in Figure 2 (a). Generally, a network includes an encoder and decoder. The encoder has multiple downsampling and local information aggregation (DLIA) modules, while the decoder has upsampling and feature propagation (UFP) modules. Note that Figure 2 (a) is a simplified case. Typically, there are 4-5 DLIAs and UFPs in the network, with a skip connection from DLIA to UFP, similar to UNet [32] .The DLIA module includes a downsampling module and a local information aggregation module. The downsampling module reduces point cloud resolution, similar to pooling layers in image processing. For example, RandLANet [22] uses random downsampling; KPConv [35] applies voxel downsampling; while PointNet [29] , PointTr [52] , and StratifiedTr [18] use farthest downsampling. The local information aggregation module aggregates point cloud features before and after downsampling. For each point after downsampling, it identifies the nearest 𝑘 points before downsampling and aggregates their features. For example, PointNet++ [30] uses max-pooling; KPConv [35] employs point convolution; and PointTr [52] utilizes a local self-attention mechanism for aggregation.The UFP module includes an upsampling module and a feature propagation module. The upsampling module increases the point cloud resolution, usually achieved by preserving points after downsampling from different layers. For example, in a network with 4 DLIAs and 4 UFPs, the input point of the 1st UFP upsampling module is the output point of the 4th DLIA downsampling module, and the output is the output point of the downsampling module in the 3rd DLIA, and so forth. The feature propagation module propagates point cloud features before upsampling to the point cloud after upsampling. Specifically, for each point 𝑥 𝑎𝑢 after upsampling, finding the nearest 𝐾 points {𝑥 𝑘,𝑏𝑢 } before upsampling, and then propagate the features of those 𝐾 points byEQUATIONIf the network has no skip connections, the initial value of 𝑥 𝑎𝑢 is an all-0 vector. Θ is usually implemented with MLPs.𝑤 𝑘 = 1/𝑑 𝑘 𝐾 𝑘=1 1/𝑑 𝑘 ,where 𝑑 𝑘 is the distance between 𝑥 𝑎𝑢 and its 𝑘-th nearest neighbor 𝑥 𝑘,𝑏𝑢 .In this section, we will delve into the details of our design. methods [2, 36, 46] . We exclude deep-learning-based methods initially because they introduce too many additional network parameters and unnecessary memory footprint. Among traditional methods, random sampling loses significant information. Despite having the lowest time complexity, in practice, we observed that the model fails to converge. Other methods such as ISS [53] , Harris3D [34] , and Sift3D [55] are locally geometry-sensitive and do not guarantee the uniform distribution of keypoints. In contrast, farthest point sampling not only ensures the uniformity of sampling but also guarantees that keypoints cover the entire scene. Moreover, the farthest sampling method makes it easier for us to set the required number of keypoints. Providing a robust initial descriptor for key points is essential for high-quality long-distance context modeling. The concept revolves around ensuring that each initial descriptor encapsulates neighborhood information at various spatial scales around the key point. This has the advantage of leveraging multi-scale information, enhancing the ability to recognize objects of different sizes within the scene, as mentioned in PatchFormer [49] . To achieve this goal, we introduce a simple module named the Multi-Scale Spatial Context Aggregation Module (MSSCA). This module queries neighbors of a key point at different scales and aggregates information from these neighbors to the key point.The detail of the module is shown in Figure 2 (e). For an input point cloud 𝑃 ∈ R 𝑁 ×6 , where the first three channels are 𝑥𝑦𝑧 coordinates and the last three channels are 𝑟𝑔𝑏, We first query multiple 𝑘-nearest neighbors of each point(specifically, 𝑘 = {8, 16, 32}) and get 𝑃 8 ∈ R 𝑁 ×8×6 , 𝑃 16 ∈ R 𝑁 ×16×6 , 𝑃 32 ∈ R 𝑁 ×32×6 . Then we use a single MLP to lift their feature dimensions. After that, we use average pooling to aggregate its nearest neighbor information and concatenate this information with the original descriptor to obtain the initial descriptor of each point. Finally, we use the farthest sampling to sample 𝑀 key points. The key points 𝑋 𝑘𝑒𝑦 ∈ R 𝑀 ×32 are fed into LRDRA and the non-key points 𝑋 𝑛𝑘𝑒𝑦 ∈ R 𝑁 ×32 are fed into DLIA modules.Reasoning and Aggregation. In the original point cloud backbone, the input feature of the latter layer of encoders is derived from the outputs of the previous layer of the network. In the previous discussion, we observed that the features from the upper layer network encode only local information, which is not sufficient for semantic segmentation of large-scale scenes. Therefore, our idea is to perform an augmentation at each layer of the encoder of the original point cloud backbone network to adequately supplement long-distance contextual information into the original point cloud backbone network.Specifically, each LDCRA module contains two parts: a Long-Distance Context Reasoning Module (LDCR) and a Long-Distance Context Aggregation Module (LDCA). Given keypoints and their descriptors, LDCR uses a standard self-attention mechanism to capture the relationship between two key points and model global contextual information. Given the global information carried by each key point, as well as the non-keypoint information, LDCA aggregates the global information from the key points to the nonkeypoints.Long-Distance Context Reasoning. Details of the long-distance context modeling are illustrated in Figure 2 (f). For an input key point 𝑋 𝑘𝑒𝑦 , we use standard vector self-attention [52] as follows:𝑥 𝑖 = 𝑀 ∑︁ 𝑚=1 𝑊 𝑤 (𝑊 𝑞 (𝑥 𝑖 ) -𝑊 𝑘 (𝑥 𝑚 ) + 𝑃 𝑟 ) ⊙ (𝑊 𝑣 (𝑥 𝑚 ) + 𝑃 𝑟 ) (2)where 𝑥 𝑖 ∈ R 𝐶 represent the descriptor of the i-th key point, 𝑊 𝑞 ,𝑊 𝑘 ,𝑊 𝑣 represent 𝑞𝑢𝑒𝑟𝑦, 𝑘𝑒𝑦 and 𝑣𝑎𝑙𝑢𝑒 matrices respectively as defined in [38] . 𝑊 𝑤 represents a weight layer that generates an attention vector. Following PointTr [52] , we add a position encoding 𝑃 𝑟 = 𝜃 (𝑝 𝑖 -𝑝 𝑚 ) to both the attention vector and the transformed features 𝑊 𝑣 (𝑥 𝑚 ) to keep position information intact, where 𝑝 𝑖 is the 𝑥𝑦𝑧 coordinate of the 𝑖-th key point.Long-Distance Context Aggregation. The entire process is illustrated in Figure 2 (f). A simple intuition is that adjacent points are more relevant. For each non-key point, we identify the nearest key point and incorporate the information carried by that key point to the non-key point. To further reduce noise and eliminate redundant information, we compute the difference between the two using a channel-wise subtraction operation. We then employ MLP and Softmax to generate a channel-wise soft weight. This soft weight is multiplied with the keypoint features and added to the non-keypoint features, resulting in the final augmented nonkeypoint features. Specifically, given a key point 𝑋 𝑖 𝑘𝑒𝑦 ∈ R 𝑀 ×𝐶 updated by self-attention at the 𝑖-th layer and a non-key point 𝑋 𝑖 𝑛𝑘𝑒𝑦 ∈ R 𝑁 𝑖 ×𝐶 from the 𝑖-th DLIA, the entire process can be formulated as:EQUATIONwhere 𝑥 𝑖,𝑛𝑒 represents the 𝑖-th non-key point feature, 𝑥 𝑗,𝑒 represents the 𝑗-th key point feature. Also, 𝑥 𝑗,𝑒 is the of the point closest to 𝑖-th non-key point. Θ is implemented with MLPs but not the same as in Equation1.In this subsection, we discuss in detail the impact of each parameter on our approach and the design choices. Memory Footprint. Here we focus on analyzing the additional memory footprint we introduced. Compared to applying the standard self-attention module globally to all points, since our selfattention mechanism is executed at key points and uses a vector attention mechanism, the total space complexity is reduced from O (𝑁 2 𝐶) to O (𝑀 2 𝐶)(𝑀 << 𝑁 ), where 𝑁 and 𝑀 are the numbers of origin point clouds and key points. Number of 𝑀. Increasing 𝑀 theoretically allows for better capture of long-range context; however, the algorithm's space complexity scales with 𝑀 2 𝐶, leading to a tradeoff. Our experiments show some performance gains (Table 4 ), but considering the additional memories, we find this improvement less satisfying. Given that 256 keypoints already achieve a 3% mIoU boost compared to the original backbone network, we opt for the more memory-effective 256 keypoints.  
experiments: 
 We conducted experiments on six datasets including one unpublished dataset CHNRockery3D, as well as five public datasets: S3DIS [1] , ScanNet [7] , ScanNet200 [33] , Semantic3D [13] , and SemanticKitti We adapted and modified several related methods from their official codebases to suit this dataset. For the other five public datasets, we follow a common evaluation protocol [14, 52] . Evaluation metrics include mean classwise intersection over union (mIoU), mean of classwise accuracy (mAcc), and overall pointwise accuracy (OA). Implementation Detail. We mainly use PointTr [52] as our backbone because both methods are Transformer-based. In the original paper, PointTr is a 5-layer setup. To evaluate the effectiveness of our method, we use a 4-layer setup. To further show the adaptability of our method to existing methods, we also choose a more classical method, KPConv [35] , and kept all the settings (number of layers, etc.) from the original paper. Unless otherwise mentioned, we use PointTr as the backbone. All experiments are conducted on a single RTX3090Ti GPU. The number of key points is set to 256 in most experiments.To verify the effectiveness of our long-distance context modeling, we discuss and illustrate the following experiments. Results on CHNRockery3D. We compared our method with several existing open-source methods. Table 2 illustrates that our approach achieves a 3-4% improvement over the baseline method when using both KPConv and PointTr as the backbone. This suggests that our method exhibits some level of generalization within existing frameworks. Notably, our method attains the best results among these methods when utilizing PointTr as the backbone. It's worth highlighting that our method outperforms existing methods in both the buildings and rocks categories. This is particularly significant as buildings and rocks typically occupy substantial regions within a scene, emphasizing the importance of a larger contextual range for such cases.The visualization results are presented in Figure 3 . Notably, our method demonstrates improvements in the building and rock categories compared to the previous network. In the second row of Figure 3 , it's evident that both KPConv and PointTr misclassify large areas of rocks as buildings, whereas our method accurately segments the rock area. However, we observed a decrease in our method's performance on vegetation compared to other methods. This might be attributed to the relatively small size of vegetation compared to buildings and rocks. Consequently, a larger contextual information might introduce some noise.Ponds constitute a special class of objects that share geometric similarities with rocks, but their distribution in the scene differs from that of rocks. Consequently, recognizing this object presents a considerable challenge. As indicated in Table 2 , effectively segmenting this object is a daunting task for existing methods. However, our approach shows some progress in this category. The potential reason for this lies in the network's ability to model long-distance context for object analysis. This enables the network to identify the object's location in the scene and may even infer its function within the entire scene, providing further validation of the effectiveness of our method. Visualization of Effective Receptive Field. To understand how our method identifies ponds, we visualize the effective receptive field (ERF), as suggested in [22] , in Figure 5 . It is evident that our method associates the pond with a larger region of the scene during inference. This implies that our method may implicitly learn relationships between objects. The visualization underscores the effectiveness of our approach in modeling long-distance contexts. Memory Footprint Comparison. To demonstrate the memory efficiency of our approach, we experimented with different settings of PointTr, including a 4-layer setting, a 5-layer setting, and the use of more neighboring points for DLIA. Additionally, we conducted a comparison with StratifiedTr, considering its utilization of a larger window to capture longer contexts.As shown in Table 3 , the aggregation of information using more neighborhood points leads to a significant increase in memory. This is because, as the number of neighborhood points grows, the network must compute and store a larger attention map during the forward propagation process. However, despite this increase, the performance does not see significant improvement. Similarly, the addition of layers to PointTr results in a sharp rise in the number of parameters and memory usage. While this brings some performance gains, our approach achieves relatively greater improvements without introducing excessive memory footprint. Although StratifiedTr outperforms PointTr, it comes with the highest number of network parameters, and memory usage. It's important to note that Strati-fiedTr is configured with 4 layers. This experiment underscores the superiority of our method.We also compare our method with existing approaches on various public datasets to demonstrate the potential of our methods. Results on S3DIS. The segmentation results depicted in Figure 4 demonstrate the notably accurate performance of our method in a large area containing numerous chairs, underscoring the effectiveness of our approach for such scenes. Additionally, we present the results for Area-5 and the 6-fold cross-validation in Table 1 . It is evident from the table that our method exhibits performance gains compared to the baseline used, and achieves comparable results to other recent methods. Results on Semantic3D. We present the segmentation results on this dataset in Figure 6 and Table 5 . Our method demonstrates a performance improvement over the baseline. Additionally, we observed an interesting phenomenon, depicted in Fig. 7 . As the input scene range becomes larger (We simulate different sizes of input scenes by cropping the scene from small to large ranges during the testing phase), our method achieves increasingly better performance, while the baseline method reaches a bottleneck at a certain level of scene range. This indicates that our method's capacity for modeling context can scale with the size of the scene, i.e. when the scene gets larger, our method can naturally extend the context-aware distance to a wider distance. This phenomenon validates the effectiveness of our long-distance context modeling.Results on other datasets. We also report results of our method on Scannet [7] , Scannet200 [33] and SemanticKitti [3] dataset in Table 5 . In all three datasets, we made improvements from the baseline.We perform ablation experiments on CHNRockery3D to verify the effectiveness of each part of our method. Effectiveness of MSSCA. Since this module is used to generate initial key points descriptors, removing this module means that only information from a single point is used when modeling longdistance dependencies. As shown in Table 4 , the method performance drops a lot after removing the module, but it's still better than the 4-layer PointTr, which suggests that using a single-point feature can also be effective. Figure 7 : Experiments on the ability of the method for context perception over long distances. It can be seen that our method can better extend the context-aware distance to a wider distance as the input scene range increases. For the size of the scene, we simply take the maximum value of the scene length and width.Effectiveness of LDCRA. We remove all LDCRA modules. In this case, the only difference between our method and PointTr is the input; PointTr uses the original point cloud as input, whereas removing LDCRA from our method is equivalent to using the initial descriptors extracted by MSSCA as input to PointTr. As shown in Table 4 , we achieve a little bit of improvement compared to the origin PointTr. This demonstrates both the effectiveness of the MSSCA and the LDCRA. Effectiveness of Soft Weight(SW). In our design, the soft weight is used to eliminate the redundancy between the critical point information and the non-critical point information, and after removing this weight, the performance is slightly degraded as can be seen in Table 4 .Where to model long-distance context? Our encoder has four layers, and in our design, we perform long-distance context modeling and fuse information with local contextual information at each layer. However, is it necessary to do so? For this purpose, we performed ablation experiments. As shown in Table 4 (i.e. LDC loc.), doing it at every layer is required to fully utilize the capabilities of our approach.Impacts of the number of the keypoints. The impacts on 𝑀 is shown in Table 4 . As the number of points increases, the memory overhead increases dramatically, and the performance gain it provides is relatively modest. 256 achieves a performance-efficiency tradeoff.Although our approach models contextual information over longer distances with only a slight increase in memory overhead, the extensive use of 𝑘NN searches, as well as farthest point sampling algorithms throughout the network's inference, results in a significant time overhead. One avenue for future improvement is the serialization of the point cloud. Point cloud serialization [21, 40, 43] is a recent technique that has emerged, showing promise in greatly enhancing overall inference speed by appropriately ordering the point cloud so that nearest neighbor points are adjacent to each other in memory. This approach leads to an almost 𝑂 (1) complexity compared to the 𝑂 (𝑁 2 ) complexity required for 𝑘NN and farthest point sampling. 
conclusion: 
 This paper introduces a novel approach to explicitly model longdistance contextual information throughout the entire scene, effectively addressing the ambiguity challenges inherent in large-scale scenes. Our method achieves this with minimal memory footprint.Experimental results demonstrate the effectiveness of our approach across a range of scene cases, particularly excelling in challenging scenarios involving irregular large-sized objects. In future work, further exploration of the definition and challenges associated with large-scale scenarios is needed.  
: 
 rockery garden from the CHNRockery3D dataset. Previous methods such as SPG [19] , RandLANet [14] and PointTr [52] give similar results as shown in the 3rd column.scenes, the network can effectively segment individual chairs almost without ambiguity. However, when multiple chairs are placed together, the network tends to classify them as clutter(Figure 1 (a)).Similarly, In outdoor scenes, such as Chinese classical gardens featuring rockeries and ponds made of stones, the overall look of a pond may resemble a local region of a rockery. This similarity poses a challenge for the network in distinguishing between the two (Figure 1 (b)). Additionally, some parts of buildings exhibit high geometric similarity with local areas of rockeries, leading to confusion. This ambiguity hinders the network's ability to effectively handle such scenarios.Our observation is that modeling long-distance contexts to capture the spatial layout of the entire scene and the relation between objects can effectively mitigate the issue of ambiguity. For instance, within a classroom setting, chairs are typically positioned at the center of the scene. Additionally, in garden scenarios, ponds are usually positioned at lower elevations, while buildings generally surround the entire scene. Furthermore, rockeries are typically situated near the buildings.To model long-distance context, early methods [11, 19] segmented the scene into blocks and employed Recurrent Neural Networks (RNNs) to capture contextual information between these blocks. However, due to the limited long-term modeling capacity of RNNs, extending the context to cover the entire scene was challenging. Inspired by the impressive long-distance modeling capabilities of Transformers in 2D image and NLP domains [6, 10, 25, 27, 37, 48] , many methods explored using Transformers for capturing longdistance context in point clouds. However, as the memory footprint of Transformers grows quadratically with the number of points, existing methods typically fall into two categories. One confines the input point cloud to a small block (e.g., 2m x 2m) and captures global context on a sparse subset of points (usually 1024 or 2048 points) within these small blocks [23, 50] . The other attempts to use a larger number of points (e.g., 100,000) [17, 18, 52] . To address the cost issue, these methods attempt to use Transformers locally (e.g., 16 points in k-neighbor or 3d window), with the long-distance context being implicitly acquired during the network's forward propagation. These methods either restrict the context range to small blocks or require very deep networks to extend the context to cover the entire scene.To this end, we propose Long-Distance Context Network(LDCNet). Our idea is simple and intuitive, the key points are sufficient to reason about the layout of the scene. Hence, we transform largescale scenes into a representation based on key points and model long-distance context on these key points. We then aggregate contextual information from key points into non-key points for dense prediction. With the number of key points significantly fewer than the original point cloud, it's feasible to model long-distance context across the entire scene. Experiments demonstrate that our method extends the context length beyond 10 meters without adding too many network parameters and memory footprint. Our network accurately perceives the full view of irregular large-sized objects and provides correct predictions.Our main contributions are as follows• We address the problem of high inter-class similarity in semantic segmentation of large-scale scene point clouds. We introduce a new Transformer-based method that efficiently models the context of the entire scene and reasons about the overall scene layout relationships. • We model the long-distance context by first extracting key points from a point cloud and carefully initiating a descriptor for each key point. Then we apply Transformer globally on all key points to extract long-distance contextual information. Finally, we aggregate this information into non-key points for dense prediction. • Experiments show our approach can effectively mitigate ambiguities by expanding the contextual range to the entire scene (over 10m). Our method performs well on large, irregular objects in large-scale scenes, and it also exhibits good generalization for typical scenarios. 
 
 Appendix:  


Decision: Reasonable quality but lacks convincing performance.

Reviews: review: paper_summary: paper_summary: This paper introduces a Long-Distance Context Modeling Network (LDCNet) for Large-Scale 3D Point Cloud Scene Semantic Segmentation. LDCNet aims to resolve ambiguities arising from locally high inter-class similarity by extracting key points along with a local descriptor.Ths work is highly followed by PointTr. The performance was evaluated on six datasets. The idea and topic of this manuscript are interesting and have potential significance
strength_weakness: strength_weakness: The presentation of the paper is clear. The experiments are conducted in a correct manner, and both qualitative and quantitative results are compared with an ablation study.
limitations: limitations: 1. It is mentioned in line 109 that the transformer model increases the computational footprint, but this work heavily follows the transformer-based PointTr model, which is a significant contradiction. It is expected to explore something new beyond transformer-based models.
2. The contribution of this work is claimed to be the extraction of key points using conventional descriptors rather than DL-based models to reduce computational complexity. However, no theoretical support or discussion is provided for key point extraction.
3. Key point-based methods often focus on local geometric features, so how the model maps or learns global features in such complex, cluttered environments need to be discussed.
4. Another factor is the selection of key points, which is limited to 256 in this paper. How the model selects dynamic key points in large-scale contexts, especially given that point clouds can vary greatly in scale and point density, needs to be considered and discussed further.
paper_summary: paper_summary: This paper deals with the problem of 3D point cloud semantic segmentation in large-scale scenes. A Long-Distance Context modeling method, named LDCNet, is proposed. The method first extracts key points and encodes contextual information using Transformer to facilitate semantic segmentation. Extensive experiments demonstrate the effectiveness of the proposed method. The manuscript is well-written and structured.
strength_weakness: strength_weakness: The task of large-scale point cloud scene semantic segmentation is of vital importance to autonomous driving. This work tries to improve the performance of the task from the perceptive of long-distance contextual information encoding to under the scene layout. The mechanism effectively mitigates the issue of ambiguity during scene feature encoding. Experiments on several dataset show the strength of the work.
limitations: limitations: I believe the manuscript can be further improved regarding the following concerns:
1. The basic idea of the work is that the extracted key points can be utilized to model or encode the entire scene. Such an idea is not new. Please refer to PointNet++ used to encode the geometric information of a 3D object, while the work is an extension to the large-scale 3D scene.
2. The definition of the loss function associated with the training of the network is missing.
3. The depiction of Figure(e) should be corrected, regarding the feature extraction using 8, 16, 32 neighbors followed by MPL.
paper_summary: paper_summary: This paper introduces LDCNet, a method for large-scale 3D point cloud scene semantic segmentation, which effectively models long-distance contextual information by using key points and local descriptors to understand the overall scene layout. Several experiments are conducted, demonstrating this approach can effectively mitigate ambiguities and has good generalization for typical scenarios
strength_weakness: strength_weakness: 1) The paper introduces a new branch that models long-distance contextual information and aggregates this information into the point cloud backbone, enhancing semantic segmentation accuracy for large-scale 3D point cloud scenes.

2) The proposed method explicitly model long-distance context across the entire scene with Transformer, rather than within local blocks, achieving better results while saving computational resources.

3) The method description of the article is easy to understand, and there are much visualization to help understand.
limitations: limitations: 1) Point sampling for keypoint is important in the MSSCA module, and the comparison of different sampling methods are discussed in the method part. However, there is no ablation experiment on the influence of different sampling methods in the experiment part to support the discussion in the method part.

2) About the LDCRA module, since the data will be downsampled in each stage of the encoder, will the keypoints also be downsampled in each stage of the LDCRA module? In addition, the right part of table 4 shows that the relationship between the number of LDCRA modules and the performance of the whole model does not increase linearly. There is a lack of theoretical analysis for this phenomenon.

3) Although the method proposed in this paper has some improvements over the chosen backbones, the experimental results on some datasets shown in the experiment part have no obvious advantages compared with SOTA. More experiments on the performance improvement of this method under different backbone are needed.

4) The experiment part only introduces the dataset CHNRockery3D in detail, and lacks a brief description of the content of several other datasets.
paper_summary: paper_summary: The paper tackles the point cloud semantic segmentation task and focuses on the long-distance context modeling problem. The proposed method is intuitive and effective, performing well on several representative datasets.
strength_weakness: strength_weakness: The stated problem in the task is both authentic and crucial. The proposed method directly and convincingly addresses the problem. The writing is clear and easy to follow, and the experiments are abundant, accompanied by in-depth analysis.
limitations: limitations: 1. Time Consumption Analysis: Although L907 mentions significant time overhead, the paper lacks related statistics, which is unacceptable.
2. CHNRockery3D Dataset Availability: Will the CHNRockery3D dataset be made public? It would serve as a valuable testbed for the community.
3. Exploring Key Point Extraction Methods: Beyond fps, it's crucial to explore various sampling methods and understand their impact on the proposed long-distance context modeling approach.