Spherical Frustum Sparse Convolution Network for LiDAR Point Cloud Semantic Segmentation

Authors: yu  zheng
guangming  wang
jiuming  liu
marc  pollefeys
hesheng  wang
spherical  frustum
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 12:20:39

Abstract: LiDAR point cloud semantic segmentation enables the robots to obtain fine-grained semantic information of the surrounding environment. Recently, many works project the point cloud onto the 2D image and adopt the 2D Convolutional Neural Networks (CNNs) or vision transformer for LiDAR point cloud semantic segmentation. However, since more than one point can be projected onto the same 2D position but only one point can be preserved, the previous 2D projection-based segmentation methods suffer from inevitable quantized information loss, which results in incomplete geometric structure, especially for small objects. To avoid quantized information loss, in this paper, we propose a novel spherical frustum structure, which preserves all points projected onto the same 2D position. Additionally, a hash-based representation is proposed for memory-efficient spherical frustum storage. Based on the spherical frustum structure, the Spherical Frustum sparse Convolution (SFC) and Frustum Farthest Point Sampling (F2PS) are proposed to convolve and sample the points stored in spherical frustums respectively. Finally, we present the Spherical Frustum sparse Convolution Network (SFCNet) to adopt 2D CNNs for LiDAR point cloud semantic segmentation without quantized information loss. Extensive experiments on the SemanticKITTI and nuScenes datasets demonstrate that our SFCNet outperforms previous 2D projection-based semantic segmentation methods based on conventional spherical projection and shows better performance on small object segmentation by preserving complete geometric structure. Codes will be available at https://github.com/IRMVLab/SFCNet.

Main paper: 
 introduction: 
 Nowadays, 3D LiDAR point clouds are widely used sensor data in autonomous robot systems. Many recent works focus on resolving perception [1, 2] and localization [3, 4, 5] tasks on autonomous robot systems using LiDAR point clouds. Among them, semantic segmentation on the LiDAR point cloud enables the robot a fine-grained understanding of the surrounding environment. In addition, the semantic segmentation results can be adopted for the reconstruction of the semantic map [6, 7, 8, 9, 10] of the environments.Inspired by the achievements of deep learning in image semantic segmentation, researchers focus on searching for effective approaches to transfer the achievements to the field of point cloud semantic segmentation. Most previous works convert the raw point cloud to regular grids, like 2D images [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21] and 3D voxels [22, 23, 24, 25] , to exploit Convolutional Neural Figure 1 : Difference between our spherical frustum and conventional spherical projection. In conventional spherical projection, the points projected onto the same 2D grid are dropped, which leads to quantized information loss, e.g., dropping the boundary between the person, a small object, and the road, and results in incorrect prediction of the 2D projection-based method RangeViT [21] for the person. In contrast, our spherical frustum preserves all points in the frustum, which eliminates quantized information loss and makes SFCNet correctly segment the person.Networks (CNNs) and transformers in the field of point cloud semantic segmentation. The CNNs and transformer can easily process the regular grids to effectively segment the point cloud. However, due to the limited resolution, more than one point can be projected onto the same grid, and only one point is preserved, which results in quantized information loss to the regular grid-based point cloud semantic segmentation methods. The quantized information loss poses a challenge for small object segmentation since most points belonging to the small objects can be dropped during the projection. A few methods [13, 17] are proposed to compensate for the quantized information loss by restoring complete semantic predictions from partial predictions. However, quantized information loss still exists in the feature aggregation.To overcome quantized information loss during 2D projection, in this paper, a novel spherical frustum structure is proposed. Fig. 1 shows the comparison between the conventional spherical projection [11] and spherical frustum. Through spherical frustum, all the points projected onto the same 2D grid are preserved. Therefore, spherical frustum can avoid quantized information loss during the projection and improve the segmentation of small objects. However, without specific designs, the spherical frustum is an irregular structure and can not be processed by CNNs. Using dense grids to store the spherical frustums is an intuitive method to regularize the spherical frustum. However, since the point number of the spherical frustums is different, each point set is required to be padded to the maximal point number of the spherical frustums before being stored in the dense grid, which results in many redundant memory costs. To avoid redundant memory occupancy, we propose a hash-based spherical frustum representation, which stores spherical frustums in a memory-efficient way. In the hash-based spherical frustum representation, the neighbor relationship of spherical frustums and points is represented through the hash table, which enables the points to be simply stored in the original irregular point set.In the hash-based representation, each point is uniquely identified by the hash key, which consists of the 2D coordinates of the corresponding spherical frustum and the point index in the spherical frustum point set. Thus, the points projected onto any specific 2D grids can be efficiently queried. Based on the hash-based representation, we propose the Spherical Frustum sparse Convolution (SFC) to exploit 2D CNNs on spherical frustums. SFC aggregates point features of nearby spherical frustums to obtain the local feature of the center point.Moreover, previous 2D projection-based segmentation methods downsample the projected point cloud based on stride-based 2D sampling, which is unable to uniformly sample the 3D point cloud. However, the stride-based 2D sampling uniformly samples the spherical frustums. Therefore, we propose a novel uniform point cloud sampling method, Frustum Farthest Point Sampling (F2PS).F2PS firstly samples spherical frustums by stride, and then uniformly samples the point set inside each sampled spherical frustum by Farthest Point Sampling (FPS) [26] . Since the computing complexity of sampling points in each spherical frustum is constant-level, F2PS is an efficient sampling algorithm with a linear computing complexity.In summary, our contributions are:• We propose a novel spherical frustum structure with a memory-efficient hash-based representation. Spherical frustum avoids quantized information loss of spherical projection and preserves complete geometric structure.• We integrate spherical frustum structure into 2D sparse convolution, and propose a novel Spherical Frustum sparse Convolution Network (SFCNet) for LiDAR point cloud semantic segmentation.• An efficient and uniform 3D point cloud sampling named Frustum Farthest Point Sampling (F2PS) is proposed based on the spherical frustum structure.• SFCNet is evaluated on the SemanticKITTI [1] and nuScenes [27] datasets. The experiment results show that SFCNet outperforms previous 2D projection-based methods and can better segment small objects. 
related work: 
 Point-Based Semantic Segmentation. A group of works [26, 28, 29, 30, 31, 32, 33] learn to segment point cloud based on the raw unstructured point cloud. However, learning of raw point cloud requires the neighborhood query with high computing complexity to learn effective features from the local point cloud structure. Therefore, the efficiency of these point-based methods is limited.3D Sparse Voxel-Based Semantic Segmentation. Storing large-scale LiDAR point clouds in dense 3D voxels requires huge memory consumption. Therefore, Graham et al [34] proposes the 3D sparse voxel structure. Instead of dense grids, the hash table is adopted to represent the neighborhood relations of the 3D sparse grids. Based on the hash table, the convolved grids are recorded in the rule book. According to the rule book, the 3D sparse convolution is performed.Based on the sparse 3D voxel architecture, the methods of 3D sparse convolution and 3D attention mechanisms [22, 23, 24, 35, 36, 25, 37] are proposed.2D Projection-Based Semantic Segmentation. The research of image semantic segmentation [38, 39, 40, 41, 42] has gained great achievement. Thus, many works [11, 12, 13, 14, 15, 43, 18, 19, 20, 21, 16, 17] project the point cloud onto the 2D plane and utilize 2D neural networks to process the projected point cloud. Spherical projection is a widely used projection method first introduced by SqueezeSeg [11] . The subsequent works [11, 12, 13, 14, 43, 20, 21] effectively segment the point cloud with the image semantic segmentation architecture including 2D CNNs and vision transformers.Due to the limited resolution, the 2D projection-based segmentation methods suffer from quantized information loss. With quantized information loss, networks can only process the incomplete geometric structure and output partial semantic predictions, which results in the penalty of segmentation performance. The previous works only focus on restoring complete semantic predictions from the partial predictions of 2D neural networks. RangeNet++ [13] proposes a post-processing strategy to restore the complete predictions. The semantic predictions of dropped points are voted by the predictions of their K-Nearest Neighbors (KNN). In addition to KNN-based post-processing, KPRNet [17] directly reprojects incomplete predictions to the complete point cloud and adopts point-based network KPConv [29] to refine the predictions. However, few works explore the method of preserving the complete geometric structure during projection.In this paper, we propose the spherical frustum which avoids the quantized information loss of spherical projection. Our spherical frustum structure can not only preserve the complete geometric structure but also output the complete semantic predictions without any post-processing or point-based network refinement. 
sfcnet: 
 In this section, the spherical frustum and the hash-based representation will be first illustrated in Sec. 3.1. Based on the hash-based spherical frustum representation, the spherical frustum sparse convolution and frustum farthest point sampling for LiDAR point cloud semantic segmentation will be introduced in Sec. 3.2 and 3.3 respectively. Finally, the architecture of the Spherical Frustum sparse Convolution Network (SFCNet) is illustrated in Sec. 3.4.Conventional Spherical Projection. The LiDAR point cloud P is composed of N points. The k-th point in P is represented by its 3D coordinates x k = [x k , y k , z k ] T and the input point features f k ∈ R Cin , where C in represents the channel dimension of the features. The conventional spherical projection [11] first calculates the 2D spherical coordinates of each point:EQUATIONwhere (H, W ) is the height and width of the projected image.r k = x 2 k + y 2 k + z 2k is the range of the point. f = f up + f down is the vertical field-of-view of the LiDAR sensor, where f up and f down are the up and down vertical field-of-views respectively. According to the computed 2D spherical coordinates, the point features {f k } N k=1 are projected onto the 2D dense image. If multiple points have the same 2D coordinates, the conventional spherical projection only projects the point closest to the origin and drops the other points, which results in the quantized information loss.From Spherical Projection to Spherical Frustum. Since dropping the redundant points projected onto the same 2D position results in quantized information loss, we propose the spherical frustum to preserve all the points projected onto the same 2D position. Specifically, we organize these points as a point set and assign each point with the unique index m k in the point set. In addition, the 3D coordinates of each point {(x k , y k , z k )} N k=1 are preserved as the 3D geometric information for the subsequent modules.Hash-Based Spherical Frustum Representation. The irregular spherical frustums can not be directly processed by the 2D CNNs. A natural idea to regularize the spherical frustums is putting them in dense grids. To store the point set of each spherical frustum in the dense grids, an extra grid dimension is required. The size of this dimension should be the maximal point number M of each spherical frustum point set. However, since most of the spherical frustum point numbers are much less than M , many grids are empty. To avoid saving these empty grids in memory, we propose the hash-based spherical frustum representation to regularize the spherical frustum, where the hash table replaces the dense grids to map the 2D coordinates to the corresponding spherical frustums and points. In the hash table, the index k of any point in the original point cloud can be queried using the key (u k , v k , m k ), which is the combination of the 2D spherical coordinates and the point index in the spherical frustum point set. Based on the hash-based representation, the spherical frustums are regularly stored in a memory-efficient way.Since multiple points are stored in a single spherical frustum, the conventional 2D convolution can not be directly performed on the spherical frustum structure. Therefore, we propose Spherical Frustum sparse Convolution (SFC). As shown in Fig. 2 , SFC can be seen as the sparse convolution on the virtual spherical plane of the center point. The feature of each convolved 2D position on the virtual spherical plane is filled with the feature of the nearest point in the corresponding spherical frustum.Selecting Convolved Spherical Frustums. SFC first selects the convolved spherical frustums for each center point p. The 3D coordinates and the 2D spherical coordinates of the center point p are (x, y, z) and (u, v) respectively. The conventional 2D convolution convolves the features of the grids in the convolution kernel. Similar to the conventional convolution, the spherical frustum of each 2D position in the convolution kernel is selected to perform the convolution. The coordinates of the 2D positions are {(u+ ∆u i , v + ∆v i )} K 2 i=1, where K is the kernel size and (∆u i , ∆v i ) are the shift inside the kernel. Then, the points inside each spherical frustum are queried through the hash table. Meanwhile, the features {f j } Mi j=1 of these points are obtained, where M i is the number of the points Then, the nearest point in each spherical frustum is determined by the 3D geometric information. Finally, the sparse convolution is performed on the selected point features.in the i-th spherical frustum. Notably, M i can be zero, which means that no points are projected onto the corresponding 2D position, and the spherical frustum is invalid. The invalid spherical frustums are ignored in the subsequent convolution.Selecting the Nearest Point in Each Spherical Frustum. After identifying the points in the spherical frustum, a feature should be selected from the features of the frustum point set for 2D convolution. PointNet++ [26] emphasizes that the local feature of the center point is expected to be aggregated from the 3D neighboring points. Inspired by PointNet++, we select the feature of the nearest point to the center point in each spherical frustum. Specifically, based on the 3D geometric information, the 3D coordinates {(x j , y j , z j )} Mi j=1 of the frustum points are obtained for the nearest point selection. Inspired by the post-processing of RangeNet++ [13] , we select the distance of range r j = x 2 j + y 2 j + z 2 j rather than the Euclidean distance as the metric of the nearest point for efficient distance calculation. Therefore, the selected point index of each spherical frustum is arg min j |r j -r|, where r is the range of the center point. According to the indexes, the convolved features {f i } K ′ i=1 are obtained, where K ′ is the number of valid spherical frustums.Sparse Convolution. Finally, the sparse convolution is performed as:EQUATIONwhere W i is the convolution weight of the i-th valid 2D position, and f ′ is the aggregated feature.Through the proposed spherical frustum sparse convolution, we realize effective regularization and 2D convolution-based feature aggregation for all points in the unstructured point cloud.Sampling is a significant process of point cloud semantic segmentation. Through sampling, the network can aggregate the features of different scales and recognize objects of different sizes. Moreover, the sampling should uniformly sample the point cloud to avoid key information loss. The previous 2D projection-based methods sample the projected point cloud using stride-based 2D sampling. This sampling ignores the 3D geometric structure of the point cloud. In contrast, as shown in Fig. 3 [26] is adopted to uniformly sample the point set in the downsampled spherical frustum. Since the point number of each downsampled spherical frustum is much smaller than the point number of the point cloud, performing FPS is not time-consuming. Specifically, the 3D coordinates of each point in {p l } L l=1 are first acquired from the 3D geometric information for 3D distance calculation. Then, the ⌈L/(S h × S w )⌉ points are iteratively sampled from the original point set. At each iteration, the distance of each non-sampled point towards the sampled point set is calculated. The point that has the maximal distance is added to the sampled set. Finally, the uniformly sampled spherical frustum point set is obtained.F2PS integrates the stride-based spherical frustum sampling with the FPS-based frustum point set sampling. Thus, F2PS can sample the original point cloud uniformly. In addition, since performing FPS on the frustum point set costs O(1) time, the computing complexity of F2PS is O(N ). Thus, F2PS is an efficient point cloud sampling algorithm.Based on the Spherical Frustum sparse Convolution (SFC) and Frustum Farthest Point Sampling (F2PS), the Spherical Frustum sparse Convolution Network (SFCNet) is constructed. SFCNet is an encoder-decoder architecture. The hash-based spherical frustum representation is first built for convolution and sampling in the subsequent modules. Then, the point features {f ∈ R C } are extracted through the encoder of SFCNet, where C is the channel dimension. The encoder consists of the residual convolutional blocks from ResNet [44] , where the convolutions are replaced by the proposed SFC. In addition, the point cloud is downsampled based on F2PS to extract the features of different scales. After each downsampling operation, an SFC layer is adopted to aggregate the neighbor features for each sampled point. Since F2PS can uniformly sample the point cloud, the information of the original point cloud can be fully gathered in the downsampled point cloud. In the decoder, the extracted features are upsampled, concatenated, and fed into the head layer to output the prediction of semantic segmentation. 
experiments: 
 In this section, we first introduce the datasets adopted in the experiments and the implementation details of the SFCNet. Then, the quantitative results of the two datasets and the qualitative results of the SemanticKITTI dataset are presented. Finally, the ablation studies and comparison with restoring-based methods are conducted to validate the effectiveness of the proposed modules. We train and evaluate SFCNet on the SemanticKITTI [1] and nuScenes [27] datasets, which provide point-wise semantic labels for large-scale LiDAR point clouds.SemanticKITTI [1] sets. In addition, in nuScenes dataset, 16-class point-wise semantic annotations are provided for the LiDAR semantic segmentation task.On both datasets, the performance of LiDAR point cloud semantic segmentation is evaluated by mean Intersection-over-Union (mIoU) [45] .SFCNet is implemented through PyTorch [46] framework. For spherical frustum, the height and width in the calculation of spherical coordinates are set as H = 64, W = 1800 for the SemanticKITTI dataset, and H = 32, W = 1024 for the nuScenes dataset. The channel dimensions C of the extracted point features for SemanticKITTI and nuScenes datasets are set as 128 and 256 respectively. The strides (S h , S w ) of the F2PS are all set as (2, 2). The multi-layer weighted cross-entropy loss and Lovász-Softmax loss [47] are adopted for network optimization. Adam [48] with the initial learning rate 0.001 is treated as the optimizer. The learning rate is delayed by 5% in every epoch. Random rotation, flipping, translation, and scaling are utilized for data augmentation on both datasets. Model training is conducted on a single NVIDIA Quadro RTX 8000. The training batch size is set as 4.We compare our SFCNet to the State-of-The-Art (SoTA) 2D projection-based, point-based and 3D voxel-based segmentation methods on the SemanticKITTI and nuScenes datasets.As shown in Tabs. 1 and 2, SFCNet outperforms the previous SoTA 2D convolution-based segmentation methods CENet [20] and SalsaNext [15] on the SemanticKITTI and nuScenes datasets respectively. In addition, SFCNet also has better performance than the vision transformer-based segmentation method RangeViT [21] on both two datasets. SFCNet also outperforms the point-based methods and realizes a smaller performance gap between the 2D projection-based methods and the 3D voxel-based methods. As for the per-class IoU comparison, SFCNet has better IoU than the other 2D projection-based methods on the small 3D objects, including the motorcycle, person (which is pedestrian in nuScenes), bicyclist, trunk, and pole. The performance improvement on these small objects results from the elimination of quantized information loss. Without quantized information loss, the complete geometric structure of the small 3D objects can be preserved, which enables more accurate segmentation. We also observe the slightly weaker performances on wide-range classes, e.g., road, parking, and terrain, on the SemanticKITTI dataset. However, since preserving complete points significantly improves the accuracies of the hard small objects, SFCNet has a higher mean IoU than the previous 2D projection-based methods.Fig. 4 presents the qualitative comparison between our SFCNet and the 2D projection-based segmentation methods CENet [20] and RangeViT [21] . The comparison shows that the predictions of SFCNet have the minimal segmentation error among the three methods. Moreover, the circled objects in the three rows of Fig. 4 demonstrate the accurate segmentation of SFCNet to the persons, poles, and trunks respectively. This result further indicates our better segmentation performance of 3D small objects by eliminating the quantized information loss.In this section, we conduct the ablation study on the SemanticKITTI dataset to validate the effectiveness of the proposed modules. We adopt the baseline network using the conventional spherical projection and stride-based sampling. The results of ablation studies are shown in Tab. 3. Spherical Frustum Sparse Convolution (SFC). First, we replace spherical projection in the baseline with spherical frustum and adopt spherical frustum sparse convolution for feature aggregation. After replacement, the mIoU increases 4.3%, which indicates that spherical frustum structure can avoid the quantized information loss, and thus prevent segmentation error from incomplete predictions.Frustum Farthest Point Sampling (F2PS). After replacing the stride-based 2D sampling with F2PS, the mIoU increases 2.4%. F2PS uniformly samples the point cloud and preserves the key information. Thus, the performance of semantic segmentation has been improved. Based on the same baseline network in Sec. 4.5, we compare our SFCNet with the methods that compensate for the quantized information loss by restoring complete predictions from partial predictions, including the KNN-based postprocessing [13] and KPConv refinement [17] .Tab. 4 shows that SFCNet has 3.2% mIoU improvement to the KNN-based post-processing and 2.8% mIoU improvement to KPConv refinement. Compared to the restoring-based methods, SFCNet preserves the complete geometric structure for the feature aggregation rather than compensating for the information loss by post-processing or refinement, which results in higher performance of semantic segmentation. 
conclusion: 
 In this paper, we present the Spherical Frustum sparse Convolutional Network (SFCNet), a 2D convolution-based LiDAR point cloud segmentation method without quantized information loss. The quantized information loss is eliminated through the novel spherical frustum structure, which preserves all the points projected onto the same 2D position. Moreover, the novel spherical frustum sparse convolution and frustum farthest point sampling are proposed for effective convolution and sampling of the points stored in the spherical frustums. Experiment results on SemanticKITTI and nuScenes datasets show the better semantic segmentation performance of SFCNet compared to the previous 2D projection-based semantic segmentation methods, especially on small objects. The results show the great potential of SFCNet for safe autonomous driving perception due to the accurate segmentation of small targets.Limitations and future work. To implement the 2D convolution on the spherical frustum, only the nearest points in the neighbor spherical frustums are adopted in the spherical frustum sparse convolution. This design may limit the receptive field of the network and thus result in a slightly weaker performance of the wide-range classes. To maintain the performance on both the wide-range classes and small classes, the improvement direction is to expand the receptive field based on our spherical frustum structure. To realize this, future work can lie in combining the vision network architecture with a larger receptive field, like the vision transformer [49] or vision mamba [50] , with our spherical frustum structure. In addition, our work mainly focuses on the supervised and unimodal point cloud semantic segmentation. Future work can also lie in adopting the spherical frustum structure on the weakly-supervised [51] and multi-modal [52] point cloud semantic segmentation. Moreover, applying the spherical frustum structure to more tasks on the LiDAR point cloud, like point cloud registration [53, 54] and scene flow estimation [55] , is also a direction for future work. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: The dataset splits, hyperparameters, and optimizer choice are described in Sec. 4.1 and 4.2.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Some results need to be uploaded to the dataset's official website for evaluation since the ground truths of the test set are not publicly available. Thus, it is not feasible for multiple evaluations to obtain the error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The work is aimed at scene understanding from the LiDAR sensor data. There are no potential risks of misusing.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited the papers providing the public datasets used in this paper. The licenses of the datasets will be included in the released source code.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: This paper does not include new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This work is irrelevant to human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: This work is irrelevant to human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In the appendix, we first introduce the detailed architecture of the Spherical Frustum sparse Convolution Network (SFCNet) in Sec. A. Then, the additional implementation details of SFCNet are presented in Sec. B. Next, the additional experimental results are illustrated in Sec. C. Finally, more visualization of the semantic segmentation results on the SemanticKITTI [1] and nuScenes [27] datasets are presented in Sec. D. A Detailed Architecture Basic Modules of SFCNet. Specifically, the SFC layer is composed of the SFC, batch normalization, and the activation function. Inspired by [20] , we use Hardswish [56] as the activation function. The formula of Hardswish is: EQUATIONData Normalization. For the k-th point in the LiDAR point cloud P, the combination of the 3D coordinatesx k = [x k , y k , z k ] T , the range r k = x 2 k + y 2 k + z 2 k, and the intensity is treated as the input point feature f k . Because of the different units of the different data categories, the input features should be normalized. Specifically, for the SemanticKITTI [1] dataset, like RangeNet++ [13] , we minus the features by the mean and divide the features by the standard deviation to obtain the normalized features. The mean and standard deviation are obtained from the statistics of each input data category on the SemanticKITTI dataset, which are presented in Tab. 6.For the nuScenes [27] dataset, like Cylinder3D [23] , a batch normalization layer is applied on the input point features to record the mean and standard deviation of the nuScenes dataset during training.During inferencing, the recorded mean and standard deviation are used to normalize the input point features.Spherical Frustum Construction. We construct the spherical frustum structure by assigning each point with the 2D spherical coordinates (u k , v k ) and the point index m k in the spherical frustum point set, where k is the index of the point in the original point cloud. The 2D spherical coordinates can be calculated through Eq. 1. Thus, the key process is to assign the point index m k for each point based on the 2D spherical coordinates.We implement this by sorting the 2D coordinates (u k , v k ) of the points. The points with smaller u k and v k are ranked ahead of the points with larger u k and v k . Thus, the points with the same 2D coordinates are neighbors in the sorted point cloud. For each point, we count the number of points that have the same 2D coordinates and appear ahead or behind the point in the sorted point cloud separately. The number of the points appearing ahead is treated as the point index m k of each point.In addition, we assign each point an indicator ξ k ∈ {0, 1} according to the number of the points appearing behind. The point with zero point appearing behind is assigned a zero indicator. Otherwise, the point is assigned with an indicator equal to one. The indicator indicates the end of the frustum point set and is used for the subsequent spherical frustum point set visiting.The sorting and the point number counting are implemented through the Graphics Processing Unit (GPU)-based parallel computing using Compute Unified Device Architecture (CUDA). Thus, the construction is efficient in practice.Hash-Based Spherical Frustum Representation. After the construction of the spherical frustum structure, we build the hash-based spherical frustum representation. Specifically, we construct the key-value pairs between the key (u k , v k , m k ) and the value k. The key-value pairs are inserted into a hash table, which represents the neighbor relationship of spherical frustums and points.In practice, we adopt an efficient GPU-based hash table [58] . The GPU-based hash table requires both key and value to be an integer. The value k satisfies the integer requirement. However, the key (u k , v k , m k ) in the hash-based spherical frustum representation is not an integer.To adopt the GPU-based hash table for efficient processing,(u k , v k , m k ) is transferred to an integer as v k • (W • M ) + u k • M + m k ,where W is the width of the spherical projection, M is the maximal point number of the spherical frustum point sets. Through this process, any point represented by the coordinates (u k , v k , m k ) can be efficiently queried through the GPU-based hash table.Spherical Frustum Point Set Visiting. Both the SFC and F2PS require visiting all the points in any spherical frustums. Thus, we propose the spherical frustum point set visiting algorithm. The visiting obtains all the points in the given spherical frustum, whose 2D coordinates are (u, v), by sequentially querying the points in the hash table.Specifically, we first query the first point in the spherical frustum using the key (u, v, 0). If the key (u, v, 0) is not in the hash table, the spherical frustum on (u, v) is invalid. Otherwise, the first point in the spherical frustum can be queried through the hash table.Then, the points in the spherical frustum are sequentially visited. We first initialize the point index m = 0 in the spherical frustum. At each step, the point index m increases by one. Through the hash table, the point with m-th index in the spherical frustum is queried using the key (u, v, m). Meanwhile, the indicator ξ of this point is obtained. ξ indicates whether (u, v, m + 1) refers to a valid point. Thus, the visiting ends when the indicator of the current point is zero.Detailed Implementation of Frustum Farthest Point Sampling. In F2PS, we first sample the spherical frustums by stride. Then, we sample the points in each sampled spherical frustum by Farthest Point Sampling (FPS) [26] . As mentioned in Sec. 3.3, FPS is an iterative algorithm. The detailed process of the j-th iteration can be expressed by the following formula:EQUATIONwhere P s is the spherical frustum point set to be sampled, S j and S j-1 are the sampled point sets in j-th and (j -1)-th iterations respectively. Notably, S 0 contains the point randomly sampled from P s . In addition, dist(p, s) is the distance between point p and point s in 3D space. The iteration starts at j = 1, and ends when the size of S k equals the number of sampling points.Moreover, since the distances between the points in S j-2 and the points in P s \S j-1 have been calculated before the j-th iteration, we just need to calculate the distance between each p in P s \S j-1 and the point sampled in (j -1)-th iteration for the calculation of min s∈Sj-1 dist(p, s), which is the minimal distance from point p to the point set S j-1 . Thus, the computing complexity of FPS for P s of size n is O(n 2 ). Since the point number of each spherical frustum is O(1), the computing complexity of FPS for the spherical frustum is also O(1), which ensures the efficiency of F2PS.Loss Function. We use multi-layer weighted cross-entropy loss and Lovász-Softmax loss [47] to help the network learn the semantic information from different scales. To get the semantic predictions of extraction layers 1 to 4, we apply a linear layer to decode the extracted point features of each extraction layer into the semantic predictions.Specifically, for extraction layer 1, the linear layer is applied on the extracted point features F 1 to gain the prediction L1 . For the other extraction layers, the linear layer is applied on the upsampled point features F ′ 2 , F ′ 3 , and F ′ 4 to obtain the predictions L2 , L3 , and L4 respectively. Based on the predictions of each layer and the final predictions of SFCNet L1 , the loss function is calculated as:EQUATIONwhere L wce is the weighted cross-entropy loss, L Lov is the Lovász-Softmax loss, and L is the ground truth. In addition, the weights of weighted cross-entropy loss are calculated as w c = (f c + ϵ) -1 , where c is the semantic class, f c is the frequency of class c in the dataset, and ϵ is a small positive value to avoid zero division.We evaluate the efficiency of the proposed SFCNet with the previous works and our 2D projectionbased baseline model on a single Geforce RTX 4090Ti GPU.We adopt the same baseline model used in Sec. 4.5. For RangeViT, we adopt the official code for efficiency evaluation. Notably, in the inference, RangeViT splits the projected LiDAR image, inputs each image slice into the network to gain the predictions, and merges the predictions to gain the prediction of the entire projected LiDAR image. Thus, the inference time of RangeViT includes the time of all the processes. In addition, since RangeViT adopts the KPConv refinement [17] , which restores the complete predictions from the partial predictions, we use the point number of the entire point cloud as the processed point number. For PointNet++ [26] , we sample 45K points from the point cloud before inputting into the network as its original setting. For 3D voxel-based methods, Cylinder3D [23] and SphereFormer [25] , only the points preserved after the voxelization are counted since these points are exactly processed in the 3D sparse convolution network. Since the limited projection resolution is the reason for quantized information loss, expanding the resolution of the projected range image can preserve more points during the spherical projection and ease the quantized information loss. However, expanding the resolution increases the sparsity of the projected points and makes the convolution hard to aggregate the local features. Thus, resolution expansion is not a feasible solution for resolving quantized information loss. To validate this, we expand the image horizon resolution of the baseline model to 2048 and 4096 and conduct the ablation studies of different resolutions on the SemanticKITTI validation set to show the effect of a larger resolution. As shown in Tab. 8, the increment of resolution preserves more points but results in worse performances. In contrast, SFCNet not only overcomes quantized information loss but also effectively aggregates local features with a suitable resolution by preserving all points using spherical frustum.In this subsection, we conduct additional ablation studies to evaluate the sensitivity of our SFCNet to the key parameters.Stride Sizes in Frustum Farthest Point Sampling (F2PS). The ablation studies of four different settings of the stride sizes in the F2PS on the three downsampling layers are conducted, including (1, 2), (2, 1), (2, 4) , and (4, 2). The results are shown in Tab. 9. The results show on all the downsampling layers, the (2, 2) stride sizes show a better segmentation performance than the other stride size settings. (2, 2) stride sizes suitably downsample the point cloud in the vertical and horizon dimensions. Higher or lower downsampling rates result in the oversampling or undersampling of the point cloud respectively.Frustums. In the spherical frustum structure, the number of points in the frustum is unlimited and only depends on how many points are projected onto the corresponding 2D location. To analyze the effect of the number of points in the frustum, we set the maximal number of points in each spherical frustum and the points exceeding the maximal point number are dropped. As shown in Tab. 10, preserving more points in the spherical frustum results in better segmentation performance, since more complete geometry information is preserved. These We further validate the effectiveness and efficiency of the proposed Frustum Farthest Point Sampling (F2PS) by the qualitative comparison with stride-based 2D sampling and the comparison of time consumption with Farthest Point Sampling (FPS).Qualitive Comparison. As shown in Fig. 6 (a), Stride-Based 2D Sampling (SBS) only samples the point cloud based on 2D stride. The visualization shows that the stride-based sampled point cloud is relatively rough. Due to the lack of 3D geometric information, SBS fails to sample the 3D point cloud uniformly. Thus, the loss of geometric structure in the sampled point cloud is obvious, such as many broken lines on the ground. Our F2PS takes into account the 3D geometric information based on the FPS in the spherical frustum, which enables F2PS to sample the 3D point cloud uniformly and preserve the significant 3D geometric structure during the sampling.As shown in Fig. 6(b) , with the increment of sampled point number, the cost time of our F2PS increases slowly, while the cost time of FPS increases dramatically.This result shows performing FPS on the frustum point sets is efficient and does not increase the computing burden.To further show the improvement of small object segmentation, we compare the quantitative results on the small object categories between SFCNet and the baseline model with the KNN-based postprocessing [13] on SemanticKITTI and nuScenes validation sets. As shown in Tabs. 12 and 13, our SFCNet has higher performances on all small object categories compared to the baseline model. The results show overcoming quantized information loss preserves complete geometric information of the small objects and thus makes them better recognized and segmented by our SFCNet. To better demonstrate the effectiveness of SFCNet for LiDAR point cloud semantic segmentation, we conduct more visualization on the SemanticKITTI and nuScenes datasets. The results are shown in Figs. 7, 8, 9, 10 and the supplementary video SupplementaryVideo.mp4.Qualitative Comparison on NuScenes Validation Set. The results of qualitative comparison between our SFCNet and RangeViT [21] are shown in Fig. 7 . On the nuScenes validation set, SFCNet can also have fewer segmentation errors than RangeViT as the results in the SemanticKITTI dataset. Moreover, the better segmentation accuracy of the 3D small objects, like pedestrians and motorcycles, can also be observed on the nuScenes validation set. The results once more demonstrate semantic segmentation improvement of SFCNet due to the overcoming of quantized information loss.More Qualitative Comparison on SemanticKITTI Test Set. The ground truths on the Se-manticKITTI test set are not available. Thus, we search for the corresponding RGB image and project the semantic predictions on the image to compare the semantic segmentation accuracy between the state-of-the-art 2D image-based method CENet [20] and our SFCNet on the SemanticKITTI test set. As shown in Figs. 8 and 9 , compared to CENet, SFCNet can more accurately segment the LiDAR point cloud in various challenging scenes on the SemanticKITTI test set.Specifically, SFCNet recognizes the thin poles in distance on the rural road of Fig. 8 (a) and in the complex intersections of Fig. 9 (c), while CENet predicts the poles as wrong classes. In addition, SFCNet recognizes the thin trunks inside the vegetation on the rural scenes of Fig. 8 (b) and Fig. 9 (b) while CENet wrongly predicts the trunk as the fetch and vegetation respectively. Moreover, SFCNet successfully segments the boxed persons in the complex intersection of Fig. 8(c ) and in the urban scene of Fig. 9 (a) while CENet gives wrong predictions due to the information loss of the distant persons during 2D projection. These results further validate the better segmentation performance of SFCNet to 3D small objects.More Qualitative Comparison on NuScenes Validation Set. As the visualization on the Se-manticKITTI test set, we provide the additional qualitative comparison between our SFCNet and RangeViT on the nuScene validation set with the projected predictions illustrated in Fig. 10 . The results further demonstrate the better semantic segmentation of SFCNet for the challenging street scenes on the nuScenes validation set compared to RangeViT.Specifically, in the first scene, the close motorcycle can be correctly segmented by SFCNet, while RangeViT recognizes the motorcycle as a car, which shows that SFCNet can help the autonomous car correctly recognize the type of close obstacles, and enable the car to make appropriate decisions. In the second scene, the distant pedestrians on the other side of the crossing can also be correctly segmented by SFCNet due to the elimination of quantized information loss. In contrast, RangeViT wrongly predicts the pedestrians as traffic cones.In the third scene, since the boxed pedestrian is close to the manmade, RangeViT confuses it with the manmade and does not segment the pedestrian, while our SFCNet can clearly recognize the boundary and successfully segments the pedestrian.Sequential Qualitative Comparison on SemanticKITTI Validation Set. We demonstrate the qualitative comparison between our SFCNet and the SoTA 2D projection-based segmentation methods, CENet and RangeViT, on a continuous sequence on the SemanticKITTI validation set in the supplementary video SupplementaryVideo.mp4. In this video, the semantic predictions in both the 3D point cloud view and the RGB image view (where the colored point cloud is projected onto the RGB images) are presented. The results show that our SFCNet can consistently show higher segmentation accuracy on the point cloud of each frame in the sequence than 2D projection-based methods, which further indicates the stronger semantic segmentation capability of our SFCNet. [20] on the urban, rural, and complex intersection scenes of the SemanticKITTI test set. As in Fig. 8 , the predictions projected on the corresponding RGB images are also illustrated. In addition, the same color boxes are adopted to point out the same objects in the point clouds and images for each scene. Meanwhile, the zoomed-in view of some boxed objects is illustrated for clear visualization. Moreover, the reference from point color to the semantic class in the predictions is shown at the bottom of the figure. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 


Decision: After rebuttal and discussion, all four reviewers are positive, but to a varying degree: all scores between borderline accept and strong accept were given. The main strengths are novelty of the representation, the efficiency, and the performance, where the least positive reviewer criticizes the lack of experimental evidence on the latter.

Reviews: review: paper_summary: paper_summary: This paper introduces a novel spherical frustum structure for LiDAR point cloud semantic segmentation, addressing the quantized information loss in 2D projection-based methods. By preserving all points within frustums and employing a memory-efficient hash-based representation, the Spherical Frustum sparse Convolution Network (SFCNet) achieves superior segmentation accuracy, particularly for small objects. The proposed Frustum Farthest Point Sampling (F2PS) method ensures uniform sampling and computational efficiency. Extensive experiments on the SemanticKITTI and nuScenes datasets demonstrate SFCNet's improved performance over existing methods.
strength_weakness: strength_weakness: - The proposed methods, including the spherical frustum structure and hash-based representation, add layers of complexity that might pose challenges for practical implementation. Interesting to see the analyze on memory usage, parameter size, and inference time.
- The paper focuses primarily on comparisons with 2D projection-based methods. While it mentions the performance gap with 3D voxel-based methods, a more detailed analysis and direct comparison with state-of-the-art 3D voxel-based approaches would provide a clearer picture.
- The paper lacks a sensitivity analysis of key parameters, such as the stride size in the F2PS, the number of points in the spherical frustums, and the impact of different hash table configurations, etc. Understanding how sensitive the model performance is to these parameters would be valuable for tuning the model for specific applications.
- LiDAR data often contains noise and outliers. The robustness of the proposed method to such imperfections is not thoroughly evaluated. Experiments that introduce varying levels of noise and assess the impact on segmentation performance would be beneficial in demonstrating the method’s robustness.
- The qualitative results, although helpful, are somewhat limited in scope. Providing a more diverse set of visualizations, including different types of scenes (e.g., urban, rural, complex intersections), would offer a clearer picture of the method’s practical performance across various real-world scenarios.
questions: questions: I overall like the idea of this paper. However, my concerns are focused on the experimental results. If the authors can address my concerns regarding the experiments, I am inclined to raise my score.
limitations: limitations: no potential negative social impact
paper_summary: paper_summary: The paper introduces a Spherical Frustum Sparse Convolution Network (SFCNet), a novel approach for LiDAR point cloud semantic segmentation that addresses the quantized information loss existing in 2D projection-based methods. By utilizing a spherical frustum structure, the SFCNet preserves all points within a 2D projection area, thus maintaining the complete geometric structure of the data. This model improves the segmentation accuracy, particularly for small objects, through the innovative use of hash-based spherical frustum representation and a custom sampling method called Frustum Farthest Point Sampling (F2PS). Extensive evaluations on the SemanticKITTI and nuScenes datasets demonstrate superior performance over existing methods.
strength_weakness: strength_weakness: W1. Performance Decrease on Large Classes. While SFCNet introduces significant advancements in semantic segmentation of LiDAR data, the model demonstrates slightly weaker performance on wide-range classes, such as roads and parking areas. This suggests that while the model excels in handling small object segmentation, its techniques might not be as effective for broader landscapes or larger objects.
questions: questions: Q1a. Performance Change on Large Classes. Could the authors provide insights into why SFCNet shows slightly weaker performance on wide-range classes, such as roads and parking areas (see also W1)?

Q1b. Maintaining performance on large Classes. Are there any thoughts how the model could improve the small classes while maintaining performance on larger classes?

Q2. Performance Comparison toward tail-end of Distribution. Given a different performance w.r.t. class sizes, additional tests on accuracy w.r.t. data changes might be helpful. How does SFCNet adapt its segmentation accuracy in non-standard environments (like in night settings, heavy rain, etc.)? Insights here could greatly influence its applicability in real-world settings.
limitations: limitations: Investigating how the system performs in unpredictable, non-standad settings would offer a clearer picture of its real-world readiness. The paper can also benefit from a deeper exploration of SFCNet's slightly weaker performance on wide-range classes such as roads and parking areas. Understanding the factors that contribute to these discrepancies could lead to improvements in the model's overall accuracy and applicability in real-world scenarios.
paper_summary: paper_summary: This paper introduces SFCNet, a spherical frustum sparse convolution network designed for semantic segmentation of LiDAR point clouds. Traditional 2D projection methods suffer from quantized information loss when multiple points project onto the same pixel, leading to sub-optimal segmentation. To address this, the authors propose a spherical frustum structure that retains all points within the same pixel by using a hash-based representation. This structure is then utilized in spherical frustum sparse convolution, which considers both neighboring frustums and the nearest point within each frustum for convolution. Additionally, Frustum Farthest Point Sampling (F2PS) is used to sample points stored in spherical frustums. Experimental results on the SemanticKITTI and nuScenes datasets demonstrate that SFCNet outperforms conventional 2D projection-based methods, particularly in the segmentation of small objects, while maintaining complete point cloud information.
strength_weakness: strength_weakness: 1. The experimental results do not demonstrate a clear superiority of the proposed method. While the SFCNet shows some mIoU improvements over prior projection methods on the SemanticKITTI and nuScenes datasets, these gains are marginal. Furthermore, the performance of SFCNet is still significantly below that of SOTA 3D voxel-based methods.
2. The paper does not provide an analysis of the computational efficiency of the proposed method. Given that SFCNet involves specific point sampling for convolution, assessing the computational efficiency is crucial to understand its practical significance and potential trade-offs in real-world applications.
questions: questions: In Table 1, it is clear  that 3D voxel-based methods achieve significantly better results compared to point-based and 2D projection-based methods. Does this suggest that 3D voxel-based methods are inherently superior?

Could you provide a comparison of the computational efficiency of SFCNet against other baseline methods? This information is crucial to evaluate the practical applicability and performance trade-offs of SFCNet.
limitations: limitations: No limitations are discussed in the paper.
paper_summary: paper_summary: This paper proposes a Spherical Frustum Sparse Convolution Network to address the challenge of LiDAR point cloud semantic segmentation. Traditional approaches often project point clouds into 2D images and apply 2D convolutional neural networks (CNNs) or vision transformers, leading to quantization information loss due to the overlap of multiple points projected onto the same 2D location. To overcome this limitation, the paper introduces a novel spherical frustum structure that allows for direct processing of the 3D point cloud data, preserving spatial and geometric information. The proposed Spherical Frustum Sparse Convolution Network achieves more accurate semantic segmentation of LiDAR point clouds, offering a promising approach for robot perception and scene understanding.
strength_weakness: strength_weakness: 1. **Lack of Conceptual Novelty**: The use of spherical structures, while effective, appears to have been partially explored in SphereFormer. This limits the novelty of the proposed SFCNet in terms of the core concept.

2. **Performance Gap**: Based on the information provided, the performance of SFCNet for semantic segmentation of LiDAR point clouds lags far behind that of SphereFormer, whose radial window transformer structure may achieve higher recognition accuracy, especially for distant objects.

3. **Potential for Further Optimization**: Since SFCNet based on 2D projections lags behind SphereFormer based on 3D voxels, there may be room for optimization in terms of data input or network architecture. This paper does not discuss potential future avenues to address this performance gap or to work on different application scenarios.
questions: questions: Despite the fact that the paper is full of content, it always gives a discomfort in the presentation. Obviously, the authors are requested to resize Figure 1, remove the captions in the figures of Figures 2 and 3 and optimize their presentation. It is recommended that the internal spacing of Table 1 be increased appropriately. The authors are invited to review more carefully.

In addition, similar to the work on semantic segmentation of large scenes, the authors should compare it with state-of-the-art methods such as DRQNet (ACMMM'2023) and JoSS (INFFUS'2024).
limitations: limitations: This paper proposes a method SFCNet for point cloud semantic segmentation by directly projecting 3D point clouds into 2D spherical frustums, the main limitation of which lies in the large performance gap. Of course, the proposed method still has a large potential in specific scenarios.

Comments: title: Raised my rating to ACCEPT
Response from Reviewer
comment: Thank you very much for your time and your recognition of our method and response. We promise to incorporate our responses to your Q3 and Q5 into the supplementary material in the final version of our paper.
I appreciate the authors' rebuttal and additional experiments. I have read the comments from other reviewers and the related rebuttal. The rebuttal has effectively addressed my main concerns. Additionally, I highly appreciate the responses to Q3 and Q5, and I suggest incorporating them into the supplementary material.

Overall, I believe this is a technically solid paper, and I am inclined to accept it - I have raised my rating from borderline accept to **ACCEPT**.
Thank you very much for your time and your recognition of our method and response. We promise to include the discussion of Q1 in the final version of our paper. Your valuable suggestions really help us improve the illustration of our conceptual novelty in a more dialectical way.
Many thanks for the clarification which addressed all my open questions!

I would really encourage the authors to include a discussion along Q1 into the paper to illustrate the limitations of the current approach, too. This may also help to emphasize the Conceptual Novelty in a more dialectic way (c.f. reviewer BSNT).
Thank you very much for your time and your recognition of our method and response. We promise to make the appropriate changes, including adding the discussions, revising our presentation according to your suggestions, etc., on the revised version of our paper.
Thanks to the reviewers for their serious response! A lot of my questions and concerns have been addressed. I hope the authors will make the appropriate changes on the revised version of the paper. In return, I will further improve my rating.