Original: Spatial Pruned Sparse Convolution for Efficient 3D Object Detection
Revised: Spatial Pruned Sparse Convolution for Efficient 3D Object Detection

Original: Anonymous Author(s)

Original: AffiliationAddress email Abstract 3D scenes are dominated by a large number of background points, which is redundant for the detection task that mainly needs to focus on foreground objects.
Revised: Abstract 3D scenes are dominated by a large number of background points, which is redundant for the detection task that mainly needs to focus on foreground objects.

Original: In thispaper, we analyze major components of existing sparse 3D CNNs and find that 3D CNNs ignore the redundancy of data and further amplify it in the down-samplingprocess, which brings a huge amount of extra and unnecessary computationaloverhead.
Revised: In this paper, we analyze major components of existing sparse 3D CNNs and find that 3D CNNs ignore the redundancy of data and further amplify it in the down-sampling process, which brings a huge amount of extra and unnecessary computational overhead.

Original: Inspired by this, we propose a new convolution operator named spatialpruned sparse convolution (SPS-Conv), which includes two variants, spatial prunedsubmanifold sparse convolution (SPSS-Conv) and spatial pruned regular sparseconvolution (SPRS-Conv), both of which are based on the idea of dynamically determine crucial areas for performing computations to reduce redundancy.
Revised: Inspired by this, we propose a new convolution operator named spatial pruned sparse convolution (SPS-Conv), which includes two variants, spatial pruned submanifold sparse convolution (SPSS-Conv) and spatial pruned regular sparse convolution (SPRS-Conv), both of which are based on the idea of dynamically determining crucial areas for redundancy reduction.

Original: We empirically find that  magnitudeof features can serve as important cues to determinecrucial areas which get rid of the heavy computations of learning-based methods.
Revised: We validate that the magnitude can serve as important cues to determine crucial areas which get rid of the extra computations of learning-based methods.

Original: The proposed modules can easily be incorporated into existing sparse 3D CNNs without extra architectural modifications.
Revised: The proposed modules can easily be incorporated into existing sparse 3D CNNs without extra architectural modifications.

Original: Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method can achieve more than 50%reduction in GFLOPs without compromising the performance. 
Revised: Extensive experiments on the KITTI, Waymo and nuScenes datasets demonstrate that our method can achieve more than 50% reduction in GFLOPs without compromising the performance. Code is available at: https://github.com/CVMI-Lab/SPS-Conv .

Original:  Introduction
Revised:  Introduction

Original: 3D object detection has always been a research field of great interest due to its wide applicationsin autonomous driving, virtual reality, and robotics.
Revised: 3D object detection has always been a research field of great interest due to its wide applications in autonomous driving, virtual reality, and robotics.

Original: However, compared with the 2D detectiontask, LiDAR-based 3D detection is more challenging due to the inherent characteristics of LiDARpoints such as disorders, irregularity and non-uniformity.
Revised: However, compared with the 2D detection task, LiDAR-based 3D detection is more challenging due to the inherent characteristics of LiDAR points such as disorders, irregularity, and non-uniformity.

Original: Recently, voxel-based methods [23, 10,2, 36, 30, 18] with deep 3D sparse convolutional neural networks (CNNs) have become one of themajor research streams to tackle 3D object detection problem due to its effectiveness and simplicity.
Revised: Recently, voxel-based methods [23, 10, 2, 37, 31, 18] with deep 3D sparse convolutional neural networks (CNNs) have become one of the major research streams to tackle 3D object detection problem due to its effectiveness and simplicity.

Original: However, the improved accuracy is often accompanied by increased computational costs [29, 35, 3],limiting its applicability in practical systems.
Revised: However, the improved accuracy is often accompanied by increased computational costs [30, 36, 3], limiting its applicability in practical systems.

Original: This motivates us to investigate potential redundanciesin the detection model that can be safely avoided to improve efficiency without sacrificing accuracy.
Revised: This motivates us to investigate potential redundancies in the detection model that can be safely avoided to improve efficiency without sacrificing accuracy.

Original: When delving into the task of 3D detection, we found that 3D data itself has high redundancy asshown in Fig. 1(a).
Revised: When delving into the task of 3D detection, we found that 3D data itself has high redundancy as shown in Fig. 1(a).

Original: Notably, compared with background points, the proportion of foreground points inthe entire scene is extremely low (around 5%), which shows that non-informative background pointsoccupy the major areas of a scene.
Revised: Notably, compared with background points in each stage of the sparse CNN, the proportion of foreground points in the entire scene is extremely low (around 5%), which shows that less-informative background points occupy the major areas of a scene.

Original: However, the existing 3D sparse CNNs are applied uniformlyto the whole scene, causing a considerable amount of computation on the background areas, whichmight be potentially redundant.
Revised: However, the existing 3D sparse CNNs are applied uniformly to the whole scene, causing a considerable amount of computation on the background areas, which might be potentially redundant.

Original: Intuitively, if such areas can be identified and selectively skipped,the computational costs have the potential to be dramatically reduced without deteriorating theperformance.
Revised: Intuitively, if such areas can be identified and selectively skipped, the computational costs have the potential to be dramatically reduced without deteriorating the performance.

Original: Besides the redundancy of the data, the model design itself also brings redundancy.
Revised: Besides the redundancy of the data, the model design itself also brings redundancy.

Original: To avoidaggressive down-sampling, 3D sparse CNNsalso adopt a dilation-like design when applied to 
Revised: To avoid aggressive down-sampling, 3D sparse CNNs adopt a dilation-like design when applied to perform the

Original: Stage
Revised: Stage

Original: Stage Stage
Revised: Stage Stage

Original: Stage×
Revised: Stage×

Original: Foreground Background (a) 1.
Revised: Foreground Background (a) 1.

Original: Stage
Revised: Stage

Original: Stage Stage
Revised: Stage Stage

Original: Stage×
Revised: Stage×

Original: Sparse CNN SP Sparse CNN(
Revised: Sparse CNN SP Sparse CNN

Original: [Figure]
Revised: [Figure]

Original: perform the down-sampling operation.
Revised: down-sampling operation.

Original: Specifically, it will compute features for adjacent empty voxels as shown in Fig.
Revised: Specifically, it will compute features for adjacent empty voxels as shown in Fig.

Original: 2 (b): the yellow voxels which are initially empty will be assigned withcomputed features.
Revised: 2 (b): the green voxels which are initially empty will be assigned with computed features.

Original: Consequently, after convolutional (stride > 1) down-sampling, the number ofnon-empty voxels might be increased rather than decreased as shown in Fig.
Revised: Consequently, after convolutional (stride > 1) down-sampling, the number of non-empty voxels might be increased rather than decreased as shown in Fig.

Original: 1 (b): the number of nonempty voxels even doubled (see stage 2) compared to the input (see stage1).
Revised: 1 (b): the number of non-empty voxels even doubled (see stage 2) compared to the input (see stage1).

Original: This will undoubtedlyincrease unnecessary computational costs for follow-up stages.
Revised: This will undoubtedly increase unnecessary computational costs for follow-up stages.

Original: The above observations prompt us to ask whether there is a way to identify redundant computationsthat can be pruned to improve efficiency.
Revised: The above observations prompt us to ask whether there is a way to identify redundant computations that can be pruned to improve efficiency.

Original: A solution that has been attempted in the 2D imagedomain [28, 11] is to add an additional learnable module to predict a soft mask [21, 27] that locatesareas to be skipped for computational efficiency.
Revised: A solution that has been attempted in the 2D image domain [29, 11] is to add an auxiliary learnable module to predict a soft mask [21, 28] that locates areas to be skipped for computational efficiency.

Original: The module often requires additional post-finetuning, auxiliary costs for integration, and incurs non-negligible computational overheads.
Revised: The module often requires additional post-finetuning, auxiliary costs for integration, and incurs non-negligible computational overheads.

Original: With the aforementioned considerations, we propose a new simple and efficient sparse convolutionoperator named Spatial Pruned Sparse Convolution (SPS-Conv), to alleviate redundancies caused bydata and model design.
Revised: With the aforementioned considerations, we propose a new simple and efficient sparse convolution operator named Spatial Pruned Sparse Convolution (SPS-Conv), to alleviate redundancies caused by data and model design.

Original: The core idea of SPS-Conv is to find redundancy in the model dynamically.
Revised: The core idea of SPS-Conv is to find redundancy in the model dynamically.

Original: Toavoid the overheads of learning-based methods for simplicity and efficiency, we investigate whetherthe features from the model itself contain useful cues that can be leveraged to identify redundancies.
Revised: To avoid the overheads of learning-based methods for simplicity and efficiency, we investigate whether the features from the model itself contain useful cues that can be leveraged to identify redundancies.

Original: Fortunately, we find that the magnitude of features could be a robust signal to reflect the importancewhere locations with smaller magnitude are more likely to be redundant areas.
Revised: Fortunately, we find that the magnitude of features could be a robust signal to reflect the importance where locations with smaller magnitudes are more likely to be redundant areas.

Original: This leads us toa magnitude-based sampling module which is incorporated into 3D convolution layers to reduceredundancy from data and model.
Revised: This leads us to a magnitude-based sampling module which is incorporated into 3D convolution layers to reduce redundancy in data and model.

Original: Specifically, to address data redundancy, the submanifold variant of SPS-Conv, termed as spatialpruned submanifold sparse convolution (SPSS-Conv) can adaptively calculate important positions inthe light of the magnitude of features and perform convolutions on these locations, leaving featuresof redundant locations unchanged.
Revised: Specifically, to address data redundancy, the submanifold variant of SPS-Conv, termed as spatial pruned submanifold sparse convolution (SPSS-Conv) can adaptively calculate important positions in the light of the magnitude of features and perform convolutions on these locations, leaving features of redundant locations unchanged.

Original: As for model redundancy in the down-sampling process, anothervariant named spatial pruned regular sparse convolution (SPRS-Conv), which can dynamicallydetermine the position that needs to be inflated.
Revised: As for model redundancy in the down-sampling process, another variant named spatial pruned regular sparse convolution (SPRS-Conv), which can dynamically determine the position that needs to be inflated.

Original: As Tab. 1(b) shows, our method effectively reducesthe meaningless expansion caused by convolution (around 50% in stage2).
Revised: As Tab. 1(b) shows, our method effectively reduces the meaningless expansion caused by convolution (around 50% in stage2).

Original: In general, SPS-Convonly keeps the essence and discards the irrelevant ones for pursuing a higher efficiency withoutcompromising performance.
Revised: In general, SPS-Conv only keeps the essence and discards the irrelevant ones for pursuing a higher efficiency without compromising performance.

Original: In summary, we propose an efficient convolution operator for spatial redundancy pruning, whichcan be easily incorporated into existing sparse 3D CNNs for 3D detection.
Revised: In summary, we propose an efficient convolution operator for spatial redundancy pruning, which can be easily incorporated into existing sparse 3D CNNs for 3D detection.

Original: We conduct extensiveexperiments on both KITTI and nuScenes dataset.
Revised: We conduct extensive experiments on both KITTI, Waymo, and nuScenes datasets.

Original: The result shows that we can achieve comparableperformance with SOTA methods while enjoying 52.4% and 46.5% GFLOPs reduction.
Revised: The result shows that we can achieve comparable performance with SOTA methods while enjoying 52.4%, 62.46%, and 46.5% GFLOPs reduction.

Original:  Related Work
Revised:  Related Work

Original: Sparse Convolution.
Revised: Sparse Convolution.

Original: Depending on how it is used, sparse convolution[15] can be subdividedinto regular sparse convolution and submanifold sparse convolution.
Revised: Depending on how it is used, sparse convolution[15] can be subdivided into regular sparse convolution and submanifold sparse convolution.

Original: Regular sparse convolution isoften used for the down-sample layer, which dilates all input features to its kernel-size neighbors,
Revised: Regular sparse convolution is

Original: [Figure]
Revised: [Figure]

Original:  sacrificing computational efficiency in exchange for a wider interaction of information.
Revised: often used for the down-sample layer, which dilates all input features to its kernel-size neighbors, sacrificing computational efficiency in exchange for a wider interaction of information.

Original: In contrast,the submanifold convolution is more like a simplified version of the regular sparse convolution,frequently used in residual blocks[17], only calculates on valid positions, avoiding meaninglessexpansion and achieving efficient computation.
Revised: In contrast, the submanifold convolution is more like a simplified version of the regular sparse convolution, frequently used in residual blocks[17], only calculates on valid positions, avoiding meaningless expansion and achieving efficient computation.

Original: Voxel-based Detectors.
Revised: Voxel-based Detectors.

Original: In order to process irregular point cloud data, voxel-based methods firstconvert the point cloud into regular voxel, and then use mature CNNs for feature extraction.
Revised: In order to process irregular point cloud data, voxel-based methods first convert the point cloud into regular voxel, and then use mature CNNs for feature extraction.

Original: However,the computational cost and memory requirement both increase cubically along with the voxelresolution.
Revised: However, the computational cost and memory requirement both increase cubically along with the voxel resolution.

Original: Thus, it is infeasible to train a voxel-based model with high-resolution inputs.
Revised: Thus, it is infeasible to train a voxel-based model with high-resolution inputs.

Original: Benjaminet al.[15] proposed a novel convolution operator named submanifold sparse convolution, by reducingcalculation on invalid position,  greatly improves the computing efficiency and memory.
Revised: Benjamin et al.[15] proposed a novel convolution operator named submanifold sparse convolution, by reducing calculation on invalid position, which greatly improves the computing efficiency and memory.

Original: Approaches for voxel-based detection methods can be grouped into two categories, i.e., singlestage[12, 16,33, 34, 38, 39] and two-stage detectors[10, 23, 25, 22, 24].
Revised: Approaches for voxel-based detection methods can be grouped into two categories, i.e., singlestage[12, 16, 34, 35, 39, 40] and two-stage detectors[10, 23, 25, 22, 24].

Original: Single-stage detectorsare relatively simple,  directly predict final bounding boxes based on the features extracted bysparse CNN.
Revised: Single-stage detectors are relatively simple, and directly predict final bounding boxes based on the features extracted by sparse CNN.

Original: VoxelNet[40] utilizes a Sparse CNN to extract voxel features from a dense grid.
Revised: VoxelNet[41] utilizes a Sparse CNN to extract voxel features from a dense grid.

Original: SECOND[30] proposes 3D sparse convolutions to efficiently extract voxel features.
Revised: SECOND[31] proposes 3D sparse convolutions to efficiently extract voxel features.

Original: HVNet[33]designs a convolutional network that attentively aggregate and project the multi-scale feature maps toachieve better performance.
Revised: HVNet[34] designs a convolutional network that attentively aggregates and projects the multi-scale feature maps to achieve better performance.

Original: In contrast, two-stage detectors are more complex, but can get higherperformance.
Revised: In contrast, two-stage detectors are more complex but can get higher performance.

Original: Part -A 2 [24] proposed a part-aware and aggregation module to exploit the intra-objectpart location.
Revised: Part -A 2 [24] proposed a part-aware and aggregation module to exploit the intra-object part location.

Original: PV-RCNN[23] uses keypoints to extract voxel features for better boxes refinement.
Revised: PV-RCNN[23] uses key points to extract voxel features for better box refinement.

Original: Dynamic kernel shape design.
Revised: Dynamic Kernel Shape Design.

Original: Many literature work on dynamic kernel design for adapting varioustasks.
Revised: Much literature work on dynamic kernel design for adapting various tasks.

Original: Deformable convolution [9] predicts offets for feature sampling.
Revised: Deformable convolution [9] predicts offsets for feature sampling.

Original: For 3D scene understanding,
Revised: For 3D scene understanding,

Original: KPConv [26] constructs dynamic graph for kernel points.
Revised: KPConv [27] constructs dynamic graph for kernel points.

Original: Deformable PV-RCNN [3] applies offsetprediction for feature sampling in 3D object detection.
Revised: Deformable PV-RCNN [3] applies offset prediction for feature sampling in 3D object detection.

Original: Focals Conv [8] learns a dynamic spatiallysparsity which enhances  network for spatial modeling capabilities,
Revised: Focals Conv [8] learns a dynamic spatially sparsity which enhances the network for spatial modeling capabilities,

Original:  Background and Motivation
Revised:  Background and Motivation

Original: In this section, we will introduce the mechanism of sparse convolution, then analyze its redundancyand introduce our motivation.
Revised: In this section, we will introduce the mechanism of sparse convolution, then analyze its redundancy and introduce our motivation.

Original: 3.1 Preliminary of Sparse Convolution
Revised: 3.1 Preliminary of Sparse Convolution

Original: To better illustrate sparse convolution, we first introduce its general expression.
Revised: To better illustrate sparse convolution, we first introduce its general expression.

Original: We denote x p as aninput feature with c in dimension at position p , w ∈ R K d × c in × c out is the weight of convolution kernel,where d and K d refers to the dimension of the spatial space and spatial size of the kernel respectively.
Revised: We denote x p as an input feature with c in dimension at position p , w ∈ R K d × c in × c out is the weight of convolution kernel, where d and K d refers to the dimension of the spatial space and spatial size of the kernel respectively.

Original: Thus we can formulate the convolution process as follows.
Revised: Thus we can formulate the convolution process as follows.

Original: [Equation]
Revised: [Equation]

Original: , where K d ( p, P in ) = (cid:8) k | p + k ∈ Pin , k ∈ K d (cid:9) .
Revised: p [Equation] P

Original: [Equation]
Revised: [Equation]

Original: Here, P in and P out refers to the input and output feature space, k is the kernel offset that correspondsto all the valid locations in kernel space K d , ¯ p k = p + k denotes all non-empty neighbor voxelsaround center p .
Revised: Here, P in and P out refer to the input and output feature space, k is the kernel offset that corresponds to all the valid locations in kernel space K d , ¯ p k = p + k denotes all non-empty neighbor voxels around center p .

Original: Due to the discrete data distribution in 3d space, we define K d ( p, P in ) as a subset of K d , leaving out the empty position, which is constrained by position p and input feature space P in .
Revised: Due to the discrete data distribution in 3d space, we define K d ( p, P in ) as a subset of K d , leaving out the empty position, which is constrained by position p and input feature space P in .

Original: There are two types of sparse convolutions, namely, regular sparse convolution [14] and submanifoldsparse convolution [15], their major difference lies in the output position of the convolution.
Revised: There are two types of sparse convolutions, namely, regular sparse convolution [14] and submanifold sparse convolution [15], their major difference lies in the output position of the convolution.

Original: Forregular sparse convolution, as shown in Fig. 2, the position related to the input will be activatedduring the convolution process.
Revised: For regular sparse convolution, as shown in Fig. 2, the position related to the input will be activated during the convolution process.

Original: Specifically, these position will be assigned value and combined withthe P in to form a new P out .
Revised: Specifically, these positions are assigned value and combined with the P in to form a new P out .

Original: This process can be easily formulated as
Revised: This process can be easily formulated as

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where Q (cid:0)  p 
Revised: Q (cid:0) p, K d (cid:1) = (cid:8) p [Equation] k

Original: [Equation]
Revised: [Equation]

Original:  where P conv is a subset of P in , representing the position where the operation needs to be performed.
Revised: k [Equation] where P conv is a subset of P in , representing the position where the convolution needs to be performed.

Original: Q (cid:0) p, K d (cid:1) refers to the position related to P conv which is restricted by kernel size.
Revised: Q (cid:0) p, K d (cid:1) refers to the position related to P conv which is restricted by kernel size.

Original: Regular sparse convolution can effectively expand the receptive field like 2D convolution which isbeneficial for 3D sparse data.
Revised: Regular sparse convolution can effectively expand the receptive field like 2D convolution, which is beneficial for 3D sparse data.

Original: However, it also brings  computation burden due to the generation oftoo many active locations, which can lead to a decrease in speed in subsequent convolution layersdue to the large number of active points.
Revised: However, it also brings a computation burden due to the generation of too many active locations, which can lead to a decrease in speed in subsequent convolution layers due to a large number of active points.

Original: In order to achieve a better trade-off between efficiency andreceptive field, regular sparse convolution is often adopted in the down-sample layer in sparse CNNs.
Revised: In order to achieve a better trade-off between efficiency and receptive field, regular sparse convolution is often adopted in the down-sample layer in sparse CNNs.

Original: In contrast, submanifold sparse convolution[15] restricts an output location to be active if and onlyif the corresponding input location is active, which avoid the dilation in regular sparse convolution,although the receptive field is limited.
Revised: In contrast, submanifold sparse convolution[15] restricts an output location to be active if and only if the corresponding input location is active, which avoids the dilation in regular sparse convolution, although the receptive field is limited.

Original: With a simple modification on Eq. 2, P in =
Revised: With a simple modification on Eq. 2, P out =

Original: P out , regular modecan be convert to submanifold mode.
Revised: P in , regular mode can be convert to submanifold mode.

Original: 3.2 Redundancy Analysis and Motivations
Revised: 3.2 Redundancy Analysis and Motivations

Original: Despite the wide adoption of sparse CNNs in 3D object detection due to their generality and efficiency,spatial redundancy caused by sparse CNNs for this task is still under-explored.
Revised: Despite the wide adoption of sparse CNNs in 3D object detection due to their generality and efficiency, spatial redundancy caused by sparse CNNs for this task is still under-explored.

Original: In Fig.

Original: 5, we can findthat the most common component of sparse CNN is the submanifold convolution.

Original: Considering thelarge number of task-independent background points in the 3D scene, a lot of redundant computationwill be generated in each stage.
Revised: Considering the large number of task-independent background points in the 3D scene, a lot of redundant computation will be generated in each stage.

Original: In addition, due to the expansion of regular convolution, thebackground points will become denser, bringing more computational burden to the following stages.
Revised: In addition, due to the expansion of regular convolution, the background points will become denser, bringing more computational burden to the following stages.

Original: Therefore, in response to these problems, we have targeted the improvement of the sparse convolutionoperator.
Revised: Therefore, in response to these problems, we have targeted the improvement of the sparse convolution operator.

Original: Details are shown in Sec.
Revised: Details are shown in Sec.

Original:  Spatial Pruned  ConvolutionNetwork
Revised:  Spatial Pruned Sparse Convolution

Original: In this section, we propose a new efficient sparse convolution operator, named spatial pruned sparseconvolution (SPS-Conv).
Revised: In this section, we propose a new efficient sparse convolution operator, named spatial pruned sparse convolution (SPS-Conv).

Original: Intending to effectively bring down the unnecessary computation costscaused by the 3D spatial redundancy, two variants of SPS-Conv are investigated, i.e., spatial prunedsubmanifold sparse convolution (SPSS-Conv) and spatial pruned regular sparse convolution (SPRSConv), and both of them are based on the same idea that the redundancy should be dynamicallyameliorated in 3D space according to the specific contexts of different individuals.
Revised: Intending to effectively bring down the unnecessary computation costs caused by the 3D spatial redundancy, two variants of SPS-Conv are investigated, i.e., spatial pruned submanifold sparse convolution (SPSS-Conv) and spatial pruned regular sparse convolution (SPRSConv), and both of them are based on the same idea that the redundancy should be dynamically ameliorated in 3D space according to the specific contexts of different individuals.

Original: In the following,the magnitude-based sampling strategy is presented in Sec. 4.1, followed by the introduction of
Revised: In the following, the magnitude-based sampling strategy is presented in Sec. 4.1, followed by the introduction of

Original: SPSS-Conv and SPRS-Conv in Sec.
Revised: SPSS-Conv and SPRS-Conv in Sec.

Original: 4.2 and Sec. 4.3 respectively.
Revised: 4.2 and Sec. 4.3 respectively.

Original: Then, Sec. 4.4 manifests thegeneralization ability by showing how our method is applied to generic sparse CNNs.
Revised: Then, Sec. 4.4 manifests the generalization ability by showing how our method is applied to generic sparse CNNs.

Original: 4.1 Magnitude-guided spatial sampling
Revised: 4.1 Magnitude-guided Spatial Sampling

Original: In previous works [28, 11, 5], sampling module is often designed in a learnable manner, i.e. , aconvolution layer is trained to yield soft masks for sampling, while, at the time of inference, the softmasks are converted to hard ones where elements with lower responses will be simply discarded.
Revised: In previous works [29, 11, 5], sampling module is often designed in a learnable manner, i.e. , a convolution layer is trained to yield soft masks for sampling, while, at the time of inference, the soft masks are converted to hard ones where elements with lower responses will be simply discarded.

Original: Although these methods have been empirically shown effective in 2D tasks, the following issues stillexist: 1) Additional supervision is required on the prediction branch, otherwise it is easy to fall intotrivial solutions, i.e. , the values in the soft mask are all close to 1. 2)
Revised: Although these methods have been empirically shown effective in 2D tasks, the following issues still exist: 1) Additional supervision is required on the prediction branch, otherwise it is easy to fall into trivial solutions, i.e. , the values in the soft mask are all close to 1. 2)

Original: The proportion of pruning is[Figure] hard to be fixed, and it is instead automatically determined throughout the learning process, makingit hard to satisfy the specific requirement with certain limited computation budget.
Revised: The proportion of pruning is hard to be fixed, and it is instead automatically determined throughout the learning process, making it hard to satisfy the specific requirement with certain limited computation budget.

Original: 3) Two-stagefine-tuning is required to compensate for the performance loss of converting from soft mask to hardmask, consuming additional time and resources.
Revised: 3) Two-stage [Figure] fine-tuning is required to compensate for the performance loss of converting from soft mask to hard mask, consuming additional time and resources.

Original: Recent literature has shown that the attention mechanism is able to reveal what the model really caresabout.
Revised: Recent literature has shown that the attention mechanism is able to reveal what the model cares about.

Original: Following [37, 32], the feature magnitude is adopted for yielding the model’s “attention” tohighlight the informative elements.
Revised: Following [38, 33], the feature magnitude is adopted for yielding the model’s “attention” to highlight the informative elements.

Original: For the purpose of getting the magnitude-based attention mask,we first calculate the  absolute mean values on different channels, and then the sigmoid function isapplied to get the normalized output, which can be formulated as[Equation] where
Revised: For the purpose of getting the magnitude-based attention mask, we first calculate the channel-wise absolute mean values on different voxels, and then the sigmoid function is applied to get the normalized output, which can be formulated as

Original: [Equation]
Revised: [Equation]

Original: where F and C denote the input feature and its channel.
Revised: where F and C denote the input feature and its channel.

Original: F c is the feature of the c -th dimension.
Revised: F c is the feature of the c -th dimension.

Original: Gand M refer to the spatial magnitude map and magnitude mask.
Revised: G and M refer to the spatial magnitude map and magnitude mask.

Original: 4.2 Spatial Pruned Submanifold Sparse Convolution
Revised: 4.2 Spatial Pruned Submanifold Sparse Convolution

Original: Considering the fact that the background regions often take the majority in 3D scenes, directlyperforming the submanifold convolution everywhere of the input volume inevitably involves substantial unnecessary calculations, leading to computational redundancy.
Revised: Considering the fact that the background regions often take the majority in 3D scenes, directly performing the submanifold convolution everywhere of the input volume inevitably involves substantial unnecessary calculations, leading to computational redundancy.

Original: Therefore, with a focus onalleviating this issue, we propose the spatial pruned submanifold sparse convolution (SPSS-Conv)that dynamically examines the regions of interest.
Revised: Therefore, with a focus on alleviating this issue, we propose the spatial pruned submanifold sparse convolution (SPSS-Conv) that dynamically examines the regions of interest.

Original: Specifically, with the guidance of the magnitude-based attention mask, we separate the elements P ininto two disjoint subsets, i.e., the important set P im and the unimportant set P nim , where P im ∪ P uim =
Revised: Specifically, with the guidance of the magnitude-based attention mask, we separate the elements P in into two disjoint subsets, i.e., the important set P im and the unimportant set P nim , where P im ∪ P uim =

Original: P in .
Revised: P in .

Original: We note that various methods can be incorporated for accomplishing the division, such as using afixed threshold or simply taking the elements with top-k scores.
Revised: We note that various methods can be incorporated for accomplishing the division, such as using a fixed threshold or simply taking the elements with top-k scores.

Original: Then we re-weight the input featuremap by multiplying it with the magnitude mask, applying submanifold convolution on importantpositions P im according to Eq. 1, and concatenate it with the features of unimportant positions P nimas a residual, as shown in Fig.
Revised: Then we re-weight the input feature map by multiplying it with the magnitude mask, applying submanifold convolution on important positions P im according to Eq. 1, and concatenate it with the features of unimportant positions P nim as a residual, as shown in Fig.

Original: To this end, the reconstructed output feature can be written as:
Revised: To this end, the reconstructed output feature can be written as:

Original: [Equation]
Revised: [Equation]

Original: Since P im is sparser than P in , convolution enables task-beneficial locations to be highlighted.
Revised: Since P im is sparser than P in , convolution enables task-beneficial locations to be highlighted.

Original: In contrast, those unimportant positions have relatively small activation values, thus the skip connection cannot only reduce the computational overhead but also suppress the diffusion of redundant information.
Revised: In contrast, those unimportant positions have relatively small activation values, thus the skip connection can not only reduce the computational overhead but also suppress the diffusion of redundant information.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 4.3 Spatial Pruned Regular Sparse Convolution
Revised: 4.3 Spatial Pruned Regular Sparse Convolution

Original: The “dilation” effect of the regular sparse convolution inevitably causes a large number of adjacentpositions to be activated, which brings a greater computational burden to subsequent layers.
Revised: The “dilation” effect of the regular sparse convolution inevitably causes a large number of adjacent positions to be activated, which brings a greater computational burden to subsequent layers.

Original: Recentwork [8] shows that the spatially dynamic sparsity in sparse convolution is essential for sophisticated3D object detection.
Revised: Recent work [8] shows that the spatially dynamic sparsity in sparse convolution is essential for sophisticated 3D object detection.

Original: Motivated by this, SPRS-Conv is designed as a magnitude-based dynamicprediction down-sampling module, which can effectively suppress the amplification effect of spatialredundancy.
Revised: Motivated by this, SPRS-Conv is designed as a magnitude-based dynamic prediction down-sampling module, which can effectively suppress the amplification effect of spatial redundancy.

Original: Similar to SPSS-Conv, P in is first divided into two disjoint subsets, i.e , P im and P nim .
Revised: Similar to SPSS-Conv, P in is first divided into two disjoint subsets, i.e , P im and P nim .

Original: Unlike theregular sparse convolution that dilates all possible neighborhood locations, leading to sub-optimalefficiency, we instead dynamically choose whether to expand for each position based on the magnitudemask yielded by Eq. 3.
Revised: Unlike the regular sparse convolution that dilates all possible neighborhood locations, leading to sub-optimal efficiency, we instead dynamically choose whether to expand for each position based on the magnitude mask yielded by Eq. 3.

Original: Further, we define P dilate , which is a set composed of all P im positions and thepositions within their kernel range.
Revised: Further, we define P dilate , which is a set composed of all P im positions and the positions within their kernel range.

Original: The equation can be written as:
Revised: The equation can be written as:

Original: [Equation]
Revised: [Equation]

Original: Considering theproblem of stride during down-sampling, we designed a stride mask E ( P ) to ignorethose that should be skipped by the stride, which can be formulated as below:
Revised: Considering the stride during down-sampling, we designed a stride mask E ( P ) to ignore those that should be skipped by the stride, which can be formulated as below:

Original: [Equation]
Revised: [Equation]

Original: where s  refers to the stride of convolution, % denotes mod operation.
Revised: where s i refers to the stride of the convolution on a specific dimension, % denotes mod operation.

Original: With E ( P ) , we can furthermodify Eq.
Revised: With E ( P ) , we can further modify Eq.

Original: 2 as:
Revised: 2 as:

Original: [Equation]
Revised: [Equation]

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, we operate at the specified positions according to Eq.
Revised: 4, we operate at the specified positions according to Eq.

Original: 1 to get the outputfeature map.
Revised: 1 to get the output feature map.

Original: Then we down-sample it to the correct size according to the stride.
Revised: Then we down-sample it to the correct size according to the stride.

Original: Compared to theregular sparse convolution, our method dynamically prunes P out conditioned on its specific content,effectively suppressing meaningless computations without compromising the performance.
Revised: Compared to the regular sparse convolution, our method dynamically prunes P out conditioned on its specific content. As

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Revised: shown in Fig. 1(b), SPRS-Conv greatly reduces the number of voxels in each stage, especially stage (about 50%), thus reducing the computational burden caused by background points

Original: 4.4 Spatial Pruned Convolution Network
Revised: 4.4 Spatial Pruned Convolution Network

Original: Our method is model-agnostic thus it can be easily plugged into any existing sparse CNNs.
Revised: Our method is model-agnostic thus it can be easily plugged into any existing sparse CNNs.

Original: As shownin Fig 5, a regular frame of sparse CNN is composed of a stem layer and four stages, each of whichcontains a down-sampling layer and two submanifold sparse convolution blocks.
Revised: A regular frame of sparse CNN is composed of a stem layer and four stages, each of which contains a downsampling layer and two submanifold sparse convolution blocks.

Original: To demonstrate theeffectiveness and generalization ability, we replace all regular sparse convolutions and submanifoldsparse convolutions in sparse CNN, except those in the stem layer, with our proposed SPRS-Convand SPSS-Conv, respectively.
Revised: To demonstrate the effectiveness and generalization ability, we replace all regular sparse convolutions and submanifold sparse convolutions in sparse CNN, except those in the stem layer, with our proposed SPRS-Conv and SPSS-Conv, respectively.

Original: Detailed experimental results and analysis are presented in Sec. 5.
Revised: Detailed experimental results and analysis are presented in Sec. 5.

Original:  Experiments5.1 Experimental Setting
Revised:  Experiments 5.1 Experimental Setting

Original: 3D Object Detection Datasets .
Revised: 3D Object Detection Datasets .

Original: We evaluate our method on two challenging benchmarks KITTI[13]and nuScenes[4].
Revised: We evaluate our method on three challenging benchmarks KITTI [13], Waymo [26] and nuScenes [4].

Original: KITTI dataset consists of 7,481 samples and 7,518 testing samples, where thetraining samples are generally divided into the train split (3, 712 samples) and the val split (3, 769samples).
Revised: KITTI dataset consists of 7,481 samples and 7,518 testing samples, where the training samples are generally divided into the train split (3,712 samples) and the val split (3,769 samples). Waymo dataset contains 1,000 sequences in total, with 798 for training andfor validation. As the Waymo dataset is really large-scale, we use 1 5 data for training. Results in

Revised: Waymo are evaluated in difficulty LEVEL_1 (L1) and LEVEL_2 (L2) objects.

Original: nuScenes dataset is a large-scale autonomous driving dataset, which contains 1,000 drivingsequences in total.
Revised: nuScenes dataset is a large-scale autonomous driving dataset, which contains 1,000 driving sequences in total.

Original: It is split into 700 scenes for training, 150 scenes for validation, and 150 scenesfor testing.
Revised: It is split into 700 scenes for training, 150 scenes for validation, and 150 scenes for testing.

Original: It is collected using a 32-beam synced LIDAR and 6 cameras with the complete 360 oenvironment coverage.
Revised: It is collected using a 32-beam synced LIDAR and 6 cameras with the complete 360 o environment coverage.

Original: Implementation Detail .
Revised: Model Configurations .

Original: We conduct experiments on different detection frameworks, i.e., VoxelRCNN [10], PV-RCNN [23] and SECOND [30] for KITTI, CenterPoint [36] for nuScenes. We replacethe submanifold convolution and regular convolution in sparse CNNs with SPSS-Conv and SPRS-
Revised: i.e., VoxelR-CNN [10], PV-RCNN [23] and SECOND [31] for KITTI, CenterPoint [37] for nuScenes and Waymo. We replace the submanifold convolution and regular convolution in sparse CNNs with SPSS-Conv and SPRS-Conv, respectively, other experimental hyperparameters flowing the default settings of the baseline methods [10, 23, 31, 37, 31].

Original: [Table]
Revised: Training .

Original: [Table] [Table]
Revised: Unlike existing 2D approaches [29, 11, 5] that design the sampling module in a learnable manner which needs a two-stage fine-tune. We train the model in an end-to-end manner.

Original: [Table]
Revised: Our

Original: Conv, respectively, other experimental hyperparameters flowing the default settings of the baselinemethods [10, 23, 30, 36, 30].
Revised: SPS-Conv is purely embedded into sparse CNNs without any modifications on network architectures or parameters ( i.e., feature dimensions). Our modules contain one hyperparameter, i.e., pruning ratio and we choose to use top-k to divide the pruning part.

Original: For KITTI, we set pruning ratio in SPSS-Conv and SPRS-Conv as 0.5and 0.5 respectively, as for nuscenes they are set  as  0.3 and 0.5.
Revised: For spatial pruned submanifold sparse convolution (SPSS-Conv) it refers to the proportion of positions in each stage that can be ignored for calculation, which can be symbolized as { r s 0 , r s 1 , r s 2 , r s 3 }. We set it as {0.3, 0.3, 0.3, 0.3 } [Table] [Table]

Original: More implementation details aregiven in the supplementary file.
Revised: [Table] for nuScenes and {0.5, 0.5, 0.5, 0.5} for Waymo and KITTI. As for spatial pruned regular sparse convolution (SPRS-Conv), the pruning ratio is used for controlling the amount that needs to be inflated when downsampling. We symbolized it as { r d 1 , r d 2 , r d 2 }, which are set as {0.5, 0.5, 0.5} in nuScenes and Waymo and {0.7, 0.5, 0.3} in KITTI.

Original: 5.2 Main Results
Revised: 5.2 Main Results

Original: nuScenes  For the sake of fairness, we choose LIDAR-only methods for comparison, model ensembling and additional augmentation are not included during inference.
Revised: nuScenes . For the sake of fairness, we choose LIDAR-only methods for comparison, model ensembling and additional augmentation are not included during inference.

Original: Our method achieves SOTAperformance on both mAP and NDS on nuScenes test split, in Tab. 1.
Revised: Our method achieves competitive performance on both mAP and NDS on nuScenes test split, in Tab. 1.

Original: Besides, We apply the methodto the base model for a more detailed comparison, as shown in Tab. 2, our method can help themodel to preserve the original performance while skipping redundant computation, specifically, SPCenterPoint’s GFLOPs are only 54.5% of the base model.
Revised: Besides, We apply the method to the base model for a more detailed comparison, as shown in Tab. 2, our method can help the model to preserve the original performance while skipping redundant computation, specifically, SP-CenterPoint’s GFLOPs are only 54.5% of the base model.

Original: It is worth noting that although our methodremoves and ignores redundant locations in 3D space, the mAP does not decrease significantly onsmall object categories, which indicates that the network can learn the importance of spatial locationsand distinguish them accordingly.

Original: KITTI .
Revised: KITTI .

Original: We make comparisons with recent state-of-the-art methods on KITTI val split.
Revised: We make comparisons with recent state-of-the-art methods on KITTI val split.

Original: As shownin Tab. 3, Our Spatial Pruned Voxel R-CNN achieves competitive results with other methods.
Revised: As shown in Tab. 4, Our Spatial Pruned Voxel R-CNN achieves competitive results with other methods.

Original: Tobetter demonstrate the effectiveness of our method, we validate our method on popular voxel-baseddetectors, i.e., Second, PVRCNN, and Voxel-RCNN.
Revised: To better demonstrate the effectiveness of our method, we validate our method on popular voxel-based detectors, i.e., Second, PVRCNN, and Voxel-RCNN.

Revised: Tab. 5 shows that with our method, the GFLOPs of the sparse CNN are reduced by more than 50%, and the matching performance can still be obtained on KITTI val split in AP 3D in recall 11 positions.

Revised: Waymo .

Revised: We present SPS-Conv based on the CenterPoint on the outdoor dataset Waymo [26] in

Original: Tab. 
Revised: Tab. 3.

Original: 4 shows that with our method, the GFLOPsof the sparse CNN are reduced by more than 50%, and the matching performance can still be obtainedon KITTI val split in AP 3D in recall 11 positions.
Revised: The results demonstrate the generality of SPS-Conv on large-scale datasets with dense point clouds: SPS-conv still maintains the high performance while significantly reduces FLOPs. [Table]

Original: 5.3 Ablation Study
Revised: [Table] Method mAP [Table] Bar

Original: SPSS-Conv pruning ratio .
Revised: [Table] [Table] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original:  The pruning ratio  of SPSS-Conv is used to control the proportion ofunimportant positions selected in each stage, the higher the proportion, the fewer positions areinvolved in the calculation.
Revised: [Figure] 5.3 Ablation Study SPS-Conv Pruning Ratio . The pruning ratio (PR.) of the SPS-Conv is used to control the proportion of unimportant positions selected in each block, the higher the proportion, the fewer positions are involved in the calculation.

Original: We evaluate on nuScenes val split upon CenterPoint.
Revised: We evaluate on nuScenes val split upon CenterPoint.

Original: As shown inTab.
Revised: As shown in

Original: 7with the pruning ratio increases, there is a relatively obvious and proportional decrease in GFLOPs, incontrast, the performance drop is not so severe, except when the pruning ratio increases to 0.9.
Revised: Tab. 6 and Tab. 7, there is a relatively obvious and proportional decrease in GFLOPs, in contrast, the performance drop is not so severe, except when the pruning ratio increases to 0.9.

Original: Thisalso reveals that 3D Scenes contain spatial redundancy, and when we selectively skip these redundantpositions, it will not affect the performance.[Table]
Revised: This also reveals that 3D Scenes contain spatial redundancy, and when we selectively skip these redundant positions, it will not affect the performance.

Original: [Table]
Revised: [Table]

Original: SPRS-Conv pruning ratio .

Original: Due to the expansion of the convolution, some non-redundant positionsin the 3d sparse CNNs are expanded, increasing the computational burden.

Original: Therefore, we explorethe influence of the pruning ratio in our designed SPRS-Conv on this problem.

Original: We tried differentcombinations of pruning ratios when down-sampling between different stages.

Original: Notably, we foundthat the performance of the network is not very sensitive to the pruning rate.

Original: Tab. 8 shows that even if70% of the position dilation is suppressed during down-sampling, the impact on performance is notfatal.

Original: Sampling and interpolation strategy in SPSS-Conv .
Revised: Sampling and Interpolation Strategy in SPSS-Conv .

Original: In order to verify the effectiveness of ourmethod, we conduct a combination of several sampling and interpolation methods, where the pruningratio is limited to 0.5.
Revised: In order to verify the effectiveness of our method, we conduct a combination of several sampling and interpolation methods, where the pruning ratio is limited to 0.5.

Original: Results are shown in Tab. 5, when we compare the sampling strategy, it is notdifficult to find that the learnable methods consume more computation than the magnitude methodwith similar or worse accuracy. It makes us think that magnitude is another manifestation of networkattention, and regions with high magnitude represent high attention, so we select important regionsaccording to this logic, which conforms to the inherent performance of the network.
Revised: Results are shown in Tab. 9, it is not difficult to find that the learnable method and magnitude method have comparable results which makes us think that magnitude is another manifestation of network attention, so we select important regions according to this logic, which conforms to the inherent performance of the network.

Original: In addition, forthe interpolation operation, compared to the average pooling operation, a simple skip connectioncan play a good role.
Revised: In addition, for the interpolation operation, compared to the average pooling operation, a simple skip connection can play a good role.

Original: A possible explanation is that these predicted unimportant regions are nottask-essential, so even using the original values would not have much impact on the results
Revised: A possible explanation is that these predicted unimportant regions are not task-essential, so even using the original values would not have much impact on the results.

Original: Importance of position with high magnitude .
Revised: Importance of Position with High Magnitude .

Original: SPSS-Conv dynamically selects important positionsin the space to perform convolution operations based on magnitude. To verify that the location chosenby this mechanism is task-friendly, we exchanged the calculation method of positions with highmagnitude and other positions, that is, convolution for low magnitude positions, and skip connectionfor low magnitude positions.
Revised: To verify that the location chosen by this mechanism is task-friendly, we exchanged the calculation method of positions with high magnitude and other positions. As shown in Tab. 8, for SPSS-Conv, it can be seen that when we reverse the important position, there is considerable performance drop in all categories. This phenomenon is more obvious in small objects, especially for the traffic cone and bicycle categories (even reaching about a 7 % performance drop).

Original: As shown in Tab. 6, a dramatic performance drop occurs when we onlytake positions with low activation into consideration.
Revised: Large object categories (such as vehicles) with more points are less sensitive to the sampling method but still have a notable performance drop. Compared with the experimental results of SPSS-Conv, the inversion experiment of SPRS has a more exaggerated performance loss. When inversion is not performed, SPRS will suppress the irrelevant features with low magnitude which reduced in the downsample part, showing that there is no obvious performance loss in the result. However, when we choose to suppress positions with large magnitudes, the spatial redundancy is amplified, and important features cannot be effectively expanded, and even discarded due to downsampling.

Original: This further proves that the value of featuremagnitude is positively correlated with importance.
Revised: After the above features are converted to Bird’s Eye View (BEV), since the number of foreground points from the input is weakened, the effective features extracted by BEVbackone are very limited, resulting in a great degree of performance degradation. Magnitude-base Pruning vs Random Drop . Compared to magnitude-based pruning, we observe using random drop as an indicator will lead to a certain loss in performance (around 2%), shown in Tab. 8.

Revised: This is caused by the randomness, part of the foreground is discarded, resulting in performance degradation.

Revised: However, the important part still has a high probability of being selected, which also guarantees performance to a certain extent.

Revised: Despite its degraded performance, the random drop method also has a certain degree of randomness.

Revised: This is not desirable in practical applications as it may have the chance to lose some safety-critical areas which will cause problems in safety-critical applications.

Revised: 5.4 Visualization

Revised: Visual Analysis .

Original: Concluding Remarks
Revised: To better understand points pruned by our magnitude criterion, we visualize point clouds before and after pruning. The comparison results are shown in Fig. 6(a), the original images and the pruned images are provided respectively. We observe that most of the foreground points are preserved. For the background areas, points that fall in vertical structures, such as light, poles, and trees, are also preserved as they tend to be hard negatives, and easily confused with foreground objects.

Revised: These points require a deep neural network with a certain capability to process in order to recognize them as background.

Revised: In contrast, background points in flat structures such as road points are largely removed because they are easily identifiable redundant points.

Revised: Why foreground points with high feature magnitude?

Revised: We visualize magnitude in BEV for a clearer comparison, as shown in Fig.

Revised: 6(b), It is not difficult to see that the feature magnitude of most foreground points is relatively large (vehicles).

Revised: In contrast, the simple background points (ground) show in small magnitude, which are easier to be distinguished by the network.

Revised: To gain more insights into why high feature magnitude corresponds to the above patterns, we conjecture that this is caused by the training objective in 3D object detection.

Revised: When training a 3D object detection model, the focal loss is adopted as default in 3D object detection.

Revised: When we look closer at the focal loss, it will incur a loss on positive samples and hard negatives while easy negatives are removed from the loss.

Revised: Thus, this will generate gradients in the direction that can incur an update of features for areas with positive samples and hard negatives.

Revised: This can eventually make a difference in their feature magnitudes in comparison with areas for easy negatives which are less frequently considered in the optimization objective.

Original: In this paper, we investigate the spatial redundancy in sparse CNNs and propose a new simpleand efficient convolution operator SPSS-Conv with two variants, i.e., SPSS-ConV and SPRS-Conv.
Revised: In this paper, we investigate the spatial redundancy in sparse CNNs and propose a new simple and efficient convolution operator SPSS-Conv with two variants, i.e., SPSS-ConV and SPRS-Conv.

Original: Extensive experiments have proved the following conclusions: 1) Magnitude can serve as a goodindicator for sparse CNNs to dynamically identify the informative points inthe diverse scenes.
Revised: Extensive experiments have proved the following conclusions: 1) Magnitude can serve as a good indicator for sparse CNNs to dynamically identify the informative points in diverse scenes.

Original: Theoverwhelming background region in the 3D scene results in spatial redundancy, which can be prunedby our method without adversely affecting the performance.
Revised: The overwhelming background region in the 3D scene results in spatial redundancy, which can be pruned by our method without adversely affecting the performance.

Original: Selectively inflating the elements nearthe region of interest during the down-sampling process not only saves computation but also keepsthe necessary information intact.
Revised: Selectively inflating the elements near the region of interest during the down-sampling process not only saves computation but also keeps the necessary information intact.

Original: The proposed SPSS-Conv can be effortlessly applied to genericsparse CNNs without specific structural constraints.
Revised: The proposed SPSS-Conv can be effortlessly applied to generic sparse CNNs without specific structural constraints.

Original: We hope our work can provide new thoughts forinspiring future  the community.
Revised: We hope our work can provide new thoughts for inspiring future research in the community.

Revised:  Acknowledgement

Revised: This work has been supported by Hong Kong Research Grant Council - Early Career Scheme (Grant

Revised: No. 27209621), HKU Startup Fund, and HKU Seed Fund for Basic Research.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?[Yes]
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original:  (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] Code will be released upon acceptance.
Revised: [No] Code will be released upon acceptance.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?
Revised: g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e
Revised: [Yes] (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi- ments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/ curating contains personally identifiableinformation or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A
Revised: [N/A] 5.

Original: If  crowdsourcing or conducted research with human subjects a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A

Original:  (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A
Revised: [N/A]