Original: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds
Revised: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds

Original: (s)
Revised: (s)

Original: AffiliationAddress
Revised: AffiliationAddress

Original: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomousdriving.
Revised: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomousdriving.

Original: Large-scale outdoor scenes usually feature significant variance in instancescales, thus requiring features rich in long-range and fine-grained information tosupport accurate detection.
Revised: Large-scale outdoor scenes usually feature significant variance in instancescales, thus requiring features rich in long-range and fine-grained information tosupport accurate detection.

Original: Recent detectors leverage the power of window-basedtransformers to model long-range dependencies but tend to blur out fine-graineddetails.
Revised: Recent detectors leverage the power of window-basedtransformers to model long-range dependencies but tend to blur out fine-graineddetails.

Original: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.
Revised: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.

Original: Specifically, MsSVT explicitlydivides attention heads into multiple groups, each in charge of attending to information within a particular range.
Revised: Specifically, MsSVT explicitlydivides attention heads into multiple groups, each in charge of attending to information within a particular range.

Original: All groups’ output is merged to obtain the finalmixed-scale features.
Revised: All groups’ output is merged to obtain the finalmixed-scale features.

Original: Moreover, we provide a novel chessboard sampling strategyto reduce the computational complexity of applying a window-based transformer in3D voxel space.
Revised: Moreover, we provide a novel chessboard sampling strategyto reduce the computational complexity of applying a window-based transformer in3D voxel space.

Original: To improve efficiency, we also implement the voxel sampling andgathering operations sparsely with a hash map.
Revised: To improve efficiency, we also implement the voxel sampling andgathering operations sparsely with a hash map.

Original: Endowed by the powerful capabilityand high efficiency of modeling mixed-scale information, our single-stage detectorbuilt on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectorson Waymo.
Revised: Endowed by the powerful capabilityand high efficiency of modeling mixed-scale information, our single-stage detectorbuilt on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectorson Waymo.

Original: Code will soon be available.
Revised: Code will soon be available.

Original:  Introduction
Revised:  Introduction

Original: 3D object detection has received increasing attention due to its successful autonomous drivingapplications.
Revised: 3D object detection has received increasing attention due to its successful autonomous drivingapplications.

Original: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturallyirregular and unordered.
Revised: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturallyirregular and unordered.

Original: Hence directly applying CNN-like operations [11, 12] to them can bedifficult.
Revised: Hence directly applying CNN-like operations [11, 12] to them can bedifficult.

Original: To solve this, many researchers have rasterized point clouds into regular voxel grids [24]and employed 3D CNNs to extract high-dimensional voxel features.
Revised: To solve this, many researchers have rasterized point clouds into regular voxel grids [24]and employed 3D CNNs to extract high-dimensional voxel features.

Original: With the recent rise of visiontransformer (ViT) [42] on 2D images, some attempts have been made to generalize global or moreefficient window-based transformers to 3D voxels [23] or pillars [7].
Revised: With the recent rise of visiontransformer (ViT) [42] on 2D images, some attempts have been made to generalize global or moreefficient window-based transformers to 3D voxels [23] or pillars [7].

Original: These methods successfully seeklong-range context by utilizing transformers’ powerful abilities in modeling long-range information.
Revised: These methods successfully seeklong-range context by utilizing transformers’ powerful abilities in modeling long-range information.

Original: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details,especially in sparse 3D space, crucial to accurate object recognition and localization.
Revised: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details,especially in sparse 3D space, crucial to accurate object recognition and localization.

Original: Standard window-based transformers update the features of queries in a local window by attending tokeys from the same window.
Revised: Standard window-based transformers update the features of queries in a local window by attending tokeys from the same window.

Original: Hence simultaneously aggregating long-range context and fine-graineddetails require enlarging the window size to embrace local and distant voxels.
Revised: Hence simultaneously aggregating long-range context and fine-graineddetails require enlarging the window size to embrace local and distant voxels.

Original: Nevertheless, directlygathering all the voxels within the window as keys suffers a cubical growth of the computationalload with the window size.
Revised: Nevertheless, directlygathering all the voxels within the window as keys suffers a cubical growth of the computationalload with the window size.

Original: Some attempts alleviate this by sampling only a certain number of keyvoxels [23].
Revised: Some attempts alleviate this by sampling only a certain number of keyvoxels [23].

Original: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels(Fig.
Revised: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels(Fig.

Original: 1 b), thus bias mainly on long-range context.
Revised: 1 b), thus bias mainly on long-range context.

Original: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.
Revised: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.

Original: As a result, we can keep finergranularity in the local region to retain fine-grained details while collecting distant voxels roughly toenlarge the receptive field (Fig.
Revised: As a result, we can keep finergranularity in the local region to retain fine-grained details while collecting distant voxels roughly toenlarge the receptive field (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: With the sampled voxels ready, the next question is how to effectively attend to voxels from differentwindows and simultaneously capture long-range context and fine-grained details.
Revised: With the sampled voxels ready, the next question is how to effectively attend to voxels from differentwindows and simultaneously capture long-range context and fine-grained details.

Original: We argue that thedivide-and-conquer philosophy can satisfactorily resolve this issue.
Revised: We argue that thedivide-and-conquer philosophy can satisfactorily resolve this issue.

Original: Specifically, inspired by therecent findings [42, 56, 26] that transformers learn different levels of self-attention by different heads,we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides thetransformer heads into multiple groups.
Revised: Specifically, inspired by therecent findings [42, 56, 26] that transformers learn different levels of self-attention by different heads,we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides thetransformer heads into multiple groups.

Original: Different head groups accept voxels sampled from windowsof different sizes, so they are each in charge of capturing information of a particular scale.
Revised: Different head groups accept voxels sampled from windowsof different sizes, so they are each in charge of capturing information of a particular scale.

Original: Combiningthe outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context,and fine-grained details.
Revised: Combiningthe outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context,and fine-grained details.

Original: We also design a novel scale-aware relative position encoding strategy toadaptively adjust the position encoding used in each head group according to the range of the keys.
Revised: We also design a novel scale-aware relative position encoding strategy toadaptively adjust the position encoding used in each head group according to the range of the keys.

Original: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).
Revised: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).

Original: It is alsoworth mentioning that the mixed-scale attention enables information exchange across local windows,making MsSVT more compact by saving additional shift window operation commonly required bywindow-based transformers [21, 7].
Revised: It is alsoworth mentioning that the mixed-scale attention enables information exchange across local windows,making MsSVT more compact by saving additional shift window operation commonly required bywindow-based transformers [21, 7].

Original: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reducecomputational costs in two ways.
Revised: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reducecomputational costs in two ways.

Original: First, we propose a novel chessboard sampling (CBS) strategyto reduce the number of query voxels that need to be sampled within the query window, to reducecomputational costs without losing information.
Revised: First, we propose a novel chessboard sampling (CBS) strategyto reduce the number of query voxels that need to be sampled within the query window, to reducecomputational costs without losing information.

Original: Specifically, we partition the query window intochess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.
Revised: Specifically, we partition the query window intochess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.

Original: During each attentionlayer, only one specific position of voxels is sampled and updated by serving as queries, and theupdates of the other voxels can be obtained by interpolation.
Revised: During each attentionlayer, only one specific position of voxels is sampled and updated by serving as queries, and theupdates of the other voxels can be obtained by interpolation.

Original: Four positions are selected in the circular.
Revised: Four positions are selected in the circular.

Original: Thus, we can update all the voxels without introducing deviation.
Revised: Thus, we can update all the voxels without introducing deviation.

Original: Second, we take advantage ofnon-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-emptysites in 3D space.
Revised: Second, we take advantage ofnon-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-emptysites in 3D space.

Original: Furthermore, we parallelize the search and feature gathering for non-empty voxelsusing hash mapping for further acceleration.
Revised: Furthermore, we parallelize the search and feature gathering for non-empty voxelsusing hash mapping for further acceleration.

Original: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [47] with our
Revised: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [47] with our

Original: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].
Revised: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].

Original: Benefitingfrom the powerful capability of abstracting mixed-scale voxel features, our single-stage detectorbased on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.
Revised: Benefitingfrom the powerful capability of abstracting mixed-scale voxel features, our single-stage detectorbased on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.

Original: We summarize ourcontributions as follows:
Revised: We summarize ourcontributions as follows:

Original: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneouslyabstracts voxel features with long-range context and fine-grained details.•
Revised: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneouslyabstracts voxel features with long-range context and fine-grained details.•

Original: We design an efficient chessboard sampling strategy to vastly reduce the computational costof applying a voxel-based transformer in 3D space and sparsely implement all operations toimprove efficiency.• Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors onWaymo.
Revised: We design an efficient chessboard sampling strategy to vastly reduce the computational costof applying a voxel-based transformer in 3D space and sparsely implement all operations toimprove efficiency.• Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors onWaymo.

Original:  Related work
Revised:  Related work

Original: Outdoor 3D detection on point clouds.
Revised: Outdoor 3D detection on point clouds.

Original: The mainstream outdoor 3D object detectors are basedon voxels [3, 48, 49, 37, 17, 14, 57, 47] or pillars [16, 32].
Revised: The mainstream outdoor 3D object detectors are basedon voxels [3, 48, 49, 37, 17, 14, 57, 47] or pillars [16, 32].

Original: VoxelNet [57] utilizes PointNet [27,32] to aggregate features within each voxel and then apply sparse 3D convolution to generatedetection results.
Revised: VoxelNet [57] utilizes PointNet [27,32] to aggregate features within each voxel and then apply sparse 3D convolution to generatedetection results.

Original: SECOND [47] investigates improved sparse convolution to further improve speed.
Revised: SECOND [47] investigates improved sparse convolution to further improve speed.

Original: Pointpillar [16] quantifies the point cloud into pillars such that 2D CNNs can be applied to trade offbetween accuracy and efficiency.
Revised: Pointpillar [16] quantifies the point cloud into pillars such that 2D CNNs can be applied to trade offbetween accuracy and efficiency.

Original: Rapoport-Lavie et al .
Revised: Rapoport-Lavie et al .

Original: [28] introduce the Cylindrical Coordinatesto leverage the natural scanning pattern from LiDAR sensors.
Revised: [28] introduce the Cylindrical Coordinatesto leverage the natural scanning pattern from LiDAR sensors.

Original: Chen et al .
Revised: Chen et al .

Original: [2] further operate onboth Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS)voxel representation.
Revised: [2] further operate onboth Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS)voxel representation.

Original: Reconfigurable Voxels [43] improves the local neighbors searching of eachvoxel using a random walk scheme.
Revised: Reconfigurable Voxels [43] improves the local neighbors searching of eachvoxel using a random walk scheme.

Original: Voxel-FPN
Revised: Voxel-FPN

Original: [15] and Pillar-in-Pillar [41] adopt a multi-scalevoxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate
Revised: [15] and Pillar-in-Pillar [41] adopt a multi-scalevoxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate

Original: FPN [20] to aggregate multi-resolution feature maps.
Revised: FPN [20] to aggregate multi-resolution feature maps.

Original: Two-stage detectors [19, 35, 36, 31] refine thebounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features,yielding state-of-the-art performance.
Revised: Two-stage detectors [19, 35, 36, 31] refine thebounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features,yielding state-of-the-art performance.

Original: Vision transformer.
Revised: Vision transformer.

Original: Transformer [42, 5] has recently achieved great success in computer vision [6,1 , 21, 18, 58, 50, 44].
Revised: Transformer [42, 5] has recently achieved great success in computer vision [6,1 , 21, 18, 58, 50, 44].

Original: Swin-transformer [21] restricts self-attention to non-overlapping local windowswhile allowing cross-window connection to improve efficiency.
Revised: Swin-transformer [21] restricts self-attention to non-overlapping local windowswhile allowing cross-window connection to improve efficiency.

Original: SSA [29] divides attention headsinto multiple groups to aggregate image features with different granularities.
Revised: SSA [29] divides attention headsinto multiple groups to aggregate image features with different granularities.

Original: Guo et al.
Revised: Guo et al.

Original: [9] and
Revised: [9] and

Original: Zhao et al.
Revised: Zhao et al.

Original: [55] make the first step towards introducing the transformer for point cloud analysis.
Revised: [55] make the first step towards introducing the transformer for point cloud analysis.

Original: Recently, many approaches [45, 54, 25, 10, 22] apply local self-attention on voxels to learn richerfeature representation.
Revised: Recently, many approaches [45, 54, 25, 10, 22] apply local self-attention on voxels to learn richerfeature representation.

Original: Our work extends the window-based attention on 3D voxels by introducingscale-aware attention learning equipped with novel sampling strategies for the queries and the keys toimprove both accuracy and efficiency.
Revised: Our work extends the window-based attention on 3D voxels by introducingscale-aware attention learning equipped with novel sampling strategies for the queries and the keys toimprove both accuracy and efficiency.

Original: Voxel transformer for 3D detection.
Revised: Voxel transformer for 3D detection.

Original: VoTr [23] introduces a voxel-based transformer backbonethat performs self-attention on sparse voxels with the local and the dilated attention mechanisms.
Revised: VoTr [23] introduces a voxel-based transformer backbonethat performs self-attention on sparse voxels with the local and the dilated attention mechanisms.

Original: Our work improves VoTr by introducing the window-based attention and optimizing the sparseoperation.
Revised: Our work improves VoTr by introducing the window-based attention and optimizing the sparseoperation.

Original: The recent SST [7] follows a single-stride design and the swin-transformer architecture,which performs well on small objects.
Revised: The recent SST [7] follows a single-stride design and the swin-transformer architecture,which performs well on small objects.

Original: Nevertheless, SST is implemented based on pillars.
Revised: Nevertheless, SST is implemented based on pillars.

Original: Thesingle window size is not conducive to capturing multi-scale features, resulting in unsatisfactoryperformance on Vehicle when simultaneously detecting multiple categories.
Revised: Thesingle window size is not conducive to capturing multi-scale features, resulting in unsatisfactoryperformance on Vehicle when simultaneously detecting multiple categories.

Original: In comparison, our
Revised: In comparison, our

Original: MsSVT can capture mixed-scale information to boost detection of objects of various scales.
Revised: MsSVT can capture mixed-scale information to boost detection of objects of various scales.

Original:  Method
Revised:  Method

Original: This section first gives details of the MsSVT block, then its efficient sparse implementation, andfinally the 3D detector based on MsSVT.
Revised: This section first gives details of the MsSVT block, then its efficient sparse implementation, andfinally the 3D detector based on MsSVT.

Original: 3.1 Mixed-scale Sparse Voxel Transformer
Revised: 3.1 Mixed-scale Sparse Voxel Transformer

Original: Fig.
Revised: Fig.

Original: 2 illustrates the overall architecture of the MsSVT block.
Revised: 2 illustrates the overall architecture of the MsSVT block.

Original: We first gather the query and the keyvoxels via chessboard sampling and balanced multi-window sampling, respectively.
Revised: We first gather the query and the keyvoxels via chessboard sampling and balanced multi-window sampling, respectively.

Original: The obtainedqueries and keys are then fed into multiple head groups to capture mixed-scale information throughscale-aware attention learning.
Revised: The obtainedqueries and keys are then fed into multiple head groups to capture mixed-scale information throughscale-aware attention learning.

Original: We further incorporate scale-aware relative position encoding to makebetter use of position information in different head groups.
Revised: We further incorporate scale-aware relative position encoding to makebetter use of position information in different head groups.

Original: [Figure]
Revised: [Figure]

Original: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.
Revised: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.

Original: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.
Revised: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.

Original: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.
Revised: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

Original: 3.1.1 Balanced Multi-window Sampling
Revised: 3.1.1 Balanced Multi-window Sampling

Original: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query windowand s 1 ,...,M are the sizes of M successively larger key windows.
Revised: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query windowand s 1 ,...,M are the sizes of M successively larger key windows.

Original: Let V
Revised: Let V

Original: = { v i | v i =
Revised: = { v i | v i =

Original: ( x i , f i ) } |V| i =1 bethe input voxel set, with xyz coordinates x i ∈
Revised: ( x i , f i ) } |V| i =1 bethe input voxel set, with xyz coordinates x i ∈

Original: Z 3 and feature vector f i ∈ R C for voxel
Revised: Z 3 and feature vector f i ∈ R C for voxel

Original: i .
Revised: i .

Original: We firstpartition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-emptyones as query windows with their centers denoted by { c i
Revised: We firstpartition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-emptyones as query windows with their centers denoted by { c i

Original: | c i ∈
Revised: | c i ∈

Original: Z 3 } Li =0 , where L is the total numberof query windows.
Revised: Z 3 } Li =0 , where L is the total numberof query windows.

Original: To get query voxels V c i , s 0 for the query window centered on c
Revised: To get query voxels V c i , s 0 for the query window centered on c

Original: i , one can simplygather all the non-empty voxels within the window as the queries.
Revised: i , one can simplygather all the non-empty voxels within the window as the queries.

Original: While keeping efficiency in mind,we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.
Revised: While keeping efficiency in mind,we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.

Original: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for theneighbors for each center c i within multiple key windows of different sizes.
Revised: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for theneighbors for each center c i within multiple key windows of different sizes.

Original: For the key window ofsize s k , we gather no more than N P non-empty voxels V c
Revised: For the key window ofsize s k , we gather no more than N P non-empty voxels V c

Original: i , s k
Revised: i , s k

Original: = { v j | − s k < x j − c
Revised: = { v j | − s k < x j − c

Original: i < s k } N P j =1 ,where N P is a pre-set number.
Revised: i < s k } N P j =1 ,where N P is a pre-set number.

Original: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from
Revised: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from

Original: V c i , s k to obtain the final key voxels
Revised: V c i , s k to obtain the final key voxels

Original: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-setmaximum number of sampled voxels.
Revised: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-setmaximum number of sampled voxels.

Original: Benefitting from the multi-window strategy equipped withuniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, whichis crucial to capturing mixed-scale information.
Revised: Benefitting from the multi-window strategy equipped withuniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, whichis crucial to capturing mixed-scale information.

Original: 3.1.2 Chessboard Sampling
Revised: 3.1.2 Chessboard Sampling

Original: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels andonly need to select some representative ones to reduce computational redundancy.
Revised: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels andonly need to select some representative ones to reduce computational redundancy.

Original: While thingsare different for the queries.
Revised: While thingsare different for the queries.

Original: On the one hand, we should retain and update every query voxel afteran attention layer; otherwise, the information could be irreversibly lost.
Revised: On the one hand, we should retain and update every query voxel afteran attention layer; otherwise, the information could be irreversibly lost.

Original: But on the other hand,the computational cost and memory footprint grow dramatically with the window size, makingimplementation impractical if the queries are not reduced.
Revised: But on the other hand,the computational cost and memory footprint grow dramatically with the window size, makingimplementation impractical if the queries are not reduced.

Original: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxelseach time for feature updating, and then use thesampled voxels to update those unsampled.
Revised: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxelseach time for feature updating, and then use thesampled voxels to update those unsampled.

Original: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.
Revised: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.

Original: Each voxel inthe query window is marked with " × ", " ⃝ ", " △ ",or " □ ", in an evenly spaced pattern.
Revised: Each voxel inthe query window is marked with " × ", " ⃝ ", " △ ",or " □ ", in an evenly spaced pattern.

Original: In each MsSVTblock, we sample queries from the non-empty voxelswith one specific type of the marks, and update thequeries’ features through attention learning.
Revised: In each MsSVTblock, we sample queries from the non-empty voxelswith one specific type of the marks, and update thequeries’ features through attention learning.

Original: Thenfor each unsampled non-empty voxel, we update itsfeature by finding the K-nearest query voxels (bydefault K = 3 ) and by linearly interpolating theirupdated features.
Revised: Thenfor each unsampled non-empty voxel, we update itsfeature by finding the K-nearest query voxels (bydefault K = 3 ) and by linearly interpolating theirupdated features.

Original: The four masks are selected in turn to sample query voxels in stacked blocks.
Revised: The four masks are used in the circular to sample query voxels in stacked blocks.

Original: Thisway, we retain the original structure and cover all voxels as comprehensively as possible.
Revised: This way, we retain the original structure and cover all voxels as comprehensively as possible.

Original: Note thatwe can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2, 1/4, or1/8.
Revised: Notethat we can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2,1/4, or 1/8.

Original: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).
Revised: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).

Original: 3.1.3 Scale-aware Head Attention
Revised: 3.1.3 Scale-aware Head Attention

Original: Given the query voxels V c
Revised: Given the query voxels V c

Original: i , s 

Revised: i , s

Original: =
Revised: =

Original: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and featurevectors F 0 ∈ R N Q × C , and the multi-scale key voxels
Revised: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and featurevectors F 0 ∈ R N Q × C , and the multi-scale key voxels

Original: V fps c i , s k =
Revised: V fps c i , s k =

Original: ( X k , F k ) , k = 1 , ..., M with voxelcoordinates X
Revised: ( X k , F k ) , k = 1 , ..., M with voxelcoordinates X

Original: k ∈
Revised: k ∈

Original: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk =1and values { V k } Mk =1 as
Revised: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk =1and values { V k } Mk =1 as

Original: [Equation]
Revised: [Equation]

Original: , M (1) where W Q , W K , 
Revised: where W Q , W K , W V ∈ R C

Original: W V ∈ R C × C are linear projections.
Revised: × C are linear projections.

Original: To achieve scare-aware attention learning, wedividethe multiple attention heads into M groups and assign  keys from windows of different sizes todifferent head groups.
Revised: To achieve scare-aware attention learning, wedivide multiple attention heads into M groups and assign the keys from windows of different sizes todifferent head groups.

Original: Accordingly, we also split the feature channels of the queries Q into M groups.
Revised: Accordingly, we also split the feature channels of the queries Q into M groups.

Original: The k -th channel group of Q , denoted by Q k =
Revised: The k -th channel group of Q , denoted by Q k =

Original: Q
Revised: Q

Original: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M ,is fed into the k -th head group.
Revised: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M ,is fed into the k -th head group.

Original: As a result, each head group is responsible for learning to attendinformation of a specific scale.
Revised: As a result, each head group is responsible for learning to attendinformation at a specific scale.

Original: The attended feature for the k -th head group is
Revised: The attended feature for the k -th head group is

Original: [Equation]
Revised: [Equation]

Original: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents  newly proposed relativeposition encoding which will be discussed in Section 3.1.4.
Revised: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents the newly proposedrelative position encoding which will be detailed in Section 3.1.4.

Original: Each window size corresponds to a headgroup with one or more attention heads.
Revised: Each window size corresponds toa head group with one or more attention heads.

Original: We concatenate the output from all heads groups { ˜ Y k } Mk =1 to˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implemented by multi-layerperceptron (MLP) to obtain the final mixed-scale feature Y :
Revised: We concatenate the output from all heads groups{ ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implementedby multi-layer perceptron (MLP) to obtain the final mixed-scale feature Y :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where LN ( · ) represents layer normalization.
Revised: where LN ( · ) represents layer normalization.

Original: 3.1.4 Scale-aware Relative Position Encoding
Revised: 3.1.4 Scale-aware Relative Position Encoding

Original: Relative position encoding is necessary for transformer-based networks because fine-grained positioninformation may be lost in high-level features with the deepening of the network.
Revised: Relative position encoding is necessary for transformer-based networks because fine-grained positioninformation may be lost in high-level features with the deepening of the network.

Original: To make better useof position information to facilitate multi-scale feature learning in our case, we adopt a scale-awareadaptive relative position encoding strategy inspired by [30, 46, 52], which can generate the positionalbias dynamically with scales for different head groups.
Revised: To make better useof position information to facilitate multi-scale feature learning in our case, we adopt a scale-awareadaptive relative position encoding strategy inspired by [30, 46, 52], which can generate the positionalbias dynamically with scales for different head groups.

Original: Specifically, we establish a learnable embedding table T k ∈ R D × R for  k -th head group according tothe size of the largest key window [21], where R is the number of possible relative position pairs and D is the feature dimension.
Revised: Specifically, we establish a learnable embedding table T k ∈ R D × R for the k -th head group accordingto the size of the largest key window [21], where R is the number of possible relative position pairsand D is the feature dimension.

Original: We get the relative positional bias for the queries as:
Revised: We get the relative positional bias for the queries as:

Original: [Equation]
Revised: [Equation]

Original: where I
Revised: where I

Original: k ∈
Revised: k ∈

Original: Z L × N Q × N K represent the table indices corresponding to the actual relative positionbetween the queries and  keys, G ( · ) is the operation of gathering features via indices.
Revised: Z L × N Q × N K represents the table indices corresponding to the actual relative positionbetween the queries and the keys, G ( · ) is the operation of gathering features according to the indices.

Original: Similarly, wecan ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.
Revised: Similarly, we can ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.

Original: Finally, the biases act directly onthe attention weights, and Eq.
Revised: Finally, the biasesact directly on the attention weights, and Eq.

Original: (2) can be rewritten as:
Revised: (2) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) represents softmax.
Revised: where σ ( · ) represents softmax.

Original: This way, position embeddings can be adjusted adaptively accordingto different scales to guide our scale-ware head attention better.
Revised: This way, position embeddings can be adjusted adaptively accordingto different scales to guide our scale-ware head attention better.

Original: 3.2 Sparse Implementation
Revised: 3.2 Sparse Implementation

Original: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implementall our window center searching, window gathering, and balanced window sampling into CUDAoperations.
Revised: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implementall our window center searching, window gathering, and balanced window sampling into CUDAoperations.

Original: These operations are mainly based on a hash map that establishes the mapping fromcoordinate space to voxel index [23].
Revised: These operations are mainly based on a hash map that establishes the mapping fromcoordinate space to voxel index [23].

Original: Take the window gathering as an example, we query eachpossible position w.r.t.
Revised: Take the window gathering as an example, we query eachpossible position w.r.t.

Original: the given center within the window and retrieve the corresponding features ifthe position is a valid key in our pre-built hash map.
Revised: the given center within the window and retrieve the corresponding features ifthe position is a valid key in our pre-built hash map.

Original: More details are available in the supplementarymaterials.
Revised: More details are available in the supplementarymaterials.

Original: 3.3 Detector Establishment
Revised: 3.3 Detector Establishment

Original: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.
Revised: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.

Original: Noted that weset both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3Dvoxels into a 2D feature map, where the query is the average voxel features within the pillar window.
Revised: Noted that weset both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3Dvoxels into a 2D feature map, where the query is the average voxel features within the pillar window.

Original: We replace the 3D backbone in SECOND [47] with MsSVT and keep the other network componentsunchanged.
Revised: We replace the 3D backbone in SECOND [47] with MsSVT and keep the other network componentsunchanged.

Original: We discard the down sampling process because MsSVT can already capture features atdifferent scales.
Revised: We discard the down sampling process because MsSVT can already capture features atdifferent scales.

Original: The input point cloud is first converted into regular voxels and fed into our MsSVTbackbone to get mixed-scale voxel features.
Revised: The input point cloud is first converted into regular voxels and fed into our MsSVTbackbone to get mixed-scale voxel features.

Original: The features are then compressed vertically and sent tothe subsequent 2D RPN and detection head to get detection results.
Revised: The features are then compressed vertically and sent tothe subsequent 2D RPN and detection head to get detection results.

Original:  Experiments
Revised:  Experiments

Original:  In this section, we first provide architectural details of MsSVT, then compare our model with recentstate-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.
Revised: [Figure] In this section, we first provide architectural details of MsSVT, then compare our model with recentstate-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.

Original: Thorough ablation studiesand in-depth analysis are further provided to validate our design choices.
Revised: Thorough ablation studiesand in-depth analysis are further provided to validate our design choices.

Original: 4.1 Architectural Details
Revised: 4.1 Architectural Details

Original: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key windowsizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are setas 1 × 1 pillars as mentioned in 3.3.
Revised: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key windowsizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are setas 1 × 1 pillars as mentioned in 3.3.

Original: We divide 8 attention heads into 2 head groups.
Revised: We divide 8 attention heads into 2 head groups.

Original: The samplingrate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .
Revised: The samplingrate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .

Original: Weuse the center head [53] to generate single-stage bounding boxes.
Revised: Weuse the center head [53] to generate single-stage bounding boxes.

Original: In addition, we also provide atwo-stage version with CT3D [31].
Revised: In addition, we also provide atwo-stage version with CT3D [31].

Original: More details can be found in OpenPCDet [40] since we conductall experiments with this toolbox.
Revised: More details can be found in OpenPCDet [40] since we conductall experiments with this toolbox.

Original: 4.2 Results on Waymo
Revised: 4.2 Results on Waymo

Original: Setups.
Revised: Setups.

Original: We first evaluate our model on large-scale Waymo Open Dataset [38].
Revised: We first evaluate our model on large-scale Waymo Open Dataset [38].

Original: The input is asingle-frame point cloud with 150m × 150m detection range.
Revised: The input is asingle-frame point cloud with 150m × 150m detection range.

Original: We set the detection range in thehorizontal and the vertical direction as [ − 75 .
Revised: We set the detection range in thehorizontal and the vertical direction as [ − 75 .

Original: 2m , 75 . 2m] and [ − 2 .
Revised: 2m , 75 . 2m] and [ − 2 .

Original: 0m , 4 . 0m] , respectively.
Revised: 0m , 4 . 0m] , respectively.

Original: The voxelsize is (0 . 4m , 0 .
Revised: The voxelsize is (0 . 4m , 0 .

Original: 4m , 0 .
Revised: 4m , 0 .

Original: 6m) .
Revised: 6m) .

Original: We follow the same training strategy as in [23].
Revised: We follow the same training strategy as in [23].

Original: Specifically, we train ourmodel for 80 epochs on 20% Waymo data using the Adam optimizer [13].
Revised: Specifically, we train ourmodel for 80 epochs on 20% Waymo data using the Adam optimizer [13].

Original: We apply the cyclic decayscheme [47], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochsand further decreased to 1 e- 5 in the remaining epochs.
Revised: We apply the cyclic decayscheme [47], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochsand further decreased to 1 e- 5 in the remaining epochs.

Original: We also report the results of training for 30epochs on 100% Waymo data using the same optimizer and learning rate scheme.
Revised: We also report the results of training for 30epochs on 100% Waymo data using the same optimizer and learning rate scheme.

Original: The evaluationmetric is the 3D mean Average Precision (mAP)for difficulty levels of LEVEL 1 and LEVEL 2.
Revised: The evaluationmetric is the 3D mean Average Precision (mAP)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Main results.
Revised: Main results.

Original: We compare our model with state-of-the-art priors in Table 1 and Table 2 .
Revised: We compare our model with state-of-the-art priors in Table 1 and Table 2 .

Original: It is worthnoting that our model performs simultaneous detection of three object categories, which is morechallenging than detecting a single category.
Revised: It is worthnoting that our model performs simultaneous detection of three object categories, which is morechallenging than detecting a single category.

Original: Using 20% training data, our single-stage MsSVT-SSsignificantly outperforms other single-stage detectors, even though some of them are trained speciallyfor one specific category.
Revised: Using 20% training data, our single-stage MsSVT-SSsignificantly outperforms other single-stage detectors, even though some of them are trained speciallyfor one specific category.

Original: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage
Revised: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage

Original: PV-RCNN++ [36] on Vehicle , and even better, i.e. , 0 .
Revised: PV-RCNN++ [36] on Vehicle , and even better, i.e. , 0 .

Original: 8 - 1 . 5 mAP higher, on Pedestrian and Cyclist .
Revised: 8 - 1 . 5 mAP higher, on Pedestrian and Cyclist .

Original: Inaddition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperformsprevious best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.
Revised: Inaddition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperformsprevious best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.

Original: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achievethe best results compared to their corresponding counterparts.
Revised: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achievethe best results compared to their corresponding counterparts.

Original: Noticeably, our MsSVT-SS surpassesthe recent transformer-based detector SST [7] that is dedicated to small object detection by largemargins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.
Revised: Noticeably, our MsSVT-SS surpassesthe recent transformer-based detector SST [7] that is dedicated to small object detection by largemargins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.

Original: This clear evidences the superiorityof our MsSVT in capturing multi-scale information over the conventional transformer design.
Revised: This clear evidences the superiorityof our MsSVT in capturing multi-scale information over the conventional transformer design.

Original: Visualization.
Revised: Visualization.

Original: We provide some qualitative results in Fig.
Revised: We provide some qualitative results in Fig.

Original: Our MsSVT can still predict accuratebounding boxes in the scenes beyond the range of 50m where the points become extremely sparse(Fig.
Revised: Our MsSVT can still predict accuratebounding boxes in the scenes beyond the range of 50m where the points become extremely sparse(Fig.

Original: 4 (a)).
Revised: 4 (a)).

Original: This evidences that MsSVT successfully captures valuable contextual information in theabsence of fine-grained details for long-range object detection.
Revised: This evidences that MsSVT successfully captures valuable contextual information in theabsence of fine-grained details for long-range object detection.

Original: In addition, our model also performswell in complex scenes in the presence of dense objects with large scale variations (Fig.
Revised: In addition, our model also performswell in complex scenes in the presence of dense objects with large scale variations (Fig.

Original: 4 (b).
Revised: 4 (b).

Original: Itshould be owed to the flexibility and robustness of MsSVT.
Revised: Itshould be owed to the flexibility and robustness of MsSVT.

Original: Analysis.
Revised: Analysis.

Original: We further visualize some attention maps which indicate what have been focused on byour MsSVT to understand its behavior better.
Revised: We further visualize some attention maps which indicate what have been focused on byour MsSVT to understand its behavior better.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, the head group with a smaller keywindow focuses on the local foreground information, while the head group with a larger key windowattends more to longer-range context.
Revised: 5, the head group with a smaller keywindow focuses on the local foreground information, while the head group with a larger key windowattends more to longer-range context.

Original: As a result, the two head groups complement each other, wellcapturing mixed-scale information to support the detection of objects of various scales.
Revised: As a result, the two head groups complement each other, wellcapturing mixed-scale information to support the detection of objects of various scales.

Original: [Table]
Revised: [Table]

Original: 4.3 Results on KITTI
Revised: 4.3 Results on KITTI

Original: Setups.
Revised: Setups.

Original: We also evaluate our model on KITTI [8].
Revised: We also evaluate our model on KITTI [8].

Original: Given an input point cloud, we reserve thepoints within the range of [0m , 70 . 4m] , [ − 40 .
Revised: Given an input point cloud, we reserve thepoints within the range of [0m , 70 . 4m] , [ − 40 .

Original: 0m , 40 .
Revised: 0m , 40 .

Original: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes,respectively.
Revised: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes,respectively.

Original: The voxel size is set to (0 .
Revised: The voxel size is set to (0 .

Original: 32m , 0 . 32m , 0 .
Revised: 32m , 0 . 32m , 0 .

Original: 4m)
Revised: 4m)

Original: and other settings remain the same asin the experiments on Waymo.
Revised: and other settings remain the same asin the experiments on Waymo.

Original: We train the model for 100 epochs with the Adam optimizer.
Revised: We train the model for 100 epochs with the Adam optimizer.

Original: Thelearning rate is 0 . 003 , decayed by the cyclic scheme [47].
Revised: Thelearning rate is 0 . 003 , decayed by the cyclic scheme [47].

Original: The evaluation metric is 3D mAP for threedifficulty levels (easy, moderate, and hard).
Revised: The evaluation metric is 3D mAP for threedifficulty levels (easy, moderate, and hard).

Original: Main results.
Revised: Main results.

Original: Table 3 depicts that our model achieves competitive performance on all threecategories.
Revised: Table 3 depicts that our model achieves competitive performance on all threecategories.

Original: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by0 . 5 mAP.
Revised: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by0 . 5 mAP.

Original: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminenttwo-stage detectors.
Revised: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminenttwo-stage detectors.

Original: Moreover, our two-stage MsSVT-TS further increases the lead.
Revised: Moreover, our two-stage MsSVT-TS further increases the lead.

Original: These resultsdemonstrate that our MsSVT can be well generalized to various datasets.
Revised: These resultsdemonstrate that our MsSVT can be well generalized to various datasets.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: All models are trained for 12 epochs on 20% Waymo data.
Revised: All models are trained for 12 epochs on 20% Waymo data.

Original: More implementation details and ablationson hyperparameters are available in supplementary materials.
Revised: More implementation details and ablationson hyperparameters are available in supplementary materials.

Original: [Table]
Revised: [Table]

Original: Balanced multi-window sampling.
Revised: Balanced multi-window sampling.

Original: We first validate our balanced multiwindow sampling strategy in Table 4.
Revised: We first validate our balanced multiwindow sampling strategy in Table 4.

Original: The base model listed in the first rowemploys a 3D version of standardwindow-based attention [21] withwindow size (3 , 3 , 5) and without shiftwindow scheme, adopts dilated keysampling [23], and gathers all non-emptyvoxels within the window as queries.
Revised: The base model listed in the first rowemploys a 3D version of standardwindow-based attention [21] withwindow size (3 , 3 , 5) and without shiftwindow scheme, adopts dilated keysampling [23], and gathers all non-emptyvoxels within the window as queries.

Original: We build a model variant by simplyreplacing the dilated key sampling withour proposed balanced multi-window sampling.
Revised: We build a model variant by simplyreplacing the dilated key sampling withour proposed balanced multi-window sampling.

Original: Results in the second row demonstrate that samplingkey voxels from multiple windows of different sizes brings noticeable performance gains, i.e. ,71 . 

Revised: Results in the second row demonstrate that samplingkey voxels from multiple windows of different sizes brings noticeable performance gains, i.e. ,71 .

Original: → 73 . 44 on Pedestrian and 63 .
Revised: → 73 . 44 on Pedestrian and 63 .

Original: 31 → 66 . 88 on Cyclist .
Revised: 31 → 66 . 88 on Cyclist .

Original: Scale-aware head attention.
Revised: Scale-aware head attention.

Original: We also validate the effectiveness of our proposed scale-aware headattention in Table 4.
Revised: We also validate the effectiveness of our proposed scale-aware headattention in Table 4.

Original: By comparing the second and the third rows, one can find that enabling multiplehead groups to capture information of different scales clearly boots the performance.
Revised: By comparing the second and the third rows, one can find that enabling multiplehead groups to capture information of different scales clearly boots the performance.

Original: Scale-aware relative position encoding.
Revised: Scale-aware relative position encoding.

Original: Table 4 shows that incorporating scale-aware relativeposition encoding (listed in the last row) further improves the performance compared to the modelvariant that adopts scale-agnostic position encoding.
Revised: Table 4 shows that incorporating scale-aware relativeposition encoding (listed in the last row) further improves the performance compared to the modelvariant that adopts scale-agnostic position encoding.

Original: It well supports our design motivation that theencoded position information should vary with different scales.
Revised: It well supports our design motivation that theencoded position information should vary with different scales.

Original: [Table]
Revised: [Table]

Original: Chessboard sampling.
Revised: Chessboard sampling.

Original: Table 5 reportsthe performance of the model variants thatapply chessboard sampling with different sampling rates and of the model without performing any sampling.
Revised: Table 5 reportsthe performance of the model variants thatapply chessboard sampling with different sampling rates and of the model without performing any sampling.

Original: We measure the latency on Tesla-V100 GPU and
Revised: We measure the latency on Tesla-V100 GPU and

Original: Unbuntu-16.04, Python 3.7, Cuda-10.2,and Pytorch-1.8.
Revised: Unbuntu-16.04, Python 3.7, Cuda-10.2,and Pytorch-1.8.

Original: Our model is not sensitive to varying sampling rates.
Revised: Our model is not sensitive to varying sampling rates.

Original: Incorporating chessboard sampling with a samplingrate of 1/4 results in only negligible performance degradation compared to the model variant withoutsampling, yet greatly improves the computational efficiency by a significant reduction in memoryfootprint (up to 33% decrease) and in latency (up to 28% decrease).
Revised: Incorporating chessboard sampling with a samplingrate of 1/4 results in only negligible performance degradation compared to the model variant withoutsampling, yet greatly improves the computational efficiency by a significant reduction in memoryfootprint (up to 33% decrease) and in latency (up to 28% decrease).

Original:  Conclusion
Revised:  Conclusion

Original: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scaleinformation to boost 3D object detection.
Revised: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scaleinformation to boost 3D object detection.

Original: We begin with sampling the queries and the keys at differentscales through the novel chessboard and the balanced multi-window sampling strategy, respectively.
Revised: We begin with sampling the queries and the keys at differentscales through the novel chessboard and the balanced multi-window sampling strategy, respectively.

Original: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge ofcapturing information at a specific scale.
Revised: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge ofcapturing information at a specific scale.

Original: Extensive experiments have shown that MsSVT is superiorin detecting objects of various scales and sparsity thanks to the learnd mixed-scale information.
Revised: Extensive experiments have shown that MsSVT is superiorin detecting objects of various scales and sparsity thanks to the learnd mixed-scale information.

Original: Limitations.
Revised: Limitations.

Original: MsSVT considers fine-grained details and long-range contextual information in differentwindows and achieves promising performance, but still needs to set hand-crafted window sizes.
Revised: MsSVT considers fine-grained details and long-range contextual information in differentwindows and achieves promising performance, but still needs to set hand-crafted window sizes.

Original: Wewill explore the adaptive window version of MsSVT in the future.
Revised: Wewill explore the adaptive window version of MsSVT in the future.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Our work
Revised: [No] Our work

Original: is only for academic research purpose.
Revised: is only for academic research purpose.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] Our implemen-
Revised: [Yes] Our implemen-

Original: tation details are posted on Section 4.1 and the supplemental materials.
Revised: tation details are posted on Section 4.1 and the supplemental materials.

Original: The data arepublic datasets and our code will be available.
Revised: The data arepublic datasets and our code will be available.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?
Revised: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Section 4.1 and setup in 4.2 and 4.
Revised: [Yes] See Section 4.1 and setup in 4.2 and 4.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report average performance of 5 trials by default.
Revised: [Yes] We report average performance of 5 trials by default.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] Waymo [38] and
Revised: [Yes] Waymo [38] and

Original: KITTI [8]
Revised: KITTI [8]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: We don’t include any new assets.
Revised: We don’t include any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] Waymo [38] and KITTI [8] are two standardbenchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standardbenchmarks.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots,
Revised: (a) Did you include the full text of instructions given to participants and screenshots,

Original: if applicable?
Revised: if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human
Revised: We didn’t use crowdsourcing or conduct research with human

Original: subjects...
Revised: subjects...

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct
Revised: We didn’t use crowdsourcing or conduct

Original: research with human subjects...
Revised: research with human subjects...

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A] We didn’t use crowdsourcing or conduct
Revised: [N/A] We didn’t use crowdsourcing or conduct

Original: research with human subjects...
Revised: research with human subjects...