Original: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds(s)
Revised: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds

Original: AffiliationAddress

Original: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomousdriving.
Revised: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomous driving.

Original: Large-scale outdoor scenes usually feature significant variance in instancescales, thus requiring features rich in long-range and fine-grained information tosupport accurate detection.
Revised: Large-scale outdoor scenes usually feature significant variance in instance scales, thus requiring features rich in long-range and fine-grained information to support accurate detection.

Original: Recent detectors leverage the power of window-basedtransformers to model long-range dependencies but tend to blur out fine-graineddetails.
Revised: Recent detectors leverage the power of window-based transformers to model long-range dependencies but tend to blur out fine-grained details.

Original: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.
Revised: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.

Original: Specifically, MsSVT explicitlydivides attention heads into multiple groups, each in charge of attending to information within a particular range.
Revised: Specifically, MsSVT explicitly divides attention heads into multiple groups, each in charge of attending to information within a particular range.

Original: All groups’ output is merged to obtain the finalmixed-scale features.
Revised: All groups’ output is merged to obtain the final mixed-scale features.

Original: Moreover, we provide a novel chessboard sampling strategyto reduce the computational complexity of applying a window-based transformer in3D voxel space.
Revised: Moreover, we provide a novel chessboard sampling strategy to reduce the computational complexity of applying a window-based transformer in 3D voxel space.

Original: To improve efficiency, we also implement the voxel sampling andgathering operations sparsely with a hash map.
Revised: To improve efficiency, we also implement the voxel sampling and gathering operations sparsely with a hash map.

Original: Endowed by the powerful capabilityand high efficiency of modeling mixed-scale information, our single-stage detectorbuilt on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectorson Waymo.
Revised: Endowed by the powerful capability and high efficiency of modeling mixed-scale information, our single-stage detector built on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectors on Waymo.

Original: Code will soon be available.
Revised: Our project page: https://github.com/dscdyc/MsSVT .

Original:  Introduction
Revised:  Introduction

Original: 3D object detection has received increasing attention due to its successful autonomous drivingapplications.
Revised: 3D object detection has received increasing attention due to its successful autonomous driving applications.

Original: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturallyirregular and unordered.
Revised: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturally irregular and unordered.

Original: Hence directly applying CNN-like operations [11, 12] to them can bedifficult.
Revised: Hence directly applying CNN-like operations [11, 12] to them can be difficult.

Original: To solve this, many researchers have rasterized point clouds into regular voxel grids [24]and employed 3D CNNs to extract high-dimensional voxel features.
Revised: To solve this, many researchers have rasterized point clouds into regular voxel grids [24] and employed 3D CNNs to extract high-dimensional voxel features.

Original: With the recent rise of visiontransformer (ViT) [42] on 2D images, some attempts have been made to generalize global or moreefficient window-based transformers to 3D voxels [23] or pillars [7].
Revised: With the recent rise of vision transformer (ViT) [42] on 2D images, some attempts have been made to generalize global or more efficient window-based transformers to 3D voxels [23] or pillars [7].

Original: These methods successfully seeklong-range context by utilizing transformers’ powerful abilities in modeling long-range information.
Revised: These methods successfully seek long-range context by utilizing transformers’ powerful abilities in modeling long-range information.

Original: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details,especially in sparse 3D space, crucial to accurate object recognition and localization.
Revised: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details, especially in sparse 3D space, crucial to accurate object recognition and localization.

Original: Standard window-based transformers update the features of queries in a local window by attending tokeys from the same window.
Revised: Standard window-based transformers update the features of queries in a local window by attending to keys from the same window.

Original: Hence simultaneously aggregating long-range context and fine-graineddetails require enlarging the window size to embrace local and distant voxels.
Revised: Hence simultaneously aggregating long-range context and fine-grained details require enlarging the window size to embrace local and distant voxels.

Original: Nevertheless, directlygathering all the voxels within the window as keys suffers a cubical growth of the computationalload with the window size.
Revised: Nevertheless, directly gathering all the voxels within the window as keys suffers a cubical growth of the computational load with the window size.

Original: Some attempts alleviate this by sampling only a certain number of keyvoxels [23].
Revised: Some attempts alleviate this by sampling only a certain number of key voxels [23].

Original: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels(Fig. 1 b), thus bias mainly on long-range context.
Revised: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels (Fig. 1 b), thus bias mainly on long-range context.

Original: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.
Revised: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.

Original: As a result, we can keep finergranularity in the local region to retain fine-grained details while collecting distant voxels roughly toenlarge the receptive field (Fig.
Revised: As a result, we can keep finer granularity in the local region to retain fine-grained details while collecting distant voxels roughly to enlarge the receptive field (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: With the sampled voxels ready, the next question is how to effectively attend to voxels from differentwindows and simultaneously capture long-range context and fine-grained details.
Revised: With the sampled voxels ready, the next question is how to effectively attend to voxels from different windows and simultaneously capture long-range context and fine-grained details.

Original: We argue that thedivide-and-conquer philosophy can satisfactorily resolve this issue.
Revised: We argue that the divide-and-conquer philosophy can satisfactorily resolve this issue.

Original: Specifically, inspired by therecent findings [42, 56, 26] that transformers learn different levels of self-attention by different heads,we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides thetransformer heads into multiple groups.
Revised: Specifically, inspired by the recent findings [42, 59, 26] that transformers learn different levels of self-attention by different heads, we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides the transformer heads into multiple groups.

Original: Different head groups accept voxels sampled from windowsof different sizes, so they are each in charge of capturing information of a particular scale.
Revised: Different head groups accept voxels sampled from windows of different sizes, so they are each in charge of capturing information of a particular scale.

Original: Combiningthe outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context,and fine-grained details.
Revised: Combining the outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context, and fine-grained details.

Original: We also design a novel scale-aware relative position encoding strategy toadaptively adjust the position encoding used in each head group according to the range of the keys.
Revised: We also design a novel scale-aware relative position encoding strategy to adaptively adjust the position encoding used in each head group according to the range of the keys.

Original: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).
Revised: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).

Original: It is alsoworth mentioning that the mixed-scale attention enables information exchange across local windows,making MsSVT more compact by saving additional shift window operation commonly required bywindow-based transformers [21, 7].
Revised: It is also worth mentioning that the mixed-scale attention enables information exchange across local windows, making MsSVT more compact by saving additional shift window operation commonly required by window-based transformers [21, 7].

Original: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reducecomputational costs in two ways.
Revised: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reduce computational costs in two ways.

Original: First, we propose a novel chessboard sampling (CBS) strategyto reduce the number of query voxels that need to be sampled within the query window, to reducecomputational costs without losing information.
Revised: First, we propose a novel chessboard sampling (CBS) strategy to reduce the number of query voxels that need to be sampled within the query window, to reduce computational costs without losing information.

Original: Specifically, we partition the query window intochess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.
Revised: Specifically, we partition the query window into chess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.

Original: During each attentionlayer, only one specific position of voxels is sampled and updated by serving as queries, and theupdates of the other voxels can be obtained by interpolation.
Revised: During each attention layer, only one specific position of voxels is sampled and updated by serving as queries, and the updates of the other voxels can be obtained by interpolation.

Original: Four positions are selected in the circular.
Revised: Four positions are selected in the circular.

Original: Thus, we can update all the voxels without introducing deviation.
Revised: Thus, we can update all the voxels without introducing deviation.

Original: Second, we take advantage ofnon-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-emptysites in 3D space.
Revised: Second, we take advantage of non-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-empty sites in 3D space.

Original: Furthermore, we parallelize the search and feature gathering for non-empty voxelsusing hash mapping for further acceleration.
Revised: Furthermore, we parallelize the search and feature gathering for non-empty voxels using hash mapping for further acceleration.

Original: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [47] with our
Revised: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [50] with our

Original: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].
Revised: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].

Original: Benefitingfrom the powerful capability of abstracting mixed-scale voxel features, our single-stage detectorbased on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.
Revised: Benefiting from the powerful capability of abstracting mixed-scale voxel features, our single-stage detector based on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.

Original: We summarize ourcontributions as follows:
Revised: We summarize our contributions as follows:

Original: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneouslyabstracts voxel features with long-range context and fine-grained details.•
Revised: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneously abstracts voxel features with long-range context and fine-grained details.

Original:  We design an efficient chessboard sampling strategy to vastly reduce the computational costof applying a voxel-based transformer in 3D space and sparsely implement all operations toimprove efficiency.• Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors onWaymo.
Revised: • We design an efficient chessboard sampling strategy to vastly reduce the computational cost of applying a voxel-based transformer in 3D space and sparsely implement all operations to improve efficiency. • Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors on Waymo.

Original:  Related work
Revised:  Related work

Original: Outdoor 3D detection on point clouds.
Revised: 3D detection on point clouds.

Original: The mainstreamoutdoor 3D object detectors are basedon voxels [3, 48, 49, 37, 17, 14, 57, 47] or pillars [16, 32].
Revised: The mainstream 3D object detectors are based on voxels [3, 51, 52, 37, 17, 14, 60, 50, 44, 43] or pillars [16, 32].

Original: VoxelNet [57] utilizes PointNet [27,32] to aggregate features within each voxel and then apply sparse 3D convolution to generatedetection results.
Revised: VoxelNet [60] utilizes PointNet [27, 32] to aggregate features within each voxel and then apply sparse 3D convolution to generate detection results.

Original: SECOND [47] investigates improved sparse convolution to further improve speed.
Revised: SECOND [50] investigates improved sparse convolution to further improve speed.

Original: Pointpillar [16] quantifies the point cloud into pillars such that 2D CNNs can be applied to trade offbetween accuracy and efficiency.
Revised: Pointpillar [16] converts the point cloud into pillars such that 2D CNNs can be applied to trade off between accuracy and efficiency.

Original: Rapoport-Lavie et al .
Revised: Rapoport-Lavie et al .

Original: [28] introduce the Cylindrical Coordinatesto leverage the natural scanning pattern from LiDAR sensors.
Revised: [28] introduce the Cylindrical Coordinates to leverage the natural scanning pattern from LiDAR sensors.

Original: Chen et al .
Revised: Chen et al .

Original: [2] further operate onboth Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS)voxel representation.
Revised: [2] further operate on both Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS) voxel representation.

Original: Reconfigurable Voxels [43] improves the local neighbors searching of eachvoxel using a random walk scheme.
Revised: Reconfigurable Voxels [45] improves the local neighbors searching of each voxel using a random walk scheme.

Original: Voxel-FPN
Revised: Voxel-FPN

Original: [15] and Pillar-in-Pillar [41] adopt a multi-scalevoxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate FPN [20] to aggregate multi-resolution feature maps.
Revised: [15] and Pillar-in-Pillar [41] adopt a multi-scale voxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate FPN [20] to aggregate multi-resolution feature maps.

Original: Two-stage detectors [19, 35, 36, 31] refine thebounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features,yielding state-of-the-art performance.
Revised: Two-stage detectors [19, 35, 36, 31] refine the bounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features, yielding state-of-the-art performance.

Original: Vision transformer.
Revised: Vision transformer.

Original: Transformer [42, 5] has recently achieved great success in computer vision [6,1 , 21, 18, 58, 50, 44].
Revised: Transformer [42, 5] has recently achieved great success in computer vision [6, 1, 21, 18, 61, 53, 46].

Original: Swin-transformer [21] restricts self-attention to non-overlapping local windowswhile allowing cross-window connection to improve efficiency.
Revised: Swin-transformer [21] restricts self-attention to non-overlapping local windows while allowing cross-window connection to improve efficiency.

Original: SSA [29] divides attention headsinto multiple groups to aggregate image features with different granularities.
Revised: SSA [29] divides attention heads into multiple groups to aggregate image features with different granularities.

Original: Guo et al.
Revised: Guo et al.

Original: [9] and
Revised: [9] and

Original: Zhao et al.
Revised: Zhao et al.

Original: [55] make the first step towards introducing the transformer for point cloud analysis.
Revised: [58] make the first step towards introducing the transformer for point cloud analysis.

Original: Recently, many approaches [45, 54, 25, 10, 22] apply local self-attentionon voxels to learn richerfeature representation.
Revised: Recently, many approaches [47, 57, 25, 10, 22, 49] apply local self-attention to learn richer 3D feature representation.

Original: Our work extends the window-based attention on 3D voxels by introducingscale-aware attention learning equipped with novel sampling strategies for the queries and the keys toimprove both accuracy and efficiency.
Revised: Our work extends the window-based attention on 3D voxels by introducing scale-aware attention learning equipped with novel sampling strategies for the queries and the keys to improve both accuracy and efficiency.

Original: Voxel transformer for 3D detection.
Revised: Voxel transformer for 3D detection.

Original: VoTr [23] introduces a voxel-based transformer backbonethat performs self-attention on sparse voxels withthe local andthe dilated attention mechanisms.
Revised: VoTr [23] introduces a voxel-based transformer backbone that performs self-attention on sparse voxels with local and dilated attention mechanisms.

Original: Our work improves VoTr by introducingthe window-based attention and optimizing the sparseoperation.
Revised: Our work improves VoTr by introducing window-based attention and optimizing sparse operation.

Original: The recent SST [7] follows a single-stride design and the swin-transformer architecture,which performs well on small objects.
Revised: The recent SST [7] follows a single-stride design and the swin-transformer architecture, which performs well on small objects.

Original: Nevertheless, SST is implemented based on pillars.
Revised: Nevertheless, SST is implemented based on pillars.

Original: Thesingle window size is not conducive to capturing multi-scale features, resulting in unsatisfactoryperformance on Vehicle when simultaneously detecting multiple categories.
Revised: The single window size is not conducive to capturing multi-scale features, resulting in unsatisfactory performance on Vehicle when simultaneously detecting multiple categories.

Original: In comparison, our MsSVT can capture mixed-scale information to boost  detection of objects of various scales.
Revised: In comparison, our MsSVT can capture mixed-scale information to boost the detection of objects of various scales.

Original:  Method
Revised:  Method

Original: This section first gives details of the MsSVT block, then its efficient sparse implementation, andfinally the 3D detector based on MsSVT.
Revised: This section first gives details of the MsSVT block, then its efficient sparse implementation, and finally the 3D detector based on MsSVT.

Revised: [Figure]

Revised: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.

Revised: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.

Revised: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

Original: 3.1 Mixed-scale Sparse Voxel Transformer
Revised: 3.1 Mixed-scale Sparse Voxel Transformer

Original: Fig.
Revised: Fig.

Original: 2 illustrates the overall architecture of the MsSVT block.
Revised: 2 illustrates the overall architecture of the MsSVT block.

Original: We first gather the query and the keyvoxels via chessboard sampling and balanced multi-window sampling, respectively.
Revised: We first gather the query and the key voxels via chessboard sampling and balanced multi-window sampling, respectively.

Original: The obtainedqueries and keys are then fed into multiple head groups to capture mixed-scale information throughscale-aware attention learning.
Revised: The obtained queries and keys are then fed into multiple head groups to capture mixed-scale information through scale-aware attention learning.

Original: We further incorporate scale-aware relative position encoding to makebetter use of position information in different head groups.
Revised: We further incorporate scale-aware relative position encoding to make better use of position information in different head groups.

Original: [Figure]

Original: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.

Original: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.

Original: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

Original: 3.1.1 Balanced Multi-window Sampling
Revised: 3.1.1 Balanced Multi-window Sampling

Original: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query windowand s 1 ,...,M are the sizes of M successively larger key windows.
Revised: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query window and s 1 ,...,M are the sizes of M successively larger key windows.

Original: Let V
Revised: Let V

Original: = { v i | v i =
Revised: = { v i | v i =

Original: ( x i , f i ) } |V| i =1 bethe input voxel set, with xyz coordinates x i ∈
Revised: ( x i , f i ) } |V| i =1 be the input voxel set, with xyz coordinates x i ∈

Original: Z 3 and feature vector f i ∈ R C for voxel
Revised: Z 3 and feature vector f i ∈ R C for voxel

Original: i .
Revised: i .

Original: We firstpartition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-emptyones as query windows with their centers denoted by { c i
Revised: We first partition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-empty ones as query windows with their centers denoted by { c i

Original: | c i ∈
Revised: | c i ∈

Original: Z 3 } Li =0 , where L is the total numberof query windows.
Revised: Z 3 } Li =0 , where L is the total number of query windows.

Original: To get query voxels V c i , s 0 for the query window centered on c
Revised: To get query voxels V c i , s 0 for the query window centered on c

Original: i , one can simplygather all the non-empty voxels within the window as the queries.
Revised: i , one can simply gather all the non-empty voxels within the window as the queries.

Original: While keeping efficiency in mind,we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.
Revised: While keeping efficiency in mind, we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.

Original: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for theneighbors for each center c i within multiple key windows of different sizes.
Revised: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for the neighbors for each center c i within multiple key windows of different sizes.

Original: For the key window ofsize s k , we gather no more than N P non-empty voxels V c
Revised: For the key window of size s k , we gather no more than N P non-empty voxels V c

Original: i , s k
Revised: i , s k

Original: = { v j | − s k < x j − c
Revised: = { v j | − s k < x j − c

Original: i < s k } N P j =1 ,where N P is a pre-set number.
Revised: i < s k } N P j =1 , where N P is a pre-set number.

Original: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from
Revised: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from

Original: V c i , s k to obtain the final key voxels
Revised: V c i , s k to obtain the final key voxels

Original: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-setmaximum number of sampled voxels.
Revised: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-set maximum number of sampled voxels.

Original: Benefitting from the multi-window strategy equipped withuniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, whichis crucial to capturing mixed-scale information.
Revised: Benefitting from the multi-window strategy equipped with uniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, which is crucial to capturing mixed-scale information.

Original: 3.1.2 Chessboard Sampling
Revised: 3.1.2 Chessboard Sampling

Original: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels andonly need to select some representative ones to reduce computational redundancy.
Revised: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels and only need to select some representative ones to reduce computational redundancy.

Original: While thingsare different for the queries.
Revised: While things are different for the queries.

Original: On the one hand, we should retain and update every query voxel afteran attention layer; otherwise, the information could be irreversibly lost.
Revised: On the one hand, we should retain and update every query voxel after an attention layer; otherwise, the information could be irreversibly lost.

Original: But on the other hand,the computational cost and memory footprint grow dramatically with the window size, makingimplementation impractical if the queries are not reduced.
Revised: But on the other hand, the computational cost and memory footprint grow dramatically with the window size, making implementation impractical if the queries are not reduced.

Original: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxelseach time for feature updating, and then use thesampled voxels to update those unsampled.
Revised: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxels each time for feature updating, and then use the sampled voxels to update those unsampled.

Original: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.
Revised: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.

Original: Each voxel inthe query window is marked with " × ", " ⃝ ", " △ ",or " □ ", in an evenly spaced pattern.
Revised: Each voxel in the query window is marked with " × ", " ⃝ ", " △ ", or " □ ", in an evenly spaced pattern.

Original: In each MsSVTblock, we sample queries from the non-empty voxelswith one specific type of the marks, and update thequeries’ features through attention learning.
Revised: In each MsSVT block, we sample queries from the non-empty voxels with one specific type of the marks, and update the queries’ features through attention learning.

Original: Thenfor each unsampled non-empty voxel, we update itsfeature by finding the K-nearest query voxels (bydefault K = 3 ) and by linearly interpolating theirupdated features.
Revised: Then for each unsampled non-empty voxel, we update its feature by finding the K-nearest query voxels (by default K = 3 ) and by linearly interpolating their updated features.

Original: The four masks are used in the circular to sample query voxels in stacked blocks.
Revised: The four masks are used in the circular to sample query voxels in stacked blocks.

Original: This way, we retain the original structure and cover all voxels as comprehensively as possible.
Revised: This way, we retain the original structure and cover all voxels as comprehensively as possible.

Original: Notethat we can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2,1/4, or 1/8.
Revised: Note that we can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2, 1/4, or 1/8.

Original: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).
Revised: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).

Original: 3.1.3 Scale-aware Head Attention
Revised: 3.1.3 Scale-aware Head Attention

Original: Given the query voxels V c
Revised: Given the query voxels V c

Original: i , s 

Revised: i , s

Original: =
Revised: =

Original: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and featurevectors F 0 ∈ R N Q × C , and the multi-scale key voxels
Revised: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and feature vectors F 0 ∈ R N Q × C , and the multi-scale key voxels

Original: V fps c i , s k =
Revised: V fps c i , s k =

Original: ( X k , F k ) , k = 1 , ..., M with voxelcoordinates X
Revised: ( X k , F k ) , k = 1 , ..., M with voxel coordinates

Original:  k ∈
Revised: X k ∈

Original: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk =1and values { V k } Mk =1 as
Revised: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk = and values { V k } Mk =1 as

Original: [Equation]
Revised: [Equation]

Original: where W Q , W K , W V ∈ R C
Revised: where W Q , W K , W V ∈ R C

Original: ×
Revised: ×

Original: C are linear projections.
Revised: C are linear projections.

Original: To achieve scare-aware attention learning, wedivide multiple attention heads into M groups and assign the keys from windows of different sizes todifferent head groups.
Revised: To achieve scare-aware attention learning, we divide multiple attention heads into M groups and assign the keys from windows of different sizes to different head groups.

Original: Accordingly, we also split the feature channels of the queries Q into M groups.
Revised: Accordingly, we also split the feature channels of the queries Q into M groups.

Original: The k -th channel group of Q , denoted by Q k =
Revised: The k -th channel group of Q , denoted by Q k =

Original: Q
Revised: Q

Original: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M ,is fed into the k -th head group.
Revised: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M , is fed into the k -th head group.

Original: As a result, each head group is responsible for learning to attendinformation at a specific scale.
Revised: As a result, each head group is responsible for learning to attend information at a specific scale.

Original: The attended feature for the k -th head group is
Revised: The attended feature for the k -th head group is

Original: [Equation]
Revised: [Equation]

Original: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents the newly proposedrelative position encoding which will be detailed in Section 3.1.4.
Revised: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents the newly proposed relative position encoding which will be detailed in Section 3.1.4.

Original: Each window size corresponds toa head group with one or more attention heads.
Revised: Each window size corresponds to a head group with one or more attention heads.

Original: We concatenate the output from all heads groups{ ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implementedby multi-layer perceptron (MLP) to obtain the final mixed-scale feature Y :[Equation]
Revised: We concatenate the output from all heads groups { ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implemented by multi-layer perceptron (MLP) to obtain the final mixed-scale feature Y :

Original: [Equation]
Revised: [Equation]

Original: where LN ( · ) represents layer normalization.
Revised: where LN ( · ) represents layer normalization.

Original: 3.1.4 Scale-aware Relative Position Encoding
Revised: 3.1.4 Scale-aware Relative Position Encoding

Original: Relative position encoding is necessary for transformer-based networks because fine-grained positioninformation may be lost in high-level features with the deepening of the network.
Revised: Relative position encoding is necessary for transformer-based networks because fine-grained position information may be lost in high-level features with the deepening of the network.

Original: To make better useof position information to facilitate multi-scale feature learning in our case, we adopt a scale-awareadaptive relative position encoding strategy inspired by [30, 46, 52], which can generate the positionalbias dynamically with scales for different head groups.
Revised: To make better use of position information to facilitate multi-scale feature learning in our case, we adopt a scale-aware adaptive relative position encoding strategy inspired by [30, 48, 55], which can generate the positional bias dynamically with scales for different head groups.

Original: Specifically, we establish a learnable embedding table T k ∈ R D × R for the k -th head group accordingto the size of the largest key window [21], where R is the number of possible relative position pairsand D is the feature dimension.
Revised: Specifically, we establish a learnable embedding table T k ∈ R D × R for the k -th head group according to the size of the largest key window [21], where R is the number of possible relative position pairs and D is the feature dimension.

Original: We get the relative positional bias for the queries as:
Revised: We get the relative positional bias for the queries as:

Original: [Equation]
Revised: [Equation]

Original: where I
Revised: where I

Original: k ∈
Revised: k ∈

Original: Z L × N Q × N K represents the table indices corresponding to the actual relative positionbetween the queries and the keys, G ( · ) is the operation of gathering features according to the indices.
Revised: Z L × N Q × N K represents the table indices corresponding to the actual relative position between the queries and the keys, G ( · ) is the operation of gathering features according to the indices.

Original: Similarly, we can ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.
Revised: Similarly, we can ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.

Original: Finally, the biasesact directly on the attention weights, and Eq.
Revised: Finally, the biases act directly on the attention weights, and Eq.

Original: (2) can be rewritten as:
Revised: (2) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) represents softmax.
Revised: where σ ( · ) represents softmax.

Original: This way, position embeddings can be adjusted adaptively accordingto different scales to guide our scale-ware head attention better.
Revised: This way, position embeddings can be adjusted adaptively according to different scales to guide the scale-ware head attention better.

Original: 3.2 Sparse Implementation
Revised: 3.2 Sparse Implementation

Original: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implementall our window center searching, window gathering, and balanced window sampling into CUDAoperations.
Revised: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implement all our window center searching, window gathering, and balanced window sampling into CUDA operations.

Original: These operations are mainly based on a hash map that establishes the mapping fromcoordinate space to voxel index [23].
Revised: These operations are mainly based on a hash map that establishes the mapping from coordinate space to voxel index [23].

Original: Take the window gathering  as an example, we query eachpossible position w.r.t.
Revised: Taking the window gathering operation as an example, we query each possible position w.r.t.

Original: the given center within the window and retrieve the corresponding features ifthe position is a valid key in our pre-built hash map.
Revised: the given center within the window, and retrieve the corresponding features if the position is a valid key in the pre-built hash map.

Original: More details are available in the supplementarymaterials.
Revised: More details are available in the supplementary materials.

Original: 3.3 Detector Establishment
Revised: 3.3 Detector Establishment

Original: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.
Revised: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.

Original: Noted that weset both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3Dvoxels into a 2D feature map, where the query is the average voxel features within the pillar window.
Revised: Noted that we set both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3D voxels into a 2D feature map, where the query is the average voxel features within the pillar window.

Original: We replace the 3D backbone in SECOND [47] with MsSVT and keep the other network componentsunchanged.
Revised: We replace the 3D backbone in SECOND [50] with MsSVT and keep the other network components unchanged.

Original: We discard the down sampling process because MsSVT can already capture features atdifferent scales.
Revised: We discard the down sampling process because MsSVT can already capture features at different scales.

Original: The input point cloud is first converted into regular voxels and fed into our MsSVTbackbone to get mixed-scale voxel features.
Revised: The input point cloud is first converted into regular voxels and fed into our MsSVT backbone to get mixed-scale voxel features.

Original: The features are then compressed vertically and sent tothe subsequent 2D RPN and detection head to get detection results.
Revised: The features are then compressed vertically and sent to the subsequent 2D RPN and detection head to get detection results.

Original:  Experiments
Revised:  Experiments

Original: [Figure] In this section, we first provide architectural details of MsSVT, then compare our model with recentstate-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.
Revised: In this section, we first provide architectural details of MsSVT, then compare our model with recent state-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.

Original: Thorough ablation studiesand in-depth analysis are further provided to validate our design choices.
Revised: Thorough ablation studies and in-depth analysis are further provided to validate our design choices.

Original: 4.1 Architectural Details
Revised: 4.1 Architectural Details

Original: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key windowsizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are setas 1 × 1 pillars as mentioned in 3.3.
Revised: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key window sizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are set as 1 × 1 pillar as mentioned in 3.3.

Original: We divide 8 attention heads into 2 head groups.
Revised: We divide 8 attention heads into 2 head groups.

Original: The samplingrate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .
Revised: The sampling rate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .

Original: Weuse the center head [53] to generate single-stage bounding boxes.
Revised: We use the center head [56] to generate single-stage bounding boxes.

Original: In addition, we also provide atwo-stage version with CT3D [31].
Revised: In addition, we also provide a two-stage version with CT3D [31].

Original: More details can be found in OpenPCDet [40] since we conductall experiments with this toolbox. 
Revised: More details can be found in OpenPCDet [40] since we conduct all experiments with this toolbox. [Table] [Table]

Original: 4.2 Results on Waymo
Revised: 4.2 Results on Waymo

Original: Setups.
Revised: Setups.

Original: We first evaluate our model on large-scale Waymo Open Dataset [38].
Revised: We first evaluate our model on large-scale Waymo Open Dataset [38].

Original: The input is asingle-frame point cloud with 150m × 150m detection range.
Revised: The input is a single-frame point cloud with 150m × 150m detection range.

Original: We set the detection range in thehorizontal and the vertical direction as [ − 75 .
Revised: We set the detection range in the horizontal and the vertical direction as [ − 75 .

Original: 2m , 75 . 2m] and [ − 2 .
Revised: 2m , 75 . 2m] and [ − 2 .

Original: 0m , 4 . 0m] , respectively.
Revised: 0m , 4 . 0m] , respectively.

Original: The voxelsize is (0 . 4m , 0 .
Revised: The voxel size is (0 . 4m , 0 .

Original: 4m , 0 .
Revised: 4m , 0 .

Original: 6m) .
Revised: 6m) .

Original: We follow the same training strategy as in [23].
Revised: We follow the same training strategy as in [23].

Original: Specifically, we train ourmodel for 80 epochs on 20% Waymo data using the Adam optimizer [13].
Revised: Specifically, we train our model for 80 epochs on 20% Waymo data using the Adam optimizer [13].

Original: We apply the cyclic decayscheme [47], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochsand further decreased to 1 e- 5 in the remaining epochs.
Revised: We apply the cyclic decay scheme [50], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochs and further decreased to 1 e- 5 in the remaining epochs.

Original: We also report the results of training for 30epochs on 100% Waymo data using the same optimizer and learning rate scheme.
Revised: We also report the results of training forepochs on 100% Waymo data using the same optimizer and learning rate scheme.

Original: The evaluationmetric is the 3D mean Average Precision (mAP) [Table] [Table]
Revised: The evaluation metric is the 3D mean Average Precision (mAP) for difficulty levels of LEVEL 1 and LEVEL 2.

Original: Main results.
Revised: Main results.

Original: We compare our model with state-of-the-art priors in Table 1 and Table 2 .
Revised: We compare our model with state-of-the-art priors in Table 1 and Table 2 .

Original: It is worthnoting that our model performs simultaneous detection of three object categories, which is morechallenging than detecting a single category.
Revised: It is worth noting that our model performs simultaneous detection of three object categories, which is more challenging than detecting a single category.

Original: Using 20% training data, our single-stage MsSVT-SSsignificantly outperforms other single-stage detectors, even though some of them are trained speciallyfor one specific category.
Revised: Using 20% training data, our single-stage MsSVT-SS significantly surpasses other single-stage counterparts, even though some of them are trained specially for one specific category.

Original: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage
Revised: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage

Original: PV-RCNN++ [36] on Vehicle , and even better, i.e. , 0 .
Revised: PV-RCNN++ [36] on Vehicle , and even better ( 0 .

Original: 8 - 1 . 5 mAP higher, on Pedestrian and Cyclist .
Revised: 8 - 1 . 5 mAP higher) on Pedestrian and Cyclist .

Original: Inaddition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperformsprevious best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.
Revised: In addition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperforms previous best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.

Original: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achievethe best results compared to their corresponding counterparts.
Revised: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achieve the best results compared to their corresponding counterparts.

Original: Noticeably, our MsSVT-SS surpassesthe recent transformer-based detector SST [7]that is dedicated to small object detection by largemargins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.
Revised: Noticeably, our MsSVT-SS surpasses the recent transformer-based detector SST [7] dedicated to small object detection by large margins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.

Original: This clear evidences the superiorityof our MsSVT in capturing multi-scale information overthe conventional transformer design.
Revised: This clearly evidences the superiority of our MsSVT in capturing mixed-scale information over conventional transformer designs.

Original: Visualization.
Revised: Visualization.

Original: We provide some qualitative results in Fig.
Revised: We provide some qualitative results in Fig.

Original: Our MsSVT can still predict accuratebounding boxes inthe scenes beyond the range of 50m where the points become extremely sparse(Fig. 4 (a)).
Revised: Our MsSVT can still predict accurate bounding boxes in scenes beyond the range of 50m where the points become extremely sparse (Fig.(a)).

Original: This evidences that MsSVT successfully captures valuable contextual information in theabsence of fine-grained details for long-range object detection.
Revised: This evidences that MsSVT successfully captures valuable contextual information in the absence of fine-grained details for long-range object detection.

Original: In addition, our model also performswell in complex scenes in the presence of dense objects with large scale variations (Fig. 4 (b).
Revised: In addition, our model also performs well in complex scenes in the presence of dense objects with large-scale variations (Fig. 4 (b).

Original: Itshould be owed to the flexibility and robustness of MsSVT.
Revised: It should be owed to the flexibility and robustness of MsSVT.

Original: Analysis.
Revised: Analysis.

Original: We further visualize some attention maps which indicate what have been focused on byour MsSVT to understand its behavior better.
Revised: We further visualize some attention maps which indicate what has been focused on by our MsSVT to understand its behavior better.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, the head group with a smaller keywindow focuses on the local foreground information, while the head group with a larger key windowattends more to  longer-range context.
Revised: 5, the head group with a smaller key window focuses on the local foreground information, while the head group with a larger key window attends more to the longer-range context.

Original: As a result, the two head groups complement each other, wellcapturing mixed-scale information to support the detection of objects of various scales.
Revised: As a result, the two head groups complement each other, well capturing mixed-scale information to support the detection of objects of various scales

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: 4.3 Results on KITTI
Revised: 4.3 Results on KITTI

Original: Setups.
Revised: Setups.

Original: We also evaluate our model on KITTI [8].
Revised: We also evaluate our model on KITTI [8].

Original: Given an input point cloud, we reserve thepoints within the range of [0m , 70 . 4m] , [ − 40 .
Revised: Given an input point cloud, we reserve the points within the range of [0m , 70 . 4m] , [ − 40 .

Original: 0m , 40 .
Revised: 0m , 40 .

Original: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes,respectively.
Revised: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes, respectively.

Original: The voxel size is set to (0 .
Revised: The voxel size is set to (0 .

Original: 32m , 0 . 32m , 0 .
Revised: 32m , 0 . 32m , 0 .

Original: 4m)
Revised: 4m)

Original: and other settings remain the same asin the experiments on Waymo.
Revised: and other settings remain the same as in the experiments on Waymo.

Original: We train the model for 100 epochs with the Adam optimizer.
Revised: We train the model for 100 epochs with the Adam optimizer.

Original: Thelearning rate is 0 . 003 , decayed by the cyclic scheme [47].
Revised: The learning rate is 0 . 003 , decayed by the cyclic scheme [50].

Original: The evaluation metric is 3D mAP for threedifficulty levels (easy, moderate, and hard).
Revised: The evaluation metric is 3D mAP for three difficulty levels (easy, moderate, and hard).

Original: Main results.
Revised: Main results.

Original: Table 3 depicts that our model achieves competitive performance on all threecategories.
Revised: Table 3 depicts that our model achieves competitive performance on all three categories.

Original: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by0 . 5 mAP.
Revised: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by 0 . 5 mAP.

Original: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminenttwo-stage detectors.
Revised: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminent two-stage detectors.

Original: Moreover, our two-stage MsSVT-TS further increases the lead.
Revised: Moreover, our two-stage MsSVT-TS further increases the lead.

Original: These resultsdemonstrate that our MsSVT can be well generalized to various datasets.
Revised: These results demonstrate that our MsSVT can be well generalized to various datasets.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: All  models are trained for 12 epochs on 20% Waymo data.
Revised: All ablation models are trained for 12 epochs on 20% Waymo data.

Original: More implementation details and ablationson hyperparameters are available in supplementary materials.
Revised: More implementation details and ablations on hyperparameters are available in supplementary materials.

Original: [Table]

Original: Balanced multi-window sampling.
Revised: Balanced multi-window sampling.

Original: We first validate our balanced multiwindow sampling strategy in Table 4.
Revised: We first validate our balanced multi-window sampling strategy in Table 4.

Original: The base model listed in the first rowemploys a 3D version of standardwindow-based attention [21] withwindow size (3 , 3 , 5) and without shiftwindow scheme, adopts dilated keysampling [23], and gathers all non-emptyvoxels within the window as queries.
Revised: The base model listed in the first row employs a 3D version of standard windowbased attention [21] with window size (3 , 3 , 5) and without shift window scheme, adopts dilated key sampling [23], and gathers all non-empty voxels within the window as queries.

Original: We build a model variant by simplyreplacing the dilated key sampling withour proposed balanced multi-window sampling.
Revised: We build a model variant by simply replacing the dilated key sampling with our proposed balanced multiwindow sampling.

Original: Results in the second row demonstrate that samplingkey voxels from multiple windows of different sizes brings noticeable performance gains, i.e. ,71 . 

Revised: Results in the second row demonstrate that sampling key voxels from multiple windows of different sizes brings noticeable performance gains, i [Table] e. , 71 .

Original: → 73 . 44 on Pedestrian and 63 .
Revised: → 73 . 44 on Pedestrian and 63 .

Original: 31 → 66 . 88 on Cyclist .
Revised: 31 → 66 . 88 on Cyclist .

Revised: [Table]

Original: Scale-aware head attention.
Revised: Scale-aware head attention.

Original: We also validate the effectiveness of our proposed scale-aware headattention in Table 4.
Revised: We also validate the effectiveness of our proposed scale-aware head attention in Table 4.

Original: By comparing the  second and the third rows, one can find that enabling multiplehead groups to capture information of different scales clearly boots the performance.
Revised: By comparing the results in the second and the third rows, one can find that enabling multiple head groups to capture information at different scales clearly boots the performance.

Original: Scale-aware relative position encoding.
Revised: [Table]

Original: Table 4 shows that incorporating scale-aware relativeposition encoding (listed in the last row) further improves the performance compared to the modelvariant that adopts scale-agnostic position encoding.
Revised: Table 4 shows that incorporating scaleaware relative position encoding (listed in the last row) further improves the performance compared to the model variant that adopts scale-agnostic position encoding.

Original: It well supports our design motivation that theencoded position information should vary with different scales.
Revised: It well supports our design motivation that the position encoding should vary with different scales

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: Chessboard sampling.
Revised: Chessboard sampling.

Original: Table 5 reportsthe performance of the model variants thatapply chessboard sampling with different sampling rates and of the model without performing any sampling.
Revised: Table 5 reports the performance of the model variants that apply chessboard sampling with different sampling rates and of the model without performing any sampling.

Original: We measure the latency on Tesla-V100 GPU and
Revised: We measure the latency on Tesla-V100 GPU and

Original: Unbuntu-16.04, Python 3.7, Cuda-10.2,and Pytorch-1.8.
Revised: Unbuntu-16.04, Python 3.7, Cuda-10.2, and Pytorch-1.8.

Original: Our model is not sensitive to varying sampling rates.
Revised: Our model is not sensitive to varying sampling rates.

Original: Incorporating chessboard sampling with a samplingrate of 1/4 results in only negligible performance degradation compared to the model variant withoutsampling, yet greatly improves the computational efficiency by a significant reduction in memoryfootprint (up to 33% decrease) and in latency (up to 28% decrease).
Revised: Incorporating chessboard sampling with a sampling rate of 1/4 results in only negligible performance degradation compared to the model variant without sampling, yet largely improves the computational efficiency by a significant reduction in memory footprint (up to 33% decrease) and in latency (up to 28% decrease).

Original:  Conclusion
Revised:  Conclusion

Original: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scaleinformation to boost 3D object detection.
Revised: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scale information to boost 3D object detection.

Original: We begin with sampling the queries and the keys at differentscales through the novel chessboard and the balanced multi-window sampling strategy, respectively.
Revised: We begin with sampling the queries and the keys at different scales through the novel chessboard and the balanced multi-window sampling strategy, respectively.

Original: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge ofcapturing information at a specific scale.
Revised: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge of capturing information at a specific scale.

Original: Extensive experiments have shown that  MsSVT is superiorin detecting objects of various scales and sparsity thanks to the learnd mixed-scale information.
Revised: Extensive experiments have shown that our MsSVT is superior in detecting objects of various scales and granularities thanks to the learned mixed-scale information.

Original: Limitations.
Revised: Limitations.

Original: MsSVT considers fine-grained details and long-range contextual information in differentwindows and achieves promising performance, but still needs to set hand-crafted window sizes.
Revised: MsSVT can capture mixed-scale information from multiple local windows and achieves promising performance. Yet, the sizes of the windows need to be pre-set manually.

Original: Wewill explore the adaptive window version of MsSVT in the future.
Revised: We will explore the adaptive-window version of MsSVT in future work.

Revised: Acknowledgements

Revised: This work was financially supported by the National Natural Science Foundation of China (No.

Revised: 62101032), the Postdoctoral Science Foundation of China (Nos. 2021M690015, 2022T150050) and

Revised: Beijing Institute of Technology Research Fund Program for Young Scholars (No. 3040011182111)

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Our work is only for academic research purpose.
Revised: [No] Our work is only for academic research purpose.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] Our implemen- tation details are posted on Section 4.1 and the supplemental materials.
Revised: [Yes] Our implementation details are posted on Section 4.1 and the supplemental materials.

Original: The data arepublic datasets and our code will be available.
Revised: The data are public datasets and our code will be available.

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Section 4.1 and setup in 4.2 and 4.
Revised: [Yes] See Section 4.1 and setup in 4.2 and 4.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report average performance of 5 trials by default.
Revised: [Yes] We report average performance of 5 trials by default.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] Waymo [38] and KITTI [8] (b) Did you mention the license of the assets?
Revised: [Yes] Waymo [38] and KITTI [8] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] We don’t include any new assets.
Revised: [No] We don’t include any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] Waymo [38] and KITTI [8] are two standardbenchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human subjects...
Revised: We didn’t use crowdsourcing or conduct research with human subjects...

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human subjects...
Revised: We didn’t use crowdsourcing or conduct research with human subjects...

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A] We didn’t use crowdsourcing or conduct research with human subjects...
Revised: [N/A] We didn’t use crowdsourcing or conduct research with human subjects...