Original: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds
Revised: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds

Original: Author(s)
Revised: (s)

Original: AffiliationAddress
Revised: AffiliationAddress

Original: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomousdriving.
Revised: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomousdriving.

Original: Large-scale outdoor scenes usually feature significant variance in instancescales, thus requiring features rich in long-range and fine-grained information tosupport accurate detection.
Revised: Large-scale outdoor scenes usually feature significant variance in instancescales, thus requiring features rich in long-range and fine-grained information tosupport accurate detection.

Original: Recent detectors leverage the power of window-basedtransformers to model long-range dependencies but, unfortunately, tend to blurout fine-grained details.
Revised: Recent detectors leverage the power of window-basedtransformers to model long-range dependencies but tend to blur out fine-graineddetails.

Original: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types ofinformation simultaneously by the divide-and-conquer philosophy.
Revised: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.

Original: Specifically, MsSVT explicitly divides attention heads into multiple groups, each in charge ofattending to information within a particular range.
Revised: Specifically, MsSVT explicitlydivides attention heads into multiple groups, each in charge of attending to information within a particular range.

Original: The output from all the groupsis merged to obtain the final mixed-scale features.
Revised: All groups’ output is merged to obtain the finalmixed-scale features.

Original: Moreover, we provide a novelchessboard sampling strategy to reduce the computational complexity of applyinga window-based transformer in 3D voxel space and implement the voxel samplingand gathering operations sparsely with a hash map to improve efficiency.
Revised: Moreover, we provide a novel chessboard sampling strategyto reduce the computational complexity of applying a window-based transformer in3D voxel space. To improve efficiency, we also implement the voxel sampling andgathering operations sparsely with a hash map.

Original: Endowedby the powerful capability and high efficiency of modeling mixed-scale information,our single-stage detector built on top of MsSVT surprisingly outperforms state-ofthe-art two-stage detectors on Waymo.
Revised: Endowed by the powerful capabilityand high efficiency of modeling mixed-scale information, our single-stage detectorbuilt on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectorson Waymo.

Original: Code will soon be available.
Revised: Code will soon be available.

Original:  Introduction
Revised:  Introduction

Original: 3D object detection has received increasing attention due to its successful autonomous drivingapplications.
Revised: 3D object detection has received increasing attention due to its successful autonomous drivingapplications.

Original: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturallyirregular and unordered.
Revised: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturallyirregular and unordered.

Original: Hence directly applying CNN-like operations [10, 11] to them can bedifficult.
Revised: Hence directly applying CNN-like operations [11, 12] to them can bedifficult.

Original: To solve this, many researchers have rasterized point clouds into regular voxel grids [21]and employed 3D CNNs to extract high-dimensional voxel features.
Revised: To solve this, many researchers have rasterized point clouds into regular voxel grids [24]and employed 3D CNNs to extract high-dimensional voxel features.

Original: With the recent rise of visiontransformer (ViT) [35] on 2D images, some attempts have been made to generalize global or moreefficient window-based transformers to 3D voxels [20] or pillars [6].
Revised: With the recent rise of visiontransformer (ViT) [42] on 2D images, some attempts have been made to generalize global or moreefficient window-based transformers to 3D voxels [23] or pillars [7].

Original: These methods successfully seeklong-range context by utilizing transformers’ powerful abilities in modeling long-range information.
Revised: These methods successfully seeklong-range context by utilizing transformers’ powerful abilities in modeling long-range information.

Original: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details,especially in sparse 3D space, crucial to accurate object recognition and localization.
Revised: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details,especially in sparse 3D space, crucial to accurate object recognition and localization.

Original: Standard window-based transformers update the features of queries in a local window by attending tokeys from the same window.
Revised: Standard window-based transformers update the features of queries in a local window by attending tokeys from the same window.

Original: Hence simultaneously aggregating long-range context and fine-graineddetails require enlarging the window size to embrace local and distant voxels.
Revised: Hence simultaneously aggregating long-range context and fine-graineddetails require enlarging the window size to embrace local and distant voxels.

Original: Nevertheless, directlygathering all the voxels within the window as keys suffer a cubical growth in computational cost w.r.t.the window size.
Revised: Nevertheless, directlygathering all the voxels within the window as keys suffers a cubical growth of the computationalload with the window size.

Original: Some attempts[20] alleviate this by sampling only a certain number of key voxels.
Revised: Some attempts alleviate this by sampling only a certain number of keyvoxels [23].

Original: While a trivial sampling strategy quickly leads to  sparse sampling of local voxels (Fig. 1 b), thusbias mainly on long-range context.
Revised: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels(Fig. 1 b), thus bias mainly on long-range context.

Original: To mitigate the above, we try to set up multiple key windowsof varying sizes centered on a query window and sample the same number of local and distant key voxels  from the smaller and larger windows separately.
Revised: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.

Original: As a result, we can keep finer granularity inthe local region to retain fine-grained details while collecting distant voxels roughly to enlarge thereceptive field (Fig.
Revised: As a result, we can keep finergranularity in the local region to retain fine-grained details while collecting distant voxels roughly toenlarge the receptive field (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: With the sampled voxels ready, the next question is how to  attend to voxels from different windowseffectively and  capture long-range context and fine-grained details simultaneously.
Revised: With the sampled voxels ready, the next question is how to effectively attend to voxels from differentwindows and simultaneously capture long-range context and fine-grained details.

Original: We argue thatthe divide-and-conquer philosophy can satisfactorily resolve this issue.
Revised: We argue that thedivide-and-conquer philosophy can satisfactorily resolve this issue.

Original: Specifically, inspired by therecent findings [35, 48, 23] that transformers learn different levels of self-attention by different heads,we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides thetransformer heads into multiple groups.
Revised: Specifically, inspired by therecent findings [42, 56, 26] that transformers learn different levels of self-attention by different heads,we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides thetransformer heads into multiple groups.

Original: Different head groups accept voxels sampled from windowsof different sizes, such that they are each in charge of capturing information of a particular scale.
Revised: Different head groups accept voxels sampled from windowsof different sizes, so they are each in charge of capturing information of a particular scale.

Original: Bycombining the outputs from all the head groups,  mixed-scale information, i.e. , long-range context,and fine-grained details can be well captured simultaneously.
Revised: Combiningthe outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context,and fine-grained details.

Original: We also design a novel scale-awarerelative position encoding strategy to adaptively adjust the position encoding used in each headgroup according to the range of  keys.
Revised: We also design a novel scale-aware relative position encoding strategy toadaptively adjust the position encoding used in each head group according to the range of the keys.

Original: We provide some resulting attention maps by two differenthead groups (Fig. 1 (d), (e)).
Revised: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).

Original: It is also worth mentioning that the mixed-scale attention meanwhileenables information exchange across local windows, which makes MsSVT more compact by savingadditional shift window operationthat is commonly required by window-based transformers [18, 6].
Revised: It is alsoworth mentioning that the mixed-scale attention enables information exchange across local windows,making MsSVT more compact by saving additional shift window operation commonly required bywindow-based transformers [21, 7].

Original: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reducecomputational costs in two ways.
Revised: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reducecomputational costs in two ways.

Original: First, we propose a novel chessboard sampling (CBS) strategyto reduce the number of query voxels that need to be sampled within the query window, to reducecomputational costs without losing information.
Revised: First, we propose a novel chessboard sampling (CBS) strategyto reduce the number of query voxels that need to be sampled within the query window, to reducecomputational costs without losing information.

Original: Specifically, we partition the query window intochess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.
Revised: Specifically, we partition the query window intochess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.

Original: During each attentionlayer, only one specific position of voxels is sampled and updated by serving as queries, and theupdates of the other voxels can be obtained by interpolation.
Revised: During each attentionlayer, only one specific position of voxels is sampled and updated by serving as queries, and theupdates of the other voxels can be obtained by interpolation.

Original: Four positions are selected in the circular.
Revised: Four positions are selected in the circular.

Original: Thus, we can update all the voxels without introducing deviation.
Revised: Thus, we can update all the voxels without introducing deviation.

Original: Second, we take advantage ofnon-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-emptysites in 3D space and parallelizing the search and feature gathering for non-empty voxels using hashmapping for further acceleration.
Revised: Second, we take advantage ofnon-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-emptysites in 3D space. Furthermore, we parallelize the search and feature gathering for non-empty voxelsusing hash mapping for further acceleration.

Original: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [39] with our
Revised: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [47] with our

Original: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [32].
Revised: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].

Original: Benefitingfrom the powerful capability of abstracting  voxel features of mixed-scales, our single-stage detectorbased on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.
Revised: Benefitingfrom the powerful capability of abstracting mixed-scale voxel features, our single-stage detectorbased on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.

Original: We summarize ourcontributions as follows:
Revised: We summarize ourcontributions as follows:

Original: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which abstracts voxelfeatures withboth long-range context and fine-grained details simultaneously.•
Revised: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneouslyabstracts voxel features with long-range context and fine-grained details.•

Original: We design an efficient chessboard sampling strategy to vastly reduce the computational costof applying a voxel-based transformer in 3D space and sparsely implement all operations toimprove efficiency.• Our  single-stage detectorbased on MsSVT even outperforms state-of-the-art two-stagedetectors on Waymo.
Revised: We design an efficient chessboard sampling strategy to vastly reduce the computational costof applying a voxel-based transformer in 3D space and sparsely implement all operations toimprove efficiency.• Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors onWaymo.

Original:  Related work
Revised:  Related work

Original: Outdoor 3D detection on point cloud.
Revised: Outdoor 3D detection on point clouds.

Original: The mainstream outdoor 3D detection models are based onvoxel [2, 40, 41, 31, 15, 13, 49, 39] or pillar [14].
Revised: The mainstream outdoor 3D object detectors are basedon voxels [3, 48, 49, 37, 17, 14, 57, 47] or pillars [16, 32].

Original: VoxelNet [49] utilizes a point net to aggregate thefeatures within each voxel and is further processed by sparse 3D convolution and 2D convolutionto generate detection results.
Revised: VoxelNet [57] utilizes PointNet [27,32] to aggregate features within each voxel and then apply sparse 3D convolution to generatedetection results.

Original: SECOND [39] investigatesan improved sparse convolution method,which dramatically improves the model’s speed.
Revised: SECOND [47] investigates improved sparse convolution to further improve speed.

Original: Pointpillar [14] utilizes a pillar (voxel with infinitelength in the vertical direction) to quantify the point cloud, so it can fully apply the mature 2D 
Revised: Pointpillar [16] quantifies the point cloud into pillars such that 2D CNNs can be applied to trade offbetween accuracy and efficiency. Rapoport-Lavie et al .

Original: CNN, considering both efficiency and performance.
Revised: [28] introduce the Cylindrical Coordinatesto leverage the natural scanning pattern from LiDAR sensors. Chen et al . [2] further operate onboth Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS)voxel representation. Reconfigurable Voxels [43] improves the local neighbors searching of eachvoxel using a random walk scheme.

Original: Two-stage models [17, 29, 30, 26] aggregatethe original point cloud or voxel features on the proposal of the single-stage model, which is used toadjust the bounding box.
Revised: Voxel-FPN [15] and Pillar-in-Pillar [41] adopt a multi-scalevoxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate

Original: At present, they have state-of-the-art performance.
Revised: FPN [20] to aggregate multi-resolution feature maps. Two-stage detectors [19, 35, 36, 31] refine thebounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features,yielding state-of-the-art performance.

Original: Vision transformer.
Revised: Vision transformer.

Original: Inspired by the great successof Transformer [35, 4] in NLP, some worksemploy it inthe field of computer vision [5, 1, 18, 16, 50, 42, 36].
Revised: Transformer [42, 5] has recently achieved great success in computer vision [6,1 , 21, 18, 58, 50, 44].

Original: Swin-transformer [18] increasesefficiency by limiting self-attentioncomputation to non-overlapping local windows while allowing forcross-window connection.
Revised: Swin-transformer [21] restricts self-attention to non-overlapping local windowswhile allowing cross-window connection to improve efficiency.

Original: SSA [24] dividesthe attention head into multiple groups and aggregatesimage features with different granularity, respectively, which has achieved excellent performance.
Revised: SSA [29] divides attention headsinto multiple groups to aggregate image features with different granularities.

Original: In the field of point cloud, Guo et al.
Revised: Guo et al.

Original: [8] and Zhao et al.
Revised: [9] and Zhao et al.

Original: [47] introduce transformer paradigm forpoint cloud classification and segmentation.
Revised: [55] make the first step towards introducing the transformer for point cloud analysis.

Original: Recently, many methods [37, 46, 22, 9, 19] applylocal self-attentionmechanism on voxels to learn richer feature representations.
Revised: Recently, many approaches [45, 54, 25, 10, 22] apply local self-attention on voxels to learn richerfeature representation.

Original: Our model extendswindow attention to 3D voxels and flexibly combines different window sizes to capture multi-scaleand multi-granularity features while shunting them to different attention heads in SSA.
Revised: Our work extends the window-based attention on 3D voxels by introducingscale-aware attention learning equipped with novel sampling strategies for the queries and the keys toimprove both accuracy and efficiency.

Original: Voxel transformer backbone in 3D detection.
Revised: Voxel transformer for 3D detection.

Original: VoTr [20] introduces two kinds of sparse voxelattention, including localattention and extended attention. Each voxel serves as a query andattendswith the neighbor voxels.
Revised: VoTr [23] introduces a voxel-based transformer backbonethat performs self-attention on sparse voxels with the local and the dilated attention mechanisms.

Original: We optimize the sparse operation and introduce window attention, whichsignificantly improves efficiency and performance.
Revised: Our work improves VoTr by introducing the window-based attention and optimizing the sparseoperation.

Original: The recent SST [6] adopts a single-stride designand swin-transformer structure, which performs well on small objects.
Revised: The recent SST [7] follows a single-stride design and the swin-transformer architecture,which performs well on small objects.

Original: However, SST is implementedbased on a pillar.
Revised: Nevertheless, SST is implemented based on pillars.

Original: The single window size is not conducive to capturing multi-scale features, resultingin the low performance of vehicles when  multiple categories are detected simultaneously.
Revised: Thesingle window size is not conducive to capturing multi-scale features, resulting in unsatisfactoryperformance on Vehicle when simultaneously detecting multiple categories.

Original: In contrast,our MsSVT can capture fine and global features and better performance on large and small objects.
Revised: In comparison, our MsSVT can capture mixed-scale information to boost detection of objects of various scales.

Original:  Method
Revised:  Method

Original: [Figure] In this section, We first detail the pipeline of the Mixed-scale Sparse Voxel Transformer (MsSVT),then move to the novel chessboard sampling strategy and efficient sparse implementation, and build acomplete 3D detector based on the above components.
Revised: This section first gives details of the MsSVT block, then its efficient sparse implementation, andfinally the 3D detector based on MsSVT.

Original: 3.1 Mixed-scale Sparse Voxel Transformer
Revised: 3.1 Mixed-scale Sparse Voxel Transformer

Original: [Figure]

Original: Fig.
Revised: Fig.

Original: 2 illustrates the overall framework of our proposed MsSVT, consisting of two core modules: abalanced multi-window samplingmodule and a scale-aware head attention module.
Revised: 2 illustrates the overall architecture of the MsSVT block. We first gather the query and the keyvoxels via chessboard sampling and balanced multi-window sampling, respectively.

Original: We first get thequery, key, and value voxels balanced sampled from multiple windows through the former and thenfeed them into our scale-aware head attention tosimultaneously capture mixed-scale information.
Revised: The obtainedqueries and keys are then fed into multiple head groups to capture mixed-scale information throughscale-aware attention learning.

Original: We also add the scale-aware relative position encoding to the attention mechanism so that even thesame relative position informationcan play various roles in different scales.
Revised: We further incorporate scale-aware relative position encoding to makebetter use of position information in different head groups. [Figure]

Original: In MsSVT block, we gather the  voxels with the query window[-size (e.g. s 
 = (2 , 2 , 2) )-] and  the [-key window sizes (e.g. s 
 =-]
Revised: We gather the non-empty voxels within the query window[-size (e.g. s 
 = (2 , 2 , 2) )-] and apply Chessboard Sampling (CBS) to sample the [-key window sizes (e.g. s 
 =-]queries.

Original: [-(2 , 2 , 2) and s 
 = (4 , 4 , 4) ), respectively. Then,-] we apply CBS to obtain sampled queries, while employ FPS to get sampled keys and values of same number (e.g. N K = 3 ) among all key windows.
Revised: [-(2 , 2 , 2) and s 
 = (4 , 4 , 4) ), respectively. Then,-]For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.

Original: Finally, we feed the same query and the different keys into differentattention head groups, respectively.
Revised: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

Original: 3.1.1 Balanced Multi-window Sampling
Revised: 3.1.1 Balanced Multi-window Sampling

Original: Given a series of window sizes { s k | s k ∈ Z 3 } Mk =0 , where s 0 denotes thequery window size ands 1 ,...,M denote M different key window sizes from small to large.
Revised: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query windowand s 1 ,...,M are the sizes of M successively larger key windows.

Original: Let V
Revised: Let V

Original: = { v i | v i =
Revised: = { v i | v i =

Original: ( x i , f i ) } |V| i =
Revised: ( x i , f i ) } |V| i =1 bethe input voxel set, with xyz coordinates x i ∈ Z 3 and feature vector f i ∈ R C for voxel

Original: 1be
Revised: i .

Original: the input voxel set, where x i ∈ Z 3 denotes voxel coordinates and f i ∈ R C denotes voxel features. We first partition the voxel set into non-overlap 3D windows  s 0  by finding non-empty window centers{ c i
Revised: We firstpartition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-emptyones as query windows with their centers denoted by { c i

Original: | c i ∈
Revised: | c i ∈

Original: Z 3 } Li =0 , where L is the number of non-empty windows.
Revised: Z 3 } Li =0 , where L is the total numberof query windows.

Original: To get queryvoxels, we can simplygather all the non-empty voxels V c i , s 0  centered on c i  within query windows 0 to guarantee that everyvoxel can serve as a query and be updated after one attention layer.
Revised: To get query voxels V c i , s 0 for the query window centered on c i , one can simplygather all the non-empty voxels within the window as the queries.

Original: We also prsent another novelchessboard sampling strategy for the query voxel sampling  which will be discussed in Section 3.2.
Revised: While keeping efficiency in mind,we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.

Original: For easy understanding, readers can regrard all the non-empty voxels in each query window as thequeries. As for the key voxels, instead of sampling in a single large window at once like previous methods [20],which inevitably bias on  local or distant voxels, we simultaneously research the neighboors for eachcenter c 
Revised: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for theneighbors for each center c i within multiple key windows of different sizes.

Original: i with different window size s k and gather no more than m non-empty voxels 
Revised: For the key window ofsize s k , we gather no more than N P non-empty voxels V c

Original: V c i , s k
Revised: i , s k

Original: ={ v j |− s k < x j − c
Revised: = { v j | − s k < x j − c

Original: i < s k } mj =1 , where m is a pre-defined number for all key windows.
Revised: i < s k } N P j =1 ,where N P is a pre-set number.

Original: Furthermore, toreduce the computational cost and keep balanced sampling, we adopt  farthest point sampling (FPS) touniformly sample N K voxels for each window s k to obatain the final key voxels
Revised: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from V c i , s k to obtain the final key voxels

Original: V fps c i , s k , k = 1 , ..., Mat different scales, where N K is a predefined maximum number of sampled voxels.
Revised: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-setmaximum number of sampled voxels.

Original: Benefit fromthemulti-window sampling combined with the uniform FPS operation, we can achieve more balancedkey voxels sampling from various scales which is crucial to capturing mixed-scale information.
Revised: Benefitting from the multi-window strategy equipped withuniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, whichis crucial to capturing mixed-scale information.

Original: 3.1.2 Scale-aware Head Attention
Revised: 3.1.2 Chessboard Sampling

Revised: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels andonly need to select some representative ones to reduce computational redundancy. While thingsare different for the queries. On the one hand, we should retain and update every query voxel afteran attention layer; otherwise, the information could be irreversibly lost.

Revised: But on the other hand,the computational cost and memory footprint grow dramatically with the window size, makingimplementation impractical if the queries are not reduced.

Revised: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxelseach time for feature updating, and then use thesampled voxels to update those unsampled.

Revised: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.

Revised: Each voxel inthe query window is marked with " × ", " ⃝ ", " △ ",or " □ ", in an evenly spaced pattern.

Revised: In each MsSVTblock, we sample queries from the non-empty voxelswith one specific type of the marks, and update thequeries’ features through attention learning.

Revised: Thenfor each unsampled non-empty voxel, we update itsfeature by finding the K-nearest query voxels (bydefault K = 3 ) and by linearly interpolating theirupdated features.

Revised: The four masks are selected in turn to sample query voxels in stacked blocks.

Revised: Thisway, we retain the original structure and cover all voxels as comprehensively as possible.

Revised: Note thatwe can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2, 1/4, or1/8.

Revised: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).

Revised: 3.1.3 Scale-aware Head Attention

Original: Once receiving the query voxels V c
Revised: Given the query voxels V c

Original: i , s 

Revised: i , s

Original: = ( X 0 , F 0 )  and sampled key voxels
Revised: = ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and featurevectors F 0 ∈ R N Q × C , and the multi-scale key voxels

Original: V fps c i , s k =(
Revised: V fps c i , s k =

Original:  X k , F k ) , k = 1 , ..., M at different scales, where X 0 ∈ Z N Q × 3 , F 0 ∈ R N Q × C denote queryvoxel coordinates and features, X k ∈Z N K × 3 ,
Revised: ( X k , F k ) , k = 1 , ..., M with voxelcoordinates X k ∈

Original:  F k ∈ R N K × C denote key voxel coordinates andfeatures, we first get queries Q , keys { K k } Mk =1 and values { V k } Mk =1 as
Revised: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk =1and values { V k } Mk =1 as

Original: [Equation]
Revised: [Equation]

Original:  where W Q , W K ,W V ∈ R C
Revised: , M (1) where W Q , W K ,

Original:  × C are linear projection.
Revised: W V ∈ R C × C are linear projections.

Original: To make our attention module aware todifferent scales, we divide the  attention heads into Mmultiple groups and assign keys from differentwindows to different  head groups so that the queries Q should only attend K k in the k -th head group.
Revised: To achieve scare-aware attention learning, wedivide the multiple attention heads into M groups and assign keys from windows of different sizes todifferent head groups.

Original: We also split the feature channel into M parts and explicitly make each part represent the informationof a specific scale. We denote Q k =
Revised: Accordingly, we also split the feature channels of the queries Q into M groups. The k -th channel group of Q , denoted by Q k =

Original: Q
Revised: Q

Original: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M as the k -thscale features for queries and the same is true for K and V .
Revised: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M ,is fed into the k -th head group. As a result, each head group is responsible for learning to attendinformation of a specific scale.

Original: So for the k -th head group, we can getthe attended feature as:
Revised: The attended feature for the k -th head group is

Original: [Equation]
Revised: [Equation]

Original: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) stands for our relative positionencoding which will be discussed in Section 3.1.3.
Revised: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents newly proposed relativeposition encoding which will be discussed in Section 3.1.4.

Original: Each window size corresponds to a "headgroup" that includes one or more attention heads.
Revised: Each window size corresponds to a headgroup with one or more attention heads.

Original: Finally, we concatenate the output of all headsgroup { ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , and feed into FFN, where L denotes the total number ofnon-empty query windows. After that, we obtain the  mixed-scale features Y :
Revised: We concatenate the output from all heads groups { ˜ Y k } Mk =1 to˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implemented by multi-layerperceptron (MLP) to obtain the final mixed-scale feature Y :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Revised: where LN ( · ) represents layer normalization.

Original: 3.1.3 Scale-aware Relative Position Encoding
Revised: 3.1.4 Scale-aware Relative Position Encoding

Original: The 3D point cloud feature generally contains the original coordinates information, which voxelswill inherit. However, the fine-grained location information may be blurred with the deepening ofthe network, so the relative position encoding is necessary.
Revised: Relative position encoding is necessary for transformer-based networks because fine-grained positioninformation may be lost in high-level features with the deepening of the network.

Original: Furthermore, since MsSVT can extractmulti-scale features, the position-coding should differ at different scales, even for the same relativeposition. In light of these, we adopt an adaptive, scale-aware relative position encoding strategyinspired by [25, 38, 44].
Revised: To make better useof position information to facilitate multi-scale feature learning in our case, we adopt a scale-awareadaptive relative position encoding strategy inspired by [30, 46, 52], which can generate the positionalbias dynamically with scales for different head groups.

Original: Specifically,according to the size of the maximum key window, we establish a learnable embeddingtable T k ∈ R D × R for k -th head group like [18], where R is the number of possible relativeposition pairs and D is the feature dimension.
Revised: Specifically, we establish a learnable embedding table T k ∈ R D × R for k -th head group according tothe size of the largest key window [21], where R is the number of possible relative position pairs and D is the feature dimension.

Original: Then we can get thequery relative position bias 
Revised: We get the relative positional bias for the queries as:

Original: B kQ ∈ R L × N Q × N K as:

Original: [Equation]
Revised: [Equation]

Original: where I
Revised: where I

Original: k ∈
Revised: k ∈

Original: Z L × N Q × N K means the table indices corresponding to the actual relative position of thequeries and keys, G ( · ) means the operation of gather features via indices.
Revised: Z L × N Q × N K represent the table indices corresponding to the actual relative positionbetween the queries and keys, G ( · ) is the operation of gathering features via indices.

Original: We get the  B kK in the sameway.
Revised: Similarly, wecan ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.

Original: Finally, the bias acts directly on the attention, and Eq.
Revised: Finally, the biases act directly onthe attention weights, and Eq.

Original: (2) can be rewrited as:
Revised: (2) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) represents softmax.
Revised: where σ ( · ) represents softmax.

Original: In this way, position embeddings can be adjusted adaptively accordingto different scales to guide our scale-ware head attention better.
Revised: This way, position embeddings can be adjusted adaptively accordingto different scales to guide our scale-ware head attention better.

Original: 3.2 Chessboard Sampling

Original: In Section 3.1, the prerequisite for us to sample key voxels to a specific number is that we do notneed to preserve all the key voxels and only need to select some representative ones.

Original: However, thingsare different for queries.

Original: On the one hand, we should maintain and update every query voxel after anattention layer; otherwise, the information could be irreversibly lost, and sampling queries seemsimpossible.

Original: On the other hand, the computational cost grows cubically with window size, and itwould bring unacceptable memory costs if we can not reduce the number of queries.

Original: However, since the non-empty voxel positions are not changed during a sequential of attentionblocks, an effective way is to iteratively sample a subset of voxels from all the queries andthen use the sampled voxels to update those unsampled.

Original: In the light of this, we present Chessboard Sampling (CBS) as shown in Fig.

Original: In an attention module, we select one " × ", " ⃝ "," △ ", " □ " as the sampling location.

Original: The sampled non-empty voxels are sent to the subsequentnetwork for updating.

Original: The updates of other non-empty voxels can be obtained by linear interpolation with their top k nearest neighbors (default by 3).

Original: Four positions will be selected inturn in stacked modules.

Original: In this way, we retain the original structure and cover all voxels ascomprehensively as possible.

Original: Note that we can apply the interval to any axis to obtain the sampling rate of 1/2, 1/4, or 1/8.

Original: Generally, we apply it to the horizontal xy plane (like Fig. 3).

Original: 3.3 Sparse Implementation
Revised: 3.2 Sparse Implementation

Original: [Figure] We implement all our window center searching,window gathering, and balanced window samplingsparsely in cuda operations to leverage the natural sparsity of point clouds and  improve efficiency.
Revised: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implementall our window center searching, window gathering, and balanced window sampling into CUDAoperations.

Original: These operations are mainly based on a hash mapwhich establishes the mapping from coordinatespace to voxel index as in [20].
Revised: These operations are mainly based on a hash map that establishes the mapping fromcoordinate space to voxel index [23].

Original: For example,forwindow gathering, we query each possible positionwrt the given center within the window and retrievethe feature if the  position is a valid key in our prebuilt hash map.
Revised: Take the window gathering as an example, we query eachpossible position w.r.t. the given center within the window and retrieve the corresponding features ifthe position is a valid key in our pre-built hash map.

Original: More details are available in the supplementary materials.
Revised: More details are available in the supplementarymaterials.

Original: 3.4 Detector Establishment
Revised: 3.3 Detector Establishment

Original: As shown in Fig 2, we build our 3D backbone by stacking Mixed-scale Sparse Voxel Transformer(MsSVT) blocks.
Revised: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.

Original: It is worth noting that  both the query and  key window size in the lastblock of MsSVT are set to (1 , 1 , ∞ )so as to compressing the 3D voxels into  2D feature map, where the queryis the averageof all the voxel features within the pillar window.
Revised: Noted that weset both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3Dvoxels into a 2D feature map, where the query is the average voxel features within the pillar window.

Original: We replace the 3D backbonewithour MsSVT in SECOND [39] and keep the other architecture the same except for abandoning thedown-sampling process because MsSVT can already capture features at different scales.
Revised: We replace the 3D backbone in SECOND [47] with MsSVT and keep the other network componentsunchanged. We discard the down sampling process because MsSVT can already capture features atdifferent scales.

Original: The inputpoint cloud is first converted into regular voxels and fed into our MsSVT to getthe mixed-scale voxelfeatures.
Revised: The input point cloud is first converted into regular voxels and fed into our MsSVTbackbone to get mixed-scale voxel features.

Original: Next, features are  compressed vertically and sent to the following 2D RPN and detectionhead, outputting detection results.
Revised: The features are then compressed vertically and sent tothe subsequent 2D RPN and detection head to get detection results.

Original:  Experiments
Revised:  Experiments

Original: In this section, we  provide architectural details of MsSVT and compare our model with recentstate-of-the-art detectors on Waymo Open Dataset [32] and KITTI [7].
Revised: In this section, we first provide architectural details of MsSVT, then compare our model with recentstate-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.

Original: Thorough ablation studiesand in-depth analysis are further provided to validate our design choices.
Revised: Thorough ablation studiesand in-depth analysis are further provided to validate our design choices.

Original: 4.1 Architectural Details
Revised: 4.1 Architectural Details

Original: MsSVT includes four regular MsSVTattention blocks and an additional special one where thewindows are set as 1 × 1 pillars as mentioned in 3.4. The query window size ofthe normal MsSVTattention blocks is set to (3 , 3 , 5), andthe key windows are set to (3 , 3 , 5) and (7 , 7 , 7) .
Revised: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key windowsizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are setas 1 × 1 pillars as mentioned in 3.3.

Original: We dividetotal 8 attention heads into 2groups in the scale-aware head attention layer.
Revised: We divide 8 attention heads into 2 head groups.

Original: The queries are obtainedvia alternating chessboard sampling with 0.25 sampling ratio, and thekeys are balanced sampledwith a maximum number of  32 in each window.
Revised: The samplingrate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .

Original: We use center head [45] as the detection head togenerate a single-stagedetection bounding box.
Revised: Weuse the center head [53] to generate single-stage bounding boxes.

Original: In addition, we also provide a two-stage detectorwith CT3D [26].
Revised: In addition, we also provide atwo-stage version with CT3D [31].

Original: Please refer to OpenPCDet [34] for more details  since we conduct our experimentswith this toolbox.
Revised: More details can be found in OpenPCDet [40] since we conductall experiments with this toolbox.

Original: 4.2 Results on Waymo
Revised: 4.2 Results on Waymo

Original: Setup.
Revised: Setups.

Original: We first evaluate our model on large-scale Waymo Open Dataset [32].
Revised: We first evaluate our model on large-scale Waymo Open Dataset [38].

Original: The input is a singleframe point cloud with 150m × 150m detection range.
Revised: The input is asingle-frame point cloud with 150m × 150m detection range.

Original: We set the detection range in the horizontaland the vertical direction as [ − 75 .
Revised: We set the detection range in thehorizontal and the vertical direction as [ − 75 .

Original: 2m , 75 . 2m] and [ − 2 .
Revised: 2m , 75 . 2m] and [ − 2 .

Original: 0m , 4 . 0m] , respectively.
Revised: 0m , 4 . 0m] , respectively.

Original: The voxel size is(0 . 4m , 0 .
Revised: The voxelsize is (0 . 4m , 0 .

Original: 4m , 0 .
Revised: 4m , 0 .

Original: 6m) .
Revised: 6m) .

Original: We follow the same training strategy as in [20].
Revised: We follow the same training strategy as in [23].

Original: Specifically, we train the modelon Tesla-V100 using the Adam [12] optimizer for 80 epochs on 20% Waymo data.
Revised: Specifically, we train ourmodel for 80 epochs on 20% Waymo data using the Adam optimizer [13].

Original: We apply thecyclic decay scheme [39], by which the learning rate is increased from 1 e − 4 to 1 e − 3 during the first40% epochs and further decreased to 1e-5 in the remaining epochs.
Revised: We apply the cyclic decayscheme [47], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochsand further decreased to 1 e- 5 in the remaining epochs.

Original: We also provide the results of30epochs training  on 100%  data with the same optimizer and  scheme.
Revised: We also report the results of training for 30epochs on 100% Waymo data using the same optimizer and learning rate scheme.

Original: The evaluation metric is the 3Dmean Average Precision (mAP) for difficulty levels of LEVEL 1 and LEVEL 2.
Revised: The evaluationmetric is the 3D mean Average Precision (mAP) for difficulty levels of LEVEL 1 and LEVEL 2.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Main results.
Revised: Main results.

Original: Our main results are shown in Table 1 and Table 2 .
Revised: We compare our model with state-of-the-art priors in Table 1 and Table 2 .

Original: It is worth noting that ourmodel detects three categories simultaneously, which is more challenging than detecting a singlecategory.
Revised: It is worthnoting that our model performs simultaneous detection of three object categories, which is morechallenging than detecting a single category.

Original: On 20%  data, MsSVT-SS significantly outperforms other single-stage detectors, eventhough some methods areonly trained on a single category.
Revised: Using 20% training data, our single-stage MsSVT-SSsignificantly outperforms other single-stage detectors, even though some of them are trained speciallyfor one specific category.

Original: The single-stage results can almost matchthe state-of-the-art two-stagemethods PV-RCNN++ [30] on the vehicle and even outperform 0.8-1.5mAP on pedestrians and cyclists.
Revised: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage PV-RCNN++ [36] on Vehicle , and even better, i.e. , 0 . 8 - 1 . 5 mAP higher, on Pedestrian and Cyclist .

Original:  MsSVT-TS achieves the best performance in all categories and hasabsolute superiority over pedestrians and cyclists.
Revised: Inaddition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperformsprevious best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.

Original: On 100% data,  our model also achieves state-of-the-art performance.
Revised: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achievethe best results compared to their corresponding counterparts.

Original: Compared with the recenttransformer work SST [6], which is adept at the small object detection, MsSVT not only takes intoaccount all categories but also has better performance on small objects.
Revised: Noticeably, our MsSVT-SS surpassesthe recent transformer-based detector SST [7] that is dedicated to small object detection by largemargins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.

Original: This result proves MsSVT’ssuperiority over general transformers in capturing multi-scale features.
Revised: This clear evidences the superiorityof our MsSVT in capturing multi-scale information over the conventional transformer design.

Original: Visualization.
Revised: Visualization.

Original: Fig. 4 shows some visualization of our results.
Revised: We provide some qualitative results in Fig.

Original: As shown in Fig. 4 (a), in the rangeof > 50 m that the points are very sparse, the bounding boxes are still accurate, which means that
Revised: Our MsSVT can still predict accuratebounding boxes in the scenes beyond the range of 50m where the points become extremely sparse(Fig. 4 (a)).

Original:  MsSVT can rely on context information in the absence of fine-grained information. As shown in
Revised: This evidences that MsSVT successfully captures valuable contextual information in theabsence of fine-grained details for long-range object detection.

Original: Fig. 4 (b), even ina complex scene with many different categories of  objects at the same time, themodel still perform well, which should be owed to the flexibility and robustness of MsSVT.
Revised: In addition, our model also performswell in complex scenes in the presence of dense objects with large scale variations (Fig. 4 (b). Itshould be owed to the flexibility and robustness of MsSVT.

Original: Analysis.
Revised: Analysis.

Original: We visualized some attention weights.
Revised: We further visualize some attention maps which indicate what have been focused on byour MsSVT to understand its behavior better.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, the  smaller window modelfocuses on the local foreground information, while the  larger window model tends to focus on thecontext information.
Revised: 5, the head group with a smaller keywindow focuses on the local foreground information, while the head group with a larger key windowattends more to longer-range context.

Original: The two windows can complement each other. This proves the ability of MsSVTto capture mixed-scale features.
Revised: As a result, the two head groups complement each other, wellcapturing mixed-scale information to support the detection of objects of various scales.

Original: [Table]
Revised: [Table]

Original: 4.3 Results on KITTI
Revised: 4.3 Results on KITTI

Original: Setup.
Revised: Setups.

Original: We further evaluate our model on KITTI [7].
Revised: We also evaluate our model on KITTI [8].

Original: Given an input point cloud, we reserve thepoints within the range of [0m , 70 . 4m] , [ − 40 .
Revised: Given an input point cloud, we reserve thepoints within the range of [0m , 70 . 4m] , [ − 40 .

Original: 0m , 40 .
Revised: 0m , 40 .

Original: 0m] and [ − 3 . 0m , 1 . 0m] in the X, Y, and Zdirections, respectively.
Revised: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes,respectively.

Original: The voxel size is set to (0 .
Revised: The voxel size is set to (0 .

Original: 32m , 0 . 32m , 0 .
Revised: 32m , 0 . 32m , 0 .

Original: 4m)
Revised: 4m)

Original: and other settings are sameas 4.2.
Revised: and other settings remain the same asin the experiments on Waymo.

Original: We train the model for 100 epochs with the Adam optimizer.
Revised: We train the model for 100 epochs with the Adam optimizer.

Original: The learning rate is 0 . 003 ,decayed by the cyclic scheme [39].
Revised: Thelearning rate is 0 . 003 , decayed by the cyclic scheme [47].

Original: The evaluation metric is 3D mAP for three difficulty levels (easy,moderate, and hard).
Revised: The evaluation metric is 3D mAP for threedifficulty levels (easy, moderate, and hard).

Original: Main results.
Revised: Main results.

Original: Table 3 depicts  our MsSVT achieves competitive performance on allthe threecategories.
Revised: Table 3 depicts that our model achieves competitive performance on all threecategories.

Original: Specifically, on Car ,  MsSVT-SS beats the superior VoTr [20] by 0 . 5 mAP.
Revised: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by0 . 5 mAP.

Original: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminent two-stage detectors.
Revised: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminenttwo-stage detectors.

Original: Moreover,our two-stage MsSVT-TS further increases the lead.
Revised: Moreover, our two-stage MsSVT-TS further increases the lead.

Original: The results on KITTI demonstrate that our MsSVT has a good generalization ability to adapt to different datasets.
Revised: These resultsdemonstrate that our MsSVT can be well generalized to various datasets.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: We perform ablation studies to validate our design choices.

Original: All models are trained for 12 epochs on20% Waymo data.
Revised: All models are trained for 12 epochs on 20% Waymo data.

Original: More implementation details and ablations on hyperparameters are available insupplementary materials.
Revised: More implementation details and ablationson hyperparameters are available in supplementary materials.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table] Balanced [Table] window sampling
Revised: Balanced multi-window sampling.

Original: [Table]

Original: We first validate our balanced multiwindow sampling strategy in Table 4.
Revised: We first validate our balanced multiwindow sampling strategy in Table 4.

Original: The base model, listed in the firstrow, employs 3D version of standardwindow-based attention [18] (withwindow size (3 , 3 , 5) and without shiftwindow scheme), adopts dilated keysampling strategy [20], and gathers allnon-empty voxels within the windowas queries.
Revised: The base model listed in the first rowemploys a 3D version of standardwindow-based attention [21] withwindow size (3 , 3 , 5) and without shiftwindow scheme, adopts dilated keysampling [23], and gathers all non-emptyvoxels within the window as queries.

Original: We build a model variantby simply replacing the dilated keysampling in the base model with our balanced multi-window sampling strategy.
Revised: We build a model variant by simplyreplacing the dilated key sampling withour proposed balanced multi-window sampling.

Original: The results inthe second row shows sampling key voxels from  windows of multiple sizes brings noticeableperformance gains, i.e. , 71 . 66 → 73 . {+
+}
Revised: Results in the second row demonstrate that samplingkey voxels from multiple windows of different sizes brings noticeable performance gains, i.e. ,71 . {+
+}

Original:  44 on Pedestrian and 63 .
Revised: → 73 . 44 on Pedestrian and 63 .

Original: 31 → 66 . 88 on Cyclist .
Revised: 31 → 66 . 88 on Cyclist .

Original: Scale-aware head attention.
Revised: Scale-aware head attention.

Original: Table 4 also validates the effectiveness of our  scale-aware headattention module.
Revised: We also validate the effectiveness of our proposed scale-aware headattention in Table 4.

Original: By comparing the second and the third rows, we find that making different attentionhead groupsspecialized in different scales helps to capture mixed-scale features and thus bringssignificant performance gains. We further visualize some attention maps by different head groups in Fig.
Revised: By comparing the second and the third rows, one can find that enabling multiplehead groups to capture information of different scales clearly boots the performance.

Original: One can observe that the query point pays more attention to fine-grained details in a smallwindow and is more interested in long-range contextual information in a large window.

Original: Scale-aware relative position encoding.
Revised: Scale-aware relative position encoding.

Original: Table 4 shows compared with the model variant thatadopts scale-agnostic position encoding (listed in the second row), our scale-aware relative positionencoding gives better performance, which well supports our design motivation that the importance ofpositional information varies with different scales.
Revised: Table 4 shows that incorporating scale-aware relativeposition encoding (listed in the last row) further improves the performance compared to the modelvariant that adopts scale-agnostic position encoding. It well supports our design motivation that theencoded position information should vary with different scales.

Original: [Table]
Revised: [Table]

Original: Chessboard sampling.
Revised: Chessboard sampling.

Original: Table 5 reportsthe performance of the model variants using chessboard sampling with differentsampling ratios, and of the model without performing any sampling.
Revised: Table 5 reportsthe performance of the model variants thatapply chessboard sampling with different sampling rates and of the model without performing any sampling.

Revised: We measure the latency on Tesla-V100 GPU and

Revised: Unbuntu-16.04, Python 3.7, Cuda-10.2,and Pytorch-1.8.

Original: One can seethat our model is not sensitive to the variation of sampling ratio.
Revised: Our model is not sensitive to varying sampling rates.

Original: The model variant with samplingratio 1/4 performs evenon par with  the model without sampling,while bringing a significant reduction(upto 6.1G) in memory footprint.
Revised: Incorporating chessboard sampling with a samplingrate of 1/4 results in only negligible performance degradation compared to the model variant withoutsampling, yet greatly improves the computational efficiency by a significant reduction in memoryfootprint (up to 33% decrease) and in latency (up to 28% decrease).

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose MsSVT, a novel3D sparse voxel transformer backbone for 3D detectionto capture mixed-scale information simultaneously.
Revised: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scaleinformation to boost 3D object detection.

Original: We first utilize the novel chessboard samplingand balanced multi-window sampling to obtain queries and a set of keys with different scales.
Revised: We begin with sampling the queries and the keys at differentscales through the novel chessboard and the balanced multi-window sampling strategy, respectively.

Original: Then MsSVT explicitly divides attention heads into multiple scale-aware groups and each in charge ofcapturing information at a specific scale.
Revised: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge ofcapturing information at a specific scale.

Original: Equipped with the scale-aware relative position encoding, MsSVT finally gets the  mixed-scale features which is crucial for accurate detection.
Revised: Extensive experiments have shown that MsSVT is superiorin detecting objects of various scales and sparsity thanks to the learnd mixed-scale information.

Original: Limitations.
Revised: Limitations.

Original: MsSVT considersboth fine-grained details and long-range contextual information indifferent windows and achieves promising performance, but still needs to set hand-crafted windowsizes, we will explore  adaptive window version of MsSVT in the future.
Revised: MsSVT considers fine-grained details and long-range contextual information in differentwindows and achieves promising performance, but still needs to set hand-crafted window sizes. Wewill explore the adaptive window version of MsSVT in the future.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes](b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.(c)
Revised: [Yes] See Section 5.

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Our workis only for academic research purpose.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [No] Our work is only for academic research purpose. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]2.
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b)
Revised: [N/A]

Original:  Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?

Original: [Yes] Our implementation details are posted on Section 4.1 and the supplemental materials.
Revised: [Yes] Our implemen- tation details are posted on Section 4.1 and the supplemental materials.

Original: The data arepublic datasets and our code will be available.(b)
Revised: The data arepublic datasets and our code will be available.

Original:  Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Section 4.1 and setup in 4.2 and [-4.
-]
Revised: [Yes] See Section 4.1 and setup in 4.2 and [-4.
-]4.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report average performance of 5 trials by default.
Revised: [Yes] We report average performance of 5 trials by default.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.4.
Revised: [Yes] See Section 4.1.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] Waymo [32] and
Revised: [Yes] Waymo [38] and

Original: KITTI [7]
Revised: KITTI [8]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: We don’t include any new assets.
Revised: We don’t include any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [Yes] Waymo [32] and KITTI [7] are two standard benchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] Waymo [32] and KITTI [7] are two standard benchmarks.
Revised: [Yes] Waymo [38] and KITTI [8] are two standardbenchmarks.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots,
Revised: (a) Did you include the full text of instructions given to participants and screenshots,

Original: if applicable?
Revised: if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human
Revised: We didn’t use crowdsourcing or conduct research with human

Original: subjects...
Revised: subjects...

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: We didn’t use crowdsourcing or conduct
Revised: We didn’t use crowdsourcing or conduct

Original: research with human subjects...
Revised: research with human subjects...

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A] We didn’t use crowdsourcing or conduct
Revised: [N/A] We didn’t use crowdsourcing or conduct

Original: research with human subjects...
Revised: research with human subjects...